2023-09-30T14:17:12.339 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-30T14:17:12.340 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627 --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} --name yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627/orig.config.yaml 2023-09-30T14:17:12.395 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-30T14:17:12.488 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627 branch: main description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} email: yweinste@redhat.com first_in_suite: false job_id: '7407627' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: main ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 1d10b71792f3be8887a7631e69851ac2df3585af subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 1d10b71792f3be8887a7631e69851ac2df3585af kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 1d10b71792f3be8887a7631e69851ac2df3585af owner: scheduled_yuriw@teuthology priority: 76 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6478 sha1: 1d10b71792f3be8887a7631e69851ac2df3585af sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_1d10b71792f3be8887a7631e69851ac2df3585af/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 1d10b71792f3be8887a7631e69851ac2df3585af syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi156.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-29_19:46:13 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-09-30T14:17:12.488 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_1d10b71792f3be8887a7631e69851ac2df3585af/qa; will attempt to use it 2023-09-30T14:17:12.489 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_1d10b71792f3be8887a7631e69851ac2df3585af/qa/tasks 2023-09-30T14:17:12.489 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-30T14:17:12.494 INFO:teuthology.task.internal:Checking packages... 2023-09-30T14:17:12.517 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '1d10b71792f3be8887a7631e69851ac2df3585af' 2023-09-30T14:17:12.517 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T14:17:12.517 INFO:teuthology.packaging:ref: None 2023-09-30T14:17:12.517 INFO:teuthology.packaging:tag: None 2023-09-30T14:17:12.517 INFO:teuthology.packaging:branch: main 2023-09-30T14:17:12.517 INFO:teuthology.packaging:sha1: 1d10b71792f3be8887a7631e69851ac2df3585af 2023-09-30T14:17:12.518 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=main 2023-09-30T14:17:12.767 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6447.g1d10b717 2023-09-30T14:17:12.767 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-30T14:17:12.775 INFO:teuthology.task.internal:no buildpackages task found 2023-09-30T14:17:12.775 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-30T14:17:12.848 INFO:teuthology.task.internal:Saving configuration 2023-09-30T14:17:12.887 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-30T14:17:12.939 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T14:17:12.965 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 14:11:51.467913', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-30T14:17:12.991 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 14:11:51.470920', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-30T14:17:13.012 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 14:11:51.469733', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-30T14:17:13.012 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-30T14:17:13.019 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-09-30T14:17:13.019 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-09-30T14:17:13.019 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-09-30T14:17:13.019 INFO:teuthology.run_tasks:Running task console_log... 2023-09-30T14:17:13.120 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3b0263eee0>, signals=[15]) 2023-09-30T14:17:13.120 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-30T14:17:13.127 INFO:teuthology.task.internal:Opening connections... 2023-09-30T14:17:13.127 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T14:17:13.129 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T14:17:13.197 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2023-09-30T14:17:13.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T14:17:13.265 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2023-09-30T14:17:13.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T14:17:13.331 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-30T14:17:13.338 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2023-09-30T14:17:13.358 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2023-09-30T14:17:13.359 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2023-09-30T14:17:13.418 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Red Hat Enterprise Linux" 2023-09-30T14:17:13.419 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="8.6 (Ootpa)" 2023-09-30T14:17:13.419 INFO:teuthology.orchestra.run.smithi032.stdout:ID="rhel" 2023-09-30T14:17:13.419 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="fedora" 2023-09-30T14:17:13.419 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="8.6" 2023-09-30T14:17:13.419 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el8" 2023-09-30T14:17:13.419 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-30T14:17:13.419 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2023-09-30T14:17:13.419 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-30T14:17:13.419 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.redhat.com/" 2023-09-30T14:17:13.419 INFO:teuthology.orchestra.run.smithi032.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-30T14:17:13.420 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-30T14:17:13.420 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:17:13.420 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-30T14:17:13.420 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-30T14:17:13.420 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-30T14:17:13.420 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-30T14:17:13.421 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2023-09-30T14:17:13.442 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2023-09-30T14:17:13.462 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2023-09-30T14:17:13.462 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2023-09-30T14:17:13.523 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="Red Hat Enterprise Linux" 2023-09-30T14:17:13.523 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="8.6 (Ootpa)" 2023-09-30T14:17:13.523 INFO:teuthology.orchestra.run.smithi148.stdout:ID="rhel" 2023-09-30T14:17:13.523 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="fedora" 2023-09-30T14:17:13.523 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="8.6" 2023-09-30T14:17:13.524 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el8" 2023-09-30T14:17:13.524 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-30T14:17:13.524 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2023-09-30T14:17:13.524 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-30T14:17:13.524 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://www.redhat.com/" 2023-09-30T14:17:13.524 INFO:teuthology.orchestra.run.smithi148.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-30T14:17:13.525 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-30T14:17:13.525 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-09-30T14:17:13.525 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-30T14:17:13.525 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-30T14:17:13.525 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-30T14:17:13.525 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-30T14:17:13.525 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2023-09-30T14:17:13.550 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2023-09-30T14:17:13.571 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2023-09-30T14:17:13.571 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2023-09-30T14:17:13.631 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="Red Hat Enterprise Linux" 2023-09-30T14:17:13.631 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="8.6 (Ootpa)" 2023-09-30T14:17:13.631 INFO:teuthology.orchestra.run.smithi156.stdout:ID="rhel" 2023-09-30T14:17:13.631 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="fedora" 2023-09-30T14:17:13.632 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="8.6" 2023-09-30T14:17:13.632 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el8" 2023-09-30T14:17:13.632 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-30T14:17:13.632 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2023-09-30T14:17:13.632 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-30T14:17:13.632 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://www.redhat.com/" 2023-09-30T14:17:13.632 INFO:teuthology.orchestra.run.smithi156.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-30T14:17:13.632 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-30T14:17:13.632 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-30T14:17:13.632 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-30T14:17:13.632 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-30T14:17:13.632 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-30T14:17:13.633 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-30T14:17:13.633 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2023-09-30T14:17:13.658 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-30T14:17:13.669 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-30T14:17:13.674 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-30T14:17:13.674 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2023-09-30T14:17:13.677 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2023-09-30T14:17:13.679 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2023-09-30T14:17:13.697 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-30T14:17:13.703 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-30T14:17:13.703 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2023-09-30T14:17:13.737 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2023-09-30T14:17:13.739 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2023-09-30T14:17:13.765 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-30T14:17:13.834 INFO:teuthology.run_tasks:Running task kernel... 2023-09-30T14:17:13.848 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-09-30T14:17:13.849 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-09-30T14:17:13.849 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-09-30T14:17:13.849 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-30T14:17:13.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T14:17:13.869 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2023-09-30T14:17:13.930 INFO:teuthology.orchestra.run.smithi032.stdout:4.18.0-372.9.1.el8.x86_64 2023-09-30T14:17:13.930 INFO:teuthology.task.kernel:Running kernel on smithi032: 4.18.0-372.9.1.el8.x86_64 2023-09-30T14:17:13.930 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum install -y kernel 2023-09-30T14:17:15.118 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-09-30T14:17:15.119 INFO:teuthology.orchestra.run.smithi032.stdout:Unable to read consumer identity 2023-09-30T14:17:16.487 INFO:teuthology.orchestra.run.smithi032.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-09-30T14:17:17.188 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 23 MB/s | 16 MB 00:00 2023-09-30T14:17:20.838 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 516 kB/s | 24 kB 00:00 2023-09-30T14:17:21.549 INFO:teuthology.orchestra.run.smithi032.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-09-30T14:17:21.606 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2023-09-30T14:17:21.607 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2023-09-30T14:17:21.607 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2023-09-30T14:17:21.735 DEBUG:teuthology.orchestra.run.smithi032:> echo no | sudo yum reinstall kernel || true 2023-09-30T14:17:22.085 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-09-30T14:17:22.085 INFO:teuthology.orchestra.run.smithi032.stdout:Unable to read consumer identity 2023-09-30T14:17:22.366 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:02 ago on Sat 30 Sep 2023 02:17:20 PM UTC. 2023-09-30T14:17:22.411 INFO:teuthology.orchestra.run.smithi032.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-09-30T14:17:22.565 INFO:teuthology.orchestra.run.smithi032.stderr:Error: No packages marked for reinstall. 2023-09-30T14:17:22.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum reinstall -y kernel || true 2023-09-30T14:17:22.962 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-09-30T14:17:22.962 INFO:teuthology.orchestra.run.smithi032.stdout:Unable to read consumer identity 2023-09-30T14:17:23.242 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:03 ago on Sat 30 Sep 2023 02:17:20 PM UTC. 2023-09-30T14:17:23.290 INFO:teuthology.orchestra.run.smithi032.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-09-30T14:17:23.314 INFO:teuthology.orchestra.run.smithi032.stderr:Error: No packages marked for reinstall. 2023-09-30T14:17:23.366 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q kernel | sort -rV | head -n 1 2023-09-30T14:17:23.448 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-09-30T14:17:23.449 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-09-30T14:17:23.449 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-09-30T14:17:23.449 DEBUG:teuthology.parallel:result is None 2023-09-30T14:17:23.450 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-30T14:17:23.459 INFO:teuthology.task.internal:Creating test directory... 2023-09-30T14:17:23.459 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-30T14:17:23.462 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-30T14:17:23.464 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-30T14:17:23.484 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-30T14:17:23.491 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-30T14:17:23.501 INFO:teuthology.task.internal:Creating archive directory... 2023-09-30T14:17:23.501 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-30T14:17:23.519 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-30T14:17:23.524 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-30T14:17:23.575 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-30T14:17:23.582 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-30T14:17:23.582 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-30T14:17:23.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T14:17:23.616 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-30T14:17:23.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T14:17:23.635 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-30T14:17:23.637 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-30T14:17:23.659 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-30T14:17:23.670 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T14:17:23.688 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T14:17:23.697 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T14:17:23.715 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T14:17:23.736 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T14:17:23.758 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T14:17:23.759 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-30T14:17:23.767 INFO:teuthology.task.internal:Configuring sudo... 2023-09-30T14:17:23.767 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-30T14:17:23.769 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-30T14:17:23.772 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-30T14:17:23.800 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-30T14:17:23.809 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-30T14:17:23.809 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-30T14:17:23.838 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-30T14:17:23.840 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-30T14:17:23.858 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T14:17:23.935 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T14:17:23.990 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T14:17:24.074 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T14:17:24.117 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:17:24.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-30T14:17:24.185 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T14:17:24.230 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T14:17:24.281 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T14:17:24.365 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T14:17:24.409 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-09-30T14:17:24.409 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-30T14:17:24.477 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T14:17:24.521 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T14:17:24.585 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T14:17:24.631 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T14:17:24.674 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-30T14:17:24.674 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-30T14:17:24.743 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2023-09-30T14:17:24.745 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2023-09-30T14:17:24.747 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2023-09-30T14:17:24.804 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T14:17:24.813 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T14:17:24.842 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T14:17:25.151 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-30T14:17:25.159 INFO:teuthology.task.internal:Starting timer... 2023-09-30T14:17:25.159 INFO:teuthology.run_tasks:Running task pcp... 2023-09-30T14:17:25.171 INFO:teuthology.run_tasks:Running task selinux... 2023-09-30T14:17:25.180 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-09-30T14:17:25.180 DEBUG:teuthology.orchestra.run.smithi032:> sudo service auditd rotate 2023-09-30T14:17:25.240 INFO:teuthology.orchestra.run.smithi032.stdout:Rotating logs: 2023-09-30T14:17:25.242 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2023-09-30T14:17:25.299 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2023-09-30T14:17:25.300 DEBUG:teuthology.orchestra.run.smithi156:> sudo service auditd rotate 2023-09-30T14:17:25.371 INFO:teuthology.orchestra.run.smithi156.stdout:Rotating logs: 2023-09-30T14:17:25.373 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-30T14:17:25.373 DEBUG:teuthology.orchestra.run.smithi032:> /usr/sbin/getenforce 2023-09-30T14:17:25.400 INFO:teuthology.orchestra.run.smithi032.stdout:Permissive 2023-09-30T14:17:25.400 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2023-09-30T14:17:25.426 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2023-09-30T14:17:25.426 DEBUG:teuthology.orchestra.run.smithi156:> /usr/sbin/getenforce 2023-09-30T14:17:25.445 INFO:teuthology.orchestra.run.smithi156.stdout:Permissive 2023-09-30T14:17:25.446 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi032.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi148.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi156.front.sepia.ceph.com': 'permissive'} 2023-09-30T14:17:25.446 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T14:17:25.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T14:17:25.474 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T14:17:25.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T14:17:25.501 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T14:17:25.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T14:17:25.528 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-30T14:17:25.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo /usr/sbin/setenforce permissive 2023-09-30T14:17:25.559 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2023-09-30T14:17:25.585 DEBUG:teuthology.orchestra.run.smithi156:> sudo /usr/sbin/setenforce permissive 2023-09-30T14:17:25.614 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-30T14:17:25.624 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-09-30T14:17:25.624 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-30T14:17:25.639 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-30T14:17:25.640 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi148.front.sepia.ceph.com,smithi156.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-30T14:26:32.738 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi148.front.sepia.ceph.com'), Remote(name='ubuntu@smithi156.front.sepia.ceph.com')] 2023-09-30T14:26:32.739 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-09-30T14:26:32.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T14:26:32.810 DEBUG:teuthology.orchestra.run.smithi032:> true 2023-09-30T14:26:32.876 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-09-30T14:26:32.876 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-09-30T14:26:32.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T14:26:32.945 DEBUG:teuthology.orchestra.run.smithi148:> true 2023-09-30T14:26:33.012 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-09-30T14:26:33.012 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-09-30T14:26:33.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T14:26:33.081 DEBUG:teuthology.orchestra.run.smithi156:> true 2023-09-30T14:26:33.152 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-09-30T14:26:33.152 INFO:teuthology.run_tasks:Running task clock... 2023-09-30T14:26:33.162 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-30T14:26:33.162 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-30T14:26:33.163 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T14:26:33.165 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-30T14:26:33.165 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T14:26:33.168 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-30T14:26:33.168 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T14:26:33.200 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-30T14:26:33.200 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-30T14:26:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-30T14:26:33.217 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-30T14:26:33.222 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-30T14:26:33.237 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-30T14:26:33.295 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2023-09-30T14:26:33.298 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: ntpd: command not found 2023-09-30T14:26:33.301 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: ntpd: command not found 2023-09-30T14:26:33.307 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2023-09-30T14:26:33.311 INFO:teuthology.orchestra.run.smithi032.stdout:506 Cannot talk to daemon 2023-09-30T14:26:33.313 INFO:teuthology.orchestra.run.smithi156.stdout:506 Cannot talk to daemon 2023-09-30T14:26:33.321 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-30T14:26:33.327 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-30T14:26:33.327 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-30T14:26:33.335 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-30T14:26:33.341 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-30T14:26:33.343 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-30T14:26:33.413 INFO:teuthology.orchestra.run.smithi148.stderr:bash: ntpq: command not found 2023-09-30T14:26:33.415 INFO:teuthology.orchestra.run.smithi032.stderr:bash: ntpq: command not found 2023-09-30T14:26:33.415 INFO:teuthology.orchestra.run.smithi156.stderr:bash: ntpq: command not found 2023-09-30T14:26:33.417 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T14:26:33.417 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2023-09-30T14:26:33.417 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T14:26:33.417 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T14:26:33.417 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T14:26:33.417 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T14:26:33.418 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T14:26:33.418 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2023-09-30T14:26:33.418 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T14:26:33.418 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T14:26:33.419 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T14:26:33.419 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T14:26:33.420 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T14:26:33.420 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-09-30T14:26:33.420 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T14:26:33.420 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T14:26:33.420 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T14:26:33.420 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T14:26:33.421 INFO:teuthology.run_tasks:Running task pexec... 2023-09-30T14:26:33.430 INFO:teuthology.task.pexec:Executing custom commands... 2023-09-30T14:26:33.430 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T14:26:33.431 DEBUG:teuthology.orchestra.run.smithi032:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-30T14:26:33.431 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi148.front.sepia.ceph.com 2023-09-30T14:26:33.431 DEBUG:teuthology.orchestra.run.smithi148:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-30T14:26:33.431 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi156.front.sepia.ceph.com 2023-09-30T14:26:33.431 DEBUG:teuthology.orchestra.run.smithi156:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-30T14:26:33.829 INFO:teuthology.orchestra.run.smithi148.stdout:Updating Subscription Management repositories. 2023-09-30T14:26:33.833 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-09-30T14:26:33.851 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-09-30T14:26:34.818 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:39 ago on Sat 30 Sep 2023 02:24:55 PM UTC. 2023-09-30T14:26:34.819 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:40 ago on Sat 30 Sep 2023 02:24:54 PM UTC. 2023-09-30T14:26:34.836 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:30 ago on Sat 30 Sep 2023 02:25:04 PM UTC. 2023-09-30T14:26:36.540 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-09-30T14:26:36.540 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-09-30T14:26:36.541 INFO:teuthology.orchestra.run.smithi156.stdout: Package Architecture Version Repository Size 2023-09-30T14:26:36.541 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-09-30T14:26:36.541 INFO:teuthology.orchestra.run.smithi156.stdout:Resetting modules: 2023-09-30T14:26:36.541 INFO:teuthology.orchestra.run.smithi156.stdout: container-tools 2023-09-30T14:26:36.541 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-30T14:26:36.541 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-09-30T14:26:36.541 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-09-30T14:26:36.541 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-30T14:26:36.555 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2023-09-30T14:26:36.556 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2023-09-30T14:26:36.556 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2023-09-30T14:26:36.556 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2023-09-30T14:26:36.556 INFO:teuthology.orchestra.run.smithi148.stdout:Resetting modules: 2023-09-30T14:26:36.557 INFO:teuthology.orchestra.run.smithi148.stdout: container-tools 2023-09-30T14:26:36.557 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-09-30T14:26:36.557 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2023-09-30T14:26:36.557 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2023-09-30T14:26:36.557 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-09-30T14:26:36.818 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2023-09-30T14:26:36.820 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2023-09-30T14:26:36.820 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2023-09-30T14:26:36.820 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2023-09-30T14:26:36.820 INFO:teuthology.orchestra.run.smithi032.stdout:Resetting modules: 2023-09-30T14:26:36.820 INFO:teuthology.orchestra.run.smithi032.stdout: container-tools 2023-09-30T14:26:36.820 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:26:36.820 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2023-09-30T14:26:36.820 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2023-09-30T14:26:36.820 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:26:37.486 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:43 ago on Sat 30 Sep 2023 02:24:54 PM UTC. 2023-09-30T14:26:37.522 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:42 ago on Sat 30 Sep 2023 02:24:55 PM UTC. 2023-09-30T14:26:37.826 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:33 ago on Sat 30 Sep 2023 02:25:04 PM UTC. 2023-09-30T14:26:40.577 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-09-30T14:26:40.643 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2023-09-30T14:26:40.753 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2023-09-30T14:26:41.068 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-09-30T14:26:41.127 INFO:teuthology.orchestra.run.smithi148.stdout:Updating Subscription Management repositories. 2023-09-30T14:26:41.229 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-09-30T14:26:41.880 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:47 ago on Sat 30 Sep 2023 02:24:54 PM UTC. 2023-09-30T14:26:41.947 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:46 ago on Sat 30 Sep 2023 02:24:55 PM UTC. 2023-09-30T14:26:42.113 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:37 ago on Sat 30 Sep 2023 02:25:04 PM UTC. 2023-09-30T14:26:43.603 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-09-30T14:26:43.658 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2023-09-30T14:26:43.695 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================================ 2023-09-30T14:26:43.695 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2023-09-30T14:26:43.695 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================================ 2023-09-30T14:26:43.695 INFO:teuthology.orchestra.run.smithi156.stdout:Installing group/module packages: 2023-09-30T14:26:43.695 INFO:teuthology.orchestra.run.smithi156.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-30T14:26:43.696 INFO:teuthology.orchestra.run.smithi156.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-30T14:26:43.696 INFO:teuthology.orchestra.run.smithi156.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-30T14:26:43.696 INFO:teuthology.orchestra.run.smithi156.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-30T14:26:43.696 INFO:teuthology.orchestra.run.smithi156.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-30T14:26:43.696 INFO:teuthology.orchestra.run.smithi156.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-30T14:26:43.696 INFO:teuthology.orchestra.run.smithi156.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-30T14:26:43.696 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2023-09-30T14:26:43.696 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-30T14:26:43.696 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-30T14:26:43.696 INFO:teuthology.orchestra.run.smithi156.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-30T14:26:43.696 INFO:teuthology.orchestra.run.smithi156.stdout:Installing module profiles: 2023-09-30T14:26:43.696 INFO:teuthology.orchestra.run.smithi156.stdout: container-tools/common 2023-09-30T14:26:43.697 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-30T14:26:43.697 INFO:teuthology.orchestra.run.smithi156.stdout:Enabling module streams: 2023-09-30T14:26:43.697 INFO:teuthology.orchestra.run.smithi156.stdout: container-tools rhel8 2023-09-30T14:26:43.697 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-30T14:26:43.697 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-09-30T14:26:43.697 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================================ 2023-09-30T14:26:43.697 INFO:teuthology.orchestra.run.smithi156.stdout:Install 10 Packages 2023-09-30T14:26:43.697 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-30T14:26:43.698 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 18 M 2023-09-30T14:26:43.698 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 65 M 2023-09-30T14:26:43.698 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2023-09-30T14:26:43.749 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================================================ 2023-09-30T14:26:43.749 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2023-09-30T14:26:43.749 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================================================ 2023-09-30T14:26:43.749 INFO:teuthology.orchestra.run.smithi148.stdout:Installing group/module packages: 2023-09-30T14:26:43.749 INFO:teuthology.orchestra.run.smithi148.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-30T14:26:43.750 INFO:teuthology.orchestra.run.smithi148.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-30T14:26:43.750 INFO:teuthology.orchestra.run.smithi148.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-30T14:26:43.750 INFO:teuthology.orchestra.run.smithi148.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-30T14:26:43.750 INFO:teuthology.orchestra.run.smithi148.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-30T14:26:43.750 INFO:teuthology.orchestra.run.smithi148.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-30T14:26:43.750 INFO:teuthology.orchestra.run.smithi148.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-30T14:26:43.750 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2023-09-30T14:26:43.750 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-30T14:26:43.750 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-30T14:26:43.750 INFO:teuthology.orchestra.run.smithi148.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-30T14:26:43.750 INFO:teuthology.orchestra.run.smithi148.stdout:Installing module profiles: 2023-09-30T14:26:43.751 INFO:teuthology.orchestra.run.smithi148.stdout: container-tools/common 2023-09-30T14:26:43.751 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-09-30T14:26:43.751 INFO:teuthology.orchestra.run.smithi148.stdout:Enabling module streams: 2023-09-30T14:26:43.751 INFO:teuthology.orchestra.run.smithi148.stdout: container-tools rhel8 2023-09-30T14:26:43.751 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-09-30T14:26:43.751 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2023-09-30T14:26:43.751 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================================================ 2023-09-30T14:26:43.751 INFO:teuthology.orchestra.run.smithi148.stdout:Install 10 Packages 2023-09-30T14:26:43.751 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-09-30T14:26:43.752 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 18 M 2023-09-30T14:26:43.752 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 65 M 2023-09-30T14:26:43.752 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2023-09-30T14:26:43.774 INFO:teuthology.orchestra.run.smithi156.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.2 MB/s | 94 kB 00:00 2023-09-30T14:26:43.799 INFO:teuthology.orchestra.run.smithi156.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 245 kB/s | 25 kB 00:00 2023-09-30T14:26:43.824 INFO:teuthology.orchestra.run.smithi156.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 322 kB/s | 41 kB 00:00 2023-09-30T14:26:43.834 INFO:teuthology.orchestra.run.smithi148.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 496 kB/s | 41 kB 00:00 2023-09-30T14:26:43.850 INFO:teuthology.orchestra.run.smithi156.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 654 kB/s | 49 kB 00:00 2023-09-30T14:26:43.859 INFO:teuthology.orchestra.run.smithi148.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 231 kB/s | 25 kB 00:00 2023-09-30T14:26:43.884 INFO:teuthology.orchestra.run.smithi148.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 707 kB/s | 94 kB 00:00 2023-09-30T14:26:43.891 INFO:teuthology.orchestra.run.smithi156.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.6 MB/s | 524 kB 00:00 2023-09-30T14:26:43.909 INFO:teuthology.orchestra.run.smithi148.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 656 kB/s | 49 kB 00:00 2023-09-30T14:26:43.917 INFO:teuthology.orchestra.run.smithi156.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-09-30T14:26:43.943 INFO:teuthology.orchestra.run.smithi148.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-09-30T14:26:43.967 INFO:teuthology.orchestra.run.smithi156.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.8 MB/s | 213 kB 00:00 2023-09-30T14:26:44.007 INFO:teuthology.orchestra.run.smithi148.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.5 MB/s | 149 kB 00:00 2023-09-30T14:26:44.038 INFO:teuthology.orchestra.run.smithi148.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.3 MB/s | 213 kB 00:00 2023-09-30T14:26:44.050 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2023-09-30T14:26:44.067 INFO:teuthology.orchestra.run.smithi156.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-09-30T14:26:44.128 INFO:teuthology.orchestra.run.smithi148.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-09-30T14:26:44.151 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================================================ 2023-09-30T14:26:44.151 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2023-09-30T14:26:44.151 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================================================ 2023-09-30T14:26:44.151 INFO:teuthology.orchestra.run.smithi032.stdout:Installing group/module packages: 2023-09-30T14:26:44.151 INFO:teuthology.orchestra.run.smithi032.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-30T14:26:44.151 INFO:teuthology.orchestra.run.smithi032.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-30T14:26:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-30T14:26:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-30T14:26:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-30T14:26:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-30T14:26:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-30T14:26:44.152 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2023-09-30T14:26:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-30T14:26:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-30T14:26:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-30T14:26:44.152 INFO:teuthology.orchestra.run.smithi032.stdout:Installing module profiles: 2023-09-30T14:26:44.152 INFO:teuthology.orchestra.run.smithi032.stdout: container-tools/common 2023-09-30T14:26:44.153 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:26:44.153 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling module streams: 2023-09-30T14:26:44.153 INFO:teuthology.orchestra.run.smithi032.stdout: container-tools rhel8 2023-09-30T14:26:44.153 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:26:44.153 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2023-09-30T14:26:44.153 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================================================ 2023-09-30T14:26:44.153 INFO:teuthology.orchestra.run.smithi032.stdout:Install 10 Packages 2023-09-30T14:26:44.153 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:26:44.154 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 18 M 2023-09-30T14:26:44.154 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 65 M 2023-09-30T14:26:44.154 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2023-09-30T14:26:44.176 INFO:teuthology.orchestra.run.smithi156.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 27 MB/s | 7.0 MB 00:00 2023-09-30T14:26:44.225 INFO:teuthology.orchestra.run.smithi032.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 352 kB/s | 25 kB 00:00 2023-09-30T14:26:44.244 INFO:teuthology.orchestra.run.smithi148.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 30 MB/s | 7.0 MB 00:00 2023-09-30T14:26:44.250 INFO:teuthology.orchestra.run.smithi032.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 425 kB/s | 41 kB 00:00 2023-09-30T14:26:44.267 INFO:teuthology.orchestra.run.smithi156.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-09-30T14:26:44.268 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2023-09-30T14:26:44.268 INFO:teuthology.orchestra.run.smithi156.stdout:Total 32 MB/s | 18 MB 00:00 2023-09-30T14:26:44.275 INFO:teuthology.orchestra.run.smithi032.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 774 kB/s | 94 kB 00:00 2023-09-30T14:26:44.300 INFO:teuthology.orchestra.run.smithi032.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 653 kB/s | 49 kB 00:00 2023-09-30T14:26:44.334 INFO:teuthology.orchestra.run.smithi032.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-09-30T14:26:44.336 INFO:teuthology.orchestra.run.smithi148.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-09-30T14:26:44.337 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2023-09-30T14:26:44.337 INFO:teuthology.orchestra.run.smithi148.stdout:Total 31 MB/s | 18 MB 00:00 2023-09-30T14:26:44.359 INFO:teuthology.orchestra.run.smithi032.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-09-30T14:26:44.384 INFO:teuthology.orchestra.run.smithi032.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.2 MB/s | 213 kB 00:00 2023-09-30T14:26:44.467 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2023-09-30T14:26:44.485 INFO:teuthology.orchestra.run.smithi032.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-09-30T14:26:44.486 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2023-09-30T14:26:44.486 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2023-09-30T14:26:44.539 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2023-09-30T14:26:44.557 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2023-09-30T14:26:44.558 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2023-09-30T14:26:44.602 INFO:teuthology.orchestra.run.smithi032.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 29 MB/s | 7.0 MB 00:00 2023-09-30T14:26:44.671 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2023-09-30T14:26:44.675 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2023-09-30T14:26:44.685 INFO:teuthology.orchestra.run.smithi032.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-09-30T14:26:44.686 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2023-09-30T14:26:44.686 INFO:teuthology.orchestra.run.smithi032.stdout:Total 34 MB/s | 18 MB 00:00 2023-09-30T14:26:44.739 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2023-09-30T14:26:44.743 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2023-09-30T14:26:44.891 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2023-09-30T14:26:44.912 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2023-09-30T14:26:44.912 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2023-09-30T14:26:45.106 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2023-09-30T14:26:45.110 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2023-09-30T14:26:45.174 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2023-09-30T14:26:45.269 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2023-09-30T14:26:45.345 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-30T14:26:45.433 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-30T14:26:45.506 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-30T14:26:45.611 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-30T14:26:45.612 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2023-09-30T14:26:45.699 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-30T14:26:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-30T14:26:45.792 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-30T14:26:45.883 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-30T14:26:45.971 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-30T14:26:45.974 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-30T14:26:46.154 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-30T14:26:46.235 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-30T14:26:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-30T14:26:46.345 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-30T14:26:46.732 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-30T14:26:47.054 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-30T14:26:47.222 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-30T14:26:47.513 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-30T14:26:47.914 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-30T14:26:48.097 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-30T14:26:48.142 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-30T14:26:48.320 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-30T14:26:48.326 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-30T14:26:48.418 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-30T14:26:48.471 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-30T14:26:48.520 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-30T14:26:48.625 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-30T14:26:48.639 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-30T14:26:48.789 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-30T14:26:48.793 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-30T14:26:48.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-30T14:26:48.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-30T14:26:48.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-30T14:26:48.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-30T14:26:48.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-30T14:26:48.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-30T14:26:48.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-30T14:26:48.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-30T14:26:48.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-30T14:26:48.905 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-30T14:26:48.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-30T14:26:48.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-30T14:26:48.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-30T14:26:48.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-30T14:26:48.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-30T14:26:48.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-30T14:26:48.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-30T14:26:48.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-30T14:26:48.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-30T14:26:49.008 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-30T14:26:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-30T14:26:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-30T14:26:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-30T14:26:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-30T14:26:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-30T14:26:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-30T14:26:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-30T14:26:49.295 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-30T14:26:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-30T14:26:49.296 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-30T14:26:49.985 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-30T14:26:49.985 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:54 ago on Sat 30 Sep 2023 02:24:55 PM UTC. 2023-09-30T14:26:50.054 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-30T14:26:50.054 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:01:55 ago on Sat 30 Sep 2023 02:24:54 PM UTC. 2023-09-30T14:26:50.422 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-30T14:26:50.431 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:45 ago on Sat 30 Sep 2023 02:25:04 PM UTC. 2023-09-30T14:26:53.359 INFO:teuthology.orchestra.run.smithi148.stdout:Installed products updated. 2023-09-30T14:26:53.549 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-09-30T14:26:53.549 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2023-09-30T14:26:53.549 INFO:teuthology.orchestra.run.smithi148.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-30T14:26:53.549 INFO:teuthology.orchestra.run.smithi148.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-30T14:26:53.549 INFO:teuthology.orchestra.run.smithi148.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-30T14:26:53.550 INFO:teuthology.orchestra.run.smithi148.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-30T14:26:53.550 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-30T14:26:53.550 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-30T14:26:53.550 INFO:teuthology.orchestra.run.smithi148.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-30T14:26:53.550 INFO:teuthology.orchestra.run.smithi148.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-30T14:26:53.550 INFO:teuthology.orchestra.run.smithi148.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-30T14:26:53.550 INFO:teuthology.orchestra.run.smithi148.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-30T14:26:53.550 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-09-30T14:26:53.550 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2023-09-30T14:26:53.584 INFO:teuthology.orchestra.run.smithi156.stdout:Installed products updated. 2023-09-30T14:26:53.691 DEBUG:teuthology.parallel:result is None 2023-09-30T14:26:53.766 INFO:teuthology.orchestra.run.smithi032.stdout:Installed products updated. 2023-09-30T14:26:53.782 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-30T14:26:53.782 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2023-09-30T14:26:53.782 INFO:teuthology.orchestra.run.smithi156.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-30T14:26:53.782 INFO:teuthology.orchestra.run.smithi156.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-30T14:26:53.782 INFO:teuthology.orchestra.run.smithi156.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-30T14:26:53.782 INFO:teuthology.orchestra.run.smithi156.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-30T14:26:53.782 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-30T14:26:53.783 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-30T14:26:53.783 INFO:teuthology.orchestra.run.smithi156.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-30T14:26:53.783 INFO:teuthology.orchestra.run.smithi156.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-30T14:26:53.783 INFO:teuthology.orchestra.run.smithi156.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-30T14:26:53.783 INFO:teuthology.orchestra.run.smithi156.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-30T14:26:53.783 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-30T14:26:53.783 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-09-30T14:26:53.957 DEBUG:teuthology.parallel:result is None 2023-09-30T14:26:53.998 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:26:53.999 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2023-09-30T14:26:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-30T14:26:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-30T14:26:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-30T14:26:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-30T14:26:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-30T14:26:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-30T14:26:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-30T14:26:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-30T14:26:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-30T14:26:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-30T14:26:53.999 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:26:54.000 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2023-09-30T14:26:54.153 DEBUG:teuthology.parallel:result is None 2023-09-30T14:26:54.153 INFO:teuthology.run_tasks:Running task install... 2023-09-30T14:26:54.162 DEBUG:teuthology.task.install:project ceph 2023-09-30T14:26:54.162 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1d10b71792f3be8887a7631e69851ac2df3585af'}} 2023-09-30T14:26:54.162 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1d10b71792f3be8887a7631e69851ac2df3585af'} 2023-09-30T14:26:54.162 INFO:teuthology.task.install:Using flavor: default 2023-09-30T14:26:54.168 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-30T14:26:54.168 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-09-30T14:26:54.168 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1d10b71792f3be8887a7631e69851ac2df3585af', 'tag': None, 'wait_for_package': False} 2023-09-30T14:26:54.169 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1d10b71792f3be8887a7631e69851ac2df3585af 2023-09-30T14:26:54.170 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1d10b71792f3be8887a7631e69851ac2df3585af', 'tag': None, 'wait_for_package': False} 2023-09-30T14:26:54.171 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1d10b71792f3be8887a7631e69851ac2df3585af 2023-09-30T14:26:54.172 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1d10b71792f3be8887a7631e69851ac2df3585af', 'tag': None, 'wait_for_package': False} 2023-09-30T14:26:54.172 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1d10b71792f3be8887a7631e69851ac2df3585af 2023-09-30T14:26:54.310 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/flavors/default/ 2023-09-30T14:26:54.310 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6447.g1d10b717 2023-09-30T14:26:54.317 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/flavors/default/ 2023-09-30T14:26:54.317 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6447.g1d10b717 2023-09-30T14:26:54.324 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/flavors/default/ 2023-09-30T14:26:54.325 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6447.g1d10b717 2023-09-30T14:26:54.420 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-30T14:26:54.420 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:26:54.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-30T14:26:54.439 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-30T14:26:54.439 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-09-30T14:26:54.439 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-30T14:26:54.449 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-30T14:26:54.449 DEBUG:teuthology.orchestra.run.smithi032:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1d10b71792f3be8887a7631e69851ac2df3585af/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-30T14:26:54.471 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-30T14:26:54.472 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1d10b71792f3be8887a7631e69851ac2df3585af/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-30T14:26:54.488 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-30T14:26:54.488 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-30T14:26:54.488 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-30T14:26:54.516 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-30T14:26:54.516 DEBUG:teuthology.orchestra.run.smithi156:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1d10b71792f3be8887a7631e69851ac2df3585af/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-30T14:26:54.517 DEBUG:teuthology.orchestra.run.smithi032:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-30T14:26:54.543 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-30T14:26:54.585 DEBUG:teuthology.orchestra.run.smithi156:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-30T14:26:54.595 DEBUG:teuthology.orchestra.run.smithi032:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-30T14:26:54.622 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-30T14:26:54.662 DEBUG:teuthology.orchestra.run.smithi156:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-30T14:26:54.662 INFO:teuthology.orchestra.run.smithi032.stdout:check_obsoletes = 1 2023-09-30T14:26:54.663 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum clean all 2023-09-30T14:26:54.692 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2023-09-30T14:26:54.693 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2023-09-30T14:26:54.727 INFO:teuthology.orchestra.run.smithi156.stdout:check_obsoletes = 1 2023-09-30T14:26:54.729 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum clean all 2023-09-30T14:26:55.056 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-09-30T14:26:55.070 INFO:teuthology.orchestra.run.smithi148.stdout:Updating Subscription Management repositories. 2023-09-30T14:26:55.101 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-09-30T14:26:55.335 INFO:teuthology.orchestra.run.smithi032.stdout:44 files removed 2023-09-30T14:26:55.356 INFO:teuthology.orchestra.run.smithi148.stdout:44 files removed 2023-09-30T14:26:55.374 INFO:teuthology.orchestra.run.smithi156.stdout:44 files removed 2023-09-30T14:26:55.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-30T14:26:55.403 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-30T14:26:55.421 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-30T14:26:55.734 INFO:teuthology.orchestra.run.smithi148.stdout:Updating Subscription Management repositories. 2023-09-30T14:26:55.735 INFO:teuthology.orchestra.run.smithi032.stdout:Updating Subscription Management repositories. 2023-09-30T14:26:55.755 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-09-30T14:26:56.428 INFO:teuthology.orchestra.run.smithi156.stdout:ceph packages for x86_64 322 kB/s | 81 kB 00:00 2023-09-30T14:26:56.471 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 273 kB/s | 81 kB 00:00 2023-09-30T14:26:56.504 INFO:teuthology.orchestra.run.smithi032.stdout:ceph packages for x86_64 279 kB/s | 81 kB 00:00 2023-09-30T14:26:56.796 INFO:teuthology.orchestra.run.smithi032.stdout:ceph noarch packages 36 kB/s | 9.4 kB 00:00 2023-09-30T14:26:56.805 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 31 kB/s | 9.4 kB 00:00 2023-09-30T14:26:56.812 INFO:teuthology.orchestra.run.smithi156.stdout:ceph noarch packages 27 kB/s | 9.4 kB 00:00 2023-09-30T14:26:57.085 INFO:teuthology.orchestra.run.smithi156.stdout:ceph source packages 5.9 kB/s | 1.5 kB 00:00 2023-09-30T14:26:57.147 INFO:teuthology.orchestra.run.smithi032.stdout:ceph source packages 4.6 kB/s | 1.5 kB 00:00 2023-09-30T14:26:57.164 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 4.4 kB/s | 1.5 kB 00:00 2023-09-30T14:26:57.230 INFO:teuthology.orchestra.run.smithi156.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-09-30T14:26:57.290 INFO:teuthology.orchestra.run.smithi032.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-09-30T14:26:57.315 INFO:teuthology.orchestra.run.smithi148.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-09-30T14:26:58.217 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 18 MB/s | 16 MB 00:00 2023-09-30T14:26:58.576 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2023-09-30T14:26:59.706 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 6.5 MB/s | 16 MB 00:02 2023-09-30T14:27:01.924 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 554 kB/s | 24 kB 00:00 2023-09-30T14:27:02.058 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 430 kB/s | 24 kB 00:00 2023-09-30T14:27:02.875 INFO:teuthology.orchestra.run.smithi032.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 54 MB/s | 47 MB 00:00 2023-09-30T14:27:02.943 INFO:teuthology.orchestra.run.smithi148.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-09-30T14:27:03.173 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 384 kB/s | 24 kB 00:00 2023-09-30T14:27:04.058 INFO:teuthology.orchestra.run.smithi156.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-09-30T14:27:10.927 INFO:teuthology.orchestra.run.smithi148.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 58 MB/s | 53 MB 00:00 2023-09-30T14:27:11.401 INFO:teuthology.orchestra.run.smithi032.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 55 MB/s | 53 MB 00:00 2023-09-30T14:27:11.984 INFO:teuthology.orchestra.run.smithi156.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 57 MB/s | 53 MB 00:00 2023-09-30T14:27:22.208 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T14:27:22.209 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T14:27:22.223 INFO:teuthology.orchestra.run.smithi148.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-30T14:27:22.224 INFO:teuthology.orchestra.run.smithi148.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-30T14:27:22.225 INFO:teuthology.orchestra.run.smithi148.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-30T14:27:22.226 INFO:teuthology.orchestra.run.smithi148.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-30T14:27:22.228 INFO:teuthology.orchestra.run.smithi148.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-30T14:27:22.231 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-30T14:27:22.232 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-30T14:27:22.365 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2023-09-30T14:27:22.377 INFO:teuthology.orchestra.run.smithi148.stdout:=========================================================================================================================== 2023-09-30T14:27:22.377 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2023-09-30T14:27:22.377 INFO:teuthology.orchestra.run.smithi148.stdout:=========================================================================================================================== 2023-09-30T14:27:22.377 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2023-09-30T14:27:22.377 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-30T14:27:22.377 INFO:teuthology.orchestra.run.smithi148.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-30T14:27:22.377 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 6.4 k 2023-09-30T14:27:22.377 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 5.4 M 2023-09-30T14:27:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 918 k 2023-09-30T14:27:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 167 k 2023-09-30T14:27:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 1.6 M 2023-09-30T14:27:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 132 k 2023-09-30T14:27:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 1.8 M 2023-09-30T14:27:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 7.4 M 2023-09-30T14:27:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 47 k 2023-09-30T14:27:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 14 M 2023-09-30T14:27:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 49 M 2023-09-30T14:27:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 261 k 2023-09-30T14:27:22.378 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 185 k 2023-09-30T14:27:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 261 k 2023-09-30T14:27:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 18 k 2023-09-30T14:27:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-30T14:27:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-30T14:27:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-30T14:27:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 31 k 2023-09-30T14:27:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 839 k 2023-09-30T14:27:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 139 k 2023-09-30T14:27:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-30T14:27:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-30T14:27:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-30T14:27:22.379 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-30T14:27:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-30T14:27:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-30T14:27:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 197 k 2023-09-30T14:27:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 371 k 2023-09-30T14:27:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 365 k 2023-09-30T14:27:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 106 k 2023-09-30T14:27:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 94 k 2023-09-30T14:27:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 3.6 M 2023-09-30T14:27:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 180 k 2023-09-30T14:27:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-30T14:27:22.380 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2023-09-30T14:27:22.380 INFO:teuthology.orchestra.run.smithi148.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-30T14:27:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 3.7 M 2023-09-30T14:27:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 3.7 M 2023-09-30T14:27:22.381 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2023-09-30T14:27:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-30T14:27:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 23 M 2023-09-30T14:27:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 23 k 2023-09-30T14:27:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 2.3 M 2023-09-30T14:27:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 242 k 2023-09-30T14:27:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 4.6 M 2023-09-30T14:27:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 18 M 2023-09-30T14:27:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 15 k 2023-09-30T14:27:22.381 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 24 k 2023-09-30T14:27:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-30T14:27:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-30T14:27:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-30T14:27:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-30T14:27:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-30T14:27:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 184 k 2023-09-30T14:27:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-30T14:27:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-30T14:27:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-30T14:27:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-30T14:27:22.382 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-30T14:27:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-30T14:27:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 519 k 2023-09-30T14:27:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-30T14:27:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 6.9 M 2023-09-30T14:27:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-30T14:27:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-30T14:27:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-30T14:27:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-30T14:27:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-30T14:27:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-30T14:27:22.383 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-30T14:27:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-30T14:27:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-30T14:27:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-30T14:27:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-30T14:27:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-30T14:27:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-30T14:27:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-30T14:27:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-30T14:27:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-30T14:27:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 45 k 2023-09-30T14:27:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 125 k 2023-09-30T14:27:22.384 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-30T14:27:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-30T14:27:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-30T14:27:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-30T14:27:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-30T14:27:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-30T14:27:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-30T14:27:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-30T14:27:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-30T14:27:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-30T14:27:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-30T14:27:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-30T14:27:22.385 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-30T14:27:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-30T14:27:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-30T14:27:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-30T14:27:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-30T14:27:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-30T14:27:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-30T14:27:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-30T14:27:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-30T14:27:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-30T14:27:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-30T14:27:22.386 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-30T14:27:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-30T14:27:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-30T14:27:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-30T14:27:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-30T14:27:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-30T14:27:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-30T14:27:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-30T14:27:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-30T14:27:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-30T14:27:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-30T14:27:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-30T14:27:22.387 INFO:teuthology.orchestra.run.smithi148.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-30T14:27:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-30T14:27:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-30T14:27:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-30T14:27:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-30T14:27:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-30T14:27:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-30T14:27:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-30T14:27:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-30T14:27:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-30T14:27:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-30T14:27:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-30T14:27:22.388 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-30T14:27:22.389 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-30T14:27:22.389 INFO:teuthology.orchestra.run.smithi148.stdout:Enabling module streams: 2023-09-30T14:27:22.389 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql 10 2023-09-30T14:27:22.389 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-09-30T14:27:22.389 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2023-09-30T14:27:22.389 INFO:teuthology.orchestra.run.smithi148.stdout:=========================================================================================================================== 2023-09-30T14:27:22.389 INFO:teuthology.orchestra.run.smithi148.stdout:Install 128 Packages 2023-09-30T14:27:22.389 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 3 Packages 2023-09-30T14:27:22.389 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-09-30T14:27:22.390 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 208 M 2023-09-30T14:27:22.390 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2023-09-30T14:27:22.710 INFO:teuthology.orchestra.run.smithi148.stdout:(1/131): ceph-18.0.0-6447.g1d10b717.el8.x86_64. 36 kB/s | 6.4 kB 00:00 2023-09-30T14:27:23.217 INFO:teuthology.orchestra.run.smithi156.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T14:27:23.218 INFO:teuthology.orchestra.run.smithi156.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T14:27:23.233 INFO:teuthology.orchestra.run.smithi156.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-30T14:27:23.234 INFO:teuthology.orchestra.run.smithi156.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-30T14:27:23.235 INFO:teuthology.orchestra.run.smithi156.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-30T14:27:23.236 INFO:teuthology.orchestra.run.smithi156.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-30T14:27:23.239 INFO:teuthology.orchestra.run.smithi156.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-30T14:27:23.242 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-30T14:27:23.243 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-30T14:27:23.250 INFO:teuthology.orchestra.run.smithi032.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T14:27:23.251 INFO:teuthology.orchestra.run.smithi032.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T14:27:23.265 INFO:teuthology.orchestra.run.smithi032.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-30T14:27:23.266 INFO:teuthology.orchestra.run.smithi032.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-30T14:27:23.267 INFO:teuthology.orchestra.run.smithi032.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-30T14:27:23.269 INFO:teuthology.orchestra.run.smithi032.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-30T14:27:23.271 INFO:teuthology.orchestra.run.smithi032.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-30T14:27:23.274 INFO:teuthology.orchestra.run.smithi032.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-30T14:27:23.275 INFO:teuthology.orchestra.run.smithi032.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-30T14:27:23.372 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-09-30T14:27:23.385 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-09-30T14:27:23.385 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2023-09-30T14:27:23.385 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-09-30T14:27:23.385 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2023-09-30T14:27:23.385 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-30T14:27:23.385 INFO:teuthology.orchestra.run.smithi156.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-30T14:27:23.385 INFO:teuthology.orchestra.run.smithi156.stdout: ceph x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 6.4 k 2023-09-30T14:27:23.385 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 5.4 M 2023-09-30T14:27:23.385 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 918 k 2023-09-30T14:27:23.386 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 167 k 2023-09-30T14:27:23.386 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 1.6 M 2023-09-30T14:27:23.386 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 132 k 2023-09-30T14:27:23.386 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 1.8 M 2023-09-30T14:27:23.386 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 7.4 M 2023-09-30T14:27:23.386 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 47 k 2023-09-30T14:27:23.386 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 14 M 2023-09-30T14:27:23.386 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 49 M 2023-09-30T14:27:23.386 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 261 k 2023-09-30T14:27:23.386 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 185 k 2023-09-30T14:27:23.387 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 261 k 2023-09-30T14:27:23.387 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 18 k 2023-09-30T14:27:23.387 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-30T14:27:23.387 INFO:teuthology.orchestra.run.smithi156.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-30T14:27:23.387 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-30T14:27:23.387 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 31 k 2023-09-30T14:27:23.387 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2 x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 839 k 2023-09-30T14:27:23.387 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 139 k 2023-09-30T14:27:23.387 INFO:teuthology.orchestra.run.smithi156.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-30T14:27:23.387 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-30T14:27:23.387 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-30T14:27:23.387 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-30T14:27:23.387 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-30T14:27:23.387 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-30T14:27:23.388 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 197 k 2023-09-30T14:27:23.388 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 371 k 2023-09-30T14:27:23.388 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 365 k 2023-09-30T14:27:23.388 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 106 k 2023-09-30T14:27:23.388 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 94 k 2023-09-30T14:27:23.388 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 3.6 M 2023-09-30T14:27:23.388 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 180 k 2023-09-30T14:27:23.388 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-30T14:27:23.388 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrading: 2023-09-30T14:27:23.388 INFO:teuthology.orchestra.run.smithi156.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-30T14:27:23.388 INFO:teuthology.orchestra.run.smithi156.stdout: librados2 x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 3.7 M 2023-09-30T14:27:23.388 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1 x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 3.7 M 2023-09-30T14:27:23.389 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2023-09-30T14:27:23.389 INFO:teuthology.orchestra.run.smithi156.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-30T14:27:23.389 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 23 M 2023-09-30T14:27:23.389 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 23 k 2023-09-30T14:27:23.389 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 2.3 M 2023-09-30T14:27:23.389 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 242 k 2023-09-30T14:27:23.389 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 4.6 M 2023-09-30T14:27:23.389 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 18 M 2023-09-30T14:27:23.389 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 15 k 2023-09-30T14:27:23.389 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 24 k 2023-09-30T14:27:23.389 INFO:teuthology.orchestra.run.smithi156.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-30T14:27:23.389 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-30T14:27:23.389 INFO:teuthology.orchestra.run.smithi156.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-30T14:27:23.390 INFO:teuthology.orchestra.run.smithi156.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-30T14:27:23.390 INFO:teuthology.orchestra.run.smithi156.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-30T14:27:23.390 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 184 k 2023-09-30T14:27:23.390 INFO:teuthology.orchestra.run.smithi156.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-30T14:27:23.390 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-30T14:27:23.390 INFO:teuthology.orchestra.run.smithi156.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-30T14:27:23.390 INFO:teuthology.orchestra.run.smithi156.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-30T14:27:23.390 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-30T14:27:23.390 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-30T14:27:23.390 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1 x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 519 k 2023-09-30T14:27:23.391 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-30T14:27:23.391 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2 x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 6.9 M 2023-09-30T14:27:23.391 INFO:teuthology.orchestra.run.smithi156.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-30T14:27:23.391 INFO:teuthology.orchestra.run.smithi156.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-30T14:27:23.391 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-30T14:27:23.391 INFO:teuthology.orchestra.run.smithi156.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-30T14:27:23.391 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-30T14:27:23.391 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-30T14:27:23.391 INFO:teuthology.orchestra.run.smithi156.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-30T14:27:23.391 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-30T14:27:23.391 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-30T14:27:23.391 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-30T14:27:23.391 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-30T14:27:23.392 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-30T14:27:23.392 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-30T14:27:23.392 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-30T14:27:23.392 INFO:teuthology.orchestra.run.smithi156.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-30T14:27:23.392 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-30T14:27:23.392 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 45 k 2023-09-30T14:27:23.392 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 125 k 2023-09-30T14:27:23.392 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-30T14:27:23.392 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-30T14:27:23.392 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-30T14:27:23.392 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-30T14:27:23.392 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-30T14:27:23.393 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-30T14:27:23.393 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-30T14:27:23.393 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-30T14:27:23.393 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-30T14:27:23.393 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-30T14:27:23.393 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-30T14:27:23.393 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-30T14:27:23.393 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-30T14:27:23.393 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-30T14:27:23.393 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-30T14:27:23.393 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-30T14:27:23.393 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-30T14:27:23.393 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-30T14:27:23.394 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-30T14:27:23.394 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-30T14:27:23.394 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-30T14:27:23.394 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-30T14:27:23.394 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-30T14:27:23.394 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-30T14:27:23.394 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-30T14:27:23.394 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-30T14:27:23.394 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-30T14:27:23.394 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-30T14:27:23.394 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-30T14:27:23.395 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-30T14:27:23.395 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-30T14:27:23.395 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-30T14:27:23.395 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-30T14:27:23.395 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-30T14:27:23.395 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-30T14:27:23.395 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-30T14:27:23.395 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-30T14:27:23.395 INFO:teuthology.orchestra.run.smithi156.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-30T14:27:23.395 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-30T14:27:23.395 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-30T14:27:23.395 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-30T14:27:23.396 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-30T14:27:23.396 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-30T14:27:23.396 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-30T14:27:23.396 INFO:teuthology.orchestra.run.smithi156.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-30T14:27:23.396 INFO:teuthology.orchestra.run.smithi156.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-30T14:27:23.396 INFO:teuthology.orchestra.run.smithi156.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-30T14:27:23.396 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-30T14:27:23.396 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-30T14:27:23.396 INFO:teuthology.orchestra.run.smithi156.stdout:Enabling module streams: 2023-09-30T14:27:23.396 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql 10 2023-09-30T14:27:23.396 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-30T14:27:23.396 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-09-30T14:27:23.397 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-09-30T14:27:23.397 INFO:teuthology.orchestra.run.smithi156.stdout:Install 128 Packages 2023-09-30T14:27:23.397 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrade 3 Packages 2023-09-30T14:27:23.397 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-30T14:27:23.397 INFO:teuthology.orchestra.run.smithi148.stdout:(2/131): ceph-fuse-18.0.0-6447.g1d10b717.el8.x8 1.3 MB/s | 918 kB 00:00 2023-09-30T14:27:23.397 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 208 M 2023-09-30T14:27:23.398 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2023-09-30T14:27:23.419 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2023-09-30T14:27:23.433 INFO:teuthology.orchestra.run.smithi032.stdout:=========================================================================================================================== 2023-09-30T14:27:23.433 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2023-09-30T14:27:23.433 INFO:teuthology.orchestra.run.smithi032.stdout:=========================================================================================================================== 2023-09-30T14:27:23.433 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2023-09-30T14:27:23.433 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-30T14:27:23.433 INFO:teuthology.orchestra.run.smithi032.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-30T14:27:23.433 INFO:teuthology.orchestra.run.smithi032.stdout: ceph x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 6.4 k 2023-09-30T14:27:23.433 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 5.4 M 2023-09-30T14:27:23.433 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 918 k 2023-09-30T14:27:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 167 k 2023-09-30T14:27:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 1.6 M 2023-09-30T14:27:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 132 k 2023-09-30T14:27:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 1.8 M 2023-09-30T14:27:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 7.4 M 2023-09-30T14:27:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 47 k 2023-09-30T14:27:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 14 M 2023-09-30T14:27:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 49 M 2023-09-30T14:27:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-volume noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 261 k 2023-09-30T14:27:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 185 k 2023-09-30T14:27:23.434 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 261 k 2023-09-30T14:27:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-top noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 18 k 2023-09-30T14:27:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-30T14:27:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-30T14:27:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-30T14:27:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 31 k 2023-09-30T14:27:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2 x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 839 k 2023-09-30T14:27:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 139 k 2023-09-30T14:27:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-30T14:27:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-30T14:27:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-30T14:27:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-30T14:27:23.435 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-30T14:27:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-30T14:27:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 197 k 2023-09-30T14:27:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 371 k 2023-09-30T14:27:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 365 k 2023-09-30T14:27:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 106 k 2023-09-30T14:27:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 94 k 2023-09-30T14:27:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 3.6 M 2023-09-30T14:27:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 180 k 2023-09-30T14:27:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-30T14:27:23.436 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrading: 2023-09-30T14:27:23.436 INFO:teuthology.orchestra.run.smithi032.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-30T14:27:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 3.7 M 2023-09-30T14:27:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1 x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 3.7 M 2023-09-30T14:27:23.437 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2023-09-30T14:27:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-30T14:27:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 23 M 2023-09-30T14:27:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 23 k 2023-09-30T14:27:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 2.3 M 2023-09-30T14:27:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 242 k 2023-09-30T14:27:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 4.6 M 2023-09-30T14:27:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 18 M 2023-09-30T14:27:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6447.g1d10b717.el8 ceph-noarch 15 k 2023-09-30T14:27:23.437 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 24 k 2023-09-30T14:27:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-30T14:27:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-30T14:27:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-30T14:27:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-30T14:27:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-30T14:27:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 184 k 2023-09-30T14:27:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-30T14:27:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-30T14:27:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-30T14:27:23.438 INFO:teuthology.orchestra.run.smithi032.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-30T14:27:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-30T14:27:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-30T14:27:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1 x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 519 k 2023-09-30T14:27:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-30T14:27:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2 x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 6.9 M 2023-09-30T14:27:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-30T14:27:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-30T14:27:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-30T14:27:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-30T14:27:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-30T14:27:23.439 INFO:teuthology.orchestra.run.smithi032.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-30T14:27:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-30T14:27:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-30T14:27:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-30T14:27:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-30T14:27:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-30T14:27:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-30T14:27:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-30T14:27:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-30T14:27:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-30T14:27:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-30T14:27:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 45 k 2023-09-30T14:27:23.440 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common x86_64 2:18.0.0-6447.g1d10b717.el8 ceph 125 k 2023-09-30T14:27:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-30T14:27:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-30T14:27:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-30T14:27:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-30T14:27:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-30T14:27:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-30T14:27:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-30T14:27:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-30T14:27:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-30T14:27:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-30T14:27:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-30T14:27:23.441 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-30T14:27:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-30T14:27:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-30T14:27:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-30T14:27:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-30T14:27:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-30T14:27:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-30T14:27:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-30T14:27:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-30T14:27:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-30T14:27:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-30T14:27:23.442 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-30T14:27:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-30T14:27:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-30T14:27:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-30T14:27:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-30T14:27:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-30T14:27:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-30T14:27:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-30T14:27:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-30T14:27:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-30T14:27:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-30T14:27:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-30T14:27:23.443 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-30T14:27:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-30T14:27:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-30T14:27:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-30T14:27:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-30T14:27:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-30T14:27:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-30T14:27:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-30T14:27:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-30T14:27:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-30T14:27:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-30T14:27:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-30T14:27:23.444 INFO:teuthology.orchestra.run.smithi032.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-30T14:27:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-30T14:27:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-30T14:27:23.445 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling module streams: 2023-09-30T14:27:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql 10 2023-09-30T14:27:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:27:23.445 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2023-09-30T14:27:23.445 INFO:teuthology.orchestra.run.smithi032.stdout:=========================================================================================================================== 2023-09-30T14:27:23.445 INFO:teuthology.orchestra.run.smithi032.stdout:Install 128 Packages 2023-09-30T14:27:23.445 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrade 3 Packages 2023-09-30T14:27:23.445 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:27:23.446 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 208 M 2023-09-30T14:27:23.446 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2023-09-30T14:27:23.594 INFO:teuthology.orchestra.run.smithi148.stdout:(3/131): ceph-immutable-object-cache-18.0.0-644 835 kB/s | 167 kB 00:00 2023-09-30T14:27:23.753 INFO:teuthology.orchestra.run.smithi148.stdout:(4/131): ceph-base-18.0.0-6447.g1d10b717.el8.x8 4.4 MB/s | 5.4 MB 00:01 2023-09-30T14:27:23.801 INFO:teuthology.orchestra.run.smithi156.stdout:(1/131): ceph-18.0.0-6447.g1d10b717.el8.x86_64. 26 kB/s | 6.4 kB 00:00 2023-09-30T14:27:23.852 INFO:teuthology.orchestra.run.smithi032.stdout:(1/131): ceph-18.0.0-6447.g1d10b717.el8.x86_64. 18 kB/s | 6.4 kB 00:00 2023-09-30T14:27:23.970 INFO:teuthology.orchestra.run.smithi148.stdout:(5/131): ceph-common-18.0.0-6447.g1d10b717.el8. 16 MB/s | 23 MB 00:01 2023-09-30T14:27:24.028 INFO:teuthology.orchestra.run.smithi148.stdout:(6/131): ceph-mgr-18.0.0-6447.g1d10b717.el8.x86 5.9 MB/s | 1.6 MB 00:00 2023-09-30T14:27:24.494 INFO:teuthology.orchestra.run.smithi032.stdout:(2/131): ceph-base-18.0.0-6447.g1d10b717.el8.x8 5.5 MB/s | 5.4 MB 00:00 2023-09-30T14:27:24.586 INFO:teuthology.orchestra.run.smithi032.stdout:(3/131): ceph-immutable-object-cache-18.0.0-644 1.8 MB/s | 167 kB 00:00 2023-09-30T14:27:24.668 INFO:teuthology.orchestra.run.smithi156.stdout:(2/131): ceph-fuse-18.0.0-6447.g1d10b717.el8.x8 1.0 MB/s | 918 kB 00:00 2023-09-30T14:27:24.893 INFO:teuthology.orchestra.run.smithi156.stdout:(3/131): ceph-immutable-object-cache-18.0.0-644 743 kB/s | 167 kB 00:00 2023-09-30T14:27:24.953 INFO:teuthology.orchestra.run.smithi032.stdout:(4/131): ceph-mds-18.0.0-6447.g1d10b717.el8.x86 6.3 MB/s | 2.3 MB 00:00 2023-09-30T14:27:25.004 INFO:teuthology.orchestra.run.smithi148.stdout:(7/131): ceph-mon-18.0.0-6447.g1d10b717.el8.x86 4.5 MB/s | 4.6 MB 00:01 2023-09-30T14:27:25.270 INFO:teuthology.orchestra.run.smithi032.stdout:(5/131): ceph-mgr-18.0.0-6447.g1d10b717.el8.x86 5.1 MB/s | 1.6 MB 00:00 2023-09-30T14:27:25.337 INFO:teuthology.orchestra.run.smithi148.stdout:(8/131): ceph-mds-18.0.0-6447.g1d10b717.el8.x86 1.3 MB/s | 2.3 MB 00:01 2023-09-30T14:27:25.454 INFO:teuthology.orchestra.run.smithi148.stdout:(9/131): ceph-selinux-18.0.0-6447.g1d10b717.el8 208 kB/s | 24 kB 00:00 2023-09-30T14:27:25.762 INFO:teuthology.orchestra.run.smithi032.stdout:(6/131): ceph-fuse-18.0.0-6447.g1d10b717.el8.x8 481 kB/s | 918 kB 00:01 2023-09-30T14:27:26.079 INFO:teuthology.orchestra.run.smithi032.stdout:(7/131): ceph-mon-18.0.0-6447.g1d10b717.el8.x86 5.7 MB/s | 4.6 MB 00:00 2023-09-30T14:27:26.460 INFO:teuthology.orchestra.run.smithi156.stdout:(4/131): ceph-common-18.0.0-6447.g1d10b717.el8. 7.8 MB/s | 23 MB 00:02 2023-09-30T14:27:26.785 INFO:teuthology.orchestra.run.smithi156.stdout:(5/131): ceph-mgr-18.0.0-6447.g1d10b717.el8.x86 5.0 MB/s | 1.6 MB 00:00 2023-09-30T14:27:27.304 INFO:teuthology.orchestra.run.smithi148.stdout:(10/131): ceph-radosgw-18.0.0-6447.g1d10b717.el 6.0 MB/s | 14 MB 00:02 2023-09-30T14:27:27.396 INFO:teuthology.orchestra.run.smithi148.stdout:(11/131): cephfs-mirror-18.0.0-6447.g1d10b717.e 2.8 MB/s | 261 kB 00:00 2023-09-30T14:27:27.469 INFO:teuthology.orchestra.run.smithi156.stdout:(6/131): ceph-mds-18.0.0-6447.g1d10b717.el8.x86 918 kB/s | 2.3 MB 00:02 2023-09-30T14:27:27.480 INFO:teuthology.orchestra.run.smithi148.stdout:(12/131): libcephfs-devel-18.0.0-6447.g1d10b717 371 kB/s | 31 kB 00:00 2023-09-30T14:27:27.536 INFO:teuthology.orchestra.run.smithi156.stdout:(7/131): ceph-mon-18.0.0-6447.g1d10b717.el8.x86 6.2 MB/s | 4.6 MB 00:00 2023-09-30T14:27:27.622 INFO:teuthology.orchestra.run.smithi148.stdout:(13/131): libcephfs2-18.0.0-6447.g1d10b717.el8. 5.8 MB/s | 839 kB 00:00 2023-09-30T14:27:27.706 INFO:teuthology.orchestra.run.smithi148.stdout:(14/131): libcephsqlite-18.0.0-6447.g1d10b717.e 2.2 MB/s | 184 kB 00:00 2023-09-30T14:27:27.789 INFO:teuthology.orchestra.run.smithi148.stdout:(15/131): librados-devel-18.0.0-6447.g1d10b717. 1.6 MB/s | 139 kB 00:00 2023-09-30T14:27:27.906 INFO:teuthology.orchestra.run.smithi148.stdout:(16/131): libradosstriper1-18.0.0-6447.g1d10b71 4.4 MB/s | 519 kB 00:00 2023-09-30T14:27:27.971 INFO:teuthology.orchestra.run.smithi032.stdout:(8/131): ceph-radosgw-18.0.0-6447.g1d10b717.el8 7.3 MB/s | 14 MB 00:01 2023-09-30T14:27:28.054 INFO:teuthology.orchestra.run.smithi032.stdout:(9/131): ceph-selinux-18.0.0-6447.g1d10b717.el8 292 kB/s | 24 kB 00:00 2023-09-30T14:27:28.415 INFO:teuthology.orchestra.run.smithi148.stdout:(17/131): ceph-osd-18.0.0-6447.g1d10b717.el8.x8 4.1 MB/s | 18 MB 00:04 2023-09-30T14:27:28.523 INFO:teuthology.orchestra.run.smithi148.stdout:(18/131): python3-ceph-argparse-18.0.0-6447.g1d 414 kB/s | 45 kB 00:00 2023-09-30T14:27:28.632 INFO:teuthology.orchestra.run.smithi148.stdout:(19/131): python3-ceph-common-18.0.0-6447.g1d10 1.1 MB/s | 125 kB 00:00 2023-09-30T14:27:28.740 INFO:teuthology.orchestra.run.smithi148.stdout:(20/131): python3-cephfs-18.0.0-6447.g1d10b717. 1.8 MB/s | 197 kB 00:00 2023-09-30T14:27:28.958 INFO:teuthology.orchestra.run.smithi148.stdout:(21/131): python3-rados-18.0.0-6447.g1d10b717.e 1.7 MB/s | 371 kB 00:00 2023-09-30T14:27:29.041 INFO:teuthology.orchestra.run.smithi148.stdout:(22/131): librgw2-18.0.0-6447.g1d10b717.el8.x86 6.1 MB/s | 6.9 MB 00:01 2023-09-30T14:27:29.117 INFO:teuthology.orchestra.run.smithi148.stdout:(23/131): python3-rgw-18.0.0-6447.g1d10b717.el8 1.4 MB/s | 106 kB 00:00 2023-09-30T14:27:29.142 INFO:teuthology.orchestra.run.smithi148.stdout:(24/131): python3-rbd-18.0.0-6447.g1d10b717.el8 1.9 MB/s | 365 kB 00:00 2023-09-30T14:27:29.192 INFO:teuthology.orchestra.run.smithi148.stdout:(25/131): rbd-fuse-18.0.0-6447.g1d10b717.el8.x8 1.2 MB/s | 94 kB 00:00 2023-09-30T14:27:29.284 INFO:teuthology.orchestra.run.smithi148.stdout:(26/131): rbd-nbd-18.0.0-6447.g1d10b717.el8.x86 1.9 MB/s | 180 kB 00:00 2023-09-30T14:27:29.359 INFO:teuthology.orchestra.run.smithi148.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6447.g 307 kB/s | 23 kB 00:00 2023-09-30T14:27:29.451 INFO:teuthology.orchestra.run.smithi148.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6447.g1d10b71 1.4 MB/s | 132 kB 00:00 2023-09-30T14:27:29.603 INFO:teuthology.orchestra.run.smithi156.stdout:(8/131): ceph-radosgw-18.0.0-6447.g1d10b717.el8 6.7 MB/s | 14 MB 00:02 2023-09-30T14:27:29.670 INFO:teuthology.orchestra.run.smithi156.stdout:(9/131): ceph-base-18.0.0-6447.g1d10b717.el8.x8 905 kB/s | 5.4 MB 00:06 2023-09-30T14:27:29.695 INFO:teuthology.orchestra.run.smithi156.stdout:(10/131): ceph-selinux-18.0.0-6447.g1d10b717.el 265 kB/s | 24 kB 00:00 2023-09-30T14:27:29.727 INFO:teuthology.orchestra.run.smithi148.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6447.g1d10b 6.4 MB/s | 1.8 MB 00:00 2023-09-30T14:27:29.800 INFO:teuthology.orchestra.run.smithi156.stdout:(11/131): cephfs-mirror-18.0.0-6447.g1d10b717.e 2.4 MB/s | 261 kB 00:00 2023-09-30T14:27:29.887 INFO:teuthology.orchestra.run.smithi156.stdout:(12/131): libcephfs-devel-18.0.0-6447.g1d10b717 354 kB/s | 31 kB 00:00 2023-09-30T14:27:30.002 INFO:teuthology.orchestra.run.smithi148.stdout:(30/131): rbd-mirror-18.0.0-6447.g1d10b717.el8. 4.2 MB/s | 3.6 MB 00:00 2023-09-30T14:27:30.062 INFO:teuthology.orchestra.run.smithi156.stdout:(13/131): libcephfs2-18.0.0-6447.g1d10b717.el8. 4.7 MB/s | 839 kB 00:00 2023-09-30T14:27:30.119 INFO:teuthology.orchestra.run.smithi148.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6447.g1d 2.0 MB/s | 242 kB 00:00 2023-09-30T14:27:30.155 INFO:teuthology.orchestra.run.smithi156.stdout:(14/131): libcephsqlite-18.0.0-6447.g1d10b717.e 2.0 MB/s | 184 kB 00:00 2023-09-30T14:27:30.219 INFO:teuthology.orchestra.run.smithi148.stdout:(32/131): ceph-mgr-rook-18.0.0-6447.g1d10b717.e 476 kB/s | 47 kB 00:00 2023-09-30T14:27:30.238 INFO:teuthology.orchestra.run.smithi156.stdout:(15/131): librados-devel-18.0.0-6447.g1d10b717. 1.6 MB/s | 139 kB 00:00 2023-09-30T14:27:30.303 INFO:teuthology.orchestra.run.smithi148.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6447.g1 174 kB/s | 15 kB 00:00 2023-09-30T14:27:30.380 INFO:teuthology.orchestra.run.smithi156.stdout:(16/131): libradosstriper1-18.0.0-6447.g1d10b71 3.6 MB/s | 519 kB 00:00 2023-09-30T14:27:30.403 INFO:teuthology.orchestra.run.smithi148.stdout:(34/131): ceph-volume-18.0.0-6447.g1d10b717.el8 2.6 MB/s | 261 kB 00:00 2023-09-30T14:27:30.512 INFO:teuthology.orchestra.run.smithi148.stdout:(35/131): cephadm-18.0.0-6447.g1d10b717.el8.noa 1.7 MB/s | 185 kB 00:00 2023-09-30T14:27:30.612 INFO:teuthology.orchestra.run.smithi148.stdout:(36/131): cephfs-top-18.0.0-6447.g1d10b717.el8. 181 kB/s | 18 kB 00:00 2023-09-30T14:27:30.829 INFO:teuthology.orchestra.run.smithi148.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 6.7 MB/s | 7.4 MB 00:01 2023-09-30T14:27:30.871 INFO:teuthology.orchestra.run.smithi148.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-09-30T14:27:31.047 INFO:teuthology.orchestra.run.smithi148.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 439 kB/s | 75 kB 00:00 2023-09-30T14:27:31.130 INFO:teuthology.orchestra.run.smithi148.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 196 kB/s | 59 kB 00:00 2023-09-30T14:27:31.155 INFO:teuthology.orchestra.run.smithi148.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.3 MB/s | 476 kB 00:00 2023-09-30T14:27:31.181 INFO:teuthology.orchestra.run.smithi148.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 871 kB/s | 44 kB 00:00 2023-09-30T14:27:31.223 INFO:teuthology.orchestra.run.smithi148.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.7 MB/s | 185 kB 00:00 2023-09-30T14:27:31.248 INFO:teuthology.orchestra.run.smithi148.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 492 kB/s | 33 kB 00:00 2023-09-30T14:27:31.273 INFO:teuthology.orchestra.run.smithi148.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2023-09-30T14:27:31.315 INFO:teuthology.orchestra.run.smithi148.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2023-09-30T14:27:31.347 INFO:teuthology.orchestra.run.smithi156.stdout:(17/131): librgw2-18.0.0-6447.g1d10b717.el8.x86 7.2 MB/s | 6.9 MB 00:00 2023-09-30T14:27:31.365 INFO:teuthology.orchestra.run.smithi148.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-09-30T14:27:31.399 INFO:teuthology.orchestra.run.smithi148.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.0 MB/s | 384 kB 00:00 2023-09-30T14:27:31.405 INFO:teuthology.orchestra.run.smithi156.stdout:(18/131): python3-ceph-argparse-18.0.0-6447.g1d 773 kB/s | 45 kB 00:00 2023-09-30T14:27:31.424 INFO:teuthology.orchestra.run.smithi148.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-09-30T14:27:31.458 INFO:teuthology.orchestra.run.smithi148.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 301 kB/s | 18 kB 00:00 2023-09-30T14:27:31.464 INFO:teuthology.orchestra.run.smithi156.stdout:(19/131): python3-ceph-common-18.0.0-6447.g1d10 2.1 MB/s | 125 kB 00:00 2023-09-30T14:27:31.500 INFO:teuthology.orchestra.run.smithi148.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2023-09-30T14:27:31.531 INFO:teuthology.orchestra.run.smithi156.stdout:(20/131): python3-cephfs-18.0.0-6447.g1d10b717. 2.9 MB/s | 197 kB 00:00 2023-09-30T14:27:31.542 INFO:teuthology.orchestra.run.smithi148.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 10 MB/s | 1.2 MB 00:00 2023-09-30T14:27:31.567 INFO:teuthology.orchestra.run.smithi148.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 879 kB/s | 59 kB 00:00 2023-09-30T14:27:31.606 INFO:teuthology.orchestra.run.smithi156.stdout:(21/131): python3-rados-18.0.0-6447.g1d10b717.e 4.8 MB/s | 371 kB 00:00 2023-09-30T14:27:31.615 INFO:teuthology.orchestra.run.smithi148.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 823 kB/s | 60 kB 00:00 2023-09-30T14:27:31.660 INFO:teuthology.orchestra.run.smithi148.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 357 kB/s | 16 kB 00:00 2023-09-30T14:27:31.682 INFO:teuthology.orchestra.run.smithi156.stdout:(22/131): python3-rbd-18.0.0-6447.g1d10b717.el8 4.8 MB/s | 365 kB 00:00 2023-09-30T14:27:31.684 INFO:teuthology.orchestra.run.smithi148.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2023-09-30T14:27:31.710 INFO:teuthology.orchestra.run.smithi148.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 672 kB/s | 34 kB 00:00 2023-09-30T14:27:31.740 INFO:teuthology.orchestra.run.smithi156.stdout:(23/131): python3-rgw-18.0.0-6447.g1d10b717.el8 1.8 MB/s | 106 kB 00:00 2023-09-30T14:27:31.760 INFO:teuthology.orchestra.run.smithi148.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-09-30T14:27:31.785 INFO:teuthology.orchestra.run.smithi148.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2023-09-30T14:27:31.799 INFO:teuthology.orchestra.run.smithi156.stdout:(24/131): rbd-fuse-18.0.0-6447.g1d10b717.el8.x8 1.6 MB/s | 94 kB 00:00 2023-09-30T14:27:31.811 INFO:teuthology.orchestra.run.smithi148.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 374 kB/s | 19 kB 00:00 2023-09-30T14:27:31.836 INFO:teuthology.orchestra.run.smithi148.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 477 kB/s | 24 kB 00:00 2023-09-30T14:27:31.861 INFO:teuthology.orchestra.run.smithi148.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 581 kB/s | 29 kB 00:00 2023-09-30T14:27:31.878 INFO:teuthology.orchestra.run.smithi148.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 657 kB/s | 27 kB 00:00 2023-09-30T14:27:31.912 INFO:teuthology.orchestra.run.smithi148.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-09-30T14:27:31.954 INFO:teuthology.orchestra.run.smithi148.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 1.4 MB/s | 61 kB 00:00 2023-09-30T14:27:31.996 INFO:teuthology.orchestra.run.smithi148.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2023-09-30T14:27:32.021 INFO:teuthology.orchestra.run.smithi148.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.3 MB/s | 85 kB 00:00 2023-09-30T14:27:32.047 INFO:teuthology.orchestra.run.smithi148.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 456 kB/s | 23 kB 00:00 2023-09-30T14:27:32.072 INFO:teuthology.orchestra.run.smithi148.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-09-30T14:27:32.114 INFO:teuthology.orchestra.run.smithi148.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 19 MB/s | 1.7 MB 00:00 2023-09-30T14:27:32.174 INFO:teuthology.orchestra.run.smithi148.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.0 MB/s | 110 kB 00:00 2023-09-30T14:27:32.206 INFO:teuthology.orchestra.run.smithi148.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-09-30T14:27:32.248 INFO:teuthology.orchestra.run.smithi148.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-09-30T14:27:32.274 INFO:teuthology.orchestra.run.smithi156.stdout:(25/131): rbd-mirror-18.0.0-6447.g1d10b717.el8. 7.5 MB/s | 3.6 MB 00:00 2023-09-30T14:27:32.290 INFO:teuthology.orchestra.run.smithi148.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-09-30T14:27:32.332 INFO:teuthology.orchestra.run.smithi148.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 934 kB/s | 39 kB 00:00 2023-09-30T14:27:32.341 INFO:teuthology.orchestra.run.smithi156.stdout:(26/131): rbd-nbd-18.0.0-6447.g1d10b717.el8.x86 2.6 MB/s | 180 kB 00:00 2023-09-30T14:27:32.391 INFO:teuthology.orchestra.run.smithi148.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-09-30T14:27:32.391 INFO:teuthology.orchestra.run.smithi156.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6447.g 463 kB/s | 23 kB 00:00 2023-09-30T14:27:32.441 INFO:teuthology.orchestra.run.smithi148.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 11 MB/s | 538 kB 00:00 2023-09-30T14:27:32.467 INFO:teuthology.orchestra.run.smithi156.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6447.g1d10b71 1.7 MB/s | 132 kB 00:00 2023-09-30T14:27:32.483 INFO:teuthology.orchestra.run.smithi148.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-09-30T14:27:32.625 INFO:teuthology.orchestra.run.smithi148.stdout:(79/131): python3-scipy-1.0.0-21.module+el8.5.0 25 MB/s | 14 MB 00:00 2023-09-30T14:27:32.650 INFO:teuthology.orchestra.run.smithi148.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2023-09-30T14:27:32.692 INFO:teuthology.orchestra.run.smithi148.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-09-30T14:27:32.709 INFO:teuthology.orchestra.run.smithi156.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6447.g1d10b 7.3 MB/s | 1.8 MB 00:00 2023-09-30T14:27:32.734 INFO:teuthology.orchestra.run.smithi148.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 635 kB/s | 27 kB 00:00 2023-09-30T14:27:32.776 INFO:teuthology.orchestra.run.smithi148.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-09-30T14:27:32.868 INFO:teuthology.orchestra.run.smithi148.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 22 MB/s | 4.8 MB 00:00 2023-09-30T14:27:32.885 INFO:teuthology.orchestra.run.smithi148.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 404 kB/s | 44 kB 00:00 2023-09-30T14:27:32.935 INFO:teuthology.orchestra.run.smithi148.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 8.9 MB/s | 457 kB 00:00 2023-09-30T14:27:33.011 INFO:teuthology.orchestra.run.smithi148.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 26 MB/s | 3.7 MB 00:00 2023-09-30T14:27:33.036 INFO:teuthology.orchestra.run.smithi148.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.2 MB/s | 225 kB 00:00 2023-09-30T14:27:33.053 INFO:teuthology.orchestra.run.smithi148.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-09-30T14:27:33.078 INFO:teuthology.orchestra.run.smithi148.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.9 MB/s | 126 kB 00:00 2023-09-30T14:27:33.103 INFO:teuthology.orchestra.run.smithi148.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 397 kB/s | 20 kB 00:00 2023-09-30T14:27:33.120 INFO:teuthology.orchestra.run.smithi148.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2023-09-30T14:27:33.154 INFO:teuthology.orchestra.run.smithi148.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 14 MB/s | 709 kB 00:00 2023-09-30T14:27:33.197 INFO:teuthology.orchestra.run.smithi148.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-09-30T14:27:33.288 INFO:teuthology.orchestra.run.smithi148.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 29 MB/s | 4.9 MB 00:00 2023-09-30T14:27:33.380 INFO:teuthology.orchestra.run.smithi148.stdout:(96/131): openblas-0.3.15-3.el8.x86_64.rpm 26 MB/s | 4.7 MB 00:00 2023-09-30T14:27:33.431 INFO:teuthology.orchestra.run.smithi148.stdout:(97/131): jq-1.6-3.el8.x86_64.rpm 1.4 MB/s | 202 kB 00:00 2023-09-30T14:27:33.463 INFO:teuthology.orchestra.run.smithi032.stdout:(10/131): ceph-test-18.0.0-6447.g1d10b717.el8.x 9.0 MB/s | 49 MB 00:05 2023-09-30T14:27:33.464 INFO:teuthology.orchestra.run.smithi148.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 4.1 MB/s | 354 kB 00:00 2023-09-30T14:27:33.514 INFO:teuthology.orchestra.run.smithi148.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 16 MB/s | 808 kB 00:00 2023-09-30T14:27:33.529 INFO:teuthology.orchestra.run.smithi032.stdout:(11/131): cephfs-mirror-18.0.0-6447.g1d10b717.e 3.9 MB/s | 261 kB 00:00 2023-09-30T14:27:33.571 INFO:teuthology.orchestra.run.smithi032.stdout:(12/131): libcephfs-devel-18.0.0-6447.g1d10b717 745 kB/s | 31 kB 00:00 2023-09-30T14:27:33.606 INFO:teuthology.orchestra.run.smithi148.stdout:(100/131): postgresql-server-10.21-2.module+el8 29 MB/s | 5.1 MB 00:00 2023-09-30T14:27:33.618 INFO:teuthology.orchestra.run.smithi156.stdout:(30/131): ceph-mgr-diskprediction-local-18.0.0- 8.1 MB/s | 7.4 MB 00:00 2023-09-30T14:27:33.648 INFO:teuthology.orchestra.run.smithi148.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-09-30T14:27:33.663 INFO:teuthology.orchestra.run.smithi032.stdout:(13/131): libcephfs2-18.0.0-6447.g1d10b717.el8. 8.9 MB/s | 839 kB 00:00 2023-09-30T14:27:33.673 INFO:teuthology.orchestra.run.smithi148.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 701 kB/s | 47 kB 00:00 2023-09-30T14:27:33.698 INFO:teuthology.orchestra.run.smithi148.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-09-30T14:27:33.710 INFO:teuthology.orchestra.run.smithi156.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6447.g1d 2.6 MB/s | 242 kB 00:00 2023-09-30T14:27:33.722 INFO:teuthology.orchestra.run.smithi032.stdout:(14/131): libcephsqlite-18.0.0-6447.g1d10b717.e 3.1 MB/s | 184 kB 00:00 2023-09-30T14:27:33.724 INFO:teuthology.orchestra.run.smithi148.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 851 kB/s | 43 kB 00:00 2023-09-30T14:27:33.749 INFO:teuthology.orchestra.run.smithi148.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-09-30T14:27:33.760 INFO:teuthology.orchestra.run.smithi156.stdout:(32/131): ceph-mgr-rook-18.0.0-6447.g1d10b717.e 951 kB/s | 47 kB 00:00 2023-09-30T14:27:33.772 INFO:teuthology.orchestra.run.smithi032.stdout:(15/131): librados-devel-18.0.0-6447.g1d10b717. 2.7 MB/s | 139 kB 00:00 2023-09-30T14:27:33.774 INFO:teuthology.orchestra.run.smithi148.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 956 kB/s | 48 kB 00:00 2023-09-30T14:27:33.800 INFO:teuthology.orchestra.run.smithi148.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-09-30T14:27:33.802 INFO:teuthology.orchestra.run.smithi156.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6447.g1 349 kB/s | 15 kB 00:00 2023-09-30T14:27:33.825 INFO:teuthology.orchestra.run.smithi148.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 850 kB/s | 43 kB 00:00 2023-09-30T14:27:33.850 INFO:teuthology.orchestra.run.smithi148.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-09-30T14:27:33.856 INFO:teuthology.orchestra.run.smithi032.stdout:(16/131): libradosstriper1-18.0.0-6447.g1d10b71 6.1 MB/s | 519 kB 00:00 2023-09-30T14:27:33.860 INFO:teuthology.orchestra.run.smithi156.stdout:(34/131): ceph-volume-18.0.0-6447.g1d10b717.el8 4.4 MB/s | 261 kB 00:00 2023-09-30T14:27:33.875 INFO:teuthology.orchestra.run.smithi148.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-09-30T14:27:33.909 INFO:teuthology.orchestra.run.smithi148.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-09-30T14:27:33.927 INFO:teuthology.orchestra.run.smithi156.stdout:(35/131): cephadm-18.0.0-6447.g1d10b717.el8.noa 2.7 MB/s | 185 kB 00:00 2023-09-30T14:27:33.934 INFO:teuthology.orchestra.run.smithi148.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 319 kB/s | 19 kB 00:00 2023-09-30T14:27:33.960 INFO:teuthology.orchestra.run.smithi148.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-09-30T14:27:33.986 INFO:teuthology.orchestra.run.smithi156.stdout:(36/131): cephfs-top-18.0.0-6447.g1d10b717.el8. 311 kB/s | 18 kB 00:00 2023-09-30T14:27:33.994 INFO:teuthology.orchestra.run.smithi148.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 9.3 MB/s | 559 kB 00:00 2023-09-30T14:27:34.019 INFO:teuthology.orchestra.run.smithi148.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 304 kB/s | 18 kB 00:00 2023-09-30T14:27:34.052 INFO:teuthology.orchestra.run.smithi148.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-09-30T14:27:34.078 INFO:teuthology.orchestra.run.smithi148.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.9 MB/s | 238 kB 00:00 2023-09-30T14:27:34.120 INFO:teuthology.orchestra.run.smithi148.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-09-30T14:27:34.161 INFO:teuthology.orchestra.run.smithi156.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.7 MB/s | 306 kB 00:00 2023-09-30T14:27:34.170 INFO:teuthology.orchestra.run.smithi148.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 20 MB/s | 2.3 MB 00:00 2023-09-30T14:27:34.195 INFO:teuthology.orchestra.run.smithi148.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-09-30T14:27:34.212 INFO:teuthology.orchestra.run.smithi156.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.2 MB/s | 59 kB 00:00 2023-09-30T14:27:34.212 INFO:teuthology.orchestra.run.smithi148.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.4 MB/s | 60 kB 00:00 2023-09-30T14:27:34.237 INFO:teuthology.orchestra.run.smithi148.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 2.3 MB/s | 98 kB 00:00 2023-09-30T14:27:34.254 INFO:teuthology.orchestra.run.smithi156.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2023-09-30T14:27:34.254 INFO:teuthology.orchestra.run.smithi148.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.3 MB/s | 58 kB 00:00 2023-09-30T14:27:34.272 INFO:teuthology.orchestra.run.smithi148.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-09-30T14:27:34.313 INFO:teuthology.orchestra.run.smithi156.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 8.0 MB/s | 476 kB 00:00 2023-09-30T14:27:34.322 INFO:teuthology.orchestra.run.smithi148.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.3 MB/s | 644 kB 00:00 2023-09-30T14:27:34.346 INFO:teuthology.orchestra.run.smithi156.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.3 MB/s | 44 kB 00:00 2023-09-30T14:27:34.347 INFO:teuthology.orchestra.run.smithi148.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.2 MB/s | 171 kB 00:00 2023-09-30T14:27:34.372 INFO:teuthology.orchestra.run.smithi148.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-09-30T14:27:34.388 INFO:teuthology.orchestra.run.smithi156.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 4.4 MB/s | 185 kB 00:00 2023-09-30T14:27:34.430 INFO:teuthology.orchestra.run.smithi156.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 796 kB/s | 33 kB 00:00 2023-09-30T14:27:34.472 INFO:teuthology.orchestra.run.smithi156.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 390 kB/s | 16 kB 00:00 2023-09-30T14:27:34.489 INFO:teuthology.orchestra.run.smithi032.stdout:(17/131): librgw2-18.0.0-6447.g1d10b717.el8.x86 11 MB/s | 6.9 MB 00:00 2023-09-30T14:27:34.514 INFO:teuthology.orchestra.run.smithi156.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 4.1 MB/s | 173 kB 00:00 2023-09-30T14:27:34.540 INFO:teuthology.orchestra.run.smithi032.stdout:(18/131): python3-ceph-argparse-18.0.0-6447.g1d 898 kB/s | 45 kB 00:00 2023-09-30T14:27:34.564 INFO:teuthology.orchestra.run.smithi156.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 7.5 MB/s | 384 kB 00:00 2023-09-30T14:27:34.606 INFO:teuthology.orchestra.run.smithi156.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.9 MB/s | 81 kB 00:00 2023-09-30T14:27:34.648 INFO:teuthology.orchestra.run.smithi156.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 267 kB/s | 11 kB 00:00 2023-09-30T14:27:34.653 INFO:teuthology.orchestra.run.smithi032.stdout:(19/131): python3-ceph-common-18.0.0-6447.g1d10 1.1 MB/s | 125 kB 00:00 2023-09-30T14:27:34.690 INFO:teuthology.orchestra.run.smithi156.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 421 kB/s | 18 kB 00:00 2023-09-30T14:27:34.749 INFO:teuthology.orchestra.run.smithi032.stdout:(20/131): python3-cephfs-18.0.0-6447.g1d10b717. 2.0 MB/s | 197 kB 00:00 2023-09-30T14:27:34.833 INFO:teuthology.orchestra.run.smithi032.stdout:(21/131): python3-rados-18.0.0-6447.g1d10b717.e 4.4 MB/s | 371 kB 00:00 2023-09-30T14:27:34.890 INFO:teuthology.orchestra.run.smithi156.stdout:(50/131): ceph-osd-18.0.0-6447.g1d10b717.el8.x8 2.4 MB/s | 18 MB 00:07 2023-09-30T14:27:34.916 INFO:teuthology.orchestra.run.smithi032.stdout:(22/131): python3-rbd-18.0.0-6447.g1d10b717.el8 4.3 MB/s | 365 kB 00:00 2023-09-30T14:27:34.923 INFO:teuthology.orchestra.run.smithi148.stdout:(128/131): librbd1-18.0.0-6447.g1d10b717.el8.x8 6.7 MB/s | 3.7 MB 00:00 2023-09-30T14:27:34.924 INFO:teuthology.orchestra.run.smithi156.stdout:(51/131): python3-kubernetes-11.0.0-6.el8.noarc 5.0 MB/s | 1.2 MB 00:00 2023-09-30T14:27:34.966 INFO:teuthology.orchestra.run.smithi032.stdout:(23/131): python3-rgw-18.0.0-6447.g1d10b717.el8 2.1 MB/s | 106 kB 00:00 2023-09-30T14:27:34.973 INFO:teuthology.orchestra.run.smithi148.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-09-30T14:27:34.974 INFO:teuthology.orchestra.run.smithi156.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.3 MB/s | 59 kB 00:00 2023-09-30T14:27:35.016 INFO:teuthology.orchestra.run.smithi156.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2023-09-30T14:27:35.017 INFO:teuthology.orchestra.run.smithi032.stdout:(24/131): rbd-fuse-18.0.0-6447.g1d10b717.el8.x8 1.8 MB/s | 94 kB 00:00 2023-09-30T14:27:35.067 INFO:teuthology.orchestra.run.smithi156.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-09-30T14:27:35.092 INFO:teuthology.orchestra.run.smithi156.stdout:(55/131): python3-logutils-0.3.5-11.el8.noarch. 242 kB/s | 49 kB 00:00 2023-09-30T14:27:35.109 INFO:teuthology.orchestra.run.smithi156.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 372 kB/s | 16 kB 00:00 2023-09-30T14:27:35.126 INFO:teuthology.orchestra.run.smithi156.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2023-09-30T14:27:35.151 INFO:teuthology.orchestra.run.smithi156.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 4.6 MB/s | 196 kB 00:00 2023-09-30T14:27:35.176 INFO:teuthology.orchestra.run.smithi156.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-09-30T14:27:35.193 INFO:teuthology.orchestra.run.smithi156.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 448 kB/s | 19 kB 00:00 2023-09-30T14:27:35.211 INFO:teuthology.orchestra.run.smithi156.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 711 kB/s | 24 kB 00:00 2023-09-30T14:27:35.227 INFO:teuthology.orchestra.run.smithi156.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 868 kB/s | 29 kB 00:00 2023-09-30T14:27:35.240 INFO:teuthology.orchestra.run.smithi148.stdout:(130/131): librados2-18.0.0-6447.g1d10b717.el8. 4.2 MB/s | 3.7 MB 00:00 2023-09-30T14:27:35.244 INFO:teuthology.orchestra.run.smithi156.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 825 kB/s | 27 kB 00:00 2023-09-30T14:27:35.278 INFO:teuthology.orchestra.run.smithi032.stdout:(25/131): ceph-common-18.0.0-6447.g1d10b717.el8 1.9 MB/s | 23 MB 00:11 2023-09-30T14:27:35.290 INFO:teuthology.orchestra.run.smithi156.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 4.0 MB/s | 255 kB 00:00 2023-09-30T14:27:35.320 INFO:teuthology.orchestra.run.smithi156.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.3 MB/s | 251 kB 00:00 2023-09-30T14:27:35.354 INFO:teuthology.orchestra.run.smithi156.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 962 kB/s | 61 kB 00:00 2023-09-30T14:27:35.370 INFO:teuthology.orchestra.run.smithi156.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-09-30T14:27:35.396 INFO:teuthology.orchestra.run.smithi156.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 547 kB/s | 23 kB 00:00 2023-09-30T14:27:35.411 INFO:teuthology.orchestra.run.smithi032.stdout:(26/131): rbd-mirror-18.0.0-6447.g1d10b717.el8. 9.1 MB/s | 3.6 MB 00:00 2023-09-30T14:27:35.421 INFO:teuthology.orchestra.run.smithi156.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-09-30T14:27:35.436 INFO:teuthology.orchestra.run.smithi032.stdout:(27/131): rbd-nbd-18.0.0-6447.g1d10b717.el8.x86 1.1 MB/s | 180 kB 00:00 2023-09-30T14:27:35.467 INFO:teuthology.orchestra.run.smithi032.stdout:(28/131): ceph-grafana-dashboards-18.0.0-6447.g 552 kB/s | 23 kB 00:00 2023-09-30T14:27:35.488 INFO:teuthology.orchestra.run.smithi156.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 15 MB/s | 1.7 MB 00:00 2023-09-30T14:27:35.504 INFO:teuthology.orchestra.run.smithi032.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6447.g1d10b71 1.9 MB/s | 132 kB 00:00 2023-09-30T14:27:35.572 INFO:teuthology.orchestra.run.smithi156.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-09-30T14:27:35.614 INFO:teuthology.orchestra.run.smithi156.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-09-30T14:27:35.620 INFO:teuthology.orchestra.run.smithi032.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6447.g1d10b 11 MB/s | 1.8 MB 00:00 2023-09-30T14:27:35.655 INFO:teuthology.orchestra.run.smithi156.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-09-30T14:27:35.679 INFO:teuthology.orchestra.run.smithi032.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6447.g1d 4.1 MB/s | 242 kB 00:00 2023-09-30T14:27:35.706 INFO:teuthology.orchestra.run.smithi156.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-09-30T14:27:35.748 INFO:teuthology.orchestra.run.smithi156.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 939 kB/s | 39 kB 00:00 2023-09-30T14:27:35.806 INFO:teuthology.orchestra.run.smithi156.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-09-30T14:27:35.940 INFO:teuthology.orchestra.run.smithi156.stdout:(77/131): python3-scipy-1.0.0-21.module+el8.5.0 26 MB/s | 14 MB 00:00 2023-09-30T14:27:35.974 INFO:teuthology.orchestra.run.smithi156.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.1 MB/s | 538 kB 00:00 2023-09-30T14:27:35.982 INFO:teuthology.orchestra.run.smithi032.stdout:(32/131): ceph-osd-18.0.0-6447.g1d10b717.el8.x8 1.8 MB/s | 18 MB 00:10 2023-09-30T14:27:35.999 INFO:teuthology.orchestra.run.smithi156.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-09-30T14:27:36.024 INFO:teuthology.orchestra.run.smithi156.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-09-30T14:27:36.047 INFO:teuthology.orchestra.run.smithi032.stdout:(33/131): ceph-mgr-rook-18.0.0-6447.g1d10b717.e 129 kB/s | 47 kB 00:00 2023-09-30T14:27:36.066 INFO:teuthology.orchestra.run.smithi156.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-09-30T14:27:36.072 INFO:teuthology.orchestra.run.smithi032.stdout:(34/131): ceph-prometheus-alerts-18.0.0-6447.g1 163 kB/s | 15 kB 00:00 2023-09-30T14:27:36.117 INFO:teuthology.orchestra.run.smithi156.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 528 kB/s | 27 kB 00:00 2023-09-30T14:27:36.122 INFO:teuthology.orchestra.run.smithi032.stdout:(35/131): cephadm-18.0.0-6447.g1d10b717.el8.noa 3.6 MB/s | 185 kB 00:00 2023-09-30T14:27:36.148 INFO:teuthology.orchestra.run.smithi032.stdout:(36/131): ceph-volume-18.0.0-6447.g1d10b717.el8 2.5 MB/s | 261 kB 00:00 2023-09-30T14:27:36.159 INFO:teuthology.orchestra.run.smithi156.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-09-30T14:27:36.173 INFO:teuthology.orchestra.run.smithi032.stdout:(37/131): cephfs-top-18.0.0-6447.g1d10b717.el8. 361 kB/s | 18 kB 00:00 2023-09-30T14:27:36.234 INFO:teuthology.orchestra.run.smithi156.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 23 MB/s | 4.8 MB 00:00 2023-09-30T14:27:36.259 INFO:teuthology.orchestra.run.smithi156.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 438 kB/s | 44 kB 00:00 2023-09-30T14:27:36.310 INFO:teuthology.orchestra.run.smithi156.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-09-30T14:27:36.402 INFO:teuthology.orchestra.run.smithi156.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 22 MB/s | 3.7 MB 00:00 2023-09-30T14:27:36.427 INFO:teuthology.orchestra.run.smithi156.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.9 MB/s | 225 kB 00:00 2023-09-30T14:27:36.452 INFO:teuthology.orchestra.run.smithi156.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-09-30T14:27:36.478 INFO:teuthology.orchestra.run.smithi156.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.4 MB/s | 126 kB 00:00 2023-09-30T14:27:36.505 INFO:teuthology.orchestra.run.smithi156.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-09-30T14:27:36.515 INFO:teuthology.orchestra.run.smithi032.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 172 kB/s | 59 kB 00:00 2023-09-30T14:27:36.528 INFO:teuthology.orchestra.run.smithi156.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-09-30T14:27:36.562 INFO:teuthology.orchestra.run.smithi156.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-09-30T14:27:36.604 INFO:teuthology.orchestra.run.smithi156.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-09-30T14:27:36.699 INFO:teuthology.orchestra.run.smithi032.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 556 kB/s | 306 kB 00:00 2023-09-30T14:27:36.704 INFO:teuthology.orchestra.run.smithi156.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 28 MB/s | 4.9 MB 00:00 2023-09-30T14:27:36.724 INFO:teuthology.orchestra.run.smithi032.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 359 kB/s | 75 kB 00:00 2023-09-30T14:27:36.746 INFO:teuthology.orchestra.run.smithi156.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-09-30T14:27:36.788 INFO:teuthology.orchestra.run.smithi156.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-09-30T14:27:36.833 INFO:teuthology.orchestra.run.smithi032.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 403 kB/s | 44 kB 00:00 2023-09-30T14:27:36.855 INFO:teuthology.orchestra.run.smithi156.stdout:(98/131): openblas-0.3.15-3.el8.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2023-09-30T14:27:36.900 INFO:teuthology.orchestra.run.smithi032.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.3 MB/s | 476 kB 00:00 2023-09-30T14:27:36.905 INFO:teuthology.orchestra.run.smithi156.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 16 MB/s | 808 kB 00:00 2023-09-30T14:27:36.983 INFO:teuthology.orchestra.run.smithi032.stdout:(43/131): ceph-mgr-diskprediction-local-18.0.0- 5.0 MB/s | 7.4 MB 00:01 2023-09-30T14:27:37.006 INFO:teuthology.orchestra.run.smithi156.stdout:(100/131): postgresql-server-10.21-2.module+el8 23 MB/s | 5.1 MB 00:00 2023-09-30T14:27:37.008 INFO:teuthology.orchestra.run.smithi032.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 304 kB/s | 33 kB 00:00 2023-09-30T14:27:37.039 INFO:teuthology.orchestra.run.smithi156.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-09-30T14:27:37.059 INFO:teuthology.orchestra.run.smithi032.stdout:(45/131): python3-beautifulsoup4-4.6.3-2.el8.1. 820 kB/s | 185 kB 00:00 2023-09-30T14:27:37.064 INFO:teuthology.orchestra.run.smithi156.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 798 kB/s | 47 kB 00:00 2023-09-30T14:27:37.090 INFO:teuthology.orchestra.run.smithi156.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-09-30T14:27:37.115 INFO:teuthology.orchestra.run.smithi156.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 850 kB/s | 43 kB 00:00 2023-09-30T14:27:37.117 INFO:teuthology.orchestra.run.smithi032.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.6 MB/s | 173 kB 00:00 2023-09-30T14:27:37.140 INFO:teuthology.orchestra.run.smithi156.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-09-30T14:27:37.166 INFO:teuthology.orchestra.run.smithi156.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 956 kB/s | 48 kB 00:00 2023-09-30T14:27:37.191 INFO:teuthology.orchestra.run.smithi156.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-09-30T14:27:37.216 INFO:teuthology.orchestra.run.smithi156.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 847 kB/s | 43 kB 00:00 2023-09-30T14:27:37.226 INFO:teuthology.orchestra.run.smithi032.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 749 kB/s | 81 kB 00:00 2023-09-30T14:27:37.242 INFO:teuthology.orchestra.run.smithi156.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-09-30T14:27:37.251 INFO:teuthology.orchestra.run.smithi032.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.0 MB/s | 384 kB 00:00 2023-09-30T14:27:37.267 INFO:teuthology.orchestra.run.smithi156.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-09-30T14:27:37.301 INFO:teuthology.orchestra.run.smithi156.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-09-30T14:27:37.301 INFO:teuthology.orchestra.run.smithi032.stdout:(49/131): python3-certifi-2018.10.15-7.el8.noar 51 kB/s | 16 kB 00:00 2023-09-30T14:27:37.326 INFO:teuthology.orchestra.run.smithi156.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 319 kB/s | 19 kB 00:00 2023-09-30T14:27:37.335 INFO:teuthology.orchestra.run.smithi032.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2023-09-30T14:27:37.351 INFO:teuthology.orchestra.run.smithi156.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-09-30T14:27:37.369 INFO:teuthology.orchestra.run.smithi032.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 150 kB/s | 18 kB 00:00 2023-09-30T14:27:37.385 INFO:teuthology.orchestra.run.smithi156.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 9.3 MB/s | 559 kB 00:00 2023-09-30T14:27:37.410 INFO:teuthology.orchestra.run.smithi156.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 305 kB/s | 18 kB 00:00 2023-09-30T14:27:37.453 INFO:teuthology.orchestra.run.smithi032.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 416 kB/s | 49 kB 00:00 2023-09-30T14:27:37.470 INFO:teuthology.orchestra.run.smithi032.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 583 kB/s | 59 kB 00:00 2023-09-30T14:27:37.471 INFO:teuthology.orchestra.run.smithi156.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.9 MB/s | 605 kB 00:00 2023-09-30T14:27:37.502 INFO:teuthology.orchestra.run.smithi156.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-09-30T14:27:37.544 INFO:teuthology.orchestra.run.smithi156.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-09-30T14:27:37.561 INFO:teuthology.orchestra.run.smithi032.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 555 kB/s | 60 kB 00:00 2023-09-30T14:27:37.587 INFO:teuthology.orchestra.run.smithi032.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2023-09-30T14:27:37.595 INFO:teuthology.orchestra.run.smithi156.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 19 MB/s | 2.3 MB 00:00 2023-09-30T14:27:37.611 INFO:teuthology.orchestra.run.smithi156.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-09-30T14:27:37.637 INFO:teuthology.orchestra.run.smithi156.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.4 MB/s | 60 kB 00:00 2023-09-30T14:27:37.662 INFO:teuthology.orchestra.run.smithi156.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-09-30T14:27:37.663 INFO:teuthology.orchestra.run.smithi032.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-09-30T14:27:37.679 INFO:teuthology.orchestra.run.smithi032.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 365 kB/s | 34 kB 00:00 2023-09-30T14:27:37.687 INFO:teuthology.orchestra.run.smithi156.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-09-30T14:27:37.713 INFO:teuthology.orchestra.run.smithi156.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 769 kB/s | 39 kB 00:00 2023-09-30T14:27:37.746 INFO:teuthology.orchestra.run.smithi156.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-09-30T14:27:37.771 INFO:teuthology.orchestra.run.smithi032.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2023-09-30T14:27:37.772 INFO:teuthology.orchestra.run.smithi156.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-09-30T14:27:37.788 INFO:teuthology.orchestra.run.smithi032.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 591 kB/s | 64 kB 00:00 2023-09-30T14:27:37.797 INFO:teuthology.orchestra.run.smithi156.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-09-30T14:27:37.847 INFO:teuthology.orchestra.run.smithi032.stdout:(60/131): python3-kubernetes-11.0.0-6.el8.noarc 2.1 MB/s | 1.2 MB 00:00 2023-09-30T14:27:37.872 INFO:teuthology.orchestra.run.smithi032.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 186 kB/s | 19 kB 00:00 2023-09-30T14:27:37.889 INFO:teuthology.orchestra.run.smithi032.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2023-09-30T14:27:37.948 INFO:teuthology.orchestra.run.smithi032.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 291 kB/s | 29 kB 00:00 2023-09-30T14:27:37.998 INFO:teuthology.orchestra.run.smithi032.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 219 kB/s | 27 kB 00:00 2023-09-30T14:27:38.023 INFO:teuthology.orchestra.run.smithi032.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2023-09-30T14:27:38.048 INFO:teuthology.orchestra.run.smithi148.stdout:(131/131): ceph-test-18.0.0-6447.g1d10b717.el8. 3.9 MB/s | 49 MB 00:12 2023-09-30T14:27:38.049 INFO:teuthology.orchestra.run.smithi032.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.4 MB/s | 251 kB 00:00 2023-09-30T14:27:38.053 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2023-09-30T14:27:38.053 INFO:teuthology.orchestra.run.smithi148.stdout:Total 13 MB/s | 208 MB 00:15 2023-09-30T14:27:38.107 INFO:teuthology.orchestra.run.smithi032.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 562 kB/s | 61 kB 00:00 2023-09-30T14:27:38.132 INFO:teuthology.orchestra.run.smithi032.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 784 kB/s | 85 kB 00:00 2023-09-30T14:27:38.158 INFO:teuthology.orchestra.run.smithi032.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 210 kB/s | 23 kB 00:00 2023-09-30T14:27:38.172 INFO:teuthology.orchestra.run.smithi156.stdout:(128/131): librbd1-18.0.0-6447.g1d10b717.el8.x8 9.9 MB/s | 3.7 MB 00:00 2023-09-30T14:27:38.175 INFO:teuthology.orchestra.run.smithi032.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2023-09-30T14:27:38.231 INFO:teuthology.orchestra.run.smithi156.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-09-30T14:27:38.250 INFO:teuthology.orchestra.run.smithi032.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.6 MB/s | 110 kB 00:00 2023-09-30T14:27:38.292 INFO:teuthology.orchestra.run.smithi032.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-09-30T14:27:38.343 INFO:teuthology.orchestra.run.smithi032.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-09-30T14:27:38.393 INFO:teuthology.orchestra.run.smithi032.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-09-30T14:27:38.435 INFO:teuthology.orchestra.run.smithi032.stdout:(75/131): thrift-0.13.0-2.el8.x86_64.rpm 5.3 MB/s | 1.7 MB 00:00 2023-09-30T14:27:38.460 INFO:teuthology.orchestra.run.smithi032.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 580 kB/s | 39 kB 00:00 2023-09-30T14:27:38.498 INFO:teuthology.orchestra.run.smithi156.stdout:(130/131): librados2-18.0.0-6447.g1d10b717.el8. 5.2 MB/s | 3.7 MB 00:00 2023-09-30T14:27:38.544 INFO:teuthology.orchestra.run.smithi032.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 6.2 MB/s | 688 kB 00:00 2023-09-30T14:27:38.577 INFO:teuthology.orchestra.run.smithi032.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 4.8 MB/s | 538 kB 00:00 2023-09-30T14:27:38.611 INFO:teuthology.orchestra.run.smithi032.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-09-30T14:27:38.636 INFO:teuthology.orchestra.run.smithi032.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-09-30T14:27:38.678 INFO:teuthology.orchestra.run.smithi032.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-09-30T14:27:38.720 INFO:teuthology.orchestra.run.smithi032.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 638 kB/s | 27 kB 00:00 2023-09-30T14:27:38.886 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2023-09-30T14:27:38.904 INFO:teuthology.orchestra.run.smithi032.stdout:(83/131): python3-scipy-1.0.0-21.module+el8.5.0 18 MB/s | 14 MB 00:00 2023-09-30T14:27:38.937 INFO:teuthology.orchestra.run.smithi032.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 1.4 MB/s | 320 kB 00:00 2023-09-30T14:27:38.963 INFO:teuthology.orchestra.run.smithi032.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 754 kB/s | 44 kB 00:00 2023-09-30T14:27:39.006 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2023-09-30T14:27:39.007 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2023-09-30T14:27:39.013 INFO:teuthology.orchestra.run.smithi032.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-09-30T14:27:39.088 INFO:teuthology.orchestra.run.smithi032.stdout:(87/131): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2023-09-30T14:27:39.114 INFO:teuthology.orchestra.run.smithi032.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.2 MB/s | 225 kB 00:00 2023-09-30T14:27:39.139 INFO:teuthology.orchestra.run.smithi032.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-09-30T14:27:39.164 INFO:teuthology.orchestra.run.smithi032.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-09-30T14:27:39.189 INFO:teuthology.orchestra.run.smithi032.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 397 kB/s | 20 kB 00:00 2023-09-30T14:27:39.215 INFO:teuthology.orchestra.run.smithi032.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-09-30T14:27:39.257 INFO:teuthology.orchestra.run.smithi032.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-09-30T14:27:39.324 INFO:teuthology.orchestra.run.smithi032.stdout:(94/131): python3-numpy-1.14.3-10.el8.x86_64.rp 9.7 MB/s | 3.7 MB 00:00 2023-09-30T14:27:39.349 INFO:teuthology.orchestra.run.smithi032.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.1 MB/s | 198 kB 00:00 2023-09-30T14:27:39.399 INFO:teuthology.orchestra.run.smithi032.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-09-30T14:27:39.450 INFO:teuthology.orchestra.run.smithi032.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-09-30T14:27:39.533 INFO:teuthology.orchestra.run.smithi032.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 15 MB/s | 4.9 MB 00:00 2023-09-30T14:27:39.600 INFO:teuthology.orchestra.run.smithi032.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-09-30T14:27:39.642 INFO:teuthology.orchestra.run.smithi032.stdout:(100/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-09-30T14:27:39.684 INFO:teuthology.orchestra.run.smithi032.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-09-30T14:27:39.743 INFO:teuthology.orchestra.run.smithi032.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-09-30T14:27:39.759 INFO:teuthology.orchestra.run.smithi032.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.6 MB/s | 200 kB 00:00 2023-09-30T14:27:39.777 INFO:teuthology.orchestra.run.smithi032.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.2 MB/s | 43 kB 00:00 2023-09-30T14:27:39.802 INFO:teuthology.orchestra.run.smithi032.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.1 MB/s | 220 kB 00:00 2023-09-30T14:27:39.819 INFO:teuthology.orchestra.run.smithi032.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 1.1 MB/s | 48 kB 00:00 2023-09-30T14:27:39.836 INFO:teuthology.orchestra.run.smithi032.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.5 MB/s | 87 kB 00:00 2023-09-30T14:27:39.861 INFO:teuthology.orchestra.run.smithi032.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 1.0 MB/s | 43 kB 00:00 2023-09-30T14:27:39.878 INFO:teuthology.orchestra.run.smithi032.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.5 MB/s | 109 kB 00:00 2023-09-30T14:27:39.912 INFO:teuthology.orchestra.run.smithi032.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-09-30T14:27:39.946 INFO:teuthology.orchestra.run.smithi032.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.2 MB/s | 560 kB 00:00 2023-09-30T14:27:39.971 INFO:teuthology.orchestra.run.smithi032.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 319 kB/s | 19 kB 00:00 2023-09-30T14:27:39.996 INFO:teuthology.orchestra.run.smithi032.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-09-30T14:27:40.030 INFO:teuthology.orchestra.run.smithi032.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 9.3 MB/s | 559 kB 00:00 2023-09-30T14:27:40.055 INFO:teuthology.orchestra.run.smithi032.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 304 kB/s | 18 kB 00:00 2023-09-30T14:27:40.122 INFO:teuthology.orchestra.run.smithi032.stdout:(116/131): postgresql-server-10.21-2.module+el8 7.5 MB/s | 5.1 MB 00:00 2023-09-30T14:27:40.131 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2023-09-30T14:27:40.135 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2023-09-30T14:27:40.147 INFO:teuthology.orchestra.run.smithi032.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-09-30T14:27:40.181 INFO:teuthology.orchestra.run.smithi032.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.9 MB/s | 605 kB 00:00 2023-09-30T14:27:40.206 INFO:teuthology.orchestra.run.smithi032.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.2 MB/s | 72 kB 00:00 2023-09-30T14:27:40.232 INFO:teuthology.orchestra.run.smithi032.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-09-30T14:27:40.257 INFO:teuthology.orchestra.run.smithi032.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-09-30T14:27:40.274 INFO:teuthology.orchestra.run.smithi032.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 2.3 MB/s | 98 kB 00:00 2023-09-30T14:27:40.299 INFO:teuthology.orchestra.run.smithi032.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.3 MB/s | 58 kB 00:00 2023-09-30T14:27:40.325 INFO:teuthology.orchestra.run.smithi032.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 767 kB/s | 39 kB 00:00 2023-09-30T14:27:40.358 INFO:teuthology.orchestra.run.smithi032.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-09-30T14:27:40.384 INFO:teuthology.orchestra.run.smithi032.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-09-30T14:27:40.409 INFO:teuthology.orchestra.run.smithi032.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-09-30T14:27:40.459 INFO:teuthology.orchestra.run.smithi032.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.9 MB/s | 2.3 MB 00:00 2023-09-30T14:27:40.518 INFO:teuthology.orchestra.run.smithi032.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-09-30T14:27:41.735 INFO:teuthology.orchestra.run.smithi032.stdout:(130/131): librados2-18.0.0-6447.g1d10b717.el8. 2.8 MB/s | 3.7 MB 00:01 2023-09-30T14:27:44.943 INFO:teuthology.orchestra.run.smithi032.stdout:(131/131): librbd1-18.0.0-6447.g1d10b717.el8.x8 838 kB/s | 3.7 MB 00:04 2023-09-30T14:27:44.947 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2023-09-30T14:27:44.947 INFO:teuthology.orchestra.run.smithi032.stdout:Total 9.7 MB/s | 208 MB 00:21 2023-09-30T14:27:45.183 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2023-09-30T14:27:45.354 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-30T14:27:45.526 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-30T14:27:45.729 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-09-30T14:27:45.791 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2023-09-30T14:27:45.897 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-30T14:27:45.924 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2023-09-30T14:27:45.924 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2023-09-30T14:27:46.006 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-30T14:27:46.289 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-30T14:27:46.697 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-30T14:27:47.159 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2023-09-30T14:27:47.164 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:18.0.0-6447.g1d10b717.el8.x86_64 6/134 2023-09-30T14:27:47.164 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2023-09-30T14:27:47.556 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:18.0.0-6447.g1d10b717.el8.x86_64 6/134 2023-09-30T14:27:47.730 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:18.0.0-6447.g1d10b717.el8.x86_64 7/134 2023-09-30T14:27:47.862 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:18.0.0-6447.g1d10b717.el8.x86_64 7/134 2023-09-30T14:27:48.047 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:18.0.0-6447.g1d10b717.el8.x86_64 8/134 2023-09-30T14:27:48.145 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:18.0.0-6447.g1d10b717.el8.x86_64 8/134 2023-09-30T14:27:48.327 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:18.0.0-6447.g1d10b717.el8.x86_64 9/134 2023-09-30T14:27:48.422 INFO:teuthology.orchestra.run.smithi156.stdout:(131/131): ceph-test-18.0.0-6447.g1d10b717.el8. 2.6 MB/s | 49 MB 00:18 2023-09-30T14:27:48.427 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2023-09-30T14:27:48.427 INFO:teuthology.orchestra.run.smithi156.stdout:Total 8.3 MB/s | 208 MB 00:25 2023-09-30T14:27:48.568 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-30T14:27:48.800 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-30T14:27:49.203 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-30T14:27:49.259 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2023-09-30T14:27:49.381 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2023-09-30T14:27:49.381 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2023-09-30T14:27:49.437 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-30T14:27:49.563 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:18.0.0-6447.g1d10b717.el8.x86_64 14/134 2023-09-30T14:27:49.653 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6447.g1d10b717.el8.x86_64 14/134 2023-09-30T14:27:49.788 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:18.0.0-6447.g1d10b717.el8.x86 15/134 2023-09-30T14:27:49.864 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6447.g1d10b717.el8.x86 15/134 2023-09-30T14:27:50.005 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-30T14:27:50.148 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-30T14:27:50.313 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-30T14:27:50.511 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2023-09-30T14:27:50.515 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2023-09-30T14:27:50.919 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-30T14:27:51.096 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-30T14:27:51.686 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-30T14:27:51.888 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-09-30T14:27:52.088 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2023-09-30T14:27:52.493 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-30T14:27:52.589 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-09-30T14:27:52.657 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-30T14:27:52.861 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-09-30T14:27:52.862 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-09-30T14:27:53.052 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-30T14:27:53.082 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-09-30T14:27:53.168 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-30T14:27:53.334 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-09-30T14:27:53.385 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-30T14:27:53.535 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-09-30T14:27:53.736 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-09-30T14:27:53.825 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-30T14:27:53.974 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-30T14:27:54.035 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librados2-2:18.0.0-6447.g1d10b717.el8.x86_64 6/134 2023-09-30T14:27:54.169 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:18.0.0-6447.g1d10b717.el8. 26/134 2023-09-30T14:27:54.291 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:18.0.0-6447.g1d10b717.el 27/134 2023-09-30T14:27:54.474 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:18.0.0-6447.g1d10b717.el8.x86_64 6/134 2023-09-30T14:27:54.660 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librbd1-2:18.0.0-6447.g1d10b717.el8.x86_64 7/134 2023-09-30T14:27:54.795 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:18.0.0-6447.g1d10b717.el8.x86_64 7/134 2023-09-30T14:27:54.976 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs2-2:18.0.0-6447.g1d10b717.el8.x86_64 8/134 2023-09-30T14:27:55.059 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephfs2-2:18.0.0-6447.g1d10b717.el8.x86_64 8/134 2023-09-30T14:27:55.190 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:18.0.0-6447.g1d10b717.el8.x86_6 28/134 2023-09-30T14:27:55.229 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rados-2:18.0.0-6447.g1d10b717.el8.x86_64 9/134 2023-09-30T14:27:55.292 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2023-09-30T14:27:55.451 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-30T14:27:55.479 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-30T14:27:55.503 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-09-30T14:27:55.647 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-30T14:27:55.672 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-30T14:27:55.758 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-09-30T14:27:55.837 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-09-30T14:27:56.007 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-09-30T14:27:56.027 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-30T14:27:56.087 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-30T14:27:56.146 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-30T14:27:56.195 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-30T14:27:56.327 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-30T14:27:56.381 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-30T14:27:56.493 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephsqlite-2:18.0.0-6447.g1d10b717.el8.x86_64 14/134 2023-09-30T14:27:56.603 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6447.g1d10b717.el8.x86_64 14/134 2023-09-30T14:27:56.751 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libradosstriper1-2:18.0.0-6447.g1d10b717.el8.x86 15/134 2023-09-30T14:27:56.781 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-30T14:27:56.817 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6447.g1d10b717.el8.x86 15/134 2023-09-30T14:27:56.943 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-30T14:27:56.981 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librados2-2:18.0.0-6447.g1d10b717.el8.x86_64 6/134 2023-09-30T14:27:57.090 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-30T14:27:57.276 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-30T14:27:57.378 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:18.0.0-6447.g1d10b717.el8.x86_64 6/134 2023-09-30T14:27:57.548 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librbd1-2:18.0.0-6447.g1d10b717.el8.x86_64 7/134 2023-09-30T14:27:57.677 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:18.0.0-6447.g1d10b717.el8.x86_64 7/134 2023-09-30T14:27:57.872 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs2-2:18.0.0-6447.g1d10b717.el8.x86_64 8/134 2023-09-30T14:27:57.924 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-30T14:27:57.969 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephfs2-2:18.0.0-6447.g1d10b717.el8.x86_64 8/134 2023-09-30T14:27:57.986 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-30T14:27:58.126 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-30T14:27:58.140 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rados-2:18.0.0-6447.g1d10b717.el8.x86_64 9/134 2023-09-30T14:27:58.305 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-09-30T14:27:58.385 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-30T14:27:58.515 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-30T14:27:58.584 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-30T14:27:58.720 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-30T14:27:58.806 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-30T14:27:58.900 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:18.0.0-6447.g1d10b717.el8.x86_64 36/134 2023-09-30T14:27:58.965 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-30T14:27:59.034 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-09-30T14:27:59.057 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:18.0.0-6447.g1d10b717.el8.x86_6 37/134 2023-09-30T14:27:59.196 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-30T14:27:59.211 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-09-30T14:27:59.364 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephsqlite-2:18.0.0-6447.g1d10b717.el8.x86_64 14/134 2023-09-30T14:27:59.375 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-09-30T14:27:59.452 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6447.g1d10b717.el8.x86_64 14/134 2023-09-30T14:27:59.597 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libradosstriper1-2:18.0.0-6447.g1d10b717.el8.x86 15/134 2023-09-30T14:27:59.597 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-09-30T14:27:59.682 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6447.g1d10b717.el8.x86 15/134 2023-09-30T14:27:59.740 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-09-30T14:27:59.787 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-09-30T14:27:59.822 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-30T14:27:59.936 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-09-30T14:27:59.959 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-30T14:28:00.033 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-09-30T14:28:00.117 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-09-30T14:28:00.122 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-30T14:28:00.243 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-09-30T14:28:00.430 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-09-30T14:28:00.449 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-09-30T14:28:00.641 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-09-30T14:28:00.723 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-09-30T14:28:00.739 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-30T14:28:00.838 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-09-30T14:28:00.905 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-09-30T14:28:00.914 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-30T14:28:01.074 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-30T14:28:01.116 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-09-30T14:28:01.224 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-common-2:18.0.0-6447.g1d10b717.el8. 26/134 2023-09-30T14:28:01.342 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-argparse-2:18.0.0-6447.g1d10b717.el 27/134 2023-09-30T14:28:01.425 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-09-30T14:28:01.513 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-30T14:28:01.688 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-09-30T14:28:01.747 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-09-30T14:28:01.953 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-09-30T14:28:02.172 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-09-30T14:28:02.307 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cephfs-2:18.0.0-6447.g1d10b717.el8.x86_6 28/134 2023-09-30T14:28:02.409 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-09-30T14:28:02.424 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-09-30T14:28:02.590 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-09-30T14:28:02.616 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-09-30T14:28:02.670 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-09-30T14:28:02.747 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-09-30T14:28:02.857 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-09-30T14:28:02.889 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-09-30T14:28:03.037 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-09-30T14:28:03.101 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-09-30T14:28:03.154 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-09-30T14:28:03.311 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-09-30T14:28:03.318 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-09-30T14:28:03.350 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-30T14:28:03.528 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-09-30T14:28:03.576 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-09-30T14:28:03.793 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-09-30T14:28:03.807 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-30T14:28:03.969 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-09-30T14:28:03.978 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-common-2:18.0.0-6447.g1d10b717.el8. 26/134 2023-09-30T14:28:04.099 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-argparse-2:18.0.0-6447.g1d10b717.el 27/134 2023-09-30T14:28:04.138 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-09-30T14:28:04.327 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-09-30T14:28:04.397 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-09-30T14:28:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-09-30T14:28:04.814 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-09-30T14:28:04.994 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cephfs-2:18.0.0-6447.g1d10b717.el8.x86_6 28/134 2023-09-30T14:28:04.995 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-30T14:28:05.075 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-30T14:28:05.234 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-30T14:28:05.290 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-09-30T14:28:05.311 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-09-30T14:28:05.479 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-09-30T14:28:05.563 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-09-30T14:28:05.591 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-09-30T14:28:05.644 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-30T14:28:05.758 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-30T14:28:05.853 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-09-30T14:28:05.864 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-30T14:28:05.933 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-30T14:28:06.054 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-30T14:28:06.115 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rbd-2:18.0.0-6447.g1d10b717.el8.x86_64 36/134 2023-09-30T14:28:06.139 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-30T14:28:06.288 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librados-devel-2:18.0.0-6447.g1d10b717.el8.x86_6 37/134 2023-09-30T14:28:06.359 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-30T14:28:06.492 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-09-30T14:28:06.506 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-30T14:28:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-30T14:28:06.669 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-09-30T14:28:06.861 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-09-30T14:28:06.869 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-30T14:28:07.022 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-09-30T14:28:07.063 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-30T14:28:07.228 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-30T14:28:07.261 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-09-30T14:28:07.388 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-30T14:28:07.461 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-09-30T14:28:07.533 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-30T14:28:07.773 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-30T14:28:07.817 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-30T14:28:07.871 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-09-30T14:28:07.988 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-30T14:28:08.147 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-09-30T14:28:08.172 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-09-30T14:28:08.199 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-30T14:28:08.337 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-09-30T14:28:08.374 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-30T14:28:08.438 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-30T14:28:08.553 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-09-30T14:28:08.561 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-30T14:28:08.685 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-30T14:28:08.734 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rbd-2:18.0.0-6447.g1d10b717.el8.x86_64 36/134 2023-09-30T14:28:08.824 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-09-30T14:28:08.842 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-30T14:28:08.908 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librados-devel-2:18.0.0-6447.g1d10b717.el8.x86_6 37/134 2023-09-30T14:28:09.025 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-30T14:28:09.086 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-09-30T14:28:09.155 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-09-30T14:28:09.272 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-09-30T14:28:09.358 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-09-30T14:28:09.457 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-09-30T14:28:09.566 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-09-30T14:28:09.599 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-09-30T14:28:09.800 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-09-30T14:28:09.803 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-09-30T14:28:09.964 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-09-30T14:28:09.992 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-09-30T14:28:10.110 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-30T14:28:10.113 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-09-30T14:28:10.358 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-09-30T14:28:10.389 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-30T14:28:10.409 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-09-30T14:28:10.644 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-30T14:28:10.657 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-09-30T14:28:10.688 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-09-30T14:28:10.847 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-09-30T14:28:10.950 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-09-30T14:28:10.982 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-30T14:28:11.049 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-09-30T14:28:11.182 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-09-30T14:28:11.262 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-30T14:28:11.342 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-09-30T14:28:11.374 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-09-30T14:28:11.477 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-30T14:28:11.510 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-09-30T14:28:11.670 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-09-30T14:28:11.674 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-09-30T14:28:11.750 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-09-30T14:28:11.878 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-09-30T14:28:12.001 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-09-30T14:28:12.063 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-09-30T14:28:12.230 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-09-30T14:28:12.258 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-09-30T14:28:12.287 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-30T14:28:12.366 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-30T14:28:12.432 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-09-30T14:28:12.436 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-30T14:28:12.443 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:18.0.0-6447.g1d10b717.el8.x86_64 92/134 2023-09-30T14:28:12.502 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:18.0.0-6447.g1d10b717.el8.x86_64 92/134 2023-09-30T14:28:12.631 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-09-30T14:28:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-09-30T14:28:12.668 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:18.0.0-6447.g1d10b717.el8.x86_64 93/134 2023-09-30T14:28:12.844 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-09-30T14:28:12.910 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-09-30T14:28:13.034 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-30T14:28:13.159 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-09-30T14:28:13.215 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-30T14:28:13.391 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-09-30T14:28:13.462 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-30T14:28:13.643 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-09-30T14:28:13.665 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-30T14:28:13.802 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-09-30T14:28:13.863 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-30T14:28:13.946 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-09-30T14:28:14.020 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-30T14:28:14.128 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-09-30T14:28:14.200 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-09-30T14:28:14.221 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-30T14:28:14.431 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-30T14:28:14.454 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-09-30T14:28:14.605 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-30T14:28:14.644 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-09-30T14:28:14.794 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-30T14:28:14.811 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-30T14:28:14.894 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-30T14:28:14.956 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-30T14:28:15.116 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-09-30T14:28:15.190 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-30T14:28:15.287 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-09-30T14:28:15.387 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-30T14:28:15.428 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-30T14:28:15.584 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-30T14:28:15.589 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-30T14:28:15.821 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-30T14:28:15.861 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-30T14:28:16.032 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-30T14:28:16.060 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-30T14:28:16.199 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-30T14:28:16.210 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-30T14:28:16.356 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-30T14:28:16.446 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 94/134 2023-09-30T14:28:16.453 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-30T14:28:16.555 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-30T14:28:16.685 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 94/134 2023-09-30T14:28:16.750 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-30T14:28:16.965 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-30T14:28:17.147 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-30T14:28:17.284 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-30T14:28:17.284 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 94/134 2023-09-30T14:28:17.502 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-30T14:28:17.761 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-30T14:28:17.762 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-30T14:28:17.762 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:18.0.0-6447.g1d10b717.el8.x86_64 95/134 2023-09-30T14:28:17.762 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:18.0.0-6447.g1d10b717.el8.x86_64 95/134 2023-09-30T14:28:17.762 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:18.0.0-6447.g1d10b717.el8.x86_64 96/134 2023-09-30T14:28:17.900 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-30T14:28:17.927 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-30T14:28:18.159 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-30T14:28:18.184 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-30T14:28:18.393 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-30T14:28:18.510 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-30T14:28:18.568 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-30T14:28:18.770 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-30T14:28:18.785 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-30T14:28:19.009 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-30T14:28:19.840 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-30T14:28:19.872 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-30T14:28:19.990 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librgw2-2:18.0.0-6447.g1d10b717.el8.x86_64 92/134 2023-09-30T14:28:20.048 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librgw2-2:18.0.0-6447.g1d10b717.el8.x86_64 92/134 2023-09-30T14:28:20.139 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-30T14:28:20.182 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rgw-2:18.0.0-6447.g1d10b717.el8.x86_64 93/134 2023-09-30T14:28:20.394 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-30T14:28:20.688 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-30T14:28:20.958 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-30T14:28:21.202 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-30T14:28:22.021 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-30T14:28:22.177 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librgw2-2:18.0.0-6447.g1d10b717.el8.x86_64 92/134 2023-09-30T14:28:22.247 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librgw2-2:18.0.0-6447.g1d10b717.el8.x86_64 92/134 2023-09-30T14:28:22.419 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rgw-2:18.0.0-6447.g1d10b717.el8.x86_64 93/134 2023-09-30T14:28:24.143 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 94/134 2023-09-30T14:28:24.348 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 94/134 2023-09-30T14:28:24.970 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 94/134 2023-09-30T14:28:25.223 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-base-2:18.0.0-6447.g1d10b717.el8.x86_64 95/134 2023-09-30T14:28:25.310 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-base-2:18.0.0-6447.g1d10b717.el8.x86_64 95/134 2023-09-30T14:28:25.469 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-selinux-2:18.0.0-6447.g1d10b717.el8.x86_64 96/134 2023-09-30T14:28:26.199 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 94/134 2023-09-30T14:28:26.702 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 94/134 2023-09-30T14:28:27.293 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 94/134 2023-09-30T14:28:27.596 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-base-2:18.0.0-6447.g1d10b717.el8.x86_64 95/134 2023-09-30T14:28:27.666 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-base-2:18.0.0-6447.g1d10b717.el8.x86_64 95/134 2023-09-30T14:28:27.810 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-selinux-2:18.0.0-6447.g1d10b717.el8.x86_64 96/134 2023-09-30T14:28:36.656 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6447.g1d10b717.el8.x86_64 96/134 2023-09-30T14:28:36.656 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2023-09-30T14:28:36.656 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2023-09-30T14:28:36.656 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /dev 2023-09-30T14:28:36.656 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /run 2023-09-30T14:28:36.656 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2023-09-30T14:28:36.656 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2023-09-30T14:28:36.656 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2023-09-30T14:28:36.656 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2023-09-30T14:28:36.657 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /dev 2023-09-30T14:28:36.657 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-09-30T14:28:36.998 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-volume-2:18.0.0-6447.g1d10b717.el8.noarch 97/134 2023-09-30T14:28:38.615 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-volume-2:18.0.0-6447.g1d10b717.el8.noarch 97/134 2023-09-30T14:28:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:18.0.0-6447.g1d10b717.el8.x86_64 98/134 2023-09-30T14:28:39.016 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:18.0.0-6447.g1d10b717.el8.x86_64 98/134 2023-09-30T14:28:39.215 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:18.0.0-6447.g1d10b717.el8.x86_64 99/134 2023-09-30T14:28:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:18.0.0-6447.g1d10b717.el8.x86_64 99/134 2023-09-30T14:28:39.814 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:18.0.0-6447.g1d10b717.el8.x86_64 100/134 2023-09-30T14:28:40.004 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:18.0.0-6447.g1d10b717.el8.x86_64 100/134 2023-09-30T14:28:41.529 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:18.0.0-6447.g1d10b717.el8.noarch 101/134 2023-09-30T14:28:41.705 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:18.0.0-6447.g1d10b717.el8.noarch 101/134 2023-09-30T14:28:41.837 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6447.g1d10b717.e 102/134 2023-09-30T14:28:42.055 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6447.g1d10b717. 103/134 2023-09-30T14:28:42.255 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6447.g1d10b717.el8.noa 104/134 2023-09-30T14:28:42.478 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6447.g1d10b717.el8.noa 104/134 2023-09-30T14:28:42.760 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6447.g1d10b717.el8.n 105/134 2023-09-30T14:28:43.733 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6447.g1d10b717.el8.n 105/134 2023-09-30T14:28:43.955 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6447.g1d1 106/134 2023-09-30T14:28:44.047 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6447.g1d1 106/134 2023-09-30T14:28:44.395 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6447.g1d10b717.el 107/134 2023-09-30T14:28:44.599 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:18.0.0-6447.g1d10b717.el8.x86_64 108/134 2023-09-30T14:28:44.634 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6447.g1d10b717.el8.x86_64 108/134 2023-09-30T14:28:44.813 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:18.0.0-6447.g1d10b717.el8.noarch 109/134 2023-09-30T14:28:44.835 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6447.g1d10b717.el8.noarch 109/134 2023-09-30T14:28:44.970 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6447.g1d10b717.el8.x86_64 96/134 2023-09-30T14:28:44.970 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /sys 2023-09-30T14:28:44.970 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /proc 2023-09-30T14:28:44.970 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /dev 2023-09-30T14:28:44.970 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /run 2023-09-30T14:28:44.970 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /mnt 2023-09-30T14:28:44.970 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /var/tmp 2023-09-30T14:28:44.970 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /home 2023-09-30T14:28:44.970 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /tmp 2023-09-30T14:28:44.971 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /dev 2023-09-30T14:28:44.971 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:28:45.321 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-volume-2:18.0.0-6447.g1d10b717.el8.noarch 97/134 2023-09-30T14:28:46.176 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:18.0.0-6447.g1d10b717.el8.x86_64 110/134 2023-09-30T14:28:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:18.0.0-6447.g1d10b717.el8.x86_64 111/134 2023-09-30T14:28:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6447.g1d10b717.el8.x86_64 111/134 2023-09-30T14:28:46.597 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6447.g1d10b 112/134 2023-09-30T14:28:46.638 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6447.g1d10b 112/134 2023-09-30T14:28:46.788 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 113/134 2023-09-30T14:28:46.881 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6447.g1d10b717.el8.x86_64 96/134 2023-09-30T14:28:46.882 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /sys 2023-09-30T14:28:46.882 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /proc 2023-09-30T14:28:46.882 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /dev 2023-09-30T14:28:46.882 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /run 2023-09-30T14:28:46.882 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /mnt 2023-09-30T14:28:46.882 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /var/tmp 2023-09-30T14:28:46.882 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /home 2023-09-30T14:28:46.882 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /tmp 2023-09-30T14:28:46.882 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /dev 2023-09-30T14:28:46.882 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-30T14:28:47.020 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-volume-2:18.0.0-6447.g1d10b717.el8.noarch 97/134 2023-09-30T14:28:47.151 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 113/134 2023-09-30T14:28:47.187 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-osd-2:18.0.0-6447.g1d10b717.el8.x86_64 98/134 2023-09-30T14:28:47.240 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-volume-2:18.0.0-6447.g1d10b717.el8.noarch 97/134 2023-09-30T14:28:47.313 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 114/134 2023-09-30T14:28:47.420 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-osd-2:18.0.0-6447.g1d10b717.el8.x86_64 98/134 2023-09-30T14:28:47.595 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mds-2:18.0.0-6447.g1d10b717.el8.x86_64 99/134 2023-09-30T14:28:48.079 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mds-2:18.0.0-6447.g1d10b717.el8.x86_64 99/134 2023-09-30T14:28:48.253 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mon-2:18.0.0-6447.g1d10b717.el8.x86_64 100/134 2023-09-30T14:28:48.271 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mon-2:18.0.0-6447.g1d10b717.el8.x86_64 100/134 2023-09-30T14:28:48.836 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-volume-2:18.0.0-6447.g1d10b717.el8.noarch 97/134 2023-09-30T14:28:49.018 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-osd-2:18.0.0-6447.g1d10b717.el8.x86_64 98/134 2023-09-30T14:28:49.237 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-osd-2:18.0.0-6447.g1d10b717.el8.x86_64 98/134 2023-09-30T14:28:49.414 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mds-2:18.0.0-6447.g1d10b717.el8.x86_64 99/134 2023-09-30T14:28:49.870 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mds-2:18.0.0-6447.g1d10b717.el8.x86_64 99/134 2023-09-30T14:28:50.006 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mon-2:18.0.0-6447.g1d10b717.el8.x86_64 100/134 2023-09-30T14:28:50.023 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mon-2:18.0.0-6447.g1d10b717.el8.x86_64 100/134 2023-09-30T14:28:50.044 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephadm-2:18.0.0-6447.g1d10b717.el8.noarch 101/134 2023-09-30T14:28:50.223 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephadm-2:18.0.0-6447.g1d10b717.el8.noarch 101/134 2023-09-30T14:28:50.334 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6447.g1d10b717.e 102/134 2023-09-30T14:28:50.536 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6447.g1d10b717. 103/134 2023-09-30T14:28:50.761 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6447.g1d10b717.el8.noa 104/134 2023-09-30T14:28:50.989 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6447.g1d10b717.el8.noa 104/134 2023-09-30T14:28:51.303 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6447.g1d10b717.el8.n 105/134 2023-09-30T14:28:51.716 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephadm-2:18.0.0-6447.g1d10b717.el8.noarch 101/134 2023-09-30T14:28:51.911 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephadm-2:18.0.0-6447.g1d10b717.el8.noarch 101/134 2023-09-30T14:28:51.992 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 114/134 2023-09-30T14:28:52.028 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6447.g1d10b717.e 102/134 2023-09-30T14:28:52.230 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6447.g1d10b717. 103/134 2023-09-30T14:28:52.273 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:18.0.0-6447.g1d10b717.el8.x86_64 115/134 2023-09-30T14:28:52.329 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6447.g1d10b717.el8.n 105/134 2023-09-30T14:28:52.422 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6447.g1d10b717.el8.noa 104/134 2023-09-30T14:28:52.437 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-09-30T14:28:52.527 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6447.g1d1 106/134 2023-09-30T14:28:52.623 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6447.g1d1 106/134 2023-09-30T14:28:52.633 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-09-30T14:28:52.645 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6447.g1d10b717.el8.noa 104/134 2023-09-30T14:28:52.925 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6447.g1d10b717.el8.n 105/134 2023-09-30T14:28:52.955 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6447.g1d10b717.el 107/134 2023-09-30T14:28:53.183 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-2:18.0.0-6447.g1d10b717.el8.x86_64 108/134 2023-09-30T14:28:53.221 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6447.g1d10b717.el8.x86_64 108/134 2023-09-30T14:28:53.378 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-rook-2:18.0.0-6447.g1d10b717.el8.noarch 109/134 2023-09-30T14:28:53.390 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6447.g1d10b717.el8.noarch 109/134 2023-09-30T14:28:53.553 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-09-30T14:28:53.882 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-09-30T14:28:53.887 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6447.g1d10b717.el8.n 105/134 2023-09-30T14:28:54.073 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 119/134 2023-09-30T14:28:54.088 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6447.g1d1 106/134 2023-09-30T14:28:54.180 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6447.g1d1 106/134 2023-09-30T14:28:54.241 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-30T14:28:54.400 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:18.0.0-6447.g1d10b717.el8.x86_ 121/134 2023-09-30T14:28:54.509 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6447.g1d10b717.el 107/134 2023-09-30T14:28:54.521 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-30T14:28:54.743 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-2:18.0.0-6447.g1d10b717.el8.x86_64 108/134 2023-09-30T14:28:54.778 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6447.g1d10b717.el8.x86_64 108/134 2023-09-30T14:28:54.799 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-2:18.0.0-6447.g1d10b717.el8.x86_64 110/134 2023-09-30T14:28:54.930 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-rook-2:18.0.0-6447.g1d10b717.el8.noarch 109/134 2023-09-30T14:28:54.942 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6447.g1d10b717.el8.noarch 109/134 2023-09-30T14:28:55.010 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-radosgw-2:18.0.0-6447.g1d10b717.el8.x86_64 111/134 2023-09-30T14:28:55.045 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6447.g1d10b717.el8.x86_64 111/134 2023-09-30T14:28:55.203 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6447.g1d10b 112/134 2023-09-30T14:28:55.246 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6447.g1d10b 112/134 2023-09-30T14:28:55.411 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 113/134 2023-09-30T14:28:55.793 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 113/134 2023-09-30T14:28:56.002 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 114/134 2023-09-30T14:28:56.279 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-2:18.0.0-6447.g1d10b717.el8.x86_64 110/134 2023-09-30T14:28:56.443 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-radosgw-2:18.0.0-6447.g1d10b717.el8.x86_64 111/134 2023-09-30T14:28:56.477 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6447.g1d10b717.el8.x86_64 111/134 2023-09-30T14:28:56.605 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6447.g1d10b 112/134 2023-09-30T14:28:56.647 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6447.g1d10b 112/134 2023-09-30T14:28:56.730 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-30T14:28:56.821 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 113/134 2023-09-30T14:28:56.970 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-30T14:28:56.987 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-30T14:28:57.153 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-top-2:18.0.0-6447.g1d10b717.el8.noarch 124/134 2023-09-30T14:28:57.180 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 113/134 2023-09-30T14:28:57.346 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 114/134 2023-09-30T14:28:57.470 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 125/134 2023-09-30T14:28:57.760 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:18.0.0-6447.g1d10b717.el8.x86_64 126/134 2023-09-30T14:28:58.055 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-30T14:28:58.186 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-30T14:28:58.194 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-30T14:28:58.413 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-09-30T14:28:58.611 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-09-30T14:28:58.692 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-09-30T14:28:58.878 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-09-30T14:28:58.890 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-09-30T14:28:58.890 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-30T14:28:59.073 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-30T14:28:59.073 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-30T14:28:59.316 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-30T14:28:59.518 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-30T14:29:00.056 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-30T14:29:00.056 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:18.0.0-6447.g1d10b717.el8.x86_64 1/134 2023-09-30T14:29:00.056 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:18.0.0-6447.g1d10b717.el8.x86_64 2/134 2023-09-30T14:29:00.056 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 3/134 2023-09-30T14:29:00.056 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 4/134 2023-09-30T14:29:00.056 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6447.g1d10b 5/134 2023-09-30T14:29:00.056 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:18.0.0-6447.g1d10b717.el8.x86_64 6/134 2023-09-30T14:29:00.056 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:18.0.0-6447.g1d10b717.el8.x86_64 7/134 2023-09-30T14:29:00.056 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:18.0.0-6447.g1d10b717.el8.x86_64 8/134 2023-09-30T14:29:00.056 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:18.0.0-6447.g1d10b717.el8.x86_64 9/134 2023-09-30T14:29:00.057 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:18.0.0-6447.g1d10b717.el8.x86_64 10/134 2023-09-30T14:29:00.057 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:18.0.0-6447.g1d10b717.el8.x86_64 11/134 2023-09-30T14:29:00.057 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:18.0.0-6447.g1d10b717.el8.x86_64 12/134 2023-09-30T14:29:00.057 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 13/134 2023-09-30T14:29:00.057 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:18.0.0-6447.g1d10b717.el8.x86_ 14/134 2023-09-30T14:29:00.057 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:18.0.0-6447.g1d10b717.el8.x86_64 15/134 2023-09-30T14:29:00.057 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:18.0.0-6447.g1d10b717.el8.x86_64 16/134 2023-09-30T14:29:00.057 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:18.0.0-6447.g1d10b717.el8.x86_6 17/134 2023-09-30T14:29:00.057 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:18.0.0-6447.g1d10b717.el8.x86 18/134 2023-09-30T14:29:00.057 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:18.0.0-6447.g1d10b717.el8.x86_64 19/134 2023-09-30T14:29:00.057 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6447.g1d10b717.el 20/134 2023-09-30T14:29:00.057 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:18.0.0-6447.g1d10b717.el8. 21/134 2023-09-30T14:29:00.058 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:18.0.0-6447.g1d10b717.el8.x86_6 22/134 2023-09-30T14:29:00.058 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:18.0.0-6447.g1d10b717.el8.x86_64 23/134 2023-09-30T14:29:00.058 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:18.0.0-6447.g1d10b717.el8.x86_64 24/134 2023-09-30T14:29:00.058 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:18.0.0-6447.g1d10b717.el8.x86_64 25/134 2023-09-30T14:29:00.058 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 26/134 2023-09-30T14:29:00.058 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 27/134 2023-09-30T14:29:00.058 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:18.0.0-6447.g1d10b717.el8.x86_64 28/134 2023-09-30T14:29:00.058 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6447.g1d10b717. 29/134 2023-09-30T14:29:00.058 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6447.g1d10b717.el8.noa 30/134 2023-09-30T14:29:00.058 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6447.g1d10b717.el8.n 31/134 2023-09-30T14:29:00.058 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6447.g1d1 32/134 2023-09-30T14:29:00.058 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6447.g1d10b717.el 33/134 2023-09-30T14:29:00.059 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6447.g1d10b717.el8.noarch 34/134 2023-09-30T14:29:00.059 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6447.g1d10b717.e 35/134 2023-09-30T14:29:00.059 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-volume-2:18.0.0-6447.g1d10b717.el8.noarch 36/134 2023-09-30T14:29:00.059 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:18.0.0-6447.g1d10b717.el8.noarch 37/134 2023-09-30T14:29:00.059 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-top-2:18.0.0-6447.g1d10b717.el8.noarch 38/134 2023-09-30T14:29:00.059 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-30T14:29:00.059 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-30T14:29:00.059 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-30T14:29:00.059 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-30T14:29:00.059 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-30T14:29:00.059 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-30T14:29:00.059 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-30T14:29:00.060 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-30T14:29:00.060 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-30T14:29:00.060 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-30T14:29:00.060 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-30T14:29:00.060 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-30T14:29:00.060 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-30T14:29:00.060 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-30T14:29:00.060 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-30T14:29:00.060 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-30T14:29:00.060 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-30T14:29:00.060 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-30T14:29:00.060 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-30T14:29:00.060 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-30T14:29:00.061 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-30T14:29:00.061 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-30T14:29:00.061 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-30T14:29:00.061 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-30T14:29:00.061 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-30T14:29:00.061 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-30T14:29:00.061 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-30T14:29:00.061 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-30T14:29:00.061 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-30T14:29:00.061 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-30T14:29:00.061 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-30T14:29:00.061 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-30T14:29:00.062 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-30T14:29:00.062 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-09-30T14:29:00.062 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-09-30T14:29:00.062 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-09-30T14:29:00.062 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-09-30T14:29:00.062 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-09-30T14:29:00.062 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-09-30T14:29:00.062 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-09-30T14:29:00.062 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-09-30T14:29:00.062 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-09-30T14:29:00.062 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-09-30T14:29:00.063 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-09-30T14:29:00.063 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-09-30T14:29:00.063 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-09-30T14:29:00.063 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-09-30T14:29:00.063 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-09-30T14:29:00.063 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-09-30T14:29:00.063 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-09-30T14:29:00.063 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-09-30T14:29:00.064 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-09-30T14:29:00.064 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-09-30T14:29:00.064 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-09-30T14:29:00.064 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-09-30T14:29:00.065 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-09-30T14:29:00.065 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-09-30T14:29:00.065 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-09-30T14:29:00.065 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-09-30T14:29:00.065 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-09-30T14:29:00.065 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-09-30T14:29:00.065 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-09-30T14:29:00.065 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-09-30T14:29:00.065 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-09-30T14:29:00.065 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-09-30T14:29:00.065 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-09-30T14:29:00.065 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-09-30T14:29:00.066 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-09-30T14:29:00.066 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-09-30T14:29:00.066 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-09-30T14:29:00.066 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-09-30T14:29:00.066 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-09-30T14:29:00.066 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-09-30T14:29:00.066 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-09-30T14:29:00.066 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-09-30T14:29:00.066 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-09-30T14:29:00.066 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-09-30T14:29:00.066 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-09-30T14:29:00.067 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-09-30T14:29:00.067 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-09-30T14:29:00.067 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-09-30T14:29:00.067 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-30T14:29:00.067 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-09-30T14:29:00.067 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-30T14:29:00.067 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-09-30T14:29:00.067 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-09-30T14:29:00.067 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-09-30T14:29:00.067 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-09-30T14:29:00.067 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-09-30T14:29:00.067 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-30T14:29:00.068 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:18.0.0-6447.g1d10b717.el8.x86_64 129/134 2023-09-30T14:29:00.068 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-30T14:29:00.068 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:18.0.0-6447.g1d10b717.el8.x86_64 131/134 2023-09-30T14:29:00.068 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-30T14:29:00.068 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-30T14:29:00.943 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 114/134 2023-09-30T14:29:01.210 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-test-2:18.0.0-6447.g1d10b717.el8.x86_64 115/134 2023-09-30T14:29:01.376 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-09-30T14:29:01.585 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-09-30T14:29:01.979 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 114/134 2023-09-30T14:29:02.277 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-test-2:18.0.0-6447.g1d10b717.el8.x86_64 115/134 2023-09-30T14:29:02.429 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-09-30T14:29:02.524 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-09-30T14:29:02.564 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-30T14:29:02.564 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:52 ago on Sat 30 Sep 2023 02:27:10 PM UTC. 2023-09-30T14:29:02.621 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-09-30T14:29:02.852 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-09-30T14:29:03.055 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 119/134 2023-09-30T14:29:03.264 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-30T14:29:03.414 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs-devel-2:18.0.0-6447.g1d10b717.el8.x86_ 121/134 2023-09-30T14:29:03.508 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-09-30T14:29:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-30T14:29:03.824 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-09-30T14:29:04.024 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 119/134 2023-09-30T14:29:04.224 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-30T14:29:04.416 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs-devel-2:18.0.0-6447.g1d10b717.el8.x86_ 121/134 2023-09-30T14:29:04.571 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-30T14:29:05.177 INFO:teuthology.orchestra.run.smithi148.stdout:Installed products updated. 2023-09-30T14:29:05.393 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-09-30T14:29:05.394 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2023-09-30T14:29:05.394 INFO:teuthology.orchestra.run.smithi148.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-30T14:29:05.394 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.394 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.394 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2023-09-30T14:29:05.394 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-30T14:29:05.394 INFO:teuthology.orchestra.run.smithi148.stdout: automake-1.16.1-7.el8.noarch 2023-09-30T14:29:05.394 INFO:teuthology.orchestra.run.smithi148.stdout: bison-3.0.4-10.el8.x86_64 2023-09-30T14:29:05.394 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.394 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:05.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:05.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:05.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:05.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:05.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:05.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.396 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.396 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:05.396 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.396 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.396 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.396 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:05.396 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:05.396 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.396 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:05.396 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-30T14:29:05.396 INFO:teuthology.orchestra.run.smithi148.stdout: flex-2.6.1-9.el8.x86_64 2023-09-30T14:29:05.396 INFO:teuthology.orchestra.run.smithi148.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-30T14:29:05.397 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-30T14:29:05.397 INFO:teuthology.orchestra.run.smithi148.stdout: jq-1.6-3.el8.x86_64 2023-09-30T14:29:05.397 INFO:teuthology.orchestra.run.smithi148.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-30T14:29:05.397 INFO:teuthology.orchestra.run.smithi148.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-30T14:29:05.397 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-30T14:29:05.397 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.397 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.397 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.397 INFO:teuthology.orchestra.run.smithi148.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-30T14:29:05.397 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-30T14:29:05.397 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-30T14:29:05.397 INFO:teuthology.orchestra.run.smithi148.stdout: libpq-13.5-1.el8.x86_64 2023-09-30T14:29:05.398 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-30T14:29:05.398 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-30T14:29:05.398 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.398 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.398 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-30T14:29:05.398 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.398 INFO:teuthology.orchestra.run.smithi148.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-30T14:29:05.398 INFO:teuthology.orchestra.run.smithi148.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-30T14:29:05.398 INFO:teuthology.orchestra.run.smithi148.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-30T14:29:05.398 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-30T14:29:05.398 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-30T14:29:05.398 INFO:teuthology.orchestra.run.smithi148.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-30T14:29:05.399 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-30T14:29:05.399 INFO:teuthology.orchestra.run.smithi148.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-30T14:29:05.399 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-30T14:29:05.399 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-30T14:29:05.399 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-30T14:29:05.399 INFO:teuthology.orchestra.run.smithi148.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-30T14:29:05.399 INFO:teuthology.orchestra.run.smithi148.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-30T14:29:05.399 INFO:teuthology.orchestra.run.smithi148.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-30T14:29:05.399 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-30T14:29:05.399 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-30T14:29:05.399 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-30T14:29:05.399 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-30T14:29:05.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-30T14:29:05.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-30T14:29:05.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-30T14:29:05.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-30T14:29:05.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-30T14:29:05.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-30T14:29:05.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-30T14:29:05.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-30T14:29:05.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-30T14:29:05.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-30T14:29:05.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-30T14:29:05.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-30T14:29:05.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-30T14:29:05.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-30T14:29:05.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-30T14:29:05.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-30T14:29:05.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-30T14:29:05.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-30T14:29:05.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-30T14:29:05.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-30T14:29:05.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-30T14:29:05.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-30T14:29:05.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-30T14:29:05.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-30T14:29:05.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-30T14:29:05.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-30T14:29:05.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-2.6-1.el8.noarch 2023-09-30T14:29:05.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-30T14:29:05.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-30T14:29:05.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-30T14:29:05.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-30T14:29:05.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-30T14:29:05.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-30T14:29:05.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.403 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.403 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-30T14:29:05.403 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-30T14:29:05.403 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.403 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-30T14:29:05.403 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-30T14:29:05.403 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-30T14:29:05.403 INFO:teuthology.orchestra.run.smithi148.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-30T14:29:05.403 INFO:teuthology.orchestra.run.smithi148.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-30T14:29:05.403 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-30T14:29:05.403 INFO:teuthology.orchestra.run.smithi148.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-30T14:29:05.403 INFO:teuthology.orchestra.run.smithi148.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-30T14:29:05.403 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-30T14:29:05.404 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-30T14:29:05.404 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-30T14:29:05.404 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-30T14:29:05.404 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-30T14:29:05.404 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.404 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.404 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:05.404 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-30T14:29:05.404 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-30T14:29:05.404 INFO:teuthology.orchestra.run.smithi148.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-30T14:29:05.404 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-30T14:29:05.404 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-30T14:29:05.405 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-30T14:29:05.405 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-09-30T14:29:05.405 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2023-09-30T14:29:05.582 DEBUG:teuthology.parallel:result is None 2023-09-30T14:29:05.880 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-30T14:29:06.118 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-30T14:29:06.136 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-30T14:29:06.285 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-top-2:18.0.0-6447.g1d10b717.el8.noarch 124/134 2023-09-30T14:29:06.593 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 125/134 2023-09-30T14:29:06.811 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-30T14:29:06.901 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-nbd-2:18.0.0-6447.g1d10b717.el8.x86_64 126/134 2023-09-30T14:29:07.070 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-30T14:29:07.087 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-30T14:29:07.178 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-30T14:29:07.245 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-top-2:18.0.0-6447.g1d10b717.el8.noarch 124/134 2023-09-30T14:29:07.351 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-30T14:29:07.360 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-30T14:29:07.536 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 125/134 2023-09-30T14:29:07.544 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-09-30T14:29:07.734 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-09-30T14:29:07.818 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-09-30T14:29:07.869 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-nbd-2:18.0.0-6447.g1d10b717.el8.x86_64 126/134 2023-09-30T14:29:08.000 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-09-30T14:29:08.013 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-09-30T14:29:08.014 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-30T14:29:08.138 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-30T14:29:08.246 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-30T14:29:08.246 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-30T14:29:08.278 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-30T14:29:08.286 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-30T14:29:08.475 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-09-30T14:29:08.480 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-30T14:29:08.652 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-09-30T14:29:08.675 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-30T14:29:08.734 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-09-30T14:29:08.969 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-09-30T14:29:08.982 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-09-30T14:29:08.982 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-30T14:29:09.172 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-30T14:29:09.173 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-30T14:29:09.209 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-30T14:29:09.209 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-2:18.0.0-6447.g1d10b717.el8.x86_64 1/134 2023-09-30T14:29:09.209 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-base-2:18.0.0-6447.g1d10b717.el8.x86_64 2/134 2023-09-30T14:29:09.209 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 3/134 2023-09-30T14:29:09.209 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 4/134 2023-09-30T14:29:09.209 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6447.g1d10b 5/134 2023-09-30T14:29:09.209 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mds-2:18.0.0-6447.g1d10b717.el8.x86_64 6/134 2023-09-30T14:29:09.209 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-2:18.0.0-6447.g1d10b717.el8.x86_64 7/134 2023-09-30T14:29:09.209 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mon-2:18.0.0-6447.g1d10b717.el8.x86_64 8/134 2023-09-30T14:29:09.210 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-osd-2:18.0.0-6447.g1d10b717.el8.x86_64 9/134 2023-09-30T14:29:09.210 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-radosgw-2:18.0.0-6447.g1d10b717.el8.x86_64 10/134 2023-09-30T14:29:09.210 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-selinux-2:18.0.0-6447.g1d10b717.el8.x86_64 11/134 2023-09-30T14:29:09.210 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-test-2:18.0.0-6447.g1d10b717.el8.x86_64 12/134 2023-09-30T14:29:09.210 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 13/134 2023-09-30T14:29:09.210 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs-devel-2:18.0.0-6447.g1d10b717.el8.x86_ 14/134 2023-09-30T14:29:09.210 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs2-2:18.0.0-6447.g1d10b717.el8.x86_64 15/134 2023-09-30T14:29:09.210 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephsqlite-2:18.0.0-6447.g1d10b717.el8.x86_64 16/134 2023-09-30T14:29:09.210 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados-devel-2:18.0.0-6447.g1d10b717.el8.x86_6 17/134 2023-09-30T14:29:09.210 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libradosstriper1-2:18.0.0-6447.g1d10b717.el8.x86 18/134 2023-09-30T14:29:09.210 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librgw2-2:18.0.0-6447.g1d10b717.el8.x86_64 19/134 2023-09-30T14:29:09.211 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6447.g1d10b717.el 20/134 2023-09-30T14:29:09.211 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-common-2:18.0.0-6447.g1d10b717.el8. 21/134 2023-09-30T14:29:09.211 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cephfs-2:18.0.0-6447.g1d10b717.el8.x86_6 22/134 2023-09-30T14:29:09.211 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rados-2:18.0.0-6447.g1d10b717.el8.x86_64 23/134 2023-09-30T14:29:09.211 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rbd-2:18.0.0-6447.g1d10b717.el8.x86_64 24/134 2023-09-30T14:29:09.211 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rgw-2:18.0.0-6447.g1d10b717.el8.x86_64 25/134 2023-09-30T14:29:09.211 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 26/134 2023-09-30T14:29:09.211 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 27/134 2023-09-30T14:29:09.211 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-nbd-2:18.0.0-6447.g1d10b717.el8.x86_64 28/134 2023-09-30T14:29:09.220 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6447.g1d10b717. 29/134 2023-09-30T14:29:09.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6447.g1d10b717.el8.noa 30/134 2023-09-30T14:29:09.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6447.g1d10b717.el8.n 31/134 2023-09-30T14:29:09.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6447.g1d1 32/134 2023-09-30T14:29:09.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6447.g1d10b717.el 33/134 2023-09-30T14:29:09.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6447.g1d10b717.el8.noarch 34/134 2023-09-30T14:29:09.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6447.g1d10b717.e 35/134 2023-09-30T14:29:09.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-volume-2:18.0.0-6447.g1d10b717.el8.noarch 36/134 2023-09-30T14:29:09.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephadm-2:18.0.0-6447.g1d10b717.el8.noarch 37/134 2023-09-30T14:29:09.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-top-2:18.0.0-6447.g1d10b717.el8.noarch 38/134 2023-09-30T14:29:09.221 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-30T14:29:09.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-30T14:29:09.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-30T14:29:09.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-30T14:29:09.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-30T14:29:09.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-30T14:29:09.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-30T14:29:09.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-30T14:29:09.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-30T14:29:09.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-30T14:29:09.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-30T14:29:09.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-30T14:29:09.222 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-30T14:29:09.223 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-30T14:29:09.223 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-30T14:29:09.223 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-30T14:29:09.223 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-30T14:29:09.223 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-30T14:29:09.223 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-30T14:29:09.223 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-30T14:29:09.223 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-30T14:29:09.223 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-30T14:29:09.223 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-30T14:29:09.223 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-30T14:29:09.224 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-30T14:29:09.224 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-30T14:29:09.224 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-30T14:29:09.224 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-30T14:29:09.224 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-30T14:29:09.224 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-30T14:29:09.224 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-30T14:29:09.224 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-30T14:29:09.224 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-30T14:29:09.224 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-09-30T14:29:09.224 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-09-30T14:29:09.224 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-09-30T14:29:09.225 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-09-30T14:29:09.225 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-09-30T14:29:09.225 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-09-30T14:29:09.225 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-09-30T14:29:09.225 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-09-30T14:29:09.225 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-09-30T14:29:09.225 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-09-30T14:29:09.225 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-09-30T14:29:09.225 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-09-30T14:29:09.225 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-09-30T14:29:09.225 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-09-30T14:29:09.225 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-09-30T14:29:09.226 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-09-30T14:29:09.226 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-09-30T14:29:09.226 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-09-30T14:29:09.226 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-09-30T14:29:09.226 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-09-30T14:29:09.226 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-09-30T14:29:09.226 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-09-30T14:29:09.226 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-09-30T14:29:09.226 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-09-30T14:29:09.226 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-09-30T14:29:09.226 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-09-30T14:29:09.227 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-09-30T14:29:09.227 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-09-30T14:29:09.227 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-09-30T14:29:09.227 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-09-30T14:29:09.227 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-09-30T14:29:09.227 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-09-30T14:29:09.227 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-09-30T14:29:09.227 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-09-30T14:29:09.227 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-09-30T14:29:09.227 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-09-30T14:29:09.227 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-09-30T14:29:09.227 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-09-30T14:29:09.228 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-09-30T14:29:09.228 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-09-30T14:29:09.228 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-09-30T14:29:09.228 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-09-30T14:29:09.228 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-09-30T14:29:09.228 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-09-30T14:29:09.228 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-09-30T14:29:09.228 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-09-30T14:29:09.228 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-09-30T14:29:09.228 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-09-30T14:29:09.228 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-30T14:29:09.228 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-09-30T14:29:09.228 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-30T14:29:09.229 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-09-30T14:29:09.229 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-09-30T14:29:09.229 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-09-30T14:29:09.229 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-09-30T14:29:09.229 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-09-30T14:29:09.229 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-30T14:29:09.229 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:18.0.0-6447.g1d10b717.el8.x86_64 129/134 2023-09-30T14:29:09.229 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-30T14:29:09.229 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:18.0.0-6447.g1d10b717.el8.x86_64 131/134 2023-09-30T14:29:09.229 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-30T14:29:09.229 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-30T14:29:09.458 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-30T14:29:09.677 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-30T14:29:10.189 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-30T14:29:10.189 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-2:18.0.0-6447.g1d10b717.el8.x86_64 1/134 2023-09-30T14:29:10.189 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-base-2:18.0.0-6447.g1d10b717.el8.x86_64 2/134 2023-09-30T14:29:10.189 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 3/134 2023-09-30T14:29:10.189 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 4/134 2023-09-30T14:29:10.190 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6447.g1d10b 5/134 2023-09-30T14:29:10.190 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mds-2:18.0.0-6447.g1d10b717.el8.x86_64 6/134 2023-09-30T14:29:10.190 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-2:18.0.0-6447.g1d10b717.el8.x86_64 7/134 2023-09-30T14:29:10.190 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mon-2:18.0.0-6447.g1d10b717.el8.x86_64 8/134 2023-09-30T14:29:10.190 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-osd-2:18.0.0-6447.g1d10b717.el8.x86_64 9/134 2023-09-30T14:29:10.190 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-radosgw-2:18.0.0-6447.g1d10b717.el8.x86_64 10/134 2023-09-30T14:29:10.190 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-selinux-2:18.0.0-6447.g1d10b717.el8.x86_64 11/134 2023-09-30T14:29:10.190 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-test-2:18.0.0-6447.g1d10b717.el8.x86_64 12/134 2023-09-30T14:29:10.190 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 13/134 2023-09-30T14:29:10.190 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs-devel-2:18.0.0-6447.g1d10b717.el8.x86_ 14/134 2023-09-30T14:29:10.190 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs2-2:18.0.0-6447.g1d10b717.el8.x86_64 15/134 2023-09-30T14:29:10.190 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephsqlite-2:18.0.0-6447.g1d10b717.el8.x86_64 16/134 2023-09-30T14:29:10.190 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados-devel-2:18.0.0-6447.g1d10b717.el8.x86_6 17/134 2023-09-30T14:29:10.191 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libradosstriper1-2:18.0.0-6447.g1d10b717.el8.x86 18/134 2023-09-30T14:29:10.191 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librgw2-2:18.0.0-6447.g1d10b717.el8.x86_64 19/134 2023-09-30T14:29:10.191 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6447.g1d10b717.el 20/134 2023-09-30T14:29:10.191 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-common-2:18.0.0-6447.g1d10b717.el8. 21/134 2023-09-30T14:29:10.191 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cephfs-2:18.0.0-6447.g1d10b717.el8.x86_6 22/134 2023-09-30T14:29:10.191 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rados-2:18.0.0-6447.g1d10b717.el8.x86_64 23/134 2023-09-30T14:29:10.191 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rbd-2:18.0.0-6447.g1d10b717.el8.x86_64 24/134 2023-09-30T14:29:10.191 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rgw-2:18.0.0-6447.g1d10b717.el8.x86_64 25/134 2023-09-30T14:29:10.191 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 26/134 2023-09-30T14:29:10.191 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 27/134 2023-09-30T14:29:10.191 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-nbd-2:18.0.0-6447.g1d10b717.el8.x86_64 28/134 2023-09-30T14:29:10.191 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6447.g1d10b717. 29/134 2023-09-30T14:29:10.192 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6447.g1d10b717.el8.noa 30/134 2023-09-30T14:29:10.192 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6447.g1d10b717.el8.n 31/134 2023-09-30T14:29:10.200 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6447.g1d1 32/134 2023-09-30T14:29:10.200 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6447.g1d10b717.el 33/134 2023-09-30T14:29:10.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6447.g1d10b717.el8.noarch 34/134 2023-09-30T14:29:10.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6447.g1d10b717.e 35/134 2023-09-30T14:29:10.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-volume-2:18.0.0-6447.g1d10b717.el8.noarch 36/134 2023-09-30T14:29:10.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephadm-2:18.0.0-6447.g1d10b717.el8.noarch 37/134 2023-09-30T14:29:10.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-top-2:18.0.0-6447.g1d10b717.el8.noarch 38/134 2023-09-30T14:29:10.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-30T14:29:10.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-30T14:29:10.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-30T14:29:10.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-30T14:29:10.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-30T14:29:10.201 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-30T14:29:10.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-30T14:29:10.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-30T14:29:10.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-30T14:29:10.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-30T14:29:10.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-30T14:29:10.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-30T14:29:10.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-30T14:29:10.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-30T14:29:10.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-30T14:29:10.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-30T14:29:10.202 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-30T14:29:10.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-30T14:29:10.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-30T14:29:10.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-30T14:29:10.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-30T14:29:10.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-30T14:29:10.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-30T14:29:10.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-30T14:29:10.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-30T14:29:10.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-30T14:29:10.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-30T14:29:10.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-30T14:29:10.203 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-30T14:29:10.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-30T14:29:10.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-30T14:29:10.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-30T14:29:10.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-30T14:29:10.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-09-30T14:29:10.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-09-30T14:29:10.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-09-30T14:29:10.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-09-30T14:29:10.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-09-30T14:29:10.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-09-30T14:29:10.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-09-30T14:29:10.204 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-09-30T14:29:10.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-09-30T14:29:10.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-09-30T14:29:10.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-09-30T14:29:10.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-09-30T14:29:10.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-09-30T14:29:10.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-09-30T14:29:10.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-09-30T14:29:10.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-09-30T14:29:10.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-09-30T14:29:10.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-09-30T14:29:10.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-09-30T14:29:10.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-09-30T14:29:10.205 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-09-30T14:29:10.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-09-30T14:29:10.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-09-30T14:29:10.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-09-30T14:29:10.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-09-30T14:29:10.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-09-30T14:29:10.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-09-30T14:29:10.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-09-30T14:29:10.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-09-30T14:29:10.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-09-30T14:29:10.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-09-30T14:29:10.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-09-30T14:29:10.206 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-09-30T14:29:10.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-09-30T14:29:10.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-09-30T14:29:10.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-09-30T14:29:10.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-09-30T14:29:10.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-09-30T14:29:10.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-09-30T14:29:10.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-09-30T14:29:10.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-09-30T14:29:10.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-09-30T14:29:10.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-09-30T14:29:10.207 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-09-30T14:29:10.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-09-30T14:29:10.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-09-30T14:29:10.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-09-30T14:29:10.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-09-30T14:29:10.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-30T14:29:10.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-09-30T14:29:10.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-30T14:29:10.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-09-30T14:29:10.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-09-30T14:29:10.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-09-30T14:29:10.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-09-30T14:29:10.208 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-09-30T14:29:10.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-30T14:29:10.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:18.0.0-6447.g1d10b717.el8.x86_64 129/134 2023-09-30T14:29:10.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-30T14:29:10.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:18.0.0-6447.g1d10b717.el8.x86_64 131/134 2023-09-30T14:29:10.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-30T14:29:10.209 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-30T14:29:11.609 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-30T14:29:11.610 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:00 ago on Sat 30 Sep 2023 02:27:11 PM UTC. 2023-09-30T14:29:12.601 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-30T14:29:12.601 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:02:01 ago on Sat 30 Sep 2023 02:27:11 PM UTC. 2023-09-30T14:29:14.393 INFO:teuthology.orchestra.run.smithi032.stdout:Installed products updated. 2023-09-30T14:29:14.630 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:29:14.630 INFO:teuthology.orchestra.run.smithi032.stdout:Upgraded: 2023-09-30T14:29:14.630 INFO:teuthology.orchestra.run.smithi032.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-30T14:29:14.630 INFO:teuthology.orchestra.run.smithi032.stdout: librados2-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.631 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.631 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2023-09-30T14:29:14.631 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-30T14:29:14.631 INFO:teuthology.orchestra.run.smithi032.stdout: automake-1.16.1-7.el8.noarch 2023-09-30T14:29:14.631 INFO:teuthology.orchestra.run.smithi032.stdout: bison-3.0.4-10.el8.x86_64 2023-09-30T14:29:14.631 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.632 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.632 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.632 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.632 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:14.632 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.633 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.633 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.633 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:14.633 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:14.633 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:14.633 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:14.634 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:14.634 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.634 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.634 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:14.634 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.635 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.635 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.637 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-volume-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:14.637 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:14.637 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.637 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-top-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:14.638 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-30T14:29:14.638 INFO:teuthology.orchestra.run.smithi032.stdout: flex-2.6.1-9.el8.x86_64 2023-09-30T14:29:14.638 INFO:teuthology.orchestra.run.smithi032.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-30T14:29:14.638 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-30T14:29:14.638 INFO:teuthology.orchestra.run.smithi032.stdout: jq-1.6-3.el8.x86_64 2023-09-30T14:29:14.638 INFO:teuthology.orchestra.run.smithi032.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-30T14:29:14.638 INFO:teuthology.orchestra.run.smithi032.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-30T14:29:14.639 INFO:teuthology.orchestra.run.smithi032.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-30T14:29:14.639 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.639 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.639 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.639 INFO:teuthology.orchestra.run.smithi032.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-30T14:29:14.639 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-30T14:29:14.639 INFO:teuthology.orchestra.run.smithi032.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-30T14:29:14.639 INFO:teuthology.orchestra.run.smithi032.stdout: libpq-13.5-1.el8.x86_64 2023-09-30T14:29:14.639 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-30T14:29:14.640 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-30T14:29:14.640 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.640 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.640 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-30T14:29:14.640 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.641 INFO:teuthology.orchestra.run.smithi032.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-30T14:29:14.641 INFO:teuthology.orchestra.run.smithi032.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-30T14:29:14.641 INFO:teuthology.orchestra.run.smithi032.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-30T14:29:14.641 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-30T14:29:14.641 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-30T14:29:14.641 INFO:teuthology.orchestra.run.smithi032.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-30T14:29:14.641 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-30T14:29:14.641 INFO:teuthology.orchestra.run.smithi032.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-30T14:29:14.641 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-30T14:29:14.642 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-30T14:29:14.642 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-30T14:29:14.642 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-30T14:29:14.642 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-30T14:29:14.642 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-30T14:29:14.642 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-30T14:29:14.642 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-30T14:29:14.642 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-30T14:29:14.643 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-30T14:29:14.643 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-30T14:29:14.643 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-30T14:29:14.643 INFO:teuthology.orchestra.run.smithi032.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-30T14:29:14.643 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-30T14:29:14.643 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.643 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.643 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.643 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-30T14:29:14.644 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-30T14:29:14.644 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-30T14:29:14.644 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-30T14:29:14.644 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-30T14:29:14.644 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-30T14:29:14.644 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-30T14:29:14.644 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-30T14:29:14.644 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-30T14:29:14.645 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-30T14:29:14.645 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-30T14:29:14.645 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-30T14:29:14.645 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-30T14:29:14.645 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-30T14:29:14.645 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-30T14:29:14.645 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-30T14:29:14.645 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-30T14:29:14.646 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-30T14:29:14.646 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-30T14:29:14.646 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-30T14:29:14.646 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-30T14:29:14.646 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-30T14:29:14.647 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend-2.6-1.el8.noarch 2023-09-30T14:29:14.647 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-30T14:29:14.647 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-30T14:29:14.647 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-30T14:29:14.647 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-30T14:29:14.647 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-30T14:29:14.648 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-30T14:29:14.648 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.648 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.648 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-30T14:29:14.648 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-30T14:29:14.649 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.649 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-30T14:29:14.649 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-30T14:29:14.649 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-30T14:29:14.649 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-30T14:29:14.649 INFO:teuthology.orchestra.run.smithi032.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-30T14:29:14.650 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-30T14:29:14.650 INFO:teuthology.orchestra.run.smithi032.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-30T14:29:14.650 INFO:teuthology.orchestra.run.smithi032.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-30T14:29:14.650 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-30T14:29:14.650 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-30T14:29:14.650 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-30T14:29:14.651 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-30T14:29:14.651 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-30T14:29:14.651 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.651 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.651 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:14.652 INFO:teuthology.orchestra.run.smithi032.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-30T14:29:14.652 INFO:teuthology.orchestra.run.smithi032.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-30T14:29:14.652 INFO:teuthology.orchestra.run.smithi032.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-30T14:29:14.652 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-30T14:29:14.652 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-30T14:29:14.653 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-30T14:29:14.654 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:29:14.654 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2023-09-30T14:29:14.947 DEBUG:teuthology.parallel:result is None 2023-09-30T14:29:15.286 INFO:teuthology.orchestra.run.smithi156.stdout:Installed products updated. 2023-09-30T14:29:15.503 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-30T14:29:15.504 INFO:teuthology.orchestra.run.smithi156.stdout:Upgraded: 2023-09-30T14:29:15.504 INFO:teuthology.orchestra.run.smithi156.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-30T14:29:15.504 INFO:teuthology.orchestra.run.smithi156.stdout: librados2-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.504 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.504 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2023-09-30T14:29:15.504 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-30T14:29:15.504 INFO:teuthology.orchestra.run.smithi156.stdout: automake-1.16.1-7.el8.noarch 2023-09-30T14:29:15.504 INFO:teuthology.orchestra.run.smithi156.stdout: bison-3.0.4-10.el8.x86_64 2023-09-30T14:29:15.504 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.504 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.504 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.505 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.505 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:15.505 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.505 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.505 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.505 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:15.505 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:15.505 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:15.505 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:15.505 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:15.505 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.505 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.506 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:15.506 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.506 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.506 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.506 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:15.506 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:15.506 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.506 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top-2:18.0.0-6447.g1d10b717.el8.noarch 2023-09-30T14:29:15.506 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-30T14:29:15.506 INFO:teuthology.orchestra.run.smithi156.stdout: flex-2.6.1-9.el8.x86_64 2023-09-30T14:29:15.506 INFO:teuthology.orchestra.run.smithi156.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-30T14:29:15.506 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-30T14:29:15.506 INFO:teuthology.orchestra.run.smithi156.stdout: jq-1.6-3.el8.x86_64 2023-09-30T14:29:15.507 INFO:teuthology.orchestra.run.smithi156.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-30T14:29:15.507 INFO:teuthology.orchestra.run.smithi156.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-30T14:29:15.507 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-30T14:29:15.507 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.507 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.507 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.507 INFO:teuthology.orchestra.run.smithi156.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-30T14:29:15.507 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-30T14:29:15.507 INFO:teuthology.orchestra.run.smithi156.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-30T14:29:15.507 INFO:teuthology.orchestra.run.smithi156.stdout: libpq-13.5-1.el8.x86_64 2023-09-30T14:29:15.507 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-30T14:29:15.507 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-30T14:29:15.508 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.508 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.508 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-30T14:29:15.508 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.508 INFO:teuthology.orchestra.run.smithi156.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-30T14:29:15.508 INFO:teuthology.orchestra.run.smithi156.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-30T14:29:15.508 INFO:teuthology.orchestra.run.smithi156.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-30T14:29:15.508 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-30T14:29:15.508 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-30T14:29:15.508 INFO:teuthology.orchestra.run.smithi156.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-30T14:29:15.508 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-30T14:29:15.508 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-30T14:29:15.508 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-30T14:29:15.509 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-30T14:29:15.509 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-30T14:29:15.509 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-30T14:29:15.509 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-30T14:29:15.509 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-30T14:29:15.509 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-30T14:29:15.509 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-30T14:29:15.509 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-30T14:29:15.509 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-30T14:29:15.509 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-30T14:29:15.509 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-30T14:29:15.509 INFO:teuthology.orchestra.run.smithi156.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-30T14:29:15.509 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-30T14:29:15.510 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.510 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.510 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.510 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-30T14:29:15.510 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-30T14:29:15.510 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-30T14:29:15.510 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-30T14:29:15.510 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-30T14:29:15.511 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-30T14:29:15.511 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-30T14:29:15.511 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-30T14:29:15.511 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-30T14:29:15.511 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-30T14:29:15.511 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-30T14:29:15.511 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-30T14:29:15.511 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-30T14:29:15.511 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-30T14:29:15.511 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-30T14:29:15.511 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-30T14:29:15.511 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-30T14:29:15.511 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-30T14:29:15.512 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-30T14:29:15.512 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-30T14:29:15.512 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-30T14:29:15.512 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-30T14:29:15.512 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend-2.6-1.el8.noarch 2023-09-30T14:29:15.512 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-30T14:29:15.512 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-30T14:29:15.512 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-30T14:29:15.512 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-30T14:29:15.512 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-30T14:29:15.512 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-30T14:29:15.512 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.512 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.513 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-30T14:29:15.513 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-30T14:29:15.513 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.513 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-30T14:29:15.513 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-30T14:29:15.513 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-30T14:29:15.513 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-30T14:29:15.513 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-30T14:29:15.513 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-30T14:29:15.513 INFO:teuthology.orchestra.run.smithi156.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-30T14:29:15.513 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-30T14:29:15.513 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-30T14:29:15.514 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-30T14:29:15.514 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-30T14:29:15.514 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-30T14:29:15.514 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-30T14:29:15.514 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.514 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.514 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd-2:18.0.0-6447.g1d10b717.el8.x86_64 2023-09-30T14:29:15.514 INFO:teuthology.orchestra.run.smithi156.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-30T14:29:15.514 INFO:teuthology.orchestra.run.smithi156.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-30T14:29:15.514 INFO:teuthology.orchestra.run.smithi156.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-30T14:29:15.514 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-30T14:29:15.514 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-30T14:29:15.515 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-30T14:29:15.515 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-30T14:29:15.515 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-09-30T14:29:15.691 DEBUG:teuthology.parallel:result is None 2023-09-30T14:29:15.691 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1d10b71792f3be8887a7631e69851ac2df3585af 2023-09-30T14:29:15.834 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-30T14:29:15.862 INFO:teuthology.orchestra.run.smithi032.stdout:18.0.0-6447.g1d10b717.el8 2023-09-30T14:29:15.863 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6447.g1d10b717.el8 2023-09-30T14:29:15.863 INFO:teuthology.task.install:The correct ceph version 18.0.0-6447.g1d10b717 is installed. 2023-09-30T14:29:15.863 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1d10b71792f3be8887a7631e69851ac2df3585af 2023-09-30T14:29:16.086 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-30T14:29:16.115 INFO:teuthology.orchestra.run.smithi148.stdout:18.0.0-6447.g1d10b717.el8 2023-09-30T14:29:16.115 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6447.g1d10b717.el8 2023-09-30T14:29:16.116 INFO:teuthology.task.install:The correct ceph version 18.0.0-6447.g1d10b717 is installed. 2023-09-30T14:29:16.116 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1d10b71792f3be8887a7631e69851ac2df3585af 2023-09-30T14:29:16.252 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-30T14:29:16.276 INFO:teuthology.orchestra.run.smithi156.stdout:18.0.0-6447.g1d10b717.el8 2023-09-30T14:29:16.277 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6447.g1d10b717.el8 2023-09-30T14:29:16.277 INFO:teuthology.task.install:The correct ceph version 18.0.0-6447.g1d10b717 is installed. 2023-09-30T14:29:16.277 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-30T14:29:16.277 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:29:16.277 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-30T14:29:16.327 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-09-30T14:29:16.327 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-30T14:29:16.375 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-30T14:29:16.375 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-30T14:29:16.416 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-30T14:29:16.416 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:29:16.417 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2023-09-30T14:29:16.450 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-30T14:29:16.521 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-09-30T14:29:16.521 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2023-09-30T14:29:16.552 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-30T14:29:16.623 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-30T14:29:16.623 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2023-09-30T14:29:16.650 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-30T14:29:16.720 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-30T14:29:16.720 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:29:16.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-30T14:29:16.753 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-30T14:29:16.825 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-09-30T14:29:16.826 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-30T14:29:16.854 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-30T14:29:16.927 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-30T14:29:16.928 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-30T14:29:16.958 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-30T14:29:17.026 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-30T14:29:17.026 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:29:17.027 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2023-09-30T14:29:17.059 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-30T14:29:17.133 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-09-30T14:29:17.133 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2023-09-30T14:29:17.165 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-30T14:29:17.237 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-30T14:29:17.237 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2023-09-30T14:29:17.265 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-30T14:29:17.334 INFO:teuthology.run_tasks:Running task cephadm... 2023-09-30T14:29:17.442 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '1d10b71792f3be8887a7631e69851ac2df3585af', 'subvols': {'create': 2}} 2023-09-30T14:29:17.442 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af 2023-09-30T14:29:17.443 INFO:tasks.cephadm:Cluster fsid is bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:29:17.443 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-09-30T14:29:17.443 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.32', 'mon.b': '172.21.15.148', 'mon.c': '172.21.15.156'} 2023-09-30T14:29:17.443 INFO:tasks.cephadm:First mon is mon.a on smithi032 2023-09-30T14:29:17.443 INFO:tasks.cephadm:First mgr is x 2023-09-30T14:29:17.443 INFO:tasks.cephadm:Normalizing hostnames... 2023-09-30T14:29:17.444 DEBUG:teuthology.orchestra.run.smithi032:> sudo hostname $(hostname -s) 2023-09-30T14:29:17.478 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname $(hostname -s) 2023-09-30T14:29:17.513 DEBUG:teuthology.orchestra.run.smithi156:> sudo hostname $(hostname -s) 2023-09-30T14:29:17.543 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-09-30T14:29:17.543 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1d10b71792f3be8887a7631e69851ac2df3585af 2023-09-30T14:29:17.757 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1d10b71792f3be8887a7631e69851ac2df3585af', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/72884/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6447-g1d10b717', 'node_name': '172.21.4.65+confusa03', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6447.g1d10b717'}, 'url': 'https://2.chacra.ceph.com/r/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-09-29 19:15:20.339137', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'main', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/flavors/default/', 'archs': ['source', 'x86_64', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1d10b71792f3be8887a7631e69851ac2df3585af', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/53447/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6447-g1d10b717', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6447.g1d10b717'}, 'url': 'https://1.chacra.ceph.com/r/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-09-29 16:03:13.858775', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'main', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-09-30T14:29:17.897 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref main, sha1 1d10b71792f3be8887a7631e69851ac2df3585af from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=1d10b71792f3be8887a7631e69851ac2df3585af 2023-09-30T14:29:17.898 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/x86_64/flavors/default/cephadm 2023-09-30T14:29:17.898 DEBUG:teuthology.orchestra.run.smithi032:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-30T14:29:18.459 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-rw-r--. 1 ubuntu ubuntu 749537 Sep 30 14:29 /home/ubuntu/cephtest/cephadm 2023-09-30T14:29:18.460 DEBUG:teuthology.orchestra.run.smithi148:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-30T14:29:18.903 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-rw-r--. 1 ubuntu ubuntu 749537 Sep 30 14:29 /home/ubuntu/cephtest/cephadm 2023-09-30T14:29:18.903 DEBUG:teuthology.orchestra.run.smithi156:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/main/1d10b71792f3be8887a7631e69851ac2df3585af/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-30T14:29:19.280 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-rw-r--. 1 ubuntu ubuntu 749537 Sep 30 14:29 /home/ubuntu/cephtest/cephadm 2023-09-30T14:29:19.281 DEBUG:teuthology.orchestra.run.smithi032:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-30T14:29:19.303 DEBUG:teuthology.orchestra.run.smithi148:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-30T14:29:19.326 DEBUG:teuthology.orchestra.run.smithi156:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-30T14:29:19.363 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af on all hosts... 2023-09-30T14:29:19.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af pull 2023-09-30T14:29:19.366 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af pull 2023-09-30T14:29:19.368 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af pull 2023-09-30T14:29:22.720 INFO:teuthology.orchestra.run.smithi148.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af... 2023-09-30T14:29:25.160 INFO:teuthology.orchestra.run.smithi156.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af... 2023-09-30T14:29:25.177 INFO:teuthology.orchestra.run.smithi032.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af... 2023-09-30T14:29:54.325 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-09-30T14:29:54.325 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 18.0.0-6447-g1d10b717 (1d10b71792f3be8887a7631e69851ac2df3585af) reef (dev)", 2023-09-30T14:29:54.325 INFO:teuthology.orchestra.run.smithi148.stdout: "image_id": "dda7fab67785080dcbfe9f290678eade9da15e3c952b7b83e7d43fbdc40ee966", 2023-09-30T14:29:54.325 INFO:teuthology.orchestra.run.smithi148.stdout: "repo_digests": [ 2023-09-30T14:29:54.326 INFO:teuthology.orchestra.run.smithi148.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:842fd637775ede364d8d63f7b11a1a5940cb8e14653cebf5ae1ff346603854c0" 2023-09-30T14:29:54.326 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-09-30T14:29:54.326 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-09-30T14:30:01.163 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-09-30T14:30:01.163 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_version": "ceph version 18.0.0-6447-g1d10b717 (1d10b71792f3be8887a7631e69851ac2df3585af) reef (dev)", 2023-09-30T14:30:01.163 INFO:teuthology.orchestra.run.smithi156.stdout: "image_id": "dda7fab67785080dcbfe9f290678eade9da15e3c952b7b83e7d43fbdc40ee966", 2023-09-30T14:30:01.163 INFO:teuthology.orchestra.run.smithi156.stdout: "repo_digests": [ 2023-09-30T14:30:01.163 INFO:teuthology.orchestra.run.smithi156.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:842fd637775ede364d8d63f7b11a1a5940cb8e14653cebf5ae1ff346603854c0" 2023-09-30T14:30:01.164 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2023-09-30T14:30:01.164 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-09-30T14:30:02.024 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T14:30:02.024 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 18.0.0-6447-g1d10b717 (1d10b71792f3be8887a7631e69851ac2df3585af) reef (dev)", 2023-09-30T14:30:02.024 INFO:teuthology.orchestra.run.smithi032.stdout: "image_id": "dda7fab67785080dcbfe9f290678eade9da15e3c952b7b83e7d43fbdc40ee966", 2023-09-30T14:30:02.024 INFO:teuthology.orchestra.run.smithi032.stdout: "repo_digests": [ 2023-09-30T14:30:02.024 INFO:teuthology.orchestra.run.smithi032.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:842fd637775ede364d8d63f7b11a1a5940cb8e14653cebf5ae1ff346603854c0" 2023-09-30T14:30:02.024 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T14:30:02.024 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T14:30:02.044 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph 2023-09-30T14:30:02.075 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph 2023-09-30T14:30:02.113 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph 2023-09-30T14:30:02.146 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /etc/ceph 2023-09-30T14:30:02.174 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /etc/ceph 2023-09-30T14:30:02.207 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /etc/ceph 2023-09-30T14:30:02.235 INFO:tasks.cephadm:Writing seed config... 2023-09-30T14:30:02.236 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-09-30T14:30:02.236 INFO:tasks.cephadm: override: [client] debug client = 20 2023-09-30T14:30:02.237 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-09-30T14:30:02.237 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-09-30T14:30:02.237 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-09-30T14:30:02.237 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-09-30T14:30:02.237 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-09-30T14:30:02.237 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-09-30T14:30:02.237 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-09-30T14:30:02.237 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-09-30T14:30:02.237 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-09-30T14:30:02.237 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-09-30T14:30:02.237 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-09-30T14:30:02.238 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-09-30T14:30:02.238 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-09-30T14:30:02.238 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-09-30T14:30:02.238 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-09-30T14:30:02.238 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-09-30T14:30:02.238 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-09-30T14:30:02.238 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-09-30T14:30:02.238 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-09-30T14:30:02.238 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-09-30T14:30:02.238 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-09-30T14:30:02.238 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-09-30T14:30:02.239 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-09-30T14:30:02.239 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-09-30T14:30:02.239 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-09-30T14:30:02.239 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2023-09-30T14:30:02.239 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2023-09-30T14:30:02.239 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2023-09-30T14:30:02.239 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-09-30T14:30:02.239 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-09-30T14:30:02.239 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2023-09-30T14:30:02.239 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2023-09-30T14:30:02.239 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-09-30T14:30:02.239 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-09-30T14:30:02.239 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2023-09-30T14:30:02.240 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-09-30T14:30:02.240 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-09-30T14:30:02.240 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-09-30T14:30:02.240 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-09-30T14:30:02.240 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-09-30T14:30:02.240 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-09-30T14:30:02.240 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-09-30T14:30:02.240 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-09-30T14:30:02.241 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:30:02.241 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-09-30T14:30:02.258 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-09-30T14:30:02.258 DEBUG:teuthology.orchestra.run.smithi032:mon.a> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mon.a.service 2023-09-30T14:30:02.301 DEBUG:teuthology.orchestra.run.smithi032:mgr.x> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mgr.x.service 2023-09-30T14:30:02.327 INFO:journalctl@ceph.mon.a.smithi032.stdout:-- Logs begin at Sat 2023-09-30 14:15:36 UTC. -- 2023-09-30T14:30:02.328 INFO:tasks.cephadm:Bootstrapping... 2023-09-30T14:30:02.328 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af -v bootstrap --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.32 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-09-30T14:30:02.354 INFO:journalctl@ceph.mgr.x.smithi032.stdout:-- Logs begin at Sat 2023-09-30 14:15:36 UTC. -- 2023-09-30T14:30:02.497 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2023-09-30T14:30:02.497 INFO:teuthology.orchestra.run.smithi032.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af', '-v', 'bootstrap', '--fsid', 'bca3f5c6-5f9d-11ee-8db5-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.32', '--skip-admin-label'] 2023-09-30T14:30:02.649 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 4.1.1 2023-09-30T14:30:02.649 INFO:teuthology.orchestra.run.smithi032.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-09-30T14:30:02.649 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying podman|docker is present... 2023-09-30T14:30:02.796 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 4.1.1 2023-09-30T14:30:02.796 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying lvm2 is present... 2023-09-30T14:30:02.796 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying time synchronization is in place... 2023-09-30T14:30:02.803 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-30T14:30:02.803 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-30T14:30:02.809 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-30T14:30:02.809 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2023-09-30T14:30:02.815 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout enabled 2023-09-30T14:30:02.821 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout active 2023-09-30T14:30:02.821 INFO:teuthology.orchestra.run.smithi032.stdout:Unit chronyd.service is enabled and running 2023-09-30T14:30:02.822 INFO:teuthology.orchestra.run.smithi032.stdout:Repeating the final host check... 2023-09-30T14:30:02.969 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 4.1.1 2023-09-30T14:30:02.969 INFO:teuthology.orchestra.run.smithi032.stdout:podman (/bin/podman) version 4.1.1 is present 2023-09-30T14:30:02.969 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl is present 2023-09-30T14:30:02.970 INFO:teuthology.orchestra.run.smithi032.stdout:lvcreate is present 2023-09-30T14:30:02.975 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-30T14:30:02.975 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-30T14:30:02.981 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-30T14:30:02.981 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2023-09-30T14:30:02.987 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout enabled 2023-09-30T14:30:02.993 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout active 2023-09-30T14:30:02.993 INFO:teuthology.orchestra.run.smithi032.stdout:Unit chronyd.service is enabled and running 2023-09-30T14:30:02.994 INFO:teuthology.orchestra.run.smithi032.stdout:Host looks OK 2023-09-30T14:30:02.994 INFO:teuthology.orchestra.run.smithi032.stdout:Cluster fsid: bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:30:02.994 INFO:teuthology.orchestra.run.smithi032.stdout:Acquiring lock 139661551094248 on /run/cephadm/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7.lock 2023-09-30T14:30:02.994 INFO:teuthology.orchestra.run.smithi032.stdout:Lock 139661551094248 acquired on /run/cephadm/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7.lock 2023-09-30T14:30:02.994 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying IP 172.21.15.32 port 3300 ... 2023-09-30T14:30:02.995 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying IP 172.21.15.32 port 6789 ... 2023-09-30T14:30:02.995 INFO:teuthology.orchestra.run.smithi032.stdout:Base mon IP(s) is [172.21.15.32:3300, 172.21.15.32:6789], mon addrv is [v2:172.21.15.32:3300,v1:172.21.15.32:6789] 2023-09-30T14:30:02.999 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-09-30T14:30:02.999 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.32 metric 100 2023-09-30T14:30:03.002 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-09-30T14:30:03.002 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-09-30T14:30:03.002 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1762sec hoplimit 64 pref medium 2023-09-30T14:30:03.005 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-09-30T14:30:03.005 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-09-30T14:30:03.005 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-30T14:30:03.005 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-09-30T14:30:03.005 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-09-30T14:30:03.005 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-30T14:30:03.006 INFO:teuthology.orchestra.run.smithi032.stdout:Mon IP `172.21.15.32` is in CIDR network `172.21.0.0/20` 2023-09-30T14:30:03.006 INFO:teuthology.orchestra.run.smithi032.stdout:Mon IP `172.21.15.32` is in CIDR network `172.21.0.0/20` 2023-09-30T14:30:03.006 INFO:teuthology.orchestra.run.smithi032.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-09-30T14:30:03.006 INFO:teuthology.orchestra.run.smithi032.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-09-30T14:30:03.007 INFO:teuthology.orchestra.run.smithi032.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af... 2023-09-30T14:30:03.551 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout dda7fab67785080dcbfe9f290678eade9da15e3c952b7b83e7d43fbdc40ee966 2023-09-30T14:30:03.552 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af... 2023-09-30T14:30:03.552 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Getting image source signatures 2023-09-30T14:30:03.552 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying blob sha256:dd82b8f163378c666c65b5d0bf191efe73118a446cff6e1c36ede2c3d7cf557f 2023-09-30T14:30:03.552 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying blob sha256:c16254b0810587cdd81d0647eccf1c3e4d200bf4eeba6596492e593ad063d4ee 2023-09-30T14:30:03.552 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying config sha256:dda7fab67785080dcbfe9f290678eade9da15e3c952b7b83e7d43fbdc40ee966 2023-09-30T14:30:03.552 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Writing manifest to image destination 2023-09-30T14:30:03.552 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Storing signatures 2023-09-30T14:30:04.599 INFO:teuthology.orchestra.run.smithi032.stdout:ceph: stdout ceph version 18.0.0-6447-g1d10b717 (1d10b71792f3be8887a7631e69851ac2df3585af) reef (dev) 2023-09-30T14:30:04.599 INFO:teuthology.orchestra.run.smithi032.stdout:Ceph version: ceph version 18.0.0-6447-g1d10b717 (1d10b71792f3be8887a7631e69851ac2df3585af) reef (dev) 2023-09-30T14:30:04.599 INFO:teuthology.orchestra.run.smithi032.stdout:Extracting ceph user uid/gid from container image... 2023-09-30T14:30:05.460 INFO:teuthology.orchestra.run.smithi032.stdout:stat: stdout 167 167 2023-09-30T14:30:05.460 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial keys... 2023-09-30T14:30:06.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQDtMBhlrqYlMxAAGIdT/4h2DxjYjyL8sjvwjA== 2023-09-30T14:30:07.169 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQDuMBhlLSvEMBAADO9D3rVWH/RuTyGBSTSFOw== 2023-09-30T14:30:07.914 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQDvMBhlYltfHxAADDrumEdTol3nQSLKPYQlsw== 2023-09-30T14:30:07.914 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial monmap... 2023-09-30T14:30:08.664 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-30T14:30:08.664 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-09-30T14:30:08.665 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:30:08.665 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-30T14:30:08.665 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool for a [v2:172.21.15.32:3300,v1:172.21.15.32:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-30T14:30:08.665 INFO:teuthology.orchestra.run.smithi032.stdout:setting min_mon_release = pacific 2023-09-30T14:30:08.665 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: set fsid to bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:30:08.665 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-30T14:30:08.665 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:30:08.665 INFO:teuthology.orchestra.run.smithi032.stdout:Creating mon... 2023-09-30T14:30:09.516 INFO:teuthology.orchestra.run.smithi032.stdout:create mon.a on 2023-09-30T14:30:09.639 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-09-30T14:30:09.730 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-09-30T14:30:09.861 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7.target. 2023-09-30T14:30:09.862 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7.target. 2023-09-30T14:30:10.339 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mon.a 2023-09-30T14:30:10.339 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mon.a.service: Unit ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mon.a.service not loaded. 2023-09-30T14:30:10.433 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7.target.wants/ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@.service. 2023-09-30T14:30:10.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:10 smithi032 systemd[1]: Starting Ceph mon.a for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:30:11.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:11 smithi032 podman[98053]: 2023-09-30T14:30:12.349 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-30T14:30:12.349 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2023-09-30T14:30:12.358 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-30T14:30:12.358 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2023-09-30T14:30:12.358 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2023-09-30T14:30:12.358 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to enable service . firewalld.service is not available 2023-09-30T14:30:12.358 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mon to start... 2023-09-30T14:30:12.359 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mon... 2023-09-30T14:30:12.564 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:12 smithi032 bash[98053]: e4a0fcdb1c0ba8b27c4ce27ffa53120413944d1ab4f9d64734c73b38e8ae8bc8 2023-09-30T14:30:12.564 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:12 smithi032 systemd[1]: Started Ceph mon.a for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:30:12.564 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:12 smithi032 ceph-mon[98093]: mkfs bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:30:12.564 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:12 smithi032 ceph-mon[98093]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-30T14:30:13.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout cluster: 2023-09-30T14:30:13.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout id: bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:30:13.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-09-30T14:30:13.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-09-30T14:30:13.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout services: 2023-09-30T14:30:13.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.635809s) 2023-09-30T14:30:13.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-09-30T14:30:13.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-09-30T14:30:13.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-09-30T14:30:13.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout data: 2023-09-30T14:30:13.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-09-30T14:30:13.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-09-30T14:30:13.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-09-30T14:30:13.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout pgs: 2023-09-30T14:30:13.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-09-30T14:30:13.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.830+0000 7f73179ef700 1 Processor -- start 2023-09-30T14:30:13.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.830+0000 7f73179ef700 1 -- start start 2023-09-30T14:30:13.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.831+0000 7f73179ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7310106b40 0x7f7310106f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:13.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.831+0000 7f73179ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7310107530 con 0x7f7310106b40 2023-09-30T14:30:13.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.831+0000 7f731578b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7310106b40 0x7f7310106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:13.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.831+0000 7f731578b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7310106b40 0x7f7310106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60644/0 (socket says 172.21.15.32:60644) 2023-09-30T14:30:13.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.831+0000 7f731578b700 1 -- 172.21.15.32:0/708877127 learned_addr learned my addr 172.21.15.32:0/708877127 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:13.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.831+0000 7f731578b700 1 -- 172.21.15.32:0/708877127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7310107670 con 0x7f7310106b40 2023-09-30T14:30:13.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.832+0000 7f731578b700 1 --2- 172.21.15.32:0/708877127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7310106b40 0x7f7310106f60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f730c0098a0 tx=0x7f730c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b956c0a6f8e75d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:13.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.832+0000 7f7303fff700 1 -- 172.21.15.32:0/708877127 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f730c004160 con 0x7f7310106b40 2023-09-30T14:30:13.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.832+0000 7f7303fff700 1 -- 172.21.15.32:0/708877127 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730c0042c0 con 0x7f7310106b40 2023-09-30T14:30:13.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.832+0000 7f7303fff700 1 -- 172.21.15.32:0/708877127 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f730c0045a0 con 0x7f7310106b40 2023-09-30T14:30:13.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.832+0000 7f73179ef700 1 -- 172.21.15.32:0/708877127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7310106b40 msgr2=0x7f7310106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:13.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.832+0000 7f73179ef700 1 --2- 172.21.15.32:0/708877127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7310106b40 0x7f7310106f60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f730c0098a0 tx=0x7f730c009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:13.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.832+0000 7f73179ef700 1 -- 172.21.15.32:0/708877127 shutdown_connections 2023-09-30T14:30:13.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.832+0000 7f73179ef700 1 --2- 172.21.15.32:0/708877127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7310106b40 0x7f7310106f60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:13.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.832+0000 7f73179ef700 1 -- 172.21.15.32:0/708877127 >> 172.21.15.32:0/708877127 conn(0x7f73101002f0 msgr2=0x7f7310102710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:13.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.833+0000 7f73179ef700 1 -- 172.21.15.32:0/708877127 shutdown_connections 2023-09-30T14:30:13.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.833+0000 7f73179ef700 1 -- 172.21.15.32:0/708877127 wait complete. 2023-09-30T14:30:13.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.833+0000 7f73179ef700 1 Processor -- start 2023-09-30T14:30:13.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.833+0000 7f73179ef700 1 -- start start 2023-09-30T14:30:13.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.833+0000 7f73179ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7310106b40 0x7f731018e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:13.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.833+0000 7f73179ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731018e870 con 0x7f7310106b40 2023-09-30T14:30:13.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.833+0000 7f731578b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7310106b40 0x7f731018e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:13.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.833+0000 7f731578b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7310106b40 0x7f731018e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60646/0 (socket says 172.21.15.32:60646) 2023-09-30T14:30:13.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.834+0000 7f731578b700 1 -- 172.21.15.32:0/3552554783 learned_addr learned my addr 172.21.15.32:0/3552554783 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:13.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.834+0000 7f731578b700 1 -- 172.21.15.32:0/3552554783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730c009550 con 0x7f7310106b40 2023-09-30T14:30:13.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.834+0000 7f731578b700 1 --2- 172.21.15.32:0/3552554783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7310106b40 0x7f731018e330 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f730c004000 tx=0x7f730c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:13.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.834+0000 7f73027fc700 1 -- 172.21.15.32:0/3552554783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f730c011630 con 0x7f7310106b40 2023-09-30T14:30:13.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.834+0000 7f73027fc700 1 -- 172.21.15.32:0/3552554783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730c011790 con 0x7f7310106b40 2023-09-30T14:30:13.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.834+0000 7f73027fc700 1 -- 172.21.15.32:0/3552554783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f730c011a30 con 0x7f7310106b40 2023-09-30T14:30:13.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.834+0000 7f73179ef700 1 -- 172.21.15.32:0/3552554783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731018c980 con 0x7f7310106b40 2023-09-30T14:30:13.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.834+0000 7f73179ef700 1 -- 172.21.15.32:0/3552554783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731018ce20 con 0x7f7310106b40 2023-09-30T14:30:13.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.834+0000 7f73027fc700 1 -- 172.21.15.32:0/3552554783 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f730c022070 con 0x7f7310106b40 2023-09-30T14:30:13.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.834+0000 7f73027fc700 1 -- 172.21.15.32:0/3552554783 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f730c01c450 con 0x7f7310106b40 2023-09-30T14:30:13.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.835+0000 7f73179ef700 1 -- 172.21.15.32:0/3552554783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7310062450 con 0x7f7310106b40 2023-09-30T14:30:13.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.838+0000 7f73027fc700 1 -- 172.21.15.32:0/3552554783 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f730c01b450 con 0x7f7310106b40 2023-09-30T14:30:13.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.971+0000 7f73179ef700 1 -- 172.21.15.32:0/3552554783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f7310105b70 con 0x7f7310106b40 2023-09-30T14:30:13.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.972+0000 7f73027fc700 1 -- 172.21.15.32:0/3552554783 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f730c01b5f0 con 0x7f7310106b40 2023-09-30T14:30:13.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.973+0000 7f73179ef700 1 -- 172.21.15.32:0/3552554783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7310106b40 msgr2=0x7f731018e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:13.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.973+0000 7f73179ef700 1 --2- 172.21.15.32:0/3552554783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7310106b40 0x7f731018e330 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f730c004000 tx=0x7f730c004750 comp rx=0 tx=0).stop 2023-09-30T14:30:13.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.973+0000 7f73179ef700 1 -- 172.21.15.32:0/3552554783 shutdown_connections 2023-09-30T14:30:13.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.973+0000 7f73179ef700 1 --2- 172.21.15.32:0/3552554783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7310106b40 0x7f731018e330 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:13.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.973+0000 7f73179ef700 1 -- 172.21.15.32:0/3552554783 >> 172.21.15.32:0/3552554783 conn(0x7f73101002f0 msgr2=0x7f7310104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:13.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.973+0000 7f73179ef700 1 -- 172.21.15.32:0/3552554783 shutdown_connections 2023-09-30T14:30:13.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:12.973+0000 7f73179ef700 1 -- 172.21.15.32:0/3552554783 wait complete. 2023-09-30T14:30:13.321 INFO:teuthology.orchestra.run.smithi032.stdout:mon is available 2023-09-30T14:30:13.321 INFO:teuthology.orchestra.run.smithi032.stdout:Assimilating anything we can from ceph.conf... 2023-09-30T14:30:13.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:13 smithi032 ceph-mon[98093]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-30T14:30:13.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:13 smithi032 ceph-mon[98093]: monmap e1: 1 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0]} removed_ranks: {} 2023-09-30T14:30:13.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:13 smithi032 ceph-mon[98093]: fsmap 2023-09-30T14:30:13.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:13 smithi032 ceph-mon[98093]: osdmap e1: 0 total, 0 up, 0 in 2023-09-30T14:30:13.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:13 smithi032 ceph-mon[98093]: mgrmap e1: no daemons active 2023-09-30T14:30:13.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:13 smithi032 ceph-mon[98093]: from='client.? 172.21.15.32:0/3552554783' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-09-30T14:30:14.364 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-09-30T14:30:14.364 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [global] 2023-09-30T14:30:14.364 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout fsid = bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:30:14.364 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.32:3300,v1:172.21.15.32:6789] 2023-09-30T14:30:14.364 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-30T14:30:14.364 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-30T14:30:14.364 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-30T14:30:14.364 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-30T14:30:14.364 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-09-30T14:30:14.364 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-30T14:30:14.365 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-30T14:30:14.365 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-09-30T14:30:14.365 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [osd] 2023-09-30T14:30:14.365 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-30T14:30:14.365 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-30T14:30:14.365 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-30T14:30:14.365 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-30T14:30:14.365 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-30T14:30:14.365 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.807+0000 7f4a721e7700 1 Processor -- start 2023-09-30T14:30:14.365 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.807+0000 7f4a721e7700 1 -- start start 2023-09-30T14:30:14.365 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.807+0000 7f4a721e7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a6c1055c0 0x7f4a6c1059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:14.366 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.807+0000 7f4a721e7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c105fb0 con 0x7f4a6c1055c0 2023-09-30T14:30:14.366 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.808+0000 7f4a6b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a6c1055c0 0x7f4a6c1059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:14.366 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.808+0000 7f4a6b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a6c1055c0 0x7f4a6c1059e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60648/0 (socket says 172.21.15.32:60648) 2023-09-30T14:30:14.366 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.808+0000 7f4a6b7fe700 1 -- 172.21.15.32:0/3140841957 learned_addr learned my addr 172.21.15.32:0/3140841957 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:14.366 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.808+0000 7f4a6b7fe700 1 -- 172.21.15.32:0/3140841957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c1067c0 con 0x7f4a6c1055c0 2023-09-30T14:30:14.366 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.808+0000 7f4a6b7fe700 1 --2- 172.21.15.32:0/3140841957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a6c1055c0 0x7f4a6c1059e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c0098a0 tx=0x7f4a5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0f2ed0575c0d6e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:14.366 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.808+0000 7f4a6a7fc700 1 -- 172.21.15.32:0/3140841957 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a5c004160 con 0x7f4a6c1055c0 2023-09-30T14:30:14.366 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.808+0000 7f4a6a7fc700 1 -- 172.21.15.32:0/3140841957 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5c0042c0 con 0x7f4a6c1055c0 2023-09-30T14:30:14.366 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.808+0000 7f4a6a7fc700 1 -- 172.21.15.32:0/3140841957 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a5c0045a0 con 0x7f4a6c1055c0 2023-09-30T14:30:14.366 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.809+0000 7f4a721e7700 1 -- 172.21.15.32:0/3140841957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a6c1055c0 msgr2=0x7f4a6c1059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:14.366 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.809+0000 7f4a721e7700 1 --2- 172.21.15.32:0/3140841957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a6c1055c0 0x7f4a6c1059e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c0098a0 tx=0x7f4a5c009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:14.366 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.809+0000 7f4a721e7700 1 -- 172.21.15.32:0/3140841957 shutdown_connections 2023-09-30T14:30:14.367 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.809+0000 7f4a721e7700 1 --2- 172.21.15.32:0/3140841957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a6c1055c0 0x7f4a6c1059e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:14.367 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.809+0000 7f4a721e7700 1 -- 172.21.15.32:0/3140841957 >> 172.21.15.32:0/3140841957 conn(0x7f4a6c100b80 msgr2=0x7f4a6c102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:14.367 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.809+0000 7f4a721e7700 1 -- 172.21.15.32:0/3140841957 shutdown_connections 2023-09-30T14:30:14.367 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.809+0000 7f4a721e7700 1 -- 172.21.15.32:0/3140841957 wait complete. 2023-09-30T14:30:14.367 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.809+0000 7f4a721e7700 1 Processor -- start 2023-09-30T14:30:14.367 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.810+0000 7f4a721e7700 1 -- start start 2023-09-30T14:30:14.367 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.810+0000 7f4a721e7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a6c1055c0 0x7f4a6c0742e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:14.367 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.810+0000 7f4a721e7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c19c880 con 0x7f4a6c1055c0 2023-09-30T14:30:14.367 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.810+0000 7f4a6b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a6c1055c0 0x7f4a6c0742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:14.367 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.810+0000 7f4a6b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a6c1055c0 0x7f4a6c0742e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60650/0 (socket says 172.21.15.32:60650) 2023-09-30T14:30:14.367 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.810+0000 7f4a6b7fe700 1 -- 172.21.15.32:0/1702908465 learned_addr learned my addr 172.21.15.32:0/1702908465 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:14.368 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.810+0000 7f4a6b7fe700 1 -- 172.21.15.32:0/1702908465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c009550 con 0x7f4a6c1055c0 2023-09-30T14:30:14.368 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.810+0000 7f4a6b7fe700 1 --2- 172.21.15.32:0/1702908465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a6c1055c0 0x7f4a6c0742e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c004000 tx=0x7f4a5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:14.368 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.811+0000 7f4a68ff9700 1 -- 172.21.15.32:0/1702908465 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a5c011630 con 0x7f4a6c1055c0 2023-09-30T14:30:14.368 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.811+0000 7f4a68ff9700 1 -- 172.21.15.32:0/1702908465 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5c011790 con 0x7f4a6c1055c0 2023-09-30T14:30:14.368 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.811+0000 7f4a721e7700 1 -- 172.21.15.32:0/1702908465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6c19c9c0 con 0x7f4a6c1055c0 2023-09-30T14:30:14.368 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.811+0000 7f4a68ff9700 1 -- 172.21.15.32:0/1702908465 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a5c011a30 con 0x7f4a6c1055c0 2023-09-30T14:30:14.368 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.811+0000 7f4a721e7700 1 -- 172.21.15.32:0/1702908465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6c19ce60 con 0x7f4a6c1055c0 2023-09-30T14:30:14.368 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.811+0000 7f4a68ff9700 1 -- 172.21.15.32:0/1702908465 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4a5c022070 con 0x7f4a6c1055c0 2023-09-30T14:30:14.368 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.811+0000 7f4a68ff9700 1 -- 172.21.15.32:0/1702908465 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4a5c01c450 con 0x7f4a6c1055c0 2023-09-30T14:30:14.368 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.812+0000 7f4a721e7700 1 -- 172.21.15.32:0/1702908465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6c1922e0 con 0x7f4a6c1055c0 2023-09-30T14:30:14.368 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.814+0000 7f4a68ff9700 1 -- 172.21.15.32:0/1702908465 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f4a5c01b450 con 0x7f4a6c1055c0 2023-09-30T14:30:14.369 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.941+0000 7f4a721e7700 1 -- 172.21.15.32:0/1702908465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f4a6c106d80 con 0x7f4a6c1055c0 2023-09-30T14:30:14.369 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.944+0000 7f4a68ff9700 1 -- 172.21.15.32:0/1702908465 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f4a5c01b5f0 con 0x7f4a6c1055c0 2023-09-30T14:30:14.369 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.945+0000 7f4a68ff9700 1 -- 172.21.15.32:0/1702908465 <== mon.0 v2:172.21.15.32:3300/0 8 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4a5c01b790 con 0x7f4a6c1055c0 2023-09-30T14:30:14.369 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.946+0000 7f4a721e7700 1 -- 172.21.15.32:0/1702908465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a6c1055c0 msgr2=0x7f4a6c0742e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:14.369 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.946+0000 7f4a721e7700 1 --2- 172.21.15.32:0/1702908465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a6c1055c0 0x7f4a6c0742e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c004000 tx=0x7f4a5c004750 comp rx=0 tx=0).stop 2023-09-30T14:30:14.369 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.946+0000 7f4a721e7700 1 -- 172.21.15.32:0/1702908465 shutdown_connections 2023-09-30T14:30:14.369 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.946+0000 7f4a721e7700 1 --2- 172.21.15.32:0/1702908465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a6c1055c0 0x7f4a6c0742e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:14.369 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.946+0000 7f4a721e7700 1 -- 172.21.15.32:0/1702908465 >> 172.21.15.32:0/1702908465 conn(0x7f4a6c100b80 msgr2=0x7f4a6c18cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:14.369 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.946+0000 7f4a721e7700 1 -- 172.21.15.32:0/1702908465 shutdown_connections 2023-09-30T14:30:14.370 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:13.946+0000 7f4a721e7700 1 -- 172.21.15.32:0/1702908465 wait complete. 2023-09-30T14:30:14.370 INFO:teuthology.orchestra.run.smithi032.stdout:Generating new minimal ceph.conf... 2023-09-30T14:30:14.577 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:14 smithi032 ceph-mon[98093]: from='client.? 172.21.15.32:0/1702908465' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-30T14:30:14.577 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:14 smithi032 ceph-mon[98093]: from='client.? 172.21.15.32:0/1702908465' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-09-30T14:30:15.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.826+0000 7fa7cd405700 1 Processor -- start 2023-09-30T14:30:15.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.826+0000 7fa7cd405700 1 -- start start 2023-09-30T14:30:15.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.826+0000 7fa7cd405700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c8107130 0x7fa7c8109520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:15.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.826+0000 7fa7cd405700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c8079e90 con 0x7fa7c8107130 2023-09-30T14:30:15.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.826+0000 7fa7c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c8107130 0x7fa7c8109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:15.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.826+0000 7fa7c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c8107130 0x7fa7c8109520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60652/0 (socket says 172.21.15.32:60652) 2023-09-30T14:30:15.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.826+0000 7fa7c6ffd700 1 -- 172.21.15.32:0/3648928819 learned_addr learned my addr 172.21.15.32:0/3648928819 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:15.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.827+0000 7fa7c6ffd700 1 -- 172.21.15.32:0/3648928819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c8109a60 con 0x7fa7c8107130 2023-09-30T14:30:15.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.827+0000 7fa7c6ffd700 1 --2- 172.21.15.32:0/3648928819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c8107130 0x7fa7c8109520 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0098a0 tx=0x7fa7bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c755817c6001dc8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:15.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.827+0000 7fa7c5ffb700 1 -- 172.21.15.32:0/3648928819 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7bc004160 con 0x7fa7c8107130 2023-09-30T14:30:15.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.827+0000 7fa7c5ffb700 1 -- 172.21.15.32:0/3648928819 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa7bc0042c0 con 0x7fa7c8107130 2023-09-30T14:30:15.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.827+0000 7fa7c5ffb700 1 -- 172.21.15.32:0/3648928819 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7bc011a00 con 0x7fa7c8107130 2023-09-30T14:30:15.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.827+0000 7fa7cd405700 1 -- 172.21.15.32:0/3648928819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c8107130 msgr2=0x7fa7c8109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:15.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.827+0000 7fa7cd405700 1 --2- 172.21.15.32:0/3648928819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c8107130 0x7fa7c8109520 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0098a0 tx=0x7fa7bc009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:15.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.828+0000 7fa7cd405700 1 -- 172.21.15.32:0/3648928819 shutdown_connections 2023-09-30T14:30:15.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.828+0000 7fa7cd405700 1 --2- 172.21.15.32:0/3648928819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c8107130 0x7fa7c8109520 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:15.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.828+0000 7fa7cd405700 1 -- 172.21.15.32:0/3648928819 >> 172.21.15.32:0/3648928819 conn(0x7fa7c8100b00 msgr2=0x7fa7c8102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:15.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.828+0000 7fa7cd405700 1 -- 172.21.15.32:0/3648928819 shutdown_connections 2023-09-30T14:30:15.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.828+0000 7fa7cd405700 1 -- 172.21.15.32:0/3648928819 wait complete. 2023-09-30T14:30:15.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.828+0000 7fa7cd405700 1 Processor -- start 2023-09-30T14:30:15.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.828+0000 7fa7cd405700 1 -- start start 2023-09-30T14:30:15.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.829+0000 7fa7cd405700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c8107130 0x7fa7c8106a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:15.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.829+0000 7fa7cd405700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c8106fd0 con 0x7fa7c8107130 2023-09-30T14:30:15.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.829+0000 7fa7c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c8107130 0x7fa7c8106a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:15.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.829+0000 7fa7c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c8107130 0x7fa7c8106a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60654/0 (socket says 172.21.15.32:60654) 2023-09-30T14:30:15.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.829+0000 7fa7c6ffd700 1 -- 172.21.15.32:0/1364894520 learned_addr learned my addr 172.21.15.32:0/1364894520 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:15.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.829+0000 7fa7c6ffd700 1 -- 172.21.15.32:0/1364894520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc009550 con 0x7fa7c8107130 2023-09-30T14:30:15.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.829+0000 7fa7c6ffd700 1 --2- 172.21.15.32:0/1364894520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c8107130 0x7fa7c8106a90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc006af0 tx=0x7fa7bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:15.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.829+0000 7fa7affff700 1 -- 172.21.15.32:0/1364894520 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7bc003a40 con 0x7fa7c8107130 2023-09-30T14:30:15.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.829+0000 7fa7cd405700 1 -- 172.21.15.32:0/1364894520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c81050e0 con 0x7fa7c8107130 2023-09-30T14:30:15.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.829+0000 7fa7cd405700 1 -- 172.21.15.32:0/1364894520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c8105580 con 0x7fa7c8107130 2023-09-30T14:30:15.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.829+0000 7fa7affff700 1 -- 172.21.15.32:0/1364894520 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa7bc003ba0 con 0x7fa7c8107130 2023-09-30T14:30:15.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.829+0000 7fa7affff700 1 -- 172.21.15.32:0/1364894520 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7bc01a6e0 con 0x7fa7c8107130 2023-09-30T14:30:15.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.830+0000 7fa7affff700 1 -- 172.21.15.32:0/1364894520 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa7bc01ac40 con 0x7fa7c8107130 2023-09-30T14:30:15.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.830+0000 7fa7affff700 1 -- 172.21.15.32:0/1364894520 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa7bc019700 con 0x7fa7c8107130 2023-09-30T14:30:15.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.830+0000 7fa7cd405700 1 -- 172.21.15.32:0/1364894520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b4005320 con 0x7fa7c8107130 2023-09-30T14:30:15.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.833+0000 7fa7affff700 1 -- 172.21.15.32:0/1364894520 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fa7bc024a60 con 0x7fa7c8107130 2023-09-30T14:30:15.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.961+0000 7fa7cd405700 1 -- 172.21.15.32:0/1364894520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fa7b40058a0 con 0x7fa7c8107130 2023-09-30T14:30:15.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.961+0000 7fa7affff700 1 -- 172.21.15.32:0/1364894520 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fa7bc024c00 con 0x7fa7c8107130 2023-09-30T14:30:15.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.962+0000 7fa7cd405700 1 -- 172.21.15.32:0/1364894520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c8107130 msgr2=0x7fa7c8106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:15.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.962+0000 7fa7cd405700 1 --2- 172.21.15.32:0/1364894520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c8107130 0x7fa7c8106a90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc006af0 tx=0x7fa7bc011ad0 comp rx=0 tx=0).stop 2023-09-30T14:30:15.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.962+0000 7fa7cd405700 1 -- 172.21.15.32:0/1364894520 shutdown_connections 2023-09-30T14:30:15.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.962+0000 7fa7cd405700 1 --2- 172.21.15.32:0/1364894520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c8107130 0x7fa7c8106a90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:15.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.962+0000 7fa7cd405700 1 -- 172.21.15.32:0/1364894520 >> 172.21.15.32:0/1364894520 conn(0x7fa7c8100b00 msgr2=0x7fa7c81018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:15.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.962+0000 7fa7cd405700 1 -- 172.21.15.32:0/1364894520 shutdown_connections 2023-09-30T14:30:15.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:14.962+0000 7fa7cd405700 1 -- 172.21.15.32:0/1364894520 wait complete. 2023-09-30T14:30:15.310 INFO:teuthology.orchestra.run.smithi032.stdout:Restarting the monitor... 2023-09-30T14:30:15.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:15 smithi032 systemd[1]: Stopping Ceph mon.a for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:30:15.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:15 smithi032 ceph-mon[98093]: from='client.? 172.21.15.32:0/1364894520' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:30:15.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:15 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-a[98069]: 2023-09-30T14:30:15.667+0000 7f6fdb0b3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T14:30:15.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:15 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-a[98069]: 2023-09-30T14:30:15.667+0000 7f6fdb0b3700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-09-30T14:30:16.260 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 bash[98431]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-a 2023-09-30T14:30:16.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 systemd[1]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mon.a.service: Succeeded. 2023-09-30T14:30:16.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 systemd[1]: Stopped Ceph mon.a for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:30:16.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 systemd[1]: Starting Ceph mon.a for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:30:16.928 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 podman[98552]: 2023-09-30T14:30:16.928 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: set uid:gid to 167:167 (ceph:ceph) 2023-09-30T14:30:16.928 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: ceph version 18.0.0-6447-g1d10b717 (1d10b71792f3be8887a7631e69851ac2df3585af) reef (dev), process ceph-mon, pid 7 2023-09-30T14:30:16.929 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: pidfile_write: ignore empty --pid-file 2023-09-30T14:30:16.929 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 bash[98552]: 539a940bd728f8eb97cbf7099309ec82af0d06650cb72ad20b530d9b8b0d5c30 2023-09-30T14:30:16.929 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: load: jerasure load: lrc 2023-09-30T14:30:16.929 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: RocksDB version: 7.9.2 2023-09-30T14:30:16.929 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Git sha 0 2023-09-30T14:30:16.929 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Compile date 2023-09-29 16:35:50 2023-09-30T14:30:16.930 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: DB SUMMARY 2023-09-30T14:30:16.930 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: DB Session ID: 5XTF0Q250VVO6KD6ZKM0 2023-09-30T14:30:16.930 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: CURRENT file: CURRENT 2023-09-30T14:30:16.930 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: IDENTITY file: IDENTITY 2023-09-30T14:30:16.930 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-09-30T14:30:16.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-09-30T14:30:16.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99135 ; 2023-09-30T14:30:16.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.error_if_exists: 0 2023-09-30T14:30:16.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.create_if_missing: 0 2023-09-30T14:30:16.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.paranoid_checks: 1 2023-09-30T14:30:16.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-30T14:30:16.932 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-30T14:30:16.932 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-30T14:30:16.932 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.env: 0x55935cf3dda0 2023-09-30T14:30:16.932 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.fs: PosixFileSystem 2023-09-30T14:30:16.932 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.info_log: 0x55935e8d95a0 2023-09-30T14:30:16.933 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_file_opening_threads: 16 2023-09-30T14:30:16.933 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.statistics: (nil) 2023-09-30T14:30:16.933 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.use_fsync: 0 2023-09-30T14:30:16.933 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_log_file_size: 0 2023-09-30T14:30:16.933 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-30T14:30:16.933 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-30T14:30:16.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.keep_log_file_num: 1000 2023-09-30T14:30:16.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.recycle_log_file_num: 0 2023-09-30T14:30:16.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.allow_fallocate: 1 2023-09-30T14:30:16.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.allow_mmap_reads: 0 2023-09-30T14:30:16.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.allow_mmap_writes: 0 2023-09-30T14:30:16.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.use_direct_reads: 0 2023-09-30T14:30:16.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-30T14:30:16.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.create_missing_column_families: 0 2023-09-30T14:30:16.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.db_log_dir: 2023-09-30T14:30:16.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.wal_dir: 2023-09-30T14:30:16.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-30T14:30:16.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-30T14:30:16.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-30T14:30:16.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-30T14:30:16.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-30T14:30:16.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-30T14:30:16.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.advise_random_on_open: 1 2023-09-30T14:30:16.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.db_write_buffer_size: 0 2023-09-30T14:30:16.936 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.write_buffer_manager: 0x55935e891360 2023-09-30T14:30:16.936 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-30T14:30:16.936 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-30T14:30:16.936 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-30T14:30:16.936 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.rate_limiter: (nil) 2023-09-30T14:30:16.936 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-30T14:30:16.936 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.wal_recovery_mode: 2 2023-09-30T14:30:16.936 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.enable_thread_tracking: 0 2023-09-30T14:30:16.936 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.enable_pipelined_write: 0 2023-09-30T14:30:16.936 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.unordered_write: 0 2023-09-30T14:30:16.936 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-30T14:30:16.936 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-30T14:30:16.936 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-30T14:30:16.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-30T14:30:16.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.row_cache: None 2023-09-30T14:30:16.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.wal_filter: None 2023-09-30T14:30:16.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-30T14:30:16.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.allow_ingest_behind: 0 2023-09-30T14:30:16.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.two_write_queues: 0 2023-09-30T14:30:16.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.manual_wal_flush: 0 2023-09-30T14:30:16.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.wal_compression: 0 2023-09-30T14:30:16.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.atomic_flush: 0 2023-09-30T14:30:16.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-30T14:30:16.938 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-30T14:30:16.938 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-30T14:30:16.938 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.log_readahead_size: 0 2023-09-30T14:30:16.938 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-30T14:30:16.938 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.best_efforts_recovery: 0 2023-09-30T14:30:16.938 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-30T14:30:16.938 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-30T14:30:16.938 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.allow_data_in_errors: 0 2023-09-30T14:30:16.938 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.db_host_id: __hostname__ 2023-09-30T14:30:16.938 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-30T14:30:16.938 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_background_jobs: 2 2023-09-30T14:30:16.939 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_background_compactions: -1 2023-09-30T14:30:16.939 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_subcompactions: 1 2023-09-30T14:30:16.939 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-30T14:30:16.939 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-30T14:30:16.939 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-30T14:30:16.939 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_total_wal_size: 0 2023-09-30T14:30:16.939 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-30T14:30:16.939 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-30T14:30:16.939 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-30T14:30:16.939 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-30T14:30:16.939 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_open_files: -1 2023-09-30T14:30:16.939 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.bytes_per_sync: 0 2023-09-30T14:30:16.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-30T14:30:16.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-30T14:30:16.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compaction_readahead_size: 0 2023-09-30T14:30:16.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_background_flushes: -1 2023-09-30T14:30:16.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Compression algorithms supported: 2023-09-30T14:30:16.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-30T14:30:16.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: kZSTD supported: 0 2023-09-30T14:30:16.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: kXpressCompression supported: 0 2023-09-30T14:30:16.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: kLZ4HCCompression supported: 1 2023-09-30T14:30:16.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: kZlibCompression supported: 1 2023-09-30T14:30:16.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: kSnappyCompression supported: 1 2023-09-30T14:30:16.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: kLZ4Compression supported: 1 2023-09-30T14:30:16.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: kBZip2Compression supported: 0 2023-09-30T14:30:16.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-30T14:30:16.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-30T14:30:16.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-09-30T14:30:16.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-30T14:30:16.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-30T14:30:16.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.merge_operator: 2023-09-30T14:30:16.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compaction_filter: None 2023-09-30T14:30:16.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compaction_filter_factory: None 2023-09-30T14:30:16.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.sst_partitioner_factory: None 2023-09-30T14:30:16.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-30T14:30:16.941 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-30T14:30:16.942 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55935e8d96a0) 2023-09-30T14:30:16.943 INFO:journalctl@ceph.mon.a.smithi032.stdout: cache_index_and_filter_blocks: 1 2023-09-30T14:30:16.943 INFO:journalctl@ceph.mon.a.smithi032.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-30T14:30:16.943 INFO:journalctl@ceph.mon.a.smithi032.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-30T14:30:16.943 INFO:journalctl@ceph.mon.a.smithi032.stdout: pin_top_level_index_and_filter: 1 2023-09-30T14:30:16.943 INFO:journalctl@ceph.mon.a.smithi032.stdout: index_type: 0 2023-09-30T14:30:16.943 INFO:journalctl@ceph.mon.a.smithi032.stdout: data_block_index_type: 0 2023-09-30T14:30:16.943 INFO:journalctl@ceph.mon.a.smithi032.stdout: index_shortening: 1 2023-09-30T14:30:16.943 INFO:journalctl@ceph.mon.a.smithi032.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-30T14:30:16.943 INFO:journalctl@ceph.mon.a.smithi032.stdout: checksum: 4 2023-09-30T14:30:16.943 INFO:journalctl@ceph.mon.a.smithi032.stdout: no_block_cache: 0 2023-09-30T14:30:16.944 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache: 0x55935e92f1f0 2023-09-30T14:30:16.944 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache_name: BinnedLRUCache 2023-09-30T14:30:16.944 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache_options: 2023-09-30T14:30:16.944 INFO:journalctl@ceph.mon.a.smithi032.stdout: capacity : 536870912 2023-09-30T14:30:16.944 INFO:journalctl@ceph.mon.a.smithi032.stdout: num_shard_bits : 4 2023-09-30T14:30:16.944 INFO:journalctl@ceph.mon.a.smithi032.stdout: strict_capacity_limit : 0 2023-09-30T14:30:16.944 INFO:journalctl@ceph.mon.a.smithi032.stdout: high_pri_pool_ratio: 0.000 2023-09-30T14:30:16.944 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache_compressed: (nil) 2023-09-30T14:30:16.944 INFO:journalctl@ceph.mon.a.smithi032.stdout: persistent_cache: (nil) 2023-09-30T14:30:16.944 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_size: 4096 2023-09-30T14:30:16.945 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_size_deviation: 10 2023-09-30T14:30:16.945 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_restart_interval: 16 2023-09-30T14:30:16.945 INFO:journalctl@ceph.mon.a.smithi032.stdout: index_block_restart_interval: 1 2023-09-30T14:30:16.945 INFO:journalctl@ceph.mon.a.smithi032.stdout: metadata_block_size: 4096 2023-09-30T14:30:16.945 INFO:journalctl@ceph.mon.a.smithi032.stdout: partition_filters: 0 2023-09-30T14:30:16.945 INFO:journalctl@ceph.mon.a.smithi032.stdout: use_delta_encoding: 1 2023-09-30T14:30:16.945 INFO:journalctl@ceph.mon.a.smithi032.stdout: filter_policy: bloomfilter 2023-09-30T14:30:16.945 INFO:journalctl@ceph.mon.a.smithi032.stdout: whole_key_filtering: 1 2023-09-30T14:30:16.946 INFO:journalctl@ceph.mon.a.smithi032.stdout: verify_compression: 0 2023-09-30T14:30:16.946 INFO:journalctl@ceph.mon.a.smithi032.stdout: read_amp_bytes_per_bit: 0 2023-09-30T14:30:16.946 INFO:journalctl@ceph.mon.a.smithi032.stdout: format_version: 5 2023-09-30T14:30:16.946 INFO:journalctl@ceph.mon.a.smithi032.stdout: enable_index_compression: 1 2023-09-30T14:30:16.946 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_align: 0 2023-09-30T14:30:16.946 INFO:journalctl@ceph.mon.a.smithi032.stdout: max_auto_readahead_size: 262144 2023-09-30T14:30:16.946 INFO:journalctl@ceph.mon.a.smithi032.stdout: prepopulate_block_cache: 0 2023-09-30T14:30:16.947 INFO:journalctl@ceph.mon.a.smithi032.stdout: initial_auto_readahead_size: 8192 2023-09-30T14:30:16.947 INFO:journalctl@ceph.mon.a.smithi032.stdout: num_file_reads_for_auto_readahead: 2 2023-09-30T14:30:16.947 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.write_buffer_size: 33554432 2023-09-30T14:30:16.947 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_write_buffer_number: 2 2023-09-30T14:30:16.947 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compression: NoCompression 2023-09-30T14:30:16.947 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.bottommost_compression: Disabled 2023-09-30T14:30:16.947 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.prefix_extractor: nullptr 2023-09-30T14:30:16.947 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-30T14:30:16.948 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.num_levels: 7 2023-09-30T14:30:16.948 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-30T14:30:16.948 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-30T14:30:16.948 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-30T14:30:16.948 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-30T14:30:16.948 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-30T14:30:16.949 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-30T14:30:16.949 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-30T14:30:16.949 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-30T14:30:16.949 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-30T14:30:16.949 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-30T14:30:16.949 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-30T14:30:16.950 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-30T14:30:16.950 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-30T14:30:16.950 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compression_opts.level: 32767 2023-09-30T14:30:16.950 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compression_opts.strategy: 0 2023-09-30T14:30:16.950 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-30T14:30:16.950 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-30T14:30:16.951 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-30T14:30:16.954 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-30T14:30:16.954 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compression_opts.enabled: false 2023-09-30T14:30:16.954 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-30T14:30:16.954 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-30T14:30:16.954 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-30T14:30:16.955 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-30T14:30:16.955 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.target_file_size_base: 67108864 2023-09-30T14:30:16.955 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-30T14:30:16.955 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-30T14:30:16.955 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-30T14:30:16.956 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-30T14:30:16.956 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-30T14:30:16.956 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-30T14:30:16.956 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-30T14:30:16.956 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-30T14:30:16.956 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-30T14:30:16.957 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-30T14:30:16.957 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-30T14:30:16.957 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-30T14:30:16.957 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-30T14:30:16.957 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-30T14:30:16.957 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.arena_block_size: 1048576 2023-09-30T14:30:16.958 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-30T14:30:16.958 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-30T14:30:16.958 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.disable_auto_compactions: 0 2023-09-30T14:30:16.958 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-30T14:30:16.958 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-30T14:30:16.958 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-30T14:30:16.958 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-30T14:30:16.958 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-30T14:30:16.958 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-30T14:30:16.959 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-30T14:30:16.959 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-30T14:30:16.959 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-30T14:30:16.959 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-30T14:30:16.959 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.table_properties_collectors: 2023-09-30T14:30:16.959 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.inplace_update_support: 0 2023-09-30T14:30:16.959 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-30T14:30:16.960 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-30T14:30:16.960 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-30T14:30:16.960 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-30T14:30:16.960 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.bloom_locality: 0 2023-09-30T14:30:16.960 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.max_successive_merges: 0 2023-09-30T14:30:16.960 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-30T14:30:16.960 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.paranoid_file_checks: 0 2023-09-30T14:30:16.960 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.force_consistency_checks: 1 2023-09-30T14:30:16.961 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.report_bg_io_stats: 0 2023-09-30T14:30:16.961 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.ttl: 2592000 2023-09-30T14:30:16.961 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-30T14:30:16.961 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-30T14:30:16.961 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-30T14:30:16.961 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.enable_blob_files: false 2023-09-30T14:30:16.961 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.min_blob_size: 0 2023-09-30T14:30:16.962 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.blob_file_size: 268435456 2023-09-30T14:30:16.962 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-30T14:30:16.962 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-30T14:30:16.962 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-30T14:30:16.962 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-30T14:30:16.962 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-30T14:30:16.962 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.blob_file_starting_level: 0 2023-09-30T14:30:16.962 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-30T14:30:16.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-09-30T14:30:16.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-09-30T14:30:16.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c8038542-1ea8-463a-bb5f-1f76c9c28fa7 2023-09-30T14:30:16.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696084216869485, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-09-30T14:30:16.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-09-30T14:30:16.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696084216870565, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94354, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92352, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13747, "raw_average_key_size": 49, "raw_value_size": 84849, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696084216, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c8038542-1ea8-463a-bb5f-1f76c9c28fa7", "db_session_id": "5XTF0Q250VVO6KD6ZKM0", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-09-30T14:30:16.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696084216870636, "job": 1, "event": "recovery_finished"} 2023-09-30T14:30:16.963 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-09-30T14:30:16.964 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-30T14:30:16.964 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55935e9cc000 2023-09-30T14:30:16.964 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: DB pointer 0x55935e9b8000 2023-09-30T14:30:16.964 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-30T14:30:16.964 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-30T14:30:16.964 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** DB Stats ** 2023-09-30T14:30:16.964 INFO:journalctl@ceph.mon.a.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-30T14:30:16.965 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-30T14:30:16.965 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-30T14:30:16.965 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-30T14:30:16.965 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-30T14:30:16.965 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-30T14:30:16.965 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-30T14:30:16.965 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2023-09-30T14:30:16.965 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** Compaction Stats [default] ** 2023-09-30T14:30:16.966 INFO:journalctl@ceph.mon.a.smithi032.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-30T14:30:16.966 INFO:journalctl@ceph.mon.a.smithi032.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-30T14:30:16.966 INFO:journalctl@ceph.mon.a.smithi032.stdout: L0 2/0 93.98 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-30T14:30:16.966 INFO:journalctl@ceph.mon.a.smithi032.stdout: Sum 2/0 93.98 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-30T14:30:16.966 INFO:journalctl@ceph.mon.a.smithi032.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-30T14:30:16.966 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2023-09-30T14:30:16.966 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** Compaction Stats [default] ** 2023-09-30T14:30:16.966 INFO:journalctl@ceph.mon.a.smithi032.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-30T14:30:16.966 INFO:journalctl@ceph.mon.a.smithi032.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-30T14:30:16.966 INFO:journalctl@ceph.mon.a.smithi032.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 129.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-30T14:30:16.966 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2023-09-30T14:30:16.967 INFO:journalctl@ceph.mon.a.smithi032.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-30T14:30:16.967 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2023-09-30T14:30:16.967 INFO:journalctl@ceph.mon.a.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-30T14:30:16.967 INFO:journalctl@ceph.mon.a.smithi032.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-30T14:30:16.967 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-30T14:30:16.967 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-30T14:30:16.967 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-30T14:30:16.967 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-30T14:30:16.967 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative compaction: 0.00 GB write, 28.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-30T14:30:16.967 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval compaction: 0.00 GB write, 28.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-30T14:30:16.967 INFO:journalctl@ceph.mon.a.smithi032.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-30T14:30:16.967 INFO:journalctl@ceph.mon.a.smithi032.stdout: Block cache BinnedLRUCache@0x55935e92f1f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-09-30T14:30:16.968 INFO:journalctl@ceph.mon.a.smithi032.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-09-30T14:30:16.968 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2023-09-30T14:30:16.968 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-30T14:30:16.968 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: starting mon.a rank 0 at public addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] at bind addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:30:16.968 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: mon.a@-1(???) e1 preinit fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:30:16.968 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: mon.a@-1(???).mds e1 new map 2023-09-30T14:30:16.968 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: mon.a@-1(???).mds e1 print_map 2023-09-30T14:30:16.968 INFO:journalctl@ceph.mon.a.smithi032.stdout: e1 2023-09-30T14:30:16.968 INFO:journalctl@ceph.mon.a.smithi032.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-30T14:30:16.968 INFO:journalctl@ceph.mon.a.smithi032.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-30T14:30:16.968 INFO:journalctl@ceph.mon.a.smithi032.stdout: legacy client fscid: -1 2023-09-30T14:30:16.968 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2023-09-30T14:30:16.969 INFO:journalctl@ceph.mon.a.smithi032.stdout: No filesystems configured 2023-09-30T14:30:16.969 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-30T14:30:16.969 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-30T14:30:16.969 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-30T14:30:16.969 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-30T14:30:16.969 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-09-30T14:30:16.969 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-30T14:30:16.969 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: monmap e1: 1 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0]} removed_ranks: {} 2023-09-30T14:30:16.969 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: fsmap 2023-09-30T14:30:16.969 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: osdmap e1: 0 total, 0 up, 0 in 2023-09-30T14:30:16.969 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 ceph-mon[98592]: mgrmap e1: no daemons active 2023-09-30T14:30:16.970 INFO:teuthology.orchestra.run.smithi032.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-09-30T14:30:17.186 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:16 smithi032 systemd[1]: Started Ceph mon.a for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:30:17.920 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.431+0000 7ffa4c182700 1 Processor -- start 2023-09-30T14:30:17.920 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.431+0000 7ffa4c182700 1 -- start start 2023-09-30T14:30:17.920 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.431+0000 7ffa4c182700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa44104f60 0x7ffa44105380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:17.920 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.431+0000 7ffa4c182700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa44074620 con 0x7ffa44104f60 2023-09-30T14:30:17.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.432+0000 7ffa49f1e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa44104f60 0x7ffa44105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:17.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.432+0000 7ffa49f1e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa44104f60 0x7ffa44105380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60656/0 (socket says 172.21.15.32:60656) 2023-09-30T14:30:17.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.432+0000 7ffa49f1e700 1 -- 172.21.15.32:0/3830750023 learned_addr learned my addr 172.21.15.32:0/3830750023 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:17.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.432+0000 7ffa49f1e700 1 -- 172.21.15.32:0/3830750023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa441058c0 con 0x7ffa44104f60 2023-09-30T14:30:17.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.432+0000 7ffa49f1e700 1 --2- 172.21.15.32:0/3830750023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa44104f60 0x7ffa44105380 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffa340098a0 tx=0x7ffa34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26b36ad8ffbee91 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:17.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.432+0000 7ffa48f1c700 1 -- 172.21.15.32:0/3830750023 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa34004160 con 0x7ffa44104f60 2023-09-30T14:30:17.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.433+0000 7ffa48f1c700 1 -- 172.21.15.32:0/3830750023 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffa340042c0 con 0x7ffa44104f60 2023-09-30T14:30:17.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.433+0000 7ffa48f1c700 1 -- 172.21.15.32:0/3830750023 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa34011a00 con 0x7ffa44104f60 2023-09-30T14:30:17.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.433+0000 7ffa4c182700 1 -- 172.21.15.32:0/3830750023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa44104f60 msgr2=0x7ffa44105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:17.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.433+0000 7ffa4c182700 1 --2- 172.21.15.32:0/3830750023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa44104f60 0x7ffa44105380 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffa340098a0 tx=0x7ffa34009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:17.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.433+0000 7ffa4c182700 1 -- 172.21.15.32:0/3830750023 shutdown_connections 2023-09-30T14:30:17.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.433+0000 7ffa4c182700 1 --2- 172.21.15.32:0/3830750023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa44104f60 0x7ffa44105380 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:17.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.433+0000 7ffa4c182700 1 -- 172.21.15.32:0/3830750023 >> 172.21.15.32:0/3830750023 conn(0x7ffa44100b80 msgr2=0x7ffa44102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:17.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.433+0000 7ffa4c182700 1 -- 172.21.15.32:0/3830750023 shutdown_connections 2023-09-30T14:30:17.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.433+0000 7ffa4c182700 1 -- 172.21.15.32:0/3830750023 wait complete. 2023-09-30T14:30:17.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.434+0000 7ffa4c182700 1 Processor -- start 2023-09-30T14:30:17.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.434+0000 7ffa4c182700 1 -- start start 2023-09-30T14:30:17.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.434+0000 7ffa4c182700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa44104f60 0x7ffa4419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:17.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.434+0000 7ffa4c182700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4419c9f0 con 0x7ffa44104f60 2023-09-30T14:30:17.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.434+0000 7ffa49f1e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa44104f60 0x7ffa4419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:17.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.434+0000 7ffa49f1e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa44104f60 0x7ffa4419c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60658/0 (socket says 172.21.15.32:60658) 2023-09-30T14:30:17.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.434+0000 7ffa49f1e700 1 -- 172.21.15.32:0/2288975767 learned_addr learned my addr 172.21.15.32:0/2288975767 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:17.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.435+0000 7ffa49f1e700 1 -- 172.21.15.32:0/2288975767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa34009550 con 0x7ffa44104f60 2023-09-30T14:30:17.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.435+0000 7ffa49f1e700 1 --2- 172.21.15.32:0/2288975767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa44104f60 0x7ffa4419c4b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ffa34006af0 tx=0x7ffa34011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:17.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.435+0000 7ffa3affd700 1 -- 172.21.15.32:0/2288975767 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa34003a40 con 0x7ffa44104f60 2023-09-30T14:30:17.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.435+0000 7ffa3affd700 1 -- 172.21.15.32:0/2288975767 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffa34003ba0 con 0x7ffa44104f60 2023-09-30T14:30:17.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.435+0000 7ffa4c182700 1 -- 172.21.15.32:0/2288975767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4419cbf0 con 0x7ffa44104f60 2023-09-30T14:30:17.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.435+0000 7ffa3affd700 1 -- 172.21.15.32:0/2288975767 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa3401a6e0 con 0x7ffa44104f60 2023-09-30T14:30:17.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.435+0000 7ffa4c182700 1 -- 172.21.15.32:0/2288975767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4419d090 con 0x7ffa44104f60 2023-09-30T14:30:17.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.435+0000 7ffa3affd700 1 -- 172.21.15.32:0/2288975767 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffa3401ab60 con 0x7ffa44104f60 2023-09-30T14:30:17.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.435+0000 7ffa3affd700 1 -- 172.21.15.32:0/2288975767 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffa34019700 con 0x7ffa44104f60 2023-09-30T14:30:17.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.436+0000 7ffa4c182700 1 -- 172.21.15.32:0/2288975767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa44195d60 con 0x7ffa44104f60 2023-09-30T14:30:17.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.439+0000 7ffa3affd700 1 -- 172.21.15.32:0/2288975767 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7ffa3402dde0 con 0x7ffa44104f60 2023-09-30T14:30:17.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.565+0000 7ffa4c182700 1 -- 172.21.15.32:0/2288975767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7ffa44106760 con 0x7ffa44104f60 2023-09-30T14:30:17.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.567+0000 7ffa3affd700 1 -- 172.21.15.32:0/2288975767 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7ffa34024a60 con 0x7ffa44104f60 2023-09-30T14:30:17.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.568+0000 7ffa4c182700 1 -- 172.21.15.32:0/2288975767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa44104f60 msgr2=0x7ffa4419c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:17.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.568+0000 7ffa4c182700 1 --2- 172.21.15.32:0/2288975767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa44104f60 0x7ffa4419c4b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ffa34006af0 tx=0x7ffa34011ad0 comp rx=0 tx=0).stop 2023-09-30T14:30:17.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.568+0000 7ffa4c182700 1 -- 172.21.15.32:0/2288975767 shutdown_connections 2023-09-30T14:30:17.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.568+0000 7ffa4c182700 1 --2- 172.21.15.32:0/2288975767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa44104f60 0x7ffa4419c4b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:17.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.568+0000 7ffa4c182700 1 -- 172.21.15.32:0/2288975767 >> 172.21.15.32:0/2288975767 conn(0x7ffa44100b80 msgr2=0x7ffa44190b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:17.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.568+0000 7ffa4c182700 1 -- 172.21.15.32:0/2288975767 shutdown_connections 2023-09-30T14:30:17.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:17.568+0000 7ffa4c182700 1 -- 172.21.15.32:0/2288975767 wait complete. 2023-09-30T14:30:17.924 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote config to /etc/ceph/ceph.conf 2023-09-30T14:30:17.938 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-09-30T14:30:17.939 INFO:teuthology.orchestra.run.smithi032.stdout:Creating mgr... 2023-09-30T14:30:17.939 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying port 0.0.0.0:9283 ... 2023-09-30T14:30:17.940 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying port 0.0.0.0:8765 ... 2023-09-30T14:30:18.152 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mgr.x 2023-09-30T14:30:18.153 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mgr.x.service: Unit ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mgr.x.service not loaded. 2023-09-30T14:30:18.247 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7.target.wants/ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@.service. 2023-09-30T14:30:18.542 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:18 smithi032 systemd[1]: Starting Ceph mgr.x for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:30:18.799 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:18 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2288975767' entity='client.admin' 2023-09-30T14:30:18.799 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:18 smithi032 podman[98819]: 2023-09-30T14:30:18.857 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-30T14:30:18.858 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2023-09-30T14:30:18.865 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-30T14:30:18.865 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2023-09-30T14:30:18.865 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2023-09-30T14:30:18.866 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to enable service . firewalld.service is not available 2023-09-30T14:30:18.873 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-30T14:30:18.873 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2023-09-30T14:30:18.881 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-30T14:30:18.881 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2023-09-30T14:30:18.881 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2023-09-30T14:30:18.881 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-09-30T14:30:18.882 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr to start... 2023-09-30T14:30:18.882 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr... 2023-09-30T14:30:19.055 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:18 smithi032 bash[98819]: 2a8fe76dc9c3f491bdd911bdab47b2629797c157c9aef96207232eeb2bc6228a 2023-09-30T14:30:19.055 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:18 smithi032 systemd[1]: Started Ceph mgr.x for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:30:19.055 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:18 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:18.955+0000 7f88f8bfa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-30T14:30:19.055 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:19 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:19.054+0000 7f88f8bfa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-30T14:30:19.387 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:19 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:19.169+0000 7f88f8bfa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-30T14:30:19.388 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:19 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:19.284+0000 7f88f8bfa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-30T14:30:19.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:19 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1251192796' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-30T14:30:19.888 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:19 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:19.498+0000 7f88f8bfa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-30T14:30:19.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-09-30T14:30:19.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2023-09-30T14:30:19.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "bca3f5c6-5f9d-11ee-8db5-212e2dc638e7", 2023-09-30T14:30:19.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2023-09-30T14:30:19.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-30T14:30:19.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-30T14:30:19.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-30T14:30:19.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:19.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-30T14:30:19.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-30T14:30:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2023-09-30T14:30:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-09-30T14:30:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-30T14:30:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2023-09-30T14:30:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-09-30T14:30:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-09-30T14:30:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-30T14:30:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-30T14:30:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-30T14:30:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-30T14:30:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-30T14:30:19.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-30T14:30:19.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-30T14:30:19.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-30T14:30:19.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-30T14:30:19.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-30T14:30:19.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-30T14:30:19.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-30T14:30:19.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:19.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-30T14:30:19.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-30T14:30:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-30T14:30:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-30T14:30:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-30T14:30:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-30T14:30:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-30T14:30:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-30T14:30:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-30T14:30:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-30T14:30:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-30T14:30:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-30T14:30:19.952 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-30T14:30:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-30T14:30:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-30T14:30:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-30T14:30:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-30T14:30:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-30T14:30:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-30T14:30:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2023-09-30T14:30:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-09-30T14:30:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-30T14:30:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-30T14:30:19.953 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-30T14:30:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2023-09-30T14:30:12.351855+0000", 2023-09-30T14:30:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-30T14:30:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-30T14:30:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2023-09-30T14:30:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.408+0000 7f77de23c700 1 Processor -- start 2023-09-30T14:30:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.408+0000 7f77de23c700 1 -- start start 2023-09-30T14:30:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.409+0000 7f77de23c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d81066c0 0x7f77d8106ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.409+0000 7f77de23c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81070b0 con 0x7f77d81066c0 2023-09-30T14:30:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.409+0000 7f77d77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d81066c0 0x7f77d8106ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.409+0000 7f77d77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d81066c0 0x7f77d8106ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60664/0 (socket says 172.21.15.32:60664) 2023-09-30T14:30:19.954 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.409+0000 7f77d77fe700 1 -- 172.21.15.32:0/788710093 learned_addr learned my addr 172.21.15.32:0/788710093 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.409+0000 7f77d77fe700 1 -- 172.21.15.32:0/788710093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d81078c0 con 0x7f77d81066c0 2023-09-30T14:30:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.409+0000 7f77d77fe700 1 --2- 172.21.15.32:0/788710093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d81066c0 0x7f77d8106ae0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f77c80098a0 tx=0x7f77c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9079f5ef2c1cc3d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.410+0000 7f77d67fc700 1 -- 172.21.15.32:0/788710093 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77c8004160 con 0x7f77d81066c0 2023-09-30T14:30:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.410+0000 7f77d67fc700 1 -- 172.21.15.32:0/788710093 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f77c80042c0 con 0x7f77d81066c0 2023-09-30T14:30:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.410+0000 7f77d67fc700 1 -- 172.21.15.32:0/788710093 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77c8011a00 con 0x7f77d81066c0 2023-09-30T14:30:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.410+0000 7f77de23c700 1 -- 172.21.15.32:0/788710093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d81066c0 msgr2=0x7f77d8106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.410+0000 7f77de23c700 1 --2- 172.21.15.32:0/788710093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d81066c0 0x7f77d8106ae0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f77c80098a0 tx=0x7f77c8009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.410+0000 7f77de23c700 1 -- 172.21.15.32:0/788710093 shutdown_connections 2023-09-30T14:30:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.410+0000 7f77de23c700 1 --2- 172.21.15.32:0/788710093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d81066c0 0x7f77d8106ae0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.411+0000 7f77de23c700 1 -- 172.21.15.32:0/788710093 >> 172.21.15.32:0/788710093 conn(0x7f77d8101c80 msgr2=0x7f77d81040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.411+0000 7f77de23c700 1 -- 172.21.15.32:0/788710093 shutdown_connections 2023-09-30T14:30:19.955 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.411+0000 7f77de23c700 1 -- 172.21.15.32:0/788710093 wait complete. 2023-09-30T14:30:19.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.411+0000 7f77de23c700 1 Processor -- start 2023-09-30T14:30:19.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.412+0000 7f77de23c700 1 -- start start 2023-09-30T14:30:19.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.412+0000 7f77de23c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d81066c0 0x7f77d819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:19.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.412+0000 7f77de23c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d819a850 con 0x7f77d81066c0 2023-09-30T14:30:19.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.412+0000 7f77d77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d81066c0 0x7f77d819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:19.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.412+0000 7f77d77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d81066c0 0x7f77d819a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60666/0 (socket says 172.21.15.32:60666) 2023-09-30T14:30:19.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.412+0000 7f77d77fe700 1 -- 172.21.15.32:0/1251192796 learned_addr learned my addr 172.21.15.32:0/1251192796 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:19.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.412+0000 7f77d77fe700 1 -- 172.21.15.32:0/1251192796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c8009550 con 0x7f77d81066c0 2023-09-30T14:30:19.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.412+0000 7f77d77fe700 1 --2- 172.21.15.32:0/1251192796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d81066c0 0x7f77d819a310 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009fb0 tx=0x7f77c8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:19.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.413+0000 7f77d4ff9700 1 -- 172.21.15.32:0/1251192796 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77c8003930 con 0x7f77d81066c0 2023-09-30T14:30:19.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.413+0000 7f77d4ff9700 1 -- 172.21.15.32:0/1251192796 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f77c8003a90 con 0x7f77d81066c0 2023-09-30T14:30:19.956 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.413+0000 7f77de23c700 1 -- 172.21.15.32:0/1251192796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d819aa50 con 0x7f77d81066c0 2023-09-30T14:30:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.413+0000 7f77d4ff9700 1 -- 172.21.15.32:0/1251192796 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77c80195e0 con 0x7f77d81066c0 2023-09-30T14:30:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.413+0000 7f77de23c700 1 -- 172.21.15.32:0/1251192796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d819aef0 con 0x7f77d81066c0 2023-09-30T14:30:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.413+0000 7f77d4ff9700 1 -- 172.21.15.32:0/1251192796 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f77c8019b70 con 0x7f77d81066c0 2023-09-30T14:30:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.413+0000 7f77d4ff9700 1 -- 172.21.15.32:0/1251192796 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f77c80243f0 con 0x7f77d81066c0 2023-09-30T14:30:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.414+0000 7f77de23c700 1 -- 172.21.15.32:0/1251192796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d8062450 con 0x7f77d81066c0 2023-09-30T14:30:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.416+0000 7f77d4ff9700 1 -- 172.21.15.32:0/1251192796 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f77c802de30 con 0x7f77d81066c0 2023-09-30T14:30:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.550+0000 7f77de23c700 1 -- 172.21.15.32:0/1251192796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f77d810aac0 con 0x7f77d81066c0 2023-09-30T14:30:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.551+0000 7f77d4ff9700 1 -- 172.21.15.32:0/1251192796 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f77c8003c00 con 0x7f77d81066c0 2023-09-30T14:30:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.552+0000 7f77de23c700 1 -- 172.21.15.32:0/1251192796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d81066c0 msgr2=0x7f77d819a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.552+0000 7f77de23c700 1 --2- 172.21.15.32:0/1251192796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d81066c0 0x7f77d819a310 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009fb0 tx=0x7f77c8011ad0 comp rx=0 tx=0).stop 2023-09-30T14:30:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.552+0000 7f77de23c700 1 -- 172.21.15.32:0/1251192796 shutdown_connections 2023-09-30T14:30:19.957 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.552+0000 7f77de23c700 1 --2- 172.21.15.32:0/1251192796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77d81066c0 0x7f77d819a310 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:19.958 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.552+0000 7f77de23c700 1 -- 172.21.15.32:0/1251192796 >> 172.21.15.32:0/1251192796 conn(0x7f77d8101c80 msgr2=0x7f77d8102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:19.958 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.552+0000 7f77de23c700 1 -- 172.21.15.32:0/1251192796 shutdown_connections 2023-09-30T14:30:19.958 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:19.552+0000 7f77de23c700 1 -- 172.21.15.32:0/1251192796 wait complete. 2023-09-30T14:30:19.958 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (1/15)... 2023-09-30T14:30:20.258 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:19 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:19.991+0000 7f88f8bfa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-30T14:30:20.258 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:20 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:20.149+0000 7f88f8bfa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-30T14:30:20.637 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:20 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:20.257+0000 7f88f8bfa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-30T14:30:20.638 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:20 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:20.372+0000 7f88f8bfa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-30T14:30:21.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:20 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:20.884+0000 7f88f8bfa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-30T14:30:22.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:21 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:21.676+0000 7f88f8bfa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-30T14:30:22.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:21 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:21.774+0000 7f88f8bfa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-30T14:30:22.876 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:22 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:22.773+0000 7f88f8bfa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-30T14:30:22.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-09-30T14:30:22.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2023-09-30T14:30:22.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "bca3f5c6-5f9d-11ee-8db5-212e2dc638e7", 2023-09-30T14:30:22.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2023-09-30T14:30:22.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-30T14:30:22.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-30T14:30:22.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-30T14:30:22.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:22.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-30T14:30:22.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-30T14:30:22.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2023-09-30T14:30:22.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-09-30T14:30:22.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-30T14:30:22.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2023-09-30T14:30:22.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-09-30T14:30:22.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2023-09-30T14:30:22.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-30T14:30:22.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-30T14:30:22.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-30T14:30:22.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-30T14:30:22.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:22.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-30T14:30:22.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-30T14:30:22.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-30T14:30:22.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-30T14:30:22.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-30T14:30:23.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-30T14:30:23.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-30T14:30:23.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-30T14:30:23.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:23.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-30T14:30:23.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-30T14:30:23.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-30T14:30:23.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-30T14:30:23.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-30T14:30:23.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-30T14:30:23.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-30T14:30:23.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-30T14:30:23.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-30T14:30:23.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:23.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-30T14:30:23.004 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-30T14:30:23.004 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-30T14:30:23.004 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-30T14:30:23.004 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:23.004 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-30T14:30:23.004 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-30T14:30:23.004 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-30T14:30:23.004 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-30T14:30:23.004 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-30T14:30:23.004 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-30T14:30:23.004 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2023-09-30T14:30:23.004 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-09-30T14:30:23.005 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-30T14:30:23.005 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:23.005 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-30T14:30:23.005 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-30T14:30:23.005 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2023-09-30T14:30:12.351855+0000", 2023-09-30T14:30:23.005 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-30T14:30:23.005 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:23.005 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-30T14:30:23.005 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2023-09-30T14:30:23.005 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.447+0000 7fbb094d5700 1 Processor -- start 2023-09-30T14:30:23.005 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.447+0000 7fbb094d5700 1 -- start start 2023-09-30T14:30:23.005 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.448+0000 7fbb094d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb041055b0 0x7fbb041059d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:23.006 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.448+0000 7fbb094d5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb04105fa0 con 0x7fbb041055b0 2023-09-30T14:30:23.006 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.448+0000 7fbb02ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb041055b0 0x7fbb041059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:23.006 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.448+0000 7fbb02ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb041055b0 0x7fbb041059d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60668/0 (socket says 172.21.15.32:60668) 2023-09-30T14:30:23.006 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.448+0000 7fbb02ffd700 1 -- 172.21.15.32:0/3074482764 learned_addr learned my addr 172.21.15.32:0/3074482764 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:23.006 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.448+0000 7fbb02ffd700 1 -- 172.21.15.32:0/3074482764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb041067b0 con 0x7fbb041055b0 2023-09-30T14:30:23.006 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.448+0000 7fbb02ffd700 1 --2- 172.21.15.32:0/3074482764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb041055b0 0x7fbb041059d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40098a0 tx=0x7fbaf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aff3a6b88ac94fa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:23.006 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.448+0000 7fbb01ffb700 1 -- 172.21.15.32:0/3074482764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbaf4004160 con 0x7fbb041055b0 2023-09-30T14:30:23.006 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.449+0000 7fbb01ffb700 1 -- 172.21.15.32:0/3074482764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbaf40042c0 con 0x7fbb041055b0 2023-09-30T14:30:23.006 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.449+0000 7fbb01ffb700 1 -- 172.21.15.32:0/3074482764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbaf4011a00 con 0x7fbb041055b0 2023-09-30T14:30:23.006 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.449+0000 7fbb094d5700 1 -- 172.21.15.32:0/3074482764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb041055b0 msgr2=0x7fbb041059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:23.006 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.449+0000 7fbb094d5700 1 --2- 172.21.15.32:0/3074482764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb041055b0 0x7fbb041059d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40098a0 tx=0x7fbaf4009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:23.007 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.449+0000 7fbb094d5700 1 -- 172.21.15.32:0/3074482764 shutdown_connections 2023-09-30T14:30:23.007 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.449+0000 7fbb094d5700 1 --2- 172.21.15.32:0/3074482764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb041055b0 0x7fbb041059d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:23.007 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.449+0000 7fbb094d5700 1 -- 172.21.15.32:0/3074482764 >> 172.21.15.32:0/3074482764 conn(0x7fbb04100b70 msgr2=0x7fbb04102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:23.007 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.449+0000 7fbb094d5700 1 -- 172.21.15.32:0/3074482764 shutdown_connections 2023-09-30T14:30:23.007 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.449+0000 7fbb094d5700 1 -- 172.21.15.32:0/3074482764 wait complete. 2023-09-30T14:30:23.007 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.450+0000 7fbb094d5700 1 Processor -- start 2023-09-30T14:30:23.007 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.450+0000 7fbb094d5700 1 -- start start 2023-09-30T14:30:23.007 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.450+0000 7fbb094d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb041055b0 0x7fbb04190740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:23.007 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.450+0000 7fbb094d5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb04190c80 con 0x7fbb041055b0 2023-09-30T14:30:23.007 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.450+0000 7fbb02ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb041055b0 0x7fbb04190740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:23.007 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.450+0000 7fbb02ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb041055b0 0x7fbb04190740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60670/0 (socket says 172.21.15.32:60670) 2023-09-30T14:30:23.007 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.450+0000 7fbb02ffd700 1 -- 172.21.15.32:0/4169692498 learned_addr learned my addr 172.21.15.32:0/4169692498 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:23.008 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.451+0000 7fbb02ffd700 1 -- 172.21.15.32:0/4169692498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf4009550 con 0x7fbb041055b0 2023-09-30T14:30:23.008 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.451+0000 7fbb02ffd700 1 --2- 172.21.15.32:0/4169692498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb041055b0 0x7fbb04190740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4006af0 tx=0x7fbaf4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:23.008 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.451+0000 7fbaebfff700 1 -- 172.21.15.32:0/4169692498 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbaf4003a40 con 0x7fbb041055b0 2023-09-30T14:30:23.008 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.451+0000 7fbaebfff700 1 -- 172.21.15.32:0/4169692498 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbaf4003ba0 con 0x7fbb041055b0 2023-09-30T14:30:23.008 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.451+0000 7fbb094d5700 1 -- 172.21.15.32:0/4169692498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb0418ed90 con 0x7fbb041055b0 2023-09-30T14:30:23.008 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.451+0000 7fbaebfff700 1 -- 172.21.15.32:0/4169692498 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbaf401a700 con 0x7fbb041055b0 2023-09-30T14:30:23.008 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.451+0000 7fbb094d5700 1 -- 172.21.15.32:0/4169692498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0418f230 con 0x7fbb041055b0 2023-09-30T14:30:23.008 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.451+0000 7fbaebfff700 1 -- 172.21.15.32:0/4169692498 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbaf401ac80 con 0x7fbb041055b0 2023-09-30T14:30:23.008 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.451+0000 7fbaebfff700 1 -- 172.21.15.32:0/4169692498 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbaf4019700 con 0x7fbb041055b0 2023-09-30T14:30:23.008 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.452+0000 7fbb094d5700 1 -- 172.21.15.32:0/4169692498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb04062450 con 0x7fbb041055b0 2023-09-30T14:30:23.008 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.455+0000 7fbaebfff700 1 -- 172.21.15.32:0/4169692498 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fbaf4024ad0 con 0x7fbb041055b0 2023-09-30T14:30:23.008 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.588+0000 7fbb094d5700 1 -- 172.21.15.32:0/4169692498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbb0418f470 con 0x7fbb041055b0 2023-09-30T14:30:23.009 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.588+0000 7fbaebfff700 1 -- 172.21.15.32:0/4169692498 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fbb0418f470 con 0x7fbb041055b0 2023-09-30T14:30:23.009 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.589+0000 7fbb094d5700 1 -- 172.21.15.32:0/4169692498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb041055b0 msgr2=0x7fbb04190740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:23.009 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.589+0000 7fbb094d5700 1 --2- 172.21.15.32:0/4169692498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb041055b0 0x7fbb04190740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4006af0 tx=0x7fbaf4011ad0 comp rx=0 tx=0).stop 2023-09-30T14:30:23.009 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.589+0000 7fbb094d5700 1 -- 172.21.15.32:0/4169692498 shutdown_connections 2023-09-30T14:30:23.009 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.589+0000 7fbb094d5700 1 --2- 172.21.15.32:0/4169692498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb041055b0 0x7fbb04190740 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:23.009 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.589+0000 7fbb094d5700 1 -- 172.21.15.32:0/4169692498 >> 172.21.15.32:0/4169692498 conn(0x7fbb04100b70 msgr2=0x7fbb04079940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:23.009 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.590+0000 7fbb094d5700 1 -- 172.21.15.32:0/4169692498 shutdown_connections 2023-09-30T14:30:23.009 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:22.590+0000 7fbb094d5700 1 -- 172.21.15.32:0/4169692498 wait complete. 2023-09-30T14:30:23.009 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (2/15)... 2023-09-30T14:30:23.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:22 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/4169692498' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-30T14:30:23.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:22 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:22.885+0000 7f88f8bfa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-30T14:30:23.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:23 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:23.109+0000 7f88f8bfa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-30T14:30:23.716 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:23 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:23.462+0000 7f88f8bfa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-30T14:30:24.138 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:23 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:23.715+0000 7f88f8bfa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-30T14:30:24.138 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:23 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:23.897+0000 7f88f8bfa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-30T14:30:24.637 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:24 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:24.229+0000 7f88f8bfa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-30T14:30:25.387 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:25 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:25.063+0000 7f88f8bfa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-30T14:30:25.387 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:25 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:25.230+0000 7f88f8bfa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-30T14:30:25.782 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:25 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2014249272' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-30T14:30:25.783 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:25 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:25.482+0000 7f88f8bfa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-30T14:30:25.783 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:25 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:25.621+0000 7f88f8bfa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-30T14:30:26.049 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-09-30T14:30:26.049 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2023-09-30T14:30:26.049 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "bca3f5c6-5f9d-11ee-8db5-212e2dc638e7", 2023-09-30T14:30:26.049 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2023-09-30T14:30:26.049 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-30T14:30:26.049 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-30T14:30:26.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-30T14:30:26.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:26.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-30T14:30:26.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-30T14:30:26.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2023-09-30T14:30:26.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-09-30T14:30:26.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-30T14:30:26.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2023-09-30T14:30:26.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-09-30T14:30:26.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2023-09-30T14:30:26.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-30T14:30:26.050 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-30T14:30:26.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-30T14:30:26.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-30T14:30:26.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:26.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-30T14:30:26.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-30T14:30:26.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-30T14:30:26.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-30T14:30:26.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-30T14:30:26.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-30T14:30:26.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-30T14:30:26.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-30T14:30:26.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:26.051 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-30T14:30:26.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-30T14:30:26.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-30T14:30:26.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-30T14:30:26.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-30T14:30:26.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-30T14:30:26.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-30T14:30:26.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-30T14:30:26.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-30T14:30:26.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:26.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-30T14:30:26.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-30T14:30:26.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-30T14:30:26.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-30T14:30:26.052 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:26.053 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-30T14:30:26.053 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-30T14:30:26.053 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-30T14:30:26.053 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-30T14:30:26.053 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-30T14:30:26.053 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-30T14:30:26.053 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2023-09-30T14:30:26.053 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-09-30T14:30:26.053 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-30T14:30:26.053 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:26.053 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-30T14:30:26.053 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-30T14:30:26.054 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2023-09-30T14:30:12.351855+0000", 2023-09-30T14:30:26.054 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-30T14:30:26.054 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:26.054 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-30T14:30:26.054 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2023-09-30T14:30:26.054 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.483+0000 7fa70d3c4700 1 Processor -- start 2023-09-30T14:30:26.054 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.483+0000 7fa70d3c4700 1 -- start start 2023-09-30T14:30:26.054 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.483+0000 7fa70d3c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa708104d30 0x7fa708105150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:26.054 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.483+0000 7fa70d3c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa708105720 con 0x7fa708104d30 2023-09-30T14:30:26.054 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.483+0000 7fa706ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa708104d30 0x7fa708105150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:26.054 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.483+0000 7fa706ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa708104d30 0x7fa708105150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60672/0 (socket says 172.21.15.32:60672) 2023-09-30T14:30:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.483+0000 7fa706ffd700 1 -- 172.21.15.32:0/1183439906 learned_addr learned my addr 172.21.15.32:0/1183439906 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.484+0000 7fa706ffd700 1 -- 172.21.15.32:0/1183439906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa708105f30 con 0x7fa708104d30 2023-09-30T14:30:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.484+0000 7fa706ffd700 1 --2- 172.21.15.32:0/1183439906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa708104d30 0x7fa708105150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800aa40 tx=0x7fa6f800ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5dc34ac20c846 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.484+0000 7fa7067fc700 1 -- 172.21.15.32:0/1183439906 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6f8004160 con 0x7fa708104d30 2023-09-30T14:30:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.484+0000 7fa7067fc700 1 -- 172.21.15.32:0/1183439906 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa6f800cd90 con 0x7fa708104d30 2023-09-30T14:30:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.484+0000 7fa7067fc700 1 -- 172.21.15.32:0/1183439906 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6f8003c80 con 0x7fa708104d30 2023-09-30T14:30:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.485+0000 7fa70d3c4700 1 -- 172.21.15.32:0/1183439906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa708104d30 msgr2=0x7fa708105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.485+0000 7fa70d3c4700 1 --2- 172.21.15.32:0/1183439906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa708104d30 0x7fa708105150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800aa40 tx=0x7fa6f800ad50 comp rx=0 tx=0).stop 2023-09-30T14:30:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.485+0000 7fa70d3c4700 1 -- 172.21.15.32:0/1183439906 shutdown_connections 2023-09-30T14:30:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.485+0000 7fa70d3c4700 1 --2- 172.21.15.32:0/1183439906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa708104d30 0x7fa708105150 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.485+0000 7fa70d3c4700 1 -- 172.21.15.32:0/1183439906 >> 172.21.15.32:0/1183439906 conn(0x7fa7081002f0 msgr2=0x7fa708102710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:26.055 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.485+0000 7fa70d3c4700 1 -- 172.21.15.32:0/1183439906 shutdown_connections 2023-09-30T14:30:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.485+0000 7fa70d3c4700 1 -- 172.21.15.32:0/1183439906 wait complete. 2023-09-30T14:30:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.486+0000 7fa70d3c4700 1 Processor -- start 2023-09-30T14:30:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.486+0000 7fa70d3c4700 1 -- start start 2023-09-30T14:30:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.486+0000 7fa70d3c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa708104d30 0x7fa70819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.486+0000 7fa70d3c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70819a820 con 0x7fa708104d30 2023-09-30T14:30:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.486+0000 7fa706ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa708104d30 0x7fa70819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.486+0000 7fa706ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa708104d30 0x7fa70819a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60674/0 (socket says 172.21.15.32:60674) 2023-09-30T14:30:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.486+0000 7fa706ffd700 1 -- 172.21.15.32:0/2014249272 learned_addr learned my addr 172.21.15.32:0/2014249272 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.486+0000 7fa706ffd700 1 -- 172.21.15.32:0/2014249272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f800a6f0 con 0x7fa708104d30 2023-09-30T14:30:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.486+0000 7fa706ffd700 1 --2- 172.21.15.32:0/2014249272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa708104d30 0x7fa70819a2e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80037e0 tx=0x7fa6f80041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.486+0000 7fa704ff9700 1 -- 172.21.15.32:0/2014249272 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6f8020400 con 0x7fa708104d30 2023-09-30T14:30:26.056 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.487+0000 7fa704ff9700 1 -- 172.21.15.32:0/2014249272 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa6f8020a00 con 0x7fa708104d30 2023-09-30T14:30:26.057 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.487+0000 7fa70d3c4700 1 -- 172.21.15.32:0/2014249272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa70819aa20 con 0x7fa708104d30 2023-09-30T14:30:26.057 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.487+0000 7fa704ff9700 1 -- 172.21.15.32:0/2014249272 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6f8010930 con 0x7fa708104d30 2023-09-30T14:30:26.057 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.487+0000 7fa70d3c4700 1 -- 172.21.15.32:0/2014249272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa70819aec0 con 0x7fa708104d30 2023-09-30T14:30:26.057 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.487+0000 7fa704ff9700 1 -- 172.21.15.32:0/2014249272 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa6f8010b70 con 0x7fa708104d30 2023-09-30T14:30:26.057 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.487+0000 7fa704ff9700 1 -- 172.21.15.32:0/2014249272 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa6f8018730 con 0x7fa708104d30 2023-09-30T14:30:26.057 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.488+0000 7fa70d3c4700 1 -- 172.21.15.32:0/2014249272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7081919d0 con 0x7fa708104d30 2023-09-30T14:30:26.057 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.490+0000 7fa704ff9700 1 -- 172.21.15.32:0/2014249272 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fa6f8025070 con 0x7fa708104d30 2023-09-30T14:30:26.057 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.624+0000 7fa70d3c4700 1 -- 172.21.15.32:0/2014249272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa7081064c0 con 0x7fa708104d30 2023-09-30T14:30:26.057 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.625+0000 7fa704ff9700 1 -- 172.21.15.32:0/2014249272 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fa6f800b040 con 0x7fa708104d30 2023-09-30T14:30:26.057 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.626+0000 7fa70d3c4700 1 -- 172.21.15.32:0/2014249272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa708104d30 msgr2=0x7fa70819a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:26.057 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.626+0000 7fa70d3c4700 1 --2- 172.21.15.32:0/2014249272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa708104d30 0x7fa70819a2e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80037e0 tx=0x7fa6f80041d0 comp rx=0 tx=0).stop 2023-09-30T14:30:26.057 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.626+0000 7fa70d3c4700 1 -- 172.21.15.32:0/2014249272 shutdown_connections 2023-09-30T14:30:26.058 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.626+0000 7fa70d3c4700 1 --2- 172.21.15.32:0/2014249272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa708104d30 0x7fa70819a2e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:26.058 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.626+0000 7fa70d3c4700 1 -- 172.21.15.32:0/2014249272 >> 172.21.15.32:0/2014249272 conn(0x7fa7081002f0 msgr2=0x7fa70818c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:26.058 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.626+0000 7fa70d3c4700 1 -- 172.21.15.32:0/2014249272 shutdown_connections 2023-09-30T14:30:26.058 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:25.626+0000 7fa70d3c4700 1 -- 172.21.15.32:0/2014249272 wait complete. 2023-09-30T14:30:26.058 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (3/15)... 2023-09-30T14:30:26.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:25 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:25.901+0000 7f88f8bfa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-30T14:30:27.387 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:26 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:26.990+0000 7f88f8bfa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-30T14:30:28.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:27 smithi032 ceph-mon[98592]: Activating manager daemon x 2023-09-30T14:30:28.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:27 smithi032 ceph-mon[98592]: mgrmap e2: x(active, starting, since 0.00327904s) 2023-09-30T14:30:28.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:27 smithi032 ceph-mon[98592]: from='mgr.14100 172.21.15.32:0/1471113604' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-30T14:30:28.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:27 smithi032 ceph-mon[98592]: from='mgr.14100 172.21.15.32:0/1471113604' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-30T14:30:28.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:27 smithi032 ceph-mon[98592]: from='mgr.14100 172.21.15.32:0/1471113604' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-30T14:30:28.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:27 smithi032 ceph-mon[98592]: from='mgr.14100 172.21.15.32:0/1471113604' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:30:28.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:27 smithi032 ceph-mon[98592]: from='mgr.14100 172.21.15.32:0/1471113604' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-30T14:30:28.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:27 smithi032 ceph-mon[98592]: Manager daemon x is now available 2023-09-30T14:30:28.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:27 smithi032 ceph-mon[98592]: from='mgr.14100 172.21.15.32:0/1471113604' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-30T14:30:28.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:27 smithi032 ceph-mon[98592]: from='mgr.14100 172.21.15.32:0/1471113604' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-30T14:30:28.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:27 smithi032 ceph-mon[98592]: from='mgr.14100 172.21.15.32:0/1471113604' entity='mgr.x' 2023-09-30T14:30:28.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:27 smithi032 ceph-mon[98592]: from='mgr.14100 172.21.15.32:0/1471113604' entity='mgr.x' 2023-09-30T14:30:28.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:27 smithi032 ceph-mon[98592]: from='mgr.14100 172.21.15.32:0/1471113604' entity='mgr.x' 2023-09-30T14:30:29.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:29 smithi032 ceph-mon[98592]: mgrmap e3: x(active, since 1.00691s) 2023-09-30T14:30:31.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:31 smithi032 ceph-mon[98592]: mgrmap e4: x(active, since 3s) 2023-09-30T14:30:31.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:31 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2020621363' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-30T14:30:33.105 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-09-30T14:30:33.106 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2023-09-30T14:30:33.106 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "bca3f5c6-5f9d-11ee-8db5-212e2dc638e7", 2023-09-30T14:30:33.106 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2023-09-30T14:30:33.106 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-30T14:30:33.106 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-30T14:30:33.106 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-30T14:30:33.106 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:33.106 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-30T14:30:33.106 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-30T14:30:33.106 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2023-09-30T14:30:33.107 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-09-30T14:30:33.107 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-30T14:30:33.107 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2023-09-30T14:30:33.107 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-09-30T14:30:33.107 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2023-09-30T14:30:33.107 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-30T14:30:33.107 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-30T14:30:33.107 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-30T14:30:33.107 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-30T14:30:33.107 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:33.107 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-30T14:30:33.107 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-30T14:30:33.107 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-30T14:30:33.108 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-30T14:30:33.108 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-30T14:30:33.116 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-30T14:30:33.116 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-30T14:30:33.116 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-30T14:30:33.116 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:33.116 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-30T14:30:33.116 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-30T14:30:33.116 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-30T14:30:33.116 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-30T14:30:33.116 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-30T14:30:33.116 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-30T14:30:33.116 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-30T14:30:33.117 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-30T14:30:33.117 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-30T14:30:33.117 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:33.117 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-30T14:30:33.117 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-30T14:30:33.117 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-30T14:30:33.117 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-30T14:30:33.117 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:33.117 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-30T14:30:33.117 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-30T14:30:33.117 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-30T14:30:33.117 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-30T14:30:33.117 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-30T14:30:33.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-30T14:30:33.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2023-09-30T14:30:33.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2023-09-30T14:30:33.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-30T14:30:33.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:33.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-30T14:30:33.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-30T14:30:33.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2023-09-30T14:30:12.351855+0000", 2023-09-30T14:30:33.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-30T14:30:33.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2023-09-30T14:30:33.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-30T14:30:33.118 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2023-09-30T14:30:33.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.507+0000 7fc0159a7700 1 Processor -- start 2023-09-30T14:30:33.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.507+0000 7fc0159a7700 1 -- start start 2023-09-30T14:30:33.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.507+0000 7fc0159a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc010105ab0 0x7fc010105ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:33.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.507+0000 7fc0159a7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0101064a0 con 0x7fc010105ab0 2023-09-30T14:30:33.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.508+0000 7fc00effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc010105ab0 0x7fc010105ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:33.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.508+0000 7fc00effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc010105ab0 0x7fc010105ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60700/0 (socket says 172.21.15.32:60700) 2023-09-30T14:30:33.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.508+0000 7fc00effd700 1 -- 172.21.15.32:0/2252739952 learned_addr learned my addr 172.21.15.32:0/2252739952 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:33.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.508+0000 7fc00effd700 1 -- 172.21.15.32:0/2252739952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0101067d0 con 0x7fc010105ab0 2023-09-30T14:30:33.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.508+0000 7fc00effd700 1 --2- 172.21.15.32:0/2252739952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc010105ab0 0x7fc010105ed0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc0000098a0 tx=0x7fc000009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76dff6d6780edfaf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:33.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.508+0000 7fc00dffb700 1 -- 172.21.15.32:0/2252739952 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc000004160 con 0x7fc010105ab0 2023-09-30T14:30:33.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.508+0000 7fc00dffb700 1 -- 172.21.15.32:0/2252739952 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc0000042c0 con 0x7fc010105ab0 2023-09-30T14:30:33.119 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.508+0000 7fc00dffb700 1 -- 172.21.15.32:0/2252739952 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc000011a00 con 0x7fc010105ab0 2023-09-30T14:30:33.120 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.509+0000 7fc0159a7700 1 -- 172.21.15.32:0/2252739952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc010105ab0 msgr2=0x7fc010105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:33.120 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.509+0000 7fc0159a7700 1 --2- 172.21.15.32:0/2252739952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc010105ab0 0x7fc010105ed0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc0000098a0 tx=0x7fc000009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:33.120 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.509+0000 7fc0159a7700 1 -- 172.21.15.32:0/2252739952 shutdown_connections 2023-09-30T14:30:33.120 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.509+0000 7fc0159a7700 1 --2- 172.21.15.32:0/2252739952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc010105ab0 0x7fc010105ed0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:33.120 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.509+0000 7fc0159a7700 1 -- 172.21.15.32:0/2252739952 >> 172.21.15.32:0/2252739952 conn(0x7fc0101002f0 msgr2=0x7fc010102710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:33.120 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.509+0000 7fc0159a7700 1 -- 172.21.15.32:0/2252739952 shutdown_connections 2023-09-30T14:30:33.120 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.509+0000 7fc0159a7700 1 -- 172.21.15.32:0/2252739952 wait complete. 2023-09-30T14:30:33.120 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.510+0000 7fc0159a7700 1 Processor -- start 2023-09-30T14:30:33.120 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.510+0000 7fc0159a7700 1 -- start start 2023-09-30T14:30:33.120 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.510+0000 7fc0159a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc010105ab0 0x7fc0101051d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:33.120 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.510+0000 7fc0159a7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01019cf90 con 0x7fc010105ab0 2023-09-30T14:30:33.120 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.510+0000 7fc00effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc010105ab0 0x7fc0101051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:33.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.510+0000 7fc00effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc010105ab0 0x7fc0101051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60702/0 (socket says 172.21.15.32:60702) 2023-09-30T14:30:33.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.510+0000 7fc00effd700 1 -- 172.21.15.32:0/2020621363 learned_addr learned my addr 172.21.15.32:0/2020621363 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:33.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.510+0000 7fc00effd700 1 -- 172.21.15.32:0/2020621363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc000009550 con 0x7fc010105ab0 2023-09-30T14:30:33.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.511+0000 7fc00effd700 1 --2- 172.21.15.32:0/2020621363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc010105ab0 0x7fc0101051d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc000006af0 tx=0x7fc000011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:33.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.511+0000 7fc0149a5700 1 -- 172.21.15.32:0/2020621363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc000003a40 con 0x7fc010105ab0 2023-09-30T14:30:33.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.511+0000 7fc0149a5700 1 -- 172.21.15.32:0/2020621363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc000003ba0 con 0x7fc010105ab0 2023-09-30T14:30:33.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.511+0000 7fc0159a7700 1 -- 172.21.15.32:0/2020621363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01019d0d0 con 0x7fc010105ab0 2023-09-30T14:30:33.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.511+0000 7fc0149a5700 1 -- 172.21.15.32:0/2020621363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc00001a6e0 con 0x7fc010105ab0 2023-09-30T14:30:33.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.511+0000 7fc0159a7700 1 -- 172.21.15.32:0/2020621363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01019d570 con 0x7fc010105ab0 2023-09-30T14:30:33.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.512+0000 7fc0149a5700 1 -- 172.21.15.32:0/2020621363 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 3) v1 ==== 45375+0+0 (secure 0 0 0) 0x7fc000024430 con 0x7fc010105ab0 2023-09-30T14:30:33.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.512+0000 7fc0149a5700 1 --2- 172.21.15.32:0/2020621363 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7fbff8038550 0x7fbff803aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:33.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.512+0000 7fc0159a7700 1 -- 172.21.15.32:0/2020621363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc010062450 con 0x7fc010105ab0 2023-09-30T14:30:33.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.512+0000 7fc0149a5700 1 -- 172.21.15.32:0/2020621363 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc00004c3c0 con 0x7fc010105ab0 2023-09-30T14:30:33.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.512+0000 7fc00e7fc700 1 --2- 172.21.15.32:0/2020621363 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7fbff8038550 0x7fbff803aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:33.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.513+0000 7fc00e7fc700 1 --2- 172.21.15.32:0/2020621363 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7fbff8038550 0x7fbff803aa10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc004006fd0 tx=0x7fc004006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:33.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:29.518+0000 7fc0149a5700 1 -- 172.21.15.32:0/2020621363 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fc0000189a0 con 0x7fc010105ab0 2023-09-30T14:30:33.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:30.009+0000 7fc0149a5700 1 -- 172.21.15.32:0/2020621363 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 4) v1 ==== 45481+0+0 (secure 0 0 0) 0x7fc000024a60 con 0x7fc010105ab0 2023-09-30T14:30:33.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:30.050+0000 7fc0159a7700 1 -- 172.21.15.32:0/2020621363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc01019d7b0 con 0x7fc010105ab0 2023-09-30T14:30:33.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:30.051+0000 7fc0149a5700 1 -- 172.21.15.32:0/2020621363 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fc000018d90 con 0x7fc010105ab0 2023-09-30T14:30:33.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:30.053+0000 7fc0159a7700 1 -- 172.21.15.32:0/2020621363 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7fbff8038550 msgr2=0x7fbff803aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:33.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:30.053+0000 7fc0159a7700 1 --2- 172.21.15.32:0/2020621363 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7fbff8038550 0x7fbff803aa10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc004006fd0 tx=0x7fc004006e40 comp rx=0 tx=0).stop 2023-09-30T14:30:33.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:30.053+0000 7fc0159a7700 1 -- 172.21.15.32:0/2020621363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc010105ab0 msgr2=0x7fc0101051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:33.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:30.053+0000 7fc0159a7700 1 --2- 172.21.15.32:0/2020621363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc010105ab0 0x7fc0101051d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc000006af0 tx=0x7fc000011ad0 comp rx=0 tx=0).stop 2023-09-30T14:30:33.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:30.053+0000 7fc0159a7700 1 -- 172.21.15.32:0/2020621363 shutdown_connections 2023-09-30T14:30:33.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:30.053+0000 7fc0159a7700 1 --2- 172.21.15.32:0/2020621363 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7fbff8038550 0x7fbff803aa10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:33.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:30.053+0000 7fc0159a7700 1 --2- 172.21.15.32:0/2020621363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc010105ab0 0x7fc0101051d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:33.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:30.053+0000 7fc0159a7700 1 -- 172.21.15.32:0/2020621363 >> 172.21.15.32:0/2020621363 conn(0x7fc0101002f0 msgr2=0x7fc010101050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:33.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:30.053+0000 7fc0159a7700 1 -- 172.21.15.32:0/2020621363 shutdown_connections 2023-09-30T14:30:33.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:30.053+0000 7fc0159a7700 1 -- 172.21.15.32:0/2020621363 wait complete. 2023-09-30T14:30:33.123 INFO:teuthology.orchestra.run.smithi032.stdout:mgr is available 2023-09-30T14:30:35.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:35 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1503882790' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-30T14:30:36.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-09-30T14:30:36.667 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [global] 2023-09-30T14:30:36.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout fsid = bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:30:36.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-30T14:30:36.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-30T14:30:36.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-30T14:30:36.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-30T14:30:36.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-09-30T14:30:36.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-30T14:30:36.668 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-30T14:30:36.674 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2023-09-30T14:30:36.674 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [osd] 2023-09-30T14:30:36.674 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-30T14:30:36.674 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-30T14:30:36.674 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-30T14:30:36.674 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-30T14:30:36.674 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-30T14:30:36.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.082+0000 7f8accc9d700 1 Processor -- start 2023-09-30T14:30:36.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.082+0000 7f8accc9d700 1 -- start start 2023-09-30T14:30:36.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.082+0000 7f8accc9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ac81049a0 0x7f8ac8104dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:36.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.082+0000 7f8accc9d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac8105390 con 0x7f8ac81049a0 2023-09-30T14:30:36.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.082+0000 7f8ac659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ac81049a0 0x7f8ac8104dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:36.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.082+0000 7f8ac659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ac81049a0 0x7f8ac8104dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60706/0 (socket says 172.21.15.32:60706) 2023-09-30T14:30:36.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.082+0000 7f8ac659c700 1 -- 172.21.15.32:0/3591293566 learned_addr learned my addr 172.21.15.32:0/3591293566 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:36.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.083+0000 7f8ac659c700 1 -- 172.21.15.32:0/3591293566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac8107930 con 0x7f8ac81049a0 2023-09-30T14:30:36.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.083+0000 7f8ac659c700 1 --2- 172.21.15.32:0/3591293566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ac81049a0 0x7f8ac8104dc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80098a0 tx=0x7f8ab8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc9769de9f1900a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:36.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.083+0000 7f8ac559a700 1 -- 172.21.15.32:0/3591293566 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ab8004160 con 0x7f8ac81049a0 2023-09-30T14:30:36.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.083+0000 7f8ac559a700 1 -- 172.21.15.32:0/3591293566 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8ab80042c0 con 0x7f8ac81049a0 2023-09-30T14:30:36.675 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.083+0000 7f8ac559a700 1 -- 172.21.15.32:0/3591293566 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ab8011a00 con 0x7f8ac81049a0 2023-09-30T14:30:36.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.084+0000 7f8accc9d700 1 -- 172.21.15.32:0/3591293566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ac81049a0 msgr2=0x7f8ac8104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:36.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.084+0000 7f8accc9d700 1 --2- 172.21.15.32:0/3591293566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ac81049a0 0x7f8ac8104dc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80098a0 tx=0x7f8ab8009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:36.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.084+0000 7f8accc9d700 1 -- 172.21.15.32:0/3591293566 shutdown_connections 2023-09-30T14:30:36.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.084+0000 7f8accc9d700 1 --2- 172.21.15.32:0/3591293566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ac81049a0 0x7f8ac8104dc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:36.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.084+0000 7f8accc9d700 1 -- 172.21.15.32:0/3591293566 >> 172.21.15.32:0/3591293566 conn(0x7f8ac81002f0 msgr2=0x7f8ac8102710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:36.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.084+0000 7f8accc9d700 1 -- 172.21.15.32:0/3591293566 shutdown_connections 2023-09-30T14:30:36.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.084+0000 7f8accc9d700 1 -- 172.21.15.32:0/3591293566 wait complete. 2023-09-30T14:30:36.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.085+0000 7f8accc9d700 1 Processor -- start 2023-09-30T14:30:36.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.085+0000 7f8accc9d700 1 -- start start 2023-09-30T14:30:36.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.085+0000 7f8accc9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ac81049a0 0x7f8ac8107290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:36.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.085+0000 7f8accc9d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac81077d0 con 0x7f8ac81049a0 2023-09-30T14:30:36.676 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.085+0000 7f8ac659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ac81049a0 0x7f8ac8107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:36.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.085+0000 7f8ac659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ac81049a0 0x7f8ac8107290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60708/0 (socket says 172.21.15.32:60708) 2023-09-30T14:30:36.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.085+0000 7f8ac659c700 1 -- 172.21.15.32:0/1503882790 learned_addr learned my addr 172.21.15.32:0/1503882790 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:36.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.085+0000 7f8ac659c700 1 -- 172.21.15.32:0/1503882790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab8009550 con 0x7f8ac81049a0 2023-09-30T14:30:36.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.085+0000 7f8ac659c700 1 --2- 172.21.15.32:0/1503882790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ac81049a0 0x7f8ac8107290 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8009fb0 tx=0x7f8ab8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:36.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.085+0000 7f8ab77fe700 1 -- 172.21.15.32:0/1503882790 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ab8003930 con 0x7f8ac81049a0 2023-09-30T14:30:36.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.085+0000 7f8ab77fe700 1 -- 172.21.15.32:0/1503882790 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8ab8003a90 con 0x7f8ac81049a0 2023-09-30T14:30:36.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.086+0000 7f8accc9d700 1 -- 172.21.15.32:0/1503882790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac81058e0 con 0x7f8ac81049a0 2023-09-30T14:30:36.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.086+0000 7f8ab77fe700 1 -- 172.21.15.32:0/1503882790 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ab80195e0 con 0x7f8ac81049a0 2023-09-30T14:30:36.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.086+0000 7f8accc9d700 1 -- 172.21.15.32:0/1503882790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac8105d80 con 0x7f8ac81049a0 2023-09-30T14:30:36.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.086+0000 7f8ab77fe700 1 -- 172.21.15.32:0/1503882790 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 45481+0+0 (secure 0 0 0) 0x7f8ab8019850 con 0x7f8ac81049a0 2023-09-30T14:30:36.677 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.087+0000 7f8ab77fe700 1 --2- 172.21.15.32:0/1503882790 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7f8ab0038630 0x7f8ab003aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:36.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.087+0000 7f8accc9d700 1 -- 172.21.15.32:0/1503882790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac8062450 con 0x7f8ac81049a0 2023-09-30T14:30:36.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.087+0000 7f8ab77fe700 1 -- 172.21.15.32:0/1503882790 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8ab804c380 con 0x7f8ac81049a0 2023-09-30T14:30:36.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.087+0000 7f8ac5d9b700 1 --2- 172.21.15.32:0/1503882790 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7f8ab0038630 0x7f8ab003aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:36.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.087+0000 7f8ac5d9b700 1 --2- 172.21.15.32:0/1503882790 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7f8ab0038630 0x7f8ab003aaf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8abc006fd0 tx=0x7f8abc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:36.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.092+0000 7f8ab77fe700 1 -- 172.21.15.32:0/1503882790 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f8ab8019b00 con 0x7f8ac81049a0 2023-09-30T14:30:36.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.449+0000 7f8accc9d700 1 -- 172.21.15.32:0/1503882790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f8ac810ab30 con 0x7f8ac81049a0 2023-09-30T14:30:36.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.450+0000 7f8ab77fe700 1 -- 172.21.15.32:0/1503882790 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f8ab80172b0 con 0x7f8ac81049a0 2023-09-30T14:30:36.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.453+0000 7f8accc9d700 1 -- 172.21.15.32:0/1503882790 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7f8ab0038630 msgr2=0x7f8ab003aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:36.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.453+0000 7f8accc9d700 1 --2- 172.21.15.32:0/1503882790 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7f8ab0038630 0x7f8ab003aaf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8abc006fd0 tx=0x7f8abc006e40 comp rx=0 tx=0).stop 2023-09-30T14:30:36.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.453+0000 7f8accc9d700 1 -- 172.21.15.32:0/1503882790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ac81049a0 msgr2=0x7f8ac8107290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:36.678 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.453+0000 7f8accc9d700 1 --2- 172.21.15.32:0/1503882790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ac81049a0 0x7f8ac8107290 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8009fb0 tx=0x7f8ab8011ad0 comp rx=0 tx=0).stop 2023-09-30T14:30:36.679 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.453+0000 7f8accc9d700 1 -- 172.21.15.32:0/1503882790 shutdown_connections 2023-09-30T14:30:36.679 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.453+0000 7f8accc9d700 1 --2- 172.21.15.32:0/1503882790 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7f8ab0038630 0x7f8ab003aaf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:36.679 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.453+0000 7f8accc9d700 1 --2- 172.21.15.32:0/1503882790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ac81049a0 0x7f8ac8107290 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:36.679 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.453+0000 7f8accc9d700 1 -- 172.21.15.32:0/1503882790 >> 172.21.15.32:0/1503882790 conn(0x7f8ac81002f0 msgr2=0x7f8ac81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:36.679 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.453+0000 7f8accc9d700 1 -- 172.21.15.32:0/1503882790 shutdown_connections 2023-09-30T14:30:36.679 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:35.453+0000 7f8accc9d700 1 -- 172.21.15.32:0/1503882790 wait complete. 2023-09-30T14:30:36.679 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling cephadm module... 2023-09-30T14:30:38.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:37 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3582324474' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-09-30T14:30:38.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:37 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: ignoring --setuser ceph since I am not root 2023-09-30T14:30:38.138 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:37 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: ignoring --setgroup ceph since I am not root 2023-09-30T14:30:38.138 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:38 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:38.079+0000 7ff4ce23d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-30T14:30:38.226 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.186+0000 7efcdb77f700 1 Processor -- start 2023-09-30T14:30:38.226 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.186+0000 7efcdb77f700 1 -- start start 2023-09-30T14:30:38.226 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.186+0000 7efcdb77f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd4107130 0x7efcd4109520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:38.227 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.186+0000 7efcdb77f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd4074660 con 0x7efcd4107130 2023-09-30T14:30:38.227 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.186+0000 7efcd951b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd4107130 0x7efcd4109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:38.227 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.186+0000 7efcd951b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd4107130 0x7efcd4109520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60712/0 (socket says 172.21.15.32:60712) 2023-09-30T14:30:38.227 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.186+0000 7efcd951b700 1 -- 172.21.15.32:0/3561538726 learned_addr learned my addr 172.21.15.32:0/3561538726 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:38.227 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.186+0000 7efcd951b700 1 -- 172.21.15.32:0/3561538726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd4109a60 con 0x7efcd4107130 2023-09-30T14:30:38.227 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.186+0000 7efcd951b700 1 --2- 172.21.15.32:0/3561538726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd4107130 0x7efcd4109520 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efcd00098a0 tx=0x7efcd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b072e64f4d1403aa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:38.227 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.187+0000 7efcc7fff700 1 -- 172.21.15.32:0/3561538726 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcd0004160 con 0x7efcd4107130 2023-09-30T14:30:38.227 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.187+0000 7efcc7fff700 1 -- 172.21.15.32:0/3561538726 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efcd00042c0 con 0x7efcd4107130 2023-09-30T14:30:38.227 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.187+0000 7efcc7fff700 1 -- 172.21.15.32:0/3561538726 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcd0011a00 con 0x7efcd4107130 2023-09-30T14:30:38.227 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.187+0000 7efcdb77f700 1 -- 172.21.15.32:0/3561538726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd4107130 msgr2=0x7efcd4109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:38.227 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.187+0000 7efcdb77f700 1 --2- 172.21.15.32:0/3561538726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd4107130 0x7efcd4109520 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efcd00098a0 tx=0x7efcd0009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:38.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.187+0000 7efcdb77f700 1 -- 172.21.15.32:0/3561538726 shutdown_connections 2023-09-30T14:30:38.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.187+0000 7efcdb77f700 1 --2- 172.21.15.32:0/3561538726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd4107130 0x7efcd4109520 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:38.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.187+0000 7efcdb77f700 1 -- 172.21.15.32:0/3561538726 >> 172.21.15.32:0/3561538726 conn(0x7efcd4100b80 msgr2=0x7efcd4102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:38.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.187+0000 7efcdb77f700 1 -- 172.21.15.32:0/3561538726 shutdown_connections 2023-09-30T14:30:38.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.187+0000 7efcdb77f700 1 -- 172.21.15.32:0/3561538726 wait complete. 2023-09-30T14:30:38.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.188+0000 7efcdb77f700 1 Processor -- start 2023-09-30T14:30:38.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.188+0000 7efcdb77f700 1 -- start start 2023-09-30T14:30:38.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.188+0000 7efcdb77f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd4107130 0x7efcd419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:38.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.188+0000 7efcdb77f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd419a7a0 con 0x7efcd4107130 2023-09-30T14:30:38.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.188+0000 7efcd951b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd4107130 0x7efcd419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:38.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.188+0000 7efcd951b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd4107130 0x7efcd419a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60714/0 (socket says 172.21.15.32:60714) 2023-09-30T14:30:38.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.188+0000 7efcd951b700 1 -- 172.21.15.32:0/3582324474 learned_addr learned my addr 172.21.15.32:0/3582324474 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:38.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.189+0000 7efcd951b700 1 -- 172.21.15.32:0/3582324474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd0009550 con 0x7efcd4107130 2023-09-30T14:30:38.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.189+0000 7efcd951b700 1 --2- 172.21.15.32:0/3582324474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd4107130 0x7efcd419a260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efcd0006af0 tx=0x7efcd0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:38.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.189+0000 7efcc67fc700 1 -- 172.21.15.32:0/3582324474 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcd0003a40 con 0x7efcd4107130 2023-09-30T14:30:38.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.189+0000 7efcc67fc700 1 -- 172.21.15.32:0/3582324474 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efcd0003ba0 con 0x7efcd4107130 2023-09-30T14:30:38.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.189+0000 7efcdb77f700 1 -- 172.21.15.32:0/3582324474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd419a9a0 con 0x7efcd4107130 2023-09-30T14:30:38.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.189+0000 7efcc67fc700 1 -- 172.21.15.32:0/3582324474 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcd001a6e0 con 0x7efcd4107130 2023-09-30T14:30:38.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.189+0000 7efcdb77f700 1 -- 172.21.15.32:0/3582324474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd419ae40 con 0x7efcd4107130 2023-09-30T14:30:38.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.190+0000 7efcc67fc700 1 -- 172.21.15.32:0/3582324474 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 45481+0+0 (secure 0 0 0) 0x7efcd001a840 con 0x7efcd4107130 2023-09-30T14:30:38.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.190+0000 7efcc67fc700 1 --2- 172.21.15.32:0/3582324474 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7efcc0038680 0x7efcc003ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:38.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.190+0000 7efcc67fc700 1 -- 172.21.15.32:0/3582324474 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efcd004c5c0 con 0x7efcd4107130 2023-09-30T14:30:38.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.190+0000 7efcdb77f700 1 -- 172.21.15.32:0/3582324474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd410a860 con 0x7efcd4107130 2023-09-30T14:30:38.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.190+0000 7efcd8d1a700 1 --2- 172.21.15.32:0/3582324474 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7efcc0038680 0x7efcc003ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:38.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.191+0000 7efcd8d1a700 1 --2- 172.21.15.32:0/3582324474 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7efcc0038680 0x7efcc003ab40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efcc8006fd0 tx=0x7efcc8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:38.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.195+0000 7efcc67fc700 1 -- 172.21.15.32:0/3582324474 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7efcd001aaf0 con 0x7efcd4107130 2023-09-30T14:30:38.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.673+0000 7efcdb77f700 1 -- 172.21.15.32:0/3582324474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7efcd410aa50 con 0x7efcd4107130 2023-09-30T14:30:38.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.881+0000 7efcc67fc700 1 -- 172.21.15.32:0/3582324474 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7efcd0018350 con 0x7efcd4107130 2023-09-30T14:30:38.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.883+0000 7efcc67fc700 1 -- 172.21.15.32:0/3582324474 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 5) v1 ==== 45492+0+0 (secure 0 0 0) 0x7efcd002d910 con 0x7efcd4107130 2023-09-30T14:30:38.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.884+0000 7efcdb77f700 1 -- 172.21.15.32:0/3582324474 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7efcc0038680 msgr2=0x7efcc003ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:38.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.884+0000 7efcdb77f700 1 --2- 172.21.15.32:0/3582324474 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7efcc0038680 0x7efcc003ab40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efcc8006fd0 tx=0x7efcc8006e40 comp rx=0 tx=0).stop 2023-09-30T14:30:38.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.884+0000 7efcdb77f700 1 -- 172.21.15.32:0/3582324474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd4107130 msgr2=0x7efcd419a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:38.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.884+0000 7efcdb77f700 1 --2- 172.21.15.32:0/3582324474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd4107130 0x7efcd419a260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efcd0006af0 tx=0x7efcd0011ad0 comp rx=0 tx=0).stop 2023-09-30T14:30:38.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.884+0000 7efcdb77f700 1 -- 172.21.15.32:0/3582324474 shutdown_connections 2023-09-30T14:30:38.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.884+0000 7efcdb77f700 1 --2- 172.21.15.32:0/3582324474 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7efcc0038680 0x7efcc003ab40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:38.230 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.884+0000 7efcdb77f700 1 --2- 172.21.15.32:0/3582324474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd4107130 0x7efcd419a260 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:38.231 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.884+0000 7efcdb77f700 1 -- 172.21.15.32:0/3582324474 >> 172.21.15.32:0/3582324474 conn(0x7efcd4100b80 msgr2=0x7efcd4101860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:38.231 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.884+0000 7efcdb77f700 1 -- 172.21.15.32:0/3582324474 shutdown_connections 2023-09-30T14:30:38.231 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:37.884+0000 7efcdb77f700 1 -- 172.21.15.32:0/3582324474 wait complete. 2023-09-30T14:30:38.471 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:38 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:38.175+0000 7ff4ce23d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-30T14:30:38.471 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:38 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:38.295+0000 7ff4ce23d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-30T14:30:38.471 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:38 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:38.407+0000 7ff4ce23d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-30T14:30:38.881 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:38 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:38.624+0000 7ff4ce23d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-30T14:30:39.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:38 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3582324474' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-09-30T14:30:39.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:38 smithi032 ceph-mon[98592]: mgrmap e5: x(active, since 10s) 2023-09-30T14:30:39.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:39 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:39.106+0000 7ff4ce23d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-30T14:30:39.583 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2023-09-30T14:30:39.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-09-30T14:30:39.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-30T14:30:39.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-30T14:30:39.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-30T14:30:39.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2023-09-30T14:30:39.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.755+0000 7fed4ce53700 1 Processor -- start 2023-09-30T14:30:39.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.755+0000 7fed4ce53700 1 -- start start 2023-09-30T14:30:39.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.755+0000 7fed4ce53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed48107130 0x7fed48109520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:39.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.755+0000 7fed4ce53700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed48074660 con 0x7fed48107130 2023-09-30T14:30:39.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.755+0000 7fed4659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed48107130 0x7fed48109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:39.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.756+0000 7fed4659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed48107130 0x7fed48109520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60728/0 (socket says 172.21.15.32:60728) 2023-09-30T14:30:39.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.756+0000 7fed4659c700 1 -- 172.21.15.32:0/2972402691 learned_addr learned my addr 172.21.15.32:0/2972402691 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:39.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.756+0000 7fed4659c700 1 -- 172.21.15.32:0/2972402691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed48109a60 con 0x7fed48107130 2023-09-30T14:30:39.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.756+0000 7fed4659c700 1 --2- 172.21.15.32:0/2972402691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed48107130 0x7fed48109520 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fed380098a0 tx=0x7fed38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38843c2381004a08 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:39.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.756+0000 7fed4559a700 1 -- 172.21.15.32:0/2972402691 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed38004160 con 0x7fed48107130 2023-09-30T14:30:39.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.756+0000 7fed4559a700 1 -- 172.21.15.32:0/2972402691 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed380042c0 con 0x7fed48107130 2023-09-30T14:30:39.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.756+0000 7fed4559a700 1 -- 172.21.15.32:0/2972402691 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed38011a00 con 0x7fed48107130 2023-09-30T14:30:39.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.757+0000 7fed4ce53700 1 -- 172.21.15.32:0/2972402691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed48107130 msgr2=0x7fed48109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:39.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.757+0000 7fed4ce53700 1 --2- 172.21.15.32:0/2972402691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed48107130 0x7fed48109520 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fed380098a0 tx=0x7fed38009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:39.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.757+0000 7fed4ce53700 1 -- 172.21.15.32:0/2972402691 shutdown_connections 2023-09-30T14:30:39.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.757+0000 7fed4ce53700 1 --2- 172.21.15.32:0/2972402691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed48107130 0x7fed48109520 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:39.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.757+0000 7fed4ce53700 1 -- 172.21.15.32:0/2972402691 >> 172.21.15.32:0/2972402691 conn(0x7fed48100b80 msgr2=0x7fed48102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:39.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.757+0000 7fed4ce53700 1 -- 172.21.15.32:0/2972402691 shutdown_connections 2023-09-30T14:30:39.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.757+0000 7fed4ce53700 1 -- 172.21.15.32:0/2972402691 wait complete. 2023-09-30T14:30:39.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.758+0000 7fed4ce53700 1 Processor -- start 2023-09-30T14:30:39.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.758+0000 7fed4ce53700 1 -- start start 2023-09-30T14:30:39.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.758+0000 7fed4ce53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed48107130 0x7fed4819c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:39.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.758+0000 7fed4ce53700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4819c9b0 con 0x7fed48107130 2023-09-30T14:30:39.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.758+0000 7fed4659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed48107130 0x7fed4819c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:39.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.758+0000 7fed4659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed48107130 0x7fed4819c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60730/0 (socket says 172.21.15.32:60730) 2023-09-30T14:30:39.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.758+0000 7fed4659c700 1 -- 172.21.15.32:0/3682462554 learned_addr learned my addr 172.21.15.32:0/3682462554 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:39.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.758+0000 7fed4659c700 1 -- 172.21.15.32:0/3682462554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed38009550 con 0x7fed48107130 2023-09-30T14:30:39.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.759+0000 7fed4659c700 1 --2- 172.21.15.32:0/3682462554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed48107130 0x7fed4819c470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fed38006af0 tx=0x7fed38011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:39.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.759+0000 7fed377fe700 1 -- 172.21.15.32:0/3682462554 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed38003a40 con 0x7fed48107130 2023-09-30T14:30:39.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.759+0000 7fed377fe700 1 -- 172.21.15.32:0/3682462554 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed38003ba0 con 0x7fed48107130 2023-09-30T14:30:39.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.759+0000 7fed4ce53700 1 -- 172.21.15.32:0/3682462554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4819cbb0 con 0x7fed48107130 2023-09-30T14:30:39.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.759+0000 7fed377fe700 1 -- 172.21.15.32:0/3682462554 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed3801a6e0 con 0x7fed48107130 2023-09-30T14:30:39.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.759+0000 7fed4ce53700 1 -- 172.21.15.32:0/3682462554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed4819d050 con 0x7fed48107130 2023-09-30T14:30:39.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.760+0000 7fed377fe700 1 -- 172.21.15.32:0/3682462554 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 45492+0+0 (secure 0 0 0) 0x7fed3801a860 con 0x7fed48107130 2023-09-30T14:30:39.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.760+0000 7fed377fe700 1 --2- 172.21.15.32:0/3682462554 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7fed300386d0 0x7fed3003ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:39.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.760+0000 7fed377fe700 1 -- 172.21.15.32:0/3682462554 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fed380246b0 con 0x7fed48107130 2023-09-30T14:30:39.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.760+0000 7fed45d9b700 1 -- 172.21.15.32:0/3682462554 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7fed300386d0 msgr2=0x7fed3003ab90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2564784233 2023-09-30T14:30:39.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.760+0000 7fed45d9b700 1 --2- 172.21.15.32:0/3682462554 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7fed300386d0 0x7fed3003ab90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T14:30:39.588 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.761+0000 7fed4ce53700 1 -- 172.21.15.32:0/3682462554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed48062450 con 0x7fed48107130 2023-09-30T14:30:39.588 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.765+0000 7fed377fe700 1 -- 172.21.15.32:0/3682462554 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fed3801ab10 con 0x7fed48107130 2023-09-30T14:30:39.588 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.961+0000 7fed45d9b700 1 -- 172.21.15.32:0/3682462554 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7fed300386d0 msgr2=0x7fed3003ab90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2564784233 2023-09-30T14:30:39.588 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:38.961+0000 7fed45d9b700 1 --2- 172.21.15.32:0/3682462554 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7fed300386d0 0x7fed3003ab90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-30T14:30:39.588 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:39.239+0000 7fed4ce53700 1 -- 172.21.15.32:0/3682462554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fed4810aa10 con 0x7fed48107130 2023-09-30T14:30:39.588 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:39.240+0000 7fed377fe700 1 -- 172.21.15.32:0/3682462554 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fed3804a030 con 0x7fed48107130 2023-09-30T14:30:39.588 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:39.242+0000 7fed4ce53700 1 -- 172.21.15.32:0/3682462554 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7fed300386d0 msgr2=0x7fed3003ab90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T14:30:39.588 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:39.242+0000 7fed4ce53700 1 --2- 172.21.15.32:0/3682462554 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7fed300386d0 0x7fed3003ab90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:39.588 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:39.242+0000 7fed4ce53700 1 -- 172.21.15.32:0/3682462554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed48107130 msgr2=0x7fed4819c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:39.588 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:39.242+0000 7fed4ce53700 1 --2- 172.21.15.32:0/3682462554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed48107130 0x7fed4819c470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fed38006af0 tx=0x7fed38011ad0 comp rx=0 tx=0).stop 2023-09-30T14:30:39.588 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:39.242+0000 7fed4ce53700 1 -- 172.21.15.32:0/3682462554 shutdown_connections 2023-09-30T14:30:39.588 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:39.242+0000 7fed4ce53700 1 --2- 172.21.15.32:0/3682462554 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7fed300386d0 0x7fed3003ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:39.589 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:39.242+0000 7fed4ce53700 1 --2- 172.21.15.32:0/3682462554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed48107130 0x7fed4819c470 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:39.589 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:39.242+0000 7fed4ce53700 1 -- 172.21.15.32:0/3682462554 >> 172.21.15.32:0/3682462554 conn(0x7fed48100b80 msgr2=0x7fed48101860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:39.589 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:39.242+0000 7fed4ce53700 1 -- 172.21.15.32:0/3682462554 shutdown_connections 2023-09-30T14:30:39.589 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:39.242+0000 7fed4ce53700 1 -- 172.21.15.32:0/3682462554 wait complete. 2023-09-30T14:30:39.589 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for the mgr to restart... 2023-09-30T14:30:39.589 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr epoch 5... 2023-09-30T14:30:39.595 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:39 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:39.260+0000 7ff4ce23d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-30T14:30:39.595 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:39 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:39.374+0000 7ff4ce23d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-30T14:30:39.595 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:39 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:39.490+0000 7ff4ce23d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-30T14:30:40.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:39 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3682462554' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-30T14:30:40.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:40 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:40.020+0000 7ff4ce23d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-30T14:30:41.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:40 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:40.803+0000 7ff4ce23d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-30T14:30:41.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:40 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:40.902+0000 7ff4ce23d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-30T14:30:42.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:41 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:41.878+0000 7ff4ce23d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-30T14:30:42.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:41 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:41.989+0000 7ff4ce23d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-30T14:30:42.561 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:42 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:42.211+0000 7ff4ce23d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-30T14:30:42.814 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:42 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:42.561+0000 7ff4ce23d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-30T14:30:43.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:42 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:42.814+0000 7ff4ce23d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-30T14:30:43.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:42 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:42.997+0000 7ff4ce23d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-30T14:30:43.637 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:43 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:43.328+0000 7ff4ce23d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-30T14:30:44.555 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:44 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:44.151+0000 7ff4ce23d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-30T14:30:44.555 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:44 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:44.308+0000 7ff4ce23d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-30T14:30:44.887 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:44 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:44.555+0000 7ff4ce23d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-30T14:30:44.887 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:44 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:44.692+0000 7ff4ce23d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-30T14:30:45.387 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:44 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:44.966+0000 7ff4ce23d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-30T14:30:46.387 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:46 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:30:46.011+0000 7ff4ce23d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-30T14:30:46.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:46 smithi032 ceph-mon[98592]: Active manager daemon x restarted 2023-09-30T14:30:46.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:46 smithi032 ceph-mon[98592]: Activating manager daemon x 2023-09-30T14:30:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:46 smithi032 ceph-mon[98592]: osdmap e2: 0 total, 0 up, 0 in 2023-09-30T14:30:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:46 smithi032 ceph-mon[98592]: mgrmap e6: x(active, starting, since 0.00375005s) 2023-09-30T14:30:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:46 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:30:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:46 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-30T14:30:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:46 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-30T14:30:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:46 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-30T14:30:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:46 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-30T14:30:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:46 smithi032 ceph-mon[98592]: Manager daemon x is now available 2023-09-30T14:30:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:46 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' 2023-09-30T14:30:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:46 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' 2023-09-30T14:30:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:46 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:30:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:46 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:30:47.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:47 smithi032 ceph-mon[98592]: Found migration_current of "None". Setting to last migration. 2023-09-30T14:30:47.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:47 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:30:47.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:47 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-30T14:30:47.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:47 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-30T14:30:47.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:47 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' 2023-09-30T14:30:47.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:47 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' 2023-09-30T14:30:47.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:47 smithi032 ceph-mon[98592]: mgrmap e7: x(active, since 1.0052s) 2023-09-30T14:30:47.439 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2023-09-30T14:30:47.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-09-30T14:30:47.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-30T14:30:47.446 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2023-09-30T14:30:47.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.059+0000 7ff23af92700 1 Processor -- start 2023-09-30T14:30:47.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.059+0000 7ff23af92700 1 -- start start 2023-09-30T14:30:47.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.059+0000 7ff23af92700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff234107190 0x7ff234109580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:47.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.059+0000 7ff23af92700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff234074600 con 0x7ff234107190 2023-09-30T14:30:47.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.060+0000 7ff238d2e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff234107190 0x7ff234109580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:47.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.060+0000 7ff238d2e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff234107190 0x7ff234109580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60738/0 (socket says 172.21.15.32:60738) 2023-09-30T14:30:47.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.060+0000 7ff238d2e700 1 -- 172.21.15.32:0/3030649284 learned_addr learned my addr 172.21.15.32:0/3030649284 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:47.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.060+0000 7ff238d2e700 1 -- 172.21.15.32:0/3030649284 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff234109ac0 con 0x7ff234107190 2023-09-30T14:30:47.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.060+0000 7ff238d2e700 1 --2- 172.21.15.32:0/3030649284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff234107190 0x7ff234109580 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff2280098a0 tx=0x7ff228009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56fd4fd4c546324 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:47.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.060+0000 7ff2337fe700 1 -- 172.21.15.32:0/3030649284 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff228004160 con 0x7ff234107190 2023-09-30T14:30:47.447 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.060+0000 7ff2337fe700 1 -- 172.21.15.32:0/3030649284 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff2280042c0 con 0x7ff234107190 2023-09-30T14:30:47.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.060+0000 7ff2337fe700 1 -- 172.21.15.32:0/3030649284 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff228011a00 con 0x7ff234107190 2023-09-30T14:30:47.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.061+0000 7ff23af92700 1 -- 172.21.15.32:0/3030649284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff234107190 msgr2=0x7ff234109580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:47.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.061+0000 7ff23af92700 1 --2- 172.21.15.32:0/3030649284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff234107190 0x7ff234109580 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff2280098a0 tx=0x7ff228009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:47.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.061+0000 7ff23af92700 1 -- 172.21.15.32:0/3030649284 shutdown_connections 2023-09-30T14:30:47.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.061+0000 7ff23af92700 1 --2- 172.21.15.32:0/3030649284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff234107190 0x7ff234109580 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:47.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.061+0000 7ff23af92700 1 -- 172.21.15.32:0/3030649284 >> 172.21.15.32:0/3030649284 conn(0x7ff234100b80 msgr2=0x7ff234102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:47.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.061+0000 7ff23af92700 1 -- 172.21.15.32:0/3030649284 shutdown_connections 2023-09-30T14:30:47.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.061+0000 7ff23af92700 1 -- 172.21.15.32:0/3030649284 wait complete. 2023-09-30T14:30:47.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.062+0000 7ff23af92700 1 Processor -- start 2023-09-30T14:30:47.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.062+0000 7ff23af92700 1 -- start start 2023-09-30T14:30:47.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.062+0000 7ff23af92700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff234107190 0x7ff23419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:47.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.062+0000 7ff23af92700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23419a7e0 con 0x7ff234107190 2023-09-30T14:30:47.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.062+0000 7ff238d2e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff234107190 0x7ff23419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:47.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.062+0000 7ff238d2e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff234107190 0x7ff23419a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60740/0 (socket says 172.21.15.32:60740) 2023-09-30T14:30:47.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.063+0000 7ff238d2e700 1 -- 172.21.15.32:0/3071225450 learned_addr learned my addr 172.21.15.32:0/3071225450 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:47.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.063+0000 7ff238d2e700 1 -- 172.21.15.32:0/3071225450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff228009550 con 0x7ff234107190 2023-09-30T14:30:47.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.063+0000 7ff238d2e700 1 --2- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff234107190 0x7ff23419a2a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff228009fb0 tx=0x7ff228011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:47.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.063+0000 7ff231ffb700 1 -- 172.21.15.32:0/3071225450 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff228003930 con 0x7ff234107190 2023-09-30T14:30:47.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.063+0000 7ff23af92700 1 -- 172.21.15.32:0/3071225450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23419a9e0 con 0x7ff234107190 2023-09-30T14:30:47.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.063+0000 7ff231ffb700 1 -- 172.21.15.32:0/3071225450 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff228003a90 con 0x7ff234107190 2023-09-30T14:30:47.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.063+0000 7ff231ffb700 1 -- 172.21.15.32:0/3071225450 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff228019610 con 0x7ff234107190 2023-09-30T14:30:47.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.063+0000 7ff23af92700 1 -- 172.21.15.32:0/3071225450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23419ae80 con 0x7ff234107190 2023-09-30T14:30:47.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.064+0000 7ff231ffb700 1 -- 172.21.15.32:0/3071225450 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 45492+0+0 (secure 0 0 0) 0x7ff228019880 con 0x7ff234107190 2023-09-30T14:30:47.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.065+0000 7ff231ffb700 1 --2- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7ff224038680 0x7ff22403ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.065+0000 7ff231ffb700 1 -- 172.21.15.32:0/3071225450 --> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff22403b250 con 0x7ff224038680 2023-09-30T14:30:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.065+0000 7ff233fff700 1 -- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7ff224038680 msgr2=0x7ff22403ab40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2564784233 2023-09-30T14:30:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.065+0000 7ff231ffb700 1 -- 172.21.15.32:0/3071225450 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff22804c3e0 con 0x7ff234107190 2023-09-30T14:30:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.065+0000 7ff233fff700 1 --2- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7ff224038680 0x7ff22403ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T14:30:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.266+0000 7ff233fff700 1 -- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7ff224038680 msgr2=0x7ff22403ab40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2564784233 2023-09-30T14:30:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.266+0000 7ff233fff700 1 --2- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7ff224038680 0x7ff22403ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-30T14:30:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.667+0000 7ff233fff700 1 -- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7ff224038680 msgr2=0x7ff22403ab40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2564784233 2023-09-30T14:30:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:40.667+0000 7ff233fff700 1 --2- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7ff224038680 0x7ff22403ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-30T14:30:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:41.468+0000 7ff233fff700 1 -- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7ff224038680 msgr2=0x7ff22403ab40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2564784233 2023-09-30T14:30:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:41.468+0000 7ff233fff700 1 --2- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7ff224038680 0x7ff22403ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-30T14:30:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:43.069+0000 7ff233fff700 1 -- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7ff224038680 msgr2=0x7ff22403ab40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2564784233 2023-09-30T14:30:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:43.069+0000 7ff233fff700 1 --2- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7ff224038680 0x7ff22403ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-09-30T14:30:47.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:46.027+0000 7ff231ffb700 1 -- 172.21.15.32:0/3071225450 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mgrmap(e 6) v1 ==== 45259+0+0 (secure 0 0 0) 0x7ff22802d960 con 0x7ff234107190 2023-09-30T14:30:47.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:46.027+0000 7ff231ffb700 1 -- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7ff224038680 msgr2=0x7ff22403ab40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T14:30:47.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:46.027+0000 7ff231ffb700 1 --2- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7ff224038680 0x7ff22403ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:47.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.028+0000 7ff231ffb700 1 -- 172.21.15.32:0/3071225450 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 7) v1 ==== 45386+0+0 (secure 0 0 0) 0x7ff22802cc50 con 0x7ff234107190 2023-09-30T14:30:47.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.028+0000 7ff231ffb700 1 --2- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7ff22403bf90 0x7ff22403e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:47.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.028+0000 7ff231ffb700 1 -- 172.21.15.32:0/3071225450 --> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff22403b250 con 0x7ff22403bf90 2023-09-30T14:30:47.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.028+0000 7ff233fff700 1 --2- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7ff22403bf90 0x7ff22403e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:47.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.029+0000 7ff233fff700 1 --2- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7ff22403bf90 0x7ff22403e380 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff220003a10 tx=0x7ff2200092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:47.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.030+0000 7ff231ffb700 1 -- 172.21.15.32:0/3071225450 <== mgr.14120 v2:172.21.15.32:6800/1639474065 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7ff22403b250 con 0x7ff22403bf90 2023-09-30T14:30:47.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.040+0000 7ff23af92700 1 -- 172.21.15.32:0/3071225450 --> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff23419b5e0 con 0x7ff22403bf90 2023-09-30T14:30:47.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.041+0000 7ff231ffb700 1 -- 172.21.15.32:0/3071225450 <== mgr.14120 v2:172.21.15.32:6800/1639474065 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff23419b5e0 con 0x7ff22403bf90 2023-09-30T14:30:47.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.041+0000 7ff23af92700 1 -- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7ff22403bf90 msgr2=0x7ff22403e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:47.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.041+0000 7ff23af92700 1 --2- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7ff22403bf90 0x7ff22403e380 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff220003a10 tx=0x7ff2200092b0 comp rx=0 tx=0).stop 2023-09-30T14:30:47.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.041+0000 7ff23af92700 1 -- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff234107190 msgr2=0x7ff23419a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:47.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.041+0000 7ff23af92700 1 --2- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff234107190 0x7ff23419a2a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff228009fb0 tx=0x7ff228011ad0 comp rx=0 tx=0).stop 2023-09-30T14:30:47.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.041+0000 7ff23af92700 1 -- 172.21.15.32:0/3071225450 shutdown_connections 2023-09-30T14:30:47.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.041+0000 7ff23af92700 1 --2- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff234107190 0x7ff23419a2a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:47.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.041+0000 7ff23af92700 1 --2- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7ff22403bf90 0x7ff22403e380 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:47.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.041+0000 7ff23af92700 1 --2- 172.21.15.32:0/3071225450 >> [v2:172.21.15.32:6800/2564784233,v1:172.21.15.32:6801/2564784233] conn(0x7ff224038680 0x7ff22403ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:47.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.041+0000 7ff23af92700 1 -- 172.21.15.32:0/3071225450 >> 172.21.15.32:0/3071225450 conn(0x7ff234100b80 msgr2=0x7ff234101880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:47.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.041+0000 7ff23af92700 1 -- 172.21.15.32:0/3071225450 shutdown_connections 2023-09-30T14:30:47.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.041+0000 7ff23af92700 1 -- 172.21.15.32:0/3071225450 wait complete. 2023-09-30T14:30:47.452 INFO:teuthology.orchestra.run.smithi032.stdout:mgr epoch 5 is available 2023-09-30T14:30:47.452 INFO:teuthology.orchestra.run.smithi032.stdout:Setting orchestrator backend to cephadm... 2023-09-30T14:30:48.353 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:48 smithi032 ceph-mon[98592]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-30T14:30:48.353 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:48 smithi032 ceph-mon[98592]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-30T14:30:48.354 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:48 smithi032 ceph-mon[98592]: [30/Sep/2023:14:30:47] ENGINE Bus STARTING 2023-09-30T14:30:48.354 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:48 smithi032 ceph-mon[98592]: [30/Sep/2023:14:30:47] ENGINE Serving on http://172.21.15.32:8765 2023-09-30T14:30:48.354 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:48 smithi032 ceph-mon[98592]: [30/Sep/2023:14:30:47] ENGINE Serving on https://172.21.15.32:7150 2023-09-30T14:30:48.354 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:48 smithi032 ceph-mon[98592]: [30/Sep/2023:14:30:47] ENGINE Bus STARTED 2023-09-30T14:30:48.354 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:48 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:30:48.692 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.918+0000 7efcc46cb700 1 Processor -- start 2023-09-30T14:30:48.692 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.918+0000 7efcc46cb700 1 -- start start 2023-09-30T14:30:48.692 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.918+0000 7efcc46cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcbc107170 0x7efcbc109560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:48.692 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.918+0000 7efcc46cb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc074620 con 0x7efcbc107170 2023-09-30T14:30:48.692 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.918+0000 7efcc2467700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcbc107170 0x7efcbc109560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:48.692 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.919+0000 7efcc2467700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcbc107170 0x7efcbc109560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60792/0 (socket says 172.21.15.32:60792) 2023-09-30T14:30:48.692 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.919+0000 7efcc2467700 1 -- 172.21.15.32:0/1068918353 learned_addr learned my addr 172.21.15.32:0/1068918353 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:48.693 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.919+0000 7efcc2467700 1 -- 172.21.15.32:0/1068918353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc109aa0 con 0x7efcbc107170 2023-09-30T14:30:48.693 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.919+0000 7efcc2467700 1 --2- 172.21.15.32:0/1068918353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcbc107170 0x7efcbc109560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efcb80098a0 tx=0x7efcb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1454868f2245fa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:48.693 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.919+0000 7efcc1465700 1 -- 172.21.15.32:0/1068918353 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcb8004160 con 0x7efcbc107170 2023-09-30T14:30:48.693 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.919+0000 7efcc1465700 1 -- 172.21.15.32:0/1068918353 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efcb80042c0 con 0x7efcbc107170 2023-09-30T14:30:48.693 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.919+0000 7efcc1465700 1 -- 172.21.15.32:0/1068918353 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcb8011a00 con 0x7efcbc107170 2023-09-30T14:30:48.693 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.920+0000 7efcc46cb700 1 -- 172.21.15.32:0/1068918353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcbc107170 msgr2=0x7efcbc109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:48.693 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.920+0000 7efcc46cb700 1 --2- 172.21.15.32:0/1068918353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcbc107170 0x7efcbc109560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efcb80098a0 tx=0x7efcb8009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:48.693 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.920+0000 7efcc46cb700 1 -- 172.21.15.32:0/1068918353 shutdown_connections 2023-09-30T14:30:48.693 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.920+0000 7efcc46cb700 1 --2- 172.21.15.32:0/1068918353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcbc107170 0x7efcbc109560 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:48.693 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.920+0000 7efcc46cb700 1 -- 172.21.15.32:0/1068918353 >> 172.21.15.32:0/1068918353 conn(0x7efcbc100b80 msgr2=0x7efcbc102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:48.693 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.920+0000 7efcc46cb700 1 -- 172.21.15.32:0/1068918353 shutdown_connections 2023-09-30T14:30:48.694 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.920+0000 7efcc46cb700 1 -- 172.21.15.32:0/1068918353 wait complete. 2023-09-30T14:30:48.694 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.920+0000 7efcc46cb700 1 Processor -- start 2023-09-30T14:30:48.694 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.920+0000 7efcc46cb700 1 -- start start 2023-09-30T14:30:48.694 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.921+0000 7efcc46cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcbc107170 0x7efcbc106ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:48.694 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.921+0000 7efcc46cb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc107010 con 0x7efcbc107170 2023-09-30T14:30:48.694 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.921+0000 7efcc2467700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcbc107170 0x7efcbc106ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:48.694 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.921+0000 7efcc2467700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcbc107170 0x7efcbc106ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60794/0 (socket says 172.21.15.32:60794) 2023-09-30T14:30:48.694 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.921+0000 7efcc2467700 1 -- 172.21.15.32:0/467303711 learned_addr learned my addr 172.21.15.32:0/467303711 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:48.694 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.921+0000 7efcc2467700 1 -- 172.21.15.32:0/467303711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb8009550 con 0x7efcbc107170 2023-09-30T14:30:48.694 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.921+0000 7efcc2467700 1 --2- 172.21.15.32:0/467303711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcbc107170 0x7efcbc106ad0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efcb8006af0 tx=0x7efcb8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:48.694 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.921+0000 7efcb37fe700 1 -- 172.21.15.32:0/467303711 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcb8003a40 con 0x7efcbc107170 2023-09-30T14:30:48.695 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.921+0000 7efcb37fe700 1 -- 172.21.15.32:0/467303711 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efcb8003ba0 con 0x7efcbc107170 2023-09-30T14:30:48.695 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.921+0000 7efcc46cb700 1 -- 172.21.15.32:0/467303711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcbc105120 con 0x7efcbc107170 2023-09-30T14:30:48.695 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.921+0000 7efcb37fe700 1 -- 172.21.15.32:0/467303711 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcb801a6e0 con 0x7efcbc107170 2023-09-30T14:30:48.695 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.921+0000 7efcc46cb700 1 -- 172.21.15.32:0/467303711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcbc1055c0 con 0x7efcbc107170 2023-09-30T14:30:48.695 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.922+0000 7efcb37fe700 1 -- 172.21.15.32:0/467303711 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 7) v1 ==== 45386+0+0 (secure 0 0 0) 0x7efcb801a860 con 0x7efcbc107170 2023-09-30T14:30:48.695 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.922+0000 7efcb37fe700 1 --2- 172.21.15.32:0/467303711 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7efca80385a0 0x7efca803aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:48.695 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.922+0000 7efcb37fe700 1 -- 172.21.15.32:0/467303711 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efcb804c7e0 con 0x7efcbc107170 2023-09-30T14:30:48.695 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.923+0000 7efcc1c66700 1 --2- 172.21.15.32:0/467303711 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7efca80385a0 0x7efca803aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:48.695 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.923+0000 7efcc46cb700 1 -- 172.21.15.32:0/467303711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcbc10a840 con 0x7efcbc107170 2023-09-30T14:30:48.695 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.923+0000 7efcc1c66700 1 --2- 172.21.15.32:0/467303711 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7efca80385a0 0x7efca803aa60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efcac006fd0 tx=0x7efcac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:48.695 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:47.928+0000 7efcb37fe700 1 -- 172.21.15.32:0/467303711 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7efcb801ab10 con 0x7efcbc107170 2023-09-30T14:30:48.695 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:48.079+0000 7efcb37fe700 1 -- 172.21.15.32:0/467303711 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7efcb801acf0 con 0x7efcbc107170 2023-09-30T14:30:48.696 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:48.323+0000 7efcc46cb700 1 -- 172.21.15.32:0/467303711 --> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7efcbc105c70 con 0x7efca80385a0 2023-09-30T14:30:48.696 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:48.329+0000 7efcb37fe700 1 -- 172.21.15.32:0/467303711 <== mgr.14120 v2:172.21.15.32:6800/1639474065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7efcbc105c70 con 0x7efca80385a0 2023-09-30T14:30:48.696 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:48.332+0000 7efcc46cb700 1 -- 172.21.15.32:0/467303711 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7efca80385a0 msgr2=0x7efca803aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:48.696 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:48.332+0000 7efcc46cb700 1 --2- 172.21.15.32:0/467303711 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7efca80385a0 0x7efca803aa60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efcac006fd0 tx=0x7efcac006e40 comp rx=0 tx=0).stop 2023-09-30T14:30:48.696 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:48.332+0000 7efcc46cb700 1 -- 172.21.15.32:0/467303711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcbc107170 msgr2=0x7efcbc106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:48.696 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:48.332+0000 7efcc46cb700 1 --2- 172.21.15.32:0/467303711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcbc107170 0x7efcbc106ad0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efcb8006af0 tx=0x7efcb8011ad0 comp rx=0 tx=0).stop 2023-09-30T14:30:48.696 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:48.332+0000 7efcc46cb700 1 -- 172.21.15.32:0/467303711 shutdown_connections 2023-09-30T14:30:48.696 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:48.332+0000 7efcc46cb700 1 --2- 172.21.15.32:0/467303711 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7efca80385a0 0x7efca803aa60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:48.696 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:48.332+0000 7efcc46cb700 1 --2- 172.21.15.32:0/467303711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcbc107170 0x7efcbc106ad0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:48.696 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:48.332+0000 7efcc46cb700 1 -- 172.21.15.32:0/467303711 >> 172.21.15.32:0/467303711 conn(0x7efcbc100b80 msgr2=0x7efcbc101850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:48.696 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:48.332+0000 7efcc46cb700 1 -- 172.21.15.32:0/467303711 shutdown_connections 2023-09-30T14:30:48.696 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:48.332+0000 7efcc46cb700 1 -- 172.21.15.32:0/467303711 wait complete. 2023-09-30T14:30:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:49 smithi032 ceph-mon[98592]: mgrmap e8: x(active, since 2s) 2023-09-30T14:30:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:49 smithi032 ceph-mon[98592]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:30:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:49 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' 2023-09-30T14:30:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:49 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:30:49.938 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout value unchanged 2023-09-30T14:30:49.938 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.195+0000 7ffbd2d7e700 1 Processor -- start 2023-09-30T14:30:49.938 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.195+0000 7ffbd2d7e700 1 -- start start 2023-09-30T14:30:49.938 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.195+0000 7ffbd2d7e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc1066b0 0x7ffbcc106ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:49.938 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.195+0000 7ffbd2d7e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc1070a0 con 0x7ffbcc1066b0 2023-09-30T14:30:49.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.195+0000 7ffbd0b1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc1066b0 0x7ffbcc106ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:49.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.195+0000 7ffbd0b1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc1066b0 0x7ffbcc106ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60798/0 (socket says 172.21.15.32:60798) 2023-09-30T14:30:49.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.195+0000 7ffbd0b1a700 1 -- 172.21.15.32:0/3521418704 learned_addr learned my addr 172.21.15.32:0/3521418704 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:49.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.196+0000 7ffbd0b1a700 1 -- 172.21.15.32:0/3521418704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbcc1078b0 con 0x7ffbcc1066b0 2023-09-30T14:30:49.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.196+0000 7ffbd0b1a700 1 --2- 172.21.15.32:0/3521418704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc1066b0 0x7ffbcc106ad0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00098a0 tx=0x7ffbc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83849784de1549c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:49.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.196+0000 7ffbcb7fe700 1 -- 172.21.15.32:0/3521418704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbc0004160 con 0x7ffbcc1066b0 2023-09-30T14:30:49.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.196+0000 7ffbcb7fe700 1 -- 172.21.15.32:0/3521418704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffbc00042c0 con 0x7ffbcc1066b0 2023-09-30T14:30:49.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.196+0000 7ffbcb7fe700 1 -- 172.21.15.32:0/3521418704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbc0011a00 con 0x7ffbcc1066b0 2023-09-30T14:30:49.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.196+0000 7ffbd2d7e700 1 -- 172.21.15.32:0/3521418704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc1066b0 msgr2=0x7ffbcc106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:49.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.196+0000 7ffbd2d7e700 1 --2- 172.21.15.32:0/3521418704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc1066b0 0x7ffbcc106ad0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00098a0 tx=0x7ffbc0009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:49.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.197+0000 7ffbd2d7e700 1 -- 172.21.15.32:0/3521418704 shutdown_connections 2023-09-30T14:30:49.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.197+0000 7ffbd2d7e700 1 --2- 172.21.15.32:0/3521418704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc1066b0 0x7ffbcc106ad0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:49.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.197+0000 7ffbd2d7e700 1 -- 172.21.15.32:0/3521418704 >> 172.21.15.32:0/3521418704 conn(0x7ffbcc101c50 msgr2=0x7ffbcc104090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:49.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.197+0000 7ffbd2d7e700 1 -- 172.21.15.32:0/3521418704 shutdown_connections 2023-09-30T14:30:49.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.197+0000 7ffbd2d7e700 1 -- 172.21.15.32:0/3521418704 wait complete. 2023-09-30T14:30:49.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.197+0000 7ffbd2d7e700 1 Processor -- start 2023-09-30T14:30:49.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.197+0000 7ffbd2d7e700 1 -- start start 2023-09-30T14:30:49.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.197+0000 7ffbd2d7e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc1066b0 0x7ffbcc19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:49.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.197+0000 7ffbd2d7e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc19a8d0 con 0x7ffbcc1066b0 2023-09-30T14:30:49.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.197+0000 7ffbd0b1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc1066b0 0x7ffbcc19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:49.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.198+0000 7ffbd0b1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc1066b0 0x7ffbcc19a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60800/0 (socket says 172.21.15.32:60800) 2023-09-30T14:30:49.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.198+0000 7ffbd0b1a700 1 -- 172.21.15.32:0/2855833680 learned_addr learned my addr 172.21.15.32:0/2855833680 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:49.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.198+0000 7ffbd0b1a700 1 -- 172.21.15.32:0/2855833680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc0009550 con 0x7ffbcc1066b0 2023-09-30T14:30:49.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.198+0000 7ffbd0b1a700 1 --2- 172.21.15.32:0/2855833680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc1066b0 0x7ffbcc19a390 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009fb0 tx=0x7ffbc0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:49.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.198+0000 7ffbc9ffb700 1 -- 172.21.15.32:0/2855833680 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbc0003a40 con 0x7ffbcc1066b0 2023-09-30T14:30:49.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.198+0000 7ffbc9ffb700 1 -- 172.21.15.32:0/2855833680 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffbc0003ba0 con 0x7ffbcc1066b0 2023-09-30T14:30:49.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.198+0000 7ffbc9ffb700 1 -- 172.21.15.32:0/2855833680 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbc001a750 con 0x7ffbcc1066b0 2023-09-30T14:30:49.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.198+0000 7ffbd2d7e700 1 -- 172.21.15.32:0/2855833680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbcc19aad0 con 0x7ffbcc1066b0 2023-09-30T14:30:49.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.198+0000 7ffbd2d7e700 1 -- 172.21.15.32:0/2855833680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbcc19af70 con 0x7ffbcc1066b0 2023-09-30T14:30:49.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.199+0000 7ffbc9ffb700 1 -- 172.21.15.32:0/2855833680 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7ffbc0024b60 con 0x7ffbcc1066b0 2023-09-30T14:30:49.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.199+0000 7ffbc9ffb700 1 --2- 172.21.15.32:0/2855833680 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7ffbbc038630 0x7ffbbc03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:49.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.199+0000 7ffbc9ffb700 1 -- 172.21.15.32:0/2855833680 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffbc004c650 con 0x7ffbcc1066b0 2023-09-30T14:30:49.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.199+0000 7ffbcbfff700 1 --2- 172.21.15.32:0/2855833680 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7ffbbc038630 0x7ffbbc03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:49.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.199+0000 7ffbd2d7e700 1 -- 172.21.15.32:0/2855833680 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbcc062450 con 0x7ffbcc1066b0 2023-09-30T14:30:49.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.200+0000 7ffbcbfff700 1 --2- 172.21.15.32:0/2855833680 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7ffbbc038630 0x7ffbbc03aaf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8006fd0 tx=0x7ffbb8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:49.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.204+0000 7ffbc9ffb700 1 -- 172.21.15.32:0/2855833680 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7ffbc002d3f0 con 0x7ffbcc1066b0 2023-09-30T14:30:49.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.570+0000 7ffbd2d7e700 1 -- 172.21.15.32:0/2855833680 --> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbcc103f50 con 0x7ffbbc038630 2023-09-30T14:30:49.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.571+0000 7ffbc9ffb700 1 -- 172.21.15.32:0/2855833680 <== mgr.14120 v2:172.21.15.32:6800/1639474065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7ffbcc103f50 con 0x7ffbbc038630 2023-09-30T14:30:49.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.573+0000 7ffbd2d7e700 1 -- 172.21.15.32:0/2855833680 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7ffbbc038630 msgr2=0x7ffbbc03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:49.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.573+0000 7ffbd2d7e700 1 --2- 172.21.15.32:0/2855833680 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7ffbbc038630 0x7ffbbc03aaf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8006fd0 tx=0x7ffbb8006e40 comp rx=0 tx=0).stop 2023-09-30T14:30:49.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.573+0000 7ffbd2d7e700 1 -- 172.21.15.32:0/2855833680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc1066b0 msgr2=0x7ffbcc19a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:49.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.573+0000 7ffbd2d7e700 1 --2- 172.21.15.32:0/2855833680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc1066b0 0x7ffbcc19a390 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009fb0 tx=0x7ffbc0011ad0 comp rx=0 tx=0).stop 2023-09-30T14:30:49.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.574+0000 7ffbd2d7e700 1 -- 172.21.15.32:0/2855833680 shutdown_connections 2023-09-30T14:30:49.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.574+0000 7ffbd2d7e700 1 --2- 172.21.15.32:0/2855833680 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7ffbbc038630 0x7ffbbc03aaf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:49.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.574+0000 7ffbd2d7e700 1 --2- 172.21.15.32:0/2855833680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbcc1066b0 0x7ffbcc19a390 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:49.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.574+0000 7ffbd2d7e700 1 -- 172.21.15.32:0/2855833680 >> 172.21.15.32:0/2855833680 conn(0x7ffbcc101c50 msgr2=0x7ffbcc1032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:49.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.574+0000 7ffbd2d7e700 1 -- 172.21.15.32:0/2855833680 shutdown_connections 2023-09-30T14:30:49.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:49.574+0000 7ffbd2d7e700 1 -- 172.21.15.32:0/2855833680 wait complete. 2023-09-30T14:30:49.945 INFO:teuthology.orchestra.run.smithi032.stdout:Generating ssh key... 2023-09-30T14:30:50.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-mon[98592]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:30:51.369 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:51 smithi032 ceph-mon[98592]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:30:51.369 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:51 smithi032 ceph-mon[98592]: Generating ssh key... 2023-09-30T14:30:51.370 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:51 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' 2023-09-30T14:30:51.370 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:51 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' 2023-09-30T14:30:51.370 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: Generating public/private rsa key pair. 2023-09-30T14:30:51.370 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: Your identification has been saved in /tmp/tmpg7ralaw8/key. 2023-09-30T14:30:51.370 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: Your public key has been saved in /tmp/tmpg7ralaw8/key.pub. 2023-09-30T14:30:51.370 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: The key fingerprint is: 2023-09-30T14:30:51.370 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: SHA256:n228a5BNcaeUTHKTHgPpTUAt65mU3ClIgDb827XQBF4 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:30:51.370 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: The key's randomart image is: 2023-09-30T14:30:51.371 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: +---[RSA 3072]----+ 2023-09-30T14:30:51.371 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: | . ....oEB+o | 2023-09-30T14:30:51.371 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: | = ...=o@..| 2023-09-30T14:30:51.371 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: | . o ..* % * | 2023-09-30T14:30:51.371 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: | . o @ * | 2023-09-30T14:30:51.371 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: | So O = | 2023-09-30T14:30:51.371 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: | ..++* | 2023-09-30T14:30:51.371 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: | o.+ | 2023-09-30T14:30:51.371 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: | ... | 2023-09-30T14:30:51.371 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: | .o. | 2023-09-30T14:30:51.371 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:30:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: +----[SHA256]-----+ 2023-09-30T14:30:51.383 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.380+0000 7f045867c700 1 Processor -- start 2023-09-30T14:30:51.384 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.380+0000 7f045867c700 1 -- start start 2023-09-30T14:30:51.384 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.381+0000 7f045867c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450107150 0x7f0450109540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:51.384 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.381+0000 7f045867c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0450079e70 con 0x7f0450107150 2023-09-30T14:30:51.384 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.381+0000 7f0456418700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450107150 0x7f0450109540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:51.384 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.381+0000 7f0456418700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450107150 0x7f0450109540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60804/0 (socket says 172.21.15.32:60804) 2023-09-30T14:30:51.384 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.381+0000 7f0456418700 1 -- 172.21.15.32:0/1321381083 learned_addr learned my addr 172.21.15.32:0/1321381083 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:51.384 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.381+0000 7f0456418700 1 -- 172.21.15.32:0/1321381083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0450109a80 con 0x7f0450107150 2023-09-30T14:30:51.384 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.381+0000 7f0456418700 1 --2- 172.21.15.32:0/1321381083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450107150 0x7f0450109540 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f04400098a0 tx=0x7f0440009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ddcc551dafe8f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:51.384 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.381+0000 7f0455416700 1 -- 172.21.15.32:0/1321381083 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0440004160 con 0x7f0450107150 2023-09-30T14:30:51.384 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.381+0000 7f0455416700 1 -- 172.21.15.32:0/1321381083 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f04400042c0 con 0x7f0450107150 2023-09-30T14:30:51.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.381+0000 7f0455416700 1 -- 172.21.15.32:0/1321381083 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0440011a00 con 0x7f0450107150 2023-09-30T14:30:51.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.382+0000 7f045867c700 1 -- 172.21.15.32:0/1321381083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450107150 msgr2=0x7f0450109540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:51.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.382+0000 7f045867c700 1 --2- 172.21.15.32:0/1321381083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450107150 0x7f0450109540 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f04400098a0 tx=0x7f0440009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:51.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.382+0000 7f045867c700 1 -- 172.21.15.32:0/1321381083 shutdown_connections 2023-09-30T14:30:51.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.382+0000 7f045867c700 1 --2- 172.21.15.32:0/1321381083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450107150 0x7f0450109540 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:51.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.382+0000 7f045867c700 1 -- 172.21.15.32:0/1321381083 >> 172.21.15.32:0/1321381083 conn(0x7f0450100b00 msgr2=0x7f0450102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:51.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.382+0000 7f045867c700 1 -- 172.21.15.32:0/1321381083 shutdown_connections 2023-09-30T14:30:51.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.382+0000 7f045867c700 1 -- 172.21.15.32:0/1321381083 wait complete. 2023-09-30T14:30:51.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.383+0000 7f045867c700 1 Processor -- start 2023-09-30T14:30:51.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.383+0000 7f045867c700 1 -- start start 2023-09-30T14:30:51.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.383+0000 7f045867c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450107150 0x7f045019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:51.385 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.383+0000 7f045867c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045019a7a0 con 0x7f0450107150 2023-09-30T14:30:51.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.383+0000 7f0456418700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450107150 0x7f045019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:51.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.383+0000 7f0456418700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450107150 0x7f045019a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60806/0 (socket says 172.21.15.32:60806) 2023-09-30T14:30:51.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.383+0000 7f0456418700 1 -- 172.21.15.32:0/824982550 learned_addr learned my addr 172.21.15.32:0/824982550 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:51.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.383+0000 7f0456418700 1 -- 172.21.15.32:0/824982550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0440009550 con 0x7f0450107150 2023-09-30T14:30:51.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.384+0000 7f0456418700 1 --2- 172.21.15.32:0/824982550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450107150 0x7f045019a260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0440009fb0 tx=0x7f0440011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:51.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.384+0000 7f04477fe700 1 -- 172.21.15.32:0/824982550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0440003930 con 0x7f0450107150 2023-09-30T14:30:51.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.384+0000 7f04477fe700 1 -- 172.21.15.32:0/824982550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0440003a90 con 0x7f0450107150 2023-09-30T14:30:51.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.384+0000 7f045867c700 1 -- 172.21.15.32:0/824982550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045019a9a0 con 0x7f0450107150 2023-09-30T14:30:51.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.384+0000 7f04477fe700 1 -- 172.21.15.32:0/824982550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0440019610 con 0x7f0450107150 2023-09-30T14:30:51.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.384+0000 7f045867c700 1 -- 172.21.15.32:0/824982550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045019ae40 con 0x7f0450107150 2023-09-30T14:30:51.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.385+0000 7f04477fe700 1 -- 172.21.15.32:0/824982550 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7f04400198f0 con 0x7f0450107150 2023-09-30T14:30:51.386 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.385+0000 7f04477fe700 1 --2- 172.21.15.32:0/824982550 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f043c038630 0x7f043c03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:51.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.385+0000 7f04477fe700 1 -- 172.21.15.32:0/824982550 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f044004c910 con 0x7f0450107150 2023-09-30T14:30:51.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.385+0000 7f045867c700 1 -- 172.21.15.32:0/824982550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f045010a820 con 0x7f0450107150 2023-09-30T14:30:51.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.385+0000 7f0455c17700 1 --2- 172.21.15.32:0/824982550 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f043c038630 0x7f043c03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:51.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.386+0000 7f0455c17700 1 --2- 172.21.15.32:0/824982550 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f043c038630 0x7f043c03aaf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f044c006fd0 tx=0x7f044c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:51.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.390+0000 7f04477fe700 1 -- 172.21.15.32:0/824982550 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f0440019bf0 con 0x7f0450107150 2023-09-30T14:30:51.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.756+0000 7f045867c700 1 -- 172.21.15.32:0/824982550 --> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f045019d800 con 0x7f043c038630 2023-09-30T14:30:51.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.970+0000 7f04477fe700 1 -- 172.21.15.32:0/824982550 <== mgr.14120 v2:172.21.15.32:6800/1639474065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f045019d800 con 0x7f043c038630 2023-09-30T14:30:51.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.972+0000 7f045867c700 1 -- 172.21.15.32:0/824982550 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f043c038630 msgr2=0x7f043c03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:51.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.972+0000 7f045867c700 1 --2- 172.21.15.32:0/824982550 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f043c038630 0x7f043c03aaf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f044c006fd0 tx=0x7f044c006e40 comp rx=0 tx=0).stop 2023-09-30T14:30:51.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.972+0000 7f045867c700 1 -- 172.21.15.32:0/824982550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450107150 msgr2=0x7f045019a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:51.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.972+0000 7f045867c700 1 --2- 172.21.15.32:0/824982550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450107150 0x7f045019a260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0440009fb0 tx=0x7f0440011ad0 comp rx=0 tx=0).stop 2023-09-30T14:30:51.387 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.972+0000 7f045867c700 1 -- 172.21.15.32:0/824982550 shutdown_connections 2023-09-30T14:30:51.388 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.972+0000 7f045867c700 1 --2- 172.21.15.32:0/824982550 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f043c038630 0x7f043c03aaf0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:51.388 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.972+0000 7f045867c700 1 --2- 172.21.15.32:0/824982550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450107150 0x7f045019a260 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:51.388 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.972+0000 7f045867c700 1 -- 172.21.15.32:0/824982550 >> 172.21.15.32:0/824982550 conn(0x7f0450100b00 msgr2=0x7f0450101900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:51.388 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.973+0000 7f045867c700 1 -- 172.21.15.32:0/824982550 shutdown_connections 2023-09-30T14:30:51.388 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:50.973+0000 7f045867c700 1 -- 172.21.15.32:0/824982550 wait complete. 2023-09-30T14:30:52.724 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDodJBINELD7Ztf88FYZFaymjkdE1C/W3n/8v3aqRgdx1Cy7s1GM0BUr+BjoNBFK+JWwPUWVkNQe9kUlqCNeH+NYubmwdrCVadVrd0Rsf+1DRadO3LMldXuXhO1io+HIEk9AC9Omo9w/+KGYVrlFxAzfqtVXDo94cLFXrdar4DdUSnOAlXc/SqFyJ6i55HxGesymB4fm+/IG7qPYNm/HUFHFdComhlDoPFJ+uxFVArivrJo1RosQnThvq2J6UulOPFty+OZ4BtzFQsQ7qMdfzKslCWpmogxWBqrkk17FLXF9c4APFtONhkwoDFi8HsZRB8xZ+U5AYJhPIOiJLE3XBoK3I/bbtM91ezlvQHpoRbCo6nGqOd201Sgot+LpYQaDpPPfPiQ1IFDrSehQQhjY6yU+bztcKNUBWXHCoOPN10q+9bYnKgsR+Su3qU1ekzupuhRxJGBlWuGsaHM5oHU0H8d2QyoJDW144PnENNT0YeDBZpl6R1m1NufhxKbtPIsyns= ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:30:52.730 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.875+0000 7fd4560ae700 1 Processor -- start 2023-09-30T14:30:52.730 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.875+0000 7fd4560ae700 1 -- start start 2023-09-30T14:30:52.730 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.875+0000 7fd4560ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4501055c0 0x7fd4501059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:52.730 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.876+0000 7fd4560ae700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd450105fb0 con 0x7fd4501055c0 2023-09-30T14:30:52.730 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.876+0000 7fd44f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4501055c0 0x7fd4501059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:52.730 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.876+0000 7fd44f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4501055c0 0x7fd4501059e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60810/0 (socket says 172.21.15.32:60810) 2023-09-30T14:30:52.730 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.876+0000 7fd44f7fe700 1 -- 172.21.15.32:0/51765215 learned_addr learned my addr 172.21.15.32:0/51765215 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:52.730 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.876+0000 7fd44f7fe700 1 -- 172.21.15.32:0/51765215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4501067c0 con 0x7fd4501055c0 2023-09-30T14:30:52.730 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.876+0000 7fd44f7fe700 1 --2- 172.21.15.32:0/51765215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4501055c0 0x7fd4501059e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd4440098a0 tx=0x7fd444009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4eb15fc7873cbe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:52.730 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.877+0000 7fd44e7fc700 1 -- 172.21.15.32:0/51765215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd444004160 con 0x7fd4501055c0 2023-09-30T14:30:52.731 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.877+0000 7fd44e7fc700 1 -- 172.21.15.32:0/51765215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd4440042c0 con 0x7fd4501055c0 2023-09-30T14:30:52.731 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.877+0000 7fd44e7fc700 1 -- 172.21.15.32:0/51765215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd444011a00 con 0x7fd4501055c0 2023-09-30T14:30:52.731 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.877+0000 7fd4560ae700 1 -- 172.21.15.32:0/51765215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4501055c0 msgr2=0x7fd4501059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:52.731 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.877+0000 7fd4560ae700 1 --2- 172.21.15.32:0/51765215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4501055c0 0x7fd4501059e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd4440098a0 tx=0x7fd444009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:52.731 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.877+0000 7fd4560ae700 1 -- 172.21.15.32:0/51765215 shutdown_connections 2023-09-30T14:30:52.731 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.877+0000 7fd4560ae700 1 --2- 172.21.15.32:0/51765215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4501055c0 0x7fd4501059e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:52.731 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.877+0000 7fd4560ae700 1 -- 172.21.15.32:0/51765215 >> 172.21.15.32:0/51765215 conn(0x7fd450100b80 msgr2=0x7fd450102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:52.731 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.877+0000 7fd4560ae700 1 -- 172.21.15.32:0/51765215 shutdown_connections 2023-09-30T14:30:52.731 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.877+0000 7fd4560ae700 1 -- 172.21.15.32:0/51765215 wait complete. 2023-09-30T14:30:52.731 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.878+0000 7fd4560ae700 1 Processor -- start 2023-09-30T14:30:52.731 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.878+0000 7fd4560ae700 1 -- start start 2023-09-30T14:30:52.732 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.878+0000 7fd4560ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4501055c0 0x7fd450199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:52.732 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.878+0000 7fd4560ae700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd450199750 con 0x7fd4501055c0 2023-09-30T14:30:52.732 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.878+0000 7fd44f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4501055c0 0x7fd450199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:52.732 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.879+0000 7fd44f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4501055c0 0x7fd450199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60812/0 (socket says 172.21.15.32:60812) 2023-09-30T14:30:52.732 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.879+0000 7fd44f7fe700 1 -- 172.21.15.32:0/122040959 learned_addr learned my addr 172.21.15.32:0/122040959 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:52.732 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.879+0000 7fd44f7fe700 1 -- 172.21.15.32:0/122040959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd444009550 con 0x7fd4501055c0 2023-09-30T14:30:52.732 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.879+0000 7fd44f7fe700 1 --2- 172.21.15.32:0/122040959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4501055c0 0x7fd450199210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd444009fb0 tx=0x7fd444011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:52.732 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.880+0000 7fd44cff9700 1 -- 172.21.15.32:0/122040959 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd444003930 con 0x7fd4501055c0 2023-09-30T14:30:52.732 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.880+0000 7fd44cff9700 1 -- 172.21.15.32:0/122040959 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd444003a90 con 0x7fd4501055c0 2023-09-30T14:30:52.732 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.880+0000 7fd44cff9700 1 -- 172.21.15.32:0/122040959 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4440195e0 con 0x7fd4501055c0 2023-09-30T14:30:52.732 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.880+0000 7fd4560ae700 1 -- 172.21.15.32:0/122040959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd450199950 con 0x7fd4501055c0 2023-09-30T14:30:52.733 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.880+0000 7fd4560ae700 1 -- 172.21.15.32:0/122040959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd450199df0 con 0x7fd4501055c0 2023-09-30T14:30:52.733 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.881+0000 7fd4560ae700 1 -- 172.21.15.32:0/122040959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd450062450 con 0x7fd4501055c0 2023-09-30T14:30:52.733 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.882+0000 7fd44cff9700 1 -- 172.21.15.32:0/122040959 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7fd44401a9d0 con 0x7fd4501055c0 2023-09-30T14:30:52.733 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.882+0000 7fd44cff9700 1 --2- 172.21.15.32:0/122040959 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fd438038680 0x7fd43803ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:52.733 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.882+0000 7fd44cff9700 1 -- 172.21.15.32:0/122040959 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd44404cd80 con 0x7fd4501055c0 2023-09-30T14:30:52.733 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.882+0000 7fd44effd700 1 --2- 172.21.15.32:0/122040959 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fd438038680 0x7fd43803ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:52.733 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.883+0000 7fd44effd700 1 --2- 172.21.15.32:0/122040959 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fd438038680 0x7fd43803ab40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd440006fd0 tx=0x7fd440006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:52.733 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:51.886+0000 7fd44cff9700 1 -- 172.21.15.32:0/122040959 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fd444003c00 con 0x7fd4501055c0 2023-09-30T14:30:52.733 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:52.251+0000 7fd4560ae700 1 -- 172.21.15.32:0/122040959 --> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4501099c0 con 0x7fd438038680 2023-09-30T14:30:52.734 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:52.251+0000 7fd44cff9700 1 -- 172.21.15.32:0/122040959 <== mgr.14120 v2:172.21.15.32:6800/1639474065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fd4501099c0 con 0x7fd438038680 2023-09-30T14:30:52.734 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:52.254+0000 7fd4560ae700 1 -- 172.21.15.32:0/122040959 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fd438038680 msgr2=0x7fd43803ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:52.734 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:52.254+0000 7fd4560ae700 1 --2- 172.21.15.32:0/122040959 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fd438038680 0x7fd43803ab40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd440006fd0 tx=0x7fd440006e40 comp rx=0 tx=0).stop 2023-09-30T14:30:52.734 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:52.254+0000 7fd4560ae700 1 -- 172.21.15.32:0/122040959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4501055c0 msgr2=0x7fd450199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:52.734 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:52.254+0000 7fd4560ae700 1 --2- 172.21.15.32:0/122040959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4501055c0 0x7fd450199210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd444009fb0 tx=0x7fd444011ad0 comp rx=0 tx=0).stop 2023-09-30T14:30:52.734 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:52.254+0000 7fd4560ae700 1 -- 172.21.15.32:0/122040959 shutdown_connections 2023-09-30T14:30:52.734 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:52.254+0000 7fd4560ae700 1 --2- 172.21.15.32:0/122040959 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fd438038680 0x7fd43803ab40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:52.734 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:52.254+0000 7fd4560ae700 1 --2- 172.21.15.32:0/122040959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4501055c0 0x7fd450199210 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:52.734 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:52.254+0000 7fd4560ae700 1 -- 172.21.15.32:0/122040959 >> 172.21.15.32:0/122040959 conn(0x7fd450100b80 msgr2=0x7fd450101920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:52.735 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:52.254+0000 7fd4560ae700 1 -- 172.21.15.32:0/122040959 shutdown_connections 2023-09-30T14:30:52.735 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:52.254+0000 7fd4560ae700 1 -- 172.21.15.32:0/122040959 wait complete. 2023-09-30T14:30:52.735 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-09-30T14:30:52.735 INFO:teuthology.orchestra.run.smithi032.stdout:Adding key to root@localhost authorized_keys... 2023-09-30T14:30:52.735 INFO:teuthology.orchestra.run.smithi032.stdout:Adding host smithi032... 2023-09-30T14:30:53.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:52 smithi032 ceph-mon[98592]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:30:54.127 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:53 smithi032 ceph-mon[98592]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi032", "addr": "172.21.15.32", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:30:55.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:54 smithi032 ceph-mon[98592]: Deploying cephadm binary to smithi032 2023-09-30T14:30:56.885 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Added host 'smithi032' with addr '172.21.15.32' 2023-09-30T14:30:56.885 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.195+0000 7fbd78c7b700 1 Processor -- start 2023-09-30T14:30:56.885 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.195+0000 7fbd78c7b700 1 -- start start 2023-09-30T14:30:56.885 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.196+0000 7fbd78c7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd74107180 0x7fbd74109570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:56.885 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.196+0000 7fbd78c7b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd74079e70 con 0x7fbd74107180 2023-09-30T14:30:56.886 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.196+0000 7fbd7259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd74107180 0x7fbd74109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:56.886 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.196+0000 7fbd7259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd74107180 0x7fbd74109570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60816/0 (socket says 172.21.15.32:60816) 2023-09-30T14:30:56.886 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.196+0000 7fbd7259c700 1 -- 172.21.15.32:0/1791185856 learned_addr learned my addr 172.21.15.32:0/1791185856 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:56.886 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.196+0000 7fbd7259c700 1 -- 172.21.15.32:0/1791185856 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd74109ab0 con 0x7fbd74107180 2023-09-30T14:30:56.886 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.196+0000 7fbd7259c700 1 --2- 172.21.15.32:0/1791185856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd74107180 0x7fbd74109570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbd640098a0 tx=0x7fbd64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7465c308b839bb3e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:56.886 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.197+0000 7fbd7159a700 1 -- 172.21.15.32:0/1791185856 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd64004160 con 0x7fbd74107180 2023-09-30T14:30:56.886 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.197+0000 7fbd7159a700 1 -- 172.21.15.32:0/1791185856 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbd640042c0 con 0x7fbd74107180 2023-09-30T14:30:56.886 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.197+0000 7fbd7159a700 1 -- 172.21.15.32:0/1791185856 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd64011a00 con 0x7fbd74107180 2023-09-30T14:30:56.886 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.197+0000 7fbd78c7b700 1 -- 172.21.15.32:0/1791185856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd74107180 msgr2=0x7fbd74109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:56.886 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.197+0000 7fbd78c7b700 1 --2- 172.21.15.32:0/1791185856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd74107180 0x7fbd74109570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbd640098a0 tx=0x7fbd64009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:56.886 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.197+0000 7fbd78c7b700 1 -- 172.21.15.32:0/1791185856 shutdown_connections 2023-09-30T14:30:56.887 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.197+0000 7fbd78c7b700 1 --2- 172.21.15.32:0/1791185856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd74107180 0x7fbd74109570 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:56.887 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.197+0000 7fbd78c7b700 1 -- 172.21.15.32:0/1791185856 >> 172.21.15.32:0/1791185856 conn(0x7fbd74100b70 msgr2=0x7fbd74102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:56.887 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.197+0000 7fbd78c7b700 1 -- 172.21.15.32:0/1791185856 shutdown_connections 2023-09-30T14:30:56.887 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.197+0000 7fbd78c7b700 1 -- 172.21.15.32:0/1791185856 wait complete. 2023-09-30T14:30:56.887 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.198+0000 7fbd78c7b700 1 Processor -- start 2023-09-30T14:30:56.887 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.198+0000 7fbd78c7b700 1 -- start start 2023-09-30T14:30:56.887 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.199+0000 7fbd78c7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd74107180 0x7fbd74106ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:56.887 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.199+0000 7fbd78c7b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd74107020 con 0x7fbd74107180 2023-09-30T14:30:56.887 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.199+0000 7fbd7259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd74107180 0x7fbd74106ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:56.887 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.199+0000 7fbd7259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd74107180 0x7fbd74106ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60818/0 (socket says 172.21.15.32:60818) 2023-09-30T14:30:56.887 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.199+0000 7fbd7259c700 1 -- 172.21.15.32:0/548879387 learned_addr learned my addr 172.21.15.32:0/548879387 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:56.887 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.199+0000 7fbd7259c700 1 -- 172.21.15.32:0/548879387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd64009550 con 0x7fbd74107180 2023-09-30T14:30:56.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.199+0000 7fbd7259c700 1 --2- 172.21.15.32:0/548879387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd74107180 0x7fbd74106ae0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbd64006af0 tx=0x7fbd64011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:56.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.199+0000 7fbd637fe700 1 -- 172.21.15.32:0/548879387 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd64003a40 con 0x7fbd74107180 2023-09-30T14:30:56.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.199+0000 7fbd637fe700 1 -- 172.21.15.32:0/548879387 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbd64003ba0 con 0x7fbd74107180 2023-09-30T14:30:56.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.199+0000 7fbd78c7b700 1 -- 172.21.15.32:0/548879387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd74105130 con 0x7fbd74107180 2023-09-30T14:30:56.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.199+0000 7fbd637fe700 1 -- 172.21.15.32:0/548879387 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd6401a6e0 con 0x7fbd74107180 2023-09-30T14:30:56.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.200+0000 7fbd78c7b700 1 -- 172.21.15.32:0/548879387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd741055d0 con 0x7fbd74107180 2023-09-30T14:30:56.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.200+0000 7fbd637fe700 1 -- 172.21.15.32:0/548879387 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7fbd6401a840 con 0x7fbd74107180 2023-09-30T14:30:56.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.201+0000 7fbd637fe700 1 --2- 172.21.15.32:0/548879387 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbd5c0386d0 0x7fbd5c03ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:56.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.201+0000 7fbd637fe700 1 -- 172.21.15.32:0/548879387 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbd6404c660 con 0x7fbd74107180 2023-09-30T14:30:56.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.201+0000 7fbd71d9b700 1 --2- 172.21.15.32:0/548879387 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbd5c0386d0 0x7fbd5c03ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:56.888 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.201+0000 7fbd78c7b700 1 -- 172.21.15.32:0/548879387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd740624c0 con 0x7fbd74107180 2023-09-30T14:30:56.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.201+0000 7fbd71d9b700 1 --2- 172.21.15.32:0/548879387 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbd5c0386d0 0x7fbd5c03ab90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbd68006fd0 tx=0x7fbd68006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:56.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.206+0000 7fbd637fe700 1 -- 172.21.15.32:0/548879387 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fbd740624c0 con 0x7fbd74107180 2023-09-30T14:30:56.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:53.576+0000 7fbd78c7b700 1 -- 172.21.15.32:0/548879387 --> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi032", "addr": "172.21.15.32", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd740624c0 con 0x7fbd5c0386d0 2023-09-30T14:30:56.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:56.492+0000 7fbd637fe700 1 -- 172.21.15.32:0/548879387 <== mgr.14120 v2:172.21.15.32:6800/1639474065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fbd740624c0 con 0x7fbd5c0386d0 2023-09-30T14:30:56.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:56.495+0000 7fbd78c7b700 1 -- 172.21.15.32:0/548879387 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbd5c0386d0 msgr2=0x7fbd5c03ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:56.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:56.495+0000 7fbd78c7b700 1 --2- 172.21.15.32:0/548879387 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbd5c0386d0 0x7fbd5c03ab90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbd68006fd0 tx=0x7fbd68006e40 comp rx=0 tx=0).stop 2023-09-30T14:30:56.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:56.495+0000 7fbd78c7b700 1 -- 172.21.15.32:0/548879387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd74107180 msgr2=0x7fbd74106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:56.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:56.495+0000 7fbd78c7b700 1 --2- 172.21.15.32:0/548879387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd74107180 0x7fbd74106ae0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbd64006af0 tx=0x7fbd64011ad0 comp rx=0 tx=0).stop 2023-09-30T14:30:56.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:56.495+0000 7fbd78c7b700 1 -- 172.21.15.32:0/548879387 shutdown_connections 2023-09-30T14:30:56.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:56.495+0000 7fbd78c7b700 1 --2- 172.21.15.32:0/548879387 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbd5c0386d0 0x7fbd5c03ab90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:56.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:56.495+0000 7fbd78c7b700 1 --2- 172.21.15.32:0/548879387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd74107180 0x7fbd74106ae0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:56.889 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:56.495+0000 7fbd78c7b700 1 -- 172.21.15.32:0/548879387 >> 172.21.15.32:0/548879387 conn(0x7fbd74100b70 msgr2=0x7fbd741018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:56.890 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:56.495+0000 7fbd78c7b700 1 -- 172.21.15.32:0/548879387 shutdown_connections 2023-09-30T14:30:56.890 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:56.495+0000 7fbd78c7b700 1 -- 172.21.15.32:0/548879387 wait complete. 2023-09-30T14:30:56.890 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying unmanaged mon service... 2023-09-30T14:30:57.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:57 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' 2023-09-30T14:30:57.803 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:57 smithi032 ceph-mon[98592]: Added host smithi032 2023-09-30T14:30:57.804 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:57 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:30:58.226 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-09-30T14:30:58.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.361+0000 7fab360d9700 1 Processor -- start 2023-09-30T14:30:58.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.361+0000 7fab360d9700 1 -- start start 2023-09-30T14:30:58.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.362+0000 7fab360d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab301055c0 0x7fab301059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:58.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.362+0000 7fab360d9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab30105fb0 con 0x7fab301055c0 2023-09-30T14:30:58.228 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.362+0000 7fab2f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab301055c0 0x7fab301059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:58.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.362+0000 7fab2f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab301055c0 0x7fab301059e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60852/0 (socket says 172.21.15.32:60852) 2023-09-30T14:30:58.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.362+0000 7fab2f7fe700 1 -- 172.21.15.32:0/933718329 learned_addr learned my addr 172.21.15.32:0/933718329 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:58.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.362+0000 7fab2f7fe700 1 -- 172.21.15.32:0/933718329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab301067c0 con 0x7fab301055c0 2023-09-30T14:30:58.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.362+0000 7fab2f7fe700 1 --2- 172.21.15.32:0/933718329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab301055c0 0x7fab301059e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fab180098a0 tx=0x7fab18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1757851a0869211c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:58.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.363+0000 7fab2e7fc700 1 -- 172.21.15.32:0/933718329 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab18004160 con 0x7fab301055c0 2023-09-30T14:30:58.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.363+0000 7fab2e7fc700 1 -- 172.21.15.32:0/933718329 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fab180042c0 con 0x7fab301055c0 2023-09-30T14:30:58.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.363+0000 7fab2e7fc700 1 -- 172.21.15.32:0/933718329 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab18011a00 con 0x7fab301055c0 2023-09-30T14:30:58.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.363+0000 7fab360d9700 1 -- 172.21.15.32:0/933718329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab301055c0 msgr2=0x7fab301059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:58.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.363+0000 7fab360d9700 1 --2- 172.21.15.32:0/933718329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab301055c0 0x7fab301059e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fab180098a0 tx=0x7fab18009bb0 comp rx=0 tx=0).stop 2023-09-30T14:30:58.229 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.363+0000 7fab360d9700 1 -- 172.21.15.32:0/933718329 shutdown_connections 2023-09-30T14:30:58.233 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.363+0000 7fab360d9700 1 --2- 172.21.15.32:0/933718329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab301055c0 0x7fab301059e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:58.234 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.363+0000 7fab360d9700 1 -- 172.21.15.32:0/933718329 >> 172.21.15.32:0/933718329 conn(0x7fab30100b80 msgr2=0x7fab30102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:58.234 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.364+0000 7fab360d9700 1 -- 172.21.15.32:0/933718329 shutdown_connections 2023-09-30T14:30:58.234 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.364+0000 7fab360d9700 1 -- 172.21.15.32:0/933718329 wait complete. 2023-09-30T14:30:58.234 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.364+0000 7fab360d9700 1 Processor -- start 2023-09-30T14:30:58.234 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.364+0000 7fab360d9700 1 -- start start 2023-09-30T14:30:58.234 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.365+0000 7fab360d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab301055c0 0x7fab30074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:58.234 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.365+0000 7fab360d9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3019d040 con 0x7fab301055c0 2023-09-30T14:30:58.234 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.365+0000 7fab2f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab301055c0 0x7fab30074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:58.234 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.365+0000 7fab2f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab301055c0 0x7fab30074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60854/0 (socket says 172.21.15.32:60854) 2023-09-30T14:30:58.234 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.365+0000 7fab2f7fe700 1 -- 172.21.15.32:0/3342852656 learned_addr learned my addr 172.21.15.32:0/3342852656 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:58.234 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.365+0000 7fab2f7fe700 1 -- 172.21.15.32:0/3342852656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab18009550 con 0x7fab301055c0 2023-09-30T14:30:58.235 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.365+0000 7fab2f7fe700 1 --2- 172.21.15.32:0/3342852656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab301055c0 0x7fab30074300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fab18009fb0 tx=0x7fab18011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:58.235 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.365+0000 7fab2cff9700 1 -- 172.21.15.32:0/3342852656 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab18003930 con 0x7fab301055c0 2023-09-30T14:30:58.235 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.365+0000 7fab2cff9700 1 -- 172.21.15.32:0/3342852656 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fab18003a90 con 0x7fab301055c0 2023-09-30T14:30:58.235 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.365+0000 7fab2cff9700 1 -- 172.21.15.32:0/3342852656 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab180195e0 con 0x7fab301055c0 2023-09-30T14:30:58.235 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.365+0000 7fab360d9700 1 -- 172.21.15.32:0/3342852656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3019d180 con 0x7fab301055c0 2023-09-30T14:30:58.235 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.365+0000 7fab360d9700 1 -- 172.21.15.32:0/3342852656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3019d620 con 0x7fab301055c0 2023-09-30T14:30:58.235 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.366+0000 7fab2cff9700 1 -- 172.21.15.32:0/3342852656 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7fab1801a9d0 con 0x7fab301055c0 2023-09-30T14:30:58.235 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.367+0000 7fab2cff9700 1 --2- 172.21.15.32:0/3342852656 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fab1c038680 0x7fab1c03ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:58.235 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.367+0000 7fab2cff9700 1 -- 172.21.15.32:0/3342852656 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fab1804c750 con 0x7fab301055c0 2023-09-30T14:30:58.235 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.367+0000 7fab360d9700 1 -- 172.21.15.32:0/3342852656 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab30062450 con 0x7fab301055c0 2023-09-30T14:30:58.235 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.367+0000 7fab2effd700 1 --2- 172.21.15.32:0/3342852656 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fab1c038680 0x7fab1c03ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:58.235 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.367+0000 7fab2effd700 1 --2- 172.21.15.32:0/3342852656 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fab1c038680 0x7fab1c03ab40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fab20006fd0 tx=0x7fab20006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:58.236 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.372+0000 7fab2cff9700 1 -- 172.21.15.32:0/3342852656 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fab18019850 con 0x7fab301055c0 2023-09-30T14:30:58.236 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.769+0000 7fab360d9700 1 -- 172.21.15.32:0/3342852656 --> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fab301099c0 con 0x7fab1c038680 2023-09-30T14:30:58.236 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.773+0000 7fab2cff9700 1 -- 172.21.15.32:0/3342852656 <== mgr.14120 v2:172.21.15.32:6800/1639474065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fab301099c0 con 0x7fab1c038680 2023-09-30T14:30:58.236 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.776+0000 7fab360d9700 1 -- 172.21.15.32:0/3342852656 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fab1c038680 msgr2=0x7fab1c03ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:58.236 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.776+0000 7fab360d9700 1 --2- 172.21.15.32:0/3342852656 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fab1c038680 0x7fab1c03ab40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fab20006fd0 tx=0x7fab20006e40 comp rx=0 tx=0).stop 2023-09-30T14:30:58.236 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.776+0000 7fab360d9700 1 -- 172.21.15.32:0/3342852656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab301055c0 msgr2=0x7fab30074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:58.236 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.776+0000 7fab360d9700 1 --2- 172.21.15.32:0/3342852656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab301055c0 0x7fab30074300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fab18009fb0 tx=0x7fab18011ad0 comp rx=0 tx=0).stop 2023-09-30T14:30:58.236 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.776+0000 7fab360d9700 1 -- 172.21.15.32:0/3342852656 shutdown_connections 2023-09-30T14:30:58.236 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.776+0000 7fab360d9700 1 --2- 172.21.15.32:0/3342852656 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fab1c038680 0x7fab1c03ab40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:58.236 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.776+0000 7fab360d9700 1 --2- 172.21.15.32:0/3342852656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab301055c0 0x7fab30074300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:58.236 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.776+0000 7fab360d9700 1 -- 172.21.15.32:0/3342852656 >> 172.21.15.32:0/3342852656 conn(0x7fab30100b80 msgr2=0x7fab30101920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:58.236 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.776+0000 7fab360d9700 1 -- 172.21.15.32:0/3342852656 shutdown_connections 2023-09-30T14:30:58.237 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:57.776+0000 7fab360d9700 1 -- 172.21.15.32:0/3342852656 wait complete. 2023-09-30T14:30:58.237 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying unmanaged mgr service... 2023-09-30T14:30:59.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:58 smithi032 ceph-mon[98592]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:30:59.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:58 smithi032 ceph-mon[98592]: Saving service mon spec with placement count:5 2023-09-30T14:30:59.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:58 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' 2023-09-30T14:30:59.781 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-09-30T14:30:59.782 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.903+0000 7f9731caa700 1 Processor -- start 2023-09-30T14:30:59.782 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.903+0000 7f9731caa700 1 -- start start 2023-09-30T14:30:59.782 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.903+0000 7f9731caa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c1055c0 0x7f972c1059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:59.782 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.903+0000 7f9731caa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c105fb0 con 0x7f972c1055c0 2023-09-30T14:30:59.782 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.903+0000 7f972b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c1055c0 0x7f972c1059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:59.782 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.903+0000 7f972b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c1055c0 0x7f972c1059e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60858/0 (socket says 172.21.15.32:60858) 2023-09-30T14:30:59.782 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.903+0000 7f972b7fe700 1 -- 172.21.15.32:0/1737559077 learned_addr learned my addr 172.21.15.32:0/1737559077 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:59.782 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.904+0000 7f972b7fe700 1 -- 172.21.15.32:0/1737559077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c1067c0 con 0x7f972c1055c0 2023-09-30T14:30:59.782 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.904+0000 7f972b7fe700 1 --2- 172.21.15.32:0/1737559077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c1055c0 0x7f972c1059e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f971c00bb40 tx=0x7f971c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf71571cf68fc19 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:59.782 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.904+0000 7f972a7fc700 1 -- 172.21.15.32:0/1737559077 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f971c00dbe0 con 0x7f972c1055c0 2023-09-30T14:30:59.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.904+0000 7f972a7fc700 1 -- 172.21.15.32:0/1737559077 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f971c0044f0 con 0x7f972c1055c0 2023-09-30T14:30:59.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.904+0000 7f972a7fc700 1 -- 172.21.15.32:0/1737559077 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f971c003e60 con 0x7f972c1055c0 2023-09-30T14:30:59.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.905+0000 7f9731caa700 1 -- 172.21.15.32:0/1737559077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c1055c0 msgr2=0x7f972c1059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:59.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.905+0000 7f9731caa700 1 --2- 172.21.15.32:0/1737559077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c1055c0 0x7f972c1059e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f971c00bb40 tx=0x7f971c00be50 comp rx=0 tx=0).stop 2023-09-30T14:30:59.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.905+0000 7f9731caa700 1 -- 172.21.15.32:0/1737559077 shutdown_connections 2023-09-30T14:30:59.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.905+0000 7f9731caa700 1 --2- 172.21.15.32:0/1737559077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c1055c0 0x7f972c1059e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:59.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.905+0000 7f9731caa700 1 -- 172.21.15.32:0/1737559077 >> 172.21.15.32:0/1737559077 conn(0x7f972c100b80 msgr2=0x7f972c102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:59.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.905+0000 7f9731caa700 1 -- 172.21.15.32:0/1737559077 shutdown_connections 2023-09-30T14:30:59.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.905+0000 7f9731caa700 1 -- 172.21.15.32:0/1737559077 wait complete. 2023-09-30T14:30:59.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.906+0000 7f9731caa700 1 Processor -- start 2023-09-30T14:30:59.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.906+0000 7f9731caa700 1 -- start start 2023-09-30T14:30:59.784 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.906+0000 7f9731caa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c1055c0 0x7f972c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:59.784 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.906+0000 7f9731caa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c198640 con 0x7f972c1055c0 2023-09-30T14:30:59.784 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.906+0000 7f972b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c1055c0 0x7f972c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:59.784 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.906+0000 7f972b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c1055c0 0x7f972c198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60860/0 (socket says 172.21.15.32:60860) 2023-09-30T14:30:59.784 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.906+0000 7f972b7fe700 1 -- 172.21.15.32:0/1928077414 learned_addr learned my addr 172.21.15.32:0/1928077414 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:30:59.784 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.906+0000 7f972b7fe700 1 -- 172.21.15.32:0/1928077414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c00b7f0 con 0x7f972c1055c0 2023-09-30T14:30:59.784 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.906+0000 7f972b7fe700 1 --2- 172.21.15.32:0/1928077414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c1055c0 0x7f972c198100 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f971c006fa0 tx=0x7f971c004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:59.784 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.907+0000 7f9728ff9700 1 -- 172.21.15.32:0/1928077414 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f971c01e070 con 0x7f972c1055c0 2023-09-30T14:30:59.784 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.907+0000 7f9728ff9700 1 -- 172.21.15.32:0/1928077414 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f971c010af0 con 0x7f972c1055c0 2023-09-30T14:30:59.784 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.907+0000 7f9731caa700 1 -- 172.21.15.32:0/1928077414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972c198840 con 0x7f972c1055c0 2023-09-30T14:30:59.784 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.907+0000 7f9728ff9700 1 -- 172.21.15.32:0/1928077414 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f971c022a30 con 0x7f972c1055c0 2023-09-30T14:30:59.784 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.907+0000 7f9731caa700 1 -- 172.21.15.32:0/1928077414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972c198ce0 con 0x7f972c1055c0 2023-09-30T14:30:59.785 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.908+0000 7f9731caa700 1 -- 172.21.15.32:0/1928077414 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972c191e50 con 0x7f972c1055c0 2023-09-30T14:30:59.785 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.908+0000 7f9728ff9700 1 -- 172.21.15.32:0/1928077414 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7f971c010c60 con 0x7f972c1055c0 2023-09-30T14:30:59.785 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.908+0000 7f9728ff9700 1 --2- 172.21.15.32:0/1928077414 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f9714038680 0x7f971403ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:30:59.785 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.908+0000 7f9728ff9700 1 -- 172.21.15.32:0/1928077414 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f971c04ea10 con 0x7f972c1055c0 2023-09-30T14:30:59.785 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.909+0000 7f972affd700 1 --2- 172.21.15.32:0/1928077414 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f9714038680 0x7f971403ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:30:59.785 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.909+0000 7f972affd700 1 --2- 172.21.15.32:0/1928077414 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f9714038680 0x7f971403ab40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9720006fd0 tx=0x7f9720006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:30:59.785 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:58.913+0000 7f9728ff9700 1 -- 172.21.15.32:0/1928077414 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f971c02a540 con 0x7f972c1055c0 2023-09-30T14:30:59.785 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:59.304+0000 7f9731caa700 1 -- 172.21.15.32:0/1928077414 --> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f972c02d120 con 0x7f9714038680 2023-09-30T14:30:59.785 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:59.308+0000 7f9728ff9700 1 -- 172.21.15.32:0/1928077414 <== mgr.14120 v2:172.21.15.32:6800/1639474065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f972c02d120 con 0x7f9714038680 2023-09-30T14:30:59.785 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:59.311+0000 7f9731caa700 1 -- 172.21.15.32:0/1928077414 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f9714038680 msgr2=0x7f971403ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:59.785 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:59.311+0000 7f9731caa700 1 --2- 172.21.15.32:0/1928077414 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f9714038680 0x7f971403ab40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9720006fd0 tx=0x7f9720006e40 comp rx=0 tx=0).stop 2023-09-30T14:30:59.785 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:59.311+0000 7f9731caa700 1 -- 172.21.15.32:0/1928077414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c1055c0 msgr2=0x7f972c198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:30:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:59.311+0000 7f9731caa700 1 --2- 172.21.15.32:0/1928077414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c1055c0 0x7f972c198100 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f971c006fa0 tx=0x7f971c004020 comp rx=0 tx=0).stop 2023-09-30T14:30:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:59.311+0000 7f9731caa700 1 -- 172.21.15.32:0/1928077414 shutdown_connections 2023-09-30T14:30:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:59.311+0000 7f9731caa700 1 --2- 172.21.15.32:0/1928077414 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f9714038680 0x7f971403ab40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:59.311+0000 7f9731caa700 1 --2- 172.21.15.32:0/1928077414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c1055c0 0x7f972c198100 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:30:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:59.311+0000 7f9731caa700 1 -- 172.21.15.32:0/1928077414 >> 172.21.15.32:0/1928077414 conn(0x7f972c100b80 msgr2=0x7f972c18ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:30:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:59.311+0000 7f9731caa700 1 -- 172.21.15.32:0/1928077414 shutdown_connections 2023-09-30T14:30:59.786 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:30:59.311+0000 7f9731caa700 1 -- 172.21.15.32:0/1928077414 wait complete. 2023-09-30T14:31:00.066 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:59 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' 2023-09-30T14:31:00.066 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:59 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' 2023-09-30T14:31:00.066 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:30:59 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' 2023-09-30T14:31:01.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.298+0000 7fd3b599a700 1 Processor -- start 2023-09-30T14:31:01.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.298+0000 7fd3b599a700 1 -- start start 2023-09-30T14:31:01.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.298+0000 7fd3b599a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b0107160 0x7fd3b0109550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:01.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.298+0000 7fd3b599a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b0079e90 con 0x7fd3b0107160 2023-09-30T14:31:01.127 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.298+0000 7fd3aeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b0107160 0x7fd3b0109550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:01.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.298+0000 7fd3aeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b0107160 0x7fd3b0109550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60864/0 (socket says 172.21.15.32:60864) 2023-09-30T14:31:01.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.298+0000 7fd3aeffd700 1 -- 172.21.15.32:0/397410316 learned_addr learned my addr 172.21.15.32:0/397410316 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:01.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.299+0000 7fd3aeffd700 1 -- 172.21.15.32:0/397410316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b0109a90 con 0x7fd3b0107160 2023-09-30T14:31:01.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.299+0000 7fd3aeffd700 1 --2- 172.21.15.32:0/397410316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b0107160 0x7fd3b0109550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40098a0 tx=0x7fd3a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12eb6beef048d525 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:01.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.299+0000 7fd3adffb700 1 -- 172.21.15.32:0/397410316 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3a4004160 con 0x7fd3b0107160 2023-09-30T14:31:01.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.299+0000 7fd3adffb700 1 -- 172.21.15.32:0/397410316 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3a40042c0 con 0x7fd3b0107160 2023-09-30T14:31:01.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.299+0000 7fd3adffb700 1 -- 172.21.15.32:0/397410316 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3a40119b0 con 0x7fd3b0107160 2023-09-30T14:31:01.128 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.300+0000 7fd3b599a700 1 -- 172.21.15.32:0/397410316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b0107160 msgr2=0x7fd3b0109550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:01.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.300+0000 7fd3b599a700 1 --2- 172.21.15.32:0/397410316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b0107160 0x7fd3b0109550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40098a0 tx=0x7fd3a4009bb0 comp rx=0 tx=0).stop 2023-09-30T14:31:01.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.300+0000 7fd3b599a700 1 -- 172.21.15.32:0/397410316 shutdown_connections 2023-09-30T14:31:01.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.300+0000 7fd3b599a700 1 --2- 172.21.15.32:0/397410316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b0107160 0x7fd3b0109550 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:01.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.300+0000 7fd3b599a700 1 -- 172.21.15.32:0/397410316 >> 172.21.15.32:0/397410316 conn(0x7fd3b0100b70 msgr2=0x7fd3b0102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:01.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.300+0000 7fd3b599a700 1 -- 172.21.15.32:0/397410316 shutdown_connections 2023-09-30T14:31:01.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.300+0000 7fd3b599a700 1 -- 172.21.15.32:0/397410316 wait complete. 2023-09-30T14:31:01.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.301+0000 7fd3b599a700 1 Processor -- start 2023-09-30T14:31:01.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.301+0000 7fd3b599a700 1 -- start start 2023-09-30T14:31:01.129 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.301+0000 7fd3b599a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b0107160 0x7fd3b019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:01.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.301+0000 7fd3b599a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b019a800 con 0x7fd3b0107160 2023-09-30T14:31:01.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.301+0000 7fd3aeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b0107160 0x7fd3b019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:01.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.301+0000 7fd3aeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b0107160 0x7fd3b019a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60866/0 (socket says 172.21.15.32:60866) 2023-09-30T14:31:01.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.301+0000 7fd3aeffd700 1 -- 172.21.15.32:0/186365850 learned_addr learned my addr 172.21.15.32:0/186365850 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:01.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.301+0000 7fd3aeffd700 1 -- 172.21.15.32:0/186365850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a4009550 con 0x7fd3b0107160 2023-09-30T14:31:01.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.302+0000 7fd3aeffd700 1 --2- 172.21.15.32:0/186365850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b0107160 0x7fd3b019a2c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4011980 tx=0x7fd3a4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:01.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.302+0000 7fd3b4998700 1 -- 172.21.15.32:0/186365850 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3a4003a60 con 0x7fd3b0107160 2023-09-30T14:31:01.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.302+0000 7fd3b4998700 1 -- 172.21.15.32:0/186365850 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3a4003bc0 con 0x7fd3b0107160 2023-09-30T14:31:01.130 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.302+0000 7fd3b599a700 1 -- 172.21.15.32:0/186365850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b019aa00 con 0x7fd3b0107160 2023-09-30T14:31:01.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.302+0000 7fd3b4998700 1 -- 172.21.15.32:0/186365850 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3a401a670 con 0x7fd3b0107160 2023-09-30T14:31:01.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.302+0000 7fd3b599a700 1 -- 172.21.15.32:0/186365850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b019aea0 con 0x7fd3b0107160 2023-09-30T14:31:01.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.303+0000 7fd3b4998700 1 -- 172.21.15.32:0/186365850 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7fd3a401a8e0 con 0x7fd3b0107160 2023-09-30T14:31:01.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.303+0000 7fd3b4998700 1 --2- 172.21.15.32:0/186365850 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fd3980386d0 0x7fd39803ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:01.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.303+0000 7fd3b599a700 1 -- 172.21.15.32:0/186365850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b010a870 con 0x7fd3b0107160 2023-09-30T14:31:01.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.303+0000 7fd3b4998700 1 -- 172.21.15.32:0/186365850 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd3a404c840 con 0x7fd3b0107160 2023-09-30T14:31:01.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.303+0000 7fd3ae7fc700 1 --2- 172.21.15.32:0/186365850 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fd3980386d0 0x7fd39803ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:01.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.304+0000 7fd3ae7fc700 1 --2- 172.21.15.32:0/186365850 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fd3980386d0 0x7fd39803ab90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0006fd0 tx=0x7fd3a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:01.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.308+0000 7fd3b4998700 1 -- 172.21.15.32:0/186365850 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fd3a401ab90 con 0x7fd3b0107160 2023-09-30T14:31:01.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.678+0000 7fd3b599a700 1 -- 172.21.15.32:0/186365850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fd3b019d860 con 0x7fd3b0107160 2023-09-30T14:31:01.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.680+0000 7fd3b4998700 1 -- 172.21.15.32:0/186365850 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fd3a4018350 con 0x7fd3b0107160 2023-09-30T14:31:01.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.683+0000 7fd3b599a700 1 -- 172.21.15.32:0/186365850 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fd3980386d0 msgr2=0x7fd39803ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:01.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.683+0000 7fd3b599a700 1 --2- 172.21.15.32:0/186365850 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fd3980386d0 0x7fd39803ab90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0006fd0 tx=0x7fd3a0006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:01.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.683+0000 7fd3b599a700 1 -- 172.21.15.32:0/186365850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b0107160 msgr2=0x7fd3b019a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:01.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.683+0000 7fd3b599a700 1 --2- 172.21.15.32:0/186365850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b0107160 0x7fd3b019a2c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4011980 tx=0x7fd3a4011a60 comp rx=0 tx=0).stop 2023-09-30T14:31:01.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.683+0000 7fd3b599a700 1 -- 172.21.15.32:0/186365850 shutdown_connections 2023-09-30T14:31:01.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.683+0000 7fd3b599a700 1 --2- 172.21.15.32:0/186365850 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fd3980386d0 0x7fd39803ab90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:01.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.683+0000 7fd3b599a700 1 --2- 172.21.15.32:0/186365850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b0107160 0x7fd3b019a2c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:01.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.683+0000 7fd3b599a700 1 -- 172.21.15.32:0/186365850 >> 172.21.15.32:0/186365850 conn(0x7fd3b0100b70 msgr2=0x7fd3b0101880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:01.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.683+0000 7fd3b599a700 1 -- 172.21.15.32:0/186365850 shutdown_connections 2023-09-30T14:31:01.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:00.683+0000 7fd3b599a700 1 -- 172.21.15.32:0/186365850 wait complete. 2023-09-30T14:31:01.133 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:00 smithi032 ceph-mon[98592]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:31:01.133 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:00 smithi032 ceph-mon[98592]: Saving service mgr spec with placement count:2 2023-09-30T14:31:01.133 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:00 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/186365850' entity='client.admin' 2023-09-30T14:31:02.343 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:02 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' 2023-09-30T14:31:02.344 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:02 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/4139230101' entity='client.admin' 2023-09-30T14:31:02.344 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:02 smithi032 ceph-mon[98592]: from='mgr.14120 172.21.15.32:0/2484680896' entity='mgr.x' 2023-09-30T14:31:02.373 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.622+0000 7f74364df700 1 Processor -- start 2023-09-30T14:31:02.373 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.622+0000 7f74364df700 1 -- start start 2023-09-30T14:31:02.373 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.622+0000 7f74364df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7430107130 0x7f7430109520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:02.373 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.622+0000 7f74364df700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7430074660 con 0x7f7430107130 2023-09-30T14:31:02.373 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.622+0000 7f742ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7430107130 0x7f7430109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:02.373 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.623+0000 7f742ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7430107130 0x7f7430109520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60870/0 (socket says 172.21.15.32:60870) 2023-09-30T14:31:02.373 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.623+0000 7f742ffff700 1 -- 172.21.15.32:0/3342241509 learned_addr learned my addr 172.21.15.32:0/3342241509 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:02.373 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.623+0000 7f742ffff700 1 -- 172.21.15.32:0/3342241509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7430109a60 con 0x7f7430107130 2023-09-30T14:31:02.373 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.623+0000 7f742ffff700 1 --2- 172.21.15.32:0/3342241509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7430107130 0x7f7430109520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f74200098a0 tx=0x7f7420009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a3d923077bfc9d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:02.373 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.623+0000 7f742effd700 1 -- 172.21.15.32:0/3342241509 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7420004160 con 0x7f7430107130 2023-09-30T14:31:02.374 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.623+0000 7f742effd700 1 -- 172.21.15.32:0/3342241509 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74200042c0 con 0x7f7430107130 2023-09-30T14:31:02.374 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.623+0000 7f742effd700 1 -- 172.21.15.32:0/3342241509 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74200119b0 con 0x7f7430107130 2023-09-30T14:31:02.374 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.624+0000 7f74364df700 1 -- 172.21.15.32:0/3342241509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7430107130 msgr2=0x7f7430109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:02.374 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.624+0000 7f74364df700 1 --2- 172.21.15.32:0/3342241509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7430107130 0x7f7430109520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f74200098a0 tx=0x7f7420009bb0 comp rx=0 tx=0).stop 2023-09-30T14:31:02.374 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.624+0000 7f74364df700 1 -- 172.21.15.32:0/3342241509 shutdown_connections 2023-09-30T14:31:02.374 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.624+0000 7f74364df700 1 --2- 172.21.15.32:0/3342241509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7430107130 0x7f7430109520 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:02.374 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.624+0000 7f74364df700 1 -- 172.21.15.32:0/3342241509 >> 172.21.15.32:0/3342241509 conn(0x7f7430100b80 msgr2=0x7f7430102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:02.374 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.624+0000 7f74364df700 1 -- 172.21.15.32:0/3342241509 shutdown_connections 2023-09-30T14:31:02.375 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.624+0000 7f74364df700 1 -- 172.21.15.32:0/3342241509 wait complete. 2023-09-30T14:31:02.375 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.625+0000 7f74364df700 1 Processor -- start 2023-09-30T14:31:02.375 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.625+0000 7f74364df700 1 -- start start 2023-09-30T14:31:02.375 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.625+0000 7f74364df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7430107130 0x7f7430106a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:02.375 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.625+0000 7f74364df700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7430106fd0 con 0x7f7430107130 2023-09-30T14:31:02.375 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.625+0000 7f742ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7430107130 0x7f7430106a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:02.375 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.625+0000 7f742ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7430107130 0x7f7430106a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60872/0 (socket says 172.21.15.32:60872) 2023-09-30T14:31:02.375 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.625+0000 7f742ffff700 1 -- 172.21.15.32:0/4139230101 learned_addr learned my addr 172.21.15.32:0/4139230101 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:02.375 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.626+0000 7f742ffff700 1 -- 172.21.15.32:0/4139230101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7420009550 con 0x7f7430107130 2023-09-30T14:31:02.375 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.626+0000 7f742ffff700 1 --2- 172.21.15.32:0/4139230101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7430107130 0x7f7430106a90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7420011980 tx=0x7f7420011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:02.375 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.626+0000 7f742d7fa700 1 -- 172.21.15.32:0/4139230101 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7420003a60 con 0x7f7430107130 2023-09-30T14:31:02.375 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.626+0000 7f742d7fa700 1 -- 172.21.15.32:0/4139230101 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7420003bc0 con 0x7f7430107130 2023-09-30T14:31:02.376 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.626+0000 7f742d7fa700 1 -- 172.21.15.32:0/4139230101 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f742001a670 con 0x7f7430107130 2023-09-30T14:31:02.376 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.626+0000 7f74364df700 1 -- 172.21.15.32:0/4139230101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74301050e0 con 0x7f7430107130 2023-09-30T14:31:02.376 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.626+0000 7f74364df700 1 -- 172.21.15.32:0/4139230101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7430105580 con 0x7f7430107130 2023-09-30T14:31:02.376 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.627+0000 7f742d7fa700 1 -- 172.21.15.32:0/4139230101 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7f742001a8e0 con 0x7f7430107130 2023-09-30T14:31:02.376 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.627+0000 7f742d7fa700 1 --2- 172.21.15.32:0/4139230101 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f74180386d0 0x7f741803ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:02.376 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.627+0000 7f742d7fa700 1 -- 172.21.15.32:0/4139230101 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f742004c720 con 0x7f7430107130 2023-09-30T14:31:02.376 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.628+0000 7f742f7fe700 1 --2- 172.21.15.32:0/4139230101 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f74180386d0 0x7f741803ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:02.376 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.628+0000 7f74364df700 1 -- 172.21.15.32:0/4139230101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743010a8a0 con 0x7f7430107130 2023-09-30T14:31:02.376 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.628+0000 7f742f7fe700 1 --2- 172.21.15.32:0/4139230101 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f74180386d0 0x7f741803ab90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7424006fd0 tx=0x7f7424006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:02.376 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:01.633+0000 7f742d7fa700 1 -- 172.21.15.32:0/4139230101 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f742001ab90 con 0x7f7430107130 2023-09-30T14:31:02.376 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.007+0000 7f74364df700 1 -- 172.21.15.32:0/4139230101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f743010aa90 con 0x7f7430107130 2023-09-30T14:31:02.377 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.009+0000 7f742d7fa700 1 -- 172.21.15.32:0/4139230101 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f7420018350 con 0x7f7430107130 2023-09-30T14:31:02.377 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.012+0000 7f74364df700 1 -- 172.21.15.32:0/4139230101 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f74180386d0 msgr2=0x7f741803ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:02.377 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.012+0000 7f74364df700 1 --2- 172.21.15.32:0/4139230101 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f74180386d0 0x7f741803ab90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7424006fd0 tx=0x7f7424006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:02.377 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.012+0000 7f74364df700 1 -- 172.21.15.32:0/4139230101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7430107130 msgr2=0x7f7430106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:02.377 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.012+0000 7f74364df700 1 --2- 172.21.15.32:0/4139230101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7430107130 0x7f7430106a90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7420011980 tx=0x7f7420011a60 comp rx=0 tx=0).stop 2023-09-30T14:31:02.377 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.012+0000 7f74364df700 1 -- 172.21.15.32:0/4139230101 shutdown_connections 2023-09-30T14:31:02.377 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.012+0000 7f74364df700 1 --2- 172.21.15.32:0/4139230101 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f74180386d0 0x7f741803ab90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:02.377 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.012+0000 7f74364df700 1 --2- 172.21.15.32:0/4139230101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7430107130 0x7f7430106a90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:02.377 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.012+0000 7f74364df700 1 -- 172.21.15.32:0/4139230101 >> 172.21.15.32:0/4139230101 conn(0x7f7430100b80 msgr2=0x7f7430101860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:02.377 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.012+0000 7f74364df700 1 -- 172.21.15.32:0/4139230101 shutdown_connections 2023-09-30T14:31:02.377 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.012+0000 7f74364df700 1 -- 172.21.15.32:0/4139230101 wait complete. 2023-09-30T14:31:02.377 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling the dashboard module... 2023-09-30T14:31:03.485 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:03 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2925934657' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-09-30T14:31:03.756 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:03 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: ignoring --setuser ceph since I am not root 2023-09-30T14:31:03.756 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:03 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: ignoring --setgroup ceph since I am not root 2023-09-30T14:31:03.756 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:03 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:03.657+0000 7faa431c1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-30T14:31:03.756 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:03 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:03.754+0000 7faa431c1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-30T14:31:04.074 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:03 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:03.865+0000 7faa431c1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-30T14:31:04.075 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:03 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:03.978+0000 7faa431c1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-30T14:31:04.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.903+0000 7f4811d31700 1 Processor -- start 2023-09-30T14:31:04.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.903+0000 7f4811d31700 1 -- start start 2023-09-30T14:31:04.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.904+0000 7f4811d31700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c1066c0 0x7f480c106ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:04.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.904+0000 7f4811d31700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c1070b0 con 0x7f480c1066c0 2023-09-30T14:31:04.121 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.904+0000 7f480b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c1066c0 0x7f480c106ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:04.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.904+0000 7f480b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c1066c0 0x7f480c106ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60876/0 (socket says 172.21.15.32:60876) 2023-09-30T14:31:04.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.904+0000 7f480b7fe700 1 -- 172.21.15.32:0/534820718 learned_addr learned my addr 172.21.15.32:0/534820718 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:04.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.904+0000 7f480b7fe700 1 -- 172.21.15.32:0/534820718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c1078c0 con 0x7f480c1066c0 2023-09-30T14:31:04.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.904+0000 7f480b7fe700 1 --2- 172.21.15.32:0/534820718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c1066c0 0x7f480c106ae0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48000098a0 tx=0x7f4800009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cbba77b9510db6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:04.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.904+0000 7f480a7fc700 1 -- 172.21.15.32:0/534820718 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4800004160 con 0x7f480c1066c0 2023-09-30T14:31:04.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.904+0000 7f480a7fc700 1 -- 172.21.15.32:0/534820718 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48000042c0 con 0x7f480c1066c0 2023-09-30T14:31:04.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.904+0000 7f480a7fc700 1 -- 172.21.15.32:0/534820718 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48000119b0 con 0x7f480c1066c0 2023-09-30T14:31:04.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.905+0000 7f4811d31700 1 -- 172.21.15.32:0/534820718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c1066c0 msgr2=0x7f480c106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:04.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.905+0000 7f4811d31700 1 --2- 172.21.15.32:0/534820718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c1066c0 0x7f480c106ae0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48000098a0 tx=0x7f4800009bb0 comp rx=0 tx=0).stop 2023-09-30T14:31:04.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.905+0000 7f4811d31700 1 -- 172.21.15.32:0/534820718 shutdown_connections 2023-09-30T14:31:04.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.905+0000 7f4811d31700 1 --2- 172.21.15.32:0/534820718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c1066c0 0x7f480c106ae0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:04.122 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.905+0000 7f4811d31700 1 -- 172.21.15.32:0/534820718 >> 172.21.15.32:0/534820718 conn(0x7f480c101c80 msgr2=0x7f480c1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:04.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.905+0000 7f4811d31700 1 -- 172.21.15.32:0/534820718 shutdown_connections 2023-09-30T14:31:04.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.905+0000 7f4811d31700 1 -- 172.21.15.32:0/534820718 wait complete. 2023-09-30T14:31:04.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.906+0000 7f4811d31700 1 Processor -- start 2023-09-30T14:31:04.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.906+0000 7f4811d31700 1 -- start start 2023-09-30T14:31:04.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.906+0000 7f4811d31700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c1066c0 0x7f480c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:04.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.906+0000 7f4811d31700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c19a870 con 0x7f480c1066c0 2023-09-30T14:31:04.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.906+0000 7f480b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c1066c0 0x7f480c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:04.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.906+0000 7f480b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c1066c0 0x7f480c19a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60878/0 (socket says 172.21.15.32:60878) 2023-09-30T14:31:04.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.906+0000 7f480b7fe700 1 -- 172.21.15.32:0/2925934657 learned_addr learned my addr 172.21.15.32:0/2925934657 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:04.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.907+0000 7f480b7fe700 1 -- 172.21.15.32:0/2925934657 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4800009550 con 0x7f480c1066c0 2023-09-30T14:31:04.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.907+0000 7f480b7fe700 1 --2- 172.21.15.32:0/2925934657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c1066c0 0x7f480c19a330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4800003e60 tx=0x7f4800011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:04.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.907+0000 7f4808ff9700 1 -- 172.21.15.32:0/2925934657 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4800003970 con 0x7f480c1066c0 2023-09-30T14:31:04.123 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.907+0000 7f4811d31700 1 -- 172.21.15.32:0/2925934657 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480c19aa70 con 0x7f480c1066c0 2023-09-30T14:31:04.124 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.907+0000 7f4808ff9700 1 -- 172.21.15.32:0/2925934657 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4800003ad0 con 0x7f480c1066c0 2023-09-30T14:31:04.124 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.907+0000 7f4808ff9700 1 -- 172.21.15.32:0/2925934657 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48000195e0 con 0x7f480c1066c0 2023-09-30T14:31:04.124 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.907+0000 7f4811d31700 1 -- 172.21.15.32:0/2925934657 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480c19af10 con 0x7f480c1066c0 2023-09-30T14:31:04.124 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.908+0000 7f4808ff9700 1 -- 172.21.15.32:0/2925934657 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7f48000198a0 con 0x7f480c1066c0 2023-09-30T14:31:04.124 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.908+0000 7f4808ff9700 1 --2- 172.21.15.32:0/2925934657 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f47f4038630 0x7f47f403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:04.124 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.908+0000 7f4808ff9700 1 -- 172.21.15.32:0/2925934657 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f480004c8f0 con 0x7f480c1066c0 2023-09-30T14:31:04.124 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.908+0000 7f4811d31700 1 -- 172.21.15.32:0/2925934657 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f480c10aaf0 con 0x7f480c1066c0 2023-09-30T14:31:04.124 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.908+0000 7f480affd700 1 --2- 172.21.15.32:0/2925934657 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f47f4038630 0x7f47f403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:04.124 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.909+0000 7f480affd700 1 --2- 172.21.15.32:0/2925934657 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f47f4038630 0x7f47f403aaf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f47fc006fd0 tx=0x7f47fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:04.124 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:02.913+0000 7f4808ff9700 1 -- 172.21.15.32:0/2925934657 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f4800019ba0 con 0x7f480c1066c0 2023-09-30T14:31:04.124 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:03.399+0000 7f4811d31700 1 -- 172.21.15.32:0/2925934657 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f480c19daf0 con 0x7f480c1066c0 2023-09-30T14:31:04.125 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:03.450+0000 7f4808ff9700 1 -- 172.21.15.32:0/2925934657 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f4800018b40 con 0x7f480c1066c0 2023-09-30T14:31:04.125 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:03.452+0000 7f4808ff9700 1 -- 172.21.15.32:0/2925934657 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 9) v1 ==== 45505+0+0 (secure 0 0 0) 0x7f480002d9b0 con 0x7f480c1066c0 2023-09-30T14:31:04.125 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:03.454+0000 7f4811d31700 1 -- 172.21.15.32:0/2925934657 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f47f4038630 msgr2=0x7f47f403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:04.125 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:03.454+0000 7f4811d31700 1 --2- 172.21.15.32:0/2925934657 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f47f4038630 0x7f47f403aaf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f47fc006fd0 tx=0x7f47fc006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:04.125 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:03.454+0000 7f4811d31700 1 -- 172.21.15.32:0/2925934657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c1066c0 msgr2=0x7f480c19a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:04.125 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:03.454+0000 7f4811d31700 1 --2- 172.21.15.32:0/2925934657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c1066c0 0x7f480c19a330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4800003e60 tx=0x7f4800011aa0 comp rx=0 tx=0).stop 2023-09-30T14:31:04.125 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:03.454+0000 7f4811d31700 1 -- 172.21.15.32:0/2925934657 shutdown_connections 2023-09-30T14:31:04.125 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:03.454+0000 7f4811d31700 1 --2- 172.21.15.32:0/2925934657 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7f47f4038630 0x7f47f403aaf0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:04.125 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:03.454+0000 7f4811d31700 1 --2- 172.21.15.32:0/2925934657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c1066c0 0x7f480c19a330 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:04.125 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:03.454+0000 7f4811d31700 1 -- 172.21.15.32:0/2925934657 >> 172.21.15.32:0/2925934657 conn(0x7f480c101c80 msgr2=0x7f480c102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:04.125 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:03.454+0000 7f4811d31700 1 -- 172.21.15.32:0/2925934657 shutdown_connections 2023-09-30T14:31:04.125 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:03.454+0000 7f4811d31700 1 -- 172.21.15.32:0/2925934657 wait complete. 2023-09-30T14:31:04.326 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:04 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:04.191+0000 7faa431c1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-30T14:31:04.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:04 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2925934657' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-09-30T14:31:04.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:04 smithi032 ceph-mon[98592]: mgrmap e9: x(active, since 17s) 2023-09-30T14:31:04.939 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:04 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:04.676+0000 7faa431c1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-30T14:31:04.940 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:04 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:04.830+0000 7faa431c1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-30T14:31:05.203 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:04 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:04.938+0000 7faa431c1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-30T14:31:05.203 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:05 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:05.054+0000 7faa431c1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-30T14:31:05.467 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2023-09-30T14:31:05.472 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-09-30T14:31:05.473 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-30T14:31:05.473 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-30T14:31:05.473 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-30T14:31:05.473 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2023-09-30T14:31:05.473 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.597+0000 7efca9193700 1 Processor -- start 2023-09-30T14:31:05.473 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.597+0000 7efca9193700 1 -- start start 2023-09-30T14:31:05.473 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.597+0000 7efca9193700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4107130 0x7efca4109520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:05.473 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.597+0000 7efca9193700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca4074660 con 0x7efca4107130 2023-09-30T14:31:05.473 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.597+0000 7efca2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4107130 0x7efca4109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:05.473 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.597+0000 7efca2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4107130 0x7efca4109520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60892/0 (socket says 172.21.15.32:60892) 2023-09-30T14:31:05.474 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.597+0000 7efca2d9d700 1 -- 172.21.15.32:0/2773556622 learned_addr learned my addr 172.21.15.32:0/2773556622 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:05.474 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.598+0000 7efca2d9d700 1 -- 172.21.15.32:0/2773556622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca4109a60 con 0x7efca4107130 2023-09-30T14:31:05.474 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.598+0000 7efca2d9d700 1 --2- 172.21.15.32:0/2773556622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4107130 0x7efca4109520 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efc940098a0 tx=0x7efc94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b646db3072a7f5ae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:05.474 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.598+0000 7efca1d9b700 1 -- 172.21.15.32:0/2773556622 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc94004160 con 0x7efca4107130 2023-09-30T14:31:05.474 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.598+0000 7efca1d9b700 1 -- 172.21.15.32:0/2773556622 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc940042c0 con 0x7efca4107130 2023-09-30T14:31:05.474 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.598+0000 7efca1d9b700 1 -- 172.21.15.32:0/2773556622 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc940119b0 con 0x7efca4107130 2023-09-30T14:31:05.474 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.599+0000 7efca9193700 1 -- 172.21.15.32:0/2773556622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4107130 msgr2=0x7efca4109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:05.474 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.599+0000 7efca9193700 1 --2- 172.21.15.32:0/2773556622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4107130 0x7efca4109520 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efc940098a0 tx=0x7efc94009bb0 comp rx=0 tx=0).stop 2023-09-30T14:31:05.475 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.599+0000 7efca9193700 1 -- 172.21.15.32:0/2773556622 shutdown_connections 2023-09-30T14:31:05.475 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.599+0000 7efca9193700 1 --2- 172.21.15.32:0/2773556622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4107130 0x7efca4109520 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:05.475 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.599+0000 7efca9193700 1 -- 172.21.15.32:0/2773556622 >> 172.21.15.32:0/2773556622 conn(0x7efca4100b80 msgr2=0x7efca4102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:05.475 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.599+0000 7efca9193700 1 -- 172.21.15.32:0/2773556622 shutdown_connections 2023-09-30T14:31:05.475 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.599+0000 7efca9193700 1 -- 172.21.15.32:0/2773556622 wait complete. 2023-09-30T14:31:05.475 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.599+0000 7efca9193700 1 Processor -- start 2023-09-30T14:31:05.475 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.600+0000 7efca9193700 1 -- start start 2023-09-30T14:31:05.475 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.600+0000 7efca9193700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4107130 0x7efca4106a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:05.475 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.600+0000 7efca9193700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca4106fd0 con 0x7efca4107130 2023-09-30T14:31:05.475 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.600+0000 7efca2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4107130 0x7efca4106a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:05.475 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.600+0000 7efca2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4107130 0x7efca4106a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60894/0 (socket says 172.21.15.32:60894) 2023-09-30T14:31:05.475 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.600+0000 7efca2d9d700 1 -- 172.21.15.32:0/3132848046 learned_addr learned my addr 172.21.15.32:0/3132848046 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:05.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.600+0000 7efca2d9d700 1 -- 172.21.15.32:0/3132848046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc94009550 con 0x7efca4107130 2023-09-30T14:31:05.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.600+0000 7efca2d9d700 1 --2- 172.21.15.32:0/3132848046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4107130 0x7efca4106a90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efc94003e60 tx=0x7efc94011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:05.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.601+0000 7efc8bfff700 1 -- 172.21.15.32:0/3132848046 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc94003970 con 0x7efca4107130 2023-09-30T14:31:05.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.601+0000 7efc8bfff700 1 -- 172.21.15.32:0/3132848046 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc94003ad0 con 0x7efca4107130 2023-09-30T14:31:05.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.601+0000 7efca9193700 1 -- 172.21.15.32:0/3132848046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca41050e0 con 0x7efca4107130 2023-09-30T14:31:05.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.601+0000 7efc8bfff700 1 -- 172.21.15.32:0/3132848046 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc940195e0 con 0x7efca4107130 2023-09-30T14:31:05.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.601+0000 7efca9193700 1 -- 172.21.15.32:0/3132848046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca4105580 con 0x7efca4107130 2023-09-30T14:31:05.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.602+0000 7efc8bfff700 1 -- 172.21.15.32:0/3132848046 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 45505+0+0 (secure 0 0 0) 0x7efc940198c0 con 0x7efca4107130 2023-09-30T14:31:05.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.602+0000 7efc8bfff700 1 --2- 172.21.15.32:0/3132848046 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7efc8c038680 0x7efc8c03ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:05.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.602+0000 7efc8bfff700 1 -- 172.21.15.32:0/3132848046 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efc9404c820 con 0x7efca4107130 2023-09-30T14:31:05.476 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.602+0000 7efca259c700 1 -- 172.21.15.32:0/3132848046 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7efc8c038680 msgr2=0x7efc8c03ab40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1639474065 2023-09-30T14:31:05.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.602+0000 7efca259c700 1 --2- 172.21.15.32:0/3132848046 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7efc8c038680 0x7efc8c03ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T14:31:05.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.602+0000 7efca9193700 1 -- 172.21.15.32:0/3132848046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca4062450 con 0x7efca4107130 2023-09-30T14:31:05.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.607+0000 7efc8bfff700 1 -- 172.21.15.32:0/3132848046 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7efc94048080 con 0x7efca4107130 2023-09-30T14:31:05.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.803+0000 7efca259c700 1 -- 172.21.15.32:0/3132848046 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7efc8c038680 msgr2=0x7efc8c03ab40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1639474065 2023-09-30T14:31:05.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:04.803+0000 7efca259c700 1 --2- 172.21.15.32:0/3132848046 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7efc8c038680 0x7efc8c03ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-30T14:31:05.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.079+0000 7efca9193700 1 -- 172.21.15.32:0/3132848046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7efca410a8a0 con 0x7efca4107130 2023-09-30T14:31:05.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.079+0000 7efc8bfff700 1 -- 172.21.15.32:0/3132848046 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7efc94019c10 con 0x7efca4107130 2023-09-30T14:31:05.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.081+0000 7efca9193700 1 -- 172.21.15.32:0/3132848046 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7efc8c038680 msgr2=0x7efc8c03ab40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T14:31:05.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.082+0000 7efca9193700 1 --2- 172.21.15.32:0/3132848046 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7efc8c038680 0x7efc8c03ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:05.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.082+0000 7efca9193700 1 -- 172.21.15.32:0/3132848046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4107130 msgr2=0x7efca4106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:05.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.082+0000 7efca9193700 1 --2- 172.21.15.32:0/3132848046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4107130 0x7efca4106a90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efc94003e60 tx=0x7efc94011aa0 comp rx=0 tx=0).stop 2023-09-30T14:31:05.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.082+0000 7efca9193700 1 -- 172.21.15.32:0/3132848046 shutdown_connections 2023-09-30T14:31:05.477 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.082+0000 7efca9193700 1 --2- 172.21.15.32:0/3132848046 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7efc8c038680 0x7efc8c03ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:05.478 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.082+0000 7efca9193700 1 --2- 172.21.15.32:0/3132848046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca4107130 0x7efca4106a90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:05.478 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.082+0000 7efca9193700 1 -- 172.21.15.32:0/3132848046 >> 172.21.15.32:0/3132848046 conn(0x7efca4100b80 msgr2=0x7efca4101860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:05.478 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.082+0000 7efca9193700 1 -- 172.21.15.32:0/3132848046 shutdown_connections 2023-09-30T14:31:05.478 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.082+0000 7efca9193700 1 -- 172.21.15.32:0/3132848046 wait complete. 2023-09-30T14:31:05.478 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for the mgr to restart... 2023-09-30T14:31:05.478 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr epoch 9... 2023-09-30T14:31:05.708 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:05 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3132848046' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-30T14:31:05.709 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:05 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:05.582+0000 7faa431c1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-30T14:31:06.637 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:06 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:06.373+0000 7faa431c1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-30T14:31:06.637 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:06 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:06.472+0000 7faa431c1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-30T14:31:07.787 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:07 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:07.451+0000 7faa431c1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-30T14:31:07.787 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:07 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:07.563+0000 7faa431c1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-30T14:31:08.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:07 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:07.786+0000 7faa431c1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-30T14:31:08.394 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:08 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:08.139+0000 7faa431c1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-30T14:31:08.887 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:08 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:08.393+0000 7faa431c1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-30T14:31:08.887 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:08 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:08.575+0000 7faa431c1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-30T14:31:09.387 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:08 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:08.909+0000 7faa431c1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-30T14:31:10.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:09 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:09.767+0000 7faa431c1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-30T14:31:10.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:09 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:09.929+0000 7faa431c1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-30T14:31:10.591 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:10 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:10.177+0000 7faa431c1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-30T14:31:10.591 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:10 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:10.315+0000 7faa431c1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-30T14:31:10.887 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:10 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:10.590+0000 7faa431c1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-30T14:31:12.137 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:31:11 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:31:11.678+0000 7faa431c1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-30T14:31:12.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:11 smithi032 ceph-mon[98592]: Active manager daemon x restarted 2023-09-30T14:31:12.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:11 smithi032 ceph-mon[98592]: Activating manager daemon x 2023-09-30T14:31:12.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:11 smithi032 ceph-mon[98592]: osdmap e3: 0 total, 0 up, 0 in 2023-09-30T14:31:12.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:11 smithi032 ceph-mon[98592]: mgrmap e10: x(active, starting, since 0.00359145s) 2023-09-30T14:31:12.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:31:12.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-30T14:31:12.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-30T14:31:12.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-30T14:31:12.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-30T14:31:12.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:11 smithi032 ceph-mon[98592]: Manager daemon x is now available 2023-09-30T14:31:12.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:31:12.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:31:13.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2023-09-30T14:31:13.136 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-09-30T14:31:13.136 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-30T14:31:13.136 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2023-09-30T14:31:13.136 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.945+0000 7fbee07b0700 1 Processor -- start 2023-09-30T14:31:13.136 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.945+0000 7fbee07b0700 1 -- start start 2023-09-30T14:31:13.136 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.946+0000 7fbee07b0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbed8107130 0x7fbed8109520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:13.136 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.946+0000 7fbee07b0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8074660 con 0x7fbed8107130 2023-09-30T14:31:13.137 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.946+0000 7fbede54c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbed8107130 0x7fbed8109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:13.137 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.946+0000 7fbede54c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbed8107130 0x7fbed8109520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60902/0 (socket says 172.21.15.32:60902) 2023-09-30T14:31:13.137 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.946+0000 7fbede54c700 1 -- 172.21.15.32:0/3852007084 learned_addr learned my addr 172.21.15.32:0/3852007084 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:13.137 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.946+0000 7fbede54c700 1 -- 172.21.15.32:0/3852007084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed8109a60 con 0x7fbed8107130 2023-09-30T14:31:13.137 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.946+0000 7fbede54c700 1 --2- 172.21.15.32:0/3852007084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbed8107130 0x7fbed8109520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbed40098a0 tx=0x7fbed4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31946d9148f9d6b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:13.137 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.947+0000 7fbedd54a700 1 -- 172.21.15.32:0/3852007084 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbed4004160 con 0x7fbed8107130 2023-09-30T14:31:13.137 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.947+0000 7fbedd54a700 1 -- 172.21.15.32:0/3852007084 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbed40042c0 con 0x7fbed8107130 2023-09-30T14:31:13.137 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.947+0000 7fbedd54a700 1 -- 172.21.15.32:0/3852007084 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbed40119b0 con 0x7fbed8107130 2023-09-30T14:31:13.137 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.947+0000 7fbee07b0700 1 -- 172.21.15.32:0/3852007084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbed8107130 msgr2=0x7fbed8109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:13.137 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.947+0000 7fbee07b0700 1 --2- 172.21.15.32:0/3852007084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbed8107130 0x7fbed8109520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbed40098a0 tx=0x7fbed4009bb0 comp rx=0 tx=0).stop 2023-09-30T14:31:13.138 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.947+0000 7fbee07b0700 1 -- 172.21.15.32:0/3852007084 shutdown_connections 2023-09-30T14:31:13.138 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.947+0000 7fbee07b0700 1 --2- 172.21.15.32:0/3852007084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbed8107130 0x7fbed8109520 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:13.138 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.947+0000 7fbee07b0700 1 -- 172.21.15.32:0/3852007084 >> 172.21.15.32:0/3852007084 conn(0x7fbed8100b80 msgr2=0x7fbed8102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:13.138 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.947+0000 7fbee07b0700 1 -- 172.21.15.32:0/3852007084 shutdown_connections 2023-09-30T14:31:13.138 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.948+0000 7fbee07b0700 1 -- 172.21.15.32:0/3852007084 wait complete. 2023-09-30T14:31:13.138 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.948+0000 7fbee07b0700 1 Processor -- start 2023-09-30T14:31:13.138 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.948+0000 7fbee07b0700 1 -- start start 2023-09-30T14:31:13.139 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.948+0000 7fbee07b0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbed8107130 0x7fbed819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:13.139 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.948+0000 7fbee07b0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed819a7c0 con 0x7fbed8107130 2023-09-30T14:31:13.139 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.948+0000 7fbede54c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbed8107130 0x7fbed819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:13.139 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.949+0000 7fbede54c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbed8107130 0x7fbed819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60904/0 (socket says 172.21.15.32:60904) 2023-09-30T14:31:13.139 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.949+0000 7fbede54c700 1 -- 172.21.15.32:0/1104092571 learned_addr learned my addr 172.21.15.32:0/1104092571 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:13.139 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.949+0000 7fbede54c700 1 -- 172.21.15.32:0/1104092571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed4009550 con 0x7fbed8107130 2023-09-30T14:31:13.139 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.949+0000 7fbede54c700 1 --2- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbed8107130 0x7fbed819a280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbed4011980 tx=0x7fbed4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:13.140 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.949+0000 7fbecf7fe700 1 -- 172.21.15.32:0/1104092571 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbed4003a60 con 0x7fbed8107130 2023-09-30T14:31:13.140 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.949+0000 7fbecf7fe700 1 -- 172.21.15.32:0/1104092571 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbed4003bc0 con 0x7fbed8107130 2023-09-30T14:31:13.140 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.949+0000 7fbee07b0700 1 -- 172.21.15.32:0/1104092571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed819a9c0 con 0x7fbed8107130 2023-09-30T14:31:13.140 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.949+0000 7fbecf7fe700 1 -- 172.21.15.32:0/1104092571 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbed401a670 con 0x7fbed8107130 2023-09-30T14:31:13.140 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.949+0000 7fbee07b0700 1 -- 172.21.15.32:0/1104092571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed819ae60 con 0x7fbed8107130 2023-09-30T14:31:13.140 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.950+0000 7fbecf7fe700 1 -- 172.21.15.32:0/1104092571 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 45505+0+0 (secure 0 0 0) 0x7fbed4024a00 con 0x7fbed8107130 2023-09-30T14:31:13.140 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.951+0000 7fbecf7fe700 1 --2- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbec40386d0 0x7fbec403ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:13.140 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.951+0000 7fbecf7fe700 1 -- 172.21.15.32:0/1104092571 --> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbec403b2a0 con 0x7fbec40386d0 2023-09-30T14:31:13.140 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.951+0000 7fbecf7fe700 1 -- 172.21.15.32:0/1104092571 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbed404c7f0 con 0x7fbed8107130 2023-09-30T14:31:13.141 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.951+0000 7fbeddd4b700 1 -- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbec40386d0 msgr2=0x7fbec403ab90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1639474065 2023-09-30T14:31:13.141 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:05.951+0000 7fbeddd4b700 1 --2- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbec40386d0 0x7fbec403ab90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T14:31:13.141 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:06.152+0000 7fbeddd4b700 1 -- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbec40386d0 msgr2=0x7fbec403ab90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1639474065 2023-09-30T14:31:13.141 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:06.152+0000 7fbeddd4b700 1 --2- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbec40386d0 0x7fbec403ab90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-30T14:31:13.141 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-30T14:31:13.141 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-30T14:31:13.141 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:12 smithi032 ceph-mon[98592]: mgrmap e11: x(active, since 1.0059s) 2023-09-30T14:31:13.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:06.553+0000 7fbeddd4b700 1 -- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbec40386d0 msgr2=0x7fbec403ab90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1639474065 2023-09-30T14:31:13.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:06.553+0000 7fbeddd4b700 1 --2- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbec40386d0 0x7fbec403ab90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-30T14:31:13.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:07.354+0000 7fbeddd4b700 1 -- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbec40386d0 msgr2=0x7fbec403ab90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1639474065 2023-09-30T14:31:13.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:07.354+0000 7fbeddd4b700 1 --2- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbec40386d0 0x7fbec403ab90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-30T14:31:13.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:08.955+0000 7fbeddd4b700 1 -- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbec40386d0 msgr2=0x7fbec403ab90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1639474065 2023-09-30T14:31:13.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:08.955+0000 7fbeddd4b700 1 --2- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbec40386d0 0x7fbec403ab90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-09-30T14:31:13.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:11.694+0000 7fbecf7fe700 1 -- 172.21.15.32:0/1104092571 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mgrmap(e 10) v1 ==== 45272+0+0 (secure 0 0 0) 0x7fbed4024620 con 0x7fbed8107130 2023-09-30T14:31:13.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:11.694+0000 7fbecf7fe700 1 -- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbec40386d0 msgr2=0x7fbec403ab90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T14:31:13.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:11.694+0000 7fbecf7fe700 1 --2- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbec40386d0 0x7fbec403ab90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:13.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.695+0000 7fbecf7fe700 1 -- 172.21.15.32:0/1104092571 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 11) v1 ==== 45399+0+0 (secure 0 0 0) 0x7fbed401cda0 con 0x7fbed8107130 2023-09-30T14:31:13.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.696+0000 7fbecf7fe700 1 --2- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbec403bff0 0x7fbec403e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:13.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.696+0000 7fbecf7fe700 1 -- 172.21.15.32:0/1104092571 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbec403b2a0 con 0x7fbec403bff0 2023-09-30T14:31:13.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.696+0000 7fbeddd4b700 1 --2- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbec403bff0 0x7fbec403e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:13.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.697+0000 7fbeddd4b700 1 --2- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbec403bff0 0x7fbec403e3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbec8003a10 tx=0x7fbec80092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:13.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.697+0000 7fbecf7fe700 1 -- 172.21.15.32:0/1104092571 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fbec403b2a0 con 0x7fbec403bff0 2023-09-30T14:31:13.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.708+0000 7fbee07b0700 1 -- 172.21.15.32:0/1104092571 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fbed819b5c0 con 0x7fbec403bff0 2023-09-30T14:31:13.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.708+0000 7fbecf7fe700 1 -- 172.21.15.32:0/1104092571 <== mgr.14152 v2:172.21.15.32:6800/3273633815 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fbed819b5c0 con 0x7fbec403bff0 2023-09-30T14:31:13.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.709+0000 7fbee07b0700 1 -- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbec403bff0 msgr2=0x7fbec403e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:13.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.709+0000 7fbee07b0700 1 --2- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbec403bff0 0x7fbec403e3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbec8003a10 tx=0x7fbec80092b0 comp rx=0 tx=0).stop 2023-09-30T14:31:13.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.709+0000 7fbee07b0700 1 -- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbed8107130 msgr2=0x7fbed819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:13.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.709+0000 7fbee07b0700 1 --2- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbed8107130 0x7fbed819a280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbed4011980 tx=0x7fbed4011a60 comp rx=0 tx=0).stop 2023-09-30T14:31:13.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.709+0000 7fbee07b0700 1 -- 172.21.15.32:0/1104092571 shutdown_connections 2023-09-30T14:31:13.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.709+0000 7fbee07b0700 1 --2- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbed8107130 0x7fbed819a280 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:13.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.709+0000 7fbee07b0700 1 --2- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbec403bff0 0x7fbec403e3e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:13.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.709+0000 7fbee07b0700 1 --2- 172.21.15.32:0/1104092571 >> [v2:172.21.15.32:6800/1639474065,v1:172.21.15.32:6801/1639474065] conn(0x7fbec40386d0 0x7fbec403ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:13.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.709+0000 7fbee07b0700 1 -- 172.21.15.32:0/1104092571 >> 172.21.15.32:0/1104092571 conn(0x7fbed8100b80 msgr2=0x7fbed8101860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:13.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.709+0000 7fbee07b0700 1 -- 172.21.15.32:0/1104092571 shutdown_connections 2023-09-30T14:31:13.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:12.709+0000 7fbee07b0700 1 -- 172.21.15.32:0/1104092571 wait complete. 2023-09-30T14:31:13.146 INFO:teuthology.orchestra.run.smithi032.stdout:mgr epoch 9 is available 2023-09-30T14:31:13.146 INFO:teuthology.orchestra.run.smithi032.stdout:Generating a dashboard self-signed certificate... 2023-09-30T14:31:14.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:13 smithi032 ceph-mon[98592]: [30/Sep/2023:14:31:12] ENGINE Bus STARTING 2023-09-30T14:31:14.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:13 smithi032 ceph-mon[98592]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-30T14:31:14.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:13 smithi032 ceph-mon[98592]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-30T14:31:14.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:13 smithi032 ceph-mon[98592]: [30/Sep/2023:14:31:12] ENGINE Serving on http://172.21.15.32:8765 2023-09-30T14:31:14.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:13 smithi032 ceph-mon[98592]: [30/Sep/2023:14:31:12] ENGINE Serving on https://172.21.15.32:7150 2023-09-30T14:31:14.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:13 smithi032 ceph-mon[98592]: [30/Sep/2023:14:31:12] ENGINE Bus STARTED 2023-09-30T14:31:14.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:14.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-09-30T14:31:14.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.794+0000 7fccd89d6700 1 Processor -- start 2023-09-30T14:31:14.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.794+0000 7fccd89d6700 1 -- start start 2023-09-30T14:31:14.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.794+0000 7fccd89d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd4107170 0x7fccd4109560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:14.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.794+0000 7fccd89d6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd4074620 con 0x7fccd4107170 2023-09-30T14:31:14.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.795+0000 7fccd259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd4107170 0x7fccd4109560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:14.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.795+0000 7fccd259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd4107170 0x7fccd4109560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60958/0 (socket says 172.21.15.32:60958) 2023-09-30T14:31:14.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.795+0000 7fccd259c700 1 -- 172.21.15.32:0/920532952 learned_addr learned my addr 172.21.15.32:0/920532952 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:14.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.795+0000 7fccd259c700 1 -- 172.21.15.32:0/920532952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd4109aa0 con 0x7fccd4107170 2023-09-30T14:31:14.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.795+0000 7fccd259c700 1 --2- 172.21.15.32:0/920532952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd4107170 0x7fccd4109560 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fccc40098a0 tx=0x7fccc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8ef917b4cba8e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:14.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.795+0000 7fccd159a700 1 -- 172.21.15.32:0/920532952 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccc4004160 con 0x7fccd4107170 2023-09-30T14:31:14.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.795+0000 7fccd159a700 1 -- 172.21.15.32:0/920532952 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc40042c0 con 0x7fccd4107170 2023-09-30T14:31:14.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.795+0000 7fccd159a700 1 -- 172.21.15.32:0/920532952 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccc40119b0 con 0x7fccd4107170 2023-09-30T14:31:14.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.796+0000 7fccd89d6700 1 -- 172.21.15.32:0/920532952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd4107170 msgr2=0x7fccd4109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:14.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.796+0000 7fccd89d6700 1 --2- 172.21.15.32:0/920532952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd4107170 0x7fccd4109560 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fccc40098a0 tx=0x7fccc4009bb0 comp rx=0 tx=0).stop 2023-09-30T14:31:14.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.796+0000 7fccd89d6700 1 -- 172.21.15.32:0/920532952 shutdown_connections 2023-09-30T14:31:14.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.796+0000 7fccd89d6700 1 --2- 172.21.15.32:0/920532952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd4107170 0x7fccd4109560 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:14.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.796+0000 7fccd89d6700 1 -- 172.21.15.32:0/920532952 >> 172.21.15.32:0/920532952 conn(0x7fccd4100b80 msgr2=0x7fccd4102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:14.840 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.796+0000 7fccd89d6700 1 -- 172.21.15.32:0/920532952 shutdown_connections 2023-09-30T14:31:14.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.796+0000 7fccd89d6700 1 -- 172.21.15.32:0/920532952 wait complete. 2023-09-30T14:31:14.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.797+0000 7fccd89d6700 1 Processor -- start 2023-09-30T14:31:14.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.797+0000 7fccd89d6700 1 -- start start 2023-09-30T14:31:14.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.797+0000 7fccd89d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd4107170 0x7fccd4106ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:14.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.797+0000 7fccd89d6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd4107010 con 0x7fccd4107170 2023-09-30T14:31:14.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.797+0000 7fccd259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd4107170 0x7fccd4106ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:14.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.797+0000 7fccd259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd4107170 0x7fccd4106ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60960/0 (socket says 172.21.15.32:60960) 2023-09-30T14:31:14.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.797+0000 7fccd259c700 1 -- 172.21.15.32:0/1542697499 learned_addr learned my addr 172.21.15.32:0/1542697499 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:14.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.798+0000 7fccd259c700 1 -- 172.21.15.32:0/1542697499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc4009550 con 0x7fccd4107170 2023-09-30T14:31:14.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.798+0000 7fccd259c700 1 --2- 172.21.15.32:0/1542697499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd4107170 0x7fccd4106ad0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fccc4003e60 tx=0x7fccc4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:14.841 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.798+0000 7fccc37fe700 1 -- 172.21.15.32:0/1542697499 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccc4003970 con 0x7fccd4107170 2023-09-30T14:31:14.842 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.798+0000 7fccc37fe700 1 -- 172.21.15.32:0/1542697499 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc4003ad0 con 0x7fccd4107170 2023-09-30T14:31:14.842 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.798+0000 7fccc37fe700 1 -- 172.21.15.32:0/1542697499 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccc40195e0 con 0x7fccd4107170 2023-09-30T14:31:14.842 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.798+0000 7fccd89d6700 1 -- 172.21.15.32:0/1542697499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd4105120 con 0x7fccd4107170 2023-09-30T14:31:14.842 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.798+0000 7fccd89d6700 1 -- 172.21.15.32:0/1542697499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd41055c0 con 0x7fccd4107170 2023-09-30T14:31:14.842 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.799+0000 7fccc37fe700 1 -- 172.21.15.32:0/1542697499 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 11) v1 ==== 45399+0+0 (secure 0 0 0) 0x7fccc4019850 con 0x7fccd4107170 2023-09-30T14:31:14.842 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.799+0000 7fccc37fe700 1 --2- 172.21.15.32:0/1542697499 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fccbc0385b0 0x7fccbc03aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:14.842 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.799+0000 7fccc37fe700 1 -- 172.21.15.32:0/1542697499 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fccc404c700 con 0x7fccd4107170 2023-09-30T14:31:14.842 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.799+0000 7fccd89d6700 1 -- 172.21.15.32:0/1542697499 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd410a880 con 0x7fccd4107170 2023-09-30T14:31:14.842 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.799+0000 7fccd1d9b700 1 --2- 172.21.15.32:0/1542697499 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fccbc0385b0 0x7fccbc03aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:14.842 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.800+0000 7fccd1d9b700 1 --2- 172.21.15.32:0/1542697499 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fccbc0385b0 0x7fccbc03aa70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fccc8006fd0 tx=0x7fccc8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:14.842 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:13.805+0000 7fccc37fe700 1 -- 172.21.15.32:0/1542697499 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fccc402b340 con 0x7fccd4107170 2023-09-30T14:31:14.842 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:14.181+0000 7fccd89d6700 1 -- 172.21.15.32:0/1542697499 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fccd4105d70 con 0x7fccbc0385b0 2023-09-30T14:31:14.843 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:14.231+0000 7fccc37fe700 1 -- 172.21.15.32:0/1542697499 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fccd4105d70 con 0x7fccbc0385b0 2023-09-30T14:31:14.843 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:14.233+0000 7fccd89d6700 1 -- 172.21.15.32:0/1542697499 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fccbc0385b0 msgr2=0x7fccbc03aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:14.843 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:14.233+0000 7fccd89d6700 1 --2- 172.21.15.32:0/1542697499 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fccbc0385b0 0x7fccbc03aa70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fccc8006fd0 tx=0x7fccc8006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:14.843 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:14.233+0000 7fccd89d6700 1 -- 172.21.15.32:0/1542697499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd4107170 msgr2=0x7fccd4106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:14.843 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:14.233+0000 7fccd89d6700 1 --2- 172.21.15.32:0/1542697499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd4107170 0x7fccd4106ad0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fccc4003e60 tx=0x7fccc4011aa0 comp rx=0 tx=0).stop 2023-09-30T14:31:14.843 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:14.233+0000 7fccd89d6700 1 -- 172.21.15.32:0/1542697499 shutdown_connections 2023-09-30T14:31:14.843 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:14.233+0000 7fccd89d6700 1 --2- 172.21.15.32:0/1542697499 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fccbc0385b0 0x7fccbc03aa70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:14.843 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:14.233+0000 7fccd89d6700 1 --2- 172.21.15.32:0/1542697499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccd4107170 0x7fccd4106ad0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:14.843 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:14.233+0000 7fccd89d6700 1 -- 172.21.15.32:0/1542697499 >> 172.21.15.32:0/1542697499 conn(0x7fccd4100b80 msgr2=0x7fccd4101850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:14.843 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:14.234+0000 7fccd89d6700 1 -- 172.21.15.32:0/1542697499 shutdown_connections 2023-09-30T14:31:14.843 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:14.234+0000 7fccd89d6700 1 -- 172.21.15.32:0/1542697499 wait complete. 2023-09-30T14:31:14.843 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial admin user... 2023-09-30T14:31:15.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:15 smithi032 ceph-mon[98592]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:31:15.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:15.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:15.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:15.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:15 smithi032 ceph-mon[98592]: mgrmap e12: x(active, since 2s) 2023-09-30T14:31:16.485 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$7AwppLvzliWL0cmYjMVZbeurD6H83MVubWgU0vm10vMi266EGhjhS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1696084276, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-09-30T14:31:16.486 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.399+0000 7f6faaae6700 1 Processor -- start 2023-09-30T14:31:16.486 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.399+0000 7f6faaae6700 1 -- start start 2023-09-30T14:31:16.486 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.399+0000 7f6faaae6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa4107130 0x7f6fa4109520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:16.486 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.399+0000 7f6faaae6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4079e90 con 0x7f6fa4107130 2023-09-30T14:31:16.486 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.400+0000 7f6fa8882700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa4107130 0x7f6fa4109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:16.486 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.400+0000 7f6fa8882700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa4107130 0x7f6fa4109520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60964/0 (socket says 172.21.15.32:60964) 2023-09-30T14:31:16.486 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.400+0000 7f6fa8882700 1 -- 172.21.15.32:0/1469062357 learned_addr learned my addr 172.21.15.32:0/1469062357 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:16.486 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.400+0000 7f6fa8882700 1 -- 172.21.15.32:0/1469062357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa4109a60 con 0x7f6fa4107130 2023-09-30T14:31:16.486 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.400+0000 7f6fa8882700 1 --2- 172.21.15.32:0/1469062357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa4107130 0x7f6fa4109520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f980098a0 tx=0x7f6f98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18036e63b27b6f31 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:16.486 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.400+0000 7f6fa37fe700 1 -- 172.21.15.32:0/1469062357 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f98004160 con 0x7f6fa4107130 2023-09-30T14:31:16.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.400+0000 7f6fa37fe700 1 -- 172.21.15.32:0/1469062357 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f980042c0 con 0x7f6fa4107130 2023-09-30T14:31:16.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.400+0000 7f6fa37fe700 1 -- 172.21.15.32:0/1469062357 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f980119b0 con 0x7f6fa4107130 2023-09-30T14:31:16.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.401+0000 7f6faaae6700 1 -- 172.21.15.32:0/1469062357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa4107130 msgr2=0x7f6fa4109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:16.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.401+0000 7f6faaae6700 1 --2- 172.21.15.32:0/1469062357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa4107130 0x7f6fa4109520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f980098a0 tx=0x7f6f98009bb0 comp rx=0 tx=0).stop 2023-09-30T14:31:16.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.401+0000 7f6faaae6700 1 -- 172.21.15.32:0/1469062357 shutdown_connections 2023-09-30T14:31:16.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.401+0000 7f6faaae6700 1 --2- 172.21.15.32:0/1469062357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa4107130 0x7f6fa4109520 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:16.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.401+0000 7f6faaae6700 1 -- 172.21.15.32:0/1469062357 >> 172.21.15.32:0/1469062357 conn(0x7f6fa4100b00 msgr2=0x7f6fa4102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:16.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.401+0000 7f6faaae6700 1 -- 172.21.15.32:0/1469062357 shutdown_connections 2023-09-30T14:31:16.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.401+0000 7f6faaae6700 1 -- 172.21.15.32:0/1469062357 wait complete. 2023-09-30T14:31:16.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.402+0000 7f6faaae6700 1 Processor -- start 2023-09-30T14:31:16.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.402+0000 7f6faaae6700 1 -- start start 2023-09-30T14:31:16.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.402+0000 7f6faaae6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa4107130 0x7f6fa4106a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:16.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.402+0000 7f6faaae6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4106fd0 con 0x7f6fa4107130 2023-09-30T14:31:16.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.402+0000 7f6fa8882700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa4107130 0x7f6fa4106a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:16.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.402+0000 7f6fa8882700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa4107130 0x7f6fa4106a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60966/0 (socket says 172.21.15.32:60966) 2023-09-30T14:31:16.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.402+0000 7f6fa8882700 1 -- 172.21.15.32:0/2687575992 learned_addr learned my addr 172.21.15.32:0/2687575992 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:16.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.402+0000 7f6fa8882700 1 -- 172.21.15.32:0/2687575992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f98009550 con 0x7f6fa4107130 2023-09-30T14:31:16.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.402+0000 7f6fa8882700 1 --2- 172.21.15.32:0/2687575992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa4107130 0x7f6fa4106a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6f98003e60 tx=0x7f6f98011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:16.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.403+0000 7f6fa1ffb700 1 -- 172.21.15.32:0/2687575992 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f98003970 con 0x7f6fa4107130 2023-09-30T14:31:16.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.403+0000 7f6fa1ffb700 1 -- 172.21.15.32:0/2687575992 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f98003ad0 con 0x7f6fa4107130 2023-09-30T14:31:16.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.403+0000 7f6faaae6700 1 -- 172.21.15.32:0/2687575992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa41050e0 con 0x7f6fa4107130 2023-09-30T14:31:16.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.403+0000 7f6fa1ffb700 1 -- 172.21.15.32:0/2687575992 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f980195e0 con 0x7f6fa4107130 2023-09-30T14:31:16.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.403+0000 7f6faaae6700 1 -- 172.21.15.32:0/2687575992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa4105580 con 0x7f6fa4107130 2023-09-30T14:31:16.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.404+0000 7f6fa1ffb700 1 -- 172.21.15.32:0/2687575992 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 12) v1 ==== 45505+0+0 (secure 0 0 0) 0x7f6f98019850 con 0x7f6fa4107130 2023-09-30T14:31:16.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.404+0000 7f6fa1ffb700 1 --2- 172.21.15.32:0/2687575992 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6f940386d0 0x7f6f9403ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:16.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.404+0000 7f6fa1ffb700 1 -- 172.21.15.32:0/2687575992 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6f9804c9b0 con 0x7f6fa4107130 2023-09-30T14:31:16.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.404+0000 7f6fa3fff700 1 --2- 172.21.15.32:0/2687575992 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6f940386d0 0x7f6f9403ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:16.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.404+0000 7f6faaae6700 1 -- 172.21.15.32:0/2687575992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa410a840 con 0x7f6fa4107130 2023-09-30T14:31:16.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.405+0000 7f6fa3fff700 1 --2- 172.21.15.32:0/2687575992 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6f940386d0 0x7f6f9403ab90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f90006fd0 tx=0x7f6f90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:16.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.409+0000 7f6fa1ffb700 1 -- 172.21.15.32:0/2687575992 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f6f98019ba0 con 0x7f6fa4107130 2023-09-30T14:31:16.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:15.809+0000 7f6faaae6700 1 -- 172.21.15.32:0/2687575992 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6fa4062270 con 0x7f6f940386d0 2023-09-30T14:31:16.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.057+0000 7f6fa1ffb700 1 -- 172.21.15.32:0/2687575992 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f6fa4062270 con 0x7f6f940386d0 2023-09-30T14:31:16.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.059+0000 7f6faaae6700 1 -- 172.21.15.32:0/2687575992 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6f940386d0 msgr2=0x7f6f9403ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:16.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.059+0000 7f6faaae6700 1 --2- 172.21.15.32:0/2687575992 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6f940386d0 0x7f6f9403ab90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f90006fd0 tx=0x7f6f90006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:16.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.059+0000 7f6faaae6700 1 -- 172.21.15.32:0/2687575992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa4107130 msgr2=0x7f6fa4106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:16.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.059+0000 7f6faaae6700 1 --2- 172.21.15.32:0/2687575992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa4107130 0x7f6fa4106a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6f98003e60 tx=0x7f6f98011aa0 comp rx=0 tx=0).stop 2023-09-30T14:31:16.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.059+0000 7f6faaae6700 1 -- 172.21.15.32:0/2687575992 shutdown_connections 2023-09-30T14:31:16.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.059+0000 7f6faaae6700 1 --2- 172.21.15.32:0/2687575992 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6f940386d0 0x7f6f9403ab90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:16.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.059+0000 7f6faaae6700 1 --2- 172.21.15.32:0/2687575992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa4107130 0x7f6fa4106a90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:16.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.059+0000 7f6faaae6700 1 -- 172.21.15.32:0/2687575992 >> 172.21.15.32:0/2687575992 conn(0x7f6fa4100b00 msgr2=0x7f6fa4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:16.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.059+0000 7f6faaae6700 1 -- 172.21.15.32:0/2687575992 shutdown_connections 2023-09-30T14:31:16.490 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.059+0000 7f6faaae6700 1 -- 172.21.15.32:0/2687575992 wait complete. 2023-09-30T14:31:16.490 INFO:teuthology.orchestra.run.smithi032.stdout:Fetching dashboard port number... 2023-09-30T14:31:17.372 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:17 smithi032 ceph-mon[98592]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:31:17.372 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:17.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 8443 2023-09-30T14:31:17.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.970+0000 7f5169c40700 1 Processor -- start 2023-09-30T14:31:17.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.970+0000 7f5169c40700 1 -- start start 2023-09-30T14:31:17.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.970+0000 7f5169c40700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51641066c0 0x7f5164106ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:17.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.970+0000 7f5169c40700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51641070b0 con 0x7f51641066c0 2023-09-30T14:31:17.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.970+0000 7f51637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51641066c0 0x7f5164106ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:17.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.970+0000 7f51637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51641066c0 0x7f5164106ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60970/0 (socket says 172.21.15.32:60970) 2023-09-30T14:31:17.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.970+0000 7f51637fe700 1 -- 172.21.15.32:0/789798156 learned_addr learned my addr 172.21.15.32:0/789798156 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:17.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.970+0000 7f51637fe700 1 -- 172.21.15.32:0/789798156 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51641078c0 con 0x7f51641066c0 2023-09-30T14:31:17.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.971+0000 7f51637fe700 1 --2- 172.21.15.32:0/789798156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51641066c0 0x7f5164106ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f51540098a0 tx=0x7f5154009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a65235e626b7c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:17.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.971+0000 7f51627fc700 1 -- 172.21.15.32:0/789798156 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5154004160 con 0x7f51641066c0 2023-09-30T14:31:17.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.971+0000 7f51627fc700 1 -- 172.21.15.32:0/789798156 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51540042c0 con 0x7f51641066c0 2023-09-30T14:31:17.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.971+0000 7f51627fc700 1 -- 172.21.15.32:0/789798156 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51540119b0 con 0x7f51641066c0 2023-09-30T14:31:17.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.971+0000 7f5169c40700 1 -- 172.21.15.32:0/789798156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51641066c0 msgr2=0x7f5164106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:17.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.971+0000 7f5169c40700 1 --2- 172.21.15.32:0/789798156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51641066c0 0x7f5164106ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f51540098a0 tx=0x7f5154009bb0 comp rx=0 tx=0).stop 2023-09-30T14:31:17.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.972+0000 7f5169c40700 1 -- 172.21.15.32:0/789798156 shutdown_connections 2023-09-30T14:31:17.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.972+0000 7f5169c40700 1 --2- 172.21.15.32:0/789798156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51641066c0 0x7f5164106ae0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:17.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.972+0000 7f5169c40700 1 -- 172.21.15.32:0/789798156 >> 172.21.15.32:0/789798156 conn(0x7f5164101c80 msgr2=0x7f51641040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:17.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.972+0000 7f5169c40700 1 -- 172.21.15.32:0/789798156 shutdown_connections 2023-09-30T14:31:17.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.972+0000 7f5169c40700 1 -- 172.21.15.32:0/789798156 wait complete. 2023-09-30T14:31:17.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.972+0000 7f5169c40700 1 Processor -- start 2023-09-30T14:31:17.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.972+0000 7f5169c40700 1 -- start start 2023-09-30T14:31:17.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.973+0000 7f5169c40700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51641066c0 0x7f516407ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:17.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.973+0000 7f5169c40700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516407b0a0 con 0x7f51641066c0 2023-09-30T14:31:17.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.973+0000 7f51637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51641066c0 0x7f516407ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:17.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.973+0000 7f51637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51641066c0 0x7f516407ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60972/0 (socket says 172.21.15.32:60972) 2023-09-30T14:31:17.837 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.973+0000 7f51637fe700 1 -- 172.21.15.32:0/3197049197 learned_addr learned my addr 172.21.15.32:0/3197049197 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:17.837 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.973+0000 7f51637fe700 1 -- 172.21.15.32:0/3197049197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5154009550 con 0x7f51641066c0 2023-09-30T14:31:17.837 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.973+0000 7f51637fe700 1 --2- 172.21.15.32:0/3197049197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51641066c0 0x7f516407ab60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5154000c00 tx=0x7f5154011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:17.837 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.973+0000 7f5160ff9700 1 -- 172.21.15.32:0/3197049197 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51540039f0 con 0x7f51641066c0 2023-09-30T14:31:17.837 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.973+0000 7f5169c40700 1 -- 172.21.15.32:0/3197049197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51640791b0 con 0x7f51641066c0 2023-09-30T14:31:17.837 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.973+0000 7f5160ff9700 1 -- 172.21.15.32:0/3197049197 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5154003b50 con 0x7f51641066c0 2023-09-30T14:31:17.837 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.974+0000 7f5160ff9700 1 -- 172.21.15.32:0/3197049197 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f515401a780 con 0x7f51641066c0 2023-09-30T14:31:17.837 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.974+0000 7f5169c40700 1 -- 172.21.15.32:0/3197049197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5164079650 con 0x7f51641066c0 2023-09-30T14:31:17.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.974+0000 7f5160ff9700 1 -- 172.21.15.32:0/3197049197 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 12) v1 ==== 45505+0+0 (secure 0 0 0) 0x7f515401aa40 con 0x7f51641066c0 2023-09-30T14:31:17.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.975+0000 7f5160ff9700 1 --2- 172.21.15.32:0/3197049197 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f514c0386d0 0x7f514c03ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:17.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.975+0000 7f5169c40700 1 -- 172.21.15.32:0/3197049197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516410aaf0 con 0x7f51641066c0 2023-09-30T14:31:17.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.975+0000 7f5160ff9700 1 -- 172.21.15.32:0/3197049197 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f515404c7f0 con 0x7f51641066c0 2023-09-30T14:31:17.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.975+0000 7f5162ffd700 1 --2- 172.21.15.32:0/3197049197 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f514c0386d0 0x7f514c03ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:17.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.975+0000 7f5162ffd700 1 --2- 172.21.15.32:0/3197049197 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f514c0386d0 0x7f514c03ab90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5158006fd0 tx=0x7f5158006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:17.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:16.984+0000 7f5160ff9700 1 -- 172.21.15.32:0/3197049197 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f515401ad40 con 0x7f51641066c0 2023-09-30T14:31:17.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:17.347+0000 7f5169c40700 1 -- 172.21.15.32:0/3197049197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f516410ad00 con 0x7f51641066c0 2023-09-30T14:31:17.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:17.347+0000 7f5160ff9700 1 -- 172.21.15.32:0/3197049197 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f51540172b0 con 0x7f51641066c0 2023-09-30T14:31:17.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:17.350+0000 7f5169c40700 1 -- 172.21.15.32:0/3197049197 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f514c0386d0 msgr2=0x7f514c03ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:17.838 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:17.350+0000 7f5169c40700 1 --2- 172.21.15.32:0/3197049197 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f514c0386d0 0x7f514c03ab90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5158006fd0 tx=0x7f5158006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:17.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:17.350+0000 7f5169c40700 1 -- 172.21.15.32:0/3197049197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51641066c0 msgr2=0x7f516407ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:17.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:17.350+0000 7f5169c40700 1 --2- 172.21.15.32:0/3197049197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51641066c0 0x7f516407ab60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5154000c00 tx=0x7f5154011a30 comp rx=0 tx=0).stop 2023-09-30T14:31:17.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:17.350+0000 7f5169c40700 1 -- 172.21.15.32:0/3197049197 shutdown_connections 2023-09-30T14:31:17.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:17.350+0000 7f5169c40700 1 --2- 172.21.15.32:0/3197049197 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f514c0386d0 0x7f514c03ab90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:17.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:17.350+0000 7f5169c40700 1 --2- 172.21.15.32:0/3197049197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51641066c0 0x7f516407ab60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:17.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:17.350+0000 7f5169c40700 1 -- 172.21.15.32:0/3197049197 >> 172.21.15.32:0/3197049197 conn(0x7f5164101c80 msgr2=0x7f5164102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:17.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:17.350+0000 7f5169c40700 1 -- 172.21.15.32:0/3197049197 shutdown_connections 2023-09-30T14:31:17.839 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:17.350+0000 7f5169c40700 1 -- 172.21.15.32:0/3197049197 wait complete. 2023-09-30T14:31:17.841 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-30T14:31:17.841 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2023-09-30T14:31:17.848 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-30T14:31:17.848 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2023-09-30T14:31:17.848 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2023-09-30T14:31:17.848 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-09-30T14:31:17.850 INFO:teuthology.orchestra.run.smithi032.stdout:Ceph Dashboard is now available at: 2023-09-30T14:31:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:31:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: URL: https://smithi032:8443/ 2023-09-30T14:31:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: User: admin 2023-09-30T14:31:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: Password: rk64b92bf3 2023-09-30T14:31:17.850 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:31:17.851 INFO:teuthology.orchestra.run.smithi032.stdout:Saving cluster configuration to /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config directory 2023-09-30T14:31:17.851 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling autotune for osd_memory_target 2023-09-30T14:31:18.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:18 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3197049197' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-09-30T14:31:19.097 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.354+0000 7f113865d700 1 Processor -- start 2023-09-30T14:31:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.354+0000 7f113865d700 1 -- start start 2023-09-30T14:31:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.355+0000 7f113865d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f113007ad90 0x7f11300791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.355+0000 7f113865d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11300797c0 con 0x7f113007ad90 2023-09-30T14:31:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.355+0000 7f11363f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f113007ad90 0x7f11300791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.355+0000 7f11363f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f113007ad90 0x7f11300791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60980/0 (socket says 172.21.15.32:60980) 2023-09-30T14:31:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.355+0000 7f11363f9700 1 -- 172.21.15.32:0/1010138735 learned_addr learned my addr 172.21.15.32:0/1010138735 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.355+0000 7f11363f9700 1 -- 172.21.15.32:0/1010138735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1130079900 con 0x7f113007ad90 2023-09-30T14:31:19.098 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.355+0000 7f11363f9700 1 --2- 172.21.15.32:0/1010138735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f113007ad90 0x7f11300791f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f11200098a0 tx=0x7f1120009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3c09c4025de31c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:19.099 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.355+0000 7f11353f7700 1 -- 172.21.15.32:0/1010138735 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1120004160 con 0x7f113007ad90 2023-09-30T14:31:19.099 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.356+0000 7f11353f7700 1 -- 172.21.15.32:0/1010138735 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11200042c0 con 0x7f113007ad90 2023-09-30T14:31:19.099 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.356+0000 7f11353f7700 1 -- 172.21.15.32:0/1010138735 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11200119b0 con 0x7f113007ad90 2023-09-30T14:31:19.099 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.356+0000 7f113865d700 1 -- 172.21.15.32:0/1010138735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f113007ad90 msgr2=0x7f11300791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:19.099 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.356+0000 7f113865d700 1 --2- 172.21.15.32:0/1010138735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f113007ad90 0x7f11300791f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f11200098a0 tx=0x7f1120009bb0 comp rx=0 tx=0).stop 2023-09-30T14:31:19.099 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.356+0000 7f113865d700 1 -- 172.21.15.32:0/1010138735 shutdown_connections 2023-09-30T14:31:19.099 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.356+0000 7f113865d700 1 --2- 172.21.15.32:0/1010138735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f113007ad90 0x7f11300791f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:19.099 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.356+0000 7f113865d700 1 -- 172.21.15.32:0/1010138735 >> 172.21.15.32:0/1010138735 conn(0x7f1130101c30 msgr2=0x7f1130104070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:19.099 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.356+0000 7f113865d700 1 -- 172.21.15.32:0/1010138735 shutdown_connections 2023-09-30T14:31:19.099 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.356+0000 7f113865d700 1 -- 172.21.15.32:0/1010138735 wait complete. 2023-09-30T14:31:19.099 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.357+0000 7f113865d700 1 Processor -- start 2023-09-30T14:31:19.100 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.357+0000 7f113865d700 1 -- start start 2023-09-30T14:31:19.100 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.357+0000 7f113865d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f113007ad90 0x7f113019c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:19.100 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.357+0000 7f113865d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113019c9e0 con 0x7f113007ad90 2023-09-30T14:31:19.100 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.358+0000 7f11363f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f113007ad90 0x7f113019c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:19.100 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.358+0000 7f11363f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f113007ad90 0x7f113019c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60982/0 (socket says 172.21.15.32:60982) 2023-09-30T14:31:19.100 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.358+0000 7f11363f9700 1 -- 172.21.15.32:0/1118646542 learned_addr learned my addr 172.21.15.32:0/1118646542 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:19.100 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.358+0000 7f11363f9700 1 -- 172.21.15.32:0/1118646542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1120009550 con 0x7f113007ad90 2023-09-30T14:31:19.100 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.358+0000 7f11363f9700 1 --2- 172.21.15.32:0/1118646542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f113007ad90 0x7f113019c4a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1120003e60 tx=0x7f1120011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:19.100 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.358+0000 7f11277fe700 1 -- 172.21.15.32:0/1118646542 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1120003970 con 0x7f113007ad90 2023-09-30T14:31:19.100 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.358+0000 7f11277fe700 1 -- 172.21.15.32:0/1118646542 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1120003ad0 con 0x7f113007ad90 2023-09-30T14:31:19.100 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.358+0000 7f113865d700 1 -- 172.21.15.32:0/1118646542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f113019cbe0 con 0x7f113007ad90 2023-09-30T14:31:19.101 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.358+0000 7f11277fe700 1 -- 172.21.15.32:0/1118646542 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11200195e0 con 0x7f113007ad90 2023-09-30T14:31:19.101 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.358+0000 7f113865d700 1 -- 172.21.15.32:0/1118646542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113019d080 con 0x7f113007ad90 2023-09-30T14:31:19.101 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.359+0000 7f11277fe700 1 -- 172.21.15.32:0/1118646542 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f1120019850 con 0x7f113007ad90 2023-09-30T14:31:19.101 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.359+0000 7f11277fe700 1 --2- 172.21.15.32:0/1118646542 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f111c038770 0x7f111c03ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:19.101 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.359+0000 7f11277fe700 1 -- 172.21.15.32:0/1118646542 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f112004cac0 con 0x7f113007ad90 2023-09-30T14:31:19.101 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.359+0000 7f1135bf8700 1 --2- 172.21.15.32:0/1118646542 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f111c038770 0x7f111c03ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:19.101 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.359+0000 7f113865d700 1 -- 172.21.15.32:0/1118646542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1130062450 con 0x7f113007ad90 2023-09-30T14:31:19.101 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.360+0000 7f1135bf8700 1 --2- 172.21.15.32:0/1118646542 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f111c038770 0x7f111c03ac30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f112c006fd0 tx=0x7f112c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:19.101 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.364+0000 7f11277fe700 1 -- 172.21.15.32:0/1118646542 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f1120019ba0 con 0x7f113007ad90 2023-09-30T14:31:19.101 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.726+0000 7f113865d700 1 -- 172.21.15.32:0/1118646542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f113010cbc0 con 0x7f113007ad90 2023-09-30T14:31:19.101 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.727+0000 7f11277fe700 1 -- 172.21.15.32:0/1118646542 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f1120018b40 con 0x7f113007ad90 2023-09-30T14:31:19.101 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.729+0000 7f113865d700 1 -- 172.21.15.32:0/1118646542 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f111c038770 msgr2=0x7f111c03ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:19.102 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.729+0000 7f113865d700 1 --2- 172.21.15.32:0/1118646542 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f111c038770 0x7f111c03ac30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f112c006fd0 tx=0x7f112c006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:19.102 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.729+0000 7f113865d700 1 -- 172.21.15.32:0/1118646542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f113007ad90 msgr2=0x7f113019c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:19.102 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.729+0000 7f113865d700 1 --2- 172.21.15.32:0/1118646542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f113007ad90 0x7f113019c4a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1120003e60 tx=0x7f1120011aa0 comp rx=0 tx=0).stop 2023-09-30T14:31:19.102 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.729+0000 7f113865d700 1 -- 172.21.15.32:0/1118646542 shutdown_connections 2023-09-30T14:31:19.102 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.729+0000 7f113865d700 1 --2- 172.21.15.32:0/1118646542 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f111c038770 0x7f111c03ac30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:19.102 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.729+0000 7f113865d700 1 --2- 172.21.15.32:0/1118646542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f113007ad90 0x7f113019c4a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:19.102 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.729+0000 7f113865d700 1 -- 172.21.15.32:0/1118646542 >> 172.21.15.32:0/1118646542 conn(0x7f1130101c30 msgr2=0x7f1130102940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:19.102 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.729+0000 7f113865d700 1 -- 172.21.15.32:0/1118646542 shutdown_connections 2023-09-30T14:31:19.102 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:18.729+0000 7f113865d700 1 -- 172.21.15.32:0/1118646542 wait complete. 2023-09-30T14:31:19.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:19 smithi032 ceph-mon[98592]: mgrmap e13: x(active, since 6s) 2023-09-30T14:31:20.544 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.555+0000 7f82876a9700 1 Processor -- start 2023-09-30T14:31:20.545 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.556+0000 7f82876a9700 1 -- start start 2023-09-30T14:31:20.545 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.556+0000 7f82876a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8280105580 0x7f82801059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:20.545 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.556+0000 7f82876a9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8280105f70 con 0x7f8280105580 2023-09-30T14:31:20.545 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.556+0000 7f8285445700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8280105580 0x7f82801059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:20.545 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.556+0000 7f8285445700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8280105580 0x7f82801059a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60986/0 (socket says 172.21.15.32:60986) 2023-09-30T14:31:20.545 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.556+0000 7f8285445700 1 -- 172.21.15.32:0/35692980 learned_addr learned my addr 172.21.15.32:0/35692980 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:20.545 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.556+0000 7f8285445700 1 -- 172.21.15.32:0/35692980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8280106780 con 0x7f8280105580 2023-09-30T14:31:20.545 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.557+0000 7f8285445700 1 --2- 172.21.15.32:0/35692980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8280105580 0x7f82801059a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f827c0098a0 tx=0x7f827c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a056c022bf02f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:20.545 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.557+0000 7f8277fff700 1 -- 172.21.15.32:0/35692980 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f827c004160 con 0x7f8280105580 2023-09-30T14:31:20.545 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.557+0000 7f8277fff700 1 -- 172.21.15.32:0/35692980 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f827c0042c0 con 0x7f8280105580 2023-09-30T14:31:20.545 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.557+0000 7f8277fff700 1 -- 172.21.15.32:0/35692980 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f827c0119b0 con 0x7f8280105580 2023-09-30T14:31:20.545 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.557+0000 7f82876a9700 1 -- 172.21.15.32:0/35692980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8280105580 msgr2=0x7f82801059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:20.546 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.557+0000 7f82876a9700 1 --2- 172.21.15.32:0/35692980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8280105580 0x7f82801059a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f827c0098a0 tx=0x7f827c009bb0 comp rx=0 tx=0).stop 2023-09-30T14:31:20.546 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.557+0000 7f82876a9700 1 -- 172.21.15.32:0/35692980 shutdown_connections 2023-09-30T14:31:20.546 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.557+0000 7f82876a9700 1 --2- 172.21.15.32:0/35692980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8280105580 0x7f82801059a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:20.546 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.557+0000 7f82876a9700 1 -- 172.21.15.32:0/35692980 >> 172.21.15.32:0/35692980 conn(0x7f8280100b00 msgr2=0x7f8280102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:20.546 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.558+0000 7f82876a9700 1 -- 172.21.15.32:0/35692980 shutdown_connections 2023-09-30T14:31:20.546 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.558+0000 7f82876a9700 1 -- 172.21.15.32:0/35692980 wait complete. 2023-09-30T14:31:20.546 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.558+0000 7f82876a9700 1 Processor -- start 2023-09-30T14:31:20.546 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.558+0000 7f82876a9700 1 -- start start 2023-09-30T14:31:20.546 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.558+0000 7f82876a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8280105580 0x7f8280079bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:20.546 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.558+0000 7f82876a9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828019d000 con 0x7f8280105580 2023-09-30T14:31:20.547 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.558+0000 7f8285445700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8280105580 0x7f8280079bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:20.547 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.558+0000 7f8285445700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8280105580 0x7f8280079bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60988/0 (socket says 172.21.15.32:60988) 2023-09-30T14:31:20.547 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.558+0000 7f8285445700 1 -- 172.21.15.32:0/2307527802 learned_addr learned my addr 172.21.15.32:0/2307527802 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:20.547 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.559+0000 7f8285445700 1 -- 172.21.15.32:0/2307527802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c009550 con 0x7f8280105580 2023-09-30T14:31:20.547 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.559+0000 7f8285445700 1 --2- 172.21.15.32:0/2307527802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8280105580 0x7f8280079bb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f827c003e60 tx=0x7f827c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:20.547 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.559+0000 7f82767fc700 1 -- 172.21.15.32:0/2307527802 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f827c003970 con 0x7f8280105580 2023-09-30T14:31:20.547 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.559+0000 7f82876a9700 1 -- 172.21.15.32:0/2307527802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828019d140 con 0x7f8280105580 2023-09-30T14:31:20.547 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.559+0000 7f82767fc700 1 -- 172.21.15.32:0/2307527802 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f827c003ad0 con 0x7f8280105580 2023-09-30T14:31:20.547 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.559+0000 7f82767fc700 1 -- 172.21.15.32:0/2307527802 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f827c0195e0 con 0x7f8280105580 2023-09-30T14:31:20.547 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.559+0000 7f82876a9700 1 -- 172.21.15.32:0/2307527802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828019d5e0 con 0x7f8280105580 2023-09-30T14:31:20.547 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.560+0000 7f82767fc700 1 -- 172.21.15.32:0/2307527802 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f827c0198a0 con 0x7f8280105580 2023-09-30T14:31:20.547 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.560+0000 7f82767fc700 1 --2- 172.21.15.32:0/2307527802 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f826c038720 0x7f826c03abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:20.548 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.560+0000 7f82767fc700 1 -- 172.21.15.32:0/2307527802 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f827c04c9d0 con 0x7f8280105580 2023-09-30T14:31:20.548 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.560+0000 7f82876a9700 1 -- 172.21.15.32:0/2307527802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82801099b0 con 0x7f8280105580 2023-09-30T14:31:20.548 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.560+0000 7f8284c44700 1 --2- 172.21.15.32:0/2307527802 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f826c038720 0x7f826c03abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:20.548 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.561+0000 7f8284c44700 1 --2- 172.21.15.32:0/2307527802 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f826c038720 0x7f826c03abe0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8270006fd0 tx=0x7f8270006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:20.548 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:19.565+0000 7f82767fc700 1 -- 172.21.15.32:0/2307527802 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f827c003c40 con 0x7f8280105580 2023-09-30T14:31:20.548 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:20.144+0000 7f82876a9700 1 -- 172.21.15.32:0/2307527802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f828019d820 con 0x7f8280105580 2023-09-30T14:31:20.548 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:20.147+0000 7f82767fc700 1 -- 172.21.15.32:0/2307527802 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f827c0212a0 con 0x7f8280105580 2023-09-30T14:31:20.548 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:20.149+0000 7f82876a9700 1 -- 172.21.15.32:0/2307527802 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f826c038720 msgr2=0x7f826c03abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:20.548 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:20.149+0000 7f82876a9700 1 --2- 172.21.15.32:0/2307527802 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f826c038720 0x7f826c03abe0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8270006fd0 tx=0x7f8270006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:20.548 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:20.149+0000 7f82876a9700 1 -- 172.21.15.32:0/2307527802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8280105580 msgr2=0x7f8280079bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:20.548 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:20.149+0000 7f82876a9700 1 --2- 172.21.15.32:0/2307527802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8280105580 0x7f8280079bb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f827c003e60 tx=0x7f827c011aa0 comp rx=0 tx=0).stop 2023-09-30T14:31:20.548 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:20.150+0000 7f82876a9700 1 -- 172.21.15.32:0/2307527802 shutdown_connections 2023-09-30T14:31:20.549 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:20.150+0000 7f82876a9700 1 --2- 172.21.15.32:0/2307527802 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f826c038720 0x7f826c03abe0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:20.549 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:20.150+0000 7f82876a9700 1 --2- 172.21.15.32:0/2307527802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8280105580 0x7f8280079bb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:20.549 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:20.150+0000 7f82876a9700 1 -- 172.21.15.32:0/2307527802 >> 172.21.15.32:0/2307527802 conn(0x7f8280100b00 msgr2=0x7f82801018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:20.549 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:20.150+0000 7f82876a9700 1 -- 172.21.15.32:0/2307527802 shutdown_connections 2023-09-30T14:31:20.549 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2023-09-30T14:31:20.150+0000 7f82876a9700 1 -- 172.21.15.32:0/2307527802 wait complete. 2023-09-30T14:31:20.549 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-09-30T14:31:20.549 INFO:teuthology.orchestra.run.smithi032.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-09-30T14:31:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:31:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-09-30T14:31:20.549 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:31:20.549 INFO:teuthology.orchestra.run.smithi032.stdout:Or, if you are only running a single cluster on this host: 2023-09-30T14:31:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:31:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-09-30T14:31:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:31:20.550 INFO:teuthology.orchestra.run.smithi032.stdout:Please consider enabling telemetry to help improve Ceph: 2023-09-30T14:31:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:31:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: ceph telemetry on 2023-09-30T14:31:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:31:20.550 INFO:teuthology.orchestra.run.smithi032.stdout:For more information see: 2023-09-30T14:31:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:31:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-09-30T14:31:20.550 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:31:20.550 INFO:teuthology.orchestra.run.smithi032.stdout:Bootstrap complete. 2023-09-30T14:31:20.578 INFO:tasks.cephadm:Fetching config... 2023-09-30T14:31:20.579 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:31:20.579 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-09-30T14:31:20.596 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-09-30T14:31:20.597 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:31:20.597 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-09-30T14:31:20.656 INFO:tasks.cephadm:Fetching mon keyring... 2023-09-30T14:31:20.657 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:31:20.657 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-09-30T14:31:20.726 INFO:tasks.cephadm:Fetching pub ssh key... 2023-09-30T14:31:20.726 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:31:20.726 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-09-30T14:31:20.783 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-09-30T14:31:20.783 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDodJBINELD7Ztf88FYZFaymjkdE1C/W3n/8v3aqRgdx1Cy7s1GM0BUr+BjoNBFK+JWwPUWVkNQe9kUlqCNeH+NYubmwdrCVadVrd0Rsf+1DRadO3LMldXuXhO1io+HIEk9AC9Omo9w/+KGYVrlFxAzfqtVXDo94cLFXrdar4DdUSnOAlXc/SqFyJ6i55HxGesymB4fm+/IG7qPYNm/HUFHFdComhlDoPFJ+uxFVArivrJo1RosQnThvq2J6UulOPFty+OZ4BtzFQsQ7qMdfzKslCWpmogxWBqrkk17FLXF9c4APFtONhkwoDFi8HsZRB8xZ+U5AYJhPIOiJLE3XBoK3I/bbtM91ezlvQHpoRbCo6nGqOd201Sgot+LpYQaDpPPfPiQ1IFDrSehQQhjY6yU+bztcKNUBWXHCoOPN10q+9bYnKgsR+Su3qU1ekzupuhRxJGBlWuGsaHM5oHU0H8d2QyoJDW144PnENNT0YeDBZpl6R1m1NufhxKbtPIsyns= ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-30T14:31:20.861 INFO:teuthology.orchestra.run.smithi032.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDodJBINELD7Ztf88FYZFaymjkdE1C/W3n/8v3aqRgdx1Cy7s1GM0BUr+BjoNBFK+JWwPUWVkNQe9kUlqCNeH+NYubmwdrCVadVrd0Rsf+1DRadO3LMldXuXhO1io+HIEk9AC9Omo9w/+KGYVrlFxAzfqtVXDo94cLFXrdar4DdUSnOAlXc/SqFyJ6i55HxGesymB4fm+/IG7qPYNm/HUFHFdComhlDoPFJ+uxFVArivrJo1RosQnThvq2J6UulOPFty+OZ4BtzFQsQ7qMdfzKslCWpmogxWBqrkk17FLXF9c4APFtONhkwoDFi8HsZRB8xZ+U5AYJhPIOiJLE3XBoK3I/bbtM91ezlvQHpoRbCo6nGqOd201Sgot+LpYQaDpPPfPiQ1IFDrSehQQhjY6yU+bztcKNUBWXHCoOPN10q+9bYnKgsR+Su3qU1ekzupuhRxJGBlWuGsaHM5oHU0H8d2QyoJDW144PnENNT0YeDBZpl6R1m1NufhxKbtPIsyns= ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:31:20.876 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDodJBINELD7Ztf88FYZFaymjkdE1C/W3n/8v3aqRgdx1Cy7s1GM0BUr+BjoNBFK+JWwPUWVkNQe9kUlqCNeH+NYubmwdrCVadVrd0Rsf+1DRadO3LMldXuXhO1io+HIEk9AC9Omo9w/+KGYVrlFxAzfqtVXDo94cLFXrdar4DdUSnOAlXc/SqFyJ6i55HxGesymB4fm+/IG7qPYNm/HUFHFdComhlDoPFJ+uxFVArivrJo1RosQnThvq2J6UulOPFty+OZ4BtzFQsQ7qMdfzKslCWpmogxWBqrkk17FLXF9c4APFtONhkwoDFi8HsZRB8xZ+U5AYJhPIOiJLE3XBoK3I/bbtM91ezlvQHpoRbCo6nGqOd201Sgot+LpYQaDpPPfPiQ1IFDrSehQQhjY6yU+bztcKNUBWXHCoOPN10q+9bYnKgsR+Su3qU1ekzupuhRxJGBlWuGsaHM5oHU0H8d2QyoJDW144PnENNT0YeDBZpl6R1m1NufhxKbtPIsyns= ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-30T14:31:20.923 INFO:teuthology.orchestra.run.smithi148.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDodJBINELD7Ztf88FYZFaymjkdE1C/W3n/8v3aqRgdx1Cy7s1GM0BUr+BjoNBFK+JWwPUWVkNQe9kUlqCNeH+NYubmwdrCVadVrd0Rsf+1DRadO3LMldXuXhO1io+HIEk9AC9Omo9w/+KGYVrlFxAzfqtVXDo94cLFXrdar4DdUSnOAlXc/SqFyJ6i55HxGesymB4fm+/IG7qPYNm/HUFHFdComhlDoPFJ+uxFVArivrJo1RosQnThvq2J6UulOPFty+OZ4BtzFQsQ7qMdfzKslCWpmogxWBqrkk17FLXF9c4APFtONhkwoDFi8HsZRB8xZ+U5AYJhPIOiJLE3XBoK3I/bbtM91ezlvQHpoRbCo6nGqOd201Sgot+LpYQaDpPPfPiQ1IFDrSehQQhjY6yU+bztcKNUBWXHCoOPN10q+9bYnKgsR+Su3qU1ekzupuhRxJGBlWuGsaHM5oHU0H8d2QyoJDW144PnENNT0YeDBZpl6R1m1NufhxKbtPIsyns= ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:31:20.936 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDodJBINELD7Ztf88FYZFaymjkdE1C/W3n/8v3aqRgdx1Cy7s1GM0BUr+BjoNBFK+JWwPUWVkNQe9kUlqCNeH+NYubmwdrCVadVrd0Rsf+1DRadO3LMldXuXhO1io+HIEk9AC9Omo9w/+KGYVrlFxAzfqtVXDo94cLFXrdar4DdUSnOAlXc/SqFyJ6i55HxGesymB4fm+/IG7qPYNm/HUFHFdComhlDoPFJ+uxFVArivrJo1RosQnThvq2J6UulOPFty+OZ4BtzFQsQ7qMdfzKslCWpmogxWBqrkk17FLXF9c4APFtONhkwoDFi8HsZRB8xZ+U5AYJhPIOiJLE3XBoK3I/bbtM91ezlvQHpoRbCo6nGqOd201Sgot+LpYQaDpPPfPiQ1IFDrSehQQhjY6yU+bztcKNUBWXHCoOPN10q+9bYnKgsR+Su3qU1ekzupuhRxJGBlWuGsaHM5oHU0H8d2QyoJDW144PnENNT0YeDBZpl6R1m1NufhxKbtPIsyns= ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-30T14:31:20.983 INFO:teuthology.orchestra.run.smithi156.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDodJBINELD7Ztf88FYZFaymjkdE1C/W3n/8v3aqRgdx1Cy7s1GM0BUr+BjoNBFK+JWwPUWVkNQe9kUlqCNeH+NYubmwdrCVadVrd0Rsf+1DRadO3LMldXuXhO1io+HIEk9AC9Omo9w/+KGYVrlFxAzfqtVXDo94cLFXrdar4DdUSnOAlXc/SqFyJ6i55HxGesymB4fm+/IG7qPYNm/HUFHFdComhlDoPFJ+uxFVArivrJo1RosQnThvq2J6UulOPFty+OZ4BtzFQsQ7qMdfzKslCWpmogxWBqrkk17FLXF9c4APFtONhkwoDFi8HsZRB8xZ+U5AYJhPIOiJLE3XBoK3I/bbtM91ezlvQHpoRbCo6nGqOd201Sgot+LpYQaDpPPfPiQ1IFDrSehQQhjY6yU+bztcKNUBWXHCoOPN10q+9bYnKgsR+Su3qU1ekzupuhRxJGBlWuGsaHM5oHU0H8d2QyoJDW144PnENNT0YeDBZpl6R1m1NufhxKbtPIsyns= ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:31:20.998 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-09-30T14:31:21.567 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:31:21.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:21 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2307527802' entity='client.admin' 2023-09-30T14:31:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.406+0000 7ff64877e700 1 -- 172.21.15.32:0/2662122423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6400ffd80 msgr2=0x7ff6401001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.406+0000 7ff64877e700 1 --2- 172.21.15.32:0/2662122423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6400ffd80 0x7ff6401001a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff63c009910 tx=0x7ff63c009c20 comp rx=0 tx=0).stop 2023-09-30T14:31:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.406+0000 7ff64877e700 1 -- 172.21.15.32:0/2662122423 shutdown_connections 2023-09-30T14:31:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.406+0000 7ff64877e700 1 --2- 172.21.15.32:0/2662122423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6400ffd80 0x7ff6401001a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:23.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.406+0000 7ff64877e700 1 -- 172.21.15.32:0/2662122423 >> 172.21.15.32:0/2662122423 conn(0x7ff6400fa560 msgr2=0x7ff6400fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.406+0000 7ff64877e700 1 -- 172.21.15.32:0/2662122423 shutdown_connections 2023-09-30T14:31:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.406+0000 7ff64877e700 1 -- 172.21.15.32:0/2662122423 wait complete. 2023-09-30T14:31:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.407+0000 7ff64877e700 1 Processor -- start 2023-09-30T14:31:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.407+0000 7ff64877e700 1 -- start start 2023-09-30T14:31:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.407+0000 7ff64877e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6400ffd80 0x7ff6400ff440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.407+0000 7ff64877e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64019edc0 con 0x7ff6400ffd80 2023-09-30T14:31:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.407+0000 7ff64651a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6400ffd80 0x7ff6400ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.407+0000 7ff64651a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6400ffd80 0x7ff6400ff440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60994/0 (socket says 172.21.15.32:60994) 2023-09-30T14:31:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.407+0000 7ff64651a700 1 -- 172.21.15.32:0/4003554371 learned_addr learned my addr 172.21.15.32:0/4003554371 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.407+0000 7ff64651a700 1 -- 172.21.15.32:0/4003554371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c0095f0 con 0x7ff6400ffd80 2023-09-30T14:31:23.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.408+0000 7ff64651a700 1 --2- 172.21.15.32:0/4003554371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6400ffd80 0x7ff6400ff440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff63c006010 tx=0x7ff63c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.408+0000 7ff6377fe700 1 -- 172.21.15.32:0/4003554371 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff63c01d070 con 0x7ff6400ffd80 2023-09-30T14:31:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.408+0000 7ff6377fe700 1 -- 172.21.15.32:0/4003554371 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff63c0044e0 con 0x7ff6400ffd80 2023-09-30T14:31:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.408+0000 7ff64877e700 1 -- 172.21.15.32:0/4003554371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6401a33d0 con 0x7ff6400ffd80 2023-09-30T14:31:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.408+0000 7ff6377fe700 1 -- 172.21.15.32:0/4003554371 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff63c0039b0 con 0x7ff6400ffd80 2023-09-30T14:31:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.408+0000 7ff64877e700 1 -- 172.21.15.32:0/4003554371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6401a3730 con 0x7ff6400ffd80 2023-09-30T14:31:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.409+0000 7ff6377fe700 1 -- 172.21.15.32:0/4003554371 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45548+0+0 (secure 0 0 0) 0x7ff63c003c20 con 0x7ff6400ffd80 2023-09-30T14:31:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.409+0000 7ff64877e700 1 -- 172.21.15.32:0/4003554371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64006dff0 con 0x7ff6400ffd80 2023-09-30T14:31:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.409+0000 7ff6377fe700 1 --2- 172.21.15.32:0/4003554371 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff62c038770 0x7ff62c03ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:23.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.409+0000 7ff6377fe700 1 -- 172.21.15.32:0/4003554371 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff63c04c730 con 0x7ff6400ffd80 2023-09-30T14:31:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.409+0000 7ff645d19700 1 --2- 172.21.15.32:0/4003554371 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff62c038770 0x7ff62c03ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.409+0000 7ff645d19700 1 --2- 172.21.15.32:0/4003554371 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff62c038770 0x7ff62c03ac30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff630006fd0 tx=0x7ff630006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:23.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.414+0000 7ff6377fe700 1 -- 172.21.15.32:0/4003554371 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7ff63c02a950 con 0x7ff6400ffd80 2023-09-30T14:31:23.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.634+0000 7ff64877e700 1 -- 172.21.15.32:0/4003554371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7ff6401055b0 con 0x7ff6400ffd80 2023-09-30T14:31:23.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.637+0000 7ff6377fe700 1 -- 172.21.15.32:0/4003554371 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff63c02a360 con 0x7ff6400ffd80 2023-09-30T14:31:23.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.639+0000 7ff64877e700 1 -- 172.21.15.32:0/4003554371 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff62c038770 msgr2=0x7ff62c03ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:23.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.639+0000 7ff64877e700 1 --2- 172.21.15.32:0/4003554371 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff62c038770 0x7ff62c03ac30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff630006fd0 tx=0x7ff630006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:23.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.639+0000 7ff64877e700 1 -- 172.21.15.32:0/4003554371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6400ffd80 msgr2=0x7ff6400ff440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:23.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.639+0000 7ff64877e700 1 --2- 172.21.15.32:0/4003554371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6400ffd80 0x7ff6400ff440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff63c006010 tx=0x7ff63c0057c0 comp rx=0 tx=0).stop 2023-09-30T14:31:23.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.639+0000 7ff64877e700 1 -- 172.21.15.32:0/4003554371 shutdown_connections 2023-09-30T14:31:23.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.639+0000 7ff64877e700 1 --2- 172.21.15.32:0/4003554371 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff62c038770 0x7ff62c03ac30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:23.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.639+0000 7ff64877e700 1 --2- 172.21.15.32:0/4003554371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6400ffd80 0x7ff6400ff440 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:23.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.639+0000 7ff64877e700 1 -- 172.21.15.32:0/4003554371 >> 172.21.15.32:0/4003554371 conn(0x7ff6400fa560 msgr2=0x7ff6400fb320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:23.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.640+0000 7ff64877e700 1 -- 172.21.15.32:0/4003554371 shutdown_connections 2023-09-30T14:31:23.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:23.640+0000 7ff64877e700 1 -- 172.21.15.32:0/4003554371 wait complete. 2023-09-30T14:31:24.223 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-09-30T14:31:24.223 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-09-30T14:31:24.789 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:31:25.023 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:24 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/4003554371' entity='client.admin' 2023-09-30T14:31:25.023 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:25.023 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:25.023 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi032", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:31:25.023 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:25.023 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:31:25.023 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:26.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.084+0000 7f3ce00f1700 1 -- 172.21.15.32:0/1851708566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd80ff840 msgr2=0x7f3cd80ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:26.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.084+0000 7f3ce00f1700 1 --2- 172.21.15.32:0/1851708566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd80ff840 0x7f3cd80ffc60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009910 tx=0x7f3cc8009c20 comp rx=0 tx=0).stop 2023-09-30T14:31:26.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.084+0000 7f3ce00f1700 1 -- 172.21.15.32:0/1851708566 shutdown_connections 2023-09-30T14:31:26.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.084+0000 7f3ce00f1700 1 --2- 172.21.15.32:0/1851708566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd80ff840 0x7f3cd80ffc60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:26.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.084+0000 7f3ce00f1700 1 -- 172.21.15.32:0/1851708566 >> 172.21.15.32:0/1851708566 conn(0x7f3cd80fadc0 msgr2=0x7f3cd80fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:26.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.084+0000 7f3ce00f1700 1 -- 172.21.15.32:0/1851708566 shutdown_connections 2023-09-30T14:31:26.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.084+0000 7f3ce00f1700 1 -- 172.21.15.32:0/1851708566 wait complete. 2023-09-30T14:31:26.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.085+0000 7f3ce00f1700 1 Processor -- start 2023-09-30T14:31:26.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.085+0000 7f3ce00f1700 1 -- start start 2023-09-30T14:31:26.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.085+0000 7f3ce00f1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd80ff840 0x7f3cd8195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:26.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.085+0000 7f3ce00f1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd8195f80 con 0x7f3cd80ff840 2023-09-30T14:31:26.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.085+0000 7f3cdde8d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd80ff840 0x7f3cd8195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:26.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.085+0000 7f3cdde8d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd80ff840 0x7f3cd8195a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32768/0 (socket says 172.21.15.32:32768) 2023-09-30T14:31:26.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.085+0000 7f3cdde8d700 1 -- 172.21.15.32:0/718324044 learned_addr learned my addr 172.21.15.32:0/718324044 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:26.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.085+0000 7f3cdde8d700 1 -- 172.21.15.32:0/718324044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc80095f0 con 0x7f3cd80ff840 2023-09-30T14:31:26.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.085+0000 7f3cdde8d700 1 --2- 172.21.15.32:0/718324044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd80ff840 0x7f3cd8195a40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8004f40 tx=0x7f3cc80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:26.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.085+0000 7f3cceffd700 1 -- 172.21.15.32:0/718324044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cc801d070 con 0x7f3cd80ff840 2023-09-30T14:31:26.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.086+0000 7f3cceffd700 1 -- 172.21.15.32:0/718324044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cc800bc10 con 0x7f3cd80ff840 2023-09-30T14:31:26.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.086+0000 7f3ce00f1700 1 -- 172.21.15.32:0/718324044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd8196180 con 0x7f3cd80ff840 2023-09-30T14:31:26.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.086+0000 7f3cceffd700 1 -- 172.21.15.32:0/718324044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cc8003a70 con 0x7f3cd80ff840 2023-09-30T14:31:26.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.086+0000 7f3ce00f1700 1 -- 172.21.15.32:0/718324044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd8196620 con 0x7f3cd80ff840 2023-09-30T14:31:26.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.087+0000 7f3cceffd700 1 -- 172.21.15.32:0/718324044 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f3cc8003d30 con 0x7f3cd80ff840 2023-09-30T14:31:26.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.087+0000 7f3cceffd700 1 --2- 172.21.15.32:0/718324044 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3cc4038720 0x7f3cc403abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:26.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.087+0000 7f3cceffd700 1 -- 172.21.15.32:0/718324044 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3cc804c5b0 con 0x7f3cd80ff840 2023-09-30T14:31:26.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.087+0000 7f3cdd68c700 1 --2- 172.21.15.32:0/718324044 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3cc4038720 0x7f3cc403abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:26.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.087+0000 7f3ce00f1700 1 -- 172.21.15.32:0/718324044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd818f360 con 0x7f3cd80ff840 2023-09-30T14:31:26.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.087+0000 7f3cdd68c700 1 --2- 172.21.15.32:0/718324044 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3cc4038720 0x7f3cc403abe0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4006fd0 tx=0x7f3cd4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:26.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.092+0000 7f3cceffd700 1 -- 172.21.15.32:0/718324044 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f3cc8026070 con 0x7f3cd80ff840 2023-09-30T14:31:26.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.294+0000 7f3ce00f1700 1 -- 172.21.15.32:0/718324044 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cd802d040 con 0x7f3cc4038720 2023-09-30T14:31:26.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.298+0000 7f3cceffd700 1 -- 172.21.15.32:0/718324044 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3cd802d040 con 0x7f3cc4038720 2023-09-30T14:31:26.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.300+0000 7f3ce00f1700 1 -- 172.21.15.32:0/718324044 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3cc4038720 msgr2=0x7f3cc403abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:26.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.300+0000 7f3ce00f1700 1 --2- 172.21.15.32:0/718324044 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3cc4038720 0x7f3cc403abe0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4006fd0 tx=0x7f3cd4006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:26.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.300+0000 7f3ce00f1700 1 -- 172.21.15.32:0/718324044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd80ff840 msgr2=0x7f3cd8195a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:26.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.300+0000 7f3ce00f1700 1 --2- 172.21.15.32:0/718324044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd80ff840 0x7f3cd8195a40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8004f40 tx=0x7f3cc80057c0 comp rx=0 tx=0).stop 2023-09-30T14:31:26.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.300+0000 7f3ce00f1700 1 -- 172.21.15.32:0/718324044 shutdown_connections 2023-09-30T14:31:26.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.300+0000 7f3ce00f1700 1 --2- 172.21.15.32:0/718324044 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3cc4038720 0x7f3cc403abe0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:26.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.300+0000 7f3ce00f1700 1 --2- 172.21.15.32:0/718324044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cd80ff840 0x7f3cd8195a40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:26.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.300+0000 7f3ce00f1700 1 -- 172.21.15.32:0/718324044 >> 172.21.15.32:0/718324044 conn(0x7f3cd80fadc0 msgr2=0x7f3cd8069150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:26.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.300+0000 7f3ce00f1700 1 -- 172.21.15.32:0/718324044 shutdown_connections 2023-09-30T14:31:26.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:26.300+0000 7f3ce00f1700 1 -- 172.21.15.32:0/718324044 wait complete. 2023-09-30T14:31:26.760 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi148 2023-09-30T14:31:26.760 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-09-30T14:31:26.760 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.conf 2023-09-30T14:31:26.781 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-09-30T14:31:26.781 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-30T14:31:26.840 INFO:tasks.cephadm:Adding host smithi148 to orchestrator... 2023-09-30T14:31:26.841 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi148 2023-09-30T14:31:27.422 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:31:27.606 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:27 smithi032 ceph-mon[98592]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:31:27.606 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:27 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:27.606 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:27 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:31:27.606 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:27 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:31:27.606 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:27 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:31:27.606 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:27 smithi032 ceph-mon[98592]: Updating smithi032:/etc/ceph/ceph.conf 2023-09-30T14:31:28.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.582+0000 7fd4459a6700 1 -- 172.21.15.32:0/2651353904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4401057c0 msgr2=0x7fd440105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:28.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.582+0000 7fd4459a6700 1 --2- 172.21.15.32:0/2651353904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4401057c0 0x7fd440105ba0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd434009910 tx=0x7fd434009c20 comp rx=0 tx=0).stop 2023-09-30T14:31:28.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.582+0000 7fd4459a6700 1 -- 172.21.15.32:0/2651353904 shutdown_connections 2023-09-30T14:31:28.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.582+0000 7fd4459a6700 1 --2- 172.21.15.32:0/2651353904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4401057c0 0x7fd440105ba0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:28.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.582+0000 7fd4459a6700 1 -- 172.21.15.32:0/2651353904 >> 172.21.15.32:0/2651353904 conn(0x7fd4400fb2d0 msgr2=0x7fd4400fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:28.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.582+0000 7fd4459a6700 1 -- 172.21.15.32:0/2651353904 shutdown_connections 2023-09-30T14:31:28.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.582+0000 7fd4459a6700 1 -- 172.21.15.32:0/2651353904 wait complete. 2023-09-30T14:31:28.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.583+0000 7fd4459a6700 1 Processor -- start 2023-09-30T14:31:28.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.583+0000 7fd4459a6700 1 -- start start 2023-09-30T14:31:28.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.583+0000 7fd4459a6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4401057c0 0x7fd440194ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:28.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.583+0000 7fd4459a6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4401951e0 con 0x7fd4401057c0 2023-09-30T14:31:28.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.583+0000 7fd43effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4401057c0 0x7fd440194ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:28.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.583+0000 7fd43effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4401057c0 0x7fd440194ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32774/0 (socket says 172.21.15.32:32774) 2023-09-30T14:31:28.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.583+0000 7fd43effd700 1 -- 172.21.15.32:0/3330531028 learned_addr learned my addr 172.21.15.32:0/3330531028 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:28.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.584+0000 7fd43effd700 1 -- 172.21.15.32:0/3330531028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4340095f0 con 0x7fd4401057c0 2023-09-30T14:31:28.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.584+0000 7fd43effd700 1 --2- 172.21.15.32:0/3330531028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4401057c0 0x7fd440194ca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd434004f40 tx=0x7fd434005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:28.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.584+0000 7fd4449a4700 1 -- 172.21.15.32:0/3330531028 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd43401d070 con 0x7fd4401057c0 2023-09-30T14:31:28.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.584+0000 7fd4449a4700 1 -- 172.21.15.32:0/3330531028 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd434004370 con 0x7fd4401057c0 2023-09-30T14:31:28.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.584+0000 7fd4459a6700 1 -- 172.21.15.32:0/3330531028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd440195440 con 0x7fd4401057c0 2023-09-30T14:31:28.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.584+0000 7fd4449a4700 1 -- 172.21.15.32:0/3330531028 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd434003970 con 0x7fd4401057c0 2023-09-30T14:31:28.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.584+0000 7fd4459a6700 1 -- 172.21.15.32:0/3330531028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4401958c0 con 0x7fd4401057c0 2023-09-30T14:31:28.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.585+0000 7fd4459a6700 1 -- 172.21.15.32:0/3330531028 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd440062450 con 0x7fd4401057c0 2023-09-30T14:31:28.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.586+0000 7fd4449a4700 1 -- 172.21.15.32:0/3330531028 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fd434003be0 con 0x7fd4401057c0 2023-09-30T14:31:28.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.586+0000 7fd4449a4700 1 --2- 172.21.15.32:0/3330531028 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd428038770 0x7fd42803ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:28.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.586+0000 7fd4449a4700 1 -- 172.21.15.32:0/3330531028 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd43404c960 con 0x7fd4401057c0 2023-09-30T14:31:28.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.586+0000 7fd43e7fc700 1 --2- 172.21.15.32:0/3330531028 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd428038770 0x7fd42803ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:28.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.586+0000 7fd43e7fc700 1 --2- 172.21.15.32:0/3330531028 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd428038770 0x7fd42803ac30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd430006fd0 tx=0x7fd430006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:28.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.590+0000 7fd4449a4700 1 -- 172.21.15.32:0/3330531028 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fd43400e5c0 con 0x7fd4401057c0 2023-09-30T14:31:28.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:28 smithi032 ceph-mon[98592]: Updating smithi032:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-09-30T14:31:28.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:28 smithi032 ceph-mon[98592]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2023-09-30T14:31:28.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:28 smithi032 ceph-mon[98592]: Updating smithi032:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-30T14:31:28.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:28.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:28.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:28.795+0000 7fd4459a6700 1 -- 172.21.15.32:0/3330531028 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}) v1 -- 0x7fd440195f90 con 0x7fd428038770 2023-09-30T14:31:29.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:29 smithi032 ceph-mon[98592]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:31:29.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:29 smithi032 ceph-mon[98592]: Deploying cephadm binary to smithi148 2023-09-30T14:31:31.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:31.702+0000 7fd4449a4700 1 -- 172.21.15.32:0/3330531028 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd440195f90 con 0x7fd428038770 2023-09-30T14:31:31.702 INFO:teuthology.orchestra.run.smithi032.stdout:Added host 'smithi148' with addr '172.21.15.148' 2023-09-30T14:31:31.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:31.704+0000 7fd4459a6700 1 -- 172.21.15.32:0/3330531028 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd428038770 msgr2=0x7fd42803ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:31.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:31.704+0000 7fd4459a6700 1 --2- 172.21.15.32:0/3330531028 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd428038770 0x7fd42803ac30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd430006fd0 tx=0x7fd430006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:31.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:31.704+0000 7fd4459a6700 1 -- 172.21.15.32:0/3330531028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4401057c0 msgr2=0x7fd440194ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:31.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:31.704+0000 7fd4459a6700 1 --2- 172.21.15.32:0/3330531028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4401057c0 0x7fd440194ca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd434004f40 tx=0x7fd434005020 comp rx=0 tx=0).stop 2023-09-30T14:31:31.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:31.704+0000 7fd4459a6700 1 -- 172.21.15.32:0/3330531028 shutdown_connections 2023-09-30T14:31:31.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:31.705+0000 7fd4459a6700 1 --2- 172.21.15.32:0/3330531028 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd428038770 0x7fd42803ac30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:31.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:31.705+0000 7fd4459a6700 1 --2- 172.21.15.32:0/3330531028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4401057c0 0x7fd440194ca0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:31.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:31.705+0000 7fd4459a6700 1 -- 172.21.15.32:0/3330531028 >> 172.21.15.32:0/3330531028 conn(0x7fd4400fb2d0 msgr2=0x7fd440107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:31.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:31.705+0000 7fd4459a6700 1 -- 172.21.15.32:0/3330531028 shutdown_connections 2023-09-30T14:31:31.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:31.705+0000 7fd4459a6700 1 -- 172.21.15.32:0/3330531028 wait complete. 2023-09-30T14:31:32.092 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-09-30T14:31:32.658 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:31:33.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:32 smithi032 ceph-mon[98592]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:33.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:32 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:33.057 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:32 smithi032 ceph-mon[98592]: Added host smithi148 2023-09-30T14:31:33.058 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:32 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:31:33.058 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:32 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:33.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.967+0000 7fbfa2585700 1 -- 172.21.15.32:0/4183084381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c1057c0 msgr2=0x7fbf9c105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:33.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.967+0000 7fbfa2585700 1 --2- 172.21.15.32:0/4183084381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c1057c0 0x7fbf9c105ba0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009910 tx=0x7fbf8c009c20 comp rx=0 tx=0).stop 2023-09-30T14:31:33.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.967+0000 7fbfa2585700 1 -- 172.21.15.32:0/4183084381 shutdown_connections 2023-09-30T14:31:33.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.967+0000 7fbfa2585700 1 --2- 172.21.15.32:0/4183084381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c1057c0 0x7fbf9c105ba0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:33.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.967+0000 7fbfa2585700 1 -- 172.21.15.32:0/4183084381 >> 172.21.15.32:0/4183084381 conn(0x7fbf9c0fb2d0 msgr2=0x7fbf9c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:33.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.967+0000 7fbfa2585700 1 -- 172.21.15.32:0/4183084381 shutdown_connections 2023-09-30T14:31:33.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.967+0000 7fbfa2585700 1 -- 172.21.15.32:0/4183084381 wait complete. 2023-09-30T14:31:33.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.968+0000 7fbfa2585700 1 Processor -- start 2023-09-30T14:31:33.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.968+0000 7fbfa2585700 1 -- start start 2023-09-30T14:31:33.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.968+0000 7fbfa2585700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c1057c0 0x7fbf9c101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:33.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.968+0000 7fbfa2585700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c103310 con 0x7fbf9c1057c0 2023-09-30T14:31:33.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.968+0000 7fbf9bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c1057c0 0x7fbf9c101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:33.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.968+0000 7fbf9bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c1057c0 0x7fbf9c101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32800/0 (socket says 172.21.15.32:32800) 2023-09-30T14:31:33.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.968+0000 7fbf9bfff700 1 -- 172.21.15.32:0/3022051228 learned_addr learned my addr 172.21.15.32:0/3022051228 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:33.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.969+0000 7fbf9bfff700 1 -- 172.21.15.32:0/3022051228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c0095f0 con 0x7fbf9c1057c0 2023-09-30T14:31:33.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.969+0000 7fbf9bfff700 1 --2- 172.21.15.32:0/3022051228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c1057c0 0x7fbf9c101390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c006010 tx=0x7fbf8c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:33.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.969+0000 7fbf997fa700 1 -- 172.21.15.32:0/3022051228 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf8c01d070 con 0x7fbf9c1057c0 2023-09-30T14:31:33.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.969+0000 7fbfa2585700 1 -- 172.21.15.32:0/3022051228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c1018d0 con 0x7fbf9c1057c0 2023-09-30T14:31:33.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.969+0000 7fbf997fa700 1 -- 172.21.15.32:0/3022051228 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf8c00bc10 con 0x7fbf9c1057c0 2023-09-30T14:31:33.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.969+0000 7fbf997fa700 1 -- 172.21.15.32:0/3022051228 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf8c003910 con 0x7fbf9c1057c0 2023-09-30T14:31:33.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.969+0000 7fbfa2585700 1 -- 172.21.15.32:0/3022051228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c101d50 con 0x7fbf9c1057c0 2023-09-30T14:31:33.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.970+0000 7fbf997fa700 1 -- 172.21.15.32:0/3022051228 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fbf8c003bd0 con 0x7fbf9c1057c0 2023-09-30T14:31:33.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.970+0000 7fbfa2585700 1 -- 172.21.15.32:0/3022051228 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9c062450 con 0x7fbf9c1057c0 2023-09-30T14:31:33.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.970+0000 7fbf997fa700 1 --2- 172.21.15.32:0/3022051228 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbf84038720 0x7fbf8403abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:33.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.970+0000 7fbf997fa700 1 -- 172.21.15.32:0/3022051228 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbf8c04c560 con 0x7fbf9c1057c0 2023-09-30T14:31:33.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.970+0000 7fbf9b7fe700 1 --2- 172.21.15.32:0/3022051228 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbf84038720 0x7fbf8403abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:33.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.971+0000 7fbf9b7fe700 1 --2- 172.21.15.32:0/3022051228 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbf84038720 0x7fbf8403abe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbf90006fd0 tx=0x7fbf90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:33.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:33.975+0000 7fbf997fa700 1 -- 172.21.15.32:0/3022051228 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fbf8c02ad40 con 0x7fbf9c1057c0 2023-09-30T14:31:34.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:33 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:34.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:34.181+0000 7fbfa2585700 1 -- 172.21.15.32:0/3022051228 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbf9c1020c0 con 0x7fbf84038720 2023-09-30T14:31:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:34.182+0000 7fbf997fa700 1 -- 172.21.15.32:0/3022051228 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fbf9c1020c0 con 0x7fbf84038720 2023-09-30T14:31:34.182 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:31:34.182 INFO:teuthology.orchestra.run.smithi032.stdout:[{"addr": "172.21.15.32", "hostname": "smithi032", "labels": [], "status": ""}, {"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}] 2023-09-30T14:31:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:34.184+0000 7fbfa2585700 1 -- 172.21.15.32:0/3022051228 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbf84038720 msgr2=0x7fbf8403abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:34.184+0000 7fbfa2585700 1 --2- 172.21.15.32:0/3022051228 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbf84038720 0x7fbf8403abe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbf90006fd0 tx=0x7fbf90006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:34.184+0000 7fbfa2585700 1 -- 172.21.15.32:0/3022051228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c1057c0 msgr2=0x7fbf9c101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:34.184+0000 7fbfa2585700 1 --2- 172.21.15.32:0/3022051228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c1057c0 0x7fbf9c101390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c006010 tx=0x7fbf8c004ff0 comp rx=0 tx=0).stop 2023-09-30T14:31:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:34.184+0000 7fbfa2585700 1 -- 172.21.15.32:0/3022051228 shutdown_connections 2023-09-30T14:31:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:34.184+0000 7fbfa2585700 1 --2- 172.21.15.32:0/3022051228 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbf84038720 0x7fbf8403abe0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:34.184+0000 7fbfa2585700 1 --2- 172.21.15.32:0/3022051228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c1057c0 0x7fbf9c101390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:34.184+0000 7fbfa2585700 1 -- 172.21.15.32:0/3022051228 >> 172.21.15.32:0/3022051228 conn(0x7fbf9c0fb2d0 msgr2=0x7fbf9c107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:34.185+0000 7fbfa2585700 1 -- 172.21.15.32:0/3022051228 shutdown_connections 2023-09-30T14:31:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:34.185+0000 7fbfa2585700 1 -- 172.21.15.32:0/3022051228 wait complete. 2023-09-30T14:31:34.608 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi156 2023-09-30T14:31:34.609 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-30T14:31:34.609 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.conf 2023-09-30T14:31:34.630 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-30T14:31:34.631 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-30T14:31:34.691 INFO:tasks.cephadm:Adding host smithi156 to orchestrator... 2023-09-30T14:31:34.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi156 2023-09-30T14:31:35.256 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:31:35.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:35 smithi032 ceph-mon[98592]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:35.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:35 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:35.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:35 smithi032 ceph-mon[98592]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:31:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.447+0000 7faa57e50700 1 -- 172.21.15.32:0/2563451521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa501057c0 msgr2=0x7faa50105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.447+0000 7faa57e50700 1 --2- 172.21.15.32:0/2563451521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa501057c0 0x7faa50105ba0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faa40009910 tx=0x7faa40009c20 comp rx=0 tx=0).stop 2023-09-30T14:31:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.447+0000 7faa57e50700 1 -- 172.21.15.32:0/2563451521 shutdown_connections 2023-09-30T14:31:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.447+0000 7faa57e50700 1 --2- 172.21.15.32:0/2563451521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa501057c0 0x7faa50105ba0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:36.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.447+0000 7faa57e50700 1 -- 172.21.15.32:0/2563451521 >> 172.21.15.32:0/2563451521 conn(0x7faa500fb2d0 msgr2=0x7faa500fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.448+0000 7faa57e50700 1 -- 172.21.15.32:0/2563451521 shutdown_connections 2023-09-30T14:31:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.448+0000 7faa57e50700 1 -- 172.21.15.32:0/2563451521 wait complete. 2023-09-30T14:31:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.448+0000 7faa57e50700 1 Processor -- start 2023-09-30T14:31:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.448+0000 7faa57e50700 1 -- start start 2023-09-30T14:31:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.448+0000 7faa57e50700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa501057c0 0x7faa500fb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.448+0000 7faa57e50700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa500fd700 con 0x7faa501057c0 2023-09-30T14:31:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.448+0000 7faa55bec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa501057c0 0x7faa500fb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.448+0000 7faa55bec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa501057c0 0x7faa500fb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32806/0 (socket says 172.21.15.32:32806) 2023-09-30T14:31:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.448+0000 7faa55bec700 1 -- 172.21.15.32:0/263009092 learned_addr learned my addr 172.21.15.32:0/263009092 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:36.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.449+0000 7faa55bec700 1 -- 172.21.15.32:0/263009092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa400095f0 con 0x7faa501057c0 2023-09-30T14:31:36.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.449+0000 7faa55bec700 1 --2- 172.21.15.32:0/263009092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa501057c0 0x7faa500fb780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faa40006010 tx=0x7faa40004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:36.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.449+0000 7faa46ffd700 1 -- 172.21.15.32:0/263009092 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa4001d070 con 0x7faa501057c0 2023-09-30T14:31:36.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.449+0000 7faa46ffd700 1 -- 172.21.15.32:0/263009092 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa4000bc10 con 0x7faa501057c0 2023-09-30T14:31:36.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.449+0000 7faa57e50700 1 -- 172.21.15.32:0/263009092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa500fbcc0 con 0x7faa501057c0 2023-09-30T14:31:36.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.449+0000 7faa46ffd700 1 -- 172.21.15.32:0/263009092 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa40003910 con 0x7faa501057c0 2023-09-30T14:31:36.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.449+0000 7faa57e50700 1 -- 172.21.15.32:0/263009092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa500fc140 con 0x7faa501057c0 2023-09-30T14:31:36.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.450+0000 7faa46ffd700 1 -- 172.21.15.32:0/263009092 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45548+0+0 (secure 0 0 0) 0x7faa40003b80 con 0x7faa501057c0 2023-09-30T14:31:36.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.450+0000 7faa57e50700 1 -- 172.21.15.32:0/263009092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa50062450 con 0x7faa501057c0 2023-09-30T14:31:36.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.450+0000 7faa46ffd700 1 --2- 172.21.15.32:0/263009092 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7faa3c038770 0x7faa3c03ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:36.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.450+0000 7faa46ffd700 1 -- 172.21.15.32:0/263009092 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faa4004c4b0 con 0x7faa501057c0 2023-09-30T14:31:36.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.450+0000 7faa553eb700 1 --2- 172.21.15.32:0/263009092 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7faa3c038770 0x7faa3c03ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:36.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.451+0000 7faa553eb700 1 --2- 172.21.15.32:0/263009092 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7faa3c038770 0x7faa3c03ac30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faa4c006fd0 tx=0x7faa4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:36.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.455+0000 7faa46ffd700 1 -- 172.21.15.32:0/263009092 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7faa4000ba70 con 0x7faa501057c0 2023-09-30T14:31:36.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:36.663+0000 7faa57e50700 1 -- 172.21.15.32:0/263009092 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}) v1 -- 0x7faa500fc550 con 0x7faa3c038770 2023-09-30T14:31:37.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:37 smithi032 ceph-mon[98592]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:37.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:37.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:37.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:37.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:37.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:31:37.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:31:37.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:31:38.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:38 smithi032 ceph-mon[98592]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:31:38.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:38 smithi032 ceph-mon[98592]: Updating smithi148:/etc/ceph/ceph.conf 2023-09-30T14:31:38.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:38 smithi032 ceph-mon[98592]: Deploying cephadm binary to smithi156 2023-09-30T14:31:38.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:38 smithi032 ceph-mon[98592]: Updating smithi148:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-09-30T14:31:38.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:38 smithi032 ceph-mon[98592]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2023-09-30T14:31:39.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:39 smithi032 ceph-mon[98592]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:39.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:39 smithi032 ceph-mon[98592]: Updating smithi148:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-30T14:31:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:39.568+0000 7faa46ffd700 1 -- 172.21.15.32:0/263009092 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7faa500fc550 con 0x7faa3c038770 2023-09-30T14:31:39.568 INFO:teuthology.orchestra.run.smithi032.stdout:Added host 'smithi156' with addr '172.21.15.156' 2023-09-30T14:31:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:39.570+0000 7faa57e50700 1 -- 172.21.15.32:0/263009092 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7faa3c038770 msgr2=0x7faa3c03ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:39.570+0000 7faa57e50700 1 --2- 172.21.15.32:0/263009092 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7faa3c038770 0x7faa3c03ac30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faa4c006fd0 tx=0x7faa4c006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:39.570+0000 7faa57e50700 1 -- 172.21.15.32:0/263009092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa501057c0 msgr2=0x7faa500fb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:39.570+0000 7faa57e50700 1 --2- 172.21.15.32:0/263009092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa501057c0 0x7faa500fb780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faa40006010 tx=0x7faa40004ff0 comp rx=0 tx=0).stop 2023-09-30T14:31:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:39.570+0000 7faa57e50700 1 -- 172.21.15.32:0/263009092 shutdown_connections 2023-09-30T14:31:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:39.570+0000 7faa57e50700 1 --2- 172.21.15.32:0/263009092 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7faa3c038770 0x7faa3c03ac30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:39.570+0000 7faa57e50700 1 --2- 172.21.15.32:0/263009092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa501057c0 0x7faa500fb780 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:39.570+0000 7faa57e50700 1 -- 172.21.15.32:0/263009092 >> 172.21.15.32:0/263009092 conn(0x7faa500fb2d0 msgr2=0x7faa500688e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:39.570+0000 7faa57e50700 1 -- 172.21.15.32:0/263009092 shutdown_connections 2023-09-30T14:31:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:39.570+0000 7faa57e50700 1 -- 172.21.15.32:0/263009092 wait complete. 2023-09-30T14:31:39.928 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-09-30T14:31:40.495 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:31:40.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:40.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:40.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:40.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:40.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:31:40.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:41.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.717+0000 7f0d756ce700 1 -- 172.21.15.32:0/2288727756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70101b00 msgr2=0x7f0d70101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:41.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.717+0000 7f0d756ce700 1 --2- 172.21.15.32:0/2288727756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70101b00 0x7f0d70101ee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009910 tx=0x7f0d60009c20 comp rx=0 tx=0).stop 2023-09-30T14:31:41.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.717+0000 7f0d756ce700 1 -- 172.21.15.32:0/2288727756 shutdown_connections 2023-09-30T14:31:41.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.717+0000 7f0d756ce700 1 --2- 172.21.15.32:0/2288727756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70101b00 0x7f0d70101ee0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:41.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.717+0000 7f0d756ce700 1 -- 172.21.15.32:0/2288727756 >> 172.21.15.32:0/2288727756 conn(0x7f0d700fb2f0 msgr2=0x7f0d700fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:41.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.718+0000 7f0d756ce700 1 -- 172.21.15.32:0/2288727756 shutdown_connections 2023-09-30T14:31:41.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.718+0000 7f0d756ce700 1 -- 172.21.15.32:0/2288727756 wait complete. 2023-09-30T14:31:41.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.718+0000 7f0d756ce700 1 Processor -- start 2023-09-30T14:31:41.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.718+0000 7f0d756ce700 1 -- start start 2023-09-30T14:31:41.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.718+0000 7f0d756ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70101b00 0x7f0d70198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:41.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.718+0000 7f0d756ce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d70198570 con 0x7f0d70101b00 2023-09-30T14:31:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.719+0000 7f0d6effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70101b00 0x7f0d70198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.719+0000 7f0d6effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70101b00 0x7f0d70198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32832/0 (socket says 172.21.15.32:32832) 2023-09-30T14:31:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.719+0000 7f0d6effd700 1 -- 172.21.15.32:0/332893392 learned_addr learned my addr 172.21.15.32:0/332893392 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.719+0000 7f0d6effd700 1 -- 172.21.15.32:0/332893392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d600095f0 con 0x7f0d70101b00 2023-09-30T14:31:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.719+0000 7f0d6effd700 1 --2- 172.21.15.32:0/332893392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70101b00 0x7f0d70198030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0d60005850 tx=0x7f0d60004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.719+0000 7f0d57fff700 1 -- 172.21.15.32:0/332893392 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d6001d070 con 0x7f0d70101b00 2023-09-30T14:31:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.719+0000 7f0d756ce700 1 -- 172.21.15.32:0/332893392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d70198770 con 0x7f0d70101b00 2023-09-30T14:31:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.719+0000 7f0d57fff700 1 -- 172.21.15.32:0/332893392 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d60004370 con 0x7f0d70101b00 2023-09-30T14:31:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.719+0000 7f0d57fff700 1 -- 172.21.15.32:0/332893392 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d60003ab0 con 0x7f0d70101b00 2023-09-30T14:31:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.719+0000 7f0d756ce700 1 -- 172.21.15.32:0/332893392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d70198b90 con 0x7f0d70101b00 2023-09-30T14:31:41.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.720+0000 7f0d57fff700 1 -- 172.21.15.32:0/332893392 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f0d6000f460 con 0x7f0d70101b00 2023-09-30T14:31:41.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.720+0000 7f0d756ce700 1 -- 172.21.15.32:0/332893392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d70062450 con 0x7f0d70101b00 2023-09-30T14:31:41.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.720+0000 7f0d57fff700 1 --2- 172.21.15.32:0/332893392 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0d58038720 0x7f0d5803abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:41.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.720+0000 7f0d57fff700 1 -- 172.21.15.32:0/332893392 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0d6004c6c0 con 0x7f0d70101b00 2023-09-30T14:31:41.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.720+0000 7f0d6e7fc700 1 --2- 172.21.15.32:0/332893392 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0d58038720 0x7f0d5803abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:41.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.721+0000 7f0d6e7fc700 1 --2- 172.21.15.32:0/332893392 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0d58038720 0x7f0d5803abe0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0d64006fd0 tx=0x7f0d64006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:41.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.725+0000 7f0d57fff700 1 -- 172.21.15.32:0/332893392 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f0d60029950 con 0x7f0d70101b00 2023-09-30T14:31:41.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:41 smithi032 ceph-mon[98592]: Added host smithi156 2023-09-30T14:31:41.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:41 smithi032 ceph-mon[98592]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:41.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:41 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:41.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.930+0000 7f0d756ce700 1 -- 172.21.15.32:0/332893392 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d7010ad10 con 0x7f0d58038720 2023-09-30T14:31:41.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.931+0000 7f0d57fff700 1 -- 172.21.15.32:0/332893392 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f0d7010ad10 con 0x7f0d58038720 2023-09-30T14:31:41.931 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:31:41.932 INFO:teuthology.orchestra.run.smithi032.stdout:[{"addr": "172.21.15.32", "hostname": "smithi032", "labels": [], "status": ""}, {"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}, {"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}] 2023-09-30T14:31:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.933+0000 7f0d756ce700 1 -- 172.21.15.32:0/332893392 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0d58038720 msgr2=0x7f0d5803abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.933+0000 7f0d756ce700 1 --2- 172.21.15.32:0/332893392 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0d58038720 0x7f0d5803abe0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0d64006fd0 tx=0x7f0d64006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.933+0000 7f0d756ce700 1 -- 172.21.15.32:0/332893392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70101b00 msgr2=0x7f0d70198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.933+0000 7f0d756ce700 1 --2- 172.21.15.32:0/332893392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70101b00 0x7f0d70198030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0d60005850 tx=0x7f0d60004ff0 comp rx=0 tx=0).stop 2023-09-30T14:31:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.933+0000 7f0d756ce700 1 -- 172.21.15.32:0/332893392 shutdown_connections 2023-09-30T14:31:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.933+0000 7f0d756ce700 1 --2- 172.21.15.32:0/332893392 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0d58038720 0x7f0d5803abe0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.933+0000 7f0d756ce700 1 --2- 172.21.15.32:0/332893392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70101b00 0x7f0d70198030 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.933+0000 7f0d756ce700 1 -- 172.21.15.32:0/332893392 >> 172.21.15.32:0/332893392 conn(0x7f0d700fb2f0 msgr2=0x7f0d70107550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.934+0000 7f0d756ce700 1 -- 172.21.15.32:0/332893392 shutdown_connections 2023-09-30T14:31:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:41.934+0000 7f0d756ce700 1 -- 172.21.15.32:0/332893392 wait complete. 2023-09-30T14:31:42.354 INFO:tasks.cephadm:Setting crush tunables to default 2023-09-30T14:31:42.354 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd crush tunables default 2023-09-30T14:31:42.917 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:31:43.330 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:43 smithi032 ceph-mon[98592]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:43.330 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:43 smithi032 ceph-mon[98592]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:31:43.330 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:43 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:44.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.356+0000 7f87e93b7700 1 -- 172.21.15.32:0/2362078331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e41057c0 msgr2=0x7f87e4105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:44.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.356+0000 7f87e93b7700 1 --2- 172.21.15.32:0/2362078331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e41057c0 0x7f87e4105ba0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009910 tx=0x7f87d8009c20 comp rx=0 tx=0).stop 2023-09-30T14:31:44.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.356+0000 7f87e93b7700 1 -- 172.21.15.32:0/2362078331 shutdown_connections 2023-09-30T14:31:44.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.356+0000 7f87e93b7700 1 --2- 172.21.15.32:0/2362078331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e41057c0 0x7f87e4105ba0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:44.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.356+0000 7f87e93b7700 1 -- 172.21.15.32:0/2362078331 >> 172.21.15.32:0/2362078331 conn(0x7f87e40fb2d0 msgr2=0x7f87e40fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:44.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.356+0000 7f87e93b7700 1 -- 172.21.15.32:0/2362078331 shutdown_connections 2023-09-30T14:31:44.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.356+0000 7f87e93b7700 1 -- 172.21.15.32:0/2362078331 wait complete. 2023-09-30T14:31:44.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.356+0000 7f87e93b7700 1 Processor -- start 2023-09-30T14:31:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.357+0000 7f87e93b7700 1 -- start start 2023-09-30T14:31:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.357+0000 7f87e93b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e41057c0 0x7f87e4199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.357+0000 7f87e93b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e4199ea0 con 0x7f87e41057c0 2023-09-30T14:31:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.357+0000 7f87e2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e41057c0 0x7f87e4199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.357+0000 7f87e2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e41057c0 0x7f87e4199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32838/0 (socket says 172.21.15.32:32838) 2023-09-30T14:31:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.357+0000 7f87e2ffd700 1 -- 172.21.15.32:0/2219430363 learned_addr learned my addr 172.21.15.32:0/2219430363 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:31:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.357+0000 7f87e2ffd700 1 -- 172.21.15.32:0/2219430363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d80095f0 con 0x7f87e41057c0 2023-09-30T14:31:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.357+0000 7f87e2ffd700 1 --2- 172.21.15.32:0/2219430363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e41057c0 0x7f87e4199960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f87d8006010 tx=0x7f87d8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.358+0000 7f87cbfff700 1 -- 172.21.15.32:0/2219430363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87d801d070 con 0x7f87e41057c0 2023-09-30T14:31:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.358+0000 7f87cbfff700 1 -- 172.21.15.32:0/2219430363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87d800bbb0 con 0x7f87e41057c0 2023-09-30T14:31:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.358+0000 7f87e93b7700 1 -- 172.21.15.32:0/2219430363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e419a100 con 0x7f87e41057c0 2023-09-30T14:31:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.358+0000 7f87cbfff700 1 -- 172.21.15.32:0/2219430363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87d8003930 con 0x7f87e41057c0 2023-09-30T14:31:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.358+0000 7f87e93b7700 1 -- 172.21.15.32:0/2219430363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e41939e0 con 0x7f87e41057c0 2023-09-30T14:31:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.358+0000 7f87cbfff700 1 -- 172.21.15.32:0/2219430363 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f87d8003b20 con 0x7f87e41057c0 2023-09-30T14:31:44.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.359+0000 7f87cbfff700 1 --2- 172.21.15.32:0/2219430363 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f87cc03a9b0 0x7f87cc03ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:44.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.359+0000 7f87e93b7700 1 -- 172.21.15.32:0/2219430363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e4062450 con 0x7f87e41057c0 2023-09-30T14:31:44.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.359+0000 7f87cbfff700 1 -- 172.21.15.32:0/2219430363 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f87d804c390 con 0x7f87e41057c0 2023-09-30T14:31:44.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.359+0000 7f87e27fc700 1 --2- 172.21.15.32:0/2219430363 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f87cc03a9b0 0x7f87cc03ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:44.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.359+0000 7f87e27fc700 1 --2- 172.21.15.32:0/2219430363 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f87cc03a9b0 0x7f87cc03ce70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f87d4006fd0 tx=0x7f87d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:44.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.364+0000 7f87cbfff700 1 -- 172.21.15.32:0/2219430363 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f87d8026020 con 0x7f87e41057c0 2023-09-30T14:31:44.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:44.571+0000 7f87e93b7700 1 -- 172.21.15.32:0/2219430363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f87e4194ef0 con 0x7f87e41057c0 2023-09-30T14:31:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:45.006+0000 7f87cbfff700 1 -- 172.21.15.32:0/2219430363 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f87d802b440 con 0x7f87e41057c0 2023-09-30T14:31:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:45.009+0000 7f87e93b7700 1 -- 172.21.15.32:0/2219430363 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f87cc03a9b0 msgr2=0x7f87cc03ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:45.009+0000 7f87e93b7700 1 --2- 172.21.15.32:0/2219430363 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f87cc03a9b0 0x7f87cc03ce70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f87d4006fd0 tx=0x7f87d4006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:45.009+0000 7f87e93b7700 1 -- 172.21.15.32:0/2219430363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e41057c0 msgr2=0x7f87e4199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:45.009+0000 7f87e93b7700 1 --2- 172.21.15.32:0/2219430363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e41057c0 0x7f87e4199960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f87d8006010 tx=0x7f87d8004ff0 comp rx=0 tx=0).stop 2023-09-30T14:31:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:45.009+0000 7f87e93b7700 1 -- 172.21.15.32:0/2219430363 shutdown_connections 2023-09-30T14:31:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:45.009+0000 7f87e93b7700 1 --2- 172.21.15.32:0/2219430363 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f87cc03a9b0 0x7f87cc03ce70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:45.009+0000 7f87e93b7700 1 --2- 172.21.15.32:0/2219430363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e41057c0 0x7f87e4199960 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:45.009+0000 7f87e93b7700 1 -- 172.21.15.32:0/2219430363 >> 172.21.15.32:0/2219430363 conn(0x7f87e40fb2d0 msgr2=0x7f87e4107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:45.010+0000 7f87e93b7700 1 -- 172.21.15.32:0/2219430363 shutdown_connections 2023-09-30T14:31:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:31:45.010+0000 7f87e93b7700 1 -- 172.21.15.32:0/2219430363 wait complete. 2023-09-30T14:31:45.011 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2023-09-30T14:31:45.366 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:45 smithi032 ceph-mon[98592]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:45.366 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:45 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2219430363' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-30T14:31:45.366 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:45 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:45.366 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:45 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:45.366 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:45 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:45.367 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:45 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:45.367 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:45 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:31:45.367 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:45 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:31:45.367 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:45 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:31:45.376 INFO:tasks.cephadm:Adding mon.a on smithi032 2023-09-30T14:31:45.376 INFO:tasks.cephadm:Adding mon.b on smithi148 2023-09-30T14:31:45.376 INFO:tasks.cephadm:Adding mon.c on smithi156 2023-09-30T14:31:45.377 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch apply mon '3;smithi032:172.21.15.32=a;smithi148:172.21.15.148=b;smithi156:172.21.15.156=c' 2023-09-30T14:31:45.964 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-09-30T14:31:46.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:46 smithi032 ceph-mon[98592]: Updating smithi156:/etc/ceph/ceph.conf 2023-09-30T14:31:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:46 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2219430363' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-30T14:31:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:46 smithi032 ceph-mon[98592]: osdmap e4: 0 total, 0 up, 0 in 2023-09-30T14:31:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:46 smithi032 ceph-mon[98592]: Updating smithi156:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-09-30T14:31:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:46 smithi032 ceph-mon[98592]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-09-30T14:31:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:46 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:46 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:46.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:46 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:47.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.179+0000 7fe888612700 1 -- 172.21.15.156:0/1606212648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8801057d0 msgr2=0x7fe880105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:47.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.179+0000 7fe888612700 1 --2- 172.21.15.156:0/1606212648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8801057d0 0x7fe880105bb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe870009910 tx=0x7fe870009c20 comp rx=0 tx=0).stop 2023-09-30T14:31:47.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.179+0000 7fe888612700 1 -- 172.21.15.156:0/1606212648 shutdown_connections 2023-09-30T14:31:47.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.179+0000 7fe888612700 1 --2- 172.21.15.156:0/1606212648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8801057d0 0x7fe880105bb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:47.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.179+0000 7fe888612700 1 -- 172.21.15.156:0/1606212648 >> 172.21.15.156:0/1606212648 conn(0x7fe8800752e0 msgr2=0x7fe8800756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:47.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.179+0000 7fe888612700 1 -- 172.21.15.156:0/1606212648 shutdown_connections 2023-09-30T14:31:47.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.179+0000 7fe888612700 1 -- 172.21.15.156:0/1606212648 wait complete. 2023-09-30T14:31:47.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.180+0000 7fe888612700 1 Processor -- start 2023-09-30T14:31:47.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.180+0000 7fe888612700 1 -- start start 2023-09-30T14:31:47.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.180+0000 7fe888612700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8801057d0 0x7fe88019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:47.183 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.180+0000 7fe888612700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88019b000 con 0x7fe8801057d0 2023-09-30T14:31:47.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.181+0000 7fe8863ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8801057d0 0x7fe88019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:47.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.181+0000 7fe8863ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8801057d0 0x7fe88019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.156:56600/0 (socket says 172.21.15.156:56600) 2023-09-30T14:31:47.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.181+0000 7fe8863ae700 1 -- 172.21.15.156:0/1872918341 learned_addr learned my addr 172.21.15.156:0/1872918341 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-30T14:31:47.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.181+0000 7fe8863ae700 1 -- 172.21.15.156:0/1872918341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8700095f0 con 0x7fe8801057d0 2023-09-30T14:31:47.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.181+0000 7fe8863ae700 1 --2- 172.21.15.156:0/1872918341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8801057d0 0x7fe88019aac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe870006010 tx=0x7fe870004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:47.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.181+0000 7fe8777fe700 1 -- 172.21.15.156:0/1872918341 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe87001d070 con 0x7fe8801057d0 2023-09-30T14:31:47.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.181+0000 7fe888612700 1 -- 172.21.15.156:0/1872918341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88019b260 con 0x7fe8801057d0 2023-09-30T14:31:47.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.181+0000 7fe8777fe700 1 -- 172.21.15.156:0/1872918341 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe87000bbb0 con 0x7fe8801057d0 2023-09-30T14:31:47.184 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.181+0000 7fe8777fe700 1 -- 172.21.15.156:0/1872918341 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe870003930 con 0x7fe8801057d0 2023-09-30T14:31:47.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.181+0000 7fe888612700 1 -- 172.21.15.156:0/1872918341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe880194ab0 con 0x7fe8801057d0 2023-09-30T14:31:47.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.182+0000 7fe8777fe700 1 -- 172.21.15.156:0/1872918341 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fe87000bd20 con 0x7fe8801057d0 2023-09-30T14:31:47.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.182+0000 7fe888612700 1 -- 172.21.15.156:0/1872918341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe880062450 con 0x7fe8801057d0 2023-09-30T14:31:47.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.182+0000 7fe8777fe700 1 --2- 172.21.15.156:0/1872918341 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe86c038720 0x7fe86c03abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:47.185 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.182+0000 7fe8777fe700 1 -- 172.21.15.156:0/1872918341 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe87004c4c0 con 0x7fe8801057d0 2023-09-30T14:31:47.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.183+0000 7fe885bad700 1 --2- 172.21.15.156:0/1872918341 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe86c038720 0x7fe86c03abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:47.186 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.183+0000 7fe885bad700 1 --2- 172.21.15.156:0/1872918341 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe86c038720 0x7fe86c03abe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe87c006fd0 tx=0x7fe87c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:47.189 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.188+0000 7fe8777fe700 1 -- 172.21.15.156:0/1872918341 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fe870029360 con 0x7fe8801057d0 2023-09-30T14:31:47.386 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.385+0000 7fe888612700 1 -- 172.21.15.156:0/1872918341 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi032:172.21.15.32=a;smithi148:172.21.15.148=b;smithi156:172.21.15.156=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fe880109cc0 con 0x7fe86c038720 2023-09-30T14:31:47.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:47 smithi032 ceph-mon[98592]: Updating smithi156:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-30T14:31:47.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:47 smithi032 ceph-mon[98592]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:47.393 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.392+0000 7fe8777fe700 1 -- 172.21.15.156:0/1872918341 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe880109cc0 con 0x7fe86c038720 2023-09-30T14:31:47.393 INFO:teuthology.orchestra.run.smithi156.stdout:Scheduled mon update... 2023-09-30T14:31:47.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.394+0000 7fe888612700 1 -- 172.21.15.156:0/1872918341 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe86c038720 msgr2=0x7fe86c03abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:47.395 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.394+0000 7fe888612700 1 --2- 172.21.15.156:0/1872918341 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe86c038720 0x7fe86c03abe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe87c006fd0 tx=0x7fe87c006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:47.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.394+0000 7fe888612700 1 -- 172.21.15.156:0/1872918341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8801057d0 msgr2=0x7fe88019aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:47.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.394+0000 7fe888612700 1 --2- 172.21.15.156:0/1872918341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8801057d0 0x7fe88019aac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe870006010 tx=0x7fe870004ff0 comp rx=0 tx=0).stop 2023-09-30T14:31:47.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.394+0000 7fe888612700 1 -- 172.21.15.156:0/1872918341 shutdown_connections 2023-09-30T14:31:47.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.394+0000 7fe888612700 1 --2- 172.21.15.156:0/1872918341 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe86c038720 0x7fe86c03abe0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:47.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.394+0000 7fe888612700 1 --2- 172.21.15.156:0/1872918341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8801057d0 0x7fe88019aac0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:47.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.394+0000 7fe888612700 1 -- 172.21.15.156:0/1872918341 >> 172.21.15.156:0/1872918341 conn(0x7fe8800752e0 msgr2=0x7fe8800fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:47.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.395+0000 7fe888612700 1 -- 172.21.15.156:0/1872918341 shutdown_connections 2023-09-30T14:31:47.396 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:47.395+0000 7fe888612700 1 -- 172.21.15.156:0/1872918341 wait complete. 2023-09-30T14:31:47.829 DEBUG:teuthology.orchestra.run.smithi148:mon.b> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mon.b.service 2023-09-30T14:31:47.831 DEBUG:teuthology.orchestra.run.smithi156:mon.c> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mon.c.service 2023-09-30T14:31:47.833 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-30T14:31:47.834 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-09-30T14:31:47.863 INFO:journalctl@ceph.mon.c.smithi156.stdout:-- Logs begin at Sat 2023-09-30 14:16:23 UTC. -- 2023-09-30T14:31:47.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:-- Logs begin at Sat 2023-09-30 14:16:26 UTC. -- 2023-09-30T14:31:48.762 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-09-30T14:31:48.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:48 smithi032 ceph-mon[98592]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi032:172.21.15.32=a;smithi148:172.21.15.148=b;smithi156:172.21.15.156=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:31:48.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:48 smithi032 ceph-mon[98592]: Saving service mon spec with placement smithi032:172.21.15.32=a;smithi148:172.21.15.148=b;smithi156:172.21.15.156=c;count:3 2023-09-30T14:31:48.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:48.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:31:48.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:31:48.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:31:48.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:48.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-30T14:31:48.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:31:48.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:48 smithi032 ceph-mon[98592]: Deploying daemon mon.c on smithi156 2023-09-30T14:31:49.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:49 smithi032 ceph-mon[98592]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:50.758 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.756+0000 7f600e57f700 1 -- 172.21.15.156:0/4291381725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6008101ae0 msgr2=0x7f6008101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:50.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.756+0000 7f600e57f700 1 --2- 172.21.15.156:0/4291381725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6008101ae0 0x7f6008101ec0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009910 tx=0x7f5ff8009c20 comp rx=0 tx=0).stop 2023-09-30T14:31:50.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.756+0000 7f600e57f700 1 -- 172.21.15.156:0/4291381725 shutdown_connections 2023-09-30T14:31:50.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.756+0000 7f600e57f700 1 --2- 172.21.15.156:0/4291381725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6008101ae0 0x7f6008101ec0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:50.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.756+0000 7f600e57f700 1 -- 172.21.15.156:0/4291381725 >> 172.21.15.156:0/4291381725 conn(0x7f60080fb2d0 msgr2=0x7f60080fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:50.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.756+0000 7f600e57f700 1 -- 172.21.15.156:0/4291381725 shutdown_connections 2023-09-30T14:31:50.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.756+0000 7f600e57f700 1 -- 172.21.15.156:0/4291381725 wait complete. 2023-09-30T14:31:50.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.757+0000 7f600e57f700 1 Processor -- start 2023-09-30T14:31:50.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.757+0000 7f600e57f700 1 -- start start 2023-09-30T14:31:50.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.757+0000 7f600e57f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6008101ae0 0x7f60080ff740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:50.759 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.757+0000 7f600e57f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60081016c0 con 0x7f6008101ae0 2023-09-30T14:31:50.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.758+0000 7f6007fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6008101ae0 0x7f60080ff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:50.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.758+0000 7f6007fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6008101ae0 0x7f60080ff740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.156:56606/0 (socket says 172.21.15.156:56606) 2023-09-30T14:31:50.760 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.758+0000 7f6007fff700 1 -- 172.21.15.156:0/2057463695 learned_addr learned my addr 172.21.15.156:0/2057463695 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-30T14:31:50.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.758+0000 7f6007fff700 1 -- 172.21.15.156:0/2057463695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff80095f0 con 0x7f6008101ae0 2023-09-30T14:31:50.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.758+0000 7f6007fff700 1 --2- 172.21.15.156:0/2057463695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6008101ae0 0x7f60080ff740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8005850 tx=0x7f5ff8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:50.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.758+0000 7f60057fa700 1 -- 172.21.15.156:0/2057463695 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ff801d070 con 0x7f6008101ae0 2023-09-30T14:31:50.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.758+0000 7f600e57f700 1 -- 172.21.15.156:0/2057463695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60080ffc80 con 0x7f6008101ae0 2023-09-30T14:31:50.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.758+0000 7f60057fa700 1 -- 172.21.15.156:0/2057463695 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ff8004370 con 0x7f6008101ae0 2023-09-30T14:31:50.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.758+0000 7f60057fa700 1 -- 172.21.15.156:0/2057463695 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ff8003ab0 con 0x7f6008101ae0 2023-09-30T14:31:50.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.758+0000 7f600e57f700 1 -- 172.21.15.156:0/2057463695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6008100100 con 0x7f6008101ae0 2023-09-30T14:31:50.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.759+0000 7f60057fa700 1 -- 172.21.15.156:0/2057463695 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f5ff800f460 con 0x7f6008101ae0 2023-09-30T14:31:50.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.760+0000 7f60057fa700 1 --2- 172.21.15.156:0/2057463695 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5ff0038720 0x7f5ff003abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:50.761 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.760+0000 7f60057fa700 1 -- 172.21.15.156:0/2057463695 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5ff804c520 con 0x7f6008101ae0 2023-09-30T14:31:50.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.760+0000 7f600e57f700 1 -- 172.21.15.156:0/2057463695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6008107ff0 con 0x7f6008101ae0 2023-09-30T14:31:50.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.760+0000 7f60077fe700 1 --2- 172.21.15.156:0/2057463695 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5ff0038720 0x7f5ff003abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:50.762 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.760+0000 7f60077fe700 1 --2- 172.21.15.156:0/2057463695 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5ff0038720 0x7f5ff003abe0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc006fd0 tx=0x7f5ffc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:50.766 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:50.765+0000 7f60057fa700 1 -- 172.21.15.156:0/2057463695 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f5ff8026020 con 0x7f6008101ae0 2023-09-30T14:31:50.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:50 smithi156 systemd[1]: Starting Ceph mon.c for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:31:51.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:51.052+0000 7f600e57f700 1 -- 172.21.15.156:0/2057463695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6008062450 con 0x7f6008101ae0 2023-09-30T14:31:51.054 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:51.053+0000 7f60057fa700 1 -- 172.21.15.156:0/2057463695 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f5ff8029950 con 0x7f6008101ae0 2023-09-30T14:31:51.054 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-30T14:31:51.054 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":1,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","modified":"2023-09-30T14:30:08.312183Z","created":"2023-09-30T14:30:08.312183Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-09-30T14:31:51.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:51.055+0000 7f600e57f700 1 -- 172.21.15.156:0/2057463695 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5ff0038720 msgr2=0x7f5ff003abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:51.056 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:51.055+0000 7f600e57f700 1 --2- 172.21.15.156:0/2057463695 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5ff0038720 0x7f5ff003abe0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc006fd0 tx=0x7f5ffc006e40 comp rx=0 tx=0).stop 2023-09-30T14:31:51.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:51.055+0000 7f600e57f700 1 -- 172.21.15.156:0/2057463695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6008101ae0 msgr2=0x7f60080ff740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:51.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:51.055+0000 7f600e57f700 1 --2- 172.21.15.156:0/2057463695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6008101ae0 0x7f60080ff740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8005850 tx=0x7f5ff8004ff0 comp rx=0 tx=0).stop 2023-09-30T14:31:51.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:51.055+0000 7f600e57f700 1 -- 172.21.15.156:0/2057463695 shutdown_connections 2023-09-30T14:31:51.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:51.055+0000 7f600e57f700 1 --2- 172.21.15.156:0/2057463695 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5ff0038720 0x7f5ff003abe0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:51.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:51.055+0000 7f600e57f700 1 --2- 172.21.15.156:0/2057463695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6008101ae0 0x7f60080ff740 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:51.057 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:51.055+0000 7f600e57f700 1 -- 172.21.15.156:0/2057463695 >> 172.21.15.156:0/2057463695 conn(0x7f60080fb2d0 msgr2=0x7f6008105130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:51.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:51.055+0000 7f600e57f700 1 -- 172.21.15.156:0/2057463695 shutdown_connections 2023-09-30T14:31:51.058 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:51.055+0000 7f600e57f700 1 -- 172.21.15.156:0/2057463695 wait complete. 2023-09-30T14:31:51.058 INFO:teuthology.orchestra.run.smithi156.stderr:dumped monmap epoch 1 2023-09-30T14:31:51.250 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:50 smithi156 podman[103517]: 2023-09-30T14:31:51.250 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: set uid:gid to 167:167 (ceph:ceph) 2023-09-30T14:31:51.250 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: ceph version 18.0.0-6447-g1d10b717 (1d10b71792f3be8887a7631e69851ac2df3585af) reef (dev), process ceph-mon, pid 7 2023-09-30T14:31:51.250 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 bash[103517]: c4a4135d3eba2942949b337a2d3cbe25137aefa67c6dea50459b47ddbb52cc5c 2023-09-30T14:31:51.250 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: pidfile_write: ignore empty --pid-file 2023-09-30T14:31:51.250 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: load: jerasure load: lrc 2023-09-30T14:31:51.250 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: RocksDB version: 7.9.2 2023-09-30T14:31:51.250 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Git sha 0 2023-09-30T14:31:51.250 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Compile date 2023-09-29 16:35:50 2023-09-30T14:31:51.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: DB SUMMARY 2023-09-30T14:31:51.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: DB Session ID: 9E6BW6OAU2IMY21M8RKQ 2023-09-30T14:31:51.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: CURRENT file: CURRENT 2023-09-30T14:31:51.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: IDENTITY file: IDENTITY 2023-09-30T14:31:51.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-09-30T14:31:51.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-09-30T14:31:51.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-09-30T14:31:51.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.error_if_exists: 0 2023-09-30T14:31:51.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.create_if_missing: 0 2023-09-30T14:31:51.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.paranoid_checks: 1 2023-09-30T14:31:51.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-30T14:31:51.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-30T14:31:51.252 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-30T14:31:51.252 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.env: 0x55b9aa66cda0 2023-09-30T14:31:51.252 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.fs: PosixFileSystem 2023-09-30T14:31:51.252 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.info_log: 0x55b9abfb35a0 2023-09-30T14:31:51.252 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_file_opening_threads: 16 2023-09-30T14:31:51.252 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.statistics: (nil) 2023-09-30T14:31:51.252 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.use_fsync: 0 2023-09-30T14:31:51.252 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_log_file_size: 0 2023-09-30T14:31:51.252 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-30T14:31:51.252 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-30T14:31:51.252 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.keep_log_file_num: 1000 2023-09-30T14:31:51.252 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.recycle_log_file_num: 0 2023-09-30T14:31:51.252 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.allow_fallocate: 1 2023-09-30T14:31:51.253 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.allow_mmap_reads: 0 2023-09-30T14:31:51.253 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.allow_mmap_writes: 0 2023-09-30T14:31:51.253 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.use_direct_reads: 0 2023-09-30T14:31:51.253 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-30T14:31:51.253 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.create_missing_column_families: 0 2023-09-30T14:31:51.253 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.db_log_dir: 2023-09-30T14:31:51.255 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.wal_dir: 2023-09-30T14:31:51.255 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-30T14:31:51.255 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-30T14:31:51.255 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-30T14:31:51.255 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-30T14:31:51.256 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-30T14:31:51.256 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-30T14:31:51.256 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.advise_random_on_open: 1 2023-09-30T14:31:51.256 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.db_write_buffer_size: 0 2023-09-30T14:31:51.256 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.write_buffer_manager: 0x55b9abf6b360 2023-09-30T14:31:51.256 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-30T14:31:51.256 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-30T14:31:51.256 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-30T14:31:51.256 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.rate_limiter: (nil) 2023-09-30T14:31:51.257 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-30T14:31:51.257 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.wal_recovery_mode: 2 2023-09-30T14:31:51.257 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.enable_thread_tracking: 0 2023-09-30T14:31:51.257 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.enable_pipelined_write: 0 2023-09-30T14:31:51.257 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.unordered_write: 0 2023-09-30T14:31:51.257 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-30T14:31:51.257 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-30T14:31:51.257 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-30T14:31:51.257 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-30T14:31:51.257 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.row_cache: None 2023-09-30T14:31:51.257 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.wal_filter: None 2023-09-30T14:31:51.257 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-30T14:31:51.258 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.allow_ingest_behind: 0 2023-09-30T14:31:51.258 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.two_write_queues: 0 2023-09-30T14:31:51.258 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.manual_wal_flush: 0 2023-09-30T14:31:51.258 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.wal_compression: 0 2023-09-30T14:31:51.258 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.atomic_flush: 0 2023-09-30T14:31:51.258 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-30T14:31:51.258 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-30T14:31:51.258 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-30T14:31:51.258 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.log_readahead_size: 0 2023-09-30T14:31:51.258 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-30T14:31:51.258 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.best_efforts_recovery: 0 2023-09-30T14:31:51.259 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-30T14:31:51.259 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-30T14:31:51.259 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.allow_data_in_errors: 0 2023-09-30T14:31:51.259 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.db_host_id: __hostname__ 2023-09-30T14:31:51.259 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-30T14:31:51.259 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_background_jobs: 2 2023-09-30T14:31:51.259 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_background_compactions: -1 2023-09-30T14:31:51.259 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_subcompactions: 1 2023-09-30T14:31:51.259 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-30T14:31:51.260 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-30T14:31:51.260 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-30T14:31:51.260 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_total_wal_size: 0 2023-09-30T14:31:51.260 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-30T14:31:51.260 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-30T14:31:51.260 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-30T14:31:51.260 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-30T14:31:51.260 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_open_files: -1 2023-09-30T14:31:51.260 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.bytes_per_sync: 0 2023-09-30T14:31:51.260 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-30T14:31:51.260 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-30T14:31:51.260 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compaction_readahead_size: 0 2023-09-30T14:31:51.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_background_flushes: -1 2023-09-30T14:31:51.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Compression algorithms supported: 2023-09-30T14:31:51.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-30T14:31:51.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: kZSTD supported: 0 2023-09-30T14:31:51.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: kXpressCompression supported: 0 2023-09-30T14:31:51.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: kLZ4HCCompression supported: 1 2023-09-30T14:31:51.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: kZlibCompression supported: 1 2023-09-30T14:31:51.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: kSnappyCompression supported: 1 2023-09-30T14:31:51.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: kLZ4Compression supported: 1 2023-09-30T14:31:51.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: kBZip2Compression supported: 0 2023-09-30T14:31:51.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-30T14:31:51.261 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-30T14:31:51.262 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-09-30T14:31:51.262 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-30T14:31:51.262 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-30T14:31:51.262 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.merge_operator: 2023-09-30T14:31:51.262 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compaction_filter: None 2023-09-30T14:31:51.262 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compaction_filter_factory: None 2023-09-30T14:31:51.262 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.sst_partitioner_factory: None 2023-09-30T14:31:51.262 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-30T14:31:51.262 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-30T14:31:51.262 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b9abfb36a0) 2023-09-30T14:31:51.262 INFO:journalctl@ceph.mon.c.smithi156.stdout: cache_index_and_filter_blocks: 1 2023-09-30T14:31:51.263 INFO:journalctl@ceph.mon.c.smithi156.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-30T14:31:51.263 INFO:journalctl@ceph.mon.c.smithi156.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-30T14:31:51.263 INFO:journalctl@ceph.mon.c.smithi156.stdout: pin_top_level_index_and_filter: 1 2023-09-30T14:31:51.263 INFO:journalctl@ceph.mon.c.smithi156.stdout: index_type: 0 2023-09-30T14:31:51.263 INFO:journalctl@ceph.mon.c.smithi156.stdout: data_block_index_type: 0 2023-09-30T14:31:51.263 INFO:journalctl@ceph.mon.c.smithi156.stdout: index_shortening: 1 2023-09-30T14:31:51.263 INFO:journalctl@ceph.mon.c.smithi156.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-30T14:31:51.263 INFO:journalctl@ceph.mon.c.smithi156.stdout: checksum: 4 2023-09-30T14:31:51.263 INFO:journalctl@ceph.mon.c.smithi156.stdout: no_block_cache: 0 2023-09-30T14:31:51.263 INFO:journalctl@ceph.mon.c.smithi156.stdout: block_cache: 0x55b9ac0091f0 2023-09-30T14:31:51.263 INFO:journalctl@ceph.mon.c.smithi156.stdout: block_cache_name: BinnedLRUCache 2023-09-30T14:31:51.263 INFO:journalctl@ceph.mon.c.smithi156.stdout: block_cache_options: 2023-09-30T14:31:51.264 INFO:journalctl@ceph.mon.c.smithi156.stdout: capacity : 536870912 2023-09-30T14:31:51.264 INFO:journalctl@ceph.mon.c.smithi156.stdout: num_shard_bits : 4 2023-09-30T14:31:51.264 INFO:journalctl@ceph.mon.c.smithi156.stdout: strict_capacity_limit : 0 2023-09-30T14:31:51.264 INFO:journalctl@ceph.mon.c.smithi156.stdout: high_pri_pool_ratio: 0.000 2023-09-30T14:31:51.264 INFO:journalctl@ceph.mon.c.smithi156.stdout: block_cache_compressed: (nil) 2023-09-30T14:31:51.264 INFO:journalctl@ceph.mon.c.smithi156.stdout: persistent_cache: (nil) 2023-09-30T14:31:51.264 INFO:journalctl@ceph.mon.c.smithi156.stdout: block_size: 4096 2023-09-30T14:31:51.264 INFO:journalctl@ceph.mon.c.smithi156.stdout: block_size_deviation: 10 2023-09-30T14:31:51.264 INFO:journalctl@ceph.mon.c.smithi156.stdout: block_restart_interval: 16 2023-09-30T14:31:51.264 INFO:journalctl@ceph.mon.c.smithi156.stdout: index_block_restart_interval: 1 2023-09-30T14:31:51.264 INFO:journalctl@ceph.mon.c.smithi156.stdout: metadata_block_size: 4096 2023-09-30T14:31:51.264 INFO:journalctl@ceph.mon.c.smithi156.stdout: partition_filters: 0 2023-09-30T14:31:51.265 INFO:journalctl@ceph.mon.c.smithi156.stdout: use_delta_encoding: 1 2023-09-30T14:31:51.265 INFO:journalctl@ceph.mon.c.smithi156.stdout: filter_policy: bloomfilter 2023-09-30T14:31:51.265 INFO:journalctl@ceph.mon.c.smithi156.stdout: whole_key_filtering: 1 2023-09-30T14:31:51.265 INFO:journalctl@ceph.mon.c.smithi156.stdout: verify_compression: 0 2023-09-30T14:31:51.265 INFO:journalctl@ceph.mon.c.smithi156.stdout: read_amp_bytes_per_bit: 0 2023-09-30T14:31:51.265 INFO:journalctl@ceph.mon.c.smithi156.stdout: format_version: 5 2023-09-30T14:31:51.265 INFO:journalctl@ceph.mon.c.smithi156.stdout: enable_index_compression: 1 2023-09-30T14:31:51.265 INFO:journalctl@ceph.mon.c.smithi156.stdout: block_align: 0 2023-09-30T14:31:51.265 INFO:journalctl@ceph.mon.c.smithi156.stdout: max_auto_readahead_size: 262144 2023-09-30T14:31:51.265 INFO:journalctl@ceph.mon.c.smithi156.stdout: prepopulate_block_cache: 0 2023-09-30T14:31:51.265 INFO:journalctl@ceph.mon.c.smithi156.stdout: initial_auto_readahead_size: 8192 2023-09-30T14:31:51.265 INFO:journalctl@ceph.mon.c.smithi156.stdout: num_file_reads_for_auto_readahead: 2 2023-09-30T14:31:51.265 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.write_buffer_size: 33554432 2023-09-30T14:31:51.266 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_write_buffer_number: 2 2023-09-30T14:31:51.266 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compression: NoCompression 2023-09-30T14:31:51.266 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.bottommost_compression: Disabled 2023-09-30T14:31:51.266 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.prefix_extractor: nullptr 2023-09-30T14:31:51.266 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-30T14:31:51.266 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.num_levels: 7 2023-09-30T14:31:51.266 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-30T14:31:51.266 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-30T14:31:51.266 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-30T14:31:51.266 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-30T14:31:51.266 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-30T14:31:51.267 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-30T14:31:51.267 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-30T14:31:51.267 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-30T14:31:51.267 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-30T14:31:51.267 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-30T14:31:51.267 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-30T14:31:51.267 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-30T14:31:51.267 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-30T14:31:51.267 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compression_opts.level: 32767 2023-09-30T14:31:51.267 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compression_opts.strategy: 0 2023-09-30T14:31:51.267 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-30T14:31:51.267 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-30T14:31:51.268 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-30T14:31:51.268 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-30T14:31:51.268 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compression_opts.enabled: false 2023-09-30T14:31:51.268 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-30T14:31:51.268 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-30T14:31:51.268 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-30T14:31:51.268 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-30T14:31:51.268 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.target_file_size_base: 67108864 2023-09-30T14:31:51.268 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-30T14:31:51.268 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-30T14:31:51.268 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-30T14:31:51.268 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-30T14:31:51.269 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-30T14:31:51.269 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-30T14:31:51.269 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-30T14:31:51.269 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-30T14:31:51.269 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-30T14:31:51.269 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-30T14:31:51.269 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-30T14:31:51.269 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-30T14:31:51.271 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-30T14:31:51.271 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-30T14:31:51.271 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.arena_block_size: 1048576 2023-09-30T14:31:51.273 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-30T14:31:51.273 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-30T14:31:51.273 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.disable_auto_compactions: 0 2023-09-30T14:31:51.273 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-30T14:31:51.274 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-30T14:31:51.274 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-30T14:31:51.274 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-30T14:31:51.274 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-30T14:31:51.274 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-30T14:31:51.275 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-30T14:31:51.275 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-30T14:31:51.275 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-30T14:31:51.275 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-30T14:31:51.275 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.table_properties_collectors: 2023-09-30T14:31:51.275 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.inplace_update_support: 0 2023-09-30T14:31:51.276 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-30T14:31:51.276 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-30T14:31:51.276 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-30T14:31:51.276 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-30T14:31:51.276 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.bloom_locality: 0 2023-09-30T14:31:51.276 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.max_successive_merges: 0 2023-09-30T14:31:51.277 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-30T14:31:51.277 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.paranoid_file_checks: 0 2023-09-30T14:31:51.277 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.force_consistency_checks: 1 2023-09-30T14:31:51.277 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.report_bg_io_stats: 0 2023-09-30T14:31:51.277 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.ttl: 2592000 2023-09-30T14:31:51.278 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-30T14:31:51.278 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-30T14:31:51.278 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-30T14:31:51.278 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.enable_blob_files: false 2023-09-30T14:31:51.278 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.min_blob_size: 0 2023-09-30T14:31:51.278 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.blob_file_size: 268435456 2023-09-30T14:31:51.279 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-30T14:31:51.279 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-30T14:31:51.279 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-30T14:31:51.279 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-30T14:31:51.279 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-30T14:31:51.280 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.blob_file_starting_level: 0 2023-09-30T14:31:51.280 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-30T14:31:51.280 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-09-30T14:31:51.280 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-09-30T14:31:51.280 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 734e0e41-d35f-44d3-ba2e-c79e516a191e 2023-09-30T14:31:51.280 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696084311244665, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-09-30T14:31:51.280 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-09-30T14:31:51.281 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696084311245213, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696084311, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "734e0e41-d35f-44d3-ba2e-c79e516a191e", "db_session_id": "9E6BW6OAU2IMY21M8RKQ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-09-30T14:31:51.281 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696084311245303, "job": 1, "event": "recovery_finished"} 2023-09-30T14:31:51.281 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-09-30T14:31:51.541 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-30T14:31:51.541 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b9ac0a6000 2023-09-30T14:31:51.541 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: DB pointer 0x55b9ac092000 2023-09-30T14:31:51.541 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-30T14:31:51.541 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-30T14:31:51.542 INFO:journalctl@ceph.mon.c.smithi156.stdout: ** DB Stats ** 2023-09-30T14:31:51.542 INFO:journalctl@ceph.mon.c.smithi156.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-30T14:31:51.542 INFO:journalctl@ceph.mon.c.smithi156.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-30T14:31:51.542 INFO:journalctl@ceph.mon.c.smithi156.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-30T14:31:51.542 INFO:journalctl@ceph.mon.c.smithi156.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-30T14:31:51.542 INFO:journalctl@ceph.mon.c.smithi156.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-30T14:31:51.542 INFO:journalctl@ceph.mon.c.smithi156.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-30T14:31:51.542 INFO:journalctl@ceph.mon.c.smithi156.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-30T14:31:51.542 INFO:journalctl@ceph.mon.c.smithi156.stdout: 2023-09-30T14:31:51.542 INFO:journalctl@ceph.mon.c.smithi156.stdout: ** Compaction Stats [default] ** 2023-09-30T14:31:51.542 INFO:journalctl@ceph.mon.c.smithi156.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-30T14:31:51.542 INFO:journalctl@ceph.mon.c.smithi156.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-30T14:31:51.543 INFO:journalctl@ceph.mon.c.smithi156.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-30T14:31:51.543 INFO:journalctl@ceph.mon.c.smithi156.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-30T14:31:51.543 INFO:journalctl@ceph.mon.c.smithi156.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-30T14:31:51.543 INFO:journalctl@ceph.mon.c.smithi156.stdout: 2023-09-30T14:31:51.543 INFO:journalctl@ceph.mon.c.smithi156.stdout: ** Compaction Stats [default] ** 2023-09-30T14:31:51.543 INFO:journalctl@ceph.mon.c.smithi156.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-30T14:31:51.543 INFO:journalctl@ceph.mon.c.smithi156.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-30T14:31:51.543 INFO:journalctl@ceph.mon.c.smithi156.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-30T14:31:51.543 INFO:journalctl@ceph.mon.c.smithi156.stdout: 2023-09-30T14:31:51.543 INFO:journalctl@ceph.mon.c.smithi156.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-30T14:31:51.543 INFO:journalctl@ceph.mon.c.smithi156.stdout: 2023-09-30T14:31:51.543 INFO:journalctl@ceph.mon.c.smithi156.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-30T14:31:51.543 INFO:journalctl@ceph.mon.c.smithi156.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-30T14:31:51.544 INFO:journalctl@ceph.mon.c.smithi156.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-30T14:31:51.544 INFO:journalctl@ceph.mon.c.smithi156.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-30T14:31:51.544 INFO:journalctl@ceph.mon.c.smithi156.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-30T14:31:51.544 INFO:journalctl@ceph.mon.c.smithi156.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-30T14:31:51.544 INFO:journalctl@ceph.mon.c.smithi156.stdout: Cumulative compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-30T14:31:51.544 INFO:journalctl@ceph.mon.c.smithi156.stdout: Interval compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-30T14:31:51.544 INFO:journalctl@ceph.mon.c.smithi156.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-30T14:31:51.544 INFO:journalctl@ceph.mon.c.smithi156.stdout: Block cache BinnedLRUCache@0x55b9ac0091f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-09-30T14:31:51.544 INFO:journalctl@ceph.mon.c.smithi156.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-09-30T14:31:51.544 INFO:journalctl@ceph.mon.c.smithi156.stdout: 2023-09-30T14:31:51.544 INFO:journalctl@ceph.mon.c.smithi156.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-30T14:31:51.544 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-09-30T14:31:51.544 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: using public_addr v2:172.21.15.156:0/0 -> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] 2023-09-30T14:31:51.545 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: starting mon.c rank -1 at public addrs [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] at bind addrs [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:31:51.545 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: mon.c@-1(???) e0 preinit fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:31:51.545 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: mon.c@-1(synchronizing).mds e1 new map 2023-09-30T14:31:51.545 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: mon.c@-1(synchronizing).mds e1 print_map 2023-09-30T14:31:51.545 INFO:journalctl@ceph.mon.c.smithi156.stdout: e1 2023-09-30T14:31:51.545 INFO:journalctl@ceph.mon.c.smithi156.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-30T14:31:51.545 INFO:journalctl@ceph.mon.c.smithi156.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-30T14:31:51.545 INFO:journalctl@ceph.mon.c.smithi156.stdout: legacy client fscid: -1 2023-09-30T14:31:51.545 INFO:journalctl@ceph.mon.c.smithi156.stdout: 2023-09-30T14:31:51.545 INFO:journalctl@ceph.mon.c.smithi156.stdout: No filesystems configured 2023-09-30T14:31:51.545 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-30T14:31:51.546 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-30T14:31:51.546 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-30T14:31:51.546 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-30T14:31:51.546 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-30T14:31:51.546 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-30T14:31:51.546 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-30T14:31:51.546 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-30T14:31:51.546 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-30T14:31:51.547 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-30T14:31:51.547 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:51.547 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: Updating smithi148:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-30T14:31:51.547 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:51.547 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:51.547 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:51.547 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:51.547 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:31:51.547 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:51.548 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: Added host smithi156 2023-09-30T14:31:51.548 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:51.548 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:51.548 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:51.548 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:31:51.548 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:51.548 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:51.548 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2219430363' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-30T14:31:51.548 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:51.548 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:51.548 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:51.549 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:51.549 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:31:51.549 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:31:51.549 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:31:51.549 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: Updating smithi156:/etc/ceph/ceph.conf 2023-09-30T14:31:51.549 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2219430363' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-30T14:31:51.549 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: osdmap e4: 0 total, 0 up, 0 in 2023-09-30T14:31:51.549 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: Updating smithi156:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-09-30T14:31:51.549 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-09-30T14:31:51.549 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:51.549 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:51.550 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:51.550 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: Updating smithi156:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-30T14:31:51.550 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:51.550 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi032:172.21.15.32=a;smithi148:172.21.15.148=b;smithi156:172.21.15.156=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:31:51.550 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: Saving service mon spec with placement smithi032:172.21.15.32=a;smithi148:172.21.15.148=b;smithi156:172.21.15.156=c;count:3 2023-09-30T14:31:51.550 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:51.550 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:31:51.550 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:31:51.550 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:31:51.551 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:51.551 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-30T14:31:51.551 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:31:51.551 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: Deploying daemon mon.c on smithi156 2023-09-30T14:31:51.551 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:51.551 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-30T14:31:51.551 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: expand_channel_meta expand map: {default=false} 2023-09-30T14:31:51.551 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: expand_channel_meta from 'false' to 'false' 2023-09-30T14:31:51.551 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: expand_channel_meta expanded map: {default=false} 2023-09-30T14:31:51.551 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: expand_channel_meta expand map: {default=info} 2023-09-30T14:31:51.551 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: expand_channel_meta from 'info' to 'info' 2023-09-30T14:31:51.551 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: expand_channel_meta expanded map: {default=info} 2023-09-30T14:31:51.551 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: expand_channel_meta expand map: {default=daemon} 2023-09-30T14:31:51.552 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-30T14:31:51.552 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: expand_channel_meta expanded map: {default=daemon} 2023-09-30T14:31:51.552 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: expand_channel_meta expand map: {default=debug} 2023-09-30T14:31:51.552 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: expand_channel_meta from 'debug' to 'debug' 2023-09-30T14:31:51.552 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: expand_channel_meta expanded map: {default=debug} 2023-09-30T14:31:51.552 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 ceph-mon[103579]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-09-30T14:31:51.552 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:51 smithi156 systemd[1]: Started Ceph mon.c for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:31:52.469 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-30T14:31:52.470 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-09-30T14:31:53.042 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.c/config 2023-09-30T14:31:56.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:31:56.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:56.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: mon.a calling monitor election 2023-09-30T14:31:56.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:56.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:56.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:56.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: mon.c calling monitor election 2023-09-30T14:31:56.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:56.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:56.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:56.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:56.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-30T14:31:56.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: monmap e2: 2 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-30T14:31:56.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: fsmap 2023-09-30T14:31:56.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: osdmap e4: 0 total, 0 up, 0 in 2023-09-30T14:31:56.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: mgrmap e13: x(active, since 44s) 2023-09-30T14:31:56.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: overall HEALTH_OK 2023-09-30T14:31:56.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:56.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:56.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:56.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-30T14:31:56.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:31:56.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:56.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: mon.a calling monitor election 2023-09-30T14:31:56.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:56.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:56.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:56.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: mon.c calling monitor election 2023-09-30T14:31:56.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:56.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:56.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:56.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:56.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-30T14:31:56.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: monmap e2: 2 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-30T14:31:56.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: fsmap 2023-09-30T14:31:56.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: osdmap e4: 0 total, 0 up, 0 in 2023-09-30T14:31:56.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: mgrmap e13: x(active, since 44s) 2023-09-30T14:31:56.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: overall HEALTH_OK 2023-09-30T14:31:56.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:56.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:56.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:56.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-30T14:31:57.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.249+0000 7fa349752700 1 -- 172.21.15.156:0/1068108762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa32c0035c0 msgr2=0x7fa32c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:57.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.249+0000 7fa349752700 1 --2- 172.21.15.156:0/1068108762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa32c0035c0 0x7fa32c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa3440770c0 tx=0x7fa33800feb0 comp rx=0 tx=0).stop 2023-09-30T14:31:57.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.249+0000 7fa349752700 1 -- 172.21.15.156:0/1068108762 shutdown_connections 2023-09-30T14:31:57.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.249+0000 7fa349752700 1 --2- 172.21.15.156:0/1068108762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa32c0035c0 0x7fa32c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:57.250 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.249+0000 7fa349752700 1 -- 172.21.15.156:0/1068108762 >> 172.21.15.156:0/1068108762 conn(0x7fa3440fafc0 msgr2=0x7fa3440fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:57.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.249+0000 7fa349752700 1 -- 172.21.15.156:0/1068108762 shutdown_connections 2023-09-30T14:31:57.251 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.249+0000 7fa349752700 1 -- 172.21.15.156:0/1068108762 wait complete. 2023-09-30T14:31:57.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.250+0000 7fa349752700 1 Processor -- start 2023-09-30T14:31:57.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.250+0000 7fa349752700 1 -- start start 2023-09-30T14:31:57.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.250+0000 7fa349752700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa32c0035c0 0x7fa3440fb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:57.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.250+0000 7fa349752700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3440fb9b0 0x7fa3440fbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:57.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.250+0000 7fa349752700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3440fd3f0 con 0x7fa3440fb9b0 2023-09-30T14:31:57.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.250+0000 7fa349752700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3441a2110 con 0x7fa32c0035c0 2023-09-30T14:31:57.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.251+0000 7fa342ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa32c0035c0 0x7fa3440fb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:57.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.251+0000 7fa342ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa32c0035c0 0x7fa3440fb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35956/0 (socket says 172.21.15.156:35956) 2023-09-30T14:31:57.252 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.251+0000 7fa342ffd700 1 -- 172.21.15.156:0/966153249 learned_addr learned my addr 172.21.15.156:0/966153249 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-30T14:31:57.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.251+0000 7fa3427fc700 1 --2- 172.21.15.156:0/966153249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3440fb9b0 0x7fa3440fbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:57.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.251+0000 7fa342ffd700 1 -- 172.21.15.156:0/966153249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa32c0035c0 msgr2=0x7fa3440fb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-09-30T14:31:57.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.251+0000 7fa342ffd700 1 -- 172.21.15.156:0/966153249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa32c0035c0 msgr2=0x7fa3440fb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-30T14:31:57.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.251+0000 7fa342ffd700 1 --2- 172.21.15.156:0/966153249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa32c0035c0 0x7fa3440fb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T14:31:57.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.251+0000 7fa342ffd700 1 --2- 172.21.15.156:0/966153249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa32c0035c0 0x7fa3440fb470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T14:31:57.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.251+0000 7fa3427fc700 1 -- 172.21.15.156:0/966153249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa32c0035c0 msgr2=0x7fa3440fb470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T14:31:57.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.251+0000 7fa3427fc700 1 --2- 172.21.15.156:0/966153249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa32c0035c0 0x7fa3440fb470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:57.253 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.251+0000 7fa3427fc700 1 -- 172.21.15.156:0/966153249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33800f900 con 0x7fa3440fb9b0 2023-09-30T14:31:57.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.251+0000 7fa3427fc700 1 --2- 172.21.15.156:0/966153249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3440fb9b0 0x7fa3440fbe30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa33400b4d0 tx=0x7fa33400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:57.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.251+0000 7fa32bfff700 1 -- 172.21.15.156:0/966153249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa334011630 con 0x7fa3440fb9b0 2023-09-30T14:31:57.254 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.252+0000 7fa32bfff700 1 -- 172.21.15.156:0/966153249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa334011c50 con 0x7fa3440fb9b0 2023-09-30T14:31:57.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.252+0000 7fa349752700 1 -- 172.21.15.156:0/966153249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3441a2310 con 0x7fa3440fb9b0 2023-09-30T14:31:57.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.252+0000 7fa32bfff700 1 -- 172.21.15.156:0/966153249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa334010d90 con 0x7fa3440fb9b0 2023-09-30T14:31:57.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.252+0000 7fa349752700 1 -- 172.21.15.156:0/966153249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3441a2830 con 0x7fa3440fb9b0 2023-09-30T14:31:57.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.253+0000 7fa32bfff700 1 -- 172.21.15.156:0/966153249 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fa33401a400 con 0x7fa3440fb9b0 2023-09-30T14:31:57.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.253+0000 7fa32bfff700 1 --2- 172.21.15.156:0/966153249 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa330038500 0x7fa33003a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:31:57.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.253+0000 7fa349752700 1 -- 172.21.15.156:0/966153249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa344062450 con 0x7fa3440fb9b0 2023-09-30T14:31:57.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.253+0000 7fa32bfff700 1 -- 172.21.15.156:0/966153249 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa33404adc0 con 0x7fa3440fb9b0 2023-09-30T14:31:57.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.253+0000 7fa342ffd700 1 --2- 172.21.15.156:0/966153249 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa330038500 0x7fa33003a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:31:57.255 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.254+0000 7fa342ffd700 1 --2- 172.21.15.156:0/966153249 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa330038500 0x7fa33003a9c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa338011510 tx=0x7fa338005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:31:57.259 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.259+0000 7fa32bfff700 1 -- 172.21.15.156:0/966153249 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fa334050020 con 0x7fa3440fb9b0 2023-09-30T14:31:57.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.526+0000 7fa349752700 1 -- 172.21.15.156:0/966153249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa344107930 con 0x7fa3440fb9b0 2023-09-30T14:31:57.528 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.527+0000 7fa32bfff700 1 -- 172.21.15.156:0/966153249 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7fa3340174a0 con 0x7fa3440fb9b0 2023-09-30T14:31:57.528 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-30T14:31:57.528 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":2,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","modified":"2023-09-30T14:31:51.270584Z","created":"2023-09-30T14:30:08.312183Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-30T14:31:57.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.529+0000 7fa349752700 1 -- 172.21.15.156:0/966153249 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa330038500 msgr2=0x7fa33003a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:57.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.529+0000 7fa349752700 1 --2- 172.21.15.156:0/966153249 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa330038500 0x7fa33003a9c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa338011510 tx=0x7fa338005960 comp rx=0 tx=0).stop 2023-09-30T14:31:57.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.529+0000 7fa349752700 1 -- 172.21.15.156:0/966153249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3440fb9b0 msgr2=0x7fa3440fbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:31:57.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.529+0000 7fa349752700 1 --2- 172.21.15.156:0/966153249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3440fb9b0 0x7fa3440fbe30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa33400b4d0 tx=0x7fa33400b890 comp rx=0 tx=0).stop 2023-09-30T14:31:57.530 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.529+0000 7fa349752700 1 -- 172.21.15.156:0/966153249 shutdown_connections 2023-09-30T14:31:57.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.529+0000 7fa349752700 1 --2- 172.21.15.156:0/966153249 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa330038500 0x7fa33003a9c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:57.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.529+0000 7fa349752700 1 --2- 172.21.15.156:0/966153249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa32c0035c0 0x7fa3440fb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:57.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.529+0000 7fa349752700 1 --2- 172.21.15.156:0/966153249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3440fb9b0 0x7fa3440fbe30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:31:57.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.529+0000 7fa349752700 1 -- 172.21.15.156:0/966153249 >> 172.21.15.156:0/966153249 conn(0x7fa3440fafc0 msgr2=0x7fa3440ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:31:57.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.529+0000 7fa349752700 1 -- 172.21.15.156:0/966153249 shutdown_connections 2023-09-30T14:31:57.531 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:31:57.529+0000 7fa349752700 1 -- 172.21.15.156:0/966153249 wait complete. 2023-09-30T14:31:57.531 INFO:teuthology.orchestra.run.smithi156.stderr:dumped monmap epoch 2 2023-09-30T14:31:57.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:57 smithi032 ceph-mon[98592]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:57.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:57 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:31:57.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:57 smithi032 ceph-mon[98592]: Deploying daemon mon.b on smithi148 2023-09-30T14:31:57.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:57 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:57.780 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:57 smithi156 ceph-mon[103579]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:57.780 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:57 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:31:57.780 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:57 smithi156 ceph-mon[103579]: Deploying daemon mon.b on smithi148 2023-09-30T14:31:57.780 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:57 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:58.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:31:58 smithi032 ceph-mon[98592]: from='client.? 172.21.15.156:0/966153249' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-30T14:31:58.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:31:58 smithi156 ceph-mon[103579]: from='client.? 172.21.15.156:0/966153249' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-30T14:31:59.006 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-30T14:31:59.007 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-09-30T14:31:59.071 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:58 smithi148 systemd[1]: Starting Ceph mon.b for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:31:59.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 podman[102315]: 2023-09-30T14:31:59.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: set uid:gid to 167:167 (ceph:ceph) 2023-09-30T14:31:59.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: ceph version 18.0.0-6447-g1d10b717 (1d10b71792f3be8887a7631e69851ac2df3585af) reef (dev), process ceph-mon, pid 7 2023-09-30T14:31:59.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: pidfile_write: ignore empty --pid-file 2023-09-30T14:31:59.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 bash[102315]: 7815dedee190fd5acf83f466d3555932c429e032340e45ac2a6e16091c04117d 2023-09-30T14:31:59.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: load: jerasure load: lrc 2023-09-30T14:31:59.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: RocksDB version: 7.9.2 2023-09-30T14:31:59.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Git sha 0 2023-09-30T14:31:59.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Compile date 2023-09-29 16:35:50 2023-09-30T14:31:59.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: DB SUMMARY 2023-09-30T14:31:59.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: DB Session ID: QKL66ZYS3H1J3KNJQ9MK 2023-09-30T14:31:59.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: CURRENT file: CURRENT 2023-09-30T14:31:59.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: IDENTITY file: IDENTITY 2023-09-30T14:31:59.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-09-30T14:31:59.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-09-30T14:31:59.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-09-30T14:31:59.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.error_if_exists: 0 2023-09-30T14:31:59.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.create_if_missing: 0 2023-09-30T14:31:59.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.paranoid_checks: 1 2023-09-30T14:31:59.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-30T14:31:59.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-30T14:31:59.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-30T14:31:59.362 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.env: 0x556adebb0da0 2023-09-30T14:31:59.362 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.fs: PosixFileSystem 2023-09-30T14:31:59.362 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.info_log: 0x556ae053f5a0 2023-09-30T14:31:59.362 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_file_opening_threads: 16 2023-09-30T14:31:59.362 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.statistics: (nil) 2023-09-30T14:31:59.362 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.use_fsync: 0 2023-09-30T14:31:59.362 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_log_file_size: 0 2023-09-30T14:31:59.362 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-30T14:31:59.362 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-30T14:31:59.362 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.keep_log_file_num: 1000 2023-09-30T14:31:59.362 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.recycle_log_file_num: 0 2023-09-30T14:31:59.362 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.allow_fallocate: 1 2023-09-30T14:31:59.363 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.allow_mmap_reads: 0 2023-09-30T14:31:59.363 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.allow_mmap_writes: 0 2023-09-30T14:31:59.363 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.use_direct_reads: 0 2023-09-30T14:31:59.363 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-30T14:31:59.363 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.create_missing_column_families: 0 2023-09-30T14:31:59.363 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.db_log_dir: 2023-09-30T14:31:59.363 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.wal_dir: 2023-09-30T14:31:59.363 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-30T14:31:59.363 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-30T14:31:59.363 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-30T14:31:59.363 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-30T14:31:59.363 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-30T14:31:59.364 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-30T14:31:59.364 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.advise_random_on_open: 1 2023-09-30T14:31:59.364 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.db_write_buffer_size: 0 2023-09-30T14:31:59.364 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.write_buffer_manager: 0x556ae04f7360 2023-09-30T14:31:59.364 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-30T14:31:59.364 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-30T14:31:59.364 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-30T14:31:59.364 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.rate_limiter: (nil) 2023-09-30T14:31:59.364 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-30T14:31:59.364 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.wal_recovery_mode: 2 2023-09-30T14:31:59.364 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.enable_thread_tracking: 0 2023-09-30T14:31:59.364 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.enable_pipelined_write: 0 2023-09-30T14:31:59.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.unordered_write: 0 2023-09-30T14:31:59.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-30T14:31:59.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-30T14:31:59.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-30T14:31:59.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-30T14:31:59.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.row_cache: None 2023-09-30T14:31:59.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.wal_filter: None 2023-09-30T14:31:59.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-30T14:31:59.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.allow_ingest_behind: 0 2023-09-30T14:31:59.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.two_write_queues: 0 2023-09-30T14:31:59.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.manual_wal_flush: 0 2023-09-30T14:31:59.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.wal_compression: 0 2023-09-30T14:31:59.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.atomic_flush: 0 2023-09-30T14:31:59.366 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-30T14:31:59.366 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-30T14:31:59.366 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-30T14:31:59.366 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.log_readahead_size: 0 2023-09-30T14:31:59.366 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-30T14:31:59.366 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.best_efforts_recovery: 0 2023-09-30T14:31:59.366 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-30T14:31:59.366 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-30T14:31:59.366 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.allow_data_in_errors: 0 2023-09-30T14:31:59.366 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.db_host_id: __hostname__ 2023-09-30T14:31:59.366 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-30T14:31:59.366 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_background_jobs: 2 2023-09-30T14:31:59.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_background_compactions: -1 2023-09-30T14:31:59.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_subcompactions: 1 2023-09-30T14:31:59.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-30T14:31:59.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-30T14:31:59.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-30T14:31:59.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_total_wal_size: 0 2023-09-30T14:31:59.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-30T14:31:59.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-30T14:31:59.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-30T14:31:59.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-30T14:31:59.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_open_files: -1 2023-09-30T14:31:59.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.bytes_per_sync: 0 2023-09-30T14:31:59.367 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-30T14:31:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-30T14:31:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compaction_readahead_size: 0 2023-09-30T14:31:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_background_flushes: -1 2023-09-30T14:31:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Compression algorithms supported: 2023-09-30T14:31:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-30T14:31:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: kZSTD supported: 0 2023-09-30T14:31:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: kXpressCompression supported: 0 2023-09-30T14:31:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: kLZ4HCCompression supported: 1 2023-09-30T14:31:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: kZlibCompression supported: 1 2023-09-30T14:31:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: kSnappyCompression supported: 1 2023-09-30T14:31:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: kLZ4Compression supported: 1 2023-09-30T14:31:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: kBZip2Compression supported: 0 2023-09-30T14:31:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-30T14:31:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-30T14:31:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-09-30T14:31:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-30T14:31:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-30T14:31:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.merge_operator: 2023-09-30T14:31:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compaction_filter: None 2023-09-30T14:31:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compaction_filter_factory: None 2023-09-30T14:31:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.sst_partitioner_factory: None 2023-09-30T14:31:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-30T14:31:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-30T14:31:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556ae053f6a0) 2023-09-30T14:31:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout: cache_index_and_filter_blocks: 1 2023-09-30T14:31:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-30T14:31:59.370 INFO:journalctl@ceph.mon.b.smithi148.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-30T14:31:59.370 INFO:journalctl@ceph.mon.b.smithi148.stdout: pin_top_level_index_and_filter: 1 2023-09-30T14:31:59.370 INFO:journalctl@ceph.mon.b.smithi148.stdout: index_type: 0 2023-09-30T14:31:59.370 INFO:journalctl@ceph.mon.b.smithi148.stdout: data_block_index_type: 0 2023-09-30T14:31:59.370 INFO:journalctl@ceph.mon.b.smithi148.stdout: index_shortening: 1 2023-09-30T14:31:59.370 INFO:journalctl@ceph.mon.b.smithi148.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-30T14:31:59.370 INFO:journalctl@ceph.mon.b.smithi148.stdout: checksum: 4 2023-09-30T14:31:59.370 INFO:journalctl@ceph.mon.b.smithi148.stdout: no_block_cache: 0 2023-09-30T14:31:59.370 INFO:journalctl@ceph.mon.b.smithi148.stdout: block_cache: 0x556ae05951f0 2023-09-30T14:31:59.370 INFO:journalctl@ceph.mon.b.smithi148.stdout: block_cache_name: BinnedLRUCache 2023-09-30T14:31:59.370 INFO:journalctl@ceph.mon.b.smithi148.stdout: block_cache_options: 2023-09-30T14:31:59.371 INFO:journalctl@ceph.mon.b.smithi148.stdout: capacity : 536870912 2023-09-30T14:31:59.371 INFO:journalctl@ceph.mon.b.smithi148.stdout: num_shard_bits : 4 2023-09-30T14:31:59.371 INFO:journalctl@ceph.mon.b.smithi148.stdout: strict_capacity_limit : 0 2023-09-30T14:31:59.371 INFO:journalctl@ceph.mon.b.smithi148.stdout: high_pri_pool_ratio: 0.000 2023-09-30T14:31:59.371 INFO:journalctl@ceph.mon.b.smithi148.stdout: block_cache_compressed: (nil) 2023-09-30T14:31:59.371 INFO:journalctl@ceph.mon.b.smithi148.stdout: persistent_cache: (nil) 2023-09-30T14:31:59.371 INFO:journalctl@ceph.mon.b.smithi148.stdout: block_size: 4096 2023-09-30T14:31:59.371 INFO:journalctl@ceph.mon.b.smithi148.stdout: block_size_deviation: 10 2023-09-30T14:31:59.371 INFO:journalctl@ceph.mon.b.smithi148.stdout: block_restart_interval: 16 2023-09-30T14:31:59.371 INFO:journalctl@ceph.mon.b.smithi148.stdout: index_block_restart_interval: 1 2023-09-30T14:31:59.371 INFO:journalctl@ceph.mon.b.smithi148.stdout: metadata_block_size: 4096 2023-09-30T14:31:59.371 INFO:journalctl@ceph.mon.b.smithi148.stdout: partition_filters: 0 2023-09-30T14:31:59.372 INFO:journalctl@ceph.mon.b.smithi148.stdout: use_delta_encoding: 1 2023-09-30T14:31:59.372 INFO:journalctl@ceph.mon.b.smithi148.stdout: filter_policy: bloomfilter 2023-09-30T14:31:59.372 INFO:journalctl@ceph.mon.b.smithi148.stdout: whole_key_filtering: 1 2023-09-30T14:31:59.372 INFO:journalctl@ceph.mon.b.smithi148.stdout: verify_compression: 0 2023-09-30T14:31:59.372 INFO:journalctl@ceph.mon.b.smithi148.stdout: read_amp_bytes_per_bit: 0 2023-09-30T14:31:59.372 INFO:journalctl@ceph.mon.b.smithi148.stdout: format_version: 5 2023-09-30T14:31:59.372 INFO:journalctl@ceph.mon.b.smithi148.stdout: enable_index_compression: 1 2023-09-30T14:31:59.372 INFO:journalctl@ceph.mon.b.smithi148.stdout: block_align: 0 2023-09-30T14:31:59.372 INFO:journalctl@ceph.mon.b.smithi148.stdout: max_auto_readahead_size: 262144 2023-09-30T14:31:59.372 INFO:journalctl@ceph.mon.b.smithi148.stdout: prepopulate_block_cache: 0 2023-09-30T14:31:59.372 INFO:journalctl@ceph.mon.b.smithi148.stdout: initial_auto_readahead_size: 8192 2023-09-30T14:31:59.372 INFO:journalctl@ceph.mon.b.smithi148.stdout: num_file_reads_for_auto_readahead: 2 2023-09-30T14:31:59.372 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.write_buffer_size: 33554432 2023-09-30T14:31:59.373 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_write_buffer_number: 2 2023-09-30T14:31:59.373 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compression: NoCompression 2023-09-30T14:31:59.373 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.bottommost_compression: Disabled 2023-09-30T14:31:59.373 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.prefix_extractor: nullptr 2023-09-30T14:31:59.373 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-30T14:31:59.373 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.num_levels: 7 2023-09-30T14:31:59.373 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-30T14:31:59.373 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-30T14:31:59.373 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-30T14:31:59.374 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-30T14:31:59.374 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-30T14:31:59.374 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-30T14:31:59.375 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-30T14:31:59.375 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-30T14:31:59.375 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-30T14:31:59.375 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-30T14:31:59.375 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-30T14:31:59.376 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-30T14:31:59.376 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-30T14:31:59.376 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compression_opts.level: 32767 2023-09-30T14:31:59.376 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compression_opts.strategy: 0 2023-09-30T14:31:59.376 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-30T14:31:59.376 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-30T14:31:59.376 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-30T14:31:59.376 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-30T14:31:59.376 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compression_opts.enabled: false 2023-09-30T14:31:59.376 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-30T14:31:59.376 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-30T14:31:59.376 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-30T14:31:59.376 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-30T14:31:59.377 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.target_file_size_base: 67108864 2023-09-30T14:31:59.377 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-30T14:31:59.377 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-30T14:31:59.377 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-30T14:31:59.377 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-30T14:31:59.377 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-30T14:31:59.377 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-30T14:31:59.377 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-30T14:31:59.377 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-30T14:31:59.377 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-30T14:31:59.378 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-30T14:31:59.378 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-30T14:31:59.378 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-30T14:31:59.378 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-30T14:31:59.378 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-30T14:31:59.378 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.arena_block_size: 1048576 2023-09-30T14:31:59.378 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-30T14:31:59.378 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-30T14:31:59.378 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.disable_auto_compactions: 0 2023-09-30T14:31:59.378 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-30T14:31:59.378 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-30T14:31:59.378 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-30T14:31:59.379 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-30T14:31:59.379 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-30T14:31:59.379 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-30T14:31:59.379 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-30T14:31:59.379 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-30T14:31:59.379 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-30T14:31:59.379 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-30T14:31:59.379 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.table_properties_collectors: 2023-09-30T14:31:59.379 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.inplace_update_support: 0 2023-09-30T14:31:59.379 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-30T14:31:59.379 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-30T14:31:59.379 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-30T14:31:59.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-30T14:31:59.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.bloom_locality: 0 2023-09-30T14:31:59.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.max_successive_merges: 0 2023-09-30T14:31:59.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-30T14:31:59.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.paranoid_file_checks: 0 2023-09-30T14:31:59.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.force_consistency_checks: 1 2023-09-30T14:31:59.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.report_bg_io_stats: 0 2023-09-30T14:31:59.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.ttl: 2592000 2023-09-30T14:31:59.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-30T14:31:59.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-30T14:31:59.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-30T14:31:59.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.enable_blob_files: false 2023-09-30T14:31:59.380 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.min_blob_size: 0 2023-09-30T14:31:59.381 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.blob_file_size: 268435456 2023-09-30T14:31:59.381 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-30T14:31:59.381 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-30T14:31:59.381 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-30T14:31:59.381 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-30T14:31:59.381 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-30T14:31:59.381 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.blob_file_starting_level: 0 2023-09-30T14:31:59.381 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-30T14:31:59.381 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-09-30T14:31:59.381 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-09-30T14:31:59.381 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8457b09f-f845-4c42-afad-24038e21b9d6 2023-09-30T14:31:59.381 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696084319255842, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-09-30T14:31:59.382 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-09-30T14:31:59.382 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696084319256254, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696084319, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8457b09f-f845-4c42-afad-24038e21b9d6", "db_session_id": "QKL66ZYS3H1J3KNJQ9MK", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-09-30T14:31:59.382 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696084319256322, "job": 1, "event": "recovery_finished"} 2023-09-30T14:31:59.382 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-09-30T14:31:59.382 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-30T14:31:59.382 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556ae0632000 2023-09-30T14:31:59.382 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: DB pointer 0x556ae061e000 2023-09-30T14:31:59.382 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-30T14:31:59.382 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-30T14:31:59.382 INFO:journalctl@ceph.mon.b.smithi148.stdout: ** DB Stats ** 2023-09-30T14:31:59.382 INFO:journalctl@ceph.mon.b.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-30T14:31:59.382 INFO:journalctl@ceph.mon.b.smithi148.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-30T14:31:59.383 INFO:journalctl@ceph.mon.b.smithi148.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-30T14:31:59.383 INFO:journalctl@ceph.mon.b.smithi148.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-30T14:31:59.383 INFO:journalctl@ceph.mon.b.smithi148.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-30T14:31:59.383 INFO:journalctl@ceph.mon.b.smithi148.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-30T14:31:59.383 INFO:journalctl@ceph.mon.b.smithi148.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-30T14:31:59.383 INFO:journalctl@ceph.mon.b.smithi148.stdout: 2023-09-30T14:31:59.383 INFO:journalctl@ceph.mon.b.smithi148.stdout: ** Compaction Stats [default] ** 2023-09-30T14:31:59.383 INFO:journalctl@ceph.mon.b.smithi148.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-30T14:31:59.383 INFO:journalctl@ceph.mon.b.smithi148.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-30T14:31:59.383 INFO:journalctl@ceph.mon.b.smithi148.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-30T14:31:59.384 INFO:journalctl@ceph.mon.b.smithi148.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-30T14:31:59.384 INFO:journalctl@ceph.mon.b.smithi148.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-30T14:31:59.384 INFO:journalctl@ceph.mon.b.smithi148.stdout: 2023-09-30T14:31:59.384 INFO:journalctl@ceph.mon.b.smithi148.stdout: ** Compaction Stats [default] ** 2023-09-30T14:31:59.384 INFO:journalctl@ceph.mon.b.smithi148.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-30T14:31:59.384 INFO:journalctl@ceph.mon.b.smithi148.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-30T14:31:59.384 INFO:journalctl@ceph.mon.b.smithi148.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-30T14:31:59.384 INFO:journalctl@ceph.mon.b.smithi148.stdout: 2023-09-30T14:31:59.384 INFO:journalctl@ceph.mon.b.smithi148.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-30T14:31:59.384 INFO:journalctl@ceph.mon.b.smithi148.stdout: 2023-09-30T14:31:59.384 INFO:journalctl@ceph.mon.b.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-30T14:31:59.384 INFO:journalctl@ceph.mon.b.smithi148.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-30T14:31:59.384 INFO:journalctl@ceph.mon.b.smithi148.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-30T14:31:59.385 INFO:journalctl@ceph.mon.b.smithi148.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-30T14:31:59.385 INFO:journalctl@ceph.mon.b.smithi148.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-30T14:31:59.385 INFO:journalctl@ceph.mon.b.smithi148.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-30T14:31:59.385 INFO:journalctl@ceph.mon.b.smithi148.stdout: Cumulative compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-30T14:31:59.385 INFO:journalctl@ceph.mon.b.smithi148.stdout: Interval compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-30T14:31:59.385 INFO:journalctl@ceph.mon.b.smithi148.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-30T14:31:59.385 INFO:journalctl@ceph.mon.b.smithi148.stdout: Block cache BinnedLRUCache@0x556ae05951f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-09-30T14:31:59.385 INFO:journalctl@ceph.mon.b.smithi148.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-09-30T14:31:59.385 INFO:journalctl@ceph.mon.b.smithi148.stdout: 2023-09-30T14:31:59.385 INFO:journalctl@ceph.mon.b.smithi148.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-30T14:31:59.386 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-09-30T14:31:59.386 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: using public_addr v2:172.21.15.148:0/0 -> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] 2023-09-30T14:31:59.386 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: starting mon.b rank -1 at public addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] at bind addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:31:59.386 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.b@-1(???) e0 preinit fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:31:59.386 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.b@-1(synchronizing).mds e1 new map 2023-09-30T14:31:59.386 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.b@-1(synchronizing).mds e1 print_map 2023-09-30T14:31:59.386 INFO:journalctl@ceph.mon.b.smithi148.stdout: e1 2023-09-30T14:31:59.386 INFO:journalctl@ceph.mon.b.smithi148.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-30T14:31:59.386 INFO:journalctl@ceph.mon.b.smithi148.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-30T14:31:59.387 INFO:journalctl@ceph.mon.b.smithi148.stdout: legacy client fscid: -1 2023-09-30T14:31:59.387 INFO:journalctl@ceph.mon.b.smithi148.stdout: 2023-09-30T14:31:59.387 INFO:journalctl@ceph.mon.b.smithi148.stdout: No filesystems configured 2023-09-30T14:31:59.387 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-30T14:31:59.387 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-30T14:31:59.387 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-30T14:31:59.387 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-30T14:31:59.387 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-30T14:31:59.387 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-30T14:31:59.387 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-30T14:31:59.387 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-30T14:31:59.387 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-30T14:31:59.388 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-30T14:31:59.388 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:59.388 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: Updating smithi148:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-30T14:31:59.388 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.388 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.388 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.388 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.388 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:31:59.388 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.388 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: Added host smithi156 2023-09-30T14:31:59.388 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:59.388 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.389 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:59.389 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:31:59.389 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.389 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:59.389 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2219430363' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-30T14:31:59.389 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.389 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.389 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.389 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.389 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:31:59.389 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:31:59.389 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:31:59.389 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: Updating smithi156:/etc/ceph/ceph.conf 2023-09-30T14:31:59.390 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2219430363' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-30T14:31:59.390 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: osdmap e4: 0 total, 0 up, 0 in 2023-09-30T14:31:59.390 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: Updating smithi156:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-09-30T14:31:59.390 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-09-30T14:31:59.390 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.390 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.390 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.390 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: Updating smithi156:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-30T14:31:59.390 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:59.390 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi032:172.21.15.32=a;smithi148:172.21.15.148=b;smithi156:172.21.15.156=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:31:59.390 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: Saving service mon spec with placement smithi032:172.21.15.32=a;smithi148:172.21.15.148=b;smithi156:172.21.15.156=c;count:3 2023-09-30T14:31:59.390 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.391 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:31:59.391 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:31:59.391 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:31:59.391 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.391 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-30T14:31:59.391 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:31:59.391 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: Deploying daemon mon.c on smithi156 2023-09-30T14:31:59.391 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:59.391 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:31:59.391 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:59.391 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.a calling monitor election 2023-09-30T14:31:59.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:59.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:59.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:59.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.c calling monitor election 2023-09-30T14:31:59.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:59.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:59.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:59.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:59.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-30T14:31:59.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: monmap e2: 2 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-30T14:31:59.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: fsmap 2023-09-30T14:31:59.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: osdmap e4: 0 total, 0 up, 0 in 2023-09-30T14:31:59.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mgrmap e13: x(active, since 44s) 2023-09-30T14:31:59.393 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: overall HEALTH_OK 2023-09-30T14:31:59.393 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.393 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.393 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:31:59.393 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-30T14:31:59.393 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:31:59.393 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:31:59.393 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: Deploying daemon mon.b on smithi148 2023-09-30T14:31:59.393 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:31:59.393 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: from='client.? 172.21.15.156:0/966153249' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-30T14:31:59.393 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-30T14:31:59.394 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-09-30T14:31:59.394 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: expand_channel_meta expand map: {default=false} 2023-09-30T14:31:59.394 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: expand_channel_meta from 'false' to 'false' 2023-09-30T14:31:59.394 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: expand_channel_meta expanded map: {default=false} 2023-09-30T14:31:59.394 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: expand_channel_meta expand map: {default=info} 2023-09-30T14:31:59.394 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: expand_channel_meta from 'info' to 'info' 2023-09-30T14:31:59.394 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: expand_channel_meta expanded map: {default=info} 2023-09-30T14:31:59.394 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: expand_channel_meta expand map: {default=daemon} 2023-09-30T14:31:59.394 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-30T14:31:59.394 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: expand_channel_meta expanded map: {default=daemon} 2023-09-30T14:31:59.394 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: expand_channel_meta expand map: {default=debug} 2023-09-30T14:31:59.394 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: expand_channel_meta from 'debug' to 'debug' 2023-09-30T14:31:59.394 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 ceph-mon[102355]: expand_channel_meta expanded map: {default=debug} 2023-09-30T14:31:59.395 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:31:59 smithi148 systemd[1]: Started Ceph mon.b for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:31:59.574 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.c/config 2023-09-30T14:32:04.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.326+0000 7f2c8790d700 1 -- 172.21.15.156:0/3651914858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c801069e0 msgr2=0x7f2c6c005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:04.326 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.326+0000 7f2c8790d700 1 --2- 172.21.15.156:0/3651914858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c801069e0 0x7f2c6c005610 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2c70005fa0 tx=0x7f2c7000f4a0 comp rx=0 tx=0).stop 2023-09-30T14:32:04.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.326+0000 7f2c8790d700 1 -- 172.21.15.156:0/3651914858 shutdown_connections 2023-09-30T14:32:04.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.326+0000 7f2c8790d700 1 --2- 172.21.15.156:0/3651914858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c801069e0 0x7f2c6c005610 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:04.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.326+0000 7f2c8790d700 1 -- 172.21.15.156:0/3651914858 >> 172.21.15.156:0/3651914858 conn(0x7f2c800fc4f0 msgr2=0x7f2c800fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:32:04.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.326+0000 7f2c8790d700 1 -- 172.21.15.156:0/3651914858 shutdown_connections 2023-09-30T14:32:04.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.326+0000 7f2c8790d700 1 -- 172.21.15.156:0/3651914858 wait complete. 2023-09-30T14:32:04.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.327+0000 7f2c8790d700 1 Processor -- start 2023-09-30T14:32:04.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.327+0000 7f2c8790d700 1 -- start start 2023-09-30T14:32:04.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.327+0000 7f2c8790d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c801069e0 0x7f2c80073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:04.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.327+0000 7f2c8790d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c80073660 0x7f2c80073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:04.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.327+0000 7f2c8790d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c801a20c0 0x7f2c801a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:04.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.327+0000 7f2c8790d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c801a4b00 con 0x7f2c80073660 2023-09-30T14:32:04.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.327+0000 7f2c8790d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c801a4c70 con 0x7f2c801a20c0 2023-09-30T14:32:04.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.327+0000 7f2c8790d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c801a4e30 con 0x7f2c801069e0 2023-09-30T14:32:04.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.327+0000 7f2c856a9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c801069e0 0x7f2c80073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:04.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c84ea8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c80073660 0x7f2c80073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:04.329 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c856a9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c801069e0 0x7f2c80073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35980/0 (socket says 172.21.15.156:35980) 2023-09-30T14:32:04.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c856a9700 1 -- 172.21.15.156:0/1232310268 learned_addr learned my addr 172.21.15.156:0/1232310268 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-30T14:32:04.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c85eaa700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c801a20c0 0x7f2c801a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:04.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c856a9700 1 -- 172.21.15.156:0/1232310268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c801069e0 msgr2=0x7f2c80073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-09-30T14:32:04.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c856a9700 1 -- 172.21.15.156:0/1232310268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c801069e0 msgr2=0x7f2c80073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-30T14:32:04.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c856a9700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c801069e0 0x7f2c80073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T14:32:04.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c856a9700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c801069e0 0x7f2c80073120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T14:32:04.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c85eaa700 1 -- 172.21.15.156:0/1232310268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c801a20c0 msgr2=0x7f2c801a44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-09-30T14:32:04.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c85eaa700 1 -- 172.21.15.156:0/1232310268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c801a20c0 msgr2=0x7f2c801a44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-30T14:32:04.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c85eaa700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c801a20c0 0x7f2c801a44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T14:32:04.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c85eaa700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c801a20c0 0x7f2c801a44b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T14:32:04.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c84ea8700 1 -- 172.21.15.156:0/1232310268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c801069e0 msgr2=0x7f2c80073120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T14:32:04.330 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c84ea8700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c801069e0 0x7f2c80073120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:04.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c84ea8700 1 -- 172.21.15.156:0/1232310268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c801a20c0 msgr2=0x7f2c801a44b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T14:32:04.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c84ea8700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c801a20c0 0x7f2c801a44b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:04.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c84ea8700 1 -- 172.21.15.156:0/1232310268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c700149d0 con 0x7f2c80073660 2023-09-30T14:32:04.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c84ea8700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c80073660 0x7f2c80073ae0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00b550 tx=0x7f2c7c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:32:04.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c767fc700 1 -- 172.21.15.156:0/1232310268 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c004230 con 0x7f2c80073660 2023-09-30T14:32:04.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c8790d700 1 -- 172.21.15.156:0/1232310268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c801a5110 con 0x7f2c80073660 2023-09-30T14:32:04.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.328+0000 7f2c767fc700 1 -- 172.21.15.156:0/1232310268 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c7c004390 con 0x7f2c80073660 2023-09-30T14:32:04.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.329+0000 7f2c767fc700 1 -- 172.21.15.156:0/1232310268 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c011af0 con 0x7f2c80073660 2023-09-30T14:32:04.331 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.329+0000 7f2c8790d700 1 -- 172.21.15.156:0/1232310268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c801a5610 con 0x7f2c80073660 2023-09-30T14:32:04.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.329+0000 7f2c767fc700 1 -- 172.21.15.156:0/1232310268 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f2c7c011c50 con 0x7f2c80073660 2023-09-30T14:32:04.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.330+0000 7f2c767fc700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2c64038bf0 0x7f2c6403b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:04.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.330+0000 7f2c8790d700 1 -- 172.21.15.156:0/1232310268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c80062450 con 0x7f2c80073660 2023-09-30T14:32:04.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.330+0000 7f2c767fc700 1 -- 172.21.15.156:0/1232310268 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2c7c04aee0 con 0x7f2c80073660 2023-09-30T14:32:04.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.330+0000 7f2c856a9700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2c64038bf0 0x7f2c6403b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:04.332 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.330+0000 7f2c856a9700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2c64038bf0 0x7f2c6403b0b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2c70001eb0 tx=0x7f2c70015fe0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:32:04.335 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.335+0000 7f2c767fc700 1 -- 172.21.15.156:0/1232310268 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f2c7c00f8d0 con 0x7f2c80073660 2023-09-30T14:32:04.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.613+0000 7f2c8790d700 1 -- 172.21.15.156:0/1232310268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2c800008d0 con 0x7f2c80073660 2023-09-30T14:32:04.614 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.613+0000 7f2c767fc700 1 -- 172.21.15.156:0/1232310268 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7f2c7c01f8a0 con 0x7f2c80073660 2023-09-30T14:32:04.614 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-30T14:32:04.614 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":3,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","modified":"2023-09-30T14:31:59.286455Z","created":"2023-09-30T14:30:08.312183Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-30T14:32:04.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.616+0000 7f2c8790d700 1 -- 172.21.15.156:0/1232310268 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2c64038bf0 msgr2=0x7f2c6403b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:04.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.616+0000 7f2c8790d700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2c64038bf0 0x7f2c6403b0b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2c70001eb0 tx=0x7f2c70015fe0 comp rx=0 tx=0).stop 2023-09-30T14:32:04.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.616+0000 7f2c8790d700 1 -- 172.21.15.156:0/1232310268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c80073660 msgr2=0x7f2c80073ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:04.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.616+0000 7f2c8790d700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c80073660 0x7f2c80073ae0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00b550 tx=0x7f2c7c00b910 comp rx=0 tx=0).stop 2023-09-30T14:32:04.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.616+0000 7f2c8790d700 1 -- 172.21.15.156:0/1232310268 shutdown_connections 2023-09-30T14:32:04.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.616+0000 7f2c8790d700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2c64038bf0 0x7f2c6403b0b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:04.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.616+0000 7f2c8790d700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c801a20c0 0x7f2c801a44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:04.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.616+0000 7f2c8790d700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c801069e0 0x7f2c80073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:04.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.616+0000 7f2c8790d700 1 --2- 172.21.15.156:0/1232310268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c80073660 0x7f2c80073ae0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:04.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.616+0000 7f2c8790d700 1 -- 172.21.15.156:0/1232310268 >> 172.21.15.156:0/1232310268 conn(0x7f2c800fc4f0 msgr2=0x7f2c80102540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:32:04.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.616+0000 7f2c8790d700 1 -- 172.21.15.156:0/1232310268 shutdown_connections 2023-09-30T14:32:04.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:04.616+0000 7f2c8790d700 1 -- 172.21.15.156:0/1232310268 wait complete. 2023-09-30T14:32:04.617 INFO:teuthology.orchestra.run.smithi156.stderr:dumped monmap epoch 3 2023-09-30T14:32:04.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:32:04.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: mon.c calling monitor election 2023-09-30T14:32:04.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: mon.a calling monitor election 2023-09-30T14:32:04.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:04.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:32:04.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:04.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:04.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:04.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:04.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:04.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:04.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:04.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-30T14:32:04.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: monmap e3: 3 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],b=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-30T14:32:04.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: fsmap 2023-09-30T14:32:04.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: osdmap e4: 0 total, 0 up, 0 in 2023-09-30T14:32:04.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: mgrmap e13: x(active, since 52s) 2023-09-30T14:32:04.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-30T14:32:04.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-30T14:32:04.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-30T14:32:04.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: mon.b (rank 2) addr [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] is down (out of quorum) 2023-09-30T14:32:04.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:04.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:04.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:04.640 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:04.648 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:32:04.648 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: mon.c calling monitor election 2023-09-30T14:32:04.648 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: mon.a calling monitor election 2023-09-30T14:32:04.648 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:04.648 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:32:04.648 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:04.648 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:04.648 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:04.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:04.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:04.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:04.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:04.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-30T14:32:04.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: monmap e3: 3 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],b=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-30T14:32:04.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: fsmap 2023-09-30T14:32:04.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: osdmap e4: 0 total, 0 up, 0 in 2023-09-30T14:32:04.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: mgrmap e13: x(active, since 52s) 2023-09-30T14:32:04.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-30T14:32:04.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-30T14:32:04.649 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-30T14:32:04.650 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: mon.b (rank 2) addr [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] is down (out of quorum) 2023-09-30T14:32:04.650 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:04.650 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:04.650 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:04.650 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:05.023 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-09-30T14:32:05.023 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph config generate-minimal-conf 2023-09-30T14:32:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:32:05.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:05 smithi032 ceph-mon[98592]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:05.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:05.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:05 smithi032 ceph-mon[98592]: from='client.? 172.21.15.156:0/1232310268' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-30T14:32:05.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:05.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:05 smithi156 ceph-mon[103579]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:05.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:05.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:05 smithi156 ceph-mon[103579]: from='client.? 172.21.15.156:0/1232310268' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-30T14:32:05.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:06.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:32:06.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: mon.c calling monitor election 2023-09-30T14:32:06.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: mon.a calling monitor election 2023-09-30T14:32:06.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:06.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:32:06.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:06.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:06.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:06.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:06.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:06.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:06.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:06.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-30T14:32:06.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: monmap e3: 3 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],b=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-30T14:32:06.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: fsmap 2023-09-30T14:32:06.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: osdmap e4: 0 total, 0 up, 0 in 2023-09-30T14:32:06.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: mgrmap e13: x(active, since 52s) 2023-09-30T14:32:06.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-30T14:32:06.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-30T14:32:06.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-30T14:32:06.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: mon.b (rank 2) addr [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] is down (out of quorum) 2023-09-30T14:32:06.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:06.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:06.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:06.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:06.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:06.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:06.612 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: from='client.? 172.21.15.156:0/1232310268' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-30T14:32:06.612 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.909+0000 7f031a95d700 1 -- 172.21.15.32:0/4008027785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0314101a80 msgr2=0x7f0314101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.909+0000 7f031a95d700 1 --2- 172.21.15.32:0/4008027785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0314101a80 0x7f0314101e60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0304009910 tx=0x7f0304009c20 comp rx=0 tx=0).stop 2023-09-30T14:32:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.909+0000 7f031a95d700 1 -- 172.21.15.32:0/4008027785 shutdown_connections 2023-09-30T14:32:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.909+0000 7f031a95d700 1 --2- 172.21.15.32:0/4008027785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0314101a80 0x7f0314101e60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.909+0000 7f031a95d700 1 -- 172.21.15.32:0/4008027785 >> 172.21.15.32:0/4008027785 conn(0x7f03140fb2d0 msgr2=0x7f03140fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:32:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.909+0000 7f031a95d700 1 -- 172.21.15.32:0/4008027785 shutdown_connections 2023-09-30T14:32:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.909+0000 7f031a95d700 1 -- 172.21.15.32:0/4008027785 wait complete. 2023-09-30T14:32:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.909+0000 7f031a95d700 1 Processor -- start 2023-09-30T14:32:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.909+0000 7f031a95d700 1 -- start start 2023-09-30T14:32:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f031a95d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0314101a80 0x7f0314198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f031a95d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0314198690 0x7f031419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f031a95d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f031419d040 0x7f031419d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f031a95d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0314198d80 con 0x7f0314198690 2023-09-30T14:32:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f031a95d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03141a4db0 con 0x7f0314101a80 2023-09-30T14:32:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f031a95d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03141a4ef0 con 0x7f031419d040 2023-09-30T14:32:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f03137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0314198690 0x7f031419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f03137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0314198690 0x7f031419cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32846/0 (socket says 172.21.15.32:32846) 2023-09-30T14:32:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f03137fe700 1 -- 172.21.15.32:0/2782101879 learned_addr learned my addr 172.21.15.32:0/2782101879 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:32:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f0313fff700 1 --2- 172.21.15.32:0/2782101879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0314101a80 0x7f0314198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f0318efa700 1 --2- 172.21.15.32:0/2782101879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f031419d040 0x7f031419d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f03137fe700 1 -- 172.21.15.32:0/2782101879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f031419d040 msgr2=0x7f031419d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f03137fe700 1 --2- 172.21.15.32:0/2782101879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f031419d040 0x7f031419d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f03137fe700 1 -- 172.21.15.32:0/2782101879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0314101a80 msgr2=0x7f0314198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f03137fe700 1 --2- 172.21.15.32:0/2782101879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0314101a80 0x7f0314198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f03137fe700 1 -- 172.21.15.32:0/2782101879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03040095f0 con 0x7f0314198690 2023-09-30T14:32:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.910+0000 7f0313fff700 1 --2- 172.21.15.32:0/2782101879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0314101a80 0x7f0314198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:32:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.911+0000 7f03137fe700 1 --2- 172.21.15.32:0/2782101879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0314198690 0x7f031419cb00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f030800b780 tx=0x7f030800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:32:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.911+0000 7f03117fa700 1 -- 172.21.15.32:0/2782101879 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308004230 con 0x7f0314198690 2023-09-30T14:32:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.911+0000 7f03117fa700 1 -- 172.21.15.32:0/2782101879 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0308004390 con 0x7f0314198690 2023-09-30T14:32:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.911+0000 7f03117fa700 1 -- 172.21.15.32:0/2782101879 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308011b30 con 0x7f0314198690 2023-09-30T14:32:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.911+0000 7f031a95d700 1 -- 172.21.15.32:0/2782101879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03141a51d0 con 0x7f0314198690 2023-09-30T14:32:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.911+0000 7f031a95d700 1 -- 172.21.15.32:0/2782101879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03141a56d0 con 0x7f0314198690 2023-09-30T14:32:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.912+0000 7f03117fa700 1 -- 172.21.15.32:0/2782101879 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f0308011cd0 con 0x7f0314198690 2023-09-30T14:32:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.912+0000 7f031a95d700 1 -- 172.21.15.32:0/2782101879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0314062450 con 0x7f0314198690 2023-09-30T14:32:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.912+0000 7f03117fa700 1 --2- 172.21.15.32:0/2782101879 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f02fc038ba0 0x7f02fc03b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:06.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.912+0000 7f03117fa700 1 -- 172.21.15.32:0/2782101879 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f030804ac90 con 0x7f0314198690 2023-09-30T14:32:06.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.912+0000 7f0313fff700 1 --2- 172.21.15.32:0/2782101879 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f02fc038ba0 0x7f02fc03b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:06.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.913+0000 7f0313fff700 1 --2- 172.21.15.32:0/2782101879 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f02fc038ba0 0x7f02fc03b060 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0304006010 tx=0x7f030400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:32:06.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:06.917+0000 7f03117fa700 1 -- 172.21.15.32:0/2782101879 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f030801a400 con 0x7f0314198690 2023-09-30T14:32:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:07.119+0000 7f031a95d700 1 -- 172.21.15.32:0/2782101879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f03141994b0 con 0x7f0314198690 2023-09-30T14:32:07.120 INFO:teuthology.orchestra.run.smithi032.stdout:# minimal ceph.conf for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:32:07.120 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2023-09-30T14:32:07.120 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:32:07.121 INFO:teuthology.orchestra.run.smithi032.stdout: mon_host = [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] 2023-09-30T14:32:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:07.119+0000 7f03117fa700 1 -- 172.21.15.32:0/2782101879 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f030800f2e0 con 0x7f0314198690 2023-09-30T14:32:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:07.122+0000 7f031a95d700 1 -- 172.21.15.32:0/2782101879 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f02fc038ba0 msgr2=0x7f02fc03b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:07.122+0000 7f031a95d700 1 --2- 172.21.15.32:0/2782101879 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f02fc038ba0 0x7f02fc03b060 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0304006010 tx=0x7f030400b540 comp rx=0 tx=0).stop 2023-09-30T14:32:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:07.122+0000 7f031a95d700 1 -- 172.21.15.32:0/2782101879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0314198690 msgr2=0x7f031419cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:07.122+0000 7f031a95d700 1 --2- 172.21.15.32:0/2782101879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0314198690 0x7f031419cb00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f030800b780 tx=0x7f030800bb40 comp rx=0 tx=0).stop 2023-09-30T14:32:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:07.122+0000 7f031a95d700 1 -- 172.21.15.32:0/2782101879 shutdown_connections 2023-09-30T14:32:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:07.122+0000 7f031a95d700 1 --2- 172.21.15.32:0/2782101879 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f02fc038ba0 0x7f02fc03b060 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:07.122+0000 7f031a95d700 1 --2- 172.21.15.32:0/2782101879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f031419d040 0x7f031419d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:07.122+0000 7f031a95d700 1 --2- 172.21.15.32:0/2782101879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0314101a80 0x7f0314198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:07.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:07.122+0000 7f031a95d700 1 --2- 172.21.15.32:0/2782101879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0314198690 0x7f031419cb00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:07.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:07.122+0000 7f031a95d700 1 -- 172.21.15.32:0/2782101879 >> 172.21.15.32:0/2782101879 conn(0x7f03140fb2d0 msgr2=0x7f03141074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:32:07.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:07.122+0000 7f031a95d700 1 -- 172.21.15.32:0/2782101879 shutdown_connections 2023-09-30T14:32:07.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:07.122+0000 7f031a95d700 1 -- 172.21.15.32:0/2782101879 wait complete. 2023-09-30T14:32:07.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: mon.b calling monitor election 2023-09-30T14:32:07.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:07.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: Updating smithi032:/etc/ceph/ceph.conf 2023-09-30T14:32:07.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: Updating smithi148:/etc/ceph/ceph.conf 2023-09-30T14:32:07.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: Updating smithi156:/etc/ceph/ceph.conf 2023-09-30T14:32:07.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: mon.b calling monitor election 2023-09-30T14:32:07.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: mon.c calling monitor election 2023-09-30T14:32:07.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: mon.a calling monitor election 2023-09-30T14:32:07.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-30T14:32:07.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: monmap e3: 3 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],b=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-30T14:32:07.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: fsmap 2023-09-30T14:32:07.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: osdmap e4: 0 total, 0 up, 0 in 2023-09-30T14:32:07.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: mgrmap e13: x(active, since 54s) 2023-09-30T14:32:07.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-30T14:32:07.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: Cluster is now healthy 2023-09-30T14:32:07.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: overall HEALTH_OK 2023-09-30T14:32:07.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: Updating smithi156:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-09-30T14:32:07.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: Updating smithi032:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-09-30T14:32:07.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: Updating smithi148:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-09-30T14:32:07.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-30T14:32:07.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-30T14:32:07.464 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:07.465 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:07 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2782101879' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:07.465 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-09-30T14:32:07.466 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:32:07.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.conf 2023-09-30T14:32:07.497 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:32:07.497 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-30T14:32:07.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: mon.b calling monitor election 2023-09-30T14:32:07.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:07.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: Updating smithi032:/etc/ceph/ceph.conf 2023-09-30T14:32:07.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: Updating smithi148:/etc/ceph/ceph.conf 2023-09-30T14:32:07.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: Updating smithi156:/etc/ceph/ceph.conf 2023-09-30T14:32:07.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: mon.b calling monitor election 2023-09-30T14:32:07.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: mon.c calling monitor election 2023-09-30T14:32:07.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: mon.a calling monitor election 2023-09-30T14:32:07.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-30T14:32:07.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: monmap e3: 3 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],b=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-30T14:32:07.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: fsmap 2023-09-30T14:32:07.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: osdmap e4: 0 total, 0 up, 0 in 2023-09-30T14:32:07.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: mgrmap e13: x(active, since 54s) 2023-09-30T14:32:07.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-30T14:32:07.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: Cluster is now healthy 2023-09-30T14:32:07.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: overall HEALTH_OK 2023-09-30T14:32:07.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: Updating smithi156:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-09-30T14:32:07.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: Updating smithi032:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-09-30T14:32:07.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: Updating smithi148:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-09-30T14:32:07.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-30T14:32:07.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-30T14:32:07.559 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:07.559 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:07 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2782101879' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:07.565 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-09-30T14:32:07.566 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.conf 2023-09-30T14:32:07.597 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: mon.b calling monitor election 2023-09-30T14:32:07.599 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:07.599 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: Updating smithi032:/etc/ceph/ceph.conf 2023-09-30T14:32:07.599 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: Updating smithi148:/etc/ceph/ceph.conf 2023-09-30T14:32:07.599 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: Updating smithi156:/etc/ceph/ceph.conf 2023-09-30T14:32:07.599 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: mon.b calling monitor election 2023-09-30T14:32:07.599 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: mon.c calling monitor election 2023-09-30T14:32:07.600 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: mon.a calling monitor election 2023-09-30T14:32:07.600 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-30T14:32:07.600 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: monmap e3: 3 mons at {a=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],b=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-09-30T14:32:07.600 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: fsmap 2023-09-30T14:32:07.600 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: osdmap e4: 0 total, 0 up, 0 in 2023-09-30T14:32:07.600 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: mgrmap e13: x(active, since 54s) 2023-09-30T14:32:07.600 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-30T14:32:07.600 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: Cluster is now healthy 2023-09-30T14:32:07.600 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: overall HEALTH_OK 2023-09-30T14:32:07.600 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: Updating smithi156:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-09-30T14:32:07.600 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: Updating smithi032:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-09-30T14:32:07.600 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: Updating smithi148:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-09-30T14:32:07.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:07.602 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-30T14:32:07.602 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-30T14:32:07.602 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:07.602 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:07 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2782101879' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:07.603 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-09-30T14:32:07.603 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-30T14:32:07.673 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-30T14:32:07.673 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.conf 2023-09-30T14:32:07.708 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-30T14:32:07.709 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-30T14:32:07.779 INFO:tasks.cephadm:Adding mgr.x on smithi032 2023-09-30T14:32:07.779 INFO:tasks.cephadm:Adding mgr.y on smithi148 2023-09-30T14:32:07.779 INFO:tasks.cephadm:Adding mgr.z on smithi156 2023-09-30T14:32:07.779 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch apply mgr '3;smithi032=x;smithi148=y;smithi156=z' 2023-09-30T14:32:08.374 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.c/config 2023-09-30T14:32:08.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:08 smithi156 ceph-mon[103579]: Reconfiguring mon.a (unknown last config time)... 2023-09-30T14:32:08.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:08 smithi156 ceph-mon[103579]: Reconfiguring daemon mon.a on smithi032 2023-09-30T14:32:08.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:08 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:08.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:08 smithi148 ceph-mon[102355]: Reconfiguring mon.a (unknown last config time)... 2023-09-30T14:32:08.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:08 smithi148 ceph-mon[102355]: Reconfiguring daemon mon.a on smithi032 2023-09-30T14:32:08.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:08 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:08.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:08 smithi032 ceph-mon[98592]: Reconfiguring mon.a (unknown last config time)... 2023-09-30T14:32:08.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:08 smithi032 ceph-mon[98592]: Reconfiguring daemon mon.a on smithi032 2023-09-30T14:32:08.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:08 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:32:09.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:09 smithi148 ceph-mon[102355]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:09.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:09.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:09.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:09 smithi148 ceph-mon[102355]: Reconfiguring mon.b (monmap changed)... 2023-09-30T14:32:09.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-30T14:32:09.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-30T14:32:09.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:09.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:09 smithi148 ceph-mon[102355]: Reconfiguring daemon mon.b on smithi148 2023-09-30T14:32:09.429 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:09 smithi156 ceph-mon[103579]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:09.429 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:09.429 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:09.429 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:09 smithi156 ceph-mon[103579]: Reconfiguring mon.b (monmap changed)... 2023-09-30T14:32:09.429 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-30T14:32:09.429 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-30T14:32:09.429 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:09.429 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:09 smithi156 ceph-mon[103579]: Reconfiguring daemon mon.b on smithi148 2023-09-30T14:32:09.580 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.580+0000 7facfea0a700 1 -- 172.21.15.156:0/1706586828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf80ff970 msgr2=0x7facf80ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:09.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.580+0000 7facfea0a700 1 --2- 172.21.15.156:0/1706586828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf80ff970 0x7facf80ffd90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7face8009910 tx=0x7face8009c20 comp rx=0 tx=0).stop 2023-09-30T14:32:09.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.580+0000 7facfea0a700 1 -- 172.21.15.156:0/1706586828 shutdown_connections 2023-09-30T14:32:09.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.580+0000 7facfea0a700 1 --2- 172.21.15.156:0/1706586828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf80ff970 0x7facf80ffd90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:09.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.580+0000 7facfea0a700 1 -- 172.21.15.156:0/1706586828 >> 172.21.15.156:0/1706586828 conn(0x7facf80faf30 msgr2=0x7facf80fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:32:09.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.580+0000 7facfea0a700 1 -- 172.21.15.156:0/1706586828 shutdown_connections 2023-09-30T14:32:09.581 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.580+0000 7facfea0a700 1 -- 172.21.15.156:0/1706586828 wait complete. 2023-09-30T14:32:09.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.581+0000 7facfea0a700 1 Processor -- start 2023-09-30T14:32:09.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.581+0000 7facfea0a700 1 -- start start 2023-09-30T14:32:09.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facfea0a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facf80ff970 0x7facf8193a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:09.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facfea0a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf8193fc0 0x7facf8199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:09.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facfea0a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facf8194440 0x7facf819b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:09.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facfea0a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf819bd60 con 0x7facf8193fc0 2023-09-30T14:32:09.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facfea0a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf819bed0 con 0x7facf80ff970 2023-09-30T14:32:09.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facfea0a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf819c090 con 0x7facf8194440 2023-09-30T14:32:09.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facfcfa7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facf8194440 0x7facf819b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:09.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facfcfa7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facf8194440 0x7facf819b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35992/0 (socket says 172.21.15.156:35992) 2023-09-30T14:32:09.584 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facfcfa7700 1 -- 172.21.15.156:0/1486106638 learned_addr learned my addr 172.21.15.156:0/1486106638 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-30T14:32:09.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facf77fe700 1 --2- 172.21.15.156:0/1486106638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf8193fc0 0x7facf8199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:09.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facf7fff700 1 --2- 172.21.15.156:0/1486106638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facf80ff970 0x7facf8193a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:09.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facfcfa7700 1 -- 172.21.15.156:0/1486106638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facf80ff970 msgr2=0x7facf8193a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:09.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facfcfa7700 1 --2- 172.21.15.156:0/1486106638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facf80ff970 0x7facf8193a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:09.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facfcfa7700 1 -- 172.21.15.156:0/1486106638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf8193fc0 msgr2=0x7facf8199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:09.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facfcfa7700 1 --2- 172.21.15.156:0/1486106638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf8193fc0 0x7facf8199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:09.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facfcfa7700 1 -- 172.21.15.156:0/1486106638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face80095f0 con 0x7facf8194440 2023-09-30T14:32:09.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.582+0000 7facfcfa7700 1 --2- 172.21.15.156:0/1486106638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facf8194440 0x7facf819b5c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7facf000ba50 tx=0x7facf000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:32:09.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.583+0000 7facf57fa700 1 -- 172.21.15.156:0/1486106638 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf000c930 con 0x7facf8194440 2023-09-30T14:32:09.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.583+0000 7facf57fa700 1 -- 172.21.15.156:0/1486106638 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf0014440 con 0x7facf8194440 2023-09-30T14:32:09.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.583+0000 7facfea0a700 1 -- 172.21.15.156:0/1486106638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf819c370 con 0x7facf8194440 2023-09-30T14:32:09.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.583+0000 7facf57fa700 1 -- 172.21.15.156:0/1486106638 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf00075b0 con 0x7facf8194440 2023-09-30T14:32:09.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.583+0000 7facfea0a700 1 -- 172.21.15.156:0/1486106638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf819c8f0 con 0x7facf8194440 2023-09-30T14:32:09.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.584+0000 7facfea0a700 1 -- 172.21.15.156:0/1486106638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf818d370 con 0x7facf8194440 2023-09-30T14:32:09.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.584+0000 7facf57fa700 1 -- 172.21.15.156:0/1486106638 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 13) v1 ==== 45548+0+0 (secure 0 0 0) 0x7facf00145b0 con 0x7facf8194440 2023-09-30T14:32:09.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.584+0000 7facf57fa700 1 --2- 172.21.15.156:0/1486106638 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7face003ae00 0x7face003d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:09.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.584+0000 7facf57fa700 1 -- 172.21.15.156:0/1486106638 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7facf004ad80 con 0x7facf8194440 2023-09-30T14:32:09.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.585+0000 7facf7fff700 1 --2- 172.21.15.156:0/1486106638 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7face003ae00 0x7face003d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:09.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.585+0000 7facf7fff700 1 --2- 172.21.15.156:0/1486106638 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7face003ae00 0x7face003d2c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7face8006010 tx=0x7face80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:32:09.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.589+0000 7facf57fa700 1 -- 172.21.15.156:0/1486106638 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7facf0012910 con 0x7facf8194440 2023-09-30T14:32:09.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:09 smithi032 ceph-mon[98592]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:09.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:09.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:09 smithi032 ceph-mon[98592]: Reconfiguring mon.b (monmap changed)... 2023-09-30T14:32:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-30T14:32:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-30T14:32:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:09 smithi032 ceph-mon[98592]: Reconfiguring daemon mon.b on smithi148 2023-09-30T14:32:09.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.796+0000 7facfea0a700 1 -- 172.21.15.156:0/1486106638 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi032=x;smithi148=y;smithi156=z", "target": ["mon-mgr", ""]}) v1 -- 0x7facf802d240 con 0x7face003ae00 2023-09-30T14:32:09.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.803+0000 7facf57fa700 1 -- 172.21.15.156:0/1486106638 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7facf802d240 con 0x7face003ae00 2023-09-30T14:32:09.803 INFO:teuthology.orchestra.run.smithi156.stdout:Scheduled mgr update... 2023-09-30T14:32:09.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.805+0000 7facfea0a700 1 -- 172.21.15.156:0/1486106638 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7face003ae00 msgr2=0x7face003d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:09.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.805+0000 7facfea0a700 1 --2- 172.21.15.156:0/1486106638 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7face003ae00 0x7face003d2c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7face8006010 tx=0x7face80058e0 comp rx=0 tx=0).stop 2023-09-30T14:32:09.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.805+0000 7facfea0a700 1 -- 172.21.15.156:0/1486106638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facf8194440 msgr2=0x7facf819b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:09.805 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.805+0000 7facfea0a700 1 --2- 172.21.15.156:0/1486106638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facf8194440 0x7facf819b5c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7facf000ba50 tx=0x7facf000bd60 comp rx=0 tx=0).stop 2023-09-30T14:32:09.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.805+0000 7facfea0a700 1 -- 172.21.15.156:0/1486106638 shutdown_connections 2023-09-30T14:32:09.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.805+0000 7facfea0a700 1 --2- 172.21.15.156:0/1486106638 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7face003ae00 0x7face003d2c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:09.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.805+0000 7facfea0a700 1 --2- 172.21.15.156:0/1486106638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7facf8194440 0x7facf819b5c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:09.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.805+0000 7facfea0a700 1 --2- 172.21.15.156:0/1486106638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facf80ff970 0x7facf8193a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:09.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.805+0000 7facfea0a700 1 --2- 172.21.15.156:0/1486106638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facf8193fc0 0x7facf8199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:09.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.805+0000 7facfea0a700 1 -- 172.21.15.156:0/1486106638 >> 172.21.15.156:0/1486106638 conn(0x7facf80faf30 msgr2=0x7facf80fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:32:09.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.805+0000 7facfea0a700 1 -- 172.21.15.156:0/1486106638 shutdown_connections 2023-09-30T14:32:09.806 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:09.805+0000 7facfea0a700 1 -- 172.21.15.156:0/1486106638 wait complete. 2023-09-30T14:32:10.330 DEBUG:teuthology.orchestra.run.smithi148:mgr.y> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mgr.y.service 2023-09-30T14:32:10.333 DEBUG:teuthology.orchestra.run.smithi156:mgr.z> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mgr.z.service 2023-09-30T14:32:10.334 INFO:tasks.cephadm:Deploying OSDs... 2023-09-30T14:32:10.335 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:32:10.335 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2023-09-30T14:32:10.357 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-30T14:32:10.357 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2023-09-30T14:32:10.363 INFO:journalctl@ceph.mgr.z.smithi156.stdout:-- Logs begin at Sat 2023-09-30 14:16:23 UTC. -- 2023-09-30T14:32:10.365 INFO:journalctl@ceph.mgr.y.smithi148.stdout:-- Logs begin at Sat 2023-09-30 14:16:26 UTC. -- 2023-09-30T14:32:10.418 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-30T14:32:10.418 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T14:32:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 384656 Links: 1 2023-09-30T14:32:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T14:32:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T14:32:10.418 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-09-30 14:31:23.138559592 +0000 2023-09-30T14:32:10.419 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-09-30 14:26:03.739901879 +0000 2023-09-30T14:32:10.419 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-09-30 14:26:03.739901879 +0000 2023-09-30T14:32:10.419 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2023-09-30T14:32:10.419 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-30T14:32:10.485 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:10 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:10.486 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:10 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:10.486 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:10 smithi156 ceph-mon[103579]: Reconfiguring mon.c (monmap changed)... 2023-09-30T14:32:10.486 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:10 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-30T14:32:10.486 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:10 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-30T14:32:10.486 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:10 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:10.486 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:10 smithi156 ceph-mon[103579]: Reconfiguring daemon mon.c on smithi156 2023-09-30T14:32:10.486 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:10 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:10.490 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-09-30T14:32:10.490 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-09-30T14:32:10.490 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000132725 s, 3.9 MB/s 2023-09-30T14:32:10.491 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-30T14:32:10.553 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2023-09-30T14:32:10.612 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-30T14:32:10.612 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T14:32:10.613 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 389660 Links: 1 2023-09-30T14:32:10.613 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T14:32:10.613 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T14:32:10.613 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-09-30 14:31:23.202558398 +0000 2023-09-30T14:32:10.613 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-09-30 14:26:04.246892310 +0000 2023-09-30T14:32:10.613 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-09-30 14:26:04.246892310 +0000 2023-09-30T14:32:10.613 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2023-09-30T14:32:10.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-30T14:32:10.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:10 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:10.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:10 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:10.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:10 smithi032 ceph-mon[98592]: Reconfiguring mon.c (monmap changed)... 2023-09-30T14:32:10.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:10 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-30T14:32:10.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:10 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-30T14:32:10.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:10 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:10.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:10 smithi032 ceph-mon[98592]: Reconfiguring daemon mon.c on smithi156 2023-09-30T14:32:10.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:10 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:10.680 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-09-30T14:32:10.681 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-09-30T14:32:10.681 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000104284 s, 4.9 MB/s 2023-09-30T14:32:10.682 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-30T14:32:10.741 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2023-09-30T14:32:10.800 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-30T14:32:10.801 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T14:32:10.801 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 389711 Links: 1 2023-09-30T14:32:10.801 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T14:32:10.801 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T14:32:10.801 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-09-30 14:31:23.264557242 +0000 2023-09-30T14:32:10.801 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-09-30 14:26:04.745882892 +0000 2023-09-30T14:32:10.801 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-09-30 14:26:04.745882892 +0000 2023-09-30T14:32:10.801 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2023-09-30T14:32:10.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-30T14:32:10.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:10 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:10.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:10 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:10.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:10 smithi148 ceph-mon[102355]: Reconfiguring mon.c (monmap changed)... 2023-09-30T14:32:10.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:10 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-30T14:32:10.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:10 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-30T14:32:10.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:10 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:10.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:10 smithi148 ceph-mon[102355]: Reconfiguring daemon mon.c on smithi156 2023-09-30T14:32:10.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:10 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:10.868 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-09-30T14:32:10.868 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-09-30T14:32:10.868 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000104352 s, 4.9 MB/s 2023-09-30T14:32:10.871 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-30T14:32:10.930 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2023-09-30T14:32:10.989 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-30T14:32:10.989 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T14:32:10.989 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 393273 Links: 1 2023-09-30T14:32:10.989 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T14:32:10.989 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T14:32:10.989 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-09-30 14:31:23.322556160 +0000 2023-09-30T14:32:10.989 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-09-30 14:26:05.242873512 +0000 2023-09-30T14:32:10.989 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-09-30 14:26:05.242873512 +0000 2023-09-30T14:32:10.989 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2023-09-30T14:32:10.990 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-30T14:32:11.055 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-09-30T14:32:11.055 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-09-30T14:32:11.055 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000107503 s, 4.8 MB/s 2023-09-30T14:32:11.056 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-30T14:32:11.114 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-09-30T14:32:11.114 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2023-09-30T14:32:11.135 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-30T14:32:11.135 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2023-09-30T14:32:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-30T14:32:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T14:32:11.196 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 6h/6d Inode: 385722 Links: 1 2023-09-30T14:32:11.196 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T14:32:11.196 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T14:32:11.196 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2023-09-30 14:31:58.755677340 +0000 2023-09-30T14:32:11.196 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2023-09-30 14:25:51.922126322 +0000 2023-09-30T14:32:11.196 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2023-09-30 14:25:51.922126322 +0000 2023-09-30T14:32:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: - 2023-09-30T14:32:11.197 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-30T14:32:11.266 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2023-09-30T14:32:11.266 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2023-09-30T14:32:11.267 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000268421 s, 1.9 MB/s 2023-09-30T14:32:11.268 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-30T14:32:11.328 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2023-09-30T14:32:11.388 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-30T14:32:11.388 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T14:32:11.388 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 6h/6d Inode: 385805 Links: 1 2023-09-30T14:32:11.388 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T14:32:11.388 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T14:32:11.388 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2023-09-30 14:31:58.755677340 +0000 2023-09-30T14:32:11.388 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2023-09-30 14:25:52.426114229 +0000 2023-09-30T14:32:11.388 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2023-09-30 14:25:52.426114229 +0000 2023-09-30T14:32:11.389 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: - 2023-09-30T14:32:11.389 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-30T14:32:11.456 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2023-09-30T14:32:11.456 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2023-09-30T14:32:11.456 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000234449 s, 2.2 MB/s 2023-09-30T14:32:11.457 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-30T14:32:11.517 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2023-09-30T14:32:11.575 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-30T14:32:11.576 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T14:32:11.576 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 6h/6d Inode: 389269 Links: 1 2023-09-30T14:32:11.576 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T14:32:11.576 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T14:32:11.576 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2023-09-30 14:31:58.755677340 +0000 2023-09-30T14:32:11.576 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2023-09-30 14:25:52.926102232 +0000 2023-09-30T14:32:11.576 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2023-09-30 14:25:52.926102232 +0000 2023-09-30T14:32:11.576 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: - 2023-09-30T14:32:11.577 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-30T14:32:11.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:11 smithi148 ceph-mon[102355]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:11.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:11 smithi148 ceph-mon[102355]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi032=x;smithi148=y;smithi156=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:32:11.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:11 smithi148 ceph-mon[102355]: Saving service mgr spec with placement smithi032=x;smithi148=y;smithi156=z;count:3 2023-09-30T14:32:11.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:11 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:11.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:11 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:11.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:11 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:11.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:11 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:11.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:11 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:11.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:11 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:11.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:11 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-30T14:32:11.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:11 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-30T14:32:11.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:11 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-30T14:32:11.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:11 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:11.637 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2023-09-30T14:32:11.637 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2023-09-30T14:32:11.637 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000273289 s, 1.9 MB/s 2023-09-30T14:32:11.638 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-30T14:32:11.697 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2023-09-30T14:32:11.710 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:11 smithi156 ceph-mon[103579]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:11.710 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:11 smithi156 ceph-mon[103579]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi032=x;smithi148=y;smithi156=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:32:11.711 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:11 smithi156 ceph-mon[103579]: Saving service mgr spec with placement smithi032=x;smithi148=y;smithi156=z;count:3 2023-09-30T14:32:11.711 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:11 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:11.711 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:11 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:11.711 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:11 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:11.711 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:11 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:11.711 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:11 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:11.711 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:11 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:11.711 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:11 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-30T14:32:11.711 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:11 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-30T14:32:11.711 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:11 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-30T14:32:11.711 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:11 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:11.754 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-30T14:32:11.754 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T14:32:11.754 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 6h/6d Inode: 389099 Links: 1 2023-09-30T14:32:11.755 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T14:32:11.755 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T14:32:11.755 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2023-09-30 14:31:58.755677340 +0000 2023-09-30T14:32:11.755 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2023-09-30 14:25:53.428090187 +0000 2023-09-30T14:32:11.755 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2023-09-30 14:25:53.428090187 +0000 2023-09-30T14:32:11.755 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: - 2023-09-30T14:32:11.755 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-30T14:32:11.820 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2023-09-30T14:32:11.820 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2023-09-30T14:32:11.821 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000175496 s, 2.9 MB/s 2023-09-30T14:32:11.821 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-30T14:32:11.879 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-30T14:32:11.880 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2023-09-30T14:32:11.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:11 smithi032 ceph-mon[98592]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:11.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:11 smithi032 ceph-mon[98592]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi032=x;smithi148=y;smithi156=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:32:11.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:11 smithi032 ceph-mon[98592]: Saving service mgr spec with placement smithi032=x;smithi148=y;smithi156=z;count:3 2023-09-30T14:32:11.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-30T14:32:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-30T14:32:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-30T14:32:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:11 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:11.897 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-30T14:32:11.897 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2023-09-30T14:32:11.955 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-30T14:32:11.955 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T14:32:11.955 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 389452 Links: 1 2023-09-30T14:32:11.955 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T14:32:11.955 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T14:32:11.955 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-09-30 14:31:50.542752001 +0000 2023-09-30T14:32:11.955 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-09-30 14:25:51.940872820 +0000 2023-09-30T14:32:11.955 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-09-30 14:25:51.940872820 +0000 2023-09-30T14:32:11.955 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-09-30T14:32:11.956 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-30T14:32:12.021 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-09-30T14:32:12.022 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-09-30T14:32:12.022 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000101748 s, 5.0 MB/s 2023-09-30T14:32:12.023 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-30T14:32:12.090 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2023-09-30T14:32:12.148 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-30T14:32:12.148 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T14:32:12.148 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 392583 Links: 1 2023-09-30T14:32:12.148 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T14:32:12.148 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T14:32:12.149 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-09-30 14:32:12.115446181 +0000 2023-09-30T14:32:12.149 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-09-30 14:25:52.441865494 +0000 2023-09-30T14:32:12.149 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-09-30 14:25:52.441865494 +0000 2023-09-30T14:32:12.149 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-09-30T14:32:12.149 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-30T14:32:12.287 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-09-30T14:32:12.287 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-09-30T14:32:12.287 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.00011604 s, 4.4 MB/s 2023-09-30T14:32:12.288 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-30T14:32:12.308 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2023-09-30T14:32:12.368 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-30T14:32:12.368 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T14:32:12.368 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 389657 Links: 1 2023-09-30T14:32:12.368 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T14:32:12.368 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T14:32:12.368 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-09-30 14:32:12.241444379 +0000 2023-09-30T14:32:12.368 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-09-30 14:25:52.938858226 +0000 2023-09-30T14:32:12.369 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-09-30 14:25:52.938858226 +0000 2023-09-30T14:32:12.369 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-09-30T14:32:12.369 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-30T14:32:12.437 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-09-30T14:32:12.437 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-09-30T14:32:12.437 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000102595 s, 5.0 MB/s 2023-09-30T14:32:12.438 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-30T14:32:12.497 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2023-09-30T14:32:12.554 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-30T14:32:12.554 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T14:32:12.554 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 382959 Links: 1 2023-09-30T14:32:12.554 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T14:32:12.554 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T14:32:12.554 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-09-30 14:32:12.241444379 +0000 2023-09-30T14:32:12.554 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-09-30 14:25:53.445850811 +0000 2023-09-30T14:32:12.555 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-09-30 14:25:53.445850811 +0000 2023-09-30T14:32:12.555 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-09-30T14:32:12.555 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-30T14:32:12.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:12 smithi156 ceph-mon[103579]: Deploying daemon mgr.z on smithi156 2023-09-30T14:32:12.556 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:12 smithi156 systemd[1]: Starting Ceph mgr.z for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:32:12.580 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-09-30T14:32:12.581 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-09-30T14:32:12.581 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000112693 s, 4.5 MB/s 2023-09-30T14:32:12.582 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-30T14:32:12.639 INFO:tasks.cephadm:Deploying osd.0 on smithi032 with /dev/vg_nvme/lv_4... 2023-09-30T14:32:12.640 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-30T14:32:12.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:12 smithi032 ceph-mon[98592]: Deploying daemon mgr.z on smithi156 2023-09-30T14:32:12.859 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:12 smithi156 podman[105583]: 2023-09-30T14:32:12.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:12 smithi148 ceph-mon[102355]: Deploying daemon mgr.z on smithi156 2023-09-30T14:32:13.158 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:12 smithi156 bash[105583]: 189058b0f1d27e17f9f5610cd4bf44fc952cb360f1bd6c849f107d92b3cf1bed 2023-09-30T14:32:13.158 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:12 smithi156 ceph-mgr[105708]: -- 172.21.15.156:0/62224688 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5560c7f331e0 con 0x5560c7f08800 2023-09-30T14:32:13.158 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:12 smithi156 systemd[1]: Started Ceph mgr.z for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:32:13.158 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:12 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:12.990+0000 7f09b3132200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-30T14:32:13.159 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:13 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:13.069+0000 7f09b3132200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-30T14:32:13.159 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:13 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:13.158+0000 7f09b3132200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-30T14:32:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:32:13.411 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:13 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:13.251+0000 7f09b3132200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-30T14:32:13.719 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:13 smithi032 ceph-mon[98592]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:13.719 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:13.719 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:13.719 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:13.719 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-30T14:32:13.719 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-30T14:32:13.719 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-30T14:32:13.720 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:13.777 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:13 smithi148 ceph-mon[102355]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:13.778 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:13.778 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:13.778 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:13.778 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-30T14:32:13.778 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-30T14:32:13.778 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-30T14:32:13.778 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:13.784 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:13 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:13.421+0000 7f09b3132200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-30T14:32:13.784 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:13 smithi156 ceph-mon[103579]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:13.784 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:13.784 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:13.784 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:13.785 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-30T14:32:13.785 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-30T14:32:13.785 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-30T14:32:13.785 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:14.055 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:13 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:13.784+0000 7f09b3132200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-30T14:32:14.055 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:13 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:13.915+0000 7f09b3132200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-30T14:32:14.055 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:14 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:14.002+0000 7f09b3132200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-30T14:32:14.415 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:14 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:14.096+0000 7f09b3132200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-30T14:32:14.717 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:14 smithi148 ceph-mon[102355]: Deploying daemon mgr.y on smithi148 2023-09-30T14:32:14.718 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:14 smithi148 systemd[1]: Starting Ceph mgr.y for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:32:14.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:14 smithi156 ceph-mon[103579]: Deploying daemon mgr.y on smithi148 2023-09-30T14:32:14.805 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:14 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:14.516+0000 7f09b3132200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-30T14:32:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-30T14:32:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-30T14:32:14.868 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2023-09-30T14:32:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2023-09-30T14:32:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00797494 s, 1.3 GB/s 2023-09-30T14:32:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2023-09-30T14:32:14.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:14 smithi032 ceph-mon[98592]: Deploying daemon mgr.y on smithi148 2023-09-30T14:32:14.983 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:14 smithi148 podman[103789]: 2023-09-30T14:32:14.983 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:14 smithi148 bash[103789]: 06a5340551b0a8f867c1ed1d7ad229c8c63c6892136db6b9ad73230976c5f53c 2023-09-30T14:32:14.984 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:14 smithi148 ceph-mgr[103829]: -- 172.21.15.148:0/3159580735 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d1b49411e0 con 0x55d1b4916c00 2023-09-30T14:32:15.246 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:14 smithi148 systemd[1]: Started Ceph mgr.y for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:32:15.247 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:15 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:15.074+0000 7f8e3c6b6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-30T14:32:15.247 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:15 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:15.154+0000 7f8e3c6b6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-30T14:32:15.269 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi032:vg_nvme/lv_4 2023-09-30T14:32:15.525 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:15 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:15.246+0000 7f8e3c6b6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-30T14:32:15.526 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:15 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:15.343+0000 7f8e3c6b6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-30T14:32:15.526 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:15 smithi148 ceph-mon[102355]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:15.526 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:15 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:15.526 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:15 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:15.526 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:15 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:15.526 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:15 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:15.526 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:15 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:15.555 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:15 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:15.181+0000 7f09b3132200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-30T14:32:15.555 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:15 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:15.267+0000 7f09b3132200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-30T14:32:15.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:15 smithi156 ceph-mon[103579]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:15.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:15 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:15.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:15 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:15.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:15 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:15.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:15 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:15.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:15 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:15.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:15 smithi032 ceph-mon[98592]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:15.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:15.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:15.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:15.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:15.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:15.859 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:15 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:15.525+0000 7f8e3c6b6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-30T14:32:15.880 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:32:16.245 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:15 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:15.906+0000 7f8e3c6b6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-30T14:32:16.246 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:16.049+0000 7f8e3c6b6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-30T14:32:16.246 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:16.143+0000 7f8e3c6b6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-30T14:32:16.368 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:16 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:16.092+0000 7f09b3132200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-30T14:32:16.368 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:16 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:16.184+0000 7f09b3132200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-30T14:32:16.663 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:16 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:16.368+0000 7f09b3132200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-30T14:32:16.663 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:16 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:16.662+0000 7f09b3132200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-30T14:32:17.052 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:16 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:16.902+0000 7f09b3132200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-30T14:32:17.109 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:16.245+0000 7f8e3c6b6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-30T14:32:17.109 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:16.696+0000 7f8e3c6b6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-30T14:32:17.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.190+0000 7fec34160700 1 -- 172.21.15.32:0/909436729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2c0ffde0 msgr2=0x7fec2c100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:17.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.190+0000 7fec34160700 1 --2- 172.21.15.32:0/909436729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2c0ffde0 0x7fec2c100200 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00b0d0 tx=0x7fec1c00b3e0 comp rx=0 tx=0).stop 2023-09-30T14:32:17.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.190+0000 7fec34160700 1 -- 172.21.15.32:0/909436729 shutdown_connections 2023-09-30T14:32:17.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.190+0000 7fec34160700 1 --2- 172.21.15.32:0/909436729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c101980 0x7fec2c108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:17.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.190+0000 7fec34160700 1 --2- 172.21.15.32:0/909436729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2c0ffde0 0x7fec2c100200 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:17.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.190+0000 7fec34160700 1 --2- 172.21.15.32:0/909436729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec2c100fe0 0x7fec2c101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:17.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.190+0000 7fec34160700 1 -- 172.21.15.32:0/909436729 >> 172.21.15.32:0/909436729 conn(0x7fec2c0fb360 msgr2=0x7fec2c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:32:17.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.190+0000 7fec34160700 1 -- 172.21.15.32:0/909436729 shutdown_connections 2023-09-30T14:32:17.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.190+0000 7fec34160700 1 -- 172.21.15.32:0/909436729 wait complete. 2023-09-30T14:32:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.190+0000 7fec34160700 1 Processor -- start 2023-09-30T14:32:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.190+0000 7fec34160700 1 -- start start 2023-09-30T14:32:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.190+0000 7fec34160700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2c0ffde0 0x7fec2c196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec34160700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec2c100fe0 0x7fec2c197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec34160700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c101980 0x7fec2c19e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec34160700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c19e870 con 0x7fec2c0ffde0 2023-09-30T14:32:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec34160700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c19e9b0 con 0x7fec2c101980 2023-09-30T14:32:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec34160700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c19eb70 con 0x7fec2c100fe0 2023-09-30T14:32:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec31efc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2c0ffde0 0x7fec2c196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec31efc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2c0ffde0 0x7fec2c196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32856/0 (socket says 172.21.15.32:32856) 2023-09-30T14:32:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec31efc700 1 -- 172.21.15.32:0/688018495 learned_addr learned my addr 172.21.15.32:0/688018495 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:32:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec316fb700 1 --2- 172.21.15.32:0/688018495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec2c100fe0 0x7fec2c197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec326fd700 1 --2- 172.21.15.32:0/688018495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c101980 0x7fec2c19e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec31efc700 1 -- 172.21.15.32:0/688018495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec2c100fe0 msgr2=0x7fec2c197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec31efc700 1 --2- 172.21.15.32:0/688018495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec2c100fe0 0x7fec2c197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec31efc700 1 -- 172.21.15.32:0/688018495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c101980 msgr2=0x7fec2c19e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec31efc700 1 --2- 172.21.15.32:0/688018495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c101980 0x7fec2c19e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec31efc700 1 -- 172.21.15.32:0/688018495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c009d00 con 0x7fec2c0ffde0 2023-09-30T14:32:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec326fd700 1 --2- 172.21.15.32:0/688018495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c101980 0x7fec2c19e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:32:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec31efc700 1 --2- 172.21.15.32:0/688018495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2c0ffde0 0x7fec2c196d20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fec1c000f80 tx=0x7fec1c008f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:32:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec2affd700 1 -- 172.21.15.32:0/688018495 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c00e040 con 0x7fec2c0ffde0 2023-09-30T14:32:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec2affd700 1 -- 172.21.15.32:0/688018495 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec1c00be60 con 0x7fec2c0ffde0 2023-09-30T14:32:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec34160700 1 -- 172.21.15.32:0/688018495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec2c108c40 con 0x7fec2c0ffde0 2023-09-30T14:32:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec2affd700 1 -- 172.21.15.32:0/688018495 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c01ba50 con 0x7fec2c0ffde0 2023-09-30T14:32:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.191+0000 7fec34160700 1 -- 172.21.15.32:0/688018495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2c109140 con 0x7fec2c0ffde0 2023-09-30T14:32:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.192+0000 7fec2affd700 1 -- 172.21.15.32:0/688018495 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fec1c01d670 con 0x7fec2c0ffde0 2023-09-30T14:32:17.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.192+0000 7fec34160700 1 -- 172.21.15.32:0/688018495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec2c1073d0 con 0x7fec2c0ffde0 2023-09-30T14:32:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.192+0000 7fec2affd700 1 --2- 172.21.15.32:0/688018495 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fec14038950 0x7fec1403ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.193+0000 7fec2affd700 1 -- 172.21.15.32:0/688018495 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fec1c04c8e0 con 0x7fec2c0ffde0 2023-09-30T14:32:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.193+0000 7fec316fb700 1 --2- 172.21.15.32:0/688018495 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fec14038950 0x7fec1403ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.193+0000 7fec316fb700 1 --2- 172.21.15.32:0/688018495 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fec14038950 0x7fec1403ae10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fec20009c60 tx=0x7fec20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:32:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.198+0000 7fec2affd700 1 -- 172.21.15.32:0/688018495 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fec1c011570 con 0x7fec2c0ffde0 2023-09-30T14:32:17.305 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:17 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:17.052+0000 7f09b3132200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-30T14:32:17.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:17.412+0000 7fec34160700 1 -- 172.21.15.32:0/688018495 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fec2c02d040 con 0x7fec14038950 2023-09-30T14:32:17.675 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:17 smithi032 ceph-mon[98592]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:17.675 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:17.675 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:17.675 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:17.676 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:17.676 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:17.676 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:17.676 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:17.676 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:17.676 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-30T14:32:17.676 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-30T14:32:17.676 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:17.676 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:32:17.805 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:17 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:17.314+0000 7f09b3132200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-30T14:32:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:17 smithi156 ceph-mon[103579]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:17.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:17.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:17.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:17.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-30T14:32:17.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-30T14:32:17.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:17.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:32:17.859 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:17 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:17.379+0000 7f8e3c6b6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-30T14:32:18.103 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:17 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:17.462+0000 7f8e3c6b6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-30T14:32:18.103 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:17 smithi148 ceph-mon[102355]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:18.103 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:18.103 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:18.104 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:18.104 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:18.104 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:18.104 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:18.105 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:18.105 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:18.105 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-30T14:32:18.105 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-30T14:32:18.105 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:18.105 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:32:18.305 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:18 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:18.015+0000 7f09b3132200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-30T14:32:18.305 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:18 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:18.146+0000 7f09b3132200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-30T14:32:18.511 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:18 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:18.237+0000 7f8e3c6b6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-30T14:32:18.511 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:18 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:18.331+0000 7f8e3c6b6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-30T14:32:18.511 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:18 smithi148 ceph-mon[102355]: Reconfiguring mgr.x (unknown last config time)... 2023-09-30T14:32:18.511 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:18 smithi148 ceph-mon[102355]: Reconfiguring daemon mgr.x on smithi032 2023-09-30T14:32:18.511 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:18 smithi148 ceph-mon[102355]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:32:18.511 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:32:18.512 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:18.512 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:18.512 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:18.512 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:18.512 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:18.512 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:18.512 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:18.590 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:18 smithi032 ceph-mon[98592]: Reconfiguring mgr.x (unknown last config time)... 2023-09-30T14:32:18.727 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:18 smithi032 ceph-mon[98592]: Reconfiguring daemon mgr.x on smithi032 2023-09-30T14:32:18.728 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:18 smithi032 ceph-mon[98592]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:32:18.728 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:32:18.728 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:18.729 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:18.729 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:18.729 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:18.729 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:18.729 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:18.729 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:18.731 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:18 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:18.341+0000 7f09b3132200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-30T14:32:18.731 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:18 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:18.456+0000 7f09b3132200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-30T14:32:18.731 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:18 smithi156 ceph-mon[103579]: Reconfiguring mgr.x (unknown last config time)... 2023-09-30T14:32:18.732 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:18 smithi156 ceph-mon[103579]: Reconfiguring daemon mgr.x on smithi032 2023-09-30T14:32:18.732 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:18 smithi156 ceph-mon[103579]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:32:18.732 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:32:18.733 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:18.733 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:18.733 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:18.734 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:18.734 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:18.734 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:18.734 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:18.788 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:18 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:18.511+0000 7f8e3c6b6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-30T14:32:19.055 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:18 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:18.675+0000 7f09b3132200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-30T14:32:19.109 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:18 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:18.788+0000 7f8e3c6b6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-30T14:32:19.110 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:19 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:19.016+0000 7f8e3c6b6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-30T14:32:19.427 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:19 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:19.166+0000 7f8e3c6b6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-30T14:32:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:19 smithi156 ceph-mon[103579]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:19.805 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 14:32:19 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z[105683]: 2023-09-30T14:32:19.541+0000 7f09b3132200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-30T14:32:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:19 smithi148 ceph-mon[102355]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:19.860 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:19 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:19.433+0000 7f8e3c6b6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-30T14:32:19.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:19 smithi032 ceph-mon[98592]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:20.430 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:20 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:20.141+0000 7f8e3c6b6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-30T14:32:20.430 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:20 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:20.273+0000 7f8e3c6b6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-30T14:32:20.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:20.435+0000 7fec2affd700 1 -- 172.21.15.32:0/688018495 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 14) v1 ==== 90617+0+0 (secure 0 0 0) 0x7fec1c026410 con 0x7fec2c0ffde0 2023-09-30T14:32:20.799 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:20 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:20.469+0000 7f8e3c6b6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-30T14:32:20.800 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:20 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:20.582+0000 7f8e3c6b6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-30T14:32:20.800 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:20 smithi148 ceph-mon[102355]: Standby manager daemon z started 2023-09-30T14:32:20.800 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:20 smithi148 ceph-mon[102355]: from='mgr.? 172.21.15.156:0/3584408114' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-30T14:32:20.800 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:20 smithi148 ceph-mon[102355]: from='mgr.? 172.21.15.156:0/3584408114' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-30T14:32:20.800 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:20 smithi148 ceph-mon[102355]: from='mgr.? 172.21.15.156:0/3584408114' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-30T14:32:20.800 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:20 smithi148 ceph-mon[102355]: from='mgr.? 172.21.15.156:0/3584408114' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-30T14:32:20.800 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:20 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/174661547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf7887c5-207a-419c-8828-18179f511646"}]: dispatch 2023-09-30T14:32:20.800 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:20 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/174661547' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf7887c5-207a-419c-8828-18179f511646"}]': finished 2023-09-30T14:32:20.800 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:20 smithi148 ceph-mon[102355]: osdmap e5: 1 total, 0 up, 1 in 2023-09-30T14:32:20.801 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:20 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-30T14:32:20.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:20 smithi156 ceph-mon[103579]: Standby manager daemon z started 2023-09-30T14:32:20.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:20 smithi156 ceph-mon[103579]: from='mgr.? 172.21.15.156:0/3584408114' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-30T14:32:20.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:20 smithi156 ceph-mon[103579]: from='mgr.? 172.21.15.156:0/3584408114' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-30T14:32:20.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:20 smithi156 ceph-mon[103579]: from='mgr.? 172.21.15.156:0/3584408114' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-30T14:32:20.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:20 smithi156 ceph-mon[103579]: from='mgr.? 172.21.15.156:0/3584408114' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-30T14:32:20.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:20 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/174661547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf7887c5-207a-419c-8828-18179f511646"}]: dispatch 2023-09-30T14:32:20.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:20 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/174661547' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf7887c5-207a-419c-8828-18179f511646"}]': finished 2023-09-30T14:32:20.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:20 smithi156 ceph-mon[103579]: osdmap e5: 1 total, 0 up, 1 in 2023-09-30T14:32:20.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:20 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-30T14:32:20.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:20 smithi032 ceph-mon[98592]: Standby manager daemon z started 2023-09-30T14:32:20.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:20 smithi032 ceph-mon[98592]: from='mgr.? 172.21.15.156:0/3584408114' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-30T14:32:20.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:20 smithi032 ceph-mon[98592]: from='mgr.? 172.21.15.156:0/3584408114' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-30T14:32:20.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:20 smithi032 ceph-mon[98592]: from='mgr.? 172.21.15.156:0/3584408114' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-30T14:32:20.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:20 smithi032 ceph-mon[98592]: from='mgr.? 172.21.15.156:0/3584408114' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-30T14:32:20.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:20 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/174661547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf7887c5-207a-419c-8828-18179f511646"}]: dispatch 2023-09-30T14:32:20.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:20 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/174661547' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf7887c5-207a-419c-8828-18179f511646"}]': finished 2023-09-30T14:32:20.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:20 smithi032 ceph-mon[98592]: osdmap e5: 1 total, 0 up, 1 in 2023-09-30T14:32:20.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:20 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-30T14:32:21.109 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:20 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:20.799+0000 7f8e3c6b6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-30T14:32:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:21 smithi156 ceph-mon[103579]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:21 smithi156 ceph-mon[103579]: mgrmap e14: x(active, since 68s), standbys: z 2023-09-30T14:32:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:21 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-30T14:32:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:21 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2729782654' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:32:21.859 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 14:32:21 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y[103805]: 2023-09-30T14:32:21.664+0000 7f8e3c6b6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-30T14:32:21.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:21 smithi148 ceph-mon[102355]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:21.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:21 smithi148 ceph-mon[102355]: mgrmap e14: x(active, since 68s), standbys: z 2023-09-30T14:32:21.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:21 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-30T14:32:21.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:21 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2729782654' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:32:21.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:21 smithi032 ceph-mon[98592]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:21.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:21 smithi032 ceph-mon[98592]: mgrmap e14: x(active, since 68s), standbys: z 2023-09-30T14:32:21.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:21 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-30T14:32:21.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:21 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2729782654' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:32:22.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:22.447+0000 7fec2affd700 1 -- 172.21.15.32:0/688018495 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 15) v1 ==== 135686+0+0 (secure 0 0 0) 0x7fec1c025cb0 con 0x7fec2c0ffde0 2023-09-30T14:32:22.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:22 smithi156 ceph-mon[103579]: Standby manager daemon y started 2023-09-30T14:32:22.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:22 smithi156 ceph-mon[103579]: from='mgr.? 172.21.15.148:0/3664082811' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-30T14:32:22.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:22 smithi156 ceph-mon[103579]: from='mgr.? 172.21.15.148:0/3664082811' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-30T14:32:22.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:22 smithi156 ceph-mon[103579]: from='mgr.? 172.21.15.148:0/3664082811' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-30T14:32:22.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:22 smithi156 ceph-mon[103579]: from='mgr.? 172.21.15.148:0/3664082811' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-30T14:32:22.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:22 smithi148 ceph-mon[102355]: Standby manager daemon y started 2023-09-30T14:32:22.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:22 smithi148 ceph-mon[102355]: from='mgr.? 172.21.15.148:0/3664082811' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-30T14:32:22.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:22 smithi148 ceph-mon[102355]: from='mgr.? 172.21.15.148:0/3664082811' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-30T14:32:22.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:22 smithi148 ceph-mon[102355]: from='mgr.? 172.21.15.148:0/3664082811' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-30T14:32:22.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:22 smithi148 ceph-mon[102355]: from='mgr.? 172.21.15.148:0/3664082811' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-30T14:32:22.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:22 smithi032 ceph-mon[98592]: Standby manager daemon y started 2023-09-30T14:32:22.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:22 smithi032 ceph-mon[98592]: from='mgr.? 172.21.15.148:0/3664082811' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-30T14:32:22.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:22 smithi032 ceph-mon[98592]: from='mgr.? 172.21.15.148:0/3664082811' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-30T14:32:22.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:22 smithi032 ceph-mon[98592]: from='mgr.? 172.21.15.148:0/3664082811' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-30T14:32:22.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:22 smithi032 ceph-mon[98592]: from='mgr.? 172.21.15.148:0/3664082811' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-30T14:32:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:23 smithi156 ceph-mon[103579]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:23 smithi156 ceph-mon[103579]: mgrmap e15: x(active, since 70s), standbys: z, y 2023-09-30T14:32:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-30T14:32:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:23 smithi148 ceph-mon[102355]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:23 smithi148 ceph-mon[102355]: mgrmap e15: x(active, since 70s), standbys: z, y 2023-09-30T14:32:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-30T14:32:23.861 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:23 smithi032 ceph-mon[98592]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:23.861 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:23 smithi032 ceph-mon[98592]: mgrmap e15: x(active, since 70s), standbys: z, y 2023-09-30T14:32:23.861 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-30T14:32:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:25 smithi156 ceph-mon[103579]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:25 smithi148 ceph-mon[102355]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:25.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:25 smithi032 ceph-mon[98592]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:27 smithi032 ceph-mon[98592]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:27 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-30T14:32:27.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:27 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:27 smithi156 ceph-mon[103579]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:27 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-30T14:32:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:27 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:27 smithi148 ceph-mon[102355]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:27 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-30T14:32:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:27 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:28.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:28 smithi156 ceph-mon[103579]: Deploying daemon osd.0 on smithi032 2023-09-30T14:32:28.851 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:28 smithi032 ceph-mon[98592]: Deploying daemon osd.0 on smithi032 2023-09-30T14:32:28.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:28 smithi148 ceph-mon[102355]: Deploying daemon osd.0 on smithi032 2023-09-30T14:32:29.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:29 smithi032 ceph-mon[98592]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:29.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:29 smithi156 ceph-mon[103579]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:29.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:29 smithi148 ceph-mon[102355]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:31.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:31 smithi156 ceph-mon[103579]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:31.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:31 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:31.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:31 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:31.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:31 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:31.831 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:31 smithi032 ceph-mon[98592]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:31.831 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:31 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:31.831 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:31 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:31.832 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:31 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:31.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:31 smithi148 ceph-mon[102355]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:31.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:31 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:31.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:31 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:31.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:31 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:33.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:33 smithi032 ceph-mon[98592]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:33.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:33 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:33.570 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:33 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:33.570 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:33 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:33.570 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:33 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:33.570 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:33 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:33.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:33.614+0000 7fec2affd700 1 -- 172.21.15.32:0/688018495 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fec2c02d040 con 0x7fec14038950 2023-09-30T14:32:33.615 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 0 on host 'smithi032' 2023-09-30T14:32:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:33.616+0000 7fec34160700 1 -- 172.21.15.32:0/688018495 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fec14038950 msgr2=0x7fec1403ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:33.616+0000 7fec34160700 1 --2- 172.21.15.32:0/688018495 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fec14038950 0x7fec1403ae10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fec20009c60 tx=0x7fec20009260 comp rx=0 tx=0).stop 2023-09-30T14:32:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:33.616+0000 7fec34160700 1 -- 172.21.15.32:0/688018495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2c0ffde0 msgr2=0x7fec2c196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:33.616+0000 7fec34160700 1 --2- 172.21.15.32:0/688018495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2c0ffde0 0x7fec2c196d20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fec1c000f80 tx=0x7fec1c008f10 comp rx=0 tx=0).stop 2023-09-30T14:32:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:33.616+0000 7fec34160700 1 -- 172.21.15.32:0/688018495 shutdown_connections 2023-09-30T14:32:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:33.616+0000 7fec34160700 1 --2- 172.21.15.32:0/688018495 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fec14038950 0x7fec1403ae10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:33.616+0000 7fec34160700 1 --2- 172.21.15.32:0/688018495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c101980 0x7fec2c19e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:33.616+0000 7fec34160700 1 --2- 172.21.15.32:0/688018495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec2c0ffde0 0x7fec2c196d20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:33.616+0000 7fec34160700 1 --2- 172.21.15.32:0/688018495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec2c100fe0 0x7fec2c197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:33.616+0000 7fec34160700 1 -- 172.21.15.32:0/688018495 >> 172.21.15.32:0/688018495 conn(0x7fec2c0fb360 msgr2=0x7fec2c104510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:32:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:33.616+0000 7fec34160700 1 -- 172.21.15.32:0/688018495 shutdown_connections 2023-09-30T14:32:33.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:32:33.616+0000 7fec34160700 1 -- 172.21.15.32:0/688018495 wait complete. 2023-09-30T14:32:33.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:33 smithi156 ceph-mon[103579]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:33.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:33 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:33.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:33 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:33.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:33 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:33.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:33 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:33.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:33 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:33.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:33 smithi148 ceph-mon[102355]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:33.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:33 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:33.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:33 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:33.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:33 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:33.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:33 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:33.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:33 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:33.966 DEBUG:teuthology.orchestra.run.smithi032:osd.0> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.0.service 2023-09-30T14:32:33.968 INFO:tasks.cephadm:Deploying osd.1 on smithi148 with /dev/vg_nvme/lv_4... 2023-09-30T14:32:33.968 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-30T14:32:33.998 INFO:journalctl@ceph.osd.0.smithi032.stdout:-- Logs begin at Sat 2023-09-30 14:15:36 UTC. -- 2023-09-30T14:32:34.534 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.b/config 2023-09-30T14:32:34.791 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:34 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:34.791 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:34 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:34.791 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:34 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:34.791 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:34 smithi032 ceph-mon[98592]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:34.815 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:34 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:34.815 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:34 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:34.815 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:34 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:34.815 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:34 smithi148 ceph-mon[102355]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:35.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:34 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:35.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:34 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:35.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:34 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:35.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:34 smithi156 ceph-mon[103579]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:35.057 INFO:journalctl@ceph.osd.0.smithi032.stdout:Sep 30 14:32:34 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-0[117351]: 2023-09-30T14:32:34.992+0000 7fc70655b640 -1 osd.0 0 log_to_monitors true 2023-09-30T14:32:35.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:35 smithi148 ceph-mon[102355]: from='osd.0 [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-30T14:32:35.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:35 smithi032 ceph-mon[98592]: from='osd.0 [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-30T14:32:36.036 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-30T14:32:36.036 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-30T14:32:36.036 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2023-09-30T14:32:36.037 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2023-09-30T14:32:36.037 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0185467 s, 565 MB/s 2023-09-30T14:32:36.037 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2023-09-30T14:32:36.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:35 smithi156 ceph-mon[103579]: from='osd.0 [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-30T14:32:36.490 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi148:vg_nvme/lv_4 2023-09-30T14:32:36.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:36 smithi148 ceph-mon[102355]: from='osd.0 [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-30T14:32:36.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:36 smithi148 ceph-mon[102355]: osdmap e6: 1 total, 0 up, 1 in 2023-09-30T14:32:36.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:36 smithi148 ceph-mon[102355]: from='osd.0 [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-09-30T14:32:36.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:36 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-30T14:32:36.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:36 smithi148 ceph-mon[102355]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:36.887 INFO:journalctl@ceph.osd.0.smithi032.stdout:Sep 30 14:32:36 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-0[117351]: 2023-09-30T14:32:36.575+0000 7fc6fb3d1700 -1 osd.0 0 waiting for initial osdmap 2023-09-30T14:32:36.887 INFO:journalctl@ceph.osd.0.smithi032.stdout:Sep 30 14:32:36 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-0[117351]: 2023-09-30T14:32:36.579+0000 7fc6f59c3700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T14:32:36.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:36 smithi032 ceph-mon[98592]: from='osd.0 [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-30T14:32:36.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:36 smithi032 ceph-mon[98592]: osdmap e6: 1 total, 0 up, 1 in 2023-09-30T14:32:36.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:36 smithi032 ceph-mon[98592]: from='osd.0 [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-09-30T14:32:36.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:36 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-30T14:32:36.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:36 smithi032 ceph-mon[98592]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:37.044 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.b/config 2023-09-30T14:32:37.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:36 smithi156 ceph-mon[103579]: from='osd.0 [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-30T14:32:37.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:36 smithi156 ceph-mon[103579]: osdmap e6: 1 total, 0 up, 1 in 2023-09-30T14:32:37.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:36 smithi156 ceph-mon[103579]: from='osd.0 [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-09-30T14:32:37.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:36 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-30T14:32:37.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:36 smithi156 ceph-mon[103579]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:37.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:37 smithi148 ceph-mon[102355]: from='osd.0 [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-09-30T14:32:38.097 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:37 smithi148 ceph-mon[102355]: osdmap e7: 1 total, 0 up, 1 in 2023-09-30T14:32:38.097 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-30T14:32:38.097 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-30T14:32:38.098 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:37 smithi148 ceph-mon[102355]: Detected new or changed devices on smithi032 2023-09-30T14:32:38.098 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:38.098 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:38.098 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:32:38.098 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:37 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi032 to 17173M 2023-09-30T14:32:38.098 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:38.098 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:38.098 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:38.098 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:38.099 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:37 smithi156 ceph-mon[103579]: from='osd.0 [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-09-30T14:32:38.099 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:37 smithi156 ceph-mon[103579]: osdmap e7: 1 total, 0 up, 1 in 2023-09-30T14:32:38.099 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-30T14:32:38.099 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-30T14:32:38.099 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:37 smithi156 ceph-mon[103579]: Detected new or changed devices on smithi032 2023-09-30T14:32:38.099 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:38.100 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:38.100 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:32:38.100 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:37 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi032 to 17173M 2023-09-30T14:32:38.100 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:38.100 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:38.100 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:38.100 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:38.100 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:37 smithi032 ceph-mon[98592]: from='osd.0 [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-09-30T14:32:38.100 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:37 smithi032 ceph-mon[98592]: osdmap e7: 1 total, 0 up, 1 in 2023-09-30T14:32:38.100 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-30T14:32:38.100 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-30T14:32:38.100 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:37 smithi032 ceph-mon[98592]: Detected new or changed devices on smithi032 2023-09-30T14:32:38.101 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:38.101 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:38.101 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:32:38.101 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:37 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi032 to 17173M 2023-09-30T14:32:38.101 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:38.101 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:38.101 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:38.101 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:38.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.234+0000 7f5b878f8700 1 -- 172.21.15.148:0/3468330783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b801024e0 msgr2=0x7f5b80102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:38.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b878f8700 1 --2- 172.21.15.148:0/3468330783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b801024e0 0x7f5b80102900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009990 tx=0x7f5b7c009ca0 comp rx=0 tx=0).stop 2023-09-30T14:32:38.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b878f8700 1 -- 172.21.15.148:0/3468330783 shutdown_connections 2023-09-30T14:32:38.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b878f8700 1 --2- 172.21.15.148:0/3468330783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b800ff920 0x7f5b801094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:38.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b878f8700 1 --2- 172.21.15.148:0/3468330783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b801024e0 0x7f5b80102900 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:38.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b878f8700 1 --2- 172.21.15.148:0/3468330783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b800fef60 0x7f5b800ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:38.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b878f8700 1 -- 172.21.15.148:0/3468330783 >> 172.21.15.148:0/3468330783 conn(0x7f5b800faae0 msgr2=0x7f5b800fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:32:38.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b878f8700 1 -- 172.21.15.148:0/3468330783 shutdown_connections 2023-09-30T14:32:38.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b878f8700 1 -- 172.21.15.148:0/3468330783 wait complete. 2023-09-30T14:32:38.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b878f8700 1 Processor -- start 2023-09-30T14:32:38.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b878f8700 1 -- start start 2023-09-30T14:32:38.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b878f8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b800fef60 0x7f5b80196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:38.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b878f8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b800ff920 0x7f5b80197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:38.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b878f8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b801024e0 0x7f5b8019e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:38.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b878f8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8019e6a0 con 0x7f5b800fef60 2023-09-30T14:32:38.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b878f8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8019e7e0 con 0x7f5b800ff920 2023-09-30T14:32:38.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b878f8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8019e970 con 0x7f5b801024e0 2023-09-30T14:32:38.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.235+0000 7f5b84e93700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b800ff920 0x7f5b80197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:38.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.236+0000 7f5b84e93700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b800ff920 0x7f5b80197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51650/0 (socket says 172.21.15.148:51650) 2023-09-30T14:32:38.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.236+0000 7f5b84e93700 1 -- 172.21.15.148:0/4211484107 learned_addr learned my addr 172.21.15.148:0/4211484107 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-09-30T14:32:38.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.236+0000 7f5b85694700 1 --2- 172.21.15.148:0/4211484107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b800fef60 0x7f5b80196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:38.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.236+0000 7f5b85e95700 1 --2- 172.21.15.148:0/4211484107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b801024e0 0x7f5b8019e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:38.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.236+0000 7f5b84e93700 1 -- 172.21.15.148:0/4211484107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b801024e0 msgr2=0x7f5b8019e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:38.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.236+0000 7f5b84e93700 1 --2- 172.21.15.148:0/4211484107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b801024e0 0x7f5b8019e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:38.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.236+0000 7f5b84e93700 1 -- 172.21.15.148:0/4211484107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b800fef60 msgr2=0x7f5b80196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:38.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.236+0000 7f5b84e93700 1 --2- 172.21.15.148:0/4211484107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b800fef60 0x7f5b80196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:38.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.236+0000 7f5b84e93700 1 -- 172.21.15.148:0/4211484107 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c0095f0 con 0x7f5b800ff920 2023-09-30T14:32:38.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.236+0000 7f5b85694700 1 --2- 172.21.15.148:0/4211484107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b800fef60 0x7f5b80196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:32:38.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.236+0000 7f5b84e93700 1 --2- 172.21.15.148:0/4211484107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b800ff920 0x7f5b80197050 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000c990 tx=0x7f5b7000cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:32:38.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.236+0000 7f5b767fc700 1 -- 172.21.15.148:0/4211484107 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b70007b00 con 0x7f5b800ff920 2023-09-30T14:32:38.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.236+0000 7f5b767fc700 1 -- 172.21.15.148:0/4211484107 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b70007c60 con 0x7f5b800ff920 2023-09-30T14:32:38.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.236+0000 7f5b878f8700 1 -- 172.21.15.148:0/4211484107 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8019ec50 con 0x7f5b800ff920 2023-09-30T14:32:38.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.236+0000 7f5b767fc700 1 -- 172.21.15.148:0/4211484107 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b70018660 con 0x7f5b800ff920 2023-09-30T14:32:38.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.236+0000 7f5b878f8700 1 -- 172.21.15.148:0/4211484107 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8019f150 con 0x7f5b800ff920 2023-09-30T14:32:38.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.237+0000 7f5b878f8700 1 -- 172.21.15.148:0/4211484107 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b801085d0 con 0x7f5b800ff920 2023-09-30T14:32:38.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.238+0000 7f5b767fc700 1 -- 172.21.15.148:0/4211484107 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 15) v1 ==== 135686+0+0 (secure 0 0 0) 0x7f5b70004020 con 0x7f5b800ff920 2023-09-30T14:32:38.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.239+0000 7f5b767fc700 1 --2- 172.21.15.148:0/4211484107 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5b6c0a0f10 0x7f5b6c0a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:38.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.239+0000 7f5b767fc700 1 -- 172.21.15.148:0/4211484107 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f5b700cad70 con 0x7f5b800ff920 2023-09-30T14:32:38.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.239+0000 7f5b85694700 1 --2- 172.21.15.148:0/4211484107 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5b6c0a0f10 0x7f5b6c0a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:38.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.239+0000 7f5b85694700 1 --2- 172.21.15.148:0/4211484107 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5b6c0a0f10 0x7f5b6c0a33d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c006010 tx=0x7f5b7c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:32:38.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.242+0000 7f5b767fc700 1 -- 172.21.15.148:0/4211484107 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f5b70096070 con 0x7f5b800ff920 2023-09-30T14:32:38.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:38.440+0000 7f5b878f8700 1 -- 172.21.15.148:0/4211484107 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b8002d2a0 con 0x7f5b6c0a0f10 2023-09-30T14:32:38.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:38 smithi148 ceph-mon[102355]: purged_snaps scrub starts 2023-09-30T14:32:38.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:38 smithi148 ceph-mon[102355]: purged_snaps scrub ok 2023-09-30T14:32:38.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:38 smithi148 ceph-mon[102355]: osd.0 [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588] boot 2023-09-30T14:32:38.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:38 smithi148 ceph-mon[102355]: osdmap e8: 1 total, 1 up, 1 in 2023-09-30T14:32:38.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:38 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-30T14:32:38.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:38 smithi148 ceph-mon[102355]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:38.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:38 smithi148 ceph-mon[102355]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:32:38.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:38 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:32:38.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:38 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:32:38.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:38 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:38.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:38 smithi032 ceph-mon[98592]: purged_snaps scrub starts 2023-09-30T14:32:38.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:38 smithi032 ceph-mon[98592]: purged_snaps scrub ok 2023-09-30T14:32:38.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:38 smithi032 ceph-mon[98592]: osd.0 [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588] boot 2023-09-30T14:32:38.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:38 smithi032 ceph-mon[98592]: osdmap e8: 1 total, 1 up, 1 in 2023-09-30T14:32:38.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:38 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-30T14:32:38.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:38 smithi032 ceph-mon[98592]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:38.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:38 smithi032 ceph-mon[98592]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:32:38.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:38 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:32:38.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:38 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:32:38.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:38 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:39.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:38 smithi156 ceph-mon[103579]: purged_snaps scrub starts 2023-09-30T14:32:39.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:38 smithi156 ceph-mon[103579]: purged_snaps scrub ok 2023-09-30T14:32:39.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:38 smithi156 ceph-mon[103579]: osd.0 [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588] boot 2023-09-30T14:32:39.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:38 smithi156 ceph-mon[103579]: osdmap e8: 1 total, 1 up, 1 in 2023-09-30T14:32:39.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:38 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-30T14:32:39.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:38 smithi156 ceph-mon[103579]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-30T14:32:39.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:38 smithi156 ceph-mon[103579]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:32:39.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:38 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:32:39.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:38 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:32:39.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:38 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:39.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:39 smithi148 ceph-mon[102355]: osdmap e9: 1 total, 1 up, 1 in 2023-09-30T14:32:39.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:39 smithi032 ceph-mon[98592]: osdmap e9: 1 total, 1 up, 1 in 2023-09-30T14:32:40.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:39 smithi156 ceph-mon[103579]: osdmap e9: 1 total, 1 up, 1 in 2023-09-30T14:32:40.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:40 smithi148 ceph-mon[102355]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:40.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:40 smithi032 ceph-mon[98592]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:41.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:40 smithi156 ceph-mon[103579]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:41.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:41 smithi032 ceph-mon[98592]: from='client.? 172.21.15.148:0/1355908549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35766d2e-c839-4c77-839a-535465f18cfe"}]: dispatch 2023-09-30T14:32:41.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:41 smithi032 ceph-mon[98592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35766d2e-c839-4c77-839a-535465f18cfe"}]: dispatch 2023-09-30T14:32:41.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:41 smithi032 ceph-mon[98592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35766d2e-c839-4c77-839a-535465f18cfe"}]': finished 2023-09-30T14:32:41.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:41 smithi032 ceph-mon[98592]: osdmap e10: 2 total, 1 up, 2 in 2023-09-30T14:32:41.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:41 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-30T14:32:41.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:41 smithi032 ceph-mon[98592]: from='client.? 172.21.15.148:0/2166417082' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:32:42.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:41 smithi156 ceph-mon[103579]: from='client.? 172.21.15.148:0/1355908549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35766d2e-c839-4c77-839a-535465f18cfe"}]: dispatch 2023-09-30T14:32:42.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:41 smithi156 ceph-mon[103579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35766d2e-c839-4c77-839a-535465f18cfe"}]: dispatch 2023-09-30T14:32:42.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:41 smithi156 ceph-mon[103579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35766d2e-c839-4c77-839a-535465f18cfe"}]': finished 2023-09-30T14:32:42.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:41 smithi156 ceph-mon[103579]: osdmap e10: 2 total, 1 up, 2 in 2023-09-30T14:32:42.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:41 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-30T14:32:42.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:41 smithi156 ceph-mon[103579]: from='client.? 172.21.15.148:0/2166417082' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:32:42.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:41 smithi148 ceph-mon[102355]: from='client.? 172.21.15.148:0/1355908549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35766d2e-c839-4c77-839a-535465f18cfe"}]: dispatch 2023-09-30T14:32:42.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:41 smithi148 ceph-mon[102355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35766d2e-c839-4c77-839a-535465f18cfe"}]: dispatch 2023-09-30T14:32:42.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:41 smithi148 ceph-mon[102355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35766d2e-c839-4c77-839a-535465f18cfe"}]': finished 2023-09-30T14:32:42.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:41 smithi148 ceph-mon[102355]: osdmap e10: 2 total, 1 up, 2 in 2023-09-30T14:32:42.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:41 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-30T14:32:42.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:41 smithi148 ceph-mon[102355]: from='client.? 172.21.15.148:0/2166417082' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:32:43.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:42 smithi156 ceph-mon[103579]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:43.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:42 smithi148 ceph-mon[102355]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:43.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:42 smithi032 ceph-mon[98592]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:45.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:44 smithi148 ceph-mon[102355]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:45.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:44 smithi156 ceph-mon[103579]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:45.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:44 smithi032 ceph-mon[98592]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:47.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:46 smithi148 ceph-mon[102355]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:47.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:46 smithi156 ceph-mon[103579]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:47.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:46 smithi032 ceph-mon[98592]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:48.200 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:47 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-30T14:32:48.201 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:47 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:48.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:47 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-30T14:32:48.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:47 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:48.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:47 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-30T14:32:48.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:47 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:49.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:48 smithi156 ceph-mon[103579]: Deploying daemon osd.1 on smithi148 2023-09-30T14:32:49.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:48 smithi156 ceph-mon[103579]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:49.356 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:48 smithi148 ceph-mon[102355]: Deploying daemon osd.1 on smithi148 2023-09-30T14:32:49.356 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:48 smithi148 ceph-mon[102355]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:48 smithi032 ceph-mon[98592]: Deploying daemon osd.1 on smithi148 2023-09-30T14:32:49.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:48 smithi032 ceph-mon[98592]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:51.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:50 smithi156 ceph-mon[103579]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:51.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:50 smithi148 ceph-mon[102355]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:51.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:50 smithi032 ceph-mon[98592]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:53.063 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:52 smithi148 ceph-mon[102355]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:53.063 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:52 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:53.063 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:52 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:53.063 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:52 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:53.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:52 smithi156 ceph-mon[103579]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:53.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:52 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:53.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:52 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:53.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:52 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:53.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:52 smithi032 ceph-mon[98592]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:53.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:52 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:53.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:52 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:53.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:52 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:54.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:54.608+0000 7f5b767fc700 1 -- 172.21.15.148:0/4211484107 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5b8002d2a0 con 0x7f5b6c0a0f10 2023-09-30T14:32:54.609 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 1 on host 'smithi148' 2023-09-30T14:32:54.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:54.610+0000 7f5b878f8700 1 -- 172.21.15.148:0/4211484107 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5b6c0a0f10 msgr2=0x7f5b6c0a33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:54.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:54.610+0000 7f5b878f8700 1 --2- 172.21.15.148:0/4211484107 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5b6c0a0f10 0x7f5b6c0a33d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c006010 tx=0x7f5b7c00b540 comp rx=0 tx=0).stop 2023-09-30T14:32:54.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:54.610+0000 7f5b878f8700 1 -- 172.21.15.148:0/4211484107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b800ff920 msgr2=0x7f5b80197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:54.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:54.610+0000 7f5b878f8700 1 --2- 172.21.15.148:0/4211484107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b800ff920 0x7f5b80197050 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000c990 tx=0x7f5b7000cd50 comp rx=0 tx=0).stop 2023-09-30T14:32:54.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:54.611+0000 7f5b878f8700 1 -- 172.21.15.148:0/4211484107 shutdown_connections 2023-09-30T14:32:54.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:54.611+0000 7f5b878f8700 1 --2- 172.21.15.148:0/4211484107 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5b6c0a0f10 0x7f5b6c0a33d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:54.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:54.611+0000 7f5b878f8700 1 --2- 172.21.15.148:0/4211484107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b801024e0 0x7f5b8019e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:54.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:54.611+0000 7f5b878f8700 1 --2- 172.21.15.148:0/4211484107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b800fef60 0x7f5b80196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:54.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:54.611+0000 7f5b878f8700 1 --2- 172.21.15.148:0/4211484107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b800ff920 0x7f5b80197050 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:54.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:54.611+0000 7f5b878f8700 1 -- 172.21.15.148:0/4211484107 >> 172.21.15.148:0/4211484107 conn(0x7f5b800faae0 msgr2=0x7f5b80105710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:32:54.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:54.611+0000 7f5b878f8700 1 -- 172.21.15.148:0/4211484107 shutdown_connections 2023-09-30T14:32:54.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:32:54.611+0000 7f5b878f8700 1 -- 172.21.15.148:0/4211484107 wait complete. 2023-09-30T14:32:55.030 DEBUG:teuthology.orchestra.run.smithi148:osd.1> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.1.service 2023-09-30T14:32:55.033 INFO:tasks.cephadm:Deploying osd.2 on smithi156 with /dev/vg_nvme/lv_4... 2023-09-30T14:32:55.033 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-30T14:32:55.061 INFO:journalctl@ceph.osd.1.smithi148.stdout:-- Logs begin at Sat 2023-09-30 14:16:26 UTC. -- 2023-09-30T14:32:55.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:55 smithi148 ceph-mon[102355]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:55.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:55 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:55.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:55 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:55.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:55 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:55.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:55 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:55.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:55 smithi032 ceph-mon[98592]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:55.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:55 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:55.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:55 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:55.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:55 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:55.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:55 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:55.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:55 smithi156 ceph-mon[103579]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:55.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:55 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:55.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:55 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:55.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:55 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:55.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:55 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:55.601 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.c/config 2023-09-30T14:32:56.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:56 smithi148 ceph-mon[102355]: from='osd.1 [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-30T14:32:56.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:56 smithi148 ceph-mon[102355]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-30T14:32:56.109 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 30 14:32:55 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-1[107315]: 2023-09-30T14:32:55.948+0000 7fb9a7ac9640 -1 osd.1 0 log_to_monitors true 2023-09-30T14:32:56.369 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:56 smithi156 ceph-mon[103579]: from='osd.1 [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-30T14:32:56.369 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:56 smithi156 ceph-mon[103579]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-30T14:32:56.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:56 smithi032 ceph-mon[98592]: from='osd.1 [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-30T14:32:56.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:56 smithi032 ceph-mon[98592]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-30T14:32:57.071 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-30T14:32:57.071 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-30T14:32:57.072 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-09-30T14:32:57.072 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-09-30T14:32:57.072 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00825131 s, 1.3 GB/s 2023-09-30T14:32:57.072 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-09-30T14:32:57.359 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 30 14:32:57 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-1[107315]: 2023-09-30T14:32:57.082+0000 7fb99c93f700 -1 osd.1 0 waiting for initial osdmap 2023-09-30T14:32:57.359 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 30 14:32:57 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-1[107315]: 2023-09-30T14:32:57.085+0000 7fb996730700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T14:32:57.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:57 smithi148 ceph-mon[102355]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:57.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:57 smithi148 ceph-mon[102355]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-30T14:32:57.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:57 smithi148 ceph-mon[102355]: from='osd.1 [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:32:57.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:57 smithi148 ceph-mon[102355]: osdmap e11: 2 total, 1 up, 2 in 2023-09-30T14:32:57.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:57 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-30T14:32:57.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:57 smithi148 ceph-mon[102355]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:32:57.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:57 smithi032 ceph-mon[98592]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:57.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:57 smithi032 ceph-mon[98592]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-30T14:32:57.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:57 smithi032 ceph-mon[98592]: from='osd.1 [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:32:57.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:57 smithi032 ceph-mon[98592]: osdmap e11: 2 total, 1 up, 2 in 2023-09-30T14:32:57.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:57 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-30T14:32:57.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:57 smithi032 ceph-mon[98592]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:32:57.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:57 smithi156 ceph-mon[103579]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:57.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:57 smithi156 ceph-mon[103579]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-30T14:32:57.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:57 smithi156 ceph-mon[103579]: from='osd.1 [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:32:57.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:57 smithi156 ceph-mon[103579]: osdmap e11: 2 total, 1 up, 2 in 2023-09-30T14:32:57.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:57 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-30T14:32:57.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:57 smithi156 ceph-mon[103579]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:32:57.458 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi156:vg_nvme/lv_4 2023-09-30T14:32:58.015 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.c/config 2023-09-30T14:32:58.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2023-09-30T14:32:58.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: osdmap e12: 2 total, 1 up, 2 in 2023-09-30T14:32:58.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-30T14:32:58.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-30T14:32:58.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: Detected new or changed devices on smithi148 2023-09-30T14:32:58.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:58.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:58.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:32:58.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi148 to 17168M 2023-09-30T14:32:58.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:58.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:58.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:58.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:58.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:58.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:58.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:58.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:58.370 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2023-09-30T14:32:58.370 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: osdmap e12: 2 total, 1 up, 2 in 2023-09-30T14:32:58.370 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-30T14:32:58.370 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-30T14:32:58.370 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: Detected new or changed devices on smithi148 2023-09-30T14:32:58.370 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:58.371 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:58.371 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:32:58.371 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi148 to 17168M 2023-09-30T14:32:58.371 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:58.371 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:58.372 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:58.372 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:58.372 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:58.372 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:58.372 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:58.372 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:58.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2023-09-30T14:32:58.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: osdmap e12: 2 total, 1 up, 2 in 2023-09-30T14:32:58.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-30T14:32:58.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-30T14:32:58.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: Detected new or changed devices on smithi148 2023-09-30T14:32:58.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:58.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:58.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:32:58.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi148 to 17168M 2023-09-30T14:32:58.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:58.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:58.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:58.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:58.390 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:32:58.390 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:32:58.390 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:32:58.390 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:32:59.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.206+0000 7fe10d1a7700 1 -- 172.21.15.156:0/3886374334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe108100fe0 msgr2=0x7fe108101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:59.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.206+0000 7fe10d1a7700 1 --2- 172.21.15.156:0/3886374334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe108100fe0 0x7fe108101440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009990 tx=0x7fe0fc009ca0 comp rx=0 tx=0).stop 2023-09-30T14:32:59.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.206+0000 7fe10d1a7700 1 -- 172.21.15.156:0/3886374334 shutdown_connections 2023-09-30T14:32:59.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.206+0000 7fe10d1a7700 1 --2- 172.21.15.156:0/3886374334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe108101980 0x7fe108108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:59.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.206+0000 7fe10d1a7700 1 --2- 172.21.15.156:0/3886374334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1080ffde0 0x7fe108100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:59.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.207+0000 7fe10d1a7700 1 --2- 172.21.15.156:0/3886374334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe108100fe0 0x7fe108101440 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:59.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.207+0000 7fe10d1a7700 1 -- 172.21.15.156:0/3886374334 >> 172.21.15.156:0/3886374334 conn(0x7fe1080fb360 msgr2=0x7fe1080fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:32:59.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.207+0000 7fe10d1a7700 1 -- 172.21.15.156:0/3886374334 shutdown_connections 2023-09-30T14:32:59.208 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.207+0000 7fe10d1a7700 1 -- 172.21.15.156:0/3886374334 wait complete. 2023-09-30T14:32:59.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.207+0000 7fe10d1a7700 1 Processor -- start 2023-09-30T14:32:59.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.207+0000 7fe10d1a7700 1 -- start start 2023-09-30T14:32:59.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe10d1a7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1080ffde0 0x7fe108069490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:59.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe10d1a7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe108100fe0 0x7fe10819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:59.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe10d1a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe108101980 0x7fe10819df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:59.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe10d1a7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10819e6b0 con 0x7fe108101980 2023-09-30T14:32:59.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe10d1a7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10819e820 con 0x7fe108100fe0 2023-09-30T14:32:59.210 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe10d1a7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10819e9e0 con 0x7fe1080ffde0 2023-09-30T14:32:59.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe106d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1080ffde0 0x7fe108069490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:59.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe106d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1080ffde0 0x7fe108069490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36020/0 (socket says 172.21.15.156:36020) 2023-09-30T14:32:59.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe106d9d700 1 -- 172.21.15.156:0/1123865002 learned_addr learned my addr 172.21.15.156:0/1123865002 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-30T14:32:59.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe10659c700 1 --2- 172.21.15.156:0/1123865002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe108100fe0 0x7fe10819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:59.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe106d9d700 1 -- 172.21.15.156:0/1123865002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe108100fe0 msgr2=0x7fe10819b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:59.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe106d9d700 1 --2- 172.21.15.156:0/1123865002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe108100fe0 0x7fe10819b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:59.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe106d9d700 1 -- 172.21.15.156:0/1123865002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe108101980 msgr2=0x7fe10819df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:32:59.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe106d9d700 1 --2- 172.21.15.156:0/1123865002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe108101980 0x7fe10819df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:32:59.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe106d9d700 1 -- 172.21.15.156:0/1123865002 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc0095f0 con 0x7fe1080ffde0 2023-09-30T14:32:59.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe106d9d700 1 --2- 172.21.15.156:0/1123865002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1080ffde0 0x7fe108069490 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800c990 tx=0x7fe0f800cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:32:59.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe0effff700 1 -- 172.21.15.156:0/1123865002 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8007b00 con 0x7fe1080ffde0 2023-09-30T14:32:59.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe10d1a7700 1 -- 172.21.15.156:0/1123865002 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10819ecc0 con 0x7fe1080ffde0 2023-09-30T14:32:59.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe0effff700 1 -- 172.21.15.156:0/1123865002 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0f8007c60 con 0x7fe1080ffde0 2023-09-30T14:32:59.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe0effff700 1 -- 172.21.15.156:0/1123865002 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8018660 con 0x7fe1080ffde0 2023-09-30T14:32:59.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.208+0000 7fe10d1a7700 1 -- 172.21.15.156:0/1123865002 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10819f240 con 0x7fe1080ffde0 2023-09-30T14:32:59.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.209+0000 7fe10d1a7700 1 -- 172.21.15.156:0/1123865002 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe108107310 con 0x7fe1080ffde0 2023-09-30T14:32:59.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.210+0000 7fe0effff700 1 -- 172.21.15.156:0/1123865002 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 15) v1 ==== 135686+0+0 (secure 0 0 0) 0x7fe0f800f450 con 0x7fe1080ffde0 2023-09-30T14:32:59.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.211+0000 7fe0effff700 1 --2- 172.21.15.156:0/1123865002 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0f00a0f90 0x7fe0f00a3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:32:59.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.211+0000 7fe0effff700 1 -- 172.21.15.156:0/1123865002 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fe0f80cad10 con 0x7fe1080ffde0 2023-09-30T14:32:59.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.211+0000 7fe10659c700 1 --2- 172.21.15.156:0/1123865002 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0f00a0f90 0x7fe0f00a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:32:59.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.212+0000 7fe10659c700 1 --2- 172.21.15.156:0/1123865002 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0f00a0f90 0x7fe0f00a3450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe1080699f0 tx=0x7fe0fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:32:59.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.215+0000 7fe0effff700 1 -- 172.21.15.156:0/1123865002 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fe0f8095d60 con 0x7fe1080ffde0 2023-09-30T14:32:59.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:59 smithi156 ceph-mon[103579]: purged_snaps scrub starts 2023-09-30T14:32:59.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:59 smithi156 ceph-mon[103579]: purged_snaps scrub ok 2023-09-30T14:32:59.307 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:59 smithi156 ceph-mon[103579]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:59.307 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:59 smithi156 ceph-mon[103579]: osd.1 [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565] boot 2023-09-30T14:32:59.307 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:59 smithi156 ceph-mon[103579]: osdmap e13: 2 total, 2 up, 2 in 2023-09-30T14:32:59.307 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:32:59 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-30T14:32:59.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:59 smithi148 ceph-mon[102355]: purged_snaps scrub starts 2023-09-30T14:32:59.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:59 smithi148 ceph-mon[102355]: purged_snaps scrub ok 2023-09-30T14:32:59.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:59 smithi148 ceph-mon[102355]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:59.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:59 smithi148 ceph-mon[102355]: osd.1 [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565] boot 2023-09-30T14:32:59.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:59 smithi148 ceph-mon[102355]: osdmap e13: 2 total, 2 up, 2 in 2023-09-30T14:32:59.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:32:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-30T14:32:59.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:59 smithi032 ceph-mon[98592]: purged_snaps scrub starts 2023-09-30T14:32:59.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:59 smithi032 ceph-mon[98592]: purged_snaps scrub ok 2023-09-30T14:32:59.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:59 smithi032 ceph-mon[98592]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-30T14:32:59.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:59 smithi032 ceph-mon[98592]: osd.1 [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565] boot 2023-09-30T14:32:59.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:59 smithi032 ceph-mon[98592]: osdmap e13: 2 total, 2 up, 2 in 2023-09-30T14:32:59.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:32:59 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-30T14:32:59.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:32:59.411+0000 7fe10d1a7700 1 -- 172.21.15.156:0/1123865002 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe10802d330 con 0x7fe0f00a0f90 2023-09-30T14:33:00.341 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:00 smithi156 ceph-mon[103579]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:33:00.341 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:00 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:33:00.342 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:00 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:33:00.342 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:00 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:00.342 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:00 smithi156 ceph-mon[103579]: osdmap e14: 2 total, 2 up, 2 in 2023-09-30T14:33:00.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:00 smithi148 ceph-mon[102355]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:33:00.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:00 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:33:00.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:00 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:33:00.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:00 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:00.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:00 smithi148 ceph-mon[102355]: osdmap e14: 2 total, 2 up, 2 in 2023-09-30T14:33:00.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:00 smithi032 ceph-mon[98592]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:33:00.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:00 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:33:00.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:00 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:33:00.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:00 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:00.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:00 smithi032 ceph-mon[98592]: osdmap e14: 2 total, 2 up, 2 in 2023-09-30T14:33:01.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:01 smithi156 ceph-mon[103579]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:01.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:01 smithi148 ceph-mon[102355]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:01 smithi032 ceph-mon[98592]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:02 smithi156 ceph-mon[103579]: from='client.? 172.21.15.156:0/3318638692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94cc95d0-0fef-4e49-aefc-2de4626add70"}]: dispatch 2023-09-30T14:33:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:02 smithi156 ceph-mon[103579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94cc95d0-0fef-4e49-aefc-2de4626add70"}]: dispatch 2023-09-30T14:33:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:02 smithi156 ceph-mon[103579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94cc95d0-0fef-4e49-aefc-2de4626add70"}]': finished 2023-09-30T14:33:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:02 smithi156 ceph-mon[103579]: osdmap e15: 3 total, 2 up, 3 in 2023-09-30T14:33:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:02 smithi156 ceph-mon[103579]: from='client.? 172.21.15.156:0/1032569732' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:33:02.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:02 smithi148 ceph-mon[102355]: from='client.? 172.21.15.156:0/3318638692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94cc95d0-0fef-4e49-aefc-2de4626add70"}]: dispatch 2023-09-30T14:33:02.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:02 smithi148 ceph-mon[102355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94cc95d0-0fef-4e49-aefc-2de4626add70"}]: dispatch 2023-09-30T14:33:02.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:02 smithi148 ceph-mon[102355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94cc95d0-0fef-4e49-aefc-2de4626add70"}]': finished 2023-09-30T14:33:02.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:02 smithi148 ceph-mon[102355]: osdmap e15: 3 total, 2 up, 3 in 2023-09-30T14:33:02.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:02.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:02 smithi148 ceph-mon[102355]: from='client.? 172.21.15.156:0/1032569732' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:33:02.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:02 smithi032 ceph-mon[98592]: from='client.? 172.21.15.156:0/3318638692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94cc95d0-0fef-4e49-aefc-2de4626add70"}]: dispatch 2023-09-30T14:33:02.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:02 smithi032 ceph-mon[98592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94cc95d0-0fef-4e49-aefc-2de4626add70"}]: dispatch 2023-09-30T14:33:02.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:02 smithi032 ceph-mon[98592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94cc95d0-0fef-4e49-aefc-2de4626add70"}]': finished 2023-09-30T14:33:02.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:02 smithi032 ceph-mon[98592]: osdmap e15: 3 total, 2 up, 3 in 2023-09-30T14:33:02.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:02.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:02 smithi032 ceph-mon[98592]: from='client.? 172.21.15.156:0/1032569732' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:33:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:03 smithi156 ceph-mon[103579]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:03 smithi148 ceph-mon[102355]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:03.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:03 smithi032 ceph-mon[98592]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:04.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:04 smithi156 ceph-mon[103579]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:04.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:04 smithi148 ceph-mon[102355]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:04.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:04 smithi032 ceph-mon[98592]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:06.968 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:06 smithi156 ceph-mon[103579]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:07.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:06 smithi148 ceph-mon[102355]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:07.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:06 smithi032 ceph-mon[98592]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:09.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:08 smithi156 ceph-mon[103579]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:09.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:08 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-30T14:33:09.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:08 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:09.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:08 smithi156 ceph-mon[103579]: Deploying daemon osd.2 on smithi156 2023-09-30T14:33:09.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:08 smithi148 ceph-mon[102355]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:09.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:08 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-30T14:33:09.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:08 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:09.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:08 smithi148 ceph-mon[102355]: Deploying daemon osd.2 on smithi156 2023-09-30T14:33:09.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:08 smithi032 ceph-mon[98592]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:09.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:08 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-30T14:33:09.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:08 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:09.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:08 smithi032 ceph-mon[98592]: Deploying daemon osd.2 on smithi156 2023-09-30T14:33:11.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:10 smithi156 ceph-mon[103579]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:11.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:10 smithi148 ceph-mon[102355]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:11.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:10 smithi032 ceph-mon[98592]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:13.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:13 smithi156 ceph-mon[103579]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:14.748 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:14.748 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:14.748 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:33:14.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:13 smithi032 ceph-mon[98592]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:14.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:14.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:14.749 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:33:14.749 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:13 smithi148 ceph-mon[102355]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:14.749 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:14.750 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:14.750 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:33:15.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:33:15.324+0000 7fe0effff700 1 -- 172.21.15.156:0/1123865002 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe10802d330 con 0x7fe0f00a0f90 2023-09-30T14:33:15.325 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 2 on host 'smithi156' 2023-09-30T14:33:15.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:33:15.326+0000 7fe10d1a7700 1 -- 172.21.15.156:0/1123865002 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0f00a0f90 msgr2=0x7fe0f00a3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:33:15.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:33:15.326+0000 7fe10d1a7700 1 --2- 172.21.15.156:0/1123865002 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0f00a0f90 0x7fe0f00a3450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe1080699f0 tx=0x7fe0fc005fb0 comp rx=0 tx=0).stop 2023-09-30T14:33:15.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:33:15.326+0000 7fe10d1a7700 1 -- 172.21.15.156:0/1123865002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1080ffde0 msgr2=0x7fe108069490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:33:15.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:33:15.326+0000 7fe10d1a7700 1 --2- 172.21.15.156:0/1123865002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1080ffde0 0x7fe108069490 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800c990 tx=0x7fe0f800cca0 comp rx=0 tx=0).stop 2023-09-30T14:33:15.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:33:15.326+0000 7fe10d1a7700 1 -- 172.21.15.156:0/1123865002 shutdown_connections 2023-09-30T14:33:15.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:33:15.326+0000 7fe10d1a7700 1 --2- 172.21.15.156:0/1123865002 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0f00a0f90 0x7fe0f00a3450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:15.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:33:15.326+0000 7fe10d1a7700 1 --2- 172.21.15.156:0/1123865002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe108101980 0x7fe10819df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:15.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:33:15.326+0000 7fe10d1a7700 1 --2- 172.21.15.156:0/1123865002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1080ffde0 0x7fe108069490 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:15.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:33:15.326+0000 7fe10d1a7700 1 --2- 172.21.15.156:0/1123865002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe108100fe0 0x7fe10819b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:15.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:33:15.326+0000 7fe10d1a7700 1 -- 172.21.15.156:0/1123865002 >> 172.21.15.156:0/1123865002 conn(0x7fe1080fb360 msgr2=0x7fe108104450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:33:15.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:33:15.327+0000 7fe10d1a7700 1 -- 172.21.15.156:0/1123865002 shutdown_connections 2023-09-30T14:33:15.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:33:15.327+0000 7fe10d1a7700 1 -- 172.21.15.156:0/1123865002 wait complete. 2023-09-30T14:33:15.510 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:15 smithi156 ceph-mon[103579]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:15.510 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:15 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:15.511 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:15 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:15.511 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:15 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:15.511 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:15 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:33:15.511 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:15 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:15.511 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:15 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:15.511 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:15 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:15.511 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:15 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:33:15.800 DEBUG:teuthology.orchestra.run.smithi156:osd.2> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.2.service 2023-09-30T14:33:15.802 INFO:tasks.cephadm:Deploying osd.3 on smithi032 with /dev/vg_nvme/lv_3... 2023-09-30T14:33:15.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-30T14:33:15.830 INFO:journalctl@ceph.osd.2.smithi156.stdout:-- Logs begin at Sat 2023-09-30 14:16:23 UTC. -- 2023-09-30T14:33:15.835 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:15 smithi032 ceph-mon[98592]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:15.835 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:15.836 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:15.836 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:15.836 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:33:15.836 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:15.836 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:15.836 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:15.836 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:33:15.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:15 smithi148 ceph-mon[102355]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:15.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:15 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:15.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:15 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:15.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:15 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:15.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:15 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:33:15.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:15 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:15.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:15 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:15.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:15 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:15.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:15 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:33:16.384 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:33:16.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:16 smithi156 ceph-mon[103579]: from='osd.2 [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-30T14:33:16.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:16 smithi156 ceph-mon[103579]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-30T14:33:16.555 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 30 14:33:16 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-2[109388]: 2023-09-30T14:33:16.310+0000 7f77813e3640 -1 osd.2 0 log_to_monitors true 2023-09-30T14:33:16.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:16 smithi032 ceph-mon[98592]: from='osd.2 [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-30T14:33:16.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:16 smithi032 ceph-mon[98592]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-30T14:33:16.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:16 smithi148 ceph-mon[102355]: from='osd.2 [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-30T14:33:16.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:16 smithi148 ceph-mon[102355]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-30T14:33:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:17 smithi156 ceph-mon[103579]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:17.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:17 smithi156 ceph-mon[103579]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-30T14:33:17.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:17 smithi156 ceph-mon[103579]: from='osd.2 [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:33:17.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:17 smithi156 ceph-mon[103579]: osdmap e16: 3 total, 2 up, 3 in 2023-09-30T14:33:17.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:17.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:17 smithi156 ceph-mon[103579]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:33:17.807 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 30 14:33:17 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-2[109388]: 2023-09-30T14:33:17.426+0000 7f7776259700 -1 osd.2 0 waiting for initial osdmap 2023-09-30T14:33:17.807 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 30 14:33:17 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-2[109388]: 2023-09-30T14:33:17.431+0000 7f777204e700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T14:33:17.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:17 smithi148 ceph-mon[102355]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:17.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:17 smithi148 ceph-mon[102355]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-30T14:33:17.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:17 smithi148 ceph-mon[102355]: from='osd.2 [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:33:17.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:17 smithi148 ceph-mon[102355]: osdmap e16: 3 total, 2 up, 3 in 2023-09-30T14:33:17.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:17.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:17 smithi148 ceph-mon[102355]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:33:17.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:17 smithi032 ceph-mon[98592]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:17.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:17 smithi032 ceph-mon[98592]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-30T14:33:17.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:17 smithi032 ceph-mon[98592]: from='osd.2 [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:33:17.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:17 smithi032 ceph-mon[98592]: osdmap e16: 3 total, 2 up, 3 in 2023-09-30T14:33:17.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:17.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:17 smithi032 ceph-mon[98592]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:33:17.899 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-30T14:33:17.899 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-30T14:33:17.899 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2023-09-30T14:33:17.899 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2023-09-30T14:33:17.899 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00805896 s, 1.3 GB/s 2023-09-30T14:33:17.899 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2023-09-30T14:33:18.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi032:vg_nvme/lv_3 2023-09-30T14:33:18.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:18 smithi156 ceph-mon[103579]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-30T14:33:18.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:18 smithi156 ceph-mon[103579]: osdmap e17: 3 total, 2 up, 3 in 2023-09-30T14:33:18.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:18.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:18.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:18.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:18.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:33:18.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:18.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:18.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:33:18.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:18 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:18.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:18 smithi032 ceph-mon[98592]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-30T14:33:18.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:18 smithi032 ceph-mon[98592]: osdmap e17: 3 total, 2 up, 3 in 2023-09-30T14:33:18.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:18.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:18.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:18.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:18.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:33:18.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:18.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:18.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:33:18.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:18 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:18.849 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:33:18.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:18 smithi148 ceph-mon[102355]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-30T14:33:18.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:18 smithi148 ceph-mon[102355]: osdmap e17: 3 total, 2 up, 3 in 2023-09-30T14:33:18.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:18.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:18.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:18.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:18.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:33:18.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:18.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:18.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:33:18.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:18 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:19.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:19 smithi032 ceph-mon[98592]: purged_snaps scrub starts 2023-09-30T14:33:19.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:19 smithi032 ceph-mon[98592]: purged_snaps scrub ok 2023-09-30T14:33:19.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:19 smithi032 ceph-mon[98592]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:19.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:19 smithi032 ceph-mon[98592]: Detected new or changed devices on smithi156 2023-09-30T14:33:19.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:19 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi156 to 17168M 2023-09-30T14:33:19.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:19 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:19.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:19 smithi032 ceph-mon[98592]: osd.2 [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580] boot 2023-09-30T14:33:19.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:19 smithi032 ceph-mon[98592]: osdmap e18: 3 total, 3 up, 3 in 2023-09-30T14:33:19.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:19 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:19 smithi156 ceph-mon[103579]: purged_snaps scrub starts 2023-09-30T14:33:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:19 smithi156 ceph-mon[103579]: purged_snaps scrub ok 2023-09-30T14:33:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:19 smithi156 ceph-mon[103579]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:19 smithi156 ceph-mon[103579]: Detected new or changed devices on smithi156 2023-09-30T14:33:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:19 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi156 to 17168M 2023-09-30T14:33:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:19 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:19 smithi156 ceph-mon[103579]: osd.2 [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580] boot 2023-09-30T14:33:19.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:19 smithi156 ceph-mon[103579]: osdmap e18: 3 total, 3 up, 3 in 2023-09-30T14:33:19.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:19 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:19 smithi148 ceph-mon[102355]: purged_snaps scrub starts 2023-09-30T14:33:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:19 smithi148 ceph-mon[102355]: purged_snaps scrub ok 2023-09-30T14:33:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:19 smithi148 ceph-mon[102355]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-30T14:33:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:19 smithi148 ceph-mon[102355]: Detected new or changed devices on smithi156 2023-09-30T14:33:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:19 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi156 to 17168M 2023-09-30T14:33:19.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:19 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:19.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:19 smithi148 ceph-mon[102355]: osd.2 [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580] boot 2023-09-30T14:33:19.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:19 smithi148 ceph-mon[102355]: osdmap e18: 3 total, 3 up, 3 in 2023-09-30T14:33:19.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:19 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-30T14:33:20.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.064+0000 7f6f9f69c700 1 -- 172.21.15.32:0/1992285608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f981042a0 msgr2=0x7f6f98106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:33:20.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.064+0000 7f6f9f69c700 1 --2- 172.21.15.32:0/1992285608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f981042a0 0x7f6f98106690 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6f880098f0 tx=0x7f6f88009c00 comp rx=0 tx=0).stop 2023-09-30T14:33:20.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.065+0000 7f6f9f69c700 1 -- 172.21.15.32:0/1992285608 shutdown_connections 2023-09-30T14:33:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.065+0000 7f6f9f69c700 1 --2- 172.21.15.32:0/1992285608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f98106bd0 0x7f6f98109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.065+0000 7f6f9f69c700 1 --2- 172.21.15.32:0/1992285608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f98101970 0x7f6f98103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.065+0000 7f6f9f69c700 1 --2- 172.21.15.32:0/1992285608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f981042a0 0x7f6f98106690 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.065+0000 7f6f9f69c700 1 -- 172.21.15.32:0/1992285608 >> 172.21.15.32:0/1992285608 conn(0x7f6f980fb360 msgr2=0x7f6f980fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:33:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.065+0000 7f6f9f69c700 1 -- 172.21.15.32:0/1992285608 shutdown_connections 2023-09-30T14:33:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.065+0000 7f6f9f69c700 1 -- 172.21.15.32:0/1992285608 wait complete. 2023-09-30T14:33:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.065+0000 7f6f9f69c700 1 Processor -- start 2023-09-30T14:33:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.065+0000 7f6f9f69c700 1 -- start start 2023-09-30T14:33:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.065+0000 7f6f9f69c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98101970 0x7f6f98196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:33:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.065+0000 7f6f9f69c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f981042a0 0x7f6f98197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:33:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.065+0000 7f6f9f69c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f98106bd0 0x7f6f9819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:33:20.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.065+0000 7f6f9f69c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9819e770 con 0x7f6f981042a0 2023-09-30T14:33:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.065+0000 7f6f9f69c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9819e8b0 con 0x7f6f98101970 2023-09-30T14:33:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.065+0000 7f6f9f69c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9819ea40 con 0x7f6f98106bd0 2023-09-30T14:33:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.066+0000 7f6f9cc37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f981042a0 0x7f6f98197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:33:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.066+0000 7f6f9cc37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f981042a0 0x7f6f98197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32930/0 (socket says 172.21.15.32:32930) 2023-09-30T14:33:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.066+0000 7f6f9cc37700 1 -- 172.21.15.32:0/2633152168 learned_addr learned my addr 172.21.15.32:0/2633152168 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:33:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.066+0000 7f6f9dc39700 1 --2- 172.21.15.32:0/2633152168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f98106bd0 0x7f6f9819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:33:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.066+0000 7f6f9d438700 1 --2- 172.21.15.32:0/2633152168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98101970 0x7f6f98196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:33:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.066+0000 7f6f9cc37700 1 -- 172.21.15.32:0/2633152168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f98106bd0 msgr2=0x7f6f9819e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:33:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.066+0000 7f6f9cc37700 1 --2- 172.21.15.32:0/2633152168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f98106bd0 0x7f6f9819e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.066+0000 7f6f9cc37700 1 -- 172.21.15.32:0/2633152168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98101970 msgr2=0x7f6f98196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:33:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.066+0000 7f6f9cc37700 1 --2- 172.21.15.32:0/2633152168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98101970 0x7f6f98196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:20.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.066+0000 7f6f9cc37700 1 -- 172.21.15.32:0/2633152168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f88009520 con 0x7f6f981042a0 2023-09-30T14:33:20.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.066+0000 7f6f9cc37700 1 --2- 172.21.15.32:0/2633152168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f981042a0 0x7f6f98197120 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800c010 tx=0x7f6f8800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:33:20.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.224+0000 7f6f8e7fc700 1 -- 172.21.15.32:0/2633152168 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8801c070 con 0x7f6f981042a0 2023-09-30T14:33:20.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.224+0000 7f6f8e7fc700 1 -- 172.21.15.32:0/2633152168 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f8800bbf0 con 0x7f6f981042a0 2023-09-30T14:33:20.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.224+0000 7f6f9f69c700 1 -- 172.21.15.32:0/2633152168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9819ecc0 con 0x7f6f981042a0 2023-09-30T14:33:20.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.224+0000 7f6f8e7fc700 1 -- 172.21.15.32:0/2633152168 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f88004c30 con 0x7f6f981042a0 2023-09-30T14:33:20.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.224+0000 7f6f9f69c700 1 -- 172.21.15.32:0/2633152168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9819f130 con 0x7f6f981042a0 2023-09-30T14:33:20.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.225+0000 7f6f9f69c700 1 -- 172.21.15.32:0/2633152168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f980fcee0 con 0x7f6f981042a0 2023-09-30T14:33:20.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.226+0000 7f6f8e7fc700 1 -- 172.21.15.32:0/2633152168 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 15) v1 ==== 135686+0+0 (secure 0 0 0) 0x7f6f88005040 con 0x7f6f981042a0 2023-09-30T14:33:20.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.227+0000 7f6f8e7fc700 1 --2- 172.21.15.32:0/2633152168 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6f840a0f90 0x7f6f840a3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:33:20.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.227+0000 7f6f8e7fc700 1 -- 172.21.15.32:0/2633152168 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f6f880cc900 con 0x7f6f981042a0 2023-09-30T14:33:20.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.227+0000 7f6f9d438700 1 --2- 172.21.15.32:0/2633152168 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6f840a0f90 0x7f6f840a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:33:20.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.228+0000 7f6f9d438700 1 --2- 172.21.15.32:0/2633152168 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6f840a0f90 0x7f6f840a3450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6f94005fd0 tx=0x7f6f94009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:33:20.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.231+0000 7f6f8e7fc700 1 -- 172.21.15.32:0/2633152168 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f6f880977c0 con 0x7f6f981042a0 2023-09-30T14:33:20.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:20.440+0000 7f6f9f69c700 1 -- 172.21.15.32:0/2633152168 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f9802d1e0 con 0x7f6f840a0f90 2023-09-30T14:33:20.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:20 smithi156 ceph-mon[103579]: osdmap e19: 3 total, 3 up, 3 in 2023-09-30T14:33:20.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:20 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-30T14:33:20.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:20 smithi148 ceph-mon[102355]: osdmap e19: 3 total, 3 up, 3 in 2023-09-30T14:33:20.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:20 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-30T14:33:20.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:20 smithi032 ceph-mon[98592]: osdmap e19: 3 total, 3 up, 3 in 2023-09-30T14:33:20.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:20 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-30T14:33:21.578 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:21 smithi032 ceph-mon[98592]: pgmap v73: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:21.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:21 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-30T14:33:21.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:21 smithi032 ceph-mon[98592]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:33:21.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:21 smithi032 ceph-mon[98592]: osdmap e20: 3 total, 3 up, 3 in 2023-09-30T14:33:21.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:21 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:33:21.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:21 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-30T14:33:21.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:21 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:33:21.579 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:21 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:21 smithi156 ceph-mon[103579]: pgmap v73: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:21 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-30T14:33:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:21 smithi156 ceph-mon[103579]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:33:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:21 smithi156 ceph-mon[103579]: osdmap e20: 3 total, 3 up, 3 in 2023-09-30T14:33:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:21 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:33:21.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:21 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-30T14:33:21.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:21 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:33:21.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:21 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:21.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:21 smithi148 ceph-mon[102355]: pgmap v73: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:21.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:21 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-30T14:33:21.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:21 smithi148 ceph-mon[102355]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:33:21.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:21 smithi148 ceph-mon[102355]: osdmap e20: 3 total, 3 up, 3 in 2023-09-30T14:33:21.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:21 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:33:21.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:21 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-30T14:33:21.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:21 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:33:21.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:21 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:22.388 INFO:journalctl@ceph.osd.0.smithi032.stdout:Sep 30 14:33:21 smithi032 sudo[123659]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-30T14:33:22.388 INFO:journalctl@ceph.osd.0.smithi032.stdout:Sep 30 14:33:21 smithi032 sudo[123659]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-30T14:33:22.388 INFO:journalctl@ceph.osd.0.smithi032.stdout:Sep 30 14:33:21 smithi032 sudo[123659]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-30T14:33:22.388 INFO:journalctl@ceph.osd.0.smithi032.stdout:Sep 30 14:33:22 smithi032 sudo[123659]: pam_unix(sudo:session): session closed for user root 2023-09-30T14:33:22.723 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:22 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-30T14:33:22.723 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:22 smithi148 ceph-mon[102355]: osdmap e21: 3 total, 3 up, 3 in 2023-09-30T14:33:22.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:22 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-30T14:33:22.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:22 smithi156 ceph-mon[103579]: osdmap e21: 3 total, 3 up, 3 in 2023-09-30T14:33:22.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:22 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-30T14:33:22.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:22 smithi032 ceph-mon[98592]: osdmap e21: 3 total, 3 up, 3 in 2023-09-30T14:33:22.888 INFO:journalctl@ceph.osd.0.smithi032.stdout:Sep 30 14:33:22 smithi032 sudo[123696]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-30T14:33:22.888 INFO:journalctl@ceph.osd.0.smithi032.stdout:Sep 30 14:33:22 smithi032 sudo[123696]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-30T14:33:22.888 INFO:journalctl@ceph.osd.0.smithi032.stdout:Sep 30 14:33:22 smithi032 sudo[123696]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-30T14:33:22.888 INFO:journalctl@ceph.osd.0.smithi032.stdout:Sep 30 14:33:22 smithi032 sudo[123696]: pam_unix(sudo:session): session closed for user root 2023-09-30T14:33:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:22.919+0000 7f6f8e7fc700 1 -- 172.21.15.32:0/2633152168 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f6f88097b90 con 0x7f6f981042a0 2023-09-30T14:33:22.977 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 30 14:33:22 smithi148 sudo[112301]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-30T14:33:22.978 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 30 14:33:22 smithi148 sudo[112301]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-30T14:33:22.978 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 30 14:33:22 smithi148 sudo[112301]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-30T14:33:22.978 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 30 14:33:22 smithi148 sudo[112301]: pam_unix(sudo:session): session closed for user root 2023-09-30T14:33:23.359 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 30 14:33:22 smithi148 sudo[112304]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-30T14:33:23.359 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 30 14:33:22 smithi148 sudo[112304]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-30T14:33:23.360 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 30 14:33:22 smithi148 sudo[112304]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-30T14:33:23.360 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 30 14:33:22 smithi148 sudo[112304]: pam_unix(sudo:session): session closed for user root 2023-09-30T14:33:23.500 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 30 14:33:23 smithi156 sudo[114062]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-30T14:33:23.500 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 30 14:33:23 smithi156 sudo[114062]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-30T14:33:23.500 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 30 14:33:23 smithi156 sudo[114062]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-30T14:33:23.500 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 30 14:33:23 smithi156 sudo[114062]: pam_unix(sudo:session): session closed for user root 2023-09-30T14:33:23.501 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:23 smithi156 ceph-mon[103579]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:23.501 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:23 smithi156 ceph-mon[103579]: osdmap e22: 3 total, 3 up, 3 in 2023-09-30T14:33:23.501 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:23.501 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:23 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3320661677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e9c85a2-5720-4256-9285-f6dd719f0ce5"}]: dispatch 2023-09-30T14:33:23.501 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:23 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3320661677' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e9c85a2-5720-4256-9285-f6dd719f0ce5"}]': finished 2023-09-30T14:33:23.501 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:23 smithi156 ceph-mon[103579]: osdmap e23: 4 total, 3 up, 4 in 2023-09-30T14:33:23.501 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:23.501 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:23 smithi156 ceph-mon[103579]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-30T14:33:23.501 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:23.805 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 30 14:33:23 smithi156 sudo[114065]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-30T14:33:23.805 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 30 14:33:23 smithi156 sudo[114065]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-30T14:33:23.805 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 30 14:33:23 smithi156 sudo[114065]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-30T14:33:23.805 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 30 14:33:23 smithi156 sudo[114065]: pam_unix(sudo:session): session closed for user root 2023-09-30T14:33:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:23 smithi148 ceph-mon[102355]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:23 smithi148 ceph-mon[102355]: osdmap e22: 3 total, 3 up, 3 in 2023-09-30T14:33:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:23.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:23 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3320661677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e9c85a2-5720-4256-9285-f6dd719f0ce5"}]: dispatch 2023-09-30T14:33:23.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:23 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3320661677' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e9c85a2-5720-4256-9285-f6dd719f0ce5"}]': finished 2023-09-30T14:33:23.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:23 smithi148 ceph-mon[102355]: osdmap e23: 4 total, 3 up, 4 in 2023-09-30T14:33:23.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:23.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:23 smithi148 ceph-mon[102355]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-30T14:33:23.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:23.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:23 smithi032 ceph-mon[98592]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:23.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:23 smithi032 ceph-mon[98592]: osdmap e22: 3 total, 3 up, 3 in 2023-09-30T14:33:23.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:23.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:23 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3320661677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e9c85a2-5720-4256-9285-f6dd719f0ce5"}]: dispatch 2023-09-30T14:33:23.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:23 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3320661677' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e9c85a2-5720-4256-9285-f6dd719f0ce5"}]': finished 2023-09-30T14:33:23.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:23 smithi032 ceph-mon[98592]: osdmap e23: 4 total, 3 up, 4 in 2023-09-30T14:33:23.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:23.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:23 smithi032 ceph-mon[98592]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-30T14:33:23.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:24.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:23 smithi032 sudo[123762]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-30T14:33:24.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:23 smithi032 sudo[123762]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-30T14:33:24.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:23 smithi032 sudo[123762]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-30T14:33:24.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:23 smithi032 sudo[123762]: pam_unix(sudo:session): session closed for user root 2023-09-30T14:33:24.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:24 smithi032 sudo[123765]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-30T14:33:24.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:24 smithi032 sudo[123765]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-30T14:33:24.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:24 smithi032 sudo[123765]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-30T14:33:24.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:24 smithi032 sudo[123765]: pam_unix(sudo:session): session closed for user root 2023-09-30T14:33:24.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:24 smithi156 sudo[114068]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-30T14:33:24.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:24 smithi156 sudo[114068]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-30T14:33:24.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:24 smithi156 sudo[114068]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-30T14:33:24.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:24 smithi156 sudo[114068]: pam_unix(sudo:session): session closed for user root 2023-09-30T14:33:25.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:24 smithi156 sudo[114071]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-30T14:33:25.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:24 smithi156 sudo[114071]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-30T14:33:25.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:24 smithi156 sudo[114071]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-30T14:33:25.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:24 smithi156 sudo[114071]: pam_unix(sudo:session): session closed for user root 2023-09-30T14:33:25.407 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 sudo[112307]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-30T14:33:25.407 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 sudo[112307]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-30T14:33:25.408 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 sudo[112307]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-30T14:33:25.408 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 sudo[112307]: pam_unix(sudo:session): session closed for user root 2023-09-30T14:33:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:25 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:25 smithi156 ceph-mon[103579]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-30T14:33:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:25 smithi156 ceph-mon[103579]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-30T14:33:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:25 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:33:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:25 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1250609107' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:33:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:25 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:33:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:25 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:33:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:25 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:33:25.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:25 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:33:25.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:25 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:33:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 sudo[112310]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-30T14:33:25.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 sudo[112310]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-30T14:33:25.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 sudo[112310]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-30T14:33:25.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 sudo[112310]: pam_unix(sudo:session): session closed for user root 2023-09-30T14:33:25.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:25.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 ceph-mon[102355]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-30T14:33:25.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 ceph-mon[102355]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-30T14:33:25.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:33:25.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1250609107' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:33:25.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:33:25.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:33:25.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:33:25.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:33:25.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:25 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:33:25.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:25 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:25.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:25 smithi032 ceph-mon[98592]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-30T14:33:25.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:25 smithi032 ceph-mon[98592]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-30T14:33:25.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:25 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:33:25.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:25 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1250609107' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:33:25.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:25 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:33:25.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:25 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:33:25.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:25 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:33:25.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:25 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:33:25.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:25 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:33:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:26 smithi156 ceph-mon[103579]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:26 smithi156 ceph-mon[103579]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-30T14:33:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:26 smithi156 ceph-mon[103579]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-30T14:33:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:26 smithi156 ceph-mon[103579]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-30T14:33:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:33:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:33:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:33:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:26 smithi156 ceph-mon[103579]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-30T14:33:26.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:26 smithi148 ceph-mon[102355]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:26.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:26 smithi148 ceph-mon[102355]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-30T14:33:26.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:26 smithi148 ceph-mon[102355]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-30T14:33:26.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:26 smithi148 ceph-mon[102355]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-30T14:33:26.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:33:26.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:33:26.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:33:26.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:26 smithi148 ceph-mon[102355]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-30T14:33:26.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:26 smithi032 ceph-mon[98592]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:26.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:26 smithi032 ceph-mon[98592]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-30T14:33:26.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:26 smithi032 ceph-mon[98592]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-30T14:33:26.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:26 smithi032 ceph-mon[98592]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-30T14:33:26.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-30T14:33:26.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-30T14:33:26.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-30T14:33:26.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:26 smithi032 ceph-mon[98592]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-30T14:33:27.753 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:27 smithi032 ceph-mon[98592]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:27 smithi156 ceph-mon[103579]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:27 smithi148 ceph-mon[102355]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:29.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:29 smithi156 ceph-mon[103579]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:29.858 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:29 smithi032 ceph-mon[98592]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:29.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:29 smithi148 ceph-mon[102355]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:30.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:30 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-30T14:33:30.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:30 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:30.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:30 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-30T14:33:30.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:30 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:30.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:30 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-30T14:33:30.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:30 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:31.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:31 smithi156 ceph-mon[103579]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-09-30T14:33:31.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:31 smithi156 ceph-mon[103579]: Deploying daemon osd.3 on smithi032 2023-09-30T14:33:31.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:31 smithi148 ceph-mon[102355]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-09-30T14:33:31.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:31 smithi148 ceph-mon[102355]: Deploying daemon osd.3 on smithi032 2023-09-30T14:33:31.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:31 smithi032 ceph-mon[98592]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-09-30T14:33:31.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:31 smithi032 ceph-mon[98592]: Deploying daemon osd.3 on smithi032 2023-09-30T14:33:33.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:33 smithi032 ceph-mon[98592]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-09-30T14:33:33.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:33 smithi156 ceph-mon[103579]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-09-30T14:33:33.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:33 smithi148 ceph-mon[102355]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-09-30T14:33:35.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:35 smithi032 ceph-mon[98592]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-09-30T14:33:35.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:35 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:35.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:35 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:35.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:35 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:33:35.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:35 smithi156 ceph-mon[103579]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-09-30T14:33:35.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:35 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:35.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:35 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:35.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:35 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:33:35.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:35 smithi148 ceph-mon[102355]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-09-30T14:33:35.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:35 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:35.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:35 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:35.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:35 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:33:37.295 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 3 on host 'smithi032' 2023-09-30T14:33:37.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:37.295+0000 7f6f8e7fc700 1 -- 172.21.15.32:0/2633152168 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6f9802d1e0 con 0x7f6f840a0f90 2023-09-30T14:33:37.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:37.297+0000 7f6f9f69c700 1 -- 172.21.15.32:0/2633152168 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6f840a0f90 msgr2=0x7f6f840a3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:33:37.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:37.297+0000 7f6f9f69c700 1 --2- 172.21.15.32:0/2633152168 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6f840a0f90 0x7f6f840a3450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6f94005fd0 tx=0x7f6f94009310 comp rx=0 tx=0).stop 2023-09-30T14:33:37.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:37.297+0000 7f6f9f69c700 1 -- 172.21.15.32:0/2633152168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f981042a0 msgr2=0x7f6f98197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:33:37.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:37.297+0000 7f6f9f69c700 1 --2- 172.21.15.32:0/2633152168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f981042a0 0x7f6f98197120 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800c010 tx=0x7f6f8800b790 comp rx=0 tx=0).stop 2023-09-30T14:33:37.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:37.297+0000 7f6f9f69c700 1 -- 172.21.15.32:0/2633152168 shutdown_connections 2023-09-30T14:33:37.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:37.297+0000 7f6f9f69c700 1 --2- 172.21.15.32:0/2633152168 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6f840a0f90 0x7f6f840a3450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:37.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:37.297+0000 7f6f9f69c700 1 --2- 172.21.15.32:0/2633152168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f98106bd0 0x7f6f9819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:37.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:37.297+0000 7f6f9f69c700 1 --2- 172.21.15.32:0/2633152168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98101970 0x7f6f98196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:37.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:37.297+0000 7f6f9f69c700 1 --2- 172.21.15.32:0/2633152168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f981042a0 0x7f6f98197120 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:37.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:37.297+0000 7f6f9f69c700 1 -- 172.21.15.32:0/2633152168 >> 172.21.15.32:0/2633152168 conn(0x7f6f980fb360 msgr2=0x7f6f981025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:33:37.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:37.298+0000 7f6f9f69c700 1 -- 172.21.15.32:0/2633152168 shutdown_connections 2023-09-30T14:33:37.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:33:37.298+0000 7f6f9f69c700 1 -- 172.21.15.32:0/2633152168 wait complete. 2023-09-30T14:33:37.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:37 smithi032 ceph-mon[98592]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-30T14:33:37.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:37.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:37.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:37.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:33:37.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:37.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:37.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:37.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:33:37.762 DEBUG:teuthology.orchestra.run.smithi032:osd.3> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.3.service 2023-09-30T14:33:37.764 INFO:tasks.cephadm:Deploying osd.4 on smithi148 with /dev/vg_nvme/lv_3... 2023-09-30T14:33:37.765 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-30T14:33:37.797 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:37 smithi148 ceph-mon[102355]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-30T14:33:37.797 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:37.797 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:37.797 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:37.798 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:33:37.798 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:37.798 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:37.798 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:37.798 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:33:37.801 INFO:journalctl@ceph.osd.3.smithi032.stdout:-- Logs begin at Sat 2023-09-30 14:15:36 UTC. -- 2023-09-30T14:33:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:37 smithi156 ceph-mon[103579]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-30T14:33:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:33:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:33:38.334 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.b/config 2023-09-30T14:33:38.507 INFO:journalctl@ceph.osd.3.smithi032.stdout:Sep 30 14:33:38 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-3[125952]: 2023-09-30T14:33:38.479+0000 7fd39aece640 -1 osd.3 0 log_to_monitors true 2023-09-30T14:33:38.762 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:38 smithi032 ceph-mon[98592]: from='osd.3 [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-30T14:33:38.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:38 smithi156 ceph-mon[103579]: from='osd.3 [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-30T14:33:38.814 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:38 smithi148 ceph-mon[102355]: from='osd.3 [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-30T14:33:39.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:39 smithi148 ceph-mon[102355]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-09-30T14:33:39.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:39 smithi148 ceph-mon[102355]: from='osd.3 [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-30T14:33:39.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:39 smithi148 ceph-mon[102355]: osdmap e24: 4 total, 3 up, 4 in 2023-09-30T14:33:39.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:39 smithi148 ceph-mon[102355]: from='osd.3 [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-09-30T14:33:39.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:39 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:39 smithi156 ceph-mon[103579]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-09-30T14:33:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:39 smithi156 ceph-mon[103579]: from='osd.3 [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-30T14:33:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:39 smithi156 ceph-mon[103579]: osdmap e24: 4 total, 3 up, 4 in 2023-09-30T14:33:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:39 smithi156 ceph-mon[103579]: from='osd.3 [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-09-30T14:33:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:39 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:39.832 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-30T14:33:39.833 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-30T14:33:39.833 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2023-09-30T14:33:39.833 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2023-09-30T14:33:39.833 INFO:teuthology.orchestra.run.smithi148.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0155269 s, 675 MB/s 2023-09-30T14:33:39.833 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2023-09-30T14:33:39.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:39 smithi032 ceph-mon[98592]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-09-30T14:33:39.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:39 smithi032 ceph-mon[98592]: from='osd.3 [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-30T14:33:39.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:39 smithi032 ceph-mon[98592]: osdmap e24: 4 total, 3 up, 4 in 2023-09-30T14:33:39.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:39 smithi032 ceph-mon[98592]: from='osd.3 [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-09-30T14:33:39.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:39 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:39.888 INFO:journalctl@ceph.osd.3.smithi032.stdout:Sep 30 14:33:39 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-3[125952]: 2023-09-30T14:33:39.520+0000 7fd391547700 -1 osd.3 0 waiting for initial osdmap 2023-09-30T14:33:39.889 INFO:journalctl@ceph.osd.3.smithi032.stdout:Sep 30 14:33:39 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-3[125952]: 2023-09-30T14:33:39.526+0000 7fd38b338700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T14:33:40.183 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi148:vg_nvme/lv_3 2023-09-30T14:33:40.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:40 smithi032 ceph-mon[98592]: from='osd.3 [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-09-30T14:33:40.878 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:40 smithi032 ceph-mon[98592]: osdmap e25: 4 total, 3 up, 4 in 2023-09-30T14:33:40.879 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:40.879 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:40.879 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:40.879 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:40.879 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:33:40.879 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:33:40.879 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:40.879 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:40.879 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:33:40.880 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:40.880 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:40 smithi156 ceph-mon[103579]: from='osd.3 [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-09-30T14:33:40.881 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:40 smithi156 ceph-mon[103579]: osdmap e25: 4 total, 3 up, 4 in 2023-09-30T14:33:40.881 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:40.881 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:40.881 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:40.881 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:40.881 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:33:40.881 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:33:40.881 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:40.881 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:40.882 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:33:40.882 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:40.882 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.b/config 2023-09-30T14:33:40.882 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:40 smithi148 ceph-mon[102355]: from='osd.3 [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-09-30T14:33:40.882 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:40 smithi148 ceph-mon[102355]: osdmap e25: 4 total, 3 up, 4 in 2023-09-30T14:33:40.882 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:40.883 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:40.883 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:40.883 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:40.883 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:33:40.883 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:33:40.883 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:40.883 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:40.883 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:33:40.883 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:41.792 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:41 smithi148 ceph-mon[102355]: purged_snaps scrub starts 2023-09-30T14:33:41.792 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:41 smithi148 ceph-mon[102355]: purged_snaps scrub ok 2023-09-30T14:33:41.792 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:41 smithi148 ceph-mon[102355]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:41.792 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:41 smithi148 ceph-mon[102355]: Detected new or changed devices on smithi032 2023-09-30T14:33:41.793 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:41 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi032 to 8586M 2023-09-30T14:33:41.793 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:41 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:41.793 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:41 smithi148 ceph-mon[102355]: osd.3 [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583] boot 2023-09-30T14:33:41.793 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:41 smithi148 ceph-mon[102355]: osdmap e26: 4 total, 4 up, 4 in 2023-09-30T14:33:41.793 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:41 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:41 smithi156 ceph-mon[103579]: purged_snaps scrub starts 2023-09-30T14:33:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:41 smithi156 ceph-mon[103579]: purged_snaps scrub ok 2023-09-30T14:33:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:41 smithi156 ceph-mon[103579]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:41 smithi156 ceph-mon[103579]: Detected new or changed devices on smithi032 2023-09-30T14:33:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:41 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi032 to 8586M 2023-09-30T14:33:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:41 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:41 smithi156 ceph-mon[103579]: osd.3 [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583] boot 2023-09-30T14:33:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:41 smithi156 ceph-mon[103579]: osdmap e26: 4 total, 4 up, 4 in 2023-09-30T14:33:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:41 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:41.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:41 smithi032 ceph-mon[98592]: purged_snaps scrub starts 2023-09-30T14:33:41.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:41 smithi032 ceph-mon[98592]: purged_snaps scrub ok 2023-09-30T14:33:41.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:41 smithi032 ceph-mon[98592]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-09-30T14:33:41.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:41 smithi032 ceph-mon[98592]: Detected new or changed devices on smithi032 2023-09-30T14:33:41.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:41 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi032 to 8586M 2023-09-30T14:33:41.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:41 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:41.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:41 smithi032 ceph-mon[98592]: osd.3 [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583] boot 2023-09-30T14:33:41.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:41 smithi032 ceph-mon[98592]: osdmap e26: 4 total, 4 up, 4 in 2023-09-30T14:33:41.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:41 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-30T14:33:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.939+0000 7f36e2bb0700 1 -- 172.21.15.148:0/2148538983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36dc100760 msgr2=0x7f36dc100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:33:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.939+0000 7f36e2bb0700 1 --2- 172.21.15.148:0/2148538983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36dc100760 0x7f36dc100bc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f36c80099c0 tx=0x7f36c8009cd0 comp rx=0 tx=0).stop 2023-09-30T14:33:41.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.939+0000 7f36e2bb0700 1 -- 172.21.15.148:0/2148538983 shutdown_connections 2023-09-30T14:33:41.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.939+0000 7f36e2bb0700 1 --2- 172.21.15.148:0/2148538983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36dc101100 0x7f36dc1079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:41.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.939+0000 7f36e2bb0700 1 --2- 172.21.15.148:0/2148538983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36dc0ff560 0x7f36dc0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:41.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.939+0000 7f36e2bb0700 1 --2- 172.21.15.148:0/2148538983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36dc100760 0x7f36dc100bc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:41.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.939+0000 7f36e2bb0700 1 -- 172.21.15.148:0/2148538983 >> 172.21.15.148:0/2148538983 conn(0x7f36dc0faae0 msgr2=0x7f36dc0fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:33:41.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.939+0000 7f36e2bb0700 1 -- 172.21.15.148:0/2148538983 shutdown_connections 2023-09-30T14:33:41.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.939+0000 7f36e2bb0700 1 -- 172.21.15.148:0/2148538983 wait complete. 2023-09-30T14:33:41.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.939+0000 7f36e2bb0700 1 Processor -- start 2023-09-30T14:33:41.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e2bb0700 1 -- start start 2023-09-30T14:33:41.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e2bb0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36dc0ff560 0x7f36dc0fd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:33:41.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e2bb0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36dc100760 0x7f36dc19b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:33:41.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e2bb0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36dc101100 0x7f36dc19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:33:41.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e2bb0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc19e6b0 con 0x7f36dc101100 2023-09-30T14:33:41.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e2bb0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc19e820 con 0x7f36dc0ff560 2023-09-30T14:33:41.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e2bb0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc19e9e0 con 0x7f36dc100760 2023-09-30T14:33:41.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e094c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36dc0ff560 0x7f36dc0fd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:33:41.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e094c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36dc0ff560 0x7f36dc0fd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51726/0 (socket says 172.21.15.148:51726) 2023-09-30T14:33:41.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e094c700 1 -- 172.21.15.148:0/2119766892 learned_addr learned my addr 172.21.15.148:0/2119766892 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-09-30T14:33:41.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36dbfff700 1 --2- 172.21.15.148:0/2119766892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36dc100760 0x7f36dc19b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:33:41.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e094c700 1 -- 172.21.15.148:0/2119766892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36dc100760 msgr2=0x7f36dc19b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:33:41.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e094c700 1 --2- 172.21.15.148:0/2119766892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36dc100760 0x7f36dc19b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:41.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e094c700 1 -- 172.21.15.148:0/2119766892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36dc101100 msgr2=0x7f36dc19df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:33:41.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e094c700 1 --2- 172.21.15.148:0/2119766892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36dc101100 0x7f36dc19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:41.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e094c700 1 -- 172.21.15.148:0/2119766892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c80095f0 con 0x7f36dc0ff560 2023-09-30T14:33:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36dbfff700 1 --2- 172.21.15.148:0/2119766892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36dc100760 0x7f36dc19b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:33:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e094c700 1 --2- 172.21.15.148:0/2119766892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36dc0ff560 0x7f36dc0fd050 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f36d000d980 tx=0x7f36d000dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:33:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36d9ffb700 1 -- 172.21.15.148:0/2119766892 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d000de60 con 0x7f36dc0ff560 2023-09-30T14:33:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36d9ffb700 1 -- 172.21.15.148:0/2119766892 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36d00044e0 con 0x7f36dc0ff560 2023-09-30T14:33:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36d9ffb700 1 -- 172.21.15.148:0/2119766892 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0003be0 con 0x7f36dc0ff560 2023-09-30T14:33:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.940+0000 7f36e2bb0700 1 -- 172.21.15.148:0/2119766892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36dc19ecc0 con 0x7f36dc0ff560 2023-09-30T14:33:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.941+0000 7f36e2bb0700 1 -- 172.21.15.148:0/2119766892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36dc19f1c0 con 0x7f36dc0ff560 2023-09-30T14:33:41.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.942+0000 7f36e2bb0700 1 -- 172.21.15.148:0/2119766892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36dc1070e0 con 0x7f36dc0ff560 2023-09-30T14:33:41.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.943+0000 7f36d9ffb700 1 -- 172.21.15.148:0/2119766892 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f36d0009bb0 con 0x7f36dc0ff560 2023-09-30T14:33:41.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.943+0000 7f36d9ffb700 1 --2- 172.21.15.148:0/2119766892 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f36cc0a1020 0x7f36cc0a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:33:41.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.943+0000 7f36d9ffb700 1 -- 172.21.15.148:0/2119766892 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f36d00cb550 con 0x7f36dc0ff560 2023-09-30T14:33:41.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.943+0000 7f36dbfff700 1 --2- 172.21.15.148:0/2119766892 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f36cc0a1020 0x7f36cc0a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:33:41.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.944+0000 7f36dbfff700 1 --2- 172.21.15.148:0/2119766892 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f36cc0a1020 0x7f36cc0a34e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36c8000c00 tx=0x7f36c8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:33:41.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:41.947+0000 7f36d9ffb700 1 -- 172.21.15.148:0/2119766892 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f36d0095fd0 con 0x7f36dc0ff560 2023-09-30T14:33:42.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:42.143+0000 7f36e2bb0700 1 -- 172.21.15.148:0/2119766892 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f36dc02d310 con 0x7f36cc0a1020 2023-09-30T14:33:42.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:42 smithi156 ceph-mon[103579]: osdmap e27: 4 total, 4 up, 4 in 2023-09-30T14:33:42.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:33:42.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:33:42.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:42.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:42 smithi148 ceph-mon[102355]: osdmap e27: 4 total, 4 up, 4 in 2023-09-30T14:33:42.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:33:42.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:33:42.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:42.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:42 smithi032 ceph-mon[98592]: osdmap e27: 4 total, 4 up, 4 in 2023-09-30T14:33:42.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:33:42.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:33:42.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:43.791 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:43 smithi148 ceph-mon[102355]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:33:43.792 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:43 smithi148 ceph-mon[102355]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:33:43.792 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:43 smithi148 ceph-mon[102355]: osdmap e28: 4 total, 4 up, 4 in 2023-09-30T14:33:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:43 smithi156 ceph-mon[103579]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:33:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:43 smithi156 ceph-mon[103579]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:33:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:43 smithi156 ceph-mon[103579]: osdmap e28: 4 total, 4 up, 4 in 2023-09-30T14:33:43.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:43 smithi032 ceph-mon[98592]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:33:43.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:43 smithi032 ceph-mon[98592]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:33:43.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:43 smithi032 ceph-mon[98592]: osdmap e28: 4 total, 4 up, 4 in 2023-09-30T14:33:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:44 smithi156 ceph-mon[103579]: osdmap e29: 4 total, 4 up, 4 in 2023-09-30T14:33:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:44 smithi156 ceph-mon[103579]: from='client.? 172.21.15.148:0/3047243583' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d538f510-0f4b-477a-a7e3-99f203c349cf"}]: dispatch 2023-09-30T14:33:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:44 smithi156 ceph-mon[103579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d538f510-0f4b-477a-a7e3-99f203c349cf"}]: dispatch 2023-09-30T14:33:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:44 smithi156 ceph-mon[103579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d538f510-0f4b-477a-a7e3-99f203c349cf"}]': finished 2023-09-30T14:33:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:44 smithi156 ceph-mon[103579]: osdmap e30: 5 total, 4 up, 5 in 2023-09-30T14:33:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-30T14:33:44.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:44 smithi148 ceph-mon[102355]: osdmap e29: 4 total, 4 up, 4 in 2023-09-30T14:33:44.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:44 smithi148 ceph-mon[102355]: from='client.? 172.21.15.148:0/3047243583' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d538f510-0f4b-477a-a7e3-99f203c349cf"}]: dispatch 2023-09-30T14:33:44.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:44 smithi148 ceph-mon[102355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d538f510-0f4b-477a-a7e3-99f203c349cf"}]: dispatch 2023-09-30T14:33:44.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:44 smithi148 ceph-mon[102355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d538f510-0f4b-477a-a7e3-99f203c349cf"}]': finished 2023-09-30T14:33:44.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:44 smithi148 ceph-mon[102355]: osdmap e30: 5 total, 4 up, 5 in 2023-09-30T14:33:44.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-30T14:33:44.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:44 smithi032 ceph-mon[98592]: osdmap e29: 4 total, 4 up, 4 in 2023-09-30T14:33:44.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:44 smithi032 ceph-mon[98592]: from='client.? 172.21.15.148:0/3047243583' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d538f510-0f4b-477a-a7e3-99f203c349cf"}]: dispatch 2023-09-30T14:33:44.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:44 smithi032 ceph-mon[98592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d538f510-0f4b-477a-a7e3-99f203c349cf"}]: dispatch 2023-09-30T14:33:44.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:44 smithi032 ceph-mon[98592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d538f510-0f4b-477a-a7e3-99f203c349cf"}]': finished 2023-09-30T14:33:44.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:44 smithi032 ceph-mon[98592]: osdmap e30: 5 total, 4 up, 5 in 2023-09-30T14:33:44.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-30T14:33:45.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:45 smithi148 ceph-mon[102355]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:33:45.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:45 smithi148 ceph-mon[102355]: from='client.? 172.21.15.148:0/1024120484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:33:45.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:45 smithi032 ceph-mon[98592]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:33:45.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:45 smithi032 ceph-mon[98592]: from='client.? 172.21.15.148:0/1024120484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:33:46.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:45 smithi156 ceph-mon[103579]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:33:46.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:45 smithi156 ceph-mon[103579]: from='client.? 172.21.15.148:0/1024120484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:33:46.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:46 smithi148 ceph-mon[102355]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2023-09-30T14:33:46.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:46 smithi032 ceph-mon[98592]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2023-09-30T14:33:47.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:46 smithi156 ceph-mon[103579]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2023-09-30T14:33:49.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:48 smithi148 ceph-mon[102355]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-30T14:33:49.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:48 smithi156 ceph-mon[103579]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-30T14:33:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:48 smithi032 ceph-mon[98592]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-30T14:33:51.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:50 smithi156 ceph-mon[103579]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-09-30T14:33:51.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:50 smithi148 ceph-mon[102355]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-09-30T14:33:51.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:50 smithi032 ceph-mon[98592]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-09-30T14:33:52.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-30T14:33:52.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:52.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:51 smithi156 ceph-mon[103579]: Deploying daemon osd.4 on smithi148 2023-09-30T14:33:52.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:51 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-30T14:33:52.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:51 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:52.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:51 smithi148 ceph-mon[102355]: Deploying daemon osd.4 on smithi148 2023-09-30T14:33:52.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:51 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-30T14:33:52.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:51 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:33:52.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:51 smithi032 ceph-mon[98592]: Deploying daemon osd.4 on smithi148 2023-09-30T14:33:53.254 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:52 smithi148 ceph-mon[102355]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-09-30T14:33:53.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:52 smithi156 ceph-mon[103579]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-09-30T14:33:53.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:52 smithi032 ceph-mon[98592]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-09-30T14:33:55.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:54 smithi148 ceph-mon[102355]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-30T14:33:55.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:54 smithi156 ceph-mon[103579]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-30T14:33:55.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:54 smithi032 ceph-mon[98592]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-30T14:33:56.778 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:56 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:56.778 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:56 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:56.778 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:56 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:33:56.778 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:56 smithi148 ceph-mon[102355]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:33:57.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:57.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:57.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:33:57.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:56 smithi156 ceph-mon[103579]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:33:57.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:57.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:57.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:33:57.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:56 smithi032 ceph-mon[98592]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:33:58.262 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 4 on host 'smithi148' 2023-09-30T14:33:58.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:58.260+0000 7f36d9ffb700 1 -- 172.21.15.148:0/2119766892 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f36dc02d310 con 0x7f36cc0a1020 2023-09-30T14:33:58.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:58.263+0000 7f36e2bb0700 1 -- 172.21.15.148:0/2119766892 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f36cc0a1020 msgr2=0x7f36cc0a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:33:58.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:58.263+0000 7f36e2bb0700 1 --2- 172.21.15.148:0/2119766892 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f36cc0a1020 0x7f36cc0a34e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36c8000c00 tx=0x7f36c8005fb0 comp rx=0 tx=0).stop 2023-09-30T14:33:58.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:58.263+0000 7f36e2bb0700 1 -- 172.21.15.148:0/2119766892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36dc0ff560 msgr2=0x7f36dc0fd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:33:58.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:58.263+0000 7f36e2bb0700 1 --2- 172.21.15.148:0/2119766892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36dc0ff560 0x7f36dc0fd050 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f36d000d980 tx=0x7f36d000dd40 comp rx=0 tx=0).stop 2023-09-30T14:33:58.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:58.263+0000 7f36e2bb0700 1 -- 172.21.15.148:0/2119766892 shutdown_connections 2023-09-30T14:33:58.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:58.263+0000 7f36e2bb0700 1 --2- 172.21.15.148:0/2119766892 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f36cc0a1020 0x7f36cc0a34e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:58.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:58.263+0000 7f36e2bb0700 1 --2- 172.21.15.148:0/2119766892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36dc101100 0x7f36dc19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:58.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:58.263+0000 7f36e2bb0700 1 --2- 172.21.15.148:0/2119766892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36dc0ff560 0x7f36dc0fd050 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:58.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:58.263+0000 7f36e2bb0700 1 --2- 172.21.15.148:0/2119766892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36dc100760 0x7f36dc19b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:33:58.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:58.263+0000 7f36e2bb0700 1 -- 172.21.15.148:0/2119766892 >> 172.21.15.148:0/2119766892 conn(0x7f36dc0faae0 msgr2=0x7f36dc104220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:33:58.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:58.263+0000 7f36e2bb0700 1 -- 172.21.15.148:0/2119766892 shutdown_connections 2023-09-30T14:33:58.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:33:58.263+0000 7f36e2bb0700 1 -- 172.21.15.148:0/2119766892 wait complete. 2023-09-30T14:33:58.662 DEBUG:teuthology.orchestra.run.smithi148:osd.4> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.4.service 2023-09-30T14:33:58.664 INFO:tasks.cephadm:Deploying osd.5 on smithi156 with /dev/vg_nvme/lv_3... 2023-09-30T14:33:58.665 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-30T14:33:58.694 INFO:journalctl@ceph.osd.4.smithi148.stdout:-- Logs begin at Sat 2023-09-30 14:16:26 UTC. -- 2023-09-30T14:33:59.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:58 smithi156 ceph-mon[103579]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:33:59.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:59.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:59.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:59.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:59.225 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:58 smithi148 ceph-mon[102355]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:33:59.226 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:59.226 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:59.226 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:59.226 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:59.229 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.c/config 2023-09-30T14:33:59.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:58 smithi032 ceph-mon[98592]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:33:59.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:59.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:59.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:59.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:33:59.859 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 30 14:33:59 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-4[115377]: 2023-09-30T14:33:59.613+0000 7fba03abf640 -1 osd.4 0 log_to_monitors true 2023-09-30T14:33:59.987 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:59 smithi156 ceph-mon[103579]: from='osd.4 [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-30T14:33:59.988 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:33:59 smithi156 ceph-mon[103579]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-30T14:34:00.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:59 smithi148 ceph-mon[102355]: from='osd.4 [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-30T14:34:00.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:33:59 smithi148 ceph-mon[102355]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-30T14:34:00.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:59 smithi032 ceph-mon[98592]: from='osd.4 [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-30T14:34:00.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:33:59 smithi032 ceph-mon[98592]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-30T14:34:00.735 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-30T14:34:00.735 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-30T14:34:00.735 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-09-30T14:34:00.735 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-09-30T14:34:00.735 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00818295 s, 1.3 GB/s 2023-09-30T14:34:00.735 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-09-30T14:34:01.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:00 smithi156 ceph-mon[103579]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:34:01.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:00 smithi156 ceph-mon[103579]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-30T14:34:01.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:00 smithi156 ceph-mon[103579]: osdmap e31: 5 total, 4 up, 5 in 2023-09-30T14:34:01.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:00 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-30T14:34:01.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:00 smithi156 ceph-mon[103579]: from='osd.4 [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:34:01.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:00 smithi156 ceph-mon[103579]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:34:01.184 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 30 14:34:00 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-4[115377]: 2023-09-30T14:34:00.915+0000 7fb9f8935700 -1 osd.4 0 waiting for initial osdmap 2023-09-30T14:34:01.185 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 30 14:34:00 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-4[115377]: 2023-09-30T14:34:00.921+0000 7fb9f0f23700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T14:34:01.185 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:00 smithi148 ceph-mon[102355]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:34:01.185 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:00 smithi148 ceph-mon[102355]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-30T14:34:01.185 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:00 smithi148 ceph-mon[102355]: osdmap e31: 5 total, 4 up, 5 in 2023-09-30T14:34:01.185 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:00 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-30T14:34:01.185 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:00 smithi148 ceph-mon[102355]: from='osd.4 [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:34:01.185 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:00 smithi148 ceph-mon[102355]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:34:01.274 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi156:vg_nvme/lv_3 2023-09-30T14:34:01.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:00 smithi032 ceph-mon[98592]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:34:01.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:00 smithi032 ceph-mon[98592]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-30T14:34:01.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:00 smithi032 ceph-mon[98592]: osdmap e31: 5 total, 4 up, 5 in 2023-09-30T14:34:01.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:00 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-30T14:34:01.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:00 smithi032 ceph-mon[98592]: from='osd.4 [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:34:01.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:00 smithi032 ceph-mon[98592]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:34:01.835 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.c/config 2023-09-30T14:34:02.231 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2023-09-30T14:34:02.232 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: osdmap e32: 5 total, 4 up, 5 in 2023-09-30T14:34:02.232 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-30T14:34:02.232 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-30T14:34:02.232 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: Detected new or changed devices on smithi148 2023-09-30T14:34:02.232 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:02.232 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:02.232 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:02.233 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:02.233 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi148 to 8584M 2023-09-30T14:34:02.233 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:02.233 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:02.233 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:02.233 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:02.233 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:02.233 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:02.234 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:02.234 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:02.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2023-09-30T14:34:02.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: osdmap e32: 5 total, 4 up, 5 in 2023-09-30T14:34:02.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-30T14:34:02.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-30T14:34:02.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: Detected new or changed devices on smithi148 2023-09-30T14:34:02.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:02.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:02.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:02.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:02.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi148 to 8584M 2023-09-30T14:34:02.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:02.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:02.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:02.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:02.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:02.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:02.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:02.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:02.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2023-09-30T14:34:02.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: osdmap e32: 5 total, 4 up, 5 in 2023-09-30T14:34:02.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-30T14:34:02.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-30T14:34:02.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: Detected new or changed devices on smithi148 2023-09-30T14:34:02.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:02.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:02.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:02.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:02.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi148 to 8584M 2023-09-30T14:34:02.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:02.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:02.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:02.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:02.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:02.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:02.390 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:02.390 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:03.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.030+0000 7f8a78271700 1 -- 172.21.15.156:0/3979528926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a700ff920 msgr2=0x7f8a701094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:34:03.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.030+0000 7f8a78271700 1 --2- 172.21.15.156:0/3979528926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a700ff920 0x7f8a701094f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800b0a0 tx=0x7f8a6800b3b0 comp rx=0 tx=0).stop 2023-09-30T14:34:03.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.030+0000 7f8a78271700 1 -- 172.21.15.156:0/3979528926 shutdown_connections 2023-09-30T14:34:03.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.030+0000 7f8a78271700 1 --2- 172.21.15.156:0/3979528926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a700ff920 0x7f8a701094f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:03.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.030+0000 7f8a78271700 1 --2- 172.21.15.156:0/3979528926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a70102500 0x7f8a70102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:03.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.030+0000 7f8a78271700 1 --2- 172.21.15.156:0/3979528926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a700fef80 0x7f8a700ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:03.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.030+0000 7f8a78271700 1 -- 172.21.15.156:0/3979528926 >> 172.21.15.156:0/3979528926 conn(0x7f8a700faae0 msgr2=0x7f8a700fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:34:03.033 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.030+0000 7f8a78271700 1 -- 172.21.15.156:0/3979528926 shutdown_connections 2023-09-30T14:34:03.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.030+0000 7f8a78271700 1 -- 172.21.15.156:0/3979528926 wait complete. 2023-09-30T14:34:03.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.030+0000 7f8a78271700 1 Processor -- start 2023-09-30T14:34:03.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a78271700 1 -- start start 2023-09-30T14:34:03.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a78271700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a700fef80 0x7f8a70196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:34:03.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a78271700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a700ff920 0x7f8a701971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:34:03.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a78271700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a70102500 0x7f8a7019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:34:03.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a78271700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7019e830 con 0x7f8a700ff920 2023-09-30T14:34:03.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a78271700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7019e970 con 0x7f8a70102500 2023-09-30T14:34:03.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a78271700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7019eb30 con 0x7f8a700fef80 2023-09-30T14:34:03.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a7600d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a700fef80 0x7f8a70196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:34:03.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a7600d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a700fef80 0x7f8a70196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36100/0 (socket says 172.21.15.156:36100) 2023-09-30T14:34:03.034 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a7600d700 1 -- 172.21.15.156:0/6498281 learned_addr learned my addr 172.21.15.156:0/6498281 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-30T14:34:03.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a7580c700 1 --2- 172.21.15.156:0/6498281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a700ff920 0x7f8a701971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:34:03.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a7680e700 1 --2- 172.21.15.156:0/6498281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a70102500 0x7f8a7019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:34:03.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a7600d700 1 -- 172.21.15.156:0/6498281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a70102500 msgr2=0x7f8a7019e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:34:03.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a7600d700 1 --2- 172.21.15.156:0/6498281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a70102500 0x7f8a7019e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:03.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a7600d700 1 -- 172.21.15.156:0/6498281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a700ff920 msgr2=0x7f8a701971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:34:03.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a7600d700 1 --2- 172.21.15.156:0/6498281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a700ff920 0x7f8a701971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:03.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a7600d700 1 -- 172.21.15.156:0/6498281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a68009d00 con 0x7f8a700fef80 2023-09-30T14:34:03.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a7600d700 1 --2- 172.21.15.156:0/6498281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a700fef80 0x7f8a70196cb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00d980 tx=0x7f8a6c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:34:03.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a677fe700 1 -- 172.21.15.156:0/6498281 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c004020 con 0x7f8a700fef80 2023-09-30T14:34:03.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a677fe700 1 -- 172.21.15.156:0/6498281 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a6c009730 con 0x7f8a700fef80 2023-09-30T14:34:03.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.031+0000 7f8a78271700 1 -- 172.21.15.156:0/6498281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a701001b0 con 0x7f8a700fef80 2023-09-30T14:34:03.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.032+0000 7f8a677fe700 1 -- 172.21.15.156:0/6498281 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c003a80 con 0x7f8a700fef80 2023-09-30T14:34:03.035 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.032+0000 7f8a78271700 1 -- 172.21.15.156:0/6498281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a701006b0 con 0x7f8a700fef80 2023-09-30T14:34:03.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.032+0000 7f8a78271700 1 -- 172.21.15.156:0/6498281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a70108610 con 0x7f8a700fef80 2023-09-30T14:34:03.036 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.033+0000 7f8a677fe700 1 -- 172.21.15.156:0/6498281 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f8a6c0098a0 con 0x7f8a700fef80 2023-09-30T14:34:03.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.034+0000 7f8a677fe700 1 --2- 172.21.15.156:0/6498281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a5c0a0fa0 0x7f8a5c0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:34:03.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.034+0000 7f8a677fe700 1 -- 172.21.15.156:0/6498281 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f8a6c0cb3a0 con 0x7f8a700fef80 2023-09-30T14:34:03.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.034+0000 7f8a7580c700 1 --2- 172.21.15.156:0/6498281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a5c0a0fa0 0x7f8a5c0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:34:03.037 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.035+0000 7f8a7580c700 1 --2- 172.21.15.156:0/6498281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a5c0a0fa0 0x7f8a5c0a3460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a701981c0 tx=0x7f8a6000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:34:03.039 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.039+0000 7f8a677fe700 1 -- 172.21.15.156:0/6498281 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f8a6c095c10 con 0x7f8a700fef80 2023-09-30T14:34:03.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:03.233+0000 7f8a78271700 1 -- 172.21.15.156:0/6498281 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a7002d1b0 con 0x7f8a5c0a0fa0 2023-09-30T14:34:03.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:02 smithi156 ceph-mon[103579]: purged_snaps scrub starts 2023-09-30T14:34:03.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:02 smithi156 ceph-mon[103579]: purged_snaps scrub ok 2023-09-30T14:34:03.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:02 smithi156 ceph-mon[103579]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:34:03.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:02 smithi156 ceph-mon[103579]: osd.4 [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559] boot 2023-09-30T14:34:03.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:02 smithi156 ceph-mon[103579]: osdmap e33: 5 total, 5 up, 5 in 2023-09-30T14:34:03.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-30T14:34:03.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:02 smithi148 ceph-mon[102355]: purged_snaps scrub starts 2023-09-30T14:34:03.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:02 smithi148 ceph-mon[102355]: purged_snaps scrub ok 2023-09-30T14:34:03.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:02 smithi148 ceph-mon[102355]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:34:03.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:02 smithi148 ceph-mon[102355]: osd.4 [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559] boot 2023-09-30T14:34:03.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:02 smithi148 ceph-mon[102355]: osdmap e33: 5 total, 5 up, 5 in 2023-09-30T14:34:03.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-30T14:34:03.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:02 smithi032 ceph-mon[98592]: purged_snaps scrub starts 2023-09-30T14:34:03.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:02 smithi032 ceph-mon[98592]: purged_snaps scrub ok 2023-09-30T14:34:03.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:02 smithi032 ceph-mon[98592]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-30T14:34:03.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:02 smithi032 ceph-mon[98592]: osd.4 [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559] boot 2023-09-30T14:34:03.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:02 smithi032 ceph-mon[98592]: osdmap e33: 5 total, 5 up, 5 in 2023-09-30T14:34:03.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-30T14:34:04.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:03 smithi156 ceph-mon[103579]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:34:04.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:03 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:34:04.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:03 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:34:04.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:03 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:04.251 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:03 smithi156 ceph-mon[103579]: osdmap e34: 5 total, 5 up, 5 in 2023-09-30T14:34:04.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:03 smithi148 ceph-mon[102355]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:34:04.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:03 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:34:04.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:03 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:34:04.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:03 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:04.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:03 smithi148 ceph-mon[102355]: osdmap e34: 5 total, 5 up, 5 in 2023-09-30T14:34:04.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:03 smithi032 ceph-mon[98592]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:34:04.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:03 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:34:04.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:03 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:34:04.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:03 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:04.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:03 smithi032 ceph-mon[98592]: osdmap e34: 5 total, 5 up, 5 in 2023-09-30T14:34:05.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:05 smithi156 ceph-mon[103579]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:05.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:05 smithi148 ceph-mon[102355]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:05.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:05 smithi032 ceph-mon[98592]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:06.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:06 smithi156 ceph-mon[103579]: from='client.? 172.21.15.156:0/4258073514' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7e7db08-601c-416b-be4a-e5b72a9c00a2"}]: dispatch 2023-09-30T14:34:06.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:06 smithi156 ceph-mon[103579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7e7db08-601c-416b-be4a-e5b72a9c00a2"}]: dispatch 2023-09-30T14:34:06.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:06 smithi156 ceph-mon[103579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7e7db08-601c-416b-be4a-e5b72a9c00a2"}]': finished 2023-09-30T14:34:06.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:06 smithi156 ceph-mon[103579]: osdmap e35: 6 total, 5 up, 6 in 2023-09-30T14:34:06.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:06 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-30T14:34:06.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:06 smithi156 ceph-mon[103579]: from='client.? 172.21.15.156:0/2864899426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:34:06.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:06 smithi148 ceph-mon[102355]: from='client.? 172.21.15.156:0/4258073514' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7e7db08-601c-416b-be4a-e5b72a9c00a2"}]: dispatch 2023-09-30T14:34:06.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:06 smithi148 ceph-mon[102355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7e7db08-601c-416b-be4a-e5b72a9c00a2"}]: dispatch 2023-09-30T14:34:06.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:06 smithi148 ceph-mon[102355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7e7db08-601c-416b-be4a-e5b72a9c00a2"}]': finished 2023-09-30T14:34:06.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:06 smithi148 ceph-mon[102355]: osdmap e35: 6 total, 5 up, 6 in 2023-09-30T14:34:06.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-30T14:34:06.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:06 smithi148 ceph-mon[102355]: from='client.? 172.21.15.156:0/2864899426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:34:06.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:06 smithi032 ceph-mon[98592]: from='client.? 172.21.15.156:0/4258073514' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7e7db08-601c-416b-be4a-e5b72a9c00a2"}]: dispatch 2023-09-30T14:34:06.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:06 smithi032 ceph-mon[98592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7e7db08-601c-416b-be4a-e5b72a9c00a2"}]: dispatch 2023-09-30T14:34:06.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:06 smithi032 ceph-mon[98592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7e7db08-601c-416b-be4a-e5b72a9c00a2"}]': finished 2023-09-30T14:34:06.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:06 smithi032 ceph-mon[98592]: osdmap e35: 6 total, 5 up, 6 in 2023-09-30T14:34:06.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:06 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-30T14:34:06.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:06 smithi032 ceph-mon[98592]: from='client.? 172.21.15.156:0/2864899426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:34:07.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:07 smithi156 ceph-mon[103579]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:07.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:07 smithi148 ceph-mon[102355]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:07.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:07 smithi032 ceph-mon[98592]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:09 smithi156 ceph-mon[103579]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:09.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:09 smithi148 ceph-mon[102355]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:09.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:09 smithi032 ceph-mon[98592]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:11.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:11 smithi156 ceph-mon[103579]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:11.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:11 smithi148 ceph-mon[102355]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:11.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:11 smithi032 ceph-mon[98592]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:13.194 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:12 smithi156 ceph-mon[103579]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:13.194 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:12 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-30T14:34:13.194 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:12 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:13.194 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:12 smithi156 ceph-mon[103579]: Deploying daemon osd.5 on smithi156 2023-09-30T14:34:13.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:12 smithi148 ceph-mon[102355]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:13.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:12 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-30T14:34:13.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:12 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:13.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:12 smithi148 ceph-mon[102355]: Deploying daemon osd.5 on smithi156 2023-09-30T14:34:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:12 smithi032 ceph-mon[98592]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-30T14:34:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:12 smithi032 ceph-mon[98592]: Deploying daemon osd.5 on smithi156 2023-09-30T14:34:15.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:14 smithi156 ceph-mon[103579]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:15.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:14 smithi148 ceph-mon[102355]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:15.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:14 smithi032 ceph-mon[98592]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:17.460 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:17 smithi156 ceph-mon[103579]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:17.460 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:17.460 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:17.461 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:17.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:17 smithi148 ceph-mon[102355]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:17.632 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:17.632 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:17.632 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:17.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:17 smithi032 ceph-mon[98592]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:17.651 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:17.651 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:17.651 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:19.101 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 5 on host 'smithi156' 2023-09-30T14:34:19.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:19.101+0000 7f8a677fe700 1 -- 172.21.15.156:0/6498281 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8a7002d1b0 con 0x7f8a5c0a0fa0 2023-09-30T14:34:19.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:19.102+0000 7f8a78271700 1 -- 172.21.15.156:0/6498281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a5c0a0fa0 msgr2=0x7f8a5c0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:34:19.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:19.102+0000 7f8a78271700 1 --2- 172.21.15.156:0/6498281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a5c0a0fa0 0x7f8a5c0a3460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a701981c0 tx=0x7f8a6000a300 comp rx=0 tx=0).stop 2023-09-30T14:34:19.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:19.103+0000 7f8a78271700 1 -- 172.21.15.156:0/6498281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a700fef80 msgr2=0x7f8a70196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:34:19.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:19.103+0000 7f8a78271700 1 --2- 172.21.15.156:0/6498281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a700fef80 0x7f8a70196cb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00d980 tx=0x7f8a6c00dc90 comp rx=0 tx=0).stop 2023-09-30T14:34:19.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:19.103+0000 7f8a78271700 1 -- 172.21.15.156:0/6498281 shutdown_connections 2023-09-30T14:34:19.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:19.103+0000 7f8a78271700 1 --2- 172.21.15.156:0/6498281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a5c0a0fa0 0x7f8a5c0a3460 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:19.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:19.103+0000 7f8a78271700 1 --2- 172.21.15.156:0/6498281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a70102500 0x7f8a7019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:19.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:19.103+0000 7f8a78271700 1 --2- 172.21.15.156:0/6498281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a700fef80 0x7f8a70196cb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:19.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:19.103+0000 7f8a78271700 1 --2- 172.21.15.156:0/6498281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a700ff920 0x7f8a701971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:19.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:19.103+0000 7f8a78271700 1 -- 172.21.15.156:0/6498281 >> 172.21.15.156:0/6498281 conn(0x7f8a700faae0 msgr2=0x7f8a70105750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:34:19.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:19.103+0000 7f8a78271700 1 -- 172.21.15.156:0/6498281 shutdown_connections 2023-09-30T14:34:19.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:34:19.103+0000 7f8a78271700 1 -- 172.21.15.156:0/6498281 wait complete. 2023-09-30T14:34:19.529 DEBUG:teuthology.orchestra.run.smithi156:osd.5> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.5.service 2023-09-30T14:34:19.536 INFO:tasks.cephadm:Deploying osd.6 on smithi032 with /dev/vg_nvme/lv_2... 2023-09-30T14:34:19.536 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-30T14:34:19.585 INFO:journalctl@ceph.osd.5.smithi156.stdout:-- Logs begin at Sat 2023-09-30 14:16:23 UTC. -- 2023-09-30T14:34:19.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:19 smithi148 ceph-mon[102355]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:19.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:19 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:19.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:19 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:19.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:19 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:19.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:19 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:19.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:19 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:19.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:19 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:19.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:19 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:19.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:19 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:19.686 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:19 smithi032 ceph-mon[98592]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:19.686 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:19 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:19.686 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:19 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:19.687 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:19 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:19.687 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:19 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:19.687 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:19 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:19.687 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:19 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:19.687 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:19 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:19.687 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:19 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:19 smithi156 ceph-mon[103579]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:19.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:19 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:19.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:19 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:19.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:19 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:19.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:19 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:19.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:19 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:19.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:19 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:19.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:19 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:19.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:19 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:34:20.282 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 14:34:20 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T14:34:20.241+0000 7efce7b75640 -1 osd.5 0 log_to_monitors true 2023-09-30T14:34:20.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:20 smithi156 ceph-mon[103579]: from='osd.5 [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-30T14:34:20.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:20 smithi156 ceph-mon[103579]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-30T14:34:20.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:20 smithi148 ceph-mon[102355]: from='osd.5 [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-30T14:34:20.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:20 smithi148 ceph-mon[102355]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-30T14:34:20.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:20 smithi032 ceph-mon[98592]: from='osd.5 [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-30T14:34:20.651 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:20 smithi032 ceph-mon[98592]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-30T14:34:21.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:21 smithi148 ceph-mon[102355]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:21.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:21 smithi148 ceph-mon[102355]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-30T14:34:21.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:21 smithi148 ceph-mon[102355]: osdmap e36: 6 total, 5 up, 6 in 2023-09-30T14:34:21.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:21 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-30T14:34:21.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:21 smithi148 ceph-mon[102355]: from='osd.5 [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:34:21.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:21 smithi148 ceph-mon[102355]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:34:21.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:21 smithi032 ceph-mon[98592]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:21.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:21 smithi032 ceph-mon[98592]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-30T14:34:21.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:21 smithi032 ceph-mon[98592]: osdmap e36: 6 total, 5 up, 6 in 2023-09-30T14:34:21.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:21 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-30T14:34:21.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:21 smithi032 ceph-mon[98592]: from='osd.5 [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:34:21.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:21 smithi032 ceph-mon[98592]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:34:21.770 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 14:34:21 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T14:34:21.357+0000 7efcde1ee700 -1 osd.5 0 waiting for initial osdmap 2023-09-30T14:34:21.770 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 14:34:21 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T14:34:21.364+0000 7efcd6fdd700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T14:34:21.770 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:21 smithi156 ceph-mon[103579]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:21.771 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:21 smithi156 ceph-mon[103579]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-30T14:34:21.771 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:21 smithi156 ceph-mon[103579]: osdmap e36: 6 total, 5 up, 6 in 2023-09-30T14:34:21.771 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:21 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-30T14:34:21.771 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:21 smithi156 ceph-mon[103579]: from='osd.5 [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:34:21.771 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:21 smithi156 ceph-mon[103579]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:34:21.981 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-30T14:34:21.981 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-30T14:34:21.981 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2023-09-30T14:34:21.981 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2023-09-30T14:34:21.982 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0080894 s, 1.3 GB/s 2023-09-30T14:34:21.982 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 2023-09-30T14:34:21.982 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2023-09-30T14:34:22.380 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi032:vg_nvme/lv_2 2023-09-30T14:34:22.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:22 smithi156 ceph-mon[103579]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-30T14:34:22.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:22 smithi156 ceph-mon[103579]: osdmap e37: 6 total, 5 up, 6 in 2023-09-30T14:34:22.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:22 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-30T14:34:22.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:22 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-30T14:34:22.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:22 smithi156 ceph-mon[103579]: osd.5 [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180] boot 2023-09-30T14:34:22.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:22 smithi156 ceph-mon[103579]: osdmap e38: 6 total, 6 up, 6 in 2023-09-30T14:34:22.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:22 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-30T14:34:22.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:22 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:22.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:22 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:22.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:22 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:22.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:22 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:22.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:22 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:22.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:22 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:22.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:22 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:22.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:22 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:22.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:22 smithi032 ceph-mon[98592]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-30T14:34:22.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:22 smithi032 ceph-mon[98592]: osdmap e37: 6 total, 5 up, 6 in 2023-09-30T14:34:22.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:22 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-30T14:34:22.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:22 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-30T14:34:22.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:22 smithi032 ceph-mon[98592]: osd.5 [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180] boot 2023-09-30T14:34:22.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:22 smithi032 ceph-mon[98592]: osdmap e38: 6 total, 6 up, 6 in 2023-09-30T14:34:22.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:22 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-30T14:34:22.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:22 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:22.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:22 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:22.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:22 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:22.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:22 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:22.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:22 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:22.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:22 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:22.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:22 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:22.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:22 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:22.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:22 smithi148 ceph-mon[102355]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-30T14:34:22.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:22 smithi148 ceph-mon[102355]: osdmap e37: 6 total, 5 up, 6 in 2023-09-30T14:34:22.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:22 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-30T14:34:22.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:22 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-30T14:34:22.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:22 smithi148 ceph-mon[102355]: osd.5 [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180] boot 2023-09-30T14:34:22.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:22 smithi148 ceph-mon[102355]: osdmap e38: 6 total, 6 up, 6 in 2023-09-30T14:34:22.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:22 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-30T14:34:22.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:22 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:22.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:22 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:22.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:22 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:22.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:22 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:22.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:22 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:22.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:22 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:22.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:22 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:22.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:22 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:34:23.469 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:23 smithi032 ceph-mon[98592]: purged_snaps scrub starts 2023-09-30T14:34:23.469 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:23 smithi032 ceph-mon[98592]: purged_snaps scrub ok 2023-09-30T14:34:23.469 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:23 smithi032 ceph-mon[98592]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:23.469 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:23 smithi032 ceph-mon[98592]: Detected new or changed devices on smithi156 2023-09-30T14:34:23.469 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:23 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi156 to 8584M 2023-09-30T14:34:23.469 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:23 smithi032 ceph-mon[98592]: osdmap e39: 6 total, 6 up, 6 in 2023-09-30T14:34:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:23 smithi156 ceph-mon[103579]: purged_snaps scrub starts 2023-09-30T14:34:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:23 smithi156 ceph-mon[103579]: purged_snaps scrub ok 2023-09-30T14:34:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:23 smithi156 ceph-mon[103579]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:23 smithi156 ceph-mon[103579]: Detected new or changed devices on smithi156 2023-09-30T14:34:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:23 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi156 to 8584M 2023-09-30T14:34:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:23 smithi156 ceph-mon[103579]: osdmap e39: 6 total, 6 up, 6 in 2023-09-30T14:34:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:23 smithi148 ceph-mon[102355]: purged_snaps scrub starts 2023-09-30T14:34:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:23 smithi148 ceph-mon[102355]: purged_snaps scrub ok 2023-09-30T14:34:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:23 smithi148 ceph-mon[102355]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-30T14:34:23.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:23 smithi148 ceph-mon[102355]: Detected new or changed devices on smithi156 2023-09-30T14:34:23.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:23 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi156 to 8584M 2023-09-30T14:34:23.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:23 smithi148 ceph-mon[102355]: osdmap e39: 6 total, 6 up, 6 in 2023-09-30T14:34:24.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.178+0000 7f201c23b700 1 -- 172.21.15.32:0/2723321670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2014100030 msgr2=0x7f20141004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:34:24.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.178+0000 7f201c23b700 1 --2- 172.21.15.32:0/2723321670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2014100030 0x7f20141004b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2004009990 tx=0x7f2004009ca0 comp rx=0 tx=0).stop 2023-09-30T14:34:24.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.178+0000 7f201c23b700 1 -- 172.21.15.32:0/2723321670 shutdown_connections 2023-09-30T14:34:24.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.178+0000 7f201c23b700 1 --2- 172.21.15.32:0/2723321670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20141009f0 0x7f2014108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:24.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.178+0000 7f201c23b700 1 --2- 172.21.15.32:0/2723321670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2014101460 0x7f2014101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:24.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.178+0000 7f201c23b700 1 --2- 172.21.15.32:0/2723321670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2014100030 0x7f20141004b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:24.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.178+0000 7f201c23b700 1 -- 172.21.15.32:0/2723321670 >> 172.21.15.32:0/2723321670 conn(0x7f20140faae0 msgr2=0x7f20140fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:34:24.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.178+0000 7f201c23b700 1 -- 172.21.15.32:0/2723321670 shutdown_connections 2023-09-30T14:34:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.178+0000 7f201c23b700 1 -- 172.21.15.32:0/2723321670 wait complete. 2023-09-30T14:34:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f201c23b700 1 Processor -- start 2023-09-30T14:34:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f201c23b700 1 -- start start 2023-09-30T14:34:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f201c23b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2014100030 0x7f2014196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:34:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f201c23b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20141009f0 0x7f20141970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:34:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f201c23b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2014101460 0x7f201419e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:34:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f201c23b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201419e730 con 0x7f2014100030 2023-09-30T14:34:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f201c23b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201419e870 con 0x7f20141009f0 2023-09-30T14:34:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f201c23b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201419ea00 con 0x7f2014101460 2023-09-30T14:34:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f2019fd7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2014100030 0x7f2014196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:34:24.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f2019fd7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2014100030 0x7f2014196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33064/0 (socket says 172.21.15.32:33064) 2023-09-30T14:34:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f2019fd7700 1 -- 172.21.15.32:0/692759850 learned_addr learned my addr 172.21.15.32:0/692759850 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:34:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f201a7d8700 1 --2- 172.21.15.32:0/692759850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2014101460 0x7f201419e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:34:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f20197d6700 1 --2- 172.21.15.32:0/692759850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20141009f0 0x7f20141970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:34:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f2019fd7700 1 -- 172.21.15.32:0/692759850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2014101460 msgr2=0x7f201419e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:34:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f2019fd7700 1 --2- 172.21.15.32:0/692759850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2014101460 0x7f201419e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f2019fd7700 1 -- 172.21.15.32:0/692759850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20141009f0 msgr2=0x7f20141970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:34:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f2019fd7700 1 --2- 172.21.15.32:0/692759850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20141009f0 0x7f20141970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f2019fd7700 1 -- 172.21.15.32:0/692759850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20040095f0 con 0x7f2014100030 2023-09-30T14:34:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.179+0000 7f2019fd7700 1 --2- 172.21.15.32:0/692759850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2014100030 0x7f2014196ba0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f201000d980 tx=0x7f201000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:34:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.180+0000 7f200affd700 1 -- 172.21.15.32:0/692759850 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010009b40 con 0x7f2014100030 2023-09-30T14:34:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.180+0000 7f200affd700 1 -- 172.21.15.32:0/692759850 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f201000b6e0 con 0x7f2014100030 2023-09-30T14:34:24.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.180+0000 7f201c23b700 1 -- 172.21.15.32:0/692759850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f201419ece0 con 0x7f2014100030 2023-09-30T14:34:24.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.180+0000 7f201c23b700 1 -- 172.21.15.32:0/692759850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f201419f190 con 0x7f2014100030 2023-09-30T14:34:24.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.180+0000 7f200affd700 1 -- 172.21.15.32:0/692759850 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201000f620 con 0x7f2014100030 2023-09-30T14:34:24.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.181+0000 7f201c23b700 1 -- 172.21.15.32:0/692759850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f201410b5e0 con 0x7f2014100030 2023-09-30T14:34:24.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.182+0000 7f200affd700 1 -- 172.21.15.32:0/692759850 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f2010010460 con 0x7f2014100030 2023-09-30T14:34:24.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.182+0000 7f200affd700 1 --2- 172.21.15.32:0/692759850 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f20000a0fa0 0x7f20000a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:34:24.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.183+0000 7f200affd700 1 -- 172.21.15.32:0/692759850 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f20100cbb00 con 0x7f2014100030 2023-09-30T14:34:24.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.183+0000 7f20197d6700 1 --2- 172.21.15.32:0/692759850 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f20000a0fa0 0x7f20000a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:34:24.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.183+0000 7f20197d6700 1 --2- 172.21.15.32:0/692759850 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f20000a0fa0 0x7f20000a3460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f20040095c0 tx=0x7f2004009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:34:24.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.186+0000 7f200affd700 1 -- 172.21.15.32:0/692759850 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f20100961a0 con 0x7f2014100030 2023-09-30T14:34:24.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:24.398+0000 7f201c23b700 1 -- 172.21.15.32:0/692759850 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f20140fbb20 con 0x7f20000a0fa0 2023-09-30T14:34:25.217 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:24 smithi032 ceph-mon[98592]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:25.217 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:24 smithi032 ceph-mon[98592]: osdmap e40: 6 total, 6 up, 6 in 2023-09-30T14:34:25.217 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:24 smithi032 ceph-mon[98592]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:34:25.217 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:34:25.217 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:34:25.217 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:25.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:24 smithi156 ceph-mon[103579]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:25.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:24 smithi156 ceph-mon[103579]: osdmap e40: 6 total, 6 up, 6 in 2023-09-30T14:34:25.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:24 smithi156 ceph-mon[103579]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:34:25.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:24 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:34:25.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:24 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:34:25.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:24 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:25.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:24 smithi148 ceph-mon[102355]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:25.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:24 smithi148 ceph-mon[102355]: osdmap e40: 6 total, 6 up, 6 in 2023-09-30T14:34:25.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:24 smithi148 ceph-mon[102355]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:34:25.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:24 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:34:25.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:24 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:34:25.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:24 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:26.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:25 smithi156 ceph-mon[103579]: osdmap e41: 6 total, 6 up, 6 in 2023-09-30T14:34:26.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:25 smithi148 ceph-mon[102355]: osdmap e41: 6 total, 6 up, 6 in 2023-09-30T14:34:26.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:25 smithi032 ceph-mon[98592]: osdmap e41: 6 total, 6 up, 6 in 2023-09-30T14:34:27.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:26 smithi156 ceph-mon[103579]: pgmap v128: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-30T14:34:27.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:26 smithi156 ceph-mon[103579]: osdmap e42: 6 total, 6 up, 6 in 2023-09-30T14:34:27.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:26 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2539645733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14fb18d1-a89b-45da-ac5a-deab8fad9681"}]: dispatch 2023-09-30T14:34:27.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:26 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2539645733' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14fb18d1-a89b-45da-ac5a-deab8fad9681"}]': finished 2023-09-30T14:34:27.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:26 smithi156 ceph-mon[103579]: osdmap e43: 7 total, 6 up, 7 in 2023-09-30T14:34:27.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-30T14:34:27.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:26 smithi148 ceph-mon[102355]: pgmap v128: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-30T14:34:27.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:26 smithi148 ceph-mon[102355]: osdmap e42: 6 total, 6 up, 6 in 2023-09-30T14:34:27.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:26 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2539645733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14fb18d1-a89b-45da-ac5a-deab8fad9681"}]: dispatch 2023-09-30T14:34:27.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:26 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2539645733' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14fb18d1-a89b-45da-ac5a-deab8fad9681"}]': finished 2023-09-30T14:34:27.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:26 smithi148 ceph-mon[102355]: osdmap e43: 7 total, 6 up, 7 in 2023-09-30T14:34:27.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-30T14:34:27.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:26 smithi032 ceph-mon[98592]: pgmap v128: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-30T14:34:27.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:26 smithi032 ceph-mon[98592]: osdmap e42: 6 total, 6 up, 6 in 2023-09-30T14:34:27.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:26 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2539645733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14fb18d1-a89b-45da-ac5a-deab8fad9681"}]: dispatch 2023-09-30T14:34:27.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:26 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2539645733' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14fb18d1-a89b-45da-ac5a-deab8fad9681"}]': finished 2023-09-30T14:34:27.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:26 smithi032 ceph-mon[98592]: osdmap e43: 7 total, 6 up, 7 in 2023-09-30T14:34:27.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-30T14:34:28.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:27 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2177448268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:34:28.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:27 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2177448268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:34:28.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:27 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2177448268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:34:29.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:28 smithi156 ceph-mon[103579]: pgmap v131: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-30T14:34:29.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:28 smithi148 ceph-mon[102355]: pgmap v131: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-30T14:34:29.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:28 smithi032 ceph-mon[98592]: pgmap v131: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-30T14:34:31.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:30 smithi032 ceph-mon[98592]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-09-30T14:34:31.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:30 smithi156 ceph-mon[103579]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-09-30T14:34:31.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:30 smithi148 ceph-mon[102355]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-09-30T14:34:33.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:32 smithi156 ceph-mon[103579]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:33.351 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:32 smithi032 ceph-mon[98592]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:33.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:32 smithi148 ceph-mon[102355]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:34.240 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:33 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-30T14:34:34.240 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:33 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:34.240 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:33 smithi032 ceph-mon[98592]: Deploying daemon osd.6 on smithi032 2023-09-30T14:34:34.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:33 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-30T14:34:34.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:33 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:34.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:33 smithi156 ceph-mon[103579]: Deploying daemon osd.6 on smithi032 2023-09-30T14:34:34.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:33 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-30T14:34:34.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:33 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:34.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:33 smithi148 ceph-mon[102355]: Deploying daemon osd.6 on smithi032 2023-09-30T14:34:35.248 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:34 smithi032 ceph-mon[98592]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:35.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:34 smithi156 ceph-mon[103579]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:35.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:34 smithi148 ceph-mon[102355]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:37.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:36 smithi156 ceph-mon[103579]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:37.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:36 smithi148 ceph-mon[102355]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:37.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:36 smithi032 ceph-mon[98592]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:38.959 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:38 smithi032 ceph-mon[98592]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:38.959 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:38 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:38.959 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:38 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:38.959 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:38 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:39.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:38 smithi148 ceph-mon[102355]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:39.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:38 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:39.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:38 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:39.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:38 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:39.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:38 smithi156 ceph-mon[103579]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:39.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:38 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:39.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:38 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:39.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:38 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:40.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:40.652+0000 7f200affd700 1 -- 172.21.15.32:0/692759850 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f20140fbb20 con 0x7f20000a0fa0 2023-09-30T14:34:40.654 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 6 on host 'smithi032' 2023-09-30T14:34:40.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:40.655+0000 7f201c23b700 1 -- 172.21.15.32:0/692759850 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f20000a0fa0 msgr2=0x7f20000a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:34:40.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:40.655+0000 7f201c23b700 1 --2- 172.21.15.32:0/692759850 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f20000a0fa0 0x7f20000a3460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f20040095c0 tx=0x7f2004009530 comp rx=0 tx=0).stop 2023-09-30T14:34:40.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:40.655+0000 7f201c23b700 1 -- 172.21.15.32:0/692759850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2014100030 msgr2=0x7f2014196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:34:40.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:40.655+0000 7f201c23b700 1 --2- 172.21.15.32:0/692759850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2014100030 0x7f2014196ba0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f201000d980 tx=0x7f201000dd40 comp rx=0 tx=0).stop 2023-09-30T14:34:40.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:40.655+0000 7f201c23b700 1 -- 172.21.15.32:0/692759850 shutdown_connections 2023-09-30T14:34:40.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:40.655+0000 7f201c23b700 1 --2- 172.21.15.32:0/692759850 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f20000a0fa0 0x7f20000a3460 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:40.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:40.655+0000 7f201c23b700 1 --2- 172.21.15.32:0/692759850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2014101460 0x7f201419e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:40.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:40.655+0000 7f201c23b700 1 --2- 172.21.15.32:0/692759850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2014100030 0x7f2014196ba0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:40.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:40.655+0000 7f201c23b700 1 --2- 172.21.15.32:0/692759850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20141009f0 0x7f20141970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:40.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:40.655+0000 7f201c23b700 1 -- 172.21.15.32:0/692759850 >> 172.21.15.32:0/692759850 conn(0x7f20140faae0 msgr2=0x7f20140fb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:34:40.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:40.655+0000 7f201c23b700 1 -- 172.21.15.32:0/692759850 shutdown_connections 2023-09-30T14:34:40.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:34:40.655+0000 7f201c23b700 1 -- 172.21.15.32:0/692759850 wait complete. 2023-09-30T14:34:41.033 DEBUG:teuthology.orchestra.run.smithi032:osd.6> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.6.service 2023-09-30T14:34:41.035 INFO:tasks.cephadm:Deploying osd.7 on smithi148 with /dev/vg_nvme/lv_2... 2023-09-30T14:34:41.036 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-30T14:34:41.072 INFO:journalctl@ceph.osd.6.smithi032.stdout:-- Logs begin at Sat 2023-09-30 14:15:36 UTC. -- 2023-09-30T14:34:41.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:41 smithi148 ceph-mon[102355]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:41.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:41 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:41.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:41 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:41.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:41 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:41.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:41 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:41.619 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.b/config 2023-09-30T14:34:41.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:41 smithi032 ceph-mon[98592]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:41.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:41 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:41.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:41 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:41.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:41 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:41.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:41 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:41 smithi156 ceph-mon[103579]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:41 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:41 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:41 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:41 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:41.950 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 14:34:41 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T14:34:41.827+0000 7fe34b720640 -1 osd.6 0 log_to_monitors true 2023-09-30T14:34:42.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:42 smithi148 ceph-mon[102355]: from='osd.6 [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-30T14:34:42.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:42 smithi032 ceph-mon[98592]: from='osd.6 [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-30T14:34:42.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:42 smithi156 ceph-mon[103579]: from='osd.6 [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-30T14:34:43.123 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-30T14:34:43.123 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-30T14:34:43.124 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2023-09-30T14:34:43.124 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2023-09-30T14:34:43.124 INFO:teuthology.orchestra.run.smithi148.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128381 s, 817 MB/s 2023-09-30T14:34:43.124 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2023-09-30T14:34:43.519 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:43 smithi148 ceph-mon[102355]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:43.520 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:43 smithi148 ceph-mon[102355]: from='osd.6 [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-30T14:34:43.520 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:43 smithi148 ceph-mon[102355]: osdmap e44: 7 total, 6 up, 7 in 2023-09-30T14:34:43.520 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:43 smithi148 ceph-mon[102355]: from='osd.6 [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-09-30T14:34:43.520 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:43 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-30T14:34:43.564 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi148:vg_nvme/lv_2 2023-09-30T14:34:43.637 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 14:34:43 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T14:34:43.338+0000 7fe340596700 -1 osd.6 0 waiting for initial osdmap 2023-09-30T14:34:43.650 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 14:34:43 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T14:34:43.349+0000 7fe33ab88700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T14:34:43.650 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:43 smithi032 ceph-mon[98592]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:43.650 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:43 smithi032 ceph-mon[98592]: from='osd.6 [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-30T14:34:43.650 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:43 smithi032 ceph-mon[98592]: osdmap e44: 7 total, 6 up, 7 in 2023-09-30T14:34:43.650 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:43 smithi032 ceph-mon[98592]: from='osd.6 [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-09-30T14:34:43.650 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:43 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-30T14:34:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:43 smithi156 ceph-mon[103579]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:43 smithi156 ceph-mon[103579]: from='osd.6 [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-30T14:34:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:43 smithi156 ceph-mon[103579]: osdmap e44: 7 total, 6 up, 7 in 2023-09-30T14:34:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:43 smithi156 ceph-mon[103579]: from='osd.6 [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-09-30T14:34:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:43 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-30T14:34:44.119 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.b/config 2023-09-30T14:34:44.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: from='osd.6 [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-09-30T14:34:44.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: osdmap e45: 7 total, 6 up, 7 in 2023-09-30T14:34:44.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-30T14:34:44.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-30T14:34:44.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:44.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: from='osd.6 [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-09-30T14:34:44.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: osdmap e45: 7 total, 6 up, 7 in 2023-09-30T14:34:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-30T14:34:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-30T14:34:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:44.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:44.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:44.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:44.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:44.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: from='osd.6 [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-09-30T14:34:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: osdmap e45: 7 total, 6 up, 7 in 2023-09-30T14:34:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-30T14:34:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-30T14:34:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:34:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:34:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:45.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.282+0000 7fc7779d0700 1 -- 172.21.15.148:0/4279480609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc770100fe0 msgr2=0x7fc770101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:34:45.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.282+0000 7fc7779d0700 1 --2- 172.21.15.148:0/4279480609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc770100fe0 0x7fc770101440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0099c0 tx=0x7fc76c009cd0 comp rx=0 tx=0).stop 2023-09-30T14:34:45.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.282+0000 7fc7779d0700 1 -- 172.21.15.148:0/4279480609 shutdown_connections 2023-09-30T14:34:45.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.282+0000 7fc7779d0700 1 --2- 172.21.15.148:0/4279480609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc770101980 0x7fc770108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:45.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.282+0000 7fc7779d0700 1 --2- 172.21.15.148:0/4279480609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7700ffde0 0x7fc770100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:45.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.282+0000 7fc7779d0700 1 --2- 172.21.15.148:0/4279480609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc770100fe0 0x7fc770101440 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:45.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.282+0000 7fc7779d0700 1 -- 172.21.15.148:0/4279480609 >> 172.21.15.148:0/4279480609 conn(0x7fc7700fb360 msgr2=0x7fc7700fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:34:45.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.282+0000 7fc7779d0700 1 -- 172.21.15.148:0/4279480609 shutdown_connections 2023-09-30T14:34:45.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.283+0000 7fc7779d0700 1 -- 172.21.15.148:0/4279480609 wait complete. 2023-09-30T14:34:45.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.283+0000 7fc7779d0700 1 Processor -- start 2023-09-30T14:34:45.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.283+0000 7fc7779d0700 1 -- start start 2023-09-30T14:34:45.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.283+0000 7fc7779d0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7700ffde0 0x7fc770103d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:34:45.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.283+0000 7fc7779d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc770100fe0 0x7fc7701023c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:34:45.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.283+0000 7fc7779d0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc770101980 0x7fc770102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:34:45.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.283+0000 7fc7779d0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc770103080 con 0x7fc770100fe0 2023-09-30T14:34:45.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.283+0000 7fc7779d0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7701031f0 con 0x7fc770101980 2023-09-30T14:34:45.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.283+0000 7fc7779d0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7701a0ba0 con 0x7fc7700ffde0 2023-09-30T14:34:45.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc775f6d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc770101980 0x7fc770102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:34:45.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc775f6d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc770101980 0x7fc770102900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51838/0 (socket says 172.21.15.148:51838) 2023-09-30T14:34:45.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc775f6d700 1 -- 172.21.15.148:0/478181627 learned_addr learned my addr 172.21.15.148:0/478181627 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-09-30T14:34:45.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc77576c700 1 --2- 172.21.15.148:0/478181627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7700ffde0 0x7fc770103d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:34:45.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc774f6b700 1 --2- 172.21.15.148:0/478181627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc770100fe0 0x7fc7701023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:34:45.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc775f6d700 1 -- 172.21.15.148:0/478181627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7700ffde0 msgr2=0x7fc770103d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:34:45.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc775f6d700 1 --2- 172.21.15.148:0/478181627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7700ffde0 0x7fc770103d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:45.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc775f6d700 1 -- 172.21.15.148:0/478181627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc770100fe0 msgr2=0x7fc7701023c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:34:45.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc775f6d700 1 --2- 172.21.15.148:0/478181627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc770100fe0 0x7fc7701023c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:34:45.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc775f6d700 1 -- 172.21.15.148:0/478181627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76c0095f0 con 0x7fc770101980 2023-09-30T14:34:45.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc774f6b700 1 --2- 172.21.15.148:0/478181627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc770100fe0 0x7fc7701023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:34:45.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc775f6d700 1 --2- 172.21.15.148:0/478181627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc770101980 0x7fc770102900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc768007d70 tx=0x7fc76800a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:34:45.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc7667fc700 1 -- 172.21.15.148:0/478181627 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76800ad40 con 0x7fc770101980 2023-09-30T14:34:45.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc7667fc700 1 -- 172.21.15.148:0/478181627 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7680128e0 con 0x7fc770101980 2023-09-30T14:34:45.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc7779d0700 1 -- 172.21.15.148:0/478181627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7701a0e80 con 0x7fc770101980 2023-09-30T14:34:45.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc7667fc700 1 -- 172.21.15.148:0/478181627 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768011780 con 0x7fc770101980 2023-09-30T14:34:45.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.284+0000 7fc7779d0700 1 -- 172.21.15.148:0/478181627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7701a12a0 con 0x7fc770101980 2023-09-30T14:34:45.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.285+0000 7fc7779d0700 1 -- 172.21.15.148:0/478181627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77010b5e0 con 0x7fc770101980 2023-09-30T14:34:45.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.286+0000 7fc7667fc700 1 -- 172.21.15.148:0/478181627 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fc768012a50 con 0x7fc770101980 2023-09-30T14:34:45.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.287+0000 7fc7667fc700 1 --2- 172.21.15.148:0/478181627 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc75c0a0fa0 0x7fc75c0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:34:45.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.287+0000 7fc7667fc700 1 -- 172.21.15.148:0/478181627 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fc7680cc1a0 con 0x7fc770101980 2023-09-30T14:34:45.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.287+0000 7fc77576c700 1 --2- 172.21.15.148:0/478181627 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc75c0a0fa0 0x7fc75c0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:34:45.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.288+0000 7fc77576c700 1 --2- 172.21.15.148:0/478181627 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc75c0a0fa0 0x7fc75c0a3460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc760005fd0 tx=0x7fc760005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:34:45.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.290+0000 7fc7667fc700 1 -- 172.21.15.148:0/478181627 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fc768096610 con 0x7fc770101980 2023-09-30T14:34:45.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:34:45.490+0000 7fc7779d0700 1 -- 172.21.15.148:0/478181627 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7700698f0 con 0x7fc75c0a0fa0 2023-09-30T14:34:45.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:45 smithi148 ceph-mon[102355]: purged_snaps scrub starts 2023-09-30T14:34:45.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:45 smithi148 ceph-mon[102355]: purged_snaps scrub ok 2023-09-30T14:34:45.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:45 smithi148 ceph-mon[102355]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:45.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:45 smithi148 ceph-mon[102355]: Detected new or changed devices on smithi032 2023-09-30T14:34:45.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:45 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi032 to 5724M 2023-09-30T14:34:45.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:45 smithi148 ceph-mon[102355]: osd.6 [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433] boot 2023-09-30T14:34:45.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:45 smithi148 ceph-mon[102355]: osdmap e46: 7 total, 7 up, 7 in 2023-09-30T14:34:45.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:45 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-30T14:34:45.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:45 smithi032 ceph-mon[98592]: purged_snaps scrub starts 2023-09-30T14:34:45.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:45 smithi032 ceph-mon[98592]: purged_snaps scrub ok 2023-09-30T14:34:45.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:45 smithi032 ceph-mon[98592]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:45.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:45 smithi032 ceph-mon[98592]: Detected new or changed devices on smithi032 2023-09-30T14:34:45.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:45 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi032 to 5724M 2023-09-30T14:34:45.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:45 smithi032 ceph-mon[98592]: osd.6 [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433] boot 2023-09-30T14:34:45.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:45 smithi032 ceph-mon[98592]: osdmap e46: 7 total, 7 up, 7 in 2023-09-30T14:34:45.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:45 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-30T14:34:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:45 smithi156 ceph-mon[103579]: purged_snaps scrub starts 2023-09-30T14:34:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:45 smithi156 ceph-mon[103579]: purged_snaps scrub ok 2023-09-30T14:34:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:45 smithi156 ceph-mon[103579]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-30T14:34:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:45 smithi156 ceph-mon[103579]: Detected new or changed devices on smithi032 2023-09-30T14:34:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:45 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi032 to 5724M 2023-09-30T14:34:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:45 smithi156 ceph-mon[103579]: osd.6 [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433] boot 2023-09-30T14:34:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:45 smithi156 ceph-mon[103579]: osdmap e46: 7 total, 7 up, 7 in 2023-09-30T14:34:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:45 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-30T14:34:46.578 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:46 smithi148 ceph-mon[102355]: osdmap e47: 7 total, 7 up, 7 in 2023-09-30T14:34:46.578 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:46 smithi148 ceph-mon[102355]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:34:46.578 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:46 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:34:46.579 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:46 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:34:46.579 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:46 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:46.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:46 smithi032 ceph-mon[98592]: osdmap e47: 7 total, 7 up, 7 in 2023-09-30T14:34:46.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:46 smithi032 ceph-mon[98592]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:34:46.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:46 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:34:46.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:46 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:34:46.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:46 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:46.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:46 smithi156 ceph-mon[103579]: osdmap e47: 7 total, 7 up, 7 in 2023-09-30T14:34:46.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:46 smithi156 ceph-mon[103579]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:34:46.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:46 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:34:46.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:46 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:34:46.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:46 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:47.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:47 smithi148 ceph-mon[102355]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:47.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:47 smithi032 ceph-mon[98592]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:47.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:47 smithi156 ceph-mon[103579]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:48.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:48 smithi032 ceph-mon[98592]: from='client.? 172.21.15.148:0/3015870578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15059921-bebf-46fc-87ba-415e7241af36"}]: dispatch 2023-09-30T14:34:48.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:48 smithi032 ceph-mon[98592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15059921-bebf-46fc-87ba-415e7241af36"}]: dispatch 2023-09-30T14:34:48.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:48 smithi032 ceph-mon[98592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15059921-bebf-46fc-87ba-415e7241af36"}]': finished 2023-09-30T14:34:48.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:48 smithi032 ceph-mon[98592]: osdmap e48: 8 total, 7 up, 8 in 2023-09-30T14:34:48.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-30T14:34:48.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:48 smithi156 ceph-mon[103579]: from='client.? 172.21.15.148:0/3015870578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15059921-bebf-46fc-87ba-415e7241af36"}]: dispatch 2023-09-30T14:34:48.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:48 smithi156 ceph-mon[103579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15059921-bebf-46fc-87ba-415e7241af36"}]: dispatch 2023-09-30T14:34:48.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:48 smithi156 ceph-mon[103579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15059921-bebf-46fc-87ba-415e7241af36"}]': finished 2023-09-30T14:34:48.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:48 smithi156 ceph-mon[103579]: osdmap e48: 8 total, 7 up, 8 in 2023-09-30T14:34:48.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-30T14:34:48.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:48 smithi148 ceph-mon[102355]: from='client.? 172.21.15.148:0/3015870578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15059921-bebf-46fc-87ba-415e7241af36"}]: dispatch 2023-09-30T14:34:48.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:48 smithi148 ceph-mon[102355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15059921-bebf-46fc-87ba-415e7241af36"}]: dispatch 2023-09-30T14:34:48.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:48 smithi148 ceph-mon[102355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15059921-bebf-46fc-87ba-415e7241af36"}]': finished 2023-09-30T14:34:48.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:48 smithi148 ceph-mon[102355]: osdmap e48: 8 total, 7 up, 8 in 2023-09-30T14:34:48.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-30T14:34:49.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:49 smithi032 ceph-mon[98592]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:50.038 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:49 smithi032 ceph-mon[98592]: from='client.? 172.21.15.148:0/3706157547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:34:50.038 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:49 smithi148 ceph-mon[102355]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:50.040 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:49 smithi148 ceph-mon[102355]: from='client.? 172.21.15.148:0/3706157547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:34:50.040 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:49 smithi156 ceph-mon[103579]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:50.040 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:49 smithi156 ceph-mon[103579]: from='client.? 172.21.15.148:0/3706157547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:34:51.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:51 smithi032 ceph-mon[98592]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:51.788 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:51 smithi148 ceph-mon[102355]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:51.804 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:51 smithi156 ceph-mon[103579]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:53.470 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:53 smithi148 ceph-mon[102355]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:53.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:53 smithi032 ceph-mon[98592]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:53.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:53 smithi156 ceph-mon[103579]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:55.443 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:55 smithi148 ceph-mon[102355]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:55.444 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:55 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-30T14:34:55.444 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:55 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:55.444 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:55 smithi148 ceph-mon[102355]: Deploying daemon osd.7 on smithi148 2023-09-30T14:34:55.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:55 smithi032 ceph-mon[98592]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:55.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:55 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-30T14:34:55.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:55 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:55.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:55 smithi032 ceph-mon[98592]: Deploying daemon osd.7 on smithi148 2023-09-30T14:34:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:55 smithi156 ceph-mon[103579]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:55 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-30T14:34:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:55 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:34:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:55 smithi156 ceph-mon[103579]: Deploying daemon osd.7 on smithi148 2023-09-30T14:34:57.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:57 smithi032 ceph-mon[98592]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:57 smithi156 ceph-mon[103579]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:57 smithi148 ceph-mon[102355]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:59.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:59 smithi148 ceph-mon[102355]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:59.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:59.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:59.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:34:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:59.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:59 smithi032 ceph-mon[98592]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:59.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:59 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:59.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:59 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:59.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:34:59 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:59 smithi156 ceph-mon[103579]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:34:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:59 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:34:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:59 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:34:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:34:59 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:01.526 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:01 smithi148 ceph-mon[102355]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:35:01.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:01 smithi032 ceph-mon[98592]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:35:01.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:01 smithi156 ceph-mon[103579]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:35:01.859 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 7 on host 'smithi148' 2023-09-30T14:35:01.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:01.858+0000 7fc7667fc700 1 -- 172.21.15.148:0/478181627 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc7700698f0 con 0x7fc75c0a0fa0 2023-09-30T14:35:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:01.861+0000 7fc7779d0700 1 -- 172.21.15.148:0/478181627 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc75c0a0fa0 msgr2=0x7fc75c0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:35:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:01.862+0000 7fc7779d0700 1 --2- 172.21.15.148:0/478181627 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc75c0a0fa0 0x7fc75c0a3460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc760005fd0 tx=0x7fc760005dd0 comp rx=0 tx=0).stop 2023-09-30T14:35:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:01.862+0000 7fc7779d0700 1 -- 172.21.15.148:0/478181627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc770101980 msgr2=0x7fc770102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:35:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:01.862+0000 7fc7779d0700 1 --2- 172.21.15.148:0/478181627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc770101980 0x7fc770102900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc768007d70 tx=0x7fc76800a340 comp rx=0 tx=0).stop 2023-09-30T14:35:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:01.862+0000 7fc7779d0700 1 -- 172.21.15.148:0/478181627 shutdown_connections 2023-09-30T14:35:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:01.862+0000 7fc7779d0700 1 --2- 172.21.15.148:0/478181627 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc75c0a0fa0 0x7fc75c0a3460 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:01.862+0000 7fc7779d0700 1 --2- 172.21.15.148:0/478181627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc770101980 0x7fc770102900 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:01.862+0000 7fc7779d0700 1 --2- 172.21.15.148:0/478181627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7700ffde0 0x7fc770103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:01.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:01.862+0000 7fc7779d0700 1 --2- 172.21.15.148:0/478181627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc770100fe0 0x7fc7701023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:01.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:01.862+0000 7fc7779d0700 1 -- 172.21.15.148:0/478181627 >> 172.21.15.148:0/478181627 conn(0x7fc7700fb360 msgr2=0x7fc7700691e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:35:01.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:01.862+0000 7fc7779d0700 1 -- 172.21.15.148:0/478181627 shutdown_connections 2023-09-30T14:35:01.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:01.862+0000 7fc7779d0700 1 -- 172.21.15.148:0/478181627 wait complete. 2023-09-30T14:35:02.302 DEBUG:teuthology.orchestra.run.smithi148:osd.7> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.7.service 2023-09-30T14:35:02.304 INFO:tasks.cephadm:Deploying osd.8 on smithi156 with /dev/vg_nvme/lv_2... 2023-09-30T14:35:02.305 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-30T14:35:02.336 INFO:journalctl@ceph.osd.7.smithi148.stdout:-- Logs begin at Sat 2023-09-30 14:16:26 UTC. -- 2023-09-30T14:35:02.605 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:02.605 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:02.605 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:02.606 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:02.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:02.859 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 14:35:02 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T14:35:02.604+0000 7fcc9aeb6640 -1 osd.7 0 log_to_monitors true 2023-09-30T14:35:02.874 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.c/config 2023-09-30T14:35:02.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:02.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:02.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:02.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:03 smithi156 ceph-mon[103579]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:35:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:03 smithi156 ceph-mon[103579]: from='osd.7 [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-30T14:35:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:03 smithi156 ceph-mon[103579]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-30T14:35:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:03 smithi148 ceph-mon[102355]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:35:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:03 smithi148 ceph-mon[102355]: from='osd.7 [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-30T14:35:03.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:03 smithi148 ceph-mon[102355]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-30T14:35:03.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:03 smithi032 ceph-mon[98592]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:35:03.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:03 smithi032 ceph-mon[98592]: from='osd.7 [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-30T14:35:03.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:03 smithi032 ceph-mon[98592]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-30T14:35:04.378 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-30T14:35:04.379 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-30T14:35:04.379 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-09-30T14:35:04.379 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-09-30T14:35:04.379 INFO:teuthology.orchestra.run.smithi156.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00819788 s, 1.3 GB/s 2023-09-30T14:35:04.379 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-09-30T14:35:04.727 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:04 smithi156 ceph-mon[103579]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-30T14:35:04.727 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:04 smithi156 ceph-mon[103579]: osdmap e49: 8 total, 7 up, 8 in 2023-09-30T14:35:04.727 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-30T14:35:04.727 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:04 smithi156 ceph-mon[103579]: from='osd.7 [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:35:04.727 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:04 smithi156 ceph-mon[103579]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:35:04.728 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi156:vg_nvme/lv_2 2023-09-30T14:35:04.857 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 14:35:04 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T14:35:04.494+0000 7fcc8fd2c700 -1 osd.7 0 waiting for initial osdmap 2023-09-30T14:35:04.858 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 14:35:04 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T14:35:04.505+0000 7fcc8c322700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T14:35:04.858 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:04 smithi148 ceph-mon[102355]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-30T14:35:04.858 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:04 smithi148 ceph-mon[102355]: osdmap e49: 8 total, 7 up, 8 in 2023-09-30T14:35:04.858 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:04 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-30T14:35:04.858 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:04 smithi148 ceph-mon[102355]: from='osd.7 [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:35:04.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:04 smithi148 ceph-mon[102355]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:35:04.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:04 smithi032 ceph-mon[98592]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-30T14:35:04.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:04 smithi032 ceph-mon[98592]: osdmap e49: 8 total, 7 up, 8 in 2023-09-30T14:35:04.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-30T14:35:04.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:04 smithi032 ceph-mon[98592]: from='osd.7 [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:35:04.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:04 smithi032 ceph-mon[98592]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:35:05.288 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.c/config 2023-09-30T14:35:05.775 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:35:05.775 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2023-09-30T14:35:05.775 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: osdmap e50: 8 total, 7 up, 8 in 2023-09-30T14:35:05.775 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-30T14:35:05.775 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-30T14:35:05.775 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:05.775 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:05.775 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:05.776 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:05.776 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:05.776 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:05.776 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:05.776 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:05.776 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:05.776 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:35:05.776 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:05.776 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:05.776 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:05 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:05.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:35:05.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2023-09-30T14:35:05.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: osdmap e50: 8 total, 7 up, 8 in 2023-09-30T14:35:05.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-30T14:35:05.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-30T14:35:05.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:05.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:05.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:05.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:05.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:05.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:05.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:05.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:05.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:05.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:35:05.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:05.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:05.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:05 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:05.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-30T14:35:05.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2023-09-30T14:35:05.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: osdmap e50: 8 total, 7 up, 8 in 2023-09-30T14:35:05.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-30T14:35:05.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-30T14:35:05.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:05.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:05.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:05.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:05.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:05.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:05.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:05.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:05.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:05.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:35:05.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:05.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:05.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:05 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:06.465 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.462+0000 7f06eb6d5700 1 -- 172.21.15.156:0/3270243976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06e40ffde0 msgr2=0x7f06e4100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:35:06.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.462+0000 7f06eb6d5700 1 --2- 172.21.15.156:0/3270243976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06e40ffde0 0x7f06e4100200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009990 tx=0x7f06d4009ca0 comp rx=0 tx=0).stop 2023-09-30T14:35:06.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.462+0000 7f06eb6d5700 1 -- 172.21.15.156:0/3270243976 shutdown_connections 2023-09-30T14:35:06.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.462+0000 7f06eb6d5700 1 --2- 172.21.15.156:0/3270243976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06e4101980 0x7f06e4108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:06.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.462+0000 7f06eb6d5700 1 --2- 172.21.15.156:0/3270243976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06e40ffde0 0x7f06e4100200 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:06.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.462+0000 7f06eb6d5700 1 --2- 172.21.15.156:0/3270243976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06e4100fe0 0x7f06e4101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:06.466 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.462+0000 7f06eb6d5700 1 -- 172.21.15.156:0/3270243976 >> 172.21.15.156:0/3270243976 conn(0x7f06e40fb360 msgr2=0x7f06e40fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:35:06.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.462+0000 7f06eb6d5700 1 -- 172.21.15.156:0/3270243976 shutdown_connections 2023-09-30T14:35:06.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.462+0000 7f06eb6d5700 1 -- 172.21.15.156:0/3270243976 wait complete. 2023-09-30T14:35:06.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06eb6d5700 1 Processor -- start 2023-09-30T14:35:06.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06eb6d5700 1 -- start start 2023-09-30T14:35:06.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06eb6d5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06e40ffde0 0x7f06e4198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:35:06.467 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06eb6d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06e4100fe0 0x7f06e4199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:35:06.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06eb6d5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06e4101980 0x7f06e41a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:35:06.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06eb6d5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e41a09a0 con 0x7f06e4100fe0 2023-09-30T14:35:06.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06eb6d5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e41a0ae0 con 0x7f06e4101980 2023-09-30T14:35:06.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06eb6d5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e41a0c70 con 0x7f06e40ffde0 2023-09-30T14:35:06.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06e9471700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06e40ffde0 0x7f06e4198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:35:06.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06e9471700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06e40ffde0 0x7f06e4198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36214/0 (socket says 172.21.15.156:36214) 2023-09-30T14:35:06.468 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06e9471700 1 -- 172.21.15.156:0/3792025905 learned_addr learned my addr 172.21.15.156:0/3792025905 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-30T14:35:06.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06e8c70700 1 --2- 172.21.15.156:0/3792025905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06e4100fe0 0x7f06e4199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:35:06.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06e9c72700 1 --2- 172.21.15.156:0/3792025905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06e4101980 0x7f06e41a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:35:06.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06e9471700 1 -- 172.21.15.156:0/3792025905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06e4101980 msgr2=0x7f06e41a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:35:06.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06e9471700 1 --2- 172.21.15.156:0/3792025905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06e4101980 0x7f06e41a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:06.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06e9471700 1 -- 172.21.15.156:0/3792025905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06e4100fe0 msgr2=0x7f06e4199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:35:06.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06e9471700 1 --2- 172.21.15.156:0/3792025905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06e4100fe0 0x7f06e4199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:06.469 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.463+0000 7f06e9471700 1 -- 172.21.15.156:0/3792025905 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d40095f0 con 0x7f06e40ffde0 2023-09-30T14:35:06.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.464+0000 7f06e9471700 1 --2- 172.21.15.156:0/3792025905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06e40ffde0 0x7f06e4198e10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f06d4004730 tx=0x7f06d4004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:35:06.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.464+0000 7f06da7fc700 1 -- 172.21.15.156:0/3792025905 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d401d070 con 0x7f06e40ffde0 2023-09-30T14:35:06.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.464+0000 7f06da7fc700 1 -- 172.21.15.156:0/3792025905 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06d40054f0 con 0x7f06e40ffde0 2023-09-30T14:35:06.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.464+0000 7f06eb6d5700 1 -- 172.21.15.156:0/3792025905 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e41a0ef0 con 0x7f06e40ffde0 2023-09-30T14:35:06.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.464+0000 7f06da7fc700 1 -- 172.21.15.156:0/3792025905 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4004c60 con 0x7f06e40ffde0 2023-09-30T14:35:06.470 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.464+0000 7f06eb6d5700 1 -- 172.21.15.156:0/3792025905 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e41a1250 con 0x7f06e40ffde0 2023-09-30T14:35:06.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.465+0000 7f06eb6d5700 1 -- 172.21.15.156:0/3792025905 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e410b650 con 0x7f06e40ffde0 2023-09-30T14:35:06.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.466+0000 7f06da7fc700 1 -- 172.21.15.156:0/3792025905 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f06d400fbb0 con 0x7f06e40ffde0 2023-09-30T14:35:06.472 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.466+0000 7f06da7fc700 1 --2- 172.21.15.156:0/3792025905 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f06d00a0fa0 0x7f06d00a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:35:06.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.466+0000 7f06da7fc700 1 -- 172.21.15.156:0/3792025905 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5430+0+0 (secure 0 0 0) 0x7f06d40ccf00 con 0x7f06e40ffde0 2023-09-30T14:35:06.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.466+0000 7f06e8c70700 1 --2- 172.21.15.156:0/3792025905 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f06d00a0fa0 0x7f06d00a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:35:06.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.467+0000 7f06e8c70700 1 --2- 172.21.15.156:0/3792025905 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f06d00a0fa0 0x7f06d00a3460 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f06e0005950 tx=0x7f06e000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:35:06.473 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.470+0000 7f06da7fc700 1 -- 172.21.15.156:0/3792025905 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f06d405af30 con 0x7f06e40ffde0 2023-09-30T14:35:06.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:06 smithi156 ceph-mon[103579]: purged_snaps scrub starts 2023-09-30T14:35:06.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:06 smithi156 ceph-mon[103579]: purged_snaps scrub ok 2023-09-30T14:35:06.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:06 smithi156 ceph-mon[103579]: Detected new or changed devices on smithi148 2023-09-30T14:35:06.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:06 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi148 to 5722M 2023-09-30T14:35:06.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:06 smithi156 ceph-mon[103579]: osd.7 [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147] boot 2023-09-30T14:35:06.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:06 smithi156 ceph-mon[103579]: osdmap e51: 8 total, 8 up, 8 in 2023-09-30T14:35:06.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:06 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-30T14:35:06.671 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:06.670+0000 7f06eb6d5700 1 -- 172.21.15.156:0/3792025905 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f06e4069970 con 0x7f06d00a0fa0 2023-09-30T14:35:06.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:06 smithi148 ceph-mon[102355]: purged_snaps scrub starts 2023-09-30T14:35:06.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:06 smithi148 ceph-mon[102355]: purged_snaps scrub ok 2023-09-30T14:35:06.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:06 smithi148 ceph-mon[102355]: Detected new or changed devices on smithi148 2023-09-30T14:35:06.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:06 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi148 to 5722M 2023-09-30T14:35:06.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:06 smithi148 ceph-mon[102355]: osd.7 [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147] boot 2023-09-30T14:35:06.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:06 smithi148 ceph-mon[102355]: osdmap e51: 8 total, 8 up, 8 in 2023-09-30T14:35:06.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-30T14:35:06.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:06 smithi032 ceph-mon[98592]: purged_snaps scrub starts 2023-09-30T14:35:06.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:06 smithi032 ceph-mon[98592]: purged_snaps scrub ok 2023-09-30T14:35:06.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:06 smithi032 ceph-mon[98592]: Detected new or changed devices on smithi148 2023-09-30T14:35:06.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:06 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi148 to 5722M 2023-09-30T14:35:06.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:06 smithi032 ceph-mon[98592]: osd.7 [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147] boot 2023-09-30T14:35:06.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:06 smithi032 ceph-mon[98592]: osdmap e51: 8 total, 8 up, 8 in 2023-09-30T14:35:06.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:06 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-30T14:35:07.749 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:07 smithi156 ceph-mon[103579]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:07.749 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:07 smithi156 ceph-mon[103579]: osdmap e52: 8 total, 8 up, 8 in 2023-09-30T14:35:07.749 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:35:07.749 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:35:07.749 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:07 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:07.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:07 smithi148 ceph-mon[102355]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:07.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:07 smithi148 ceph-mon[102355]: osdmap e52: 8 total, 8 up, 8 in 2023-09-30T14:35:07.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:35:07.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:35:07.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:07 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:07.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:07 smithi032 ceph-mon[98592]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:07.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:07 smithi032 ceph-mon[98592]: osdmap e52: 8 total, 8 up, 8 in 2023-09-30T14:35:07.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:35:07.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:35:07.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:07 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:08.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:08 smithi156 ceph-mon[103579]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:35:08.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:08 smithi156 ceph-mon[103579]: osdmap e53: 8 total, 8 up, 8 in 2023-09-30T14:35:08.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:08 smithi148 ceph-mon[102355]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:35:08.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:08 smithi148 ceph-mon[102355]: osdmap e53: 8 total, 8 up, 8 in 2023-09-30T14:35:08.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:08 smithi032 ceph-mon[98592]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:35:08.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:08 smithi032 ceph-mon[98592]: osdmap e53: 8 total, 8 up, 8 in 2023-09-30T14:35:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:09 smithi156 ceph-mon[103579]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:09 smithi156 ceph-mon[103579]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-30T14:35:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:09 smithi156 ceph-mon[103579]: osdmap e54: 8 total, 8 up, 8 in 2023-09-30T14:35:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:09 smithi156 ceph-mon[103579]: from='client.? 172.21.15.156:0/1321695014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a853c86-fce1-450d-b902-80989d4a5f21"}]: dispatch 2023-09-30T14:35:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:09 smithi156 ceph-mon[103579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a853c86-fce1-450d-b902-80989d4a5f21"}]: dispatch 2023-09-30T14:35:09.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:09 smithi156 ceph-mon[103579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a853c86-fce1-450d-b902-80989d4a5f21"}]': finished 2023-09-30T14:35:09.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:09 smithi156 ceph-mon[103579]: osdmap e55: 9 total, 8 up, 9 in 2023-09-30T14:35:09.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-30T14:35:09.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:09 smithi148 ceph-mon[102355]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:09.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:09 smithi148 ceph-mon[102355]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-30T14:35:09.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:09 smithi148 ceph-mon[102355]: osdmap e54: 8 total, 8 up, 8 in 2023-09-30T14:35:09.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:09 smithi148 ceph-mon[102355]: from='client.? 172.21.15.156:0/1321695014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a853c86-fce1-450d-b902-80989d4a5f21"}]: dispatch 2023-09-30T14:35:09.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:09 smithi148 ceph-mon[102355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a853c86-fce1-450d-b902-80989d4a5f21"}]: dispatch 2023-09-30T14:35:09.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:09 smithi148 ceph-mon[102355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a853c86-fce1-450d-b902-80989d4a5f21"}]': finished 2023-09-30T14:35:09.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:09 smithi148 ceph-mon[102355]: osdmap e55: 9 total, 8 up, 9 in 2023-09-30T14:35:09.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-30T14:35:09.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:09 smithi032 ceph-mon[98592]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:09.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:09 smithi032 ceph-mon[98592]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-30T14:35:09.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:09 smithi032 ceph-mon[98592]: osdmap e54: 8 total, 8 up, 8 in 2023-09-30T14:35:09.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:09 smithi032 ceph-mon[98592]: from='client.? 172.21.15.156:0/1321695014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a853c86-fce1-450d-b902-80989d4a5f21"}]: dispatch 2023-09-30T14:35:09.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:09 smithi032 ceph-mon[98592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a853c86-fce1-450d-b902-80989d4a5f21"}]: dispatch 2023-09-30T14:35:09.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:09 smithi032 ceph-mon[98592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a853c86-fce1-450d-b902-80989d4a5f21"}]': finished 2023-09-30T14:35:09.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:09 smithi032 ceph-mon[98592]: osdmap e55: 9 total, 8 up, 9 in 2023-09-30T14:35:09.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-30T14:35:10.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:10 smithi156 ceph-mon[103579]: from='client.? 172.21.15.156:0/3286072985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:35:10.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:10 smithi148 ceph-mon[102355]: from='client.? 172.21.15.156:0/3286072985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:35:10.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:10 smithi032 ceph-mon[98592]: from='client.? 172.21.15.156:0/3286072985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:35:11.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:11 smithi156 ceph-mon[103579]: pgmap v164: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:11.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:11 smithi148 ceph-mon[102355]: pgmap v164: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:11.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:11 smithi032 ceph-mon[98592]: pgmap v164: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:13.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:12 smithi156 ceph-mon[103579]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:13.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:12 smithi156 ceph-mon[103579]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-09-30T14:35:13.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:12 smithi156 ceph-mon[103579]: Cluster is now healthy 2023-09-30T14:35:13.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:12 smithi148 ceph-mon[102355]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:13.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:12 smithi148 ceph-mon[102355]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-09-30T14:35:13.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:12 smithi148 ceph-mon[102355]: Cluster is now healthy 2023-09-30T14:35:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:12 smithi032 ceph-mon[98592]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:12 smithi032 ceph-mon[98592]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-09-30T14:35:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:12 smithi032 ceph-mon[98592]: Cluster is now healthy 2023-09-30T14:35:15.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:14 smithi156 ceph-mon[103579]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:15.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:14 smithi148 ceph-mon[102355]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:15.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:14 smithi032 ceph-mon[98592]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:16.260 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:15 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-30T14:35:16.260 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:15 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:16.260 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:15 smithi156 ceph-mon[103579]: Deploying daemon osd.8 on smithi156 2023-09-30T14:35:16.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:15 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-30T14:35:16.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:15 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:16.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:15 smithi148 ceph-mon[102355]: Deploying daemon osd.8 on smithi156 2023-09-30T14:35:16.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-30T14:35:16.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:15 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:16.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:15 smithi032 ceph-mon[98592]: Deploying daemon osd.8 on smithi156 2023-09-30T14:35:17.256 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:16 smithi148 ceph-mon[102355]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:17.286 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:16 smithi156 ceph-mon[103579]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:17.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:16 smithi032 ceph-mon[98592]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:19.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:18 smithi156 ceph-mon[103579]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:19.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:18 smithi148 ceph-mon[102355]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:19.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:18 smithi032 ceph-mon[98592]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:21.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:20 smithi156 ceph-mon[103579]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:21.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:20 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:21.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:20 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:21.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:20 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:35:21.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:20 smithi032 ceph-mon[98592]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:21.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:20 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:21.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:20 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:21.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:20 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:35:21.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:20 smithi148 ceph-mon[102355]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:21.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:20 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:21.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:20 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:21.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:20 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:35:22.534 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:22.533+0000 7f06da7fc700 1 -- 172.21.15.156:0/3792025905 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f06e4069970 con 0x7f06d00a0fa0 2023-09-30T14:35:22.534 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 8 on host 'smithi156' 2023-09-30T14:35:22.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:22.535+0000 7f06eb6d5700 1 -- 172.21.15.156:0/3792025905 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f06d00a0fa0 msgr2=0x7f06d00a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:35:22.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:22.535+0000 7f06eb6d5700 1 --2- 172.21.15.156:0/3792025905 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f06d00a0fa0 0x7f06d00a3460 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f06e0005950 tx=0x7f06e000a300 comp rx=0 tx=0).stop 2023-09-30T14:35:22.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:22.535+0000 7f06eb6d5700 1 -- 172.21.15.156:0/3792025905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06e40ffde0 msgr2=0x7f06e4198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:35:22.536 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:22.535+0000 7f06eb6d5700 1 --2- 172.21.15.156:0/3792025905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06e40ffde0 0x7f06e4198e10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f06d4004730 tx=0x7f06d4004760 comp rx=0 tx=0).stop 2023-09-30T14:35:22.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:22.535+0000 7f06eb6d5700 1 -- 172.21.15.156:0/3792025905 shutdown_connections 2023-09-30T14:35:22.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:22.535+0000 7f06eb6d5700 1 --2- 172.21.15.156:0/3792025905 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f06d00a0fa0 0x7f06d00a3460 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:22.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:22.535+0000 7f06eb6d5700 1 --2- 172.21.15.156:0/3792025905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06e4101980 0x7f06e41a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:22.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:22.535+0000 7f06eb6d5700 1 --2- 172.21.15.156:0/3792025905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06e40ffde0 0x7f06e4198e10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:22.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:22.535+0000 7f06eb6d5700 1 --2- 172.21.15.156:0/3792025905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06e4100fe0 0x7f06e4199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:22.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:22.535+0000 7f06eb6d5700 1 -- 172.21.15.156:0/3792025905 >> 172.21.15.156:0/3792025905 conn(0x7f06e40fb360 msgr2=0x7f06e4069260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:35:22.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:22.535+0000 7f06eb6d5700 1 -- 172.21.15.156:0/3792025905 shutdown_connections 2023-09-30T14:35:22.537 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:35:22.535+0000 7f06eb6d5700 1 -- 172.21.15.156:0/3792025905 wait complete. 2023-09-30T14:35:22.972 DEBUG:teuthology.orchestra.run.smithi156:osd.8> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.8.service 2023-09-30T14:35:22.975 INFO:tasks.cephadm:Deploying osd.9 on smithi032 with /dev/vg_nvme/lv_1... 2023-09-30T14:35:22.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-30T14:35:23.012 INFO:journalctl@ceph.osd.8.smithi156.stdout:-- Logs begin at Sat 2023-09-30 14:16:23 UTC. -- 2023-09-30T14:35:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:35:23.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:23 smithi032 ceph-mon[98592]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:23.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:23.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:23.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:23.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:23.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:23.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:23.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:23.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:35:23.670 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:23 smithi156 ceph-mon[103579]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:23.670 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:23.670 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:23.670 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:23.670 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:23.670 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:23.670 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:23.670 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:23.671 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:35:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:23 smithi148 ceph-mon[102355]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:23.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:23.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:23.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:23.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:23.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:35:24.031 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 14:35:23 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T14:35:23.783+0000 7f5360235640 -1 osd.8 0 log_to_monitors true 2023-09-30T14:35:24.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:24 smithi032 ceph-mon[98592]: from='osd.8 [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-30T14:35:24.658 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:24 smithi032 ceph-mon[98592]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-30T14:35:24.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:24 smithi156 ceph-mon[103579]: from='osd.8 [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-30T14:35:24.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:24 smithi156 ceph-mon[103579]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-30T14:35:24.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:24 smithi148 ceph-mon[102355]: from='osd.8 [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-30T14:35:24.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:24 smithi148 ceph-mon[102355]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-30T14:35:25.384 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:25 smithi156 ceph-mon[103579]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:25.384 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:25 smithi156 ceph-mon[103579]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-30T14:35:25.385 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:25 smithi156 ceph-mon[103579]: osdmap e56: 9 total, 8 up, 9 in 2023-09-30T14:35:25.385 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:25 smithi156 ceph-mon[103579]: from='osd.8 [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:35:25.385 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:25 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-30T14:35:25.385 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:25 smithi156 ceph-mon[103579]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:35:25.389 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-30T14:35:25.389 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-30T14:35:25.390 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2023-09-30T14:35:25.390 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2023-09-30T14:35:25.390 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0136919 s, 766 MB/s 2023-09-30T14:35:25.390 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2023-09-30T14:35:25.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:25 smithi032 ceph-mon[98592]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:25.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:25 smithi032 ceph-mon[98592]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-30T14:35:25.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:25 smithi032 ceph-mon[98592]: osdmap e56: 9 total, 8 up, 9 in 2023-09-30T14:35:25.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:25 smithi032 ceph-mon[98592]: from='osd.8 [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:35:25.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:25 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-30T14:35:25.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:25 smithi032 ceph-mon[98592]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:35:25.805 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 14:35:25 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T14:35:25.386+0000 7f53550ab700 -1 osd.8 0 waiting for initial osdmap 2023-09-30T14:35:25.805 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 14:35:25 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T14:35:25.398+0000 7f534de9a700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T14:35:25.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi032:vg_nvme/lv_1 2023-09-30T14:35:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:25 smithi148 ceph-mon[102355]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:25 smithi148 ceph-mon[102355]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-30T14:35:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:25 smithi148 ceph-mon[102355]: osdmap e56: 9 total, 8 up, 9 in 2023-09-30T14:35:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:25 smithi148 ceph-mon[102355]: from='osd.8 [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:35:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:25 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-30T14:35:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:25 smithi148 ceph-mon[102355]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:35:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:35:26.653 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:26 smithi032 ceph-mon[98592]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-30T14:35:26.653 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:26 smithi032 ceph-mon[98592]: osdmap e57: 9 total, 8 up, 9 in 2023-09-30T14:35:26.653 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-30T14:35:26.653 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-30T14:35:26.653 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:26 smithi032 ceph-mon[98592]: Detected new or changed devices on smithi156 2023-09-30T14:35:26.653 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:26.654 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:26.654 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:26.654 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:26.654 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:26.654 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:26.654 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:26.654 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:26.654 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:26 smithi156 ceph-mon[103579]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-30T14:35:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:26 smithi156 ceph-mon[103579]: osdmap e57: 9 total, 8 up, 9 in 2023-09-30T14:35:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-30T14:35:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-30T14:35:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:26 smithi156 ceph-mon[103579]: Detected new or changed devices on smithi156 2023-09-30T14:35:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:26.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:26.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:26.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:26.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:26.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:26.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:26.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:26.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:26.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:26 smithi148 ceph-mon[102355]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-30T14:35:26.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:26 smithi148 ceph-mon[102355]: osdmap e57: 9 total, 8 up, 9 in 2023-09-30T14:35:26.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-30T14:35:26.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-30T14:35:26.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:26 smithi148 ceph-mon[102355]: Detected new or changed devices on smithi156 2023-09-30T14:35:26.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:26.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:26.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:26.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:26.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:26.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:26.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:26.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:26.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:27.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.574+0000 7f1770d5f700 1 -- 172.21.15.32:0/1902348735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c100fe0 msgr2=0x7f176c101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:35:27.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.574+0000 7f1770d5f700 1 --2- 172.21.15.32:0/1902348735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c100fe0 0x7f176c101440 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f175c0099c0 tx=0x7f175c009cd0 comp rx=0 tx=0).stop 2023-09-30T14:35:27.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.574+0000 7f1770d5f700 1 -- 172.21.15.32:0/1902348735 shutdown_connections 2023-09-30T14:35:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.574+0000 7f1770d5f700 1 --2- 172.21.15.32:0/1902348735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f176c101980 0x7f176c108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.574+0000 7f1770d5f700 1 --2- 172.21.15.32:0/1902348735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f176c0ffde0 0x7f176c100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.574+0000 7f1770d5f700 1 --2- 172.21.15.32:0/1902348735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c100fe0 0x7f176c101440 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.574+0000 7f1770d5f700 1 -- 172.21.15.32:0/1902348735 >> 172.21.15.32:0/1902348735 conn(0x7f176c0fb360 msgr2=0x7f176c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:35:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.574+0000 7f1770d5f700 1 -- 172.21.15.32:0/1902348735 shutdown_connections 2023-09-30T14:35:27.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.574+0000 7f1770d5f700 1 -- 172.21.15.32:0/1902348735 wait complete. 2023-09-30T14:35:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.575+0000 7f1770d5f700 1 Processor -- start 2023-09-30T14:35:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.575+0000 7f1770d5f700 1 -- start start 2023-09-30T14:35:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.575+0000 7f1770d5f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f176c0ffde0 0x7f176c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:35:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.575+0000 7f1770d5f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f176c100fe0 0x7f176c19b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:35:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.575+0000 7f1770d5f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c101980 0x7f176c19dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:35:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.575+0000 7f1770d5f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c19e640 con 0x7f176c101980 2023-09-30T14:35:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.575+0000 7f1770d5f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c19e7b0 con 0x7f176c100fe0 2023-09-30T14:35:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.575+0000 7f1770d5f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c19e970 con 0x7f176c0ffde0 2023-09-30T14:35:27.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.575+0000 7f176ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c101980 0x7f176c19dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:35:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.575+0000 7f176ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c101980 0x7f176c19dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33216/0 (socket says 172.21.15.32:33216) 2023-09-30T14:35:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.575+0000 7f176ad9d700 1 -- 172.21.15.32:0/3998443783 learned_addr learned my addr 172.21.15.32:0/3998443783 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:35:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.575+0000 7f176a59c700 1 --2- 172.21.15.32:0/3998443783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f176c0ffde0 0x7f176c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:35:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.576+0000 7f1769d9b700 1 --2- 172.21.15.32:0/3998443783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f176c100fe0 0x7f176c19b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:35:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.576+0000 7f176ad9d700 1 -- 172.21.15.32:0/3998443783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f176c0ffde0 msgr2=0x7f176c069470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:35:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.576+0000 7f176ad9d700 1 --2- 172.21.15.32:0/3998443783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f176c0ffde0 0x7f176c069470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.576+0000 7f176ad9d700 1 -- 172.21.15.32:0/3998443783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f176c100fe0 msgr2=0x7f176c19b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:35:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.576+0000 7f176ad9d700 1 --2- 172.21.15.32:0/3998443783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f176c100fe0 0x7f176c19b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.576+0000 7f176ad9d700 1 -- 172.21.15.32:0/3998443783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c0095f0 con 0x7f176c101980 2023-09-30T14:35:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.576+0000 7f176ad9d700 1 --2- 172.21.15.32:0/3998443783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c101980 0x7f176c19dea0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f176400ada0 tx=0x7f176400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:35:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.576+0000 7f17637fe700 1 -- 172.21.15.32:0/3998443783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176400f070 con 0x7f176c101980 2023-09-30T14:35:27.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.576+0000 7f17637fe700 1 -- 172.21.15.32:0/3998443783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1764007b30 con 0x7f176c101980 2023-09-30T14:35:27.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.576+0000 7f17637fe700 1 -- 172.21.15.32:0/3998443783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17640053e0 con 0x7f176c101980 2023-09-30T14:35:27.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.576+0000 7f1770d5f700 1 -- 172.21.15.32:0/3998443783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176c19ec50 con 0x7f176c101980 2023-09-30T14:35:27.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.576+0000 7f1770d5f700 1 -- 172.21.15.32:0/3998443783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c19f0d0 con 0x7f176c101980 2023-09-30T14:35:27.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.577+0000 7f1770d5f700 1 -- 172.21.15.32:0/3998443783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176c10b5e0 con 0x7f176c101980 2023-09-30T14:35:27.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.578+0000 7f17637fe700 1 -- 172.21.15.32:0/3998443783 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f176400c980 con 0x7f176c101980 2023-09-30T14:35:27.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.579+0000 7f17637fe700 1 --2- 172.21.15.32:0/3998443783 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f17580a0fa0 0x7f17580a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:35:27.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.579+0000 7f176a59c700 1 --2- 172.21.15.32:0/3998443783 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f17580a0fa0 0x7f17580a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:35:27.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.579+0000 7f17637fe700 1 -- 172.21.15.32:0/3998443783 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f17640cb380 con 0x7f176c101980 2023-09-30T14:35:27.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.580+0000 7f176a59c700 1 --2- 172.21.15.32:0/3998443783 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f17580a0fa0 0x7f17580a3460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f176c100e40 tx=0x7f175400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:35:27.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.584+0000 7f17637fe700 1 -- 172.21.15.32:0/3998443783 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f17640955a0 con 0x7f176c101980 2023-09-30T14:35:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:27 smithi032 ceph-mon[98592]: purged_snaps scrub starts 2023-09-30T14:35:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:27 smithi032 ceph-mon[98592]: purged_snaps scrub ok 2023-09-30T14:35:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:27 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi156 to 5722M 2023-09-30T14:35:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:27 smithi032 ceph-mon[98592]: pgmap v174: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:27.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:27 smithi032 ceph-mon[98592]: osd.8 [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463] boot 2023-09-30T14:35:27.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:27 smithi032 ceph-mon[98592]: osdmap e58: 9 total, 9 up, 9 in 2023-09-30T14:35:27.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:27 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-30T14:35:27.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:27 smithi032 ceph-mon[98592]: osdmap e59: 9 total, 9 up, 9 in 2023-09-30T14:35:27.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:27.793+0000 7f1770d5f700 1 -- 172.21.15.32:0/3998443783 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f176c061260 con 0x7f17580a0fa0 2023-09-30T14:35:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:27 smithi156 ceph-mon[103579]: purged_snaps scrub starts 2023-09-30T14:35:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:27 smithi156 ceph-mon[103579]: purged_snaps scrub ok 2023-09-30T14:35:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:27 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi156 to 5722M 2023-09-30T14:35:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:27 smithi156 ceph-mon[103579]: pgmap v174: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:27 smithi156 ceph-mon[103579]: osd.8 [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463] boot 2023-09-30T14:35:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:27 smithi156 ceph-mon[103579]: osdmap e58: 9 total, 9 up, 9 in 2023-09-30T14:35:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:27 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-30T14:35:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:27 smithi156 ceph-mon[103579]: osdmap e59: 9 total, 9 up, 9 in 2023-09-30T14:35:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:27 smithi148 ceph-mon[102355]: purged_snaps scrub starts 2023-09-30T14:35:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:27 smithi148 ceph-mon[102355]: purged_snaps scrub ok 2023-09-30T14:35:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:27 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi156 to 5722M 2023-09-30T14:35:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:27 smithi148 ceph-mon[102355]: pgmap v174: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-30T14:35:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:27 smithi148 ceph-mon[102355]: osd.8 [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463] boot 2023-09-30T14:35:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:27 smithi148 ceph-mon[102355]: osdmap e58: 9 total, 9 up, 9 in 2023-09-30T14:35:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:27 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-30T14:35:27.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:27 smithi148 ceph-mon[102355]: osdmap e59: 9 total, 9 up, 9 in 2023-09-30T14:35:28.650 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:35:28.650 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:35:28.650 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:28.650 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:28 smithi032 ceph-mon[98592]: osdmap e60: 9 total, 9 up, 9 in 2023-09-30T14:35:28.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:35:28.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:35:28.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:28.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:28 smithi156 ceph-mon[103579]: osdmap e60: 9 total, 9 up, 9 in 2023-09-30T14:35:28.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:35:28.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:35:28.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:28.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:28 smithi148 ceph-mon[102355]: osdmap e60: 9 total, 9 up, 9 in 2023-09-30T14:35:29.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:29 smithi156 ceph-mon[103579]: pgmap v177: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-30T14:35:29.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:29 smithi156 ceph-mon[103579]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:35:29.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:29 smithi156 ceph-mon[103579]: osdmap e61: 9 total, 9 up, 9 in 2023-09-30T14:35:29.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:29 smithi148 ceph-mon[102355]: pgmap v177: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-30T14:35:29.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:29 smithi148 ceph-mon[102355]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:35:29.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:29 smithi148 ceph-mon[102355]: osdmap e61: 9 total, 9 up, 9 in 2023-09-30T14:35:29.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:29 smithi032 ceph-mon[98592]: pgmap v177: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-30T14:35:29.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:29 smithi032 ceph-mon[98592]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:35:29.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:29 smithi032 ceph-mon[98592]: osdmap e61: 9 total, 9 up, 9 in 2023-09-30T14:35:31.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:30 smithi156 ceph-mon[103579]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-30T14:35:31.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:30 smithi156 ceph-mon[103579]: osdmap e62: 9 total, 9 up, 9 in 2023-09-30T14:35:31.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:30 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1082884929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47eafd4b-4c81-4a0c-9496-272c10e2fb23"}]: dispatch 2023-09-30T14:35:31.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:30 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1082884929' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47eafd4b-4c81-4a0c-9496-272c10e2fb23"}]': finished 2023-09-30T14:35:31.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:30 smithi156 ceph-mon[103579]: osdmap e63: 10 total, 9 up, 10 in 2023-09-30T14:35:31.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:30 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-30T14:35:31.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:30 smithi148 ceph-mon[102355]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-30T14:35:31.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:30 smithi148 ceph-mon[102355]: osdmap e62: 9 total, 9 up, 9 in 2023-09-30T14:35:31.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:30 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1082884929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47eafd4b-4c81-4a0c-9496-272c10e2fb23"}]: dispatch 2023-09-30T14:35:31.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:30 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1082884929' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47eafd4b-4c81-4a0c-9496-272c10e2fb23"}]': finished 2023-09-30T14:35:31.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:30 smithi148 ceph-mon[102355]: osdmap e63: 10 total, 9 up, 10 in 2023-09-30T14:35:31.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:30 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-30T14:35:31.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:30 smithi032 ceph-mon[98592]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-30T14:35:31.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:30 smithi032 ceph-mon[98592]: osdmap e62: 9 total, 9 up, 9 in 2023-09-30T14:35:31.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:30 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1082884929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47eafd4b-4c81-4a0c-9496-272c10e2fb23"}]: dispatch 2023-09-30T14:35:31.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:30 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1082884929' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47eafd4b-4c81-4a0c-9496-272c10e2fb23"}]': finished 2023-09-30T14:35:31.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:30 smithi032 ceph-mon[98592]: osdmap e63: 10 total, 9 up, 10 in 2023-09-30T14:35:31.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:30 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-30T14:35:32.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:31 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3339610552' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:35:32.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:31 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3339610552' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:35:32.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:31 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3339610552' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:35:33.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:32 smithi156 ceph-mon[103579]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-30T14:35:33.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:32 smithi148 ceph-mon[102355]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-30T14:35:33.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:32 smithi032 ceph-mon[98592]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-30T14:35:35.215 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:34 smithi032 ceph-mon[98592]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-09-30T14:35:35.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:34 smithi156 ceph-mon[103579]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-09-30T14:35:35.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:34 smithi148 ceph-mon[102355]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-09-30T14:35:37.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:36 smithi156 ceph-mon[103579]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-09-30T14:35:37.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:36 smithi148 ceph-mon[102355]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-09-30T14:35:37.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:36 smithi032 ceph-mon[98592]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-09-30T14:35:38.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-30T14:35:38.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:38.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:37 smithi156 ceph-mon[103579]: Deploying daemon osd.9 on smithi032 2023-09-30T14:35:38.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-30T14:35:38.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:38.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:37 smithi148 ceph-mon[102355]: Deploying daemon osd.9 on smithi032 2023-09-30T14:35:38.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-30T14:35:38.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:38.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:37 smithi032 ceph-mon[98592]: Deploying daemon osd.9 on smithi032 2023-09-30T14:35:39.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:38 smithi032 ceph-mon[98592]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-30T14:35:39.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:38 smithi156 ceph-mon[103579]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-30T14:35:39.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:38 smithi148 ceph-mon[102355]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-30T14:35:41.242 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:40 smithi032 ceph-mon[98592]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-09-30T14:35:41.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:40 smithi156 ceph-mon[103579]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-09-30T14:35:41.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:40 smithi148 ceph-mon[102355]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-09-30T14:35:42.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:42.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:35:42.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:42.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:42.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:35:42.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:42.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:42.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:35:42.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:43.551 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:43 smithi032 ceph-mon[98592]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-09-30T14:35:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:43 smithi156 ceph-mon[103579]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-09-30T14:35:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:43 smithi148 ceph-mon[102355]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-09-30T14:35:44.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:44.443+0000 7f17637fe700 1 -- 172.21.15.32:0/3998443783 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f176c061260 con 0x7f17580a0fa0 2023-09-30T14:35:44.445 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 9 on host 'smithi032' 2023-09-30T14:35:44.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:44.446+0000 7f1770d5f700 1 -- 172.21.15.32:0/3998443783 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f17580a0fa0 msgr2=0x7f17580a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:35:44.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:44.446+0000 7f1770d5f700 1 --2- 172.21.15.32:0/3998443783 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f17580a0fa0 0x7f17580a3460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f176c100e40 tx=0x7f175400a3b0 comp rx=0 tx=0).stop 2023-09-30T14:35:44.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:44.446+0000 7f1770d5f700 1 -- 172.21.15.32:0/3998443783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c101980 msgr2=0x7f176c19dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:35:44.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:44.446+0000 7f1770d5f700 1 --2- 172.21.15.32:0/3998443783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c101980 0x7f176c19dea0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f176400ada0 tx=0x7f176400c330 comp rx=0 tx=0).stop 2023-09-30T14:35:44.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:44.446+0000 7f1770d5f700 1 -- 172.21.15.32:0/3998443783 shutdown_connections 2023-09-30T14:35:44.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:44.446+0000 7f1770d5f700 1 --2- 172.21.15.32:0/3998443783 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f17580a0fa0 0x7f17580a3460 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:44.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:44.446+0000 7f1770d5f700 1 --2- 172.21.15.32:0/3998443783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f176c101980 0x7f176c19dea0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:44.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:44.446+0000 7f1770d5f700 1 --2- 172.21.15.32:0/3998443783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f176c0ffde0 0x7f176c069470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:44.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:44.446+0000 7f1770d5f700 1 --2- 172.21.15.32:0/3998443783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f176c100fe0 0x7f176c19b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:44.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:44.446+0000 7f1770d5f700 1 -- 172.21.15.32:0/3998443783 >> 172.21.15.32:0/3998443783 conn(0x7f176c0fb360 msgr2=0x7f176c0fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:35:44.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:44.446+0000 7f1770d5f700 1 -- 172.21.15.32:0/3998443783 shutdown_connections 2023-09-30T14:35:44.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:35:44.446+0000 7f1770d5f700 1 -- 172.21.15.32:0/3998443783 wait complete. 2023-09-30T14:35:44.885 DEBUG:teuthology.orchestra.run.smithi032:osd.9> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.9.service 2023-09-30T14:35:44.887 INFO:tasks.cephadm:Deploying osd.10 on smithi148 with /dev/vg_nvme/lv_1... 2023-09-30T14:35:44.888 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-30T14:35:44.918 INFO:journalctl@ceph.osd.9.smithi032.stdout:-- Logs begin at Sat 2023-09-30 14:15:36 UTC. -- 2023-09-30T14:35:45.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:45 smithi032 ceph-mon[98592]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-30T14:35:45.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:45 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:45.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:45 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:45.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:45 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:45.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:45 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:45.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:45 smithi156 ceph-mon[103579]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-30T14:35:45.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:45 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:45.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:45 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:45.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:45 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:45.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:45 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:45.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:45 smithi148 ceph-mon[102355]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-30T14:35:45.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:45 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:45.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:45 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:45.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:45 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:45.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:45 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:45.461 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.b/config 2023-09-30T14:35:45.637 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 14:35:45 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T14:35:45.357+0000 7f39f6fb6640 -1 osd.9 0 log_to_monitors true 2023-09-30T14:35:46.266 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:46 smithi032 ceph-mon[98592]: from='osd.9 [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-30T14:35:46.266 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:46 smithi032 ceph-mon[98592]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-30T14:35:46.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:46 smithi156 ceph-mon[103579]: from='osd.9 [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-30T14:35:46.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:46 smithi156 ceph-mon[103579]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-30T14:35:46.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:46 smithi148 ceph-mon[102355]: from='osd.9 [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-30T14:35:46.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:46 smithi148 ceph-mon[102355]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-30T14:35:46.951 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-30T14:35:46.951 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-30T14:35:46.952 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2023-09-30T14:35:46.952 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2023-09-30T14:35:46.952 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125156 s, 838 MB/s 2023-09-30T14:35:46.952 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2023-09-30T14:35:47.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:47 smithi156 ceph-mon[103579]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-30T14:35:47.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:47 smithi156 ceph-mon[103579]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-30T14:35:47.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:47 smithi156 ceph-mon[103579]: from='osd.9 [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-09-30T14:35:47.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:47 smithi156 ceph-mon[103579]: osdmap e64: 10 total, 9 up, 10 in 2023-09-30T14:35:47.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:47 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-30T14:35:47.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:47 smithi156 ceph-mon[103579]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-09-30T14:35:47.346 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:47 smithi148 ceph-mon[102355]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-30T14:35:47.346 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:47 smithi148 ceph-mon[102355]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-30T14:35:47.346 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:47 smithi148 ceph-mon[102355]: from='osd.9 [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-09-30T14:35:47.346 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:47 smithi148 ceph-mon[102355]: osdmap e64: 10 total, 9 up, 10 in 2023-09-30T14:35:47.346 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:47 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-30T14:35:47.346 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:47 smithi148 ceph-mon[102355]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-09-30T14:35:47.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:47 smithi032 ceph-mon[98592]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-30T14:35:47.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:47 smithi032 ceph-mon[98592]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-30T14:35:47.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:47 smithi032 ceph-mon[98592]: from='osd.9 [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-09-30T14:35:47.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:47 smithi032 ceph-mon[98592]: osdmap e64: 10 total, 9 up, 10 in 2023-09-30T14:35:47.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:47 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-30T14:35:47.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:47 smithi032 ceph-mon[98592]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2023-09-30T14:35:47.388 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 14:35:47 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T14:35:47.052+0000 7f39ed62f700 -1 osd.9 0 waiting for initial osdmap 2023-09-30T14:35:47.388 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 14:35:47 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T14:35:47.068+0000 7f39e7420700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T14:35:47.416 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi148:vg_nvme/lv_1 2023-09-30T14:35:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.b/config 2023-09-30T14:35:48.357 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-09-30T14:35:48.357 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: osdmap e65: 10 total, 9 up, 10 in 2023-09-30T14:35:48.358 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-30T14:35:48.358 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-30T14:35:48.358 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:48.358 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:48.358 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:48.358 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:48.358 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:48.358 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:48.358 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:48.358 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:48.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:48.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:48.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:35:48.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:48.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:48.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:48.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-09-30T14:35:48.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: osdmap e65: 10 total, 9 up, 10 in 2023-09-30T14:35:48.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-30T14:35:48.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-30T14:35:48.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:48.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:48.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:48.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:48.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:48.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:48.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:48.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:48.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:48.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:48.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:35:48.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:48.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:48.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:48.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2023-09-30T14:35:48.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: osdmap e65: 10 total, 9 up, 10 in 2023-09-30T14:35:48.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-30T14:35:48.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-30T14:35:48.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:48.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:48.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:48.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:48.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:48.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:35:48.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:48.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:48.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:48.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:48.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:35:48.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:48.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:35:48.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:35:49.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.178+0000 7f60822cb700 1 -- 172.21.15.148:0/4130055800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0ff560 msgr2=0x7f607c0ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:35:49.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.178+0000 7f60822cb700 1 --2- 172.21.15.148:0/4130055800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0ff560 0x7f607c0ff980 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f606c0099c0 tx=0x7f606c009cd0 comp rx=0 tx=0).stop 2023-09-30T14:35:49.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.178+0000 7f60822cb700 1 -- 172.21.15.148:0/4130055800 shutdown_connections 2023-09-30T14:35:49.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.178+0000 7f60822cb700 1 --2- 172.21.15.148:0/4130055800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f607c101100 0x7f607c1079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:49.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.178+0000 7f60822cb700 1 --2- 172.21.15.148:0/4130055800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0ff560 0x7f607c0ff980 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:49.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.178+0000 7f60822cb700 1 --2- 172.21.15.148:0/4130055800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f607c100760 0x7f607c100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:49.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.178+0000 7f60822cb700 1 -- 172.21.15.148:0/4130055800 >> 172.21.15.148:0/4130055800 conn(0x7f607c0faae0 msgr2=0x7f607c0fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:35:49.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.179+0000 7f60822cb700 1 -- 172.21.15.148:0/4130055800 shutdown_connections 2023-09-30T14:35:49.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.179+0000 7f60822cb700 1 -- 172.21.15.148:0/4130055800 wait complete. 2023-09-30T14:35:49.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.179+0000 7f60822cb700 1 Processor -- start 2023-09-30T14:35:49.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.179+0000 7f60822cb700 1 -- start start 2023-09-30T14:35:49.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f60822cb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0ff560 0x7f607c1034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:35:49.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f60822cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f607c100760 0x7f607c101b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:35:49.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f60822cb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f607c101100 0x7f607c102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:35:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f60822cb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c102800 con 0x7f607c100760 2023-09-30T14:35:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f60822cb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c102970 con 0x7f607c0ff560 2023-09-30T14:35:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f60822cb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c1a0c70 con 0x7f607c101100 2023-09-30T14:35:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f607bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0ff560 0x7f607c1034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:35:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f607bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0ff560 0x7f607c1034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51974/0 (socket says 172.21.15.148:51974) 2023-09-30T14:35:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f607bfff700 1 -- 172.21.15.148:0/4110882039 learned_addr learned my addr 172.21.15.148:0/4110882039 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-09-30T14:35:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f607b7fe700 1 --2- 172.21.15.148:0/4110882039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f607c100760 0x7f607c101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:35:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f6080868700 1 --2- 172.21.15.148:0/4110882039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f607c101100 0x7f607c102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:35:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f607bfff700 1 -- 172.21.15.148:0/4110882039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f607c101100 msgr2=0x7f607c102080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:35:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f607bfff700 1 --2- 172.21.15.148:0/4110882039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f607c101100 0x7f607c102080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f607bfff700 1 -- 172.21.15.148:0/4110882039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f607c100760 msgr2=0x7f607c101b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:35:49.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f607bfff700 1 --2- 172.21.15.148:0/4110882039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f607c100760 0x7f607c101b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:35:49.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f607bfff700 1 -- 172.21.15.148:0/4110882039 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c0095f0 con 0x7f607c0ff560 2023-09-30T14:35:49.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f607bfff700 1 --2- 172.21.15.148:0/4110882039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0ff560 0x7f607c1034f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f606c009990 tx=0x7f606c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:35:49.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.180+0000 7f60797fa700 1 -- 172.21.15.148:0/4110882039 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c01d070 con 0x7f607c0ff560 2023-09-30T14:35:49.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.181+0000 7f60822cb700 1 -- 172.21.15.148:0/4110882039 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607c1a0ef0 con 0x7f607c0ff560 2023-09-30T14:35:49.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.181+0000 7f60797fa700 1 -- 172.21.15.148:0/4110882039 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f606c004010 con 0x7f607c0ff560 2023-09-30T14:35:49.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.181+0000 7f60797fa700 1 -- 172.21.15.148:0/4110882039 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c022cd0 con 0x7f607c0ff560 2023-09-30T14:35:49.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.181+0000 7f60822cb700 1 -- 172.21.15.148:0/4110882039 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c1a12a0 con 0x7f607c0ff560 2023-09-30T14:35:49.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.182+0000 7f60822cb700 1 -- 172.21.15.148:0/4110882039 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607c10b5e0 con 0x7f607c0ff560 2023-09-30T14:35:49.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.183+0000 7f60797fa700 1 -- 172.21.15.148:0/4110882039 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f606c022470 con 0x7f607c0ff560 2023-09-30T14:35:49.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.183+0000 7f60797fa700 1 --2- 172.21.15.148:0/4110882039 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f60640a0fa0 0x7f60640a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:35:49.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.183+0000 7f60797fa700 1 -- 172.21.15.148:0/4110882039 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f606c0cd210 con 0x7f607c0ff560 2023-09-30T14:35:49.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.183+0000 7f607b7fe700 1 --2- 172.21.15.148:0/4110882039 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f60640a0fa0 0x7f60640a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:35:49.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.184+0000 7f607b7fe700 1 --2- 172.21.15.148:0/4110882039 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f60640a0fa0 0x7f60640a3460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6070005950 tx=0x7f607000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:35:49.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.188+0000 7f60797fa700 1 -- 172.21.15.148:0/4110882039 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f606c0972a0 con 0x7f607c0ff560 2023-09-30T14:35:49.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:49 smithi148 ceph-mon[102355]: purged_snaps scrub starts 2023-09-30T14:35:49.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:49 smithi148 ceph-mon[102355]: purged_snaps scrub ok 2023-09-30T14:35:49.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:49 smithi148 ceph-mon[102355]: Detected new or changed devices on smithi032 2023-09-30T14:35:49.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:49 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi032 to 4293M 2023-09-30T14:35:49.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:49 smithi148 ceph-mon[102355]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-30T14:35:49.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:49 smithi148 ceph-mon[102355]: osd.9 [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479] boot 2023-09-30T14:35:49.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:49 smithi148 ceph-mon[102355]: osdmap e66: 10 total, 10 up, 10 in 2023-09-30T14:35:49.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:49 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-30T14:35:49.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:35:49.384+0000 7f60822cb700 1 -- 172.21.15.148:0/4110882039 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f607c108a90 con 0x7f60640a0fa0 2023-09-30T14:35:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:49 smithi032 ceph-mon[98592]: purged_snaps scrub starts 2023-09-30T14:35:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:49 smithi032 ceph-mon[98592]: purged_snaps scrub ok 2023-09-30T14:35:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:49 smithi032 ceph-mon[98592]: Detected new or changed devices on smithi032 2023-09-30T14:35:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:49 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi032 to 4293M 2023-09-30T14:35:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:49 smithi032 ceph-mon[98592]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-30T14:35:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:49 smithi032 ceph-mon[98592]: osd.9 [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479] boot 2023-09-30T14:35:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:49 smithi032 ceph-mon[98592]: osdmap e66: 10 total, 10 up, 10 in 2023-09-30T14:35:49.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:49 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-30T14:35:49.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:49 smithi156 ceph-mon[103579]: purged_snaps scrub starts 2023-09-30T14:35:49.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:49 smithi156 ceph-mon[103579]: purged_snaps scrub ok 2023-09-30T14:35:49.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:49 smithi156 ceph-mon[103579]: Detected new or changed devices on smithi032 2023-09-30T14:35:49.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:49 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi032 to 4293M 2023-09-30T14:35:49.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:49 smithi156 ceph-mon[103579]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-30T14:35:49.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:49 smithi156 ceph-mon[103579]: osd.9 [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479] boot 2023-09-30T14:35:49.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:49 smithi156 ceph-mon[103579]: osdmap e66: 10 total, 10 up, 10 in 2023-09-30T14:35:49.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:49 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-30T14:35:50.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:50 smithi148 ceph-mon[102355]: osdmap e67: 10 total, 10 up, 10 in 2023-09-30T14:35:50.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:50 smithi148 ceph-mon[102355]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:35:50.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:50 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:35:50.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:50 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:35:50.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:50 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:50.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:50 smithi032 ceph-mon[98592]: osdmap e67: 10 total, 10 up, 10 in 2023-09-30T14:35:50.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:50 smithi032 ceph-mon[98592]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:35:50.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:50 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:35:50.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:50 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:35:50.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:50 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:50.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:50 smithi156 ceph-mon[103579]: osdmap e67: 10 total, 10 up, 10 in 2023-09-30T14:35:50.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:50 smithi156 ceph-mon[103579]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:35:50.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:50 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:35:50.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:50 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:35:50.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:50 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:51.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:51 smithi148 ceph-mon[102355]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:35:51.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:51 smithi032 ceph-mon[98592]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:35:51.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:51 smithi156 ceph-mon[103579]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:35:52.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:52 smithi148 ceph-mon[102355]: from='client.? 172.21.15.148:0/2448058942' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c7d56d1-96b3-4d0b-8d6e-dc4587579089"}]: dispatch 2023-09-30T14:35:52.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:52 smithi148 ceph-mon[102355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c7d56d1-96b3-4d0b-8d6e-dc4587579089"}]: dispatch 2023-09-30T14:35:52.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:52 smithi148 ceph-mon[102355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c7d56d1-96b3-4d0b-8d6e-dc4587579089"}]': finished 2023-09-30T14:35:52.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:52 smithi148 ceph-mon[102355]: osdmap e68: 11 total, 10 up, 11 in 2023-09-30T14:35:52.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:52 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-30T14:35:52.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:52 smithi032 ceph-mon[98592]: from='client.? 172.21.15.148:0/2448058942' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c7d56d1-96b3-4d0b-8d6e-dc4587579089"}]: dispatch 2023-09-30T14:35:52.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:52 smithi032 ceph-mon[98592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c7d56d1-96b3-4d0b-8d6e-dc4587579089"}]: dispatch 2023-09-30T14:35:52.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:52 smithi032 ceph-mon[98592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c7d56d1-96b3-4d0b-8d6e-dc4587579089"}]': finished 2023-09-30T14:35:52.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:52 smithi032 ceph-mon[98592]: osdmap e68: 11 total, 10 up, 11 in 2023-09-30T14:35:52.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:52 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-30T14:35:52.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:52 smithi156 ceph-mon[103579]: from='client.? 172.21.15.148:0/2448058942' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c7d56d1-96b3-4d0b-8d6e-dc4587579089"}]: dispatch 2023-09-30T14:35:52.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:52 smithi156 ceph-mon[103579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c7d56d1-96b3-4d0b-8d6e-dc4587579089"}]: dispatch 2023-09-30T14:35:52.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:52 smithi156 ceph-mon[103579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c7d56d1-96b3-4d0b-8d6e-dc4587579089"}]': finished 2023-09-30T14:35:52.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:52 smithi156 ceph-mon[103579]: osdmap e68: 11 total, 10 up, 11 in 2023-09-30T14:35:52.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:52 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-30T14:35:53.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:53 smithi148 ceph-mon[102355]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:35:53.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:53 smithi148 ceph-mon[102355]: from='client.? 172.21.15.148:0/3310396516' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:35:53.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:53 smithi032 ceph-mon[98592]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:35:53.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:53 smithi032 ceph-mon[98592]: from='client.? 172.21.15.148:0/3310396516' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:35:53.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:53 smithi156 ceph-mon[103579]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:35:53.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:53 smithi156 ceph-mon[103579]: from='client.? 172.21.15.148:0/3310396516' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:35:55.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:55 smithi148 ceph-mon[102355]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:35:55.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:55 smithi032 ceph-mon[98592]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:35:55.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:55 smithi156 ceph-mon[103579]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:35:57.306 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:57 smithi148 ceph-mon[102355]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:35:57.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:57 smithi032 ceph-mon[98592]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:35:57.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:57 smithi156 ceph-mon[103579]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:35:59.200 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:59 smithi148 ceph-mon[102355]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:35:59.200 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-30T14:35:59.200 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:59.200 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:35:59 smithi148 ceph-mon[102355]: Deploying daemon osd.10 on smithi148 2023-09-30T14:35:59.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:59 smithi032 ceph-mon[98592]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:35:59.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:59 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-30T14:35:59.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:59 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:59.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:35:59 smithi032 ceph-mon[98592]: Deploying daemon osd.10 on smithi148 2023-09-30T14:35:59.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:59 smithi156 ceph-mon[103579]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:35:59.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:59 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-30T14:35:59.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:59 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:35:59.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:35:59 smithi156 ceph-mon[103579]: Deploying daemon osd.10 on smithi148 2023-09-30T14:36:01.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:01 smithi148 ceph-mon[102355]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:01.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:01 smithi032 ceph-mon[98592]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:01.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:01 smithi156 ceph-mon[103579]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:03.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:03 smithi148 ceph-mon[102355]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:03.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:03 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:03.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:03 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:36:03.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:03 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:03.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:03 smithi032 ceph-mon[98592]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:03.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:03 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:03.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:03 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:36:03.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:03 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:03.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:03 smithi156 ceph-mon[103579]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:03.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:03 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:03.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:03 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:36:03.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:03 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:05.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:36:05.207+0000 7f60797fa700 1 -- 172.21.15.148:0/4110882039 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f607c108a90 con 0x7f60640a0fa0 2023-09-30T14:36:05.208 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 10 on host 'smithi148' 2023-09-30T14:36:05.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:36:05.209+0000 7f60822cb700 1 -- 172.21.15.148:0/4110882039 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f60640a0fa0 msgr2=0x7f60640a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:05.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:36:05.209+0000 7f60822cb700 1 --2- 172.21.15.148:0/4110882039 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f60640a0fa0 0x7f60640a3460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6070005950 tx=0x7f607000a300 comp rx=0 tx=0).stop 2023-09-30T14:36:05.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:36:05.209+0000 7f60822cb700 1 -- 172.21.15.148:0/4110882039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0ff560 msgr2=0x7f607c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:05.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:36:05.209+0000 7f60822cb700 1 --2- 172.21.15.148:0/4110882039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0ff560 0x7f607c1034f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f606c009990 tx=0x7f606c004730 comp rx=0 tx=0).stop 2023-09-30T14:36:05.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:36:05.209+0000 7f60822cb700 1 -- 172.21.15.148:0/4110882039 shutdown_connections 2023-09-30T14:36:05.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:36:05.209+0000 7f60822cb700 1 --2- 172.21.15.148:0/4110882039 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f60640a0fa0 0x7f60640a3460 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:05.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:36:05.209+0000 7f60822cb700 1 --2- 172.21.15.148:0/4110882039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f607c101100 0x7f607c102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:05.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:36:05.209+0000 7f60822cb700 1 --2- 172.21.15.148:0/4110882039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0ff560 0x7f607c1034f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:05.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:36:05.209+0000 7f60822cb700 1 --2- 172.21.15.148:0/4110882039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f607c100760 0x7f607c101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:05.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:36:05.209+0000 7f60822cb700 1 -- 172.21.15.148:0/4110882039 >> 172.21.15.148:0/4110882039 conn(0x7f607c0faae0 msgr2=0x7f607c108380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:05.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:36:05.209+0000 7f60822cb700 1 -- 172.21.15.148:0/4110882039 shutdown_connections 2023-09-30T14:36:05.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-09-30T14:36:05.209+0000 7f60822cb700 1 -- 172.21.15.148:0/4110882039 wait complete. 2023-09-30T14:36:05.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:05 smithi148 ceph-mon[102355]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:05.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:05 smithi032 ceph-mon[98592]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:05.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:05 smithi156 ceph-mon[103579]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:05.651 DEBUG:teuthology.orchestra.run.smithi148:osd.10> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.10.service 2023-09-30T14:36:05.653 INFO:tasks.cephadm:Deploying osd.11 on smithi156 with /dev/vg_nvme/lv_1... 2023-09-30T14:36:05.654 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-30T14:36:05.683 INFO:journalctl@ceph.osd.10.smithi148.stdout:-- Logs begin at Sat 2023-09-30 14:16:26 UTC. -- 2023-09-30T14:36:06.226 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.c/config 2023-09-30T14:36:06.421 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:06 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:06.421 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:06 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:06.421 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:06 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:06.421 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:06 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:06.519 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:06.519 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:06.519 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:06.520 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:06.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:06 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:06.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:06 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:06.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:06 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:06.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:06 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:06.788 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 14:36:06 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T14:36:06.517+0000 7fcbdeee1640 -1 osd.10 0 log_to_monitors true 2023-09-30T14:36:07.440 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:07 smithi148 ceph-mon[102355]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:07.440 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:07 smithi148 ceph-mon[102355]: from='osd.10 [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-30T14:36:07.440 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:07 smithi148 ceph-mon[102355]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-30T14:36:07.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:07 smithi156 ceph-mon[103579]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:07.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:07 smithi156 ceph-mon[103579]: from='osd.10 [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-30T14:36:07.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:07 smithi156 ceph-mon[103579]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-30T14:36:07.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:07 smithi032 ceph-mon[98592]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:07.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:07 smithi032 ceph-mon[98592]: from='osd.10 [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-30T14:36:07.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:07 smithi032 ceph-mon[98592]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-30T14:36:07.704 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-30T14:36:07.704 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-30T14:36:07.704 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-09-30T14:36:07.704 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-09-30T14:36:07.704 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00833145 s, 1.3 GB/s 2023-09-30T14:36:07.705 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-09-30T14:36:08.068 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi156:vg_nvme/lv_1 2023-09-30T14:36:08.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:08 smithi156 ceph-mon[103579]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-30T14:36:08.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:08 smithi156 ceph-mon[103579]: from='osd.10 [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:36:08.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:08 smithi156 ceph-mon[103579]: osdmap e69: 11 total, 10 up, 11 in 2023-09-30T14:36:08.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:08 smithi156 ceph-mon[103579]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:36:08.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:08 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-30T14:36:08.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:08 smithi148 ceph-mon[102355]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-30T14:36:08.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:08 smithi148 ceph-mon[102355]: from='osd.10 [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:36:08.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:08 smithi148 ceph-mon[102355]: osdmap e69: 11 total, 10 up, 11 in 2023-09-30T14:36:08.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:08 smithi148 ceph-mon[102355]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:36:08.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:08 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-30T14:36:08.610 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 14:36:08 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T14:36:08.279+0000 7fcbd3d57700 -1 osd.10 0 waiting for initial osdmap 2023-09-30T14:36:08.610 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 14:36:08 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T14:36:08.296+0000 7fcbcc345700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T14:36:08.627 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.c/config 2023-09-30T14:36:08.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:08 smithi032 ceph-mon[98592]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-30T14:36:08.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:08 smithi032 ceph-mon[98592]: from='osd.10 [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:36:08.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:08 smithi032 ceph-mon[98592]: osdmap e69: 11 total, 10 up, 11 in 2023-09-30T14:36:08.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:08 smithi032 ceph-mon[98592]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2023-09-30T14:36:08.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:08 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-30T14:36:09.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:09.406 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2023-09-30T14:36:09.406 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: osdmap e70: 11 total, 10 up, 11 in 2023-09-30T14:36:09.406 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-30T14:36:09.406 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-30T14:36:09.406 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:09.406 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:09.406 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:09.407 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:09.407 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:09.407 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:09.407 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:09.407 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:09.407 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:36:09.407 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:09.407 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:36:09.407 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:09.407 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:36:09.407 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:09 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:09.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:09.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2023-09-30T14:36:09.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: osdmap e70: 11 total, 10 up, 11 in 2023-09-30T14:36:09.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-30T14:36:09.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-30T14:36:09.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:09.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:09.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:09.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:09.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:09.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:09.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:09.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:09.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:36:09.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:09.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:36:09.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:09.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:36:09.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:09 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:09.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:09.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2023-09-30T14:36:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: osdmap e70: 11 total, 10 up, 11 in 2023-09-30T14:36:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-30T14:36:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-30T14:36:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:09.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:36:09.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:09.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:36:09.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:09.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:36:09.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:09 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:09.797 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.795+0000 7f497a9ee700 1 -- 172.21.15.156:0/3620020171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49740ffde0 msgr2=0x7f4974100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:09.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.795+0000 7f497a9ee700 1 --2- 172.21.15.156:0/3620020171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49740ffde0 0x7f4974100200 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4964009990 tx=0x7f4964009ca0 comp rx=0 tx=0).stop 2023-09-30T14:36:09.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.795+0000 7f497a9ee700 1 -- 172.21.15.156:0/3620020171 shutdown_connections 2023-09-30T14:36:09.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.795+0000 7f497a9ee700 1 --2- 172.21.15.156:0/3620020171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4974101980 0x7f4974108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:09.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.795+0000 7f497a9ee700 1 --2- 172.21.15.156:0/3620020171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49740ffde0 0x7f4974100200 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:09.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.795+0000 7f497a9ee700 1 --2- 172.21.15.156:0/3620020171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4974100fe0 0x7f4974101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:09.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.795+0000 7f497a9ee700 1 -- 172.21.15.156:0/3620020171 >> 172.21.15.156:0/3620020171 conn(0x7f49740fb360 msgr2=0x7f49740fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:09.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.796+0000 7f497a9ee700 1 -- 172.21.15.156:0/3620020171 shutdown_connections 2023-09-30T14:36:09.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.796+0000 7f497a9ee700 1 -- 172.21.15.156:0/3620020171 wait complete. 2023-09-30T14:36:09.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.796+0000 7f497a9ee700 1 Processor -- start 2023-09-30T14:36:09.798 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.796+0000 7f497a9ee700 1 -- start start 2023-09-30T14:36:09.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.796+0000 7f497a9ee700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49740ffde0 0x7f497410a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:09.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.796+0000 7f497a9ee700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4974100fe0 0x7f4974108d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:09.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.796+0000 7f497a9ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4974101980 0x7f4974109280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:09.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.796+0000 7f497a9ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4974109a00 con 0x7f4974101980 2023-09-30T14:36:09.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.796+0000 7f497a9ee700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4974109b70 con 0x7f49740ffde0 2023-09-30T14:36:09.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.796+0000 7f497a9ee700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49741a0c50 con 0x7f4974100fe0 2023-09-30T14:36:09.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.797+0000 7f49737fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4974100fe0 0x7f4974108d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:09.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.797+0000 7f49737fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4974100fe0 0x7f4974108d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36362/0 (socket says 172.21.15.156:36362) 2023-09-30T14:36:09.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.797+0000 7f49737fe700 1 -- 172.21.15.156:0/2752422735 learned_addr learned my addr 172.21.15.156:0/2752422735 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-30T14:36:09.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.797+0000 7f4973fff700 1 --2- 172.21.15.156:0/2752422735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49740ffde0 0x7f497410a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:09.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.797+0000 7f4978f8b700 1 --2- 172.21.15.156:0/2752422735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4974101980 0x7f4974109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:09.799 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.797+0000 7f49737fe700 1 -- 172.21.15.156:0/2752422735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49740ffde0 msgr2=0x7f497410a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:09.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.797+0000 7f49737fe700 1 --2- 172.21.15.156:0/2752422735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49740ffde0 0x7f497410a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:09.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.797+0000 7f49737fe700 1 -- 172.21.15.156:0/2752422735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4974101980 msgr2=0x7f4974109280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:09.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.797+0000 7f49737fe700 1 --2- 172.21.15.156:0/2752422735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4974101980 0x7f4974109280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:09.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.797+0000 7f49737fe700 1 -- 172.21.15.156:0/2752422735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49640095f0 con 0x7f4974100fe0 2023-09-30T14:36:09.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.797+0000 7f49737fe700 1 --2- 172.21.15.156:0/2752422735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4974100fe0 0x7f4974108d40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f496800e9a0 tx=0x7f496800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:09.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.797+0000 7f49717fa700 1 -- 172.21.15.156:0/2752422735 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968009790 con 0x7f4974100fe0 2023-09-30T14:36:09.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.797+0000 7f497a9ee700 1 -- 172.21.15.156:0/2752422735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49741a0f30 con 0x7f4974100fe0 2023-09-30T14:36:09.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.797+0000 7f49717fa700 1 -- 172.21.15.156:0/2752422735 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4968010430 con 0x7f4974100fe0 2023-09-30T14:36:09.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.797+0000 7f49717fa700 1 -- 172.21.15.156:0/2752422735 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968018680 con 0x7f4974100fe0 2023-09-30T14:36:09.800 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.797+0000 7f497a9ee700 1 -- 172.21.15.156:0/2752422735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49741a13a0 con 0x7f4974100fe0 2023-09-30T14:36:09.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.798+0000 7f497a9ee700 1 -- 172.21.15.156:0/2752422735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4974107450 con 0x7f4974100fe0 2023-09-30T14:36:09.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.799+0000 7f49717fa700 1 -- 172.21.15.156:0/2752422735 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f49680105a0 con 0x7f4974100fe0 2023-09-30T14:36:09.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.800+0000 7f49717fa700 1 --2- 172.21.15.156:0/2752422735 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f495c0a0fa0 0x7f495c0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:09.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.800+0000 7f49717fa700 1 -- 172.21.15.156:0/2752422735 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6727+0+0 (secure 0 0 0) 0x7f4968014070 con 0x7f4974100fe0 2023-09-30T14:36:09.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.800+0000 7f4973fff700 1 --2- 172.21.15.156:0/2752422735 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f495c0a0fa0 0x7f495c0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:09.802 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.801+0000 7f4973fff700 1 --2- 172.21.15.156:0/2752422735 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f495c0a0fa0 0x7f495c0a3460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f496400b2e0 tx=0x7f4964005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:09.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:09.803+0000 7f49717fa700 1 -- 172.21.15.156:0/2752422735 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f49680962a0 con 0x7f4974100fe0 2023-09-30T14:36:10.000 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:10.000+0000 7f497a9ee700 1 -- 172.21.15.156:0/2752422735 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f497402d1b0 con 0x7f495c0a0fa0 2023-09-30T14:36:10.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:10 smithi156 ceph-mon[103579]: purged_snaps scrub starts 2023-09-30T14:36:10.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:10 smithi156 ceph-mon[103579]: purged_snaps scrub ok 2023-09-30T14:36:10.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:10 smithi156 ceph-mon[103579]: Detected new or changed devices on smithi148 2023-09-30T14:36:10.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:10 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi148 to 4292M 2023-09-30T14:36:10.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:10 smithi156 ceph-mon[103579]: osd.10 [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673] boot 2023-09-30T14:36:10.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:10 smithi156 ceph-mon[103579]: osdmap e71: 11 total, 11 up, 11 in 2023-09-30T14:36:10.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:10 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-30T14:36:10.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:10 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:36:10.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:10 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:36:10.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:10 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:10.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:10 smithi148 ceph-mon[102355]: purged_snaps scrub starts 2023-09-30T14:36:10.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:10 smithi148 ceph-mon[102355]: purged_snaps scrub ok 2023-09-30T14:36:10.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:10 smithi148 ceph-mon[102355]: Detected new or changed devices on smithi148 2023-09-30T14:36:10.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:10 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi148 to 4292M 2023-09-30T14:36:10.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:10 smithi148 ceph-mon[102355]: osd.10 [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673] boot 2023-09-30T14:36:10.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:10 smithi148 ceph-mon[102355]: osdmap e71: 11 total, 11 up, 11 in 2023-09-30T14:36:10.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:10 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-30T14:36:10.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:10 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:36:10.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:10 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:36:10.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:10 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:10.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:10 smithi032 ceph-mon[98592]: purged_snaps scrub starts 2023-09-30T14:36:10.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:10 smithi032 ceph-mon[98592]: purged_snaps scrub ok 2023-09-30T14:36:10.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:10 smithi032 ceph-mon[98592]: Detected new or changed devices on smithi148 2023-09-30T14:36:10.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:10 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi148 to 4292M 2023-09-30T14:36:10.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:10 smithi032 ceph-mon[98592]: osd.10 [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673] boot 2023-09-30T14:36:10.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:10 smithi032 ceph-mon[98592]: osdmap e71: 11 total, 11 up, 11 in 2023-09-30T14:36:10.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:10 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-30T14:36:10.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:10 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-30T14:36:10.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:10 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-30T14:36:10.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:10 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:11.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:11 smithi156 ceph-mon[103579]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:11.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:11 smithi156 ceph-mon[103579]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:36:11.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:11 smithi156 ceph-mon[103579]: osdmap e72: 11 total, 11 up, 11 in 2023-09-30T14:36:11.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:11 smithi148 ceph-mon[102355]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:11.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:11 smithi148 ceph-mon[102355]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:36:11.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:11 smithi148 ceph-mon[102355]: osdmap e72: 11 total, 11 up, 11 in 2023-09-30T14:36:11.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:11 smithi032 ceph-mon[98592]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-30T14:36:11.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:11 smithi032 ceph-mon[98592]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:36:11.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:11 smithi032 ceph-mon[98592]: osdmap e72: 11 total, 11 up, 11 in 2023-09-30T14:36:12.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:12 smithi156 ceph-mon[103579]: osdmap e73: 11 total, 11 up, 11 in 2023-09-30T14:36:12.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:12 smithi156 ceph-mon[103579]: osdmap e74: 11 total, 11 up, 11 in 2023-09-30T14:36:12.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:12 smithi148 ceph-mon[102355]: osdmap e73: 11 total, 11 up, 11 in 2023-09-30T14:36:12.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:12 smithi148 ceph-mon[102355]: osdmap e74: 11 total, 11 up, 11 in 2023-09-30T14:36:12.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:12 smithi032 ceph-mon[98592]: osdmap e73: 11 total, 11 up, 11 in 2023-09-30T14:36:12.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:12 smithi032 ceph-mon[98592]: osdmap e74: 11 total, 11 up, 11 in 2023-09-30T14:36:13.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:13 smithi156 ceph-mon[103579]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-30T14:36:13.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:13 smithi156 ceph-mon[103579]: from='client.? 172.21.15.156:0/1374090486' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2d6db2a-bd3d-4ed8-95fa-fe8918457114"}]: dispatch 2023-09-30T14:36:13.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:13 smithi156 ceph-mon[103579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2d6db2a-bd3d-4ed8-95fa-fe8918457114"}]: dispatch 2023-09-30T14:36:13.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:13 smithi156 ceph-mon[103579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2d6db2a-bd3d-4ed8-95fa-fe8918457114"}]': finished 2023-09-30T14:36:13.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:13 smithi156 ceph-mon[103579]: osdmap e75: 12 total, 11 up, 12 in 2023-09-30T14:36:13.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:13.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:13 smithi156 ceph-mon[103579]: from='client.? 172.21.15.156:0/1876153255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:36:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:13 smithi148 ceph-mon[102355]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-30T14:36:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:13 smithi148 ceph-mon[102355]: from='client.? 172.21.15.156:0/1374090486' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2d6db2a-bd3d-4ed8-95fa-fe8918457114"}]: dispatch 2023-09-30T14:36:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:13 smithi148 ceph-mon[102355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2d6db2a-bd3d-4ed8-95fa-fe8918457114"}]: dispatch 2023-09-30T14:36:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:13 smithi148 ceph-mon[102355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2d6db2a-bd3d-4ed8-95fa-fe8918457114"}]': finished 2023-09-30T14:36:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:13 smithi148 ceph-mon[102355]: osdmap e75: 12 total, 11 up, 12 in 2023-09-30T14:36:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:13.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:13 smithi148 ceph-mon[102355]: from='client.? 172.21.15.156:0/1876153255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:36:13.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:13 smithi032 ceph-mon[98592]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-30T14:36:13.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:13 smithi032 ceph-mon[98592]: from='client.? 172.21.15.156:0/1374090486' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2d6db2a-bd3d-4ed8-95fa-fe8918457114"}]: dispatch 2023-09-30T14:36:13.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:13 smithi032 ceph-mon[98592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2d6db2a-bd3d-4ed8-95fa-fe8918457114"}]: dispatch 2023-09-30T14:36:13.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:13 smithi032 ceph-mon[98592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2d6db2a-bd3d-4ed8-95fa-fe8918457114"}]': finished 2023-09-30T14:36:13.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:13 smithi032 ceph-mon[98592]: osdmap e75: 12 total, 11 up, 12 in 2023-09-30T14:36:13.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:13.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:13 smithi032 ceph-mon[98592]: from='client.? 172.21.15.156:0/1876153255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-30T14:36:15.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:15 smithi148 ceph-mon[102355]: pgmap v216: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-30T14:36:15.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:15 smithi032 ceph-mon[98592]: pgmap v216: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-30T14:36:15.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:15 smithi156 ceph-mon[103579]: pgmap v216: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-30T14:36:17.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:17 smithi156 ceph-mon[103579]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-09-30T14:36:17.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:17 smithi148 ceph-mon[102355]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-09-30T14:36:17.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:17 smithi032 ceph-mon[98592]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-09-30T14:36:19.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:19 smithi148 ceph-mon[102355]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-09-30T14:36:19.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:19 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-30T14:36:19.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:19 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:19.616 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:19 smithi156 ceph-mon[103579]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-09-30T14:36:19.616 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:19 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-30T14:36:19.616 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:19 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:19.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:19 smithi032 ceph-mon[98592]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-09-30T14:36:19.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:19 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-30T14:36:19.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:19 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:20.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:20 smithi156 ceph-mon[103579]: Deploying daemon osd.11 on smithi156 2023-09-30T14:36:20.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:20 smithi148 ceph-mon[102355]: Deploying daemon osd.11 on smithi156 2023-09-30T14:36:20.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:20 smithi032 ceph-mon[98592]: Deploying daemon osd.11 on smithi156 2023-09-30T14:36:21.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:21 smithi156 ceph-mon[103579]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-30T14:36:21.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:21 smithi148 ceph-mon[102355]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-30T14:36:21.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:21 smithi032 ceph-mon[98592]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-30T14:36:23.548 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:23 smithi156 ceph-mon[103579]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-09-30T14:36:23.548 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:23.548 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:23.548 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:36:23.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:23 smithi148 ceph-mon[102355]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-09-30T14:36:23.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:23.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:23.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:36:23.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:23 smithi032 ceph-mon[98592]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-09-30T14:36:23.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:23.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:23.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:36:25.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:25 smithi148 ceph-mon[102355]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-09-30T14:36:25.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:25 smithi032 ceph-mon[98592]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-09-30T14:36:25.765 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:25.764+0000 7f49717fa700 1 -- 172.21.15.156:0/2752422735 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f497402d1b0 con 0x7f495c0a0fa0 2023-09-30T14:36:25.765 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 11 on host 'smithi156' 2023-09-30T14:36:25.766 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:25 smithi156 ceph-mon[103579]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-09-30T14:36:25.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:25.767+0000 7f497a9ee700 1 -- 172.21.15.156:0/2752422735 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f495c0a0fa0 msgr2=0x7f495c0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:25.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:25.767+0000 7f497a9ee700 1 --2- 172.21.15.156:0/2752422735 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f495c0a0fa0 0x7f495c0a3460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f496400b2e0 tx=0x7f4964005fb0 comp rx=0 tx=0).stop 2023-09-30T14:36:25.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:25.767+0000 7f497a9ee700 1 -- 172.21.15.156:0/2752422735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4974100fe0 msgr2=0x7f4974108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:25.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:25.767+0000 7f497a9ee700 1 --2- 172.21.15.156:0/2752422735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4974100fe0 0x7f4974108d40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f496800e9a0 tx=0x7f496800ecb0 comp rx=0 tx=0).stop 2023-09-30T14:36:25.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:25.767+0000 7f497a9ee700 1 -- 172.21.15.156:0/2752422735 shutdown_connections 2023-09-30T14:36:25.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:25.767+0000 7f497a9ee700 1 --2- 172.21.15.156:0/2752422735 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f495c0a0fa0 0x7f495c0a3460 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:25.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:25.767+0000 7f497a9ee700 1 --2- 172.21.15.156:0/2752422735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4974101980 0x7f4974109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:25.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:25.767+0000 7f497a9ee700 1 --2- 172.21.15.156:0/2752422735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49740ffde0 0x7f497410a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:25.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:25.767+0000 7f497a9ee700 1 --2- 172.21.15.156:0/2752422735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4974100fe0 0x7f4974108d40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:25.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:25.767+0000 7f497a9ee700 1 -- 172.21.15.156:0/2752422735 >> 172.21.15.156:0/2752422735 conn(0x7f49740fb360 msgr2=0x7f4974104590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:25.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:25.767+0000 7f497a9ee700 1 -- 172.21.15.156:0/2752422735 shutdown_connections 2023-09-30T14:36:25.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:25.767+0000 7f497a9ee700 1 -- 172.21.15.156:0/2752422735 wait complete. 2023-09-30T14:36:26.217 DEBUG:teuthology.orchestra.run.smithi156:osd.11> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.11.service 2023-09-30T14:36:26.219 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-09-30T14:36:26.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-09-30T14:36:26.249 INFO:journalctl@ceph.osd.11.smithi156.stdout:-- Logs begin at Sat 2023-09-30 14:16:23 UTC. -- 2023-09-30T14:36:26.803 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:36:27.036 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:27.036 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:26 smithi156 ceph-mon[103579]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-30T14:36:27.037 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:27.037 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:27.037 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:27.043 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:27.043 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:26 smithi032 ceph-mon[98592]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-30T14:36:27.043 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:27.043 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:27.043 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:27.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:27.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:26 smithi148 ceph-mon[102355]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-30T14:36:27.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:27.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:27.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:27.305 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 14:36:27 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T14:36:27.088+0000 7fb98fd40640 -1 osd.11 0 log_to_monitors true 2023-09-30T14:36:28.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.133+0000 7f7f5fb0e700 1 -- 172.21.15.32:0/556322129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f58100030 msgr2=0x7f7f581004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:28.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.133+0000 7f7f5fb0e700 1 --2- 172.21.15.32:0/556322129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f58100030 0x7f7f581004b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009990 tx=0x7f7f54009ca0 comp rx=0 tx=0).stop 2023-09-30T14:36:28.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.133+0000 7f7f5fb0e700 1 -- 172.21.15.32:0/556322129 shutdown_connections 2023-09-30T14:36:28.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.133+0000 7f7f5fb0e700 1 --2- 172.21.15.32:0/556322129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f581009f0 0x7f7f58108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:28.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.133+0000 7f7f5fb0e700 1 --2- 172.21.15.32:0/556322129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f58101460 0x7f7f58101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.133+0000 7f7f5fb0e700 1 --2- 172.21.15.32:0/556322129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f58100030 0x7f7f581004b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.133+0000 7f7f5fb0e700 1 -- 172.21.15.32:0/556322129 >> 172.21.15.32:0/556322129 conn(0x7f7f580faae0 msgr2=0x7f7f580fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.133+0000 7f7f5fb0e700 1 -- 172.21.15.32:0/556322129 shutdown_connections 2023-09-30T14:36:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.133+0000 7f7f5fb0e700 1 -- 172.21.15.32:0/556322129 wait complete. 2023-09-30T14:36:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.134+0000 7f7f5fb0e700 1 Processor -- start 2023-09-30T14:36:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.134+0000 7f7f5fb0e700 1 -- start start 2023-09-30T14:36:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.134+0000 7f7f5fb0e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f58100030 0x7f7f58196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.134+0000 7f7f5fb0e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f581009f0 0x7f7f581970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:28.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.134+0000 7f7f5fb0e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f58101460 0x7f7f5819e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.134+0000 7f7f5fb0e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5819e6f0 con 0x7f7f58100030 2023-09-30T14:36:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.134+0000 7f7f5fb0e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5819e830 con 0x7f7f58101460 2023-09-30T14:36:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.134+0000 7f7f5fb0e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5819e9c0 con 0x7f7f581009f0 2023-09-30T14:36:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.134+0000 7f7f5d8aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f58100030 0x7f7f58196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.134+0000 7f7f5d8aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f58100030 0x7f7f58196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33368/0 (socket says 172.21.15.32:33368) 2023-09-30T14:36:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.134+0000 7f7f5d8aa700 1 -- 172.21.15.32:0/2133825458 learned_addr learned my addr 172.21.15.32:0/2133825458 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:36:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.134+0000 7f7f5d0a9700 1 --2- 172.21.15.32:0/2133825458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f581009f0 0x7f7f581970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.135+0000 7f7f5d8aa700 1 -- 172.21.15.32:0/2133825458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f581009f0 msgr2=0x7f7f581970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.135+0000 7f7f5d8aa700 1 --2- 172.21.15.32:0/2133825458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f581009f0 0x7f7f581970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:28.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.135+0000 7f7f5d8aa700 1 -- 172.21.15.32:0/2133825458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f58101460 msgr2=0x7f7f5819e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.135+0000 7f7f5d8aa700 1 --2- 172.21.15.32:0/2133825458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f58101460 0x7f7f5819e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.135+0000 7f7f5d8aa700 1 -- 172.21.15.32:0/2133825458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f540095f0 con 0x7f7f58100030 2023-09-30T14:36:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.135+0000 7f7f5d8aa700 1 --2- 172.21.15.32:0/2133825458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f58100030 0x7f7f58196b60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00e9d0 tx=0x7f7f4c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.135+0000 7f7f4affd700 1 -- 172.21.15.32:0/2133825458 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c009790 con 0x7f7f58100030 2023-09-30T14:36:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.135+0000 7f7f4affd700 1 -- 172.21.15.32:0/2133825458 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f4c00cbd0 con 0x7f7f58100030 2023-09-30T14:36:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.135+0000 7f7f5fb0e700 1 -- 172.21.15.32:0/2133825458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5819eca0 con 0x7f7f58100030 2023-09-30T14:36:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.135+0000 7f7f4affd700 1 -- 172.21.15.32:0/2133825458 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c018680 con 0x7f7f58100030 2023-09-30T14:36:28.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.135+0000 7f7f5fb0e700 1 -- 172.21.15.32:0/2133825458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5819f1a0 con 0x7f7f58100030 2023-09-30T14:36:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.136+0000 7f7f5fb0e700 1 -- 172.21.15.32:0/2133825458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5810b5e0 con 0x7f7f58100030 2023-09-30T14:36:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.137+0000 7f7f4affd700 1 -- 172.21.15.32:0/2133825458 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f7f4c00cd40 con 0x7f7f58100030 2023-09-30T14:36:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.138+0000 7f7f4affd700 1 --2- 172.21.15.32:0/2133825458 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7f440a1020 0x7f7f440a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.138+0000 7f7f4affd700 1 -- 172.21.15.32:0/2133825458 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f7f4c014070 con 0x7f7f58100030 2023-09-30T14:36:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.138+0000 7f7f5d0a9700 1 --2- 172.21.15.32:0/2133825458 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7f440a1020 0x7f7f440a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:28.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.138+0000 7f7f5d0a9700 1 --2- 172.21.15.32:0/2133825458 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7f440a1020 0x7f7f440a34e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400b2e0 tx=0x7f7f54005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:28.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.141+0000 7f7f4affd700 1 -- 172.21.15.32:0/2133825458 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f7f4c0963e0 con 0x7f7f58100030 2023-09-30T14:36:28.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:27 smithi156 ceph-mon[103579]: from='osd.11 [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-30T14:36:28.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:27 smithi156 ceph-mon[103579]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-30T14:36:28.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.341+0000 7f7f5fb0e700 1 -- 172.21.15.32:0/2133825458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f7f5819f330 con 0x7f7f58100030 2023-09-30T14:36:28.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.342+0000 7f7f4affd700 1 -- 172.21.15.32:0/2133825458 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7f7f4c0260a0 con 0x7f7f58100030 2023-09-30T14:36:28.342 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:36:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.344+0000 7f7f5fb0e700 1 -- 172.21.15.32:0/2133825458 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7f440a1020 msgr2=0x7f7f440a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.344+0000 7f7f5fb0e700 1 --2- 172.21.15.32:0/2133825458 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7f440a1020 0x7f7f440a34e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400b2e0 tx=0x7f7f54005fb0 comp rx=0 tx=0).stop 2023-09-30T14:36:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.344+0000 7f7f5fb0e700 1 -- 172.21.15.32:0/2133825458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f58100030 msgr2=0x7f7f58196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.344+0000 7f7f5fb0e700 1 --2- 172.21.15.32:0/2133825458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f58100030 0x7f7f58196b60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00e9d0 tx=0x7f7f4c00ece0 comp rx=0 tx=0).stop 2023-09-30T14:36:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.344+0000 7f7f5fb0e700 1 -- 172.21.15.32:0/2133825458 shutdown_connections 2023-09-30T14:36:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.344+0000 7f7f5fb0e700 1 --2- 172.21.15.32:0/2133825458 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7f440a1020 0x7f7f440a34e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.344+0000 7f7f5fb0e700 1 --2- 172.21.15.32:0/2133825458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f58101460 0x7f7f5819e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.344+0000 7f7f5fb0e700 1 --2- 172.21.15.32:0/2133825458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f58100030 0x7f7f58196b60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:28.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.344+0000 7f7f5fb0e700 1 --2- 172.21.15.32:0/2133825458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f581009f0 0x7f7f581970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.344+0000 7f7f5fb0e700 1 -- 172.21.15.32:0/2133825458 >> 172.21.15.32:0/2133825458 conn(0x7f7f580faae0 msgr2=0x7f7f580fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.344+0000 7f7f5fb0e700 1 -- 172.21.15.32:0/2133825458 shutdown_connections 2023-09-30T14:36:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:28.344+0000 7f7f5fb0e700 1 -- 172.21.15.32:0/2133825458 wait complete. 2023-09-30T14:36:28.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:27 smithi148 ceph-mon[102355]: from='osd.11 [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-30T14:36:28.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:27 smithi148 ceph-mon[102355]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-30T14:36:28.364 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:27 smithi032 ceph-mon[98592]: from='osd.11 [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-30T14:36:28.364 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:27 smithi032 ceph-mon[98592]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-30T14:36:28.763 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1696084569,"num_in_osds":12,"osd_in_since":1696084572,"num_remapped_pgs":0} 2023-09-30T14:36:29.055 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T14:36:28.970+0000 7fb984bb6700 -1 osd.11 0 waiting for initial osdmap 2023-09-30T14:36:29.055 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T14:36:28.985+0000 7fb9811ac700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T14:36:29.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-30T14:36:29.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-30T14:36:29.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: from='osd.11 [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:36:29.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: osdmap e76: 12 total, 11 up, 12 in 2023-09-30T14:36:29.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:29.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:36:29.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2133825458' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-30T14:36:29.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:29.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:29.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:29.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:29.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:29.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:29.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:29.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:29.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:36:29.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:29.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-30T14:36:29.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-30T14:36:29.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: from='osd.11 [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:36:29.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: osdmap e76: 12 total, 11 up, 12 in 2023-09-30T14:36:29.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:29.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:36:29.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2133825458' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-30T14:36:29.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:29.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:29.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:29.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:29.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:29.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:29.140 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:29.140 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:29.140 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:36:29.140 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:29.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-30T14:36:29.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-30T14:36:29.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: from='osd.11 [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:36:29.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: osdmap e76: 12 total, 11 up, 12 in 2023-09-30T14:36:29.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:29.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-09-30T14:36:29.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2133825458' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-30T14:36:29.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:29.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:29.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:29.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:29.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:29.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:36:29.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:29.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:29.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:36:29.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:29.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-09-30T14:36:30.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:29 smithi032 ceph-mon[98592]: Detected new or changed devices on smithi156 2023-09-30T14:36:30.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:29 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi156 to 4292M 2023-09-30T14:36:30.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:29 smithi032 ceph-mon[98592]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-30T14:36:30.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:29 smithi032 ceph-mon[98592]: osdmap e77: 12 total, 11 up, 12 in 2023-09-30T14:36:30.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:29 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:30.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:29 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:30.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:29 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:36:30.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:29 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:30.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:29 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:36:30.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:29 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:30.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:29 smithi156 ceph-mon[103579]: Detected new or changed devices on smithi156 2023-09-30T14:36:30.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:29 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi156 to 4292M 2023-09-30T14:36:30.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:29 smithi156 ceph-mon[103579]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-30T14:36:30.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:29 smithi156 ceph-mon[103579]: osdmap e77: 12 total, 11 up, 12 in 2023-09-30T14:36:30.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:29 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:30.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:29 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:30.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:29 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:36:30.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:29 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:30.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:29 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:36:30.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:29 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:36:30.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:29 smithi148 ceph-mon[102355]: Detected new or changed devices on smithi156 2023-09-30T14:36:30.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:29 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi156 to 4292M 2023-09-30T14:36:30.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:29 smithi148 ceph-mon[102355]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-09-30T14:36:30.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:29 smithi148 ceph-mon[102355]: osdmap e77: 12 total, 11 up, 12 in 2023-09-30T14:36:30.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:29 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:30.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:29 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:30.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:29 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:36:30.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:29 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:30.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:29 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:36:30.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:29 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:31.199 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:30 smithi032 ceph-mon[98592]: purged_snaps scrub starts 2023-09-30T14:36:31.199 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:30 smithi032 ceph-mon[98592]: purged_snaps scrub ok 2023-09-30T14:36:31.199 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:30 smithi032 ceph-mon[98592]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-30T14:36:31.199 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:30 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:31.199 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:30 smithi032 ceph-mon[98592]: osd.11 [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507] boot 2023-09-30T14:36:31.199 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:30 smithi032 ceph-mon[98592]: osdmap e78: 12 total, 12 up, 12 in 2023-09-30T14:36:31.199 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:30 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:31.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:30 smithi156 ceph-mon[103579]: purged_snaps scrub starts 2023-09-30T14:36:31.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:30 smithi156 ceph-mon[103579]: purged_snaps scrub ok 2023-09-30T14:36:31.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:30 smithi156 ceph-mon[103579]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-30T14:36:31.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:30 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:31.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:30 smithi156 ceph-mon[103579]: osd.11 [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507] boot 2023-09-30T14:36:31.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:30 smithi156 ceph-mon[103579]: osdmap e78: 12 total, 12 up, 12 in 2023-09-30T14:36:31.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:30 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:31.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:30 smithi148 ceph-mon[102355]: purged_snaps scrub starts 2023-09-30T14:36:31.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:30 smithi148 ceph-mon[102355]: purged_snaps scrub ok 2023-09-30T14:36:31.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:30 smithi148 ceph-mon[102355]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-30T14:36:31.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:30 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:31.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:30 smithi148 ceph-mon[102355]: osd.11 [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507] boot 2023-09-30T14:36:31.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:30 smithi148 ceph-mon[102355]: osdmap e78: 12 total, 12 up, 12 in 2023-09-30T14:36:31.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:30 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-30T14:36:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.624+0000 7f3eec115700 1 -- 172.21.15.32:0/233720229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee4101460 msgr2=0x7f3ee4101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.624+0000 7f3eec115700 1 --2- 172.21.15.32:0/233720229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee4101460 0x7f3ee4101880 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009990 tx=0x7f3ee0009ca0 comp rx=0 tx=0).stop 2023-09-30T14:36:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.624+0000 7f3eec115700 1 -- 172.21.15.32:0/233720229 shutdown_connections 2023-09-30T14:36:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.624+0000 7f3eec115700 1 --2- 172.21.15.32:0/233720229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee41009f0 0x7f3ee4108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.624+0000 7f3eec115700 1 --2- 172.21.15.32:0/233720229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee4101460 0x7f3ee4101880 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.624+0000 7f3eec115700 1 --2- 172.21.15.32:0/233720229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ee4100030 0x7f3ee41004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.624+0000 7f3eec115700 1 -- 172.21.15.32:0/233720229 >> 172.21.15.32:0/233720229 conn(0x7f3ee40faae0 msgr2=0x7f3ee40fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.625+0000 7f3eec115700 1 -- 172.21.15.32:0/233720229 shutdown_connections 2023-09-30T14:36:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.625+0000 7f3eec115700 1 -- 172.21.15.32:0/233720229 wait complete. 2023-09-30T14:36:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.625+0000 7f3eec115700 1 Processor -- start 2023-09-30T14:36:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.625+0000 7f3eec115700 1 -- start start 2023-09-30T14:36:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.625+0000 7f3eec115700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee4100030 0x7f3ee4198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:31.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.625+0000 7f3eec115700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee41009f0 0x7f3ee41993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.625+0000 7f3eec115700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ee4101460 0x7f3ee41a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.625+0000 7f3eec115700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee41a0a30 con 0x7f3ee4100030 2023-09-30T14:36:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.625+0000 7f3eec115700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee41a0b70 con 0x7f3ee41009f0 2023-09-30T14:36:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.625+0000 7f3eec115700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee41a0d00 con 0x7f3ee4101460 2023-09-30T14:36:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.625+0000 7f3ee9eb1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee4100030 0x7f3ee4198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.626+0000 7f3ee9eb1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee4100030 0x7f3ee4198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33384/0 (socket says 172.21.15.32:33384) 2023-09-30T14:36:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.626+0000 7f3ee9eb1700 1 -- 172.21.15.32:0/1149045785 learned_addr learned my addr 172.21.15.32:0/1149045785 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:36:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.626+0000 7f3ee96b0700 1 --2- 172.21.15.32:0/1149045785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee41009f0 0x7f3ee41993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.626+0000 7f3eea6b2700 1 --2- 172.21.15.32:0/1149045785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ee4101460 0x7f3ee41a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.626+0000 7f3ee9eb1700 1 -- 172.21.15.32:0/1149045785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ee4101460 msgr2=0x7f3ee41a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.626+0000 7f3ee9eb1700 1 --2- 172.21.15.32:0/1149045785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ee4101460 0x7f3ee41a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:31.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.626+0000 7f3ee9eb1700 1 -- 172.21.15.32:0/1149045785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee41009f0 msgr2=0x7f3ee41993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:31.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.626+0000 7f3ee9eb1700 1 --2- 172.21.15.32:0/1149045785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee41009f0 0x7f3ee41993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:31.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.626+0000 7f3ee9eb1700 1 -- 172.21.15.32:0/1149045785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee00095f0 con 0x7f3ee4100030 2023-09-30T14:36:31.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.626+0000 7f3ee96b0700 1 --2- 172.21.15.32:0/1149045785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee41009f0 0x7f3ee41993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:36:31.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.626+0000 7f3ee9eb1700 1 --2- 172.21.15.32:0/1149045785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee4100030 0x7f3ee4198ea0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000f690 tx=0x7f3ee000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:31.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.626+0000 7f3edaffd700 1 -- 172.21.15.32:0/1149045785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee001d070 con 0x7f3ee4100030 2023-09-30T14:36:31.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.626+0000 7f3edaffd700 1 -- 172.21.15.32:0/1149045785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee000fc50 con 0x7f3ee4100030 2023-09-30T14:36:31.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.626+0000 7f3eec115700 1 -- 172.21.15.32:0/1149045785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee41a0f80 con 0x7f3ee4100030 2023-09-30T14:36:31.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.626+0000 7f3edaffd700 1 -- 172.21.15.32:0/1149045785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0017650 con 0x7f3ee4100030 2023-09-30T14:36:31.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.626+0000 7f3eec115700 1 -- 172.21.15.32:0/1149045785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee41a13b0 con 0x7f3ee4100030 2023-09-30T14:36:31.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.627+0000 7f3eec115700 1 -- 172.21.15.32:0/1149045785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee410b730 con 0x7f3ee4100030 2023-09-30T14:36:31.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.628+0000 7f3edaffd700 1 -- 172.21.15.32:0/1149045785 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f3ee0022470 con 0x7f3ee4100030 2023-09-30T14:36:31.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.629+0000 7f3edaffd700 1 --2- 172.21.15.32:0/1149045785 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3ed00a1020 0x7f3ed00a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:31.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.629+0000 7f3ee96b0700 1 --2- 172.21.15.32:0/1149045785 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3ed00a1020 0x7f3ed00a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:31.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.629+0000 7f3edaffd700 1 -- 172.21.15.32:0/1149045785 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3ee00cd800 con 0x7f3ee4100030 2023-09-30T14:36:31.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.629+0000 7f3ee96b0700 1 --2- 172.21.15.32:0/1149045785 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3ed00a1020 0x7f3ed00a34e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4007900 tx=0x7f3ed4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:31.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.633+0000 7f3edaffd700 1 -- 172.21.15.32:0/1149045785 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f3ee00975b0 con 0x7f3ee4100030 2023-09-30T14:36:31.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.833+0000 7f3eec115700 1 -- 172.21.15.32:0/1149045785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f3ee410b970 con 0x7f3ee4100030 2023-09-30T14:36:31.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.834+0000 7f3edaffd700 1 -- 172.21.15.32:0/1149045785 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f3ee00270a0 con 0x7f3ee4100030 2023-09-30T14:36:31.835 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:36:31.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.836+0000 7f3eec115700 1 -- 172.21.15.32:0/1149045785 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3ed00a1020 msgr2=0x7f3ed00a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:31.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.836+0000 7f3eec115700 1 --2- 172.21.15.32:0/1149045785 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3ed00a1020 0x7f3ed00a34e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4007900 tx=0x7f3ed4008040 comp rx=0 tx=0).stop 2023-09-30T14:36:31.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.836+0000 7f3eec115700 1 -- 172.21.15.32:0/1149045785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee4100030 msgr2=0x7f3ee4198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:31.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.836+0000 7f3eec115700 1 --2- 172.21.15.32:0/1149045785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee4100030 0x7f3ee4198ea0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000f690 tx=0x7f3ee000f770 comp rx=0 tx=0).stop 2023-09-30T14:36:31.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.836+0000 7f3eec115700 1 -- 172.21.15.32:0/1149045785 shutdown_connections 2023-09-30T14:36:31.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.836+0000 7f3eec115700 1 --2- 172.21.15.32:0/1149045785 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3ed00a1020 0x7f3ed00a34e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:31.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.836+0000 7f3eec115700 1 --2- 172.21.15.32:0/1149045785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ee4101460 0x7f3ee41a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:31.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.836+0000 7f3eec115700 1 --2- 172.21.15.32:0/1149045785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ee4100030 0x7f3ee4198ea0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:31.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.836+0000 7f3eec115700 1 --2- 172.21.15.32:0/1149045785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee41009f0 0x7f3ee41993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:31.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.836+0000 7f3eec115700 1 -- 172.21.15.32:0/1149045785 >> 172.21.15.32:0/1149045785 conn(0x7f3ee40faae0 msgr2=0x7f3ee40fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:31.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.836+0000 7f3eec115700 1 -- 172.21.15.32:0/1149045785 shutdown_connections 2023-09-30T14:36:31.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:31.836+0000 7f3eec115700 1 -- 172.21.15.32:0/1149045785 wait complete. 2023-09-30T14:36:32.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:32 smithi032 ceph-mon[98592]: osdmap e79: 12 total, 12 up, 12 in 2023-09-30T14:36:32.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:32 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1149045785' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-30T14:36:32.291 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1696084589,"num_in_osds":12,"osd_in_since":1696084572,"num_remapped_pgs":0} 2023-09-30T14:36:32.292 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-09-30T14:36:32.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:32 smithi148 ceph-mon[102355]: osdmap e79: 12 total, 12 up, 12 in 2023-09-30T14:36:32.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:32 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1149045785' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-30T14:36:32.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:32 smithi156 ceph-mon[103579]: osdmap e79: 12 total, 12 up, 12 in 2023-09-30T14:36:32.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:32 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1149045785' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-30T14:36:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:36:33.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:33 smithi148 ceph-mon[102355]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:33.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:33 smithi032 ceph-mon[98592]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:33.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:33 smithi156 ceph-mon[103579]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:34.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.136+0000 7f1c92f71700 1 -- 172.21.15.32:0/4242573160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c8c1018b0 msgr2=0x7f1c8c10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:34.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.136+0000 7f1c92f71700 1 --2- 172.21.15.32:0/4242573160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c8c1018b0 0x7f1c8c10dd20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400b0d0 tx=0x7f1c8400b3e0 comp rx=0 tx=0).stop 2023-09-30T14:36:34.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.136+0000 7f1c92f71700 1 -- 172.21.15.32:0/4242573160 shutdown_connections 2023-09-30T14:36:34.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.136+0000 7f1c92f71700 1 --2- 172.21.15.32:0/4242573160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c8c1018b0 0x7f1c8c10dd20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:34.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.136+0000 7f1c92f71700 1 --2- 172.21.15.32:0/4242573160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c8c106f00 0x7f1c8c1072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:34.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.136+0000 7f1c92f71700 1 --2- 172.21.15.32:0/4242573160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c8c100ef0 0x7f1c8c101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:34.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.136+0000 7f1c92f71700 1 -- 172.21.15.32:0/4242573160 >> 172.21.15.32:0/4242573160 conn(0x7f1c8c078010 msgr2=0x7f1c8c078420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:34.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.136+0000 7f1c92f71700 1 -- 172.21.15.32:0/4242573160 shutdown_connections 2023-09-30T14:36:34.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.136+0000 7f1c92f71700 1 -- 172.21.15.32:0/4242573160 wait complete. 2023-09-30T14:36:34.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c92f71700 1 Processor -- start 2023-09-30T14:36:34.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c92f71700 1 -- start start 2023-09-30T14:36:34.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c92f71700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c8c100ef0 0x7f1c8c197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:34.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c92f71700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c8c1018b0 0x7f1c8c1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:34.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c92f71700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c8c106f00 0x7f1c8c19ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:34.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c92f71700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c19c020 con 0x7f1c8c1018b0 2023-09-30T14:36:34.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c92f71700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c19c190 con 0x7f1c8c106f00 2023-09-30T14:36:34.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c92f71700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c19c350 con 0x7f1c8c100ef0 2023-09-30T14:36:34.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c8bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c8c1018b0 0x7f1c8c1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:34.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c8bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c8c1018b0 0x7f1c8c1975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33406/0 (socket says 172.21.15.32:33406) 2023-09-30T14:36:34.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c8bfff700 1 -- 172.21.15.32:0/496034991 learned_addr learned my addr 172.21.15.32:0/496034991 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:36:34.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c90d0d700 1 --2- 172.21.15.32:0/496034991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c8c100ef0 0x7f1c8c197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:34.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c8bfff700 1 -- 172.21.15.32:0/496034991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c8c100ef0 msgr2=0x7f1c8c197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:34.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c8bfff700 1 --2- 172.21.15.32:0/496034991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c8c100ef0 0x7f1c8c197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:34.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c8bfff700 1 -- 172.21.15.32:0/496034991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c8c106f00 msgr2=0x7f1c8c19ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:34.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c9150e700 1 --2- 172.21.15.32:0/496034991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c8c106f00 0x7f1c8c19ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:34.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c8bfff700 1 --2- 172.21.15.32:0/496034991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c8c106f00 0x7f1c8c19ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:34.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c8bfff700 1 -- 172.21.15.32:0/496034991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c84009d00 con 0x7f1c8c1018b0 2023-09-30T14:36:34.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.138+0000 7f1c9150e700 1 --2- 172.21.15.32:0/496034991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c8c106f00 0x7f1c8c19ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:36:34.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.137+0000 7f1c90d0d700 1 --2- 172.21.15.32:0/496034991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c8c100ef0 0x7f1c8c197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:36:34.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.138+0000 7f1c8bfff700 1 --2- 172.21.15.32:0/496034991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c8c1018b0 0x7f1c8c1975a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800d950 tx=0x7f1c7800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:34.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.138+0000 7f1c89ffb700 1 -- 172.21.15.32:0/496034991 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c78009b40 con 0x7f1c8c1018b0 2023-09-30T14:36:34.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.138+0000 7f1c89ffb700 1 -- 172.21.15.32:0/496034991 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c78009ca0 con 0x7f1c8c1018b0 2023-09-30T14:36:34.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.138+0000 7f1c89ffb700 1 -- 172.21.15.32:0/496034991 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7800f620 con 0x7f1c8c1018b0 2023-09-30T14:36:34.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.138+0000 7f1c92f71700 1 -- 172.21.15.32:0/496034991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8c19c630 con 0x7f1c8c1018b0 2023-09-30T14:36:34.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.138+0000 7f1c92f71700 1 -- 172.21.15.32:0/496034991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8c1a3e90 con 0x7f1c8c1018b0 2023-09-30T14:36:34.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.139+0000 7f1c92f71700 1 -- 172.21.15.32:0/496034991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c8c04eb20 con 0x7f1c8c1018b0 2023-09-30T14:36:34.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.140+0000 7f1c89ffb700 1 -- 172.21.15.32:0/496034991 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f1c78010460 con 0x7f1c8c1018b0 2023-09-30T14:36:34.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.141+0000 7f1c89ffb700 1 --2- 172.21.15.32:0/496034991 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1c7c0a1020 0x7f1c7c0a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:34.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.141+0000 7f1c90d0d700 1 --2- 172.21.15.32:0/496034991 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1c7c0a1020 0x7f1c7c0a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:34.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.141+0000 7f1c89ffb700 1 -- 172.21.15.32:0/496034991 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1c780cc590 con 0x7f1c8c1018b0 2023-09-30T14:36:34.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.141+0000 7f1c90d0d700 1 --2- 172.21.15.32:0/496034991 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1c7c0a1020 0x7f1c7c0a34e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1c80005fd0 tx=0x7f1c80005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:34.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.146+0000 7f1c89ffb700 1 -- 172.21.15.32:0/496034991 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f1c78096230 con 0x7f1c8c1018b0 2023-09-30T14:36:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.343+0000 7f1c92f71700 1 -- 172.21.15.32:0/496034991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c8c110f00 con 0x7f1c8c1018b0 2023-09-30T14:36:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.344+0000 7f1c89ffb700 1 -- 172.21.15.32:0/496034991 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18436 (secure 0 0 0) 0x7f1c78099850 con 0x7f1c8c1018b0 2023-09-30T14:36:34.345 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:36:34.345 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":79,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:36:31.061613+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:36:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.346+0000 7f1c92f71700 1 -- 172.21.15.32:0/496034991 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1c7c0a1020 msgr2=0x7f1c7c0a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.346+0000 7f1c92f71700 1 --2- 172.21.15.32:0/496034991 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1c7c0a1020 0x7f1c7c0a34e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1c80005fd0 tx=0x7f1c80005db0 comp rx=0 tx=0).stop 2023-09-30T14:36:34.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.346+0000 7f1c92f71700 1 -- 172.21.15.32:0/496034991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c8c1018b0 msgr2=0x7f1c8c1975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:34.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.346+0000 7f1c92f71700 1 --2- 172.21.15.32:0/496034991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c8c1018b0 0x7f1c8c1975a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800d950 tx=0x7f1c7800dd10 comp rx=0 tx=0).stop 2023-09-30T14:36:34.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.346+0000 7f1c92f71700 1 -- 172.21.15.32:0/496034991 shutdown_connections 2023-09-30T14:36:34.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.346+0000 7f1c92f71700 1 --2- 172.21.15.32:0/496034991 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1c7c0a1020 0x7f1c7c0a34e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:34.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.346+0000 7f1c92f71700 1 --2- 172.21.15.32:0/496034991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c8c106f00 0x7f1c8c19ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:34.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.346+0000 7f1c92f71700 1 --2- 172.21.15.32:0/496034991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c8c100ef0 0x7f1c8c197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:34.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.346+0000 7f1c92f71700 1 --2- 172.21.15.32:0/496034991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c8c1018b0 0x7f1c8c1975a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:34.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.346+0000 7f1c92f71700 1 -- 172.21.15.32:0/496034991 >> 172.21.15.32:0/496034991 conn(0x7f1c8c078010 msgr2=0x7f1c8c0ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:34.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.346+0000 7f1c92f71700 1 -- 172.21.15.32:0/496034991 shutdown_connections 2023-09-30T14:36:34.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:34.346+0000 7f1c92f71700 1 -- 172.21.15.32:0/496034991 wait complete. 2023-09-30T14:36:34.807 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T14:33:20.224529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T14:36:34.808 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-09-30T14:36:35.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:35 smithi148 ceph-mon[102355]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:35.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:35 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/496034991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:36:35.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:35 smithi032 ceph-mon[98592]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:35.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:35 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/496034991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:36:35.470 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:36:35.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:35 smithi156 ceph-mon[103579]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:35.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:35 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/496034991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:36:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.684+0000 7f1356b45700 1 -- 172.21.15.32:0/439898753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1350105de0 msgr2=0x7f13501061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.684+0000 7f1356b45700 1 --2- 172.21.15.32:0/439898753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1350105de0 0x7f13501061c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f13440099c0 tx=0x7f1344009cd0 comp rx=0 tx=0).stop 2023-09-30T14:36:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.684+0000 7f1356b45700 1 -- 172.21.15.32:0/439898753 shutdown_connections 2023-09-30T14:36:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.684+0000 7f1356b45700 1 --2- 172.21.15.32:0/439898753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1350100790 0x7f135010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.684+0000 7f1356b45700 1 --2- 172.21.15.32:0/439898753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1350105de0 0x7f13501061c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.684+0000 7f1356b45700 1 --2- 172.21.15.32:0/439898753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13500ffdd0 0x7f1350100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.684+0000 7f1356b45700 1 -- 172.21.15.32:0/439898753 >> 172.21.15.32:0/439898753 conn(0x7f1350075d70 msgr2=0x7f1350076180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.684+0000 7f1356b45700 1 -- 172.21.15.32:0/439898753 shutdown_connections 2023-09-30T14:36:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.684+0000 7f1356b45700 1 -- 172.21.15.32:0/439898753 wait complete. 2023-09-30T14:36:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.685+0000 7f1356b45700 1 Processor -- start 2023-09-30T14:36:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.685+0000 7f1356b45700 1 -- start start 2023-09-30T14:36:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.685+0000 7f1356b45700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13500ffdd0 0x7f1350197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.685+0000 7f1356b45700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1350100790 0x7f13501982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.685+0000 7f1356b45700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1350105de0 0x7f13501a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:36.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.685+0000 7f1356b45700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13501a3370 con 0x7f13500ffdd0 2023-09-30T14:36:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.685+0000 7f1356b45700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13501a34b0 con 0x7f1350105de0 2023-09-30T14:36:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.685+0000 7f1356b45700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13501a35f0 con 0x7f1350100790 2023-09-30T14:36:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.685+0000 7f13548e1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13500ffdd0 0x7f1350197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.685+0000 7f13548e1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13500ffdd0 0x7f1350197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33418/0 (socket says 172.21.15.32:33418) 2023-09-30T14:36:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.685+0000 7f13548e1700 1 -- 172.21.15.32:0/2816067718 learned_addr learned my addr 172.21.15.32:0/2816067718 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:36:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.685+0000 7f13550e2700 1 --2- 172.21.15.32:0/2816067718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1350105de0 0x7f13501a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.685+0000 7f134ffff700 1 --2- 172.21.15.32:0/2816067718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1350100790 0x7f13501982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.686+0000 7f13548e1700 1 -- 172.21.15.32:0/2816067718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1350100790 msgr2=0x7f13501982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.686+0000 7f13548e1700 1 --2- 172.21.15.32:0/2816067718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1350100790 0x7f13501982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.686+0000 7f13548e1700 1 -- 172.21.15.32:0/2816067718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1350105de0 msgr2=0x7f13501a2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.686+0000 7f13548e1700 1 --2- 172.21.15.32:0/2816067718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1350105de0 0x7f13501a2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:36.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.686+0000 7f13548e1700 1 -- 172.21.15.32:0/2816067718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13440095f0 con 0x7f13500ffdd0 2023-09-30T14:36:36.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.686+0000 7f134ffff700 1 --2- 172.21.15.32:0/2816067718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1350100790 0x7f13501982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:36:36.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.686+0000 7f13548e1700 1 --2- 172.21.15.32:0/2816067718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13500ffdd0 0x7f1350197db0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1344005910 tx=0x7f1344004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:36.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.686+0000 7f134dffb700 1 -- 172.21.15.32:0/2816067718 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134401d070 con 0x7f13500ffdd0 2023-09-30T14:36:36.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.686+0000 7f1356b45700 1 -- 172.21.15.32:0/2816067718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13501a37f0 con 0x7f13500ffdd0 2023-09-30T14:36:36.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.686+0000 7f134dffb700 1 -- 172.21.15.32:0/2816067718 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1344004200 con 0x7f13500ffdd0 2023-09-30T14:36:36.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.686+0000 7f134dffb700 1 -- 172.21.15.32:0/2816067718 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1344022e60 con 0x7f13500ffdd0 2023-09-30T14:36:36.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.686+0000 7f1356b45700 1 -- 172.21.15.32:0/2816067718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13501a3c00 con 0x7f13500ffdd0 2023-09-30T14:36:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.687+0000 7f1356b45700 1 -- 172.21.15.32:0/2816067718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f135004eb20 con 0x7f13500ffdd0 2023-09-30T14:36:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.688+0000 7f134dffb700 1 -- 172.21.15.32:0/2816067718 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f1344003d40 con 0x7f13500ffdd0 2023-09-30T14:36:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.689+0000 7f134dffb700 1 --2- 172.21.15.32:0/2816067718 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f13400a0fa0 0x7f13400a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.689+0000 7f134ffff700 1 --2- 172.21.15.32:0/2816067718 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f13400a0fa0 0x7f13400a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.689+0000 7f134dffb700 1 -- 172.21.15.32:0/2816067718 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f13440cd570 con 0x7f13500ffdd0 2023-09-30T14:36:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.690+0000 7f134ffff700 1 --2- 172.21.15.32:0/2816067718 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f13400a0fa0 0x7f13400a3460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f13500699f0 tx=0x7f133c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:36.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.694+0000 7f134dffb700 1 -- 172.21.15.32:0/2816067718 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f134405cf30 con 0x7f13500ffdd0 2023-09-30T14:36:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.890+0000 7f1356b45700 1 -- 172.21.15.32:0/2816067718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f135010fe70 con 0x7f13500ffdd0 2023-09-30T14:36:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.891+0000 7f134dffb700 1 -- 172.21.15.32:0/2816067718 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13440972a0 con 0x7f13500ffdd0 2023-09-30T14:36:36.892 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T14:36:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.892+0000 7f1356b45700 1 -- 172.21.15.32:0/2816067718 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f13400a0fa0 msgr2=0x7f13400a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.892+0000 7f1356b45700 1 --2- 172.21.15.32:0/2816067718 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f13400a0fa0 0x7f13400a3460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f13500699f0 tx=0x7f133c008040 comp rx=0 tx=0).stop 2023-09-30T14:36:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.892+0000 7f1356b45700 1 -- 172.21.15.32:0/2816067718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13500ffdd0 msgr2=0x7f1350197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.892+0000 7f1356b45700 1 --2- 172.21.15.32:0/2816067718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13500ffdd0 0x7f1350197db0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1344005910 tx=0x7f1344004730 comp rx=0 tx=0).stop 2023-09-30T14:36:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.893+0000 7f1356b45700 1 -- 172.21.15.32:0/2816067718 shutdown_connections 2023-09-30T14:36:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.893+0000 7f1356b45700 1 --2- 172.21.15.32:0/2816067718 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f13400a0fa0 0x7f13400a3460 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.893+0000 7f1356b45700 1 --2- 172.21.15.32:0/2816067718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1350105de0 0x7f13501a2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.893+0000 7f1356b45700 1 --2- 172.21.15.32:0/2816067718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13500ffdd0 0x7f1350197db0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.893+0000 7f1356b45700 1 --2- 172.21.15.32:0/2816067718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1350100790 0x7f13501982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.893+0000 7f1356b45700 1 -- 172.21.15.32:0/2816067718 >> 172.21.15.32:0/2816067718 conn(0x7f1350075d70 msgr2=0x7f13500fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.893+0000 7f1356b45700 1 -- 172.21.15.32:0/2816067718 shutdown_connections 2023-09-30T14:36:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:36.893+0000 7f1356b45700 1 -- 172.21.15.32:0/2816067718 wait complete. 2023-09-30T14:36:37.294 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:37 smithi032 ceph-mon[98592]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:37.294 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:37 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2816067718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:36:37.330 INFO:tasks.cephadm:Adding mds.a on smithi032 2023-09-30T14:36:37.331 INFO:tasks.cephadm:Adding mds.d on smithi032 2023-09-30T14:36:37.331 INFO:tasks.cephadm:Adding mds.g on smithi032 2023-09-30T14:36:37.331 INFO:tasks.cephadm:Adding mds.j on smithi032 2023-09-30T14:36:37.331 INFO:tasks.cephadm:Adding mds.b on smithi148 2023-09-30T14:36:37.331 INFO:tasks.cephadm:Adding mds.e on smithi148 2023-09-30T14:36:37.331 INFO:tasks.cephadm:Adding mds.h on smithi148 2023-09-30T14:36:37.331 INFO:tasks.cephadm:Adding mds.k on smithi148 2023-09-30T14:36:37.331 INFO:tasks.cephadm:Adding mds.c on smithi156 2023-09-30T14:36:37.331 INFO:tasks.cephadm:Adding mds.f on smithi156 2023-09-30T14:36:37.331 INFO:tasks.cephadm:Adding mds.i on smithi156 2023-09-30T14:36:37.331 INFO:tasks.cephadm:Adding mds.l on smithi156 2023-09-30T14:36:37.332 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph orch apply mds all '12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l' 2023-09-30T14:36:37.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:37 smithi148 ceph-mon[102355]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:37.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:37 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2816067718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:36:37.366 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:37 smithi156 ceph-mon[103579]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:37.366 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:37 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2816067718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:36:37.908 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.c/config 2023-09-30T14:36:39.102 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.101+0000 7f9110ed4700 1 -- 172.21.15.156:0/3174306892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f910c102a30 msgr2=0x7f910c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:39.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.101+0000 7f9110ed4700 1 --2- 172.21.15.156:0/3174306892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f910c102a30 0x7f910c109350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f910400b0a0 tx=0x7f910400b3b0 comp rx=0 tx=0).stop 2023-09-30T14:36:39.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.101+0000 7f9110ed4700 1 -- 172.21.15.156:0/3174306892 shutdown_connections 2023-09-30T14:36:39.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.101+0000 7f9110ed4700 1 --2- 172.21.15.156:0/3174306892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f910c102a30 0x7f910c109350 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:39.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.101+0000 7f9110ed4700 1 --2- 172.21.15.156:0/3174306892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c100f10 0x7f910c101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:39.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.101+0000 7f9110ed4700 1 --2- 172.21.15.156:0/3174306892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f910c102070 0x7f910c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:39.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.101+0000 7f9110ed4700 1 -- 172.21.15.156:0/3174306892 >> 172.21.15.156:0/3174306892 conn(0x7f910c0fc4f0 msgr2=0x7f910c0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:39.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.101+0000 7f9110ed4700 1 -- 172.21.15.156:0/3174306892 shutdown_connections 2023-09-30T14:36:39.103 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.101+0000 7f9110ed4700 1 -- 172.21.15.156:0/3174306892 wait complete. 2023-09-30T14:36:39.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.102+0000 7f9110ed4700 1 Processor -- start 2023-09-30T14:36:39.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.102+0000 7f9110ed4700 1 -- start start 2023-09-30T14:36:39.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.102+0000 7f9110ed4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c100f10 0x7f910c196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:39.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.102+0000 7f9110ed4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f910c102070 0x7f910c196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:39.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.102+0000 7f9110ed4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f910c102a30 0x7f910c19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:39.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.102+0000 7f9110ed4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c19e5d0 con 0x7f910c102a30 2023-09-30T14:36:39.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.102+0000 7f9110ed4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c19e710 con 0x7f910c100f10 2023-09-30T14:36:39.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.102+0000 7f9110ed4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c19e8a0 con 0x7f910c102070 2023-09-30T14:36:39.105 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.102+0000 7f9109d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f910c102070 0x7f910c196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:39.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.102+0000 7f9109d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f910c102070 0x7f910c196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:36496/0 (socket says 172.21.15.156:36496) 2023-09-30T14:36:39.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.102+0000 7f9109d9b700 1 -- 172.21.15.156:0/3908158980 learned_addr learned my addr 172.21.15.156:0/3908158980 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-30T14:36:39.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.102+0000 7f910a59c700 1 --2- 172.21.15.156:0/3908158980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c100f10 0x7f910c196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:39.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.103+0000 7f910ad9d700 1 --2- 172.21.15.156:0/3908158980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f910c102a30 0x7f910c19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:39.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.103+0000 7f9109d9b700 1 -- 172.21.15.156:0/3908158980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c100f10 msgr2=0x7f910c196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:39.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.103+0000 7f9109d9b700 1 --2- 172.21.15.156:0/3908158980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c100f10 0x7f910c196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:39.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.103+0000 7f9109d9b700 1 -- 172.21.15.156:0/3908158980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f910c102a30 msgr2=0x7f910c19e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:39.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.103+0000 7f9109d9b700 1 --2- 172.21.15.156:0/3908158980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f910c102a30 0x7f910c19e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:39.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.103+0000 7f9109d9b700 1 -- 172.21.15.156:0/3908158980 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9104009d00 con 0x7f910c102070 2023-09-30T14:36:39.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.103+0000 7f9109d9b700 1 --2- 172.21.15.156:0/3908158980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f910c102070 0x7f910c196f80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00d980 tx=0x7f90fc00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:39.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.103+0000 7f91037fe700 1 -- 172.21.15.156:0/3908158980 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc0098d0 con 0x7f910c102070 2023-09-30T14:36:39.106 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.103+0000 7f9110ed4700 1 -- 172.21.15.156:0/3908158980 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910c0730b0 con 0x7f910c102070 2023-09-30T14:36:39.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.103+0000 7f91037fe700 1 -- 172.21.15.156:0/3908158980 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90fc010460 con 0x7f910c102070 2023-09-30T14:36:39.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.103+0000 7f91037fe700 1 -- 172.21.15.156:0/3908158980 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc00f620 con 0x7f910c102070 2023-09-30T14:36:39.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.103+0000 7f9110ed4700 1 -- 172.21.15.156:0/3908158980 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910c0735b0 con 0x7f910c102070 2023-09-30T14:36:39.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.104+0000 7f9110ed4700 1 -- 172.21.15.156:0/3908158980 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910c10c660 con 0x7f910c102070 2023-09-30T14:36:39.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.106+0000 7f91037fe700 1 -- 172.21.15.156:0/3908158980 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f90fc010860 con 0x7f910c102070 2023-09-30T14:36:39.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.106+0000 7f91037fe700 1 --2- 172.21.15.156:0/3908158980 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f90f80a0fa0 0x7f90f80a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:39.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.106+0000 7f91037fe700 1 -- 172.21.15.156:0/3908158980 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f90fc0cbfc0 con 0x7f910c102070 2023-09-30T14:36:39.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.106+0000 7f910a59c700 1 --2- 172.21.15.156:0/3908158980 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f90f80a0fa0 0x7f90f80a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:39.108 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.107+0000 7f910a59c700 1 --2- 172.21.15.156:0/3908158980 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f90f80a0fa0 0x7f90f80a3460 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f90f4005950 tx=0x7f90f400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:39.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.109+0000 7f91037fe700 1 -- 172.21.15.156:0/3908158980 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f90fc095c60 con 0x7f910c102070 2023-09-30T14:36:39.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.305+0000 7f9110ed4700 1 -- 172.21.15.156:0/3908158980 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f910c061260 con 0x7f90f80a0fa0 2023-09-30T14:36:39.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.314+0000 7f91037fe700 1 -- 172.21.15.156:0/3908158980 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f910c061260 con 0x7f90f80a0fa0 2023-09-30T14:36:39.315 INFO:teuthology.orchestra.run.smithi156.stdout:Scheduled mds.all update... 2023-09-30T14:36:39.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.316+0000 7f9110ed4700 1 -- 172.21.15.156:0/3908158980 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f90f80a0fa0 msgr2=0x7f90f80a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:39.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.316+0000 7f9110ed4700 1 --2- 172.21.15.156:0/3908158980 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f90f80a0fa0 0x7f90f80a3460 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f90f4005950 tx=0x7f90f400b410 comp rx=0 tx=0).stop 2023-09-30T14:36:39.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.316+0000 7f9110ed4700 1 -- 172.21.15.156:0/3908158980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f910c102070 msgr2=0x7f910c196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:39.317 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.316+0000 7f9110ed4700 1 --2- 172.21.15.156:0/3908158980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f910c102070 0x7f910c196f80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00d980 tx=0x7f90fc00dc90 comp rx=0 tx=0).stop 2023-09-30T14:36:39.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.316+0000 7f9110ed4700 1 -- 172.21.15.156:0/3908158980 shutdown_connections 2023-09-30T14:36:39.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.316+0000 7f9110ed4700 1 --2- 172.21.15.156:0/3908158980 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f90f80a0fa0 0x7f90f80a3460 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:39.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.316+0000 7f9110ed4700 1 --2- 172.21.15.156:0/3908158980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f910c102a30 0x7f910c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:39.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.316+0000 7f9110ed4700 1 --2- 172.21.15.156:0/3908158980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c100f10 0x7f910c196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:39.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.316+0000 7f9110ed4700 1 --2- 172.21.15.156:0/3908158980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f910c102070 0x7f910c196f80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:39.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.316+0000 7f9110ed4700 1 -- 172.21.15.156:0/3908158980 >> 172.21.15.156:0/3908158980 conn(0x7f910c0fc4f0 msgr2=0x7f910c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:39.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.316+0000 7f9110ed4700 1 -- 172.21.15.156:0/3908158980 shutdown_connections 2023-09-30T14:36:39.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-30T14:36:39.316+0000 7f9110ed4700 1 -- 172.21.15.156:0/3908158980 wait complete. 2023-09-30T14:36:39.341 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:39 smithi156 ceph-mon[103579]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:39.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:39 smithi148 ceph-mon[102355]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:39.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:39 smithi032 ceph-mon[98592]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:39.665 DEBUG:teuthology.orchestra.run.smithi032:mds.a> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.a.service 2023-09-30T14:36:39.668 DEBUG:teuthology.orchestra.run.smithi032:mds.d> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.d.service 2023-09-30T14:36:39.706 INFO:journalctl@ceph.mds.a.smithi032.stdout:-- Logs begin at Sat 2023-09-30 14:15:36 UTC. -- 2023-09-30T14:36:39.707 DEBUG:teuthology.orchestra.run.smithi032:mds.g> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.g.service 2023-09-30T14:36:39.737 INFO:journalctl@ceph.mds.d.smithi032.stdout:-- Logs begin at Sat 2023-09-30 14:15:36 UTC. -- 2023-09-30T14:36:39.738 DEBUG:teuthology.orchestra.run.smithi032:mds.j> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.j.service 2023-09-30T14:36:39.767 INFO:journalctl@ceph.mds.g.smithi032.stdout:-- Logs begin at Sat 2023-09-30 14:15:36 UTC. -- 2023-09-30T14:36:39.768 DEBUG:teuthology.orchestra.run.smithi148:mds.b> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.b.service 2023-09-30T14:36:39.771 DEBUG:teuthology.orchestra.run.smithi148:mds.e> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.e.service 2023-09-30T14:36:39.795 INFO:journalctl@ceph.mds.j.smithi032.stdout:-- Logs begin at Sat 2023-09-30 14:15:36 UTC. -- 2023-09-30T14:36:39.810 INFO:journalctl@ceph.mds.b.smithi148.stdout:-- Logs begin at Sat 2023-09-30 14:16:26 UTC. -- 2023-09-30T14:36:39.811 DEBUG:teuthology.orchestra.run.smithi148:mds.h> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.h.service 2023-09-30T14:36:39.840 INFO:journalctl@ceph.mds.e.smithi148.stdout:-- Logs begin at Sat 2023-09-30 14:16:26 UTC. -- 2023-09-30T14:36:39.841 DEBUG:teuthology.orchestra.run.smithi148:mds.k> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.k.service 2023-09-30T14:36:39.869 INFO:journalctl@ceph.mds.h.smithi148.stdout:-- Logs begin at Sat 2023-09-30 14:16:26 UTC. -- 2023-09-30T14:36:39.870 DEBUG:teuthology.orchestra.run.smithi156:mds.c> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.c.service 2023-09-30T14:36:39.872 DEBUG:teuthology.orchestra.run.smithi156:mds.f> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.f.service 2023-09-30T14:36:39.897 INFO:journalctl@ceph.mds.k.smithi148.stdout:-- Logs begin at Sat 2023-09-30 14:16:26 UTC. -- 2023-09-30T14:36:39.901 INFO:journalctl@ceph.mds.c.smithi156.stdout:-- Logs begin at Sat 2023-09-30 14:16:23 UTC. -- 2023-09-30T14:36:39.902 DEBUG:teuthology.orchestra.run.smithi156:mds.i> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.i.service 2023-09-30T14:36:39.927 INFO:journalctl@ceph.mds.f.smithi156.stdout:-- Logs begin at Sat 2023-09-30 14:16:23 UTC. -- 2023-09-30T14:36:39.928 DEBUG:teuthology.orchestra.run.smithi156:mds.l> sudo journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.l.service 2023-09-30T14:36:39.953 INFO:journalctl@ceph.mds.i.smithi156.stdout:-- Logs begin at Sat 2023-09-30 14:16:23 UTC. -- 2023-09-30T14:36:39.954 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-09-30T14:36:39.955 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T14:36:39.980 INFO:journalctl@ceph.mds.l.smithi156.stdout:-- Logs begin at Sat 2023-09-30 14:16:23 UTC. -- 2023-09-30T14:36:40.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.091+0000 7f380f9d3700 1 -- 172.21.15.32:0/999138374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38080fff40 msgr2=0x7f38081003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:40.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.091+0000 7f380f9d3700 1 --2- 172.21.15.32:0/999138374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38080fff40 0x7f38081003c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009990 tx=0x7f37fc009ca0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.091+0000 7f380f9d3700 1 -- 172.21.15.32:0/999138374 shutdown_connections 2023-09-30T14:36:40.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.091+0000 7f380f9d3700 1 --2- 172.21.15.32:0/999138374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3808100900 0x7f38081081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.091+0000 7f380f9d3700 1 --2- 172.21.15.32:0/999138374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38081012f0 0x7f3808101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.091+0000 7f380f9d3700 1 --2- 172.21.15.32:0/999138374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38080fff40 0x7f38081003c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.091+0000 7f380f9d3700 1 -- 172.21.15.32:0/999138374 >> 172.21.15.32:0/999138374 conn(0x7f38080faa30 msgr2=0x7f38080fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:40.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.091+0000 7f380f9d3700 1 -- 172.21.15.32:0/999138374 shutdown_connections 2023-09-30T14:36:40.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.091+0000 7f380f9d3700 1 -- 172.21.15.32:0/999138374 wait complete. 2023-09-30T14:36:40.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.091+0000 7f380f9d3700 1 Processor -- start 2023-09-30T14:36:40.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.091+0000 7f380f9d3700 1 -- start start 2023-09-30T14:36:40.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380f9d3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38080fff40 0x7f3808103cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:40.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380f9d3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3808100900 0x7f3808102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:40.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380f9d3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38081012f0 0x7f3808102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:40.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380f9d3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3808103010 con 0x7f3808100900 2023-09-30T14:36:40.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380f9d3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3808103180 con 0x7f38080fff40 2023-09-30T14:36:40.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380f9d3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38081a0b40 con 0x7f38081012f0 2023-09-30T14:36:40.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380cf6e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3808100900 0x7f3808102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:40.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380cf6e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3808100900 0x7f3808102330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33432/0 (socket says 172.21.15.32:33432) 2023-09-30T14:36:40.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380cf6e700 1 -- 172.21.15.32:0/3205078482 learned_addr learned my addr 172.21.15.32:0/3205078482 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:36:40.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380d76f700 1 --2- 172.21.15.32:0/3205078482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38080fff40 0x7f3808103cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380df70700 1 --2- 172.21.15.32:0/3205078482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38081012f0 0x7f3808102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380cf6e700 1 -- 172.21.15.32:0/3205078482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38081012f0 msgr2=0x7f3808102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380cf6e700 1 --2- 172.21.15.32:0/3205078482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38081012f0 0x7f3808102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380cf6e700 1 -- 172.21.15.32:0/3205078482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38080fff40 msgr2=0x7f3808103cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380cf6e700 1 --2- 172.21.15.32:0/3205078482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38080fff40 0x7f3808103cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380cf6e700 1 -- 172.21.15.32:0/3205078482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc0095f0 con 0x7f3808100900 2023-09-30T14:36:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380cf6e700 1 --2- 172.21.15.32:0/3205078482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3808100900 0x7f3808102330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37fc000c00 tx=0x7f37fc0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f37fa7fc700 1 -- 172.21.15.32:0/3205078482 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc01d070 con 0x7f3808100900 2023-09-30T14:36:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f37fa7fc700 1 -- 172.21.15.32:0/3205078482 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37fc022470 con 0x7f3808100900 2023-09-30T14:36:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380f9d3700 1 -- 172.21.15.32:0/3205078482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38081a0dc0 con 0x7f3808100900 2023-09-30T14:36:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f37fa7fc700 1 -- 172.21.15.32:0/3205078482 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc017650 con 0x7f3808100900 2023-09-30T14:36:40.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.092+0000 7f380f9d3700 1 -- 172.21.15.32:0/3205078482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38081a1190 con 0x7f3808100900 2023-09-30T14:36:40.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.093+0000 7f380f9d3700 1 -- 172.21.15.32:0/3205078482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380810b5a0 con 0x7f3808100900 2023-09-30T14:36:40.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.094+0000 7f37fa7fc700 1 -- 172.21.15.32:0/3205078482 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f37fc0225e0 con 0x7f3808100900 2023-09-30T14:36:40.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.095+0000 7f37fa7fc700 1 --2- 172.21.15.32:0/3205078482 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f37f40a0ff0 0x7f37f40a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:40.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.095+0000 7f380d76f700 1 --2- 172.21.15.32:0/3205078482 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f37f40a0ff0 0x7f37f40a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:40.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.095+0000 7f37fa7fc700 1 -- 172.21.15.32:0/3205078482 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f37fc0cdc00 con 0x7f3808100900 2023-09-30T14:36:40.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.095+0000 7f380d76f700 1 --2- 172.21.15.32:0/3205078482 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f37f40a0ff0 0x7f37f40a34b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3804005950 tx=0x7f380400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:40.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.098+0000 7f37fa7fc700 1 -- 172.21.15.32:0/3205078482 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f37fc097930 con 0x7f3808100900 2023-09-30T14:36:40.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.303+0000 7f380f9d3700 1 -- 172.21.15.32:0/3205078482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38081a1380 con 0x7f3808100900 2023-09-30T14:36:40.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.304+0000 7f37fa7fc700 1 -- 172.21.15.32:0/3205078482 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18436 (secure 0 0 0) 0x7f37fc09af50 con 0x7f3808100900 2023-09-30T14:36:40.305 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:36:40.306 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":79,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:36:31.061613+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:36:40.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.307+0000 7f380f9d3700 1 -- 172.21.15.32:0/3205078482 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f37f40a0ff0 msgr2=0x7f37f40a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:40.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.307+0000 7f380f9d3700 1 --2- 172.21.15.32:0/3205078482 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f37f40a0ff0 0x7f37f40a34b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3804005950 tx=0x7f380400b410 comp rx=0 tx=0).stop 2023-09-30T14:36:40.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.307+0000 7f380f9d3700 1 -- 172.21.15.32:0/3205078482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3808100900 msgr2=0x7f3808102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:40.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.307+0000 7f380f9d3700 1 --2- 172.21.15.32:0/3205078482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3808100900 0x7f3808102330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37fc000c00 tx=0x7f37fc0047a0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.307+0000 7f380f9d3700 1 -- 172.21.15.32:0/3205078482 shutdown_connections 2023-09-30T14:36:40.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.307+0000 7f380f9d3700 1 --2- 172.21.15.32:0/3205078482 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f37f40a0ff0 0x7f37f40a34b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.307+0000 7f380f9d3700 1 --2- 172.21.15.32:0/3205078482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38081012f0 0x7f3808102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.307+0000 7f380f9d3700 1 --2- 172.21.15.32:0/3205078482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38080fff40 0x7f3808103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.307+0000 7f380f9d3700 1 --2- 172.21.15.32:0/3205078482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3808100900 0x7f3808102330 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.307+0000 7f380f9d3700 1 -- 172.21.15.32:0/3205078482 >> 172.21.15.32:0/3205078482 conn(0x7f38080faa30 msgr2=0x7f38080feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:40.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.307+0000 7f380f9d3700 1 -- 172.21.15.32:0/3205078482 shutdown_connections 2023-09-30T14:36:40.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.307+0000 7f380f9d3700 1 -- 172.21.15.32:0/3205078482 wait complete. 2023-09-30T14:36:40.321 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T14:33:20.224529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T14:36:40.321 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T14:36:40.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.450+0000 7fc121cd5700 1 -- 172.21.15.32:0/731777085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc11c104270 msgr2=0x7fc11c106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.450+0000 7fc121cd5700 1 --2- 172.21.15.32:0/731777085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc11c104270 0x7fc11c106660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009990 tx=0x7fc10c009ca0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.450+0000 7fc121cd5700 1 -- 172.21.15.32:0/731777085 shutdown_connections 2023-09-30T14:36:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.450+0000 7fc121cd5700 1 --2- 172.21.15.32:0/731777085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc11c106ba0 0x7fc11c108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.450+0000 7fc121cd5700 1 --2- 172.21.15.32:0/731777085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc11c101940 0x7fc11c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.450+0000 7fc121cd5700 1 --2- 172.21.15.32:0/731777085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc11c104270 0x7fc11c106660 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.450+0000 7fc121cd5700 1 -- 172.21.15.32:0/731777085 >> 172.21.15.32:0/731777085 conn(0x7fc11c0fb2f0 msgr2=0x7fc11c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.450+0000 7fc121cd5700 1 -- 172.21.15.32:0/731777085 shutdown_connections 2023-09-30T14:36:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.450+0000 7fc121cd5700 1 -- 172.21.15.32:0/731777085 wait complete. 2023-09-30T14:36:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.450+0000 7fc121cd5700 1 Processor -- start 2023-09-30T14:36:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.450+0000 7fc121cd5700 1 -- start start 2023-09-30T14:36:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc121cd5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc11c101940 0x7fc11c196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc121cd5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc11c104270 0x7fc11c197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc121cd5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc11c106ba0 0x7fc11c19e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc121cd5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c19e690 con 0x7fc11c106ba0 2023-09-30T14:36:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc121cd5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c19e7d0 con 0x7fc11c101940 2023-09-30T14:36:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc121cd5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c19e960 con 0x7fc11c104270 2023-09-30T14:36:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc11bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc11c106ba0 0x7fc11c19e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc11bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc11c106ba0 0x7fc11c19e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33446/0 (socket says 172.21.15.32:33446) 2023-09-30T14:36:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc11bfff700 1 -- 172.21.15.32:0/3382622995 learned_addr learned my addr 172.21.15.32:0/3382622995 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:36:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc11affd700 1 --2- 172.21.15.32:0/3382622995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc11c104270 0x7fc11c197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc11b7fe700 1 --2- 172.21.15.32:0/3382622995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc11c101940 0x7fc11c196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc11bfff700 1 -- 172.21.15.32:0/3382622995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc11c104270 msgr2=0x7fc11c197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc11bfff700 1 --2- 172.21.15.32:0/3382622995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc11c104270 0x7fc11c197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc11bfff700 1 -- 172.21.15.32:0/3382622995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc11c101940 msgr2=0x7fc11c196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc11bfff700 1 --2- 172.21.15.32:0/3382622995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc11c101940 0x7fc11c196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc11bfff700 1 -- 172.21.15.32:0/3382622995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c0095f0 con 0x7fc11c106ba0 2023-09-30T14:36:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc11bfff700 1 --2- 172.21.15.32:0/3382622995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc11c106ba0 0x7fc11c19e0c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc114007ee0 tx=0x7fc11400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc118ff9700 1 -- 172.21.15.32:0/3382622995 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114017070 con 0x7fc11c106ba0 2023-09-30T14:36:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc118ff9700 1 -- 172.21.15.32:0/3382622995 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc11400f040 con 0x7fc11c106ba0 2023-09-30T14:36:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc121cd5700 1 -- 172.21.15.32:0/3382622995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c19ec40 con 0x7fc11c106ba0 2023-09-30T14:36:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.451+0000 7fc118ff9700 1 -- 172.21.15.32:0/3382622995 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114005740 con 0x7fc11c106ba0 2023-09-30T14:36:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.452+0000 7fc121cd5700 1 -- 172.21.15.32:0/3382622995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c19f100 con 0x7fc11c106ba0 2023-09-30T14:36:40.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.452+0000 7fc121cd5700 1 -- 172.21.15.32:0/3382622995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c0fce70 con 0x7fc11c106ba0 2023-09-30T14:36:40.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.453+0000 7fc118ff9700 1 -- 172.21.15.32:0/3382622995 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fc11400dc60 con 0x7fc11c106ba0 2023-09-30T14:36:40.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.454+0000 7fc118ff9700 1 --2- 172.21.15.32:0/3382622995 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1040a0fa0 0x7fc1040a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:40.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.454+0000 7fc11b7fe700 1 --2- 172.21.15.32:0/3382622995 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1040a0fa0 0x7fc1040a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:40.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.455+0000 7fc118ff9700 1 -- 172.21.15.32:0/3382622995 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc1140cc0d0 con 0x7fc11c106ba0 2023-09-30T14:36:40.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.455+0000 7fc11b7fe700 1 --2- 172.21.15.32:0/3382622995 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1040a0fa0 0x7fc1040a3460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc110005950 tx=0x7fc1100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:40.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.458+0000 7fc118ff9700 1 -- 172.21.15.32:0/3382622995 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fc114095d70 con 0x7fc11c106ba0 2023-09-30T14:36:40.538 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:40 smithi156 ceph-mon[103579]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:36:40.538 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:40 smithi156 ceph-mon[103579]: Saving service mds.all spec with placement smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l;count:12 2023-09-30T14:36:40.538 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:40.538 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:36:40.538 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:40.538 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:36:40.538 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:40.538 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:40.538 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:40.539 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:40.539 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:40 smithi156 ceph-mon[103579]: Deploying daemon mds.l on smithi156 2023-09-30T14:36:40.539 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:40 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3205078482' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:36:40.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:40 smithi148 ceph-mon[102355]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:36:40.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:40 smithi148 ceph-mon[102355]: Saving service mds.all spec with placement smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l;count:12 2023-09-30T14:36:40.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:40.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:36:40.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:40.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:36:40.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:40.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:40.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:40.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:40.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:40 smithi148 ceph-mon[102355]: Deploying daemon mds.l on smithi156 2023-09-30T14:36:40.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:40 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3205078482' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:36:40.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:40 smithi032 ceph-mon[98592]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:36:40.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:40 smithi032 ceph-mon[98592]: Saving service mds.all spec with placement smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l;count:12 2023-09-30T14:36:40.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:40.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:36:40.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:40.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:36:40.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:40.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:40.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:40.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:40.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:40 smithi032 ceph-mon[98592]: Deploying daemon mds.l on smithi156 2023-09-30T14:36:40.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:40 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3205078482' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:36:40.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.659+0000 7fc121cd5700 1 -- 172.21.15.32:0/3382622995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc11c10c7e0 con 0x7fc11c106ba0 2023-09-30T14:36:40.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.660+0000 7fc118ff9700 1 -- 172.21.15.32:0/3382622995 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc114099390 con 0x7fc11c106ba0 2023-09-30T14:36:40.660 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T14:36:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.662+0000 7fc121cd5700 1 -- 172.21.15.32:0/3382622995 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1040a0fa0 msgr2=0x7fc1040a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.662+0000 7fc121cd5700 1 --2- 172.21.15.32:0/3382622995 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1040a0fa0 0x7fc1040a3460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc110005950 tx=0x7fc1100058e0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.662+0000 7fc121cd5700 1 -- 172.21.15.32:0/3382622995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc11c106ba0 msgr2=0x7fc11c19e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.662+0000 7fc121cd5700 1 --2- 172.21.15.32:0/3382622995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc11c106ba0 0x7fc11c19e0c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc114007ee0 tx=0x7fc11400d3b0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.662+0000 7fc121cd5700 1 -- 172.21.15.32:0/3382622995 shutdown_connections 2023-09-30T14:36:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.662+0000 7fc121cd5700 1 --2- 172.21.15.32:0/3382622995 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1040a0fa0 0x7fc1040a3460 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.662+0000 7fc121cd5700 1 --2- 172.21.15.32:0/3382622995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc11c106ba0 0x7fc11c19e0c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.662+0000 7fc121cd5700 1 --2- 172.21.15.32:0/3382622995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc11c101940 0x7fc11c196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.662+0000 7fc121cd5700 1 --2- 172.21.15.32:0/3382622995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc11c104270 0x7fc11c197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.662+0000 7fc121cd5700 1 -- 172.21.15.32:0/3382622995 >> 172.21.15.32:0/3382622995 conn(0x7fc11c0fb2f0 msgr2=0x7fc11c1025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:40.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.662+0000 7fc121cd5700 1 -- 172.21.15.32:0/3382622995 shutdown_connections 2023-09-30T14:36:40.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.662+0000 7fc121cd5700 1 -- 172.21.15.32:0/3382622995 wait complete. 2023-09-30T14:36:40.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:36:40.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.799+0000 7fbd1d186700 1 -- 172.21.15.32:0/3514631855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd180ffd40 msgr2=0x7fbd18100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:40.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.799+0000 7fbd1d186700 1 --2- 172.21.15.32:0/3514631855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd180ffd40 0x7fbd18100160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbd000099c0 tx=0x7fbd00009cd0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.799+0000 7fbd1d186700 1 -- 172.21.15.32:0/3514631855 shutdown_connections 2023-09-30T14:36:40.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.799+0000 7fbd1d186700 1 --2- 172.21.15.32:0/3514631855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd181018d0 0x7fbd18108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.799+0000 7fbd1d186700 1 --2- 172.21.15.32:0/3514631855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd180ffd40 0x7fbd18100160 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.799+0000 7fbd1d186700 1 --2- 172.21.15.32:0/3514631855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd18100f30 0x7fbd18101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.799+0000 7fbd1d186700 1 -- 172.21.15.32:0/3514631855 >> 172.21.15.32:0/3514631855 conn(0x7fbd180fb270 msgr2=0x7fbd180fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:40.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.799+0000 7fbd1d186700 1 -- 172.21.15.32:0/3514631855 shutdown_connections 2023-09-30T14:36:40.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.799+0000 7fbd1d186700 1 -- 172.21.15.32:0/3514631855 wait complete. 2023-09-30T14:36:40.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.800+0000 7fbd1d186700 1 Processor -- start 2023-09-30T14:36:40.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.800+0000 7fbd1d186700 1 -- start start 2023-09-30T14:36:40.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.800+0000 7fbd1d186700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd180ffd40 0x7fbd18196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:40.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.800+0000 7fbd1d186700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd18100f30 0x7fbd18197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:40.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.800+0000 7fbd1d186700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd181018d0 0x7fbd1819e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:40.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.800+0000 7fbd1d186700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1819e670 con 0x7fbd18100f30 2023-09-30T14:36:40.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.800+0000 7fbd1d186700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1819e7b0 con 0x7fbd181018d0 2023-09-30T14:36:40.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.800+0000 7fbd1d186700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1819e940 con 0x7fbd180ffd40 2023-09-30T14:36:40.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.800+0000 7fbd1659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd18100f30 0x7fbd18197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:40.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.800+0000 7fbd1659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd18100f30 0x7fbd18197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33456/0 (socket says 172.21.15.32:33456) 2023-09-30T14:36:40.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.800+0000 7fbd1659c700 1 -- 172.21.15.32:0/224448867 learned_addr learned my addr 172.21.15.32:0/224448867 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:36:40.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.800+0000 7fbd16d9d700 1 --2- 172.21.15.32:0/224448867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd180ffd40 0x7fbd18196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:40.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.800+0000 7fbd1759e700 1 --2- 172.21.15.32:0/224448867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd181018d0 0x7fbd1819e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.801+0000 7fbd1659c700 1 -- 172.21.15.32:0/224448867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd180ffd40 msgr2=0x7fbd18196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.801+0000 7fbd1659c700 1 --2- 172.21.15.32:0/224448867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd180ffd40 0x7fbd18196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.801+0000 7fbd1659c700 1 -- 172.21.15.32:0/224448867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd181018d0 msgr2=0x7fbd1819e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.801+0000 7fbd1659c700 1 --2- 172.21.15.32:0/224448867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd181018d0 0x7fbd1819e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.801+0000 7fbd1659c700 1 -- 172.21.15.32:0/224448867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd000095f0 con 0x7fbd18100f30 2023-09-30T14:36:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.801+0000 7fbd1759e700 1 --2- 172.21.15.32:0/224448867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd181018d0 0x7fbd1819e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:36:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.801+0000 7fbd1659c700 1 --2- 172.21.15.32:0/224448867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd18100f30 0x7fbd18197030 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800e9a0 tx=0x7fbd0800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.801+0000 7fbd0ffff700 1 -- 172.21.15.32:0/224448867 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd08009790 con 0x7fbd18100f30 2023-09-30T14:36:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.801+0000 7fbd0ffff700 1 -- 172.21.15.32:0/224448867 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd0800cbd0 con 0x7fbd18100f30 2023-09-30T14:36:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.801+0000 7fbd1d186700 1 -- 172.21.15.32:0/224448867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1819ec20 con 0x7fbd18100f30 2023-09-30T14:36:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.801+0000 7fbd0ffff700 1 -- 172.21.15.32:0/224448867 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd08018680 con 0x7fbd18100f30 2023-09-30T14:36:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.801+0000 7fbd1d186700 1 -- 172.21.15.32:0/224448867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd1819f130 con 0x7fbd18100f30 2023-09-30T14:36:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.802+0000 7fbd1d186700 1 -- 172.21.15.32:0/224448867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1810b550 con 0x7fbd18100f30 2023-09-30T14:36:40.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.804+0000 7fbd0ffff700 1 -- 172.21.15.32:0/224448867 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fbd0800cd40 con 0x7fbd18100f30 2023-09-30T14:36:40.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.804+0000 7fbd0ffff700 1 --2- 172.21.15.32:0/224448867 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbd040a0ff0 0x7fbd040a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:40.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.804+0000 7fbd16d9d700 1 --2- 172.21.15.32:0/224448867 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbd040a0ff0 0x7fbd040a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:40.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.804+0000 7fbd0ffff700 1 -- 172.21.15.32:0/224448867 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbd08014070 con 0x7fbd18100f30 2023-09-30T14:36:40.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.805+0000 7fbd16d9d700 1 --2- 172.21.15.32:0/224448867 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbd040a0ff0 0x7fbd040a34b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009990 tx=0x7fbd00005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:40.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:40.808+0000 7fbd0ffff700 1 -- 172.21.15.32:0/224448867 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fbd08096410 con 0x7fbd18100f30 2023-09-30T14:36:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:41.055+0000 7fbd1d186700 1 -- 172.21.15.32:0/224448867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd1810b830 con 0x7fbd18100f30 2023-09-30T14:36:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:41.056+0000 7fbd0ffff700 1 -- 172.21.15.32:0/224448867 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fbd1810b830 con 0x7fbd18100f30 2023-09-30T14:36:41.057 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:36:41.057 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-09-30T14:36:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:41.058+0000 7fbd1d186700 1 -- 172.21.15.32:0/224448867 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbd040a0ff0 msgr2=0x7fbd040a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:41.058+0000 7fbd1d186700 1 --2- 172.21.15.32:0/224448867 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbd040a0ff0 0x7fbd040a34b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009990 tx=0x7fbd00005fb0 comp rx=0 tx=0).stop 2023-09-30T14:36:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:41.058+0000 7fbd1d186700 1 -- 172.21.15.32:0/224448867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd18100f30 msgr2=0x7fbd18197030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:41.058+0000 7fbd1d186700 1 --2- 172.21.15.32:0/224448867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd18100f30 0x7fbd18197030 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800e9a0 tx=0x7fbd0800ecb0 comp rx=0 tx=0).stop 2023-09-30T14:36:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:41.058+0000 7fbd1d186700 1 -- 172.21.15.32:0/224448867 shutdown_connections 2023-09-30T14:36:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:41.058+0000 7fbd1d186700 1 --2- 172.21.15.32:0/224448867 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbd040a0ff0 0x7fbd040a34b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:41.058+0000 7fbd1d186700 1 --2- 172.21.15.32:0/224448867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd181018d0 0x7fbd1819e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:41.058+0000 7fbd1d186700 1 --2- 172.21.15.32:0/224448867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd180ffd40 0x7fbd18196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:41.058+0000 7fbd1d186700 1 --2- 172.21.15.32:0/224448867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd18100f30 0x7fbd18197030 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:41.058+0000 7fbd1d186700 1 -- 172.21.15.32:0/224448867 >> 172.21.15.32:0/224448867 conn(0x7fbd180fb270 msgr2=0x7fbd180fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:41.058+0000 7fbd1d186700 1 -- 172.21.15.32:0/224448867 shutdown_connections 2023-09-30T14:36:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:41.058+0000 7fbd1d186700 1 -- 172.21.15.32:0/224448867 wait complete. 2023-09-30T14:36:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 1 2023-09-30T14:36:41.126 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 30 14:36:40 smithi156 systemd[1]: Starting Ceph mds.l for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:36:41.377 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 30 14:36:41 smithi156 podman[139587]: 2023-09-30T14:36:41.378 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 30 14:36:41 smithi156 bash[139587]: 1c6092d3b3d559b726427e048a478a8c96d29b4b023ba9e13dfb6cc48c2a1b0a 2023-09-30T14:36:41.378 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 30 14:36:41 smithi156 systemd[1]: Started Ceph mds.l for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:36:41.378 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 30 14:36:41 smithi156 ceph-mds[139626]: -- 172.21.15.156:0/290456872 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5650afa971e0 con 0x5650afa6c800 2023-09-30T14:36:41.378 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 30 14:36:41 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-l[139602]: starting mds.l at 2023-09-30T14:36:41.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:41 smithi032 ceph-mon[98592]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:41.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:41 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3382622995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:36:41.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:41 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/224448867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:41 smithi156 ceph-mon[103579]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:41 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3382622995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:36:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:41 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/224448867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:41 smithi148 ceph-mon[102355]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:41 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3382622995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:36:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:41 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/224448867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:42.593 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:42 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.156:6832/2184509909,v1:172.21.15.156:6833/2184509909] up:boot 2023-09-30T14:36:42.593 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:42 smithi148 ceph-mon[102355]: fsmap 1 up:standby 2023-09-30T14:36:42.593 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-30T14:36:42.594 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:42.594 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:42.594 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:42.594 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:42.594 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:42.594 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:42.594 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:42 smithi148 ceph-mon[102355]: Deploying daemon mds.k on smithi148 2023-09-30T14:36:42.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:42 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.156:6832/2184509909,v1:172.21.15.156:6833/2184509909] up:boot 2023-09-30T14:36:42.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:42 smithi032 ceph-mon[98592]: fsmap 1 up:standby 2023-09-30T14:36:42.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-30T14:36:42.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:42.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:42.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:42.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:42.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:42.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:42.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:42 smithi032 ceph-mon[98592]: Deploying daemon mds.k on smithi148 2023-09-30T14:36:42.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:42 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.156:6832/2184509909,v1:172.21.15.156:6833/2184509909] up:boot 2023-09-30T14:36:42.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:42 smithi156 ceph-mon[103579]: fsmap 1 up:standby 2023-09-30T14:36:42.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-30T14:36:42.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:42.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:42.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:42.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:42.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:42.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:42.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:42 smithi156 ceph-mon[103579]: Deploying daemon mds.k on smithi148 2023-09-30T14:36:42.859 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 30 14:36:42 smithi148 systemd[1]: Starting Ceph mds.k for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:36:43.072 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:36:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.212+0000 7f920b47c700 1 -- 172.21.15.32:0/4047941732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92040ffe60 msgr2=0x7f92041002e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.212+0000 7f920b47c700 1 --2- 172.21.15.32:0/4047941732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92040ffe60 0x7f92041002e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009990 tx=0x7f91f8009ca0 comp rx=0 tx=0).stop 2023-09-30T14:36:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.212+0000 7f920b47c700 1 -- 172.21.15.32:0/4047941732 shutdown_connections 2023-09-30T14:36:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.212+0000 7f920b47c700 1 --2- 172.21.15.32:0/4047941732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9204100820 0x7f92041010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:43.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.212+0000 7f920b47c700 1 --2- 172.21.15.32:0/4047941732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92040ff100 0x7f92040ff520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:43.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.212+0000 7f920b47c700 1 --2- 172.21.15.32:0/4047941732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92040ffe60 0x7f92041002e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:43.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.212+0000 7f920b47c700 1 -- 172.21.15.32:0/4047941732 >> 172.21.15.32:0/4047941732 conn(0x7f92040faa30 msgr2=0x7f92040fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:43.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.212+0000 7f920b47c700 1 -- 172.21.15.32:0/4047941732 shutdown_connections 2023-09-30T14:36:43.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.212+0000 7f920b47c700 1 -- 172.21.15.32:0/4047941732 wait complete. 2023-09-30T14:36:43.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.213+0000 7f920b47c700 1 Processor -- start 2023-09-30T14:36:43.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.213+0000 7f920b47c700 1 -- start start 2023-09-30T14:36:43.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.213+0000 7f920b47c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92040ff100 0x7f9204198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:43.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.213+0000 7f920b47c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92040ffe60 0x7f9204199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:43.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.213+0000 7f920b47c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9204100820 0x7f92041a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:43.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.213+0000 7f920b47c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92041a0830 con 0x7f9204100820 2023-09-30T14:36:43.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.213+0000 7f920b47c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92041a09a0 con 0x7f92040ffe60 2023-09-30T14:36:43.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.213+0000 7f920b47c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92041a0b60 con 0x7f92040ff100 2023-09-30T14:36:43.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.213+0000 7f9209a19700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9204100820 0x7f92041a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:43.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.213+0000 7f9209a19700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9204100820 0x7f92041a0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33472/0 (socket says 172.21.15.32:33472) 2023-09-30T14:36:43.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.213+0000 7f9209a19700 1 -- 172.21.15.32:0/1314602640 learned_addr learned my addr 172.21.15.32:0/1314602640 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:36:43.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.213+0000 7f9208a17700 1 --2- 172.21.15.32:0/1314602640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92040ffe60 0x7f9204199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:43.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.214+0000 7f9209218700 1 --2- 172.21.15.32:0/1314602640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92040ff100 0x7f9204198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:43.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.214+0000 7f9209a19700 1 -- 172.21.15.32:0/1314602640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92040ff100 msgr2=0x7f9204198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:43.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.214+0000 7f9209a19700 1 --2- 172.21.15.32:0/1314602640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92040ff100 0x7f9204198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:43.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.214+0000 7f9209a19700 1 -- 172.21.15.32:0/1314602640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92040ffe60 msgr2=0x7f9204199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:43.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.214+0000 7f9209a19700 1 --2- 172.21.15.32:0/1314602640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92040ffe60 0x7f9204199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:43.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.214+0000 7f9209a19700 1 -- 172.21.15.32:0/1314602640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f80095f0 con 0x7f9204100820 2023-09-30T14:36:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.214+0000 7f9209a19700 1 --2- 172.21.15.32:0/1314602640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9204100820 0x7f92041a0260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00ef30 tx=0x7f91fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.214+0000 7f91f67fc700 1 -- 172.21.15.32:0/1314602640 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc018070 con 0x7f9204100820 2023-09-30T14:36:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.214+0000 7f91f67fc700 1 -- 172.21.15.32:0/1314602640 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91fc00f040 con 0x7f9204100820 2023-09-30T14:36:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.214+0000 7f920b47c700 1 -- 172.21.15.32:0/1314602640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92041a0de0 con 0x7f9204100820 2023-09-30T14:36:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.214+0000 7f91f67fc700 1 -- 172.21.15.32:0/1314602640 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc00cdb0 con 0x7f9204100820 2023-09-30T14:36:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.214+0000 7f920b47c700 1 -- 172.21.15.32:0/1314602640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92041a1260 con 0x7f9204100820 2023-09-30T14:36:43.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.215+0000 7f920b47c700 1 -- 172.21.15.32:0/1314602640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920410aca0 con 0x7f9204100820 2023-09-30T14:36:43.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.216+0000 7f91f67fc700 1 -- 172.21.15.32:0/1314602640 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f91fc003a00 con 0x7f9204100820 2023-09-30T14:36:43.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.217+0000 7f91f67fc700 1 --2- 172.21.15.32:0/1314602640 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91f00a1070 0x7f91f00a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:43.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.217+0000 7f9209218700 1 --2- 172.21.15.32:0/1314602640 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91f00a1070 0x7f91f00a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:43.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.217+0000 7f91f67fc700 1 -- 172.21.15.32:0/1314602640 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f91fc0cbe00 con 0x7f9204100820 2023-09-30T14:36:43.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.217+0000 7f9209218700 1 --2- 172.21.15.32:0/1314602640 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91f00a1070 0x7f91f00a3530 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9200005950 tx=0x7f92000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:43.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.221+0000 7f91f67fc700 1 -- 172.21.15.32:0/1314602640 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f91fc095b30 con 0x7f9204100820 2023-09-30T14:36:43.359 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 30 14:36:43 smithi148 podman[137464]: 2023-09-30T14:36:43.359 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 30 14:36:43 smithi148 ceph-mds[137504]: -- 172.21.15.148:0/2366438607 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c660af11e0 con 0x55c660ac6c00 2023-09-30T14:36:43.359 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 30 14:36:43 smithi148 bash[137464]: 1e5c0d5465e94090c1c975034b109888cb91a2ccc3a34bab6d6d5fa12e590062 2023-09-30T14:36:43.359 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 30 14:36:43 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-k[137480]: starting mds.k at 2023-09-30T14:36:43.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.472+0000 7f920b47c700 1 -- 172.21.15.32:0/1314602640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f920410e890 con 0x7f9204100820 2023-09-30T14:36:43.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.473+0000 7f91f67fc700 1 -- 172.21.15.32:0/1314602640 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f91fc099150 con 0x7f9204100820 2023-09-30T14:36:43.473 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:36:43.474 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-09-30T14:36:43.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.475+0000 7f920b47c700 1 -- 172.21.15.32:0/1314602640 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91f00a1070 msgr2=0x7f91f00a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:43.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.475+0000 7f920b47c700 1 --2- 172.21.15.32:0/1314602640 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91f00a1070 0x7f91f00a3530 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9200005950 tx=0x7f92000058e0 comp rx=0 tx=0).stop 2023-09-30T14:36:43.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.475+0000 7f920b47c700 1 -- 172.21.15.32:0/1314602640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9204100820 msgr2=0x7f92041a0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:43.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.475+0000 7f920b47c700 1 --2- 172.21.15.32:0/1314602640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9204100820 0x7f92041a0260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00ef30 tx=0x7f91fc00c3a0 comp rx=0 tx=0).stop 2023-09-30T14:36:43.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.475+0000 7f920b47c700 1 -- 172.21.15.32:0/1314602640 shutdown_connections 2023-09-30T14:36:43.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.475+0000 7f920b47c700 1 --2- 172.21.15.32:0/1314602640 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91f00a1070 0x7f91f00a3530 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:43.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.475+0000 7f920b47c700 1 --2- 172.21.15.32:0/1314602640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9204100820 0x7f92041a0260 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:43.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.475+0000 7f920b47c700 1 --2- 172.21.15.32:0/1314602640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92040ff100 0x7f9204198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:43.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.475+0000 7f920b47c700 1 --2- 172.21.15.32:0/1314602640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92040ffe60 0x7f9204199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:43.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.475+0000 7f920b47c700 1 -- 172.21.15.32:0/1314602640 >> 172.21.15.32:0/1314602640 conn(0x7f92040faa30 msgr2=0x7f92040fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:43.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.475+0000 7f920b47c700 1 -- 172.21.15.32:0/1314602640 shutdown_connections 2023-09-30T14:36:43.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:43.475+0000 7f920b47c700 1 -- 172.21.15.32:0/1314602640 wait complete. 2023-09-30T14:36:43.477 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 3 2023-09-30T14:36:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:43 smithi156 ceph-mon[103579]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:43 smithi148 ceph-mon[102355]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:43.859 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 30 14:36:43 smithi148 systemd[1]: Started Ceph mds.k for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:36:43.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:43 smithi032 ceph-mon[98592]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:44.461 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:44 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.148:6832/636626698,v1:172.21.15.148:6833/636626698] up:boot 2023-09-30T14:36:44.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:44 smithi032 ceph-mon[98592]: fsmap 2 up:standby 2023-09-30T14:36:44.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-30T14:36:44.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:44.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:44 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1314602640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:44.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:44.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:44.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:44.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:44.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:44.463 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:44 smithi032 ceph-mon[98592]: Deploying daemon mds.j on smithi032 2023-09-30T14:36:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:44 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.148:6832/636626698,v1:172.21.15.148:6833/636626698] up:boot 2023-09-30T14:36:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:44 smithi156 ceph-mon[103579]: fsmap 2 up:standby 2023-09-30T14:36:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-30T14:36:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:44 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1314602640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:44.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:44.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:44 smithi156 ceph-mon[103579]: Deploying daemon mds.j on smithi032 2023-09-30T14:36:44.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:44 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.148:6832/636626698,v1:172.21.15.148:6833/636626698] up:boot 2023-09-30T14:36:44.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:44 smithi148 ceph-mon[102355]: fsmap 2 up:standby 2023-09-30T14:36:44.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-30T14:36:44.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:44.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:44 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1314602640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:44.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:44.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:44.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:44.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:44.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:44.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:44 smithi148 ceph-mon[102355]: Deploying daemon mds.j on smithi032 2023-09-30T14:36:45.388 INFO:journalctl@ceph.mds.j.smithi032.stdout:Sep 30 14:36:45 smithi032 systemd[1]: Starting Ceph mds.j for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:36:45.489 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:36:45.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.632+0000 7fd45da7c700 1 -- 172.21.15.32:0/430630455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd458106b50 msgr2=0x7fd458108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.632+0000 7fd45da7c700 1 --2- 172.21.15.32:0/430630455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd458106b50 0x7fd458108f80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd45000b0a0 tx=0x7fd45000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:36:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.632+0000 7fd45da7c700 1 -- 172.21.15.32:0/430630455 shutdown_connections 2023-09-30T14:36:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.632+0000 7fd45da7c700 1 --2- 172.21.15.32:0/430630455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd458106b50 0x7fd458108f80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.632+0000 7fd45da7c700 1 --2- 172.21.15.32:0/430630455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4581018f0 0x7fd458103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.632+0000 7fd45da7c700 1 --2- 172.21.15.32:0/430630455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd458104220 0x7fd458106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.632+0000 7fd45da7c700 1 -- 172.21.15.32:0/430630455 >> 172.21.15.32:0/430630455 conn(0x7fd4580fb280 msgr2=0x7fd4580fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.632+0000 7fd45da7c700 1 -- 172.21.15.32:0/430630455 shutdown_connections 2023-09-30T14:36:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.632+0000 7fd45da7c700 1 -- 172.21.15.32:0/430630455 wait complete. 2023-09-30T14:36:45.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.633+0000 7fd45da7c700 1 Processor -- start 2023-09-30T14:36:45.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.633+0000 7fd45da7c700 1 -- start start 2023-09-30T14:36:45.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.633+0000 7fd45da7c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4581018f0 0x7fd458196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.633+0000 7fd45da7c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd458104220 0x7fd458197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.633+0000 7fd45da7c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd458106b50 0x7fd45819e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.633+0000 7fd45da7c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45819e6b0 con 0x7fd458104220 2023-09-30T14:36:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.633+0000 7fd45da7c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45819e7f0 con 0x7fd4581018f0 2023-09-30T14:36:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.633+0000 7fd45da7c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45819e980 con 0x7fd458106b50 2023-09-30T14:36:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.633+0000 7fd456ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd458104220 0x7fd458197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.633+0000 7fd456ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd458104220 0x7fd458197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33484/0 (socket says 172.21.15.32:33484) 2023-09-30T14:36:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.633+0000 7fd456ffd700 1 -- 172.21.15.32:0/351407410 learned_addr learned my addr 172.21.15.32:0/351407410 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:36:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.633+0000 7fd4577fe700 1 --2- 172.21.15.32:0/351407410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4581018f0 0x7fd458196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.633+0000 7fd457fff700 1 --2- 172.21.15.32:0/351407410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd458106b50 0x7fd45819e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.634+0000 7fd456ffd700 1 -- 172.21.15.32:0/351407410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd458106b50 msgr2=0x7fd45819e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:45.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.634+0000 7fd456ffd700 1 --2- 172.21.15.32:0/351407410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd458106b50 0x7fd45819e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:45.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.634+0000 7fd456ffd700 1 -- 172.21.15.32:0/351407410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4581018f0 msgr2=0x7fd458196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:45.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.634+0000 7fd456ffd700 1 --2- 172.21.15.32:0/351407410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4581018f0 0x7fd458196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:45.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.634+0000 7fd456ffd700 1 -- 172.21.15.32:0/351407410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd450009d00 con 0x7fd458104220 2023-09-30T14:36:45.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.634+0000 7fd456ffd700 1 --2- 172.21.15.32:0/351407410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd458104220 0x7fd458197070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd44000c9b0 tx=0x7fd44000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:45.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.634+0000 7fd454ff9700 1 -- 172.21.15.32:0/351407410 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd440007b00 con 0x7fd458104220 2023-09-30T14:36:45.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.634+0000 7fd454ff9700 1 -- 172.21.15.32:0/351407410 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd44000ce50 con 0x7fd458104220 2023-09-30T14:36:45.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.634+0000 7fd45da7c700 1 -- 172.21.15.32:0/351407410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45819ec60 con 0x7fd458104220 2023-09-30T14:36:45.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.634+0000 7fd454ff9700 1 -- 172.21.15.32:0/351407410 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd440018660 con 0x7fd458104220 2023-09-30T14:36:45.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.634+0000 7fd45da7c700 1 -- 172.21.15.32:0/351407410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45819f1c0 con 0x7fd458104220 2023-09-30T14:36:45.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.635+0000 7fd45da7c700 1 -- 172.21.15.32:0/351407410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4580fce00 con 0x7fd458104220 2023-09-30T14:36:45.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.636+0000 7fd454ff9700 1 -- 172.21.15.32:0/351407410 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fd44000f450 con 0x7fd458104220 2023-09-30T14:36:45.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.637+0000 7fd454ff9700 1 --2- 172.21.15.32:0/351407410 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd4440a1070 0x7fd4440a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:45.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.637+0000 7fd454ff9700 1 -- 172.21.15.32:0/351407410 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd4400cc110 con 0x7fd458104220 2023-09-30T14:36:45.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.637+0000 7fd4577fe700 1 --2- 172.21.15.32:0/351407410 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd4440a1070 0x7fd4440a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:45.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.637+0000 7fd4577fe700 1 --2- 172.21.15.32:0/351407410 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd4440a1070 0x7fd4440a3530 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd448009c60 tx=0x7fd448009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:45.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.641+0000 7fd454ff9700 1 -- 172.21.15.32:0/351407410 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fd440095db0 con 0x7fd458104220 2023-09-30T14:36:45.691 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:45 smithi032 ceph-mon[98592]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:45.692 INFO:journalctl@ceph.mds.j.smithi032.stdout:Sep 30 14:36:45 smithi032 podman[149188]: 2023-09-30T14:36:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:45 smithi156 ceph-mon[103579]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:45.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:45 smithi148 ceph-mon[102355]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.898+0000 7fd45da7c700 1 -- 172.21.15.32:0/351407410 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd45810c610 con 0x7fd458104220 2023-09-30T14:36:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.899+0000 7fd454ff9700 1 -- 172.21.15.32:0/351407410 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7fd440007e60 con 0x7fd458104220 2023-09-30T14:36:45.899 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:36:45.900 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-09-30T14:36:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.901+0000 7fd45da7c700 1 -- 172.21.15.32:0/351407410 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd4440a1070 msgr2=0x7fd4440a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.901+0000 7fd45da7c700 1 --2- 172.21.15.32:0/351407410 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd4440a1070 0x7fd4440a3530 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd448009c60 tx=0x7fd448009260 comp rx=0 tx=0).stop 2023-09-30T14:36:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.901+0000 7fd45da7c700 1 -- 172.21.15.32:0/351407410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd458104220 msgr2=0x7fd458197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.901+0000 7fd45da7c700 1 --2- 172.21.15.32:0/351407410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd458104220 0x7fd458197070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd44000c9b0 tx=0x7fd44000ccc0 comp rx=0 tx=0).stop 2023-09-30T14:36:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.901+0000 7fd45da7c700 1 -- 172.21.15.32:0/351407410 shutdown_connections 2023-09-30T14:36:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.901+0000 7fd45da7c700 1 --2- 172.21.15.32:0/351407410 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd4440a1070 0x7fd4440a3530 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.901+0000 7fd45da7c700 1 --2- 172.21.15.32:0/351407410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd458106b50 0x7fd45819e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.901+0000 7fd45da7c700 1 --2- 172.21.15.32:0/351407410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4581018f0 0x7fd458196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.901+0000 7fd45da7c700 1 --2- 172.21.15.32:0/351407410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd458104220 0x7fd458197070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.901+0000 7fd45da7c700 1 -- 172.21.15.32:0/351407410 >> 172.21.15.32:0/351407410 conn(0x7fd4580fb280 msgr2=0x7fd458102560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:45.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.901+0000 7fd45da7c700 1 -- 172.21.15.32:0/351407410 shutdown_connections 2023-09-30T14:36:45.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:45.901+0000 7fd45da7c700 1 -- 172.21.15.32:0/351407410 wait complete. 2023-09-30T14:36:45.903 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 3 2023-09-30T14:36:46.137 INFO:journalctl@ceph.mds.j.smithi032.stdout:Sep 30 14:36:45 smithi032 bash[149188]: 5eabcc2d9d9d0942bae53b2085b81a63fd2f769333246534373c0bd39625f105 2023-09-30T14:36:46.137 INFO:journalctl@ceph.mds.j.smithi032.stdout:Sep 30 14:36:45 smithi032 ceph-mds[149271]: -- 172.21.15.32:0/2243912498 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c232c831e0 con 0x55c232c58c00 2023-09-30T14:36:46.138 INFO:journalctl@ceph.mds.j.smithi032.stdout:Sep 30 14:36:45 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-j[149204]: starting mds.j at 2023-09-30T14:36:46.138 INFO:journalctl@ceph.mds.j.smithi032.stdout:Sep 30 14:36:45 smithi032 systemd[1]: Started Ceph mds.j for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:36:47.015 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:46 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:47.015 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:46 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:47.015 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:46 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:47.015 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:46 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:47.015 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:46 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:47.015 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:46 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:47.015 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:46 smithi032 ceph-mon[98592]: Deploying daemon mds.d on smithi032 2023-09-30T14:36:47.015 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:46 smithi032 ceph-mon[98592]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:47.016 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:46 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/351407410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:47.016 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:46 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.32:6834/3687429810,v1:172.21.15.32:6835/3687429810] up:boot 2023-09-30T14:36:47.016 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:46 smithi032 ceph-mon[98592]: fsmap 3 up:standby 2023-09-30T14:36:47.016 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:46 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-30T14:36:47.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:46 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:47.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:46 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:47.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:46 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:47.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:46 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:47.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:46 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:47.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:46 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:47.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:46 smithi156 ceph-mon[103579]: Deploying daemon mds.d on smithi032 2023-09-30T14:36:47.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:46 smithi156 ceph-mon[103579]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:47.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:46 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/351407410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:47.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:46 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.32:6834/3687429810,v1:172.21.15.32:6835/3687429810] up:boot 2023-09-30T14:36:47.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:46 smithi156 ceph-mon[103579]: fsmap 3 up:standby 2023-09-30T14:36:47.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:46 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-30T14:36:47.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:46 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:47.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:46 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:47.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:46 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:47.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:46 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:47.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:46 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:47.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:46 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:47.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:46 smithi148 ceph-mon[102355]: Deploying daemon mds.d on smithi032 2023-09-30T14:36:47.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:46 smithi148 ceph-mon[102355]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:47.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:46 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/351407410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:47.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:46 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.32:6834/3687429810,v1:172.21.15.32:6835/3687429810] up:boot 2023-09-30T14:36:47.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:46 smithi148 ceph-mon[102355]: fsmap 3 up:standby 2023-09-30T14:36:47.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:46 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-30T14:36:47.536 INFO:journalctl@ceph.mds.d.smithi032.stdout:Sep 30 14:36:47 smithi032 systemd[1]: Starting Ceph mds.d for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:36:47.808 INFO:journalctl@ceph.mds.d.smithi032.stdout:Sep 30 14:36:47 smithi032 podman[149579]: 2023-09-30T14:36:47.915 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:36:48.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.048+0000 7f1f59b8f700 1 -- 172.21.15.32:0/614370367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f541018e0 msgr2=0x7f1f54103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:48.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.048+0000 7f1f59b8f700 1 --2- 172.21.15.32:0/614370367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f541018e0 0x7f1f54103cd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00b0a0 tx=0x7f1f4c00b3b0 comp rx=0 tx=0).stop 2023-09-30T14:36:48.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.048+0000 7f1f59b8f700 1 -- 172.21.15.32:0/614370367 shutdown_connections 2023-09-30T14:36:48.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.048+0000 7f1f59b8f700 1 --2- 172.21.15.32:0/614370367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f54106b40 0x7f1f54108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:48.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.048+0000 7f1f59b8f700 1 --2- 172.21.15.32:0/614370367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f541018e0 0x7f1f54103cd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:48.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.048+0000 7f1f59b8f700 1 --2- 172.21.15.32:0/614370367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f54104210 0x7f1f54106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:48.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.048+0000 7f1f59b8f700 1 -- 172.21.15.32:0/614370367 >> 172.21.15.32:0/614370367 conn(0x7f1f540fb2b0 msgr2=0x7f1f540fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:48.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.048+0000 7f1f59b8f700 1 -- 172.21.15.32:0/614370367 shutdown_connections 2023-09-30T14:36:48.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.048+0000 7f1f59b8f700 1 -- 172.21.15.32:0/614370367 wait complete. 2023-09-30T14:36:48.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.049+0000 7f1f59b8f700 1 Processor -- start 2023-09-30T14:36:48.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.049+0000 7f1f59b8f700 1 -- start start 2023-09-30T14:36:48.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.049+0000 7f1f59b8f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f541018e0 0x7f1f54196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:48.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.049+0000 7f1f59b8f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f54104210 0x7f1f541971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:48.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.049+0000 7f1f59b8f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f54106b40 0x7f1f5419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:48.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.049+0000 7f1f59b8f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5419e800 con 0x7f1f541018e0 2023-09-30T14:36:48.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.049+0000 7f1f59b8f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5419e970 con 0x7f1f54106b40 2023-09-30T14:36:48.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.049+0000 7f1f59b8f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5419eb30 con 0x7f1f54104210 2023-09-30T14:36:48.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.049+0000 7f1f537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f541018e0 0x7f1f54196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:48.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.049+0000 7f1f537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f541018e0 0x7f1f54196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33522/0 (socket says 172.21.15.32:33522) 2023-09-30T14:36:48.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.049+0000 7f1f537fe700 1 -- 172.21.15.32:0/3372588458 learned_addr learned my addr 172.21.15.32:0/3372588458 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:36:48.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.049+0000 7f1f53fff700 1 --2- 172.21.15.32:0/3372588458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f54106b40 0x7f1f5419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:48.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.049+0000 7f1f52ffd700 1 --2- 172.21.15.32:0/3372588458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f54104210 0x7f1f541971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:48.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.050+0000 7f1f537fe700 1 -- 172.21.15.32:0/3372588458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f54104210 msgr2=0x7f1f541971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:48.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.050+0000 7f1f537fe700 1 --2- 172.21.15.32:0/3372588458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f54104210 0x7f1f541971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:48.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.050+0000 7f1f537fe700 1 -- 172.21.15.32:0/3372588458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f54106b40 msgr2=0x7f1f5419e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:48.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.050+0000 7f1f537fe700 1 --2- 172.21.15.32:0/3372588458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f54106b40 0x7f1f5419e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:48.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.050+0000 7f1f537fe700 1 -- 172.21.15.32:0/3372588458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c009d00 con 0x7f1f541018e0 2023-09-30T14:36:48.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.050+0000 7f1f53fff700 1 --2- 172.21.15.32:0/3372588458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f54106b40 0x7f1f5419e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:36:48.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.050+0000 7f1f537fe700 1 --2- 172.21.15.32:0/3372588458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f541018e0 0x7f1f54196c90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00b070 tx=0x7f1f4c004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.050+0000 7f1f50ff9700 1 -- 172.21.15.32:0/3372588458 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00e040 con 0x7f1f541018e0 2023-09-30T14:36:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.050+0000 7f1f50ff9700 1 -- 172.21.15.32:0/3372588458 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f4c009440 con 0x7f1f541018e0 2023-09-30T14:36:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.050+0000 7f1f59b8f700 1 -- 172.21.15.32:0/3372588458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5419edb0 con 0x7f1f541018e0 2023-09-30T14:36:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.050+0000 7f1f50ff9700 1 -- 172.21.15.32:0/3372588458 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c004020 con 0x7f1f541018e0 2023-09-30T14:36:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.050+0000 7f1f59b8f700 1 -- 172.21.15.32:0/3372588458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5419f200 con 0x7f1f541018e0 2023-09-30T14:36:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.052+0000 7f1f59b8f700 1 -- 172.21.15.32:0/3372588458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f540fce30 con 0x7f1f541018e0 2023-09-30T14:36:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.052+0000 7f1f50ff9700 1 -- 172.21.15.32:0/3372588458 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f1f4c01d690 con 0x7f1f541018e0 2023-09-30T14:36:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.053+0000 7f1f50ff9700 1 --2- 172.21.15.32:0/3372588458 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1f3c0a0ff0 0x7f1f3c0a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.053+0000 7f1f52ffd700 1 --2- 172.21.15.32:0/3372588458 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1f3c0a0ff0 0x7f1f3c0a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.053+0000 7f1f50ff9700 1 -- 172.21.15.32:0/3372588458 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1f4c0cdb70 con 0x7f1f541018e0 2023-09-30T14:36:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.053+0000 7f1f52ffd700 1 --2- 172.21.15.32:0/3372588458 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1f3c0a0ff0 0x7f1f3c0a34b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1f48005950 tx=0x7f1f480058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:48.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.057+0000 7f1f50ff9700 1 -- 172.21.15.32:0/3372588458 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f1f4c097800 con 0x7f1f541018e0 2023-09-30T14:36:48.137 INFO:journalctl@ceph.mds.d.smithi032.stdout:Sep 30 14:36:47 smithi032 bash[149579]: 4c655f88641f4bc01962994a4b94c094c71a144be006164759bc8da742999c3a 2023-09-30T14:36:48.137 INFO:journalctl@ceph.mds.d.smithi032.stdout:Sep 30 14:36:47 smithi032 ceph-mds[149620]: -- 172.21.15.32:0/343003628 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558750f071e0 con 0x558750edc800 2023-09-30T14:36:48.137 INFO:journalctl@ceph.mds.d.smithi032.stdout:Sep 30 14:36:47 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-d[149595]: starting mds.d at 2023-09-30T14:36:48.137 INFO:journalctl@ceph.mds.d.smithi032.stdout:Sep 30 14:36:47 smithi032 systemd[1]: Started Ceph mds.d for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:36:48.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.303+0000 7f1f59b8f700 1 -- 172.21.15.32:0/3372588458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f5402d0f0 con 0x7f1f541018e0 2023-09-30T14:36:48.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.304+0000 7f1f50ff9700 1 -- 172.21.15.32:0/3372588458 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7f1f4c021e90 con 0x7f1f541018e0 2023-09-30T14:36:48.305 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:36:48.305 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-09-30T14:36:48.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.306+0000 7f1f59b8f700 1 -- 172.21.15.32:0/3372588458 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1f3c0a0ff0 msgr2=0x7f1f3c0a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:48.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.306+0000 7f1f59b8f700 1 --2- 172.21.15.32:0/3372588458 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1f3c0a0ff0 0x7f1f3c0a34b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1f48005950 tx=0x7f1f480058e0 comp rx=0 tx=0).stop 2023-09-30T14:36:48.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.306+0000 7f1f59b8f700 1 -- 172.21.15.32:0/3372588458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f541018e0 msgr2=0x7f1f54196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:48.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.306+0000 7f1f59b8f700 1 --2- 172.21.15.32:0/3372588458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f541018e0 0x7f1f54196c90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00b070 tx=0x7f1f4c004db0 comp rx=0 tx=0).stop 2023-09-30T14:36:48.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.306+0000 7f1f59b8f700 1 -- 172.21.15.32:0/3372588458 shutdown_connections 2023-09-30T14:36:48.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.306+0000 7f1f59b8f700 1 --2- 172.21.15.32:0/3372588458 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1f3c0a0ff0 0x7f1f3c0a34b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:48.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.306+0000 7f1f59b8f700 1 --2- 172.21.15.32:0/3372588458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f54106b40 0x7f1f5419e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:48.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.306+0000 7f1f59b8f700 1 --2- 172.21.15.32:0/3372588458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f541018e0 0x7f1f54196c90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:48.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.306+0000 7f1f59b8f700 1 --2- 172.21.15.32:0/3372588458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f54104210 0x7f1f541971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:48.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.306+0000 7f1f59b8f700 1 -- 172.21.15.32:0/3372588458 >> 172.21.15.32:0/3372588458 conn(0x7f1f540fb2b0 msgr2=0x7f1f54102550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:48.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.306+0000 7f1f59b8f700 1 -- 172.21.15.32:0/3372588458 shutdown_connections 2023-09-30T14:36:48.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:48.306+0000 7f1f59b8f700 1 -- 172.21.15.32:0/3372588458 wait complete. 2023-09-30T14:36:48.308 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2023-09-30T14:36:49.182 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:48 smithi148 ceph-mon[102355]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:49.182 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:49.183 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:49.183 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:49.183 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:49.183 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:49.183 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:49.183 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:48 smithi148 ceph-mon[102355]: Deploying daemon mds.e on smithi148 2023-09-30T14:36:49.183 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:48 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.32:6836/2728444148,v1:172.21.15.32:6837/2728444148] up:boot 2023-09-30T14:36:49.183 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:48 smithi148 ceph-mon[102355]: fsmap 4 up:standby 2023-09-30T14:36:49.183 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:48 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-30T14:36:49.184 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:48 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3372588458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:49.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:48 smithi156 ceph-mon[103579]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:49.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:49.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:49.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:49.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:49.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:49.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:49.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:48 smithi156 ceph-mon[103579]: Deploying daemon mds.e on smithi148 2023-09-30T14:36:49.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:48 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.32:6836/2728444148,v1:172.21.15.32:6837/2728444148] up:boot 2023-09-30T14:36:49.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:48 smithi156 ceph-mon[103579]: fsmap 4 up:standby 2023-09-30T14:36:49.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:48 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-30T14:36:49.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:48 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3372588458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:48 smithi032 ceph-mon[98592]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:49.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:49.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:49.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:49.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:49.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:49.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:49.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:48 smithi032 ceph-mon[98592]: Deploying daemon mds.e on smithi148 2023-09-30T14:36:49.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:48 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.32:6836/2728444148,v1:172.21.15.32:6837/2728444148] up:boot 2023-09-30T14:36:49.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:48 smithi032 ceph-mon[98592]: fsmap 4 up:standby 2023-09-30T14:36:49.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:48 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-30T14:36:49.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:48 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3372588458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:49.609 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 30 14:36:49 smithi148 systemd[1]: Starting Ceph mds.e for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:36:50.016 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 30 14:36:49 smithi148 podman[137807]: 2023-09-30T14:36:50.016 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 30 14:36:49 smithi148 ceph-mds[137847]: -- 172.21.15.148:0/2842584724 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559e182a71e0 con 0x559e1827c800 2023-09-30T14:36:50.016 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 30 14:36:49 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-e[137823]: starting mds.e at 2023-09-30T14:36:50.016 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 30 14:36:50 smithi148 bash[137807]: a261c55a36996e27537548fe75fedaca7a619b5fd6964afd33c24aa4769c8b32 2023-09-30T14:36:50.320 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:36:50.359 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 30 14:36:50 smithi148 systemd[1]: Started Ceph mds.e for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:36:50.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.460+0000 7f840e042700 1 -- 172.21.15.32:0/552448201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84081018c0 msgr2=0x7f8408108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:50.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.460+0000 7f840e042700 1 --2- 172.21.15.32:0/552448201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84081018c0 0x7f8408108170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f840000b0d0 tx=0x7f840000b3e0 comp rx=0 tx=0).stop 2023-09-30T14:36:50.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.461+0000 7f840e042700 1 -- 172.21.15.32:0/552448201 shutdown_connections 2023-09-30T14:36:50.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.461+0000 7f840e042700 1 --2- 172.21.15.32:0/552448201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84081018c0 0x7f8408108170 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:50.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.461+0000 7f840e042700 1 --2- 172.21.15.32:0/552448201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84080ffd30 0x7f8408100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:50.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.461+0000 7f840e042700 1 --2- 172.21.15.32:0/552448201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8408100f20 0x7f8408101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:50.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.461+0000 7f840e042700 1 -- 172.21.15.32:0/552448201 >> 172.21.15.32:0/552448201 conn(0x7f84080fb280 msgr2=0x7f84080fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:50.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.461+0000 7f840e042700 1 -- 172.21.15.32:0/552448201 shutdown_connections 2023-09-30T14:36:50.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.461+0000 7f840e042700 1 -- 172.21.15.32:0/552448201 wait complete. 2023-09-30T14:36:50.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.461+0000 7f840e042700 1 Processor -- start 2023-09-30T14:36:50.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.461+0000 7f840e042700 1 -- start start 2023-09-30T14:36:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f840e042700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84080ffd30 0x7f8408103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f840e042700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8408100f20 0x7f8408102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f840e042700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84081018c0 0x7f8408102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f840e042700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8408102fe0 con 0x7f8408100f20 2023-09-30T14:36:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f840e042700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8408103150 con 0x7f84081018c0 2023-09-30T14:36:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f840e042700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84081a0c40 con 0x7f84080ffd30 2023-09-30T14:36:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f8406ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8408100f20 0x7f8408102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f8406ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8408100f20 0x7f8408102300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33538/0 (socket says 172.21.15.32:33538) 2023-09-30T14:36:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f8406ffd700 1 -- 172.21.15.32:0/2961464881 learned_addr learned my addr 172.21.15.32:0/2961464881 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:36:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f8407fff700 1 --2- 172.21.15.32:0/2961464881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84081018c0 0x7f8408102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f84077fe700 1 --2- 172.21.15.32:0/2961464881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84080ffd30 0x7f8408103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f8406ffd700 1 -- 172.21.15.32:0/2961464881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84080ffd30 msgr2=0x7f8408103cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f8406ffd700 1 --2- 172.21.15.32:0/2961464881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84080ffd30 0x7f8408103cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f8406ffd700 1 -- 172.21.15.32:0/2961464881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84081018c0 msgr2=0x7f8408102840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f8406ffd700 1 --2- 172.21.15.32:0/2961464881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84081018c0 0x7f8408102840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f8406ffd700 1 -- 172.21.15.32:0/2961464881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8400009d00 con 0x7f8408100f20 2023-09-30T14:36:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f8406ffd700 1 --2- 172.21.15.32:0/2961464881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8408100f20 0x7f8408102300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00d980 tx=0x7f83fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f8404ff9700 1 -- 172.21.15.32:0/2961464881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc004220 con 0x7f8408100f20 2023-09-30T14:36:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f840e042700 1 -- 172.21.15.32:0/2961464881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84081a0f20 con 0x7f8408100f20 2023-09-30T14:36:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f8404ff9700 1 -- 172.21.15.32:0/2961464881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83fc007640 con 0x7f8408100f20 2023-09-30T14:36:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f8404ff9700 1 -- 172.21.15.32:0/2961464881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc005690 con 0x7f8408100f20 2023-09-30T14:36:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.462+0000 7f840e042700 1 -- 172.21.15.32:0/2961464881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84081a13f0 con 0x7f8408100f20 2023-09-30T14:36:50.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.463+0000 7f840e042700 1 -- 172.21.15.32:0/2961464881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840810b550 con 0x7f8408100f20 2023-09-30T14:36:50.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.465+0000 7f8404ff9700 1 -- 172.21.15.32:0/2961464881 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f83fc004750 con 0x7f8408100f20 2023-09-30T14:36:50.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.465+0000 7f8404ff9700 1 --2- 172.21.15.32:0/2961464881 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f83f00a0ff0 0x7f83f00a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:50.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.465+0000 7f8404ff9700 1 -- 172.21.15.32:0/2961464881 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f83fc0cc4e0 con 0x7f8408100f20 2023-09-30T14:36:50.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.465+0000 7f84077fe700 1 --2- 172.21.15.32:0/2961464881 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f83f00a0ff0 0x7f83f00a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:50.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.466+0000 7f84077fe700 1 --2- 172.21.15.32:0/2961464881 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f83f00a0ff0 0x7f83f00a34b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8408100d80 tx=0x7f83f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:50.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.469+0000 7f8404ff9700 1 -- 172.21.15.32:0/2961464881 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f83fc096180 con 0x7f8408100f20 2023-09-30T14:36:50.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.714+0000 7f840e042700 1 -- 172.21.15.32:0/2961464881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f840810b770 con 0x7f8408100f20 2023-09-30T14:36:50.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.715+0000 7f8404ff9700 1 -- 172.21.15.32:0/2961464881 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7f83fc016e90 con 0x7f8408100f20 2023-09-30T14:36:50.716 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:36:50.716 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-09-30T14:36:50.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.717+0000 7f840e042700 1 -- 172.21.15.32:0/2961464881 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f83f00a0ff0 msgr2=0x7f83f00a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:50.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.717+0000 7f840e042700 1 --2- 172.21.15.32:0/2961464881 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f83f00a0ff0 0x7f83f00a34b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8408100d80 tx=0x7f83f8009260 comp rx=0 tx=0).stop 2023-09-30T14:36:50.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.717+0000 7f840e042700 1 -- 172.21.15.32:0/2961464881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8408100f20 msgr2=0x7f8408102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:50.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.717+0000 7f840e042700 1 --2- 172.21.15.32:0/2961464881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8408100f20 0x7f8408102300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00d980 tx=0x7f83fc00dc90 comp rx=0 tx=0).stop 2023-09-30T14:36:50.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.717+0000 7f840e042700 1 -- 172.21.15.32:0/2961464881 shutdown_connections 2023-09-30T14:36:50.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.717+0000 7f840e042700 1 --2- 172.21.15.32:0/2961464881 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f83f00a0ff0 0x7f83f00a34b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:50.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.717+0000 7f840e042700 1 --2- 172.21.15.32:0/2961464881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84081018c0 0x7f8408102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:50.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.717+0000 7f840e042700 1 --2- 172.21.15.32:0/2961464881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84080ffd30 0x7f8408103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:50.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.717+0000 7f840e042700 1 --2- 172.21.15.32:0/2961464881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8408100f20 0x7f8408102300 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:50.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.717+0000 7f840e042700 1 -- 172.21.15.32:0/2961464881 >> 172.21.15.32:0/2961464881 conn(0x7f84080fb280 msgr2=0x7f8408067ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:50.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.717+0000 7f840e042700 1 -- 172.21.15.32:0/2961464881 shutdown_connections 2023-09-30T14:36:50.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:50.717+0000 7f840e042700 1 -- 172.21.15.32:0/2961464881 wait complete. 2023-09-30T14:36:50.719 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 5 2023-09-30T14:36:51.288 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:51 smithi156 ceph-mon[103579]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:51.288 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:51.288 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:51.288 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:51.288 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:51.288 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:51.288 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:51.288 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:51 smithi156 ceph-mon[103579]: Deploying daemon mds.f on smithi156 2023-09-30T14:36:51.288 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:51 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2961464881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:51.288 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:51 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.148:6834/4151174061,v1:172.21.15.148:6835/4151174061] up:boot 2023-09-30T14:36:51.289 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:51 smithi156 ceph-mon[103579]: fsmap 5 up:standby 2023-09-30T14:36:51.289 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:51 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-30T14:36:51.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:51 smithi148 ceph-mon[102355]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:51.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:51 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:51.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:51 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:51.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:51 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:51.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:51 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:51.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:51 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:51.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:51 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:51.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:51 smithi148 ceph-mon[102355]: Deploying daemon mds.f on smithi156 2023-09-30T14:36:51.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:51 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2961464881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:51.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:51 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.148:6834/4151174061,v1:172.21.15.148:6835/4151174061] up:boot 2023-09-30T14:36:51.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:51 smithi148 ceph-mon[102355]: fsmap 5 up:standby 2023-09-30T14:36:51.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:51 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-30T14:36:51.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:51 smithi032 ceph-mon[98592]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:51.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:51 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:51.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:51 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:51.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:51 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:51.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:51 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:51.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:51 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:51.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:51 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:51.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:51 smithi032 ceph-mon[98592]: Deploying daemon mds.f on smithi156 2023-09-30T14:36:51.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:51 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2961464881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:51.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:51 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.148:6834/4151174061,v1:172.21.15.148:6835/4151174061] up:boot 2023-09-30T14:36:51.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:51 smithi032 ceph-mon[98592]: fsmap 5 up:standby 2023-09-30T14:36:51.389 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:51 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-30T14:36:51.933 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 30 14:36:51 smithi156 systemd[1]: Starting Ceph mds.f for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:36:52.203 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 30 14:36:51 smithi156 podman[139930]: 2023-09-30T14:36:52.203 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 30 14:36:52 smithi156 ceph-mds[139970]: -- 172.21.15.156:0/1957750848 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5627b0ec31e0 con 0x5627b0e98800 2023-09-30T14:36:52.203 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 30 14:36:52 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-f[139946]: starting mds.f at 2023-09-30T14:36:52.204 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 30 14:36:52 smithi156 bash[139930]: 41da5db4517606dc2c3e82ba28aa40f3b9c7988502f5660afe83930b346176f2 2023-09-30T14:36:52.555 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 30 14:36:52 smithi156 systemd[1]: Started Ceph mds.f for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:36:52.730 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:36:52.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.870+0000 7f1f00515700 1 -- 172.21.15.32:0/588374257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ef80ffd40 msgr2=0x7f1ef8100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:52.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.870+0000 7f1f00515700 1 --2- 172.21.15.32:0/588374257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ef80ffd40 0x7f1ef8100160 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40099c0 tx=0x7f1ef4009cd0 comp rx=0 tx=0).stop 2023-09-30T14:36:52.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.870+0000 7f1f00515700 1 -- 172.21.15.32:0/588374257 shutdown_connections 2023-09-30T14:36:52.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.870+0000 7f1f00515700 1 --2- 172.21.15.32:0/588374257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ef81018d0 0x7f1ef8108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:52.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.870+0000 7f1f00515700 1 --2- 172.21.15.32:0/588374257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ef80ffd40 0x7f1ef8100160 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:52.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.870+0000 7f1f00515700 1 --2- 172.21.15.32:0/588374257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef8100f30 0x7f1ef8101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:52.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.870+0000 7f1f00515700 1 -- 172.21.15.32:0/588374257 >> 172.21.15.32:0/588374257 conn(0x7f1ef80fb2b0 msgr2=0x7f1ef80fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:52.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.870+0000 7f1f00515700 1 -- 172.21.15.32:0/588374257 shutdown_connections 2023-09-30T14:36:52.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.870+0000 7f1f00515700 1 -- 172.21.15.32:0/588374257 wait complete. 2023-09-30T14:36:52.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.870+0000 7f1f00515700 1 Processor -- start 2023-09-30T14:36:52.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.870+0000 7f1f00515700 1 -- start start 2023-09-30T14:36:52.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1f00515700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ef80ffd40 0x7f1ef8069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:52.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1f00515700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ef8100f30 0x7f1ef819b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:52.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1f00515700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef81018d0 0x7f1ef819dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:52.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1f00515700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef819e660 con 0x7f1ef8100f30 2023-09-30T14:36:52.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1f00515700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef819e7d0 con 0x7f1ef81018d0 2023-09-30T14:36:52.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1f00515700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef819e990 con 0x7f1ef80ffd40 2023-09-30T14:36:52.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1efdab0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ef8100f30 0x7f1ef819b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:52.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1efe2b1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ef80ffd40 0x7f1ef8069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:52.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1efdab0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ef8100f30 0x7f1ef819b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33552/0 (socket says 172.21.15.32:33552) 2023-09-30T14:36:52.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1efdab0700 1 -- 172.21.15.32:0/579228408 learned_addr learned my addr 172.21.15.32:0/579228408 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:36:52.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1efeab2700 1 --2- 172.21.15.32:0/579228408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef81018d0 0x7f1ef819dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:52.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1efdab0700 1 -- 172.21.15.32:0/579228408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ef80ffd40 msgr2=0x7f1ef8069d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:52.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1efdab0700 1 --2- 172.21.15.32:0/579228408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ef80ffd40 0x7f1ef8069d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:52.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1efdab0700 1 -- 172.21.15.32:0/579228408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef81018d0 msgr2=0x7f1ef819dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:52.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1efdab0700 1 --2- 172.21.15.32:0/579228408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef81018d0 0x7f1ef819dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:52.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1efdab0700 1 -- 172.21.15.32:0/579228408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef40095f0 con 0x7f1ef8100f30 2023-09-30T14:36:52.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1efe2b1700 1 --2- 172.21.15.32:0/579228408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ef80ffd40 0x7f1ef8069d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:36:52.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1efdab0700 1 --2- 172.21.15.32:0/579228408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ef8100f30 0x7f1ef819b880 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800d950 tx=0x7f1ee800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:52.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1eef7fe700 1 -- 172.21.15.32:0/579228408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee800f890 con 0x7f1ef8100f30 2023-09-30T14:36:52.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.871+0000 7f1eef7fe700 1 -- 172.21.15.32:0/579228408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ee8010460 con 0x7f1ef8100f30 2023-09-30T14:36:52.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.872+0000 7f1f00515700 1 -- 172.21.15.32:0/579228408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef819ec70 con 0x7f1ef8100f30 2023-09-30T14:36:52.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.872+0000 7f1eef7fe700 1 -- 172.21.15.32:0/579228408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee800e610 con 0x7f1ef8100f30 2023-09-30T14:36:52.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.872+0000 7f1f00515700 1 -- 172.21.15.32:0/579228408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef819f1d0 con 0x7f1ef8100f30 2023-09-30T14:36:52.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.872+0000 7f1f00515700 1 -- 172.21.15.32:0/579228408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef810b5a0 con 0x7f1ef8100f30 2023-09-30T14:36:52.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.874+0000 7f1eef7fe700 1 -- 172.21.15.32:0/579228408 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f1ee80108d0 con 0x7f1ef8100f30 2023-09-30T14:36:52.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.874+0000 7f1eef7fe700 1 --2- 172.21.15.32:0/579228408 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ee40a1070 0x7f1ee40a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:52.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.874+0000 7f1efe2b1700 1 --2- 172.21.15.32:0/579228408 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ee40a1070 0x7f1ee40a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:52.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.874+0000 7f1eef7fe700 1 -- 172.21.15.32:0/579228408 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1ee80cbf70 con 0x7f1ef8100f30 2023-09-30T14:36:52.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.875+0000 7f1efe2b1700 1 --2- 172.21.15.32:0/579228408 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ee40a1070 0x7f1ee40a3530 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009990 tx=0x7f1ef4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:52.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:52.878+0000 7f1eef7fe700 1 -- 172.21.15.32:0/579228408 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f1ee8095c10 con 0x7f1ef8100f30 2023-09-30T14:36:53.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:53.129+0000 7f1f00515700 1 -- 172.21.15.32:0/579228408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ef8106c60 con 0x7f1ef8100f30 2023-09-30T14:36:53.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:53.129+0000 7f1eef7fe700 1 -- 172.21.15.32:0/579228408 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7f1ee8099230 con 0x7f1ef8100f30 2023-09-30T14:36:53.131 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:36:53.131 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-09-30T14:36:53.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:53.131+0000 7f1f00515700 1 -- 172.21.15.32:0/579228408 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ee40a1070 msgr2=0x7f1ee40a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:53.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:53.131+0000 7f1f00515700 1 --2- 172.21.15.32:0/579228408 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ee40a1070 0x7f1ee40a3530 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009990 tx=0x7f1ef4005fb0 comp rx=0 tx=0).stop 2023-09-30T14:36:53.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:53.131+0000 7f1f00515700 1 -- 172.21.15.32:0/579228408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ef8100f30 msgr2=0x7f1ef819b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:53.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:53.131+0000 7f1f00515700 1 --2- 172.21.15.32:0/579228408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ef8100f30 0x7f1ef819b880 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800d950 tx=0x7f1ee800dc60 comp rx=0 tx=0).stop 2023-09-30T14:36:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:53.131+0000 7f1f00515700 1 -- 172.21.15.32:0/579228408 shutdown_connections 2023-09-30T14:36:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:53.131+0000 7f1f00515700 1 --2- 172.21.15.32:0/579228408 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ee40a1070 0x7f1ee40a3530 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:53.131+0000 7f1f00515700 1 --2- 172.21.15.32:0/579228408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef81018d0 0x7f1ef819dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:53.131+0000 7f1f00515700 1 --2- 172.21.15.32:0/579228408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ef80ffd40 0x7f1ef8069d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:53.131+0000 7f1f00515700 1 --2- 172.21.15.32:0/579228408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ef8100f30 0x7f1ef819b880 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:53.131+0000 7f1f00515700 1 -- 172.21.15.32:0/579228408 >> 172.21.15.32:0/579228408 conn(0x7f1ef80fb2b0 msgr2=0x7f1ef80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:53.131+0000 7f1f00515700 1 -- 172.21.15.32:0/579228408 shutdown_connections 2023-09-30T14:36:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:53.132+0000 7f1f00515700 1 -- 172.21.15.32:0/579228408 wait complete. 2023-09-30T14:36:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 7 2023-09-30T14:36:53.452 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:53 smithi156 ceph-mon[103579]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:53.452 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:53 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:53.452 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:53 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:53.452 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:53 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:53.453 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:53 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:53.453 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:53 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:53.453 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:53 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:53.453 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:53 smithi156 ceph-mon[103579]: Deploying daemon mds.c on smithi156 2023-09-30T14:36:53.453 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:53 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.156:6834/4180205116,v1:172.21.15.156:6835/4180205116] up:boot 2023-09-30T14:36:53.453 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:53 smithi156 ceph-mon[103579]: fsmap 6 up:standby 2023-09-30T14:36:53.453 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:53 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-30T14:36:53.453 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:53 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/579228408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:53.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:53 smithi148 ceph-mon[102355]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:53.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:53 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:53.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:53 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:53.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:53 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:53.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:53 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:53.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:53 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:53.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:53 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:53.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:53 smithi148 ceph-mon[102355]: Deploying daemon mds.c on smithi156 2023-09-30T14:36:53.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:53 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.156:6834/4180205116,v1:172.21.15.156:6835/4180205116] up:boot 2023-09-30T14:36:53.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:53 smithi148 ceph-mon[102355]: fsmap 6 up:standby 2023-09-30T14:36:53.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:53 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-30T14:36:53.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:53 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/579228408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:53.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:53 smithi032 ceph-mon[98592]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:53.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:53 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:53.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:53 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:53.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:53 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:53.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:53 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:53.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:53 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:53.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:53 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:53.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:53 smithi032 ceph-mon[98592]: Deploying daemon mds.c on smithi156 2023-09-30T14:36:53.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:53 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.156:6834/4180205116,v1:172.21.15.156:6835/4180205116] up:boot 2023-09-30T14:36:53.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:53 smithi032 ceph-mon[98592]: fsmap 6 up:standby 2023-09-30T14:36:53.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:53 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-30T14:36:53.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:53 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/579228408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:53.780 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 30 14:36:53 smithi156 systemd[1]: Starting Ceph mds.c for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:36:54.368 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 30 14:36:54 smithi156 podman[140274]: 2023-09-30T14:36:54.368 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 30 14:36:54 smithi156 ceph-mds[140314]: -- 172.21.15.156:0/3588663875 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d73bdad1e0 con 0x55d73bd83000 2023-09-30T14:36:54.368 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 30 14:36:54 smithi156 bash[140274]: d48c672e0ab4c30cd1dbcd3a27e113fba7df4386d4d3b7c2938b705355aa363e 2023-09-30T14:36:54.368 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 30 14:36:54 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-c[140290]: starting mds.c at 2023-09-30T14:36:54.805 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 30 14:36:54 smithi156 systemd[1]: Started Ceph mds.c for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:36:55.145 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:36:55.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.285+0000 7f4f76153700 1 -- 172.21.15.32:0/629715944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f700ffd30 msgr2=0x7f4f70100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.285+0000 7f4f76153700 1 --2- 172.21.15.32:0/629715944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f700ffd30 0x7f4f70100150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4f600099c0 tx=0x7f4f60009cd0 comp rx=0 tx=0).stop 2023-09-30T14:36:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.285+0000 7f4f76153700 1 -- 172.21.15.32:0/629715944 shutdown_connections 2023-09-30T14:36:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.285+0000 7f4f76153700 1 --2- 172.21.15.32:0/629715944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f701018c0 0x7f4f70108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.285+0000 7f4f76153700 1 --2- 172.21.15.32:0/629715944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f700ffd30 0x7f4f70100150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.285+0000 7f4f76153700 1 --2- 172.21.15.32:0/629715944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f70100f20 0x7f4f70101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.285+0000 7f4f76153700 1 -- 172.21.15.32:0/629715944 >> 172.21.15.32:0/629715944 conn(0x7f4f700fb280 msgr2=0x7f4f700fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.285+0000 7f4f76153700 1 -- 172.21.15.32:0/629715944 shutdown_connections 2023-09-30T14:36:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.285+0000 7f4f76153700 1 -- 172.21.15.32:0/629715944 wait complete. 2023-09-30T14:36:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.286+0000 7f4f76153700 1 Processor -- start 2023-09-30T14:36:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.286+0000 7f4f76153700 1 -- start start 2023-09-30T14:36:55.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.286+0000 7f4f76153700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f700ffd30 0x7f4f70196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:55.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.286+0000 7f4f76153700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f70100f20 0x7f4f70197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:55.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.286+0000 7f4f76153700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f701018c0 0x7f4f7019e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:55.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.286+0000 7f4f76153700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7019e670 con 0x7f4f701018c0 2023-09-30T14:36:55.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.286+0000 7f4f76153700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7019e7b0 con 0x7f4f700ffd30 2023-09-30T14:36:55.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.286+0000 7f4f76153700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7019e940 con 0x7f4f70100f20 2023-09-30T14:36:55.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.286+0000 7f4f6ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f701018c0 0x7f4f7019e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.286+0000 7f4f6ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f701018c0 0x7f4f7019e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33568/0 (socket says 172.21.15.32:33568) 2023-09-30T14:36:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.286+0000 7f4f6ffff700 1 -- 172.21.15.32:0/480687529 learned_addr learned my addr 172.21.15.32:0/480687529 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:36:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.286+0000 7f4f6f7fe700 1 --2- 172.21.15.32:0/480687529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f700ffd30 0x7f4f70196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.287+0000 7f4f6effd700 1 --2- 172.21.15.32:0/480687529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f70100f20 0x7f4f70197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.287+0000 7f4f6ffff700 1 -- 172.21.15.32:0/480687529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f70100f20 msgr2=0x7f4f70197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.287+0000 7f4f6ffff700 1 --2- 172.21.15.32:0/480687529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f70100f20 0x7f4f70197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.287+0000 7f4f6ffff700 1 -- 172.21.15.32:0/480687529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f700ffd30 msgr2=0x7f4f70196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.287+0000 7f4f6ffff700 1 --2- 172.21.15.32:0/480687529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f700ffd30 0x7f4f70196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.287+0000 7f4f6ffff700 1 -- 172.21.15.32:0/480687529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f600095f0 con 0x7f4f701018c0 2023-09-30T14:36:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.287+0000 7f4f6effd700 1 --2- 172.21.15.32:0/480687529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f70100f20 0x7f4f70197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:36:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.287+0000 7f4f6ffff700 1 --2- 172.21.15.32:0/480687529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f701018c0 0x7f4f7019e0a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800ef30 tx=0x7f4f6800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.287+0000 7f4f6cff9700 1 -- 172.21.15.32:0/480687529 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f68018070 con 0x7f4f701018c0 2023-09-30T14:36:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.287+0000 7f4f6cff9700 1 -- 172.21.15.32:0/480687529 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f6800f040 con 0x7f4f701018c0 2023-09-30T14:36:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.287+0000 7f4f76153700 1 -- 172.21.15.32:0/480687529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7019ebc0 con 0x7f4f701018c0 2023-09-30T14:36:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.287+0000 7f4f6cff9700 1 -- 172.21.15.32:0/480687529 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6800cdb0 con 0x7f4f701018c0 2023-09-30T14:36:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.287+0000 7f4f76153700 1 -- 172.21.15.32:0/480687529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7019f080 con 0x7f4f701018c0 2023-09-30T14:36:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.288+0000 7f4f76153700 1 -- 172.21.15.32:0/480687529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f7010c650 con 0x7f4f701018c0 2023-09-30T14:36:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.289+0000 7f4f6cff9700 1 -- 172.21.15.32:0/480687529 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f4f68003a00 con 0x7f4f701018c0 2023-09-30T14:36:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.290+0000 7f4f6cff9700 1 --2- 172.21.15.32:0/480687529 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4f580a0ff0 0x7f4f580a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.290+0000 7f4f6f7fe700 1 --2- 172.21.15.32:0/480687529 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4f580a0ff0 0x7f4f580a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.290+0000 7f4f6cff9700 1 -- 172.21.15.32:0/480687529 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4f680cc8b0 con 0x7f4f701018c0 2023-09-30T14:36:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.290+0000 7f4f6f7fe700 1 --2- 172.21.15.32:0/480687529 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4f580a0ff0 0x7f4f580a34b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009990 tx=0x7f4f60005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:55.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.293+0000 7f4f6cff9700 1 -- 172.21.15.32:0/480687529 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f4f68096540 con 0x7f4f701018c0 2023-09-30T14:36:55.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.545+0000 7f4f76153700 1 -- 172.21.15.32:0/480687529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f70106c70 con 0x7f4f701018c0 2023-09-30T14:36:55.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.546+0000 7f4f6cff9700 1 -- 172.21.15.32:0/480687529 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6080 (secure 0 0 0) 0x7f4f68099b60 con 0x7f4f701018c0 2023-09-30T14:36:55.547 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:36:55.547 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-09-30T14:36:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.548+0000 7f4f76153700 1 -- 172.21.15.32:0/480687529 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4f580a0ff0 msgr2=0x7f4f580a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.548+0000 7f4f76153700 1 --2- 172.21.15.32:0/480687529 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4f580a0ff0 0x7f4f580a34b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009990 tx=0x7f4f60005fb0 comp rx=0 tx=0).stop 2023-09-30T14:36:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.548+0000 7f4f76153700 1 -- 172.21.15.32:0/480687529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f701018c0 msgr2=0x7f4f7019e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.548+0000 7f4f76153700 1 --2- 172.21.15.32:0/480687529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f701018c0 0x7f4f7019e0a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800ef30 tx=0x7f4f6800c3a0 comp rx=0 tx=0).stop 2023-09-30T14:36:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.548+0000 7f4f76153700 1 -- 172.21.15.32:0/480687529 shutdown_connections 2023-09-30T14:36:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.548+0000 7f4f76153700 1 --2- 172.21.15.32:0/480687529 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4f580a0ff0 0x7f4f580a34b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.548+0000 7f4f76153700 1 --2- 172.21.15.32:0/480687529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f701018c0 0x7f4f7019e0a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.548+0000 7f4f76153700 1 --2- 172.21.15.32:0/480687529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f700ffd30 0x7f4f70196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.548+0000 7f4f76153700 1 --2- 172.21.15.32:0/480687529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f70100f20 0x7f4f70197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.548+0000 7f4f76153700 1 -- 172.21.15.32:0/480687529 >> 172.21.15.32:0/480687529 conn(0x7f4f700fb280 msgr2=0x7f4f700fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.548+0000 7f4f76153700 1 -- 172.21.15.32:0/480687529 shutdown_connections 2023-09-30T14:36:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:55.548+0000 7f4f76153700 1 -- 172.21.15.32:0/480687529 wait complete. 2023-09-30T14:36:55.550 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-09-30T14:36:55.560 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:55 smithi032 ceph-mon[98592]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:55.560 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:55 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:55.560 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:55 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:55.560 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:55 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:55.561 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:55 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:55.561 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:55 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:55.561 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:55 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:55.561 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:55 smithi032 ceph-mon[98592]: Deploying daemon mds.h on smithi148 2023-09-30T14:36:55.561 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:55 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.156:6836/445372881,v1:172.21.15.156:6837/445372881] up:boot 2023-09-30T14:36:55.561 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:55 smithi032 ceph-mon[98592]: fsmap 7 up:standby 2023-09-30T14:36:55.561 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:55 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-30T14:36:55.732 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:55 smithi148 ceph-mon[102355]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:55.732 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:55 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:55.732 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:55 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:55.732 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:55 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:55.732 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:55 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:55.732 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:55 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:55.732 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:55 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:55.732 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:55 smithi148 ceph-mon[102355]: Deploying daemon mds.h on smithi148 2023-09-30T14:36:55.732 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:55 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.156:6836/445372881,v1:172.21.15.156:6837/445372881] up:boot 2023-09-30T14:36:55.732 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:55 smithi148 ceph-mon[102355]: fsmap 7 up:standby 2023-09-30T14:36:55.733 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:55 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-30T14:36:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:55 smithi156 ceph-mon[103579]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:55 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:55 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:55 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:55 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:55 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:55 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:55.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:55 smithi156 ceph-mon[103579]: Deploying daemon mds.h on smithi148 2023-09-30T14:36:55.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:55 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.156:6836/445372881,v1:172.21.15.156:6837/445372881] up:boot 2023-09-30T14:36:55.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:55 smithi156 ceph-mon[103579]: fsmap 7 up:standby 2023-09-30T14:36:55.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:55 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-30T14:36:56.360 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 30 14:36:56 smithi148 systemd[1]: Starting Ceph mds.h for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:36:56.615 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:56 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/480687529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:56.615 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 30 14:36:56 smithi148 podman[138148]: 2023-09-30T14:36:56.615 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 30 14:36:56 smithi148 bash[138148]: b7af7cda2c9a2e21efc0ac8b21988cf5b74754f1daa0da36e1daeae272eaa9a5 2023-09-30T14:36:56.616 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 30 14:36:56 smithi148 ceph-mds[138188]: -- 172.21.15.148:0/2023937176 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561fa866b1e0 con 0x561fa8640800 2023-09-30T14:36:56.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:56 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/480687529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:56.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:56 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/480687529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:57.109 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 30 14:36:56 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-h[138164]: starting mds.h at 2023-09-30T14:36:57.109 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 30 14:36:56 smithi148 systemd[1]: Started Ceph mds.h for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:36:57.563 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:36:57.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.702+0000 7f7b37984700 1 -- 172.21.15.32:0/1738420544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b300ffd70 msgr2=0x7f7b30100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:57.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.702+0000 7f7b37984700 1 --2- 172.21.15.32:0/1738420544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b300ffd70 0x7f7b30100190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7b200099c0 tx=0x7f7b20009cd0 comp rx=0 tx=0).stop 2023-09-30T14:36:57.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.702+0000 7f7b37984700 1 -- 172.21.15.32:0/1738420544 shutdown_connections 2023-09-30T14:36:57.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.702+0000 7f7b37984700 1 --2- 172.21.15.32:0/1738420544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b30101900 0x7f7b301081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:57.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.702+0000 7f7b37984700 1 --2- 172.21.15.32:0/1738420544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b300ffd70 0x7f7b30100190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:57.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.702+0000 7f7b37984700 1 --2- 172.21.15.32:0/1738420544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b30100f60 0x7f7b301013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:57.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.702+0000 7f7b37984700 1 -- 172.21.15.32:0/1738420544 >> 172.21.15.32:0/1738420544 conn(0x7f7b300fb2e0 msgr2=0x7f7b300fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:57.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.702+0000 7f7b37984700 1 -- 172.21.15.32:0/1738420544 shutdown_connections 2023-09-30T14:36:57.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.702+0000 7f7b37984700 1 -- 172.21.15.32:0/1738420544 wait complete. 2023-09-30T14:36:57.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.703+0000 7f7b37984700 1 Processor -- start 2023-09-30T14:36:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.703+0000 7f7b37984700 1 -- start start 2023-09-30T14:36:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.703+0000 7f7b37984700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b300ffd70 0x7f7b30069da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.703+0000 7f7b37984700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b30100f60 0x7f7b3019b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.703+0000 7f7b37984700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b30101900 0x7f7b3019de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.703+0000 7f7b37984700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3019e610 con 0x7f7b30100f60 2023-09-30T14:36:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.703+0000 7f7b37984700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3019e780 con 0x7f7b300ffd70 2023-09-30T14:36:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.703+0000 7f7b37984700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3019e940 con 0x7f7b30101900 2023-09-30T14:36:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.703+0000 7f7b34f1f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b30100f60 0x7f7b3019b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.704+0000 7f7b34f1f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b30100f60 0x7f7b3019b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33580/0 (socket says 172.21.15.32:33580) 2023-09-30T14:36:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.704+0000 7f7b34f1f700 1 -- 172.21.15.32:0/1510822866 learned_addr learned my addr 172.21.15.32:0/1510822866 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:36:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.704+0000 7f7b35f21700 1 --2- 172.21.15.32:0/1510822866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b30101900 0x7f7b3019de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.704+0000 7f7b35720700 1 --2- 172.21.15.32:0/1510822866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b300ffd70 0x7f7b30069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.704+0000 7f7b34f1f700 1 -- 172.21.15.32:0/1510822866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b30101900 msgr2=0x7f7b3019de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.704+0000 7f7b34f1f700 1 --2- 172.21.15.32:0/1510822866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b30101900 0x7f7b3019de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.704+0000 7f7b34f1f700 1 -- 172.21.15.32:0/1510822866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b300ffd70 msgr2=0x7f7b30069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.704+0000 7f7b34f1f700 1 --2- 172.21.15.32:0/1510822866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b300ffd70 0x7f7b30069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.704+0000 7f7b34f1f700 1 -- 172.21.15.32:0/1510822866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b200095f0 con 0x7f7b30100f60 2023-09-30T14:36:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.704+0000 7f7b35720700 1 --2- 172.21.15.32:0/1510822866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b300ffd70 0x7f7b30069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:36:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.704+0000 7f7b34f1f700 1 --2- 172.21.15.32:0/1510822866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b30100f60 0x7f7b3019b7e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00d950 tx=0x7f7b2c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.704+0000 7f7b267fc700 1 -- 172.21.15.32:0/1510822866 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c009b40 con 0x7f7b30100f60 2023-09-30T14:36:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.704+0000 7f7b267fc700 1 -- 172.21.15.32:0/1510822866 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b2c009ca0 con 0x7f7b30100f60 2023-09-30T14:36:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.704+0000 7f7b37984700 1 -- 172.21.15.32:0/1510822866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3019ec20 con 0x7f7b30100f60 2023-09-30T14:36:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.704+0000 7f7b267fc700 1 -- 172.21.15.32:0/1510822866 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c00f620 con 0x7f7b30100f60 2023-09-30T14:36:57.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.704+0000 7f7b37984700 1 -- 172.21.15.32:0/1510822866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3019f130 con 0x7f7b30100f60 2023-09-30T14:36:57.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.705+0000 7f7b37984700 1 -- 172.21.15.32:0/1510822866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b30106ca0 con 0x7f7b30100f60 2023-09-30T14:36:57.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.706+0000 7f7b267fc700 1 -- 172.21.15.32:0/1510822866 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f7b2c010460 con 0x7f7b30100f60 2023-09-30T14:36:57.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.707+0000 7f7b267fc700 1 --2- 172.21.15.32:0/1510822866 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7b1c0a0ff0 0x7f7b1c0a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:36:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.707+0000 7f7b267fc700 1 -- 172.21.15.32:0/1510822866 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7b2c0cc600 con 0x7f7b30100f60 2023-09-30T14:36:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.707+0000 7f7b35720700 1 --2- 172.21.15.32:0/1510822866 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7b1c0a0ff0 0x7f7b1c0a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:36:57.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.707+0000 7f7b35720700 1 --2- 172.21.15.32:0/1510822866 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7b1c0a0ff0 0x7f7b1c0a34b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7b20006010 tx=0x7f7b20000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:36:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.711+0000 7f7b267fc700 1 -- 172.21.15.32:0/1510822866 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f7b2c0962a0 con 0x7f7b30100f60 2023-09-30T14:36:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:57 smithi156 ceph-mon[103579]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:57 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:57 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:57 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:57 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:57 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:57 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:57 smithi148 ceph-mon[102355]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:57 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:57 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:57 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:57 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:57.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:57 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:57.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:57 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:57 smithi032 ceph-mon[98592]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:57 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:57 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:57 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:57 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:57 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:57 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.959+0000 7f7b37984700 1 -- 172.21.15.32:0/1510822866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b3010b580 con 0x7f7b30100f60 2023-09-30T14:36:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.960+0000 7f7b267fc700 1 -- 172.21.15.32:0/1510822866 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6874 (secure 0 0 0) 0x7f7b2c0998c0 con 0x7f7b30100f60 2023-09-30T14:36:57.961 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:36:57.961 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-09-30T14:36:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.962+0000 7f7b37984700 1 -- 172.21.15.32:0/1510822866 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7b1c0a0ff0 msgr2=0x7f7b1c0a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.962+0000 7f7b37984700 1 --2- 172.21.15.32:0/1510822866 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7b1c0a0ff0 0x7f7b1c0a34b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7b20006010 tx=0x7f7b20000bc0 comp rx=0 tx=0).stop 2023-09-30T14:36:57.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.962+0000 7f7b37984700 1 -- 172.21.15.32:0/1510822866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b30100f60 msgr2=0x7f7b3019b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:36:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.962+0000 7f7b37984700 1 --2- 172.21.15.32:0/1510822866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b30100f60 0x7f7b3019b7e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00d950 tx=0x7f7b2c00dd10 comp rx=0 tx=0).stop 2023-09-30T14:36:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.962+0000 7f7b37984700 1 -- 172.21.15.32:0/1510822866 shutdown_connections 2023-09-30T14:36:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.962+0000 7f7b37984700 1 --2- 172.21.15.32:0/1510822866 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7b1c0a0ff0 0x7f7b1c0a34b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.962+0000 7f7b37984700 1 --2- 172.21.15.32:0/1510822866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b30101900 0x7f7b3019de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.962+0000 7f7b37984700 1 --2- 172.21.15.32:0/1510822866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b300ffd70 0x7f7b30069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.962+0000 7f7b37984700 1 --2- 172.21.15.32:0/1510822866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b30100f60 0x7f7b3019b7e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:36:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.962+0000 7f7b37984700 1 -- 172.21.15.32:0/1510822866 >> 172.21.15.32:0/1510822866 conn(0x7f7b300fb2e0 msgr2=0x7f7b300fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:36:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.962+0000 7f7b37984700 1 -- 172.21.15.32:0/1510822866 shutdown_connections 2023-09-30T14:36:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:36:57.962+0000 7f7b37984700 1 -- 172.21.15.32:0/1510822866 wait complete. 2023-09-30T14:36:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 9 2023-09-30T14:36:58.362 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 30 14:36:58 smithi156 systemd[1]: Starting Ceph mds.i for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:36:58.658 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:58 smithi156 ceph-mon[103579]: Deploying daemon mds.i on smithi156 2023-09-30T14:36:58.658 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:58 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] up:boot 2023-09-30T14:36:58.658 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:58 smithi156 ceph-mon[103579]: fsmap 8 up:standby 2023-09-30T14:36:58.658 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:58 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-30T14:36:58.658 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:58 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1510822866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:58.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:58 smithi148 ceph-mon[102355]: Deploying daemon mds.i on smithi156 2023-09-30T14:36:58.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:58 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] up:boot 2023-09-30T14:36:58.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:58 smithi148 ceph-mon[102355]: fsmap 8 up:standby 2023-09-30T14:36:58.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:58 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-30T14:36:58.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:58 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1510822866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:58.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:58 smithi032 ceph-mon[98592]: Deploying daemon mds.i on smithi156 2023-09-30T14:36:58.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:58 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] up:boot 2023-09-30T14:36:58.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:58 smithi032 ceph-mon[98592]: fsmap 8 up:standby 2023-09-30T14:36:58.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:58 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-30T14:36:58.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:58 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1510822866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:36:58.979 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 30 14:36:58 smithi156 podman[140610]: 2023-09-30T14:36:59.305 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 30 14:36:58 smithi156 ceph-mds[140651]: -- 172.21.15.156:0/2102575071 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561fdd2a91e0 con 0x561fdd27e800 2023-09-30T14:36:59.305 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 30 14:36:58 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-i[140626]: starting mds.i at 2023-09-30T14:36:59.305 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 30 14:36:59 smithi156 bash[140610]: 86be7bd051c3243d9734dbb7cb5ba81a887592103eaacf480300e82c8b61e889 2023-09-30T14:36:59.305 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 30 14:36:59 smithi156 systemd[1]: Started Ceph mds.i for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:36:59.781 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:59 smithi032 ceph-mon[98592]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:59.781 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:59 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:59.781 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:59 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:59.781 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:59 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:59.781 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:59 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:59.781 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:59 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:59.781 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:36:59 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:59 smithi156 ceph-mon[103579]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:59 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:59 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:59 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:59 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:59 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:36:59 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:59 smithi148 ceph-mon[102355]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:36:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:36:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:36:59.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:36:59.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:36:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:36:59.976 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:37:00.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.111+0000 7f51da84b700 1 -- 172.21.15.32:0/2194699411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4101900 msgr2=0x7f51d41081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:00.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.111+0000 7f51da84b700 1 --2- 172.21.15.32:0/2194699411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4101900 0x7f51d41081b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00b1d0 tx=0x7f51cc00b4e0 comp rx=0 tx=0).stop 2023-09-30T14:37:00.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.111+0000 7f51da84b700 1 -- 172.21.15.32:0/2194699411 shutdown_connections 2023-09-30T14:37:00.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.111+0000 7f51da84b700 1 --2- 172.21.15.32:0/2194699411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4101900 0x7f51d41081b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:00.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.111+0000 7f51da84b700 1 --2- 172.21.15.32:0/2194699411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51d40ffd70 0x7f51d4100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:00.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.111+0000 7f51da84b700 1 --2- 172.21.15.32:0/2194699411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51d4100f60 0x7f51d41013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:00.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.111+0000 7f51da84b700 1 -- 172.21.15.32:0/2194699411 >> 172.21.15.32:0/2194699411 conn(0x7f51d40fb2e0 msgr2=0x7f51d40fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:00.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.111+0000 7f51da84b700 1 -- 172.21.15.32:0/2194699411 shutdown_connections 2023-09-30T14:37:00.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.111+0000 7f51da84b700 1 -- 172.21.15.32:0/2194699411 wait complete. 2023-09-30T14:37:00.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.112+0000 7f51da84b700 1 Processor -- start 2023-09-30T14:37:00.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.112+0000 7f51da84b700 1 -- start start 2023-09-30T14:37:00.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.112+0000 7f51da84b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51d40ffd70 0x7f51d41948b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:00.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.112+0000 7f51da84b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51d4100f60 0x7f51d4194df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:00.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.112+0000 7f51da84b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4101900 0x7f51d419be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:00.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.112+0000 7f51da84b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d419c430 con 0x7f51d4101900 2023-09-30T14:37:00.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.112+0000 7f51da84b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d419c570 con 0x7f51d40ffd70 2023-09-30T14:37:00.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.112+0000 7f51da84b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d419c700 con 0x7f51d4100f60 2023-09-30T14:37:00.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.112+0000 7f51d8de8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4101900 0x7f51d419be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:00.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.113+0000 7f51d8de8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4101900 0x7f51d419be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33596/0 (socket says 172.21.15.32:33596) 2023-09-30T14:37:00.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.113+0000 7f51d8de8700 1 -- 172.21.15.32:0/2177229370 learned_addr learned my addr 172.21.15.32:0/2177229370 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:00.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.113+0000 7f51d8de8700 1 -- 172.21.15.32:0/2177229370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51d4100f60 msgr2=0x7f51d4194df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:00.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.113+0000 7f51d8de8700 1 --2- 172.21.15.32:0/2177229370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51d4100f60 0x7f51d4194df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:00.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.113+0000 7f51d8de8700 1 -- 172.21.15.32:0/2177229370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51d40ffd70 msgr2=0x7f51d41948b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:00.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.113+0000 7f51d3fff700 1 --2- 172.21.15.32:0/2177229370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51d40ffd70 0x7f51d41948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:00.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.113+0000 7f51d8de8700 1 --2- 172.21.15.32:0/2177229370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51d40ffd70 0x7f51d41948b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:00.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.113+0000 7f51d8de8700 1 -- 172.21.15.32:0/2177229370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51cc009d00 con 0x7f51d4101900 2023-09-30T14:37:00.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.113+0000 7f51d3fff700 1 --2- 172.21.15.32:0/2177229370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51d40ffd70 0x7f51d41948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:37:00.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.113+0000 7f51d8de8700 1 --2- 172.21.15.32:0/2177229370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4101900 0x7f51d419be60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009bb0 tx=0x7f51cc0078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:00.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.113+0000 7f51d17fa700 1 -- 172.21.15.32:0/2177229370 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc008e50 con 0x7f51d4101900 2023-09-30T14:37:00.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.113+0000 7f51d17fa700 1 -- 172.21.15.32:0/2177229370 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51cc0104d0 con 0x7f51d4101900 2023-09-30T14:37:00.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.113+0000 7f51da84b700 1 -- 172.21.15.32:0/2177229370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d419c980 con 0x7f51d4101900 2023-09-30T14:37:00.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.113+0000 7f51d17fa700 1 -- 172.21.15.32:0/2177229370 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc005330 con 0x7f51d4101900 2023-09-30T14:37:00.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.113+0000 7f51da84b700 1 -- 172.21.15.32:0/2177229370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d419ce50 con 0x7f51d4101900 2023-09-30T14:37:00.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.114+0000 7f51da84b700 1 -- 172.21.15.32:0/2177229370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d410a580 con 0x7f51d4101900 2023-09-30T14:37:00.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.115+0000 7f51d17fa700 1 -- 172.21.15.32:0/2177229370 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f51cc010640 con 0x7f51d4101900 2023-09-30T14:37:00.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.116+0000 7f51d17fa700 1 --2- 172.21.15.32:0/2177229370 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51bc0a1070 0x7f51bc0a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:00.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.116+0000 7f51d3fff700 1 --2- 172.21.15.32:0/2177229370 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51bc0a1070 0x7f51bc0a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:00.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.116+0000 7f51d17fa700 1 -- 172.21.15.32:0/2177229370 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f51cc0d1100 con 0x7f51d4101900 2023-09-30T14:37:00.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.116+0000 7f51d3fff700 1 --2- 172.21.15.32:0/2177229370 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51bc0a1070 0x7f51bc0a3530 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f51c4005fd0 tx=0x7f51c4005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:00.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.120+0000 7f51d17fa700 1 -- 172.21.15.32:0/2177229370 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f51cc09ae30 con 0x7f51d4101900 2023-09-30T14:37:00.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.368+0000 7f51da84b700 1 -- 172.21.15.32:0/2177229370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51d410a770 con 0x7f51d4101900 2023-09-30T14:37:00.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.369+0000 7f51d17fa700 1 -- 172.21.15.32:0/2177229370 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7670 (secure 0 0 0) 0x7f51cc09e450 con 0x7f51d4101900 2023-09-30T14:37:00.370 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:37:00.370 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/1445176203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1445176203},{"type":"v1","addr":"172.21.15.156:6839","nonce":1445176203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-09-30T14:37:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.371+0000 7f51da84b700 1 -- 172.21.15.32:0/2177229370 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51bc0a1070 msgr2=0x7f51bc0a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.371+0000 7f51da84b700 1 --2- 172.21.15.32:0/2177229370 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51bc0a1070 0x7f51bc0a3530 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f51c4005fd0 tx=0x7f51c4005f00 comp rx=0 tx=0).stop 2023-09-30T14:37:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.371+0000 7f51da84b700 1 -- 172.21.15.32:0/2177229370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4101900 msgr2=0x7f51d419be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.371+0000 7f51da84b700 1 --2- 172.21.15.32:0/2177229370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4101900 0x7f51d419be60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009bb0 tx=0x7f51cc0078a0 comp rx=0 tx=0).stop 2023-09-30T14:37:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.371+0000 7f51da84b700 1 -- 172.21.15.32:0/2177229370 shutdown_connections 2023-09-30T14:37:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.371+0000 7f51da84b700 1 --2- 172.21.15.32:0/2177229370 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51bc0a1070 0x7f51bc0a3530 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.371+0000 7f51da84b700 1 --2- 172.21.15.32:0/2177229370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d4101900 0x7f51d419be60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.371+0000 7f51da84b700 1 --2- 172.21.15.32:0/2177229370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51d40ffd70 0x7f51d41948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.371+0000 7f51da84b700 1 --2- 172.21.15.32:0/2177229370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51d4100f60 0x7f51d4194df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.371+0000 7f51da84b700 1 -- 172.21.15.32:0/2177229370 >> 172.21.15.32:0/2177229370 conn(0x7f51d40fb2e0 msgr2=0x7f51d40fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.371+0000 7f51da84b700 1 -- 172.21.15.32:0/2177229370 shutdown_connections 2023-09-30T14:37:00.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:00.371+0000 7f51da84b700 1 -- 172.21.15.32:0/2177229370 wait complete. 2023-09-30T14:37:00.373 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 10 2023-09-30T14:37:00.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:00 smithi032 ceph-mon[98592]: Deploying daemon mds.g on smithi032 2023-09-30T14:37:00.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:00 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] up:boot 2023-09-30T14:37:00.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:00 smithi032 ceph-mon[98592]: fsmap 9 up:standby 2023-09-30T14:37:00.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:00 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-30T14:37:00.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:00 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2177229370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:37:00.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:00 smithi156 ceph-mon[103579]: Deploying daemon mds.g on smithi032 2023-09-30T14:37:00.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:00 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] up:boot 2023-09-30T14:37:00.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:00 smithi156 ceph-mon[103579]: fsmap 9 up:standby 2023-09-30T14:37:00.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:00 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-30T14:37:00.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:00 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2177229370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:37:00.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:00 smithi148 ceph-mon[102355]: Deploying daemon mds.g on smithi032 2023-09-30T14:37:00.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:00 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] up:boot 2023-09-30T14:37:00.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:00 smithi148 ceph-mon[102355]: fsmap 9 up:standby 2023-09-30T14:37:00.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:00 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-30T14:37:00.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:00 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2177229370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:37:01.137 INFO:journalctl@ceph.mds.g.smithi032.stdout:Sep 30 14:37:00 smithi032 systemd[1]: Starting Ceph mds.g for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:37:01.434 INFO:journalctl@ceph.mds.g.smithi032.stdout:Sep 30 14:37:01 smithi032 podman[150191]: 2023-09-30T14:37:01.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:01 smithi156 ceph-mon[103579]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:01.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:01 smithi148 ceph-mon[102355]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:01 smithi032 ceph-mon[98592]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:01.888 INFO:journalctl@ceph.mds.g.smithi032.stdout:Sep 30 14:37:01 smithi032 ceph-mds[150231]: -- 172.21.15.32:0/755562135 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55619d2a71e0 con 0x55619d27d000 2023-09-30T14:37:01.888 INFO:journalctl@ceph.mds.g.smithi032.stdout:Sep 30 14:37:01 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-g[150207]: starting mds.g at 2023-09-30T14:37:01.888 INFO:journalctl@ceph.mds.g.smithi032.stdout:Sep 30 14:37:01 smithi032 bash[150191]: d730eb659aee2f4cf246cd507621216323abd2a6fbce3387058ce2576daccfe2 2023-09-30T14:37:01.888 INFO:journalctl@ceph.mds.g.smithi032.stdout:Sep 30 14:37:01 smithi032 systemd[1]: Started Ceph mds.g for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:37:02.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:37:02.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.519+0000 7f75a303a700 1 -- 172.21.15.32:0/2073899428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c0ffd30 msgr2=0x7f759c100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:02.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.519+0000 7f75a303a700 1 --2- 172.21.15.32:0/2073899428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c0ffd30 0x7f759c100150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f75900099c0 tx=0x7f7590009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:02.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.519+0000 7f75a303a700 1 -- 172.21.15.32:0/2073899428 shutdown_connections 2023-09-30T14:37:02.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.519+0000 7f75a303a700 1 --2- 172.21.15.32:0/2073899428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f759c1018c0 0x7f759c108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:02.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.519+0000 7f75a303a700 1 --2- 172.21.15.32:0/2073899428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c0ffd30 0x7f759c100150 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:02.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.519+0000 7f75a303a700 1 --2- 172.21.15.32:0/2073899428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c100f20 0x7f759c101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:02.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.519+0000 7f75a303a700 1 -- 172.21.15.32:0/2073899428 >> 172.21.15.32:0/2073899428 conn(0x7f759c0fb280 msgr2=0x7f759c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:02.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.519+0000 7f75a303a700 1 -- 172.21.15.32:0/2073899428 shutdown_connections 2023-09-30T14:37:02.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.519+0000 7f75a303a700 1 -- 172.21.15.32:0/2073899428 wait complete. 2023-09-30T14:37:02.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.520+0000 7f75a303a700 1 Processor -- start 2023-09-30T14:37:02.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.520+0000 7f75a303a700 1 -- start start 2023-09-30T14:37:02.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.520+0000 7f75a303a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c0ffd30 0x7f759c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:02.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.520+0000 7f75a303a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f759c100f20 0x7f759c19b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:02.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.520+0000 7f75a303a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c1018c0 0x7f759c19deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:02.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.520+0000 7f75a303a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c19e650 con 0x7f759c1018c0 2023-09-30T14:37:02.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.520+0000 7f75a303a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c19e7c0 con 0x7f759c0ffd30 2023-09-30T14:37:02.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.520+0000 7f75a303a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c19e980 con 0x7f759c100f20 2023-09-30T14:37:02.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.520+0000 7f75a15d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c1018c0 0x7f759c19deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:02.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.520+0000 7f75a15d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c1018c0 0x7f759c19deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33624/0 (socket says 172.21.15.32:33624) 2023-09-30T14:37:02.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.520+0000 7f75a15d7700 1 -- 172.21.15.32:0/4113635086 learned_addr learned my addr 172.21.15.32:0/4113635086 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:02.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.520+0000 7f75a0dd6700 1 --2- 172.21.15.32:0/4113635086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c0ffd30 0x7f759c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.520+0000 7f759bfff700 1 --2- 172.21.15.32:0/4113635086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f759c100f20 0x7f759c19b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.521+0000 7f75a15d7700 1 -- 172.21.15.32:0/4113635086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f759c100f20 msgr2=0x7f759c19b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.521+0000 7f75a15d7700 1 --2- 172.21.15.32:0/4113635086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f759c100f20 0x7f759c19b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.521+0000 7f75a15d7700 1 -- 172.21.15.32:0/4113635086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c0ffd30 msgr2=0x7f759c068b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.521+0000 7f75a15d7700 1 --2- 172.21.15.32:0/4113635086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c0ffd30 0x7f759c068b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.521+0000 7f75a15d7700 1 -- 172.21.15.32:0/4113635086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75900095f0 con 0x7f759c1018c0 2023-09-30T14:37:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.521+0000 7f75a0dd6700 1 --2- 172.21.15.32:0/4113635086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c0ffd30 0x7f759c068b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:37:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.521+0000 7f75a15d7700 1 --2- 172.21.15.32:0/4113635086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c1018c0 0x7f759c19deb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7594007ee0 tx=0x7f759400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.521+0000 7f7599ffb700 1 -- 172.21.15.32:0/4113635086 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594017070 con 0x7f759c1018c0 2023-09-30T14:37:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.521+0000 7f7599ffb700 1 -- 172.21.15.32:0/4113635086 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f759400f040 con 0x7f759c1018c0 2023-09-30T14:37:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.521+0000 7f75a303a700 1 -- 172.21.15.32:0/4113635086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759c19ec60 con 0x7f759c1018c0 2023-09-30T14:37:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.521+0000 7f7599ffb700 1 -- 172.21.15.32:0/4113635086 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594005740 con 0x7f759c1018c0 2023-09-30T14:37:02.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.521+0000 7f75a303a700 1 -- 172.21.15.32:0/4113635086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759c19f1c0 con 0x7f759c1018c0 2023-09-30T14:37:02.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.522+0000 7f75a303a700 1 -- 172.21.15.32:0/4113635086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f759c106cb0 con 0x7f759c1018c0 2023-09-30T14:37:02.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.523+0000 7f7599ffb700 1 -- 172.21.15.32:0/4113635086 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f75940058a0 con 0x7f759c1018c0 2023-09-30T14:37:02.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.524+0000 7f7599ffb700 1 --2- 172.21.15.32:0/4113635086 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f758c0a1070 0x7f758c0a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:02.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.524+0000 7f75a0dd6700 1 --2- 172.21.15.32:0/4113635086 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f758c0a1070 0x7f758c0a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:02.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.524+0000 7f7599ffb700 1 -- 172.21.15.32:0/4113635086 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f75940cbb00 con 0x7f759c1018c0 2023-09-30T14:37:02.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.524+0000 7f75a0dd6700 1 --2- 172.21.15.32:0/4113635086 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f758c0a1070 0x7f758c0a3530 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f759000b580 tx=0x7f7590005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:02.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.529+0000 7f7599ffb700 1 -- 172.21.15.32:0/4113635086 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f75940cde60 con 0x7f759c1018c0 2023-09-30T14:37:02.705 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:02.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:02.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:02.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:37:02.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:37:02.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:37:02.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:02 smithi032 ceph-mon[98592]: Deploying daemon mds.a on smithi032 2023-09-30T14:37:02.759 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:02 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.32:6838/3505064841,v1:172.21.15.32:6839/3505064841] up:boot 2023-09-30T14:37:02.760 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:02 smithi032 ceph-mon[98592]: fsmap 10 up:standby 2023-09-30T14:37:02.760 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-30T14:37:02.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.775+0000 7f75a303a700 1 -- 172.21.15.32:0/4113635086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f759c10b540 con 0x7f759c1018c0 2023-09-30T14:37:02.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.776+0000 7f7599ffb700 1 -- 172.21.15.32:0/4113635086 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8462 (secure 0 0 0) 0x7f7594095700 con 0x7f759c1018c0 2023-09-30T14:37:02.776 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:37:02.777 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3505064841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3505064841},{"type":"v1","addr":"172.21.15.32:6839","nonce":3505064841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/1445176203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1445176203},{"type":"v1","addr":"172.21.15.156:6839","nonce":1445176203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-09-30T14:37:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.778+0000 7f75a303a700 1 -- 172.21.15.32:0/4113635086 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f758c0a1070 msgr2=0x7f758c0a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.778+0000 7f75a303a700 1 --2- 172.21.15.32:0/4113635086 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f758c0a1070 0x7f758c0a3530 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f759000b580 tx=0x7f7590005fb0 comp rx=0 tx=0).stop 2023-09-30T14:37:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.778+0000 7f75a303a700 1 -- 172.21.15.32:0/4113635086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c1018c0 msgr2=0x7f759c19deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.778+0000 7f75a303a700 1 --2- 172.21.15.32:0/4113635086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c1018c0 0x7f759c19deb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7594007ee0 tx=0x7f759400d3b0 comp rx=0 tx=0).stop 2023-09-30T14:37:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.778+0000 7f75a303a700 1 -- 172.21.15.32:0/4113635086 shutdown_connections 2023-09-30T14:37:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.778+0000 7f75a303a700 1 --2- 172.21.15.32:0/4113635086 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f758c0a1070 0x7f758c0a3530 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.778+0000 7f75a303a700 1 --2- 172.21.15.32:0/4113635086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f759c1018c0 0x7f759c19deb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.778+0000 7f75a303a700 1 --2- 172.21.15.32:0/4113635086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f759c0ffd30 0x7f759c068b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.778+0000 7f75a303a700 1 --2- 172.21.15.32:0/4113635086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f759c100f20 0x7f759c19b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.778+0000 7f75a303a700 1 -- 172.21.15.32:0/4113635086 >> 172.21.15.32:0/4113635086 conn(0x7f759c0fb280 msgr2=0x7f759c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.778+0000 7f75a303a700 1 -- 172.21.15.32:0/4113635086 shutdown_connections 2023-09-30T14:37:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:02.778+0000 7f75a303a700 1 -- 172.21.15.32:0/4113635086 wait complete. 2023-09-30T14:37:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 11 2023-09-30T14:37:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:37:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:37:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:37:02.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:02 smithi156 ceph-mon[103579]: Deploying daemon mds.a on smithi032 2023-09-30T14:37:02.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:02 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.32:6838/3505064841,v1:172.21.15.32:6839/3505064841] up:boot 2023-09-30T14:37:02.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:02 smithi156 ceph-mon[103579]: fsmap 10 up:standby 2023-09-30T14:37:02.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-30T14:37:02.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:02.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:02.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:02.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:37:02.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:37:02.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:37:02.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:02 smithi148 ceph-mon[102355]: Deploying daemon mds.a on smithi032 2023-09-30T14:37:02.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:02 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.32:6838/3505064841,v1:172.21.15.32:6839/3505064841] up:boot 2023-09-30T14:37:02.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:02 smithi148 ceph-mon[102355]: fsmap 10 up:standby 2023-09-30T14:37:02.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-30T14:37:03.051 INFO:journalctl@ceph.mds.a.smithi032.stdout:Sep 30 14:37:03 smithi032 systemd[1]: Starting Ceph mds.a for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:37:03.614 INFO:journalctl@ceph.mds.a.smithi032.stdout:Sep 30 14:37:03 smithi032 podman[150578]: 2023-09-30T14:37:03.614 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:03 smithi032 ceph-mon[98592]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:03.614 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:03 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/4113635086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:37:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:03 smithi156 ceph-mon[103579]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:03 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/4113635086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:37:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:03 smithi148 ceph-mon[102355]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:03 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/4113635086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:37:03.887 INFO:journalctl@ceph.mds.a.smithi032.stdout:Sep 30 14:37:03 smithi032 bash[150578]: 8dedb073a3bfea8cd57fca647bd140a2a3bd693780fe76ebdd222b0dae42970d 2023-09-30T14:37:03.887 INFO:journalctl@ceph.mds.a.smithi032.stdout:Sep 30 14:37:03 smithi032 ceph-mds[150618]: -- 172.21.15.32:0/3793862733 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55671794f1e0 con 0x556717924c00 2023-09-30T14:37:03.887 INFO:journalctl@ceph.mds.a.smithi032.stdout:Sep 30 14:37:03 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-a[150594]: starting mds.a at 2023-09-30T14:37:03.887 INFO:journalctl@ceph.mds.a.smithi032.stdout:Sep 30 14:37:03 smithi032 systemd[1]: Started Ceph mds.a for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:37:04.792 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:37:04.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.937+0000 7f17027ef700 1 -- 172.21.15.32:0/87772381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16fc100f30 msgr2=0x7f16fc101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:04.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.937+0000 7f17027ef700 1 --2- 172.21.15.32:0/87772381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16fc100f30 0x7f16fc101390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f16e40099c0 tx=0x7f16e4009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:04.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.937+0000 7f17027ef700 1 -- 172.21.15.32:0/87772381 shutdown_connections 2023-09-30T14:37:04.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.937+0000 7f17027ef700 1 --2- 172.21.15.32:0/87772381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16fc1018d0 0x7f16fc108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:04.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.937+0000 7f17027ef700 1 --2- 172.21.15.32:0/87772381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16fc0ffd40 0x7f16fc100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:04.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.937+0000 7f17027ef700 1 --2- 172.21.15.32:0/87772381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16fc100f30 0x7f16fc101390 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:04.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.937+0000 7f17027ef700 1 -- 172.21.15.32:0/87772381 >> 172.21.15.32:0/87772381 conn(0x7f16fc0fb270 msgr2=0x7f16fc0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:04.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.937+0000 7f17027ef700 1 -- 172.21.15.32:0/87772381 shutdown_connections 2023-09-30T14:37:04.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.937+0000 7f17027ef700 1 -- 172.21.15.32:0/87772381 wait complete. 2023-09-30T14:37:04.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f17027ef700 1 Processor -- start 2023-09-30T14:37:04.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f17027ef700 1 -- start start 2023-09-30T14:37:04.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f17027ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16fc0ffd40 0x7f16fc1959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f17027ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16fc100f30 0x7f16fc195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f17027ef700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16fc1018d0 0x7f16fc19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f17027ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc19d550 con 0x7f16fc100f30 2023-09-30T14:37:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f17027ef700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc19d690 con 0x7f16fc1018d0 2023-09-30T14:37:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f17027ef700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc19d820 con 0x7f16fc0ffd40 2023-09-30T14:37:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f16f3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16fc100f30 0x7f16fc195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f16f3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16fc100f30 0x7f16fc195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33646/0 (socket says 172.21.15.32:33646) 2023-09-30T14:37:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f16f3fff700 1 -- 172.21.15.32:0/3156210172 learned_addr learned my addr 172.21.15.32:0/3156210172 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f16fbfff700 1 --2- 172.21.15.32:0/3156210172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16fc0ffd40 0x7f16fc1959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f1700d8c700 1 --2- 172.21.15.32:0/3156210172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16fc1018d0 0x7f16fc19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f16f3fff700 1 -- 172.21.15.32:0/3156210172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16fc0ffd40 msgr2=0x7f16fc1959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f16f3fff700 1 --2- 172.21.15.32:0/3156210172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16fc0ffd40 0x7f16fc1959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f16f3fff700 1 -- 172.21.15.32:0/3156210172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16fc1018d0 msgr2=0x7f16fc19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f16f3fff700 1 --2- 172.21.15.32:0/3156210172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16fc1018d0 0x7f16fc19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f16f3fff700 1 -- 172.21.15.32:0/3156210172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e40095f0 con 0x7f16fc100f30 2023-09-30T14:37:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.938+0000 7f16fbfff700 1 --2- 172.21.15.32:0/3156210172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16fc0ffd40 0x7f16fc1959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:37:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.939+0000 7f16f3fff700 1 --2- 172.21.15.32:0/3156210172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16fc100f30 0x7f16fc195f10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f16e40036f0 tx=0x7f16e4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.939+0000 7f16f9ffb700 1 -- 172.21.15.32:0/3156210172 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e401d070 con 0x7f16fc100f30 2023-09-30T14:37:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.939+0000 7f16f9ffb700 1 -- 172.21.15.32:0/3156210172 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16e4004010 con 0x7f16fc100f30 2023-09-30T14:37:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.939+0000 7f17027ef700 1 -- 172.21.15.32:0/3156210172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16fc19daa0 con 0x7f16fc100f30 2023-09-30T14:37:04.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.939+0000 7f16f9ffb700 1 -- 172.21.15.32:0/3156210172 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e4022da0 con 0x7f16fc100f30 2023-09-30T14:37:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.939+0000 7f17027ef700 1 -- 172.21.15.32:0/3156210172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16fc19df20 con 0x7f16fc100f30 2023-09-30T14:37:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.940+0000 7f17027ef700 1 -- 172.21.15.32:0/3156210172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16fc10b570 con 0x7f16fc100f30 2023-09-30T14:37:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.941+0000 7f16f9ffb700 1 -- 172.21.15.32:0/3156210172 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f16e4022500 con 0x7f16fc100f30 2023-09-30T14:37:04.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.941+0000 7f16f9ffb700 1 --2- 172.21.15.32:0/3156210172 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f16e80a1070 0x7f16e80a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:04.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.941+0000 7f16fbfff700 1 --2- 172.21.15.32:0/3156210172 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f16e80a1070 0x7f16e80a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:04.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.941+0000 7f16f9ffb700 1 -- 172.21.15.32:0/3156210172 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f16e40cdc70 con 0x7f16fc100f30 2023-09-30T14:37:04.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.942+0000 7f16fbfff700 1 --2- 172.21.15.32:0/3156210172 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f16e80a1070 0x7f16e80a3530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f16ec005950 tx=0x7f16ec005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:04.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:04.945+0000 7f16f9ffb700 1 -- 172.21.15.32:0/3156210172 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f16e4097a20 con 0x7f16fc100f30 2023-09-30T14:37:05.012 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:04 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:05.012 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:04 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:05.012 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:04 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:05.012 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:04 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:37:05.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:04 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:37:05.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:04 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:37:05.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:04 smithi148 ceph-mon[102355]: Deploying daemon mds.b on smithi148 2023-09-30T14:37:05.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:04 smithi148 ceph-mon[102355]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:05.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:04 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.32:6840/2262846038,v1:172.21.15.32:6841/2262846038] up:boot 2023-09-30T14:37:05.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:04 smithi148 ceph-mon[102355]: fsmap 11 up:standby 2023-09-30T14:37:05.013 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:04 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-30T14:37:05.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:05.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:05.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:05.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:37:05.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:37:05.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:37:05.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:04 smithi156 ceph-mon[103579]: Deploying daemon mds.b on smithi148 2023-09-30T14:37:05.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:04 smithi156 ceph-mon[103579]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:05.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:04 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.32:6840/2262846038,v1:172.21.15.32:6841/2262846038] up:boot 2023-09-30T14:37:05.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:04 smithi156 ceph-mon[103579]: fsmap 11 up:standby 2023-09-30T14:37:05.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-30T14:37:05.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:05.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:05.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:05.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-30T14:37:05.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-30T14:37:05.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:37:05.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:04 smithi032 ceph-mon[98592]: Deploying daemon mds.b on smithi148 2023-09-30T14:37:05.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:04 smithi032 ceph-mon[98592]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:05.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:04 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.32:6840/2262846038,v1:172.21.15.32:6841/2262846038] up:boot 2023-09-30T14:37:05.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:04 smithi032 ceph-mon[98592]: fsmap 11 up:standby 2023-09-30T14:37:05.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-30T14:37:05.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:05.193+0000 7f17027ef700 1 -- 172.21.15.32:0/3156210172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16fc106c40 con 0x7f16fc100f30 2023-09-30T14:37:05.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:05.193+0000 7f16f9ffb700 1 -- 172.21.15.32:0/3156210172 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9254 (secure 0 0 0) 0x7f16e4027090 con 0x7f16fc100f30 2023-09-30T14:37:05.193 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:37:05.194 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3505064841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3505064841},{"type":"v1","addr":"172.21.15.32:6839","nonce":3505064841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2262846038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2262846038},{"type":"v1","addr":"172.21.15.32:6841","nonce":2262846038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/1445176203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1445176203},{"type":"v1","addr":"172.21.15.156:6839","nonce":1445176203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-09-30T14:37:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:05.196+0000 7f17027ef700 1 -- 172.21.15.32:0/3156210172 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f16e80a1070 msgr2=0x7f16e80a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:05.196+0000 7f17027ef700 1 --2- 172.21.15.32:0/3156210172 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f16e80a1070 0x7f16e80a3530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f16ec005950 tx=0x7f16ec005f50 comp rx=0 tx=0).stop 2023-09-30T14:37:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:05.196+0000 7f17027ef700 1 -- 172.21.15.32:0/3156210172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16fc100f30 msgr2=0x7f16fc195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:05.196+0000 7f17027ef700 1 --2- 172.21.15.32:0/3156210172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16fc100f30 0x7f16fc195f10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f16e40036f0 tx=0x7f16e4005740 comp rx=0 tx=0).stop 2023-09-30T14:37:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:05.196+0000 7f17027ef700 1 -- 172.21.15.32:0/3156210172 shutdown_connections 2023-09-30T14:37:05.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:05.196+0000 7f17027ef700 1 --2- 172.21.15.32:0/3156210172 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f16e80a1070 0x7f16e80a3530 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:05.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:05.196+0000 7f17027ef700 1 --2- 172.21.15.32:0/3156210172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16fc1018d0 0x7f16fc19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:05.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:05.196+0000 7f17027ef700 1 --2- 172.21.15.32:0/3156210172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16fc0ffd40 0x7f16fc1959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:05.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:05.196+0000 7f17027ef700 1 --2- 172.21.15.32:0/3156210172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16fc100f30 0x7f16fc195f10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:05.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:05.196+0000 7f17027ef700 1 -- 172.21.15.32:0/3156210172 >> 172.21.15.32:0/3156210172 conn(0x7f16fc0fb270 msgr2=0x7f16fc0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:05.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:05.196+0000 7f17027ef700 1 -- 172.21.15.32:0/3156210172 shutdown_connections 2023-09-30T14:37:05.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:05.196+0000 7f17027ef700 1 -- 172.21.15.32:0/3156210172 wait complete. 2023-09-30T14:37:05.197 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 12 2023-09-30T14:37:05.262 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 30 14:37:05 smithi148 systemd[1]: Starting Ceph mds.b for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T14:37:05.822 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:05 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3156210172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:37:05.823 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 30 14:37:05 smithi148 podman[138489]: 2023-09-30T14:37:06.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:05 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3156210172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:37:06.109 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 30 14:37:05 smithi148 ceph-mds[138529]: -- 172.21.15.148:0/1594015776 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c7b9ed11e0 con 0x55c7b9ea6c00 2023-09-30T14:37:06.109 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 30 14:37:05 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-b[138505]: starting mds.b at 2023-09-30T14:37:06.109 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 30 14:37:05 smithi148 bash[138489]: c1e52d11442983cde990c6e06ecadaca33b739f49ee97be37a41865538561aa8 2023-09-30T14:37:06.109 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 30 14:37:05 smithi148 systemd[1]: Started Ceph mds.b for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T14:37:06.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:05 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3156210172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:37:07.209 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:37:07.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:06 smithi156 ceph-mon[103579]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:07.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:06 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:06 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:06 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:06 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:06 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:06 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:37:07.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:06 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] up:boot 2023-09-30T14:37:07.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:06 smithi156 ceph-mon[103579]: fsmap 12 up:standby 2023-09-30T14:37:07.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:06 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-30T14:37:07.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:06 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.341+0000 7f77952bf700 1 -- 172.21.15.32:0/44514843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7790101fa0 msgr2=0x7f7790102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:07.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.341+0000 7f77952bf700 1 --2- 172.21.15.32:0/44514843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7790101fa0 0x7f7790102420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7784009990 tx=0x7f7784009ca0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.341+0000 7f77952bf700 1 -- 172.21.15.32:0/44514843 shutdown_connections 2023-09-30T14:37:07.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.341+0000 7f77952bf700 1 --2- 172.21.15.32:0/44514843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7790102960 0x7f7790109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.341+0000 7f77952bf700 1 --2- 172.21.15.32:0/44514843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7790100e50 0x7f7790101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.341+0000 7f77952bf700 1 --2- 172.21.15.32:0/44514843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7790101fa0 0x7f7790102420 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.341+0000 7f77952bf700 1 -- 172.21.15.32:0/44514843 >> 172.21.15.32:0/44514843 conn(0x7f77900fc3a0 msgr2=0x7f77900fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:07.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.341+0000 7f77952bf700 1 -- 172.21.15.32:0/44514843 shutdown_connections 2023-09-30T14:37:07.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.341+0000 7f77952bf700 1 -- 172.21.15.32:0/44514843 wait complete. 2023-09-30T14:37:07.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.342+0000 7f77952bf700 1 Processor -- start 2023-09-30T14:37:07.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.342+0000 7f77952bf700 1 -- start start 2023-09-30T14:37:07.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.342+0000 7f77952bf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7790100e50 0x7f7790196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:07.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.342+0000 7f77952bf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7790101fa0 0x7f7790197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:07.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.342+0000 7f77952bf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7790102960 0x7f779019e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:07.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.342+0000 7f77952bf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779019e670 con 0x7f7790102960 2023-09-30T14:37:07.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.342+0000 7f77952bf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779019e7b0 con 0x7f7790101fa0 2023-09-30T14:37:07.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.342+0000 7f77952bf700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779019e940 con 0x7f7790100e50 2023-09-30T14:37:07.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.342+0000 7f778f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7790102960 0x7f779019e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.342+0000 7f778e7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7790101fa0 0x7f7790197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.342+0000 7f778e7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7790101fa0 0x7f7790197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:45136/0 (socket says 172.21.15.32:45136) 2023-09-30T14:37:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.342+0000 7f778f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7790102960 0x7f779019e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33662/0 (socket says 172.21.15.32:33662) 2023-09-30T14:37:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.342+0000 7f778e7fc700 1 -- 172.21.15.32:0/3145219878 learned_addr learned my addr 172.21.15.32:0/3145219878 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.342+0000 7f778effd700 1 --2- 172.21.15.32:0/3145219878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7790100e50 0x7f7790196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.342+0000 7f778e7fc700 1 -- 172.21.15.32:0/3145219878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7790100e50 msgr2=0x7f7790196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.342+0000 7f778e7fc700 1 --2- 172.21.15.32:0/3145219878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7790100e50 0x7f7790196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.343+0000 7f778e7fc700 1 -- 172.21.15.32:0/3145219878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7790102960 msgr2=0x7f779019e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.343+0000 7f778e7fc700 1 --2- 172.21.15.32:0/3145219878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7790102960 0x7f779019e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.343+0000 7f778e7fc700 1 -- 172.21.15.32:0/3145219878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7780009520 con 0x7f7790101fa0 2023-09-30T14:37:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.343+0000 7f778e7fc700 1 --2- 172.21.15.32:0/3145219878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7790101fa0 0x7f7790197030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7784004760 tx=0x7f7784004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:07.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.343+0000 7f7777fff700 1 -- 172.21.15.32:0/3145219878 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778401d070 con 0x7f7790101fa0 2023-09-30T14:37:07.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.343+0000 7f77952bf700 1 -- 172.21.15.32:0/3145219878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77840095f0 con 0x7f7790101fa0 2023-09-30T14:37:07.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.343+0000 7f7777fff700 1 -- 172.21.15.32:0/3145219878 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7784004830 con 0x7f7790101fa0 2023-09-30T14:37:07.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.343+0000 7f7777fff700 1 -- 172.21.15.32:0/3145219878 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784005350 con 0x7f7790101fa0 2023-09-30T14:37:07.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.343+0000 7f77952bf700 1 -- 172.21.15.32:0/3145219878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779019edf0 con 0x7f7790101fa0 2023-09-30T14:37:07.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.344+0000 7f77952bf700 1 -- 172.21.15.32:0/3145219878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f779010c6a0 con 0x7f7790101fa0 2023-09-30T14:37:07.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.345+0000 7f7777fff700 1 -- 172.21.15.32:0/3145219878 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f7784005510 con 0x7f7790101fa0 2023-09-30T14:37:07.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.346+0000 7f7777fff700 1 --2- 172.21.15.32:0/3145219878 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f77780a0ff0 0x7f77780a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:07.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.346+0000 7f778effd700 1 --2- 172.21.15.32:0/3145219878 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f77780a0ff0 0x7f77780a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:07.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.346+0000 7f7777fff700 1 -- 172.21.15.32:0/3145219878 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f77840cd4b0 con 0x7f7790101fa0 2023-09-30T14:37:07.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.346+0000 7f778effd700 1 --2- 172.21.15.32:0/3145219878 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f77780a0ff0 0x7f77780a34b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7780009df0 tx=0x7f7780009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:07.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.349+0000 7f7777fff700 1 -- 172.21.15.32:0/3145219878 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f7784097140 con 0x7f7790101fa0 2023-09-30T14:37:07.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:06 smithi148 ceph-mon[102355]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:07.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:37:07.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:06 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] up:boot 2023-09-30T14:37:07.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:06 smithi148 ceph-mon[102355]: fsmap 12 up:standby 2023-09-30T14:37:07.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-30T14:37:07.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:06 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:06 smithi032 ceph-mon[98592]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:07.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:06 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:06 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:06 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:06 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:06 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:06 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:37:07.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:06 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] up:boot 2023-09-30T14:37:07.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:06 smithi032 ceph-mon[98592]: fsmap 12 up:standby 2023-09-30T14:37:07.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:06 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-30T14:37:07.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:06 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:07.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.606+0000 7f77952bf700 1 -- 172.21.15.32:0/3145219878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7790107ce0 con 0x7f7790101fa0 2023-09-30T14:37:07.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.606+0000 7f7777fff700 1 -- 172.21.15.32:0/3145219878 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10046 (secure 0 0 0) 0x7f778409a760 con 0x7f7790101fa0 2023-09-30T14:37:07.606 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:37:07.606 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3505064841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3505064841},{"type":"v1","addr":"172.21.15.32:6839","nonce":3505064841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2262846038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2262846038},{"type":"v1","addr":"172.21.15.32:6841","nonce":2262846038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/1445176203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1445176203},{"type":"v1","addr":"172.21.15.156:6839","nonce":1445176203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6839/494821686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":494821686},{"type":"v1","addr":"172.21.15.148:6839","nonce":494821686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-09-30T14:37:07.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.608+0000 7f77952bf700 1 -- 172.21.15.32:0/3145219878 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f77780a0ff0 msgr2=0x7f77780a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:07.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.608+0000 7f77952bf700 1 --2- 172.21.15.32:0/3145219878 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f77780a0ff0 0x7f77780a34b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7780009df0 tx=0x7f7780009260 comp rx=0 tx=0).stop 2023-09-30T14:37:07.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.608+0000 7f77952bf700 1 -- 172.21.15.32:0/3145219878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7790101fa0 msgr2=0x7f7790197030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:07.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.608+0000 7f77952bf700 1 --2- 172.21.15.32:0/3145219878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7790101fa0 0x7f7790197030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7784004760 tx=0x7f7784004790 comp rx=0 tx=0).stop 2023-09-30T14:37:07.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.608+0000 7f77952bf700 1 -- 172.21.15.32:0/3145219878 shutdown_connections 2023-09-30T14:37:07.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.608+0000 7f77952bf700 1 --2- 172.21.15.32:0/3145219878 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f77780a0ff0 0x7f77780a34b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.608+0000 7f77952bf700 1 --2- 172.21.15.32:0/3145219878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7790102960 0x7f779019e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.608+0000 7f77952bf700 1 --2- 172.21.15.32:0/3145219878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7790100e50 0x7f7790196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.608+0000 7f77952bf700 1 --2- 172.21.15.32:0/3145219878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7790101fa0 0x7f7790197030 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.608+0000 7f77952bf700 1 -- 172.21.15.32:0/3145219878 >> 172.21.15.32:0/3145219878 conn(0x7f77900fc3a0 msgr2=0x7f77900fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:07.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.609+0000 7f77952bf700 1 -- 172.21.15.32:0/3145219878 shutdown_connections 2023-09-30T14:37:07.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.609+0000 7f77952bf700 1 -- 172.21.15.32:0/3145219878 wait complete. 2023-09-30T14:37:07.609 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 13 2023-09-30T14:37:07.621 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T14:37:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.752+0000 7fb96d0dd700 1 -- 172.21.15.32:0/513505838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9681018d0 msgr2=0x7fb968108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.752+0000 7fb96d0dd700 1 --2- 172.21.15.32:0/513505838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9681018d0 0x7fb968108180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb96000b0d0 tx=0x7fb96000b3e0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.752+0000 7fb96d0dd700 1 -- 172.21.15.32:0/513505838 shutdown_connections 2023-09-30T14:37:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.752+0000 7fb96d0dd700 1 --2- 172.21.15.32:0/513505838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9681018d0 0x7fb968108180 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.752+0000 7fb96d0dd700 1 --2- 172.21.15.32:0/513505838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9680ffd40 0x7fb968100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.752+0000 7fb96d0dd700 1 --2- 172.21.15.32:0/513505838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb968100f30 0x7fb968101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.752+0000 7fb96d0dd700 1 -- 172.21.15.32:0/513505838 >> 172.21.15.32:0/513505838 conn(0x7fb9680fb2b0 msgr2=0x7fb9680fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.752+0000 7fb96d0dd700 1 -- 172.21.15.32:0/513505838 shutdown_connections 2023-09-30T14:37:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.752+0000 7fb96d0dd700 1 -- 172.21.15.32:0/513505838 wait complete. 2023-09-30T14:37:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96d0dd700 1 Processor -- start 2023-09-30T14:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96d0dd700 1 -- start start 2023-09-30T14:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96d0dd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9680ffd40 0x7fb968069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96d0dd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb968100f30 0x7fb96819b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96d0dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9681018d0 0x7fb96819df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96d0dd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96819e6c0 con 0x7fb9681018d0 2023-09-30T14:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96d0dd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96819e830 con 0x7fb968100f30 2023-09-30T14:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96d0dd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96819e9f0 con 0x7fb9680ffd40 2023-09-30T14:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9681018d0 0x7fb96819df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:07.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb966d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9680ffd40 0x7fb968069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9681018d0 0x7fb96819df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33674/0 (socket says 172.21.15.32:33674) 2023-09-30T14:37:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb968100f30 0x7fb96819b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96759e700 1 -- 172.21.15.32:0/795798684 learned_addr learned my addr 172.21.15.32:0/795798684 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96759e700 1 -- 172.21.15.32:0/795798684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9680ffd40 msgr2=0x7fb968069d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96759e700 1 --2- 172.21.15.32:0/795798684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9680ffd40 0x7fb968069d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96759e700 1 -- 172.21.15.32:0/795798684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb968100f30 msgr2=0x7fb96819b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96759e700 1 --2- 172.21.15.32:0/795798684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb968100f30 0x7fb96819b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96759e700 1 -- 172.21.15.32:0/795798684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb960009d00 con 0x7fb9681018d0 2023-09-30T14:37:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.753+0000 7fb96659c700 1 --2- 172.21.15.32:0/795798684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb968100f30 0x7fb96819b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:37:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.754+0000 7fb96759e700 1 --2- 172.21.15.32:0/795798684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9681018d0 0x7fb96819df20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb960008ec0 tx=0x7fb960008ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:07.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.754+0000 7fb94ffff700 1 -- 172.21.15.32:0/795798684 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96000e040 con 0x7fb9681018d0 2023-09-30T14:37:07.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.754+0000 7fb96d0dd700 1 -- 172.21.15.32:0/795798684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96819ec70 con 0x7fb9681018d0 2023-09-30T14:37:07.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.754+0000 7fb94ffff700 1 -- 172.21.15.32:0/795798684 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb960005070 con 0x7fb9681018d0 2023-09-30T14:37:07.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.754+0000 7fb94ffff700 1 -- 172.21.15.32:0/795798684 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96001b6c0 con 0x7fb9681018d0 2023-09-30T14:37:07.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.754+0000 7fb96d0dd700 1 -- 172.21.15.32:0/795798684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96819f0a0 con 0x7fb9681018d0 2023-09-30T14:37:07.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.755+0000 7fb96d0dd700 1 -- 172.21.15.32:0/795798684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb968107240 con 0x7fb9681018d0 2023-09-30T14:37:07.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.756+0000 7fb94ffff700 1 -- 172.21.15.32:0/795798684 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fb9600051e0 con 0x7fb9681018d0 2023-09-30T14:37:07.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.756+0000 7fb94ffff700 1 --2- 172.21.15.32:0/795798684 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9500a0ff0 0x7fb9500a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:07.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.757+0000 7fb966d9d700 1 --2- 172.21.15.32:0/795798684 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9500a0ff0 0x7fb9500a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:07.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.757+0000 7fb94ffff700 1 -- 172.21.15.32:0/795798684 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb9600cd650 con 0x7fb9681018d0 2023-09-30T14:37:07.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.757+0000 7fb966d9d700 1 --2- 172.21.15.32:0/795798684 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9500a0ff0 0x7fb9500a34b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb968100d90 tx=0x7fb95c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:07.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.760+0000 7fb94ffff700 1 -- 172.21.15.32:0/795798684 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fb960097380 con 0x7fb9681018d0 2023-09-30T14:37:07.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.989+0000 7fb96d0dd700 1 -- 172.21.15.32:0/795798684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb96802d2a0 con 0x7fb9681018d0 2023-09-30T14:37:07.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.990+0000 7fb94ffff700 1 -- 172.21.15.32:0/795798684 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18436 (secure 0 0 0) 0x7fb96009a9a0 con 0x7fb9681018d0 2023-09-30T14:37:07.990 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:37:07.990 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":79,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:36:31.061613+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:37:07.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.992+0000 7fb96d0dd700 1 -- 172.21.15.32:0/795798684 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9500a0ff0 msgr2=0x7fb9500a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:07.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.992+0000 7fb96d0dd700 1 --2- 172.21.15.32:0/795798684 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9500a0ff0 0x7fb9500a34b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb968100d90 tx=0x7fb95c009260 comp rx=0 tx=0).stop 2023-09-30T14:37:07.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.992+0000 7fb96d0dd700 1 -- 172.21.15.32:0/795798684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9681018d0 msgr2=0x7fb96819df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:07.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.992+0000 7fb96d0dd700 1 --2- 172.21.15.32:0/795798684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9681018d0 0x7fb96819df20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb960008ec0 tx=0x7fb960008ef0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.992+0000 7fb96d0dd700 1 -- 172.21.15.32:0/795798684 shutdown_connections 2023-09-30T14:37:07.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.992+0000 7fb96d0dd700 1 --2- 172.21.15.32:0/795798684 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9500a0ff0 0x7fb9500a34b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.992+0000 7fb96d0dd700 1 --2- 172.21.15.32:0/795798684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9681018d0 0x7fb96819df20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.992+0000 7fb96d0dd700 1 --2- 172.21.15.32:0/795798684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9680ffd40 0x7fb968069d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.992+0000 7fb96d0dd700 1 --2- 172.21.15.32:0/795798684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb968100f30 0x7fb96819b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:07.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.992+0000 7fb96d0dd700 1 -- 172.21.15.32:0/795798684 >> 172.21.15.32:0/795798684 conn(0x7fb9680fb2b0 msgr2=0x7fb968104380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:07.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.992+0000 7fb96d0dd700 1 -- 172.21.15.32:0/795798684 shutdown_connections 2023-09-30T14:37:07.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:07.992+0000 7fb96d0dd700 1 -- 172.21.15.32:0/795798684 wait complete. 2023-09-30T14:37:08.006 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T14:33:20.224529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T14:37:08.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T14:37:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.151+0000 7f95c4345700 1 -- 172.21.15.32:0/1225020735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95bc1018e0 msgr2=0x7f95bc103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.151+0000 7f95c4345700 1 --2- 172.21.15.32:0/1225020735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95bc1018e0 0x7f95bc103cd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009990 tx=0x7f95ac009ca0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.151+0000 7f95c4345700 1 -- 172.21.15.32:0/1225020735 shutdown_connections 2023-09-30T14:37:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.151+0000 7f95c4345700 1 --2- 172.21.15.32:0/1225020735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95bc106b40 0x7f95bc108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.151+0000 7f95c4345700 1 --2- 172.21.15.32:0/1225020735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95bc1018e0 0x7f95bc103cd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.151+0000 7f95c4345700 1 --2- 172.21.15.32:0/1225020735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95bc104210 0x7f95bc106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.151+0000 7f95c4345700 1 -- 172.21.15.32:0/1225020735 >> 172.21.15.32:0/1225020735 conn(0x7f95bc0fb2b0 msgr2=0x7f95bc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.151+0000 7f95c4345700 1 -- 172.21.15.32:0/1225020735 shutdown_connections 2023-09-30T14:37:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.151+0000 7f95c4345700 1 -- 172.21.15.32:0/1225020735 wait complete. 2023-09-30T14:37:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.152+0000 7f95c4345700 1 Processor -- start 2023-09-30T14:37:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.152+0000 7f95c4345700 1 -- start start 2023-09-30T14:37:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.152+0000 7f95c4345700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95bc1018e0 0x7f95bc101250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.152+0000 7f95c4345700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95bc104210 0x7f95bc0ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.152+0000 7f95c4345700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95bc106b40 0x7f95bc0ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.152+0000 7f95c4345700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc100570 con 0x7f95bc104210 2023-09-30T14:37:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.152+0000 7f95c4345700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc1006e0 con 0x7f95bc1018e0 2023-09-30T14:37:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.152+0000 7f95c4345700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc1a0b80 con 0x7f95bc106b40 2023-09-30T14:37:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.152+0000 7f95c18e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95bc104210 0x7f95bc0ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.152+0000 7f95c18e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95bc104210 0x7f95bc0ff890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33686/0 (socket says 172.21.15.32:33686) 2023-09-30T14:37:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.152+0000 7f95c18e0700 1 -- 172.21.15.32:0/3980603344 learned_addr learned my addr 172.21.15.32:0/3980603344 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.152+0000 7f95c20e1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95bc1018e0 0x7f95bc101250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.152+0000 7f95c28e2700 1 --2- 172.21.15.32:0/3980603344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95bc106b40 0x7f95bc0ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.153+0000 7f95c18e0700 1 -- 172.21.15.32:0/3980603344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95bc106b40 msgr2=0x7f95bc0ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.153+0000 7f95c18e0700 1 --2- 172.21.15.32:0/3980603344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95bc106b40 0x7f95bc0ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.153+0000 7f95c18e0700 1 -- 172.21.15.32:0/3980603344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95bc1018e0 msgr2=0x7f95bc101250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.153+0000 7f95c18e0700 1 --2- 172.21.15.32:0/3980603344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95bc1018e0 0x7f95bc101250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.153+0000 7f95c18e0700 1 -- 172.21.15.32:0/3980603344 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0095f0 con 0x7f95bc104210 2023-09-30T14:37:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.153+0000 7f95c20e1700 1 --2- 172.21.15.32:0/3980603344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95bc1018e0 0x7f95bc101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:37:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.153+0000 7f95c18e0700 1 --2- 172.21.15.32:0/3980603344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95bc104210 0x7f95bc0ff890 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f95b800e9a0 tx=0x7f95b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.153+0000 7f95b37fe700 1 -- 172.21.15.32:0/3980603344 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8009790 con 0x7f95bc104210 2023-09-30T14:37:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.153+0000 7f95b37fe700 1 -- 172.21.15.32:0/3980603344 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95b8010430 con 0x7f95bc104210 2023-09-30T14:37:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.153+0000 7f95c4345700 1 -- 172.21.15.32:0/3980603344 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95bc1a0e60 con 0x7f95bc104210 2023-09-30T14:37:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.153+0000 7f95b37fe700 1 -- 172.21.15.32:0/3980603344 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8018680 con 0x7f95bc104210 2023-09-30T14:37:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.153+0000 7f95c4345700 1 -- 172.21.15.32:0/3980603344 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95bc1a1330 con 0x7f95bc104210 2023-09-30T14:37:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.154+0000 7f95c4345700 1 -- 172.21.15.32:0/3980603344 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95bc0fce30 con 0x7f95bc104210 2023-09-30T14:37:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.156+0000 7f95b37fe700 1 -- 172.21.15.32:0/3980603344 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f95b80105a0 con 0x7f95bc104210 2023-09-30T14:37:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.156+0000 7f95b37fe700 1 --2- 172.21.15.32:0/3980603344 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f95a80a1020 0x7f95a80a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.156+0000 7f95b37fe700 1 -- 172.21.15.32:0/3980603344 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f95b8014070 con 0x7f95bc104210 2023-09-30T14:37:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.156+0000 7f95c20e1700 1 --2- 172.21.15.32:0/3980603344 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f95a80a1020 0x7f95a80a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.157+0000 7f95c20e1700 1 --2- 172.21.15.32:0/3980603344 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f95a80a1020 0x7f95a80a34e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00b680 tx=0x7f95ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:08.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.160+0000 7f95b37fe700 1 -- 172.21.15.32:0/3980603344 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f95b80962c0 con 0x7f95bc104210 2023-09-30T14:37:08.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:07 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3145219878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:37:08.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:07 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/795798684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:08.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:07 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3145219878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:37:08.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:07 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/795798684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:08.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.364+0000 7f95c4345700 1 -- 172.21.15.32:0/3980603344 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95bc10c7d0 con 0x7f95bc104210 2023-09-30T14:37:08.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.365+0000 7f95b37fe700 1 -- 172.21.15.32:0/3980603344 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95b80998e0 con 0x7f95bc104210 2023-09-30T14:37:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T14:37:08.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.367+0000 7f95c4345700 1 -- 172.21.15.32:0/3980603344 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f95a80a1020 msgr2=0x7f95a80a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:08.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.367+0000 7f95c4345700 1 --2- 172.21.15.32:0/3980603344 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f95a80a1020 0x7f95a80a34e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00b680 tx=0x7f95ac005fd0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.367+0000 7f95c4345700 1 -- 172.21.15.32:0/3980603344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95bc104210 msgr2=0x7f95bc0ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:08.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.367+0000 7f95c4345700 1 --2- 172.21.15.32:0/3980603344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95bc104210 0x7f95bc0ff890 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f95b800e9a0 tx=0x7f95b800ecb0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.367+0000 7f95c4345700 1 -- 172.21.15.32:0/3980603344 shutdown_connections 2023-09-30T14:37:08.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.367+0000 7f95c4345700 1 --2- 172.21.15.32:0/3980603344 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f95a80a1020 0x7f95a80a34e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.367+0000 7f95c4345700 1 --2- 172.21.15.32:0/3980603344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95bc106b40 0x7f95bc0ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.367+0000 7f95c4345700 1 --2- 172.21.15.32:0/3980603344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95bc1018e0 0x7f95bc101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.367+0000 7f95c4345700 1 --2- 172.21.15.32:0/3980603344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95bc104210 0x7f95bc0ff890 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.367+0000 7f95c4345700 1 -- 172.21.15.32:0/3980603344 >> 172.21.15.32:0/3980603344 conn(0x7f95bc0fb2b0 msgr2=0x7f95bc102550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:08.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.367+0000 7f95c4345700 1 -- 172.21.15.32:0/3980603344 shutdown_connections 2023-09-30T14:37:08.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.367+0000 7f95c4345700 1 -- 172.21.15.32:0/3980603344 wait complete. 2023-09-30T14:37:08.377 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:07 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3145219878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:37:08.378 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:07 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/795798684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:08.379 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-30T14:37:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 -- 172.21.15.32:0/2154134379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce81018d0 msgr2=0x7fbce8108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 --2- 172.21.15.32:0/2154134379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce81018d0 0x7fbce8108180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbce000b0d0 tx=0x7fbce000b3e0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 -- 172.21.15.32:0/2154134379 shutdown_connections 2023-09-30T14:37:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 --2- 172.21.15.32:0/2154134379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce81018d0 0x7fbce8108180 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 --2- 172.21.15.32:0/2154134379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbce80ffd40 0x7fbce8100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 --2- 172.21.15.32:0/2154134379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce8100f30 0x7fbce8101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 -- 172.21.15.32:0/2154134379 >> 172.21.15.32:0/2154134379 conn(0x7fbce80fb270 msgr2=0x7fbce80fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 -- 172.21.15.32:0/2154134379 shutdown_connections 2023-09-30T14:37:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 -- 172.21.15.32:0/2154134379 wait complete. 2023-09-30T14:37:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 Processor -- start 2023-09-30T14:37:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 -- start start 2023-09-30T14:37:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbce80ffd40 0x7fbce8069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce8100f30 0x7fbce819b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce81018d0 0x7fbce819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce819e750 con 0x7fbce8100f30 2023-09-30T14:37:08.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce819e8c0 con 0x7fbce80ffd40 2023-09-30T14:37:08.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.512+0000 7fbceea69700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce819ea80 con 0x7fbce81018d0 2023-09-30T14:37:08.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbce7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce8100f30 0x7fbce819b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:08.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbcec805700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbce80ffd40 0x7fbce8069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:08.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbce7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce8100f30 0x7fbce819b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33700/0 (socket says 172.21.15.32:33700) 2023-09-30T14:37:08.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbce7fff700 1 -- 172.21.15.32:0/2638992945 learned_addr learned my addr 172.21.15.32:0/2638992945 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:08.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbced006700 1 --2- 172.21.15.32:0/2638992945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce81018d0 0x7fbce819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:08.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbce7fff700 1 -- 172.21.15.32:0/2638992945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce81018d0 msgr2=0x7fbce819df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:08.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbce7fff700 1 --2- 172.21.15.32:0/2638992945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce81018d0 0x7fbce819df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbce7fff700 1 -- 172.21.15.32:0/2638992945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbce80ffd40 msgr2=0x7fbce8069b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbce7fff700 1 --2- 172.21.15.32:0/2638992945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbce80ffd40 0x7fbce8069b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbce7fff700 1 -- 172.21.15.32:0/2638992945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce0009d00 con 0x7fbce8100f30 2023-09-30T14:37:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbced006700 1 --2- 172.21.15.32:0/2638992945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce81018d0 0x7fbce819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:37:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbce7fff700 1 --2- 172.21.15.32:0/2638992945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce8100f30 0x7fbce819b930 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400d950 tx=0x7fbcd400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbce5ffb700 1 -- 172.21.15.32:0/2638992945 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4009940 con 0x7fbce8100f30 2023-09-30T14:37:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbce5ffb700 1 -- 172.21.15.32:0/2638992945 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcd4010460 con 0x7fbce8100f30 2023-09-30T14:37:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbceea69700 1 -- 172.21.15.32:0/2638992945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce819ed60 con 0x7fbce8100f30 2023-09-30T14:37:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbce5ffb700 1 -- 172.21.15.32:0/2638992945 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd400f620 con 0x7fbce8100f30 2023-09-30T14:37:08.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.513+0000 7fbceea69700 1 -- 172.21.15.32:0/2638992945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce819f210 con 0x7fbce8100f30 2023-09-30T14:37:08.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.514+0000 7fbceea69700 1 -- 172.21.15.32:0/2638992945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce81071e0 con 0x7fbce8100f30 2023-09-30T14:37:08.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.515+0000 7fbce5ffb700 1 -- 172.21.15.32:0/2638992945 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fbcd40108d0 con 0x7fbce8100f30 2023-09-30T14:37:08.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.516+0000 7fbce5ffb700 1 --2- 172.21.15.32:0/2638992945 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbcd80a0ff0 0x7fbcd80a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:08.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.516+0000 7fbcec805700 1 --2- 172.21.15.32:0/2638992945 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbcd80a0ff0 0x7fbcd80a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:08.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.516+0000 7fbce5ffb700 1 -- 172.21.15.32:0/2638992945 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbcd40cc130 con 0x7fbce8100f30 2023-09-30T14:37:08.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.516+0000 7fbcec805700 1 --2- 172.21.15.32:0/2638992945 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbcd80a0ff0 0x7fbcd80a34b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbce8100d90 tx=0x7fbcdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:08.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.520+0000 7fbce5ffb700 1 -- 172.21.15.32:0/2638992945 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fbcd4095d30 con 0x7fbce8100f30 2023-09-30T14:37:08.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.747+0000 7fbceea69700 1 -- 172.21.15.32:0/2638992945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbce810b5b0 con 0x7fbce8100f30 2023-09-30T14:37:08.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.747+0000 7fbce5ffb700 1 -- 172.21.15.32:0/2638992945 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbcd4099350 con 0x7fbce8100f30 2023-09-30T14:37:08.747 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:37:08.747 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-09-30T14:37:08.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T14:37:08.747 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-09-30T14:37:08.748 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-09-30T14:37:08.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T14:37:08.748 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-09-30T14:37:08.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.749+0000 7fbceea69700 1 -- 172.21.15.32:0/2638992945 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbcd80a0ff0 msgr2=0x7fbcd80a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.749+0000 7fbceea69700 1 --2- 172.21.15.32:0/2638992945 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbcd80a0ff0 0x7fbcd80a34b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbce8100d90 tx=0x7fbcdc009260 comp rx=0 tx=0).stop 2023-09-30T14:37:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.749+0000 7fbceea69700 1 -- 172.21.15.32:0/2638992945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce8100f30 msgr2=0x7fbce819b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.749+0000 7fbceea69700 1 --2- 172.21.15.32:0/2638992945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce8100f30 0x7fbce819b930 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400d950 tx=0x7fbcd400dc60 comp rx=0 tx=0).stop 2023-09-30T14:37:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.750+0000 7fbceea69700 1 -- 172.21.15.32:0/2638992945 shutdown_connections 2023-09-30T14:37:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.750+0000 7fbceea69700 1 --2- 172.21.15.32:0/2638992945 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbcd80a0ff0 0x7fbcd80a34b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.750+0000 7fbceea69700 1 --2- 172.21.15.32:0/2638992945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce81018d0 0x7fbce819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.750+0000 7fbceea69700 1 --2- 172.21.15.32:0/2638992945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbce80ffd40 0x7fbce8069b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.750+0000 7fbceea69700 1 --2- 172.21.15.32:0/2638992945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbce8100f30 0x7fbce819b930 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.750+0000 7fbceea69700 1 -- 172.21.15.32:0/2638992945 >> 172.21.15.32:0/2638992945 conn(0x7fbce80fb270 msgr2=0x7fbce8104320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.750+0000 7fbceea69700 1 -- 172.21.15.32:0/2638992945 shutdown_connections 2023-09-30T14:37:08.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.750+0000 7fbceea69700 1 -- 172.21.15.32:0/2638992945 wait complete. 2023-09-30T14:37:08.763 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-30T14:37:08.763 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-30T14:37:08.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.896+0000 7f9b786af700 1 -- 172.21.15.32:0/1370226289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b700ffd40 msgr2=0x7f9b70100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:08.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.896+0000 7f9b786af700 1 --2- 172.21.15.32:0/1370226289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b700ffd40 0x7f9b70100160 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0099c0 tx=0x7f9b6c009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.896+0000 7f9b786af700 1 -- 172.21.15.32:0/1370226289 shutdown_connections 2023-09-30T14:37:08.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.896+0000 7f9b786af700 1 --2- 172.21.15.32:0/1370226289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b701018d0 0x7f9b70108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.896+0000 7f9b786af700 1 --2- 172.21.15.32:0/1370226289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b700ffd40 0x7f9b70100160 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0099c0 tx=0x7f9b6c009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.896+0000 7f9b786af700 1 --2- 172.21.15.32:0/1370226289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b70100f30 0x7f9b70101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.896+0000 7f9b786af700 1 -- 172.21.15.32:0/1370226289 >> 172.21.15.32:0/1370226289 conn(0x7f9b700fb2b0 msgr2=0x7f9b700fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:08.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.897+0000 7f9b786af700 1 -- 172.21.15.32:0/1370226289 shutdown_connections 2023-09-30T14:37:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.897+0000 7f9b786af700 1 -- 172.21.15.32:0/1370226289 wait complete. 2023-09-30T14:37:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b786af700 1 Processor -- start 2023-09-30T14:37:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b786af700 1 -- start start 2023-09-30T14:37:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b786af700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b70100f30 0x7f9b70196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b786af700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b701018d0 0x7f9b701972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b786af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b7019c300 0x7f9b7019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b786af700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7019ecc0 con 0x7f9b7019c300 2023-09-30T14:37:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b786af700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7019ee30 con 0x7f9b70100f30 2023-09-30T14:37:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b786af700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7019eff0 con 0x7f9b701018d0 2023-09-30T14:37:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b76c4c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b7019c300 0x7f9b7019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b76c4c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b7019c300 0x7f9b7019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33716/0 (socket says 172.21.15.32:33716) 2023-09-30T14:37:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b76c4c700 1 -- 172.21.15.32:0/1625818056 learned_addr learned my addr 172.21.15.32:0/1625818056 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b7644b700 1 --2- 172.21.15.32:0/1625818056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b70100f30 0x7f9b70196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b75c4a700 1 --2- 172.21.15.32:0/1625818056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b701018d0 0x7f9b701972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b76c4c700 1 -- 172.21.15.32:0/1625818056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b701018d0 msgr2=0x7f9b701972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b76c4c700 1 --2- 172.21.15.32:0/1625818056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b701018d0 0x7f9b701972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b76c4c700 1 -- 172.21.15.32:0/1625818056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b70100f30 msgr2=0x7f9b70196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b76c4c700 1 --2- 172.21.15.32:0/1625818056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b70100f30 0x7f9b70196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b76c4c700 1 -- 172.21.15.32:0/1625818056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c0095f0 con 0x7f9b7019c300 2023-09-30T14:37:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.898+0000 7f9b7644b700 1 --2- 172.21.15.32:0/1625818056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b70100f30 0x7f9b70196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:37:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.899+0000 7f9b76c4c700 1 --2- 172.21.15.32:0/1625818056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b7019c300 0x7f9b7019e6f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800ef30 tx=0x7f9b6800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:08.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.903+0000 7f9b677fe700 1 -- 172.21.15.32:0/1625818056 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b68018070 con 0x7f9b7019c300 2023-09-30T14:37:08.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.903+0000 7f9b786af700 1 -- 172.21.15.32:0/1625818056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b7019f270 con 0x7f9b7019c300 2023-09-30T14:37:08.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.903+0000 7f9b786af700 1 -- 172.21.15.32:0/1625818056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7019f750 con 0x7f9b7019c300 2023-09-30T14:37:08.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.903+0000 7f9b677fe700 1 -- 172.21.15.32:0/1625818056 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b6800f040 con 0x7f9b7019c300 2023-09-30T14:37:08.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.903+0000 7f9b677fe700 1 -- 172.21.15.32:0/1625818056 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6800cd00 con 0x7f9b7019c300 2023-09-30T14:37:08.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.904+0000 7f9b786af700 1 -- 172.21.15.32:0/1625818056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b54005320 con 0x7f9b7019c300 2023-09-30T14:37:08.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.906+0000 7f9b677fe700 1 -- 172.21.15.32:0/1625818056 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f9b68004ad0 con 0x7f9b7019c300 2023-09-30T14:37:08.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.906+0000 7f9b677fe700 1 --2- 172.21.15.32:0/1625818056 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9b5c0a1050 0x7f9b5c0a3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:08.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.907+0000 7f9b677fe700 1 -- 172.21.15.32:0/1625818056 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9b680cbc40 con 0x7f9b7019c300 2023-09-30T14:37:08.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.907+0000 7f9b7644b700 1 --2- 172.21.15.32:0/1625818056 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9b5c0a1050 0x7f9b5c0a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:08.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.907+0000 7f9b7644b700 1 --2- 172.21.15.32:0/1625818056 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9b5c0a1050 0x7f9b5c0a3510 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0058e0 tx=0x7f9b6c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:08.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:08.911+0000 7f9b677fe700 1 -- 172.21.15.32:0/1625818056 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f9b68095970 con 0x7f9b7019c300 2023-09-30T14:37:09.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:09.123+0000 7f9b786af700 1 -- 172.21.15.32:0/1625818056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9b54005670 con 0x7f9b7019c300 2023-09-30T14:37:09.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:08 smithi032 ceph-mon[98592]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:09.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:08 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3980603344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:37:09.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:08 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2638992945' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-30T14:37:09.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:08 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:08 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:08 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:08 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:08 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:08 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:08 smithi156 ceph-mon[103579]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:09.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:08 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3980603344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:37:09.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:08 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2638992945' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-30T14:37:09.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:08 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:08 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:08 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:08 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:08 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:08 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:08 smithi148 ceph-mon[102355]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:09.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:08 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3980603344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:37:09.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:08 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2638992945' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-30T14:37:09.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:08 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:08 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:08 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:08 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:08 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:09.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:08 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.005+0000 7f9b677fe700 1 -- 172.21.15.32:0/1625818056 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9b68098f90 con 0x7f9b7019c300 2023-09-30T14:37:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.009+0000 7f9b786af700 1 -- 172.21.15.32:0/1625818056 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9b5c0a1050 msgr2=0x7f9b5c0a3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.009+0000 7f9b786af700 1 --2- 172.21.15.32:0/1625818056 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9b5c0a1050 0x7f9b5c0a3510 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0058e0 tx=0x7f9b6c00b540 comp rx=0 tx=0).stop 2023-09-30T14:37:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.009+0000 7f9b786af700 1 -- 172.21.15.32:0/1625818056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b7019c300 msgr2=0x7f9b7019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.009+0000 7f9b786af700 1 --2- 172.21.15.32:0/1625818056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b7019c300 0x7f9b7019e6f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800ef30 tx=0x7f9b6800c3a0 comp rx=0 tx=0).stop 2023-09-30T14:37:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.009+0000 7f9b786af700 1 -- 172.21.15.32:0/1625818056 shutdown_connections 2023-09-30T14:37:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.009+0000 7f9b786af700 1 --2- 172.21.15.32:0/1625818056 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9b5c0a1050 0x7f9b5c0a3510 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:10.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.009+0000 7f9b786af700 1 --2- 172.21.15.32:0/1625818056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b7019c300 0x7f9b7019e6f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:10.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.009+0000 7f9b786af700 1 --2- 172.21.15.32:0/1625818056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b70100f30 0x7f9b70196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:10.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.009+0000 7f9b786af700 1 --2- 172.21.15.32:0/1625818056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b701018d0 0x7f9b701972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:10.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.009+0000 7f9b786af700 1 -- 172.21.15.32:0/1625818056 >> 172.21.15.32:0/1625818056 conn(0x7f9b700fb2b0 msgr2=0x7f9b701045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.010+0000 7f9b786af700 1 -- 172.21.15.32:0/1625818056 shutdown_connections 2023-09-30T14:37:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.011+0000 7f9b786af700 1 -- 172.21.15.32:0/1625818056 wait complete. 2023-09-30T14:37:10.011 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-09-30T14:37:10.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-30T14:37:10.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.173+0000 7f3a1d8cb700 1 -- 172.21.15.32:0/1611595681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a18102970 msgr2=0x7f3a18109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:10.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.173+0000 7f3a1d8cb700 1 --2- 172.21.15.32:0/1611595681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a18102970 0x7f3a18109290 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000b0a0 tx=0x7f3a1000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:37:10.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.173+0000 7f3a1d8cb700 1 -- 172.21.15.32:0/1611595681 shutdown_connections 2023-09-30T14:37:10.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.173+0000 7f3a1d8cb700 1 --2- 172.21.15.32:0/1611595681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a18102970 0x7f3a18109290 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:10.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.173+0000 7f3a1d8cb700 1 --2- 172.21.15.32:0/1611595681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a18100e60 0x7f3a18101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:10.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.173+0000 7f3a1d8cb700 1 --2- 172.21.15.32:0/1611595681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a18101fb0 0x7f3a18102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:10.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.173+0000 7f3a1d8cb700 1 -- 172.21.15.32:0/1611595681 >> 172.21.15.32:0/1611595681 conn(0x7f3a180fc3d0 msgr2=0x7f3a180fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:10.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.174+0000 7f3a1d8cb700 1 -- 172.21.15.32:0/1611595681 shutdown_connections 2023-09-30T14:37:10.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.174+0000 7f3a1d8cb700 1 -- 172.21.15.32:0/1611595681 wait complete. 2023-09-30T14:37:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.174+0000 7f3a1d8cb700 1 Processor -- start 2023-09-30T14:37:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.174+0000 7f3a1d8cb700 1 -- start start 2023-09-30T14:37:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.174+0000 7f3a1d8cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a18100e60 0x7f3a18196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.174+0000 7f3a1d8cb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a18101fb0 0x7f3a181971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.174+0000 7f3a1d8cb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a18102970 0x7f3a1819e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.174+0000 7f3a1d8cb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1819e7c0 con 0x7f3a18100e60 2023-09-30T14:37:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.174+0000 7f3a1d8cb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1819e930 con 0x7f3a18101fb0 2023-09-30T14:37:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.174+0000 7f3a1d8cb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1819eaf0 con 0x7f3a18102970 2023-09-30T14:37:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.175+0000 7f3a16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a18100e60 0x7f3a18196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.175+0000 7f3a16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a18100e60 0x7f3a18196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33726/0 (socket says 172.21.15.32:33726) 2023-09-30T14:37:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.175+0000 7f3a16ffd700 1 -- 172.21.15.32:0/550823246 learned_addr learned my addr 172.21.15.32:0/550823246 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.175+0000 7f3a177fe700 1 --2- 172.21.15.32:0/550823246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a18102970 0x7f3a1819e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.175+0000 7f3a167fc700 1 --2- 172.21.15.32:0/550823246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a18101fb0 0x7f3a181971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.175+0000 7f3a16ffd700 1 -- 172.21.15.32:0/550823246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a18102970 msgr2=0x7f3a1819e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.175+0000 7f3a16ffd700 1 --2- 172.21.15.32:0/550823246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a18102970 0x7f3a1819e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.175+0000 7f3a16ffd700 1 -- 172.21.15.32:0/550823246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a18101fb0 msgr2=0x7f3a181971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.175+0000 7f3a16ffd700 1 --2- 172.21.15.32:0/550823246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a18101fb0 0x7f3a181971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.175+0000 7f3a16ffd700 1 -- 172.21.15.32:0/550823246 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a10009d00 con 0x7f3a18100e60 2023-09-30T14:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.175+0000 7f3a16ffd700 1 --2- 172.21.15.32:0/550823246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a18100e60 0x7f3a18196c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00d980 tx=0x7f3a0c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.175+0000 7f3a1c8c9700 1 -- 172.21.15.32:0/550823246 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c004220 con 0x7f3a18100e60 2023-09-30T14:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.175+0000 7f3a1d8cb700 1 -- 172.21.15.32:0/550823246 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a180730b0 con 0x7f3a18100e60 2023-09-30T14:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.175+0000 7f3a1c8c9700 1 -- 172.21.15.32:0/550823246 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a0c007640 con 0x7f3a18100e60 2023-09-30T14:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.175+0000 7f3a1c8c9700 1 -- 172.21.15.32:0/550823246 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c005690 con 0x7f3a18100e60 2023-09-30T14:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.175+0000 7f3a1d8cb700 1 -- 172.21.15.32:0/550823246 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a18073610 con 0x7f3a18100e60 2023-09-30T14:37:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.176+0000 7f3a1d8cb700 1 -- 172.21.15.32:0/550823246 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1810c6b0 con 0x7f3a18100e60 2023-09-30T14:37:10.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.178+0000 7f3a1c8c9700 1 -- 172.21.15.32:0/550823246 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f3a0c004750 con 0x7f3a18100e60 2023-09-30T14:37:10.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.179+0000 7f3a1c8c9700 1 --2- 172.21.15.32:0/550823246 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3a000a0fa0 0x7f3a000a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:10.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.179+0000 7f3a167fc700 1 --2- 172.21.15.32:0/550823246 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3a000a0fa0 0x7f3a000a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:10.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.179+0000 7f3a1c8c9700 1 -- 172.21.15.32:0/550823246 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f3a0c0cc7b0 con 0x7f3a18100e60 2023-09-30T14:37:10.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.179+0000 7f3a167fc700 1 --2- 172.21.15.32:0/550823246 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3a000a0fa0 0x7f3a000a3460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009be0 tx=0x7f3a08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:10.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.182+0000 7f3a1c8c9700 1 -- 172.21.15.32:0/550823246 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f3a0c096240 con 0x7f3a18100e60 2023-09-30T14:37:10.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:10 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1625818056' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-30T14:37:10.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:10 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:37:10.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:10 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:37:10.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:10 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:10.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:10 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:37:10.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:10 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1625818056' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-30T14:37:10.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:10 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:37:10.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:10 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:37:10.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:10 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:10.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:10 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:37:10.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:10 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1625818056' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-30T14:37:10.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:10 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:37:10.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:10 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:37:10.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:10 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:10.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:10 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:37:10.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:10.392+0000 7f3a1d8cb700 1 -- 172.21.15.32:0/550823246 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3a18073ae0 con 0x7f3a18100e60 2023-09-30T14:37:11.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.008+0000 7f3a1c8c9700 1 -- 172.21.15.32:0/550823246 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3a0c099860 con 0x7f3a18100e60 2023-09-30T14:37:11.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.011+0000 7f3a1d8cb700 1 -- 172.21.15.32:0/550823246 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3a000a0fa0 msgr2=0x7f3a000a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:11.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.011+0000 7f3a1d8cb700 1 --2- 172.21.15.32:0/550823246 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3a000a0fa0 0x7f3a000a3460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009be0 tx=0x7f3a08009260 comp rx=0 tx=0).stop 2023-09-30T14:37:11.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.011+0000 7f3a1d8cb700 1 -- 172.21.15.32:0/550823246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a18100e60 msgr2=0x7f3a18196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:11.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.011+0000 7f3a1d8cb700 1 --2- 172.21.15.32:0/550823246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a18100e60 0x7f3a18196c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00d980 tx=0x7f3a0c00dc90 comp rx=0 tx=0).stop 2023-09-30T14:37:11.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.012+0000 7f3a1d8cb700 1 -- 172.21.15.32:0/550823246 shutdown_connections 2023-09-30T14:37:11.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.012+0000 7f3a1d8cb700 1 --2- 172.21.15.32:0/550823246 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3a000a0fa0 0x7f3a000a3460 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:11.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.012+0000 7f3a1d8cb700 1 --2- 172.21.15.32:0/550823246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a18102970 0x7f3a1819e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:11.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.012+0000 7f3a1d8cb700 1 --2- 172.21.15.32:0/550823246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a18100e60 0x7f3a18196c80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:11.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.012+0000 7f3a1d8cb700 1 --2- 172.21.15.32:0/550823246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a18101fb0 0x7f3a181971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:11.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.012+0000 7f3a1d8cb700 1 -- 172.21.15.32:0/550823246 >> 172.21.15.32:0/550823246 conn(0x7f3a180fc3d0 msgr2=0x7f3a180fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:11.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.018+0000 7f3a1d8cb700 1 -- 172.21.15.32:0/550823246 shutdown_connections 2023-09-30T14:37:11.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.024+0000 7f3a1d8cb700 1 -- 172.21.15.32:0/550823246 wait complete. 2023-09-30T14:37:11.028 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-09-30T14:37:11.042 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-30T14:37:11.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.177+0000 7fb13c50e700 1 -- 172.21.15.32:0/814897346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb134104200 msgr2=0x7fb1341065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:11.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.177+0000 7fb13c50e700 1 --2- 172.21.15.32:0/814897346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb134104200 0x7fb1341065f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb130009990 tx=0x7fb130009ca0 comp rx=0 tx=0).stop 2023-09-30T14:37:11.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.177+0000 7fb13c50e700 1 -- 172.21.15.32:0/814897346 shutdown_connections 2023-09-30T14:37:11.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.177+0000 7fb13c50e700 1 --2- 172.21.15.32:0/814897346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb134106b30 0x7fb134108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:11.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.177+0000 7fb13c50e700 1 --2- 172.21.15.32:0/814897346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1341018d0 0x7fb134103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:11.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.177+0000 7fb13c50e700 1 --2- 172.21.15.32:0/814897346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb134104200 0x7fb1341065f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:11.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.177+0000 7fb13c50e700 1 -- 172.21.15.32:0/814897346 >> 172.21.15.32:0/814897346 conn(0x7fb1340fb2a0 msgr2=0x7fb1340fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:11.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.177+0000 7fb13c50e700 1 -- 172.21.15.32:0/814897346 shutdown_connections 2023-09-30T14:37:11.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.177+0000 7fb13c50e700 1 -- 172.21.15.32:0/814897346 wait complete. 2023-09-30T14:37:11.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13c50e700 1 Processor -- start 2023-09-30T14:37:11.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13c50e700 1 -- start start 2023-09-30T14:37:11.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13c50e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1341018d0 0x7fb134196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:11.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13c50e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb134104200 0x7fb134197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:11.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13c50e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb134106b30 0x7fb13419e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:11.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13c50e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13419e6a0 con 0x7fb134106b30 2023-09-30T14:37:11.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13c50e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13419e7e0 con 0x7fb134104200 2023-09-30T14:37:11.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13c50e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13419e970 con 0x7fb1341018d0 2023-09-30T14:37:11.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13aaab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb134106b30 0x7fb13419e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:11.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13aaab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb134106b30 0x7fb13419e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33742/0 (socket says 172.21.15.32:33742) 2023-09-30T14:37:11.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13aaab700 1 -- 172.21.15.32:0/1937418655 learned_addr learned my addr 172.21.15.32:0/1937418655 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:11.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb139aa9700 1 --2- 172.21.15.32:0/1937418655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb134104200 0x7fb134197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13a2aa700 1 --2- 172.21.15.32:0/1937418655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1341018d0 0x7fb134196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13aaab700 1 -- 172.21.15.32:0/1937418655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1341018d0 msgr2=0x7fb134196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13aaab700 1 --2- 172.21.15.32:0/1937418655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1341018d0 0x7fb134196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13aaab700 1 -- 172.21.15.32:0/1937418655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb134104200 msgr2=0x7fb134197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13aaab700 1 --2- 172.21.15.32:0/1937418655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb134104200 0x7fb134197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.178+0000 7fb13aaab700 1 -- 172.21.15.32:0/1937418655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1300095f0 con 0x7fb134106b30 2023-09-30T14:37:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.179+0000 7fb13aaab700 1 --2- 172.21.15.32:0/1937418655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb134106b30 0x7fb13419e0d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00ef30 tx=0x7fb12c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.179+0000 7fb12b7fe700 1 -- 172.21.15.32:0/1937418655 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c018070 con 0x7fb134106b30 2023-09-30T14:37:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.179+0000 7fb12b7fe700 1 -- 172.21.15.32:0/1937418655 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb12c00f040 con 0x7fb134106b30 2023-09-30T14:37:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.179+0000 7fb12b7fe700 1 -- 172.21.15.32:0/1937418655 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c00cdb0 con 0x7fb134106b30 2023-09-30T14:37:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.179+0000 7fb13c50e700 1 -- 172.21.15.32:0/1937418655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13419ebf0 con 0x7fb134106b30 2023-09-30T14:37:11.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.179+0000 7fb13c50e700 1 -- 172.21.15.32:0/1937418655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13419f0b0 con 0x7fb134106b30 2023-09-30T14:37:11.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.180+0000 7fb13c50e700 1 -- 172.21.15.32:0/1937418655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1340fce20 con 0x7fb134106b30 2023-09-30T14:37:11.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.181+0000 7fb12b7fe700 1 -- 172.21.15.32:0/1937418655 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fb12c003a00 con 0x7fb134106b30 2023-09-30T14:37:11.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.181+0000 7fb12b7fe700 1 --2- 172.21.15.32:0/1937418655 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb1200a1020 0x7fb1200a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:11.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.182+0000 7fb13a2aa700 1 --2- 172.21.15.32:0/1937418655 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb1200a1020 0x7fb1200a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:11.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.182+0000 7fb12b7fe700 1 -- 172.21.15.32:0/1937418655 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fb12c0cca80 con 0x7fb134106b30 2023-09-30T14:37:11.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.182+0000 7fb13a2aa700 1 --2- 172.21.15.32:0/1937418655 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb1200a1020 0x7fb1200a34e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb124009bf0 tx=0x7fb124009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:11.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.185+0000 7fb12b7fe700 1 -- 172.21.15.32:0/1937418655 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fb12c0964c0 con 0x7fb134106b30 2023-09-30T14:37:11.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:11 smithi156 ceph-mon[103579]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:11.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:11 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1625818056' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-30T14:37:11.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:11 smithi156 ceph-mon[103579]: osdmap e80: 12 total, 12 up, 12 in 2023-09-30T14:37:11.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:11 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/550823246' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-30T14:37:11.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:11 smithi148 ceph-mon[102355]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:11.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:11 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1625818056' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-30T14:37:11.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:11 smithi148 ceph-mon[102355]: osdmap e80: 12 total, 12 up, 12 in 2023-09-30T14:37:11.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:11 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/550823246' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-30T14:37:11.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:11 smithi032 ceph-mon[98592]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:11.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:11 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1625818056' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-30T14:37:11.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:11 smithi032 ceph-mon[98592]: osdmap e80: 12 total, 12 up, 12 in 2023-09-30T14:37:11.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:11 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/550823246' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-30T14:37:11.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:11.443+0000 7fb13c50e700 1 -- 172.21.15.32:0/1937418655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb13410c850 con 0x7fb134106b30 2023-09-30T14:37:12.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.032+0000 7fb12b7fe700 1 -- 172.21.15.32:0/1937418655 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb12c099ae0 con 0x7fb134106b30 2023-09-30T14:37:12.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.034+0000 7fb13c50e700 1 -- 172.21.15.32:0/1937418655 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb1200a1020 msgr2=0x7fb1200a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:12.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.034+0000 7fb13c50e700 1 --2- 172.21.15.32:0/1937418655 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb1200a1020 0x7fb1200a34e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb124009bf0 tx=0x7fb124009260 comp rx=0 tx=0).stop 2023-09-30T14:37:12.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.034+0000 7fb13c50e700 1 -- 172.21.15.32:0/1937418655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb134106b30 msgr2=0x7fb13419e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.034+0000 7fb13c50e700 1 --2- 172.21.15.32:0/1937418655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb134106b30 0x7fb13419e0d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00ef30 tx=0x7fb12c00c3a0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.034+0000 7fb13c50e700 1 -- 172.21.15.32:0/1937418655 shutdown_connections 2023-09-30T14:37:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.034+0000 7fb13c50e700 1 --2- 172.21.15.32:0/1937418655 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb1200a1020 0x7fb1200a34e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.034+0000 7fb13c50e700 1 --2- 172.21.15.32:0/1937418655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb134106b30 0x7fb13419e0d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.034+0000 7fb13c50e700 1 --2- 172.21.15.32:0/1937418655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1341018d0 0x7fb134196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.034+0000 7fb13c50e700 1 --2- 172.21.15.32:0/1937418655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb134104200 0x7fb134197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.034+0000 7fb13c50e700 1 -- 172.21.15.32:0/1937418655 >> 172.21.15.32:0/1937418655 conn(0x7fb1340fb2a0 msgr2=0x7fb134102540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.035+0000 7fb13c50e700 1 -- 172.21.15.32:0/1937418655 shutdown_connections 2023-09-30T14:37:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.035+0000 7fb13c50e700 1 -- 172.21.15.32:0/1937418655 wait complete. 2023-09-30T14:37:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 2 and data pool 3 2023-09-30T14:37:12.049 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T14:37:12.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.182+0000 7fe163d34700 1 -- 172.21.15.32:0/1873914294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c0ffd40 msgr2=0x7fe15c100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:12.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.182+0000 7fe163d34700 1 --2- 172.21.15.32:0/1873914294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c0ffd40 0x7fe15c100160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe1580099c0 tx=0x7fe158009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.182+0000 7fe163d34700 1 -- 172.21.15.32:0/1873914294 shutdown_connections 2023-09-30T14:37:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.182+0000 7fe163d34700 1 --2- 172.21.15.32:0/1873914294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe15c1018d0 0x7fe15c108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.182+0000 7fe163d34700 1 --2- 172.21.15.32:0/1873914294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c0ffd40 0x7fe15c100160 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.182+0000 7fe163d34700 1 --2- 172.21.15.32:0/1873914294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe15c100f30 0x7fe15c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.183+0000 7fe163d34700 1 -- 172.21.15.32:0/1873914294 >> 172.21.15.32:0/1873914294 conn(0x7fe15c0fb270 msgr2=0x7fe15c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.183+0000 7fe163d34700 1 -- 172.21.15.32:0/1873914294 shutdown_connections 2023-09-30T14:37:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.183+0000 7fe163d34700 1 -- 172.21.15.32:0/1873914294 wait complete. 2023-09-30T14:37:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.183+0000 7fe163d34700 1 Processor -- start 2023-09-30T14:37:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.183+0000 7fe163d34700 1 -- start start 2023-09-30T14:37:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.183+0000 7fe163d34700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe15c0ffd40 0x7fe15c195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:12.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.183+0000 7fe163d34700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c100f30 0x7fe15c195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.183+0000 7fe163d34700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe15c1018d0 0x7fe15c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.183+0000 7fe163d34700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c19d4e0 con 0x7fe15c100f30 2023-09-30T14:37:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.183+0000 7fe163d34700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c19d620 con 0x7fe15c1018d0 2023-09-30T14:37:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.183+0000 7fe163d34700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c19d7b0 con 0x7fe15c0ffd40 2023-09-30T14:37:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe1612cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c100f30 0x7fe15c195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe1612cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c100f30 0x7fe15c195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33762/0 (socket says 172.21.15.32:33762) 2023-09-30T14:37:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe1612cf700 1 -- 172.21.15.32:0/1276702578 learned_addr learned my addr 172.21.15.32:0/1276702578 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe161ad0700 1 --2- 172.21.15.32:0/1276702578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe15c0ffd40 0x7fe15c195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe1622d1700 1 --2- 172.21.15.32:0/1276702578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe15c1018d0 0x7fe15c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe1612cf700 1 -- 172.21.15.32:0/1276702578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe15c0ffd40 msgr2=0x7fe15c195960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe1612cf700 1 --2- 172.21.15.32:0/1276702578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe15c0ffd40 0x7fe15c195960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe1612cf700 1 -- 172.21.15.32:0/1276702578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe15c1018d0 msgr2=0x7fe15c19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe1612cf700 1 --2- 172.21.15.32:0/1276702578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe15c1018d0 0x7fe15c19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe1612cf700 1 -- 172.21.15.32:0/1276702578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1580095f0 con 0x7fe15c100f30 2023-09-30T14:37:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe1622d1700 1 --2- 172.21.15.32:0/1276702578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe15c1018d0 0x7fe15c19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:37:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe1612cf700 1 --2- 172.21.15.32:0/1276702578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c100f30 0x7fe15c195ea0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00c9b0 tx=0x7fe14c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe152ffd700 1 -- 172.21.15.32:0/1276702578 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c007b00 con 0x7fe15c100f30 2023-09-30T14:37:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe152ffd700 1 -- 172.21.15.32:0/1276702578 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe14c007c60 con 0x7fe15c100f30 2023-09-30T14:37:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe163d34700 1 -- 172.21.15.32:0/1276702578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15c19da90 con 0x7fe15c100f30 2023-09-30T14:37:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe152ffd700 1 -- 172.21.15.32:0/1276702578 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c018660 con 0x7fe15c100f30 2023-09-30T14:37:12.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.184+0000 7fe163d34700 1 -- 172.21.15.32:0/1276702578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15c19dfa0 con 0x7fe15c100f30 2023-09-30T14:37:12.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.185+0000 7fe163d34700 1 -- 172.21.15.32:0/1276702578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15c10b550 con 0x7fe15c100f30 2023-09-30T14:37:12.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.186+0000 7fe152ffd700 1 -- 172.21.15.32:0/1276702578 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fe14c00f450 con 0x7fe15c100f30 2023-09-30T14:37:12.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.187+0000 7fe152ffd700 1 --2- 172.21.15.32:0/1276702578 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1480a0ff0 0x7fe1480a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:12.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.187+0000 7fe161ad0700 1 --2- 172.21.15.32:0/1276702578 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1480a0ff0 0x7fe1480a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:12.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.187+0000 7fe152ffd700 1 -- 172.21.15.32:0/1276702578 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe14c0cc850 con 0x7fe15c100f30 2023-09-30T14:37:12.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.187+0000 7fe161ad0700 1 --2- 172.21.15.32:0/1276702578 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1480a0ff0 0x7fe1480a34b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe158009f50 tx=0x7fe15800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:12.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.190+0000 7fe152ffd700 1 -- 172.21.15.32:0/1276702578 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fe14c096230 con 0x7fe15c100f30 2023-09-30T14:37:12.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:12 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/550823246' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-30T14:37:12.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:12 smithi156 ceph-mon[103579]: osdmap e81: 12 total, 12 up, 12 in 2023-09-30T14:37:12.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:12 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1937418655' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-30T14:37:12.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:12 smithi156 ceph-mon[103579]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-30T14:37:12.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:12 smithi156 ceph-mon[103579]: osdmap e82: 12 total, 12 up, 12 in 2023-09-30T14:37:12.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:12 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/550823246' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-30T14:37:12.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:12 smithi148 ceph-mon[102355]: osdmap e81: 12 total, 12 up, 12 in 2023-09-30T14:37:12.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:12 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1937418655' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-30T14:37:12.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:12 smithi148 ceph-mon[102355]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-30T14:37:12.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:12 smithi148 ceph-mon[102355]: osdmap e82: 12 total, 12 up, 12 in 2023-09-30T14:37:12.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:12 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/550823246' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-30T14:37:12.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:12 smithi032 ceph-mon[98592]: osdmap e81: 12 total, 12 up, 12 in 2023-09-30T14:37:12.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:12 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1937418655' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-30T14:37:12.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:12 smithi032 ceph-mon[98592]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-30T14:37:12.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:12 smithi032 ceph-mon[98592]: osdmap e82: 12 total, 12 up, 12 in 2023-09-30T14:37:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.399+0000 7fe163d34700 1 -- 172.21.15.32:0/1276702578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe15c106bf0 con 0x7fe15c100f30 2023-09-30T14:37:12.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.401+0000 7fe152ffd700 1 -- 172.21.15.32:0/1276702578 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22146 (secure 0 0 0) 0x7fe14c099850 con 0x7fe15c100f30 2023-09-30T14:37:12.401 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:37:12.401 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":82,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:37:11.444612+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:37:12.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.403+0000 7fe163d34700 1 -- 172.21.15.32:0/1276702578 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1480a0ff0 msgr2=0x7fe1480a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:12.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.403+0000 7fe163d34700 1 --2- 172.21.15.32:0/1276702578 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1480a0ff0 0x7fe1480a34b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe158009f50 tx=0x7fe15800b540 comp rx=0 tx=0).stop 2023-09-30T14:37:12.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.403+0000 7fe163d34700 1 -- 172.21.15.32:0/1276702578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c100f30 msgr2=0x7fe15c195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:12.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.403+0000 7fe163d34700 1 --2- 172.21.15.32:0/1276702578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c100f30 0x7fe15c195ea0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00c9b0 tx=0x7fe14c00ccc0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.403+0000 7fe163d34700 1 -- 172.21.15.32:0/1276702578 shutdown_connections 2023-09-30T14:37:12.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.403+0000 7fe163d34700 1 --2- 172.21.15.32:0/1276702578 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1480a0ff0 0x7fe1480a34b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.403+0000 7fe163d34700 1 --2- 172.21.15.32:0/1276702578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe15c1018d0 0x7fe15c19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.403+0000 7fe163d34700 1 --2- 172.21.15.32:0/1276702578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe15c0ffd40 0x7fe15c195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.403+0000 7fe163d34700 1 --2- 172.21.15.32:0/1276702578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c100f30 0x7fe15c195ea0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.403+0000 7fe163d34700 1 -- 172.21.15.32:0/1276702578 >> 172.21.15.32:0/1276702578 conn(0x7fe15c0fb270 msgr2=0x7fe15c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:12.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.404+0000 7fe163d34700 1 -- 172.21.15.32:0/1276702578 shutdown_connections 2023-09-30T14:37:12.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.404+0000 7fe163d34700 1 -- 172.21.15.32:0/1276702578 wait complete. 2023-09-30T14:37:12.422 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T14:37:12.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.554+0000 7fb1de2cf700 1 -- 172.21.15.32:0/823558851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1d80fee30 msgr2=0x7fb1d80ff2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.554+0000 7fb1de2cf700 1 --2- 172.21.15.32:0/823558851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1d80fee30 0x7fb1d80ff2b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009990 tx=0x7fb1c0009ca0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.554+0000 7fb1de2cf700 1 -- 172.21.15.32:0/823558851 shutdown_connections 2023-09-30T14:37:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.554+0000 7fb1de2cf700 1 --2- 172.21.15.32:0/823558851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d80ff7f0 0x7fb1d80ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.554+0000 7fb1de2cf700 1 --2- 172.21.15.32:0/823558851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1d8101340 0x7fb1d8101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.554+0000 7fb1de2cf700 1 --2- 172.21.15.32:0/823558851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1d80fee30 0x7fb1d80ff2b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.554+0000 7fb1de2cf700 1 -- 172.21.15.32:0/823558851 >> 172.21.15.32:0/823558851 conn(0x7fb1d80fa990 msgr2=0x7fb1d80fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.554+0000 7fb1de2cf700 1 -- 172.21.15.32:0/823558851 shutdown_connections 2023-09-30T14:37:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.554+0000 7fb1de2cf700 1 -- 172.21.15.32:0/823558851 wait complete. 2023-09-30T14:37:12.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.555+0000 7fb1de2cf700 1 Processor -- start 2023-09-30T14:37:12.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.555+0000 7fb1de2cf700 1 -- start start 2023-09-30T14:37:12.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.555+0000 7fb1de2cf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1d80fee30 0x7fb1d8198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:12.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.555+0000 7fb1de2cf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d80ff7f0 0x7fb1d81991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:12.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.555+0000 7fb1de2cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1d8101340 0x7fb1d81a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:12.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.555+0000 7fb1de2cf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d81a07c0 con 0x7fb1d8101340 2023-09-30T14:37:12.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.555+0000 7fb1de2cf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d81a0930 con 0x7fb1d80ff7f0 2023-09-30T14:37:12.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.555+0000 7fb1de2cf700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d81a0af0 con 0x7fb1d80fee30 2023-09-30T14:37:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1dc86c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1d8101340 0x7fb1d81a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1dc86c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1d8101340 0x7fb1d81a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33778/0 (socket says 172.21.15.32:33778) 2023-09-30T14:37:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1dc86c700 1 -- 172.21.15.32:0/3007278422 learned_addr learned my addr 172.21.15.32:0/3007278422 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1cffff700 1 --2- 172.21.15.32:0/3007278422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d80ff7f0 0x7fb1d81991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1d7fff700 1 --2- 172.21.15.32:0/3007278422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1d80fee30 0x7fb1d8198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1dc86c700 1 -- 172.21.15.32:0/3007278422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1d80fee30 msgr2=0x7fb1d8198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1dc86c700 1 --2- 172.21.15.32:0/3007278422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1d80fee30 0x7fb1d8198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1dc86c700 1 -- 172.21.15.32:0/3007278422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d80ff7f0 msgr2=0x7fb1d81991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:12.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1dc86c700 1 --2- 172.21.15.32:0/3007278422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d80ff7f0 0x7fb1d81991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1dc86c700 1 -- 172.21.15.32:0/3007278422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c00095f0 con 0x7fb1d8101340 2023-09-30T14:37:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1cffff700 1 --2- 172.21.15.32:0/3007278422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d80ff7f0 0x7fb1d81991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:37:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1dc86c700 1 --2- 172.21.15.32:0/3007278422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1d8101340 0x7fb1d81a01f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000ef30 tx=0x7fb1d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1d5ffb700 1 -- 172.21.15.32:0/3007278422 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0018070 con 0x7fb1d8101340 2023-09-30T14:37:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1de2cf700 1 -- 172.21.15.32:0/3007278422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d81a0d70 con 0x7fb1d8101340 2023-09-30T14:37:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1d5ffb700 1 -- 172.21.15.32:0/3007278422 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1d000f040 con 0x7fb1d8101340 2023-09-30T14:37:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1d5ffb700 1 -- 172.21.15.32:0/3007278422 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d000cd00 con 0x7fb1d8101340 2023-09-30T14:37:12.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.556+0000 7fb1de2cf700 1 -- 172.21.15.32:0/3007278422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d81a11d0 con 0x7fb1d8101340 2023-09-30T14:37:12.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.557+0000 7fb1de2cf700 1 -- 172.21.15.32:0/3007278422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d810ace0 con 0x7fb1d8101340 2023-09-30T14:37:12.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.558+0000 7fb1d5ffb700 1 -- 172.21.15.32:0/3007278422 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fb1d0003a00 con 0x7fb1d8101340 2023-09-30T14:37:12.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.559+0000 7fb1d5ffb700 1 --2- 172.21.15.32:0/3007278422 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb1c40a0ff0 0x7fb1c40a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:12.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.559+0000 7fb1d7fff700 1 --2- 172.21.15.32:0/3007278422 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb1c40a0ff0 0x7fb1c40a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:12.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.559+0000 7fb1d5ffb700 1 -- 172.21.15.32:0/3007278422 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb1d00cc0e0 con 0x7fb1d8101340 2023-09-30T14:37:12.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.560+0000 7fb1d7fff700 1 --2- 172.21.15.32:0/3007278422 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb1c40a0ff0 0x7fb1c40a34b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb1d819a150 tx=0x7fb1c80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:12.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.563+0000 7fb1d5ffb700 1 -- 172.21.15.32:0/3007278422 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fb1d0095ac0 con 0x7fb1d8101340 2023-09-30T14:37:12.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.764+0000 7fb1de2cf700 1 -- 172.21.15.32:0/3007278422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1d810ea60 con 0x7fb1d8101340 2023-09-30T14:37:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.766+0000 7fb1d5ffb700 1 -- 172.21.15.32:0/3007278422 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22146 (secure 0 0 0) 0x7fb1d00990e0 con 0x7fb1d8101340 2023-09-30T14:37:12.766 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:37:12.766 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":82,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:37:11.444612+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:37:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.768+0000 7fb1de2cf700 1 -- 172.21.15.32:0/3007278422 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb1c40a0ff0 msgr2=0x7fb1c40a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.768+0000 7fb1de2cf700 1 --2- 172.21.15.32:0/3007278422 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb1c40a0ff0 0x7fb1c40a34b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb1d819a150 tx=0x7fb1c80073d0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.768+0000 7fb1de2cf700 1 -- 172.21.15.32:0/3007278422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1d8101340 msgr2=0x7fb1d81a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.768+0000 7fb1de2cf700 1 --2- 172.21.15.32:0/3007278422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1d8101340 0x7fb1d81a01f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000ef30 tx=0x7fb1d000c3a0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.768+0000 7fb1de2cf700 1 -- 172.21.15.32:0/3007278422 shutdown_connections 2023-09-30T14:37:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.768+0000 7fb1de2cf700 1 --2- 172.21.15.32:0/3007278422 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb1c40a0ff0 0x7fb1c40a34b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.768+0000 7fb1de2cf700 1 --2- 172.21.15.32:0/3007278422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1d8101340 0x7fb1d81a01f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.768+0000 7fb1de2cf700 1 --2- 172.21.15.32:0/3007278422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1d80fee30 0x7fb1d8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.768+0000 7fb1de2cf700 1 --2- 172.21.15.32:0/3007278422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d80ff7f0 0x7fb1d81991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.768+0000 7fb1de2cf700 1 -- 172.21.15.32:0/3007278422 >> 172.21.15.32:0/3007278422 conn(0x7fb1d80fa990 msgr2=0x7fb1d80fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.768+0000 7fb1de2cf700 1 -- 172.21.15.32:0/3007278422 shutdown_connections 2023-09-30T14:37:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.768+0000 7fb1de2cf700 1 -- 172.21.15.32:0/3007278422 wait complete. 2023-09-30T14:37:12.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-30T14:37:12.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.910+0000 7fc57259e700 1 -- 172.21.15.32:0/3559573647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc56c104060 msgr2=0x7fc56c106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:12.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.910+0000 7fc57259e700 1 --2- 172.21.15.32:0/3559573647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc56c104060 0x7fc56c106450 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc568009990 tx=0x7fc568009ca0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.910+0000 7fc57259e700 1 -- 172.21.15.32:0/3559573647 shutdown_connections 2023-09-30T14:37:12.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.910+0000 7fc57259e700 1 --2- 172.21.15.32:0/3559573647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc56c106990 0x7fc56c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.910+0000 7fc57259e700 1 --2- 172.21.15.32:0/3559573647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc56c101730 0x7fc56c103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.910+0000 7fc57259e700 1 --2- 172.21.15.32:0/3559573647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc56c104060 0x7fc56c106450 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.910+0000 7fc57259e700 1 -- 172.21.15.32:0/3559573647 >> 172.21.15.32:0/3559573647 conn(0x7fc56c0fb100 msgr2=0x7fc56c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.910+0000 7fc57259e700 1 -- 172.21.15.32:0/3559573647 shutdown_connections 2023-09-30T14:37:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.910+0000 7fc57259e700 1 -- 172.21.15.32:0/3559573647 wait complete. 2023-09-30T14:37:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc57259e700 1 Processor -- start 2023-09-30T14:37:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc57259e700 1 -- start start 2023-09-30T14:37:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc57259e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc56c101730 0x7fc56c1010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc57259e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc56c104060 0x7fc56c0ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc57259e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc56c106990 0x7fc56c0ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc57259e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c1003c0 con 0x7fc56c101730 2023-09-30T14:37:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc57259e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c100530 con 0x7fc56c106990 2023-09-30T14:37:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc57259e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c1a0890 con 0x7fc56c104060 2023-09-30T14:37:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc57159c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc56c101730 0x7fc56c1010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc57159c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc56c101730 0x7fc56c1010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33788/0 (socket says 172.21.15.32:33788) 2023-09-30T14:37:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc57159c700 1 -- 172.21.15.32:0/325490892 learned_addr learned my addr 172.21.15.32:0/325490892 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc571d9d700 1 --2- 172.21.15.32:0/325490892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc56c106990 0x7fc56c0ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc570d9b700 1 --2- 172.21.15.32:0/325490892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc56c104060 0x7fc56c0ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc57159c700 1 -- 172.21.15.32:0/325490892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc56c104060 msgr2=0x7fc56c0ff6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:12.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc57159c700 1 --2- 172.21.15.32:0/325490892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc56c104060 0x7fc56c0ff6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc57159c700 1 -- 172.21.15.32:0/325490892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc56c106990 msgr2=0x7fc56c0ffc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:12.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc57159c700 1 --2- 172.21.15.32:0/325490892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc56c106990 0x7fc56c0ffc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:12.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc57159c700 1 -- 172.21.15.32:0/325490892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5680095f0 con 0x7fc56c101730 2023-09-30T14:37:12.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.911+0000 7fc570d9b700 1 --2- 172.21.15.32:0/325490892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc56c104060 0x7fc56c0ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:37:12.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.912+0000 7fc57159c700 1 --2- 172.21.15.32:0/325490892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc56c101730 0x7fc56c1010a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc56000d950 tx=0x7fc56000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:12.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.912+0000 7fc55e7fc700 1 -- 172.21.15.32:0/325490892 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc560009940 con 0x7fc56c101730 2023-09-30T14:37:12.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.912+0000 7fc55e7fc700 1 -- 172.21.15.32:0/325490892 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc560010460 con 0x7fc56c101730 2023-09-30T14:37:12.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.912+0000 7fc57259e700 1 -- 172.21.15.32:0/325490892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56c1a0b70 con 0x7fc56c101730 2023-09-30T14:37:12.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.912+0000 7fc55e7fc700 1 -- 172.21.15.32:0/325490892 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56000f620 con 0x7fc56c101730 2023-09-30T14:37:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.912+0000 7fc57259e700 1 -- 172.21.15.32:0/325490892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56c1a10d0 con 0x7fc56c101730 2023-09-30T14:37:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.913+0000 7fc57259e700 1 -- 172.21.15.32:0/325490892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56c0fcc80 con 0x7fc56c101730 2023-09-30T14:37:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.914+0000 7fc55e7fc700 1 -- 172.21.15.32:0/325490892 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fc5600108d0 con 0x7fc56c101730 2023-09-30T14:37:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.914+0000 7fc55e7fc700 1 --2- 172.21.15.32:0/325490892 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5580a10c0 0x7fc5580a3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.915+0000 7fc55e7fc700 1 -- 172.21.15.32:0/325490892 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc5600cc3f0 con 0x7fc56c101730 2023-09-30T14:37:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.915+0000 7fc570d9b700 1 --2- 172.21.15.32:0/325490892 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5580a10c0 0x7fc5580a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.915+0000 7fc570d9b700 1 --2- 172.21.15.32:0/325490892 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5580a10c0 0x7fc5580a3580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc568006010 tx=0x7fc56800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:12.918+0000 7fc55e7fc700 1 -- 172.21.15.32:0/325490892 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fc560095ca0 con 0x7fc56c101730 2023-09-30T14:37:13.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:13.172+0000 7fc57259e700 1 -- 172.21.15.32:0/325490892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc56c10c750 con 0x7fc56c101730 2023-09-30T14:37:13.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: pgmap v251: 97 pgs: 17 creating+peering, 77 unknown, 3 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:13.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-30T14:37:13.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1937418655' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-30T14:37:13.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-30T14:37:13.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-30T14:37:13.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1276702578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:13.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:37:13.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:37:13.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.307 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3007278422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:13.307 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:13 smithi156 ceph-mon[103579]: osdmap e83: 12 total, 12 up, 12 in 2023-09-30T14:37:13.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: pgmap v251: 97 pgs: 17 creating+peering, 77 unknown, 3 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:13.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-30T14:37:13.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1937418655' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-30T14:37:13.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-30T14:37:13.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-30T14:37:13.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1276702578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:13.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:37:13.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:37:13.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3007278422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:13.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:13 smithi148 ceph-mon[102355]: osdmap e83: 12 total, 12 up, 12 in 2023-09-30T14:37:13.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: pgmap v251: 97 pgs: 17 creating+peering, 77 unknown, 3 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-30T14:37:13.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-30T14:37:13.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1937418655' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-30T14:37:13.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-30T14:37:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-30T14:37:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1276702578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:37:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:37:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:37:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3007278422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:13 smithi032 ceph-mon[98592]: osdmap e83: 12 total, 12 up, 12 in 2023-09-30T14:37:14.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.043+0000 7fc55e7fc700 1 -- 172.21.15.32:0/325490892 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc5600160b0 con 0x7fc56c101730 2023-09-30T14:37:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.046+0000 7fc57259e700 1 -- 172.21.15.32:0/325490892 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5580a10c0 msgr2=0x7fc5580a3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:14.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.046+0000 7fc57259e700 1 --2- 172.21.15.32:0/325490892 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5580a10c0 0x7fc5580a3580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc568006010 tx=0x7fc56800b540 comp rx=0 tx=0).stop 2023-09-30T14:37:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.046+0000 7fc57259e700 1 -- 172.21.15.32:0/325490892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc56c101730 msgr2=0x7fc56c1010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.046+0000 7fc57259e700 1 --2- 172.21.15.32:0/325490892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc56c101730 0x7fc56c1010a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc56000d950 tx=0x7fc56000dc60 comp rx=0 tx=0).stop 2023-09-30T14:37:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.046+0000 7fc57259e700 1 -- 172.21.15.32:0/325490892 shutdown_connections 2023-09-30T14:37:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.046+0000 7fc57259e700 1 --2- 172.21.15.32:0/325490892 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5580a10c0 0x7fc5580a3580 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.046+0000 7fc57259e700 1 --2- 172.21.15.32:0/325490892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc56c106990 0x7fc56c0ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.046+0000 7fc57259e700 1 --2- 172.21.15.32:0/325490892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc56c101730 0x7fc56c1010a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.046+0000 7fc57259e700 1 --2- 172.21.15.32:0/325490892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc56c104060 0x7fc56c0ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.046+0000 7fc57259e700 1 -- 172.21.15.32:0/325490892 >> 172.21.15.32:0/325490892 conn(0x7fc56c0fb100 msgr2=0x7fc56c1023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.047+0000 7fc57259e700 1 -- 172.21.15.32:0/325490892 shutdown_connections 2023-09-30T14:37:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.047+0000 7fc57259e700 1 -- 172.21.15.32:0/325490892 wait complete. 2023-09-30T14:37:14.061 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2023-09-30T14:37:14.062 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-09-30T14:37:14.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.198+0000 7f6c999ba700 1 -- 172.21.15.32:0/3818390717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c94101940 msgr2=0x7f6c941081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:14.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.198+0000 7f6c999ba700 1 --2- 172.21.15.32:0/3818390717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c94101940 0x7f6c941081f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00b0d0 tx=0x7f6c8c00b3e0 comp rx=0 tx=0).stop 2023-09-30T14:37:14.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.198+0000 7f6c999ba700 1 -- 172.21.15.32:0/3818390717 shutdown_connections 2023-09-30T14:37:14.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.198+0000 7f6c999ba700 1 --2- 172.21.15.32:0/3818390717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c94101940 0x7f6c941081f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:14.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.198+0000 7f6c999ba700 1 --2- 172.21.15.32:0/3818390717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c940ffdb0 0x7f6c941001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:14.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.198+0000 7f6c999ba700 1 --2- 172.21.15.32:0/3818390717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c94100fa0 0x7f6c94101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:14.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.198+0000 7f6c999ba700 1 -- 172.21.15.32:0/3818390717 >> 172.21.15.32:0/3818390717 conn(0x7f6c940fb320 msgr2=0x7f6c940fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:14.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.198+0000 7f6c999ba700 1 -- 172.21.15.32:0/3818390717 shutdown_connections 2023-09-30T14:37:14.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.198+0000 7f6c999ba700 1 -- 172.21.15.32:0/3818390717 wait complete. 2023-09-30T14:37:14.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.199+0000 7f6c999ba700 1 Processor -- start 2023-09-30T14:37:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.199+0000 7f6c999ba700 1 -- start start 2023-09-30T14:37:14.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.199+0000 7f6c999ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c940ffdb0 0x7f6c94196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:14.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.199+0000 7f6c999ba700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c94100fa0 0x7f6c941971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:14.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.199+0000 7f6c999ba700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c94101940 0x7f6c9419e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.199+0000 7f6c999ba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9419e7a0 con 0x7f6c940ffdb0 2023-09-30T14:37:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.199+0000 7f6c999ba700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9419e910 con 0x7f6c94100fa0 2023-09-30T14:37:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.199+0000 7f6c999ba700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9419ead0 con 0x7f6c94101940 2023-09-30T14:37:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.200+0000 7f6c92ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c940ffdb0 0x7f6c94196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.200+0000 7f6c92ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c940ffdb0 0x7f6c94196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33802/0 (socket says 172.21.15.32:33802) 2023-09-30T14:37:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.200+0000 7f6c927fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c94100fa0 0x7f6c941971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.200+0000 7f6c92ffd700 1 -- 172.21.15.32:0/1398503864 learned_addr learned my addr 172.21.15.32:0/1398503864 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.200+0000 7f6c937fe700 1 --2- 172.21.15.32:0/1398503864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c94101940 0x7f6c9419e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.200+0000 7f6c92ffd700 1 -- 172.21.15.32:0/1398503864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c94101940 msgr2=0x7f6c9419e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.200+0000 7f6c92ffd700 1 --2- 172.21.15.32:0/1398503864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c94101940 0x7f6c9419e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.200+0000 7f6c92ffd700 1 -- 172.21.15.32:0/1398503864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c94100fa0 msgr2=0x7f6c941971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:14.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.200+0000 7f6c92ffd700 1 --2- 172.21.15.32:0/1398503864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c94100fa0 0x7f6c941971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:14.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.200+0000 7f6c92ffd700 1 -- 172.21.15.32:0/1398503864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c009d00 con 0x7f6c940ffdb0 2023-09-30T14:37:14.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.200+0000 7f6c92ffd700 1 --2- 172.21.15.32:0/1398503864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c940ffdb0 0x7f6c94196c60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400c970 tx=0x7f6c8400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:14.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.200+0000 7f6c989b8700 1 -- 172.21.15.32:0/1398503864 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84004220 con 0x7f6c940ffdb0 2023-09-30T14:37:14.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.200+0000 7f6c989b8700 1 -- 172.21.15.32:0/1398503864 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c84004380 con 0x7f6c940ffdb0 2023-09-30T14:37:14.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.200+0000 7f6c989b8700 1 -- 172.21.15.32:0/1398503864 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84003b50 con 0x7f6c940ffdb0 2023-09-30T14:37:14.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.200+0000 7f6c999ba700 1 -- 172.21.15.32:0/1398503864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9419edb0 con 0x7f6c940ffdb0 2023-09-30T14:37:14.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.200+0000 7f6c999ba700 1 -- 172.21.15.32:0/1398503864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9419f1b0 con 0x7f6c940ffdb0 2023-09-30T14:37:14.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.201+0000 7f6c999ba700 1 -- 172.21.15.32:0/1398503864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9410b560 con 0x7f6c940ffdb0 2023-09-30T14:37:14.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.202+0000 7f6c989b8700 1 -- 172.21.15.32:0/1398503864 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f6c84003cd0 con 0x7f6c940ffdb0 2023-09-30T14:37:14.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.203+0000 7f6c989b8700 1 --2- 172.21.15.32:0/1398503864 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c7c0a0fa0 0x7f6c7c0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:14.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.203+0000 7f6c989b8700 1 -- 172.21.15.32:0/1398503864 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6c840cc450 con 0x7f6c940ffdb0 2023-09-30T14:37:14.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.203+0000 7f6c927fc700 1 --2- 172.21.15.32:0/1398503864 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c7c0a0fa0 0x7f6c7c0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:14.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.203+0000 7f6c927fc700 1 --2- 172.21.15.32:0/1398503864 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c7c0a0fa0 0x7f6c7c0a3460 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6c88005950 tx=0x7f6c8800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.206+0000 7f6c989b8700 1 -- 172.21.15.32:0/1398503864 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f6c84095da0 con 0x7f6c940ffdb0 2023-09-30T14:37:14.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:14 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] up:active 2023-09-30T14:37:14.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:14 smithi156 ceph-mon[103579]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-30T14:37:14.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:14 smithi156 ceph-mon[103579]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-30T14:37:14.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:14 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/325490892' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-30T14:37:14.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:14 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] up:active 2023-09-30T14:37:14.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:14 smithi148 ceph-mon[102355]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-30T14:37:14.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:14 smithi148 ceph-mon[102355]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-30T14:37:14.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:14 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/325490892' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-30T14:37:14.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:14 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] up:active 2023-09-30T14:37:14.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:14 smithi032 ceph-mon[98592]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-30T14:37:14.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:14 smithi032 ceph-mon[98592]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-30T14:37:14.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:14 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/325490892' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-30T14:37:14.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:14.458+0000 7f6c999ba700 1 -- 172.21.15.32:0/1398503864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f6c9419f400 con 0x7f6c940ffdb0 2023-09-30T14:37:15.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.045+0000 7f6c989b8700 1 -- 172.21.15.32:0/1398503864 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f6c840993c0 con 0x7f6c940ffdb0 2023-09-30T14:37:15.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.048+0000 7f6c999ba700 1 -- 172.21.15.32:0/1398503864 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c7c0a0fa0 msgr2=0x7f6c7c0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:15.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.048+0000 7f6c999ba700 1 --2- 172.21.15.32:0/1398503864 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c7c0a0fa0 0x7f6c7c0a3460 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6c88005950 tx=0x7f6c8800b410 comp rx=0 tx=0).stop 2023-09-30T14:37:15.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.048+0000 7f6c999ba700 1 -- 172.21.15.32:0/1398503864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c940ffdb0 msgr2=0x7f6c94196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:15.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.048+0000 7f6c999ba700 1 --2- 172.21.15.32:0/1398503864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c940ffdb0 0x7f6c94196c60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400c970 tx=0x7f6c8400cc80 comp rx=0 tx=0).stop 2023-09-30T14:37:15.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.048+0000 7f6c999ba700 1 -- 172.21.15.32:0/1398503864 shutdown_connections 2023-09-30T14:37:15.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.048+0000 7f6c999ba700 1 --2- 172.21.15.32:0/1398503864 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c7c0a0fa0 0x7f6c7c0a3460 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:15.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.048+0000 7f6c999ba700 1 --2- 172.21.15.32:0/1398503864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c94101940 0x7f6c9419e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:15.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.048+0000 7f6c999ba700 1 --2- 172.21.15.32:0/1398503864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c940ffdb0 0x7f6c94196c60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:15.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.048+0000 7f6c999ba700 1 --2- 172.21.15.32:0/1398503864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c94100fa0 0x7f6c941971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:15.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.048+0000 7f6c999ba700 1 -- 172.21.15.32:0/1398503864 >> 172.21.15.32:0/1398503864 conn(0x7f6c940fb320 msgr2=0x7f6c94069240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:15.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.048+0000 7f6c999ba700 1 -- 172.21.15.32:0/1398503864 shutdown_connections 2023-09-30T14:37:15.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.048+0000 7f6c999ba700 1 -- 172.21.15.32:0/1398503864 wait complete. 2023-09-30T14:37:15.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-09-30T14:37:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.201+0000 7fc850b1e700 1 -- 172.21.15.32:0/4255428485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c1006a0 msgr2=0x7fc84c100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.201+0000 7fc850b1e700 1 --2- 172.21.15.32:0/4255428485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c1006a0 0x7fc84c100b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc8400099c0 tx=0x7fc840009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.201+0000 7fc850b1e700 1 -- 172.21.15.32:0/4255428485 shutdown_connections 2023-09-30T14:37:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.201+0000 7fc850b1e700 1 --2- 172.21.15.32:0/4255428485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc84c101040 0x7fc84c1078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.201+0000 7fc850b1e700 1 --2- 172.21.15.32:0/4255428485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c0ff4b0 0x7fc84c0ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.201+0000 7fc850b1e700 1 --2- 172.21.15.32:0/4255428485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c1006a0 0x7fc84c100b00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.201+0000 7fc850b1e700 1 -- 172.21.15.32:0/4255428485 >> 172.21.15.32:0/4255428485 conn(0x7fc84c0faa20 msgr2=0x7fc84c0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.201+0000 7fc850b1e700 1 -- 172.21.15.32:0/4255428485 shutdown_connections 2023-09-30T14:37:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.201+0000 7fc850b1e700 1 -- 172.21.15.32:0/4255428485 wait complete. 2023-09-30T14:37:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.202+0000 7fc850b1e700 1 Processor -- start 2023-09-30T14:37:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.202+0000 7fc850b1e700 1 -- start start 2023-09-30T14:37:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc850b1e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc84c0ff4b0 0x7fc84c1948b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc850b1e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c1006a0 0x7fc84c194df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc850b1e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c101040 0x7fc84c19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc850b1e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c19c430 con 0x7fc84c1006a0 2023-09-30T14:37:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc850b1e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c19c570 con 0x7fc84c101040 2023-09-30T14:37:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc850b1e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c19c700 con 0x7fc84c0ff4b0 2023-09-30T14:37:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc849d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c1006a0 0x7fc84c194df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc849d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c1006a0 0x7fc84c194df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33818/0 (socket says 172.21.15.32:33818) 2023-09-30T14:37:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc849d9b700 1 -- 172.21.15.32:0/2522624515 learned_addr learned my addr 172.21.15.32:0/2522624515 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc84a59c700 1 --2- 172.21.15.32:0/2522624515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc84c0ff4b0 0x7fc84c1948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc84ad9d700 1 --2- 172.21.15.32:0/2522624515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c101040 0x7fc84c19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc849d9b700 1 -- 172.21.15.32:0/2522624515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc84c0ff4b0 msgr2=0x7fc84c1948b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc849d9b700 1 --2- 172.21.15.32:0/2522624515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc84c0ff4b0 0x7fc84c1948b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc849d9b700 1 -- 172.21.15.32:0/2522624515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c101040 msgr2=0x7fc84c19be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc849d9b700 1 --2- 172.21.15.32:0/2522624515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c101040 0x7fc84c19be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc849d9b700 1 -- 172.21.15.32:0/2522624515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8400095f0 con 0x7fc84c1006a0 2023-09-30T14:37:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc84ad9d700 1 --2- 172.21.15.32:0/2522624515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c101040 0x7fc84c19be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:37:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc849d9b700 1 --2- 172.21.15.32:0/2522624515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c1006a0 0x7fc84c194df0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc840009990 tx=0x7fc84001a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.203+0000 7fc83b7fe700 1 -- 172.21.15.32:0/2522624515 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84001f070 con 0x7fc84c1006a0 2023-09-30T14:37:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.204+0000 7fc83b7fe700 1 -- 172.21.15.32:0/2522624515 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc840004640 con 0x7fc84c1006a0 2023-09-30T14:37:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.204+0000 7fc850b1e700 1 -- 172.21.15.32:0/2522624515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84c19c980 con 0x7fc84c1006a0 2023-09-30T14:37:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.204+0000 7fc83b7fe700 1 -- 172.21.15.32:0/2522624515 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840004220 con 0x7fc84c1006a0 2023-09-30T14:37:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.204+0000 7fc850b1e700 1 -- 172.21.15.32:0/2522624515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84c19ce80 con 0x7fc84c1006a0 2023-09-30T14:37:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.204+0000 7fc850b1e700 1 -- 172.21.15.32:0/2522624515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84c1069f0 con 0x7fc84c1006a0 2023-09-30T14:37:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.206+0000 7fc83b7fe700 1 -- 172.21.15.32:0/2522624515 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fc840023070 con 0x7fc84c1006a0 2023-09-30T14:37:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.207+0000 7fc83b7fe700 1 --2- 172.21.15.32:0/2522624515 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc8340a1010 0x7fc8340a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.207+0000 7fc84a59c700 1 --2- 172.21.15.32:0/2522624515 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc8340a1010 0x7fc8340a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.207+0000 7fc83b7fe700 1 -- 172.21.15.32:0/2522624515 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc8400d00f0 con 0x7fc84c1006a0 2023-09-30T14:37:15.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.207+0000 7fc84a59c700 1 --2- 172.21.15.32:0/2522624515 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc8340a1010 0x7fc8340a34d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc84c100500 tx=0x7fc83c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:15.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.210+0000 7fc83b7fe700 1 -- 172.21.15.32:0/2522624515 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fc840099a30 con 0x7fc84c1006a0 2023-09-30T14:37:15.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:15 smithi156 ceph-mon[103579]: pgmap v254: 97 pgs: 17 creating+peering, 44 unknown, 36 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2023-09-30T14:37:15.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:15 smithi156 ceph-mon[103579]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-09-30T14:37:15.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:15 smithi156 ceph-mon[103579]: Cluster is now healthy 2023-09-30T14:37:15.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:15 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/325490892' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-30T14:37:15.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:15 smithi156 ceph-mon[103579]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-30T14:37:15.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:15 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1398503864' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-09-30T14:37:15.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:15 smithi148 ceph-mon[102355]: pgmap v254: 97 pgs: 17 creating+peering, 44 unknown, 36 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2023-09-30T14:37:15.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:15 smithi148 ceph-mon[102355]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-09-30T14:37:15.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:15 smithi148 ceph-mon[102355]: Cluster is now healthy 2023-09-30T14:37:15.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:15 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/325490892' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-30T14:37:15.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:15 smithi148 ceph-mon[102355]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-30T14:37:15.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:15 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1398503864' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-09-30T14:37:15.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:15 smithi032 ceph-mon[98592]: pgmap v254: 97 pgs: 17 creating+peering, 44 unknown, 36 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2023-09-30T14:37:15.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:15 smithi032 ceph-mon[98592]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-09-30T14:37:15.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:15 smithi032 ceph-mon[98592]: Cluster is now healthy 2023-09-30T14:37:15.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:15 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/325490892' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-30T14:37:15.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:15 smithi032 ceph-mon[98592]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-30T14:37:15.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:15 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1398503864' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-09-30T14:37:15.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:15.463+0000 7fc850b1e700 1 -- 172.21.15.32:0/2522624515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fc84c02d180 con 0x7fc84c1006a0 2023-09-30T14:37:16.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.057+0000 7fc83b7fe700 1 -- 172.21.15.32:0/2522624515 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7fc84009d050 con 0x7fc84c1006a0 2023-09-30T14:37:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.060+0000 7fc850b1e700 1 -- 172.21.15.32:0/2522624515 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc8340a1010 msgr2=0x7fc8340a34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.060+0000 7fc850b1e700 1 --2- 172.21.15.32:0/2522624515 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc8340a1010 0x7fc8340a34d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc84c100500 tx=0x7fc83c00a3b0 comp rx=0 tx=0).stop 2023-09-30T14:37:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.060+0000 7fc850b1e700 1 -- 172.21.15.32:0/2522624515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c1006a0 msgr2=0x7fc84c194df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.060+0000 7fc850b1e700 1 --2- 172.21.15.32:0/2522624515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c1006a0 0x7fc84c194df0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc840009990 tx=0x7fc84001a840 comp rx=0 tx=0).stop 2023-09-30T14:37:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.060+0000 7fc850b1e700 1 -- 172.21.15.32:0/2522624515 shutdown_connections 2023-09-30T14:37:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.060+0000 7fc850b1e700 1 --2- 172.21.15.32:0/2522624515 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc8340a1010 0x7fc8340a34d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.060+0000 7fc850b1e700 1 --2- 172.21.15.32:0/2522624515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c101040 0x7fc84c19be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.060+0000 7fc850b1e700 1 --2- 172.21.15.32:0/2522624515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc84c0ff4b0 0x7fc84c1948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.060+0000 7fc850b1e700 1 --2- 172.21.15.32:0/2522624515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc84c1006a0 0x7fc84c194df0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.060+0000 7fc850b1e700 1 -- 172.21.15.32:0/2522624515 >> 172.21.15.32:0/2522624515 conn(0x7fc84c0faa20 msgr2=0x7fc84c103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:16.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.060+0000 7fc850b1e700 1 -- 172.21.15.32:0/2522624515 shutdown_connections 2023-09-30T14:37:16.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.060+0000 7fc850b1e700 1 -- 172.21.15.32:0/2522624515 wait complete. 2023-09-30T14:37:16.080 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-09-30T14:37:16.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.215+0000 7fb3779c9700 1 -- 172.21.15.32:0/4233892421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb370100f20 msgr2=0x7fb370101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:16.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.215+0000 7fb3779c9700 1 --2- 172.21.15.32:0/4233892421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb370100f20 0x7fb370101380 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb3600099c0 tx=0x7fb360009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:16.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.215+0000 7fb3779c9700 1 -- 172.21.15.32:0/4233892421 shutdown_connections 2023-09-30T14:37:16.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.215+0000 7fb3779c9700 1 --2- 172.21.15.32:0/4233892421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3701018c0 0x7fb370108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:16.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.215+0000 7fb3779c9700 1 --2- 172.21.15.32:0/4233892421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3700ffd30 0x7fb370100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:16.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.215+0000 7fb3779c9700 1 --2- 172.21.15.32:0/4233892421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb370100f20 0x7fb370101380 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:16.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.215+0000 7fb3779c9700 1 -- 172.21.15.32:0/4233892421 >> 172.21.15.32:0/4233892421 conn(0x7fb3700fb2a0 msgr2=0x7fb3700fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:16.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.216+0000 7fb3779c9700 1 -- 172.21.15.32:0/4233892421 shutdown_connections 2023-09-30T14:37:16.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.216+0000 7fb3779c9700 1 -- 172.21.15.32:0/4233892421 wait complete. 2023-09-30T14:37:16.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.216+0000 7fb3779c9700 1 Processor -- start 2023-09-30T14:37:16.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.216+0000 7fb3779c9700 1 -- start start 2023-09-30T14:37:16.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.216+0000 7fb3779c9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3700ffd30 0x7fb370103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:16.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.216+0000 7fb3779c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb370100f20 0x7fb370102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:16.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.216+0000 7fb3779c9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3701018c0 0x7fb370102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:16.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.216+0000 7fb3779c9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb370102fe0 con 0x7fb370100f20 2023-09-30T14:37:16.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.216+0000 7fb3779c9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb370103150 con 0x7fb3700ffd30 2023-09-30T14:37:16.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.216+0000 7fb3779c9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3701a0b00 con 0x7fb3701018c0 2023-09-30T14:37:16.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.216+0000 7fb374f64700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb370100f20 0x7fb370102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:16.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.216+0000 7fb374f64700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb370100f20 0x7fb370102300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33832/0 (socket says 172.21.15.32:33832) 2023-09-30T14:37:16.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.216+0000 7fb374f64700 1 -- 172.21.15.32:0/1378680852 learned_addr learned my addr 172.21.15.32:0/1378680852 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:16.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.217+0000 7fb375765700 1 --2- 172.21.15.32:0/1378680852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3700ffd30 0x7fb370103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:16.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.217+0000 7fb375f66700 1 --2- 172.21.15.32:0/1378680852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3701018c0 0x7fb370102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:16.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.217+0000 7fb374f64700 1 -- 172.21.15.32:0/1378680852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3701018c0 msgr2=0x7fb370102840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:16.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.217+0000 7fb374f64700 1 --2- 172.21.15.32:0/1378680852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3701018c0 0x7fb370102840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:16.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.217+0000 7fb374f64700 1 -- 172.21.15.32:0/1378680852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3700ffd30 msgr2=0x7fb370103cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:16.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.217+0000 7fb374f64700 1 --2- 172.21.15.32:0/1378680852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3700ffd30 0x7fb370103cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:16.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.217+0000 7fb374f64700 1 -- 172.21.15.32:0/1378680852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3600095f0 con 0x7fb370100f20 2023-09-30T14:37:16.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.217+0000 7fb375765700 1 --2- 172.21.15.32:0/1378680852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3700ffd30 0x7fb370103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:37:16.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.217+0000 7fb374f64700 1 --2- 172.21.15.32:0/1378680852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb370100f20 0x7fb370102300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb360009990 tx=0x7fb360005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:16.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.217+0000 7fb3667fc700 1 -- 172.21.15.32:0/1378680852 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36001d070 con 0x7fb370100f20 2023-09-30T14:37:16.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.217+0000 7fb3667fc700 1 -- 172.21.15.32:0/1378680852 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb360022470 con 0x7fb370100f20 2023-09-30T14:37:16.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.217+0000 7fb3779c9700 1 -- 172.21.15.32:0/1378680852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3701a0d80 con 0x7fb370100f20 2023-09-30T14:37:16.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.217+0000 7fb3667fc700 1 -- 172.21.15.32:0/1378680852 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360004230 con 0x7fb370100f20 2023-09-30T14:37:16.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.217+0000 7fb3779c9700 1 -- 172.21.15.32:0/1378680852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3701a11d0 con 0x7fb370100f20 2023-09-30T14:37:16.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.218+0000 7fb3779c9700 1 -- 172.21.15.32:0/1378680852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb37010b540 con 0x7fb370100f20 2023-09-30T14:37:16.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.219+0000 7fb3667fc700 1 -- 172.21.15.32:0/1378680852 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fb3600225e0 con 0x7fb370100f20 2023-09-30T14:37:16.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.220+0000 7fb3667fc700 1 --2- 172.21.15.32:0/1378680852 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb35c0a0fa0 0x7fb35c0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:16.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.220+0000 7fb3667fc700 1 -- 172.21.15.32:0/1378680852 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb3600cd820 con 0x7fb370100f20 2023-09-30T14:37:16.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.220+0000 7fb375765700 1 --2- 172.21.15.32:0/1378680852 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb35c0a0fa0 0x7fb35c0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:16.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.220+0000 7fb375765700 1 --2- 172.21.15.32:0/1378680852 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb35c0a0fa0 0x7fb35c0a3460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb36c005fd0 tx=0x7fb36c005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:16.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.224+0000 7fb3667fc700 1 -- 172.21.15.32:0/1378680852 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fb360097200 con 0x7fb370100f20 2023-09-30T14:37:16.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:16 smithi156 ceph-mon[103579]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-30T14:37:16.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:16 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1398503864' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-09-30T14:37:16.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:16 smithi156 ceph-mon[103579]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-30T14:37:16.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:16 smithi156 ceph-mon[103579]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-30T14:37:16.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:16 smithi156 ceph-mon[103579]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-09-30T14:37:16.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:16 smithi156 ceph-mon[103579]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-09-30T14:37:16.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:16 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2522624515' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-30T14:37:16.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:16 smithi148 ceph-mon[102355]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-30T14:37:16.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:16 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1398503864' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-09-30T14:37:16.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:16 smithi148 ceph-mon[102355]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-30T14:37:16.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:16 smithi148 ceph-mon[102355]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-30T14:37:16.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:16 smithi148 ceph-mon[102355]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-09-30T14:37:16.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:16 smithi148 ceph-mon[102355]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-09-30T14:37:16.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:16 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2522624515' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-30T14:37:16.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:16 smithi032 ceph-mon[98592]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-30T14:37:16.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:16 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1398503864' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-09-30T14:37:16.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:16 smithi032 ceph-mon[98592]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-30T14:37:16.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:16 smithi032 ceph-mon[98592]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-30T14:37:16.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:16 smithi032 ceph-mon[98592]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-09-30T14:37:16.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:16 smithi032 ceph-mon[98592]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-09-30T14:37:16.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:16 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2522624515' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-30T14:37:16.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.473+0000 7fb3779c9700 1 -- 172.21.15.32:0/1378680852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fb3701a13c0 con 0x7fb370100f20 2023-09-30T14:37:16.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.972+0000 7fb3667fc700 1 -- 172.21.15.32:0/1378680852 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fb36009a820 con 0x7fb370100f20 2023-09-30T14:37:16.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.975+0000 7fb3779c9700 1 -- 172.21.15.32:0/1378680852 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb35c0a0fa0 msgr2=0x7fb35c0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:16.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.975+0000 7fb3779c9700 1 --2- 172.21.15.32:0/1378680852 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb35c0a0fa0 0x7fb35c0a3460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb36c005fd0 tx=0x7fb36c005e40 comp rx=0 tx=0).stop 2023-09-30T14:37:16.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.975+0000 7fb3779c9700 1 -- 172.21.15.32:0/1378680852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb370100f20 msgr2=0x7fb370102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:16.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.975+0000 7fb3779c9700 1 --2- 172.21.15.32:0/1378680852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb370100f20 0x7fb370102300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb360009990 tx=0x7fb360005e80 comp rx=0 tx=0).stop 2023-09-30T14:37:16.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.975+0000 7fb3779c9700 1 -- 172.21.15.32:0/1378680852 shutdown_connections 2023-09-30T14:37:16.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.975+0000 7fb3779c9700 1 --2- 172.21.15.32:0/1378680852 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb35c0a0fa0 0x7fb35c0a3460 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:16.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.975+0000 7fb3779c9700 1 --2- 172.21.15.32:0/1378680852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3701018c0 0x7fb370102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:16.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.975+0000 7fb3779c9700 1 --2- 172.21.15.32:0/1378680852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3700ffd30 0x7fb370103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:16.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.975+0000 7fb3779c9700 1 --2- 172.21.15.32:0/1378680852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb370100f20 0x7fb370102300 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:16.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.975+0000 7fb3779c9700 1 -- 172.21.15.32:0/1378680852 >> 172.21.15.32:0/1378680852 conn(0x7fb3700fb2a0 msgr2=0x7fb370068030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:16.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.976+0000 7fb3779c9700 1 -- 172.21.15.32:0/1378680852 shutdown_connections 2023-09-30T14:37:16.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:16.976+0000 7fb3779c9700 1 -- 172.21.15.32:0/1378680852 wait complete. 2023-09-30T14:37:16.991 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2023-09-30T14:37:16.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-09-30T14:37:17.058 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 30 14:37:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-b[138505]: 2023-09-30T14:37:16.748+0000 7f0c3c6ac700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-30T14:37:17.059 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 30 14:37:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-b[138505]: 2023-09-30T14:37:16.748+0000 7f0c3c6ac700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-09-30T14:37:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.129+0000 7f032050a700 1 -- 172.21.15.32:0/3014397180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03180ffd30 msgr2=0x7f0318100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.129+0000 7f032050a700 1 --2- 172.21.15.32:0/3014397180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03180ffd30 0x7f0318100150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f03140099c0 tx=0x7f0314009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.129+0000 7f032050a700 1 -- 172.21.15.32:0/3014397180 shutdown_connections 2023-09-30T14:37:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.129+0000 7f032050a700 1 --2- 172.21.15.32:0/3014397180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03181018c0 0x7f0318108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.129+0000 7f032050a700 1 --2- 172.21.15.32:0/3014397180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03180ffd30 0x7f0318100150 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.129+0000 7f032050a700 1 --2- 172.21.15.32:0/3014397180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0318100f20 0x7f0318101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.129+0000 7f032050a700 1 -- 172.21.15.32:0/3014397180 >> 172.21.15.32:0/3014397180 conn(0x7f03180fb2a0 msgr2=0x7f03180fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:17.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.129+0000 7f032050a700 1 -- 172.21.15.32:0/3014397180 shutdown_connections 2023-09-30T14:37:17.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.129+0000 7f032050a700 1 -- 172.21.15.32:0/3014397180 wait complete. 2023-09-30T14:37:17.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.130+0000 7f032050a700 1 Processor -- start 2023-09-30T14:37:17.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.130+0000 7f032050a700 1 -- start start 2023-09-30T14:37:17.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.130+0000 7f032050a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03180ffd30 0x7f0318195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:17.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.130+0000 7f032050a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0318100f20 0x7f0318195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:17.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.130+0000 7f032050a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03181018c0 0x7f031819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:17.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.130+0000 7f032050a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031819d5b0 con 0x7f03180ffd30 2023-09-30T14:37:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.130+0000 7f032050a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031819d6f0 con 0x7f03181018c0 2023-09-30T14:37:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.130+0000 7f032050a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031819d880 con 0x7f0318100f20 2023-09-30T14:37:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.130+0000 7f031e2a6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03180ffd30 0x7f0318195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.130+0000 7f031e2a6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03180ffd30 0x7f0318195a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33844/0 (socket says 172.21.15.32:33844) 2023-09-30T14:37:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.130+0000 7f031e2a6700 1 -- 172.21.15.32:0/179572023 learned_addr learned my addr 172.21.15.32:0/179572023 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.131+0000 7f031e2a6700 1 -- 172.21.15.32:0/179572023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0318100f20 msgr2=0x7f0318195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.131+0000 7f031e2a6700 1 --2- 172.21.15.32:0/179572023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0318100f20 0x7f0318195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.131+0000 7f031e2a6700 1 -- 172.21.15.32:0/179572023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03181018c0 msgr2=0x7f031819cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.131+0000 7f031eaa7700 1 --2- 172.21.15.32:0/179572023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03181018c0 0x7f031819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.131+0000 7f031e2a6700 1 --2- 172.21.15.32:0/179572023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03181018c0 0x7f031819cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.131+0000 7f031e2a6700 1 -- 172.21.15.32:0/179572023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03140095f0 con 0x7f03180ffd30 2023-09-30T14:37:17.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.131+0000 7f031e2a6700 1 --2- 172.21.15.32:0/179572023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03180ffd30 0x7f0318195a30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0314009990 tx=0x7f0314005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:17.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.131+0000 7f030f7fe700 1 -- 172.21.15.32:0/179572023 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031401d070 con 0x7f03180ffd30 2023-09-30T14:37:17.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.131+0000 7f030f7fe700 1 -- 172.21.15.32:0/179572023 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0314004200 con 0x7f03180ffd30 2023-09-30T14:37:17.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.131+0000 7f032050a700 1 -- 172.21.15.32:0/179572023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031819db00 con 0x7f03180ffd30 2023-09-30T14:37:17.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.131+0000 7f030f7fe700 1 -- 172.21.15.32:0/179572023 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314022e30 con 0x7f03180ffd30 2023-09-30T14:37:17.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.131+0000 7f032050a700 1 -- 172.21.15.32:0/179572023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031819e000 con 0x7f03180ffd30 2023-09-30T14:37:17.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.132+0000 7f032050a700 1 -- 172.21.15.32:0/179572023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0318106ce0 con 0x7f03180ffd30 2023-09-30T14:37:17.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.133+0000 7f030f7fe700 1 -- 172.21.15.32:0/179572023 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f0314003d40 con 0x7f03180ffd30 2023-09-30T14:37:17.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.134+0000 7f030f7fe700 1 --2- 172.21.15.32:0/179572023 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f03040a1020 0x7f03040a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:17.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.134+0000 7f031daa5700 1 --2- 172.21.15.32:0/179572023 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f03040a1020 0x7f03040a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:17.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.134+0000 7f030f7fe700 1 -- 172.21.15.32:0/179572023 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f03140cdc00 con 0x7f03180ffd30 2023-09-30T14:37:17.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.134+0000 7f031daa5700 1 --2- 172.21.15.32:0/179572023 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f03040a1020 0x7f03040a34e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0308005950 tx=0x7f03080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:17.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.138+0000 7f030f7fe700 1 -- 172.21.15.32:0/179572023 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f0314097540 con 0x7f03180ffd30 2023-09-30T14:37:17.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:17 smithi148 ceph-mon[102355]: pgmap v255: 97 pgs: 8 creating+peering, 7 unknown, 82 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 9 op/s 2023-09-30T14:37:17.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:17 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2522624515' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-30T14:37:17.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:17 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] up:active 2023-09-30T14:37:17.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:17 smithi148 ceph-mon[102355]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-30T14:37:17.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:17 smithi148 ceph-mon[102355]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-30T14:37:17.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:17 smithi148 ceph-mon[102355]: Cluster is now healthy 2023-09-30T14:37:17.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:17 smithi148 ceph-mon[102355]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-09-30T14:37:17.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:17 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-09-30T14:37:17.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:17 smithi148 ceph-mon[102355]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-09-30T14:37:17.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:17 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1378680852' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-30T14:37:17.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:17 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1378680852' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-30T14:37:17.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:17 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] up:active 2023-09-30T14:37:17.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:17 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:37:17.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.375+0000 7f032050a700 1 -- 172.21.15.32:0/179572023 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f031802d2e0 con 0x7f03040a1020 2023-09-30T14:37:17.381 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-mon[98592]: pgmap v255: 97 pgs: 8 creating+peering, 7 unknown, 82 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 9 op/s 2023-09-30T14:37:17.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2522624515' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-30T14:37:17.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] up:active 2023-09-30T14:37:17.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-mon[98592]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-30T14:37:17.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-mon[98592]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-30T14:37:17.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-mon[98592]: Cluster is now healthy 2023-09-30T14:37:17.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-mon[98592]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-09-30T14:37:17.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-09-30T14:37:17.382 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-mon[98592]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-09-30T14:37:17.383 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1378680852' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-30T14:37:17.383 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1378680852' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-30T14:37:17.383 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] up:active 2023-09-30T14:37:17.383 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:37:17.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.416+0000 7f030f7fe700 1 -- 172.21.15.32:0/179572023 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f031802d2e0 con 0x7f03040a1020 2023-09-30T14:37:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.418+0000 7f032050a700 1 -- 172.21.15.32:0/179572023 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f03040a1020 msgr2=0x7f03040a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.418+0000 7f032050a700 1 --2- 172.21.15.32:0/179572023 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f03040a1020 0x7f03040a34e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0308005950 tx=0x7f03080058e0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.418+0000 7f032050a700 1 -- 172.21.15.32:0/179572023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03180ffd30 msgr2=0x7f0318195a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.418+0000 7f032050a700 1 --2- 172.21.15.32:0/179572023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03180ffd30 0x7f0318195a30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0314009990 tx=0x7f0314005e80 comp rx=0 tx=0).stop 2023-09-30T14:37:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.418+0000 7f032050a700 1 -- 172.21.15.32:0/179572023 shutdown_connections 2023-09-30T14:37:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.418+0000 7f032050a700 1 --2- 172.21.15.32:0/179572023 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f03040a1020 0x7f03040a34e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.418+0000 7f032050a700 1 --2- 172.21.15.32:0/179572023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03181018c0 0x7f031819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.418+0000 7f032050a700 1 --2- 172.21.15.32:0/179572023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03180ffd30 0x7f0318195a30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.418+0000 7f032050a700 1 --2- 172.21.15.32:0/179572023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0318100f20 0x7f0318195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.418+0000 7f032050a700 1 -- 172.21.15.32:0/179572023 >> 172.21.15.32:0/179572023 conn(0x7f03180fb2a0 msgr2=0x7f03180fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.418+0000 7f032050a700 1 -- 172.21.15.32:0/179572023 shutdown_connections 2023-09-30T14:37:17.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.419+0000 7f032050a700 1 -- 172.21.15.32:0/179572023 wait complete. 2023-09-30T14:37:17.431 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-09-30T14:37:17.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:17 smithi156 ceph-mon[103579]: pgmap v255: 97 pgs: 8 creating+peering, 7 unknown, 82 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 9 op/s 2023-09-30T14:37:17.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:17 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2522624515' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-30T14:37:17.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:17 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] up:active 2023-09-30T14:37:17.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:17 smithi156 ceph-mon[103579]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-30T14:37:17.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:17 smithi156 ceph-mon[103579]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-30T14:37:17.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:17 smithi156 ceph-mon[103579]: Cluster is now healthy 2023-09-30T14:37:17.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:17 smithi156 ceph-mon[103579]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-09-30T14:37:17.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:17 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-09-30T14:37:17.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:17 smithi156 ceph-mon[103579]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-09-30T14:37:17.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:17 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1378680852' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-30T14:37:17.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:17 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1378680852' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-30T14:37:17.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:17 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] up:active 2023-09-30T14:37:17.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:17 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:37:17.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.560+0000 7f69b58bd700 1 -- 172.21.15.32:0/318123970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b00ffdb0 msgr2=0x7f69b01001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:17.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.560+0000 7f69b58bd700 1 --2- 172.21.15.32:0/318123970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b00ffdb0 0x7f69b01001d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f69a00099c0 tx=0x7f69a0009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.560+0000 7f69b58bd700 1 -- 172.21.15.32:0/318123970 shutdown_connections 2023-09-30T14:37:17.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.560+0000 7f69b58bd700 1 --2- 172.21.15.32:0/318123970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b0101940 0x7f69b01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.560+0000 7f69b58bd700 1 --2- 172.21.15.32:0/318123970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b00ffdb0 0x7f69b01001d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.560+0000 7f69b58bd700 1 --2- 172.21.15.32:0/318123970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69b0100fa0 0x7f69b0101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.560+0000 7f69b58bd700 1 -- 172.21.15.32:0/318123970 >> 172.21.15.32:0/318123970 conn(0x7f69b00fb340 msgr2=0x7f69b00fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:17.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.560+0000 7f69b58bd700 1 -- 172.21.15.32:0/318123970 shutdown_connections 2023-09-30T14:37:17.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.560+0000 7f69b58bd700 1 -- 172.21.15.32:0/318123970 wait complete. 2023-09-30T14:37:17.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.560+0000 7f69b58bd700 1 Processor -- start 2023-09-30T14:37:17.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.560+0000 7f69b58bd700 1 -- start start 2023-09-30T14:37:17.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69b58bd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69b00ffdb0 0x7f69b01948f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:17.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69b58bd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b0100fa0 0x7f69b0194e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:17.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69b58bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b0101940 0x7f69b019bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:17.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69b58bd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b019c470 con 0x7f69b0101940 2023-09-30T14:37:17.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69b58bd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b019c5b0 con 0x7f69b0100fa0 2023-09-30T14:37:17.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69b58bd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b019c740 con 0x7f69b00ffdb0 2023-09-30T14:37:17.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69af7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b0101940 0x7f69b019bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:17.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69af7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b0101940 0x7f69b019bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33878/0 (socket says 172.21.15.32:33878) 2023-09-30T14:37:17.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69af7fe700 1 -- 172.21.15.32:0/2253494577 learned_addr learned my addr 172.21.15.32:0/2253494577 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:17.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69ae7fc700 1 --2- 172.21.15.32:0/2253494577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b0100fa0 0x7f69b0194e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:17.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69aeffd700 1 --2- 172.21.15.32:0/2253494577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69b00ffdb0 0x7f69b01948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:17.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69af7fe700 1 -- 172.21.15.32:0/2253494577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69b00ffdb0 msgr2=0x7f69b01948f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:17.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69af7fe700 1 --2- 172.21.15.32:0/2253494577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69b00ffdb0 0x7f69b01948f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69af7fe700 1 -- 172.21.15.32:0/2253494577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b0100fa0 msgr2=0x7f69b0194e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:17.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69af7fe700 1 --2- 172.21.15.32:0/2253494577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b0100fa0 0x7f69b0194e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69af7fe700 1 -- 172.21.15.32:0/2253494577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a00095f0 con 0x7f69b0101940 2023-09-30T14:37:17.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69af7fe700 1 --2- 172.21.15.32:0/2253494577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b0101940 0x7f69b019bea0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009c10 tx=0x7f69a800e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:17.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69b48bb700 1 -- 172.21.15.32:0/2253494577 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8009e70 con 0x7f69b0101940 2023-09-30T14:37:17.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69b48bb700 1 -- 172.21.15.32:0/2253494577 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69a800ed50 con 0x7f69b0101940 2023-09-30T14:37:17.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69b58bd700 1 -- 172.21.15.32:0/2253494577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b019ca20 con 0x7f69b0101940 2023-09-30T14:37:17.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69b48bb700 1 -- 172.21.15.32:0/2253494577 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a80045e0 con 0x7f69b0101940 2023-09-30T14:37:17.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.561+0000 7f69b58bd700 1 -- 172.21.15.32:0/2253494577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b019cf30 con 0x7f69b0101940 2023-09-30T14:37:17.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.562+0000 7f69b58bd700 1 -- 172.21.15.32:0/2253494577 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b010a4a0 con 0x7f69b0101940 2023-09-30T14:37:17.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.564+0000 7f69b48bb700 1 -- 172.21.15.32:0/2253494577 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f69a8004780 con 0x7f69b0101940 2023-09-30T14:37:17.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.564+0000 7f69b48bb700 1 --2- 172.21.15.32:0/2253494577 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f69980a1020 0x7f69980a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:17.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.564+0000 7f69aeffd700 1 --2- 172.21.15.32:0/2253494577 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f69980a1020 0x7f69980a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:17.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.564+0000 7f69b48bb700 1 -- 172.21.15.32:0/2253494577 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f69a80cc4b0 con 0x7f69b0101940 2023-09-30T14:37:17.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.565+0000 7f69aeffd700 1 --2- 172.21.15.32:0/2253494577 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f69980a1020 0x7f69980a34e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009990 tx=0x7f69a001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:17.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.568+0000 7f69b48bb700 1 -- 172.21.15.32:0/2253494577 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f69a8095e00 con 0x7f69b0101940 2023-09-30T14:37:17.637 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:37:17.380+0000 7fa9a5d67700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T14:37:17.637 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:37:17.380+0000 7fa9a5d67700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T14:37:17.637 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:37:17.380+0000 7fa9a5d67700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T14:37:17.638 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:37:17.380+0000 7fa9a5d67700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T14:37:17.638 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 14:37:17 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T14:37:17.380+0000 7fa9a5d67700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T14:37:17.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.806+0000 7f69b58bd700 1 -- 172.21.15.32:0/2253494577 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f69b0061260 con 0x7f69980a1020 2023-09-30T14:37:17.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.811+0000 7f69b48bb700 1 -- 172.21.15.32:0/2253494577 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f69b0061260 con 0x7f69980a1020 2023-09-30T14:37:17.811 INFO:teuthology.orchestra.run.smithi032.stdout:/volumes/_nogroup/sv_0/c2fb4de5-6cfb-4aa7-bc93-76658a8809d9 2023-09-30T14:37:17.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.813+0000 7f69b58bd700 1 -- 172.21.15.32:0/2253494577 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f69980a1020 msgr2=0x7f69980a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:17.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.813+0000 7f69b58bd700 1 --2- 172.21.15.32:0/2253494577 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f69980a1020 0x7f69980a34e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009990 tx=0x7f69a001a040 comp rx=0 tx=0).stop 2023-09-30T14:37:17.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.813+0000 7f69b58bd700 1 -- 172.21.15.32:0/2253494577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b0101940 msgr2=0x7f69b019bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:17.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.813+0000 7f69b58bd700 1 --2- 172.21.15.32:0/2253494577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b0101940 0x7f69b019bea0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009c10 tx=0x7f69a800e5b0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.813+0000 7f69b58bd700 1 -- 172.21.15.32:0/2253494577 shutdown_connections 2023-09-30T14:37:17.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.813+0000 7f69b58bd700 1 --2- 172.21.15.32:0/2253494577 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f69980a1020 0x7f69980a34e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.813+0000 7f69b58bd700 1 --2- 172.21.15.32:0/2253494577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b0101940 0x7f69b019bea0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.813+0000 7f69b58bd700 1 --2- 172.21.15.32:0/2253494577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69b00ffdb0 0x7f69b01948f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.813+0000 7f69b58bd700 1 --2- 172.21.15.32:0/2253494577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b0100fa0 0x7f69b0194e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.813+0000 7f69b58bd700 1 -- 172.21.15.32:0/2253494577 >> 172.21.15.32:0/2253494577 conn(0x7f69b00fb340 msgr2=0x7f69b00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:17.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.813+0000 7f69b58bd700 1 -- 172.21.15.32:0/2253494577 shutdown_connections 2023-09-30T14:37:17.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.813+0000 7f69b58bd700 1 -- 172.21.15.32:0/2253494577 wait complete. 2023-09-30T14:37:17.825 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-09-30T14:37:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.956+0000 7f4d08077700 1 -- 172.21.15.32:0/2702151559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d00102970 msgr2=0x7f4d00109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.956+0000 7f4d08077700 1 --2- 172.21.15.32:0/2702151559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d00102970 0x7f4d00109290 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800b0a0 tx=0x7f4cf800b3b0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.956+0000 7f4d08077700 1 -- 172.21.15.32:0/2702151559 shutdown_connections 2023-09-30T14:37:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.956+0000 7f4d08077700 1 --2- 172.21.15.32:0/2702151559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d00102970 0x7f4d00109290 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.956+0000 7f4d08077700 1 --2- 172.21.15.32:0/2702151559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d00100e60 0x7f4d00101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.956+0000 7f4d08077700 1 --2- 172.21.15.32:0/2702151559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00101fb0 0x7f4d00102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.956+0000 7f4d08077700 1 -- 172.21.15.32:0/2702151559 >> 172.21.15.32:0/2702151559 conn(0x7f4d000fc3d0 msgr2=0x7f4d000fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.956+0000 7f4d08077700 1 -- 172.21.15.32:0/2702151559 shutdown_connections 2023-09-30T14:37:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.956+0000 7f4d08077700 1 -- 172.21.15.32:0/2702151559 wait complete. 2023-09-30T14:37:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d08077700 1 Processor -- start 2023-09-30T14:37:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d08077700 1 -- start start 2023-09-30T14:37:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d08077700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00100e60 0x7f4d00196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d08077700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d00101fb0 0x7f4d00197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d08077700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d00102970 0x7f4d0019e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d08077700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0019e770 con 0x7f4d00102970 2023-09-30T14:37:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d08077700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0019e8e0 con 0x7f4d00101fb0 2023-09-30T14:37:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d08077700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0019eaa0 con 0x7f4d00100e60 2023-09-30T14:37:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d06614700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d00102970 0x7f4d0019e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d06614700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d00102970 0x7f4d0019e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33892/0 (socket says 172.21.15.32:33892) 2023-09-30T14:37:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d06614700 1 -- 172.21.15.32:0/1781861267 learned_addr learned my addr 172.21.15.32:0/1781861267 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d05612700 1 --2- 172.21.15.32:0/1781861267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d00101fb0 0x7f4d00197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d05e13700 1 --2- 172.21.15.32:0/1781861267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00100e60 0x7f4d00196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d06614700 1 -- 172.21.15.32:0/1781861267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00100e60 msgr2=0x7f4d00196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d06614700 1 --2- 172.21.15.32:0/1781861267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00100e60 0x7f4d00196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d06614700 1 -- 172.21.15.32:0/1781861267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d00101fb0 msgr2=0x7f4d00197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d06614700 1 --2- 172.21.15.32:0/1781861267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d00101fb0 0x7f4d00197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d06614700 1 -- 172.21.15.32:0/1781861267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf8009d00 con 0x7f4d00102970 2023-09-30T14:37:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4d06614700 1 --2- 172.21.15.32:0/1781861267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d00102970 0x7f4d0019e1a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800b830 tx=0x7f4cf8004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.957+0000 7f4cf6ffd700 1 -- 172.21.15.32:0/1781861267 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf800e040 con 0x7f4d00102970 2023-09-30T14:37:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.958+0000 7f4d08077700 1 -- 172.21.15.32:0/1781861267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0019ed20 con 0x7f4d00102970 2023-09-30T14:37:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.958+0000 7f4cf6ffd700 1 -- 172.21.15.32:0/1781861267 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cf801db50 con 0x7f4d00102970 2023-09-30T14:37:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.958+0000 7f4cf6ffd700 1 -- 172.21.15.32:0/1781861267 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf801bb20 con 0x7f4d00102970 2023-09-30T14:37:17.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.958+0000 7f4d08077700 1 -- 172.21.15.32:0/1781861267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0019f1a0 con 0x7f4d00102970 2023-09-30T14:37:17.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.958+0000 7f4d08077700 1 -- 172.21.15.32:0/1781861267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0010c660 con 0x7f4d00102970 2023-09-30T14:37:17.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.960+0000 7f4cf6ffd700 1 -- 172.21.15.32:0/1781861267 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f4cf801dcc0 con 0x7f4d00102970 2023-09-30T14:37:17.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.960+0000 7f4cf6ffd700 1 --2- 172.21.15.32:0/1781861267 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4cec0a0fa0 0x7f4cec0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:17.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.960+0000 7f4cf6ffd700 1 -- 172.21.15.32:0/1781861267 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4cf80ce560 con 0x7f4d00102970 2023-09-30T14:37:17.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.960+0000 7f4d05e13700 1 --2- 172.21.15.32:0/1781861267 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4cec0a0fa0 0x7f4cec0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:17.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.961+0000 7f4d05e13700 1 --2- 172.21.15.32:0/1781861267 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4cec0a0fa0 0x7f4cec0a3460 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4d00198100 tx=0x7f4cfc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:17.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:17.964+0000 7f4cf6ffd700 1 -- 172.21.15.32:0/1781861267 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f4cf8097fc0 con 0x7f4d00102970 2023-09-30T14:37:18.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.076+0000 7f4cf6ffd700 1 -- 172.21.15.32:0/1781861267 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f4cf8098390 con 0x7f4d00102970 2023-09-30T14:37:18.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:18 smithi032 ceph-mon[98592]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:37:18.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.204+0000 7f4d08077700 1 -- 172.21.15.32:0/1781861267 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d00061260 con 0x7f4cec0a0fa0 2023-09-30T14:37:18.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.231+0000 7f4cf6ffd700 1 -- 172.21.15.32:0/1781861267 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4d00061260 con 0x7f4cec0a0fa0 2023-09-30T14:37:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.234+0000 7f4d08077700 1 -- 172.21.15.32:0/1781861267 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4cec0a0fa0 msgr2=0x7f4cec0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.234+0000 7f4d08077700 1 --2- 172.21.15.32:0/1781861267 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4cec0a0fa0 0x7f4cec0a3460 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4d00198100 tx=0x7f4cfc00b410 comp rx=0 tx=0).stop 2023-09-30T14:37:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.234+0000 7f4d08077700 1 -- 172.21.15.32:0/1781861267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d00102970 msgr2=0x7f4d0019e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.234+0000 7f4d08077700 1 --2- 172.21.15.32:0/1781861267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d00102970 0x7f4d0019e1a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800b830 tx=0x7f4cf8004c80 comp rx=0 tx=0).stop 2023-09-30T14:37:18.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.234+0000 7f4d08077700 1 -- 172.21.15.32:0/1781861267 shutdown_connections 2023-09-30T14:37:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.234+0000 7f4d08077700 1 --2- 172.21.15.32:0/1781861267 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4cec0a0fa0 0x7f4cec0a3460 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.234+0000 7f4d08077700 1 --2- 172.21.15.32:0/1781861267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d00102970 0x7f4d0019e1a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.234+0000 7f4d08077700 1 --2- 172.21.15.32:0/1781861267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d00100e60 0x7f4d00196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.234+0000 7f4d08077700 1 --2- 172.21.15.32:0/1781861267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d00101fb0 0x7f4d00197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.234+0000 7f4d08077700 1 -- 172.21.15.32:0/1781861267 >> 172.21.15.32:0/1781861267 conn(0x7f4d000fc3d0 msgr2=0x7f4d000fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.234+0000 7f4d08077700 1 -- 172.21.15.32:0/1781861267 shutdown_connections 2023-09-30T14:37:18.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.234+0000 7f4d08077700 1 -- 172.21.15.32:0/1781861267 wait complete. 2023-09-30T14:37:18.246 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-09-30T14:37:18.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:18 smithi148 ceph-mon[102355]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:37:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.375+0000 7fae5ba66700 1 -- 172.21.15.32:0/3814288457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae54100f40 msgr2=0x7fae541013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.375+0000 7fae5ba66700 1 --2- 172.21.15.32:0/3814288457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae54100f40 0x7fae541013a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fae500099c0 tx=0x7fae50009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.375+0000 7fae5ba66700 1 -- 172.21.15.32:0/3814288457 shutdown_connections 2023-09-30T14:37:18.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.375+0000 7fae5ba66700 1 --2- 172.21.15.32:0/3814288457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae541018e0 0x7fae54108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.375+0000 7fae5ba66700 1 --2- 172.21.15.32:0/3814288457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae540ffd50 0x7fae54100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.375+0000 7fae5ba66700 1 --2- 172.21.15.32:0/3814288457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae54100f40 0x7fae541013a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.375+0000 7fae5ba66700 1 -- 172.21.15.32:0/3814288457 >> 172.21.15.32:0/3814288457 conn(0x7fae540fb2a0 msgr2=0x7fae540fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:18.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.376+0000 7fae5ba66700 1 -- 172.21.15.32:0/3814288457 shutdown_connections 2023-09-30T14:37:18.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.376+0000 7fae5ba66700 1 -- 172.21.15.32:0/3814288457 wait complete. 2023-09-30T14:37:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.376+0000 7fae5ba66700 1 Processor -- start 2023-09-30T14:37:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.376+0000 7fae5ba66700 1 -- start start 2023-09-30T14:37:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae5ba66700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae540ffd50 0x7fae54195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae5ba66700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae54100f40 0x7fae54195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae5ba66700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae541018e0 0x7fae5419d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae5ba66700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5419d620 con 0x7fae541018e0 2023-09-30T14:37:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae5ba66700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5419d760 con 0x7fae540ffd50 2023-09-30T14:37:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae5ba66700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5419d8f0 con 0x7fae54100f40 2023-09-30T14:37:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae5a003700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae541018e0 0x7fae5419d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae5a003700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae541018e0 0x7fae5419d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33908/0 (socket says 172.21.15.32:33908) 2023-09-30T14:37:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae5a003700 1 -- 172.21.15.32:0/2592107021 learned_addr learned my addr 172.21.15.32:0/2592107021 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae59802700 1 --2- 172.21.15.32:0/2592107021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae540ffd50 0x7fae54195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae59001700 1 --2- 172.21.15.32:0/2592107021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae54100f40 0x7fae54195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae5a003700 1 -- 172.21.15.32:0/2592107021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae54100f40 msgr2=0x7fae54195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae5a003700 1 --2- 172.21.15.32:0/2592107021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae54100f40 0x7fae54195fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae5a003700 1 -- 172.21.15.32:0/2592107021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae540ffd50 msgr2=0x7fae54195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:18.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae5a003700 1 --2- 172.21.15.32:0/2592107021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae540ffd50 0x7fae54195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae5a003700 1 -- 172.21.15.32:0/2592107021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae500095f0 con 0x7fae541018e0 2023-09-30T14:37:18.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae5a003700 1 --2- 172.21.15.32:0/2592107021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae541018e0 0x7fae5419d050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00e580 tx=0x7fae4c00e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:18.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae4affd700 1 -- 172.21.15.32:0/2592107021 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c004db0 con 0x7fae541018e0 2023-09-30T14:37:18.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.377+0000 7fae4affd700 1 -- 172.21.15.32:0/2592107021 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae4c0090d0 con 0x7fae541018e0 2023-09-30T14:37:18.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.378+0000 7fae5ba66700 1 -- 172.21.15.32:0/2592107021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae54069040 con 0x7fae541018e0 2023-09-30T14:37:18.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.378+0000 7fae4affd700 1 -- 172.21.15.32:0/2592107021 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c004610 con 0x7fae541018e0 2023-09-30T14:37:18.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.378+0000 7fae5ba66700 1 -- 172.21.15.32:0/2592107021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae54069550 con 0x7fae541018e0 2023-09-30T14:37:18.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.379+0000 7fae5ba66700 1 -- 172.21.15.32:0/2592107021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae5410b5b0 con 0x7fae541018e0 2023-09-30T14:37:18.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.379+0000 7fae4affd700 1 -- 172.21.15.32:0/2592107021 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fae4c009240 con 0x7fae541018e0 2023-09-30T14:37:18.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.380+0000 7fae4affd700 1 --2- 172.21.15.32:0/2592107021 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fae400a1030 0x7fae400a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:18.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.380+0000 7fae59802700 1 --2- 172.21.15.32:0/2592107021 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fae400a1030 0x7fae400a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:18.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.380+0000 7fae4affd700 1 -- 172.21.15.32:0/2592107021 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fae4c016070 con 0x7fae541018e0 2023-09-30T14:37:18.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.381+0000 7fae59802700 1 --2- 172.21.15.32:0/2592107021 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fae400a1030 0x7fae400a34f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fae54100da0 tx=0x7fae44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:18.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.384+0000 7fae4affd700 1 -- 172.21.15.32:0/2592107021 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fae4c0980c0 con 0x7fae541018e0 2023-09-30T14:37:18.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:18 smithi156 ceph-mon[103579]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:37:18.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.624+0000 7fae5ba66700 1 -- 172.21.15.32:0/2592107021 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fae54069c30 con 0x7fae400a1030 2023-09-30T14:37:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.629+0000 7fae4affd700 1 -- 172.21.15.32:0/2592107021 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fae54069c30 con 0x7fae400a1030 2023-09-30T14:37:18.628 INFO:teuthology.orchestra.run.smithi032.stdout:/volumes/_nogroup/sv_1/b25fcf16-a734-410f-849e-f8fb3a417cdf 2023-09-30T14:37:18.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.631+0000 7fae5ba66700 1 -- 172.21.15.32:0/2592107021 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fae400a1030 msgr2=0x7fae400a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.631+0000 7fae5ba66700 1 --2- 172.21.15.32:0/2592107021 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fae400a1030 0x7fae400a34f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fae54100da0 tx=0x7fae44009260 comp rx=0 tx=0).stop 2023-09-30T14:37:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.631+0000 7fae5ba66700 1 -- 172.21.15.32:0/2592107021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae541018e0 msgr2=0x7fae5419d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.631+0000 7fae5ba66700 1 --2- 172.21.15.32:0/2592107021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae541018e0 0x7fae5419d050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00e580 tx=0x7fae4c00e890 comp rx=0 tx=0).stop 2023-09-30T14:37:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.631+0000 7fae5ba66700 1 -- 172.21.15.32:0/2592107021 shutdown_connections 2023-09-30T14:37:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.631+0000 7fae5ba66700 1 --2- 172.21.15.32:0/2592107021 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fae400a1030 0x7fae400a34f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.631+0000 7fae5ba66700 1 --2- 172.21.15.32:0/2592107021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae541018e0 0x7fae5419d050 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.631+0000 7fae5ba66700 1 --2- 172.21.15.32:0/2592107021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae540ffd50 0x7fae54195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.631+0000 7fae5ba66700 1 --2- 172.21.15.32:0/2592107021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae54100f40 0x7fae54195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.631+0000 7fae5ba66700 1 -- 172.21.15.32:0/2592107021 >> 172.21.15.32:0/2592107021 conn(0x7fae540fb2a0 msgr2=0x7fae540fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.631+0000 7fae5ba66700 1 -- 172.21.15.32:0/2592107021 shutdown_connections 2023-09-30T14:37:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.631+0000 7fae5ba66700 1 -- 172.21.15.32:0/2592107021 wait complete. 2023-09-30T14:37:18.643 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:37:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.769+0000 7ff7707c4700 1 -- 172.21.15.32:0/3631583341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7680ffd40 msgr2=0x7ff768100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.769+0000 7ff7707c4700 1 --2- 172.21.15.32:0/3631583341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7680ffd40 0x7ff768100160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff7640099c0 tx=0x7ff764009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.770+0000 7ff7707c4700 1 -- 172.21.15.32:0/3631583341 shutdown_connections 2023-09-30T14:37:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.770+0000 7ff7707c4700 1 --2- 172.21.15.32:0/3631583341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7681018d0 0x7ff768108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.770+0000 7ff7707c4700 1 --2- 172.21.15.32:0/3631583341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7680ffd40 0x7ff768100160 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.770+0000 7ff7707c4700 1 --2- 172.21.15.32:0/3631583341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff768100f30 0x7ff768101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.770+0000 7ff7707c4700 1 -- 172.21.15.32:0/3631583341 >> 172.21.15.32:0/3631583341 conn(0x7ff7680fb290 msgr2=0x7ff7680fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.770+0000 7ff7707c4700 1 -- 172.21.15.32:0/3631583341 shutdown_connections 2023-09-30T14:37:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.770+0000 7ff7707c4700 1 -- 172.21.15.32:0/3631583341 wait complete. 2023-09-30T14:37:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.770+0000 7ff7707c4700 1 Processor -- start 2023-09-30T14:37:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.770+0000 7ff7707c4700 1 -- start start 2023-09-30T14:37:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.770+0000 7ff7707c4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7680ffd40 0x7ff768196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.770+0000 7ff7707c4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff768100f30 0x7ff768197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.770+0000 7ff7707c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7681018d0 0x7ff76819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.770+0000 7ff7707c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76819e680 con 0x7ff7681018d0 2023-09-30T14:37:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.770+0000 7ff7707c4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76819e7c0 con 0x7ff768100f30 2023-09-30T14:37:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.770+0000 7ff7707c4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76819e950 con 0x7ff7680ffd40 2023-09-30T14:37:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff76ed61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7681018d0 0x7ff76819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff76ed61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7681018d0 0x7ff76819e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33922/0 (socket says 172.21.15.32:33922) 2023-09-30T14:37:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff76ed61700 1 -- 172.21.15.32:0/402877081 learned_addr learned my addr 172.21.15.32:0/402877081 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff76e560700 1 --2- 172.21.15.32:0/402877081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7680ffd40 0x7ff768196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff76dd5f700 1 --2- 172.21.15.32:0/402877081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff768100f30 0x7ff768197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff76ed61700 1 -- 172.21.15.32:0/402877081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7680ffd40 msgr2=0x7ff768196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff76ed61700 1 --2- 172.21.15.32:0/402877081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7680ffd40 0x7ff768196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff76ed61700 1 -- 172.21.15.32:0/402877081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff768100f30 msgr2=0x7ff768197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff76ed61700 1 --2- 172.21.15.32:0/402877081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff768100f30 0x7ff768197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff76ed61700 1 -- 172.21.15.32:0/402877081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7640095f0 con 0x7ff7681018d0 2023-09-30T14:37:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff76dd5f700 1 --2- 172.21.15.32:0/402877081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff768100f30 0x7ff768197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:37:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff76ed61700 1 --2- 172.21.15.32:0/402877081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7681018d0 0x7ff76819e0b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff76000b970 tx=0x7ff76000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff75f7fe700 1 -- 172.21.15.32:0/402877081 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76000c930 con 0x7ff7681018d0 2023-09-30T14:37:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff75f7fe700 1 -- 172.21.15.32:0/402877081 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff760014440 con 0x7ff7681018d0 2023-09-30T14:37:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff7707c4700 1 -- 172.21.15.32:0/402877081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff76819ec30 con 0x7ff7681018d0 2023-09-30T14:37:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff75f7fe700 1 -- 172.21.15.32:0/402877081 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7600075d0 con 0x7ff7681018d0 2023-09-30T14:37:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.771+0000 7ff7707c4700 1 -- 172.21.15.32:0/402877081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76819f140 con 0x7ff7681018d0 2023-09-30T14:37:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.772+0000 7ff7707c4700 1 -- 172.21.15.32:0/402877081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76810b620 con 0x7ff7681018d0 2023-09-30T14:37:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.773+0000 7ff75f7fe700 1 -- 172.21.15.32:0/402877081 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7ff7600145b0 con 0x7ff7681018d0 2023-09-30T14:37:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.774+0000 7ff75f7fe700 1 --2- 172.21.15.32:0/402877081 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff7540a10f0 0x7ff7540a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.774+0000 7ff76e560700 1 --2- 172.21.15.32:0/402877081 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff7540a10f0 0x7ff7540a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.774+0000 7ff75f7fe700 1 -- 172.21.15.32:0/402877081 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff760006e40 con 0x7ff7681018d0 2023-09-30T14:37:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.774+0000 7ff76e560700 1 --2- 172.21.15.32:0/402877081 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff7540a10f0 0x7ff7540a35b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff764009f50 tx=0x7ff7640058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:18.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:18.778+0000 7ff75f7fe700 1 -- 172.21.15.32:0/402877081 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7ff76004ef30 con 0x7ff7681018d0 2023-09-30T14:37:19.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.029+0000 7ff7707c4700 1 -- 172.21.15.32:0/402877081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff76810b810 con 0x7ff7681018d0 2023-09-30T14:37:19.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.030+0000 7ff75f7fe700 1 -- 172.21.15.32:0/402877081 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11321 (secure 0 0 0) 0x7ff760019090 con 0x7ff7681018d0 2023-09-30T14:37:19.030 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:37:19.030 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3505064841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3505064841},{"type":"v1","addr":"172.21.15.32:6839","nonce":3505064841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2262846038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2262846038},{"type":"v1","addr":"172.21.15.32:6841","nonce":2262846038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-30T14:37:11.444993+0000","modified":"2023-09-30T14:37:16.971534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6839/1445176203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1445176203},{"type":"v1","addr":"172.21.15.156:6839","nonce":1445176203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/494821686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":494821686},{"type":"v1","addr":"172.21.15.148:6839","nonce":494821686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T14:37:19.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.032+0000 7ff7707c4700 1 -- 172.21.15.32:0/402877081 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff7540a10f0 msgr2=0x7ff7540a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:19.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.032+0000 7ff7707c4700 1 --2- 172.21.15.32:0/402877081 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff7540a10f0 0x7ff7540a35b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff764009f50 tx=0x7ff7640058e0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.032+0000 7ff7707c4700 1 -- 172.21.15.32:0/402877081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7681018d0 msgr2=0x7ff76819e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:19.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.032+0000 7ff7707c4700 1 --2- 172.21.15.32:0/402877081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7681018d0 0x7ff76819e0b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff76000b970 tx=0x7ff76000bc80 comp rx=0 tx=0).stop 2023-09-30T14:37:19.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.032+0000 7ff7707c4700 1 -- 172.21.15.32:0/402877081 shutdown_connections 2023-09-30T14:37:19.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.032+0000 7ff7707c4700 1 --2- 172.21.15.32:0/402877081 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff7540a10f0 0x7ff7540a35b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.032+0000 7ff7707c4700 1 --2- 172.21.15.32:0/402877081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7681018d0 0x7ff76819e0b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.032+0000 7ff7707c4700 1 --2- 172.21.15.32:0/402877081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7680ffd40 0x7ff768196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.032+0000 7ff7707c4700 1 --2- 172.21.15.32:0/402877081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff768100f30 0x7ff768197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.033+0000 7ff7707c4700 1 -- 172.21.15.32:0/402877081 >> 172.21.15.32:0/402877081 conn(0x7ff7680fb290 msgr2=0x7ff768069230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:19.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.033+0000 7ff7707c4700 1 -- 172.21.15.32:0/402877081 shutdown_connections 2023-09-30T14:37:19.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.033+0000 7ff7707c4700 1 -- 172.21.15.32:0/402877081 wait complete. 2023-09-30T14:37:19.034 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 22 2023-09-30T14:37:19.045 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T14:37:19.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.171+0000 7fc4f5411700 1 -- 172.21.15.32:0/1960025436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f0106b70 msgr2=0x7fc4f0108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.171+0000 7fc4f5411700 1 --2- 172.21.15.32:0/1960025436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f0106b70 0x7fc4f0108f80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800b0a0 tx=0x7fc4e800b3b0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.171+0000 7fc4f5411700 1 -- 172.21.15.32:0/1960025436 shutdown_connections 2023-09-30T14:37:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.171+0000 7fc4f5411700 1 --2- 172.21.15.32:0/1960025436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f0106b70 0x7fc4f0108f80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.171+0000 7fc4f5411700 1 --2- 172.21.15.32:0/1960025436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4f0101910 0x7fc4f0103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.171+0000 7fc4f5411700 1 --2- 172.21.15.32:0/1960025436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4f0104240 0x7fc4f0106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.171+0000 7fc4f5411700 1 -- 172.21.15.32:0/1960025436 >> 172.21.15.32:0/1960025436 conn(0x7fc4f00fb280 msgr2=0x7fc4f00fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.171+0000 7fc4f5411700 1 -- 172.21.15.32:0/1960025436 shutdown_connections 2023-09-30T14:37:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.171+0000 7fc4f5411700 1 -- 172.21.15.32:0/1960025436 wait complete. 2023-09-30T14:37:19.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.172+0000 7fc4f5411700 1 Processor -- start 2023-09-30T14:37:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.172+0000 7fc4f5411700 1 -- start start 2023-09-30T14:37:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4f5411700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4f0101910 0x7fc4f0196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4f5411700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f0104240 0x7fc4f0197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4f5411700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4f0106b70 0x7fc4f019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4f5411700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f019e6b0 con 0x7fc4f0104240 2023-09-30T14:37:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4f5411700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f019e7f0 con 0x7fc4f0101910 2023-09-30T14:37:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4f5411700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f019e980 con 0x7fc4f0106b70 2023-09-30T14:37:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4ee7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f0104240 0x7fc4f0197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4ee7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f0104240 0x7fc4f0197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33934/0 (socket says 172.21.15.32:33934) 2023-09-30T14:37:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4ee7fc700 1 -- 172.21.15.32:0/3222870387 learned_addr learned my addr 172.21.15.32:0/3222870387 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4eeffd700 1 --2- 172.21.15.32:0/3222870387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4f0101910 0x7fc4f0196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4ef7fe700 1 --2- 172.21.15.32:0/3222870387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4f0106b70 0x7fc4f019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4ee7fc700 1 -- 172.21.15.32:0/3222870387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4f0106b70 msgr2=0x7fc4f019e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4ee7fc700 1 --2- 172.21.15.32:0/3222870387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4f0106b70 0x7fc4f019e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4ee7fc700 1 -- 172.21.15.32:0/3222870387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4f0101910 msgr2=0x7fc4f0196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4ee7fc700 1 --2- 172.21.15.32:0/3222870387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4f0101910 0x7fc4f0196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4ee7fc700 1 -- 172.21.15.32:0/3222870387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e8009d00 con 0x7fc4f0104240 2023-09-30T14:37:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4ee7fc700 1 --2- 172.21.15.32:0/3222870387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f0104240 0x7fc4f0197070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400d980 tx=0x7fc4e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4d7fff700 1 -- 172.21.15.32:0/3222870387 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4004220 con 0x7fc4f0104240 2023-09-30T14:37:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.173+0000 7fc4d7fff700 1 -- 172.21.15.32:0/3222870387 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4e4007640 con 0x7fc4f0104240 2023-09-30T14:37:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.174+0000 7fc4f5411700 1 -- 172.21.15.32:0/3222870387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f019ec60 con 0x7fc4f0104240 2023-09-30T14:37:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.174+0000 7fc4d7fff700 1 -- 172.21.15.32:0/3222870387 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4005690 con 0x7fc4f0104240 2023-09-30T14:37:19.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.174+0000 7fc4f5411700 1 -- 172.21.15.32:0/3222870387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f019f170 con 0x7fc4f0104240 2023-09-30T14:37:19.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.174+0000 7fc4f5411700 1 -- 172.21.15.32:0/3222870387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f00fce00 con 0x7fc4f0104240 2023-09-30T14:37:19.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.176+0000 7fc4d7fff700 1 -- 172.21.15.32:0/3222870387 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fc4e4004750 con 0x7fc4f0104240 2023-09-30T14:37:19.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.176+0000 7fc4d7fff700 1 --2- 172.21.15.32:0/3222870387 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc4d80a1070 0x7fc4d80a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.176+0000 7fc4eeffd700 1 --2- 172.21.15.32:0/3222870387 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc4d80a1070 0x7fc4d80a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.176+0000 7fc4d7fff700 1 -- 172.21.15.32:0/3222870387 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc4e40cc970 con 0x7fc4f0104240 2023-09-30T14:37:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.177+0000 7fc4eeffd700 1 --2- 172.21.15.32:0/3222870387 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc4d80a1070 0x7fc4d80a3530 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000ac50 tx=0x7fc4e000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:19.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.180+0000 7fc4d7fff700 1 -- 172.21.15.32:0/3222870387 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fc4e40961c0 con 0x7fc4f0104240 2023-09-30T14:37:19.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:19 smithi148 ceph-mon[102355]: pgmap v256: 97 pgs: 97 active+clean; 580 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2023-09-30T14:37:19.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:19 smithi148 ceph-mon[102355]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:37:19.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:19 smithi148 ceph-mon[102355]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-30T14:37:19.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:19 smithi148 ceph-mon[102355]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:37:19.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:19 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/402877081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:37:19.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.385+0000 7fc4f5411700 1 -- 172.21.15.32:0/3222870387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4f010c820 con 0x7fc4f0104240 2023-09-30T14:37:19.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.387+0000 7fc4d7fff700 1 -- 172.21.15.32:0/3222870387 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22133 (secure 0 0 0) 0x7fc4e4016020 con 0x7fc4f0104240 2023-09-30T14:37:19.387 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:37:19.387 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":83,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:37:12.972217+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:37:19.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:19 smithi032 ceph-mon[98592]: pgmap v256: 97 pgs: 97 active+clean; 580 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2023-09-30T14:37:19.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:19 smithi032 ceph-mon[98592]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:37:19.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:19 smithi032 ceph-mon[98592]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-30T14:37:19.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:19 smithi032 ceph-mon[98592]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:37:19.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:19 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/402877081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:37:19.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.389+0000 7fc4f5411700 1 -- 172.21.15.32:0/3222870387 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc4d80a1070 msgr2=0x7fc4d80a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:19.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.389+0000 7fc4f5411700 1 --2- 172.21.15.32:0/3222870387 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc4d80a1070 0x7fc4d80a3530 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000ac50 tx=0x7fc4e000a380 comp rx=0 tx=0).stop 2023-09-30T14:37:19.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.389+0000 7fc4f5411700 1 -- 172.21.15.32:0/3222870387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f0104240 msgr2=0x7fc4f0197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:19.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.389+0000 7fc4f5411700 1 --2- 172.21.15.32:0/3222870387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f0104240 0x7fc4f0197070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400d980 tx=0x7fc4e400dc90 comp rx=0 tx=0).stop 2023-09-30T14:37:19.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.389+0000 7fc4f5411700 1 -- 172.21.15.32:0/3222870387 shutdown_connections 2023-09-30T14:37:19.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.389+0000 7fc4f5411700 1 --2- 172.21.15.32:0/3222870387 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc4d80a1070 0x7fc4d80a3530 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.389+0000 7fc4f5411700 1 --2- 172.21.15.32:0/3222870387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4f0106b70 0x7fc4f019e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.389+0000 7fc4f5411700 1 --2- 172.21.15.32:0/3222870387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4f0101910 0x7fc4f0196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.389+0000 7fc4f5411700 1 --2- 172.21.15.32:0/3222870387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4f0104240 0x7fc4f0197070 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.389+0000 7fc4f5411700 1 -- 172.21.15.32:0/3222870387 >> 172.21.15.32:0/3222870387 conn(0x7fc4f00fb280 msgr2=0x7fc4f0102580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:19.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.389+0000 7fc4f5411700 1 -- 172.21.15.32:0/3222870387 shutdown_connections 2023-09-30T14:37:19.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.389+0000 7fc4f5411700 1 -- 172.21.15.32:0/3222870387 wait complete. 2023-09-30T14:37:19.402 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-30T14:37:19.402 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T14:37:19.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.528+0000 7fcd0fbfa700 1 -- 172.21.15.32:0/2942390191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd08102970 msgr2=0x7fcd08109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:19.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.528+0000 7fcd0fbfa700 1 --2- 172.21.15.32:0/2942390191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd08102970 0x7fcd08109290 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000b0a0 tx=0x7fcd0000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.529+0000 7fcd0fbfa700 1 -- 172.21.15.32:0/2942390191 shutdown_connections 2023-09-30T14:37:19.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.529+0000 7fcd0fbfa700 1 --2- 172.21.15.32:0/2942390191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd08102970 0x7fcd08109290 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.529+0000 7fcd0fbfa700 1 --2- 172.21.15.32:0/2942390191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd08100e60 0x7fcd08101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.529+0000 7fcd0fbfa700 1 --2- 172.21.15.32:0/2942390191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd08101fb0 0x7fcd08102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.529+0000 7fcd0fbfa700 1 -- 172.21.15.32:0/2942390191 >> 172.21.15.32:0/2942390191 conn(0x7fcd080fc390 msgr2=0x7fcd080fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:19.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.529+0000 7fcd0fbfa700 1 -- 172.21.15.32:0/2942390191 shutdown_connections 2023-09-30T14:37:19.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.529+0000 7fcd0fbfa700 1 -- 172.21.15.32:0/2942390191 wait complete. 2023-09-30T14:37:19.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.529+0000 7fcd0fbfa700 1 Processor -- start 2023-09-30T14:37:19.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.529+0000 7fcd0fbfa700 1 -- start start 2023-09-30T14:37:19.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.529+0000 7fcd0fbfa700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd08100e60 0x7fcd08198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:19.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.529+0000 7fcd0fbfa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd08101fb0 0x7fcd08199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:19.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.529+0000 7fcd0fbfa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd08102970 0x7fcd081a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:19.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.529+0000 7fcd0fbfa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd081a0900 con 0x7fcd08102970 2023-09-30T14:37:19.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.529+0000 7fcd0fbfa700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd081a0a70 con 0x7fcd08101fb0 2023-09-30T14:37:19.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.529+0000 7fcd0fbfa700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd081a0c30 con 0x7fcd08100e60 2023-09-30T14:37:19.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fcd0e197700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd08102970 0x7fcd081a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:19.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fcd0e197700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd08102970 0x7fcd081a0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33950/0 (socket says 172.21.15.32:33950) 2023-09-30T14:37:19.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fcd0e197700 1 -- 172.21.15.32:0/2157841192 learned_addr learned my addr 172.21.15.32:0/2157841192 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:19.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fcd0d996700 1 --2- 172.21.15.32:0/2157841192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd08100e60 0x7fcd08198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:19.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fcd0d195700 1 --2- 172.21.15.32:0/2157841192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd08101fb0 0x7fcd08199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:19.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fcd0e197700 1 -- 172.21.15.32:0/2157841192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd08100e60 msgr2=0x7fcd08198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:19.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fcd0e197700 1 --2- 172.21.15.32:0/2157841192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd08100e60 0x7fcd08198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fcd0e197700 1 -- 172.21.15.32:0/2157841192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd08101fb0 msgr2=0x7fcd08199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:19.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fcd0e197700 1 --2- 172.21.15.32:0/2157841192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd08101fb0 0x7fcd08199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fcd0e197700 1 -- 172.21.15.32:0/2157841192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd00009d00 con 0x7fcd08102970 2023-09-30T14:37:19.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fcd0d996700 1 --2- 172.21.15.32:0/2157841192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd08100e60 0x7fcd08198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:37:19.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fcd0d195700 1 --2- 172.21.15.32:0/2157841192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd08101fb0 0x7fcd08199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:37:19.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fcd0e197700 1 --2- 172.21.15.32:0/2157841192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd08102970 0x7fcd081a0330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000b830 tx=0x7fcd00004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:19.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fccfeffd700 1 -- 172.21.15.32:0/2157841192 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0000e040 con 0x7fcd08102970 2023-09-30T14:37:19.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fccfeffd700 1 -- 172.21.15.32:0/2157841192 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd000094c0 con 0x7fcd08102970 2023-09-30T14:37:19.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fcd0fbfa700 1 -- 172.21.15.32:0/2157841192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd081a0eb0 con 0x7fcd08102970 2023-09-30T14:37:19.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fccfeffd700 1 -- 172.21.15.32:0/2157841192 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0001bc70 con 0x7fcd08102970 2023-09-30T14:37:19.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.530+0000 7fcd0fbfa700 1 -- 172.21.15.32:0/2157841192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd081a1270 con 0x7fcd08102970 2023-09-30T14:37:19.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.531+0000 7fcd0fbfa700 1 -- 172.21.15.32:0/2157841192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0810c660 con 0x7fcd08102970 2023-09-30T14:37:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.532+0000 7fccfeffd700 1 -- 172.21.15.32:0/2157841192 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fcd0001d890 con 0x7fcd08102970 2023-09-30T14:37:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.533+0000 7fccfeffd700 1 --2- 172.21.15.32:0/2157841192 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fccf40a1070 0x7fccf40a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.533+0000 7fcd0d996700 1 --2- 172.21.15.32:0/2157841192 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fccf40a1070 0x7fccf40a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.533+0000 7fccfeffd700 1 -- 172.21.15.32:0/2157841192 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcd000ce910 con 0x7fcd08102970 2023-09-30T14:37:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.533+0000 7fcd0d996700 1 --2- 172.21.15.32:0/2157841192 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fccf40a1070 0x7fccf40a3530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcd04005fd0 tx=0x7fcd04005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:19.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.536+0000 7fccfeffd700 1 -- 172.21.15.32:0/2157841192 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fcd00098170 con 0x7fcd08102970 2023-09-30T14:37:19.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:19 smithi156 ceph-mon[103579]: pgmap v256: 97 pgs: 97 active+clean; 580 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2023-09-30T14:37:19.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:19 smithi156 ceph-mon[103579]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:37:19.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:19 smithi156 ceph-mon[103579]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-30T14:37:19.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:19 smithi156 ceph-mon[103579]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:37:19.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:19 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/402877081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:37:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.733+0000 7fcd0fbfa700 1 -- 172.21.15.32:0/2157841192 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd08108fb0 con 0x7fccf40a1070 2023-09-30T14:37:19.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.746+0000 7fccfeffd700 1 -- 172.21.15.32:0/2157841192 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335593 (secure 0 0 0) 0x7fcd08108fb0 con 0x7fccf40a1070 2023-09-30T14:37:19.749 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:37:19.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.749+0000 7fcd0fbfa700 1 -- 172.21.15.32:0/2157841192 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fccf40a1070 msgr2=0x7fccf40a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:19.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.749+0000 7fcd0fbfa700 1 --2- 172.21.15.32:0/2157841192 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fccf40a1070 0x7fccf40a3530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcd04005fd0 tx=0x7fcd04005f00 comp rx=0 tx=0).stop 2023-09-30T14:37:19.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.749+0000 7fcd0fbfa700 1 -- 172.21.15.32:0/2157841192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd08102970 msgr2=0x7fcd081a0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:19.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.749+0000 7fcd0fbfa700 1 --2- 172.21.15.32:0/2157841192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd08102970 0x7fcd081a0330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000b830 tx=0x7fcd00004d30 comp rx=0 tx=0).stop 2023-09-30T14:37:19.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.750+0000 7fcd0fbfa700 1 -- 172.21.15.32:0/2157841192 shutdown_connections 2023-09-30T14:37:19.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.750+0000 7fcd0fbfa700 1 --2- 172.21.15.32:0/2157841192 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fccf40a1070 0x7fccf40a3530 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.750+0000 7fcd0fbfa700 1 --2- 172.21.15.32:0/2157841192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd08102970 0x7fcd081a0330 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.750+0000 7fcd0fbfa700 1 --2- 172.21.15.32:0/2157841192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd08100e60 0x7fcd08198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.750+0000 7fcd0fbfa700 1 --2- 172.21.15.32:0/2157841192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd08101fb0 0x7fcd08199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.750+0000 7fcd0fbfa700 1 -- 172.21.15.32:0/2157841192 >> 172.21.15.32:0/2157841192 conn(0x7fcd080fc390 msgr2=0x7fcd08107890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:19.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.750+0000 7fcd0fbfa700 1 -- 172.21.15.32:0/2157841192 shutdown_connections 2023-09-30T14:37:19.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.750+0000 7fcd0fbfa700 1 -- 172.21.15.32:0/2157841192 wait complete. 2023-09-30T14:37:19.753 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-09-30T14:37:19.762 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2023-09-30T14:37:17.771311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594296,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":259,"num_write_kb":2966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":260,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336496,"kb_used_data":12252,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605328,"statfs":{"total":1151940427776,"available":1151595855872,"internally_reserved":0,"allocated":12546048,"data_stored":6869404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3928,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.762188"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014576+0000","last_change":"2023-09-30T14:37:11.988122+0000","last_active":"2023-09-30T14:37:13.014576+0000","last_peered":"2023-09-30T14:37:13.014576+0000","last_clean":"2023-09-30T14:37:13.014576+0000","last_became_active":"2023-09-30T14:37:11.986770+0000","last_became_peered":"2023-09-30T14:37:11.986770+0000","last_unstale":"2023-09-30T14:37:13.014576+0000","last_undegraded":"2023-09-30T14:37:13.014576+0000","last_fullsized":"2023-09-30T14:37:13.014576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:46:29.655309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014211+0000","last_change":"2023-09-30T14:37:11.987953+0000","last_active":"2023-09-30T14:37:13.014211+0000","last_peered":"2023-09-30T14:37:13.014211+0000","last_clean":"2023-09-30T14:37:13.014211+0000","last_became_active":"2023-09-30T14:37:11.987248+0000","last_became_peered":"2023-09-30T14:37:11.987248+0000","last_unstale":"2023-09-30T14:37:13.014211+0000","last_undegraded":"2023-09-30T14:37:13.014211+0000","last_fullsized":"2023-09-30T14:37:13.014211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:12:59.780089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014752+0000","last_change":"2023-09-30T14:37:11.987570+0000","last_active":"2023-09-30T14:37:13.014752+0000","last_peered":"2023-09-30T14:37:13.014752+0000","last_clean":"2023-09-30T14:37:13.014752+0000","last_became_active":"2023-09-30T14:37:11.987064+0000","last_became_peered":"2023-09-30T14:37:11.987064+0000","last_unstale":"2023-09-30T14:37:13.014752+0000","last_undegraded":"2023-09-30T14:37:13.014752+0000","last_fullsized":"2023-09-30T14:37:13.014752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:26:16.674605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:11.991224+0000","last_change":"2023-09-30T14:37:11.990161+0000","last_active":"2023-09-30T14:37:11.991224+0000","last_peered":"2023-09-30T14:37:11.991224+0000","last_clean":"2023-09-30T14:37:11.991224+0000","last_became_active":"2023-09-30T14:37:11.989580+0000","last_became_peered":"2023-09-30T14:37:11.989580+0000","last_unstale":"2023-09-30T14:37:11.991224+0000","last_undegraded":"2023-09-30T14:37:11.991224+0000","last_fullsized":"2023-09-30T14:37:11.991224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:03:44.023417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130616+0000","last_change":"2023-09-30T14:37:11.993902+0000","last_active":"2023-09-30T14:37:13.130616+0000","last_peered":"2023-09-30T14:37:13.130616+0000","last_clean":"2023-09-30T14:37:13.130616+0000","last_became_active":"2023-09-30T14:37:11.993622+0000","last_became_peered":"2023-09-30T14:37:11.993622+0000","last_unstale":"2023-09-30T14:37:13.130616+0000","last_undegraded":"2023-09-30T14:37:13.130616+0000","last_fullsized":"2023-09-30T14:37:13.130616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:09:49.252823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.015937+0000","last_change":"2023-09-30T14:37:11.997282+0000","last_active":"2023-09-30T14:37:13.015937+0000","last_peered":"2023-09-30T14:37:13.015937+0000","last_clean":"2023-09-30T14:37:13.015937+0000","last_became_active":"2023-09-30T14:37:11.996937+0000","last_became_peered":"2023-09-30T14:37:11.996937+0000","last_unstale":"2023-09-30T14:37:13.015937+0000","last_undegraded":"2023-09-30T14:37:13.015937+0000","last_fullsized":"2023-09-30T14:37:13.015937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:39:27.351772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014196+0000","last_change":"2023-09-30T14:37:11.997408+0000","last_active":"2023-09-30T14:37:13.014196+0000","last_peered":"2023-09-30T14:37:13.014196+0000","last_clean":"2023-09-30T14:37:13.014196+0000","last_became_active":"2023-09-30T14:37:11.997147+0000","last_became_peered":"2023-09-30T14:37:11.997147+0000","last_unstale":"2023-09-30T14:37:13.014196+0000","last_undegraded":"2023-09-30T14:37:13.014196+0000","last_fullsized":"2023-09-30T14:37:13.014196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:21:24.721417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975636+0000","last_change":"2023-09-30T14:37:11.986172+0000","last_active":"2023-09-30T14:37:12.975636+0000","last_peered":"2023-09-30T14:37:12.975636+0000","last_clean":"2023-09-30T14:37:12.975636+0000","last_became_active":"2023-09-30T14:37:11.985836+0000","last_became_peered":"2023-09-30T14:37:11.985836+0000","last_unstale":"2023-09-30T14:37:12.975636+0000","last_undegraded":"2023-09-30T14:37:12.975636+0000","last_fullsized":"2023-09-30T14:37:12.975636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:35:57.464679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976406+0000","last_change":"2023-09-30T14:37:11.990853+0000","last_active":"2023-09-30T14:37:12.976406+0000","last_peered":"2023-09-30T14:37:12.976406+0000","last_clean":"2023-09-30T14:37:12.976406+0000","last_became_active":"2023-09-30T14:37:11.990321+0000","last_became_peered":"2023-09-30T14:37:11.990321+0000","last_unstale":"2023-09-30T14:37:12.976406+0000","last_undegraded":"2023-09-30T14:37:12.976406+0000","last_fullsized":"2023-09-30T14:37:12.976406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:32:11.409927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.016101+0000","last_change":"2023-09-30T14:37:11.992094+0000","last_active":"2023-09-30T14:37:13.016101+0000","last_peered":"2023-09-30T14:37:13.016101+0000","last_clean":"2023-09-30T14:37:13.016101+0000","last_became_active":"2023-09-30T14:37:11.991762+0000","last_became_peered":"2023-09-30T14:37:11.991762+0000","last_unstale":"2023-09-30T14:37:13.016101+0000","last_undegraded":"2023-09-30T14:37:13.016101+0000","last_fullsized":"2023-09-30T14:37:13.016101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:22:50.203788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014249+0000","last_change":"2023-09-30T14:37:11.992213+0000","last_active":"2023-09-30T14:37:13.014249+0000","last_peered":"2023-09-30T14:37:13.014249+0000","last_clean":"2023-09-30T14:37:13.014249+0000","last_became_active":"2023-09-30T14:37:11.991868+0000","last_became_peered":"2023-09-30T14:37:11.991868+0000","last_unstale":"2023-09-30T14:37:13.014249+0000","last_undegraded":"2023-09-30T14:37:13.014249+0000","last_fullsized":"2023-09-30T14:37:13.014249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:20:06.061978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:11.991304+0000","last_change":"2023-09-30T14:37:11.990043+0000","last_active":"2023-09-30T14:37:11.991304+0000","last_peered":"2023-09-30T14:37:11.991304+0000","last_clean":"2023-09-30T14:37:11.991304+0000","last_became_active":"2023-09-30T14:37:11.988243+0000","last_became_peered":"2023-09-30T14:37:11.988243+0000","last_unstale":"2023-09-30T14:37:11.991304+0000","last_undegraded":"2023-09-30T14:37:11.991304+0000","last_fullsized":"2023-09-30T14:37:11.991304+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:50:58.741971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014266+0000","last_change":"2023-09-30T14:37:11.990361+0000","last_active":"2023-09-30T14:37:13.014266+0000","last_peered":"2023-09-30T14:37:13.014266+0000","last_clean":"2023-09-30T14:37:13.014266+0000","last_became_active":"2023-09-30T14:37:11.990080+0000","last_became_peered":"2023-09-30T14:37:11.990080+0000","last_unstale":"2023-09-30T14:37:13.014266+0000","last_undegraded":"2023-09-30T14:37:13.014266+0000","last_fullsized":"2023-09-30T14:37:13.014266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:27:08.830419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014485+0000","last_change":"2023-09-30T14:37:11.988119+0000","last_active":"2023-09-30T14:37:13.014485+0000","last_peered":"2023-09-30T14:37:13.014485+0000","last_clean":"2023-09-30T14:37:13.014485+0000","last_became_active":"2023-09-30T14:37:11.987704+0000","last_became_peered":"2023-09-30T14:37:11.987704+0000","last_unstale":"2023-09-30T14:37:13.014485+0000","last_undegraded":"2023-09-30T14:37:13.014485+0000","last_fullsized":"2023-09-30T14:37:13.014485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:47:23.318982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.013931+0000","last_change":"2023-09-30T14:37:11.985403+0000","last_active":"2023-09-30T14:37:13.013931+0000","last_peered":"2023-09-30T14:37:13.013931+0000","last_clean":"2023-09-30T14:37:13.013931+0000","last_became_active":"2023-09-30T14:37:11.984818+0000","last_became_peered":"2023-09-30T14:37:11.984818+0000","last_unstale":"2023-09-30T14:37:13.013931+0000","last_undegraded":"2023-09-30T14:37:13.013931+0000","last_fullsized":"2023-09-30T14:37:13.013931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:10:43.592505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976073+0000","last_change":"2023-09-30T14:37:11.989500+0000","last_active":"2023-09-30T14:37:12.976073+0000","last_peered":"2023-09-30T14:37:12.976073+0000","last_clean":"2023-09-30T14:37:12.976073+0000","last_became_active":"2023-09-30T14:37:11.989186+0000","last_became_peered":"2023-09-30T14:37:11.989186+0000","last_unstale":"2023-09-30T14:37:12.976073+0000","last_undegraded":"2023-09-30T14:37:12.976073+0000","last_fullsized":"2023-09-30T14:37:12.976073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:14:35.497319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014676+0000","last_change":"2023-09-30T14:37:11.987217+0000","last_active":"2023-09-30T14:37:13.014676+0000","last_peered":"2023-09-30T14:37:13.014676+0000","last_clean":"2023-09-30T14:37:13.014676+0000","last_became_active":"2023-09-30T14:37:11.986835+0000","last_became_peered":"2023-09-30T14:37:11.986835+0000","last_unstale":"2023-09-30T14:37:13.014676+0000","last_undegraded":"2023-09-30T14:37:13.014676+0000","last_fullsized":"2023-09-30T14:37:13.014676+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:11:19.234768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014440+0000","last_change":"2023-09-30T14:37:11.995224+0000","last_active":"2023-09-30T14:37:13.014440+0000","last_peered":"2023-09-30T14:37:13.014440+0000","last_clean":"2023-09-30T14:37:13.014440+0000","last_became_active":"2023-09-30T14:37:11.994925+0000","last_became_peered":"2023-09-30T14:37:11.994925+0000","last_unstale":"2023-09-30T14:37:13.014440+0000","last_undegraded":"2023-09-30T14:37:13.014440+0000","last_fullsized":"2023-09-30T14:37:13.014440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:08:07.453499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975873+0000","last_change":"2023-09-30T14:37:11.990182+0000","last_active":"2023-09-30T14:37:12.975873+0000","last_peered":"2023-09-30T14:37:12.975873+0000","last_clean":"2023-09-30T14:37:12.975873+0000","last_became_active":"2023-09-30T14:37:11.989862+0000","last_became_peered":"2023-09-30T14:37:11.989862+0000","last_unstale":"2023-09-30T14:37:12.975873+0000","last_undegraded":"2023-09-30T14:37:12.975873+0000","last_fullsized":"2023-09-30T14:37:12.975873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:15:31.713002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014062+0000","last_change":"2023-09-30T14:37:11.028123+0000","last_active":"2023-09-30T14:37:13.014062+0000","last_peered":"2023-09-30T14:37:13.014062+0000","last_clean":"2023-09-30T14:37:13.014062+0000","last_became_active":"2023-09-30T14:37:11.027764+0000","last_became_peered":"2023-09-30T14:37:11.027764+0000","last_unstale":"2023-09-30T14:37:13.014062+0000","last_undegraded":"2023-09-30T14:37:13.014062+0000","last_fullsized":"2023-09-30T14:37:13.014062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:15:50.818783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.977049+0000","last_change":"2023-09-30T14:37:11.994584+0000","last_active":"2023-09-30T14:37:12.977049+0000","last_peered":"2023-09-30T14:37:12.977049+0000","last_clean":"2023-09-30T14:37:12.977049+0000","last_became_active":"2023-09-30T14:37:11.992991+0000","last_became_peered":"2023-09-30T14:37:11.992991+0000","last_unstale":"2023-09-30T14:37:12.977049+0000","last_undegraded":"2023-09-30T14:37:12.977049+0000","last_fullsized":"2023-09-30T14:37:12.977049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:39:50.995717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.977224+0000","last_change":"2023-09-30T14:37:11.020653+0000","last_active":"2023-09-30T14:37:12.977224+0000","last_peered":"2023-09-30T14:37:12.977224+0000","last_clean":"2023-09-30T14:37:12.977224+0000","last_became_active":"2023-09-30T14:37:11.017621+0000","last_became_peered":"2023-09-30T14:37:11.017621+0000","last_unstale":"2023-09-30T14:37:12.977224+0000","last_undegraded":"2023-09-30T14:37:12.977224+0000","last_fullsized":"2023-09-30T14:37:12.977224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:26:59.395278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014548+0000","last_change":"2023-09-30T14:37:11.999881+0000","last_active":"2023-09-30T14:37:13.014548+0000","last_peered":"2023-09-30T14:37:13.014548+0000","last_clean":"2023-09-30T14:37:13.014548+0000","last_became_active":"2023-09-30T14:37:11.999614+0000","last_became_peered":"2023-09-30T14:37:11.999614+0000","last_unstale":"2023-09-30T14:37:13.014548+0000","last_undegraded":"2023-09-30T14:37:13.014548+0000","last_fullsized":"2023-09-30T14:37:13.014548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:59:44.198361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.071987+0000","last_change":"2023-09-30T14:37:11.027187+0000","last_active":"2023-09-30T14:37:16.071987+0000","last_peered":"2023-09-30T14:37:16.071987+0000","last_clean":"2023-09-30T14:37:16.071987+0000","last_became_active":"2023-09-30T14:37:11.026882+0000","last_became_peered":"2023-09-30T14:37:11.026882+0000","last_unstale":"2023-09-30T14:37:16.071987+0000","last_undegraded":"2023-09-30T14:37:16.071987+0000","last_fullsized":"2023-09-30T14:37:16.071987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:17:11.824002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976283+0000","last_change":"2023-09-30T14:37:11.986979+0000","last_active":"2023-09-30T14:37:12.976283+0000","last_peered":"2023-09-30T14:37:12.976283+0000","last_clean":"2023-09-30T14:37:12.976283+0000","last_became_active":"2023-09-30T14:37:11.986705+0000","last_became_peered":"2023-09-30T14:37:11.986705+0000","last_unstale":"2023-09-30T14:37:12.976283+0000","last_undegraded":"2023-09-30T14:37:12.976283+0000","last_fullsized":"2023-09-30T14:37:12.976283+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:31:40.319747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.395336+0000","last_change":"2023-09-30T14:37:11.022313+0000","last_active":"2023-09-30T14:37:12.395336+0000","last_peered":"2023-09-30T14:37:12.395336+0000","last_clean":"2023-09-30T14:37:12.395336+0000","last_became_active":"2023-09-30T14:37:11.021949+0000","last_became_peered":"2023-09-30T14:37:11.021949+0000","last_unstale":"2023-09-30T14:37:12.395336+0000","last_undegraded":"2023-09-30T14:37:12.395336+0000","last_fullsized":"2023-09-30T14:37:12.395336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:30:26.913666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014721+0000","last_change":"2023-09-30T14:37:11.988966+0000","last_active":"2023-09-30T14:37:13.014721+0000","last_peered":"2023-09-30T14:37:13.014721+0000","last_clean":"2023-09-30T14:37:13.014721+0000","last_became_active":"2023-09-30T14:37:11.988716+0000","last_became_peered":"2023-09-30T14:37:11.988716+0000","last_unstale":"2023-09-30T14:37:13.014721+0000","last_undegraded":"2023-09-30T14:37:13.014721+0000","last_fullsized":"2023-09-30T14:37:13.014721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:25:31.437584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.981848+0000","last_change":"2023-09-30T14:37:11.025855+0000","last_active":"2023-09-30T14:37:16.981848+0000","last_peered":"2023-09-30T14:37:16.981848+0000","last_clean":"2023-09-30T14:37:16.981848+0000","last_became_active":"2023-09-30T14:37:11.025402+0000","last_became_peered":"2023-09-30T14:37:11.025402+0000","last_unstale":"2023-09-30T14:37:16.981848+0000","last_undegraded":"2023-09-30T14:37:16.981848+0000","last_fullsized":"2023-09-30T14:37:16.981848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:50:37.692533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130846+0000","last_change":"2023-09-30T14:37:11.993074+0000","last_active":"2023-09-30T14:37:13.130846+0000","last_peered":"2023-09-30T14:37:13.130846+0000","last_clean":"2023-09-30T14:37:13.130846+0000","last_became_active":"2023-09-30T14:37:11.992784+0000","last_became_peered":"2023-09-30T14:37:11.992784+0000","last_unstale":"2023-09-30T14:37:13.130846+0000","last_undegraded":"2023-09-30T14:37:13.130846+0000","last_fullsized":"2023-09-30T14:37:13.130846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:28:18.454389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.051864+0000","last_change":"2023-09-30T14:37:11.022677+0000","last_active":"2023-09-30T14:37:12.051864+0000","last_peered":"2023-09-30T14:37:12.051864+0000","last_clean":"2023-09-30T14:37:12.051864+0000","last_became_active":"2023-09-30T14:37:11.022357+0000","last_became_peered":"2023-09-30T14:37:11.022357+0000","last_unstale":"2023-09-30T14:37:12.051864+0000","last_undegraded":"2023-09-30T14:37:12.051864+0000","last_fullsized":"2023-09-30T14:37:12.051864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:00:15.500968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014764+0000","last_change":"2023-09-30T14:37:11.992358+0000","last_active":"2023-09-30T14:37:13.014764+0000","last_peered":"2023-09-30T14:37:13.014764+0000","last_clean":"2023-09-30T14:37:13.014764+0000","last_became_active":"2023-09-30T14:37:11.992094+0000","last_became_peered":"2023-09-30T14:37:11.992094+0000","last_unstale":"2023-09-30T14:37:13.014764+0000","last_undegraded":"2023-09-30T14:37:13.014764+0000","last_fullsized":"2023-09-30T14:37:13.014764+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:59:36.817138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:11.983426+0000","last_change":"2023-09-30T14:37:11.030961+0000","last_active":"2023-09-30T14:37:11.983426+0000","last_peered":"2023-09-30T14:37:11.983426+0000","last_clean":"2023-09-30T14:37:11.983426+0000","last_became_active":"2023-09-30T14:37:11.030662+0000","last_became_peered":"2023-09-30T14:37:11.030662+0000","last_unstale":"2023-09-30T14:37:11.983426+0000","last_undegraded":"2023-09-30T14:37:11.983426+0000","last_fullsized":"2023-09-30T14:37:11.983426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:35:13.313382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975724+0000","last_change":"2023-09-30T14:37:11.989039+0000","last_active":"2023-09-30T14:37:12.975724+0000","last_peered":"2023-09-30T14:37:12.975724+0000","last_clean":"2023-09-30T14:37:12.975724+0000","last_became_active":"2023-09-30T14:37:11.988594+0000","last_became_peered":"2023-09-30T14:37:11.988594+0000","last_unstale":"2023-09-30T14:37:12.975724+0000","last_undegraded":"2023-09-30T14:37:12.975724+0000","last_fullsized":"2023-09-30T14:37:12.975724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:52:58.146681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014581+0000","last_change":"2023-09-30T14:37:11.026545+0000","last_active":"2023-09-30T14:37:13.014581+0000","last_peered":"2023-09-30T14:37:13.014581+0000","last_clean":"2023-09-30T14:37:13.014581+0000","last_became_active":"2023-09-30T14:37:11.026063+0000","last_became_peered":"2023-09-30T14:37:11.026063+0000","last_unstale":"2023-09-30T14:37:13.014581+0000","last_undegraded":"2023-09-30T14:37:13.014581+0000","last_fullsized":"2023-09-30T14:37:13.014581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:15:34.020336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014356+0000","last_change":"2023-09-30T14:37:11.991845+0000","last_active":"2023-09-30T14:37:13.014356+0000","last_peered":"2023-09-30T14:37:13.014356+0000","last_clean":"2023-09-30T14:37:13.014356+0000","last_became_active":"2023-09-30T14:37:11.991560+0000","last_became_peered":"2023-09-30T14:37:11.991560+0000","last_unstale":"2023-09-30T14:37:13.014356+0000","last_undegraded":"2023-09-30T14:37:13.014356+0000","last_fullsized":"2023-09-30T14:37:13.014356+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:27:39.653882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063853+0000","last_change":"2023-09-30T14:37:11.028192+0000","last_active":"2023-09-30T14:37:15.063853+0000","last_peered":"2023-09-30T14:37:15.063853+0000","last_clean":"2023-09-30T14:37:15.063853+0000","last_became_active":"2023-09-30T14:37:11.027902+0000","last_became_peered":"2023-09-30T14:37:11.027902+0000","last_unstale":"2023-09-30T14:37:15.063853+0000","last_undegraded":"2023-09-30T14:37:15.063853+0000","last_fullsized":"2023-09-30T14:37:15.063853+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:21:30.267559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976420+0000","last_change":"2023-09-30T14:37:11.997318+0000","last_active":"2023-09-30T14:37:12.976420+0000","last_peered":"2023-09-30T14:37:12.976420+0000","last_clean":"2023-09-30T14:37:12.976420+0000","last_became_active":"2023-09-30T14:37:11.996988+0000","last_became_peered":"2023-09-30T14:37:11.996988+0000","last_unstale":"2023-09-30T14:37:12.976420+0000","last_undegraded":"2023-09-30T14:37:12.976420+0000","last_fullsized":"2023-09-30T14:37:12.976420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:57:21.510952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063345+0000","last_change":"2023-09-30T14:37:11.023686+0000","last_active":"2023-09-30T14:37:15.063345+0000","last_peered":"2023-09-30T14:37:15.063345+0000","last_clean":"2023-09-30T14:37:15.063345+0000","last_became_active":"2023-09-30T14:37:11.023365+0000","last_became_peered":"2023-09-30T14:37:11.023365+0000","last_unstale":"2023-09-30T14:37:15.063345+0000","last_undegraded":"2023-09-30T14:37:15.063345+0000","last_fullsized":"2023-09-30T14:37:15.063345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:00:04.358289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014711+0000","last_change":"2023-09-30T14:37:11.989338+0000","last_active":"2023-09-30T14:37:13.014711+0000","last_peered":"2023-09-30T14:37:13.014711+0000","last_clean":"2023-09-30T14:37:13.014711+0000","last_became_active":"2023-09-30T14:37:11.989045+0000","last_became_peered":"2023-09-30T14:37:11.989045+0000","last_unstale":"2023-09-30T14:37:13.014711+0000","last_undegraded":"2023-09-30T14:37:13.014711+0000","last_fullsized":"2023-09-30T14:37:13.014711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:58:59.359684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063525+0000","last_change":"2023-09-30T14:37:11.024709+0000","last_active":"2023-09-30T14:37:15.063525+0000","last_peered":"2023-09-30T14:37:15.063525+0000","last_clean":"2023-09-30T14:37:15.063525+0000","last_became_active":"2023-09-30T14:37:11.024344+0000","last_became_peered":"2023-09-30T14:37:11.024344+0000","last_unstale":"2023-09-30T14:37:15.063525+0000","last_undegraded":"2023-09-30T14:37:15.063525+0000","last_fullsized":"2023-09-30T14:37:15.063525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:17:49.508396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976335+0000","last_change":"2023-09-30T14:37:11.989797+0000","last_active":"2023-09-30T14:37:12.976335+0000","last_peered":"2023-09-30T14:37:12.976335+0000","last_clean":"2023-09-30T14:37:12.976335+0000","last_became_active":"2023-09-30T14:37:11.989482+0000","last_became_peered":"2023-09-30T14:37:11.989482+0000","last_unstale":"2023-09-30T14:37:12.976335+0000","last_undegraded":"2023-09-30T14:37:12.976335+0000","last_fullsized":"2023-09-30T14:37:12.976335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:33:20.437706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.062892+0000","last_change":"2023-09-30T14:37:11.027322+0000","last_active":"2023-09-30T14:37:15.062892+0000","last_peered":"2023-09-30T14:37:15.062892+0000","last_clean":"2023-09-30T14:37:15.062892+0000","last_became_active":"2023-09-30T14:37:11.027007+0000","last_became_peered":"2023-09-30T14:37:11.027007+0000","last_unstale":"2023-09-30T14:37:15.062892+0000","last_undegraded":"2023-09-30T14:37:15.062892+0000","last_fullsized":"2023-09-30T14:37:15.062892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:47:22.919317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130730+0000","last_change":"2023-09-30T14:37:11.988469+0000","last_active":"2023-09-30T14:37:13.130730+0000","last_peered":"2023-09-30T14:37:13.130730+0000","last_clean":"2023-09-30T14:37:13.130730+0000","last_became_active":"2023-09-30T14:37:11.988118+0000","last_became_peered":"2023-09-30T14:37:11.988118+0000","last_unstale":"2023-09-30T14:37:13.130730+0000","last_undegraded":"2023-09-30T14:37:13.130730+0000","last_fullsized":"2023-09-30T14:37:13.130730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:11:56.750690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975788+0000","last_change":"2023-09-30T14:37:11.024345+0000","last_active":"2023-09-30T14:37:12.975788+0000","last_peered":"2023-09-30T14:37:12.975788+0000","last_clean":"2023-09-30T14:37:12.975788+0000","last_became_active":"2023-09-30T14:37:11.024039+0000","last_became_peered":"2023-09-30T14:37:11.024039+0000","last_unstale":"2023-09-30T14:37:12.975788+0000","last_undegraded":"2023-09-30T14:37:12.975788+0000","last_fullsized":"2023-09-30T14:37:12.975788+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:48:32.872814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014748+0000","last_change":"2023-09-30T14:37:11.994068+0000","last_active":"2023-09-30T14:37:13.014748+0000","last_peered":"2023-09-30T14:37:13.014748+0000","last_clean":"2023-09-30T14:37:13.014748+0000","last_became_active":"2023-09-30T14:37:11.993793+0000","last_became_peered":"2023-09-30T14:37:11.993793+0000","last_unstale":"2023-09-30T14:37:13.014748+0000","last_undegraded":"2023-09-30T14:37:13.014748+0000","last_fullsized":"2023-09-30T14:37:13.014748+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:51:28.071414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063744+0000","last_change":"2023-09-30T14:37:11.025622+0000","last_active":"2023-09-30T14:37:15.063744+0000","last_peered":"2023-09-30T14:37:15.063744+0000","last_clean":"2023-09-30T14:37:15.063744+0000","last_became_active":"2023-09-30T14:37:11.025338+0000","last_became_peered":"2023-09-30T14:37:11.025338+0000","last_unstale":"2023-09-30T14:37:15.063744+0000","last_undegraded":"2023-09-30T14:37:15.063744+0000","last_fullsized":"2023-09-30T14:37:15.063744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:32:03.166296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.015550+0000","last_change":"2023-09-30T14:37:11.990678+0000","last_active":"2023-09-30T14:37:13.015550+0000","last_peered":"2023-09-30T14:37:13.015550+0000","last_clean":"2023-09-30T14:37:13.015550+0000","last_became_active":"2023-09-30T14:37:11.990331+0000","last_became_peered":"2023-09-30T14:37:11.990331+0000","last_unstale":"2023-09-30T14:37:13.015550+0000","last_undegraded":"2023-09-30T14:37:13.015550+0000","last_fullsized":"2023-09-30T14:37:13.015550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:57:06.296415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:11.974378+0000","last_change":"2023-09-30T14:37:11.029501+0000","last_active":"2023-09-30T14:37:11.974378+0000","last_peered":"2023-09-30T14:37:11.974378+0000","last_clean":"2023-09-30T14:37:11.974378+0000","last_became_active":"2023-09-30T14:37:11.029207+0000","last_became_peered":"2023-09-30T14:37:11.029207+0000","last_unstale":"2023-09-30T14:37:11.974378+0000","last_undegraded":"2023-09-30T14:37:11.974378+0000","last_fullsized":"2023-09-30T14:37:11.974378+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:48:53.145952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014366+0000","last_change":"2023-09-30T14:37:11.993052+0000","last_active":"2023-09-30T14:37:13.014366+0000","last_peered":"2023-09-30T14:37:13.014366+0000","last_clean":"2023-09-30T14:37:13.014366+0000","last_became_active":"2023-09-30T14:37:11.992762+0000","last_became_peered":"2023-09-30T14:37:11.992762+0000","last_unstale":"2023-09-30T14:37:13.014366+0000","last_undegraded":"2023-09-30T14:37:13.014366+0000","last_fullsized":"2023-09-30T14:37:13.014366+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:09:16.440026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.075318+0000","last_change":"2023-09-30T14:37:11.027266+0000","last_active":"2023-09-30T14:37:16.075318+0000","last_peered":"2023-09-30T14:37:16.075318+0000","last_clean":"2023-09-30T14:37:16.075318+0000","last_became_active":"2023-09-30T14:37:11.026971+0000","last_became_peered":"2023-09-30T14:37:11.026971+0000","last_unstale":"2023-09-30T14:37:16.075318+0000","last_undegraded":"2023-09-30T14:37:16.075318+0000","last_fullsized":"2023-09-30T14:37:16.075318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:38:09.543391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014408+0000","last_change":"2023-09-30T14:37:11.989592+0000","last_active":"2023-09-30T14:37:13.014408+0000","last_peered":"2023-09-30T14:37:13.014408+0000","last_clean":"2023-09-30T14:37:13.014408+0000","last_became_active":"2023-09-30T14:37:11.989290+0000","last_became_peered":"2023-09-30T14:37:11.989290+0000","last_unstale":"2023-09-30T14:37:13.014408+0000","last_undegraded":"2023-09-30T14:37:13.014408+0000","last_fullsized":"2023-09-30T14:37:13.014408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:11:33.974823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130849+0000","last_change":"2023-09-30T14:37:11.028919+0000","last_active":"2023-09-30T14:37:13.130849+0000","last_peered":"2023-09-30T14:37:13.130849+0000","last_clean":"2023-09-30T14:37:13.130849+0000","last_became_active":"2023-09-30T14:37:11.028532+0000","last_became_peered":"2023-09-30T14:37:11.028532+0000","last_unstale":"2023-09-30T14:37:13.130849+0000","last_undegraded":"2023-09-30T14:37:13.130849+0000","last_fullsized":"2023-09-30T14:37:13.130849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:15:42.675409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014483+0000","last_change":"2023-09-30T14:37:11.980611+0000","last_active":"2023-09-30T14:37:13.014483+0000","last_peered":"2023-09-30T14:37:13.014483+0000","last_clean":"2023-09-30T14:37:13.014483+0000","last_became_active":"2023-09-30T14:37:11.980288+0000","last_became_peered":"2023-09-30T14:37:11.980288+0000","last_unstale":"2023-09-30T14:37:13.014483+0000","last_undegraded":"2023-09-30T14:37:13.014483+0000","last_fullsized":"2023-09-30T14:37:13.014483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:38:49.270976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014852+0000","last_change":"2023-09-30T14:37:11.025133+0000","last_active":"2023-09-30T14:37:13.014852+0000","last_peered":"2023-09-30T14:37:13.014852+0000","last_clean":"2023-09-30T14:37:13.014852+0000","last_became_active":"2023-09-30T14:37:11.024914+0000","last_became_peered":"2023-09-30T14:37:11.024914+0000","last_unstale":"2023-09-30T14:37:13.014852+0000","last_undegraded":"2023-09-30T14:37:13.014852+0000","last_fullsized":"2023-09-30T14:37:13.014852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:23:39.328773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.015965+0000","last_change":"2023-09-30T14:37:11.988434+0000","last_active":"2023-09-30T14:37:13.015965+0000","last_peered":"2023-09-30T14:37:13.015965+0000","last_clean":"2023-09-30T14:37:13.015965+0000","last_became_active":"2023-09-30T14:37:11.988156+0000","last_became_peered":"2023-09-30T14:37:11.988156+0000","last_unstale":"2023-09-30T14:37:13.015965+0000","last_undegraded":"2023-09-30T14:37:13.015965+0000","last_fullsized":"2023-09-30T14:37:13.015965+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:20:49.420026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:11.978658+0000","last_change":"2023-09-30T14:37:11.026290+0000","last_active":"2023-09-30T14:37:11.978658+0000","last_peered":"2023-09-30T14:37:11.978658+0000","last_clean":"2023-09-30T14:37:11.978658+0000","last_became_active":"2023-09-30T14:37:11.024684+0000","last_became_peered":"2023-09-30T14:37:11.024684+0000","last_unstale":"2023-09-30T14:37:11.978658+0000","last_undegraded":"2023-09-30T14:37:11.978658+0000","last_fullsized":"2023-09-30T14:37:11.978658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:43:06.463243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.016048+0000","last_change":"2023-09-30T14:37:11.987989+0000","last_active":"2023-09-30T14:37:13.016048+0000","last_peered":"2023-09-30T14:37:13.016048+0000","last_clean":"2023-09-30T14:37:13.016048+0000","last_became_active":"2023-09-30T14:37:11.987690+0000","last_became_peered":"2023-09-30T14:37:11.987690+0000","last_unstale":"2023-09-30T14:37:13.016048+0000","last_undegraded":"2023-09-30T14:37:13.016048+0000","last_fullsized":"2023-09-30T14:37:13.016048+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:51:25.825871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014864+0000","last_change":"2023-09-30T14:37:11.996852+0000","last_active":"2023-09-30T14:37:13.014864+0000","last_peered":"2023-09-30T14:37:13.014864+0000","last_clean":"2023-09-30T14:37:13.014864+0000","last_became_active":"2023-09-30T14:37:11.996596+0000","last_became_peered":"2023-09-30T14:37:11.996596+0000","last_unstale":"2023-09-30T14:37:13.014864+0000","last_undegraded":"2023-09-30T14:37:13.014864+0000","last_fullsized":"2023-09-30T14:37:13.014864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:25:04.531236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014217+0000","last_change":"2023-09-30T14:37:11.025458+0000","last_active":"2023-09-30T14:37:13.014217+0000","last_peered":"2023-09-30T14:37:13.014217+0000","last_clean":"2023-09-30T14:37:13.014217+0000","last_became_active":"2023-09-30T14:37:11.025157+0000","last_became_peered":"2023-09-30T14:37:11.025157+0000","last_unstale":"2023-09-30T14:37:13.014217+0000","last_undegraded":"2023-09-30T14:37:13.014217+0000","last_fullsized":"2023-09-30T14:37:13.014217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:31:24.436125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976151+0000","last_change":"2023-09-30T14:37:11.986840+0000","last_active":"2023-09-30T14:37:12.976151+0000","last_peered":"2023-09-30T14:37:12.976151+0000","last_clean":"2023-09-30T14:37:12.976151+0000","last_became_active":"2023-09-30T14:37:11.986483+0000","last_became_peered":"2023-09-30T14:37:11.986483+0000","last_unstale":"2023-09-30T14:37:12.976151+0000","last_undegraded":"2023-09-30T14:37:12.976151+0000","last_fullsized":"2023-09-30T14:37:12.976151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:14:32.399685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:11.974397+0000","last_change":"2023-09-30T14:37:11.025706+0000","last_active":"2023-09-30T14:37:11.974397+0000","last_peered":"2023-09-30T14:37:11.974397+0000","last_clean":"2023-09-30T14:37:11.974397+0000","last_became_active":"2023-09-30T14:37:11.025376+0000","last_became_peered":"2023-09-30T14:37:11.025376+0000","last_unstale":"2023-09-30T14:37:11.974397+0000","last_undegraded":"2023-09-30T14:37:11.974397+0000","last_fullsized":"2023-09-30T14:37:11.974397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:10:36.025273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":308,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:11.977686+0000","last_change":"2023-09-30T14:36:12.344762+0000","last_active":"2023-09-30T14:37:11.977686+0000","last_peered":"2023-09-30T14:37:11.977686+0000","last_clean":"2023-09-30T14:37:11.977686+0000","last_became_active":"2023-09-30T14:36:12.344268+0000","last_became_peered":"2023-09-30T14:36:12.344268+0000","last_unstale":"2023-09-30T14:37:11.977686+0000","last_undegraded":"2023-09-30T14:37:11.977686+0000","last_fullsized":"2023-09-30T14:37:11.977686+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:33:20.434992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:33:20.434992+0000","last_clean_scrub_stamp":"2023-09-30T14:33:20.434992+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:00:05.598961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014131+0000","last_change":"2023-09-30T14:37:11.996886+0000","last_active":"2023-09-30T14:37:13.014131+0000","last_peered":"2023-09-30T14:37:13.014131+0000","last_clean":"2023-09-30T14:37:13.014131+0000","last_became_active":"2023-09-30T14:37:11.996667+0000","last_became_peered":"2023-09-30T14:37:11.996667+0000","last_unstale":"2023-09-30T14:37:13.014131+0000","last_undegraded":"2023-09-30T14:37:13.014131+0000","last_fullsized":"2023-09-30T14:37:13.014131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:27:00.546356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.013888+0000","last_change":"2023-09-30T14:37:11.027012+0000","last_active":"2023-09-30T14:37:13.013888+0000","last_peered":"2023-09-30T14:37:13.013888+0000","last_clean":"2023-09-30T14:37:13.013888+0000","last_became_active":"2023-09-30T14:37:11.026714+0000","last_became_peered":"2023-09-30T14:37:11.026714+0000","last_unstale":"2023-09-30T14:37:13.013888+0000","last_undegraded":"2023-09-30T14:37:13.013888+0000","last_fullsized":"2023-09-30T14:37:13.013888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:17:57.860438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.062444+0000","last_change":"2023-09-30T14:37:11.027389+0000","last_active":"2023-09-30T14:37:15.062444+0000","last_peered":"2023-09-30T14:37:15.062444+0000","last_clean":"2023-09-30T14:37:15.062444+0000","last_became_active":"2023-09-30T14:37:11.027011+0000","last_became_peered":"2023-09-30T14:37:11.027011+0000","last_unstale":"2023-09-30T14:37:15.062444+0000","last_undegraded":"2023-09-30T14:37:15.062444+0000","last_fullsized":"2023-09-30T14:37:15.062444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:38:22.953936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014762+0000","last_change":"2023-09-30T14:37:11.997675+0000","last_active":"2023-09-30T14:37:13.014762+0000","last_peered":"2023-09-30T14:37:13.014762+0000","last_clean":"2023-09-30T14:37:13.014762+0000","last_became_active":"2023-09-30T14:37:11.997330+0000","last_became_peered":"2023-09-30T14:37:11.997330+0000","last_unstale":"2023-09-30T14:37:13.014762+0000","last_undegraded":"2023-09-30T14:37:13.014762+0000","last_fullsized":"2023-09-30T14:37:13.014762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:54:52.802138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014285+0000","last_change":"2023-09-30T14:37:11.023836+0000","last_active":"2023-09-30T14:37:13.014285+0000","last_peered":"2023-09-30T14:37:13.014285+0000","last_clean":"2023-09-30T14:37:13.014285+0000","last_became_active":"2023-09-30T14:37:11.023362+0000","last_became_peered":"2023-09-30T14:37:11.023362+0000","last_unstale":"2023-09-30T14:37:13.014285+0000","last_undegraded":"2023-09-30T14:37:13.014285+0000","last_fullsized":"2023-09-30T14:37:13.014285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:58:22.772546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976655+0000","last_change":"2023-09-30T14:37:11.985281+0000","last_active":"2023-09-30T14:37:12.976655+0000","last_peered":"2023-09-30T14:37:12.976655+0000","last_clean":"2023-09-30T14:37:12.976655+0000","last_became_active":"2023-09-30T14:37:11.984944+0000","last_became_peered":"2023-09-30T14:37:11.984944+0000","last_unstale":"2023-09-30T14:37:12.976655+0000","last_undegraded":"2023-09-30T14:37:12.976655+0000","last_fullsized":"2023-09-30T14:37:12.976655+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:30:19.829727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.051096+0000","last_change":"2023-09-30T14:37:11.028070+0000","last_active":"2023-09-30T14:37:12.051096+0000","last_peered":"2023-09-30T14:37:12.051096+0000","last_clean":"2023-09-30T14:37:12.051096+0000","last_became_active":"2023-09-30T14:37:11.027703+0000","last_became_peered":"2023-09-30T14:37:11.027703+0000","last_unstale":"2023-09-30T14:37:12.051096+0000","last_undegraded":"2023-09-30T14:37:12.051096+0000","last_fullsized":"2023-09-30T14:37:12.051096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:23:15.071946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014359+0000","last_change":"2023-09-30T14:37:11.989897+0000","last_active":"2023-09-30T14:37:13.014359+0000","last_peered":"2023-09-30T14:37:13.014359+0000","last_clean":"2023-09-30T14:37:13.014359+0000","last_became_active":"2023-09-30T14:37:11.989613+0000","last_became_peered":"2023-09-30T14:37:11.989613+0000","last_unstale":"2023-09-30T14:37:13.014359+0000","last_undegraded":"2023-09-30T14:37:13.014359+0000","last_fullsized":"2023-09-30T14:37:13.014359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:43:17.887766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975954+0000","last_change":"2023-09-30T14:37:11.026491+0000","last_active":"2023-09-30T14:37:12.975954+0000","last_peered":"2023-09-30T14:37:12.975954+0000","last_clean":"2023-09-30T14:37:12.975954+0000","last_became_active":"2023-09-30T14:37:11.026196+0000","last_became_peered":"2023-09-30T14:37:11.026196+0000","last_unstale":"2023-09-30T14:37:12.975954+0000","last_undegraded":"2023-09-30T14:37:12.975954+0000","last_fullsized":"2023-09-30T14:37:12.975954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:17:22.343377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130535+0000","last_change":"2023-09-30T14:37:11.988570+0000","last_active":"2023-09-30T14:37:13.130535+0000","last_peered":"2023-09-30T14:37:13.130535+0000","last_clean":"2023-09-30T14:37:13.130535+0000","last_became_active":"2023-09-30T14:37:11.988230+0000","last_became_peered":"2023-09-30T14:37:11.988230+0000","last_unstale":"2023-09-30T14:37:13.130535+0000","last_undegraded":"2023-09-30T14:37:13.130535+0000","last_fullsized":"2023-09-30T14:37:13.130535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:58:38.302343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.062480+0000","last_change":"2023-09-30T14:37:11.024587+0000","last_active":"2023-09-30T14:37:15.062480+0000","last_peered":"2023-09-30T14:37:15.062480+0000","last_clean":"2023-09-30T14:37:15.062480+0000","last_became_active":"2023-09-30T14:37:11.024253+0000","last_became_peered":"2023-09-30T14:37:11.024253+0000","last_unstale":"2023-09-30T14:37:15.062480+0000","last_undegraded":"2023-09-30T14:37:15.062480+0000","last_fullsized":"2023-09-30T14:37:15.062480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:31:07.293657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014239+0000","last_change":"2023-09-30T14:37:11.987818+0000","last_active":"2023-09-30T14:37:13.014239+0000","last_peered":"2023-09-30T14:37:13.014239+0000","last_clean":"2023-09-30T14:37:13.014239+0000","last_became_active":"2023-09-30T14:37:11.987476+0000","last_became_peered":"2023-09-30T14:37:11.987476+0000","last_unstale":"2023-09-30T14:37:13.014239+0000","last_undegraded":"2023-09-30T14:37:13.014239+0000","last_fullsized":"2023-09-30T14:37:13.014239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:01:07.802959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975601+0000","last_change":"2023-09-30T14:37:11.023223+0000","last_active":"2023-09-30T14:37:12.975601+0000","last_peered":"2023-09-30T14:37:12.975601+0000","last_clean":"2023-09-30T14:37:12.975601+0000","last_became_active":"2023-09-30T14:37:11.022932+0000","last_became_peered":"2023-09-30T14:37:11.022932+0000","last_unstale":"2023-09-30T14:37:12.975601+0000","last_undegraded":"2023-09-30T14:37:12.975601+0000","last_fullsized":"2023-09-30T14:37:12.975601+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:01:04.315669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976068+0000","last_change":"2023-09-30T14:37:11.990323+0000","last_active":"2023-09-30T14:37:12.976068+0000","last_peered":"2023-09-30T14:37:12.976068+0000","last_clean":"2023-09-30T14:37:12.976068+0000","last_became_active":"2023-09-30T14:37:11.989981+0000","last_became_peered":"2023-09-30T14:37:11.989981+0000","last_unstale":"2023-09-30T14:37:12.976068+0000","last_undegraded":"2023-09-30T14:37:12.976068+0000","last_fullsized":"2023-09-30T14:37:12.976068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:30:44.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063371+0000","last_change":"2023-09-30T14:37:11.025658+0000","last_active":"2023-09-30T14:37:15.063371+0000","last_peered":"2023-09-30T14:37:15.063371+0000","last_clean":"2023-09-30T14:37:15.063371+0000","last_became_active":"2023-09-30T14:37:11.025334+0000","last_became_peered":"2023-09-30T14:37:11.025334+0000","last_unstale":"2023-09-30T14:37:15.063371+0000","last_undegraded":"2023-09-30T14:37:15.063371+0000","last_fullsized":"2023-09-30T14:37:15.063371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:23:41.933634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014128+0000","last_change":"2023-09-30T14:37:11.989989+0000","last_active":"2023-09-30T14:37:13.014128+0000","last_peered":"2023-09-30T14:37:13.014128+0000","last_clean":"2023-09-30T14:37:13.014128+0000","last_became_active":"2023-09-30T14:37:11.989666+0000","last_became_peered":"2023-09-30T14:37:11.989666+0000","last_unstale":"2023-09-30T14:37:13.014128+0000","last_undegraded":"2023-09-30T14:37:13.014128+0000","last_fullsized":"2023-09-30T14:37:13.014128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:41:09.835210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975960+0000","last_change":"2023-09-30T14:37:11.025027+0000","last_active":"2023-09-30T14:37:12.975960+0000","last_peered":"2023-09-30T14:37:12.975960+0000","last_clean":"2023-09-30T14:37:12.975960+0000","last_became_active":"2023-09-30T14:37:11.024696+0000","last_became_peered":"2023-09-30T14:37:11.024696+0000","last_unstale":"2023-09-30T14:37:12.975960+0000","last_undegraded":"2023-09-30T14:37:12.975960+0000","last_fullsized":"2023-09-30T14:37:12.975960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:34:28.384535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014390+0000","last_change":"2023-09-30T14:37:11.992674+0000","last_active":"2023-09-30T14:37:13.014390+0000","last_peered":"2023-09-30T14:37:13.014390+0000","last_clean":"2023-09-30T14:37:13.014390+0000","last_became_active":"2023-09-30T14:37:11.992243+0000","last_became_peered":"2023-09-30T14:37:11.992243+0000","last_unstale":"2023-09-30T14:37:13.014390+0000","last_undegraded":"2023-09-30T14:37:13.014390+0000","last_fullsized":"2023-09-30T14:37:13.014390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:33:19.759637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014643+0000","last_change":"2023-09-30T14:37:11.024525+0000","last_active":"2023-09-30T14:37:13.014643+0000","last_peered":"2023-09-30T14:37:13.014643+0000","last_clean":"2023-09-30T14:37:13.014643+0000","last_became_active":"2023-09-30T14:37:11.024231+0000","last_became_peered":"2023-09-30T14:37:11.024231+0000","last_unstale":"2023-09-30T14:37:13.014643+0000","last_undegraded":"2023-09-30T14:37:13.014643+0000","last_fullsized":"2023-09-30T14:37:13.014643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:53:47.142798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014167+0000","last_change":"2023-09-30T14:37:11.991401+0000","last_active":"2023-09-30T14:37:13.014167+0000","last_peered":"2023-09-30T14:37:13.014167+0000","last_clean":"2023-09-30T14:37:13.014167+0000","last_became_active":"2023-09-30T14:37:11.991012+0000","last_became_peered":"2023-09-30T14:37:11.991012+0000","last_unstale":"2023-09-30T14:37:13.014167+0000","last_undegraded":"2023-09-30T14:37:13.014167+0000","last_fullsized":"2023-09-30T14:37:13.014167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:55:00.562030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.050302+0000","last_change":"2023-09-30T14:37:11.022893+0000","last_active":"2023-09-30T14:37:12.050302+0000","last_peered":"2023-09-30T14:37:12.050302+0000","last_clean":"2023-09-30T14:37:12.050302+0000","last_became_active":"2023-09-30T14:37:11.022505+0000","last_became_peered":"2023-09-30T14:37:11.022505+0000","last_unstale":"2023-09-30T14:37:12.050302+0000","last_undegraded":"2023-09-30T14:37:12.050302+0000","last_fullsized":"2023-09-30T14:37:12.050302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:00:17.721645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014348+0000","last_change":"2023-09-30T14:37:11.992942+0000","last_active":"2023-09-30T14:37:13.014348+0000","last_peered":"2023-09-30T14:37:13.014348+0000","last_clean":"2023-09-30T14:37:13.014348+0000","last_became_active":"2023-09-30T14:37:11.992698+0000","last_became_peered":"2023-09-30T14:37:11.992698+0000","last_unstale":"2023-09-30T14:37:13.014348+0000","last_undegraded":"2023-09-30T14:37:13.014348+0000","last_fullsized":"2023-09-30T14:37:13.014348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:15:30.239423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014256+0000","last_change":"2023-09-30T14:37:11.984435+0000","last_active":"2023-09-30T14:37:13.014256+0000","last_peered":"2023-09-30T14:37:13.014256+0000","last_clean":"2023-09-30T14:37:13.014256+0000","last_became_active":"2023-09-30T14:37:11.984137+0000","last_became_peered":"2023-09-30T14:37:11.984137+0000","last_unstale":"2023-09-30T14:37:13.014256+0000","last_undegraded":"2023-09-30T14:37:13.014256+0000","last_fullsized":"2023-09-30T14:37:13.014256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:40:41.112326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975815+0000","last_change":"2023-09-30T14:37:11.984481+0000","last_active":"2023-09-30T14:37:12.975815+0000","last_peered":"2023-09-30T14:37:12.975815+0000","last_clean":"2023-09-30T14:37:12.975815+0000","last_became_active":"2023-09-30T14:37:11.984021+0000","last_became_peered":"2023-09-30T14:37:11.984021+0000","last_unstale":"2023-09-30T14:37:12.975815+0000","last_undegraded":"2023-09-30T14:37:12.975815+0000","last_fullsized":"2023-09-30T14:37:12.975815+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:36:33.184379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975777+0000","last_change":"2023-09-30T14:37:11.990779+0000","last_active":"2023-09-30T14:37:12.975777+0000","last_peered":"2023-09-30T14:37:12.975777+0000","last_clean":"2023-09-30T14:37:12.975777+0000","last_became_active":"2023-09-30T14:37:11.989960+0000","last_became_peered":"2023-09-30T14:37:11.989960+0000","last_unstale":"2023-09-30T14:37:12.975777+0000","last_undegraded":"2023-09-30T14:37:12.975777+0000","last_fullsized":"2023-09-30T14:37:12.975777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:54:36.813191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.015049+0000","last_change":"2023-09-30T14:37:11.995750+0000","last_active":"2023-09-30T14:37:13.015049+0000","last_peered":"2023-09-30T14:37:13.015049+0000","last_clean":"2023-09-30T14:37:13.015049+0000","last_became_active":"2023-09-30T14:37:11.995434+0000","last_became_peered":"2023-09-30T14:37:11.995434+0000","last_unstale":"2023-09-30T14:37:13.015049+0000","last_undegraded":"2023-09-30T14:37:13.015049+0000","last_fullsized":"2023-09-30T14:37:13.015049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:49:51.234024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014630+0000","last_change":"2023-09-30T14:37:11.996550+0000","last_active":"2023-09-30T14:37:13.014630+0000","last_peered":"2023-09-30T14:37:13.014630+0000","last_clean":"2023-09-30T14:37:13.014630+0000","last_became_active":"2023-09-30T14:37:11.996269+0000","last_became_peered":"2023-09-30T14:37:11.996269+0000","last_unstale":"2023-09-30T14:37:13.014630+0000","last_undegraded":"2023-09-30T14:37:13.014630+0000","last_fullsized":"2023-09-30T14:37:13.014630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:40:16.537270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014992+0000","last_change":"2023-09-30T14:37:11.983495+0000","last_active":"2023-09-30T14:37:13.014992+0000","last_peered":"2023-09-30T14:37:13.014992+0000","last_clean":"2023-09-30T14:37:13.014992+0000","last_became_active":"2023-09-30T14:37:11.983241+0000","last_became_peered":"2023-09-30T14:37:11.983241+0000","last_unstale":"2023-09-30T14:37:13.014992+0000","last_undegraded":"2023-09-30T14:37:13.014992+0000","last_fullsized":"2023-09-30T14:37:13.014992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:41:46.604467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975888+0000","last_change":"2023-09-30T14:37:11.996058+0000","last_active":"2023-09-30T14:37:12.975888+0000","last_peered":"2023-09-30T14:37:12.975888+0000","last_clean":"2023-09-30T14:37:12.975888+0000","last_became_active":"2023-09-30T14:37:11.995763+0000","last_became_peered":"2023-09-30T14:37:11.995763+0000","last_unstale":"2023-09-30T14:37:12.975888+0000","last_undegraded":"2023-09-30T14:37:12.975888+0000","last_fullsized":"2023-09-30T14:37:12.975888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:42:21.306561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976498+0000","last_change":"2023-09-30T14:37:11.992080+0000","last_active":"2023-09-30T14:37:12.976498+0000","last_peered":"2023-09-30T14:37:12.976498+0000","last_clean":"2023-09-30T14:37:12.976498+0000","last_became_active":"2023-09-30T14:37:11.991747+0000","last_became_peered":"2023-09-30T14:37:11.991747+0000","last_unstale":"2023-09-30T14:37:12.976498+0000","last_undegraded":"2023-09-30T14:37:12.976498+0000","last_fullsized":"2023-09-30T14:37:12.976498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:07:25.985911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014724+0000","last_change":"2023-09-30T14:37:11.997109+0000","last_active":"2023-09-30T14:37:13.014724+0000","last_peered":"2023-09-30T14:37:13.014724+0000","last_clean":"2023-09-30T14:37:13.014724+0000","last_became_active":"2023-09-30T14:37:11.996793+0000","last_became_peered":"2023-09-30T14:37:11.996793+0000","last_unstale":"2023-09-30T14:37:13.014724+0000","last_undegraded":"2023-09-30T14:37:13.014724+0000","last_fullsized":"2023-09-30T14:37:13.014724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:13:59.838638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975786+0000","last_change":"2023-09-30T14:37:11.984083+0000","last_active":"2023-09-30T14:37:12.975786+0000","last_peered":"2023-09-30T14:37:12.975786+0000","last_clean":"2023-09-30T14:37:12.975786+0000","last_became_active":"2023-09-30T14:37:11.983352+0000","last_became_peered":"2023-09-30T14:37:11.983352+0000","last_unstale":"2023-09-30T14:37:12.975786+0000","last_undegraded":"2023-09-30T14:37:12.975786+0000","last_fullsized":"2023-09-30T14:37:12.975786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:02:03.668484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976194+0000","last_change":"2023-09-30T14:37:11.992212+0000","last_active":"2023-09-30T14:37:12.976194+0000","last_peered":"2023-09-30T14:37:12.976194+0000","last_clean":"2023-09-30T14:37:12.976194+0000","last_became_active":"2023-09-30T14:37:11.990158+0000","last_became_peered":"2023-09-30T14:37:11.990158+0000","last_unstale":"2023-09-30T14:37:12.976194+0000","last_undegraded":"2023-09-30T14:37:12.976194+0000","last_fullsized":"2023-09-30T14:37:12.976194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:14:27.976110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014402+0000","last_change":"2023-09-30T14:37:11.995884+0000","last_active":"2023-09-30T14:37:13.014402+0000","last_peered":"2023-09-30T14:37:13.014402+0000","last_clean":"2023-09-30T14:37:13.014402+0000","last_became_active":"2023-09-30T14:37:11.995551+0000","last_became_peered":"2023-09-30T14:37:11.995551+0000","last_unstale":"2023-09-30T14:37:13.014402+0000","last_undegraded":"2023-09-30T14:37:13.014402+0000","last_fullsized":"2023-09-30T14:37:13.014402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:28:26.051188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014437+0000","last_change":"2023-09-30T14:37:11.985853+0000","last_active":"2023-09-30T14:37:13.014437+0000","last_peered":"2023-09-30T14:37:13.014437+0000","last_clean":"2023-09-30T14:37:13.014437+0000","last_became_active":"2023-09-30T14:37:11.985525+0000","last_became_peered":"2023-09-30T14:37:11.985525+0000","last_unstale":"2023-09-30T14:37:13.014437+0000","last_undegraded":"2023-09-30T14:37:13.014437+0000","last_fullsized":"2023-09-30T14:37:13.014437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:07:24.392538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3928,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":10222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449099,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":425000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]}]},{"osd":10,"up_from":71,"seq":304942678031,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1503232,"data_stored":1016843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":1,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.442},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.47299999999999998}]},{"osd":2,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.61099999999999999}]},{"osd":3,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.72599999999999998}]},{"osd":4,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.50800000000000001}]},{"osd":5,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.68100000000000005}]},{"osd":6,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.41699999999999998}]},{"osd":8,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.745},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.79300000000000004}]},{"osd":9,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.872}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":9,"up_from":66,"seq":283467841554,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":425251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.378},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.47899999999999998}]},{"osd":1,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.54600000000000004}]},{"osd":2,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.45900000000000002}]},{"osd":3,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51500000000000001}]},{"osd":4,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.497},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.57499999999999996}]},{"osd":5,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.626}]},{"osd":6,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.56599999999999995}]},{"osd":7,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.73399999999999999}]},{"osd":8,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.59799999999999998}]},{"osd":10,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.64200000000000002}]}]},{"osd":8,"up_from":58,"seq":249108103192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":425738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.48699999999999999}]},{"osd":1,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.61599999999999999}]},{"osd":2,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.442},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.41799999999999998}]},{"osd":3,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59399999999999997}]},{"osd":4,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.80000000000000004}]},{"osd":5,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.70099999999999996}]},{"osd":6,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.66900000000000004}]},{"osd":7,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.84299999999999997}]},{"osd":9,"last update":"Sat Sep 30 14:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.85799999999999998}]},{"osd":10,"last update":"Sat Sep 30 14:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.873},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1,"5min":1,"15min":1},"last":0.89300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332124,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1495040,"data_stored":1016572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42999999999999999}]},{"osd":1,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63600000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.439,"15min":0.433},"min":{"1min":0.34300000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.44900000000000001}]},{"osd":2,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.38,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.74299999999999999}]},{"osd":3,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84099999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.82399999999999995,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68600000000000005}]},{"osd":4,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.496},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.379,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.56000000000000005}]},{"osd":5,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84199999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.875}]},{"osd":6,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89500000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.63},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.72999999999999998,"15min":0.72899999999999998},"min":{"1min":0.47899999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98599999999999999,"5min":1.004,"15min":1.004},"last":0.65100000000000002}]},{"osd":8,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.84999999999999998}]},{"osd":9,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.76000000000000001}]},{"osd":10,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.70999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]}]},{"osd":6,"up_from":46,"seq":197568495649,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":884736,"data_stored":424669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.373,"15min":0.36899999999999999},"min":{"1min":0.246,"5min":0.22,"15min":0.22},"max":{"1min":0.52800000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.314,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.379}]},{"osd":1,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.59499999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.441,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45100000000000001}]},{"osd":2,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76500000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63100000000000001,"15min":0.628},"min":{"1min":0.49199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85099999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.67300000000000004}]},{"osd":3,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.499,"15min":0.48499999999999999},"min":{"1min":0.33800000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.498,"15min":0.497},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71699999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.40600000000000003}]},{"osd":4,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.47899999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.48799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.60799999999999998}]},{"osd":5,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92600000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.69699999999999995,"15min":0.68600000000000005},"min":{"1min":0.57099999999999995,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.72099999999999997}]},{"osd":7,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.69899999999999995,"15min":0.68400000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.71199999999999997,"15min":0.69999999999999996},"min":{"1min":0.47999999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.69599999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:36:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.68500000000000005}]},{"osd":9,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.65000000000000002}]},{"osd":10,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.71199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":426281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53300000000000003,"15min":0.499},"min":{"1min":0.36199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73999999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54100000000000004,"15min":0.51300000000000001},"min":{"1min":0.33500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78600000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.45200000000000001}]},{"osd":2,"last update":"Sat Sep 30 14:36:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.51900000000000002,"15min":0.498},"min":{"1min":0.44,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.5},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.51400000000000001}]},{"osd":3,"last update":"Sat Sep 30 14:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.56200000000000006,"15min":0.53700000000000003},"min":{"1min":0.39900000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.436},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58899999999999997,"15min":0.56899999999999995},"min":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.41799999999999998}]},{"osd":4,"last update":"Sat Sep 30 14:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.443,"15min":0.42999999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.46800000000000003,"15min":0.45200000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.307}]},{"osd":5,"last update":"Sat Sep 30 14:36:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58999999999999997}]},{"osd":6,"last update":"Sat Sep 30 14:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46899999999999997}]},{"osd":7,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.55700000000000005,"15min":0.54000000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.378}]},{"osd":8,"last update":"Sat Sep 30 14:36:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.54700000000000004}]},{"osd":9,"last update":"Sat Sep 30 14:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.56799999999999995}]},{"osd":10,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.49099999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":424759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 14:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.54600000000000004,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79400000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.51200000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.83099999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.67100000000000004}]},{"osd":2,"last update":"Sat Sep 30 14:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.38800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.39200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.76900000000000002}]},{"osd":3,"last update":"Sat Sep 30 14:35:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.432},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.498,"5min":0.46100000000000002,"15min":0.45500000000000002},"min":{"1min":0.27700000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59099999999999997}]},{"osd":4,"last update":"Sat Sep 30 14:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.58599999999999997},"min":{"1min":0.40200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.60499999999999998,"15min":0.57699999999999996},"min":{"1min":0.47499999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.83099999999999996}]},{"osd":5,"last update":"Sat Sep 30 14:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.56399999999999995,"15min":0.54900000000000004},"min":{"1min":0.46300000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60399999999999998,"15min":0.59499999999999997},"min":{"1min":0.46999999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.79400000000000004}]},{"osd":6,"last update":"Sat Sep 30 14:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.621},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.318,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.80600000000000005}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69699999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.014,"5min":1.147,"15min":1.147},"last":0.442},{"interface":"front","average":{"1min":0.748,"5min":0.67800000000000005,"15min":0.66600000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56799999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.81999999999999995}]},{"osd":9,"last update":"Sat Sep 30 14:36:51 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.753}]},{"osd":10,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.873}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":426373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79500000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.621},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.38,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70999999999999996,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.45200000000000001}]},{"osd":1,"last update":"Sat Sep 30 14:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.51500000000000001},"min":{"1min":0.40000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.35999999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.68000000000000005}]},{"osd":3,"last update":"Sat Sep 30 14:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.40500000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82699999999999996,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.55000000000000004}]},{"osd":4,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.54400000000000004},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.432,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85899999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.60299999999999998}]},{"osd":5,"last update":"Sat Sep 30 14:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.46200000000000002,"15min":0.45400000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.65900000000000003}]},{"osd":6,"last update":"Sat Sep 30 14:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.65700000000000003,"15min":0.64600000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.70299999999999996}]},{"osd":7,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.45500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.80100000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70699999999999996,"15min":0.71299999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.86399999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.59199999999999997}]},{"osd":8,"last update":"Sat Sep 30 14:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.76500000000000001}]},{"osd":9,"last update":"Sat Sep 30 14:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64400000000000002}]},{"osd":10,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.753},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.80000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149742,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":424953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.375,"15min":0.35399999999999998},"min":{"1min":0.28199999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.308},{"interface":"front","average":{"1min":0.496,"5min":0.43099999999999999,"15min":0.41199999999999998},"min":{"1min":0.32300000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.374}]},{"osd":1,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.63200000000000001,"15min":0.58199999999999996},"min":{"1min":0.434,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":5.798,"5min":5.798,"15min":5.798},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.59999999999999998,"15min":0.56899999999999995},"min":{"1min":0.371,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":2.7530000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.57799999999999996}]},{"osd":2,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.46600000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.41799999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.64600000000000002,"15min":0.57899999999999996},"min":{"1min":0.36699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":7.1619999999999999,"5min":7.1619999999999999,"15min":7.1619999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.61199999999999999,"15min":0.57399999999999995},"min":{"1min":0.45500000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.67400000000000004}]},{"osd":5,"last update":"Sat Sep 30 14:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.49399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98499999999999999,"5min":1.071,"15min":1.071},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.55700000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91100000000000003,"5min":1.129,"15min":1.129},"last":0.70199999999999996}]},{"osd":6,"last update":"Sat Sep 30 14:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.621,"15min":0.61499999999999999},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90300000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.46100000000000002}]},{"osd":7,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.72299999999999998,"15min":0.70699999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.745}]},{"osd":8,"last update":"Sat Sep 30 14:36:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.46100000000000002}]},{"osd":9,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.42999999999999999}]},{"osd":10,"last update":"Sat Sep 30 14:37:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":7.2389999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":0.78000000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]}]},{"osd":4,"up_from":33,"seq":141733920808,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":880640,"data_stored":416452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67100000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.65000000000000002}]},{"osd":1,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.377,"15min":0.36899999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.434,"5min":0.377,"15min":0.36299999999999999},"min":{"1min":0.33700000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.38400000000000001}]},{"osd":2,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.54800000000000004,"15min":0.51600000000000001},"min":{"1min":0.41299999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.374,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.67300000000000004}]},{"osd":3,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.41799999999999998}]},{"osd":5,"last update":"Sat Sep 30 14:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.70599999999999996}]},{"osd":6,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63400000000000001,"15min":0.628},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.504},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.81799999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.439}]},{"osd":7,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54500000000000004,"15min":0.53400000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":0.54100000000000004}]},{"osd":8,"last update":"Sat Sep 30 14:36:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.66100000000000003}]},{"osd":9,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.64600000000000002}]},{"osd":10,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.58499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]},{"osd":5,"up_from":38,"seq":163208757285,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1507328,"data_stored":1016513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.39700000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.371,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.373}]},{"osd":1,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.503,"15min":0.48799999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40799999999999997}]},{"osd":2,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41699999999999998,"15min":0.40999999999999998},"min":{"1min":0.28799999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.308},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41599999999999998},"min":{"1min":0.28799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.625,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.33600000000000002}]},{"osd":3,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.45900000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.47699999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58599999999999997}]},{"osd":4,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.55200000000000005}]},{"osd":6,"last update":"Sat Sep 30 14:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.61599999999999999,"15min":0.60799999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.498}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67600000000000005,"15min":0.67800000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.69299999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.68100000000000005}]},{"osd":9,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.71499999999999997}]},{"osd":10,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T14:37:19.770 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T14:37:19.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.930+0000 7f021290e700 1 -- 172.21.15.32:0/540340184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020c100e50 msgr2=0x7f020c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:19.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.930+0000 7f021290e700 1 --2- 172.21.15.32:0/540340184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020c100e50 0x7f020c101270 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009990 tx=0x7f01fc009ca0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.930+0000 7f021290e700 1 -- 172.21.15.32:0/540340184 shutdown_connections 2023-09-30T14:37:19.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.930+0000 7f021290e700 1 --2- 172.21.15.32:0/540340184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f020c102960 0x7f020c109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.930+0000 7f021290e700 1 --2- 172.21.15.32:0/540340184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020c100e50 0x7f020c101270 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.930+0000 7f021290e700 1 --2- 172.21.15.32:0/540340184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f020c101fa0 0x7f020c102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.930+0000 7f021290e700 1 -- 172.21.15.32:0/540340184 >> 172.21.15.32:0/540340184 conn(0x7f020c0fc3a0 msgr2=0x7f020c0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:19.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.930+0000 7f021290e700 1 -- 172.21.15.32:0/540340184 shutdown_connections 2023-09-30T14:37:19.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.930+0000 7f021290e700 1 -- 172.21.15.32:0/540340184 wait complete. 2023-09-30T14:37:19.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.931+0000 7f021290e700 1 Processor -- start 2023-09-30T14:37:19.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.931+0000 7f021290e700 1 -- start start 2023-09-30T14:37:19.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.931+0000 7f021290e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f020c100e50 0x7f020c198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:19.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.931+0000 7f021290e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f020c101fa0 0x7f020c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:19.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.931+0000 7f021290e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020c102960 0x7f020c1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:19.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.931+0000 7f021290e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c1a08e0 con 0x7f020c102960 2023-09-30T14:37:19.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.931+0000 7f021290e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c1a0a20 con 0x7f020c101fa0 2023-09-30T14:37:19.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.931+0000 7f021290e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c1a0bb0 con 0x7f020c100e50 2023-09-30T14:37:19.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.931+0000 7f0210eab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020c102960 0x7f020c1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:19.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.931+0000 7f0210eab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020c102960 0x7f020c1a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33964/0 (socket says 172.21.15.32:33964) 2023-09-30T14:37:19.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.931+0000 7f0210eab700 1 -- 172.21.15.32:0/580229292 learned_addr learned my addr 172.21.15.32:0/580229292 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:19.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.931+0000 7f020bfff700 1 --2- 172.21.15.32:0/580229292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f020c100e50 0x7f020c198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:19.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.931+0000 7f0203fff700 1 --2- 172.21.15.32:0/580229292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f020c101fa0 0x7f020c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:19.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.932+0000 7f0210eab700 1 -- 172.21.15.32:0/580229292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f020c100e50 msgr2=0x7f020c198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:19.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.932+0000 7f0210eab700 1 --2- 172.21.15.32:0/580229292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f020c100e50 0x7f020c198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.932+0000 7f0210eab700 1 -- 172.21.15.32:0/580229292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f020c101fa0 msgr2=0x7f020c1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:19.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.932+0000 7f0210eab700 1 --2- 172.21.15.32:0/580229292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f020c101fa0 0x7f020c1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:19.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.932+0000 7f0210eab700 1 -- 172.21.15.32:0/580229292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc0095f0 con 0x7f020c102960 2023-09-30T14:37:19.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.932+0000 7f0210eab700 1 --2- 172.21.15.32:0/580229292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020c102960 0x7f020c1a0310 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0204007ee0 tx=0x7f020400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:19.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.932+0000 7f0209ffb700 1 -- 172.21.15.32:0/580229292 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0204017070 con 0x7f020c102960 2023-09-30T14:37:19.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.932+0000 7f0209ffb700 1 -- 172.21.15.32:0/580229292 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f020400f040 con 0x7f020c102960 2023-09-30T14:37:19.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.932+0000 7f021290e700 1 -- 172.21.15.32:0/580229292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c1a0e90 con 0x7f020c102960 2023-09-30T14:37:19.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.932+0000 7f0209ffb700 1 -- 172.21.15.32:0/580229292 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0204005740 con 0x7f020c102960 2023-09-30T14:37:19.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.932+0000 7f021290e700 1 -- 172.21.15.32:0/580229292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c1a12c0 con 0x7f020c102960 2023-09-30T14:37:19.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.933+0000 7f021290e700 1 -- 172.21.15.32:0/580229292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c193440 con 0x7f020c102960 2023-09-30T14:37:19.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.934+0000 7f0209ffb700 1 -- 172.21.15.32:0/580229292 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f020400dc60 con 0x7f020c102960 2023-09-30T14:37:19.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.935+0000 7f0209ffb700 1 --2- 172.21.15.32:0/580229292 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f01f80a1070 0x7f01f80a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:19.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.935+0000 7f020bfff700 1 --2- 172.21.15.32:0/580229292 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f01f80a1070 0x7f01f80a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:19.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.935+0000 7f0209ffb700 1 -- 172.21.15.32:0/580229292 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f02040cc590 con 0x7f020c102960 2023-09-30T14:37:19.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.935+0000 7f020bfff700 1 --2- 172.21.15.32:0/580229292 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f01f80a1070 0x7f01f80a3530 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00b5c0 tx=0x7f01fc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:19.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:19.938+0000 7f0209ffb700 1 -- 172.21.15.32:0/580229292 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f0204095ce0 con 0x7f020c102960 2023-09-30T14:37:20.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:20.142+0000 7f021290e700 1 -- 172.21.15.32:0/580229292 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f020c02d270 con 0x7f01f80a1070 2023-09-30T14:37:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:20.154+0000 7f0209ffb700 1 -- 172.21.15.32:0/580229292 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335854 (secure 0 0 0) 0x7f020c02d270 con 0x7f01f80a1070 2023-09-30T14:37:20.158 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:37:20.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:20.158+0000 7f021290e700 1 -- 172.21.15.32:0/580229292 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f01f80a1070 msgr2=0x7f01f80a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:20.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:20.158+0000 7f021290e700 1 --2- 172.21.15.32:0/580229292 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f01f80a1070 0x7f01f80a3530 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00b5c0 tx=0x7f01fc005960 comp rx=0 tx=0).stop 2023-09-30T14:37:20.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:20.158+0000 7f021290e700 1 -- 172.21.15.32:0/580229292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020c102960 msgr2=0x7f020c1a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:20.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:20.158+0000 7f021290e700 1 --2- 172.21.15.32:0/580229292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020c102960 0x7f020c1a0310 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0204007ee0 tx=0x7f020400d3b0 comp rx=0 tx=0).stop 2023-09-30T14:37:20.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:20.158+0000 7f021290e700 1 -- 172.21.15.32:0/580229292 shutdown_connections 2023-09-30T14:37:20.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:20.158+0000 7f021290e700 1 --2- 172.21.15.32:0/580229292 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f01f80a1070 0x7f01f80a3530 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:20.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:20.158+0000 7f021290e700 1 --2- 172.21.15.32:0/580229292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f020c102960 0x7f020c1a0310 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:20.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:20.158+0000 7f021290e700 1 --2- 172.21.15.32:0/580229292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f020c100e50 0x7f020c198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:20.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:20.158+0000 7f021290e700 1 --2- 172.21.15.32:0/580229292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f020c101fa0 0x7f020c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:20.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:20.158+0000 7f021290e700 1 -- 172.21.15.32:0/580229292 >> 172.21.15.32:0/580229292 conn(0x7f020c0fc3a0 msgr2=0x7f020c107930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:20.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:20.158+0000 7f021290e700 1 -- 172.21.15.32:0/580229292 shutdown_connections 2023-09-30T14:37:20.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:20.158+0000 7f021290e700 1 -- 172.21.15.32:0/580229292 wait complete. 2023-09-30T14:37:20.161 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-09-30T14:37:20.170 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2023-09-30T14:37:19.771942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595500,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":292,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336600,"kb_used_data":12396,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605224,"statfs":{"total":1151940427776,"available":1151595749376,"internally_reserved":0,"allocated":12693504,"data_stored":6950882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5132,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":111,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002303"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014576+0000","last_change":"2023-09-30T14:37:11.988122+0000","last_active":"2023-09-30T14:37:13.014576+0000","last_peered":"2023-09-30T14:37:13.014576+0000","last_clean":"2023-09-30T14:37:13.014576+0000","last_became_active":"2023-09-30T14:37:11.986770+0000","last_became_peered":"2023-09-30T14:37:11.986770+0000","last_unstale":"2023-09-30T14:37:13.014576+0000","last_undegraded":"2023-09-30T14:37:13.014576+0000","last_fullsized":"2023-09-30T14:37:13.014576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:46:29.655309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014211+0000","last_change":"2023-09-30T14:37:11.987953+0000","last_active":"2023-09-30T14:37:13.014211+0000","last_peered":"2023-09-30T14:37:13.014211+0000","last_clean":"2023-09-30T14:37:13.014211+0000","last_became_active":"2023-09-30T14:37:11.987248+0000","last_became_peered":"2023-09-30T14:37:11.987248+0000","last_unstale":"2023-09-30T14:37:13.014211+0000","last_undegraded":"2023-09-30T14:37:13.014211+0000","last_fullsized":"2023-09-30T14:37:13.014211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:12:59.780089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014752+0000","last_change":"2023-09-30T14:37:11.987570+0000","last_active":"2023-09-30T14:37:13.014752+0000","last_peered":"2023-09-30T14:37:13.014752+0000","last_clean":"2023-09-30T14:37:13.014752+0000","last_became_active":"2023-09-30T14:37:11.987064+0000","last_became_peered":"2023-09-30T14:37:11.987064+0000","last_unstale":"2023-09-30T14:37:13.014752+0000","last_undegraded":"2023-09-30T14:37:13.014752+0000","last_fullsized":"2023-09-30T14:37:13.014752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:26:16.674605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014524+0000","last_change":"2023-09-30T14:37:11.990161+0000","last_active":"2023-09-30T14:37:13.014524+0000","last_peered":"2023-09-30T14:37:13.014524+0000","last_clean":"2023-09-30T14:37:13.014524+0000","last_became_active":"2023-09-30T14:37:11.989580+0000","last_became_peered":"2023-09-30T14:37:11.989580+0000","last_unstale":"2023-09-30T14:37:13.014524+0000","last_undegraded":"2023-09-30T14:37:13.014524+0000","last_fullsized":"2023-09-30T14:37:13.014524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:13:06.986864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130616+0000","last_change":"2023-09-30T14:37:11.993902+0000","last_active":"2023-09-30T14:37:13.130616+0000","last_peered":"2023-09-30T14:37:13.130616+0000","last_clean":"2023-09-30T14:37:13.130616+0000","last_became_active":"2023-09-30T14:37:11.993622+0000","last_became_peered":"2023-09-30T14:37:11.993622+0000","last_unstale":"2023-09-30T14:37:13.130616+0000","last_undegraded":"2023-09-30T14:37:13.130616+0000","last_fullsized":"2023-09-30T14:37:13.130616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:09:49.252823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.015937+0000","last_change":"2023-09-30T14:37:11.997282+0000","last_active":"2023-09-30T14:37:13.015937+0000","last_peered":"2023-09-30T14:37:13.015937+0000","last_clean":"2023-09-30T14:37:13.015937+0000","last_became_active":"2023-09-30T14:37:11.996937+0000","last_became_peered":"2023-09-30T14:37:11.996937+0000","last_unstale":"2023-09-30T14:37:13.015937+0000","last_undegraded":"2023-09-30T14:37:13.015937+0000","last_fullsized":"2023-09-30T14:37:13.015937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:39:27.351772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014196+0000","last_change":"2023-09-30T14:37:11.997408+0000","last_active":"2023-09-30T14:37:13.014196+0000","last_peered":"2023-09-30T14:37:13.014196+0000","last_clean":"2023-09-30T14:37:13.014196+0000","last_became_active":"2023-09-30T14:37:11.997147+0000","last_became_peered":"2023-09-30T14:37:11.997147+0000","last_unstale":"2023-09-30T14:37:13.014196+0000","last_undegraded":"2023-09-30T14:37:13.014196+0000","last_fullsized":"2023-09-30T14:37:13.014196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:21:24.721417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975636+0000","last_change":"2023-09-30T14:37:11.986172+0000","last_active":"2023-09-30T14:37:12.975636+0000","last_peered":"2023-09-30T14:37:12.975636+0000","last_clean":"2023-09-30T14:37:12.975636+0000","last_became_active":"2023-09-30T14:37:11.985836+0000","last_became_peered":"2023-09-30T14:37:11.985836+0000","last_unstale":"2023-09-30T14:37:12.975636+0000","last_undegraded":"2023-09-30T14:37:12.975636+0000","last_fullsized":"2023-09-30T14:37:12.975636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:35:57.464679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976406+0000","last_change":"2023-09-30T14:37:11.990853+0000","last_active":"2023-09-30T14:37:12.976406+0000","last_peered":"2023-09-30T14:37:12.976406+0000","last_clean":"2023-09-30T14:37:12.976406+0000","last_became_active":"2023-09-30T14:37:11.990321+0000","last_became_peered":"2023-09-30T14:37:11.990321+0000","last_unstale":"2023-09-30T14:37:12.976406+0000","last_undegraded":"2023-09-30T14:37:12.976406+0000","last_fullsized":"2023-09-30T14:37:12.976406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:32:11.409927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.016101+0000","last_change":"2023-09-30T14:37:11.992094+0000","last_active":"2023-09-30T14:37:13.016101+0000","last_peered":"2023-09-30T14:37:13.016101+0000","last_clean":"2023-09-30T14:37:13.016101+0000","last_became_active":"2023-09-30T14:37:11.991762+0000","last_became_peered":"2023-09-30T14:37:11.991762+0000","last_unstale":"2023-09-30T14:37:13.016101+0000","last_undegraded":"2023-09-30T14:37:13.016101+0000","last_fullsized":"2023-09-30T14:37:13.016101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:22:50.203788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014249+0000","last_change":"2023-09-30T14:37:11.992213+0000","last_active":"2023-09-30T14:37:13.014249+0000","last_peered":"2023-09-30T14:37:13.014249+0000","last_clean":"2023-09-30T14:37:13.014249+0000","last_became_active":"2023-09-30T14:37:11.991868+0000","last_became_peered":"2023-09-30T14:37:11.991868+0000","last_unstale":"2023-09-30T14:37:13.014249+0000","last_undegraded":"2023-09-30T14:37:13.014249+0000","last_fullsized":"2023-09-30T14:37:13.014249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:20:06.061978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014693+0000","last_change":"2023-09-30T14:37:11.990043+0000","last_active":"2023-09-30T14:37:13.014693+0000","last_peered":"2023-09-30T14:37:13.014693+0000","last_clean":"2023-09-30T14:37:13.014693+0000","last_became_active":"2023-09-30T14:37:11.988243+0000","last_became_peered":"2023-09-30T14:37:11.988243+0000","last_unstale":"2023-09-30T14:37:13.014693+0000","last_undegraded":"2023-09-30T14:37:13.014693+0000","last_fullsized":"2023-09-30T14:37:13.014693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:17:58.946159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014266+0000","last_change":"2023-09-30T14:37:11.990361+0000","last_active":"2023-09-30T14:37:13.014266+0000","last_peered":"2023-09-30T14:37:13.014266+0000","last_clean":"2023-09-30T14:37:13.014266+0000","last_became_active":"2023-09-30T14:37:11.990080+0000","last_became_peered":"2023-09-30T14:37:11.990080+0000","last_unstale":"2023-09-30T14:37:13.014266+0000","last_undegraded":"2023-09-30T14:37:13.014266+0000","last_fullsized":"2023-09-30T14:37:13.014266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:27:08.830419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014485+0000","last_change":"2023-09-30T14:37:11.988119+0000","last_active":"2023-09-30T14:37:13.014485+0000","last_peered":"2023-09-30T14:37:13.014485+0000","last_clean":"2023-09-30T14:37:13.014485+0000","last_became_active":"2023-09-30T14:37:11.987704+0000","last_became_peered":"2023-09-30T14:37:11.987704+0000","last_unstale":"2023-09-30T14:37:13.014485+0000","last_undegraded":"2023-09-30T14:37:13.014485+0000","last_fullsized":"2023-09-30T14:37:13.014485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:47:23.318982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.013931+0000","last_change":"2023-09-30T14:37:11.985403+0000","last_active":"2023-09-30T14:37:13.013931+0000","last_peered":"2023-09-30T14:37:13.013931+0000","last_clean":"2023-09-30T14:37:13.013931+0000","last_became_active":"2023-09-30T14:37:11.984818+0000","last_became_peered":"2023-09-30T14:37:11.984818+0000","last_unstale":"2023-09-30T14:37:13.013931+0000","last_undegraded":"2023-09-30T14:37:13.013931+0000","last_fullsized":"2023-09-30T14:37:13.013931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:10:43.592505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976073+0000","last_change":"2023-09-30T14:37:11.989500+0000","last_active":"2023-09-30T14:37:12.976073+0000","last_peered":"2023-09-30T14:37:12.976073+0000","last_clean":"2023-09-30T14:37:12.976073+0000","last_became_active":"2023-09-30T14:37:11.989186+0000","last_became_peered":"2023-09-30T14:37:11.989186+0000","last_unstale":"2023-09-30T14:37:12.976073+0000","last_undegraded":"2023-09-30T14:37:12.976073+0000","last_fullsized":"2023-09-30T14:37:12.976073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:14:35.497319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014676+0000","last_change":"2023-09-30T14:37:11.987217+0000","last_active":"2023-09-30T14:37:13.014676+0000","last_peered":"2023-09-30T14:37:13.014676+0000","last_clean":"2023-09-30T14:37:13.014676+0000","last_became_active":"2023-09-30T14:37:11.986835+0000","last_became_peered":"2023-09-30T14:37:11.986835+0000","last_unstale":"2023-09-30T14:37:13.014676+0000","last_undegraded":"2023-09-30T14:37:13.014676+0000","last_fullsized":"2023-09-30T14:37:13.014676+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:11:19.234768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014440+0000","last_change":"2023-09-30T14:37:11.995224+0000","last_active":"2023-09-30T14:37:13.014440+0000","last_peered":"2023-09-30T14:37:13.014440+0000","last_clean":"2023-09-30T14:37:13.014440+0000","last_became_active":"2023-09-30T14:37:11.994925+0000","last_became_peered":"2023-09-30T14:37:11.994925+0000","last_unstale":"2023-09-30T14:37:13.014440+0000","last_undegraded":"2023-09-30T14:37:13.014440+0000","last_fullsized":"2023-09-30T14:37:13.014440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:08:07.453499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975873+0000","last_change":"2023-09-30T14:37:11.990182+0000","last_active":"2023-09-30T14:37:12.975873+0000","last_peered":"2023-09-30T14:37:12.975873+0000","last_clean":"2023-09-30T14:37:12.975873+0000","last_became_active":"2023-09-30T14:37:11.989862+0000","last_became_peered":"2023-09-30T14:37:11.989862+0000","last_unstale":"2023-09-30T14:37:12.975873+0000","last_undegraded":"2023-09-30T14:37:12.975873+0000","last_fullsized":"2023-09-30T14:37:12.975873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:15:31.713002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.076239+0000","last_change":"2023-09-30T14:37:11.028123+0000","last_active":"2023-09-30T14:37:16.076239+0000","last_peered":"2023-09-30T14:37:16.076239+0000","last_clean":"2023-09-30T14:37:16.076239+0000","last_became_active":"2023-09-30T14:37:11.027764+0000","last_became_peered":"2023-09-30T14:37:11.027764+0000","last_unstale":"2023-09-30T14:37:16.076239+0000","last_undegraded":"2023-09-30T14:37:16.076239+0000","last_fullsized":"2023-09-30T14:37:16.076239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:15:50.818783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.977049+0000","last_change":"2023-09-30T14:37:11.994584+0000","last_active":"2023-09-30T14:37:12.977049+0000","last_peered":"2023-09-30T14:37:12.977049+0000","last_clean":"2023-09-30T14:37:12.977049+0000","last_became_active":"2023-09-30T14:37:11.992991+0000","last_became_peered":"2023-09-30T14:37:11.992991+0000","last_unstale":"2023-09-30T14:37:12.977049+0000","last_undegraded":"2023-09-30T14:37:12.977049+0000","last_fullsized":"2023-09-30T14:37:12.977049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:39:50.995717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.977224+0000","last_change":"2023-09-30T14:37:11.020653+0000","last_active":"2023-09-30T14:37:12.977224+0000","last_peered":"2023-09-30T14:37:12.977224+0000","last_clean":"2023-09-30T14:37:12.977224+0000","last_became_active":"2023-09-30T14:37:11.017621+0000","last_became_peered":"2023-09-30T14:37:11.017621+0000","last_unstale":"2023-09-30T14:37:12.977224+0000","last_undegraded":"2023-09-30T14:37:12.977224+0000","last_fullsized":"2023-09-30T14:37:12.977224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:26:59.395278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014548+0000","last_change":"2023-09-30T14:37:11.999881+0000","last_active":"2023-09-30T14:37:13.014548+0000","last_peered":"2023-09-30T14:37:13.014548+0000","last_clean":"2023-09-30T14:37:13.014548+0000","last_became_active":"2023-09-30T14:37:11.999614+0000","last_became_peered":"2023-09-30T14:37:11.999614+0000","last_unstale":"2023-09-30T14:37:13.014548+0000","last_undegraded":"2023-09-30T14:37:13.014548+0000","last_fullsized":"2023-09-30T14:37:13.014548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:59:44.198361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.071987+0000","last_change":"2023-09-30T14:37:11.027187+0000","last_active":"2023-09-30T14:37:16.071987+0000","last_peered":"2023-09-30T14:37:16.071987+0000","last_clean":"2023-09-30T14:37:16.071987+0000","last_became_active":"2023-09-30T14:37:11.026882+0000","last_became_peered":"2023-09-30T14:37:11.026882+0000","last_unstale":"2023-09-30T14:37:16.071987+0000","last_undegraded":"2023-09-30T14:37:16.071987+0000","last_fullsized":"2023-09-30T14:37:16.071987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:17:11.824002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976283+0000","last_change":"2023-09-30T14:37:11.986979+0000","last_active":"2023-09-30T14:37:12.976283+0000","last_peered":"2023-09-30T14:37:12.976283+0000","last_clean":"2023-09-30T14:37:12.976283+0000","last_became_active":"2023-09-30T14:37:11.986705+0000","last_became_peered":"2023-09-30T14:37:11.986705+0000","last_unstale":"2023-09-30T14:37:12.976283+0000","last_undegraded":"2023-09-30T14:37:12.976283+0000","last_fullsized":"2023-09-30T14:37:12.976283+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:31:40.319747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.395336+0000","last_change":"2023-09-30T14:37:11.022313+0000","last_active":"2023-09-30T14:37:12.395336+0000","last_peered":"2023-09-30T14:37:12.395336+0000","last_clean":"2023-09-30T14:37:12.395336+0000","last_became_active":"2023-09-30T14:37:11.021949+0000","last_became_peered":"2023-09-30T14:37:11.021949+0000","last_unstale":"2023-09-30T14:37:12.395336+0000","last_undegraded":"2023-09-30T14:37:12.395336+0000","last_fullsized":"2023-09-30T14:37:12.395336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:30:26.913666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014721+0000","last_change":"2023-09-30T14:37:11.988966+0000","last_active":"2023-09-30T14:37:13.014721+0000","last_peered":"2023-09-30T14:37:13.014721+0000","last_clean":"2023-09-30T14:37:13.014721+0000","last_became_active":"2023-09-30T14:37:11.988716+0000","last_became_peered":"2023-09-30T14:37:11.988716+0000","last_unstale":"2023-09-30T14:37:13.014721+0000","last_undegraded":"2023-09-30T14:37:13.014721+0000","last_fullsized":"2023-09-30T14:37:13.014721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:25:31.437584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.981848+0000","last_change":"2023-09-30T14:37:11.025855+0000","last_active":"2023-09-30T14:37:16.981848+0000","last_peered":"2023-09-30T14:37:16.981848+0000","last_clean":"2023-09-30T14:37:16.981848+0000","last_became_active":"2023-09-30T14:37:11.025402+0000","last_became_peered":"2023-09-30T14:37:11.025402+0000","last_unstale":"2023-09-30T14:37:16.981848+0000","last_undegraded":"2023-09-30T14:37:16.981848+0000","last_fullsized":"2023-09-30T14:37:16.981848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:50:37.692533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130846+0000","last_change":"2023-09-30T14:37:11.993074+0000","last_active":"2023-09-30T14:37:13.130846+0000","last_peered":"2023-09-30T14:37:13.130846+0000","last_clean":"2023-09-30T14:37:13.130846+0000","last_became_active":"2023-09-30T14:37:11.992784+0000","last_became_peered":"2023-09-30T14:37:11.992784+0000","last_unstale":"2023-09-30T14:37:13.130846+0000","last_undegraded":"2023-09-30T14:37:13.130846+0000","last_fullsized":"2023-09-30T14:37:13.130846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:28:18.454389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.051864+0000","last_change":"2023-09-30T14:37:11.022677+0000","last_active":"2023-09-30T14:37:12.051864+0000","last_peered":"2023-09-30T14:37:12.051864+0000","last_clean":"2023-09-30T14:37:12.051864+0000","last_became_active":"2023-09-30T14:37:11.022357+0000","last_became_peered":"2023-09-30T14:37:11.022357+0000","last_unstale":"2023-09-30T14:37:12.051864+0000","last_undegraded":"2023-09-30T14:37:12.051864+0000","last_fullsized":"2023-09-30T14:37:12.051864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:00:15.500968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014764+0000","last_change":"2023-09-30T14:37:11.992358+0000","last_active":"2023-09-30T14:37:13.014764+0000","last_peered":"2023-09-30T14:37:13.014764+0000","last_clean":"2023-09-30T14:37:13.014764+0000","last_became_active":"2023-09-30T14:37:11.992094+0000","last_became_peered":"2023-09-30T14:37:11.992094+0000","last_unstale":"2023-09-30T14:37:13.014764+0000","last_undegraded":"2023-09-30T14:37:13.014764+0000","last_fullsized":"2023-09-30T14:37:13.014764+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:59:36.817138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:11.983426+0000","last_change":"2023-09-30T14:37:11.030961+0000","last_active":"2023-09-30T14:37:11.983426+0000","last_peered":"2023-09-30T14:37:11.983426+0000","last_clean":"2023-09-30T14:37:11.983426+0000","last_became_active":"2023-09-30T14:37:11.030662+0000","last_became_peered":"2023-09-30T14:37:11.030662+0000","last_unstale":"2023-09-30T14:37:11.983426+0000","last_undegraded":"2023-09-30T14:37:11.983426+0000","last_fullsized":"2023-09-30T14:37:11.983426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:35:13.313382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975724+0000","last_change":"2023-09-30T14:37:11.989039+0000","last_active":"2023-09-30T14:37:12.975724+0000","last_peered":"2023-09-30T14:37:12.975724+0000","last_clean":"2023-09-30T14:37:12.975724+0000","last_became_active":"2023-09-30T14:37:11.988594+0000","last_became_peered":"2023-09-30T14:37:11.988594+0000","last_unstale":"2023-09-30T14:37:12.975724+0000","last_undegraded":"2023-09-30T14:37:12.975724+0000","last_fullsized":"2023-09-30T14:37:12.975724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:52:58.146681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014581+0000","last_change":"2023-09-30T14:37:11.026545+0000","last_active":"2023-09-30T14:37:13.014581+0000","last_peered":"2023-09-30T14:37:13.014581+0000","last_clean":"2023-09-30T14:37:13.014581+0000","last_became_active":"2023-09-30T14:37:11.026063+0000","last_became_peered":"2023-09-30T14:37:11.026063+0000","last_unstale":"2023-09-30T14:37:13.014581+0000","last_undegraded":"2023-09-30T14:37:13.014581+0000","last_fullsized":"2023-09-30T14:37:13.014581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:15:34.020336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014356+0000","last_change":"2023-09-30T14:37:11.991845+0000","last_active":"2023-09-30T14:37:13.014356+0000","last_peered":"2023-09-30T14:37:13.014356+0000","last_clean":"2023-09-30T14:37:13.014356+0000","last_became_active":"2023-09-30T14:37:11.991560+0000","last_became_peered":"2023-09-30T14:37:11.991560+0000","last_unstale":"2023-09-30T14:37:13.014356+0000","last_undegraded":"2023-09-30T14:37:13.014356+0000","last_fullsized":"2023-09-30T14:37:13.014356+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:27:39.653882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063853+0000","last_change":"2023-09-30T14:37:11.028192+0000","last_active":"2023-09-30T14:37:15.063853+0000","last_peered":"2023-09-30T14:37:15.063853+0000","last_clean":"2023-09-30T14:37:15.063853+0000","last_became_active":"2023-09-30T14:37:11.027902+0000","last_became_peered":"2023-09-30T14:37:11.027902+0000","last_unstale":"2023-09-30T14:37:15.063853+0000","last_undegraded":"2023-09-30T14:37:15.063853+0000","last_fullsized":"2023-09-30T14:37:15.063853+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:21:30.267559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976420+0000","last_change":"2023-09-30T14:37:11.997318+0000","last_active":"2023-09-30T14:37:12.976420+0000","last_peered":"2023-09-30T14:37:12.976420+0000","last_clean":"2023-09-30T14:37:12.976420+0000","last_became_active":"2023-09-30T14:37:11.996988+0000","last_became_peered":"2023-09-30T14:37:11.996988+0000","last_unstale":"2023-09-30T14:37:12.976420+0000","last_undegraded":"2023-09-30T14:37:12.976420+0000","last_fullsized":"2023-09-30T14:37:12.976420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:57:21.510952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063345+0000","last_change":"2023-09-30T14:37:11.023686+0000","last_active":"2023-09-30T14:37:15.063345+0000","last_peered":"2023-09-30T14:37:15.063345+0000","last_clean":"2023-09-30T14:37:15.063345+0000","last_became_active":"2023-09-30T14:37:11.023365+0000","last_became_peered":"2023-09-30T14:37:11.023365+0000","last_unstale":"2023-09-30T14:37:15.063345+0000","last_undegraded":"2023-09-30T14:37:15.063345+0000","last_fullsized":"2023-09-30T14:37:15.063345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:00:04.358289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014711+0000","last_change":"2023-09-30T14:37:11.989338+0000","last_active":"2023-09-30T14:37:13.014711+0000","last_peered":"2023-09-30T14:37:13.014711+0000","last_clean":"2023-09-30T14:37:13.014711+0000","last_became_active":"2023-09-30T14:37:11.989045+0000","last_became_peered":"2023-09-30T14:37:11.989045+0000","last_unstale":"2023-09-30T14:37:13.014711+0000","last_undegraded":"2023-09-30T14:37:13.014711+0000","last_fullsized":"2023-09-30T14:37:13.014711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:58:59.359684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063525+0000","last_change":"2023-09-30T14:37:11.024709+0000","last_active":"2023-09-30T14:37:15.063525+0000","last_peered":"2023-09-30T14:37:15.063525+0000","last_clean":"2023-09-30T14:37:15.063525+0000","last_became_active":"2023-09-30T14:37:11.024344+0000","last_became_peered":"2023-09-30T14:37:11.024344+0000","last_unstale":"2023-09-30T14:37:15.063525+0000","last_undegraded":"2023-09-30T14:37:15.063525+0000","last_fullsized":"2023-09-30T14:37:15.063525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:17:49.508396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976335+0000","last_change":"2023-09-30T14:37:11.989797+0000","last_active":"2023-09-30T14:37:12.976335+0000","last_peered":"2023-09-30T14:37:12.976335+0000","last_clean":"2023-09-30T14:37:12.976335+0000","last_became_active":"2023-09-30T14:37:11.989482+0000","last_became_peered":"2023-09-30T14:37:11.989482+0000","last_unstale":"2023-09-30T14:37:12.976335+0000","last_undegraded":"2023-09-30T14:37:12.976335+0000","last_fullsized":"2023-09-30T14:37:12.976335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:33:20.437706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.062892+0000","last_change":"2023-09-30T14:37:11.027322+0000","last_active":"2023-09-30T14:37:15.062892+0000","last_peered":"2023-09-30T14:37:15.062892+0000","last_clean":"2023-09-30T14:37:15.062892+0000","last_became_active":"2023-09-30T14:37:11.027007+0000","last_became_peered":"2023-09-30T14:37:11.027007+0000","last_unstale":"2023-09-30T14:37:15.062892+0000","last_undegraded":"2023-09-30T14:37:15.062892+0000","last_fullsized":"2023-09-30T14:37:15.062892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:47:22.919317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130730+0000","last_change":"2023-09-30T14:37:11.988469+0000","last_active":"2023-09-30T14:37:13.130730+0000","last_peered":"2023-09-30T14:37:13.130730+0000","last_clean":"2023-09-30T14:37:13.130730+0000","last_became_active":"2023-09-30T14:37:11.988118+0000","last_became_peered":"2023-09-30T14:37:11.988118+0000","last_unstale":"2023-09-30T14:37:13.130730+0000","last_undegraded":"2023-09-30T14:37:13.130730+0000","last_fullsized":"2023-09-30T14:37:13.130730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:11:56.750690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.072486+0000","last_change":"2023-09-30T14:37:11.024345+0000","last_active":"2023-09-30T14:37:16.072486+0000","last_peered":"2023-09-30T14:37:16.072486+0000","last_clean":"2023-09-30T14:37:16.072486+0000","last_became_active":"2023-09-30T14:37:11.024039+0000","last_became_peered":"2023-09-30T14:37:11.024039+0000","last_unstale":"2023-09-30T14:37:16.072486+0000","last_undegraded":"2023-09-30T14:37:16.072486+0000","last_fullsized":"2023-09-30T14:37:16.072486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:48:32.872814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014748+0000","last_change":"2023-09-30T14:37:11.994068+0000","last_active":"2023-09-30T14:37:13.014748+0000","last_peered":"2023-09-30T14:37:13.014748+0000","last_clean":"2023-09-30T14:37:13.014748+0000","last_became_active":"2023-09-30T14:37:11.993793+0000","last_became_peered":"2023-09-30T14:37:11.993793+0000","last_unstale":"2023-09-30T14:37:13.014748+0000","last_undegraded":"2023-09-30T14:37:13.014748+0000","last_fullsized":"2023-09-30T14:37:13.014748+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:51:28.071414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063744+0000","last_change":"2023-09-30T14:37:11.025622+0000","last_active":"2023-09-30T14:37:15.063744+0000","last_peered":"2023-09-30T14:37:15.063744+0000","last_clean":"2023-09-30T14:37:15.063744+0000","last_became_active":"2023-09-30T14:37:11.025338+0000","last_became_peered":"2023-09-30T14:37:11.025338+0000","last_unstale":"2023-09-30T14:37:15.063744+0000","last_undegraded":"2023-09-30T14:37:15.063744+0000","last_fullsized":"2023-09-30T14:37:15.063744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:32:03.166296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.015550+0000","last_change":"2023-09-30T14:37:11.990678+0000","last_active":"2023-09-30T14:37:13.015550+0000","last_peered":"2023-09-30T14:37:13.015550+0000","last_clean":"2023-09-30T14:37:13.015550+0000","last_became_active":"2023-09-30T14:37:11.990331+0000","last_became_peered":"2023-09-30T14:37:11.990331+0000","last_unstale":"2023-09-30T14:37:13.015550+0000","last_undegraded":"2023-09-30T14:37:13.015550+0000","last_fullsized":"2023-09-30T14:37:13.015550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:57:06.296415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.982063+0000","last_change":"2023-09-30T14:37:11.029501+0000","last_active":"2023-09-30T14:37:16.982063+0000","last_peered":"2023-09-30T14:37:16.982063+0000","last_clean":"2023-09-30T14:37:16.982063+0000","last_became_active":"2023-09-30T14:37:11.029207+0000","last_became_peered":"2023-09-30T14:37:11.029207+0000","last_unstale":"2023-09-30T14:37:16.982063+0000","last_undegraded":"2023-09-30T14:37:16.982063+0000","last_fullsized":"2023-09-30T14:37:16.982063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:48:53.145952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014366+0000","last_change":"2023-09-30T14:37:11.993052+0000","last_active":"2023-09-30T14:37:13.014366+0000","last_peered":"2023-09-30T14:37:13.014366+0000","last_clean":"2023-09-30T14:37:13.014366+0000","last_became_active":"2023-09-30T14:37:11.992762+0000","last_became_peered":"2023-09-30T14:37:11.992762+0000","last_unstale":"2023-09-30T14:37:13.014366+0000","last_undegraded":"2023-09-30T14:37:13.014366+0000","last_fullsized":"2023-09-30T14:37:13.014366+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:09:16.440026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.075318+0000","last_change":"2023-09-30T14:37:11.027266+0000","last_active":"2023-09-30T14:37:16.075318+0000","last_peered":"2023-09-30T14:37:16.075318+0000","last_clean":"2023-09-30T14:37:16.075318+0000","last_became_active":"2023-09-30T14:37:11.026971+0000","last_became_peered":"2023-09-30T14:37:11.026971+0000","last_unstale":"2023-09-30T14:37:16.075318+0000","last_undegraded":"2023-09-30T14:37:16.075318+0000","last_fullsized":"2023-09-30T14:37:16.075318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:38:09.543391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014408+0000","last_change":"2023-09-30T14:37:11.989592+0000","last_active":"2023-09-30T14:37:13.014408+0000","last_peered":"2023-09-30T14:37:13.014408+0000","last_clean":"2023-09-30T14:37:13.014408+0000","last_became_active":"2023-09-30T14:37:11.989290+0000","last_became_peered":"2023-09-30T14:37:11.989290+0000","last_unstale":"2023-09-30T14:37:13.014408+0000","last_undegraded":"2023-09-30T14:37:13.014408+0000","last_fullsized":"2023-09-30T14:37:13.014408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:11:33.974823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130849+0000","last_change":"2023-09-30T14:37:11.028919+0000","last_active":"2023-09-30T14:37:13.130849+0000","last_peered":"2023-09-30T14:37:13.130849+0000","last_clean":"2023-09-30T14:37:13.130849+0000","last_became_active":"2023-09-30T14:37:11.028532+0000","last_became_peered":"2023-09-30T14:37:11.028532+0000","last_unstale":"2023-09-30T14:37:13.130849+0000","last_undegraded":"2023-09-30T14:37:13.130849+0000","last_fullsized":"2023-09-30T14:37:13.130849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:15:42.675409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014483+0000","last_change":"2023-09-30T14:37:11.980611+0000","last_active":"2023-09-30T14:37:13.014483+0000","last_peered":"2023-09-30T14:37:13.014483+0000","last_clean":"2023-09-30T14:37:13.014483+0000","last_became_active":"2023-09-30T14:37:11.980288+0000","last_became_peered":"2023-09-30T14:37:11.980288+0000","last_unstale":"2023-09-30T14:37:13.014483+0000","last_undegraded":"2023-09-30T14:37:13.014483+0000","last_fullsized":"2023-09-30T14:37:13.014483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:38:49.270976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014852+0000","last_change":"2023-09-30T14:37:11.025133+0000","last_active":"2023-09-30T14:37:13.014852+0000","last_peered":"2023-09-30T14:37:13.014852+0000","last_clean":"2023-09-30T14:37:13.014852+0000","last_became_active":"2023-09-30T14:37:11.024914+0000","last_became_peered":"2023-09-30T14:37:11.024914+0000","last_unstale":"2023-09-30T14:37:13.014852+0000","last_undegraded":"2023-09-30T14:37:13.014852+0000","last_fullsized":"2023-09-30T14:37:13.014852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:23:39.328773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.015965+0000","last_change":"2023-09-30T14:37:11.988434+0000","last_active":"2023-09-30T14:37:13.015965+0000","last_peered":"2023-09-30T14:37:13.015965+0000","last_clean":"2023-09-30T14:37:13.015965+0000","last_became_active":"2023-09-30T14:37:11.988156+0000","last_became_peered":"2023-09-30T14:37:11.988156+0000","last_unstale":"2023-09-30T14:37:13.015965+0000","last_undegraded":"2023-09-30T14:37:13.015965+0000","last_fullsized":"2023-09-30T14:37:13.015965+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:20:49.420026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:11.978658+0000","last_change":"2023-09-30T14:37:11.026290+0000","last_active":"2023-09-30T14:37:11.978658+0000","last_peered":"2023-09-30T14:37:11.978658+0000","last_clean":"2023-09-30T14:37:11.978658+0000","last_became_active":"2023-09-30T14:37:11.024684+0000","last_became_peered":"2023-09-30T14:37:11.024684+0000","last_unstale":"2023-09-30T14:37:11.978658+0000","last_undegraded":"2023-09-30T14:37:11.978658+0000","last_fullsized":"2023-09-30T14:37:11.978658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:43:06.463243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.016048+0000","last_change":"2023-09-30T14:37:11.987989+0000","last_active":"2023-09-30T14:37:13.016048+0000","last_peered":"2023-09-30T14:37:13.016048+0000","last_clean":"2023-09-30T14:37:13.016048+0000","last_became_active":"2023-09-30T14:37:11.987690+0000","last_became_peered":"2023-09-30T14:37:11.987690+0000","last_unstale":"2023-09-30T14:37:13.016048+0000","last_undegraded":"2023-09-30T14:37:13.016048+0000","last_fullsized":"2023-09-30T14:37:13.016048+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:51:25.825871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014864+0000","last_change":"2023-09-30T14:37:11.996852+0000","last_active":"2023-09-30T14:37:13.014864+0000","last_peered":"2023-09-30T14:37:13.014864+0000","last_clean":"2023-09-30T14:37:13.014864+0000","last_became_active":"2023-09-30T14:37:11.996596+0000","last_became_peered":"2023-09-30T14:37:11.996596+0000","last_unstale":"2023-09-30T14:37:13.014864+0000","last_undegraded":"2023-09-30T14:37:13.014864+0000","last_fullsized":"2023-09-30T14:37:13.014864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:25:04.531236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014217+0000","last_change":"2023-09-30T14:37:11.025458+0000","last_active":"2023-09-30T14:37:13.014217+0000","last_peered":"2023-09-30T14:37:13.014217+0000","last_clean":"2023-09-30T14:37:13.014217+0000","last_became_active":"2023-09-30T14:37:11.025157+0000","last_became_peered":"2023-09-30T14:37:11.025157+0000","last_unstale":"2023-09-30T14:37:13.014217+0000","last_undegraded":"2023-09-30T14:37:13.014217+0000","last_fullsized":"2023-09-30T14:37:13.014217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:31:24.436125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976151+0000","last_change":"2023-09-30T14:37:11.986840+0000","last_active":"2023-09-30T14:37:12.976151+0000","last_peered":"2023-09-30T14:37:12.976151+0000","last_clean":"2023-09-30T14:37:12.976151+0000","last_became_active":"2023-09-30T14:37:11.986483+0000","last_became_peered":"2023-09-30T14:37:11.986483+0000","last_unstale":"2023-09-30T14:37:12.976151+0000","last_undegraded":"2023-09-30T14:37:12.976151+0000","last_fullsized":"2023-09-30T14:37:12.976151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:14:32.399685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.979424+0000","last_change":"2023-09-30T14:37:11.025706+0000","last_active":"2023-09-30T14:37:16.979424+0000","last_peered":"2023-09-30T14:37:16.979424+0000","last_clean":"2023-09-30T14:37:16.979424+0000","last_became_active":"2023-09-30T14:37:11.025376+0000","last_became_peered":"2023-09-30T14:37:11.025376+0000","last_unstale":"2023-09-30T14:37:16.979424+0000","last_undegraded":"2023-09-30T14:37:16.979424+0000","last_fullsized":"2023-09-30T14:37:16.979424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:10:36.025273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":308,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:11.977686+0000","last_change":"2023-09-30T14:36:12.344762+0000","last_active":"2023-09-30T14:37:11.977686+0000","last_peered":"2023-09-30T14:37:11.977686+0000","last_clean":"2023-09-30T14:37:11.977686+0000","last_became_active":"2023-09-30T14:36:12.344268+0000","last_became_peered":"2023-09-30T14:36:12.344268+0000","last_unstale":"2023-09-30T14:37:11.977686+0000","last_undegraded":"2023-09-30T14:37:11.977686+0000","last_fullsized":"2023-09-30T14:37:11.977686+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:33:20.434992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:33:20.434992+0000","last_clean_scrub_stamp":"2023-09-30T14:33:20.434992+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:00:05.598961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014131+0000","last_change":"2023-09-30T14:37:11.996886+0000","last_active":"2023-09-30T14:37:13.014131+0000","last_peered":"2023-09-30T14:37:13.014131+0000","last_clean":"2023-09-30T14:37:13.014131+0000","last_became_active":"2023-09-30T14:37:11.996667+0000","last_became_peered":"2023-09-30T14:37:11.996667+0000","last_unstale":"2023-09-30T14:37:13.014131+0000","last_undegraded":"2023-09-30T14:37:13.014131+0000","last_fullsized":"2023-09-30T14:37:13.014131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:27:00.546356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.013888+0000","last_change":"2023-09-30T14:37:11.027012+0000","last_active":"2023-09-30T14:37:13.013888+0000","last_peered":"2023-09-30T14:37:13.013888+0000","last_clean":"2023-09-30T14:37:13.013888+0000","last_became_active":"2023-09-30T14:37:11.026714+0000","last_became_peered":"2023-09-30T14:37:11.026714+0000","last_unstale":"2023-09-30T14:37:13.013888+0000","last_undegraded":"2023-09-30T14:37:13.013888+0000","last_fullsized":"2023-09-30T14:37:13.013888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:17:57.860438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.062444+0000","last_change":"2023-09-30T14:37:11.027389+0000","last_active":"2023-09-30T14:37:15.062444+0000","last_peered":"2023-09-30T14:37:15.062444+0000","last_clean":"2023-09-30T14:37:15.062444+0000","last_became_active":"2023-09-30T14:37:11.027011+0000","last_became_peered":"2023-09-30T14:37:11.027011+0000","last_unstale":"2023-09-30T14:37:15.062444+0000","last_undegraded":"2023-09-30T14:37:15.062444+0000","last_fullsized":"2023-09-30T14:37:15.062444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:38:22.953936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014762+0000","last_change":"2023-09-30T14:37:11.997675+0000","last_active":"2023-09-30T14:37:13.014762+0000","last_peered":"2023-09-30T14:37:13.014762+0000","last_clean":"2023-09-30T14:37:13.014762+0000","last_became_active":"2023-09-30T14:37:11.997330+0000","last_became_peered":"2023-09-30T14:37:11.997330+0000","last_unstale":"2023-09-30T14:37:13.014762+0000","last_undegraded":"2023-09-30T14:37:13.014762+0000","last_fullsized":"2023-09-30T14:37:13.014762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:54:52.802138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.075036+0000","last_change":"2023-09-30T14:37:11.023836+0000","last_active":"2023-09-30T14:37:16.075036+0000","last_peered":"2023-09-30T14:37:16.075036+0000","last_clean":"2023-09-30T14:37:16.075036+0000","last_became_active":"2023-09-30T14:37:11.023362+0000","last_became_peered":"2023-09-30T14:37:11.023362+0000","last_unstale":"2023-09-30T14:37:16.075036+0000","last_undegraded":"2023-09-30T14:37:16.075036+0000","last_fullsized":"2023-09-30T14:37:16.075036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:58:22.772546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976655+0000","last_change":"2023-09-30T14:37:11.985281+0000","last_active":"2023-09-30T14:37:12.976655+0000","last_peered":"2023-09-30T14:37:12.976655+0000","last_clean":"2023-09-30T14:37:12.976655+0000","last_became_active":"2023-09-30T14:37:11.984944+0000","last_became_peered":"2023-09-30T14:37:11.984944+0000","last_unstale":"2023-09-30T14:37:12.976655+0000","last_undegraded":"2023-09-30T14:37:12.976655+0000","last_fullsized":"2023-09-30T14:37:12.976655+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:30:19.829727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.061631+0000","last_change":"2023-09-30T14:37:11.028070+0000","last_active":"2023-09-30T14:37:15.061631+0000","last_peered":"2023-09-30T14:37:15.061631+0000","last_clean":"2023-09-30T14:37:15.061631+0000","last_became_active":"2023-09-30T14:37:11.027703+0000","last_became_peered":"2023-09-30T14:37:11.027703+0000","last_unstale":"2023-09-30T14:37:15.061631+0000","last_undegraded":"2023-09-30T14:37:15.061631+0000","last_fullsized":"2023-09-30T14:37:15.061631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:23:15.071946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014359+0000","last_change":"2023-09-30T14:37:11.989897+0000","last_active":"2023-09-30T14:37:13.014359+0000","last_peered":"2023-09-30T14:37:13.014359+0000","last_clean":"2023-09-30T14:37:13.014359+0000","last_became_active":"2023-09-30T14:37:11.989613+0000","last_became_peered":"2023-09-30T14:37:11.989613+0000","last_unstale":"2023-09-30T14:37:13.014359+0000","last_undegraded":"2023-09-30T14:37:13.014359+0000","last_fullsized":"2023-09-30T14:37:13.014359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:43:17.887766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.062793+0000","last_change":"2023-09-30T14:37:11.026491+0000","last_active":"2023-09-30T14:37:15.062793+0000","last_peered":"2023-09-30T14:37:15.062793+0000","last_clean":"2023-09-30T14:37:15.062793+0000","last_became_active":"2023-09-30T14:37:11.026196+0000","last_became_peered":"2023-09-30T14:37:11.026196+0000","last_unstale":"2023-09-30T14:37:15.062793+0000","last_undegraded":"2023-09-30T14:37:15.062793+0000","last_fullsized":"2023-09-30T14:37:15.062793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:17:22.343377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130535+0000","last_change":"2023-09-30T14:37:11.988570+0000","last_active":"2023-09-30T14:37:13.130535+0000","last_peered":"2023-09-30T14:37:13.130535+0000","last_clean":"2023-09-30T14:37:13.130535+0000","last_became_active":"2023-09-30T14:37:11.988230+0000","last_became_peered":"2023-09-30T14:37:11.988230+0000","last_unstale":"2023-09-30T14:37:13.130535+0000","last_undegraded":"2023-09-30T14:37:13.130535+0000","last_fullsized":"2023-09-30T14:37:13.130535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:58:38.302343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.062480+0000","last_change":"2023-09-30T14:37:11.024587+0000","last_active":"2023-09-30T14:37:15.062480+0000","last_peered":"2023-09-30T14:37:15.062480+0000","last_clean":"2023-09-30T14:37:15.062480+0000","last_became_active":"2023-09-30T14:37:11.024253+0000","last_became_peered":"2023-09-30T14:37:11.024253+0000","last_unstale":"2023-09-30T14:37:15.062480+0000","last_undegraded":"2023-09-30T14:37:15.062480+0000","last_fullsized":"2023-09-30T14:37:15.062480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:31:07.293657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014239+0000","last_change":"2023-09-30T14:37:11.987818+0000","last_active":"2023-09-30T14:37:13.014239+0000","last_peered":"2023-09-30T14:37:13.014239+0000","last_clean":"2023-09-30T14:37:13.014239+0000","last_became_active":"2023-09-30T14:37:11.987476+0000","last_became_peered":"2023-09-30T14:37:11.987476+0000","last_unstale":"2023-09-30T14:37:13.014239+0000","last_undegraded":"2023-09-30T14:37:13.014239+0000","last_fullsized":"2023-09-30T14:37:13.014239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:01:07.802959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:14.444569+0000","last_change":"2023-09-30T14:37:11.023223+0000","last_active":"2023-09-30T14:37:14.444569+0000","last_peered":"2023-09-30T14:37:14.444569+0000","last_clean":"2023-09-30T14:37:14.444569+0000","last_became_active":"2023-09-30T14:37:11.022932+0000","last_became_peered":"2023-09-30T14:37:11.022932+0000","last_unstale":"2023-09-30T14:37:14.444569+0000","last_undegraded":"2023-09-30T14:37:14.444569+0000","last_fullsized":"2023-09-30T14:37:14.444569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:01:04.315669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976068+0000","last_change":"2023-09-30T14:37:11.990323+0000","last_active":"2023-09-30T14:37:12.976068+0000","last_peered":"2023-09-30T14:37:12.976068+0000","last_clean":"2023-09-30T14:37:12.976068+0000","last_became_active":"2023-09-30T14:37:11.989981+0000","last_became_peered":"2023-09-30T14:37:11.989981+0000","last_unstale":"2023-09-30T14:37:12.976068+0000","last_undegraded":"2023-09-30T14:37:12.976068+0000","last_fullsized":"2023-09-30T14:37:12.976068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:30:44.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063371+0000","last_change":"2023-09-30T14:37:11.025658+0000","last_active":"2023-09-30T14:37:15.063371+0000","last_peered":"2023-09-30T14:37:15.063371+0000","last_clean":"2023-09-30T14:37:15.063371+0000","last_became_active":"2023-09-30T14:37:11.025334+0000","last_became_peered":"2023-09-30T14:37:11.025334+0000","last_unstale":"2023-09-30T14:37:15.063371+0000","last_undegraded":"2023-09-30T14:37:15.063371+0000","last_fullsized":"2023-09-30T14:37:15.063371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:23:41.933634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014128+0000","last_change":"2023-09-30T14:37:11.989989+0000","last_active":"2023-09-30T14:37:13.014128+0000","last_peered":"2023-09-30T14:37:13.014128+0000","last_clean":"2023-09-30T14:37:13.014128+0000","last_became_active":"2023-09-30T14:37:11.989666+0000","last_became_peered":"2023-09-30T14:37:11.989666+0000","last_unstale":"2023-09-30T14:37:13.014128+0000","last_undegraded":"2023-09-30T14:37:13.014128+0000","last_fullsized":"2023-09-30T14:37:13.014128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:41:09.835210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.074595+0000","last_change":"2023-09-30T14:37:11.025027+0000","last_active":"2023-09-30T14:37:16.074595+0000","last_peered":"2023-09-30T14:37:16.074595+0000","last_clean":"2023-09-30T14:37:16.074595+0000","last_became_active":"2023-09-30T14:37:11.024696+0000","last_became_peered":"2023-09-30T14:37:11.024696+0000","last_unstale":"2023-09-30T14:37:16.074595+0000","last_undegraded":"2023-09-30T14:37:16.074595+0000","last_fullsized":"2023-09-30T14:37:16.074595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:34:28.384535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014390+0000","last_change":"2023-09-30T14:37:11.992674+0000","last_active":"2023-09-30T14:37:13.014390+0000","last_peered":"2023-09-30T14:37:13.014390+0000","last_clean":"2023-09-30T14:37:13.014390+0000","last_became_active":"2023-09-30T14:37:11.992243+0000","last_became_peered":"2023-09-30T14:37:11.992243+0000","last_unstale":"2023-09-30T14:37:13.014390+0000","last_undegraded":"2023-09-30T14:37:13.014390+0000","last_fullsized":"2023-09-30T14:37:13.014390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:33:19.759637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014643+0000","last_change":"2023-09-30T14:37:11.024525+0000","last_active":"2023-09-30T14:37:13.014643+0000","last_peered":"2023-09-30T14:37:13.014643+0000","last_clean":"2023-09-30T14:37:13.014643+0000","last_became_active":"2023-09-30T14:37:11.024231+0000","last_became_peered":"2023-09-30T14:37:11.024231+0000","last_unstale":"2023-09-30T14:37:13.014643+0000","last_undegraded":"2023-09-30T14:37:13.014643+0000","last_fullsized":"2023-09-30T14:37:13.014643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:53:47.142798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014167+0000","last_change":"2023-09-30T14:37:11.991401+0000","last_active":"2023-09-30T14:37:13.014167+0000","last_peered":"2023-09-30T14:37:13.014167+0000","last_clean":"2023-09-30T14:37:13.014167+0000","last_became_active":"2023-09-30T14:37:11.991012+0000","last_became_peered":"2023-09-30T14:37:11.991012+0000","last_unstale":"2023-09-30T14:37:13.014167+0000","last_undegraded":"2023-09-30T14:37:13.014167+0000","last_fullsized":"2023-09-30T14:37:13.014167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:55:00.562030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.050302+0000","last_change":"2023-09-30T14:37:11.022893+0000","last_active":"2023-09-30T14:37:12.050302+0000","last_peered":"2023-09-30T14:37:12.050302+0000","last_clean":"2023-09-30T14:37:12.050302+0000","last_became_active":"2023-09-30T14:37:11.022505+0000","last_became_peered":"2023-09-30T14:37:11.022505+0000","last_unstale":"2023-09-30T14:37:12.050302+0000","last_undegraded":"2023-09-30T14:37:12.050302+0000","last_fullsized":"2023-09-30T14:37:12.050302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:00:17.721645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014348+0000","last_change":"2023-09-30T14:37:11.992942+0000","last_active":"2023-09-30T14:37:13.014348+0000","last_peered":"2023-09-30T14:37:13.014348+0000","last_clean":"2023-09-30T14:37:13.014348+0000","last_became_active":"2023-09-30T14:37:11.992698+0000","last_became_peered":"2023-09-30T14:37:11.992698+0000","last_unstale":"2023-09-30T14:37:13.014348+0000","last_undegraded":"2023-09-30T14:37:13.014348+0000","last_fullsized":"2023-09-30T14:37:13.014348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:15:30.239423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014256+0000","last_change":"2023-09-30T14:37:11.984435+0000","last_active":"2023-09-30T14:37:13.014256+0000","last_peered":"2023-09-30T14:37:13.014256+0000","last_clean":"2023-09-30T14:37:13.014256+0000","last_became_active":"2023-09-30T14:37:11.984137+0000","last_became_peered":"2023-09-30T14:37:11.984137+0000","last_unstale":"2023-09-30T14:37:13.014256+0000","last_undegraded":"2023-09-30T14:37:13.014256+0000","last_fullsized":"2023-09-30T14:37:13.014256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:40:41.112326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975815+0000","last_change":"2023-09-30T14:37:11.984481+0000","last_active":"2023-09-30T14:37:12.975815+0000","last_peered":"2023-09-30T14:37:12.975815+0000","last_clean":"2023-09-30T14:37:12.975815+0000","last_became_active":"2023-09-30T14:37:11.984021+0000","last_became_peered":"2023-09-30T14:37:11.984021+0000","last_unstale":"2023-09-30T14:37:12.975815+0000","last_undegraded":"2023-09-30T14:37:12.975815+0000","last_fullsized":"2023-09-30T14:37:12.975815+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:36:33.184379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975777+0000","last_change":"2023-09-30T14:37:11.990779+0000","last_active":"2023-09-30T14:37:12.975777+0000","last_peered":"2023-09-30T14:37:12.975777+0000","last_clean":"2023-09-30T14:37:12.975777+0000","last_became_active":"2023-09-30T14:37:11.989960+0000","last_became_peered":"2023-09-30T14:37:11.989960+0000","last_unstale":"2023-09-30T14:37:12.975777+0000","last_undegraded":"2023-09-30T14:37:12.975777+0000","last_fullsized":"2023-09-30T14:37:12.975777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:54:36.813191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.015049+0000","last_change":"2023-09-30T14:37:11.995750+0000","last_active":"2023-09-30T14:37:13.015049+0000","last_peered":"2023-09-30T14:37:13.015049+0000","last_clean":"2023-09-30T14:37:13.015049+0000","last_became_active":"2023-09-30T14:37:11.995434+0000","last_became_peered":"2023-09-30T14:37:11.995434+0000","last_unstale":"2023-09-30T14:37:13.015049+0000","last_undegraded":"2023-09-30T14:37:13.015049+0000","last_fullsized":"2023-09-30T14:37:13.015049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:49:51.234024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014630+0000","last_change":"2023-09-30T14:37:11.996550+0000","last_active":"2023-09-30T14:37:13.014630+0000","last_peered":"2023-09-30T14:37:13.014630+0000","last_clean":"2023-09-30T14:37:13.014630+0000","last_became_active":"2023-09-30T14:37:11.996269+0000","last_became_peered":"2023-09-30T14:37:11.996269+0000","last_unstale":"2023-09-30T14:37:13.014630+0000","last_undegraded":"2023-09-30T14:37:13.014630+0000","last_fullsized":"2023-09-30T14:37:13.014630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:40:16.537270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014992+0000","last_change":"2023-09-30T14:37:11.983495+0000","last_active":"2023-09-30T14:37:13.014992+0000","last_peered":"2023-09-30T14:37:13.014992+0000","last_clean":"2023-09-30T14:37:13.014992+0000","last_became_active":"2023-09-30T14:37:11.983241+0000","last_became_peered":"2023-09-30T14:37:11.983241+0000","last_unstale":"2023-09-30T14:37:13.014992+0000","last_undegraded":"2023-09-30T14:37:13.014992+0000","last_fullsized":"2023-09-30T14:37:13.014992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:41:46.604467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975888+0000","last_change":"2023-09-30T14:37:11.996058+0000","last_active":"2023-09-30T14:37:12.975888+0000","last_peered":"2023-09-30T14:37:12.975888+0000","last_clean":"2023-09-30T14:37:12.975888+0000","last_became_active":"2023-09-30T14:37:11.995763+0000","last_became_peered":"2023-09-30T14:37:11.995763+0000","last_unstale":"2023-09-30T14:37:12.975888+0000","last_undegraded":"2023-09-30T14:37:12.975888+0000","last_fullsized":"2023-09-30T14:37:12.975888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:42:21.306561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976498+0000","last_change":"2023-09-30T14:37:11.992080+0000","last_active":"2023-09-30T14:37:12.976498+0000","last_peered":"2023-09-30T14:37:12.976498+0000","last_clean":"2023-09-30T14:37:12.976498+0000","last_became_active":"2023-09-30T14:37:11.991747+0000","last_became_peered":"2023-09-30T14:37:11.991747+0000","last_unstale":"2023-09-30T14:37:12.976498+0000","last_undegraded":"2023-09-30T14:37:12.976498+0000","last_fullsized":"2023-09-30T14:37:12.976498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:07:25.985911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014724+0000","last_change":"2023-09-30T14:37:11.997109+0000","last_active":"2023-09-30T14:37:13.014724+0000","last_peered":"2023-09-30T14:37:13.014724+0000","last_clean":"2023-09-30T14:37:13.014724+0000","last_became_active":"2023-09-30T14:37:11.996793+0000","last_became_peered":"2023-09-30T14:37:11.996793+0000","last_unstale":"2023-09-30T14:37:13.014724+0000","last_undegraded":"2023-09-30T14:37:13.014724+0000","last_fullsized":"2023-09-30T14:37:13.014724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:13:59.838638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975786+0000","last_change":"2023-09-30T14:37:11.984083+0000","last_active":"2023-09-30T14:37:12.975786+0000","last_peered":"2023-09-30T14:37:12.975786+0000","last_clean":"2023-09-30T14:37:12.975786+0000","last_became_active":"2023-09-30T14:37:11.983352+0000","last_became_peered":"2023-09-30T14:37:11.983352+0000","last_unstale":"2023-09-30T14:37:12.975786+0000","last_undegraded":"2023-09-30T14:37:12.975786+0000","last_fullsized":"2023-09-30T14:37:12.975786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:02:03.668484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976194+0000","last_change":"2023-09-30T14:37:11.992212+0000","last_active":"2023-09-30T14:37:12.976194+0000","last_peered":"2023-09-30T14:37:12.976194+0000","last_clean":"2023-09-30T14:37:12.976194+0000","last_became_active":"2023-09-30T14:37:11.990158+0000","last_became_peered":"2023-09-30T14:37:11.990158+0000","last_unstale":"2023-09-30T14:37:12.976194+0000","last_undegraded":"2023-09-30T14:37:12.976194+0000","last_fullsized":"2023-09-30T14:37:12.976194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:14:27.976110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014402+0000","last_change":"2023-09-30T14:37:11.995884+0000","last_active":"2023-09-30T14:37:13.014402+0000","last_peered":"2023-09-30T14:37:13.014402+0000","last_clean":"2023-09-30T14:37:13.014402+0000","last_became_active":"2023-09-30T14:37:11.995551+0000","last_became_peered":"2023-09-30T14:37:11.995551+0000","last_unstale":"2023-09-30T14:37:13.014402+0000","last_undegraded":"2023-09-30T14:37:13.014402+0000","last_fullsized":"2023-09-30T14:37:13.014402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:28:26.051188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014437+0000","last_change":"2023-09-30T14:37:11.985853+0000","last_active":"2023-09-30T14:37:13.014437+0000","last_peered":"2023-09-30T14:37:13.014437+0000","last_clean":"2023-09-30T14:37:13.014437+0000","last_became_active":"2023-09-30T14:37:11.985525+0000","last_became_peered":"2023-09-30T14:37:11.985525+0000","last_unstale":"2023-09-30T14:37:13.014437+0000","last_undegraded":"2023-09-30T14:37:13.014437+0000","last_fullsized":"2023-09-30T14:37:13.014437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:07:24.392538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5132,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":111,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290816,"data_stored":83130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449099,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":425000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]}]},{"osd":10,"up_from":71,"seq":304942678031,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1503232,"data_stored":1016843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":1,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.442},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.47299999999999998}]},{"osd":2,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.61099999999999999}]},{"osd":3,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.72599999999999998}]},{"osd":4,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.50800000000000001}]},{"osd":5,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.68100000000000005}]},{"osd":6,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.41699999999999998}]},{"osd":8,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.745},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.79300000000000004}]},{"osd":9,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.872}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":9,"up_from":66,"seq":283467841555,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":425487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.376},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.35199999999999998}]},{"osd":1,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58599999999999997}]},{"osd":2,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.53400000000000003}]},{"osd":3,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.40500000000000003}]},{"osd":4,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.70899999999999996}]},{"osd":5,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.57399999999999995}]},{"osd":6,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.496},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.55300000000000005}]},{"osd":7,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68000000000000005}]},{"osd":8,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.622}]},{"osd":10,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.78200000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":425738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.48699999999999999}]},{"osd":1,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.61599999999999999}]},{"osd":2,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.442},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.41799999999999998}]},{"osd":3,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59399999999999997}]},{"osd":4,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.80000000000000004}]},{"osd":5,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.70099999999999996}]},{"osd":6,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.66900000000000004}]},{"osd":7,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.84299999999999997}]},{"osd":9,"last update":"Sat Sep 30 14:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.85799999999999998}]},{"osd":10,"last update":"Sat Sep 30 14:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.873},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1,"5min":1,"15min":1},"last":0.89300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332124,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1495040,"data_stored":1016572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42999999999999999}]},{"osd":1,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63600000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.439,"15min":0.433},"min":{"1min":0.34300000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.44900000000000001}]},{"osd":2,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.38,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.74299999999999999}]},{"osd":3,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84099999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.82399999999999995,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68600000000000005}]},{"osd":4,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.496},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.379,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.56000000000000005}]},{"osd":5,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84199999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.875}]},{"osd":6,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89500000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.63},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.72999999999999998,"15min":0.72899999999999998},"min":{"1min":0.47899999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98599999999999999,"5min":1.004,"15min":1.004},"last":0.65100000000000002}]},{"osd":8,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.84999999999999998}]},{"osd":9,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.76000000000000001}]},{"osd":10,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.70999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]}]},{"osd":6,"up_from":46,"seq":197568495649,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":884736,"data_stored":424669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.373,"15min":0.36899999999999999},"min":{"1min":0.246,"5min":0.22,"15min":0.22},"max":{"1min":0.52800000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.314,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.379}]},{"osd":1,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.59499999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.441,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45100000000000001}]},{"osd":2,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76500000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63100000000000001,"15min":0.628},"min":{"1min":0.49199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85099999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.67300000000000004}]},{"osd":3,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.499,"15min":0.48499999999999999},"min":{"1min":0.33800000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.498,"15min":0.497},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71699999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.40600000000000003}]},{"osd":4,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.47899999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.48799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.60799999999999998}]},{"osd":5,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92600000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.69699999999999995,"15min":0.68600000000000005},"min":{"1min":0.57099999999999995,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.72099999999999997}]},{"osd":7,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.69899999999999995,"15min":0.68400000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.71199999999999997,"15min":0.69999999999999996},"min":{"1min":0.47999999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.69599999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:36:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.68500000000000005}]},{"osd":9,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.65000000000000002}]},{"osd":10,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.71199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":460954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53300000000000003,"15min":0.499},"min":{"1min":0.36199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73999999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54100000000000004,"15min":0.51300000000000001},"min":{"1min":0.33500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78600000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.83299999999999996}]},{"osd":2,"last update":"Sat Sep 30 14:36:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.51900000000000002,"15min":0.498},"min":{"1min":0.44,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56299999999999994}]},{"osd":3,"last update":"Sat Sep 30 14:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.56200000000000006,"15min":0.53700000000000003},"min":{"1min":0.39900000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.58899999999999997,"15min":0.56899999999999995},"min":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.92200000000000004}]},{"osd":4,"last update":"Sat Sep 30 14:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.443,"15min":0.42999999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.46800000000000003,"15min":0.45200000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.45400000000000001}]},{"osd":5,"last update":"Sat Sep 30 14:36:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.621}]},{"osd":6,"last update":"Sat Sep 30 14:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.755}]},{"osd":7,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.55700000000000005,"15min":0.54000000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.50600000000000001}]},{"osd":8,"last update":"Sat Sep 30 14:36:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.72099999999999997}]},{"osd":9,"last update":"Sat Sep 30 14:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.90400000000000003}]},{"osd":10,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.94899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":426147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 14:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.54600000000000004,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79400000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.37},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.51200000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.83099999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.44}]},{"osd":2,"last update":"Sat Sep 30 14:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.38800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.39200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.51300000000000001}]},{"osd":3,"last update":"Sat Sep 30 14:35:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.432},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.498,"5min":0.46100000000000002,"15min":0.45500000000000002},"min":{"1min":0.27700000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.625}]},{"osd":4,"last update":"Sat Sep 30 14:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.58599999999999997},"min":{"1min":0.40200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.60499999999999998,"15min":0.57699999999999996},"min":{"1min":0.47499999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.42099999999999999}]},{"osd":5,"last update":"Sat Sep 30 14:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.56399999999999995,"15min":0.54900000000000004},"min":{"1min":0.46300000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60399999999999998,"15min":0.59499999999999997},"min":{"1min":0.46999999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.67400000000000004}]},{"osd":6,"last update":"Sat Sep 30 14:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.495},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.318,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64200000000000002}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69699999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.014,"5min":1.147,"15min":1.147},"last":0.442},{"interface":"front","average":{"1min":0.748,"5min":0.67800000000000005,"15min":0.66600000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56799999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:36:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.69899999999999995}]},{"osd":9,"last update":"Sat Sep 30 14:36:51 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.68899999999999995}]},{"osd":10,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.73699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411379,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":462362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79500000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.38,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70999999999999996,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.35299999999999998}]},{"osd":1,"last update":"Sat Sep 30 14:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.51500000000000001},"min":{"1min":0.40000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.35999999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.76400000000000001}]},{"osd":3,"last update":"Sat Sep 30 14:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.40500000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82699999999999996,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.45700000000000002}]},{"osd":4,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.54400000000000004},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.432,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85899999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.74099999999999999}]},{"osd":5,"last update":"Sat Sep 30 14:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.46200000000000002,"15min":0.45400000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.53000000000000003}]},{"osd":6,"last update":"Sat Sep 30 14:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.65700000000000003,"15min":0.64600000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.60799999999999998}]},{"osd":7,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.45500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.80100000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70699999999999996,"15min":0.71299999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.86399999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.59199999999999997}]},{"osd":8,"last update":"Sat Sep 30 14:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.623}]},{"osd":9,"last update":"Sat Sep 30 14:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55200000000000005}]},{"osd":10,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.81100000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149742,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":424953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.375,"15min":0.35399999999999998},"min":{"1min":0.28199999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.308},{"interface":"front","average":{"1min":0.496,"5min":0.43099999999999999,"15min":0.41199999999999998},"min":{"1min":0.32300000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.374}]},{"osd":1,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.63200000000000001,"15min":0.58199999999999996},"min":{"1min":0.434,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":5.798,"5min":5.798,"15min":5.798},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.59999999999999998,"15min":0.56899999999999995},"min":{"1min":0.371,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":2.7530000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.57799999999999996}]},{"osd":2,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.46600000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.41799999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.64600000000000002,"15min":0.57899999999999996},"min":{"1min":0.36699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":7.1619999999999999,"5min":7.1619999999999999,"15min":7.1619999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.61199999999999999,"15min":0.57399999999999995},"min":{"1min":0.45500000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.67400000000000004}]},{"osd":5,"last update":"Sat Sep 30 14:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.49399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98499999999999999,"5min":1.071,"15min":1.071},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.55700000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91100000000000003,"5min":1.129,"15min":1.129},"last":0.70199999999999996}]},{"osd":6,"last update":"Sat Sep 30 14:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.621,"15min":0.61499999999999999},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90300000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.46100000000000002}]},{"osd":7,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.72299999999999998,"15min":0.70699999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.745}]},{"osd":8,"last update":"Sat Sep 30 14:36:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.46100000000000002}]},{"osd":9,"last update":"Sat Sep 30 14:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.42999999999999999}]},{"osd":10,"last update":"Sat Sep 30 14:37:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":7.2389999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":0.78000000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]}]},{"osd":4,"up_from":33,"seq":141733920809,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":425644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67100000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.65000000000000002}]},{"osd":1,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.377,"15min":0.36899999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.377,"15min":0.36299999999999999},"min":{"1min":0.33700000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.45700000000000002}]},{"osd":2,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.54800000000000004,"15min":0.51600000000000001},"min":{"1min":0.41299999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.374,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55400000000000005}]},{"osd":3,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.504}]},{"osd":5,"last update":"Sat Sep 30 14:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.68000000000000005}]},{"osd":6,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63400000000000001,"15min":0.628},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.81799999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60299999999999998}]},{"osd":7,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54500000000000004,"15min":0.53400000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":0.63700000000000001}]},{"osd":8,"last update":"Sat Sep 30 14:36:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.83799999999999997}]},{"osd":9,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.76300000000000001}]},{"osd":10,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.85099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757285,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1507328,"data_stored":1016513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.39700000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.371,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.373}]},{"osd":1,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.503,"15min":0.48799999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40799999999999997}]},{"osd":2,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41699999999999998,"15min":0.40999999999999998},"min":{"1min":0.28799999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.308},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41599999999999998},"min":{"1min":0.28799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.625,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.33600000000000002}]},{"osd":3,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.45900000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.47699999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58599999999999997}]},{"osd":4,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.55200000000000005}]},{"osd":6,"last update":"Sat Sep 30 14:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.61599999999999999,"15min":0.60799999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.498}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67600000000000005,"15min":0.67800000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.69299999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.68100000000000005}]},{"osd":9,"last update":"Sat Sep 30 14:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.71499999999999997}]},{"osd":10,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":36285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":37635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T14:37:20.179 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-30T14:37:20.179 INFO:tasks.cephadm:Setting up client nodes... 2023-09-30T14:37:20.179 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-09-30T14:37:20.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:20 smithi148 ceph-mon[102355]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:37:20.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:20 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3222870387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:20.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:20 smithi032 ceph-mon[98592]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:37:20.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:20 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3222870387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:20.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:20 smithi156 ceph-mon[103579]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:37:20.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:20 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3222870387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:20.768 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:21.300 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:21 smithi032 ceph-mon[98592]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:37:21.301 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:21 smithi032 ceph-mon[98592]: pgmap v257: 97 pgs: 97 active+clean; 582 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.6 KiB/s wr, 16 op/s 2023-09-30T14:37:21.301 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:21 smithi032 ceph-mon[98592]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:37:21.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:21 smithi148 ceph-mon[102355]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:37:21.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:21 smithi148 ceph-mon[102355]: pgmap v257: 97 pgs: 97 active+clean; 582 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.6 KiB/s wr, 16 op/s 2023-09-30T14:37:21.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:21 smithi148 ceph-mon[102355]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:37:21.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:21 smithi156 ceph-mon[103579]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:37:21.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:21 smithi156 ceph-mon[103579]: pgmap v257: 97 pgs: 97 active+clean; 582 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.6 KiB/s wr, 16 op/s 2023-09-30T14:37:21.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:21 smithi156 ceph-mon[103579]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:37:22.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.072+0000 7fca361e9700 1 -- 172.21.15.32:0/3306304878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca300ffe10 msgr2=0x7fca30100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:22.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.072+0000 7fca361e9700 1 --2- 172.21.15.32:0/3306304878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca300ffe10 0x7fca30100230 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fca20009990 tx=0x7fca20009ca0 comp rx=0 tx=0).stop 2023-09-30T14:37:22.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.072+0000 7fca361e9700 1 -- 172.21.15.32:0/3306304878 shutdown_connections 2023-09-30T14:37:22.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.072+0000 7fca361e9700 1 --2- 172.21.15.32:0/3306304878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca30101930 0x7fca30108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:22.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.072+0000 7fca361e9700 1 --2- 172.21.15.32:0/3306304878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca300ffe10 0x7fca30100230 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:22.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.072+0000 7fca361e9700 1 --2- 172.21.15.32:0/3306304878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca30100f70 0x7fca301013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:22.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.072+0000 7fca361e9700 1 -- 172.21.15.32:0/3306304878 >> 172.21.15.32:0/3306304878 conn(0x7fca300fb390 msgr2=0x7fca300fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:22.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.072+0000 7fca361e9700 1 -- 172.21.15.32:0/3306304878 shutdown_connections 2023-09-30T14:37:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.072+0000 7fca361e9700 1 -- 172.21.15.32:0/3306304878 wait complete. 2023-09-30T14:37:22.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.073+0000 7fca361e9700 1 Processor -- start 2023-09-30T14:37:22.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca361e9700 1 -- start start 2023-09-30T14:37:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca361e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca300ffe10 0x7fca30196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca361e9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca30100f70 0x7fca30197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca361e9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca30101930 0x7fca3019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca361e9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3019e6d0 con 0x7fca300ffe10 2023-09-30T14:37:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca361e9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3019e810 con 0x7fca30100f70 2023-09-30T14:37:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca361e9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3019e9a0 con 0x7fca30101930 2023-09-30T14:37:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca2f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca300ffe10 0x7fca30196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca2f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca300ffe10 0x7fca30196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33972/0 (socket says 172.21.15.32:33972) 2023-09-30T14:37:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca2f7fe700 1 -- 172.21.15.32:0/1178831428 learned_addr learned my addr 172.21.15.32:0/1178831428 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca26ffd700 1 --2- 172.21.15.32:0/1178831428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca30100f70 0x7fca30197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca2ffff700 1 --2- 172.21.15.32:0/1178831428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca30101930 0x7fca3019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca2f7fe700 1 -- 172.21.15.32:0/1178831428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca30101930 msgr2=0x7fca3019e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca2f7fe700 1 --2- 172.21.15.32:0/1178831428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca30101930 0x7fca3019e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca2f7fe700 1 -- 172.21.15.32:0/1178831428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca30100f70 msgr2=0x7fca30197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca2f7fe700 1 --2- 172.21.15.32:0/1178831428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca30100f70 0x7fca30197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca2f7fe700 1 -- 172.21.15.32:0/1178831428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca200095f0 con 0x7fca300ffe10 2023-09-30T14:37:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.074+0000 7fca2ffff700 1 --2- 172.21.15.32:0/1178831428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca30101930 0x7fca3019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:37:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.075+0000 7fca2f7fe700 1 --2- 172.21.15.32:0/1178831428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca300ffe10 0x7fca30196b40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fca200057a0 tx=0x7fca200057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.075+0000 7fca2d7fa700 1 -- 172.21.15.32:0/1178831428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2001d070 con 0x7fca300ffe10 2023-09-30T14:37:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.075+0000 7fca2d7fa700 1 -- 172.21.15.32:0/1178831428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca20004060 con 0x7fca300ffe10 2023-09-30T14:37:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.075+0000 7fca361e9700 1 -- 172.21.15.32:0/1178831428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3019ec20 con 0x7fca300ffe10 2023-09-30T14:37:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.075+0000 7fca2d7fa700 1 -- 172.21.15.32:0/1178831428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20022e30 con 0x7fca300ffe10 2023-09-30T14:37:22.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.075+0000 7fca361e9700 1 -- 172.21.15.32:0/1178831428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3019f090 con 0x7fca300ffe10 2023-09-30T14:37:22.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.076+0000 7fca361e9700 1 -- 172.21.15.32:0/1178831428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3010b620 con 0x7fca300ffe10 2023-09-30T14:37:22.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.077+0000 7fca2d7fa700 1 -- 172.21.15.32:0/1178831428 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fca20003ba0 con 0x7fca300ffe10 2023-09-30T14:37:22.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.078+0000 7fca2d7fa700 1 --2- 172.21.15.32:0/1178831428 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fca1c0a10b0 0x7fca1c0a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:22.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.078+0000 7fca26ffd700 1 --2- 172.21.15.32:0/1178831428 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fca1c0a10b0 0x7fca1c0a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:22.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.078+0000 7fca2d7fa700 1 -- 172.21.15.32:0/1178831428 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fca200cd9a0 con 0x7fca300ffe10 2023-09-30T14:37:22.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.078+0000 7fca26ffd700 1 --2- 172.21.15.32:0/1178831428 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fca1c0a10b0 0x7fca1c0a3570 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca18005950 tx=0x7fca1800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:22.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.082+0000 7fca2d7fa700 1 -- 172.21.15.32:0/1178831428 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fca20097280 con 0x7fca300ffe10 2023-09-30T14:37:22.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.365+0000 7fca361e9700 1 -- 172.21.15.32:0/1178831428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fca3010b860 con 0x7fca300ffe10 2023-09-30T14:37:22.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.367+0000 7fca2d7fa700 1 -- 172.21.15.32:0/1178831428 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fca20027050 con 0x7fca300ffe10 2023-09-30T14:37:22.367 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2023-09-30T14:37:22.367 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCiMhhlzbvSFRAANUVRRA/j8t1wQO5HuEvTFg== 2023-09-30T14:37:22.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.369+0000 7fca361e9700 1 -- 172.21.15.32:0/1178831428 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fca1c0a10b0 msgr2=0x7fca1c0a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:22.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.369+0000 7fca361e9700 1 --2- 172.21.15.32:0/1178831428 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fca1c0a10b0 0x7fca1c0a3570 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca18005950 tx=0x7fca1800b410 comp rx=0 tx=0).stop 2023-09-30T14:37:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.369+0000 7fca361e9700 1 -- 172.21.15.32:0/1178831428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca300ffe10 msgr2=0x7fca30196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.369+0000 7fca361e9700 1 --2- 172.21.15.32:0/1178831428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca300ffe10 0x7fca30196b40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fca200057a0 tx=0x7fca200057d0 comp rx=0 tx=0).stop 2023-09-30T14:37:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.370+0000 7fca361e9700 1 -- 172.21.15.32:0/1178831428 shutdown_connections 2023-09-30T14:37:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.370+0000 7fca361e9700 1 --2- 172.21.15.32:0/1178831428 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fca1c0a10b0 0x7fca1c0a3570 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.370+0000 7fca361e9700 1 --2- 172.21.15.32:0/1178831428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca30101930 0x7fca3019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.370+0000 7fca361e9700 1 --2- 172.21.15.32:0/1178831428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca300ffe10 0x7fca30196b40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.370+0000 7fca361e9700 1 --2- 172.21.15.32:0/1178831428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca30100f70 0x7fca30197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.370+0000 7fca361e9700 1 -- 172.21.15.32:0/1178831428 >> 172.21.15.32:0/1178831428 conn(0x7fca300fb390 msgr2=0x7fca300fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.370+0000 7fca361e9700 1 -- 172.21.15.32:0/1178831428 shutdown_connections 2023-09-30T14:37:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:22.370+0000 7fca361e9700 1 -- 172.21.15.32:0/1178831428 wait complete. 2023-09-30T14:37:22.829 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:37:22.830 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-09-30T14:37:22.830 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-30T14:37:22.871 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-30T14:37:22.871 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-09-30T14:37:22.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph mgr dump --format=json 2023-09-30T14:37:23.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:23 smithi148 ceph-mon[102355]: pgmap v258: 97 pgs: 97 active+clean; 616 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2023-09-30T14:37:23.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:23 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1178831428' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-30T14:37:23.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:23 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1178831428' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-30T14:37:23.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:23 smithi032 ceph-mon[98592]: pgmap v258: 97 pgs: 97 active+clean; 616 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2023-09-30T14:37:23.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:23 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1178831428' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-30T14:37:23.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:23 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1178831428' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-30T14:37:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:23.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:23 smithi156 ceph-mon[103579]: pgmap v258: 97 pgs: 97 active+clean; 616 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2023-09-30T14:37:23.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:23 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1178831428' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-30T14:37:23.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:23 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1178831428' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-30T14:37:24.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.830+0000 7f4cfc437700 1 -- 172.21.15.32:0/2258987176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf40ffdd0 msgr2=0x7f4cf4100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:24.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.830+0000 7f4cfc437700 1 --2- 172.21.15.32:0/2258987176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf40ffdd0 0x7f4cf4100250 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40099c0 tx=0x7f4ce4009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:24.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.830+0000 7f4cfc437700 1 -- 172.21.15.32:0/2258987176 shutdown_connections 2023-09-30T14:37:24.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.830+0000 7f4cfc437700 1 --2- 172.21.15.32:0/2258987176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cf4100790 0x7f4cf410cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:24.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.830+0000 7f4cfc437700 1 --2- 172.21.15.32:0/2258987176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf4105de0 0x7f4cf41061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:24.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.830+0000 7f4cfc437700 1 --2- 172.21.15.32:0/2258987176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf40ffdd0 0x7f4cf4100250 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:24.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.830+0000 7f4cfc437700 1 -- 172.21.15.32:0/2258987176 >> 172.21.15.32:0/2258987176 conn(0x7f4cf4075d70 msgr2=0x7f4cf4076180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:24.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.830+0000 7f4cfc437700 1 -- 172.21.15.32:0/2258987176 shutdown_connections 2023-09-30T14:37:24.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.830+0000 7f4cfc437700 1 -- 172.21.15.32:0/2258987176 wait complete. 2023-09-30T14:37:24.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.830+0000 7f4cfc437700 1 Processor -- start 2023-09-30T14:37:24.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.830+0000 7f4cfc437700 1 -- start start 2023-09-30T14:37:24.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cfc437700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cf40ffdd0 0x7f4cf4197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:24.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cfc437700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf4100790 0x7f4cf4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:24.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cfc437700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf4105de0 0x7f4cf41a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:24.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cfc437700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf4198ba0 con 0x7f4cf4105de0 2023-09-30T14:37:24.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cfc437700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf41a33a0 con 0x7f4cf4100790 2023-09-30T14:37:24.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cfc437700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf41a34e0 con 0x7f4cf40ffdd0 2023-09-30T14:37:24.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cfa9d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf4105de0 0x7f4cf41a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:24.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cfa9d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf4105de0 0x7f4cf41a2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33988/0 (socket says 172.21.15.32:33988) 2023-09-30T14:37:24.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cfa9d4700 1 -- 172.21.15.32:0/3102507400 learned_addr learned my addr 172.21.15.32:0/3102507400 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:24.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cf99d2700 1 --2- 172.21.15.32:0/3102507400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf4100790 0x7f4cf4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:24.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cfa1d3700 1 --2- 172.21.15.32:0/3102507400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cf40ffdd0 0x7f4cf4197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:24.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cfa9d4700 1 -- 172.21.15.32:0/3102507400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cf40ffdd0 msgr2=0x7f4cf4197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:24.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cfa9d4700 1 --2- 172.21.15.32:0/3102507400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cf40ffdd0 0x7f4cf4197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:24.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cfa9d4700 1 -- 172.21.15.32:0/3102507400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf4100790 msgr2=0x7f4cf4198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:24.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cfa9d4700 1 --2- 172.21.15.32:0/3102507400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf4100790 0x7f4cf4198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:24.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cfa9d4700 1 -- 172.21.15.32:0/3102507400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce40095f0 con 0x7f4cf4105de0 2023-09-30T14:37:24.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4cfa9d4700 1 --2- 172.21.15.32:0/3102507400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf4105de0 0x7f4cf41a2e60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4cec007ee0 tx=0x7f4cec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:24.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.831+0000 7f4ceb7fe700 1 -- 172.21.15.32:0/3102507400 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec017070 con 0x7f4cf4105de0 2023-09-30T14:37:24.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.832+0000 7f4cfc437700 1 -- 172.21.15.32:0/3102507400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf41a3770 con 0x7f4cf4105de0 2023-09-30T14:37:24.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.832+0000 7f4ceb7fe700 1 -- 172.21.15.32:0/3102507400 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cec00f040 con 0x7f4cf4105de0 2023-09-30T14:37:24.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.832+0000 7f4ceb7fe700 1 -- 172.21.15.32:0/3102507400 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec005740 con 0x7f4cf4105de0 2023-09-30T14:37:24.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.832+0000 7f4cfc437700 1 -- 172.21.15.32:0/3102507400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf41a3c40 con 0x7f4cf4105de0 2023-09-30T14:37:24.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.832+0000 7f4cfc437700 1 -- 172.21.15.32:0/3102507400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf404eb20 con 0x7f4cf4105de0 2023-09-30T14:37:24.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.834+0000 7f4ceb7fe700 1 -- 172.21.15.32:0/3102507400 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f4cec004c40 con 0x7f4cf4105de0 2023-09-30T14:37:24.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.834+0000 7f4ceb7fe700 1 --2- 172.21.15.32:0/3102507400 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4ce00a10b0 0x7f4ce00a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:24.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.834+0000 7f4cfa1d3700 1 --2- 172.21.15.32:0/3102507400 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4ce00a10b0 0x7f4ce00a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:24.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.835+0000 7f4ceb7fe700 1 -- 172.21.15.32:0/3102507400 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4cec0cc4b0 con 0x7f4cf4105de0 2023-09-30T14:37:24.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.835+0000 7f4cfa1d3700 1 --2- 172.21.15.32:0/3102507400 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4ce00a10b0 0x7f4ce00a3570 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0005950 tx=0x7f4cf00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:24.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:24.838+0000 7f4ceb7fe700 1 -- 172.21.15.32:0/3102507400 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f4cec095d00 con 0x7f4cf4105de0 2023-09-30T14:37:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:25.099+0000 7f4cfc437700 1 -- 172.21.15.32:0/3102507400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f4cf40008d0 con 0x7f4cf4105de0 2023-09-30T14:37:25.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:25.103+0000 7f4ceb7fe700 1 -- 172.21.15.32:0/3102507400 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+259817 (secure 0 0 0) 0x7f4cec099320 con 0x7f4cf4105de0 2023-09-30T14:37:25.107 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:37:25.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:25.106+0000 7f4cfc437700 1 -- 172.21.15.32:0/3102507400 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4ce00a10b0 msgr2=0x7f4ce00a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:25.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:25.106+0000 7f4cfc437700 1 --2- 172.21.15.32:0/3102507400 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4ce00a10b0 0x7f4ce00a3570 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0005950 tx=0x7f4cf00058e0 comp rx=0 tx=0).stop 2023-09-30T14:37:25.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:25.107+0000 7f4cfc437700 1 -- 172.21.15.32:0/3102507400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf4105de0 msgr2=0x7f4cf41a2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:25.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:25.107+0000 7f4cfc437700 1 --2- 172.21.15.32:0/3102507400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf4105de0 0x7f4cf41a2e60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4cec007ee0 tx=0x7f4cec00d3b0 comp rx=0 tx=0).stop 2023-09-30T14:37:25.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:25.107+0000 7f4cfc437700 1 -- 172.21.15.32:0/3102507400 shutdown_connections 2023-09-30T14:37:25.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:25.107+0000 7f4cfc437700 1 --2- 172.21.15.32:0/3102507400 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4ce00a10b0 0x7f4ce00a3570 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:25.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:25.107+0000 7f4cfc437700 1 --2- 172.21.15.32:0/3102507400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf4105de0 0x7f4cf41a2e60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:25.107+0000 7f4cfc437700 1 --2- 172.21.15.32:0/3102507400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cf40ffdd0 0x7f4cf4197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:25.107+0000 7f4cfc437700 1 --2- 172.21.15.32:0/3102507400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf4100790 0x7f4cf4198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:25.107+0000 7f4cfc437700 1 -- 172.21.15.32:0/3102507400 >> 172.21.15.32:0/3102507400 conn(0x7f4cf4075d70 msgr2=0x7f4cf40fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:25.107+0000 7f4cfc437700 1 -- 172.21.15.32:0/3102507400 shutdown_connections 2023-09-30T14:37:25.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:25.107+0000 7f4cfc437700 1 -- 172.21.15.32:0/3102507400 wait complete. 2023-09-30T14:37:25.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:25 smithi148 ceph-mon[102355]: pgmap v259: 97 pgs: 97 active+clean; 616 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2023-09-30T14:37:25.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:25 smithi032 ceph-mon[98592]: pgmap v259: 97 pgs: 97 active+clean; 616 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2023-09-30T14:37:25.520 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":3273633815},{"type":"v1","addr":"172.21.15.32:6801","nonce":3273633815}]},"active_addr":"172.21.15.32:6801/3273633815","active_change":"2023-09-30T14:31:11.693659+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.32:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":3798689053}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1806059555}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":3077947686}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1294075219}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2545368235}]}]} 2023-09-30T14:37:25.526 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-09-30T14:37:25.526 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-09-30T14:37:25.526 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-09-30T14:37:25.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:25 smithi156 ceph-mon[103579]: pgmap v259: 97 pgs: 97 active+clean; 616 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2023-09-30T14:37:26.088 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:26.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:26 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3102507400' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-30T14:37:26.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:26 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3102507400' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-30T14:37:26.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:26 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3102507400' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-30T14:37:27.294 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:27 smithi032 ceph-mon[98592]: pgmap v260: 97 pgs: 97 active+clean; 620 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2023-09-30T14:37:27.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:27 smithi148 ceph-mon[102355]: pgmap v260: 97 pgs: 97 active+clean; 620 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2023-09-30T14:37:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.478+0000 7fa96ab2e700 1 -- 172.21.15.32:0/438419982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa964106ef0 msgr2=0x7fa9641072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.478+0000 7fa96ab2e700 1 --2- 172.21.15.32:0/438419982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa964106ef0 0x7fa9641072d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa958009990 tx=0x7fa958009ca0 comp rx=0 tx=0).stop 2023-09-30T14:37:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.478+0000 7fa96ab2e700 1 -- 172.21.15.32:0/438419982 shutdown_connections 2023-09-30T14:37:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.478+0000 7fa96ab2e700 1 --2- 172.21.15.32:0/438419982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa964069b80 0x7fa96410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.478+0000 7fa96ab2e700 1 --2- 172.21.15.32:0/438419982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa964106ef0 0x7fa9641072d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.478+0000 7fa96ab2e700 1 --2- 172.21.15.32:0/438419982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9640691e0 0x7fa964069640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.478+0000 7fa96ab2e700 1 -- 172.21.15.32:0/438419982 >> 172.21.15.32:0/438419982 conn(0x7fa964076d30 msgr2=0x7fa964077140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.478+0000 7fa96ab2e700 1 -- 172.21.15.32:0/438419982 shutdown_connections 2023-09-30T14:37:27.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.478+0000 7fa96ab2e700 1 -- 172.21.15.32:0/438419982 wait complete. 2023-09-30T14:37:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa96ab2e700 1 Processor -- start 2023-09-30T14:37:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa96ab2e700 1 -- start start 2023-09-30T14:37:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa96ab2e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9640691e0 0x7fa964196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa96ab2e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa964069b80 0x7fa964197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa96ab2e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa964106ef0 0x7fa96419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa96ab2e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96419bfb0 con 0x7fa964069b80 2023-09-30T14:37:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa96ab2e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96419c120 con 0x7fa964106ef0 2023-09-30T14:37:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa96ab2e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96419c2e0 con 0x7fa9640691e0 2023-09-30T14:37:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa963fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa964069b80 0x7fa964197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa9688ca700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9640691e0 0x7fa964196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa9688ca700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9640691e0 0x7fa964196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:48380/0 (socket says 172.21.15.32:48380) 2023-09-30T14:37:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa963fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa964069b80 0x7fa964197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34000/0 (socket says 172.21.15.32:34000) 2023-09-30T14:37:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa9688ca700 1 -- 172.21.15.32:0/2624694752 learned_addr learned my addr 172.21.15.32:0/2624694752 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa9690cb700 1 --2- 172.21.15.32:0/2624694752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa964106ef0 0x7fa96419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa963fff700 1 -- 172.21.15.32:0/2624694752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9640691e0 msgr2=0x7fa964196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa963fff700 1 --2- 172.21.15.32:0/2624694752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9640691e0 0x7fa964196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa963fff700 1 -- 172.21.15.32:0/2624694752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa964106ef0 msgr2=0x7fa96419b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa963fff700 1 --2- 172.21.15.32:0/2624694752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa964106ef0 0x7fa96419b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.479+0000 7fa963fff700 1 -- 172.21.15.32:0/2624694752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9580095f0 con 0x7fa964069b80 2023-09-30T14:37:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.480+0000 7fa963fff700 1 --2- 172.21.15.32:0/2624694752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa964069b80 0x7fa964197530 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa95000c970 tx=0x7fa95000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.480+0000 7fa961ffb700 1 -- 172.21.15.32:0/2624694752 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950004a30 con 0x7fa964069b80 2023-09-30T14:37:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.480+0000 7fa961ffb700 1 -- 172.21.15.32:0/2624694752 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9500054e0 con 0x7fa964069b80 2023-09-30T14:37:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.480+0000 7fa961ffb700 1 -- 172.21.15.32:0/2624694752 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000eb40 con 0x7fa964069b80 2023-09-30T14:37:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.480+0000 7fa96ab2e700 1 -- 172.21.15.32:0/2624694752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96419c5c0 con 0x7fa964069b80 2023-09-30T14:37:27.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.480+0000 7fa96ab2e700 1 -- 172.21.15.32:0/2624694752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9641a3e20 con 0x7fa964069b80 2023-09-30T14:37:27.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.481+0000 7fa96ab2e700 1 -- 172.21.15.32:0/2624694752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96404eb20 con 0x7fa964069b80 2023-09-30T14:37:27.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.482+0000 7fa961ffb700 1 -- 172.21.15.32:0/2624694752 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fa95000eca0 con 0x7fa964069b80 2023-09-30T14:37:27.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.482+0000 7fa961ffb700 1 --2- 172.21.15.32:0/2624694752 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa9540a1030 0x7fa9540a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:27.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.483+0000 7fa9688ca700 1 --2- 172.21.15.32:0/2624694752 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa9540a1030 0x7fa9540a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:27.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.483+0000 7fa961ffb700 1 -- 172.21.15.32:0/2624694752 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa9500cc4b0 con 0x7fa964069b80 2023-09-30T14:37:27.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.483+0000 7fa9688ca700 1 --2- 172.21.15.32:0/2624694752 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa9540a1030 0x7fa9540a34f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa958006010 tx=0x7fa958005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:27.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.487+0000 7fa961ffb700 1 -- 172.21.15.32:0/2624694752 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fa950095d00 con 0x7fa964069b80 2023-09-30T14:37:27.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:27 smithi156 ceph-mon[103579]: pgmap v260: 97 pgs: 97 active+clean; 620 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2023-09-30T14:37:27.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.686+0000 7fa96ab2e700 1 -- 172.21.15.32:0/2624694752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa964110e70 con 0x7fa964069b80 2023-09-30T14:37:27.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.687+0000 7fa961ffb700 1 -- 172.21.15.32:0/2624694752 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22133 (secure 0 0 0) 0x7fa950099320 con 0x7fa964069b80 2023-09-30T14:37:27.688 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:37:27.688 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":83,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:37:12.972217+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:37:27.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.689+0000 7fa96ab2e700 1 -- 172.21.15.32:0/2624694752 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa9540a1030 msgr2=0x7fa9540a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:27.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.689+0000 7fa96ab2e700 1 --2- 172.21.15.32:0/2624694752 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa9540a1030 0x7fa9540a34f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa958006010 tx=0x7fa958005960 comp rx=0 tx=0).stop 2023-09-30T14:37:27.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.689+0000 7fa96ab2e700 1 -- 172.21.15.32:0/2624694752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa964069b80 msgr2=0x7fa964197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:27.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.689+0000 7fa96ab2e700 1 --2- 172.21.15.32:0/2624694752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa964069b80 0x7fa964197530 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa95000c970 tx=0x7fa95000cc80 comp rx=0 tx=0).stop 2023-09-30T14:37:27.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.689+0000 7fa96ab2e700 1 -- 172.21.15.32:0/2624694752 shutdown_connections 2023-09-30T14:37:27.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.689+0000 7fa96ab2e700 1 --2- 172.21.15.32:0/2624694752 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa9540a1030 0x7fa9540a34f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:27.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.689+0000 7fa96ab2e700 1 --2- 172.21.15.32:0/2624694752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa964106ef0 0x7fa96419b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:27.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.689+0000 7fa96ab2e700 1 --2- 172.21.15.32:0/2624694752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9640691e0 0x7fa964196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:27.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.689+0000 7fa96ab2e700 1 --2- 172.21.15.32:0/2624694752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa964069b80 0x7fa964197530 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:27.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.689+0000 7fa96ab2e700 1 -- 172.21.15.32:0/2624694752 >> 172.21.15.32:0/2624694752 conn(0x7fa964076d30 msgr2=0x7fa9640fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:27.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.689+0000 7fa96ab2e700 1 -- 172.21.15.32:0/2624694752 shutdown_connections 2023-09-30T14:37:27.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:27.690+0000 7fa96ab2e700 1 -- 172.21.15.32:0/2624694752 wait complete. 2023-09-30T14:37:28.039 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-09-30T14:37:28.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-09-30T14:37:28.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:28 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2624694752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:28.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:28 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2624694752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:28.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:28 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2624694752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:28.605 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:29.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:29 smithi148 ceph-mon[102355]: pgmap v261: 97 pgs: 97 active+clean; 620 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2023-09-30T14:37:29.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:29 smithi032 ceph-mon[98592]: pgmap v261: 97 pgs: 97 active+clean; 620 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2023-09-30T14:37:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:29 smithi156 ceph-mon[103579]: pgmap v261: 97 pgs: 97 active+clean; 620 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.5 KiB/s wr, 13 op/s 2023-09-30T14:37:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.703+0000 7f3229b10700 1 -- 172.21.15.32:0/2301573310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32240ffd80 msgr2=0x7f3224100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.703+0000 7f3229b10700 1 --2- 172.21.15.32:0/2301573310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32240ffd80 0x7f3224100200 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3218009990 tx=0x7f3218009ca0 comp rx=0 tx=0).stop 2023-09-30T14:37:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.704+0000 7f3229b10700 1 -- 172.21.15.32:0/2301573310 shutdown_connections 2023-09-30T14:37:29.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.704+0000 7f3229b10700 1 --2- 172.21.15.32:0/2301573310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3224100740 0x7f322410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.704+0000 7f3229b10700 1 --2- 172.21.15.32:0/2301573310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3224105e30 0x7f3224106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.704+0000 7f3229b10700 1 --2- 172.21.15.32:0/2301573310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32240ffd80 0x7f3224100200 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.704+0000 7f3229b10700 1 -- 172.21.15.32:0/2301573310 >> 172.21.15.32:0/2301573310 conn(0x7f3224076d30 msgr2=0x7f3224077140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.704+0000 7f3229b10700 1 -- 172.21.15.32:0/2301573310 shutdown_connections 2023-09-30T14:37:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.704+0000 7f3229b10700 1 -- 172.21.15.32:0/2301573310 wait complete. 2023-09-30T14:37:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.704+0000 7f3229b10700 1 Processor -- start 2023-09-30T14:37:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3229b10700 1 -- start start 2023-09-30T14:37:29.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3229b10700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32240ffd80 0x7f3224195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:29.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3229b10700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3224100740 0x7f32241962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:29.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3229b10700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3224105e30 0x7f322419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:29.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3229b10700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322419ad50 con 0x7f3224100740 2023-09-30T14:37:29.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3229b10700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322419aec0 con 0x7f3224105e30 2023-09-30T14:37:29.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3229b10700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322419b080 con 0x7f32240ffd80 2023-09-30T14:37:29.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3222ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3224100740 0x7f32241962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:29.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3222ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3224100740 0x7f32241962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34014/0 (socket says 172.21.15.32:34014) 2023-09-30T14:37:29.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3222ffd700 1 -- 172.21.15.32:0/2509982838 learned_addr learned my addr 172.21.15.32:0/2509982838 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:29.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f32237fe700 1 --2- 172.21.15.32:0/2509982838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32240ffd80 0x7f3224195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:29.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3223fff700 1 --2- 172.21.15.32:0/2509982838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3224105e30 0x7f322419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:29.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3222ffd700 1 -- 172.21.15.32:0/2509982838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32240ffd80 msgr2=0x7f3224195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:29.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3222ffd700 1 --2- 172.21.15.32:0/2509982838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32240ffd80 0x7f3224195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:29.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3222ffd700 1 -- 172.21.15.32:0/2509982838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3224105e30 msgr2=0x7f322419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:29.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3222ffd700 1 --2- 172.21.15.32:0/2509982838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3224105e30 0x7f322419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:29.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3222ffd700 1 -- 172.21.15.32:0/2509982838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32180095f0 con 0x7f3224100740 2023-09-30T14:37:29.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.705+0000 7f3223fff700 1 --2- 172.21.15.32:0/2509982838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3224105e30 0x7f322419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:37:29.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.706+0000 7f3222ffd700 1 --2- 172.21.15.32:0/2509982838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3224100740 0x7f32241962d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f32180095c0 tx=0x7f321800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:29.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.706+0000 7f3220ff9700 1 -- 172.21.15.32:0/2509982838 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321801d070 con 0x7f3224100740 2023-09-30T14:37:29.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.706+0000 7f3229b10700 1 -- 172.21.15.32:0/2509982838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322419b300 con 0x7f3224100740 2023-09-30T14:37:29.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.706+0000 7f3220ff9700 1 -- 172.21.15.32:0/2509982838 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f321800fb50 con 0x7f3224100740 2023-09-30T14:37:29.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.706+0000 7f3220ff9700 1 -- 172.21.15.32:0/2509982838 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32180173f0 con 0x7f3224100740 2023-09-30T14:37:29.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.706+0000 7f3229b10700 1 -- 172.21.15.32:0/2509982838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32240691a0 con 0x7f3224100740 2023-09-30T14:37:29.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.707+0000 7f3229b10700 1 -- 172.21.15.32:0/2509982838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322404eb20 con 0x7f3224100740 2023-09-30T14:37:29.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.708+0000 7f3220ff9700 1 -- 172.21.15.32:0/2509982838 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f321800fcc0 con 0x7f3224100740 2023-09-30T14:37:29.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.708+0000 7f3220ff9700 1 --2- 172.21.15.32:0/2509982838 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f320c0a1030 0x7f320c0a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:29.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.709+0000 7f3220ff9700 1 -- 172.21.15.32:0/2509982838 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f32180cda20 con 0x7f3224100740 2023-09-30T14:37:29.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.709+0000 7f32237fe700 1 --2- 172.21.15.32:0/2509982838 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f320c0a1030 0x7f320c0a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:29.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.709+0000 7f32237fe700 1 --2- 172.21.15.32:0/2509982838 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f320c0a1030 0x7f320c0a34f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3214005950 tx=0x7f321400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:29.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.712+0000 7f3220ff9700 1 -- 172.21.15.32:0/2509982838 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f3218097300 con 0x7f3224100740 2023-09-30T14:37:29.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.913+0000 7f3229b10700 1 -- 172.21.15.32:0/2509982838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f322410fe10 con 0x7f3224100740 2023-09-30T14:37:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.914+0000 7f3220ff9700 1 -- 172.21.15.32:0/2509982838 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22133 (secure 0 0 0) 0x7f321809a920 con 0x7f3224100740 2023-09-30T14:37:29.916 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:37:29.916 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":83,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:37:12.972217+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:37:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.917+0000 7f3229b10700 1 -- 172.21.15.32:0/2509982838 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f320c0a1030 msgr2=0x7f320c0a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.917+0000 7f3229b10700 1 --2- 172.21.15.32:0/2509982838 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f320c0a1030 0x7f320c0a34f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3214005950 tx=0x7f321400a300 comp rx=0 tx=0).stop 2023-09-30T14:37:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.917+0000 7f3229b10700 1 -- 172.21.15.32:0/2509982838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3224100740 msgr2=0x7f32241962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.917+0000 7f3229b10700 1 --2- 172.21.15.32:0/2509982838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3224100740 0x7f32241962d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f32180095c0 tx=0x7f321800f710 comp rx=0 tx=0).stop 2023-09-30T14:37:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.917+0000 7f3229b10700 1 -- 172.21.15.32:0/2509982838 shutdown_connections 2023-09-30T14:37:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.917+0000 7f3229b10700 1 --2- 172.21.15.32:0/2509982838 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f320c0a1030 0x7f320c0a34f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.917+0000 7f3229b10700 1 --2- 172.21.15.32:0/2509982838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3224105e30 0x7f322419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.917+0000 7f3229b10700 1 --2- 172.21.15.32:0/2509982838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32240ffd80 0x7f3224195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.917+0000 7f3229b10700 1 --2- 172.21.15.32:0/2509982838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3224100740 0x7f32241962d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:29.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.917+0000 7f3229b10700 1 -- 172.21.15.32:0/2509982838 >> 172.21.15.32:0/2509982838 conn(0x7f3224076d30 msgr2=0x7f32240fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:29.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.917+0000 7f3229b10700 1 -- 172.21.15.32:0/2509982838 shutdown_connections 2023-09-30T14:37:29.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:29.917+0000 7f3229b10700 1 -- 172.21.15.32:0/2509982838 wait complete. 2023-09-30T14:37:30.250 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:30 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2509982838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:30.252 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-09-30T14:37:30.253 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-09-30T14:37:30.253 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-09-30T14:37:30.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-09-30T14:37:30.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-09-30T14:37:30.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-09-30T14:37:30.255 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-09-30T14:37:30.255 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-09-30T14:37:30.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-09-30T14:37:30.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-09-30T14:37:30.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-09-30T14:37:30.257 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-09-30T14:37:30.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:30 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2509982838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:30.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:30 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2509982838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:37:31.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:31 smithi148 ceph-mon[102355]: pgmap v262: 97 pgs: 97 active+clean; 620 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2023-09-30T14:37:31.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:31 smithi032 ceph-mon[98592]: pgmap v262: 97 pgs: 97 active+clean; 620 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2023-09-30T14:37:31.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:31 smithi156 ceph-mon[103579]: pgmap v262: 97 pgs: 97 active+clean; 620 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2023-09-30T14:37:31.780 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:31.921 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:31.950 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:32.376 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:32.545 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:33.280 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:33 smithi032 ceph-mon[98592]: pgmap v263: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2023-09-30T14:37:33.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:33 smithi148 ceph-mon[102355]: pgmap v263: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2023-09-30T14:37:33.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:33 smithi156 ceph-mon[103579]: pgmap v263: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2023-09-30T14:37:34.501 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:35.327 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:35 smithi032 ceph-mon[98592]: pgmap v264: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2023-09-30T14:37:35.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:35 smithi148 ceph-mon[102355]: pgmap v264: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2023-09-30T14:37:35.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.649+0000 7f1b16d79700 1 -- 172.21.15.32:0/1783439340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b10106220 msgr2=0x7f1b1010dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:35.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.649+0000 7f1b16d79700 1 --2- 172.21.15.32:0/1783439340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b10106220 0x7f1b1010dda0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800b0a0 tx=0x7f1b0800b3b0 comp rx=0 tx=0).stop 2023-09-30T14:37:35.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.650+0000 7f1b16d79700 1 -- 172.21.15.32:0/1783439340 shutdown_connections 2023-09-30T14:37:35.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.650+0000 7f1b16d79700 1 --2- 172.21.15.32:0/1783439340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b10106220 0x7f1b1010dda0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:35.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.650+0000 7f1b16d79700 1 --2- 172.21.15.32:0/1783439340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b10068b70 0x7f1b10068f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:35.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.650+0000 7f1b16d79700 1 --2- 172.21.15.32:0/1783439340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b10069520 0x7f1b10105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.650+0000 7f1b16d79700 1 -- 172.21.15.32:0/1783439340 >> 172.21.15.32:0/1783439340 conn(0x7f1b10075d70 msgr2=0x7f1b10076180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.650+0000 7f1b16d79700 1 -- 172.21.15.32:0/1783439340 shutdown_connections 2023-09-30T14:37:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.650+0000 7f1b16d79700 1 -- 172.21.15.32:0/1783439340 wait complete. 2023-09-30T14:37:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.650+0000 7f1b16d79700 1 Processor -- start 2023-09-30T14:37:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.650+0000 7f1b16d79700 1 -- start start 2023-09-30T14:37:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.650+0000 7f1b16d79700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b10068b70 0x7f1b10197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b16d79700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b10069520 0x7f1b101976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b16d79700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b10106220 0x7f1b1019bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b16d79700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1019c150 con 0x7f1b10069520 2023-09-30T14:37:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b16d79700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1019c2c0 con 0x7f1b10106220 2023-09-30T14:37:35.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b16d79700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1019c480 con 0x7f1b10068b70 2023-09-30T14:37:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b0ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b10069520 0x7f1b101976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b0ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b10069520 0x7f1b101976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34024/0 (socket says 172.21.15.32:34024) 2023-09-30T14:37:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b0ffff700 1 -- 172.21.15.32:0/3349954641 learned_addr learned my addr 172.21.15.32:0/3349954641 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b15316700 1 --2- 172.21.15.32:0/3349954641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b10106220 0x7f1b1019bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b14b15700 1 --2- 172.21.15.32:0/3349954641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b10068b70 0x7f1b10197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b0ffff700 1 -- 172.21.15.32:0/3349954641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b10068b70 msgr2=0x7f1b10197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b0ffff700 1 --2- 172.21.15.32:0/3349954641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b10068b70 0x7f1b10197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b0ffff700 1 -- 172.21.15.32:0/3349954641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b10106220 msgr2=0x7f1b1019bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b0ffff700 1 --2- 172.21.15.32:0/3349954641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b10106220 0x7f1b1019bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b0ffff700 1 -- 172.21.15.32:0/3349954641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b08009d00 con 0x7f1b10069520 2023-09-30T14:37:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b0ffff700 1 --2- 172.21.15.32:0/3349954641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b10069520 0x7f1b101976d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00d950 tx=0x7f1afc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:35.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b0dffb700 1 -- 172.21.15.32:0/3349954641 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc009b40 con 0x7f1b10069520 2023-09-30T14:37:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b0dffb700 1 -- 172.21.15.32:0/3349954641 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1afc00de30 con 0x7f1b10069520 2023-09-30T14:37:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b0dffb700 1 -- 172.21.15.32:0/3349954641 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc00f620 con 0x7f1b10069520 2023-09-30T14:37:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.651+0000 7f1b16d79700 1 -- 172.21.15.32:0/3349954641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1019c760 con 0x7f1b10069520 2023-09-30T14:37:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.652+0000 7f1b16d79700 1 -- 172.21.15.32:0/3349954641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b101a3fc0 con 0x7f1b10069520 2023-09-30T14:37:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.652+0000 7f1b16d79700 1 -- 172.21.15.32:0/3349954641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1b10111090 con 0x7f1b10069520 2023-09-30T14:37:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.653+0000 7f1b0dffb700 1 -- 172.21.15.32:0/3349954641 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f1afc010460 con 0x7f1b10069520 2023-09-30T14:37:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.654+0000 7f1b0dffb700 1 --2- 172.21.15.32:0/3349954641 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1b000a1030 0x7f1b000a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.654+0000 7f1b14b15700 1 --2- 172.21.15.32:0/3349954641 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1b000a1030 0x7f1b000a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:35.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.654+0000 7f1b0dffb700 1 -- 172.21.15.32:0/3349954641 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1afc0cc7a0 con 0x7f1b10069520 2023-09-30T14:37:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.654+0000 7f1b0dffb700 1 --2- 172.21.15.32:0/3349954641 >> [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463] conn(0x7f1b000aa810 0x7f1b000acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.654+0000 7f1b0dffb700 1 -- 172.21.15.32:0/3349954641 --> [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b000ad2e0 con 0x7f1b000aa810 2023-09-30T14:37:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.654+0000 7f1b0dffb700 1 -- 172.21.15.32:0/3349954641 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1afc0ccb10 con 0x7f1b10069520 2023-09-30T14:37:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.654+0000 7f1b14b15700 1 --2- 172.21.15.32:0/3349954641 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1b000a1030 0x7f1b000a34f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1b04005950 tx=0x7f1b040058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.655+0000 7f1b15316700 1 --2- 172.21.15.32:0/3349954641 >> [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463] conn(0x7f1b000aa810 0x7f1b000acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.655+0000 7f1b15316700 1 --2- 172.21.15.32:0/3349954641 >> [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463] conn(0x7f1b000aa810 0x7f1b000acc30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.656+0000 7f1b0dffb700 1 -- 172.21.15.32:0/3349954641 <== osd.8 v2:172.21.15.156:6816/2338383463 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1b000ad2e0 con 0x7f1b000aa810 2023-09-30T14:37:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.672+0000 7f1b16d79700 1 -- 172.21.15.32:0/3349954641 --> [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1b1004eb20 con 0x7f1b000aa810 2023-09-30T14:37:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.672+0000 7f1b0dffb700 1 -- 172.21.15.32:0/3349954641 <== osd.8 v2:172.21.15.156:6816/2338383463 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1b1004eb20 con 0x7f1b000aa810 2023-09-30T14:37:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.673+0000 7f1b16d79700 1 -- 172.21.15.32:0/3349954641 >> [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463] conn(0x7f1b000aa810 msgr2=0x7f1b000acc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.673+0000 7f1b16d79700 1 --2- 172.21.15.32:0/3349954641 >> [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463] conn(0x7f1b000aa810 0x7f1b000acc30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.673+0000 7f1b16d79700 1 -- 172.21.15.32:0/3349954641 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1b000a1030 msgr2=0x7f1b000a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.673+0000 7f1b16d79700 1 --2- 172.21.15.32:0/3349954641 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1b000a1030 0x7f1b000a34f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1b04005950 tx=0x7f1b040058e0 comp rx=0 tx=0).stop 2023-09-30T14:37:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.673+0000 7f1b16d79700 1 -- 172.21.15.32:0/3349954641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b10069520 msgr2=0x7f1b101976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.673+0000 7f1b16d79700 1 --2- 172.21.15.32:0/3349954641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b10069520 0x7f1b101976d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00d950 tx=0x7f1afc00dd10 comp rx=0 tx=0).stop 2023-09-30T14:37:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.673+0000 7f1b15316700 1 -- 172.21.15.32:0/3349954641 reap_dead start 2023-09-30T14:37:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.673+0000 7f1b16d79700 1 -- 172.21.15.32:0/3349954641 shutdown_connections 2023-09-30T14:37:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.673+0000 7f1b16d79700 1 -- 172.21.15.32:0/3349954641 >> 172.21.15.32:0/3349954641 conn(0x7f1b10075d70 msgr2=0x7f1b100fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.673+0000 7f1b16d79700 1 -- 172.21.15.32:0/3349954641 shutdown_connections 2023-09-30T14:37:35.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:34.673+0000 7f1b16d79700 1 -- 172.21.15.32:0/3349954641 wait complete. 2023-09-30T14:37:35.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:35 smithi156 ceph-mon[103579]: pgmap v264: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2023-09-30T14:37:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:36.236 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:37.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.127+0000 7f6bbeaa3700 1 -- 172.21.15.32:0/918206681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb8105de0 msgr2=0x7f6bb81061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:37.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.127+0000 7f6bbeaa3700 1 --2- 172.21.15.32:0/918206681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb8105de0 0x7f6bb81061c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0099c0 tx=0x7f6bac009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:37.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.127+0000 7f6bbeaa3700 1 -- 172.21.15.32:0/918206681 shutdown_connections 2023-09-30T14:37:37.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.127+0000 7f6bbeaa3700 1 --2- 172.21.15.32:0/918206681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bb8100790 0x7f6bb810cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:37.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.127+0000 7f6bbeaa3700 1 --2- 172.21.15.32:0/918206681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb8105de0 0x7f6bb81061c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:37.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.127+0000 7f6bbeaa3700 1 --2- 172.21.15.32:0/918206681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bb80ffdd0 0x7f6bb8100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:37.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.127+0000 7f6bbeaa3700 1 -- 172.21.15.32:0/918206681 >> 172.21.15.32:0/918206681 conn(0x7f6bb8075d70 msgr2=0x7f6bb8076180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:37.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.127+0000 7f6bbeaa3700 1 -- 172.21.15.32:0/918206681 shutdown_connections 2023-09-30T14:37:37.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.127+0000 7f6bbeaa3700 1 -- 172.21.15.32:0/918206681 wait complete. 2023-09-30T14:37:37.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.128+0000 7f6bbeaa3700 1 Processor -- start 2023-09-30T14:37:37.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.128+0000 7f6bbeaa3700 1 -- start start 2023-09-30T14:37:37.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbeaa3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb80ffdd0 0x7f6bb8197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:37.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbeaa3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bb8100790 0x7f6bb81982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:37.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbeaa3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bb8105de0 0x7f6bb81a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:37.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbeaa3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb8198bd0 con 0x7f6bb80ffdd0 2023-09-30T14:37:37.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbeaa3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb81a3340 con 0x7f6bb8100790 2023-09-30T14:37:37.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbeaa3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb81a34b0 con 0x7f6bb8105de0 2023-09-30T14:37:37.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbc83f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb80ffdd0 0x7f6bb8197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:37.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbc83f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb80ffdd0 0x7f6bb8197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34038/0 (socket says 172.21.15.32:34038) 2023-09-30T14:37:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbc83f700 1 -- 172.21.15.32:0/1657473256 learned_addr learned my addr 172.21.15.32:0/1657473256 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bb7fff700 1 --2- 172.21.15.32:0/1657473256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bb8100790 0x7f6bb81982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbd040700 1 --2- 172.21.15.32:0/1657473256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bb8105de0 0x7f6bb81a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbc83f700 1 -- 172.21.15.32:0/1657473256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bb8105de0 msgr2=0x7f6bb81a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbc83f700 1 --2- 172.21.15.32:0/1657473256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bb8105de0 0x7f6bb81a2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbc83f700 1 -- 172.21.15.32:0/1657473256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bb8100790 msgr2=0x7f6bb81982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbc83f700 1 --2- 172.21.15.32:0/1657473256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bb8100790 0x7f6bb81982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbc83f700 1 -- 172.21.15.32:0/1657473256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac0095f0 con 0x7f6bb80ffdd0 2023-09-30T14:37:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bb7fff700 1 --2- 172.21.15.32:0/1657473256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bb8100790 0x7f6bb81982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:37:37.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbc83f700 1 --2- 172.21.15.32:0/1657473256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb80ffdd0 0x7f6bb8197d80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00b580 tx=0x7f6bac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bb5ffb700 1 -- 172.21.15.32:0/1657473256 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac01d070 con 0x7f6bb80ffdd0 2023-09-30T14:37:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bb5ffb700 1 -- 172.21.15.32:0/1657473256 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bac004010 con 0x7f6bb80ffdd0 2023-09-30T14:37:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bbeaa3700 1 -- 172.21.15.32:0/1657473256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb81a3730 con 0x7f6bb80ffdd0 2023-09-30T14:37:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.129+0000 7f6bb5ffb700 1 -- 172.21.15.32:0/1657473256 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac022db0 con 0x7f6bb80ffdd0 2023-09-30T14:37:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.130+0000 7f6bbeaa3700 1 -- 172.21.15.32:0/1657473256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb81a3ba0 con 0x7f6bb80ffdd0 2023-09-30T14:37:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.130+0000 7f6bbeaa3700 1 -- 172.21.15.32:0/1657473256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6bb804eb20 con 0x7f6bb80ffdd0 2023-09-30T14:37:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.132+0000 7f6bb5ffb700 1 -- 172.21.15.32:0/1657473256 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f6bac022470 con 0x7f6bb80ffdd0 2023-09-30T14:37:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.132+0000 7f6bb5ffb700 1 --2- 172.21.15.32:0/1657473256 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6ba80a10b0 0x7f6ba80a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.132+0000 7f6bb7fff700 1 --2- 172.21.15.32:0/1657473256 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6ba80a10b0 0x7f6ba80a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.132+0000 7f6bb5ffb700 1 -- 172.21.15.32:0/1657473256 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6bac0ce5a0 con 0x7f6bb80ffdd0 2023-09-30T14:37:37.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.133+0000 7f6bb5ffb700 1 --2- 172.21.15.32:0/1657473256 >> [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588] conn(0x7f6ba80aa890 0x7f6ba80accb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:37.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.133+0000 7f6bb5ffb700 1 -- 172.21.15.32:0/1657473256 --> [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ba80ad360 con 0x7f6ba80aa890 2023-09-30T14:37:37.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.133+0000 7f6bb5ffb700 1 -- 172.21.15.32:0/1657473256 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bac0cea40 con 0x7f6bb80ffdd0 2023-09-30T14:37:37.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.133+0000 7f6bbd040700 1 --2- 172.21.15.32:0/1657473256 >> [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588] conn(0x7f6ba80aa890 0x7f6ba80accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:37.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.133+0000 7f6bb7fff700 1 --2- 172.21.15.32:0/1657473256 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6ba80a10b0 0x7f6ba80a3570 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4005950 tx=0x7f6ba40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:37.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.133+0000 7f6bbd040700 1 --2- 172.21.15.32:0/1657473256 >> [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588] conn(0x7f6ba80aa890 0x7f6ba80accb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:37.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.134+0000 7f6bb5ffb700 1 -- 172.21.15.32:0/1657473256 <== osd.0 v2:172.21.15.32:6802/3314475588 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f6ba80ad360 con 0x7f6ba80aa890 2023-09-30T14:37:37.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.150+0000 7f6bbeaa3700 1 -- 172.21.15.32:0/1657473256 --> [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6bb8110020 con 0x7f6ba80aa890 2023-09-30T14:37:37.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.150+0000 7f6bb5ffb700 1 -- 172.21.15.32:0/1657473256 <== osd.0 v2:172.21.15.32:6802/3314475588 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6bb0009a70 con 0x7f6ba80aa890 2023-09-30T14:37:37.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.151+0000 7f6bbeaa3700 1 -- 172.21.15.32:0/1657473256 >> [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588] conn(0x7f6ba80aa890 msgr2=0x7f6ba80accb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:37.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.151+0000 7f6bbeaa3700 1 --2- 172.21.15.32:0/1657473256 >> [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588] conn(0x7f6ba80aa890 0x7f6ba80accb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:37.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.151+0000 7f6bbeaa3700 1 -- 172.21.15.32:0/1657473256 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6ba80a10b0 msgr2=0x7f6ba80a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:37.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.151+0000 7f6bbeaa3700 1 --2- 172.21.15.32:0/1657473256 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6ba80a10b0 0x7f6ba80a3570 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4005950 tx=0x7f6ba40058e0 comp rx=0 tx=0).stop 2023-09-30T14:37:37.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.151+0000 7f6bbeaa3700 1 -- 172.21.15.32:0/1657473256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb80ffdd0 msgr2=0x7f6bb8197d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:37.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.151+0000 7f6bbeaa3700 1 --2- 172.21.15.32:0/1657473256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb80ffdd0 0x7f6bb8197d80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00b580 tx=0x7f6bac0057f0 comp rx=0 tx=0).stop 2023-09-30T14:37:37.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.151+0000 7f6bbd040700 1 -- 172.21.15.32:0/1657473256 reap_dead start 2023-09-30T14:37:37.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.151+0000 7f6bbeaa3700 1 -- 172.21.15.32:0/1657473256 shutdown_connections 2023-09-30T14:37:37.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.151+0000 7f6bbeaa3700 1 -- 172.21.15.32:0/1657473256 >> 172.21.15.32:0/1657473256 conn(0x7f6bb8075d70 msgr2=0x7f6bb8103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:37.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.151+0000 7f6bbeaa3700 1 -- 172.21.15.32:0/1657473256 shutdown_connections 2023-09-30T14:37:37.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:37.151+0000 7f6bbeaa3700 1 -- 172.21.15.32:0/1657473256 wait complete. 2023-09-30T14:37:37.220 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:37 smithi032 ceph-mon[98592]: pgmap v265: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2023-09-30T14:37:37.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:37 smithi148 ceph-mon[102355]: pgmap v265: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2023-09-30T14:37:37.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:37 smithi156 ceph-mon[103579]: pgmap v265: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2023-09-30T14:37:38.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.470+0000 7f878f77c700 1 -- 172.21.15.32:0/1832186282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8788104da0 msgr2=0x7f8788105180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:38.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.470+0000 7f878f77c700 1 --2- 172.21.15.32:0/1832186282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8788104da0 0x7f8788105180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f87840099c0 tx=0x7f8784009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:38.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.470+0000 7f878f77c700 1 -- 172.21.15.32:0/1832186282 shutdown_connections 2023-09-30T14:37:38.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.470+0000 7f878f77c700 1 --2- 172.21.15.32:0/1832186282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8788100e50 0x7f878810cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:38.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.470+0000 7f878f77c700 1 --2- 172.21.15.32:0/1832186282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8788104da0 0x7f8788105180 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:38.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.470+0000 7f878f77c700 1 --2- 172.21.15.32:0/1832186282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8788100490 0x7f8788100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:38.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.470+0000 7f878f77c700 1 -- 172.21.15.32:0/1832186282 >> 172.21.15.32:0/1832186282 conn(0x7f87880762d0 msgr2=0x7f87880fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:38.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.470+0000 7f878f77c700 1 -- 172.21.15.32:0/1832186282 shutdown_connections 2023-09-30T14:37:38.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.470+0000 7f878f77c700 1 -- 172.21.15.32:0/1832186282 wait complete. 2023-09-30T14:37:38.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.471+0000 7f878f77c700 1 Processor -- start 2023-09-30T14:37:38.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.471+0000 7f878f77c700 1 -- start start 2023-09-30T14:37:38.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.471+0000 7f878f77c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8788100490 0x7f8788106ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:38.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.471+0000 7f878f77c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8788100e50 0x7f8788107220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:38.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.471+0000 7f878f77c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8788104da0 0x7f87881077a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:38.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.471+0000 7f878f77c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87881a5340 con 0x7f8788104da0 2023-09-30T14:37:38.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.471+0000 7f878f77c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87881a54b0 con 0x7f8788100e50 2023-09-30T14:37:38.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.471+0000 7f878f77c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87881a5670 con 0x7f8788100490 2023-09-30T14:37:38.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.471+0000 7f878dd19700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8788104da0 0x7f87881077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:38.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.471+0000 7f878dd19700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8788104da0 0x7f87881077a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34056/0 (socket says 172.21.15.32:34056) 2023-09-30T14:37:38.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.471+0000 7f878dd19700 1 -- 172.21.15.32:0/3575805189 learned_addr learned my addr 172.21.15.32:0/3575805189 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:38.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.471+0000 7f878cd17700 1 --2- 172.21.15.32:0/3575805189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8788100e50 0x7f8788107220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:38.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.472+0000 7f878d518700 1 --2- 172.21.15.32:0/3575805189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8788100490 0x7f8788106ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:38.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.472+0000 7f878dd19700 1 -- 172.21.15.32:0/3575805189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8788100490 msgr2=0x7f8788106ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:38.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.472+0000 7f878dd19700 1 --2- 172.21.15.32:0/3575805189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8788100490 0x7f8788106ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:38.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.472+0000 7f878dd19700 1 -- 172.21.15.32:0/3575805189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8788100e50 msgr2=0x7f8788107220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:38.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.472+0000 7f878dd19700 1 --2- 172.21.15.32:0/3575805189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8788100e50 0x7f8788107220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:38.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.472+0000 7f878dd19700 1 -- 172.21.15.32:0/3575805189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87840095f0 con 0x7f8788104da0 2023-09-30T14:37:38.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.472+0000 7f878d518700 1 --2- 172.21.15.32:0/3575805189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8788100490 0x7f8788106ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:37:38.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.472+0000 7f878dd19700 1 --2- 172.21.15.32:0/3575805189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8788104da0 0x7f87881077a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8780007ee0 tx=0x7f878000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:38.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.472+0000 7f877e7fc700 1 -- 172.21.15.32:0/3575805189 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878000ddd0 con 0x7f8788104da0 2023-09-30T14:37:38.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.472+0000 7f878f77c700 1 -- 172.21.15.32:0/3575805189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87881a5950 con 0x7f8788104da0 2023-09-30T14:37:38.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.472+0000 7f877e7fc700 1 -- 172.21.15.32:0/3575805189 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f878000f040 con 0x7f8788104da0 2023-09-30T14:37:38.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.472+0000 7f877e7fc700 1 -- 172.21.15.32:0/3575805189 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8780005a40 con 0x7f8788104da0 2023-09-30T14:37:38.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.472+0000 7f878f77c700 1 -- 172.21.15.32:0/3575805189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87881a5e00 con 0x7f8788104da0 2023-09-30T14:37:38.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.473+0000 7f878f77c700 1 -- 172.21.15.32:0/3575805189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f878810ffc0 con 0x7f8788104da0 2023-09-30T14:37:38.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.475+0000 7f877e7fc700 1 -- 172.21.15.32:0/3575805189 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f8780004ad0 con 0x7f8788104da0 2023-09-30T14:37:38.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.476+0000 7f877e7fc700 1 --2- 172.21.15.32:0/3575805189 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f87740a1030 0x7f87740a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:38.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.476+0000 7f877e7fc700 1 -- 172.21.15.32:0/3575805189 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f87800ccb00 con 0x7f8788104da0 2023-09-30T14:37:38.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.476+0000 7f878d518700 1 --2- 172.21.15.32:0/3575805189 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f87740a1030 0x7f87740a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:38.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.476+0000 7f877e7fc700 1 --2- 172.21.15.32:0/3575805189 >> [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507] conn(0x7f87740aa810 0x7f87740acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:38.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.476+0000 7f877e7fc700 1 -- 172.21.15.32:0/3575805189 --> [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87740ad2e0 con 0x7f87740aa810 2023-09-30T14:37:38.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.476+0000 7f877e7fc700 1 -- 172.21.15.32:0/3575805189 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87800ccfa0 con 0x7f8788104da0 2023-09-30T14:37:38.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.476+0000 7f878d518700 1 --2- 172.21.15.32:0/3575805189 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f87740a1030 0x7f87740a34f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8784009fd0 tx=0x7f8784005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:38.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.476+0000 7f878cd17700 1 --2- 172.21.15.32:0/3575805189 >> [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507] conn(0x7f87740aa810 0x7f87740acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:38.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.477+0000 7f878cd17700 1 --2- 172.21.15.32:0/3575805189 >> [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507] conn(0x7f87740aa810 0x7f87740acc30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:38.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.478+0000 7f877e7fc700 1 -- 172.21.15.32:0/3575805189 <== osd.11 v2:172.21.15.156:6824/2619594507 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f87740ad2e0 con 0x7f87740aa810 2023-09-30T14:37:38.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.495+0000 7f878f77c700 1 -- 172.21.15.32:0/3575805189 --> [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f878804eb20 con 0x7f87740aa810 2023-09-30T14:37:38.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.495+0000 7f877e7fc700 1 -- 172.21.15.32:0/3575805189 <== osd.11 v2:172.21.15.156:6824/2619594507 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f878804eb20 con 0x7f87740aa810 2023-09-30T14:37:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.496+0000 7f878f77c700 1 -- 172.21.15.32:0/3575805189 >> [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507] conn(0x7f87740aa810 msgr2=0x7f87740acc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.496+0000 7f878f77c700 1 --2- 172.21.15.32:0/3575805189 >> [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507] conn(0x7f87740aa810 0x7f87740acc30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.496+0000 7f878f77c700 1 -- 172.21.15.32:0/3575805189 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f87740a1030 msgr2=0x7f87740a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.496+0000 7f878f77c700 1 --2- 172.21.15.32:0/3575805189 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f87740a1030 0x7f87740a34f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8784009fd0 tx=0x7f8784005fb0 comp rx=0 tx=0).stop 2023-09-30T14:37:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.496+0000 7f878f77c700 1 -- 172.21.15.32:0/3575805189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8788104da0 msgr2=0x7f87881077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.496+0000 7f878f77c700 1 --2- 172.21.15.32:0/3575805189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8788104da0 0x7f87881077a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8780007ee0 tx=0x7f878000d3b0 comp rx=0 tx=0).stop 2023-09-30T14:37:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.496+0000 7f878dd19700 1 -- 172.21.15.32:0/3575805189 reap_dead start 2023-09-30T14:37:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.496+0000 7f878f77c700 1 -- 172.21.15.32:0/3575805189 shutdown_connections 2023-09-30T14:37:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.496+0000 7f878f77c700 1 -- 172.21.15.32:0/3575805189 >> 172.21.15.32:0/3575805189 conn(0x7f87880762d0 msgr2=0x7f8788102460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.496+0000 7f878f77c700 1 -- 172.21.15.32:0/3575805189 shutdown_connections 2023-09-30T14:37:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:38.496+0000 7f878f77c700 1 -- 172.21.15.32:0/3575805189 wait complete. 2023-09-30T14:37:39.111 INFO:teuthology.orchestra.run.smithi032.stdout:34359738431 2023-09-30T14:37:39.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-09-30T14:37:39.313 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:39 smithi032 ceph-mon[98592]: pgmap v266: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-09-30T14:37:39.515 INFO:teuthology.orchestra.run.smithi032.stdout:249108103196 2023-09-30T14:37:39.515 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-09-30T14:37:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.527+0000 7fa020a7b700 1 -- 172.21.15.32:0/361874627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01c0ffed0 msgr2=0x7fa01c10c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.527+0000 7fa020a7b700 1 --2- 172.21.15.32:0/361874627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01c0ffed0 0x7fa01c10c390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa01400b0d0 tx=0x7fa01400b3e0 comp rx=0 tx=0).stop 2023-09-30T14:37:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.527+0000 7fa020a7b700 1 -- 172.21.15.32:0/361874627 shutdown_connections 2023-09-30T14:37:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.527+0000 7fa020a7b700 1 --2- 172.21.15.32:0/361874627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01c0ffed0 0x7fa01c10c390 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:39.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.527+0000 7fa020a7b700 1 --2- 172.21.15.32:0/361874627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa01c1055c0 0x7fa01c1059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.527+0000 7fa020a7b700 1 --2- 172.21.15.32:0/361874627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa01c0ff510 0x7fa01c0ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.527+0000 7fa020a7b700 1 -- 172.21.15.32:0/361874627 >> 172.21.15.32:0/361874627 conn(0x7fa01c0762d0 msgr2=0x7fa01c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.527+0000 7fa020a7b700 1 -- 172.21.15.32:0/361874627 shutdown_connections 2023-09-30T14:37:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.527+0000 7fa020a7b700 1 -- 172.21.15.32:0/361874627 wait complete. 2023-09-30T14:37:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.528+0000 7fa020a7b700 1 Processor -- start 2023-09-30T14:37:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.528+0000 7fa020a7b700 1 -- start start 2023-09-30T14:37:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa020a7b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa01c0ff510 0x7fa01c194d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa020a7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01c0ffed0 0x7fa01c195290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa020a7b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa01c1055c0 0x7fa01c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa020a7b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c199d10 con 0x7fa01c0ffed0 2023-09-30T14:37:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa020a7b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c199e80 con 0x7fa01c1055c0 2023-09-30T14:37:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa020a7b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c19a040 con 0x7fa01c0ff510 2023-09-30T14:37:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa019d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01c0ffed0 0x7fa01c195290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa019d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01c0ffed0 0x7fa01c195290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34070/0 (socket says 172.21.15.32:34070) 2023-09-30T14:37:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa019d9b700 1 -- 172.21.15.32:0/3425044305 learned_addr learned my addr 172.21.15.32:0/3425044305 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa01a59c700 1 --2- 172.21.15.32:0/3425044305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa01c0ff510 0x7fa01c194d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa01ad9d700 1 --2- 172.21.15.32:0/3425044305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa01c1055c0 0x7fa01c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa019d9b700 1 -- 172.21.15.32:0/3425044305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa01c0ff510 msgr2=0x7fa01c194d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa019d9b700 1 --2- 172.21.15.32:0/3425044305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa01c0ff510 0x7fa01c194d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa019d9b700 1 -- 172.21.15.32:0/3425044305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa01c1055c0 msgr2=0x7fa01c199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa019d9b700 1 --2- 172.21.15.32:0/3425044305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa01c1055c0 0x7fa01c199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa019d9b700 1 -- 172.21.15.32:0/3425044305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa014009d00 con 0x7fa01c0ffed0 2023-09-30T14:37:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa01ad9d700 1 --2- 172.21.15.32:0/3425044305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa01c1055c0 0x7fa01c199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:37:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa019d9b700 1 --2- 172.21.15.32:0/3425044305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01c0ffed0 0x7fa01c195290 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa01000ddc0 tx=0x7fa01000a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.529+0000 7fa00b7fe700 1 -- 172.21.15.32:0/3425044305 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01000ad40 con 0x7fa01c0ffed0 2023-09-30T14:37:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.530+0000 7fa00b7fe700 1 -- 172.21.15.32:0/3425044305 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa01000e910 con 0x7fa01c0ffed0 2023-09-30T14:37:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.530+0000 7fa00b7fe700 1 -- 172.21.15.32:0/3425044305 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0100168d0 con 0x7fa01c0ffed0 2023-09-30T14:37:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.530+0000 7fa020a7b700 1 -- 172.21.15.32:0/3425044305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01c19a320 con 0x7fa01c0ffed0 2023-09-30T14:37:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.530+0000 7fa020a7b700 1 -- 172.21.15.32:0/3425044305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c1a1b80 con 0x7fa01c0ffed0 2023-09-30T14:37:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.531+0000 7fa020a7b700 1 -- 172.21.15.32:0/3425044305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa01c04eb20 con 0x7fa01c0ffed0 2023-09-30T14:37:39.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.532+0000 7fa00b7fe700 1 -- 172.21.15.32:0/3425044305 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fa01000ea80 con 0x7fa01c0ffed0 2023-09-30T14:37:39.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.533+0000 7fa00b7fe700 1 --2- 172.21.15.32:0/3425044305 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa0040a1030 0x7fa0040a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:39.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.533+0000 7fa01a59c700 1 --2- 172.21.15.32:0/3425044305 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa0040a1030 0x7fa0040a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:39.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.533+0000 7fa00b7fe700 1 -- 172.21.15.32:0/3425044305 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa0100cc820 con 0x7fa01c0ffed0 2023-09-30T14:37:39.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.534+0000 7fa00b7fe700 1 --2- 172.21.15.32:0/3425044305 >> [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583] conn(0x7fa0040aa810 0x7fa0040acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:39.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.534+0000 7fa00b7fe700 1 -- 172.21.15.32:0/3425044305 --> [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0040ad2e0 con 0x7fa0040aa810 2023-09-30T14:37:39.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.534+0000 7fa00b7fe700 1 -- 172.21.15.32:0/3425044305 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0100ccb90 con 0x7fa01c0ffed0 2023-09-30T14:37:39.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.534+0000 7fa01ad9d700 1 --2- 172.21.15.32:0/3425044305 >> [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583] conn(0x7fa0040aa810 0x7fa0040acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:39.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.534+0000 7fa01a59c700 1 --2- 172.21.15.32:0/3425044305 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa0040a1030 0x7fa0040a34f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa00c005950 tx=0x7fa00c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:39.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.534+0000 7fa01ad9d700 1 --2- 172.21.15.32:0/3425044305 >> [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583] conn(0x7fa0040aa810 0x7fa0040acc30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:39.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.535+0000 7fa00b7fe700 1 -- 172.21.15.32:0/3425044305 <== osd.3 v2:172.21.15.32:6810/626048583 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fa0040ad2e0 con 0x7fa0040aa810 2023-09-30T14:37:39.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:39 smithi156 ceph-mon[103579]: pgmap v266: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-09-30T14:37:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.555+0000 7fa020a7b700 1 -- 172.21.15.32:0/3425044305 --> [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa01c10ef40 con 0x7fa0040aa810 2023-09-30T14:37:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.556+0000 7fa00b7fe700 1 -- 172.21.15.32:0/3425044305 <== osd.3 v2:172.21.15.32:6810/626048583 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa014009370 con 0x7fa0040aa810 2023-09-30T14:37:39.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.556+0000 7fa020a7b700 1 -- 172.21.15.32:0/3425044305 >> [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583] conn(0x7fa0040aa810 msgr2=0x7fa0040acc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:39.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.556+0000 7fa020a7b700 1 --2- 172.21.15.32:0/3425044305 >> [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583] conn(0x7fa0040aa810 0x7fa0040acc30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:39.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.556+0000 7fa020a7b700 1 -- 172.21.15.32:0/3425044305 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa0040a1030 msgr2=0x7fa0040a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.556+0000 7fa020a7b700 1 --2- 172.21.15.32:0/3425044305 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa0040a1030 0x7fa0040a34f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa00c005950 tx=0x7fa00c00b410 comp rx=0 tx=0).stop 2023-09-30T14:37:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.556+0000 7fa020a7b700 1 -- 172.21.15.32:0/3425044305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01c0ffed0 msgr2=0x7fa01c195290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.556+0000 7fa020a7b700 1 --2- 172.21.15.32:0/3425044305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01c0ffed0 0x7fa01c195290 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa01000ddc0 tx=0x7fa01000a530 comp rx=0 tx=0).stop 2023-09-30T14:37:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.556+0000 7fa01ad9d700 1 -- 172.21.15.32:0/3425044305 reap_dead start 2023-09-30T14:37:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.556+0000 7fa020a7b700 1 -- 172.21.15.32:0/3425044305 shutdown_connections 2023-09-30T14:37:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.556+0000 7fa020a7b700 1 -- 172.21.15.32:0/3425044305 >> 172.21.15.32:0/3425044305 conn(0x7fa01c0762d0 msgr2=0x7fa01c106340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.556+0000 7fa020a7b700 1 -- 172.21.15.32:0/3425044305 shutdown_connections 2023-09-30T14:37:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.556+0000 7fa020a7b700 1 -- 172.21.15.32:0/3425044305 wait complete. 2023-09-30T14:37:39.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:39 smithi148 ceph-mon[102355]: pgmap v266: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-09-30T14:37:40.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.691+0000 7f4443631700 1 -- 172.21.15.32:0/2036824129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f443c10d730 msgr2=0x7f443c10fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.691+0000 7f4443631700 1 --2- 172.21.15.32:0/2036824129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f443c10d730 0x7f443c10fb40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f443400b0a0 tx=0x7f443400b3b0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.691+0000 7f4443631700 1 -- 172.21.15.32:0/2036824129 shutdown_connections 2023-09-30T14:37:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.691+0000 7f4443631700 1 --2- 172.21.15.32:0/2036824129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f443c10d730 0x7f443c10fb40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.691+0000 7f4443631700 1 --2- 172.21.15.32:0/2036824129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f443c0731f0 0x7f443c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.691+0000 7f4443631700 1 --2- 172.21.15.32:0/2036824129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f443c073b10 0x7f443c10d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.691+0000 7f4443631700 1 -- 172.21.15.32:0/2036824129 >> 172.21.15.32:0/2036824129 conn(0x7f443c0fc940 msgr2=0x7f443c0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.692+0000 7f4443631700 1 -- 172.21.15.32:0/2036824129 shutdown_connections 2023-09-30T14:37:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.692+0000 7f4443631700 1 -- 172.21.15.32:0/2036824129 wait complete. 2023-09-30T14:37:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.692+0000 7f4443631700 1 Processor -- start 2023-09-30T14:37:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.692+0000 7f4443631700 1 -- start start 2023-09-30T14:37:40.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.692+0000 7f4443631700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f443c0731f0 0x7f443c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.692+0000 7f4443631700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f443c073b10 0x7f443c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.692+0000 7f4443631700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f443c10d730 0x7f443c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.692+0000 7f4443631700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c19e1d0 con 0x7f443c0731f0 2023-09-30T14:37:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.692+0000 7f4443631700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c19e340 con 0x7f443c073b10 2023-09-30T14:37:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.692+0000 7f4443631700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c19e500 con 0x7f443c10d730 2023-09-30T14:37:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.693+0000 7f44413cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f443c0731f0 0x7f443c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.693+0000 7f44413cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f443c0731f0 0x7f443c199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34084/0 (socket says 172.21.15.32:34084) 2023-09-30T14:37:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.693+0000 7f44413cd700 1 -- 172.21.15.32:0/4109388396 learned_addr learned my addr 172.21.15.32:0/4109388396 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.693+0000 7f4440bcc700 1 --2- 172.21.15.32:0/4109388396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f443c073b10 0x7f443c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.693+0000 7f4441bce700 1 --2- 172.21.15.32:0/4109388396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f443c10d730 0x7f443c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.693+0000 7f44413cd700 1 -- 172.21.15.32:0/4109388396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f443c10d730 msgr2=0x7f443c19dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.693+0000 7f44413cd700 1 --2- 172.21.15.32:0/4109388396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f443c10d730 0x7f443c19dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.693+0000 7f44413cd700 1 -- 172.21.15.32:0/4109388396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f443c073b10 msgr2=0x7f443c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.693+0000 7f44413cd700 1 --2- 172.21.15.32:0/4109388396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f443c073b10 0x7f443c199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.693+0000 7f44413cd700 1 -- 172.21.15.32:0/4109388396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4434009d00 con 0x7f443c0731f0 2023-09-30T14:37:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.693+0000 7f44413cd700 1 --2- 172.21.15.32:0/4109388396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f443c0731f0 0x7f443c199210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f443800d980 tx=0x7f443800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.693+0000 7f44327fc700 1 -- 172.21.15.32:0/4109388396 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438004220 con 0x7f443c0731f0 2023-09-30T14:37:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.693+0000 7f44327fc700 1 -- 172.21.15.32:0/4109388396 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4438007640 con 0x7f443c0731f0 2023-09-30T14:37:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.693+0000 7f44327fc700 1 -- 172.21.15.32:0/4109388396 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438005610 con 0x7f443c0731f0 2023-09-30T14:37:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.693+0000 7f4443631700 1 -- 172.21.15.32:0/4109388396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443c19e7e0 con 0x7f443c0731f0 2023-09-30T14:37:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.693+0000 7f4443631700 1 -- 172.21.15.32:0/4109388396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443c1a6040 con 0x7f443c0731f0 2023-09-30T14:37:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.694+0000 7f4443631700 1 -- 172.21.15.32:0/4109388396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f443c1132e0 con 0x7f443c0731f0 2023-09-30T14:37:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.696+0000 7f44327fc700 1 -- 172.21.15.32:0/4109388396 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f4438004750 con 0x7f443c0731f0 2023-09-30T14:37:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.697+0000 7f44327fc700 1 --2- 172.21.15.32:0/4109388396 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44280a1030 0x7f44280a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.697+0000 7f4440bcc700 1 --2- 172.21.15.32:0/4109388396 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44280a1030 0x7f44280a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.697+0000 7f44327fc700 1 -- 172.21.15.32:0/4109388396 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f44380ccbf0 con 0x7f443c0731f0 2023-09-30T14:37:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.697+0000 7f44327fc700 1 --2- 172.21.15.32:0/4109388396 >> [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565] conn(0x7f44280aa810 0x7f44280acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.697+0000 7f44327fc700 1 -- 172.21.15.32:0/4109388396 --> [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44280ad2e0 con 0x7f44280aa810 2023-09-30T14:37:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.697+0000 7f44327fc700 1 -- 172.21.15.32:0/4109388396 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44380cd090 con 0x7f443c0731f0 2023-09-30T14:37:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.697+0000 7f4440bcc700 1 --2- 172.21.15.32:0/4109388396 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44280a1030 0x7f44280a34f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f443c19a830 tx=0x7f442c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.698+0000 7f4441bce700 1 --2- 172.21.15.32:0/4109388396 >> [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565] conn(0x7f44280aa810 0x7f44280acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.698+0000 7f4441bce700 1 --2- 172.21.15.32:0/4109388396 >> [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565] conn(0x7f44280aa810 0x7f44280acc30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.699+0000 7f44327fc700 1 -- 172.21.15.32:0/4109388396 <== osd.1 v2:172.21.15.148:6800/2368718565 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f44280ad2e0 con 0x7f44280aa810 2023-09-30T14:37:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.718+0000 7f4443631700 1 -- 172.21.15.32:0/4109388396 --> [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f443c04eb20 con 0x7f44280aa810 2023-09-30T14:37:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.718+0000 7f44327fc700 1 -- 172.21.15.32:0/4109388396 <== osd.1 v2:172.21.15.148:6800/2368718565 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f443c04eb20 con 0x7f44280aa810 2023-09-30T14:37:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.719+0000 7f4443631700 1 -- 172.21.15.32:0/4109388396 >> [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565] conn(0x7f44280aa810 msgr2=0x7f44280acc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.719+0000 7f4443631700 1 --2- 172.21.15.32:0/4109388396 >> [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565] conn(0x7f44280aa810 0x7f44280acc30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.719+0000 7f4443631700 1 -- 172.21.15.32:0/4109388396 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44280a1030 msgr2=0x7f44280a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.719+0000 7f4443631700 1 --2- 172.21.15.32:0/4109388396 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44280a1030 0x7f44280a34f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f443c19a830 tx=0x7f442c009260 comp rx=0 tx=0).stop 2023-09-30T14:37:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.719+0000 7f4443631700 1 -- 172.21.15.32:0/4109388396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f443c0731f0 msgr2=0x7f443c199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.719+0000 7f4443631700 1 --2- 172.21.15.32:0/4109388396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f443c0731f0 0x7f443c199210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f443800d980 tx=0x7f443800dc90 comp rx=0 tx=0).stop 2023-09-30T14:37:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.719+0000 7f4441bce700 1 -- 172.21.15.32:0/4109388396 reap_dead start 2023-09-30T14:37:40.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.719+0000 7f4443631700 1 -- 172.21.15.32:0/4109388396 shutdown_connections 2023-09-30T14:37:40.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.719+0000 7f4443631700 1 -- 172.21.15.32:0/4109388396 >> 172.21.15.32:0/4109388396 conn(0x7f443c0fc940 msgr2=0x7f443c105190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:40.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.719+0000 7f4443631700 1 -- 172.21.15.32:0/4109388396 shutdown_connections 2023-09-30T14:37:40.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.719+0000 7f4443631700 1 -- 172.21.15.32:0/4109388396 wait complete. 2023-09-30T14:37:40.064 INFO:teuthology.orchestra.run.smithi032.stdout:335007449104 2023-09-30T14:37:40.064 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-09-30T14:37:40.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.201+0000 7fe0acd8e700 1 -- 172.21.15.32:0/3030547057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8105de0 msgr2=0x7fe0a81061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.201+0000 7fe0acd8e700 1 --2- 172.21.15.32:0/3030547057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8105de0 0x7fe0a81061c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe0980099c0 tx=0x7fe098009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.201+0000 7fe0acd8e700 1 -- 172.21.15.32:0/3030547057 shutdown_connections 2023-09-30T14:37:40.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.201+0000 7fe0acd8e700 1 --2- 172.21.15.32:0/3030547057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0a8100790 0x7fe0a810cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.201+0000 7fe0acd8e700 1 --2- 172.21.15.32:0/3030547057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8105de0 0x7fe0a81061c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.201+0000 7fe0acd8e700 1 --2- 172.21.15.32:0/3030547057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a80ffdd0 0x7fe0a8100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.201+0000 7fe0acd8e700 1 -- 172.21.15.32:0/3030547057 >> 172.21.15.32:0/3030547057 conn(0x7fe0a8075d70 msgr2=0x7fe0a8076180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:40.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.201+0000 7fe0acd8e700 1 -- 172.21.15.32:0/3030547057 shutdown_connections 2023-09-30T14:37:40.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.201+0000 7fe0acd8e700 1 -- 172.21.15.32:0/3030547057 wait complete. 2023-09-30T14:37:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.202+0000 7fe0acd8e700 1 Processor -- start 2023-09-30T14:37:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.202+0000 7fe0acd8e700 1 -- start start 2023-09-30T14:37:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.202+0000 7fe0acd8e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a80ffdd0 0x7fe0a8195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.202+0000 7fe0acd8e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0a8100790 0x7fe0a8196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.202+0000 7fe0acd8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8105de0 0x7fe0a819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.202+0000 7fe0acd8e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a819ad90 con 0x7fe0a8105de0 2023-09-30T14:37:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.202+0000 7fe0acd8e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a819af00 con 0x7fe0a80ffdd0 2023-09-30T14:37:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.202+0000 7fe0acd8e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a819b0c0 con 0x7fe0a8100790 2023-09-30T14:37:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.202+0000 7fe0a6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8105de0 0x7fe0a819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.202+0000 7fe0a6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8105de0 0x7fe0a819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34120/0 (socket says 172.21.15.32:34120) 2023-09-30T14:37:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.202+0000 7fe0a6d9d700 1 -- 172.21.15.32:0/4027909467 learned_addr learned my addr 172.21.15.32:0/4027909467 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.202+0000 7fe0a5d9b700 1 --2- 172.21.15.32:0/4027909467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0a8100790 0x7fe0a8196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.202+0000 7fe0a659c700 1 --2- 172.21.15.32:0/4027909467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a80ffdd0 0x7fe0a8195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.203+0000 7fe0a6d9d700 1 -- 172.21.15.32:0/4027909467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0a8100790 msgr2=0x7fe0a8196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.203+0000 7fe0a6d9d700 1 --2- 172.21.15.32:0/4027909467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0a8100790 0x7fe0a8196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.203+0000 7fe0a6d9d700 1 -- 172.21.15.32:0/4027909467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a80ffdd0 msgr2=0x7fe0a8195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.203+0000 7fe0a6d9d700 1 --2- 172.21.15.32:0/4027909467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a80ffdd0 0x7fe0a8195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.203+0000 7fe0a6d9d700 1 -- 172.21.15.32:0/4027909467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0980095f0 con 0x7fe0a8105de0 2023-09-30T14:37:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.203+0000 7fe0a6d9d700 1 --2- 172.21.15.32:0/4027909467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8105de0 0x7fe0a819a780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000ef30 tx=0x7fe0a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.203+0000 7fe0977fe700 1 -- 172.21.15.32:0/4027909467 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0004bb0 con 0x7fe0a8105de0 2023-09-30T14:37:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.203+0000 7fe0977fe700 1 -- 172.21.15.32:0/4027909467 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0a000f040 con 0x7fe0a8105de0 2023-09-30T14:37:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.203+0000 7fe0977fe700 1 -- 172.21.15.32:0/4027909467 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0004020 con 0x7fe0a8105de0 2023-09-30T14:37:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.203+0000 7fe0acd8e700 1 -- 172.21.15.32:0/4027909467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a819b3a0 con 0x7fe0a8105de0 2023-09-30T14:37:40.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.203+0000 7fe0acd8e700 1 -- 172.21.15.32:0/4027909467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a8068ae0 con 0x7fe0a8105de0 2023-09-30T14:37:40.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.204+0000 7fe0acd8e700 1 -- 172.21.15.32:0/4027909467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe0a804eb20 con 0x7fe0a8105de0 2023-09-30T14:37:40.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.206+0000 7fe0977fe700 1 -- 172.21.15.32:0/4027909467 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fe0a0003a00 con 0x7fe0a8105de0 2023-09-30T14:37:40.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.207+0000 7fe0977fe700 1 --2- 172.21.15.32:0/4027909467 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0900a1030 0x7fe0900a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:40.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.207+0000 7fe0a659c700 1 --2- 172.21.15.32:0/4027909467 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0900a1030 0x7fe0900a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:40.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.207+0000 7fe0977fe700 1 -- 172.21.15.32:0/4027909467 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe0a0016070 con 0x7fe0a8105de0 2023-09-30T14:37:40.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.207+0000 7fe0977fe700 1 --2- 172.21.15.32:0/4027909467 >> [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580] conn(0x7fe0900aa810 0x7fe0900acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:40.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.207+0000 7fe0977fe700 1 -- 172.21.15.32:0/4027909467 --> [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0900ad2e0 con 0x7fe0900aa810 2023-09-30T14:37:40.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.207+0000 7fe0977fe700 1 -- 172.21.15.32:0/4027909467 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0a00ce640 con 0x7fe0a8105de0 2023-09-30T14:37:40.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.207+0000 7fe0a659c700 1 --2- 172.21.15.32:0/4027909467 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0900a1030 0x7fe0900a34f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe098009990 tx=0x7fe098005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:40.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.208+0000 7fe0a5d9b700 1 --2- 172.21.15.32:0/4027909467 >> [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580] conn(0x7fe0900aa810 0x7fe0900acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:40.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.208+0000 7fe0a5d9b700 1 --2- 172.21.15.32:0/4027909467 >> [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580] conn(0x7fe0900aa810 0x7fe0900acc30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:40.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.209+0000 7fe0977fe700 1 -- 172.21.15.32:0/4027909467 <== osd.2 v2:172.21.15.156:6800/2843236580 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe0900ad2e0 con 0x7fe0900aa810 2023-09-30T14:37:40.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.225+0000 7fe0acd8e700 1 -- 172.21.15.32:0/4027909467 --> [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe0a810ffb0 con 0x7fe0900aa810 2023-09-30T14:37:40.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.226+0000 7fe0977fe700 1 -- 172.21.15.32:0/4027909467 <== osd.2 v2:172.21.15.156:6800/2843236580 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe09c009990 con 0x7fe0900aa810 2023-09-30T14:37:40.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.226+0000 7fe0acd8e700 1 -- 172.21.15.32:0/4027909467 >> [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580] conn(0x7fe0900aa810 msgr2=0x7fe0900acc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.226+0000 7fe0acd8e700 1 --2- 172.21.15.32:0/4027909467 >> [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580] conn(0x7fe0900aa810 0x7fe0900acc30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.226+0000 7fe0acd8e700 1 -- 172.21.15.32:0/4027909467 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0900a1030 msgr2=0x7fe0900a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.226+0000 7fe0acd8e700 1 --2- 172.21.15.32:0/4027909467 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0900a1030 0x7fe0900a34f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe098009990 tx=0x7fe098005fb0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.226+0000 7fe0acd8e700 1 -- 172.21.15.32:0/4027909467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8105de0 msgr2=0x7fe0a819a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.226+0000 7fe0acd8e700 1 --2- 172.21.15.32:0/4027909467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0a8105de0 0x7fe0a819a780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000ef30 tx=0x7fe0a000c3a0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.226+0000 7fe0a6d9d700 1 -- 172.21.15.32:0/4027909467 reap_dead start 2023-09-30T14:37:40.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.226+0000 7fe0acd8e700 1 -- 172.21.15.32:0/4027909467 shutdown_connections 2023-09-30T14:37:40.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.226+0000 7fe0acd8e700 1 -- 172.21.15.32:0/4027909467 >> 172.21.15.32:0/4027909467 conn(0x7fe0a8075d70 msgr2=0x7fe0a80fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:40.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.227+0000 7fe0acd8e700 1 -- 172.21.15.32:0/4027909467 shutdown_connections 2023-09-30T14:37:40.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:40.227+0000 7fe0acd8e700 1 -- 172.21.15.32:0/4027909467 wait complete. 2023-09-30T14:37:40.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.807+0000 7f770dc3a700 1 -- 172.21.15.32:0/2980865206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77081018b0 msgr2=0x7f770810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.807+0000 7f770dc3a700 1 --2- 172.21.15.32:0/2980865206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77081018b0 0x7f770810dd20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f770000b0d0 tx=0x7f770000b3e0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.807+0000 7f770dc3a700 1 -- 172.21.15.32:0/2980865206 shutdown_connections 2023-09-30T14:37:40.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.807+0000 7f770dc3a700 1 --2- 172.21.15.32:0/2980865206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77081018b0 0x7f770810dd20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.807+0000 7f770dc3a700 1 --2- 172.21.15.32:0/2980865206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7708106f00 0x7f77081072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.807+0000 7f770dc3a700 1 --2- 172.21.15.32:0/2980865206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7708100ef0 0x7f7708101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.807+0000 7f770dc3a700 1 -- 172.21.15.32:0/2980865206 >> 172.21.15.32:0/2980865206 conn(0x7f7708078010 msgr2=0x7f7708078420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:40.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.807+0000 7f770dc3a700 1 -- 172.21.15.32:0/2980865206 shutdown_connections 2023-09-30T14:37:40.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.807+0000 7f770dc3a700 1 -- 172.21.15.32:0/2980865206 wait complete. 2023-09-30T14:37:40.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.808+0000 7f770dc3a700 1 Processor -- start 2023-09-30T14:37:40.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.808+0000 7f770dc3a700 1 -- start start 2023-09-30T14:37:40.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.808+0000 7f770dc3a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7708100ef0 0x7f7708199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:40.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.808+0000 7f770dc3a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77081018b0 0x7f7708199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:40.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.808+0000 7f770dc3a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7708106f00 0x7f770819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:40.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.808+0000 7f770dc3a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770819e1d0 con 0x7f7708106f00 2023-09-30T14:37:40.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.808+0000 7f770dc3a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770819e340 con 0x7f7708100ef0 2023-09-30T14:37:40.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.808+0000 7f770dc3a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770819e500 con 0x7f77081018b0 2023-09-30T14:37:40.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.808+0000 7f7707fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7708106f00 0x7f770819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:40.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.808+0000 7f7707fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7708106f00 0x7f770819dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34104/0 (socket says 172.21.15.32:34104) 2023-09-30T14:37:40.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.808+0000 7f7707fff700 1 -- 172.21.15.32:0/665524389 learned_addr learned my addr 172.21.15.32:0/665524389 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:40.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.808+0000 7f7706ffd700 1 --2- 172.21.15.32:0/665524389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77081018b0 0x7f7708199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:40.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.808+0000 7f77077fe700 1 --2- 172.21.15.32:0/665524389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7708100ef0 0x7f7708199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:40.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.809+0000 7f7707fff700 1 -- 172.21.15.32:0/665524389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77081018b0 msgr2=0x7f7708199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.809+0000 7f7707fff700 1 --2- 172.21.15.32:0/665524389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77081018b0 0x7f7708199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.809+0000 7f7707fff700 1 -- 172.21.15.32:0/665524389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7708100ef0 msgr2=0x7f7708199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.809+0000 7f7707fff700 1 --2- 172.21.15.32:0/665524389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7708100ef0 0x7f7708199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.809+0000 7f7707fff700 1 -- 172.21.15.32:0/665524389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7700009d00 con 0x7f7708106f00 2023-09-30T14:37:40.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.809+0000 7f7707fff700 1 --2- 172.21.15.32:0/665524389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7708106f00 0x7f770819dbc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7700007870 tx=0x7f77000078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:40.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.809+0000 7f7704ff9700 1 -- 172.21.15.32:0/665524389 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770000e040 con 0x7f7708106f00 2023-09-30T14:37:40.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.809+0000 7f7704ff9700 1 -- 172.21.15.32:0/665524389 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7700003e10 con 0x7f7708106f00 2023-09-30T14:37:40.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.809+0000 7f7704ff9700 1 -- 172.21.15.32:0/665524389 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770001b870 con 0x7f7708106f00 2023-09-30T14:37:40.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.809+0000 7f770dc3a700 1 -- 172.21.15.32:0/665524389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770819e780 con 0x7f7708106f00 2023-09-30T14:37:40.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.809+0000 7f770dc3a700 1 -- 172.21.15.32:0/665524389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77081a5e70 con 0x7f7708106f00 2023-09-30T14:37:40.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.810+0000 7f770dc3a700 1 -- 172.21.15.32:0/665524389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f77081110c0 con 0x7f7708106f00 2023-09-30T14:37:40.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.811+0000 7f7704ff9700 1 -- 172.21.15.32:0/665524389 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f7700005090 con 0x7f7708106f00 2023-09-30T14:37:40.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.812+0000 7f7704ff9700 1 --2- 172.21.15.32:0/665524389 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f76f00a1030 0x7f76f00a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:40.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.812+0000 7f77077fe700 1 --2- 172.21.15.32:0/665524389 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f76f00a1030 0x7f76f00a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:40.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.812+0000 7f7704ff9700 1 -- 172.21.15.32:0/665524389 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f77000cdff0 con 0x7f7708106f00 2023-09-30T14:37:40.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.812+0000 7f7704ff9700 1 --2- 172.21.15.32:0/665524389 >> [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673] conn(0x7f76f00aa810 0x7f76f00acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:40.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.812+0000 7f7704ff9700 1 -- 172.21.15.32:0/665524389 --> [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76f00ad2e0 con 0x7f76f00aa810 2023-09-30T14:37:40.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.812+0000 7f7704ff9700 1 -- 172.21.15.32:0/665524389 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77000ce490 con 0x7f7708106f00 2023-09-30T14:37:40.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.812+0000 7f77077fe700 1 --2- 172.21.15.32:0/665524389 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f76f00a1030 0x7f76f00a34f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f76f8007b60 tx=0x7f76f80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:40.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.812+0000 7f7706ffd700 1 --2- 172.21.15.32:0/665524389 >> [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673] conn(0x7f76f00aa810 0x7f76f00acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:40.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.813+0000 7f7706ffd700 1 --2- 172.21.15.32:0/665524389 >> [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673] conn(0x7f76f00aa810 0x7f76f00acc30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:40.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.813+0000 7f7704ff9700 1 -- 172.21.15.32:0/665524389 <== osd.10 v2:172.21.15.148:6824/2093868673 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f76f00ad2e0 con 0x7f76f00aa810 2023-09-30T14:37:40.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.830+0000 7f770dc3a700 1 -- 172.21.15.32:0/665524389 --> [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f770804eb20 con 0x7f76f00aa810 2023-09-30T14:37:40.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.830+0000 7f7704ff9700 1 -- 172.21.15.32:0/665524389 <== osd.10 v2:172.21.15.148:6824/2093868673 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f770804eb20 con 0x7f76f00aa810 2023-09-30T14:37:40.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.831+0000 7f770dc3a700 1 -- 172.21.15.32:0/665524389 >> [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673] conn(0x7f76f00aa810 msgr2=0x7f76f00acc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.831+0000 7f770dc3a700 1 --2- 172.21.15.32:0/665524389 >> [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673] conn(0x7f76f00aa810 0x7f76f00acc30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.831+0000 7f770dc3a700 1 -- 172.21.15.32:0/665524389 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f76f00a1030 msgr2=0x7f76f00a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.831+0000 7f770dc3a700 1 --2- 172.21.15.32:0/665524389 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f76f00a1030 0x7f76f00a34f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f76f8007b60 tx=0x7f76f80073d0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.831+0000 7f770dc3a700 1 -- 172.21.15.32:0/665524389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7708106f00 msgr2=0x7f770819dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:40.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.831+0000 7f770dc3a700 1 --2- 172.21.15.32:0/665524389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7708106f00 0x7f770819dbc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7700007870 tx=0x7f77000078a0 comp rx=0 tx=0).stop 2023-09-30T14:37:40.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.831+0000 7f7707fff700 1 -- 172.21.15.32:0/665524389 reap_dead start 2023-09-30T14:37:40.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.831+0000 7f770dc3a700 1 -- 172.21.15.32:0/665524389 shutdown_connections 2023-09-30T14:37:40.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.831+0000 7f770dc3a700 1 -- 172.21.15.32:0/665524389 >> 172.21.15.32:0/665524389 conn(0x7f7708078010 msgr2=0x7f770810c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:40.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.831+0000 7f770dc3a700 1 -- 172.21.15.32:0/665524389 shutdown_connections 2023-09-30T14:37:40.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:39.831+0000 7f770dc3a700 1 -- 172.21.15.32:0/665524389 wait complete. 2023-09-30T14:37:40.985 INFO:teuthology.orchestra.run.smithi032.stdout:55834574908 2023-09-30T14:37:40.985 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-09-30T14:37:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.058+0000 7f5626306700 1 -- 172.21.15.32:0/1487660217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56201018b0 msgr2=0x7f562010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.058+0000 7f5626306700 1 --2- 172.21.15.32:0/1487660217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56201018b0 0x7f562010dd20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f561800b0d0 tx=0x7f561800b3e0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.058+0000 7f5626306700 1 -- 172.21.15.32:0/1487660217 shutdown_connections 2023-09-30T14:37:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.058+0000 7f5626306700 1 --2- 172.21.15.32:0/1487660217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56201018b0 0x7f562010dd20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.058+0000 7f5626306700 1 --2- 172.21.15.32:0/1487660217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5620106f00 0x7f56201072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.058+0000 7f5626306700 1 --2- 172.21.15.32:0/1487660217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5620100ef0 0x7f5620101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.058+0000 7f5626306700 1 -- 172.21.15.32:0/1487660217 >> 172.21.15.32:0/1487660217 conn(0x7f5620078010 msgr2=0x7f5620078420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.059+0000 7f5626306700 1 -- 172.21.15.32:0/1487660217 shutdown_connections 2023-09-30T14:37:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.059+0000 7f5626306700 1 -- 172.21.15.32:0/1487660217 wait complete. 2023-09-30T14:37:41.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.059+0000 7f5626306700 1 Processor -- start 2023-09-30T14:37:41.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.059+0000 7f5626306700 1 -- start start 2023-09-30T14:37:41.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.060+0000 7f5626306700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5620100ef0 0x7f5620196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.060+0000 7f5626306700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56201018b0 0x7f56201974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.060+0000 7f5626306700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5620106f00 0x7f562019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.060+0000 7f5626306700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562019bf50 con 0x7f56201018b0 2023-09-30T14:37:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.060+0000 7f5626306700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562019c0c0 con 0x7f5620106f00 2023-09-30T14:37:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.060+0000 7f5626306700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562019c280 con 0x7f5620100ef0 2023-09-30T14:37:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.060+0000 7f561f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56201018b0 0x7f56201974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.060+0000 7f561f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56201018b0 0x7f56201974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34132/0 (socket says 172.21.15.32:34132) 2023-09-30T14:37:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.060+0000 7f561f7fe700 1 -- 172.21.15.32:0/2682074402 learned_addr learned my addr 172.21.15.32:0/2682074402 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.060+0000 7f561ffff700 1 --2- 172.21.15.32:0/2682074402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5620100ef0 0x7f5620196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.060+0000 7f56248a3700 1 --2- 172.21.15.32:0/2682074402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5620106f00 0x7f562019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.060+0000 7f561f7fe700 1 -- 172.21.15.32:0/2682074402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5620100ef0 msgr2=0x7f5620196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.060+0000 7f561f7fe700 1 --2- 172.21.15.32:0/2682074402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5620100ef0 0x7f5620196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.060+0000 7f561f7fe700 1 -- 172.21.15.32:0/2682074402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5620106f00 msgr2=0x7f562019b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.060+0000 7f561f7fe700 1 --2- 172.21.15.32:0/2682074402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5620106f00 0x7f562019b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.060+0000 7f561f7fe700 1 -- 172.21.15.32:0/2682074402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5618009d00 con 0x7f56201018b0 2023-09-30T14:37:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.061+0000 7f561f7fe700 1 --2- 172.21.15.32:0/2682074402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56201018b0 0x7f56201974d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f561400d980 tx=0x7f561400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.061+0000 7f561d7fa700 1 -- 172.21.15.32:0/2682074402 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561400bb60 con 0x7f56201018b0 2023-09-30T14:37:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.061+0000 7f5626306700 1 -- 172.21.15.32:0/2682074402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562019c560 con 0x7f56201018b0 2023-09-30T14:37:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.061+0000 7f561d7fa700 1 -- 172.21.15.32:0/2682074402 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5614007640 con 0x7f56201018b0 2023-09-30T14:37:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.061+0000 7f561d7fa700 1 -- 172.21.15.32:0/2682074402 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614004e90 con 0x7f56201018b0 2023-09-30T14:37:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.061+0000 7f5626306700 1 -- 172.21.15.32:0/2682074402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56201a3dc0 con 0x7f56201018b0 2023-09-30T14:37:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.062+0000 7f5626306700 1 -- 172.21.15.32:0/2682074402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f562004eb20 con 0x7f56201018b0 2023-09-30T14:37:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.063+0000 7f561d7fa700 1 -- 172.21.15.32:0/2682074402 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f561400bcc0 con 0x7f56201018b0 2023-09-30T14:37:41.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.064+0000 7f561d7fa700 1 --2- 172.21.15.32:0/2682074402 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f56080a1030 0x7f56080a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:41.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.064+0000 7f561d7fa700 1 -- 172.21.15.32:0/2682074402 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f56140ccc70 con 0x7f56201018b0 2023-09-30T14:37:41.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.064+0000 7f561ffff700 1 --2- 172.21.15.32:0/2682074402 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f56080a1030 0x7f56080a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:41.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.064+0000 7f561d7fa700 1 --2- 172.21.15.32:0/2682074402 >> [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433] conn(0x7f56080aa6a0 0x7f56080acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:41.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.064+0000 7f561d7fa700 1 -- 172.21.15.32:0/2682074402 --> [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56080ad170 con 0x7f56080aa6a0 2023-09-30T14:37:41.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.064+0000 7f561d7fa700 1 -- 172.21.15.32:0/2682074402 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56140cd110 con 0x7f56201018b0 2023-09-30T14:37:41.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.064+0000 7f56248a3700 1 --2- 172.21.15.32:0/2682074402 >> [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433] conn(0x7f56080aa6a0 0x7f56080acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:41.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.064+0000 7f561ffff700 1 --2- 172.21.15.32:0/2682074402 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f56080a1030 0x7f56080a34f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5610009bd0 tx=0x7f5610009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:41.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.065+0000 7f56248a3700 1 --2- 172.21.15.32:0/2682074402 >> [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433] conn(0x7f56080aa6a0 0x7f56080acac0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:41.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.066+0000 7f561d7fa700 1 -- 172.21.15.32:0/2682074402 <== osd.6 v2:172.21.15.32:6818/4025142433 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f56080ad170 con 0x7f56080aa6a0 2023-09-30T14:37:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.082+0000 7f5626306700 1 -- 172.21.15.32:0/2682074402 --> [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f56201110c0 con 0x7f56080aa6a0 2023-09-30T14:37:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.082+0000 7f561d7fa700 1 -- 172.21.15.32:0/2682074402 <== osd.6 v2:172.21.15.32:6818/4025142433 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5618009400 con 0x7f56080aa6a0 2023-09-30T14:37:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.083+0000 7f5626306700 1 -- 172.21.15.32:0/2682074402 >> [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433] conn(0x7f56080aa6a0 msgr2=0x7f56080acac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.083+0000 7f5626306700 1 --2- 172.21.15.32:0/2682074402 >> [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433] conn(0x7f56080aa6a0 0x7f56080acac0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.083+0000 7f5626306700 1 -- 172.21.15.32:0/2682074402 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f56080a1030 msgr2=0x7f56080a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.083+0000 7f5626306700 1 --2- 172.21.15.32:0/2682074402 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f56080a1030 0x7f56080a34f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5610009bd0 tx=0x7f5610009260 comp rx=0 tx=0).stop 2023-09-30T14:37:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.083+0000 7f5626306700 1 -- 172.21.15.32:0/2682074402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56201018b0 msgr2=0x7f56201974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.083+0000 7f5626306700 1 --2- 172.21.15.32:0/2682074402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56201018b0 0x7f56201974d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f561400d980 tx=0x7f561400dd40 comp rx=0 tx=0).stop 2023-09-30T14:37:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.083+0000 7f56248a3700 1 -- 172.21.15.32:0/2682074402 reap_dead start 2023-09-30T14:37:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.083+0000 7f5626306700 1 -- 172.21.15.32:0/2682074402 shutdown_connections 2023-09-30T14:37:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.083+0000 7f5626306700 1 -- 172.21.15.32:0/2682074402 >> 172.21.15.32:0/2682074402 conn(0x7f5620078010 msgr2=0x7f5620104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.083+0000 7f5626306700 1 -- 172.21.15.32:0/2682074402 shutdown_connections 2023-09-30T14:37:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.083+0000 7f5626306700 1 -- 172.21.15.32:0/2682074402 wait complete. 2023-09-30T14:37:41.180 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:41 smithi032 ceph-mon[98592]: pgmap v267: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-09-30T14:37:41.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.217+0000 7f321210a700 1 -- 172.21.15.32:0/351829955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f320c0ffdd0 msgr2=0x7f320c100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.217+0000 7f321210a700 1 --2- 172.21.15.32:0/351829955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f320c0ffdd0 0x7f320c100250 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f32000099c0 tx=0x7f3200009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.217+0000 7f321210a700 1 -- 172.21.15.32:0/351829955 shutdown_connections 2023-09-30T14:37:41.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.217+0000 7f321210a700 1 --2- 172.21.15.32:0/351829955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f320c100790 0x7f320c10cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.217+0000 7f321210a700 1 --2- 172.21.15.32:0/351829955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f320c105de0 0x7f320c1061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.217+0000 7f321210a700 1 --2- 172.21.15.32:0/351829955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f320c0ffdd0 0x7f320c100250 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.217+0000 7f321210a700 1 -- 172.21.15.32:0/351829955 >> 172.21.15.32:0/351829955 conn(0x7f320c075d70 msgr2=0x7f320c076180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:41.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.217+0000 7f321210a700 1 -- 172.21.15.32:0/351829955 shutdown_connections 2023-09-30T14:37:41.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.217+0000 7f321210a700 1 -- 172.21.15.32:0/351829955 wait complete. 2023-09-30T14:37:41.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.217+0000 7f321210a700 1 Processor -- start 2023-09-30T14:37:41.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f321210a700 1 -- start start 2023-09-30T14:37:41.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f321210a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f320c0ffdd0 0x7f320c10d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:41.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f321210a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f320c100790 0x7f320c10db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:41.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f321210a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f320c105de0 0x7f320c10e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:41.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f321210a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c1a5350 con 0x7f320c100790 2023-09-30T14:37:41.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f321210a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c1a54c0 con 0x7f320c105de0 2023-09-30T14:37:41.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f321210a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c1a5680 con 0x7f320c0ffdd0 2023-09-30T14:37:41.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f320affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f320c100790 0x7f320c10db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f320affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f320c100790 0x7f320c10db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34148/0 (socket says 172.21.15.32:34148) 2023-09-30T14:37:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f320affd700 1 -- 172.21.15.32:0/2039220025 learned_addr learned my addr 172.21.15.32:0/2039220025 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f320bfff700 1 --2- 172.21.15.32:0/2039220025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f320c105de0 0x7f320c10e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f320b7fe700 1 --2- 172.21.15.32:0/2039220025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f320c0ffdd0 0x7f320c10d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f320affd700 1 -- 172.21.15.32:0/2039220025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f320c0ffdd0 msgr2=0x7f320c10d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f320affd700 1 --2- 172.21.15.32:0/2039220025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f320c0ffdd0 0x7f320c10d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f320affd700 1 -- 172.21.15.32:0/2039220025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f320c105de0 msgr2=0x7f320c10e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f320affd700 1 --2- 172.21.15.32:0/2039220025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f320c105de0 0x7f320c10e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.218+0000 7f320affd700 1 -- 172.21.15.32:0/2039220025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32000095f0 con 0x7f320c100790 2023-09-30T14:37:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.219+0000 7f320affd700 1 --2- 172.21.15.32:0/2039220025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f320c100790 0x7f320c10db10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3200009990 tx=0x7f3200009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.219+0000 7f3208ff9700 1 -- 172.21.15.32:0/2039220025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320001d070 con 0x7f320c100790 2023-09-30T14:37:41.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.219+0000 7f321210a700 1 -- 172.21.15.32:0/2039220025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320c1a5900 con 0x7f320c100790 2023-09-30T14:37:41.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.219+0000 7f3208ff9700 1 -- 172.21.15.32:0/2039220025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3200003f90 con 0x7f320c100790 2023-09-30T14:37:41.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.219+0000 7f3208ff9700 1 -- 172.21.15.32:0/2039220025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3200022ce0 con 0x7f320c100790 2023-09-30T14:37:41.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.219+0000 7f321210a700 1 -- 172.21.15.32:0/2039220025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320c1a5d70 con 0x7f320c100790 2023-09-30T14:37:41.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.220+0000 7f31f27fc700 1 -- 172.21.15.32:0/2039220025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f31ec000fc0 con 0x7f320c100790 2023-09-30T14:37:41.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.222+0000 7f3208ff9700 1 -- 172.21.15.32:0/2039220025 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f3200022470 con 0x7f320c100790 2023-09-30T14:37:41.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.223+0000 7f3208ff9700 1 --2- 172.21.15.32:0/2039220025 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31f40a1030 0x7f31f40a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:41.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.223+0000 7f320b7fe700 1 --2- 172.21.15.32:0/2039220025 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31f40a1030 0x7f31f40a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:41.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.223+0000 7f3208ff9700 1 -- 172.21.15.32:0/2039220025 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f32000cdee0 con 0x7f320c100790 2023-09-30T14:37:41.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.223+0000 7f3208ff9700 1 --2- 172.21.15.32:0/2039220025 >> [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559] conn(0x7f31f40aa810 0x7f31f40acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:41.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.223+0000 7f3208ff9700 1 -- 172.21.15.32:0/2039220025 --> [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31f40ad2e0 con 0x7f31f40aa810 2023-09-30T14:37:41.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.223+0000 7f3208ff9700 1 -- 172.21.15.32:0/2039220025 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32000ce380 con 0x7f320c100790 2023-09-30T14:37:41.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.223+0000 7f320b7fe700 1 --2- 172.21.15.32:0/2039220025 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31f40a1030 0x7f31f40a34f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31fc005950 tx=0x7f31fc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:41.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.223+0000 7f320bfff700 1 --2- 172.21.15.32:0/2039220025 >> [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559] conn(0x7f31f40aa810 0x7f31f40acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:41.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.224+0000 7f320bfff700 1 --2- 172.21.15.32:0/2039220025 >> [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559] conn(0x7f31f40aa810 0x7f31f40acc30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:41.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.225+0000 7f3208ff9700 1 -- 172.21.15.32:0/2039220025 <== osd.4 v2:172.21.15.148:6808/1987304559 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f31f40ad2e0 con 0x7f31f40aa810 2023-09-30T14:37:41.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.242+0000 7f31f27fc700 1 -- 172.21.15.32:0/2039220025 --> [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f31ec002d40 con 0x7f31f40aa810 2023-09-30T14:37:41.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.242+0000 7f3208ff9700 1 -- 172.21.15.32:0/2039220025 <== osd.4 v2:172.21.15.148:6808/1987304559 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f31ec002d40 con 0x7f31f40aa810 2023-09-30T14:37:41.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.242+0000 7f31f27fc700 1 -- 172.21.15.32:0/2039220025 >> [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559] conn(0x7f31f40aa810 msgr2=0x7f31f40acc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.242+0000 7f31f27fc700 1 --2- 172.21.15.32:0/2039220025 >> [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559] conn(0x7f31f40aa810 0x7f31f40acc30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.242+0000 7f31f27fc700 1 -- 172.21.15.32:0/2039220025 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31f40a1030 msgr2=0x7f31f40a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.242+0000 7f31f27fc700 1 --2- 172.21.15.32:0/2039220025 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31f40a1030 0x7f31f40a34f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31fc005950 tx=0x7f31fc0058e0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.242+0000 7f31f27fc700 1 -- 172.21.15.32:0/2039220025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f320c100790 msgr2=0x7f320c10db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.242+0000 7f31f27fc700 1 --2- 172.21.15.32:0/2039220025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f320c100790 0x7f320c10db10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3200009990 tx=0x7f3200009580 comp rx=0 tx=0).stop 2023-09-30T14:37:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.243+0000 7f320bfff700 1 -- 172.21.15.32:0/2039220025 reap_dead start 2023-09-30T14:37:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.243+0000 7f31f27fc700 1 -- 172.21.15.32:0/2039220025 shutdown_connections 2023-09-30T14:37:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.243+0000 7f31f27fc700 1 -- 172.21.15.32:0/2039220025 >> 172.21.15.32:0/2039220025 conn(0x7f320c075d70 msgr2=0x7f320c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.243+0000 7f31f27fc700 1 -- 172.21.15.32:0/2039220025 shutdown_connections 2023-09-30T14:37:41.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.243+0000 7f31f27fc700 1 -- 172.21.15.32:0/2039220025 wait complete. 2023-09-30T14:37:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.258+0000 7f3dfe15c700 1 -- 172.21.15.32:0/4208897132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df81010c0 msgr2=0x7f3df81014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.258+0000 7f3dfe15c700 1 --2- 172.21.15.32:0/4208897132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df81010c0 0x7f3df81014a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3de800b0a0 tx=0x7f3de800b3b0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.258+0000 7f3dfe15c700 1 -- 172.21.15.32:0/4208897132 shutdown_connections 2023-09-30T14:37:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.258+0000 7f3dfe15c700 1 --2- 172.21.15.32:0/4208897132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3df81061a0 0x7f3df810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.258+0000 7f3dfe15c700 1 --2- 172.21.15.32:0/4208897132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df81010c0 0x7f3df81014a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.258+0000 7f3dfe15c700 1 --2- 172.21.15.32:0/4208897132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8101a70 0x7f3df8105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.258+0000 7f3dfe15c700 1 -- 172.21.15.32:0/4208897132 >> 172.21.15.32:0/4208897132 conn(0x7f3df80fc910 msgr2=0x7f3df80fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:41.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.258+0000 7f3dfe15c700 1 -- 172.21.15.32:0/4208897132 shutdown_connections 2023-09-30T14:37:41.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.258+0000 7f3dfe15c700 1 -- 172.21.15.32:0/4208897132 wait complete. 2023-09-30T14:37:41.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.259+0000 7f3dfe15c700 1 Processor -- start 2023-09-30T14:37:41.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.259+0000 7f3dfe15c700 1 -- start start 2023-09-30T14:37:41.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.259+0000 7f3dfe15c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df81010c0 0x7f3df81970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:41.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.259+0000 7f3dfe15c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8101a70 0x7f3df8197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:41.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.259+0000 7f3dfe15c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3df81061a0 0x7f3df819ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:41.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.259+0000 7f3dfe15c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df819c080 con 0x7f3df8101a70 2023-09-30T14:37:41.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.259+0000 7f3dfe15c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df819c1f0 con 0x7f3df81010c0 2023-09-30T14:37:41.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.259+0000 7f3dfe15c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df819c3b0 con 0x7f3df81061a0 2023-09-30T14:37:41.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3df6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8101a70 0x7f3df8197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:41.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3df6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8101a70 0x7f3df8197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34164/0 (socket says 172.21.15.32:34164) 2023-09-30T14:37:41.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3df6ffd700 1 -- 172.21.15.32:0/903932008 learned_addr learned my addr 172.21.15.32:0/903932008 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:41.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3df77fe700 1 --2- 172.21.15.32:0/903932008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df81010c0 0x7f3df81970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:41.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3df7fff700 1 --2- 172.21.15.32:0/903932008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3df81061a0 0x7f3df819ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:41.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3df6ffd700 1 -- 172.21.15.32:0/903932008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3df81061a0 msgr2=0x7f3df819ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3df6ffd700 1 --2- 172.21.15.32:0/903932008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3df81061a0 0x7f3df819ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3df6ffd700 1 -- 172.21.15.32:0/903932008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df81010c0 msgr2=0x7f3df81970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3df6ffd700 1 --2- 172.21.15.32:0/903932008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df81010c0 0x7f3df81970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3df6ffd700 1 -- 172.21.15.32:0/903932008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de8009d00 con 0x7f3df8101a70 2023-09-30T14:37:41.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3df7fff700 1 --2- 172.21.15.32:0/903932008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3df81061a0 0x7f3df819ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:37:41.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3df6ffd700 1 --2- 172.21.15.32:0/903932008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8101a70 0x7f3df8197600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00e970 tx=0x7f3dec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:41.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3df4ff9700 1 -- 172.21.15.32:0/903932008 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec009790 con 0x7f3df8101a70 2023-09-30T14:37:41.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3df4ff9700 1 -- 172.21.15.32:0/903932008 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dec00eec0 con 0x7f3df8101a70 2023-09-30T14:37:41.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3dfe15c700 1 -- 172.21.15.32:0/903932008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df819c690 con 0x7f3df8101a70 2023-09-30T14:37:41.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3df4ff9700 1 -- 172.21.15.32:0/903932008 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec005590 con 0x7f3df8101a70 2023-09-30T14:37:41.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.260+0000 7f3dfe15c700 1 -- 172.21.15.32:0/903932008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df81a3ef0 con 0x7f3df8101a70 2023-09-30T14:37:41.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.261+0000 7f3dfe15c700 1 -- 172.21.15.32:0/903932008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3df804eb20 con 0x7f3df8101a70 2023-09-30T14:37:41.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.262+0000 7f3df4ff9700 1 -- 172.21.15.32:0/903932008 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f3dec00cbd0 con 0x7f3df8101a70 2023-09-30T14:37:41.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.263+0000 7f3df4ff9700 1 --2- 172.21.15.32:0/903932008 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3de00a1030 0x7f3de00a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:41.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.263+0000 7f3df77fe700 1 --2- 172.21.15.32:0/903932008 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3de00a1030 0x7f3de00a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:41.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.263+0000 7f3df4ff9700 1 -- 172.21.15.32:0/903932008 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3dec014070 con 0x7f3df8101a70 2023-09-30T14:37:41.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.263+0000 7f3df4ff9700 1 --2- 172.21.15.32:0/903932008 >> [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147] conn(0x7f3de00aa810 0x7f3de00acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:41.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.263+0000 7f3df4ff9700 1 -- 172.21.15.32:0/903932008 --> [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3de00ad2e0 con 0x7f3de00aa810 2023-09-30T14:37:41.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.263+0000 7f3df4ff9700 1 -- 172.21.15.32:0/903932008 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dec0cd060 con 0x7f3df8101a70 2023-09-30T14:37:41.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.264+0000 7f3df7fff700 1 --2- 172.21.15.32:0/903932008 >> [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147] conn(0x7f3de00aa810 0x7f3de00acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:41.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.264+0000 7f3df77fe700 1 --2- 172.21.15.32:0/903932008 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3de00a1030 0x7f3de00a34f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009fd0 tx=0x7f3de800b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:41.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.264+0000 7f3df7fff700 1 --2- 172.21.15.32:0/903932008 >> [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147] conn(0x7f3de00aa810 0x7f3de00acc30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:41.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.265+0000 7f3df4ff9700 1 -- 172.21.15.32:0/903932008 <== osd.7 v2:172.21.15.148:6816/3841614147 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f3de00ad2e0 con 0x7f3de00aa810 2023-09-30T14:37:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.282+0000 7f3dfe15c700 1 -- 172.21.15.32:0/903932008 --> [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3df81110b0 con 0x7f3de00aa810 2023-09-30T14:37:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.282+0000 7f3df4ff9700 1 -- 172.21.15.32:0/903932008 <== osd.7 v2:172.21.15.148:6816/3841614147 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3df000b7f0 con 0x7f3de00aa810 2023-09-30T14:37:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.283+0000 7f3dfe15c700 1 -- 172.21.15.32:0/903932008 >> [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147] conn(0x7f3de00aa810 msgr2=0x7f3de00acc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.283+0000 7f3dfe15c700 1 --2- 172.21.15.32:0/903932008 >> [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147] conn(0x7f3de00aa810 0x7f3de00acc30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.283+0000 7f3dfe15c700 1 -- 172.21.15.32:0/903932008 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3de00a1030 msgr2=0x7f3de00a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.283+0000 7f3dfe15c700 1 --2- 172.21.15.32:0/903932008 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3de00a1030 0x7f3de00a34f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009fd0 tx=0x7f3de800b700 comp rx=0 tx=0).stop 2023-09-30T14:37:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.283+0000 7f3dfe15c700 1 -- 172.21.15.32:0/903932008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8101a70 msgr2=0x7f3df8197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.283+0000 7f3dfe15c700 1 --2- 172.21.15.32:0/903932008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df8101a70 0x7f3df8197600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00e970 tx=0x7f3dec00ec80 comp rx=0 tx=0).stop 2023-09-30T14:37:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.283+0000 7f3df7fff700 1 -- 172.21.15.32:0/903932008 reap_dead start 2023-09-30T14:37:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.283+0000 7f3dfe15c700 1 -- 172.21.15.32:0/903932008 shutdown_connections 2023-09-30T14:37:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.283+0000 7f3dfe15c700 1 -- 172.21.15.32:0/903932008 >> 172.21.15.32:0/903932008 conn(0x7f3df80fc910 msgr2=0x7f3df8104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.283+0000 7f3dfe15c700 1 -- 172.21.15.32:0/903932008 shutdown_connections 2023-09-30T14:37:41.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:41.283+0000 7f3dfe15c700 1 -- 172.21.15.32:0/903932008 wait complete. 2023-09-30T14:37:41.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:41 smithi156 ceph-mon[103579]: pgmap v267: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-09-30T14:37:41.583 INFO:teuthology.orchestra.run.smithi032.stdout:77309411384 2023-09-30T14:37:41.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-09-30T14:37:41.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:41 smithi148 ceph-mon[102355]: pgmap v267: 97 pgs: 97 active+clean; 633 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-09-30T14:37:42.491 INFO:teuthology.orchestra.run.smithi032.stdout:197568495655 2023-09-30T14:37:42.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-09-30T14:37:42.690 INFO:teuthology.orchestra.run.smithi032.stdout:304942678036 2023-09-30T14:37:42.690 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-09-30T14:37:42.747 INFO:teuthology.orchestra.run.smithi032.stdout:111669149747 2023-09-30T14:37:42.747 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-09-30T14:37:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.977+0000 7fc3a83ba700 1 -- 172.21.15.32:0/3617693431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3a0105de0 msgr2=0x7fc3a01061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.977+0000 7fc3a83ba700 1 --2- 172.21.15.32:0/3617693431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3a0105de0 0x7fc3a01061c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0099c0 tx=0x7fc39c009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.977+0000 7fc3a83ba700 1 -- 172.21.15.32:0/3617693431 shutdown_connections 2023-09-30T14:37:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.977+0000 7fc3a83ba700 1 --2- 172.21.15.32:0/3617693431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3a0100790 0x7fc3a010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.977+0000 7fc3a83ba700 1 --2- 172.21.15.32:0/3617693431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3a0105de0 0x7fc3a01061c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.977+0000 7fc3a83ba700 1 --2- 172.21.15.32:0/3617693431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a00ffdd0 0x7fc3a0100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.977+0000 7fc3a83ba700 1 -- 172.21.15.32:0/3617693431 >> 172.21.15.32:0/3617693431 conn(0x7fc3a0075d70 msgr2=0x7fc3a0076180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.978+0000 7fc3a83ba700 1 -- 172.21.15.32:0/3617693431 shutdown_connections 2023-09-30T14:37:42.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.978+0000 7fc3a83ba700 1 -- 172.21.15.32:0/3617693431 wait complete. 2023-09-30T14:37:42.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.978+0000 7fc3a83ba700 1 Processor -- start 2023-09-30T14:37:42.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.978+0000 7fc3a83ba700 1 -- start start 2023-09-30T14:37:42.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a83ba700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a00ffdd0 0x7fc3a0195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:42.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a83ba700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3a0100790 0x7fc3a0196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:42.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a83ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3a0105de0 0x7fc3a019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:42.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a83ba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a019ad90 con 0x7fc3a0105de0 2023-09-30T14:37:42.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a83ba700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a019af00 con 0x7fc3a0100790 2023-09-30T14:37:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a83ba700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a019b0c0 con 0x7fc3a00ffdd0 2023-09-30T14:37:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a6957700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3a0105de0 0x7fc3a019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a5955700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3a0100790 0x7fc3a0196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a6957700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3a0105de0 0x7fc3a019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34180/0 (socket says 172.21.15.32:34180) 2023-09-30T14:37:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a6957700 1 -- 172.21.15.32:0/476637567 learned_addr learned my addr 172.21.15.32:0/476637567 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a6156700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a00ffdd0 0x7fc3a0195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a6957700 1 -- 172.21.15.32:0/476637567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a00ffdd0 msgr2=0x7fc3a0195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a6957700 1 --2- 172.21.15.32:0/476637567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a00ffdd0 0x7fc3a0195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a6957700 1 -- 172.21.15.32:0/476637567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3a0100790 msgr2=0x7fc3a0196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a6957700 1 --2- 172.21.15.32:0/476637567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3a0100790 0x7fc3a0196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a6957700 1 -- 172.21.15.32:0/476637567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39c0095f0 con 0x7fc3a0105de0 2023-09-30T14:37:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.979+0000 7fc3a6957700 1 --2- 172.21.15.32:0/476637567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3a0105de0 0x7fc3a019a780 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc39800ada0 tx=0x7fc39800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.980+0000 7fc3937fe700 1 -- 172.21.15.32:0/476637567 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39800f070 con 0x7fc3a0105de0 2023-09-30T14:37:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.980+0000 7fc3937fe700 1 -- 172.21.15.32:0/476637567 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc398007b30 con 0x7fc3a0105de0 2023-09-30T14:37:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.980+0000 7fc3a83ba700 1 -- 172.21.15.32:0/476637567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a019b3a0 con 0x7fc3a0105de0 2023-09-30T14:37:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.980+0000 7fc3937fe700 1 -- 172.21.15.32:0/476637567 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398005280 con 0x7fc3a0105de0 2023-09-30T14:37:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.980+0000 7fc3a83ba700 1 -- 172.21.15.32:0/476637567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a0068ae0 con 0x7fc3a0105de0 2023-09-30T14:37:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.980+0000 7fc3a83ba700 1 -- 172.21.15.32:0/476637567 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc3a004eb20 con 0x7fc3a0105de0 2023-09-30T14:37:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.982+0000 7fc3937fe700 1 -- 172.21.15.32:0/476637567 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fc39800a080 con 0x7fc3a0105de0 2023-09-30T14:37:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.982+0000 7fc3937fe700 1 --2- 172.21.15.32:0/476637567 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc38c0a10b0 0x7fc38c0a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.983+0000 7fc3a6156700 1 --2- 172.21.15.32:0/476637567 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc38c0a10b0 0x7fc38c0a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.983+0000 7fc3937fe700 1 -- 172.21.15.32:0/476637567 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc3980cc4c0 con 0x7fc3a0105de0 2023-09-30T14:37:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.983+0000 7fc3937fe700 1 --2- 172.21.15.32:0/476637567 >> [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479] conn(0x7fc38c0aa890 0x7fc38c0accb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.983+0000 7fc3937fe700 1 -- 172.21.15.32:0/476637567 --> [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc38c0ad360 con 0x7fc38c0aa890 2023-09-30T14:37:42.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.983+0000 7fc3937fe700 1 -- 172.21.15.32:0/476637567 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3980cc960 con 0x7fc3a0105de0 2023-09-30T14:37:42.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.983+0000 7fc3a5955700 1 --2- 172.21.15.32:0/476637567 >> [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479] conn(0x7fc38c0aa890 0x7fc38c0accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:42.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.983+0000 7fc3a6156700 1 --2- 172.21.15.32:0/476637567 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc38c0a10b0 0x7fc38c0a3570 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00b580 tx=0x7fc39c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:42.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.983+0000 7fc3a5955700 1 --2- 172.21.15.32:0/476637567 >> [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479] conn(0x7fc38c0aa890 0x7fc38c0accb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:42.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:42.984+0000 7fc3937fe700 1 -- 172.21.15.32:0/476637567 <== osd.9 v2:172.21.15.32:6826/496139479 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fc38c0ad360 con 0x7fc38c0aa890 2023-09-30T14:37:43.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:43.000+0000 7fc3a83ba700 1 -- 172.21.15.32:0/476637567 --> [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc3a0110000 con 0x7fc38c0aa890 2023-09-30T14:37:43.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:43.000+0000 7fc3937fe700 1 -- 172.21.15.32:0/476637567 <== osd.9 v2:172.21.15.32:6826/496139479 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc39400a8a0 con 0x7fc38c0aa890 2023-09-30T14:37:43.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:43.001+0000 7fc3a83ba700 1 -- 172.21.15.32:0/476637567 >> [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479] conn(0x7fc38c0aa890 msgr2=0x7fc38c0accb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:43.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:43.001+0000 7fc3a83ba700 1 --2- 172.21.15.32:0/476637567 >> [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479] conn(0x7fc38c0aa890 0x7fc38c0accb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:43.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:43.001+0000 7fc3a83ba700 1 -- 172.21.15.32:0/476637567 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc38c0a10b0 msgr2=0x7fc38c0a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:43.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:43.001+0000 7fc3a83ba700 1 --2- 172.21.15.32:0/476637567 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc38c0a10b0 0x7fc38c0a3570 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00b580 tx=0x7fc39c0058e0 comp rx=0 tx=0).stop 2023-09-30T14:37:43.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:43.001+0000 7fc3a83ba700 1 -- 172.21.15.32:0/476637567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3a0105de0 msgr2=0x7fc3a019a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:43.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:43.001+0000 7fc3a83ba700 1 --2- 172.21.15.32:0/476637567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3a0105de0 0x7fc3a019a780 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc39800ada0 tx=0x7fc39800c330 comp rx=0 tx=0).stop 2023-09-30T14:37:43.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:43.001+0000 7fc3a6957700 1 -- 172.21.15.32:0/476637567 reap_dead start 2023-09-30T14:37:43.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:43.001+0000 7fc3a83ba700 1 -- 172.21.15.32:0/476637567 shutdown_connections 2023-09-30T14:37:43.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:43.001+0000 7fc3a83ba700 1 -- 172.21.15.32:0/476637567 >> 172.21.15.32:0/476637567 conn(0x7fc3a0075d70 msgr2=0x7fc3a00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:43.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:43.001+0000 7fc3a83ba700 1 -- 172.21.15.32:0/476637567 shutdown_connections 2023-09-30T14:37:43.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:43.001+0000 7fc3a83ba700 1 -- 172.21.15.32:0/476637567 wait complete. 2023-09-30T14:37:43.274 INFO:teuthology.orchestra.run.smithi032.stdout:219043332130 2023-09-30T14:37:43.274 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-09-30T14:37:43.284 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:43 smithi032 ceph-mon[98592]: pgmap v268: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-09-30T14:37:43.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:43 smithi156 ceph-mon[103579]: pgmap v268: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-09-30T14:37:43.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:43 smithi148 ceph-mon[102355]: pgmap v268: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-09-30T14:37:44.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.007+0000 7f941da62700 1 -- 172.21.15.32:0/1420692079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94181018b0 msgr2=0x7f941810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.007+0000 7f941da62700 1 --2- 172.21.15.32:0/1420692079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94181018b0 0x7f941810dd20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f941000b0d0 tx=0x7f941000b3e0 comp rx=0 tx=0).stop 2023-09-30T14:37:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.007+0000 7f941da62700 1 -- 172.21.15.32:0/1420692079 shutdown_connections 2023-09-30T14:37:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.007+0000 7f941da62700 1 --2- 172.21.15.32:0/1420692079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94181018b0 0x7f941810dd20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.007+0000 7f941da62700 1 --2- 172.21.15.32:0/1420692079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9418106f00 0x7f94181072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.007+0000 7f941da62700 1 --2- 172.21.15.32:0/1420692079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9418100ef0 0x7f9418101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.007+0000 7f941da62700 1 -- 172.21.15.32:0/1420692079 >> 172.21.15.32:0/1420692079 conn(0x7f9418078010 msgr2=0x7f9418078420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.007+0000 7f941da62700 1 -- 172.21.15.32:0/1420692079 shutdown_connections 2023-09-30T14:37:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.007+0000 7f941da62700 1 -- 172.21.15.32:0/1420692079 wait complete. 2023-09-30T14:37:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.008+0000 7f941da62700 1 Processor -- start 2023-09-30T14:37:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.008+0000 7f941da62700 1 -- start start 2023-09-30T14:37:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.008+0000 7f941da62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9418100ef0 0x7f9418197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.008+0000 7f941da62700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94181018b0 0x7f9418197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.008+0000 7f941da62700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9418106f00 0x7f941819b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.008+0000 7f941da62700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941819bf80 con 0x7f9418100ef0 2023-09-30T14:37:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.008+0000 7f941da62700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941819c0f0 con 0x7f94181018b0 2023-09-30T14:37:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.008+0000 7f941da62700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941819c2b0 con 0x7f9418106f00 2023-09-30T14:37:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.008+0000 7f94177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9418100ef0 0x7f9418197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.008+0000 7f94177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9418100ef0 0x7f9418197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34190/0 (socket says 172.21.15.32:34190) 2023-09-30T14:37:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.008+0000 7f94177fe700 1 -- 172.21.15.32:0/3396279661 learned_addr learned my addr 172.21.15.32:0/3396279661 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.009+0000 7f9416ffd700 1 --2- 172.21.15.32:0/3396279661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94181018b0 0x7f9418197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.009+0000 7f9417fff700 1 --2- 172.21.15.32:0/3396279661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9418106f00 0x7f941819b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.009+0000 7f94177fe700 1 -- 172.21.15.32:0/3396279661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9418106f00 msgr2=0x7f941819b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.009+0000 7f94177fe700 1 --2- 172.21.15.32:0/3396279661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9418106f00 0x7f941819b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.009+0000 7f94177fe700 1 -- 172.21.15.32:0/3396279661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94181018b0 msgr2=0x7f9418197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.009+0000 7f94177fe700 1 --2- 172.21.15.32:0/3396279661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94181018b0 0x7f9418197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.009+0000 7f94177fe700 1 -- 172.21.15.32:0/3396279661 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9410009d00 con 0x7f9418100ef0 2023-09-30T14:37:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.009+0000 7f9416ffd700 1 --2- 172.21.15.32:0/3396279661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94181018b0 0x7f9418197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:37:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.009+0000 7f94177fe700 1 --2- 172.21.15.32:0/3396279661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9418100ef0 0x7f9418197010 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f940800e9a0 tx=0x7f940800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.009+0000 7f9414ff9700 1 -- 172.21.15.32:0/3396279661 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408009790 con 0x7f9418100ef0 2023-09-30T14:37:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.009+0000 7f9414ff9700 1 -- 172.21.15.32:0/3396279661 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f940800cbd0 con 0x7f9418100ef0 2023-09-30T14:37:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.009+0000 7f9414ff9700 1 -- 172.21.15.32:0/3396279661 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408018680 con 0x7f9418100ef0 2023-09-30T14:37:44.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.009+0000 7f941da62700 1 -- 172.21.15.32:0/3396279661 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941819c590 con 0x7f9418100ef0 2023-09-30T14:37:44.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.009+0000 7f941da62700 1 -- 172.21.15.32:0/3396279661 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94181a3df0 con 0x7f9418100ef0 2023-09-30T14:37:44.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.010+0000 7f941da62700 1 -- 172.21.15.32:0/3396279661 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9418111080 con 0x7f9418100ef0 2023-09-30T14:37:44.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.011+0000 7f9414ff9700 1 -- 172.21.15.32:0/3396279661 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f940800cd40 con 0x7f9418100ef0 2023-09-30T14:37:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.012+0000 7f9414ff9700 1 --2- 172.21.15.32:0/3396279661 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f94000a1030 0x7f94000a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.012+0000 7f9414ff9700 1 -- 172.21.15.32:0/3396279661 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9408014070 con 0x7f9418100ef0 2023-09-30T14:37:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.012+0000 7f9416ffd700 1 --2- 172.21.15.32:0/3396279661 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f94000a1030 0x7f94000a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.012+0000 7f9414ff9700 1 --2- 172.21.15.32:0/3396279661 >> [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180] conn(0x7f94000aa810 0x7f94000acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.012+0000 7f9414ff9700 1 -- 172.21.15.32:0/3396279661 --> [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94000ad2e0 con 0x7f94000aa810 2023-09-30T14:37:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.012+0000 7f9414ff9700 1 -- 172.21.15.32:0/3396279661 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94080cd200 con 0x7f9418100ef0 2023-09-30T14:37:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.013+0000 7f9416ffd700 1 --2- 172.21.15.32:0/3396279661 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f94000a1030 0x7f94000a34f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f94181985e0 tx=0x7f940c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.013+0000 7f9417fff700 1 --2- 172.21.15.32:0/3396279661 >> [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180] conn(0x7f94000aa810 0x7f94000acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:44.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.013+0000 7f9417fff700 1 --2- 172.21.15.32:0/3396279661 >> [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180] conn(0x7f94000aa810 0x7f94000acc30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.014+0000 7f9414ff9700 1 -- 172.21.15.32:0/3396279661 <== osd.5 v2:172.21.15.156:6808/2465554180 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f94000ad2e0 con 0x7f94000aa810 2023-09-30T14:37:44.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.033+0000 7f941da62700 1 -- 172.21.15.32:0/3396279661 --> [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f941804eb20 con 0x7f94000aa810 2023-09-30T14:37:44.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.034+0000 7f9414ff9700 1 -- 172.21.15.32:0/3396279661 <== osd.5 v2:172.21.15.156:6808/2465554180 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f941804eb20 con 0x7f94000aa810 2023-09-30T14:37:44.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.034+0000 7f941da62700 1 -- 172.21.15.32:0/3396279661 >> [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180] conn(0x7f94000aa810 msgr2=0x7f94000acc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:44.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.034+0000 7f941da62700 1 --2- 172.21.15.32:0/3396279661 >> [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180] conn(0x7f94000aa810 0x7f94000acc30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:44.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.034+0000 7f941da62700 1 -- 172.21.15.32:0/3396279661 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f94000a1030 msgr2=0x7f94000a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:44.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.034+0000 7f941da62700 1 --2- 172.21.15.32:0/3396279661 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f94000a1030 0x7f94000a34f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f94181985e0 tx=0x7f940c009260 comp rx=0 tx=0).stop 2023-09-30T14:37:44.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.034+0000 7f941da62700 1 -- 172.21.15.32:0/3396279661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9418100ef0 msgr2=0x7f9418197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:44.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.034+0000 7f941da62700 1 --2- 172.21.15.32:0/3396279661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9418100ef0 0x7f9418197010 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f940800e9a0 tx=0x7f940800ecb0 comp rx=0 tx=0).stop 2023-09-30T14:37:44.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.034+0000 7f9417fff700 1 -- 172.21.15.32:0/3396279661 reap_dead start 2023-09-30T14:37:44.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.034+0000 7f941da62700 1 -- 172.21.15.32:0/3396279661 shutdown_connections 2023-09-30T14:37:44.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.034+0000 7f941da62700 1 -- 172.21.15.32:0/3396279661 >> 172.21.15.32:0/3396279661 conn(0x7f9418078010 msgr2=0x7f9418104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:44.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.034+0000 7f941da62700 1 -- 172.21.15.32:0/3396279661 shutdown_connections 2023-09-30T14:37:44.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:44.034+0000 7f941da62700 1 -- 172.21.15.32:0/3396279661 wait complete. 2023-09-30T14:37:44.073 INFO:teuthology.orchestra.run.smithi032.stdout:141733920814 2023-09-30T14:37:44.073 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-09-30T14:37:44.188 INFO:teuthology.orchestra.run.smithi032.stdout:283467841560 2023-09-30T14:37:44.189 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-09-30T14:37:44.581 INFO:teuthology.orchestra.run.smithi032.stdout:163208757291 2023-09-30T14:37:44.582 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-09-30T14:37:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:44.868 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:45.207 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:45 smithi032 ceph-mon[98592]: pgmap v269: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-30T14:37:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:45.525 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:45.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:45 smithi156 ceph-mon[103579]: pgmap v269: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-30T14:37:45.609 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:45.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:45 smithi148 ceph-mon[102355]: pgmap v269: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-30T14:37:45.696 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:45.723 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:47.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:47 smithi032 ceph-mon[98592]: pgmap v270: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-09-30T14:37:47.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:47 smithi156 ceph-mon[103579]: pgmap v270: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-09-30T14:37:47.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:47 smithi148 ceph-mon[102355]: pgmap v270: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-09-30T14:37:48.154 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.529+0000 7f35e4e66700 1 -- 172.21.15.32:0/3432778528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e01061d0 msgr2=0x7f35e010dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.529+0000 7f35e4e66700 1 --2- 172.21.15.32:0/3432778528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e01061d0 0x7f35e010dda0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f35d800b0a0 tx=0x7f35d800b3b0 comp rx=0 tx=0).stop 2023-09-30T14:37:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.529+0000 7f35e4e66700 1 -- 172.21.15.32:0/3432778528 shutdown_connections 2023-09-30T14:37:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.529+0000 7f35e4e66700 1 --2- 172.21.15.32:0/3432778528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e01061d0 0x7f35e010dda0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.529+0000 7f35e4e66700 1 --2- 172.21.15.32:0/3432778528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e01010f0 0x7f35e01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.529+0000 7f35e4e66700 1 --2- 172.21.15.32:0/3432778528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35e0101aa0 0x7f35e0105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.529+0000 7f35e4e66700 1 -- 172.21.15.32:0/3432778528 >> 172.21.15.32:0/3432778528 conn(0x7f35e00fc940 msgr2=0x7f35e00fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.530+0000 7f35e4e66700 1 -- 172.21.15.32:0/3432778528 shutdown_connections 2023-09-30T14:37:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.530+0000 7f35e4e66700 1 -- 172.21.15.32:0/3432778528 wait complete. 2023-09-30T14:37:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.530+0000 7f35e4e66700 1 Processor -- start 2023-09-30T14:37:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.530+0000 7f35e4e66700 1 -- start start 2023-09-30T14:37:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.530+0000 7f35e4e66700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e01010f0 0x7f35e0197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.530+0000 7f35e4e66700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35e0101aa0 0x7f35e01975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35e4e66700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e01061d0 0x7f35e019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35e4e66700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e019c040 con 0x7f35e01010f0 2023-09-30T14:37:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35e4e66700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e019c1b0 con 0x7f35e01061d0 2023-09-30T14:37:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35e4e66700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e019c370 con 0x7f35e0101aa0 2023-09-30T14:37:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35de59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e01010f0 0x7f35e0197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35de59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e01010f0 0x7f35e0197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34204/0 (socket says 172.21.15.32:34204) 2023-09-30T14:37:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35de59c700 1 -- 172.21.15.32:0/2876666140 learned_addr learned my addr 172.21.15.32:0/2876666140 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35de59c700 1 -- 172.21.15.32:0/2876666140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35e0101aa0 msgr2=0x7f35e01975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35ddd9b700 1 --2- 172.21.15.32:0/2876666140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35e0101aa0 0x7f35e01975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35ded9d700 1 --2- 172.21.15.32:0/2876666140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e01061d0 0x7f35e019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35de59c700 1 --2- 172.21.15.32:0/2876666140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35e0101aa0 0x7f35e01975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35de59c700 1 -- 172.21.15.32:0/2876666140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e01061d0 msgr2=0x7f35e019ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35de59c700 1 --2- 172.21.15.32:0/2876666140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e01061d0 0x7f35e019ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35de59c700 1 -- 172.21.15.32:0/2876666140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d8009d00 con 0x7f35e01010f0 2023-09-30T14:37:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35ddd9b700 1 --2- 172.21.15.32:0/2876666140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35e0101aa0 0x7f35e01975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:37:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35de59c700 1 --2- 172.21.15.32:0/2876666140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e01010f0 0x7f35e0197080 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f35d000d950 tx=0x7f35d000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35d77fe700 1 -- 172.21.15.32:0/2876666140 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0009b40 con 0x7f35e01010f0 2023-09-30T14:37:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35d77fe700 1 -- 172.21.15.32:0/2876666140 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35d0009ca0 con 0x7f35e01010f0 2023-09-30T14:37:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35e4e66700 1 -- 172.21.15.32:0/2876666140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e019c650 con 0x7f35e01010f0 2023-09-30T14:37:48.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35d77fe700 1 -- 172.21.15.32:0/2876666140 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d000f620 con 0x7f35e01010f0 2023-09-30T14:37:48.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.531+0000 7f35e4e66700 1 -- 172.21.15.32:0/2876666140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e01a3eb0 con 0x7f35e01010f0 2023-09-30T14:37:48.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.532+0000 7f35e4e66700 1 -- 172.21.15.32:0/2876666140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e004eb20 con 0x7f35e01010f0 2023-09-30T14:37:48.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.533+0000 7f35d77fe700 1 -- 172.21.15.32:0/2876666140 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f35d0010460 con 0x7f35e01010f0 2023-09-30T14:37:48.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.534+0000 7f35d77fe700 1 --2- 172.21.15.32:0/2876666140 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f35cc0a1030 0x7f35cc0a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:48.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.534+0000 7f35ddd9b700 1 --2- 172.21.15.32:0/2876666140 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f35cc0a1030 0x7f35cc0a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:48.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.534+0000 7f35d77fe700 1 -- 172.21.15.32:0/2876666140 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f35d00cca90 con 0x7f35e01010f0 2023-09-30T14:37:48.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.534+0000 7f35ddd9b700 1 --2- 172.21.15.32:0/2876666140 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f35cc0a1030 0x7f35cc0a34f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f35c8005fd0 tx=0x7f35c8005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:48.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.537+0000 7f35d77fe700 1 -- 172.21.15.32:0/2876666140 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f35d00962e0 con 0x7f35e01010f0 2023-09-30T14:37:48.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.740+0000 7f35e4e66700 1 -- 172.21.15.32:0/2876666140 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f35e01983e0 con 0x7f35e01010f0 2023-09-30T14:37:48.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.740+0000 7f35d77fe700 1 -- 172.21.15.32:0/2876666140 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f35d0016090 con 0x7f35e01010f0 2023-09-30T14:37:48.741 INFO:teuthology.orchestra.run.smithi032.stdout:249108103199 2023-09-30T14:37:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.742+0000 7f35e4e66700 1 -- 172.21.15.32:0/2876666140 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f35cc0a1030 msgr2=0x7f35cc0a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.742+0000 7f35e4e66700 1 --2- 172.21.15.32:0/2876666140 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f35cc0a1030 0x7f35cc0a34f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f35c8005fd0 tx=0x7f35c8005d40 comp rx=0 tx=0).stop 2023-09-30T14:37:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.742+0000 7f35e4e66700 1 -- 172.21.15.32:0/2876666140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e01010f0 msgr2=0x7f35e0197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.742+0000 7f35e4e66700 1 --2- 172.21.15.32:0/2876666140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e01010f0 0x7f35e0197080 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f35d000d950 tx=0x7f35d000dd10 comp rx=0 tx=0).stop 2023-09-30T14:37:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.742+0000 7f35e4e66700 1 -- 172.21.15.32:0/2876666140 shutdown_connections 2023-09-30T14:37:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.742+0000 7f35e4e66700 1 --2- 172.21.15.32:0/2876666140 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f35cc0a1030 0x7f35cc0a34f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.742+0000 7f35e4e66700 1 --2- 172.21.15.32:0/2876666140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e01061d0 0x7f35e019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:48.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.742+0000 7f35e4e66700 1 --2- 172.21.15.32:0/2876666140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35e01010f0 0x7f35e0197080 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:48.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.742+0000 7f35e4e66700 1 --2- 172.21.15.32:0/2876666140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35e0101aa0 0x7f35e01975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:48.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.742+0000 7f35e4e66700 1 -- 172.21.15.32:0/2876666140 >> 172.21.15.32:0/2876666140 conn(0x7f35e00fc940 msgr2=0x7f35e00fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:48.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.742+0000 7f35e4e66700 1 -- 172.21.15.32:0/2876666140 shutdown_connections 2023-09-30T14:37:48.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:48.742+0000 7f35e4e66700 1 -- 172.21.15.32:0/2876666140 wait complete. 2023-09-30T14:37:49.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.031+0000 7f426dd6e700 1 -- 172.21.15.32:0/3595764569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4268100790 msgr2=0x7f426810cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:49.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.031+0000 7f426dd6e700 1 --2- 172.21.15.32:0/3595764569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4268100790 0x7f426810cc00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f426000b0d0 tx=0x7f426000b3e0 comp rx=0 tx=0).stop 2023-09-30T14:37:49.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.031+0000 7f426dd6e700 1 -- 172.21.15.32:0/3595764569 shutdown_connections 2023-09-30T14:37:49.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.031+0000 7f426dd6e700 1 --2- 172.21.15.32:0/3595764569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4268100790 0x7f426810cc00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:49.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.031+0000 7f426dd6e700 1 --2- 172.21.15.32:0/3595764569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4268105de0 0x7f42681061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:49.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.031+0000 7f426dd6e700 1 --2- 172.21.15.32:0/3595764569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42680ffdd0 0x7f4268100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:49.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.031+0000 7f426dd6e700 1 -- 172.21.15.32:0/3595764569 >> 172.21.15.32:0/3595764569 conn(0x7f4268075d70 msgr2=0x7f4268076180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:49.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.031+0000 7f426dd6e700 1 -- 172.21.15.32:0/3595764569 shutdown_connections 2023-09-30T14:37:49.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.031+0000 7f426dd6e700 1 -- 172.21.15.32:0/3595764569 wait complete. 2023-09-30T14:37:49.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.032+0000 7f426dd6e700 1 Processor -- start 2023-09-30T14:37:49.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.032+0000 7f426dd6e700 1 -- start start 2023-09-30T14:37:49.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.032+0000 7f426dd6e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42680ffdd0 0x7f4268197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:49.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.032+0000 7f426dd6e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4268100790 0x7f4268198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:49.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.032+0000 7f426dd6e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4268105de0 0x7f42681a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:49.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.032+0000 7f426dd6e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42681a33b0 con 0x7f42680ffdd0 2023-09-30T14:37:49.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.032+0000 7f426dd6e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42681a34f0 con 0x7f4268105de0 2023-09-30T14:37:49.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.032+0000 7f426dd6e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42681a3630 con 0x7f4268100790 2023-09-30T14:37:49.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.033+0000 7f42677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42680ffdd0 0x7f4268197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:49.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.033+0000 7f42677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42680ffdd0 0x7f4268197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34218/0 (socket says 172.21.15.32:34218) 2023-09-30T14:37:49.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.033+0000 7f42677fe700 1 -- 172.21.15.32:0/350249978 learned_addr learned my addr 172.21.15.32:0/350249978 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:49.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.033+0000 7f4267fff700 1 --2- 172.21.15.32:0/350249978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4268105de0 0x7f42681a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:49.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.033+0000 7f4266ffd700 1 --2- 172.21.15.32:0/350249978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4268100790 0x7f4268198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:49.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.033+0000 7f42677fe700 1 -- 172.21.15.32:0/350249978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4268100790 msgr2=0x7f4268198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:49.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.033+0000 7f42677fe700 1 --2- 172.21.15.32:0/350249978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4268100790 0x7f4268198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:49.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.033+0000 7f42677fe700 1 -- 172.21.15.32:0/350249978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4268105de0 msgr2=0x7f42681a2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:49.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.033+0000 7f42677fe700 1 --2- 172.21.15.32:0/350249978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4268105de0 0x7f42681a2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:49.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.033+0000 7f42677fe700 1 -- 172.21.15.32:0/350249978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4260009d00 con 0x7f42680ffdd0 2023-09-30T14:37:49.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.033+0000 7f42677fe700 1 --2- 172.21.15.32:0/350249978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42680ffdd0 0x7f4268197df0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f425800e9d0 tx=0x7f425800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:49.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.033+0000 7f4264ff9700 1 -- 172.21.15.32:0/350249978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258009790 con 0x7f42680ffdd0 2023-09-30T14:37:49.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.033+0000 7f426dd6e700 1 -- 172.21.15.32:0/350249978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42681a3890 con 0x7f42680ffdd0 2023-09-30T14:37:49.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.033+0000 7f4264ff9700 1 -- 172.21.15.32:0/350249978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f425800ee70 con 0x7f42680ffdd0 2023-09-30T14:37:49.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.033+0000 7f4264ff9700 1 -- 172.21.15.32:0/350249978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258010ea0 con 0x7f42680ffdd0 2023-09-30T14:37:49.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.033+0000 7f426dd6e700 1 -- 172.21.15.32:0/350249978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42681a3de0 con 0x7f42680ffdd0 2023-09-30T14:37:49.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.034+0000 7f426dd6e700 1 -- 172.21.15.32:0/350249978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426804eb20 con 0x7f42680ffdd0 2023-09-30T14:37:49.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.035+0000 7f4264ff9700 1 -- 172.21.15.32:0/350249978 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f425800cbd0 con 0x7f42680ffdd0 2023-09-30T14:37:49.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.036+0000 7f4264ff9700 1 --2- 172.21.15.32:0/350249978 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f42500a1030 0x7f42500a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:49.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.036+0000 7f4264ff9700 1 -- 172.21.15.32:0/350249978 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4258014070 con 0x7f42680ffdd0 2023-09-30T14:37:49.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.036+0000 7f4266ffd700 1 --2- 172.21.15.32:0/350249978 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f42500a1030 0x7f42500a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:49.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.036+0000 7f4266ffd700 1 --2- 172.21.15.32:0/350249978 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f42500a1030 0x7f42500a34f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4268069560 tx=0x7f425c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:49.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.040+0000 7f4264ff9700 1 -- 172.21.15.32:0/350249978 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f4258096520 con 0x7f42680ffdd0 2023-09-30T14:37:49.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.248+0000 7f426dd6e700 1 -- 172.21.15.32:0/350249978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f426810fdf0 con 0x7f42680ffdd0 2023-09-30T14:37:49.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.248+0000 7f4264ff9700 1 -- 172.21.15.32:0/350249978 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4258099b40 con 0x7f42680ffdd0 2023-09-30T14:37:49.249 INFO:teuthology.orchestra.run.smithi032.stdout:219043332131 2023-09-30T14:37:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.250+0000 7f426dd6e700 1 -- 172.21.15.32:0/350249978 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f42500a1030 msgr2=0x7f42500a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.250+0000 7f426dd6e700 1 --2- 172.21.15.32:0/350249978 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f42500a1030 0x7f42500a34f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4268069560 tx=0x7f425c009260 comp rx=0 tx=0).stop 2023-09-30T14:37:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.250+0000 7f426dd6e700 1 -- 172.21.15.32:0/350249978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42680ffdd0 msgr2=0x7f4268197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.250+0000 7f426dd6e700 1 --2- 172.21.15.32:0/350249978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42680ffdd0 0x7f4268197df0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f425800e9d0 tx=0x7f425800ece0 comp rx=0 tx=0).stop 2023-09-30T14:37:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.250+0000 7f426dd6e700 1 -- 172.21.15.32:0/350249978 shutdown_connections 2023-09-30T14:37:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.250+0000 7f426dd6e700 1 --2- 172.21.15.32:0/350249978 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f42500a1030 0x7f42500a34f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.250+0000 7f426dd6e700 1 --2- 172.21.15.32:0/350249978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4268105de0 0x7f42681a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.250+0000 7f426dd6e700 1 --2- 172.21.15.32:0/350249978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42680ffdd0 0x7f4268197df0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.250+0000 7f426dd6e700 1 --2- 172.21.15.32:0/350249978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4268100790 0x7f4268198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.250+0000 7f426dd6e700 1 -- 172.21.15.32:0/350249978 >> 172.21.15.32:0/350249978 conn(0x7f4268075d70 msgr2=0x7f42680fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.251+0000 7f426dd6e700 1 -- 172.21.15.32:0/350249978 shutdown_connections 2023-09-30T14:37:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:49.251+0000 7f426dd6e700 1 -- 172.21.15.32:0/350249978 wait complete. 2023-09-30T14:37:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:49 smithi032 ceph-mon[98592]: pgmap v271: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-30T14:37:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:49 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2876666140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-30T14:37:49.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:49 smithi156 ceph-mon[103579]: pgmap v271: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-30T14:37:49.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:49 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2876666140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-30T14:37:49.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:49 smithi148 ceph-mon[102355]: pgmap v271: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-30T14:37:49.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:49 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2876666140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-30T14:37:50.151 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103196 got 249108103199 for osd.8 2023-09-30T14:37:50.152 DEBUG:teuthology.parallel:result is None 2023-09-30T14:37:50.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:50 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/350249978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-30T14:37:50.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.408+0000 7fa226c4d700 1 -- 172.21.15.32:0/2990107520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa220100970 msgr2=0x7fa2201049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:50.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.408+0000 7fa226c4d700 1 --2- 172.21.15.32:0/2990107520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa220100970 0x7fa2201049c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009990 tx=0x7fa20c009ca0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.408+0000 7fa226c4d700 1 -- 172.21.15.32:0/2990107520 shutdown_connections 2023-09-30T14:37:50.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.408+0000 7fa226c4d700 1 --2- 172.21.15.32:0/2990107520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2201050a0 0x7fa22010cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.408+0000 7fa226c4d700 1 --2- 172.21.15.32:0/2990107520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2200fffc0 0x7fa2201003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.408+0000 7fa226c4d700 1 --2- 172.21.15.32:0/2990107520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa220100970 0x7fa2201049c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.408+0000 7fa226c4d700 1 -- 172.21.15.32:0/2990107520 >> 172.21.15.32:0/2990107520 conn(0x7fa2200fb890 msgr2=0x7fa2200fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:50.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.408+0000 7fa226c4d700 1 -- 172.21.15.32:0/2990107520 shutdown_connections 2023-09-30T14:37:50.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.408+0000 7fa226c4d700 1 -- 172.21.15.32:0/2990107520 wait complete. 2023-09-30T14:37:50.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.409+0000 7fa226c4d700 1 Processor -- start 2023-09-30T14:37:50.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.409+0000 7fa226c4d700 1 -- start start 2023-09-30T14:37:50.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.409+0000 7fa226c4d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2200fffc0 0x7fa220101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:50.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.409+0000 7fa226c4d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa220100970 0x7fa220101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:50.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.409+0000 7fa226c4d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2201050a0 0x7fa220102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:50.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.409+0000 7fa226c4d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2201a54c0 con 0x7fa2200fffc0 2023-09-30T14:37:50.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.409+0000 7fa226c4d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2201a5630 con 0x7fa220100970 2023-09-30T14:37:50.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.409+0000 7fa226c4d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2201a57f0 con 0x7fa2201050a0 2023-09-30T14:37:50.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.409+0000 7fa2249e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2200fffc0 0x7fa220101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:50.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.409+0000 7fa2249e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2200fffc0 0x7fa220101a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34230/0 (socket says 172.21.15.32:34230) 2023-09-30T14:37:50.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.409+0000 7fa2249e9700 1 -- 172.21.15.32:0/3298923826 learned_addr learned my addr 172.21.15.32:0/3298923826 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:50.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.409+0000 7fa21ffff700 1 --2- 172.21.15.32:0/3298923826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa220100970 0x7fa220101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:50.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.410+0000 7fa2249e9700 1 -- 172.21.15.32:0/3298923826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2201050a0 msgr2=0x7fa220102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:50.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.410+0000 7fa2249e9700 1 --2- 172.21.15.32:0/3298923826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2201050a0 0x7fa220102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.410+0000 7fa2249e9700 1 -- 172.21.15.32:0/3298923826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa220100970 msgr2=0x7fa220101fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:50.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.410+0000 7fa2249e9700 1 --2- 172.21.15.32:0/3298923826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa220100970 0x7fa220101fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.410+0000 7fa2249e9700 1 -- 172.21.15.32:0/3298923826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c0095f0 con 0x7fa2200fffc0 2023-09-30T14:37:50.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.410+0000 7fa2249e9700 1 --2- 172.21.15.32:0/3298923826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2200fffc0 0x7fa220101a60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa21400d950 tx=0x7fa21400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:50.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.410+0000 7fa21dffb700 1 -- 172.21.15.32:0/3298923826 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214009940 con 0x7fa2200fffc0 2023-09-30T14:37:50.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.410+0000 7fa21dffb700 1 -- 172.21.15.32:0/3298923826 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa214010460 con 0x7fa2200fffc0 2023-09-30T14:37:50.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.410+0000 7fa21dffb700 1 -- 172.21.15.32:0/3298923826 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21400f620 con 0x7fa2200fffc0 2023-09-30T14:37:50.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.410+0000 7fa226c4d700 1 -- 172.21.15.32:0/3298923826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2201a5ad0 con 0x7fa2200fffc0 2023-09-30T14:37:50.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.411+0000 7fa226c4d700 1 -- 172.21.15.32:0/3298923826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2201a5f50 con 0x7fa2200fffc0 2023-09-30T14:37:50.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.412+0000 7fa226c4d700 1 -- 172.21.15.32:0/3298923826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa220068ac0 con 0x7fa2200fffc0 2023-09-30T14:37:50.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.414+0000 7fa21dffb700 1 -- 172.21.15.32:0/3298923826 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fa2140108d0 con 0x7fa2200fffc0 2023-09-30T14:37:50.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.415+0000 7fa21dffb700 1 --2- 172.21.15.32:0/3298923826 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa2100a10b0 0x7fa2100a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:50.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.415+0000 7fa21dffb700 1 -- 172.21.15.32:0/3298923826 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa2140cc490 con 0x7fa2200fffc0 2023-09-30T14:37:50.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.415+0000 7fa21ffff700 1 --2- 172.21.15.32:0/3298923826 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa2100a10b0 0x7fa2100a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:50.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.415+0000 7fa21ffff700 1 --2- 172.21.15.32:0/3298923826 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa2100a10b0 0x7fa2100a3570 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0036c0 tx=0x7fa20c00b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.418+0000 7fa21dffb700 1 -- 172.21.15.32:0/3298923826 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fa214095ce0 con 0x7fa2200fffc0 2023-09-30T14:37:50.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:50 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/350249978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-30T14:37:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.570+0000 7f30ce097700 1 -- 172.21.15.32:0/1492975381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c8100790 msgr2=0x7f30c810cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.570+0000 7f30ce097700 1 --2- 172.21.15.32:0/1492975381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c8100790 0x7f30c810cc00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f30c000b0d0 tx=0x7f30c000b3e0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.571+0000 7f30ce097700 1 -- 172.21.15.32:0/1492975381 shutdown_connections 2023-09-30T14:37:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.571+0000 7f30ce097700 1 --2- 172.21.15.32:0/1492975381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c8100790 0x7f30c810cc00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.571+0000 7f30ce097700 1 --2- 172.21.15.32:0/1492975381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30c8105de0 0x7f30c81061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.571+0000 7f30ce097700 1 --2- 172.21.15.32:0/1492975381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c80ffdd0 0x7f30c8100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.571+0000 7f30ce097700 1 -- 172.21.15.32:0/1492975381 >> 172.21.15.32:0/1492975381 conn(0x7f30c8075d70 msgr2=0x7f30c8076180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.571+0000 7f30ce097700 1 -- 172.21.15.32:0/1492975381 shutdown_connections 2023-09-30T14:37:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.571+0000 7f30ce097700 1 -- 172.21.15.32:0/1492975381 wait complete. 2023-09-30T14:37:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.572+0000 7f30ce097700 1 Processor -- start 2023-09-30T14:37:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.572+0000 7f30ce097700 1 -- start start 2023-09-30T14:37:50.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.572+0000 7f30ce097700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c80ffdd0 0x7f30c8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:50.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.572+0000 7f30ce097700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30c8100790 0x7f30c81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:50.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.572+0000 7f30ce097700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c8105de0 0x7f30c81a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:50.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.572+0000 7f30ce097700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8198ca0 con 0x7f30c80ffdd0 2023-09-30T14:37:50.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.572+0000 7f30ce097700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c81a3460 con 0x7f30c8105de0 2023-09-30T14:37:50.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.572+0000 7f30ce097700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c81a35a0 con 0x7f30c8100790 2023-09-30T14:37:50.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.572+0000 7f30c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c80ffdd0 0x7f30c8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:50.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.572+0000 7f30c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c80ffdd0 0x7f30c8197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34258/0 (socket says 172.21.15.32:34258) 2023-09-30T14:37:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.572+0000 7f30c77fe700 1 -- 172.21.15.32:0/4149106820 learned_addr learned my addr 172.21.15.32:0/4149106820 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.572+0000 7f30c6ffd700 1 --2- 172.21.15.32:0/4149106820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30c8100790 0x7f30c81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.572+0000 7f30c7fff700 1 --2- 172.21.15.32:0/4149106820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c8105de0 0x7f30c81a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.573+0000 7f30c77fe700 1 -- 172.21.15.32:0/4149106820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30c8100790 msgr2=0x7f30c81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.573+0000 7f30c77fe700 1 --2- 172.21.15.32:0/4149106820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30c8100790 0x7f30c81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.573+0000 7f30c77fe700 1 -- 172.21.15.32:0/4149106820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c8105de0 msgr2=0x7f30c81a2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.573+0000 7f30c77fe700 1 --2- 172.21.15.32:0/4149106820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c8105de0 0x7f30c81a2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.573+0000 7f30c77fe700 1 -- 172.21.15.32:0/4149106820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c0009d00 con 0x7f30c80ffdd0 2023-09-30T14:37:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.573+0000 7f30c77fe700 1 --2- 172.21.15.32:0/4149106820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c80ffdd0 0x7f30c8197ea0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f30b800e9c0 tx=0x7f30b800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.573+0000 7f30c4ff9700 1 -- 172.21.15.32:0/4149106820 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b80097b0 con 0x7f30c80ffdd0 2023-09-30T14:37:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.573+0000 7f30ce097700 1 -- 172.21.15.32:0/4149106820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c8195db0 con 0x7f30c80ffdd0 2023-09-30T14:37:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.573+0000 7f30c4ff9700 1 -- 172.21.15.32:0/4149106820 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30b800ee60 con 0x7f30c80ffdd0 2023-09-30T14:37:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.573+0000 7f30c4ff9700 1 -- 172.21.15.32:0/4149106820 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b80054f0 con 0x7f30c80ffdd0 2023-09-30T14:37:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.573+0000 7f30ce097700 1 -- 172.21.15.32:0/4149106820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c8196260 con 0x7f30c80ffdd0 2023-09-30T14:37:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.574+0000 7f30ce097700 1 -- 172.21.15.32:0/4149106820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c804eb20 con 0x7f30c80ffdd0 2023-09-30T14:37:50.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.576+0000 7f30c4ff9700 1 -- 172.21.15.32:0/4149106820 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f30b8004750 con 0x7f30c80ffdd0 2023-09-30T14:37:50.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.577+0000 7f30c4ff9700 1 --2- 172.21.15.32:0/4149106820 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f30b00a1030 0x7f30b00a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:50.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.577+0000 7f30c6ffd700 1 --2- 172.21.15.32:0/4149106820 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f30b00a1030 0x7f30b00a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:50.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.577+0000 7f30c4ff9700 1 -- 172.21.15.32:0/4149106820 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f30b8014070 con 0x7f30c80ffdd0 2023-09-30T14:37:50.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.578+0000 7f30c6ffd700 1 --2- 172.21.15.32:0/4149106820 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f30b00a1030 0x7f30b00a34f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f30c8069170 tx=0x7f30bc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:50.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.581+0000 7f30c4ff9700 1 -- 172.21.15.32:0/4149106820 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f30b8096300 con 0x7f30c80ffdd0 2023-09-30T14:37:50.608 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:50 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/350249978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-30T14:37:50.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.634+0000 7fa226c4d700 1 -- 172.21.15.32:0/3298923826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fa220102f90 con 0x7fa2200fffc0 2023-09-30T14:37:50.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.635+0000 7fa21dffb700 1 -- 172.21.15.32:0/3298923826 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fa214016090 con 0x7fa2200fffc0 2023-09-30T14:37:50.635 INFO:teuthology.orchestra.run.smithi032.stdout:335007449106 2023-09-30T14:37:50.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.637+0000 7fa226c4d700 1 -- 172.21.15.32:0/3298923826 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa2100a10b0 msgr2=0x7fa2100a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:50.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.637+0000 7fa226c4d700 1 --2- 172.21.15.32:0/3298923826 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa2100a10b0 0x7fa2100a3570 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0036c0 tx=0x7fa20c00b560 comp rx=0 tx=0).stop 2023-09-30T14:37:50.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.637+0000 7fa226c4d700 1 -- 172.21.15.32:0/3298923826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2200fffc0 msgr2=0x7fa220101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:50.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.637+0000 7fa226c4d700 1 --2- 172.21.15.32:0/3298923826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2200fffc0 0x7fa220101a60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa21400d950 tx=0x7fa21400dc60 comp rx=0 tx=0).stop 2023-09-30T14:37:50.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.637+0000 7fa226c4d700 1 -- 172.21.15.32:0/3298923826 shutdown_connections 2023-09-30T14:37:50.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.637+0000 7fa226c4d700 1 --2- 172.21.15.32:0/3298923826 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa2100a10b0 0x7fa2100a3570 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.637+0000 7fa226c4d700 1 --2- 172.21.15.32:0/3298923826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2201050a0 0x7fa220102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.637+0000 7fa226c4d700 1 --2- 172.21.15.32:0/3298923826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2200fffc0 0x7fa220101a60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.637+0000 7fa226c4d700 1 --2- 172.21.15.32:0/3298923826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa220100970 0x7fa220101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.637+0000 7fa226c4d700 1 -- 172.21.15.32:0/3298923826 >> 172.21.15.32:0/3298923826 conn(0x7fa2200fb890 msgr2=0x7fa220069350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:50.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.637+0000 7fa226c4d700 1 -- 172.21.15.32:0/3298923826 shutdown_connections 2023-09-30T14:37:50.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.637+0000 7fa226c4d700 1 -- 172.21.15.32:0/3298923826 wait complete. 2023-09-30T14:37:50.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.790+0000 7f30ce097700 1 -- 172.21.15.32:0/4149106820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f30c810fe50 con 0x7f30c80ffdd0 2023-09-30T14:37:50.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.791+0000 7f30c4ff9700 1 -- 172.21.15.32:0/4149106820 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f30b8099920 con 0x7f30c80ffdd0 2023-09-30T14:37:50.792 INFO:teuthology.orchestra.run.smithi032.stdout:34359738434 2023-09-30T14:37:50.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.792+0000 7f30ce097700 1 -- 172.21.15.32:0/4149106820 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f30b00a1030 msgr2=0x7f30b00a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:50.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.792+0000 7f30ce097700 1 --2- 172.21.15.32:0/4149106820 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f30b00a1030 0x7f30b00a34f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f30c8069170 tx=0x7f30bc0073d0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.792+0000 7f30ce097700 1 -- 172.21.15.32:0/4149106820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c80ffdd0 msgr2=0x7f30c8197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:50.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.792+0000 7f30ce097700 1 --2- 172.21.15.32:0/4149106820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c80ffdd0 0x7f30c8197ea0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f30b800e9c0 tx=0x7f30b800ecd0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.793+0000 7f30ce097700 1 -- 172.21.15.32:0/4149106820 shutdown_connections 2023-09-30T14:37:50.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.793+0000 7f30ce097700 1 --2- 172.21.15.32:0/4149106820 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f30b00a1030 0x7f30b00a34f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.793+0000 7f30ce097700 1 --2- 172.21.15.32:0/4149106820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c8105de0 0x7f30c81a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.793+0000 7f30ce097700 1 --2- 172.21.15.32:0/4149106820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30c80ffdd0 0x7f30c8197ea0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.793+0000 7f30ce097700 1 --2- 172.21.15.32:0/4149106820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30c8100790 0x7f30c81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:50.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.793+0000 7f30ce097700 1 -- 172.21.15.32:0/4149106820 >> 172.21.15.32:0/4149106820 conn(0x7f30c8075d70 msgr2=0x7f30c8103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:50.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.793+0000 7f30ce097700 1 -- 172.21.15.32:0/4149106820 shutdown_connections 2023-09-30T14:37:50.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.793+0000 7f30ce097700 1 -- 172.21.15.32:0/4149106820 wait complete. 2023-09-30T14:37:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.491+0000 7f6dcbe70700 1 -- 172.21.15.32:0/1953629591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc40fff90 msgr2=0x7f6dc4105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.491+0000 7f6dcbe70700 1 --2- 172.21.15.32:0/1953629591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc40fff90 0x7f6dc4105ad0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009990 tx=0x7f6db4009ca0 comp rx=0 tx=0).stop 2023-09-30T14:37:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.492+0000 7f6dcbe70700 1 -- 172.21.15.32:0/1953629591 shutdown_connections 2023-09-30T14:37:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.492+0000 7f6dcbe70700 1 --2- 172.21.15.32:0/1953629591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dc41061b0 0x7f6dc410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.492+0000 7f6dcbe70700 1 --2- 172.21.15.32:0/1953629591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc40ff5e0 0x7f6dc40ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.492+0000 7f6dcbe70700 1 --2- 172.21.15.32:0/1953629591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc40fff90 0x7f6dc4105ad0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.492+0000 7f6dcbe70700 1 -- 172.21.15.32:0/1953629591 >> 172.21.15.32:0/1953629591 conn(0x7f6dc40762d0 msgr2=0x7f6dc40fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.492+0000 7f6dcbe70700 1 -- 172.21.15.32:0/1953629591 shutdown_connections 2023-09-30T14:37:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.492+0000 7f6dcbe70700 1 -- 172.21.15.32:0/1953629591 wait complete. 2023-09-30T14:37:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.492+0000 7f6dcbe70700 1 Processor -- start 2023-09-30T14:37:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.492+0000 7f6dcbe70700 1 -- start start 2023-09-30T14:37:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dcbe70700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc40ff5e0 0x7f6dc4196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dcbe70700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc40fff90 0x7f6dc4197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dcbe70700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dc41061b0 0x7f6dc419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dcbe70700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc419bf00 con 0x7f6dc40fff90 2023-09-30T14:37:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dcbe70700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc419c070 con 0x7f6dc40ff5e0 2023-09-30T14:37:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dcbe70700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc419c230 con 0x7f6dc41061b0 2023-09-30T14:37:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dc940b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc40fff90 0x7f6dc4197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dc940b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc40fff90 0x7f6dc4197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34246/0 (socket says 172.21.15.32:34246) 2023-09-30T14:37:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dc940b700 1 -- 172.21.15.32:0/2198323298 learned_addr learned my addr 172.21.15.32:0/2198323298 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dc9c0c700 1 --2- 172.21.15.32:0/2198323298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc40ff5e0 0x7f6dc4196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dca40d700 1 --2- 172.21.15.32:0/2198323298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dc41061b0 0x7f6dc419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dc940b700 1 -- 172.21.15.32:0/2198323298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dc41061b0 msgr2=0x7f6dc419b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dc940b700 1 --2- 172.21.15.32:0/2198323298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dc41061b0 0x7f6dc419b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dc940b700 1 -- 172.21.15.32:0/2198323298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc40ff5e0 msgr2=0x7f6dc4196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dc940b700 1 --2- 172.21.15.32:0/2198323298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc40ff5e0 0x7f6dc4196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dc940b700 1 -- 172.21.15.32:0/2198323298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db40095f0 con 0x7f6dc40fff90 2023-09-30T14:37:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dc940b700 1 --2- 172.21.15.32:0/2198323298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc40fff90 0x7f6dc4197480 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6db4000c00 tx=0x7f6db400f6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dbaffd700 1 -- 172.21.15.32:0/2198323298 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db401d070 con 0x7f6dc40fff90 2023-09-30T14:37:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.493+0000 7f6dcbe70700 1 -- 172.21.15.32:0/2198323298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc419c4b0 con 0x7f6dc40fff90 2023-09-30T14:37:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.494+0000 7f6dbaffd700 1 -- 172.21.15.32:0/2198323298 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6db4022470 con 0x7f6dc40fff90 2023-09-30T14:37:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.494+0000 7f6dbaffd700 1 -- 172.21.15.32:0/2198323298 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4017650 con 0x7f6dc40fff90 2023-09-30T14:37:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.494+0000 7f6dcbe70700 1 -- 172.21.15.32:0/2198323298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc41a3d70 con 0x7f6dc40fff90 2023-09-30T14:37:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.494+0000 7f6dcbe70700 1 -- 172.21.15.32:0/2198323298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc404eb20 con 0x7f6dc40fff90 2023-09-30T14:37:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.496+0000 7f6dbaffd700 1 -- 172.21.15.32:0/2198323298 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f6db40225e0 con 0x7f6dc40fff90 2023-09-30T14:37:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.496+0000 7f6dbaffd700 1 --2- 172.21.15.32:0/2198323298 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6db00a1030 0x7f6db00a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.497+0000 7f6dc9c0c700 1 --2- 172.21.15.32:0/2198323298 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6db00a1030 0x7f6db00a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.497+0000 7f6dbaffd700 1 -- 172.21.15.32:0/2198323298 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6db40cdbf0 con 0x7f6dc40fff90 2023-09-30T14:37:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.497+0000 7f6dc9c0c700 1 --2- 172.21.15.32:0/2198323298 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6db00a1030 0x7f6db00a34f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005950 tx=0x7f6dc000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.504+0000 7f6dbaffd700 1 -- 172.21.15.32:0/2198323298 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f6db40974d0 con 0x7f6dc40fff90 2023-09-30T14:37:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.714+0000 7f6dcbe70700 1 -- 172.21.15.32:0/2198323298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6dc41982a0 con 0x7f6dc40fff90 2023-09-30T14:37:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.714+0000 7f6dbaffd700 1 -- 172.21.15.32:0/2198323298 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6db409aaf0 con 0x7f6dc40fff90 2023-09-30T14:37:51.029 INFO:teuthology.orchestra.run.smithi032.stdout:111669149749 2023-09-30T14:37:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.717+0000 7f6dcbe70700 1 -- 172.21.15.32:0/2198323298 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6db00a1030 msgr2=0x7f6db00a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.717+0000 7f6dcbe70700 1 --2- 172.21.15.32:0/2198323298 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6db00a1030 0x7f6db00a34f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005950 tx=0x7f6dc000a300 comp rx=0 tx=0).stop 2023-09-30T14:37:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.717+0000 7f6dcbe70700 1 -- 172.21.15.32:0/2198323298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc40fff90 msgr2=0x7f6dc4197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.717+0000 7f6dcbe70700 1 --2- 172.21.15.32:0/2198323298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc40fff90 0x7f6dc4197480 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6db4000c00 tx=0x7f6db400f6e0 comp rx=0 tx=0).stop 2023-09-30T14:37:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.717+0000 7f6dcbe70700 1 -- 172.21.15.32:0/2198323298 shutdown_connections 2023-09-30T14:37:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.717+0000 7f6dcbe70700 1 --2- 172.21.15.32:0/2198323298 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6db00a1030 0x7f6db00a34f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.717+0000 7f6dcbe70700 1 --2- 172.21.15.32:0/2198323298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dc41061b0 0x7f6dc419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.717+0000 7f6dcbe70700 1 --2- 172.21.15.32:0/2198323298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc40ff5e0 0x7f6dc4196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.717+0000 7f6dcbe70700 1 --2- 172.21.15.32:0/2198323298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dc40fff90 0x7f6dc4197480 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.717+0000 7f6dcbe70700 1 -- 172.21.15.32:0/2198323298 >> 172.21.15.32:0/2198323298 conn(0x7f6dc40762d0 msgr2=0x7f6dc4104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.717+0000 7f6dcbe70700 1 -- 172.21.15.32:0/2198323298 shutdown_connections 2023-09-30T14:37:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:50.717+0000 7f6dcbe70700 1 -- 172.21.15.32:0/2198323298 wait complete. 2023-09-30T14:37:51.133 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332130 got 219043332131 for osd.7 2023-09-30T14:37:51.134 DEBUG:teuthology.parallel:result is None 2023-09-30T14:37:51.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:51 smithi032 ceph-mon[98592]: pgmap v272: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-30T14:37:51.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:51 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3298923826' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-30T14:37:51.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:51 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2198323298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-30T14:37:51.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:51 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/4149106820' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-30T14:37:51.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:51 smithi156 ceph-mon[103579]: pgmap v272: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-30T14:37:51.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:51 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3298923826' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-30T14:37:51.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:51 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2198323298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-30T14:37:51.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:51 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/4149106820' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-30T14:37:51.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:51 smithi148 ceph-mon[102355]: pgmap v272: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-09-30T14:37:51.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:51 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3298923826' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-30T14:37:51.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:51 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2198323298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-30T14:37:51.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:51 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/4149106820' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-30T14:37:51.952 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738431 got 34359738434 for osd.0 2023-09-30T14:37:51.953 DEBUG:teuthology.parallel:result is None 2023-09-30T14:37:51.986 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449104 got 335007449106 for osd.11 2023-09-30T14:37:51.986 DEBUG:teuthology.parallel:result is None 2023-09-30T14:37:52.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.561+0000 7f7c55c71700 1 -- 172.21.15.32:0/4136343562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c500ffdd0 msgr2=0x7f7c50100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:52.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.561+0000 7f7c55c71700 1 --2- 172.21.15.32:0/4136343562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c500ffdd0 0x7f7c50100250 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7c440099c0 tx=0x7f7c44009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.562+0000 7f7c55c71700 1 -- 172.21.15.32:0/4136343562 shutdown_connections 2023-09-30T14:37:52.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.562+0000 7f7c55c71700 1 --2- 172.21.15.32:0/4136343562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c50100790 0x7f7c5010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.562+0000 7f7c55c71700 1 --2- 172.21.15.32:0/4136343562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c50105de0 0x7f7c501061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.562+0000 7f7c55c71700 1 --2- 172.21.15.32:0/4136343562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c500ffdd0 0x7f7c50100250 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.562+0000 7f7c55c71700 1 -- 172.21.15.32:0/4136343562 >> 172.21.15.32:0/4136343562 conn(0x7f7c50075d70 msgr2=0x7f7c50076180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:52.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.562+0000 7f7c55c71700 1 -- 172.21.15.32:0/4136343562 shutdown_connections 2023-09-30T14:37:52.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.562+0000 7f7c55c71700 1 -- 172.21.15.32:0/4136343562 wait complete. 2023-09-30T14:37:52.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.563+0000 7f7c55c71700 1 Processor -- start 2023-09-30T14:37:52.565 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149747 got 111669149749 for osd.3 2023-09-30T14:37:52.566 DEBUG:teuthology.parallel:result is None 2023-09-30T14:37:52.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.563+0000 7f7c55c71700 1 -- start start 2023-09-30T14:37:52.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.563+0000 7f7c55c71700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c500ffdd0 0x7f7c5019dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:52.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.563+0000 7f7c55c71700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c50100790 0x7f7c5019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:52.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.563+0000 7f7c55c71700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c50105de0 0x7f7c50199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:52.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.563+0000 7f7c55c71700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c50199eb0 con 0x7f7c50100790 2023-09-30T14:37:52.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.563+0000 7f7c55c71700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5019a020 con 0x7f7c500ffdd0 2023-09-30T14:37:52.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.563+0000 7f7c55c71700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5019a1e0 con 0x7f7c50105de0 2023-09-30T14:37:52.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.563+0000 7f7c4effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c50100790 0x7f7c5019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.563+0000 7f7c4effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c50100790 0x7f7c5019e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34274/0 (socket says 172.21.15.32:34274) 2023-09-30T14:37:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.563+0000 7f7c4effd700 1 -- 172.21.15.32:0/3977011106 learned_addr learned my addr 172.21.15.32:0/3977011106 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.563+0000 7f7c4f7fe700 1 --2- 172.21.15.32:0/3977011106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c500ffdd0 0x7f7c5019dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.564+0000 7f7c4ffff700 1 --2- 172.21.15.32:0/3977011106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c50105de0 0x7f7c50199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.564+0000 7f7c4effd700 1 -- 172.21.15.32:0/3977011106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c50105de0 msgr2=0x7f7c50199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.564+0000 7f7c4effd700 1 --2- 172.21.15.32:0/3977011106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c50105de0 0x7f7c50199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.564+0000 7f7c4effd700 1 -- 172.21.15.32:0/3977011106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c500ffdd0 msgr2=0x7f7c5019dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.564+0000 7f7c4effd700 1 --2- 172.21.15.32:0/3977011106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c500ffdd0 0x7f7c5019dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.564+0000 7f7c4effd700 1 -- 172.21.15.32:0/3977011106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c440095f0 con 0x7f7c50100790 2023-09-30T14:37:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.564+0000 7f7c4effd700 1 --2- 172.21.15.32:0/3977011106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c50100790 0x7f7c5019e320 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7c44004730 tx=0x7f7c44004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.564+0000 7f7c4cff9700 1 -- 172.21.15.32:0/3977011106 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4401d070 con 0x7f7c50100790 2023-09-30T14:37:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.564+0000 7f7c4cff9700 1 -- 172.21.15.32:0/3977011106 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c44004cd0 con 0x7f7c50100790 2023-09-30T14:37:52.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.564+0000 7f7c55c71700 1 -- 172.21.15.32:0/3977011106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5019a400 con 0x7f7c50100790 2023-09-30T14:37:52.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.564+0000 7f7c4cff9700 1 -- 172.21.15.32:0/3977011106 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c44005290 con 0x7f7c50100790 2023-09-30T14:37:52.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.564+0000 7f7c55c71700 1 -- 172.21.15.32:0/3977011106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c501a5ee0 con 0x7f7c50100790 2023-09-30T14:37:52.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.565+0000 7f7c55c71700 1 -- 172.21.15.32:0/3977011106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5004eb20 con 0x7f7c50100790 2023-09-30T14:37:52.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.566+0000 7f7c4cff9700 1 -- 172.21.15.32:0/3977011106 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f7c44003b90 con 0x7f7c50100790 2023-09-30T14:37:52.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.567+0000 7f7c4cff9700 1 --2- 172.21.15.32:0/3977011106 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7c380a1030 0x7f7c380a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:52.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.567+0000 7f7c4f7fe700 1 --2- 172.21.15.32:0/3977011106 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7c380a1030 0x7f7c380a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:52.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.567+0000 7f7c4cff9700 1 -- 172.21.15.32:0/3977011106 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7c440cd990 con 0x7f7c50100790 2023-09-30T14:37:52.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.568+0000 7f7c4f7fe700 1 --2- 172.21.15.32:0/3977011106 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7c380a1030 0x7f7c380a34f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000abd0 tx=0x7f7c4000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:52.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.571+0000 7f7c4cff9700 1 -- 172.21.15.32:0/3977011106 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f7c44097270 con 0x7f7c50100790 2023-09-30T14:37:52.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.788+0000 7f7c55c71700 1 -- 172.21.15.32:0/3977011106 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7c5010fd00 con 0x7f7c50100790 2023-09-30T14:37:52.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.788+0000 7f7c4cff9700 1 -- 172.21.15.32:0/3977011106 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7c4409a890 con 0x7f7c50100790 2023-09-30T14:37:52.789 INFO:teuthology.orchestra.run.smithi032.stdout:55834574910 2023-09-30T14:37:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.790+0000 7f7c55c71700 1 -- 172.21.15.32:0/3977011106 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7c380a1030 msgr2=0x7f7c380a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.790+0000 7f7c55c71700 1 --2- 172.21.15.32:0/3977011106 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7c380a1030 0x7f7c380a34f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000abd0 tx=0x7f7c4000a380 comp rx=0 tx=0).stop 2023-09-30T14:37:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.790+0000 7f7c55c71700 1 -- 172.21.15.32:0/3977011106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c50100790 msgr2=0x7f7c5019e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.790+0000 7f7c55c71700 1 --2- 172.21.15.32:0/3977011106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c50100790 0x7f7c5019e320 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7c44004730 tx=0x7f7c44004760 comp rx=0 tx=0).stop 2023-09-30T14:37:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.790+0000 7f7c55c71700 1 -- 172.21.15.32:0/3977011106 shutdown_connections 2023-09-30T14:37:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.791+0000 7f7c55c71700 1 --2- 172.21.15.32:0/3977011106 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7c380a1030 0x7f7c380a34f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.791+0000 7f7c55c71700 1 --2- 172.21.15.32:0/3977011106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c50105de0 0x7f7c50199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.791+0000 7f7c55c71700 1 --2- 172.21.15.32:0/3977011106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c500ffdd0 0x7f7c5019dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.791+0000 7f7c55c71700 1 --2- 172.21.15.32:0/3977011106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c50100790 0x7f7c5019e320 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.791+0000 7f7c55c71700 1 -- 172.21.15.32:0/3977011106 >> 172.21.15.32:0/3977011106 conn(0x7f7c50075d70 msgr2=0x7f7c500fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.791+0000 7f7c55c71700 1 -- 172.21.15.32:0/3977011106 shutdown_connections 2023-09-30T14:37:52.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.791+0000 7f7c55c71700 1 -- 172.21.15.32:0/3977011106 wait complete. 2023-09-30T14:37:52.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.879+0000 7f80d5c2e700 1 -- 172.21.15.32:0/4110276981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d0101aa0 msgr2=0x7f80d0105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:52.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.879+0000 7f80d5c2e700 1 --2- 172.21.15.32:0/4110276981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d0101aa0 0x7f80d0105af0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009990 tx=0x7f80c4009ca0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.879+0000 7f80d5c2e700 1 -- 172.21.15.32:0/4110276981 shutdown_connections 2023-09-30T14:37:52.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.879+0000 7f80d5c2e700 1 --2- 172.21.15.32:0/4110276981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80d01061d0 0x7f80d010dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.879+0000 7f80d5c2e700 1 --2- 172.21.15.32:0/4110276981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80d01010f0 0x7f80d01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.879+0000 7f80d5c2e700 1 --2- 172.21.15.32:0/4110276981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d0101aa0 0x7f80d0105af0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.879+0000 7f80d5c2e700 1 -- 172.21.15.32:0/4110276981 >> 172.21.15.32:0/4110276981 conn(0x7f80d00fc940 msgr2=0x7f80d00fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:52.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.879+0000 7f80d5c2e700 1 -- 172.21.15.32:0/4110276981 shutdown_connections 2023-09-30T14:37:52.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.880+0000 7f80d5c2e700 1 -- 172.21.15.32:0/4110276981 wait complete. 2023-09-30T14:37:52.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.880+0000 7f80d5c2e700 1 Processor -- start 2023-09-30T14:37:52.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.881+0000 7f80d5c2e700 1 -- start start 2023-09-30T14:37:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.881+0000 7f80d5c2e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d01010f0 0x7f80d0073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.881+0000 7f80d5c2e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80d0101aa0 0x7f80d0073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.881+0000 7f80d5c2e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80d01061d0 0x7f80d0073be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.881+0000 7f80d5c2e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d01a53f0 con 0x7f80d01010f0 2023-09-30T14:37:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.881+0000 7f80d5c2e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d01a5560 con 0x7f80d01061d0 2023-09-30T14:37:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.881+0000 7f80d5c2e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d01a5720 con 0x7f80d0101aa0 2023-09-30T14:37:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.881+0000 7f80cf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d01010f0 0x7f80d0073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.881+0000 7f80cf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d01010f0 0x7f80d0073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34286/0 (socket says 172.21.15.32:34286) 2023-09-30T14:37:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.881+0000 7f80cf7fe700 1 -- 172.21.15.32:0/3209394751 learned_addr learned my addr 172.21.15.32:0/3209394751 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.881+0000 7f80cffff700 1 --2- 172.21.15.32:0/3209394751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80d01061d0 0x7f80d0073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.881+0000 7f80ceffd700 1 --2- 172.21.15.32:0/3209394751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80d0101aa0 0x7f80d0073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.881+0000 7f80cf7fe700 1 -- 172.21.15.32:0/3209394751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80d0101aa0 msgr2=0x7f80d0073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.882+0000 7f80cf7fe700 1 --2- 172.21.15.32:0/3209394751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80d0101aa0 0x7f80d0073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.882+0000 7f80cf7fe700 1 -- 172.21.15.32:0/3209394751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80d01061d0 msgr2=0x7f80d0073be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.882+0000 7f80cf7fe700 1 --2- 172.21.15.32:0/3209394751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80d01061d0 0x7f80d0073be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.882+0000 7f80cf7fe700 1 -- 172.21.15.32:0/3209394751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c40095f0 con 0x7f80d01010f0 2023-09-30T14:37:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.882+0000 7f80cf7fe700 1 --2- 172.21.15.32:0/3209394751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d01010f0 0x7f80d0073120 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f80c000e9a0 tx=0x7f80c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.882+0000 7f80ccff9700 1 -- 172.21.15.32:0/3209394751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c0009790 con 0x7f80d01010f0 2023-09-30T14:37:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.882+0000 7f80ccff9700 1 -- 172.21.15.32:0/3209394751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80c0010430 con 0x7f80d01010f0 2023-09-30T14:37:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.882+0000 7f80ccff9700 1 -- 172.21.15.32:0/3209394751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c0018680 con 0x7f80d01010f0 2023-09-30T14:37:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.882+0000 7f80d5c2e700 1 -- 172.21.15.32:0/3209394751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d01a5a00 con 0x7f80d01010f0 2023-09-30T14:37:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.882+0000 7f80d5c2e700 1 -- 172.21.15.32:0/3209394751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d01a5eb0 con 0x7f80d01010f0 2023-09-30T14:37:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.883+0000 7f80d5c2e700 1 -- 172.21.15.32:0/3209394751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d004eb20 con 0x7f80d01010f0 2023-09-30T14:37:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.885+0000 7f80ccff9700 1 -- 172.21.15.32:0/3209394751 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f80c00105a0 con 0x7f80d01010f0 2023-09-30T14:37:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.885+0000 7f80ccff9700 1 --2- 172.21.15.32:0/3209394751 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f80b80a10b0 0x7f80b80a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.886+0000 7f80ceffd700 1 --2- 172.21.15.32:0/3209394751 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f80b80a10b0 0x7f80b80a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.886+0000 7f80ccff9700 1 -- 172.21.15.32:0/3209394751 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f80c0014070 con 0x7f80d01010f0 2023-09-30T14:37:52.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.886+0000 7f80ceffd700 1 --2- 172.21.15.32:0/3209394751 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f80b80a10b0 0x7f80b80a3570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f80d0074910 tx=0x7f80c4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:52.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.891+0000 7f80ccff9700 1 -- 172.21.15.32:0/3209394751 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f80c0051f30 con 0x7f80d01010f0 2023-09-30T14:37:52.940 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:52.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.941+0000 7fbae1d1f700 1 -- 172.21.15.32:0/330668625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbadc1008e0 msgr2=0x7fbadc100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:52.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.941+0000 7fbae1d1f700 1 --2- 172.21.15.32:0/330668625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbadc1008e0 0x7fbadc100d60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbad00098f0 tx=0x7fbad0009c00 comp rx=0 tx=0).stop 2023-09-30T14:37:52.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.941+0000 7fbae1d1f700 1 -- 172.21.15.32:0/330668625 shutdown_connections 2023-09-30T14:37:52.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.941+0000 7fbae1d1f700 1 --2- 172.21.15.32:0/330668625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc1012a0 0x7fbadc10cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.941+0000 7fbae1d1f700 1 --2- 172.21.15.32:0/330668625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc104dd0 0x7fbadc1051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.941+0000 7fbae1d1f700 1 --2- 172.21.15.32:0/330668625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbadc1008e0 0x7fbadc100d60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.941+0000 7fbae1d1f700 1 -- 172.21.15.32:0/330668625 >> 172.21.15.32:0/330668625 conn(0x7fbadc076320 msgr2=0x7fbadc076730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:52.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.941+0000 7fbae1d1f700 1 -- 172.21.15.32:0/330668625 shutdown_connections 2023-09-30T14:37:52.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.942+0000 7fbae1d1f700 1 -- 172.21.15.32:0/330668625 wait complete. 2023-09-30T14:37:52.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.942+0000 7fbae1d1f700 1 Processor -- start 2023-09-30T14:37:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbae1d1f700 1 -- start start 2023-09-30T14:37:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbae1d1f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbadc1008e0 0x7fbadc195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbae1d1f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc1012a0 0x7fbadc196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbae1d1f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc104dd0 0x7fbadc19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbae1d1f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc19adc0 con 0x7fbadc1012a0 2023-09-30T14:37:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbae1d1f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc19af30 con 0x7fbadc1008e0 2023-09-30T14:37:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbae1d1f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc19b0f0 con 0x7fbadc104dd0 2023-09-30T14:37:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbadaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc1012a0 0x7fbadc196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbadaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc1012a0 0x7fbadc196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34302/0 (socket says 172.21.15.32:34302) 2023-09-30T14:37:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbadb7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbadc1008e0 0x7fbadc195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbadaffd700 1 -- 172.21.15.32:0/2431951628 learned_addr learned my addr 172.21.15.32:0/2431951628 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbadbfff700 1 --2- 172.21.15.32:0/2431951628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc104dd0 0x7fbadc19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbadaffd700 1 -- 172.21.15.32:0/2431951628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc104dd0 msgr2=0x7fbadc19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbadaffd700 1 --2- 172.21.15.32:0/2431951628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc104dd0 0x7fbadc19a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbadaffd700 1 -- 172.21.15.32:0/2431951628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbadc1008e0 msgr2=0x7fbadc195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbadaffd700 1 --2- 172.21.15.32:0/2431951628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbadc1008e0 0x7fbadc195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.943+0000 7fbadaffd700 1 -- 172.21.15.32:0/2431951628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad0009520 con 0x7fbadc1012a0 2023-09-30T14:37:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.944+0000 7fbadb7fe700 1 --2- 172.21.15.32:0/2431951628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbadc1008e0 0x7fbadc195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:37:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.944+0000 7fbadaffd700 1 --2- 172.21.15.32:0/2431951628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc1012a0 0x7fbadc196340 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009230 tx=0x7fbad000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.944+0000 7fbad8ff9700 1 -- 172.21.15.32:0/2431951628 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad001c070 con 0x7fbadc1012a0 2023-09-30T14:37:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.944+0000 7fbae1d1f700 1 -- 172.21.15.32:0/2431951628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbadc19b370 con 0x7fbadc1012a0 2023-09-30T14:37:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.944+0000 7fbad8ff9700 1 -- 172.21.15.32:0/2431951628 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbad00058e0 con 0x7fbadc1012a0 2023-09-30T14:37:52.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.944+0000 7fbae1d1f700 1 -- 172.21.15.32:0/2431951628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbadc1a2c30 con 0x7fbadc1012a0 2023-09-30T14:37:52.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.945+0000 7fbae1d1f700 1 -- 172.21.15.32:0/2431951628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbadc04eb20 con 0x7fbadc1012a0 2023-09-30T14:37:52.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.945+0000 7fbad8ff9700 1 -- 172.21.15.32:0/2431951628 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad00038e0 con 0x7fbadc1012a0 2023-09-30T14:37:52.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.947+0000 7fbad8ff9700 1 -- 172.21.15.32:0/2431951628 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fbad0020030 con 0x7fbadc1012a0 2023-09-30T14:37:52.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.948+0000 7fbad8ff9700 1 --2- 172.21.15.32:0/2431951628 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbac40a10b0 0x7fbac40a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:52.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.948+0000 7fbadb7fe700 1 --2- 172.21.15.32:0/2431951628 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbac40a10b0 0x7fbac40a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:52.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.948+0000 7fbad8ff9700 1 -- 172.21.15.32:0/2431951628 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbad00cd550 con 0x7fbadc1012a0 2023-09-30T14:37:52.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.948+0000 7fbadb7fe700 1 --2- 172.21.15.32:0/2431951628 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbac40a10b0 0x7fbac40a3570 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbadc073380 tx=0x7fbacc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:52.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:52.952+0000 7fbad8ff9700 1 -- 172.21.15.32:0/2431951628 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fbad0096e30 con 0x7fbadc1012a0 2023-09-30T14:37:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.138+0000 7f80d5c2e700 1 -- 172.21.15.32:0/3209394751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f80d00008d0 con 0x7f80d01010f0 2023-09-30T14:37:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.138+0000 7f80ccff9700 1 -- 172.21.15.32:0/3209394751 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f80c0096070 con 0x7f80d01010f0 2023-09-30T14:37:53.139 INFO:teuthology.orchestra.run.smithi032.stdout:304942678039 2023-09-30T14:37:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.141+0000 7f80d5c2e700 1 -- 172.21.15.32:0/3209394751 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f80b80a10b0 msgr2=0x7f80b80a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.141+0000 7f80d5c2e700 1 --2- 172.21.15.32:0/3209394751 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f80b80a10b0 0x7f80b80a3570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f80d0074910 tx=0x7f80c4005fd0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.141+0000 7f80d5c2e700 1 -- 172.21.15.32:0/3209394751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d01010f0 msgr2=0x7f80d0073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.141+0000 7f80d5c2e700 1 --2- 172.21.15.32:0/3209394751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d01010f0 0x7f80d0073120 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f80c000e9a0 tx=0x7f80c000ecb0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.141+0000 7f80d5c2e700 1 -- 172.21.15.32:0/3209394751 shutdown_connections 2023-09-30T14:37:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.141+0000 7f80d5c2e700 1 --2- 172.21.15.32:0/3209394751 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f80b80a10b0 0x7f80b80a3570 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.141+0000 7f80d5c2e700 1 --2- 172.21.15.32:0/3209394751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80d01061d0 0x7f80d0073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.141+0000 7f80d5c2e700 1 --2- 172.21.15.32:0/3209394751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d01010f0 0x7f80d0073120 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.141+0000 7f80d5c2e700 1 --2- 172.21.15.32:0/3209394751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80d0101aa0 0x7f80d0073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.141+0000 7f80d5c2e700 1 -- 172.21.15.32:0/3209394751 >> 172.21.15.32:0/3209394751 conn(0x7f80d00fc940 msgr2=0x7f80d00fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:53.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.141+0000 7f80d5c2e700 1 -- 172.21.15.32:0/3209394751 shutdown_connections 2023-09-30T14:37:53.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.141+0000 7f80d5c2e700 1 -- 172.21.15.32:0/3209394751 wait complete. 2023-09-30T14:37:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.182+0000 7fbae1d1f700 1 -- 172.21.15.32:0/2431951628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fbadc10ffb0 con 0x7fbadc1012a0 2023-09-30T14:37:53.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.183+0000 7fbad8ff9700 1 -- 172.21.15.32:0/2431951628 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbad009a450 con 0x7fbadc1012a0 2023-09-30T14:37:53.184 INFO:teuthology.orchestra.run.smithi032.stdout:197568495657 2023-09-30T14:37:53.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.185+0000 7fbae1d1f700 1 -- 172.21.15.32:0/2431951628 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbac40a10b0 msgr2=0x7fbac40a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.185+0000 7fbae1d1f700 1 --2- 172.21.15.32:0/2431951628 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbac40a10b0 0x7fbac40a3570 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbadc073380 tx=0x7fbacc009260 comp rx=0 tx=0).stop 2023-09-30T14:37:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.185+0000 7fbae1d1f700 1 -- 172.21.15.32:0/2431951628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc1012a0 msgr2=0x7fbadc196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.185+0000 7fbae1d1f700 1 --2- 172.21.15.32:0/2431951628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc1012a0 0x7fbadc196340 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009230 tx=0x7fbad000fa00 comp rx=0 tx=0).stop 2023-09-30T14:37:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.185+0000 7fbae1d1f700 1 -- 172.21.15.32:0/2431951628 shutdown_connections 2023-09-30T14:37:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.185+0000 7fbae1d1f700 1 --2- 172.21.15.32:0/2431951628 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbac40a10b0 0x7fbac40a3570 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.185+0000 7fbae1d1f700 1 --2- 172.21.15.32:0/2431951628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbadc104dd0 0x7fbadc19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.185+0000 7fbae1d1f700 1 --2- 172.21.15.32:0/2431951628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbadc1008e0 0x7fbadc195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.185+0000 7fbae1d1f700 1 --2- 172.21.15.32:0/2431951628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbadc1012a0 0x7fbadc196340 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.185+0000 7fbae1d1f700 1 -- 172.21.15.32:0/2431951628 >> 172.21.15.32:0/2431951628 conn(0x7fbadc076320 msgr2=0x7fbadc0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.185+0000 7fbae1d1f700 1 -- 172.21.15.32:0/2431951628 shutdown_connections 2023-09-30T14:37:53.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.185+0000 7fbae1d1f700 1 -- 172.21.15.32:0/2431951628 wait complete. 2023-09-30T14:37:53.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:53 smithi032 ceph-mon[98592]: pgmap v273: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-30T14:37:53.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:53 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3977011106' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-30T14:37:53.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:53 smithi156 ceph-mon[103579]: pgmap v273: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-30T14:37:53.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:53 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3977011106' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-30T14:37:53.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:53 smithi148 ceph-mon[102355]: pgmap v273: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-30T14:37:53.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:53 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3977011106' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-30T14:37:53.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.508+0000 7fdc56732700 1 -- 172.21.15.32:0/4013585756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc50106f00 msgr2=0x7fdc501072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:53.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.508+0000 7fdc56732700 1 --2- 172.21.15.32:0/4013585756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc50106f00 0x7fdc501072e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdc400099c0 tx=0x7fdc40009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.508+0000 7fdc56732700 1 -- 172.21.15.32:0/4013585756 shutdown_connections 2023-09-30T14:37:53.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.508+0000 7fdc56732700 1 --2- 172.21.15.32:0/4013585756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc500692d0 0x7fdc5010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.508+0000 7fdc56732700 1 --2- 172.21.15.32:0/4013585756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc50106f00 0x7fdc501072e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.508+0000 7fdc56732700 1 --2- 172.21.15.32:0/4013585756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc50068910 0x7fdc50068d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.508+0000 7fdc56732700 1 -- 172.21.15.32:0/4013585756 >> 172.21.15.32:0/4013585756 conn(0x7fdc50075d70 msgr2=0x7fdc50076180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:53.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.508+0000 7fdc56732700 1 -- 172.21.15.32:0/4013585756 shutdown_connections 2023-09-30T14:37:53.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.508+0000 7fdc56732700 1 -- 172.21.15.32:0/4013585756 wait complete. 2023-09-30T14:37:53.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.508+0000 7fdc56732700 1 Processor -- start 2023-09-30T14:37:53.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.508+0000 7fdc56732700 1 -- start start 2023-09-30T14:37:53.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc56732700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc50068910 0x7fdc500ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:53.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc56732700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc500692d0 0x7fdc501003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:53.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc56732700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc50106f00 0x7fdc50100960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:53.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc56732700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc501a5390 con 0x7fdc500692d0 2023-09-30T14:37:53.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc56732700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc501a5500 con 0x7fdc50068910 2023-09-30T14:37:53.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc56732700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc501a56c0 con 0x7fdc50106f00 2023-09-30T14:37:53.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc4f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc500692d0 0x7fdc501003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:53.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc4f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc500692d0 0x7fdc501003e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34316/0 (socket says 172.21.15.32:34316) 2023-09-30T14:37:53.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc4f7fe700 1 -- 172.21.15.32:0/590415073 learned_addr learned my addr 172.21.15.32:0/590415073 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:53.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc4ffff700 1 --2- 172.21.15.32:0/590415073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc50068910 0x7fdc500ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:53.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc54ccf700 1 --2- 172.21.15.32:0/590415073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc50106f00 0x7fdc50100960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:53.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc4f7fe700 1 -- 172.21.15.32:0/590415073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc50106f00 msgr2=0x7fdc50100960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:53.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc4f7fe700 1 --2- 172.21.15.32:0/590415073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc50106f00 0x7fdc50100960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc4f7fe700 1 -- 172.21.15.32:0/590415073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc50068910 msgr2=0x7fdc500ffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:53.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc4f7fe700 1 --2- 172.21.15.32:0/590415073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc50068910 0x7fdc500ffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc4f7fe700 1 -- 172.21.15.32:0/590415073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc400095f0 con 0x7fdc500692d0 2023-09-30T14:37:53.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc4ffff700 1 --2- 172.21.15.32:0/590415073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc50068910 0x7fdc500ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:37:53.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc4f7fe700 1 --2- 172.21.15.32:0/590415073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc500692d0 0x7fdc501003e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400c920 tx=0x7fdc4400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:53.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc4d7fa700 1 -- 172.21.15.32:0/590415073 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc44007b00 con 0x7fdc500692d0 2023-09-30T14:37:53.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc4d7fa700 1 -- 172.21.15.32:0/590415073 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc44004cf0 con 0x7fdc500692d0 2023-09-30T14:37:53.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc56732700 1 -- 172.21.15.32:0/590415073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc501a59a0 con 0x7fdc500692d0 2023-09-30T14:37:53.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.509+0000 7fdc4d7fa700 1 -- 172.21.15.32:0/590415073 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc44005610 con 0x7fdc500692d0 2023-09-30T14:37:53.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.510+0000 7fdc56732700 1 -- 172.21.15.32:0/590415073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc501a5ea0 con 0x7fdc500692d0 2023-09-30T14:37:53.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.510+0000 7fdc56732700 1 -- 172.21.15.32:0/590415073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc5004eb20 con 0x7fdc500692d0 2023-09-30T14:37:53.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.512+0000 7fdc4d7fa700 1 -- 172.21.15.32:0/590415073 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fdc44004750 con 0x7fdc500692d0 2023-09-30T14:37:53.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.512+0000 7fdc4d7fa700 1 --2- 172.21.15.32:0/590415073 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdc380a1030 0x7fdc380a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:53.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.512+0000 7fdc4ffff700 1 --2- 172.21.15.32:0/590415073 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdc380a1030 0x7fdc380a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:53.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.512+0000 7fdc4d7fa700 1 -- 172.21.15.32:0/590415073 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdc440cc6c0 con 0x7fdc500692d0 2023-09-30T14:37:53.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.513+0000 7fdc4ffff700 1 --2- 172.21.15.32:0/590415073 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdc380a1030 0x7fdc380a34f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009fd0 tx=0x7fdc40005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:53.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.516+0000 7fdc4d7fa700 1 -- 172.21.15.32:0/590415073 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fdc44095f10 con 0x7fdc500692d0 2023-09-30T14:37:53.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.727+0000 7fdc56732700 1 -- 172.21.15.32:0/590415073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fdc50110ea0 con 0x7fdc500692d0 2023-09-30T14:37:53.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.728+0000 7fdc4d7fa700 1 -- 172.21.15.32:0/590415073 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc4401d090 con 0x7fdc500692d0 2023-09-30T14:37:53.729 INFO:teuthology.orchestra.run.smithi032.stdout:141733920816 2023-09-30T14:37:53.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.730+0000 7fdc56732700 1 -- 172.21.15.32:0/590415073 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdc380a1030 msgr2=0x7fdc380a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:53.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.730+0000 7fdc56732700 1 --2- 172.21.15.32:0/590415073 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdc380a1030 0x7fdc380a34f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009fd0 tx=0x7fdc40005fb0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.730+0000 7fdc56732700 1 -- 172.21.15.32:0/590415073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc500692d0 msgr2=0x7fdc501003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:53.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.730+0000 7fdc56732700 1 --2- 172.21.15.32:0/590415073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc500692d0 0x7fdc501003e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400c920 tx=0x7fdc4400cce0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.730+0000 7fdc56732700 1 -- 172.21.15.32:0/590415073 shutdown_connections 2023-09-30T14:37:53.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.730+0000 7fdc56732700 1 --2- 172.21.15.32:0/590415073 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdc380a1030 0x7fdc380a34f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.730+0000 7fdc56732700 1 --2- 172.21.15.32:0/590415073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc50106f00 0x7fdc50100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.730+0000 7fdc56732700 1 --2- 172.21.15.32:0/590415073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc50068910 0x7fdc500ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.730+0000 7fdc56732700 1 --2- 172.21.15.32:0/590415073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc500692d0 0x7fdc501003e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:53.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.730+0000 7fdc56732700 1 -- 172.21.15.32:0/590415073 >> 172.21.15.32:0/590415073 conn(0x7fdc50075d70 msgr2=0x7fdc500fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:53.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.730+0000 7fdc56732700 1 -- 172.21.15.32:0/590415073 shutdown_connections 2023-09-30T14:37:53.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:53.730+0000 7fdc56732700 1 -- 172.21.15.32:0/590415073 wait complete. 2023-09-30T14:37:53.891 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495655 got 197568495657 for osd.6 2023-09-30T14:37:53.892 DEBUG:teuthology.parallel:result is None 2023-09-30T14:37:54.439 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:54 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3209394751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-30T14:37:54.440 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:54 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2431951628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-30T14:37:54.440 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:54 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/590415073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-30T14:37:54.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:54 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3209394751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-30T14:37:54.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:54 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2431951628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-30T14:37:54.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:54 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/590415073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-30T14:37:54.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:54 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3209394751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-30T14:37:54.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:54 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2431951628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-30T14:37:54.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:54 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/590415073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-30T14:37:54.821 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678036 got 304942678039 for osd.10 2023-09-30T14:37:54.821 DEBUG:teuthology.parallel:result is None 2023-09-30T14:37:54.889 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574908 got 55834574910 for osd.1 2023-09-30T14:37:54.889 DEBUG:teuthology.parallel:result is None 2023-09-30T14:37:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.799+0000 7f6520d20700 1 -- 172.21.15.32:0/1918569253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f651c0fff90 msgr2=0x7f651c105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.799+0000 7f6520d20700 1 --2- 172.21.15.32:0/1918569253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f651c0fff90 0x7f651c105ad0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6510009990 tx=0x7f6510009ca0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.799+0000 7f6520d20700 1 -- 172.21.15.32:0/1918569253 shutdown_connections 2023-09-30T14:37:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.799+0000 7f6520d20700 1 --2- 172.21.15.32:0/1918569253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f651c1061b0 0x7f651c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.799+0000 7f6520d20700 1 --2- 172.21.15.32:0/1918569253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f651c0ff5e0 0x7f651c0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.799+0000 7f6520d20700 1 --2- 172.21.15.32:0/1918569253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f651c0fff90 0x7f651c105ad0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.799+0000 7f6520d20700 1 -- 172.21.15.32:0/1918569253 >> 172.21.15.32:0/1918569253 conn(0x7f651c0762d0 msgr2=0x7f651c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.799+0000 7f6520d20700 1 -- 172.21.15.32:0/1918569253 shutdown_connections 2023-09-30T14:37:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.799+0000 7f6520d20700 1 -- 172.21.15.32:0/1918569253 wait complete. 2023-09-30T14:37:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.800+0000 7f6520d20700 1 Processor -- start 2023-09-30T14:37:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.800+0000 7f6520d20700 1 -- start start 2023-09-30T14:37:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.800+0000 7f6520d20700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f651c0ff5e0 0x7f651c196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.800+0000 7f6520d20700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f651c0fff90 0x7f651c197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.800+0000 7f6520d20700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f651c1061b0 0x7f651c19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.800+0000 7f6520d20700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c19bef0 con 0x7f651c0fff90 2023-09-30T14:37:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.800+0000 7f6520d20700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c19c060 con 0x7f651c0ff5e0 2023-09-30T14:37:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.800+0000 7f6520d20700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c19c220 con 0x7f651c1061b0 2023-09-30T14:37:55.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.800+0000 7f6519d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f651c0fff90 0x7f651c197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:55.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.800+0000 7f6519d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f651c0fff90 0x7f651c197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34330/0 (socket says 172.21.15.32:34330) 2023-09-30T14:37:55.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.800+0000 7f6519d9b700 1 -- 172.21.15.32:0/547373558 learned_addr learned my addr 172.21.15.32:0/547373558 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:55.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.801+0000 7f651a59c700 1 --2- 172.21.15.32:0/547373558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f651c0ff5e0 0x7f651c196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:55.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.801+0000 7f651ad9d700 1 --2- 172.21.15.32:0/547373558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f651c1061b0 0x7f651c19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:55.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.801+0000 7f6519d9b700 1 -- 172.21.15.32:0/547373558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f651c1061b0 msgr2=0x7f651c19b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:55.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.801+0000 7f6519d9b700 1 --2- 172.21.15.32:0/547373558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f651c1061b0 0x7f651c19b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.801+0000 7f6519d9b700 1 -- 172.21.15.32:0/547373558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f651c0ff5e0 msgr2=0x7f651c196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:55.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.801+0000 7f6519d9b700 1 --2- 172.21.15.32:0/547373558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f651c0ff5e0 0x7f651c196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.801+0000 7f6519d9b700 1 -- 172.21.15.32:0/547373558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65100095f0 con 0x7f651c0fff90 2023-09-30T14:37:55.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.801+0000 7f651ad9d700 1 --2- 172.21.15.32:0/547373558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f651c1061b0 0x7f651c19b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:37:55.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.801+0000 7f6519d9b700 1 --2- 172.21.15.32:0/547373558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f651c0fff90 0x7f651c197470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6510003730 tx=0x7f651000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:55.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.801+0000 7f650b7fe700 1 -- 172.21.15.32:0/547373558 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651001d070 con 0x7f651c0fff90 2023-09-30T14:37:55.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.801+0000 7f650b7fe700 1 -- 172.21.15.32:0/547373558 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f651000fc30 con 0x7f651c0fff90 2023-09-30T14:37:55.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.801+0000 7f6520d20700 1 -- 172.21.15.32:0/547373558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c19c4a0 con 0x7f651c0fff90 2023-09-30T14:37:55.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.801+0000 7f650b7fe700 1 -- 172.21.15.32:0/547373558 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65100173f0 con 0x7f651c0fff90 2023-09-30T14:37:55.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.801+0000 7f6520d20700 1 -- 172.21.15.32:0/547373558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c1a3d60 con 0x7f651c0fff90 2023-09-30T14:37:55.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.802+0000 7f6520d20700 1 -- 172.21.15.32:0/547373558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651c04eb20 con 0x7f651c0fff90 2023-09-30T14:37:55.327 INFO:teuthology.orchestra.run.smithi032.stdout:77309411386 2023-09-30T14:37:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.803+0000 7f650b7fe700 1 -- 172.21.15.32:0/547373558 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f6510022470 con 0x7f651c0fff90 2023-09-30T14:37:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.804+0000 7f650b7fe700 1 --2- 172.21.15.32:0/547373558 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f65040a1030 0x7f65040a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.804+0000 7f651a59c700 1 --2- 172.21.15.32:0/547373558 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f65040a1030 0x7f65040a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.804+0000 7f650b7fe700 1 -- 172.21.15.32:0/547373558 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f65100cd9f0 con 0x7f651c0fff90 2023-09-30T14:37:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.805+0000 7f651a59c700 1 --2- 172.21.15.32:0/547373558 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f65040a1030 0x7f65040a34f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f650c005fd0 tx=0x7f650c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:54.808+0000 7f650b7fe700 1 -- 172.21.15.32:0/547373558 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f651005cf30 con 0x7f651c0fff90 2023-09-30T14:37:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.013+0000 7f6520d20700 1 -- 172.21.15.32:0/547373558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f651c110da0 con 0x7f651c0fff90 2023-09-30T14:37:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.014+0000 7f650b7fe700 1 -- 172.21.15.32:0/547373558 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f65100971d0 con 0x7f651c0fff90 2023-09-30T14:37:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.016+0000 7f6520d20700 1 -- 172.21.15.32:0/547373558 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f65040a1030 msgr2=0x7f65040a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.016+0000 7f6520d20700 1 --2- 172.21.15.32:0/547373558 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f65040a1030 0x7f65040a34f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f650c005fd0 tx=0x7f650c005f60 comp rx=0 tx=0).stop 2023-09-30T14:37:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.016+0000 7f6520d20700 1 -- 172.21.15.32:0/547373558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f651c0fff90 msgr2=0x7f651c197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:55.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.016+0000 7f6520d20700 1 --2- 172.21.15.32:0/547373558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f651c0fff90 0x7f651c197470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6510003730 tx=0x7f651000f740 comp rx=0 tx=0).stop 2023-09-30T14:37:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.016+0000 7f6520d20700 1 -- 172.21.15.32:0/547373558 shutdown_connections 2023-09-30T14:37:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.016+0000 7f6520d20700 1 --2- 172.21.15.32:0/547373558 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f65040a1030 0x7f65040a34f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.016+0000 7f6520d20700 1 --2- 172.21.15.32:0/547373558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f651c1061b0 0x7f651c19b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.016+0000 7f6520d20700 1 --2- 172.21.15.32:0/547373558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f651c0ff5e0 0x7f651c196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.016+0000 7f6520d20700 1 --2- 172.21.15.32:0/547373558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f651c0fff90 0x7f651c197470 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.016+0000 7f6520d20700 1 -- 172.21.15.32:0/547373558 >> 172.21.15.32:0/547373558 conn(0x7f651c0762d0 msgr2=0x7f651c104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.016+0000 7f6520d20700 1 -- 172.21.15.32:0/547373558 shutdown_connections 2023-09-30T14:37:55.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.016+0000 7f6520d20700 1 -- 172.21.15.32:0/547373558 wait complete. 2023-09-30T14:37:55.334 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:55 smithi032 ceph-mon[98592]: pgmap v274: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-09-30T14:37:55.335 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:55 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/547373558' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-30T14:37:55.451 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920814 got 141733920816 for osd.4 2023-09-30T14:37:55.451 DEBUG:teuthology.parallel:result is None 2023-09-30T14:37:55.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.484+0000 7f39216c4700 1 -- 172.21.15.32:0/991187405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f391c069520 msgr2=0x7f391c105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.484+0000 7f39216c4700 1 --2- 172.21.15.32:0/991187405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f391c069520 0x7f391c105af0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3904009990 tx=0x7f3904009ca0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.485+0000 7f39216c4700 1 -- 172.21.15.32:0/991187405 shutdown_connections 2023-09-30T14:37:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.485+0000 7f39216c4700 1 --2- 172.21.15.32:0/991187405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c106220 0x7f391c10dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.485+0000 7f39216c4700 1 --2- 172.21.15.32:0/991187405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f391c068b70 0x7f391c068f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.485+0000 7f39216c4700 1 --2- 172.21.15.32:0/991187405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f391c069520 0x7f391c105af0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.485+0000 7f39216c4700 1 -- 172.21.15.32:0/991187405 >> 172.21.15.32:0/991187405 conn(0x7f391c075d70 msgr2=0x7f391c076180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.485+0000 7f39216c4700 1 -- 172.21.15.32:0/991187405 shutdown_connections 2023-09-30T14:37:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.485+0000 7f39216c4700 1 -- 172.21.15.32:0/991187405 wait complete. 2023-09-30T14:37:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.485+0000 7f39216c4700 1 Processor -- start 2023-09-30T14:37:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.485+0000 7f39216c4700 1 -- start start 2023-09-30T14:37:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f39216c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f391c068b70 0x7f391c196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f39216c4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f391c069520 0x7f391c1974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f39216c4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c106220 0x7f391c19b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f39216c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c19bf40 con 0x7f391c068b70 2023-09-30T14:37:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f39216c4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c19c0b0 con 0x7f391c106220 2023-09-30T14:37:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f39216c4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c19c270 con 0x7f391c069520 2023-09-30T14:37:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f391affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f391c068b70 0x7f391c196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f391affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f391c068b70 0x7f391c196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34342/0 (socket says 172.21.15.32:34342) 2023-09-30T14:37:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f391affd700 1 -- 172.21.15.32:0/2001904396 learned_addr learned my addr 172.21.15.32:0/2001904396 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f391b7fe700 1 --2- 172.21.15.32:0/2001904396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c106220 0x7f391c19b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f3913fff700 1 --2- 172.21.15.32:0/2001904396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f391c069520 0x7f391c1974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f391b7fe700 1 -- 172.21.15.32:0/2001904396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f391c069520 msgr2=0x7f391c1974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f391b7fe700 1 --2- 172.21.15.32:0/2001904396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f391c069520 0x7f391c1974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f391b7fe700 1 -- 172.21.15.32:0/2001904396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f391c068b70 msgr2=0x7f391c196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f391b7fe700 1 --2- 172.21.15.32:0/2001904396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f391c068b70 0x7f391c196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f391b7fe700 1 -- 172.21.15.32:0/2001904396 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39040095f0 con 0x7f391c106220 2023-09-30T14:37:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.486+0000 7f391affd700 1 --2- 172.21.15.32:0/2001904396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f391c068b70 0x7f391c196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:37:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.487+0000 7f391b7fe700 1 --2- 172.21.15.32:0/2001904396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c106220 0x7f391c19b930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f391400cc50 tx=0x7f391400cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.487+0000 7f3918ff9700 1 -- 172.21.15.32:0/2001904396 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391400aa90 con 0x7f391c106220 2023-09-30T14:37:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.487+0000 7f39216c4700 1 -- 172.21.15.32:0/2001904396 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391c19c550 con 0x7f391c106220 2023-09-30T14:37:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.487+0000 7f3918ff9700 1 -- 172.21.15.32:0/2001904396 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f391400abf0 con 0x7f391c106220 2023-09-30T14:37:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.487+0000 7f3918ff9700 1 -- 172.21.15.32:0/2001904396 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39140115e0 con 0x7f391c106220 2023-09-30T14:37:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.487+0000 7f39216c4700 1 -- 172.21.15.32:0/2001904396 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391c1a3db0 con 0x7f391c106220 2023-09-30T14:37:55.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.488+0000 7f39216c4700 1 -- 172.21.15.32:0/2001904396 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391c04eb20 con 0x7f391c106220 2023-09-30T14:37:55.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.489+0000 7f3918ff9700 1 -- 172.21.15.32:0/2001904396 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f3914012420 con 0x7f391c106220 2023-09-30T14:37:55.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.490+0000 7f3918ff9700 1 --2- 172.21.15.32:0/2001904396 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f39080a1030 0x7f39080a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:55.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.490+0000 7f391affd700 1 --2- 172.21.15.32:0/2001904396 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f39080a1030 0x7f39080a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:55.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.490+0000 7f3918ff9700 1 -- 172.21.15.32:0/2001904396 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f39140ccba0 con 0x7f391c106220 2023-09-30T14:37:55.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.490+0000 7f391affd700 1 --2- 172.21.15.32:0/2001904396 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f39080a1030 0x7f39080a34f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f390c005950 tx=0x7f390c00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:55.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.494+0000 7f3918ff9700 1 -- 172.21.15.32:0/2001904396 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f3914096400 con 0x7f391c106220 2023-09-30T14:37:55.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:55 smithi156 ceph-mon[103579]: pgmap v274: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-09-30T14:37:55.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:55 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/547373558' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-30T14:37:55.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:55 smithi148 ceph-mon[102355]: pgmap v274: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-09-30T14:37:55.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:55 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/547373558' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-30T14:37:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.699+0000 7f39216c4700 1 -- 172.21.15.32:0/2001904396 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f391c110e90 con 0x7f391c106220 2023-09-30T14:37:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.700+0000 7f3918ff9700 1 -- 172.21.15.32:0/2001904396 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3914016090 con 0x7f391c106220 2023-09-30T14:37:55.701 INFO:teuthology.orchestra.run.smithi032.stdout:283467841562 2023-09-30T14:37:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.702+0000 7f39216c4700 1 -- 172.21.15.32:0/2001904396 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f39080a1030 msgr2=0x7f39080a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.702+0000 7f39216c4700 1 --2- 172.21.15.32:0/2001904396 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f39080a1030 0x7f39080a34f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f390c005950 tx=0x7f390c00b500 comp rx=0 tx=0).stop 2023-09-30T14:37:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.702+0000 7f39216c4700 1 -- 172.21.15.32:0/2001904396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c106220 msgr2=0x7f391c19b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.702+0000 7f39216c4700 1 --2- 172.21.15.32:0/2001904396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c106220 0x7f391c19b930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f391400cc50 tx=0x7f391400cf60 comp rx=0 tx=0).stop 2023-09-30T14:37:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.702+0000 7f39216c4700 1 -- 172.21.15.32:0/2001904396 shutdown_connections 2023-09-30T14:37:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.702+0000 7f39216c4700 1 --2- 172.21.15.32:0/2001904396 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f39080a1030 0x7f39080a34f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.702+0000 7f39216c4700 1 --2- 172.21.15.32:0/2001904396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391c106220 0x7f391c19b930 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.702+0000 7f39216c4700 1 --2- 172.21.15.32:0/2001904396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f391c068b70 0x7f391c196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.702+0000 7f39216c4700 1 --2- 172.21.15.32:0/2001904396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f391c069520 0x7f391c1974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.702+0000 7f39216c4700 1 -- 172.21.15.32:0/2001904396 >> 172.21.15.32:0/2001904396 conn(0x7f391c075d70 msgr2=0x7f391c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.702+0000 7f39216c4700 1 -- 172.21.15.32:0/2001904396 shutdown_connections 2023-09-30T14:37:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.702+0000 7f39216c4700 1 -- 172.21.15.32:0/2001904396 wait complete. 2023-09-30T14:37:55.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.718+0000 7f0032347700 1 -- 172.21.15.32:0/3018689059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c10b480 msgr2=0x7f002c10d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:55.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.718+0000 7f0032347700 1 --2- 172.21.15.32:0/3018689059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c10b480 0x7f002c10d940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f002400b0a0 tx=0x7f002400b3b0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.718+0000 7f0032347700 1 -- 172.21.15.32:0/3018689059 shutdown_connections 2023-09-30T14:37:55.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.718+0000 7f0032347700 1 --2- 172.21.15.32:0/3018689059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c10b480 0x7f002c10d940 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.718+0000 7f0032347700 1 --2- 172.21.15.32:0/3018689059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f002c102050 0x7f002c102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.718+0000 7f0032347700 1 --2- 172.21.15.32:0/3018689059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f002c102970 0x7f002c10ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.718+0000 7f0032347700 1 -- 172.21.15.32:0/3018689059 >> 172.21.15.32:0/3018689059 conn(0x7f002c0fb880 msgr2=0x7f002c0fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:55.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.718+0000 7f0032347700 1 -- 172.21.15.32:0/3018689059 shutdown_connections 2023-09-30T14:37:55.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.718+0000 7f0032347700 1 -- 172.21.15.32:0/3018689059 wait complete. 2023-09-30T14:37:55.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.719+0000 7f0032347700 1 Processor -- start 2023-09-30T14:37:55.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.719+0000 7f0032347700 1 -- start start 2023-09-30T14:37:55.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f0032347700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c102050 0x7f002c197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:55.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f0032347700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f002c102970 0x7f002c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:55.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f0032347700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f002c10b480 0x7f002c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:55.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f0032347700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c19c030 con 0x7f002c102050 2023-09-30T14:37:55.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f0032347700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c19c1a0 con 0x7f002c102970 2023-09-30T14:37:55.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f0032347700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c19c360 con 0x7f002c10b480 2023-09-30T14:37:55.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f002bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c102050 0x7f002c197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:55.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f002bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c102050 0x7f002c197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34356/0 (socket says 172.21.15.32:34356) 2023-09-30T14:37:55.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f002bfff700 1 -- 172.21.15.32:0/3743234539 learned_addr learned my addr 172.21.15.32:0/3743234539 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:55.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f002b7fe700 1 --2- 172.21.15.32:0/3743234539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f002c102970 0x7f002c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:55.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f00308e4700 1 --2- 172.21.15.32:0/3743234539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f002c10b480 0x7f002c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:55.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f002bfff700 1 -- 172.21.15.32:0/3743234539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f002c10b480 msgr2=0x7f002c19ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:55.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f002bfff700 1 --2- 172.21.15.32:0/3743234539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f002c10b480 0x7f002c19ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f002bfff700 1 -- 172.21.15.32:0/3743234539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f002c102970 msgr2=0x7f002c1975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:55.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f002bfff700 1 --2- 172.21.15.32:0/3743234539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f002c102970 0x7f002c1975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f002bfff700 1 -- 172.21.15.32:0/3743234539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0024009d00 con 0x7f002c102050 2023-09-30T14:37:55.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f00308e4700 1 --2- 172.21.15.32:0/3743234539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f002c10b480 0x7f002c19ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:37:55.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f002b7fe700 1 --2- 172.21.15.32:0/3743234539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f002c102970 0x7f002c1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:37:55.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f002bfff700 1 --2- 172.21.15.32:0/3743234539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c102050 0x7f002c197070 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f001c00d950 tx=0x7f001c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:55.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f00297fa700 1 -- 172.21.15.32:0/3743234539 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c009b40 con 0x7f002c102050 2023-09-30T14:37:55.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f00297fa700 1 -- 172.21.15.32:0/3743234539 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f001c009ca0 con 0x7f002c102050 2023-09-30T14:37:55.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f0032347700 1 -- 172.21.15.32:0/3743234539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002c19c640 con 0x7f002c102050 2023-09-30T14:37:55.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.720+0000 7f00297fa700 1 -- 172.21.15.32:0/3743234539 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c00f620 con 0x7f002c102050 2023-09-30T14:37:55.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.721+0000 7f0032347700 1 -- 172.21.15.32:0/3743234539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002c1a3ea0 con 0x7f002c102050 2023-09-30T14:37:55.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.722+0000 7f0032347700 1 -- 172.21.15.32:0/3743234539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002c105cc0 con 0x7f002c102050 2023-09-30T14:37:55.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.722+0000 7f00297fa700 1 -- 172.21.15.32:0/3743234539 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f001c010460 con 0x7f002c102050 2023-09-30T14:37:55.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.723+0000 7f00297fa700 1 --2- 172.21.15.32:0/3743234539 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f00140a10b0 0x7f00140a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:55.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.723+0000 7f002b7fe700 1 --2- 172.21.15.32:0/3743234539 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f00140a10b0 0x7f00140a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:55.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.723+0000 7f00297fa700 1 -- 172.21.15.32:0/3743234539 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f001c0cca00 con 0x7f002c102050 2023-09-30T14:37:55.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.724+0000 7f002b7fe700 1 --2- 172.21.15.32:0/3743234539 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f00140a10b0 0x7f00140a3570 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0020005e10 tx=0x7f0020005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:55.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.728+0000 7f00297fa700 1 -- 172.21.15.32:0/3743234539 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f001c096250 con 0x7f002c102050 2023-09-30T14:37:55.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.936+0000 7f0032347700 1 -- 172.21.15.32:0/3743234539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f002c110e30 con 0x7f002c102050 2023-09-30T14:37:55.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.936+0000 7f00297fa700 1 -- 172.21.15.32:0/3743234539 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f001c020090 con 0x7f002c102050 2023-09-30T14:37:55.938 INFO:teuthology.orchestra.run.smithi032.stdout:163208757293 2023-09-30T14:37:55.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.938+0000 7f0032347700 1 -- 172.21.15.32:0/3743234539 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f00140a10b0 msgr2=0x7f00140a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:55.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.938+0000 7f0032347700 1 --2- 172.21.15.32:0/3743234539 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f00140a10b0 0x7f00140a3570 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0020005e10 tx=0x7f0020005d80 comp rx=0 tx=0).stop 2023-09-30T14:37:55.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.938+0000 7f0032347700 1 -- 172.21.15.32:0/3743234539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c102050 msgr2=0x7f002c197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.938+0000 7f0032347700 1 --2- 172.21.15.32:0/3743234539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c102050 0x7f002c197070 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f001c00d950 tx=0x7f001c00dd10 comp rx=0 tx=0).stop 2023-09-30T14:37:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.939+0000 7f0032347700 1 -- 172.21.15.32:0/3743234539 shutdown_connections 2023-09-30T14:37:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.939+0000 7f0032347700 1 --2- 172.21.15.32:0/3743234539 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f00140a10b0 0x7f00140a3570 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.939+0000 7f0032347700 1 --2- 172.21.15.32:0/3743234539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f002c10b480 0x7f002c19ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.939+0000 7f0032347700 1 --2- 172.21.15.32:0/3743234539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c102050 0x7f002c197070 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.939+0000 7f0032347700 1 --2- 172.21.15.32:0/3743234539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f002c102970 0x7f002c1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.939+0000 7f0032347700 1 -- 172.21.15.32:0/3743234539 >> 172.21.15.32:0/3743234539 conn(0x7f002c0fb880 msgr2=0x7f002c102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.939+0000 7f0032347700 1 -- 172.21.15.32:0/3743234539 shutdown_connections 2023-09-30T14:37:55.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:55.939+0000 7f0032347700 1 -- 172.21.15.32:0/3743234539 wait complete. 2023-09-30T14:37:56.059 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411384 got 77309411386 for osd.2 2023-09-30T14:37:56.059 DEBUG:teuthology.parallel:result is None 2023-09-30T14:37:56.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:56 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2001904396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-30T14:37:56.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:56 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3743234539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-30T14:37:56.502 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757291 got 163208757293 for osd.5 2023-09-30T14:37:56.502 DEBUG:teuthology.parallel:result is None 2023-09-30T14:37:56.554 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841560 got 283467841562 for osd.9 2023-09-30T14:37:56.554 DEBUG:teuthology.parallel:result is None 2023-09-30T14:37:56.555 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-09-30T14:37:56.555 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-09-30T14:37:56.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:56 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2001904396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-30T14:37:56.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:56 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3743234539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-30T14:37:56.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:56 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2001904396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-30T14:37:56.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:56 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3743234539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-30T14:37:57.120 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:57.523 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:57 smithi032 ceph-mon[98592]: pgmap v275: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:37:57.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:57 smithi156 ceph-mon[103579]: pgmap v275: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:37:57.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:57 smithi148 ceph-mon[102355]: pgmap v275: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:37:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.342+0000 7feca0d8b700 1 -- 172.21.15.32:0/2007593425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c104da0 msgr2=0x7fec9c105180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.342+0000 7feca0d8b700 1 --2- 172.21.15.32:0/2007593425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c104da0 0x7fec9c105180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fec900099c0 tx=0x7fec90009cd0 comp rx=0 tx=0).stop 2023-09-30T14:37:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.342+0000 7feca0d8b700 1 -- 172.21.15.32:0/2007593425 shutdown_connections 2023-09-30T14:37:58.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.342+0000 7feca0d8b700 1 --2- 172.21.15.32:0/2007593425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec9c100e50 0x7fec9c10cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.342+0000 7feca0d8b700 1 --2- 172.21.15.32:0/2007593425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c104da0 0x7fec9c105180 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.342+0000 7feca0d8b700 1 --2- 172.21.15.32:0/2007593425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec9c100490 0x7fec9c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.342+0000 7feca0d8b700 1 -- 172.21.15.32:0/2007593425 >> 172.21.15.32:0/2007593425 conn(0x7fec9c0762d0 msgr2=0x7fec9c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.342+0000 7feca0d8b700 1 -- 172.21.15.32:0/2007593425 shutdown_connections 2023-09-30T14:37:58.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.342+0000 7feca0d8b700 1 -- 172.21.15.32:0/2007593425 wait complete. 2023-09-30T14:37:58.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.343+0000 7feca0d8b700 1 Processor -- start 2023-09-30T14:37:58.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.343+0000 7feca0d8b700 1 -- start start 2023-09-30T14:37:58.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.343+0000 7feca0d8b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec9c100490 0x7fec9c196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:58.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.343+0000 7feca0d8b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c100e50 0x7fec9c197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:58.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.343+0000 7feca0d8b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec9c104da0 0x7fec9c19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:58.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.343+0000 7feca0d8b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c19bef0 con 0x7fec9c100e50 2023-09-30T14:37:58.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.343+0000 7feca0d8b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c19c060 con 0x7fec9c104da0 2023-09-30T14:37:58.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.343+0000 7feca0d8b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c19c220 con 0x7fec9c100490 2023-09-30T14:37:58.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.343+0000 7fec99d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c100e50 0x7fec9c197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:58.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.343+0000 7fec99d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c100e50 0x7fec9c197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34372/0 (socket says 172.21.15.32:34372) 2023-09-30T14:37:58.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.343+0000 7fec99d9b700 1 -- 172.21.15.32:0/381612258 learned_addr learned my addr 172.21.15.32:0/381612258 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:37:58.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.343+0000 7fec9a59c700 1 --2- 172.21.15.32:0/381612258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec9c100490 0x7fec9c196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:58.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.343+0000 7fec9ad9d700 1 --2- 172.21.15.32:0/381612258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec9c104da0 0x7fec9c19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:58.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.344+0000 7fec99d9b700 1 -- 172.21.15.32:0/381612258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec9c100490 msgr2=0x7fec9c196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:58.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.344+0000 7fec99d9b700 1 --2- 172.21.15.32:0/381612258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec9c100490 0x7fec9c196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:58.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.344+0000 7fec99d9b700 1 -- 172.21.15.32:0/381612258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec9c104da0 msgr2=0x7fec9c19b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:58.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.344+0000 7fec99d9b700 1 --2- 172.21.15.32:0/381612258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec9c104da0 0x7fec9c19b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:58.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.344+0000 7fec99d9b700 1 -- 172.21.15.32:0/381612258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec900095f0 con 0x7fec9c100e50 2023-09-30T14:37:58.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.344+0000 7fec99d9b700 1 --2- 172.21.15.32:0/381612258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c100e50 0x7fec9c197470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00c9b0 tx=0x7fec8c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:58.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.344+0000 7fec8b7fe700 1 -- 172.21.15.32:0/381612258 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c007b00 con 0x7fec9c100e50 2023-09-30T14:37:58.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.344+0000 7fec8b7fe700 1 -- 172.21.15.32:0/381612258 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec8c007c60 con 0x7fec9c100e50 2023-09-30T14:37:58.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.344+0000 7feca0d8b700 1 -- 172.21.15.32:0/381612258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9c19c500 con 0x7fec9c100e50 2023-09-30T14:37:58.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.344+0000 7fec8b7fe700 1 -- 172.21.15.32:0/381612258 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c018660 con 0x7fec9c100e50 2023-09-30T14:37:58.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.344+0000 7feca0d8b700 1 -- 172.21.15.32:0/381612258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c1a3d60 con 0x7fec9c100e50 2023-09-30T14:37:58.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.345+0000 7feca0d8b700 1 -- 172.21.15.32:0/381612258 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9c04eb20 con 0x7fec9c100e50 2023-09-30T14:37:58.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.347+0000 7fec8b7fe700 1 -- 172.21.15.32:0/381612258 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fec8c018880 con 0x7fec9c100e50 2023-09-30T14:37:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.347+0000 7fec8b7fe700 1 --2- 172.21.15.32:0/381612258 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fec840a10b0 0x7fec840a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:37:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.348+0000 7fec8b7fe700 1 -- 172.21.15.32:0/381612258 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fec8c0cca80 con 0x7fec9c100e50 2023-09-30T14:37:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.348+0000 7fec9a59c700 1 --2- 172.21.15.32:0/381612258 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fec840a10b0 0x7fec840a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:37:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.348+0000 7fec9a59c700 1 --2- 172.21.15.32:0/381612258 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fec840a10b0 0x7fec840a3570 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fec90000c00 tx=0x7fec90005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:37:58.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.351+0000 7fec8b7fe700 1 -- 172.21.15.32:0/381612258 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fec8c0962e0 con 0x7fec9c100e50 2023-09-30T14:37:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.549+0000 7feca0d8b700 1 -- 172.21.15.32:0/381612258 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec9c10feb0 con 0x7fec840a10b0 2023-09-30T14:37:58.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.561+0000 7fec8b7fe700 1 -- 172.21.15.32:0/381612258 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340181 (secure 0 0 0) 0x7fec9c10feb0 con 0x7fec840a10b0 2023-09-30T14:37:58.564 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:37:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.564+0000 7feca0d8b700 1 -- 172.21.15.32:0/381612258 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fec840a10b0 msgr2=0x7fec840a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.564+0000 7feca0d8b700 1 --2- 172.21.15.32:0/381612258 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fec840a10b0 0x7fec840a3570 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fec90000c00 tx=0x7fec90005fb0 comp rx=0 tx=0).stop 2023-09-30T14:37:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.564+0000 7feca0d8b700 1 -- 172.21.15.32:0/381612258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c100e50 msgr2=0x7fec9c197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:37:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.564+0000 7feca0d8b700 1 --2- 172.21.15.32:0/381612258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c100e50 0x7fec9c197470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00c9b0 tx=0x7fec8c00cd70 comp rx=0 tx=0).stop 2023-09-30T14:37:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.564+0000 7feca0d8b700 1 -- 172.21.15.32:0/381612258 shutdown_connections 2023-09-30T14:37:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.564+0000 7feca0d8b700 1 --2- 172.21.15.32:0/381612258 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fec840a10b0 0x7fec840a3570 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.564+0000 7feca0d8b700 1 --2- 172.21.15.32:0/381612258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec9c104da0 0x7fec9c19b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.564+0000 7feca0d8b700 1 --2- 172.21.15.32:0/381612258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec9c100490 0x7fec9c196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.564+0000 7feca0d8b700 1 --2- 172.21.15.32:0/381612258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec9c100e50 0x7fec9c197470 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:37:58.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.564+0000 7feca0d8b700 1 -- 172.21.15.32:0/381612258 >> 172.21.15.32:0/381612258 conn(0x7fec9c0762d0 msgr2=0x7fec9c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:37:58.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.564+0000 7feca0d8b700 1 -- 172.21.15.32:0/381612258 shutdown_connections 2023-09-30T14:37:58.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:37:58.564+0000 7feca0d8b700 1 -- 172.21.15.32:0/381612258 wait complete. 2023-09-30T14:37:58.568 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-09-30T14:37:58.980 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2023-09-30T14:37:57.782981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648932,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":384,"num_read_kb":300,"num_write":330,"num_write_kb":3038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":268,"ondisk_log_size":268,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336736,"kb_used_data":12544,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605088,"statfs":{"total":1151940427776,"available":1151595610112,"internally_reserved":0,"allocated":12845056,"data_stored":7044206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003222"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014576+0000","last_change":"2023-09-30T14:37:11.988122+0000","last_active":"2023-09-30T14:37:13.014576+0000","last_peered":"2023-09-30T14:37:13.014576+0000","last_clean":"2023-09-30T14:37:13.014576+0000","last_became_active":"2023-09-30T14:37:11.986770+0000","last_became_peered":"2023-09-30T14:37:11.986770+0000","last_unstale":"2023-09-30T14:37:13.014576+0000","last_undegraded":"2023-09-30T14:37:13.014576+0000","last_fullsized":"2023-09-30T14:37:13.014576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:46:29.655309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014211+0000","last_change":"2023-09-30T14:37:11.987953+0000","last_active":"2023-09-30T14:37:13.014211+0000","last_peered":"2023-09-30T14:37:13.014211+0000","last_clean":"2023-09-30T14:37:13.014211+0000","last_became_active":"2023-09-30T14:37:11.987248+0000","last_became_peered":"2023-09-30T14:37:11.987248+0000","last_unstale":"2023-09-30T14:37:13.014211+0000","last_undegraded":"2023-09-30T14:37:13.014211+0000","last_fullsized":"2023-09-30T14:37:13.014211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:12:59.780089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014752+0000","last_change":"2023-09-30T14:37:11.987570+0000","last_active":"2023-09-30T14:37:13.014752+0000","last_peered":"2023-09-30T14:37:13.014752+0000","last_clean":"2023-09-30T14:37:13.014752+0000","last_became_active":"2023-09-30T14:37:11.987064+0000","last_became_peered":"2023-09-30T14:37:11.987064+0000","last_unstale":"2023-09-30T14:37:13.014752+0000","last_undegraded":"2023-09-30T14:37:13.014752+0000","last_fullsized":"2023-09-30T14:37:13.014752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:26:16.674605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014524+0000","last_change":"2023-09-30T14:37:11.990161+0000","last_active":"2023-09-30T14:37:13.014524+0000","last_peered":"2023-09-30T14:37:13.014524+0000","last_clean":"2023-09-30T14:37:13.014524+0000","last_became_active":"2023-09-30T14:37:11.989580+0000","last_became_peered":"2023-09-30T14:37:11.989580+0000","last_unstale":"2023-09-30T14:37:13.014524+0000","last_undegraded":"2023-09-30T14:37:13.014524+0000","last_fullsized":"2023-09-30T14:37:13.014524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:13:06.986864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130616+0000","last_change":"2023-09-30T14:37:11.993902+0000","last_active":"2023-09-30T14:37:13.130616+0000","last_peered":"2023-09-30T14:37:13.130616+0000","last_clean":"2023-09-30T14:37:13.130616+0000","last_became_active":"2023-09-30T14:37:11.993622+0000","last_became_peered":"2023-09-30T14:37:11.993622+0000","last_unstale":"2023-09-30T14:37:13.130616+0000","last_undegraded":"2023-09-30T14:37:13.130616+0000","last_fullsized":"2023-09-30T14:37:13.130616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:09:49.252823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.015937+0000","last_change":"2023-09-30T14:37:11.997282+0000","last_active":"2023-09-30T14:37:13.015937+0000","last_peered":"2023-09-30T14:37:13.015937+0000","last_clean":"2023-09-30T14:37:13.015937+0000","last_became_active":"2023-09-30T14:37:11.996937+0000","last_became_peered":"2023-09-30T14:37:11.996937+0000","last_unstale":"2023-09-30T14:37:13.015937+0000","last_undegraded":"2023-09-30T14:37:13.015937+0000","last_fullsized":"2023-09-30T14:37:13.015937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:39:27.351772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014196+0000","last_change":"2023-09-30T14:37:11.997408+0000","last_active":"2023-09-30T14:37:13.014196+0000","last_peered":"2023-09-30T14:37:13.014196+0000","last_clean":"2023-09-30T14:37:13.014196+0000","last_became_active":"2023-09-30T14:37:11.997147+0000","last_became_peered":"2023-09-30T14:37:11.997147+0000","last_unstale":"2023-09-30T14:37:13.014196+0000","last_undegraded":"2023-09-30T14:37:13.014196+0000","last_fullsized":"2023-09-30T14:37:13.014196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:21:24.721417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975636+0000","last_change":"2023-09-30T14:37:11.986172+0000","last_active":"2023-09-30T14:37:12.975636+0000","last_peered":"2023-09-30T14:37:12.975636+0000","last_clean":"2023-09-30T14:37:12.975636+0000","last_became_active":"2023-09-30T14:37:11.985836+0000","last_became_peered":"2023-09-30T14:37:11.985836+0000","last_unstale":"2023-09-30T14:37:12.975636+0000","last_undegraded":"2023-09-30T14:37:12.975636+0000","last_fullsized":"2023-09-30T14:37:12.975636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:35:57.464679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976406+0000","last_change":"2023-09-30T14:37:11.990853+0000","last_active":"2023-09-30T14:37:12.976406+0000","last_peered":"2023-09-30T14:37:12.976406+0000","last_clean":"2023-09-30T14:37:12.976406+0000","last_became_active":"2023-09-30T14:37:11.990321+0000","last_became_peered":"2023-09-30T14:37:11.990321+0000","last_unstale":"2023-09-30T14:37:12.976406+0000","last_undegraded":"2023-09-30T14:37:12.976406+0000","last_fullsized":"2023-09-30T14:37:12.976406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:32:11.409927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.016101+0000","last_change":"2023-09-30T14:37:11.992094+0000","last_active":"2023-09-30T14:37:13.016101+0000","last_peered":"2023-09-30T14:37:13.016101+0000","last_clean":"2023-09-30T14:37:13.016101+0000","last_became_active":"2023-09-30T14:37:11.991762+0000","last_became_peered":"2023-09-30T14:37:11.991762+0000","last_unstale":"2023-09-30T14:37:13.016101+0000","last_undegraded":"2023-09-30T14:37:13.016101+0000","last_fullsized":"2023-09-30T14:37:13.016101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:22:50.203788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014249+0000","last_change":"2023-09-30T14:37:11.992213+0000","last_active":"2023-09-30T14:37:13.014249+0000","last_peered":"2023-09-30T14:37:13.014249+0000","last_clean":"2023-09-30T14:37:13.014249+0000","last_became_active":"2023-09-30T14:37:11.991868+0000","last_became_peered":"2023-09-30T14:37:11.991868+0000","last_unstale":"2023-09-30T14:37:13.014249+0000","last_undegraded":"2023-09-30T14:37:13.014249+0000","last_fullsized":"2023-09-30T14:37:13.014249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:20:06.061978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014693+0000","last_change":"2023-09-30T14:37:11.990043+0000","last_active":"2023-09-30T14:37:13.014693+0000","last_peered":"2023-09-30T14:37:13.014693+0000","last_clean":"2023-09-30T14:37:13.014693+0000","last_became_active":"2023-09-30T14:37:11.988243+0000","last_became_peered":"2023-09-30T14:37:11.988243+0000","last_unstale":"2023-09-30T14:37:13.014693+0000","last_undegraded":"2023-09-30T14:37:13.014693+0000","last_fullsized":"2023-09-30T14:37:13.014693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:17:58.946159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014266+0000","last_change":"2023-09-30T14:37:11.990361+0000","last_active":"2023-09-30T14:37:13.014266+0000","last_peered":"2023-09-30T14:37:13.014266+0000","last_clean":"2023-09-30T14:37:13.014266+0000","last_became_active":"2023-09-30T14:37:11.990080+0000","last_became_peered":"2023-09-30T14:37:11.990080+0000","last_unstale":"2023-09-30T14:37:13.014266+0000","last_undegraded":"2023-09-30T14:37:13.014266+0000","last_fullsized":"2023-09-30T14:37:13.014266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:27:08.830419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014485+0000","last_change":"2023-09-30T14:37:11.988119+0000","last_active":"2023-09-30T14:37:13.014485+0000","last_peered":"2023-09-30T14:37:13.014485+0000","last_clean":"2023-09-30T14:37:13.014485+0000","last_became_active":"2023-09-30T14:37:11.987704+0000","last_became_peered":"2023-09-30T14:37:11.987704+0000","last_unstale":"2023-09-30T14:37:13.014485+0000","last_undegraded":"2023-09-30T14:37:13.014485+0000","last_fullsized":"2023-09-30T14:37:13.014485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:47:23.318982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.013931+0000","last_change":"2023-09-30T14:37:11.985403+0000","last_active":"2023-09-30T14:37:13.013931+0000","last_peered":"2023-09-30T14:37:13.013931+0000","last_clean":"2023-09-30T14:37:13.013931+0000","last_became_active":"2023-09-30T14:37:11.984818+0000","last_became_peered":"2023-09-30T14:37:11.984818+0000","last_unstale":"2023-09-30T14:37:13.013931+0000","last_undegraded":"2023-09-30T14:37:13.013931+0000","last_fullsized":"2023-09-30T14:37:13.013931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:10:43.592505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976073+0000","last_change":"2023-09-30T14:37:11.989500+0000","last_active":"2023-09-30T14:37:12.976073+0000","last_peered":"2023-09-30T14:37:12.976073+0000","last_clean":"2023-09-30T14:37:12.976073+0000","last_became_active":"2023-09-30T14:37:11.989186+0000","last_became_peered":"2023-09-30T14:37:11.989186+0000","last_unstale":"2023-09-30T14:37:12.976073+0000","last_undegraded":"2023-09-30T14:37:12.976073+0000","last_fullsized":"2023-09-30T14:37:12.976073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:14:35.497319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014676+0000","last_change":"2023-09-30T14:37:11.987217+0000","last_active":"2023-09-30T14:37:13.014676+0000","last_peered":"2023-09-30T14:37:13.014676+0000","last_clean":"2023-09-30T14:37:13.014676+0000","last_became_active":"2023-09-30T14:37:11.986835+0000","last_became_peered":"2023-09-30T14:37:11.986835+0000","last_unstale":"2023-09-30T14:37:13.014676+0000","last_undegraded":"2023-09-30T14:37:13.014676+0000","last_fullsized":"2023-09-30T14:37:13.014676+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:11:19.234768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014440+0000","last_change":"2023-09-30T14:37:11.995224+0000","last_active":"2023-09-30T14:37:13.014440+0000","last_peered":"2023-09-30T14:37:13.014440+0000","last_clean":"2023-09-30T14:37:13.014440+0000","last_became_active":"2023-09-30T14:37:11.994925+0000","last_became_peered":"2023-09-30T14:37:11.994925+0000","last_unstale":"2023-09-30T14:37:13.014440+0000","last_undegraded":"2023-09-30T14:37:13.014440+0000","last_fullsized":"2023-09-30T14:37:13.014440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:08:07.453499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975873+0000","last_change":"2023-09-30T14:37:11.990182+0000","last_active":"2023-09-30T14:37:12.975873+0000","last_peered":"2023-09-30T14:37:12.975873+0000","last_clean":"2023-09-30T14:37:12.975873+0000","last_became_active":"2023-09-30T14:37:11.989862+0000","last_became_peered":"2023-09-30T14:37:11.989862+0000","last_unstale":"2023-09-30T14:37:12.975873+0000","last_undegraded":"2023-09-30T14:37:12.975873+0000","last_fullsized":"2023-09-30T14:37:12.975873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:15:31.713002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.076239+0000","last_change":"2023-09-30T14:37:11.028123+0000","last_active":"2023-09-30T14:37:16.076239+0000","last_peered":"2023-09-30T14:37:16.076239+0000","last_clean":"2023-09-30T14:37:16.076239+0000","last_became_active":"2023-09-30T14:37:11.027764+0000","last_became_peered":"2023-09-30T14:37:11.027764+0000","last_unstale":"2023-09-30T14:37:16.076239+0000","last_undegraded":"2023-09-30T14:37:16.076239+0000","last_fullsized":"2023-09-30T14:37:16.076239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:15:50.818783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.977049+0000","last_change":"2023-09-30T14:37:11.994584+0000","last_active":"2023-09-30T14:37:12.977049+0000","last_peered":"2023-09-30T14:37:12.977049+0000","last_clean":"2023-09-30T14:37:12.977049+0000","last_became_active":"2023-09-30T14:37:11.992991+0000","last_became_peered":"2023-09-30T14:37:11.992991+0000","last_unstale":"2023-09-30T14:37:12.977049+0000","last_undegraded":"2023-09-30T14:37:12.977049+0000","last_fullsized":"2023-09-30T14:37:12.977049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:39:50.995717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.977224+0000","last_change":"2023-09-30T14:37:11.020653+0000","last_active":"2023-09-30T14:37:12.977224+0000","last_peered":"2023-09-30T14:37:12.977224+0000","last_clean":"2023-09-30T14:37:12.977224+0000","last_became_active":"2023-09-30T14:37:11.017621+0000","last_became_peered":"2023-09-30T14:37:11.017621+0000","last_unstale":"2023-09-30T14:37:12.977224+0000","last_undegraded":"2023-09-30T14:37:12.977224+0000","last_fullsized":"2023-09-30T14:37:12.977224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:26:59.395278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014548+0000","last_change":"2023-09-30T14:37:11.999881+0000","last_active":"2023-09-30T14:37:13.014548+0000","last_peered":"2023-09-30T14:37:13.014548+0000","last_clean":"2023-09-30T14:37:13.014548+0000","last_became_active":"2023-09-30T14:37:11.999614+0000","last_became_peered":"2023-09-30T14:37:11.999614+0000","last_unstale":"2023-09-30T14:37:13.014548+0000","last_undegraded":"2023-09-30T14:37:13.014548+0000","last_fullsized":"2023-09-30T14:37:13.014548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:59:44.198361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.071987+0000","last_change":"2023-09-30T14:37:11.027187+0000","last_active":"2023-09-30T14:37:16.071987+0000","last_peered":"2023-09-30T14:37:16.071987+0000","last_clean":"2023-09-30T14:37:16.071987+0000","last_became_active":"2023-09-30T14:37:11.026882+0000","last_became_peered":"2023-09-30T14:37:11.026882+0000","last_unstale":"2023-09-30T14:37:16.071987+0000","last_undegraded":"2023-09-30T14:37:16.071987+0000","last_fullsized":"2023-09-30T14:37:16.071987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:17:11.824002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976283+0000","last_change":"2023-09-30T14:37:11.986979+0000","last_active":"2023-09-30T14:37:12.976283+0000","last_peered":"2023-09-30T14:37:12.976283+0000","last_clean":"2023-09-30T14:37:12.976283+0000","last_became_active":"2023-09-30T14:37:11.986705+0000","last_became_peered":"2023-09-30T14:37:11.986705+0000","last_unstale":"2023-09-30T14:37:12.976283+0000","last_undegraded":"2023-09-30T14:37:12.976283+0000","last_fullsized":"2023-09-30T14:37:12.976283+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:31:40.319747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.395336+0000","last_change":"2023-09-30T14:37:11.022313+0000","last_active":"2023-09-30T14:37:12.395336+0000","last_peered":"2023-09-30T14:37:12.395336+0000","last_clean":"2023-09-30T14:37:12.395336+0000","last_became_active":"2023-09-30T14:37:11.021949+0000","last_became_peered":"2023-09-30T14:37:11.021949+0000","last_unstale":"2023-09-30T14:37:12.395336+0000","last_undegraded":"2023-09-30T14:37:12.395336+0000","last_fullsized":"2023-09-30T14:37:12.395336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:30:26.913666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014721+0000","last_change":"2023-09-30T14:37:11.988966+0000","last_active":"2023-09-30T14:37:13.014721+0000","last_peered":"2023-09-30T14:37:13.014721+0000","last_clean":"2023-09-30T14:37:13.014721+0000","last_became_active":"2023-09-30T14:37:11.988716+0000","last_became_peered":"2023-09-30T14:37:11.988716+0000","last_unstale":"2023-09-30T14:37:13.014721+0000","last_undegraded":"2023-09-30T14:37:13.014721+0000","last_fullsized":"2023-09-30T14:37:13.014721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:25:31.437584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":61,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:56.042046+0000","last_change":"2023-09-30T14:37:11.025855+0000","last_active":"2023-09-30T14:37:56.042046+0000","last_peered":"2023-09-30T14:37:56.042046+0000","last_clean":"2023-09-30T14:37:56.042046+0000","last_became_active":"2023-09-30T14:37:11.025402+0000","last_became_peered":"2023-09-30T14:37:11.025402+0000","last_unstale":"2023-09-30T14:37:56.042046+0000","last_undegraded":"2023-09-30T14:37:56.042046+0000","last_fullsized":"2023-09-30T14:37:56.042046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:50:37.692533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130846+0000","last_change":"2023-09-30T14:37:11.993074+0000","last_active":"2023-09-30T14:37:13.130846+0000","last_peered":"2023-09-30T14:37:13.130846+0000","last_clean":"2023-09-30T14:37:13.130846+0000","last_became_active":"2023-09-30T14:37:11.992784+0000","last_became_peered":"2023-09-30T14:37:11.992784+0000","last_unstale":"2023-09-30T14:37:13.130846+0000","last_undegraded":"2023-09-30T14:37:13.130846+0000","last_fullsized":"2023-09-30T14:37:13.130846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:28:18.454389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.073464+0000","last_change":"2023-09-30T14:37:11.022677+0000","last_active":"2023-09-30T14:37:16.073464+0000","last_peered":"2023-09-30T14:37:16.073464+0000","last_clean":"2023-09-30T14:37:16.073464+0000","last_became_active":"2023-09-30T14:37:11.022357+0000","last_became_peered":"2023-09-30T14:37:11.022357+0000","last_unstale":"2023-09-30T14:37:16.073464+0000","last_undegraded":"2023-09-30T14:37:16.073464+0000","last_fullsized":"2023-09-30T14:37:16.073464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:00:15.500968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014764+0000","last_change":"2023-09-30T14:37:11.992358+0000","last_active":"2023-09-30T14:37:13.014764+0000","last_peered":"2023-09-30T14:37:13.014764+0000","last_clean":"2023-09-30T14:37:13.014764+0000","last_became_active":"2023-09-30T14:37:11.992094+0000","last_became_peered":"2023-09-30T14:37:11.992094+0000","last_unstale":"2023-09-30T14:37:13.014764+0000","last_undegraded":"2023-09-30T14:37:13.014764+0000","last_fullsized":"2023-09-30T14:37:13.014764+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:59:36.817138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.075956+0000","last_change":"2023-09-30T14:37:11.030961+0000","last_active":"2023-09-30T14:37:16.075956+0000","last_peered":"2023-09-30T14:37:16.075956+0000","last_clean":"2023-09-30T14:37:16.075956+0000","last_became_active":"2023-09-30T14:37:11.030662+0000","last_became_peered":"2023-09-30T14:37:11.030662+0000","last_unstale":"2023-09-30T14:37:16.075956+0000","last_undegraded":"2023-09-30T14:37:16.075956+0000","last_fullsized":"2023-09-30T14:37:16.075956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:35:13.313382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975724+0000","last_change":"2023-09-30T14:37:11.989039+0000","last_active":"2023-09-30T14:37:12.975724+0000","last_peered":"2023-09-30T14:37:12.975724+0000","last_clean":"2023-09-30T14:37:12.975724+0000","last_became_active":"2023-09-30T14:37:11.988594+0000","last_became_peered":"2023-09-30T14:37:11.988594+0000","last_unstale":"2023-09-30T14:37:12.975724+0000","last_undegraded":"2023-09-30T14:37:12.975724+0000","last_fullsized":"2023-09-30T14:37:12.975724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:52:58.146681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014581+0000","last_change":"2023-09-30T14:37:11.026545+0000","last_active":"2023-09-30T14:37:13.014581+0000","last_peered":"2023-09-30T14:37:13.014581+0000","last_clean":"2023-09-30T14:37:13.014581+0000","last_became_active":"2023-09-30T14:37:11.026063+0000","last_became_peered":"2023-09-30T14:37:11.026063+0000","last_unstale":"2023-09-30T14:37:13.014581+0000","last_undegraded":"2023-09-30T14:37:13.014581+0000","last_fullsized":"2023-09-30T14:37:13.014581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:15:34.020336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014356+0000","last_change":"2023-09-30T14:37:11.991845+0000","last_active":"2023-09-30T14:37:13.014356+0000","last_peered":"2023-09-30T14:37:13.014356+0000","last_clean":"2023-09-30T14:37:13.014356+0000","last_became_active":"2023-09-30T14:37:11.991560+0000","last_became_peered":"2023-09-30T14:37:11.991560+0000","last_unstale":"2023-09-30T14:37:13.014356+0000","last_undegraded":"2023-09-30T14:37:13.014356+0000","last_fullsized":"2023-09-30T14:37:13.014356+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:27:39.653882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":55,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:54.038504+0000","last_change":"2023-09-30T14:37:11.028192+0000","last_active":"2023-09-30T14:37:54.038504+0000","last_peered":"2023-09-30T14:37:54.038504+0000","last_clean":"2023-09-30T14:37:54.038504+0000","last_became_active":"2023-09-30T14:37:11.027902+0000","last_became_peered":"2023-09-30T14:37:11.027902+0000","last_unstale":"2023-09-30T14:37:54.038504+0000","last_undegraded":"2023-09-30T14:37:54.038504+0000","last_fullsized":"2023-09-30T14:37:54.038504+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:21:30.267559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976420+0000","last_change":"2023-09-30T14:37:11.997318+0000","last_active":"2023-09-30T14:37:12.976420+0000","last_peered":"2023-09-30T14:37:12.976420+0000","last_clean":"2023-09-30T14:37:12.976420+0000","last_became_active":"2023-09-30T14:37:11.996988+0000","last_became_peered":"2023-09-30T14:37:11.996988+0000","last_unstale":"2023-09-30T14:37:12.976420+0000","last_undegraded":"2023-09-30T14:37:12.976420+0000","last_fullsized":"2023-09-30T14:37:12.976420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:57:21.510952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063345+0000","last_change":"2023-09-30T14:37:11.023686+0000","last_active":"2023-09-30T14:37:15.063345+0000","last_peered":"2023-09-30T14:37:15.063345+0000","last_clean":"2023-09-30T14:37:15.063345+0000","last_became_active":"2023-09-30T14:37:11.023365+0000","last_became_peered":"2023-09-30T14:37:11.023365+0000","last_unstale":"2023-09-30T14:37:15.063345+0000","last_undegraded":"2023-09-30T14:37:15.063345+0000","last_fullsized":"2023-09-30T14:37:15.063345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:00:04.358289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014711+0000","last_change":"2023-09-30T14:37:11.989338+0000","last_active":"2023-09-30T14:37:13.014711+0000","last_peered":"2023-09-30T14:37:13.014711+0000","last_clean":"2023-09-30T14:37:13.014711+0000","last_became_active":"2023-09-30T14:37:11.989045+0000","last_became_peered":"2023-09-30T14:37:11.989045+0000","last_unstale":"2023-09-30T14:37:13.014711+0000","last_undegraded":"2023-09-30T14:37:13.014711+0000","last_fullsized":"2023-09-30T14:37:13.014711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:58:59.359684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063525+0000","last_change":"2023-09-30T14:37:11.024709+0000","last_active":"2023-09-30T14:37:15.063525+0000","last_peered":"2023-09-30T14:37:15.063525+0000","last_clean":"2023-09-30T14:37:15.063525+0000","last_became_active":"2023-09-30T14:37:11.024344+0000","last_became_peered":"2023-09-30T14:37:11.024344+0000","last_unstale":"2023-09-30T14:37:15.063525+0000","last_undegraded":"2023-09-30T14:37:15.063525+0000","last_fullsized":"2023-09-30T14:37:15.063525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:17:49.508396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976335+0000","last_change":"2023-09-30T14:37:11.989797+0000","last_active":"2023-09-30T14:37:12.976335+0000","last_peered":"2023-09-30T14:37:12.976335+0000","last_clean":"2023-09-30T14:37:12.976335+0000","last_became_active":"2023-09-30T14:37:11.989482+0000","last_became_peered":"2023-09-30T14:37:11.989482+0000","last_unstale":"2023-09-30T14:37:12.976335+0000","last_undegraded":"2023-09-30T14:37:12.976335+0000","last_fullsized":"2023-09-30T14:37:12.976335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:33:20.437706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'26","reported_seq":124,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:55.046663+0000","last_change":"2023-09-30T14:37:11.027322+0000","last_active":"2023-09-30T14:37:55.046663+0000","last_peered":"2023-09-30T14:37:55.046663+0000","last_clean":"2023-09-30T14:37:55.046663+0000","last_became_active":"2023-09-30T14:37:11.027007+0000","last_became_peered":"2023-09-30T14:37:11.027007+0000","last_unstale":"2023-09-30T14:37:55.046663+0000","last_undegraded":"2023-09-30T14:37:55.046663+0000","last_fullsized":"2023-09-30T14:37:55.046663+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:47:22.919317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54708,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":85,"num_read_kb":95,"num_write":27,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130730+0000","last_change":"2023-09-30T14:37:11.988469+0000","last_active":"2023-09-30T14:37:13.130730+0000","last_peered":"2023-09-30T14:37:13.130730+0000","last_clean":"2023-09-30T14:37:13.130730+0000","last_became_active":"2023-09-30T14:37:11.988118+0000","last_became_peered":"2023-09-30T14:37:11.988118+0000","last_unstale":"2023-09-30T14:37:13.130730+0000","last_undegraded":"2023-09-30T14:37:13.130730+0000","last_fullsized":"2023-09-30T14:37:13.130730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:11:56.750690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.072486+0000","last_change":"2023-09-30T14:37:11.024345+0000","last_active":"2023-09-30T14:37:16.072486+0000","last_peered":"2023-09-30T14:37:16.072486+0000","last_clean":"2023-09-30T14:37:16.072486+0000","last_became_active":"2023-09-30T14:37:11.024039+0000","last_became_peered":"2023-09-30T14:37:11.024039+0000","last_unstale":"2023-09-30T14:37:16.072486+0000","last_undegraded":"2023-09-30T14:37:16.072486+0000","last_fullsized":"2023-09-30T14:37:16.072486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:48:32.872814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014748+0000","last_change":"2023-09-30T14:37:11.994068+0000","last_active":"2023-09-30T14:37:13.014748+0000","last_peered":"2023-09-30T14:37:13.014748+0000","last_clean":"2023-09-30T14:37:13.014748+0000","last_became_active":"2023-09-30T14:37:11.993793+0000","last_became_peered":"2023-09-30T14:37:11.993793+0000","last_unstale":"2023-09-30T14:37:13.014748+0000","last_undegraded":"2023-09-30T14:37:13.014748+0000","last_fullsized":"2023-09-30T14:37:13.014748+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:51:28.071414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063744+0000","last_change":"2023-09-30T14:37:11.025622+0000","last_active":"2023-09-30T14:37:15.063744+0000","last_peered":"2023-09-30T14:37:15.063744+0000","last_clean":"2023-09-30T14:37:15.063744+0000","last_became_active":"2023-09-30T14:37:11.025338+0000","last_became_peered":"2023-09-30T14:37:11.025338+0000","last_unstale":"2023-09-30T14:37:15.063744+0000","last_undegraded":"2023-09-30T14:37:15.063744+0000","last_fullsized":"2023-09-30T14:37:15.063744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:32:03.166296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.015550+0000","last_change":"2023-09-30T14:37:11.990678+0000","last_active":"2023-09-30T14:37:13.015550+0000","last_peered":"2023-09-30T14:37:13.015550+0000","last_clean":"2023-09-30T14:37:13.015550+0000","last_became_active":"2023-09-30T14:37:11.990331+0000","last_became_peered":"2023-09-30T14:37:11.990331+0000","last_unstale":"2023-09-30T14:37:13.015550+0000","last_undegraded":"2023-09-30T14:37:13.015550+0000","last_fullsized":"2023-09-30T14:37:13.015550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:57:06.296415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:51.034168+0000","last_change":"2023-09-30T14:37:11.029501+0000","last_active":"2023-09-30T14:37:51.034168+0000","last_peered":"2023-09-30T14:37:51.034168+0000","last_clean":"2023-09-30T14:37:51.034168+0000","last_became_active":"2023-09-30T14:37:11.029207+0000","last_became_peered":"2023-09-30T14:37:11.029207+0000","last_unstale":"2023-09-30T14:37:51.034168+0000","last_undegraded":"2023-09-30T14:37:51.034168+0000","last_fullsized":"2023-09-30T14:37:51.034168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:48:53.145952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014366+0000","last_change":"2023-09-30T14:37:11.993052+0000","last_active":"2023-09-30T14:37:13.014366+0000","last_peered":"2023-09-30T14:37:13.014366+0000","last_clean":"2023-09-30T14:37:13.014366+0000","last_became_active":"2023-09-30T14:37:11.992762+0000","last_became_peered":"2023-09-30T14:37:11.992762+0000","last_unstale":"2023-09-30T14:37:13.014366+0000","last_undegraded":"2023-09-30T14:37:13.014366+0000","last_fullsized":"2023-09-30T14:37:13.014366+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:09:16.440026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.075318+0000","last_change":"2023-09-30T14:37:11.027266+0000","last_active":"2023-09-30T14:37:16.075318+0000","last_peered":"2023-09-30T14:37:16.075318+0000","last_clean":"2023-09-30T14:37:16.075318+0000","last_became_active":"2023-09-30T14:37:11.026971+0000","last_became_peered":"2023-09-30T14:37:11.026971+0000","last_unstale":"2023-09-30T14:37:16.075318+0000","last_undegraded":"2023-09-30T14:37:16.075318+0000","last_fullsized":"2023-09-30T14:37:16.075318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:38:09.543391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014408+0000","last_change":"2023-09-30T14:37:11.989592+0000","last_active":"2023-09-30T14:37:13.014408+0000","last_peered":"2023-09-30T14:37:13.014408+0000","last_clean":"2023-09-30T14:37:13.014408+0000","last_became_active":"2023-09-30T14:37:11.989290+0000","last_became_peered":"2023-09-30T14:37:11.989290+0000","last_unstale":"2023-09-30T14:37:13.014408+0000","last_undegraded":"2023-09-30T14:37:13.014408+0000","last_fullsized":"2023-09-30T14:37:13.014408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:11:33.974823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.980060+0000","last_change":"2023-09-30T14:37:11.028919+0000","last_active":"2023-09-30T14:37:16.980060+0000","last_peered":"2023-09-30T14:37:16.980060+0000","last_clean":"2023-09-30T14:37:16.980060+0000","last_became_active":"2023-09-30T14:37:11.028532+0000","last_became_peered":"2023-09-30T14:37:11.028532+0000","last_unstale":"2023-09-30T14:37:16.980060+0000","last_undegraded":"2023-09-30T14:37:16.980060+0000","last_fullsized":"2023-09-30T14:37:16.980060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:15:42.675409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014483+0000","last_change":"2023-09-30T14:37:11.980611+0000","last_active":"2023-09-30T14:37:13.014483+0000","last_peered":"2023-09-30T14:37:13.014483+0000","last_clean":"2023-09-30T14:37:13.014483+0000","last_became_active":"2023-09-30T14:37:11.980288+0000","last_became_peered":"2023-09-30T14:37:11.980288+0000","last_unstale":"2023-09-30T14:37:13.014483+0000","last_undegraded":"2023-09-30T14:37:13.014483+0000","last_fullsized":"2023-09-30T14:37:13.014483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:38:49.270976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014852+0000","last_change":"2023-09-30T14:37:11.025133+0000","last_active":"2023-09-30T14:37:13.014852+0000","last_peered":"2023-09-30T14:37:13.014852+0000","last_clean":"2023-09-30T14:37:13.014852+0000","last_became_active":"2023-09-30T14:37:11.024914+0000","last_became_peered":"2023-09-30T14:37:11.024914+0000","last_unstale":"2023-09-30T14:37:13.014852+0000","last_undegraded":"2023-09-30T14:37:13.014852+0000","last_fullsized":"2023-09-30T14:37:13.014852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:23:39.328773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.286974+0000","last_change":"2023-09-30T14:37:11.988434+0000","last_active":"2023-09-30T14:37:15.286974+0000","last_peered":"2023-09-30T14:37:15.286974+0000","last_clean":"2023-09-30T14:37:15.286974+0000","last_became_active":"2023-09-30T14:37:11.988156+0000","last_became_peered":"2023-09-30T14:37:11.988156+0000","last_unstale":"2023-09-30T14:37:15.286974+0000","last_undegraded":"2023-09-30T14:37:15.286974+0000","last_fullsized":"2023-09-30T14:37:15.286974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:20:49.420026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014160+0000","last_change":"2023-09-30T14:37:11.026290+0000","last_active":"2023-09-30T14:37:13.014160+0000","last_peered":"2023-09-30T14:37:13.014160+0000","last_clean":"2023-09-30T14:37:13.014160+0000","last_became_active":"2023-09-30T14:37:11.024684+0000","last_became_peered":"2023-09-30T14:37:11.024684+0000","last_unstale":"2023-09-30T14:37:13.014160+0000","last_undegraded":"2023-09-30T14:37:13.014160+0000","last_fullsized":"2023-09-30T14:37:13.014160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:43:06.463243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.016048+0000","last_change":"2023-09-30T14:37:11.987989+0000","last_active":"2023-09-30T14:37:13.016048+0000","last_peered":"2023-09-30T14:37:13.016048+0000","last_clean":"2023-09-30T14:37:13.016048+0000","last_became_active":"2023-09-30T14:37:11.987690+0000","last_became_peered":"2023-09-30T14:37:11.987690+0000","last_unstale":"2023-09-30T14:37:13.016048+0000","last_undegraded":"2023-09-30T14:37:13.016048+0000","last_fullsized":"2023-09-30T14:37:13.016048+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:51:25.825871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014864+0000","last_change":"2023-09-30T14:37:11.996852+0000","last_active":"2023-09-30T14:37:13.014864+0000","last_peered":"2023-09-30T14:37:13.014864+0000","last_clean":"2023-09-30T14:37:13.014864+0000","last_became_active":"2023-09-30T14:37:11.996596+0000","last_became_peered":"2023-09-30T14:37:11.996596+0000","last_unstale":"2023-09-30T14:37:13.014864+0000","last_undegraded":"2023-09-30T14:37:13.014864+0000","last_fullsized":"2023-09-30T14:37:13.014864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:25:04.531236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:54.037759+0000","last_change":"2023-09-30T14:37:11.025458+0000","last_active":"2023-09-30T14:37:54.037759+0000","last_peered":"2023-09-30T14:37:54.037759+0000","last_clean":"2023-09-30T14:37:54.037759+0000","last_became_active":"2023-09-30T14:37:11.025157+0000","last_became_peered":"2023-09-30T14:37:11.025157+0000","last_unstale":"2023-09-30T14:37:54.037759+0000","last_undegraded":"2023-09-30T14:37:54.037759+0000","last_fullsized":"2023-09-30T14:37:54.037759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:31:24.436125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976151+0000","last_change":"2023-09-30T14:37:11.986840+0000","last_active":"2023-09-30T14:37:12.976151+0000","last_peered":"2023-09-30T14:37:12.976151+0000","last_clean":"2023-09-30T14:37:12.976151+0000","last_became_active":"2023-09-30T14:37:11.986483+0000","last_became_peered":"2023-09-30T14:37:11.986483+0000","last_unstale":"2023-09-30T14:37:12.976151+0000","last_undegraded":"2023-09-30T14:37:12.976151+0000","last_fullsized":"2023-09-30T14:37:12.976151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:14:32.399685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.979424+0000","last_change":"2023-09-30T14:37:11.025706+0000","last_active":"2023-09-30T14:37:16.979424+0000","last_peered":"2023-09-30T14:37:16.979424+0000","last_clean":"2023-09-30T14:37:16.979424+0000","last_became_active":"2023-09-30T14:37:11.025376+0000","last_became_peered":"2023-09-30T14:37:11.025376+0000","last_unstale":"2023-09-30T14:37:16.979424+0000","last_undegraded":"2023-09-30T14:37:16.979424+0000","last_fullsized":"2023-09-30T14:37:16.979424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:10:36.025273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":308,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:11.977686+0000","last_change":"2023-09-30T14:36:12.344762+0000","last_active":"2023-09-30T14:37:11.977686+0000","last_peered":"2023-09-30T14:37:11.977686+0000","last_clean":"2023-09-30T14:37:11.977686+0000","last_became_active":"2023-09-30T14:36:12.344268+0000","last_became_peered":"2023-09-30T14:36:12.344268+0000","last_unstale":"2023-09-30T14:37:11.977686+0000","last_undegraded":"2023-09-30T14:37:11.977686+0000","last_fullsized":"2023-09-30T14:37:11.977686+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:33:20.434992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:33:20.434992+0000","last_clean_scrub_stamp":"2023-09-30T14:33:20.434992+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:00:05.598961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014131+0000","last_change":"2023-09-30T14:37:11.996886+0000","last_active":"2023-09-30T14:37:13.014131+0000","last_peered":"2023-09-30T14:37:13.014131+0000","last_clean":"2023-09-30T14:37:13.014131+0000","last_became_active":"2023-09-30T14:37:11.996667+0000","last_became_peered":"2023-09-30T14:37:11.996667+0000","last_unstale":"2023-09-30T14:37:13.014131+0000","last_undegraded":"2023-09-30T14:37:13.014131+0000","last_fullsized":"2023-09-30T14:37:13.014131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:27:00.546356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.074562+0000","last_change":"2023-09-30T14:37:11.027012+0000","last_active":"2023-09-30T14:37:16.074562+0000","last_peered":"2023-09-30T14:37:16.074562+0000","last_clean":"2023-09-30T14:37:16.074562+0000","last_became_active":"2023-09-30T14:37:11.026714+0000","last_became_peered":"2023-09-30T14:37:11.026714+0000","last_unstale":"2023-09-30T14:37:16.074562+0000","last_undegraded":"2023-09-30T14:37:16.074562+0000","last_fullsized":"2023-09-30T14:37:16.074562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:17:57.860438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.062444+0000","last_change":"2023-09-30T14:37:11.027389+0000","last_active":"2023-09-30T14:37:15.062444+0000","last_peered":"2023-09-30T14:37:15.062444+0000","last_clean":"2023-09-30T14:37:15.062444+0000","last_became_active":"2023-09-30T14:37:11.027011+0000","last_became_peered":"2023-09-30T14:37:11.027011+0000","last_unstale":"2023-09-30T14:37:15.062444+0000","last_undegraded":"2023-09-30T14:37:15.062444+0000","last_fullsized":"2023-09-30T14:37:15.062444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:38:22.953936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014762+0000","last_change":"2023-09-30T14:37:11.997675+0000","last_active":"2023-09-30T14:37:13.014762+0000","last_peered":"2023-09-30T14:37:13.014762+0000","last_clean":"2023-09-30T14:37:13.014762+0000","last_became_active":"2023-09-30T14:37:11.997330+0000","last_became_peered":"2023-09-30T14:37:11.997330+0000","last_unstale":"2023-09-30T14:37:13.014762+0000","last_undegraded":"2023-09-30T14:37:13.014762+0000","last_fullsized":"2023-09-30T14:37:13.014762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:54:52.802138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:20.829473+0000","last_change":"2023-09-30T14:37:11.023836+0000","last_active":"2023-09-30T14:37:20.829473+0000","last_peered":"2023-09-30T14:37:20.829473+0000","last_clean":"2023-09-30T14:37:20.829473+0000","last_became_active":"2023-09-30T14:37:11.023362+0000","last_became_peered":"2023-09-30T14:37:11.023362+0000","last_unstale":"2023-09-30T14:37:20.829473+0000","last_undegraded":"2023-09-30T14:37:20.829473+0000","last_fullsized":"2023-09-30T14:37:20.829473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:58:22.772546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976655+0000","last_change":"2023-09-30T14:37:11.985281+0000","last_active":"2023-09-30T14:37:12.976655+0000","last_peered":"2023-09-30T14:37:12.976655+0000","last_clean":"2023-09-30T14:37:12.976655+0000","last_became_active":"2023-09-30T14:37:11.984944+0000","last_became_peered":"2023-09-30T14:37:11.984944+0000","last_unstale":"2023-09-30T14:37:12.976655+0000","last_undegraded":"2023-09-30T14:37:12.976655+0000","last_fullsized":"2023-09-30T14:37:12.976655+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:30:19.829727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.061631+0000","last_change":"2023-09-30T14:37:11.028070+0000","last_active":"2023-09-30T14:37:15.061631+0000","last_peered":"2023-09-30T14:37:15.061631+0000","last_clean":"2023-09-30T14:37:15.061631+0000","last_became_active":"2023-09-30T14:37:11.027703+0000","last_became_peered":"2023-09-30T14:37:11.027703+0000","last_unstale":"2023-09-30T14:37:15.061631+0000","last_undegraded":"2023-09-30T14:37:15.061631+0000","last_fullsized":"2023-09-30T14:37:15.061631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:23:15.071946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014359+0000","last_change":"2023-09-30T14:37:11.989897+0000","last_active":"2023-09-30T14:37:13.014359+0000","last_peered":"2023-09-30T14:37:13.014359+0000","last_clean":"2023-09-30T14:37:13.014359+0000","last_became_active":"2023-09-30T14:37:11.989613+0000","last_became_peered":"2023-09-30T14:37:11.989613+0000","last_unstale":"2023-09-30T14:37:13.014359+0000","last_undegraded":"2023-09-30T14:37:13.014359+0000","last_fullsized":"2023-09-30T14:37:13.014359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:43:17.887766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.062793+0000","last_change":"2023-09-30T14:37:11.026491+0000","last_active":"2023-09-30T14:37:15.062793+0000","last_peered":"2023-09-30T14:37:15.062793+0000","last_clean":"2023-09-30T14:37:15.062793+0000","last_became_active":"2023-09-30T14:37:11.026196+0000","last_became_peered":"2023-09-30T14:37:11.026196+0000","last_unstale":"2023-09-30T14:37:15.062793+0000","last_undegraded":"2023-09-30T14:37:15.062793+0000","last_fullsized":"2023-09-30T14:37:15.062793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:17:22.343377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130535+0000","last_change":"2023-09-30T14:37:11.988570+0000","last_active":"2023-09-30T14:37:13.130535+0000","last_peered":"2023-09-30T14:37:13.130535+0000","last_clean":"2023-09-30T14:37:13.130535+0000","last_became_active":"2023-09-30T14:37:11.988230+0000","last_became_peered":"2023-09-30T14:37:11.988230+0000","last_unstale":"2023-09-30T14:37:13.130535+0000","last_undegraded":"2023-09-30T14:37:13.130535+0000","last_fullsized":"2023-09-30T14:37:13.130535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:58:38.302343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.062480+0000","last_change":"2023-09-30T14:37:11.024587+0000","last_active":"2023-09-30T14:37:15.062480+0000","last_peered":"2023-09-30T14:37:15.062480+0000","last_clean":"2023-09-30T14:37:15.062480+0000","last_became_active":"2023-09-30T14:37:11.024253+0000","last_became_peered":"2023-09-30T14:37:11.024253+0000","last_unstale":"2023-09-30T14:37:15.062480+0000","last_undegraded":"2023-09-30T14:37:15.062480+0000","last_fullsized":"2023-09-30T14:37:15.062480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:31:07.293657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014239+0000","last_change":"2023-09-30T14:37:11.987818+0000","last_active":"2023-09-30T14:37:13.014239+0000","last_peered":"2023-09-30T14:37:13.014239+0000","last_clean":"2023-09-30T14:37:13.014239+0000","last_became_active":"2023-09-30T14:37:11.987476+0000","last_became_peered":"2023-09-30T14:37:11.987476+0000","last_unstale":"2023-09-30T14:37:13.014239+0000","last_undegraded":"2023-09-30T14:37:13.014239+0000","last_fullsized":"2023-09-30T14:37:13.014239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:01:07.802959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:14.444569+0000","last_change":"2023-09-30T14:37:11.023223+0000","last_active":"2023-09-30T14:37:14.444569+0000","last_peered":"2023-09-30T14:37:14.444569+0000","last_clean":"2023-09-30T14:37:14.444569+0000","last_became_active":"2023-09-30T14:37:11.022932+0000","last_became_peered":"2023-09-30T14:37:11.022932+0000","last_unstale":"2023-09-30T14:37:14.444569+0000","last_undegraded":"2023-09-30T14:37:14.444569+0000","last_fullsized":"2023-09-30T14:37:14.444569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:01:04.315669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976068+0000","last_change":"2023-09-30T14:37:11.990323+0000","last_active":"2023-09-30T14:37:12.976068+0000","last_peered":"2023-09-30T14:37:12.976068+0000","last_clean":"2023-09-30T14:37:12.976068+0000","last_became_active":"2023-09-30T14:37:11.989981+0000","last_became_peered":"2023-09-30T14:37:11.989981+0000","last_unstale":"2023-09-30T14:37:12.976068+0000","last_undegraded":"2023-09-30T14:37:12.976068+0000","last_fullsized":"2023-09-30T14:37:12.976068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:30:44.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063371+0000","last_change":"2023-09-30T14:37:11.025658+0000","last_active":"2023-09-30T14:37:15.063371+0000","last_peered":"2023-09-30T14:37:15.063371+0000","last_clean":"2023-09-30T14:37:15.063371+0000","last_became_active":"2023-09-30T14:37:11.025334+0000","last_became_peered":"2023-09-30T14:37:11.025334+0000","last_unstale":"2023-09-30T14:37:15.063371+0000","last_undegraded":"2023-09-30T14:37:15.063371+0000","last_fullsized":"2023-09-30T14:37:15.063371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:23:41.933634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014128+0000","last_change":"2023-09-30T14:37:11.989989+0000","last_active":"2023-09-30T14:37:13.014128+0000","last_peered":"2023-09-30T14:37:13.014128+0000","last_clean":"2023-09-30T14:37:13.014128+0000","last_became_active":"2023-09-30T14:37:11.989666+0000","last_became_peered":"2023-09-30T14:37:11.989666+0000","last_unstale":"2023-09-30T14:37:13.014128+0000","last_undegraded":"2023-09-30T14:37:13.014128+0000","last_fullsized":"2023-09-30T14:37:13.014128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:41:09.835210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.074595+0000","last_change":"2023-09-30T14:37:11.025027+0000","last_active":"2023-09-30T14:37:16.074595+0000","last_peered":"2023-09-30T14:37:16.074595+0000","last_clean":"2023-09-30T14:37:16.074595+0000","last_became_active":"2023-09-30T14:37:11.024696+0000","last_became_peered":"2023-09-30T14:37:11.024696+0000","last_unstale":"2023-09-30T14:37:16.074595+0000","last_undegraded":"2023-09-30T14:37:16.074595+0000","last_fullsized":"2023-09-30T14:37:16.074595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:34:28.384535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014390+0000","last_change":"2023-09-30T14:37:11.992674+0000","last_active":"2023-09-30T14:37:13.014390+0000","last_peered":"2023-09-30T14:37:13.014390+0000","last_clean":"2023-09-30T14:37:13.014390+0000","last_became_active":"2023-09-30T14:37:11.992243+0000","last_became_peered":"2023-09-30T14:37:11.992243+0000","last_unstale":"2023-09-30T14:37:13.014390+0000","last_undegraded":"2023-09-30T14:37:13.014390+0000","last_fullsized":"2023-09-30T14:37:13.014390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:33:19.759637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014643+0000","last_change":"2023-09-30T14:37:11.024525+0000","last_active":"2023-09-30T14:37:13.014643+0000","last_peered":"2023-09-30T14:37:13.014643+0000","last_clean":"2023-09-30T14:37:13.014643+0000","last_became_active":"2023-09-30T14:37:11.024231+0000","last_became_peered":"2023-09-30T14:37:11.024231+0000","last_unstale":"2023-09-30T14:37:13.014643+0000","last_undegraded":"2023-09-30T14:37:13.014643+0000","last_fullsized":"2023-09-30T14:37:13.014643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:53:47.142798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.512369+0000","last_change":"2023-09-30T14:37:11.991401+0000","last_active":"2023-09-30T14:37:15.512369+0000","last_peered":"2023-09-30T14:37:15.512369+0000","last_clean":"2023-09-30T14:37:15.512369+0000","last_became_active":"2023-09-30T14:37:11.991012+0000","last_became_peered":"2023-09-30T14:37:11.991012+0000","last_unstale":"2023-09-30T14:37:15.512369+0000","last_undegraded":"2023-09-30T14:37:15.512369+0000","last_fullsized":"2023-09-30T14:37:15.512369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:55:00.562030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.050302+0000","last_change":"2023-09-30T14:37:11.022893+0000","last_active":"2023-09-30T14:37:12.050302+0000","last_peered":"2023-09-30T14:37:12.050302+0000","last_clean":"2023-09-30T14:37:12.050302+0000","last_became_active":"2023-09-30T14:37:11.022505+0000","last_became_peered":"2023-09-30T14:37:11.022505+0000","last_unstale":"2023-09-30T14:37:12.050302+0000","last_undegraded":"2023-09-30T14:37:12.050302+0000","last_fullsized":"2023-09-30T14:37:12.050302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:00:17.721645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014348+0000","last_change":"2023-09-30T14:37:11.992942+0000","last_active":"2023-09-30T14:37:13.014348+0000","last_peered":"2023-09-30T14:37:13.014348+0000","last_clean":"2023-09-30T14:37:13.014348+0000","last_became_active":"2023-09-30T14:37:11.992698+0000","last_became_peered":"2023-09-30T14:37:11.992698+0000","last_unstale":"2023-09-30T14:37:13.014348+0000","last_undegraded":"2023-09-30T14:37:13.014348+0000","last_fullsized":"2023-09-30T14:37:13.014348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:15:30.239423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014256+0000","last_change":"2023-09-30T14:37:11.984435+0000","last_active":"2023-09-30T14:37:13.014256+0000","last_peered":"2023-09-30T14:37:13.014256+0000","last_clean":"2023-09-30T14:37:13.014256+0000","last_became_active":"2023-09-30T14:37:11.984137+0000","last_became_peered":"2023-09-30T14:37:11.984137+0000","last_unstale":"2023-09-30T14:37:13.014256+0000","last_undegraded":"2023-09-30T14:37:13.014256+0000","last_fullsized":"2023-09-30T14:37:13.014256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:40:41.112326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975815+0000","last_change":"2023-09-30T14:37:11.984481+0000","last_active":"2023-09-30T14:37:12.975815+0000","last_peered":"2023-09-30T14:37:12.975815+0000","last_clean":"2023-09-30T14:37:12.975815+0000","last_became_active":"2023-09-30T14:37:11.984021+0000","last_became_peered":"2023-09-30T14:37:11.984021+0000","last_unstale":"2023-09-30T14:37:12.975815+0000","last_undegraded":"2023-09-30T14:37:12.975815+0000","last_fullsized":"2023-09-30T14:37:12.975815+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:36:33.184379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975777+0000","last_change":"2023-09-30T14:37:11.990779+0000","last_active":"2023-09-30T14:37:12.975777+0000","last_peered":"2023-09-30T14:37:12.975777+0000","last_clean":"2023-09-30T14:37:12.975777+0000","last_became_active":"2023-09-30T14:37:11.989960+0000","last_became_peered":"2023-09-30T14:37:11.989960+0000","last_unstale":"2023-09-30T14:37:12.975777+0000","last_undegraded":"2023-09-30T14:37:12.975777+0000","last_fullsized":"2023-09-30T14:37:12.975777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:54:36.813191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.015049+0000","last_change":"2023-09-30T14:37:11.995750+0000","last_active":"2023-09-30T14:37:13.015049+0000","last_peered":"2023-09-30T14:37:13.015049+0000","last_clean":"2023-09-30T14:37:13.015049+0000","last_became_active":"2023-09-30T14:37:11.995434+0000","last_became_peered":"2023-09-30T14:37:11.995434+0000","last_unstale":"2023-09-30T14:37:13.015049+0000","last_undegraded":"2023-09-30T14:37:13.015049+0000","last_fullsized":"2023-09-30T14:37:13.015049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:49:51.234024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014630+0000","last_change":"2023-09-30T14:37:11.996550+0000","last_active":"2023-09-30T14:37:13.014630+0000","last_peered":"2023-09-30T14:37:13.014630+0000","last_clean":"2023-09-30T14:37:13.014630+0000","last_became_active":"2023-09-30T14:37:11.996269+0000","last_became_peered":"2023-09-30T14:37:11.996269+0000","last_unstale":"2023-09-30T14:37:13.014630+0000","last_undegraded":"2023-09-30T14:37:13.014630+0000","last_fullsized":"2023-09-30T14:37:13.014630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:40:16.537270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014992+0000","last_change":"2023-09-30T14:37:11.983495+0000","last_active":"2023-09-30T14:37:13.014992+0000","last_peered":"2023-09-30T14:37:13.014992+0000","last_clean":"2023-09-30T14:37:13.014992+0000","last_became_active":"2023-09-30T14:37:11.983241+0000","last_became_peered":"2023-09-30T14:37:11.983241+0000","last_unstale":"2023-09-30T14:37:13.014992+0000","last_undegraded":"2023-09-30T14:37:13.014992+0000","last_fullsized":"2023-09-30T14:37:13.014992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:41:46.604467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975888+0000","last_change":"2023-09-30T14:37:11.996058+0000","last_active":"2023-09-30T14:37:12.975888+0000","last_peered":"2023-09-30T14:37:12.975888+0000","last_clean":"2023-09-30T14:37:12.975888+0000","last_became_active":"2023-09-30T14:37:11.995763+0000","last_became_peered":"2023-09-30T14:37:11.995763+0000","last_unstale":"2023-09-30T14:37:12.975888+0000","last_undegraded":"2023-09-30T14:37:12.975888+0000","last_fullsized":"2023-09-30T14:37:12.975888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:42:21.306561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976498+0000","last_change":"2023-09-30T14:37:11.992080+0000","last_active":"2023-09-30T14:37:12.976498+0000","last_peered":"2023-09-30T14:37:12.976498+0000","last_clean":"2023-09-30T14:37:12.976498+0000","last_became_active":"2023-09-30T14:37:11.991747+0000","last_became_peered":"2023-09-30T14:37:11.991747+0000","last_unstale":"2023-09-30T14:37:12.976498+0000","last_undegraded":"2023-09-30T14:37:12.976498+0000","last_fullsized":"2023-09-30T14:37:12.976498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:07:25.985911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014724+0000","last_change":"2023-09-30T14:37:11.997109+0000","last_active":"2023-09-30T14:37:13.014724+0000","last_peered":"2023-09-30T14:37:13.014724+0000","last_clean":"2023-09-30T14:37:13.014724+0000","last_became_active":"2023-09-30T14:37:11.996793+0000","last_became_peered":"2023-09-30T14:37:11.996793+0000","last_unstale":"2023-09-30T14:37:13.014724+0000","last_undegraded":"2023-09-30T14:37:13.014724+0000","last_fullsized":"2023-09-30T14:37:13.014724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:13:59.838638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975786+0000","last_change":"2023-09-30T14:37:11.984083+0000","last_active":"2023-09-30T14:37:12.975786+0000","last_peered":"2023-09-30T14:37:12.975786+0000","last_clean":"2023-09-30T14:37:12.975786+0000","last_became_active":"2023-09-30T14:37:11.983352+0000","last_became_peered":"2023-09-30T14:37:11.983352+0000","last_unstale":"2023-09-30T14:37:12.975786+0000","last_undegraded":"2023-09-30T14:37:12.975786+0000","last_fullsized":"2023-09-30T14:37:12.975786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:02:03.668484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976194+0000","last_change":"2023-09-30T14:37:11.992212+0000","last_active":"2023-09-30T14:37:12.976194+0000","last_peered":"2023-09-30T14:37:12.976194+0000","last_clean":"2023-09-30T14:37:12.976194+0000","last_became_active":"2023-09-30T14:37:11.990158+0000","last_became_peered":"2023-09-30T14:37:11.990158+0000","last_unstale":"2023-09-30T14:37:12.976194+0000","last_undegraded":"2023-09-30T14:37:12.976194+0000","last_fullsized":"2023-09-30T14:37:12.976194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:14:27.976110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014402+0000","last_change":"2023-09-30T14:37:11.995884+0000","last_active":"2023-09-30T14:37:13.014402+0000","last_peered":"2023-09-30T14:37:13.014402+0000","last_clean":"2023-09-30T14:37:13.014402+0000","last_became_active":"2023-09-30T14:37:11.995551+0000","last_became_peered":"2023-09-30T14:37:11.995551+0000","last_unstale":"2023-09-30T14:37:13.014402+0000","last_undegraded":"2023-09-30T14:37:13.014402+0000","last_fullsized":"2023-09-30T14:37:13.014402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:28:26.051188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014437+0000","last_change":"2023-09-30T14:37:11.985853+0000","last_active":"2023-09-30T14:37:13.014437+0000","last_peered":"2023-09-30T14:37:13.014437+0000","last_clean":"2023-09-30T14:37:13.014437+0000","last_became_active":"2023-09-30T14:37:11.985525+0000","last_became_peered":"2023-09-30T14:37:11.985525+0000","last_unstale":"2023-09-30T14:37:13.014437+0000","last_undegraded":"2023-09-30T14:37:13.014437+0000","last_fullsized":"2023-09-30T14:37:13.014437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:07:24.392538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58318,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":258,"num_read_kb":191,"num_write":147,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":174954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449108,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":425582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.433},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.38600000000000001}]},{"osd":1,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.69299999999999995}]},{"osd":2,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.55200000000000005}]},{"osd":5,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.623},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.71499999999999997}]},{"osd":6,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.48299999999999998}]},{"osd":7,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.749},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.59799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":9,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.80700000000000005}]},{"osd":10,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.76000000000000001}]}]},{"osd":10,"up_from":71,"seq":304942678040,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1581056,"data_stored":1071446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":1,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.45400000000000001}]},{"osd":2,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.66700000000000004}]},{"osd":3,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.47599999999999998}]},{"osd":4,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51900000000000002}]},{"osd":5,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.68000000000000005}]},{"osd":6,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.41699999999999998}]},{"osd":8,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.75}]},{"osd":9,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.72499999999999998}]},{"osd":11,"last update":"Sat Sep 30 14:37:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.78600000000000003}]}]},{"osd":9,"up_from":66,"seq":283467841563,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":425487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.46500000000000002,"15min":0.47499999999999998},"min":{"1min":0.28499999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60299999999999998,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.503,"15min":0.51800000000000002},"min":{"1min":0.32600000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65700000000000003,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.32100000000000001}]},{"osd":1,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77400000000000002,"5min":1.069,"15min":1.069},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.64500000000000002,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.749,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65700000000000003}]},{"osd":2,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.61799999999999999,"15min":0.628},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.38,"5min":0.376,"15min":0.376},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.61799999999999999}]},{"osd":3,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.54900000000000004,"15min":0.56599999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69899999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.86199999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.375}]},{"osd":4,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.69299999999999995,"15min":0.70999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80500000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.71399999999999997,"15min":0.72699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86399999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.51200000000000001}]},{"osd":5,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.70499999999999996,"15min":0.72199999999999998},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":1.137,"15min":1.137},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.73299999999999998,"15min":0.749},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0860000000000001,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.60199999999999998}]},{"osd":6,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.69499999999999995,"15min":0.71799999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.046,"5min":1.21,"15min":1.21},"last":0.443},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.71599999999999997,"15min":0.73599999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96299999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.56000000000000005}]},{"osd":7,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.79700000000000004,"15min":0.81599999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94099999999999995,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.81200000000000006,"15min":0.82799999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.72899999999999998}]},{"osd":8,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81200000000000006,"15min":0.82599999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.753},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.81599999999999995,"15min":0.83599999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.028,"5min":1.466,"15min":1.466},"last":0.68799999999999994}]},{"osd":10,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.63400000000000001}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":425738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.64600000000000002}]},{"osd":1,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56200000000000006,"15min":0.55500000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.52700000000000002}]},{"osd":2,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.44800000000000001,"15min":0.44600000000000001},"min":{"1min":0.32600000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.45600000000000002},"min":{"1min":0.29999999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35999999999999999}]},{"osd":3,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.35599999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61499999999999999,"15min":0.61599999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83099999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.66500000000000004}]},{"osd":4,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.44,"5min":0.378,"15min":0.378},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63400000000000001,"15min":0.628},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.61299999999999999}]},{"osd":5,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59799999999999998,"15min":0.59899999999999998},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.44900000000000001}]},{"osd":6,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70599999999999996,"15min":0.70799999999999996},"min":{"1min":0.42899999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.99399999999999999,"5min":1.133,"15min":1.133},"last":0.63300000000000001}]},{"osd":7,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.022,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71899999999999997,"15min":0.71699999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.56200000000000006}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.749,"15min":0.747},"min":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95299999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.72899999999999998}]},{"osd":10,"last update":"Sat Sep 30 14:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1,"5min":1,"15min":1},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1503232,"data_stored":1017283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42999999999999999}]},{"osd":1,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63600000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.439,"15min":0.433},"min":{"1min":0.34300000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.58199999999999996}]},{"osd":2,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.38,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.748},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.72399999999999998}]},{"osd":3,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84099999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.82399999999999995,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.81200000000000006}]},{"osd":4,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.496},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.379,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64000000000000001}]},{"osd":5,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84199999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.83399999999999996}]},{"osd":6,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89500000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.72999999999999998,"15min":0.72899999999999998},"min":{"1min":0.47899999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98599999999999999,"5min":1.004,"15min":1.004},"last":2.9340000000000002}]},{"osd":8,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.48999999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.872},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.92300000000000004}]},{"osd":9,"last update":"Sat Sep 30 14:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77600000000000002,"15min":0.77900000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.57299999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96499999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.90500000000000003}]},{"osd":10,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.94399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":424882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38100000000000001,"15min":0.371},"min":{"1min":0.27700000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.40799999999999997,"15min":0.39000000000000001},"min":{"1min":0.28100000000000003,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.41399999999999998}]},{"osd":1,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.75700000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.72799999999999998}]},{"osd":2,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.628,"15min":0.627},"min":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.75700000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.63800000000000001}]},{"osd":3,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.502,"15min":0.48599999999999999},"min":{"1min":0.35299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.435},{"interface":"front","average":{"1min":0.504,"5min":0.499,"15min":0.497},"min":{"1min":0.35799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.35799999999999998}]},{"osd":4,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.46800000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.503},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67300000000000004,"15min":0.65100000000000002},"min":{"1min":0.46899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.89600000000000002,"5min":1.056,"15min":1.056},"last":0.55300000000000005}]},{"osd":5,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64600000000000002,"15min":0.64700000000000002},"min":{"1min":0.499,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.48099999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1060000000000001,"5min":1.228,"15min":1.228},"last":0.54100000000000004}]},{"osd":7,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.69899999999999995,"15min":0.68400000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.71199999999999997,"15min":0.69999999999999996},"min":{"1min":0.47999999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.79300000000000004}]},{"osd":8,"last update":"Sat Sep 30 14:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.55000000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.81599999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.624},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.70999999999999996}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.748,"15min":0.76000000000000001},"min":{"1min":0.45500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72499999999999998,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93200000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69599999999999995}]},{"osd":10,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.747}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":479377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53300000000000003,"15min":0.499},"min":{"1min":0.36199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73999999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54100000000000004,"15min":0.51300000000000001},"min":{"1min":0.33500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78600000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.66200000000000003}]},{"osd":2,"last update":"Sat Sep 30 14:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.55900000000000005,"15min":0.51100000000000001},"min":{"1min":0.435,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.55400000000000005,"15min":0.51600000000000001},"min":{"1min":0.46200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92400000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68700000000000006}]},{"osd":3,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57899999999999996,"15min":0.54300000000000004},"min":{"1min":0.436,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60599999999999998,"15min":0.57499999999999996},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92200000000000004,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.60799999999999998}]},{"osd":4,"last update":"Sat Sep 30 14:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.443,"15min":0.42999999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.46800000000000003,"15min":0.45200000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.40000000000000002}]},{"osd":5,"last update":"Sat Sep 30 14:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58899999999999997,"15min":0.57799999999999996},"min":{"1min":0.40200000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.63900000000000001,"15min":0.61299999999999999},"min":{"1min":0.51700000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.746}]},{"osd":6,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88300000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69499999999999995,"15min":0.67500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.69999999999999996}]},{"osd":7,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.55700000000000005,"15min":0.54000000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.436}]},{"osd":8,"last update":"Sat Sep 30 14:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.74399999999999999,"15min":0.73599999999999999},"min":{"1min":0.55700000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.48799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.71799999999999997}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.81799999999999995,"15min":0.81399999999999995},"min":{"1min":0.56799999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.78700000000000003}]},{"osd":10,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.77100000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":426147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 14:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.54600000000000004,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79400000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.51200000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.83099999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.78400000000000003}]},{"osd":2,"last update":"Sat Sep 30 14:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.372,"5min":0.317,"15min":0.317},"max":{"1min":0.76300000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57199999999999995,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91500000000000004,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59099999999999997}]},{"osd":3,"last update":"Sat Sep 30 14:35:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.432},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.503},{"interface":"front","average":{"1min":0.498,"5min":0.46100000000000002,"15min":0.45500000000000002},"min":{"1min":0.27700000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.45800000000000002}]},{"osd":4,"last update":"Sat Sep 30 14:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.58599999999999997},"min":{"1min":0.40200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.749},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.60499999999999998,"15min":0.57699999999999996},"min":{"1min":0.47499999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.55500000000000005}]},{"osd":5,"last update":"Sat Sep 30 14:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.47299999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.45100000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.79400000000000004,"5min":0.996,"15min":0.996},"last":0.80400000000000005}]},{"osd":6,"last update":"Sat Sep 30 14:37:47 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69199999999999995,"5min":1.03,"15min":1.03},"last":0.438},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.317,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80600000000000005,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.54200000000000004}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69699999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.014,"5min":1.147,"15min":1.147},"last":0.442},{"interface":"front","average":{"1min":0.748,"5min":0.67800000000000005,"15min":0.66600000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56799999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.52300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85199999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.745,"15min":0.753},"min":{"1min":0.51700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85299999999999998,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.72299999999999998}]},{"osd":9,"last update":"Sat Sep 30 14:36:51 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.60699999999999998}]},{"osd":10,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.626}]},{"osd":11,"last update":"Sat Sep 30 14:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.64200000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":480785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.501,"15min":0.495},"min":{"1min":0.308,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.47799999999999998}]},{"osd":1,"last update":"Sat Sep 30 14:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.40799999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.76200000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.57599999999999996,"15min":0.53400000000000003},"min":{"1min":0.40699999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.76100000000000001}]},{"osd":3,"last update":"Sat Sep 30 14:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.35599999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.56499999999999995},"min":{"1min":0.42699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.56799999999999995}]},{"osd":4,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.54400000000000004},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.432,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85899999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.747}]},{"osd":5,"last update":"Sat Sep 30 14:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29299999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66300000000000003,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.48199999999999998},"min":{"1min":0.37,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73499999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.501}]},{"osd":6,"last update":"Sat Sep 30 14:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66600000000000004,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1000000000000001,"5min":1.405,"15min":1.405},"last":0.629},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.72399999999999998}]},{"osd":7,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.45500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.80100000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70699999999999996,"15min":0.71299999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.86399999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.59199999999999997}]},{"osd":8,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.623,"15min":0.621},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.41099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55300000000000005}]},{"osd":9,"last update":"Sat Sep 30 14:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.64100000000000001}]},{"osd":10,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.79000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":425199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.35799999999999998},"min":{"1min":0.27200000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.432,"15min":0.41199999999999998},"min":{"1min":0.316,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54800000000000004,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.307}]},{"osd":1,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64200000000000002,"15min":0.58599999999999997},"min":{"1min":0.442,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.002,"5min":5.798,"15min":5.798},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.39100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.70999999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58399999999999996,"15min":0.55100000000000005},"min":{"1min":0.442,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.52100000000000002}]},{"osd":4,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.64600000000000002,"15min":0.57899999999999996},"min":{"1min":0.36699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":7.1619999999999999,"5min":7.1619999999999999,"15min":7.1619999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.61199999999999999,"15min":0.57399999999999995},"min":{"1min":0.45500000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.58699999999999997}]},{"osd":5,"last update":"Sat Sep 30 14:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.45800000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85699999999999998,"5min":1.071,"15min":1.071},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.66000000000000003,"15min":0.65300000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79700000000000004,"5min":1.129,"15min":1.129},"last":0.49399999999999999}]},{"osd":6,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72399999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.377}]},{"osd":7,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.72299999999999998,"15min":0.70699999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.66200000000000003}]},{"osd":8,"last update":"Sat Sep 30 14:36:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.46100000000000002}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.69499999999999995,"15min":0.71399999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81299999999999994,"5min":1.161,"15min":1.161},"last":0.442}]},{"osd":10,"last update":"Sat Sep 30 14:37:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":7.2389999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":0.61299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":425644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67100000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.65000000000000002}]},{"osd":1,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.377,"15min":0.36899999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.434,"5min":0.377,"15min":0.36299999999999999},"min":{"1min":0.33700000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.56499999999999995}]},{"osd":2,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.54800000000000004,"15min":0.51600000000000001},"min":{"1min":0.41299999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.871},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.374,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.65500000000000003}]},{"osd":3,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.90900000000000003}]},{"osd":5,"last update":"Sat Sep 30 14:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58599999999999997,"15min":0.56399999999999995},"min":{"1min":0.42999999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91700000000000004,"5min":1.484,"15min":1.484},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.57399999999999995,"15min":0.55400000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83599999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.78800000000000003}]},{"osd":6,"last update":"Sat Sep 30 14:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.626,"15min":0.626},"min":{"1min":0.501,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78300000000000003,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.439,"5min":0.435,"15min":0.435},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.73999999999999999}]},{"osd":7,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54500000000000004,"15min":0.53400000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":0.97299999999999998}]},{"osd":8,"last update":"Sat Sep 30 14:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.66200000000000003,"15min":0.65600000000000003},"min":{"1min":0.51800000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88800000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.46800000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.83799999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.89200000000000002}]},{"osd":9,"last update":"Sat Sep 30 14:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.55100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.85299999999999998,"5min":1.038,"15min":1.038},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.58599999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.95199999999999996,"5min":1.677,"15min":1.677},"last":0.94499999999999995}]},{"osd":10,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.0009999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1016636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51200000000000001,"15min":0.49199999999999999},"min":{"1min":0.34399999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52600000000000002,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78100000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.747}]},{"osd":1,"last update":"Sat Sep 30 14:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.52600000000000002,"15min":0.496},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.626},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.51700000000000002,"15min":0.49099999999999999},"min":{"1min":0.38900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.71999999999999997}]},{"osd":2,"last update":"Sat Sep 30 14:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.432,"15min":0.41499999999999998},"min":{"1min":0.308,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.64200000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.434,"15min":0.41999999999999998},"min":{"1min":0.32700000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.61099999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.40300000000000002}]},{"osd":3,"last update":"Sat Sep 30 14:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60699999999999998,"15min":0.58799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.77900000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.73299999999999998}]},{"osd":4,"last update":"Sat Sep 30 14:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.56499999999999995,"15min":0.53900000000000003},"min":{"1min":0.39700000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.56599999999999995},"min":{"1min":0.46200000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.78000000000000003,"5min":0.875,"15min":0.875},"last":0.64500000000000002}]},{"osd":6,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.69499999999999995,"15min":0.66000000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":4.125,"5min":4.125,"15min":4.125},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":0.73499999999999999,"15min":0.64700000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":12.24,"5min":12.24,"15min":12.24},"last":0.57099999999999995}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67600000000000005,"15min":0.67800000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.65800000000000003}]},{"osd":8,"last update":"Sat Sep 30 14:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42499999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.629,"15min":0.621},"min":{"1min":0.46999999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.54500000000000004}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.78400000000000003,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.503,"15min":0.503},"max":{"1min":5.1520000000000001,"5min":5.1520000000000001,"15min":5.1520000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.3580000000000001,"5min":0.874,"15min":0.79300000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":12.198,"5min":12.198,"15min":12.198},"last":0.79800000000000004}]},{"osd":10,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.83199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T14:37:58.987 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-09-30T14:37:59.553 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:37:59.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:59 smithi156 ceph-mon[103579]: pgmap v276: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:37:59.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:37:59 smithi156 ceph-mon[103579]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:37:59.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:59 smithi148 ceph-mon[102355]: pgmap v276: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:37:59.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:37:59 smithi148 ceph-mon[102355]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:37:59.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:59 smithi032 ceph-mon[98592]: pgmap v276: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:37:59.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:37:59 smithi032 ceph-mon[98592]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:38:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.831+0000 7fae438ca700 1 -- 172.21.15.32:0/4144609537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c105de0 msgr2=0x7fae3c1061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.831+0000 7fae438ca700 1 --2- 172.21.15.32:0/4144609537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c105de0 0x7fae3c1061c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0099c0 tx=0x7fae2c009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.831+0000 7fae438ca700 1 -- 172.21.15.32:0/4144609537 shutdown_connections 2023-09-30T14:38:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.831+0000 7fae438ca700 1 --2- 172.21.15.32:0/4144609537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae3c100790 0x7fae3c10cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.831+0000 7fae438ca700 1 --2- 172.21.15.32:0/4144609537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c105de0 0x7fae3c1061c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.831+0000 7fae438ca700 1 --2- 172.21.15.32:0/4144609537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3c0ffdd0 0x7fae3c100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.831+0000 7fae438ca700 1 -- 172.21.15.32:0/4144609537 >> 172.21.15.32:0/4144609537 conn(0x7fae3c075d70 msgr2=0x7fae3c076180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.831+0000 7fae438ca700 1 -- 172.21.15.32:0/4144609537 shutdown_connections 2023-09-30T14:38:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.831+0000 7fae438ca700 1 -- 172.21.15.32:0/4144609537 wait complete. 2023-09-30T14:38:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.832+0000 7fae438ca700 1 Processor -- start 2023-09-30T14:38:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.832+0000 7fae438ca700 1 -- start start 2023-09-30T14:38:00.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.832+0000 7fae438ca700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3c0ffdd0 0x7fae3c196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:00.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.832+0000 7fae438ca700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae3c100790 0x7fae3c197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:00.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.832+0000 7fae438ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c105de0 0x7fae3c19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:00.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.832+0000 7fae438ca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c19bec0 con 0x7fae3c105de0 2023-09-30T14:38:00.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.832+0000 7fae438ca700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c19c030 con 0x7fae3c100790 2023-09-30T14:38:00.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.832+0000 7fae438ca700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c19c1f0 con 0x7fae3c0ffdd0 2023-09-30T14:38:00.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.832+0000 7fae41e67700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c105de0 0x7fae3c19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:00.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.832+0000 7fae41e67700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c105de0 0x7fae3c19b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34386/0 (socket says 172.21.15.32:34386) 2023-09-30T14:38:00.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.832+0000 7fae41e67700 1 -- 172.21.15.32:0/3243191281 learned_addr learned my addr 172.21.15.32:0/3243191281 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:00.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.832+0000 7fae41666700 1 --2- 172.21.15.32:0/3243191281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3c0ffdd0 0x7fae3c196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:00.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.833+0000 7fae40e65700 1 --2- 172.21.15.32:0/3243191281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae3c100790 0x7fae3c197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:00.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.833+0000 7fae41e67700 1 -- 172.21.15.32:0/3243191281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3c0ffdd0 msgr2=0x7fae3c196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:00.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.833+0000 7fae41e67700 1 --2- 172.21.15.32:0/3243191281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3c0ffdd0 0x7fae3c196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:00.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.833+0000 7fae41e67700 1 -- 172.21.15.32:0/3243191281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae3c100790 msgr2=0x7fae3c197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:00.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.833+0000 7fae41e67700 1 --2- 172.21.15.32:0/3243191281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae3c100790 0x7fae3c197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:00.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.833+0000 7fae41e67700 1 -- 172.21.15.32:0/3243191281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c0095f0 con 0x7fae3c105de0 2023-09-30T14:38:00.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.833+0000 7fae41e67700 1 --2- 172.21.15.32:0/3243191281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c105de0 0x7fae3c19b8b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fae34007ee0 tx=0x7fae3400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:00.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.833+0000 7fae327fc700 1 -- 172.21.15.32:0/3243191281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34017070 con 0x7fae3c105de0 2023-09-30T14:38:00.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.833+0000 7fae327fc700 1 -- 172.21.15.32:0/3243191281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae3400f040 con 0x7fae3c105de0 2023-09-30T14:38:00.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.833+0000 7fae438ca700 1 -- 172.21.15.32:0/3243191281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c19c4d0 con 0x7fae3c105de0 2023-09-30T14:38:00.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.833+0000 7fae327fc700 1 -- 172.21.15.32:0/3243191281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34005740 con 0x7fae3c105de0 2023-09-30T14:38:00.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.833+0000 7fae438ca700 1 -- 172.21.15.32:0/3243191281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c1a3d30 con 0x7fae3c105de0 2023-09-30T14:38:00.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.834+0000 7fae438ca700 1 -- 172.21.15.32:0/3243191281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c04eb20 con 0x7fae3c105de0 2023-09-30T14:38:00.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.836+0000 7fae327fc700 1 -- 172.21.15.32:0/3243191281 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fae3400dc60 con 0x7fae3c105de0 2023-09-30T14:38:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.836+0000 7fae327fc700 1 --2- 172.21.15.32:0/3243191281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fae280a10b0 0x7fae280a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.836+0000 7fae41666700 1 --2- 172.21.15.32:0/3243191281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fae280a10b0 0x7fae280a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.836+0000 7fae327fc700 1 -- 172.21.15.32:0/3243191281 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fae340cbfc0 con 0x7fae3c105de0 2023-09-30T14:38:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.837+0000 7fae41666700 1 --2- 172.21.15.32:0/3243191281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fae280a10b0 0x7fae280a3570 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fae2c000c00 tx=0x7fae2c003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:00.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:00.839+0000 7fae327fc700 1 -- 172.21.15.32:0/3243191281 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fae34095810 con 0x7fae3c105de0 2023-09-30T14:38:01.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:01.035+0000 7fae438ca700 1 -- 172.21.15.32:0/3243191281 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fae3c10fdc0 con 0x7fae280a10b0 2023-09-30T14:38:01.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:01.047+0000 7fae327fc700 1 -- 172.21.15.32:0/3243191281 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340239 (secure 0 0 0) 0x7fae3c10fdc0 con 0x7fae280a10b0 2023-09-30T14:38:01.050 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:01.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:01.050+0000 7fae438ca700 1 -- 172.21.15.32:0/3243191281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fae280a10b0 msgr2=0x7fae280a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:01.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:01.050+0000 7fae438ca700 1 --2- 172.21.15.32:0/3243191281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fae280a10b0 0x7fae280a3570 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fae2c000c00 tx=0x7fae2c003680 comp rx=0 tx=0).stop 2023-09-30T14:38:01.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:01.050+0000 7fae438ca700 1 -- 172.21.15.32:0/3243191281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c105de0 msgr2=0x7fae3c19b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:01.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:01.050+0000 7fae438ca700 1 --2- 172.21.15.32:0/3243191281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c105de0 0x7fae3c19b8b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fae34007ee0 tx=0x7fae3400d3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:01.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:01.050+0000 7fae438ca700 1 -- 172.21.15.32:0/3243191281 shutdown_connections 2023-09-30T14:38:01.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:01.050+0000 7fae438ca700 1 --2- 172.21.15.32:0/3243191281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fae280a10b0 0x7fae280a3570 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:01.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:01.050+0000 7fae438ca700 1 --2- 172.21.15.32:0/3243191281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c105de0 0x7fae3c19b8b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:01.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:01.050+0000 7fae438ca700 1 --2- 172.21.15.32:0/3243191281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3c0ffdd0 0x7fae3c196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:01.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:01.050+0000 7fae438ca700 1 --2- 172.21.15.32:0/3243191281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae3c100790 0x7fae3c197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:01.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:01.050+0000 7fae438ca700 1 -- 172.21.15.32:0/3243191281 >> 172.21.15.32:0/3243191281 conn(0x7fae3c075d70 msgr2=0x7fae3c103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:01.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:01.050+0000 7fae438ca700 1 -- 172.21.15.32:0/3243191281 shutdown_connections 2023-09-30T14:38:01.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:01.050+0000 7fae438ca700 1 -- 172.21.15.32:0/3243191281 wait complete. 2023-09-30T14:38:01.054 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-09-30T14:38:01.337 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:01 smithi032 ceph-mon[98592]: pgmap v277: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:01.400 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2023-09-30T14:37:59.783568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648932,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":389,"num_read_kb":300,"num_write":330,"num_write_kb":3038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":268,"ondisk_log_size":268,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336736,"kb_used_data":12544,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605088,"statfs":{"total":1151940427776,"available":1151595610112,"internally_reserved":0,"allocated":12845056,"data_stored":7044206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003322"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014576+0000","last_change":"2023-09-30T14:37:11.988122+0000","last_active":"2023-09-30T14:37:13.014576+0000","last_peered":"2023-09-30T14:37:13.014576+0000","last_clean":"2023-09-30T14:37:13.014576+0000","last_became_active":"2023-09-30T14:37:11.986770+0000","last_became_peered":"2023-09-30T14:37:11.986770+0000","last_unstale":"2023-09-30T14:37:13.014576+0000","last_undegraded":"2023-09-30T14:37:13.014576+0000","last_fullsized":"2023-09-30T14:37:13.014576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:46:29.655309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014211+0000","last_change":"2023-09-30T14:37:11.987953+0000","last_active":"2023-09-30T14:37:13.014211+0000","last_peered":"2023-09-30T14:37:13.014211+0000","last_clean":"2023-09-30T14:37:13.014211+0000","last_became_active":"2023-09-30T14:37:11.987248+0000","last_became_peered":"2023-09-30T14:37:11.987248+0000","last_unstale":"2023-09-30T14:37:13.014211+0000","last_undegraded":"2023-09-30T14:37:13.014211+0000","last_fullsized":"2023-09-30T14:37:13.014211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:12:59.780089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014752+0000","last_change":"2023-09-30T14:37:11.987570+0000","last_active":"2023-09-30T14:37:13.014752+0000","last_peered":"2023-09-30T14:37:13.014752+0000","last_clean":"2023-09-30T14:37:13.014752+0000","last_became_active":"2023-09-30T14:37:11.987064+0000","last_became_peered":"2023-09-30T14:37:11.987064+0000","last_unstale":"2023-09-30T14:37:13.014752+0000","last_undegraded":"2023-09-30T14:37:13.014752+0000","last_fullsized":"2023-09-30T14:37:13.014752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:26:16.674605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014524+0000","last_change":"2023-09-30T14:37:11.990161+0000","last_active":"2023-09-30T14:37:13.014524+0000","last_peered":"2023-09-30T14:37:13.014524+0000","last_clean":"2023-09-30T14:37:13.014524+0000","last_became_active":"2023-09-30T14:37:11.989580+0000","last_became_peered":"2023-09-30T14:37:11.989580+0000","last_unstale":"2023-09-30T14:37:13.014524+0000","last_undegraded":"2023-09-30T14:37:13.014524+0000","last_fullsized":"2023-09-30T14:37:13.014524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:13:06.986864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130616+0000","last_change":"2023-09-30T14:37:11.993902+0000","last_active":"2023-09-30T14:37:13.130616+0000","last_peered":"2023-09-30T14:37:13.130616+0000","last_clean":"2023-09-30T14:37:13.130616+0000","last_became_active":"2023-09-30T14:37:11.993622+0000","last_became_peered":"2023-09-30T14:37:11.993622+0000","last_unstale":"2023-09-30T14:37:13.130616+0000","last_undegraded":"2023-09-30T14:37:13.130616+0000","last_fullsized":"2023-09-30T14:37:13.130616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:09:49.252823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.015937+0000","last_change":"2023-09-30T14:37:11.997282+0000","last_active":"2023-09-30T14:37:13.015937+0000","last_peered":"2023-09-30T14:37:13.015937+0000","last_clean":"2023-09-30T14:37:13.015937+0000","last_became_active":"2023-09-30T14:37:11.996937+0000","last_became_peered":"2023-09-30T14:37:11.996937+0000","last_unstale":"2023-09-30T14:37:13.015937+0000","last_undegraded":"2023-09-30T14:37:13.015937+0000","last_fullsized":"2023-09-30T14:37:13.015937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:39:27.351772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014196+0000","last_change":"2023-09-30T14:37:11.997408+0000","last_active":"2023-09-30T14:37:13.014196+0000","last_peered":"2023-09-30T14:37:13.014196+0000","last_clean":"2023-09-30T14:37:13.014196+0000","last_became_active":"2023-09-30T14:37:11.997147+0000","last_became_peered":"2023-09-30T14:37:11.997147+0000","last_unstale":"2023-09-30T14:37:13.014196+0000","last_undegraded":"2023-09-30T14:37:13.014196+0000","last_fullsized":"2023-09-30T14:37:13.014196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:21:24.721417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975636+0000","last_change":"2023-09-30T14:37:11.986172+0000","last_active":"2023-09-30T14:37:12.975636+0000","last_peered":"2023-09-30T14:37:12.975636+0000","last_clean":"2023-09-30T14:37:12.975636+0000","last_became_active":"2023-09-30T14:37:11.985836+0000","last_became_peered":"2023-09-30T14:37:11.985836+0000","last_unstale":"2023-09-30T14:37:12.975636+0000","last_undegraded":"2023-09-30T14:37:12.975636+0000","last_fullsized":"2023-09-30T14:37:12.975636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:35:57.464679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976406+0000","last_change":"2023-09-30T14:37:11.990853+0000","last_active":"2023-09-30T14:37:12.976406+0000","last_peered":"2023-09-30T14:37:12.976406+0000","last_clean":"2023-09-30T14:37:12.976406+0000","last_became_active":"2023-09-30T14:37:11.990321+0000","last_became_peered":"2023-09-30T14:37:11.990321+0000","last_unstale":"2023-09-30T14:37:12.976406+0000","last_undegraded":"2023-09-30T14:37:12.976406+0000","last_fullsized":"2023-09-30T14:37:12.976406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:32:11.409927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.016101+0000","last_change":"2023-09-30T14:37:11.992094+0000","last_active":"2023-09-30T14:37:13.016101+0000","last_peered":"2023-09-30T14:37:13.016101+0000","last_clean":"2023-09-30T14:37:13.016101+0000","last_became_active":"2023-09-30T14:37:11.991762+0000","last_became_peered":"2023-09-30T14:37:11.991762+0000","last_unstale":"2023-09-30T14:37:13.016101+0000","last_undegraded":"2023-09-30T14:37:13.016101+0000","last_fullsized":"2023-09-30T14:37:13.016101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:22:50.203788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014249+0000","last_change":"2023-09-30T14:37:11.992213+0000","last_active":"2023-09-30T14:37:13.014249+0000","last_peered":"2023-09-30T14:37:13.014249+0000","last_clean":"2023-09-30T14:37:13.014249+0000","last_became_active":"2023-09-30T14:37:11.991868+0000","last_became_peered":"2023-09-30T14:37:11.991868+0000","last_unstale":"2023-09-30T14:37:13.014249+0000","last_undegraded":"2023-09-30T14:37:13.014249+0000","last_fullsized":"2023-09-30T14:37:13.014249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:20:06.061978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014693+0000","last_change":"2023-09-30T14:37:11.990043+0000","last_active":"2023-09-30T14:37:13.014693+0000","last_peered":"2023-09-30T14:37:13.014693+0000","last_clean":"2023-09-30T14:37:13.014693+0000","last_became_active":"2023-09-30T14:37:11.988243+0000","last_became_peered":"2023-09-30T14:37:11.988243+0000","last_unstale":"2023-09-30T14:37:13.014693+0000","last_undegraded":"2023-09-30T14:37:13.014693+0000","last_fullsized":"2023-09-30T14:37:13.014693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:17:58.946159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014266+0000","last_change":"2023-09-30T14:37:11.990361+0000","last_active":"2023-09-30T14:37:13.014266+0000","last_peered":"2023-09-30T14:37:13.014266+0000","last_clean":"2023-09-30T14:37:13.014266+0000","last_became_active":"2023-09-30T14:37:11.990080+0000","last_became_peered":"2023-09-30T14:37:11.990080+0000","last_unstale":"2023-09-30T14:37:13.014266+0000","last_undegraded":"2023-09-30T14:37:13.014266+0000","last_fullsized":"2023-09-30T14:37:13.014266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:27:08.830419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014485+0000","last_change":"2023-09-30T14:37:11.988119+0000","last_active":"2023-09-30T14:37:13.014485+0000","last_peered":"2023-09-30T14:37:13.014485+0000","last_clean":"2023-09-30T14:37:13.014485+0000","last_became_active":"2023-09-30T14:37:11.987704+0000","last_became_peered":"2023-09-30T14:37:11.987704+0000","last_unstale":"2023-09-30T14:37:13.014485+0000","last_undegraded":"2023-09-30T14:37:13.014485+0000","last_fullsized":"2023-09-30T14:37:13.014485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:47:23.318982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.013931+0000","last_change":"2023-09-30T14:37:11.985403+0000","last_active":"2023-09-30T14:37:13.013931+0000","last_peered":"2023-09-30T14:37:13.013931+0000","last_clean":"2023-09-30T14:37:13.013931+0000","last_became_active":"2023-09-30T14:37:11.984818+0000","last_became_peered":"2023-09-30T14:37:11.984818+0000","last_unstale":"2023-09-30T14:37:13.013931+0000","last_undegraded":"2023-09-30T14:37:13.013931+0000","last_fullsized":"2023-09-30T14:37:13.013931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:10:43.592505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976073+0000","last_change":"2023-09-30T14:37:11.989500+0000","last_active":"2023-09-30T14:37:12.976073+0000","last_peered":"2023-09-30T14:37:12.976073+0000","last_clean":"2023-09-30T14:37:12.976073+0000","last_became_active":"2023-09-30T14:37:11.989186+0000","last_became_peered":"2023-09-30T14:37:11.989186+0000","last_unstale":"2023-09-30T14:37:12.976073+0000","last_undegraded":"2023-09-30T14:37:12.976073+0000","last_fullsized":"2023-09-30T14:37:12.976073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:14:35.497319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014676+0000","last_change":"2023-09-30T14:37:11.987217+0000","last_active":"2023-09-30T14:37:13.014676+0000","last_peered":"2023-09-30T14:37:13.014676+0000","last_clean":"2023-09-30T14:37:13.014676+0000","last_became_active":"2023-09-30T14:37:11.986835+0000","last_became_peered":"2023-09-30T14:37:11.986835+0000","last_unstale":"2023-09-30T14:37:13.014676+0000","last_undegraded":"2023-09-30T14:37:13.014676+0000","last_fullsized":"2023-09-30T14:37:13.014676+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:11:19.234768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014440+0000","last_change":"2023-09-30T14:37:11.995224+0000","last_active":"2023-09-30T14:37:13.014440+0000","last_peered":"2023-09-30T14:37:13.014440+0000","last_clean":"2023-09-30T14:37:13.014440+0000","last_became_active":"2023-09-30T14:37:11.994925+0000","last_became_peered":"2023-09-30T14:37:11.994925+0000","last_unstale":"2023-09-30T14:37:13.014440+0000","last_undegraded":"2023-09-30T14:37:13.014440+0000","last_fullsized":"2023-09-30T14:37:13.014440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:08:07.453499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975873+0000","last_change":"2023-09-30T14:37:11.990182+0000","last_active":"2023-09-30T14:37:12.975873+0000","last_peered":"2023-09-30T14:37:12.975873+0000","last_clean":"2023-09-30T14:37:12.975873+0000","last_became_active":"2023-09-30T14:37:11.989862+0000","last_became_peered":"2023-09-30T14:37:11.989862+0000","last_unstale":"2023-09-30T14:37:12.975873+0000","last_undegraded":"2023-09-30T14:37:12.975873+0000","last_fullsized":"2023-09-30T14:37:12.975873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:15:31.713002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.076239+0000","last_change":"2023-09-30T14:37:11.028123+0000","last_active":"2023-09-30T14:37:16.076239+0000","last_peered":"2023-09-30T14:37:16.076239+0000","last_clean":"2023-09-30T14:37:16.076239+0000","last_became_active":"2023-09-30T14:37:11.027764+0000","last_became_peered":"2023-09-30T14:37:11.027764+0000","last_unstale":"2023-09-30T14:37:16.076239+0000","last_undegraded":"2023-09-30T14:37:16.076239+0000","last_fullsized":"2023-09-30T14:37:16.076239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:15:50.818783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.977049+0000","last_change":"2023-09-30T14:37:11.994584+0000","last_active":"2023-09-30T14:37:12.977049+0000","last_peered":"2023-09-30T14:37:12.977049+0000","last_clean":"2023-09-30T14:37:12.977049+0000","last_became_active":"2023-09-30T14:37:11.992991+0000","last_became_peered":"2023-09-30T14:37:11.992991+0000","last_unstale":"2023-09-30T14:37:12.977049+0000","last_undegraded":"2023-09-30T14:37:12.977049+0000","last_fullsized":"2023-09-30T14:37:12.977049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:39:50.995717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.977224+0000","last_change":"2023-09-30T14:37:11.020653+0000","last_active":"2023-09-30T14:37:12.977224+0000","last_peered":"2023-09-30T14:37:12.977224+0000","last_clean":"2023-09-30T14:37:12.977224+0000","last_became_active":"2023-09-30T14:37:11.017621+0000","last_became_peered":"2023-09-30T14:37:11.017621+0000","last_unstale":"2023-09-30T14:37:12.977224+0000","last_undegraded":"2023-09-30T14:37:12.977224+0000","last_fullsized":"2023-09-30T14:37:12.977224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:26:59.395278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014548+0000","last_change":"2023-09-30T14:37:11.999881+0000","last_active":"2023-09-30T14:37:13.014548+0000","last_peered":"2023-09-30T14:37:13.014548+0000","last_clean":"2023-09-30T14:37:13.014548+0000","last_became_active":"2023-09-30T14:37:11.999614+0000","last_became_peered":"2023-09-30T14:37:11.999614+0000","last_unstale":"2023-09-30T14:37:13.014548+0000","last_undegraded":"2023-09-30T14:37:13.014548+0000","last_fullsized":"2023-09-30T14:37:13.014548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:59:44.198361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.071987+0000","last_change":"2023-09-30T14:37:11.027187+0000","last_active":"2023-09-30T14:37:16.071987+0000","last_peered":"2023-09-30T14:37:16.071987+0000","last_clean":"2023-09-30T14:37:16.071987+0000","last_became_active":"2023-09-30T14:37:11.026882+0000","last_became_peered":"2023-09-30T14:37:11.026882+0000","last_unstale":"2023-09-30T14:37:16.071987+0000","last_undegraded":"2023-09-30T14:37:16.071987+0000","last_fullsized":"2023-09-30T14:37:16.071987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:17:11.824002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976283+0000","last_change":"2023-09-30T14:37:11.986979+0000","last_active":"2023-09-30T14:37:12.976283+0000","last_peered":"2023-09-30T14:37:12.976283+0000","last_clean":"2023-09-30T14:37:12.976283+0000","last_became_active":"2023-09-30T14:37:11.986705+0000","last_became_peered":"2023-09-30T14:37:11.986705+0000","last_unstale":"2023-09-30T14:37:12.976283+0000","last_undegraded":"2023-09-30T14:37:12.976283+0000","last_fullsized":"2023-09-30T14:37:12.976283+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:31:40.319747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.395336+0000","last_change":"2023-09-30T14:37:11.022313+0000","last_active":"2023-09-30T14:37:12.395336+0000","last_peered":"2023-09-30T14:37:12.395336+0000","last_clean":"2023-09-30T14:37:12.395336+0000","last_became_active":"2023-09-30T14:37:11.021949+0000","last_became_peered":"2023-09-30T14:37:11.021949+0000","last_unstale":"2023-09-30T14:37:12.395336+0000","last_undegraded":"2023-09-30T14:37:12.395336+0000","last_fullsized":"2023-09-30T14:37:12.395336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:30:26.913666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014721+0000","last_change":"2023-09-30T14:37:11.988966+0000","last_active":"2023-09-30T14:37:13.014721+0000","last_peered":"2023-09-30T14:37:13.014721+0000","last_clean":"2023-09-30T14:37:13.014721+0000","last_became_active":"2023-09-30T14:37:11.988716+0000","last_became_peered":"2023-09-30T14:37:11.988716+0000","last_unstale":"2023-09-30T14:37:13.014721+0000","last_undegraded":"2023-09-30T14:37:13.014721+0000","last_fullsized":"2023-09-30T14:37:13.014721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:25:31.437584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":61,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:56.042046+0000","last_change":"2023-09-30T14:37:11.025855+0000","last_active":"2023-09-30T14:37:56.042046+0000","last_peered":"2023-09-30T14:37:56.042046+0000","last_clean":"2023-09-30T14:37:56.042046+0000","last_became_active":"2023-09-30T14:37:11.025402+0000","last_became_peered":"2023-09-30T14:37:11.025402+0000","last_unstale":"2023-09-30T14:37:56.042046+0000","last_undegraded":"2023-09-30T14:37:56.042046+0000","last_fullsized":"2023-09-30T14:37:56.042046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:50:37.692533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130846+0000","last_change":"2023-09-30T14:37:11.993074+0000","last_active":"2023-09-30T14:37:13.130846+0000","last_peered":"2023-09-30T14:37:13.130846+0000","last_clean":"2023-09-30T14:37:13.130846+0000","last_became_active":"2023-09-30T14:37:11.992784+0000","last_became_peered":"2023-09-30T14:37:11.992784+0000","last_unstale":"2023-09-30T14:37:13.130846+0000","last_undegraded":"2023-09-30T14:37:13.130846+0000","last_fullsized":"2023-09-30T14:37:13.130846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:28:18.454389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.073464+0000","last_change":"2023-09-30T14:37:11.022677+0000","last_active":"2023-09-30T14:37:16.073464+0000","last_peered":"2023-09-30T14:37:16.073464+0000","last_clean":"2023-09-30T14:37:16.073464+0000","last_became_active":"2023-09-30T14:37:11.022357+0000","last_became_peered":"2023-09-30T14:37:11.022357+0000","last_unstale":"2023-09-30T14:37:16.073464+0000","last_undegraded":"2023-09-30T14:37:16.073464+0000","last_fullsized":"2023-09-30T14:37:16.073464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:00:15.500968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014764+0000","last_change":"2023-09-30T14:37:11.992358+0000","last_active":"2023-09-30T14:37:13.014764+0000","last_peered":"2023-09-30T14:37:13.014764+0000","last_clean":"2023-09-30T14:37:13.014764+0000","last_became_active":"2023-09-30T14:37:11.992094+0000","last_became_peered":"2023-09-30T14:37:11.992094+0000","last_unstale":"2023-09-30T14:37:13.014764+0000","last_undegraded":"2023-09-30T14:37:13.014764+0000","last_fullsized":"2023-09-30T14:37:13.014764+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:59:36.817138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.075956+0000","last_change":"2023-09-30T14:37:11.030961+0000","last_active":"2023-09-30T14:37:16.075956+0000","last_peered":"2023-09-30T14:37:16.075956+0000","last_clean":"2023-09-30T14:37:16.075956+0000","last_became_active":"2023-09-30T14:37:11.030662+0000","last_became_peered":"2023-09-30T14:37:11.030662+0000","last_unstale":"2023-09-30T14:37:16.075956+0000","last_undegraded":"2023-09-30T14:37:16.075956+0000","last_fullsized":"2023-09-30T14:37:16.075956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:35:13.313382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975724+0000","last_change":"2023-09-30T14:37:11.989039+0000","last_active":"2023-09-30T14:37:12.975724+0000","last_peered":"2023-09-30T14:37:12.975724+0000","last_clean":"2023-09-30T14:37:12.975724+0000","last_became_active":"2023-09-30T14:37:11.988594+0000","last_became_peered":"2023-09-30T14:37:11.988594+0000","last_unstale":"2023-09-30T14:37:12.975724+0000","last_undegraded":"2023-09-30T14:37:12.975724+0000","last_fullsized":"2023-09-30T14:37:12.975724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:52:58.146681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014581+0000","last_change":"2023-09-30T14:37:11.026545+0000","last_active":"2023-09-30T14:37:13.014581+0000","last_peered":"2023-09-30T14:37:13.014581+0000","last_clean":"2023-09-30T14:37:13.014581+0000","last_became_active":"2023-09-30T14:37:11.026063+0000","last_became_peered":"2023-09-30T14:37:11.026063+0000","last_unstale":"2023-09-30T14:37:13.014581+0000","last_undegraded":"2023-09-30T14:37:13.014581+0000","last_fullsized":"2023-09-30T14:37:13.014581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:15:34.020336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014356+0000","last_change":"2023-09-30T14:37:11.991845+0000","last_active":"2023-09-30T14:37:13.014356+0000","last_peered":"2023-09-30T14:37:13.014356+0000","last_clean":"2023-09-30T14:37:13.014356+0000","last_became_active":"2023-09-30T14:37:11.991560+0000","last_became_peered":"2023-09-30T14:37:11.991560+0000","last_unstale":"2023-09-30T14:37:13.014356+0000","last_undegraded":"2023-09-30T14:37:13.014356+0000","last_fullsized":"2023-09-30T14:37:13.014356+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:27:39.653882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":55,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:54.038504+0000","last_change":"2023-09-30T14:37:11.028192+0000","last_active":"2023-09-30T14:37:54.038504+0000","last_peered":"2023-09-30T14:37:54.038504+0000","last_clean":"2023-09-30T14:37:54.038504+0000","last_became_active":"2023-09-30T14:37:11.027902+0000","last_became_peered":"2023-09-30T14:37:11.027902+0000","last_unstale":"2023-09-30T14:37:54.038504+0000","last_undegraded":"2023-09-30T14:37:54.038504+0000","last_fullsized":"2023-09-30T14:37:54.038504+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:21:30.267559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976420+0000","last_change":"2023-09-30T14:37:11.997318+0000","last_active":"2023-09-30T14:37:12.976420+0000","last_peered":"2023-09-30T14:37:12.976420+0000","last_clean":"2023-09-30T14:37:12.976420+0000","last_became_active":"2023-09-30T14:37:11.996988+0000","last_became_peered":"2023-09-30T14:37:11.996988+0000","last_unstale":"2023-09-30T14:37:12.976420+0000","last_undegraded":"2023-09-30T14:37:12.976420+0000","last_fullsized":"2023-09-30T14:37:12.976420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:57:21.510952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063345+0000","last_change":"2023-09-30T14:37:11.023686+0000","last_active":"2023-09-30T14:37:15.063345+0000","last_peered":"2023-09-30T14:37:15.063345+0000","last_clean":"2023-09-30T14:37:15.063345+0000","last_became_active":"2023-09-30T14:37:11.023365+0000","last_became_peered":"2023-09-30T14:37:11.023365+0000","last_unstale":"2023-09-30T14:37:15.063345+0000","last_undegraded":"2023-09-30T14:37:15.063345+0000","last_fullsized":"2023-09-30T14:37:15.063345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:00:04.358289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014711+0000","last_change":"2023-09-30T14:37:11.989338+0000","last_active":"2023-09-30T14:37:13.014711+0000","last_peered":"2023-09-30T14:37:13.014711+0000","last_clean":"2023-09-30T14:37:13.014711+0000","last_became_active":"2023-09-30T14:37:11.989045+0000","last_became_peered":"2023-09-30T14:37:11.989045+0000","last_unstale":"2023-09-30T14:37:13.014711+0000","last_undegraded":"2023-09-30T14:37:13.014711+0000","last_fullsized":"2023-09-30T14:37:13.014711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:58:59.359684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063525+0000","last_change":"2023-09-30T14:37:11.024709+0000","last_active":"2023-09-30T14:37:15.063525+0000","last_peered":"2023-09-30T14:37:15.063525+0000","last_clean":"2023-09-30T14:37:15.063525+0000","last_became_active":"2023-09-30T14:37:11.024344+0000","last_became_peered":"2023-09-30T14:37:11.024344+0000","last_unstale":"2023-09-30T14:37:15.063525+0000","last_undegraded":"2023-09-30T14:37:15.063525+0000","last_fullsized":"2023-09-30T14:37:15.063525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:17:49.508396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976335+0000","last_change":"2023-09-30T14:37:11.989797+0000","last_active":"2023-09-30T14:37:12.976335+0000","last_peered":"2023-09-30T14:37:12.976335+0000","last_clean":"2023-09-30T14:37:12.976335+0000","last_became_active":"2023-09-30T14:37:11.989482+0000","last_became_peered":"2023-09-30T14:37:11.989482+0000","last_unstale":"2023-09-30T14:37:12.976335+0000","last_undegraded":"2023-09-30T14:37:12.976335+0000","last_fullsized":"2023-09-30T14:37:12.976335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:33:20.437706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'26","reported_seq":124,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:55.046663+0000","last_change":"2023-09-30T14:37:11.027322+0000","last_active":"2023-09-30T14:37:55.046663+0000","last_peered":"2023-09-30T14:37:55.046663+0000","last_clean":"2023-09-30T14:37:55.046663+0000","last_became_active":"2023-09-30T14:37:11.027007+0000","last_became_peered":"2023-09-30T14:37:11.027007+0000","last_unstale":"2023-09-30T14:37:55.046663+0000","last_undegraded":"2023-09-30T14:37:55.046663+0000","last_fullsized":"2023-09-30T14:37:55.046663+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:47:22.919317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54708,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":85,"num_read_kb":95,"num_write":27,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130730+0000","last_change":"2023-09-30T14:37:11.988469+0000","last_active":"2023-09-30T14:37:13.130730+0000","last_peered":"2023-09-30T14:37:13.130730+0000","last_clean":"2023-09-30T14:37:13.130730+0000","last_became_active":"2023-09-30T14:37:11.988118+0000","last_became_peered":"2023-09-30T14:37:11.988118+0000","last_unstale":"2023-09-30T14:37:13.130730+0000","last_undegraded":"2023-09-30T14:37:13.130730+0000","last_fullsized":"2023-09-30T14:37:13.130730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:11:56.750690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.072486+0000","last_change":"2023-09-30T14:37:11.024345+0000","last_active":"2023-09-30T14:37:16.072486+0000","last_peered":"2023-09-30T14:37:16.072486+0000","last_clean":"2023-09-30T14:37:16.072486+0000","last_became_active":"2023-09-30T14:37:11.024039+0000","last_became_peered":"2023-09-30T14:37:11.024039+0000","last_unstale":"2023-09-30T14:37:16.072486+0000","last_undegraded":"2023-09-30T14:37:16.072486+0000","last_fullsized":"2023-09-30T14:37:16.072486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:48:32.872814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014748+0000","last_change":"2023-09-30T14:37:11.994068+0000","last_active":"2023-09-30T14:37:13.014748+0000","last_peered":"2023-09-30T14:37:13.014748+0000","last_clean":"2023-09-30T14:37:13.014748+0000","last_became_active":"2023-09-30T14:37:11.993793+0000","last_became_peered":"2023-09-30T14:37:11.993793+0000","last_unstale":"2023-09-30T14:37:13.014748+0000","last_undegraded":"2023-09-30T14:37:13.014748+0000","last_fullsized":"2023-09-30T14:37:13.014748+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:51:28.071414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063744+0000","last_change":"2023-09-30T14:37:11.025622+0000","last_active":"2023-09-30T14:37:15.063744+0000","last_peered":"2023-09-30T14:37:15.063744+0000","last_clean":"2023-09-30T14:37:15.063744+0000","last_became_active":"2023-09-30T14:37:11.025338+0000","last_became_peered":"2023-09-30T14:37:11.025338+0000","last_unstale":"2023-09-30T14:37:15.063744+0000","last_undegraded":"2023-09-30T14:37:15.063744+0000","last_fullsized":"2023-09-30T14:37:15.063744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:32:03.166296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.015550+0000","last_change":"2023-09-30T14:37:11.990678+0000","last_active":"2023-09-30T14:37:13.015550+0000","last_peered":"2023-09-30T14:37:13.015550+0000","last_clean":"2023-09-30T14:37:13.015550+0000","last_became_active":"2023-09-30T14:37:11.990331+0000","last_became_peered":"2023-09-30T14:37:11.990331+0000","last_unstale":"2023-09-30T14:37:13.015550+0000","last_undegraded":"2023-09-30T14:37:13.015550+0000","last_fullsized":"2023-09-30T14:37:13.015550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:57:06.296415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":59,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:56.041448+0000","last_change":"2023-09-30T14:37:11.029501+0000","last_active":"2023-09-30T14:37:56.041448+0000","last_peered":"2023-09-30T14:37:56.041448+0000","last_clean":"2023-09-30T14:37:56.041448+0000","last_became_active":"2023-09-30T14:37:11.029207+0000","last_became_peered":"2023-09-30T14:37:11.029207+0000","last_unstale":"2023-09-30T14:37:56.041448+0000","last_undegraded":"2023-09-30T14:37:56.041448+0000","last_fullsized":"2023-09-30T14:37:56.041448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:48:53.145952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014366+0000","last_change":"2023-09-30T14:37:11.993052+0000","last_active":"2023-09-30T14:37:13.014366+0000","last_peered":"2023-09-30T14:37:13.014366+0000","last_clean":"2023-09-30T14:37:13.014366+0000","last_became_active":"2023-09-30T14:37:11.992762+0000","last_became_peered":"2023-09-30T14:37:11.992762+0000","last_unstale":"2023-09-30T14:37:13.014366+0000","last_undegraded":"2023-09-30T14:37:13.014366+0000","last_fullsized":"2023-09-30T14:37:13.014366+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:09:16.440026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.075318+0000","last_change":"2023-09-30T14:37:11.027266+0000","last_active":"2023-09-30T14:37:16.075318+0000","last_peered":"2023-09-30T14:37:16.075318+0000","last_clean":"2023-09-30T14:37:16.075318+0000","last_became_active":"2023-09-30T14:37:11.026971+0000","last_became_peered":"2023-09-30T14:37:11.026971+0000","last_unstale":"2023-09-30T14:37:16.075318+0000","last_undegraded":"2023-09-30T14:37:16.075318+0000","last_fullsized":"2023-09-30T14:37:16.075318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:38:09.543391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014408+0000","last_change":"2023-09-30T14:37:11.989592+0000","last_active":"2023-09-30T14:37:13.014408+0000","last_peered":"2023-09-30T14:37:13.014408+0000","last_clean":"2023-09-30T14:37:13.014408+0000","last_became_active":"2023-09-30T14:37:11.989290+0000","last_became_peered":"2023-09-30T14:37:11.989290+0000","last_unstale":"2023-09-30T14:37:13.014408+0000","last_undegraded":"2023-09-30T14:37:13.014408+0000","last_fullsized":"2023-09-30T14:37:13.014408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:11:33.974823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.980060+0000","last_change":"2023-09-30T14:37:11.028919+0000","last_active":"2023-09-30T14:37:16.980060+0000","last_peered":"2023-09-30T14:37:16.980060+0000","last_clean":"2023-09-30T14:37:16.980060+0000","last_became_active":"2023-09-30T14:37:11.028532+0000","last_became_peered":"2023-09-30T14:37:11.028532+0000","last_unstale":"2023-09-30T14:37:16.980060+0000","last_undegraded":"2023-09-30T14:37:16.980060+0000","last_fullsized":"2023-09-30T14:37:16.980060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:15:42.675409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014483+0000","last_change":"2023-09-30T14:37:11.980611+0000","last_active":"2023-09-30T14:37:13.014483+0000","last_peered":"2023-09-30T14:37:13.014483+0000","last_clean":"2023-09-30T14:37:13.014483+0000","last_became_active":"2023-09-30T14:37:11.980288+0000","last_became_peered":"2023-09-30T14:37:11.980288+0000","last_unstale":"2023-09-30T14:37:13.014483+0000","last_undegraded":"2023-09-30T14:37:13.014483+0000","last_fullsized":"2023-09-30T14:37:13.014483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:38:49.270976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014852+0000","last_change":"2023-09-30T14:37:11.025133+0000","last_active":"2023-09-30T14:37:13.014852+0000","last_peered":"2023-09-30T14:37:13.014852+0000","last_clean":"2023-09-30T14:37:13.014852+0000","last_became_active":"2023-09-30T14:37:11.024914+0000","last_became_peered":"2023-09-30T14:37:11.024914+0000","last_unstale":"2023-09-30T14:37:13.014852+0000","last_undegraded":"2023-09-30T14:37:13.014852+0000","last_fullsized":"2023-09-30T14:37:13.014852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:23:39.328773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.286974+0000","last_change":"2023-09-30T14:37:11.988434+0000","last_active":"2023-09-30T14:37:15.286974+0000","last_peered":"2023-09-30T14:37:15.286974+0000","last_clean":"2023-09-30T14:37:15.286974+0000","last_became_active":"2023-09-30T14:37:11.988156+0000","last_became_peered":"2023-09-30T14:37:11.988156+0000","last_unstale":"2023-09-30T14:37:15.286974+0000","last_undegraded":"2023-09-30T14:37:15.286974+0000","last_fullsized":"2023-09-30T14:37:15.286974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:20:49.420026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014160+0000","last_change":"2023-09-30T14:37:11.026290+0000","last_active":"2023-09-30T14:37:13.014160+0000","last_peered":"2023-09-30T14:37:13.014160+0000","last_clean":"2023-09-30T14:37:13.014160+0000","last_became_active":"2023-09-30T14:37:11.024684+0000","last_became_peered":"2023-09-30T14:37:11.024684+0000","last_unstale":"2023-09-30T14:37:13.014160+0000","last_undegraded":"2023-09-30T14:37:13.014160+0000","last_fullsized":"2023-09-30T14:37:13.014160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:43:06.463243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.016048+0000","last_change":"2023-09-30T14:37:11.987989+0000","last_active":"2023-09-30T14:37:13.016048+0000","last_peered":"2023-09-30T14:37:13.016048+0000","last_clean":"2023-09-30T14:37:13.016048+0000","last_became_active":"2023-09-30T14:37:11.987690+0000","last_became_peered":"2023-09-30T14:37:11.987690+0000","last_unstale":"2023-09-30T14:37:13.016048+0000","last_undegraded":"2023-09-30T14:37:13.016048+0000","last_fullsized":"2023-09-30T14:37:13.016048+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:51:25.825871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014864+0000","last_change":"2023-09-30T14:37:11.996852+0000","last_active":"2023-09-30T14:37:13.014864+0000","last_peered":"2023-09-30T14:37:13.014864+0000","last_clean":"2023-09-30T14:37:13.014864+0000","last_became_active":"2023-09-30T14:37:11.996596+0000","last_became_peered":"2023-09-30T14:37:11.996596+0000","last_unstale":"2023-09-30T14:37:13.014864+0000","last_undegraded":"2023-09-30T14:37:13.014864+0000","last_fullsized":"2023-09-30T14:37:13.014864+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:25:04.531236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:54.037759+0000","last_change":"2023-09-30T14:37:11.025458+0000","last_active":"2023-09-30T14:37:54.037759+0000","last_peered":"2023-09-30T14:37:54.037759+0000","last_clean":"2023-09-30T14:37:54.037759+0000","last_became_active":"2023-09-30T14:37:11.025157+0000","last_became_peered":"2023-09-30T14:37:11.025157+0000","last_unstale":"2023-09-30T14:37:54.037759+0000","last_undegraded":"2023-09-30T14:37:54.037759+0000","last_fullsized":"2023-09-30T14:37:54.037759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:31:24.436125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976151+0000","last_change":"2023-09-30T14:37:11.986840+0000","last_active":"2023-09-30T14:37:12.976151+0000","last_peered":"2023-09-30T14:37:12.976151+0000","last_clean":"2023-09-30T14:37:12.976151+0000","last_became_active":"2023-09-30T14:37:11.986483+0000","last_became_peered":"2023-09-30T14:37:11.986483+0000","last_unstale":"2023-09-30T14:37:12.976151+0000","last_undegraded":"2023-09-30T14:37:12.976151+0000","last_fullsized":"2023-09-30T14:37:12.976151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:14:32.399685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.979424+0000","last_change":"2023-09-30T14:37:11.025706+0000","last_active":"2023-09-30T14:37:16.979424+0000","last_peered":"2023-09-30T14:37:16.979424+0000","last_clean":"2023-09-30T14:37:16.979424+0000","last_became_active":"2023-09-30T14:37:11.025376+0000","last_became_peered":"2023-09-30T14:37:11.025376+0000","last_unstale":"2023-09-30T14:37:16.979424+0000","last_undegraded":"2023-09-30T14:37:16.979424+0000","last_fullsized":"2023-09-30T14:37:16.979424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:10:36.025273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":308,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:11.977686+0000","last_change":"2023-09-30T14:36:12.344762+0000","last_active":"2023-09-30T14:37:11.977686+0000","last_peered":"2023-09-30T14:37:11.977686+0000","last_clean":"2023-09-30T14:37:11.977686+0000","last_became_active":"2023-09-30T14:36:12.344268+0000","last_became_peered":"2023-09-30T14:36:12.344268+0000","last_unstale":"2023-09-30T14:37:11.977686+0000","last_undegraded":"2023-09-30T14:37:11.977686+0000","last_fullsized":"2023-09-30T14:37:11.977686+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:33:20.434992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:33:20.434992+0000","last_clean_scrub_stamp":"2023-09-30T14:33:20.434992+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:00:05.598961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014131+0000","last_change":"2023-09-30T14:37:11.996886+0000","last_active":"2023-09-30T14:37:13.014131+0000","last_peered":"2023-09-30T14:37:13.014131+0000","last_clean":"2023-09-30T14:37:13.014131+0000","last_became_active":"2023-09-30T14:37:11.996667+0000","last_became_peered":"2023-09-30T14:37:11.996667+0000","last_unstale":"2023-09-30T14:37:13.014131+0000","last_undegraded":"2023-09-30T14:37:13.014131+0000","last_fullsized":"2023-09-30T14:37:13.014131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:27:00.546356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.074562+0000","last_change":"2023-09-30T14:37:11.027012+0000","last_active":"2023-09-30T14:37:16.074562+0000","last_peered":"2023-09-30T14:37:16.074562+0000","last_clean":"2023-09-30T14:37:16.074562+0000","last_became_active":"2023-09-30T14:37:11.026714+0000","last_became_peered":"2023-09-30T14:37:11.026714+0000","last_unstale":"2023-09-30T14:37:16.074562+0000","last_undegraded":"2023-09-30T14:37:16.074562+0000","last_fullsized":"2023-09-30T14:37:16.074562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:17:57.860438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.062444+0000","last_change":"2023-09-30T14:37:11.027389+0000","last_active":"2023-09-30T14:37:15.062444+0000","last_peered":"2023-09-30T14:37:15.062444+0000","last_clean":"2023-09-30T14:37:15.062444+0000","last_became_active":"2023-09-30T14:37:11.027011+0000","last_became_peered":"2023-09-30T14:37:11.027011+0000","last_unstale":"2023-09-30T14:37:15.062444+0000","last_undegraded":"2023-09-30T14:37:15.062444+0000","last_fullsized":"2023-09-30T14:37:15.062444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:38:22.953936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014762+0000","last_change":"2023-09-30T14:37:11.997675+0000","last_active":"2023-09-30T14:37:13.014762+0000","last_peered":"2023-09-30T14:37:13.014762+0000","last_clean":"2023-09-30T14:37:13.014762+0000","last_became_active":"2023-09-30T14:37:11.997330+0000","last_became_peered":"2023-09-30T14:37:11.997330+0000","last_unstale":"2023-09-30T14:37:13.014762+0000","last_undegraded":"2023-09-30T14:37:13.014762+0000","last_fullsized":"2023-09-30T14:37:13.014762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:54:52.802138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:20.829473+0000","last_change":"2023-09-30T14:37:11.023836+0000","last_active":"2023-09-30T14:37:20.829473+0000","last_peered":"2023-09-30T14:37:20.829473+0000","last_clean":"2023-09-30T14:37:20.829473+0000","last_became_active":"2023-09-30T14:37:11.023362+0000","last_became_peered":"2023-09-30T14:37:11.023362+0000","last_unstale":"2023-09-30T14:37:20.829473+0000","last_undegraded":"2023-09-30T14:37:20.829473+0000","last_fullsized":"2023-09-30T14:37:20.829473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:58:22.772546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976655+0000","last_change":"2023-09-30T14:37:11.985281+0000","last_active":"2023-09-30T14:37:12.976655+0000","last_peered":"2023-09-30T14:37:12.976655+0000","last_clean":"2023-09-30T14:37:12.976655+0000","last_became_active":"2023-09-30T14:37:11.984944+0000","last_became_peered":"2023-09-30T14:37:11.984944+0000","last_unstale":"2023-09-30T14:37:12.976655+0000","last_undegraded":"2023-09-30T14:37:12.976655+0000","last_fullsized":"2023-09-30T14:37:12.976655+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:30:19.829727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.061631+0000","last_change":"2023-09-30T14:37:11.028070+0000","last_active":"2023-09-30T14:37:15.061631+0000","last_peered":"2023-09-30T14:37:15.061631+0000","last_clean":"2023-09-30T14:37:15.061631+0000","last_became_active":"2023-09-30T14:37:11.027703+0000","last_became_peered":"2023-09-30T14:37:11.027703+0000","last_unstale":"2023-09-30T14:37:15.061631+0000","last_undegraded":"2023-09-30T14:37:15.061631+0000","last_fullsized":"2023-09-30T14:37:15.061631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:23:15.071946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014359+0000","last_change":"2023-09-30T14:37:11.989897+0000","last_active":"2023-09-30T14:37:13.014359+0000","last_peered":"2023-09-30T14:37:13.014359+0000","last_clean":"2023-09-30T14:37:13.014359+0000","last_became_active":"2023-09-30T14:37:11.989613+0000","last_became_peered":"2023-09-30T14:37:11.989613+0000","last_unstale":"2023-09-30T14:37:13.014359+0000","last_undegraded":"2023-09-30T14:37:13.014359+0000","last_fullsized":"2023-09-30T14:37:13.014359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:43:17.887766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.062793+0000","last_change":"2023-09-30T14:37:11.026491+0000","last_active":"2023-09-30T14:37:15.062793+0000","last_peered":"2023-09-30T14:37:15.062793+0000","last_clean":"2023-09-30T14:37:15.062793+0000","last_became_active":"2023-09-30T14:37:11.026196+0000","last_became_peered":"2023-09-30T14:37:11.026196+0000","last_unstale":"2023-09-30T14:37:15.062793+0000","last_undegraded":"2023-09-30T14:37:15.062793+0000","last_fullsized":"2023-09-30T14:37:15.062793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:17:22.343377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.130535+0000","last_change":"2023-09-30T14:37:11.988570+0000","last_active":"2023-09-30T14:37:13.130535+0000","last_peered":"2023-09-30T14:37:13.130535+0000","last_clean":"2023-09-30T14:37:13.130535+0000","last_became_active":"2023-09-30T14:37:11.988230+0000","last_became_peered":"2023-09-30T14:37:11.988230+0000","last_unstale":"2023-09-30T14:37:13.130535+0000","last_undegraded":"2023-09-30T14:37:13.130535+0000","last_fullsized":"2023-09-30T14:37:13.130535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:58:38.302343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.062480+0000","last_change":"2023-09-30T14:37:11.024587+0000","last_active":"2023-09-30T14:37:15.062480+0000","last_peered":"2023-09-30T14:37:15.062480+0000","last_clean":"2023-09-30T14:37:15.062480+0000","last_became_active":"2023-09-30T14:37:11.024253+0000","last_became_peered":"2023-09-30T14:37:11.024253+0000","last_unstale":"2023-09-30T14:37:15.062480+0000","last_undegraded":"2023-09-30T14:37:15.062480+0000","last_fullsized":"2023-09-30T14:37:15.062480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:31:07.293657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014239+0000","last_change":"2023-09-30T14:37:11.987818+0000","last_active":"2023-09-30T14:37:13.014239+0000","last_peered":"2023-09-30T14:37:13.014239+0000","last_clean":"2023-09-30T14:37:13.014239+0000","last_became_active":"2023-09-30T14:37:11.987476+0000","last_became_peered":"2023-09-30T14:37:11.987476+0000","last_unstale":"2023-09-30T14:37:13.014239+0000","last_undegraded":"2023-09-30T14:37:13.014239+0000","last_fullsized":"2023-09-30T14:37:13.014239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:01:07.802959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:14.444569+0000","last_change":"2023-09-30T14:37:11.023223+0000","last_active":"2023-09-30T14:37:14.444569+0000","last_peered":"2023-09-30T14:37:14.444569+0000","last_clean":"2023-09-30T14:37:14.444569+0000","last_became_active":"2023-09-30T14:37:11.022932+0000","last_became_peered":"2023-09-30T14:37:11.022932+0000","last_unstale":"2023-09-30T14:37:14.444569+0000","last_undegraded":"2023-09-30T14:37:14.444569+0000","last_fullsized":"2023-09-30T14:37:14.444569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:01:04.315669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976068+0000","last_change":"2023-09-30T14:37:11.990323+0000","last_active":"2023-09-30T14:37:12.976068+0000","last_peered":"2023-09-30T14:37:12.976068+0000","last_clean":"2023-09-30T14:37:12.976068+0000","last_became_active":"2023-09-30T14:37:11.989981+0000","last_became_peered":"2023-09-30T14:37:11.989981+0000","last_unstale":"2023-09-30T14:37:12.976068+0000","last_undegraded":"2023-09-30T14:37:12.976068+0000","last_fullsized":"2023-09-30T14:37:12.976068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:30:44.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.063371+0000","last_change":"2023-09-30T14:37:11.025658+0000","last_active":"2023-09-30T14:37:15.063371+0000","last_peered":"2023-09-30T14:37:15.063371+0000","last_clean":"2023-09-30T14:37:15.063371+0000","last_became_active":"2023-09-30T14:37:11.025334+0000","last_became_peered":"2023-09-30T14:37:11.025334+0000","last_unstale":"2023-09-30T14:37:15.063371+0000","last_undegraded":"2023-09-30T14:37:15.063371+0000","last_fullsized":"2023-09-30T14:37:15.063371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:23:41.933634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014128+0000","last_change":"2023-09-30T14:37:11.989989+0000","last_active":"2023-09-30T14:37:13.014128+0000","last_peered":"2023-09-30T14:37:13.014128+0000","last_clean":"2023-09-30T14:37:13.014128+0000","last_became_active":"2023-09-30T14:37:11.989666+0000","last_became_peered":"2023-09-30T14:37:11.989666+0000","last_unstale":"2023-09-30T14:37:13.014128+0000","last_undegraded":"2023-09-30T14:37:13.014128+0000","last_fullsized":"2023-09-30T14:37:13.014128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:41:09.835210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:16.074595+0000","last_change":"2023-09-30T14:37:11.025027+0000","last_active":"2023-09-30T14:37:16.074595+0000","last_peered":"2023-09-30T14:37:16.074595+0000","last_clean":"2023-09-30T14:37:16.074595+0000","last_became_active":"2023-09-30T14:37:11.024696+0000","last_became_peered":"2023-09-30T14:37:11.024696+0000","last_unstale":"2023-09-30T14:37:16.074595+0000","last_undegraded":"2023-09-30T14:37:16.074595+0000","last_fullsized":"2023-09-30T14:37:16.074595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:34:28.384535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014390+0000","last_change":"2023-09-30T14:37:11.992674+0000","last_active":"2023-09-30T14:37:13.014390+0000","last_peered":"2023-09-30T14:37:13.014390+0000","last_clean":"2023-09-30T14:37:13.014390+0000","last_became_active":"2023-09-30T14:37:11.992243+0000","last_became_peered":"2023-09-30T14:37:11.992243+0000","last_unstale":"2023-09-30T14:37:13.014390+0000","last_undegraded":"2023-09-30T14:37:13.014390+0000","last_fullsized":"2023-09-30T14:37:13.014390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:33:19.759637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014643+0000","last_change":"2023-09-30T14:37:11.024525+0000","last_active":"2023-09-30T14:37:13.014643+0000","last_peered":"2023-09-30T14:37:13.014643+0000","last_clean":"2023-09-30T14:37:13.014643+0000","last_became_active":"2023-09-30T14:37:11.024231+0000","last_became_peered":"2023-09-30T14:37:11.024231+0000","last_unstale":"2023-09-30T14:37:13.014643+0000","last_undegraded":"2023-09-30T14:37:13.014643+0000","last_fullsized":"2023-09-30T14:37:13.014643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:53:47.142798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:15.512369+0000","last_change":"2023-09-30T14:37:11.991401+0000","last_active":"2023-09-30T14:37:15.512369+0000","last_peered":"2023-09-30T14:37:15.512369+0000","last_clean":"2023-09-30T14:37:15.512369+0000","last_became_active":"2023-09-30T14:37:11.991012+0000","last_became_peered":"2023-09-30T14:37:11.991012+0000","last_unstale":"2023-09-30T14:37:15.512369+0000","last_undegraded":"2023-09-30T14:37:15.512369+0000","last_fullsized":"2023-09-30T14:37:15.512369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:55:00.562030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.050302+0000","last_change":"2023-09-30T14:37:11.022893+0000","last_active":"2023-09-30T14:37:12.050302+0000","last_peered":"2023-09-30T14:37:12.050302+0000","last_clean":"2023-09-30T14:37:12.050302+0000","last_became_active":"2023-09-30T14:37:11.022505+0000","last_became_peered":"2023-09-30T14:37:11.022505+0000","last_unstale":"2023-09-30T14:37:12.050302+0000","last_undegraded":"2023-09-30T14:37:12.050302+0000","last_fullsized":"2023-09-30T14:37:12.050302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:00:17.721645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014348+0000","last_change":"2023-09-30T14:37:11.992942+0000","last_active":"2023-09-30T14:37:13.014348+0000","last_peered":"2023-09-30T14:37:13.014348+0000","last_clean":"2023-09-30T14:37:13.014348+0000","last_became_active":"2023-09-30T14:37:11.992698+0000","last_became_peered":"2023-09-30T14:37:11.992698+0000","last_unstale":"2023-09-30T14:37:13.014348+0000","last_undegraded":"2023-09-30T14:37:13.014348+0000","last_fullsized":"2023-09-30T14:37:13.014348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:15:30.239423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014256+0000","last_change":"2023-09-30T14:37:11.984435+0000","last_active":"2023-09-30T14:37:13.014256+0000","last_peered":"2023-09-30T14:37:13.014256+0000","last_clean":"2023-09-30T14:37:13.014256+0000","last_became_active":"2023-09-30T14:37:11.984137+0000","last_became_peered":"2023-09-30T14:37:11.984137+0000","last_unstale":"2023-09-30T14:37:13.014256+0000","last_undegraded":"2023-09-30T14:37:13.014256+0000","last_fullsized":"2023-09-30T14:37:13.014256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:40:41.112326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975815+0000","last_change":"2023-09-30T14:37:11.984481+0000","last_active":"2023-09-30T14:37:12.975815+0000","last_peered":"2023-09-30T14:37:12.975815+0000","last_clean":"2023-09-30T14:37:12.975815+0000","last_became_active":"2023-09-30T14:37:11.984021+0000","last_became_peered":"2023-09-30T14:37:11.984021+0000","last_unstale":"2023-09-30T14:37:12.975815+0000","last_undegraded":"2023-09-30T14:37:12.975815+0000","last_fullsized":"2023-09-30T14:37:12.975815+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:36:33.184379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975777+0000","last_change":"2023-09-30T14:37:11.990779+0000","last_active":"2023-09-30T14:37:12.975777+0000","last_peered":"2023-09-30T14:37:12.975777+0000","last_clean":"2023-09-30T14:37:12.975777+0000","last_became_active":"2023-09-30T14:37:11.989960+0000","last_became_peered":"2023-09-30T14:37:11.989960+0000","last_unstale":"2023-09-30T14:37:12.975777+0000","last_undegraded":"2023-09-30T14:37:12.975777+0000","last_fullsized":"2023-09-30T14:37:12.975777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:54:36.813191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.015049+0000","last_change":"2023-09-30T14:37:11.995750+0000","last_active":"2023-09-30T14:37:13.015049+0000","last_peered":"2023-09-30T14:37:13.015049+0000","last_clean":"2023-09-30T14:37:13.015049+0000","last_became_active":"2023-09-30T14:37:11.995434+0000","last_became_peered":"2023-09-30T14:37:11.995434+0000","last_unstale":"2023-09-30T14:37:13.015049+0000","last_undegraded":"2023-09-30T14:37:13.015049+0000","last_fullsized":"2023-09-30T14:37:13.015049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:49:51.234024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014630+0000","last_change":"2023-09-30T14:37:11.996550+0000","last_active":"2023-09-30T14:37:13.014630+0000","last_peered":"2023-09-30T14:37:13.014630+0000","last_clean":"2023-09-30T14:37:13.014630+0000","last_became_active":"2023-09-30T14:37:11.996269+0000","last_became_peered":"2023-09-30T14:37:11.996269+0000","last_unstale":"2023-09-30T14:37:13.014630+0000","last_undegraded":"2023-09-30T14:37:13.014630+0000","last_fullsized":"2023-09-30T14:37:13.014630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:40:16.537270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014992+0000","last_change":"2023-09-30T14:37:11.983495+0000","last_active":"2023-09-30T14:37:13.014992+0000","last_peered":"2023-09-30T14:37:13.014992+0000","last_clean":"2023-09-30T14:37:13.014992+0000","last_became_active":"2023-09-30T14:37:11.983241+0000","last_became_peered":"2023-09-30T14:37:11.983241+0000","last_unstale":"2023-09-30T14:37:13.014992+0000","last_undegraded":"2023-09-30T14:37:13.014992+0000","last_fullsized":"2023-09-30T14:37:13.014992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:41:46.604467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975888+0000","last_change":"2023-09-30T14:37:11.996058+0000","last_active":"2023-09-30T14:37:12.975888+0000","last_peered":"2023-09-30T14:37:12.975888+0000","last_clean":"2023-09-30T14:37:12.975888+0000","last_became_active":"2023-09-30T14:37:11.995763+0000","last_became_peered":"2023-09-30T14:37:11.995763+0000","last_unstale":"2023-09-30T14:37:12.975888+0000","last_undegraded":"2023-09-30T14:37:12.975888+0000","last_fullsized":"2023-09-30T14:37:12.975888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:42:21.306561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976498+0000","last_change":"2023-09-30T14:37:11.992080+0000","last_active":"2023-09-30T14:37:12.976498+0000","last_peered":"2023-09-30T14:37:12.976498+0000","last_clean":"2023-09-30T14:37:12.976498+0000","last_became_active":"2023-09-30T14:37:11.991747+0000","last_became_peered":"2023-09-30T14:37:11.991747+0000","last_unstale":"2023-09-30T14:37:12.976498+0000","last_undegraded":"2023-09-30T14:37:12.976498+0000","last_fullsized":"2023-09-30T14:37:12.976498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:07:25.985911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014724+0000","last_change":"2023-09-30T14:37:11.997109+0000","last_active":"2023-09-30T14:37:13.014724+0000","last_peered":"2023-09-30T14:37:13.014724+0000","last_clean":"2023-09-30T14:37:13.014724+0000","last_became_active":"2023-09-30T14:37:11.996793+0000","last_became_peered":"2023-09-30T14:37:11.996793+0000","last_unstale":"2023-09-30T14:37:13.014724+0000","last_undegraded":"2023-09-30T14:37:13.014724+0000","last_fullsized":"2023-09-30T14:37:13.014724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:13:59.838638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.975786+0000","last_change":"2023-09-30T14:37:11.984083+0000","last_active":"2023-09-30T14:37:12.975786+0000","last_peered":"2023-09-30T14:37:12.975786+0000","last_clean":"2023-09-30T14:37:12.975786+0000","last_became_active":"2023-09-30T14:37:11.983352+0000","last_became_peered":"2023-09-30T14:37:11.983352+0000","last_unstale":"2023-09-30T14:37:12.975786+0000","last_undegraded":"2023-09-30T14:37:12.975786+0000","last_fullsized":"2023-09-30T14:37:12.975786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:02:03.668484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:12.976194+0000","last_change":"2023-09-30T14:37:11.992212+0000","last_active":"2023-09-30T14:37:12.976194+0000","last_peered":"2023-09-30T14:37:12.976194+0000","last_clean":"2023-09-30T14:37:12.976194+0000","last_became_active":"2023-09-30T14:37:11.990158+0000","last_became_peered":"2023-09-30T14:37:11.990158+0000","last_unstale":"2023-09-30T14:37:12.976194+0000","last_undegraded":"2023-09-30T14:37:12.976194+0000","last_fullsized":"2023-09-30T14:37:12.976194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:14:27.976110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014402+0000","last_change":"2023-09-30T14:37:11.995884+0000","last_active":"2023-09-30T14:37:13.014402+0000","last_peered":"2023-09-30T14:37:13.014402+0000","last_clean":"2023-09-30T14:37:13.014402+0000","last_became_active":"2023-09-30T14:37:11.995551+0000","last_became_peered":"2023-09-30T14:37:11.995551+0000","last_unstale":"2023-09-30T14:37:13.014402+0000","last_undegraded":"2023-09-30T14:37:13.014402+0000","last_fullsized":"2023-09-30T14:37:13.014402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:28:26.051188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-30T14:37:13.014437+0000","last_change":"2023-09-30T14:37:11.985853+0000","last_active":"2023-09-30T14:37:13.014437+0000","last_peered":"2023-09-30T14:37:13.014437+0000","last_clean":"2023-09-30T14:37:13.014437+0000","last_became_active":"2023-09-30T14:37:11.985525+0000","last_became_peered":"2023-09-30T14:37:11.985525+0000","last_unstale":"2023-09-30T14:37:13.014437+0000","last_undegraded":"2023-09-30T14:37:13.014437+0000","last_fullsized":"2023-09-30T14:37:13.014437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:07:24.392538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58318,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":263,"num_read_kb":191,"num_write":147,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":174954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449108,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":425582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.433},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.38600000000000001}]},{"osd":1,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.69299999999999995}]},{"osd":2,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.55200000000000005}]},{"osd":5,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.623},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.71499999999999997}]},{"osd":6,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.48299999999999998}]},{"osd":7,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.749},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.59799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":9,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.80700000000000005}]},{"osd":10,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.76000000000000001}]}]},{"osd":10,"up_from":71,"seq":304942678040,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28560,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716592,"statfs":{"total":95995035648,"available":95965790208,"internally_reserved":0,"allocated":1581056,"data_stored":1071446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":1,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.45400000000000001}]},{"osd":2,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.66700000000000004}]},{"osd":3,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.47599999999999998}]},{"osd":4,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51900000000000002}]},{"osd":5,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.68000000000000005}]},{"osd":6,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.41699999999999998}]},{"osd":8,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.75}]},{"osd":9,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.72499999999999998}]},{"osd":11,"last update":"Sat Sep 30 14:37:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.78600000000000003}]}]},{"osd":9,"up_from":66,"seq":283467841564,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":425487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.46500000000000002,"15min":0.47499999999999998},"min":{"1min":0.28499999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60299999999999998,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.503,"15min":0.51800000000000002},"min":{"1min":0.32600000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65700000000000003,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.65900000000000003}]},{"osd":1,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77400000000000002,"5min":1.069,"15min":1.069},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.64500000000000002,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.749,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.622}]},{"osd":2,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.61799999999999999,"15min":0.628},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.38,"5min":0.376,"15min":0.376},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.67700000000000005}]},{"osd":3,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.54900000000000004,"15min":0.56599999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69899999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.86199999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.64200000000000002}]},{"osd":4,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.69299999999999995,"15min":0.70999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80500000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.71399999999999997,"15min":0.72699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86399999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.753}]},{"osd":5,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.70499999999999996,"15min":0.72199999999999998},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":1.137,"15min":1.137},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.73299999999999998,"15min":0.749},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0860000000000001,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.69499999999999995,"15min":0.71799999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.046,"5min":1.21,"15min":1.21},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.71599999999999997,"15min":0.73599999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96299999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.79700000000000004,"15min":0.81599999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94099999999999995,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.81200000000000006,"15min":0.82799999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.81999999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81200000000000006,"15min":0.82599999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.748},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.81599999999999995,"15min":0.83599999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.028,"5min":1.466,"15min":1.466},"last":0.876}]},{"osd":10,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.91200000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":425738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.64600000000000002}]},{"osd":1,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56200000000000006,"15min":0.55500000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.52700000000000002}]},{"osd":2,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.44800000000000001,"15min":0.44600000000000001},"min":{"1min":0.32600000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45500000000000002,"15min":0.45600000000000002},"min":{"1min":0.29999999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35999999999999999}]},{"osd":3,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.35599999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61499999999999999,"15min":0.61599999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83099999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.66500000000000004}]},{"osd":4,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.44,"5min":0.378,"15min":0.378},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63400000000000001,"15min":0.628},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.61299999999999999}]},{"osd":5,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59799999999999998,"15min":0.59899999999999998},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.44900000000000001}]},{"osd":6,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70599999999999996,"15min":0.70799999999999996},"min":{"1min":0.42899999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.99399999999999999,"5min":1.133,"15min":1.133},"last":0.63300000000000001}]},{"osd":7,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.022,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71899999999999997,"15min":0.71699999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.56200000000000006}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.749,"15min":0.747},"min":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95299999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.72899999999999998}]},{"osd":10,"last update":"Sat Sep 30 14:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1,"5min":1,"15min":1},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1503232,"data_stored":1017283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42999999999999999}]},{"osd":1,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.31,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63600000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.439,"15min":0.433},"min":{"1min":0.34300000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.58199999999999996}]},{"osd":2,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.38,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.748},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.72399999999999998}]},{"osd":3,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84099999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.82399999999999995,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.81200000000000006}]},{"osd":4,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.496},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.53000000000000003,"15min":0.51800000000000002},"min":{"1min":0.379,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64000000000000001}]},{"osd":5,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84199999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.83399999999999996}]},{"osd":6,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89500000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.72999999999999998,"15min":0.72899999999999998},"min":{"1min":0.47899999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98599999999999999,"5min":1.004,"15min":1.004},"last":2.9340000000000002}]},{"osd":8,"last update":"Sat Sep 30 14:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.48999999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.872},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.92300000000000004}]},{"osd":9,"last update":"Sat Sep 30 14:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77600000000000002,"15min":0.77900000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.57299999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96499999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.90500000000000003}]},{"osd":10,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.94399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":424882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38100000000000001,"15min":0.371},"min":{"1min":0.27700000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.40799999999999997,"15min":0.39000000000000001},"min":{"1min":0.28100000000000003,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.41399999999999998}]},{"osd":1,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.75700000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.72799999999999998}]},{"osd":2,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.628,"15min":0.627},"min":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.75700000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.63800000000000001}]},{"osd":3,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.502,"15min":0.48599999999999999},"min":{"1min":0.35299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.435},{"interface":"front","average":{"1min":0.504,"5min":0.499,"15min":0.497},"min":{"1min":0.35799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.35799999999999998}]},{"osd":4,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.46800000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.503},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67300000000000004,"15min":0.65100000000000002},"min":{"1min":0.46899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.89600000000000002,"5min":1.056,"15min":1.056},"last":0.55300000000000005}]},{"osd":5,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64600000000000002,"15min":0.64700000000000002},"min":{"1min":0.499,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.48099999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1060000000000001,"5min":1.228,"15min":1.228},"last":0.54100000000000004}]},{"osd":7,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.69899999999999995,"15min":0.68400000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.71199999999999997,"15min":0.69999999999999996},"min":{"1min":0.47999999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.79300000000000004}]},{"osd":8,"last update":"Sat Sep 30 14:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.55000000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.81599999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.624},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.70999999999999996}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.748,"15min":0.76000000000000001},"min":{"1min":0.45500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72499999999999998,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93200000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69599999999999995}]},{"osd":10,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.747}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":479377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53300000000000003,"15min":0.499},"min":{"1min":0.36199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73999999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54100000000000004,"15min":0.51300000000000001},"min":{"1min":0.33500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78600000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.71299999999999997}]},{"osd":2,"last update":"Sat Sep 30 14:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.55900000000000005,"15min":0.51100000000000001},"min":{"1min":0.435,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.55400000000000005,"15min":0.51600000000000001},"min":{"1min":0.46200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92400000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.60899999999999999}]},{"osd":3,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57899999999999996,"15min":0.54300000000000004},"min":{"1min":0.436,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60599999999999998,"15min":0.57499999999999996},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92200000000000004,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.80900000000000005}]},{"osd":4,"last update":"Sat Sep 30 14:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.443,"15min":0.42999999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.46800000000000003,"15min":0.45200000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.45500000000000002}]},{"osd":5,"last update":"Sat Sep 30 14:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58899999999999997,"15min":0.57799999999999996},"min":{"1min":0.40200000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.63900000000000001,"15min":0.61299999999999999},"min":{"1min":0.51700000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.78100000000000003}]},{"osd":6,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88300000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69499999999999995,"15min":0.67500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.82099999999999995}]},{"osd":7,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.55700000000000005,"15min":0.54000000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.49299999999999999}]},{"osd":8,"last update":"Sat Sep 30 14:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.74399999999999999,"15min":0.73599999999999999},"min":{"1min":0.55700000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.48799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.86799999999999999}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.81799999999999995,"15min":0.81399999999999995},"min":{"1min":0.56799999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.90500000000000003}]},{"osd":10,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.83899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":426147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 14:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.54600000000000004,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79400000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.435},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.55800000000000005,"15min":0.53000000000000003},"min":{"1min":0.51200000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.83099999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.52400000000000002}]},{"osd":2,"last update":"Sat Sep 30 14:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.372,"5min":0.317,"15min":0.317},"max":{"1min":0.76300000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57199999999999995,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91500000000000004,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.629}]},{"osd":3,"last update":"Sat Sep 30 14:35:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.432},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.307},{"interface":"front","average":{"1min":0.498,"5min":0.46100000000000002,"15min":0.45500000000000002},"min":{"1min":0.27700000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56399999999999995}]},{"osd":4,"last update":"Sat Sep 30 14:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.58599999999999997},"min":{"1min":0.40200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.60499999999999998,"15min":0.57699999999999996},"min":{"1min":0.47499999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.48299999999999998}]},{"osd":5,"last update":"Sat Sep 30 14:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.47299999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.496},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.45100000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.79400000000000004,"5min":0.996,"15min":0.996},"last":0.68999999999999995}]},{"osd":6,"last update":"Sat Sep 30 14:37:47 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69199999999999995,"5min":1.03,"15min":1.03},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.317,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80600000000000005,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.42099999999999999}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69699999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.014,"5min":1.147,"15min":1.147},"last":0.442},{"interface":"front","average":{"1min":0.748,"5min":0.67800000000000005,"15min":0.66600000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56799999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.52300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85199999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.745,"15min":0.753},"min":{"1min":0.51700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85299999999999998,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.65800000000000003}]},{"osd":9,"last update":"Sat Sep 30 14:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.71999999999999997,"15min":0.73899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80200000000000005,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.78200000000000003,"15min":0.80400000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.79900000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.55200000000000005}]},{"osd":10,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.54300000000000004}]},{"osd":11,"last update":"Sat Sep 30 14:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.71899999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":480785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.501,"15min":0.495},"min":{"1min":0.308,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.36599999999999999}]},{"osd":1,"last update":"Sat Sep 30 14:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.40799999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.76200000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.57599999999999996,"15min":0.53400000000000003},"min":{"1min":0.40699999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.72999999999999998}]},{"osd":3,"last update":"Sat Sep 30 14:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.35599999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.56499999999999995},"min":{"1min":0.42699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.625}]},{"osd":4,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.54400000000000004},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.432,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85899999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.74299999999999999}]},{"osd":5,"last update":"Sat Sep 30 14:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29299999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66300000000000003,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.48199999999999998},"min":{"1min":0.37,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73499999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.58999999999999997}]},{"osd":6,"last update":"Sat Sep 30 14:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66600000000000004,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1000000000000001,"5min":1.405,"15min":1.405},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65300000000000002}]},{"osd":7,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.45500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.80100000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70699999999999996,"15min":0.71299999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.86399999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.59199999999999997}]},{"osd":8,"last update":"Sat Sep 30 14:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.623,"15min":0.621},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.501},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.41099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.46999999999999997}]},{"osd":9,"last update":"Sat Sep 30 14:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.55000000000000004}]},{"osd":10,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.78300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":425199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.35799999999999998},"min":{"1min":0.27200000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.432,"15min":0.41199999999999998},"min":{"1min":0.316,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54800000000000004,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.307}]},{"osd":1,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64200000000000002,"15min":0.58599999999999997},"min":{"1min":0.442,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.002,"5min":5.798,"15min":5.798},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.39100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.70999999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58399999999999996,"15min":0.55100000000000005},"min":{"1min":0.442,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.52100000000000002}]},{"osd":4,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.64600000000000002,"15min":0.57899999999999996},"min":{"1min":0.36699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":7.1619999999999999,"5min":7.1619999999999999,"15min":7.1619999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.61199999999999999,"15min":0.57399999999999995},"min":{"1min":0.45500000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.58699999999999997}]},{"osd":5,"last update":"Sat Sep 30 14:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.45800000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85699999999999998,"5min":1.071,"15min":1.071},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.66000000000000003,"15min":0.65300000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79700000000000004,"5min":1.129,"15min":1.129},"last":0.49399999999999999}]},{"osd":6,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72399999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.377}]},{"osd":7,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.72299999999999998,"15min":0.70699999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.66200000000000003}]},{"osd":8,"last update":"Sat Sep 30 14:36:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.46100000000000002}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.69499999999999995,"15min":0.71399999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81299999999999994,"5min":1.161,"15min":1.161},"last":0.442}]},{"osd":10,"last update":"Sat Sep 30 14:37:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":7.2389999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":0.61299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":425644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67100000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.65000000000000002}]},{"osd":1,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.377,"15min":0.36899999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.377,"15min":0.36299999999999999},"min":{"1min":0.33700000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.45500000000000002}]},{"osd":2,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.54800000000000004,"15min":0.51600000000000001},"min":{"1min":0.41299999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.374,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56200000000000006}]},{"osd":3,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.871}]},{"osd":5,"last update":"Sat Sep 30 14:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58599999999999997,"15min":0.56399999999999995},"min":{"1min":0.42999999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91700000000000004,"5min":1.484,"15min":1.484},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.57399999999999995,"15min":0.55400000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83599999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.77500000000000002}]},{"osd":6,"last update":"Sat Sep 30 14:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.626,"15min":0.626},"min":{"1min":0.501,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78300000000000003,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.439,"5min":0.435,"15min":0.435},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.71499999999999997}]},{"osd":7,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54500000000000004,"15min":0.53400000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.62},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":0.66900000000000004}]},{"osd":8,"last update":"Sat Sep 30 14:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.66200000000000003,"15min":0.65600000000000003},"min":{"1min":0.51800000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88800000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.46800000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.83799999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.85299999999999998}]},{"osd":9,"last update":"Sat Sep 30 14:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.55100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.85299999999999998,"5min":1.038,"15min":1.038},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.58599999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.95199999999999996,"5min":1.677,"15min":1.677},"last":0.89600000000000002}]},{"osd":10,"last update":"Sat Sep 30 14:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.57999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1016636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51200000000000001,"15min":0.49199999999999999},"min":{"1min":0.34399999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52600000000000002,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78100000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.747}]},{"osd":1,"last update":"Sat Sep 30 14:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.52600000000000002,"15min":0.496},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.626},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.51700000000000002,"15min":0.49099999999999999},"min":{"1min":0.38900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.71999999999999997}]},{"osd":2,"last update":"Sat Sep 30 14:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.432,"15min":0.41499999999999998},"min":{"1min":0.308,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.64200000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.434,"15min":0.41999999999999998},"min":{"1min":0.32700000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.61099999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.40300000000000002}]},{"osd":3,"last update":"Sat Sep 30 14:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60699999999999998,"15min":0.58799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.77900000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.73299999999999998}]},{"osd":4,"last update":"Sat Sep 30 14:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.56499999999999995,"15min":0.53900000000000003},"min":{"1min":0.39700000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.56599999999999995},"min":{"1min":0.46200000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.78000000000000003,"5min":0.875,"15min":0.875},"last":0.64500000000000002}]},{"osd":6,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.69499999999999995,"15min":0.66000000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":4.125,"5min":4.125,"15min":4.125},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":0.73499999999999999,"15min":0.64700000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":12.24,"5min":12.24,"15min":12.24},"last":0.57099999999999995}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67600000000000005,"15min":0.67800000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.65800000000000003}]},{"osd":8,"last update":"Sat Sep 30 14:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42499999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.629,"15min":0.621},"min":{"1min":0.46999999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.54500000000000004}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.78400000000000003,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.503,"15min":0.503},"max":{"1min":5.1520000000000001,"5min":5.1520000000000001,"15min":5.1520000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.3580000000000001,"5min":0.874,"15min":0.79300000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":12.198,"5min":12.198,"15min":12.198},"last":0.79800000000000004}]},{"osd":10,"last update":"Sat Sep 30 14:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.83199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T14:38:01.407 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-09-30T14:38:01.408 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-30T14:38:01.408 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-09-30T14:38:01.408 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph health --format=json 2023-09-30T14:38:01.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:01 smithi156 ceph-mon[103579]: pgmap v277: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:01.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:01 smithi148 ceph-mon[102355]: pgmap v277: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:38:02.426 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:02 smithi032 ceph-mon[98592]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:38:02.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:02 smithi156 ceph-mon[103579]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:38:02.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:02 smithi148 ceph-mon[102355]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:38:03.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.299+0000 7fa1246b9700 1 -- 172.21.15.32:0/4026137285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c106f00 msgr2=0x7fa11c1072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:03.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.299+0000 7fa1246b9700 1 --2- 172.21.15.32:0/4026137285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c106f00 0x7fa11c1072e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa1180099c0 tx=0x7fa118009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:03.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.299+0000 7fa1246b9700 1 -- 172.21.15.32:0/4026137285 shutdown_connections 2023-09-30T14:38:03.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.299+0000 7fa1246b9700 1 --2- 172.21.15.32:0/4026137285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa11c1018b0 0x7fa11c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:03.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.299+0000 7fa1246b9700 1 --2- 172.21.15.32:0/4026137285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c106f00 0x7fa11c1072e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:03.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.299+0000 7fa1246b9700 1 --2- 172.21.15.32:0/4026137285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c100ef0 0x7fa11c101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:03.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.299+0000 7fa1246b9700 1 -- 172.21.15.32:0/4026137285 >> 172.21.15.32:0/4026137285 conn(0x7fa11c078010 msgr2=0x7fa11c078420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:03.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.300+0000 7fa1246b9700 1 -- 172.21.15.32:0/4026137285 shutdown_connections 2023-09-30T14:38:03.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.300+0000 7fa1246b9700 1 -- 172.21.15.32:0/4026137285 wait complete. 2023-09-30T14:38:03.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.300+0000 7fa1246b9700 1 Processor -- start 2023-09-30T14:38:03.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.300+0000 7fa1246b9700 1 -- start start 2023-09-30T14:38:03.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.300+0000 7fa1246b9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c100ef0 0x7fa11c196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:03.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.300+0000 7fa1246b9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa11c1018b0 0x7fa11c197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:03.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.300+0000 7fa1246b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c106f00 0x7fa11c19b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:03.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.300+0000 7fa1246b9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c19beb0 con 0x7fa11c106f00 2023-09-30T14:38:03.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.300+0000 7fa1246b9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c19c020 con 0x7fa11c100ef0 2023-09-30T14:38:03.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.300+0000 7fa1246b9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c19c1e0 con 0x7fa11c1018b0 2023-09-30T14:38:03.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.301+0000 7fa122c56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c106f00 0x7fa11c19b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:03.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.301+0000 7fa122c56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c106f00 0x7fa11c19b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34400/0 (socket says 172.21.15.32:34400) 2023-09-30T14:38:03.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.301+0000 7fa122c56700 1 -- 172.21.15.32:0/3663707910 learned_addr learned my addr 172.21.15.32:0/3663707910 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:03.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.301+0000 7fa122455700 1 --2- 172.21.15.32:0/3663707910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c100ef0 0x7fa11c196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:03.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.301+0000 7fa121c54700 1 --2- 172.21.15.32:0/3663707910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa11c1018b0 0x7fa11c197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:03.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.301+0000 7fa122c56700 1 -- 172.21.15.32:0/3663707910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa11c1018b0 msgr2=0x7fa11c197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:03.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.301+0000 7fa122c56700 1 --2- 172.21.15.32:0/3663707910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa11c1018b0 0x7fa11c197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:03.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.301+0000 7fa122c56700 1 -- 172.21.15.32:0/3663707910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c100ef0 msgr2=0x7fa11c196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:03.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.301+0000 7fa122c56700 1 --2- 172.21.15.32:0/3663707910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c100ef0 0x7fa11c196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:03.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.301+0000 7fa122c56700 1 -- 172.21.15.32:0/3663707910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1180095f0 con 0x7fa11c106f00 2023-09-30T14:38:03.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.301+0000 7fa122c56700 1 --2- 172.21.15.32:0/3663707910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c106f00 0x7fa11c19b8a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa114007f10 tx=0x7fa11400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:03.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.301+0000 7fa1137fe700 1 -- 172.21.15.32:0/3663707910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114017070 con 0x7fa11c106f00 2023-09-30T14:38:03.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.301+0000 7fa1246b9700 1 -- 172.21.15.32:0/3663707910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c19c4c0 con 0x7fa11c106f00 2023-09-30T14:38:03.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.301+0000 7fa1137fe700 1 -- 172.21.15.32:0/3663707910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa11400f040 con 0x7fa11c106f00 2023-09-30T14:38:03.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.301+0000 7fa1137fe700 1 -- 172.21.15.32:0/3663707910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114005740 con 0x7fa11c106f00 2023-09-30T14:38:03.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.301+0000 7fa1246b9700 1 -- 172.21.15.32:0/3663707910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c1a3d20 con 0x7fa11c106f00 2023-09-30T14:38:03.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.302+0000 7fa1246b9700 1 -- 172.21.15.32:0/3663707910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c04eb20 con 0x7fa11c106f00 2023-09-30T14:38:03.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.303+0000 7fa1137fe700 1 -- 172.21.15.32:0/3663707910 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fa11400dc60 con 0x7fa11c106f00 2023-09-30T14:38:03.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.304+0000 7fa1137fe700 1 --2- 172.21.15.32:0/3663707910 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa1080a1030 0x7fa1080a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:03.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.304+0000 7fa122455700 1 --2- 172.21.15.32:0/3663707910 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa1080a1030 0x7fa1080a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:03.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.304+0000 7fa1137fe700 1 -- 172.21.15.32:0/3663707910 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa1140cc4e0 con 0x7fa11c106f00 2023-09-30T14:38:03.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.304+0000 7fa122455700 1 --2- 172.21.15.32:0/3663707910 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa1080a1030 0x7fa1080a34f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa118000c00 tx=0x7fa118003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:03.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.308+0000 7fa1137fe700 1 -- 172.21.15.32:0/3663707910 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fa114095d30 con 0x7fa11c106f00 2023-09-30T14:38:03.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:03 smithi032 ceph-mon[98592]: pgmap v278: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:38:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.552+0000 7fa1246b9700 1 -- 172.21.15.32:0/3663707910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa11c198250 con 0x7fa11c106f00 2023-09-30T14:38:03.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.553+0000 7fa1137fe700 1 -- 172.21.15.32:0/3663707910 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa114099350 con 0x7fa11c106f00 2023-09-30T14:38:03.553 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:03.553 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-30T14:38:03.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.555+0000 7fa1246b9700 1 -- 172.21.15.32:0/3663707910 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa1080a1030 msgr2=0x7fa1080a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.555+0000 7fa1246b9700 1 --2- 172.21.15.32:0/3663707910 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa1080a1030 0x7fa1080a34f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa118000c00 tx=0x7fa118003680 comp rx=0 tx=0).stop 2023-09-30T14:38:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.555+0000 7fa1246b9700 1 -- 172.21.15.32:0/3663707910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c106f00 msgr2=0x7fa11c19b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.555+0000 7fa1246b9700 1 --2- 172.21.15.32:0/3663707910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c106f00 0x7fa11c19b8a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa114007f10 tx=0x7fa11400d3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.555+0000 7fa1246b9700 1 -- 172.21.15.32:0/3663707910 shutdown_connections 2023-09-30T14:38:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.555+0000 7fa1246b9700 1 --2- 172.21.15.32:0/3663707910 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa1080a1030 0x7fa1080a34f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.555+0000 7fa1246b9700 1 --2- 172.21.15.32:0/3663707910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c106f00 0x7fa11c19b8a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.555+0000 7fa1246b9700 1 --2- 172.21.15.32:0/3663707910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c100ef0 0x7fa11c196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.555+0000 7fa1246b9700 1 --2- 172.21.15.32:0/3663707910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa11c1018b0 0x7fa11c197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.555+0000 7fa1246b9700 1 -- 172.21.15.32:0/3663707910 >> 172.21.15.32:0/3663707910 conn(0x7fa11c078010 msgr2=0x7fa11c0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.555+0000 7fa1246b9700 1 -- 172.21.15.32:0/3663707910 shutdown_connections 2023-09-30T14:38:03.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:03.555+0000 7fa1246b9700 1 -- 172.21.15.32:0/3663707910 wait complete. 2023-09-30T14:38:03.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:03 smithi156 ceph-mon[103579]: pgmap v278: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:38:03.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:03 smithi148 ceph-mon[102355]: pgmap v278: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:38:03.939 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-09-30T14:38:03.940 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T14:38:04.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.072+0000 7f0c871c4700 1 -- 172.21.15.32:0/438963749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c80101930 msgr2=0x7f0c80103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:04.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.072+0000 7f0c871c4700 1 --2- 172.21.15.32:0/438963749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c80101930 0x7f0c80103d20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009990 tx=0x7f0c74009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.072+0000 7f0c871c4700 1 -- 172.21.15.32:0/438963749 shutdown_connections 2023-09-30T14:38:04.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.072+0000 7f0c871c4700 1 --2- 172.21.15.32:0/438963749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c80106b90 0x7f0c80108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.072+0000 7f0c871c4700 1 --2- 172.21.15.32:0/438963749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c80101930 0x7f0c80103d20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.072+0000 7f0c871c4700 1 --2- 172.21.15.32:0/438963749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c80104260 0x7f0c80106650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.072+0000 7f0c871c4700 1 -- 172.21.15.32:0/438963749 >> 172.21.15.32:0/438963749 conn(0x7f0c800fb2a0 msgr2=0x7f0c800fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:04.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.072+0000 7f0c871c4700 1 -- 172.21.15.32:0/438963749 shutdown_connections 2023-09-30T14:38:04.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.072+0000 7f0c871c4700 1 -- 172.21.15.32:0/438963749 wait complete. 2023-09-30T14:38:04.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c871c4700 1 Processor -- start 2023-09-30T14:38:04.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c871c4700 1 -- start start 2023-09-30T14:38:04.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c871c4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c80101930 0x7f0c801969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:04.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c871c4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c80104260 0x7f0c80196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:04.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c871c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c80106b90 0x7f0c8019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:04.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c871c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8019e570 con 0x7f0c80106b90 2023-09-30T14:38:04.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c871c4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8019e6b0 con 0x7f0c80101930 2023-09-30T14:38:04.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c871c4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8019e840 con 0x7f0c80104260 2023-09-30T14:38:04.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c85761700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c80106b90 0x7f0c8019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:04.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c85761700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c80106b90 0x7f0c8019dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34412/0 (socket says 172.21.15.32:34412) 2023-09-30T14:38:04.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c85761700 1 -- 172.21.15.32:0/3138445913 learned_addr learned my addr 172.21.15.32:0/3138445913 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c84f60700 1 --2- 172.21.15.32:0/3138445913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c80101930 0x7f0c801969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c7ffff700 1 --2- 172.21.15.32:0/3138445913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c80104260 0x7f0c80196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c85761700 1 -- 172.21.15.32:0/3138445913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c80104260 msgr2=0x7f0c80196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c85761700 1 --2- 172.21.15.32:0/3138445913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c80104260 0x7f0c80196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c85761700 1 -- 172.21.15.32:0/3138445913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c80101930 msgr2=0x7f0c801969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c85761700 1 --2- 172.21.15.32:0/3138445913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c80101930 0x7f0c801969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.073+0000 7f0c85761700 1 -- 172.21.15.32:0/3138445913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c740095f0 con 0x7f0c80106b90 2023-09-30T14:38:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.074+0000 7f0c85761700 1 --2- 172.21.15.32:0/3138445913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c80106b90 0x7f0c8019dfa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0c78007ee0 tx=0x7f0c7800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.074+0000 7f0c7dffb700 1 -- 172.21.15.32:0/3138445913 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c78017070 con 0x7f0c80106b90 2023-09-30T14:38:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.074+0000 7f0c7dffb700 1 -- 172.21.15.32:0/3138445913 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c7800f040 con 0x7f0c80106b90 2023-09-30T14:38:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.074+0000 7f0c871c4700 1 -- 172.21.15.32:0/3138445913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8019eb20 con 0x7f0c80106b90 2023-09-30T14:38:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.074+0000 7f0c7dffb700 1 -- 172.21.15.32:0/3138445913 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c78005740 con 0x7f0c80106b90 2023-09-30T14:38:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.074+0000 7f0c871c4700 1 -- 172.21.15.32:0/3138445913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8019f030 con 0x7f0c80106b90 2023-09-30T14:38:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.075+0000 7f0c871c4700 1 -- 172.21.15.32:0/3138445913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c800fce20 con 0x7f0c80106b90 2023-09-30T14:38:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.076+0000 7f0c7dffb700 1 -- 172.21.15.32:0/3138445913 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f0c7800dc60 con 0x7f0c80106b90 2023-09-30T14:38:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.076+0000 7f0c7dffb700 1 --2- 172.21.15.32:0/3138445913 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0c700a10f0 0x7f0c700a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.077+0000 7f0c7dffb700 1 -- 172.21.15.32:0/3138445913 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0c780cbe80 con 0x7f0c80106b90 2023-09-30T14:38:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.077+0000 7f0c84f60700 1 --2- 172.21.15.32:0/3138445913 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0c700a10f0 0x7f0c700a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.077+0000 7f0c84f60700 1 --2- 172.21.15.32:0/3138445913 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0c700a10f0 0x7f0c700a35b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400b5c0 tx=0x7f0c74005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:04.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.081+0000 7f0c7dffb700 1 -- 172.21.15.32:0/3138445913 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f0c780956d0 con 0x7f0c80106b90 2023-09-30T14:38:04.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.286+0000 7f0c871c4700 1 -- 172.21.15.32:0/3138445913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c8010c630 con 0x7f0c80106b90 2023-09-30T14:38:04.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.287+0000 7f0c7dffb700 1 -- 172.21.15.32:0/3138445913 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22133 (secure 0 0 0) 0x7f0c78098cf0 con 0x7f0c80106b90 2023-09-30T14:38:04.287 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:04.287 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":83,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:37:12.972217+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:38:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.289+0000 7f0c871c4700 1 -- 172.21.15.32:0/3138445913 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0c700a10f0 msgr2=0x7f0c700a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.289+0000 7f0c871c4700 1 --2- 172.21.15.32:0/3138445913 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0c700a10f0 0x7f0c700a35b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400b5c0 tx=0x7f0c74005fd0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.289+0000 7f0c871c4700 1 -- 172.21.15.32:0/3138445913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c80106b90 msgr2=0x7f0c8019dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.289+0000 7f0c871c4700 1 --2- 172.21.15.32:0/3138445913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c80106b90 0x7f0c8019dfa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0c78007ee0 tx=0x7f0c7800d3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.289+0000 7f0c871c4700 1 -- 172.21.15.32:0/3138445913 shutdown_connections 2023-09-30T14:38:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.289+0000 7f0c871c4700 1 --2- 172.21.15.32:0/3138445913 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0c700a10f0 0x7f0c700a35b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.289+0000 7f0c871c4700 1 --2- 172.21.15.32:0/3138445913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c80106b90 0x7f0c8019dfa0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.289+0000 7f0c871c4700 1 --2- 172.21.15.32:0/3138445913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c80101930 0x7f0c801969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.289+0000 7f0c871c4700 1 --2- 172.21.15.32:0/3138445913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c80104260 0x7f0c80196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.289+0000 7f0c871c4700 1 -- 172.21.15.32:0/3138445913 >> 172.21.15.32:0/3138445913 conn(0x7f0c800fb2a0 msgr2=0x7f0c801025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.289+0000 7f0c871c4700 1 -- 172.21.15.32:0/3138445913 shutdown_connections 2023-09-30T14:38:04.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.289+0000 7f0c871c4700 1 -- 172.21.15.32:0/3138445913 wait complete. 2023-09-30T14:38:04.302 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T14:33:20.224529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T14:37:09.573219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T14:37:10.844458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T14:38:04.302 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T14:38:04.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.432+0000 7f6d923d0700 1 -- 172.21.15.32:0/2867527719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8c100f20 msgr2=0x7f6d8c101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:04.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.432+0000 7f6d923d0700 1 --2- 172.21.15.32:0/2867527719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8c100f20 0x7f6d8c101380 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6d800099c0 tx=0x7f6d80009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.432+0000 7f6d923d0700 1 -- 172.21.15.32:0/2867527719 shutdown_connections 2023-09-30T14:38:04.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.432+0000 7f6d923d0700 1 --2- 172.21.15.32:0/2867527719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d8c1018c0 0x7f6d8c108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.432+0000 7f6d923d0700 1 --2- 172.21.15.32:0/2867527719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d8c0ffd30 0x7f6d8c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.432+0000 7f6d923d0700 1 --2- 172.21.15.32:0/2867527719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8c100f20 0x7f6d8c101380 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.432+0000 7f6d923d0700 1 -- 172.21.15.32:0/2867527719 >> 172.21.15.32:0/2867527719 conn(0x7f6d8c0fb2a0 msgr2=0x7f6d8c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:04.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.432+0000 7f6d923d0700 1 -- 172.21.15.32:0/2867527719 shutdown_connections 2023-09-30T14:38:04.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.432+0000 7f6d923d0700 1 -- 172.21.15.32:0/2867527719 wait complete. 2023-09-30T14:38:04.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.432+0000 7f6d923d0700 1 Processor -- start 2023-09-30T14:38:04.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.432+0000 7f6d923d0700 1 -- start start 2023-09-30T14:38:04.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d923d0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d8c0ffd30 0x7f6d8c195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:04.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d923d0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d8c100f20 0x7f6d8c195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:04.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d923d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8c1018c0 0x7f6d8c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:04.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d923d0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c19d480 con 0x7f6d8c1018c0 2023-09-30T14:38:04.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d923d0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c19d5c0 con 0x7f6d8c0ffd30 2023-09-30T14:38:04.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d923d0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c19d750 con 0x7f6d8c100f20 2023-09-30T14:38:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d9096d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8c1018c0 0x7f6d8c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d8c0ffd30 0x7f6d8c195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d9096d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8c1018c0 0x7f6d8c19ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34424/0 (socket says 172.21.15.32:34424) 2023-09-30T14:38:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d9096d700 1 -- 172.21.15.32:0/1469101877 learned_addr learned my addr 172.21.15.32:0/1469101877 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d8b7fe700 1 --2- 172.21.15.32:0/1469101877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d8c100f20 0x7f6d8c195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d9096d700 1 -- 172.21.15.32:0/1469101877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d8c100f20 msgr2=0x7f6d8c195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d9096d700 1 --2- 172.21.15.32:0/1469101877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d8c100f20 0x7f6d8c195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d9096d700 1 -- 172.21.15.32:0/1469101877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d8c0ffd30 msgr2=0x7f6d8c195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d9096d700 1 --2- 172.21.15.32:0/1469101877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d8c0ffd30 0x7f6d8c195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d9096d700 1 -- 172.21.15.32:0/1469101877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d800095f0 con 0x7f6d8c1018c0 2023-09-30T14:38:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d8b7fe700 1 --2- 172.21.15.32:0/1469101877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d8c100f20 0x7f6d8c195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d9096d700 1 --2- 172.21.15.32:0/1469101877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8c1018c0 0x7f6d8c19ceb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400efb0 tx=0x7f6d8400c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:04.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.433+0000 7f6d897fa700 1 -- 172.21.15.32:0/1469101877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84011670 con 0x7f6d8c1018c0 2023-09-30T14:38:04.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.434+0000 7f6d897fa700 1 -- 172.21.15.32:0/1469101877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d84011c90 con 0x7f6d8c1018c0 2023-09-30T14:38:04.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.434+0000 7f6d897fa700 1 -- 172.21.15.32:0/1469101877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8401ac70 con 0x7f6d8c1018c0 2023-09-30T14:38:04.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.434+0000 7f6d923d0700 1 -- 172.21.15.32:0/1469101877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c068010 con 0x7f6d8c1018c0 2023-09-30T14:38:04.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.434+0000 7f6d923d0700 1 -- 172.21.15.32:0/1469101877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c0684d0 con 0x7f6d8c1018c0 2023-09-30T14:38:04.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.435+0000 7f6d923d0700 1 -- 172.21.15.32:0/1469101877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8c067bd0 con 0x7f6d8c1018c0 2023-09-30T14:38:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.436+0000 7f6d897fa700 1 -- 172.21.15.32:0/1469101877 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f6d840117d0 con 0x7f6d8c1018c0 2023-09-30T14:38:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.436+0000 7f6d897fa700 1 --2- 172.21.15.32:0/1469101877 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6d740a1130 0x7f6d740a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.436+0000 7f6d8bfff700 1 --2- 172.21.15.32:0/1469101877 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6d740a1130 0x7f6d740a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.437+0000 7f6d897fa700 1 -- 172.21.15.32:0/1469101877 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6d840ceee0 con 0x7f6d8c1018c0 2023-09-30T14:38:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.437+0000 7f6d8bfff700 1 --2- 172.21.15.32:0/1469101877 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6d740a1130 0x7f6d740a35f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c100d80 tx=0x7f6d7c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:04.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.441+0000 7f6d897fa700 1 -- 172.21.15.32:0/1469101877 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f6d84098630 con 0x7f6d8c1018c0 2023-09-30T14:38:04.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:04 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3663707910' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-30T14:38:04.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:04 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3663707910' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-30T14:38:04.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:04 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3663707910' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-30T14:38:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.642+0000 7f6d923d0700 1 -- 172.21.15.32:0/1469101877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d8c10b540 con 0x7f6d8c1018c0 2023-09-30T14:38:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.642+0000 7f6d897fa700 1 -- 172.21.15.32:0/1469101877 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d8409bc50 con 0x7f6d8c1018c0 2023-09-30T14:38:04.643 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T14:38:04.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.645+0000 7f6d923d0700 1 -- 172.21.15.32:0/1469101877 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6d740a1130 msgr2=0x7f6d740a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:04.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.645+0000 7f6d923d0700 1 --2- 172.21.15.32:0/1469101877 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6d740a1130 0x7f6d740a35f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c100d80 tx=0x7f6d7c00a3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.645+0000 7f6d923d0700 1 -- 172.21.15.32:0/1469101877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8c1018c0 msgr2=0x7f6d8c19ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:04.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.645+0000 7f6d923d0700 1 --2- 172.21.15.32:0/1469101877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8c1018c0 0x7f6d8c19ceb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400efb0 tx=0x7f6d8400c550 comp rx=0 tx=0).stop 2023-09-30T14:38:04.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.645+0000 7f6d923d0700 1 -- 172.21.15.32:0/1469101877 shutdown_connections 2023-09-30T14:38:04.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.645+0000 7f6d923d0700 1 --2- 172.21.15.32:0/1469101877 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6d740a1130 0x7f6d740a35f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.645+0000 7f6d923d0700 1 --2- 172.21.15.32:0/1469101877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8c1018c0 0x7f6d8c19ceb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.645+0000 7f6d923d0700 1 --2- 172.21.15.32:0/1469101877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d8c0ffd30 0x7f6d8c195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.645+0000 7f6d923d0700 1 --2- 172.21.15.32:0/1469101877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d8c100f20 0x7f6d8c195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.645+0000 7f6d923d0700 1 -- 172.21.15.32:0/1469101877 >> 172.21.15.32:0/1469101877 conn(0x7f6d8c0fb2a0 msgr2=0x7f6d8c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:04.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.645+0000 7f6d923d0700 1 -- 172.21.15.32:0/1469101877 shutdown_connections 2023-09-30T14:38:04.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.645+0000 7f6d923d0700 1 -- 172.21.15.32:0/1469101877 wait complete. 2023-09-30T14:38:04.657 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T14:38:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.786+0000 7fa6ab287700 1 -- 172.21.15.32:0/2177127250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a41018c0 msgr2=0x7fa6a4108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.786+0000 7fa6ab287700 1 --2- 172.21.15.32:0/2177127250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a41018c0 0x7fa6a4108170 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00b0d0 tx=0x7fa69c00b3e0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.786+0000 7fa6ab287700 1 -- 172.21.15.32:0/2177127250 shutdown_connections 2023-09-30T14:38:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.786+0000 7fa6ab287700 1 --2- 172.21.15.32:0/2177127250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a41018c0 0x7fa6a4108170 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.786+0000 7fa6ab287700 1 --2- 172.21.15.32:0/2177127250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a40ffd30 0x7fa6a4100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.786+0000 7fa6ab287700 1 --2- 172.21.15.32:0/2177127250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a4100f20 0x7fa6a4101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.786+0000 7fa6ab287700 1 -- 172.21.15.32:0/2177127250 >> 172.21.15.32:0/2177127250 conn(0x7fa6a40fb2a0 msgr2=0x7fa6a40fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.786+0000 7fa6ab287700 1 -- 172.21.15.32:0/2177127250 shutdown_connections 2023-09-30T14:38:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.786+0000 7fa6ab287700 1 -- 172.21.15.32:0/2177127250 wait complete. 2023-09-30T14:38:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6ab287700 1 Processor -- start 2023-09-30T14:38:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6ab287700 1 -- start start 2023-09-30T14:38:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6ab287700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a40ffd30 0x7fa6a4103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6ab287700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a4100f20 0x7fa6a4102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6ab287700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a41018c0 0x7fa6a4102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6ab287700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a4102fe0 con 0x7fa6a4100f20 2023-09-30T14:38:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6ab287700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a4103150 con 0x7fa6a40ffd30 2023-09-30T14:38:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6ab287700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a41a0c10 con 0x7fa6a41018c0 2023-09-30T14:38:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6a8822700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a4100f20 0x7fa6a4102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6a8822700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a4100f20 0x7fa6a4102300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34436/0 (socket says 172.21.15.32:34436) 2023-09-30T14:38:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6a8822700 1 -- 172.21.15.32:0/39753728 learned_addr learned my addr 172.21.15.32:0/39753728 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6a9023700 1 --2- 172.21.15.32:0/39753728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a40ffd30 0x7fa6a4103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6a9824700 1 --2- 172.21.15.32:0/39753728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a41018c0 0x7fa6a4102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6a8822700 1 -- 172.21.15.32:0/39753728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a41018c0 msgr2=0x7fa6a4102840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6a8822700 1 --2- 172.21.15.32:0/39753728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a41018c0 0x7fa6a4102840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6a8822700 1 -- 172.21.15.32:0/39753728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a40ffd30 msgr2=0x7fa6a4103cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6a8822700 1 --2- 172.21.15.32:0/39753728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a40ffd30 0x7fa6a4103cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6a8822700 1 -- 172.21.15.32:0/39753728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c009d00 con 0x7fa6a4100f20 2023-09-30T14:38:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.787+0000 7fa6a9824700 1 --2- 172.21.15.32:0/39753728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a41018c0 0x7fa6a4102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.788+0000 7fa6a8822700 1 --2- 172.21.15.32:0/39753728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a4100f20 0x7fa6a4102300 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa69800c9b0 tx=0x7fa69800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.788+0000 7fa6967fc700 1 -- 172.21.15.32:0/39753728 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698007b00 con 0x7fa6a4100f20 2023-09-30T14:38:04.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.788+0000 7fa6967fc700 1 -- 172.21.15.32:0/39753728 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa698007c60 con 0x7fa6a4100f20 2023-09-30T14:38:04.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.788+0000 7fa6ab287700 1 -- 172.21.15.32:0/39753728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a41a0ef0 con 0x7fa6a4100f20 2023-09-30T14:38:04.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.788+0000 7fa6967fc700 1 -- 172.21.15.32:0/39753728 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698018660 con 0x7fa6a4100f20 2023-09-30T14:38:04.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.788+0000 7fa6ab287700 1 -- 172.21.15.32:0/39753728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a41a13c0 con 0x7fa6a4100f20 2023-09-30T14:38:04.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.789+0000 7fa6ab287700 1 -- 172.21.15.32:0/39753728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a410b550 con 0x7fa6a4100f20 2023-09-30T14:38:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.790+0000 7fa6967fc700 1 -- 172.21.15.32:0/39753728 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fa698018880 con 0x7fa6a4100f20 2023-09-30T14:38:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.790+0000 7fa6967fc700 1 --2- 172.21.15.32:0/39753728 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa6900a10b0 0x7fa6900a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.791+0000 7fa6a9023700 1 --2- 172.21.15.32:0/39753728 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa6900a10b0 0x7fa6900a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.791+0000 7fa6967fc700 1 -- 172.21.15.32:0/39753728 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa6980cc700 con 0x7fa6a4100f20 2023-09-30T14:38:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.791+0000 7fa6a9023700 1 --2- 172.21.15.32:0/39753728 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa6900a10b0 0x7fa6900a3570 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0005950 tx=0x7fa6a00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:04.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.795+0000 7fa6967fc700 1 -- 172.21.15.32:0/39753728 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fa698095f50 con 0x7fa6a4100f20 2023-09-30T14:38:04.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.997+0000 7fa6ab287700 1 -- 172.21.15.32:0/39753728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa6a41a1640 con 0x7fa6a4100f20 2023-09-30T14:38:04.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:04.998+0000 7fa6967fc700 1 -- 172.21.15.32:0/39753728 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa698099570 con 0x7fa6a4100f20 2023-09-30T14:38:04.998 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-09-30T14:38:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.000+0000 7fa6ab287700 1 -- 172.21.15.32:0/39753728 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa6900a10b0 msgr2=0x7fa6900a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.000+0000 7fa6ab287700 1 --2- 172.21.15.32:0/39753728 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa6900a10b0 0x7fa6900a3570 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0005950 tx=0x7fa6a00058e0 comp rx=0 tx=0).stop 2023-09-30T14:38:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.000+0000 7fa6ab287700 1 -- 172.21.15.32:0/39753728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a4100f20 msgr2=0x7fa6a4102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.000+0000 7fa6ab287700 1 --2- 172.21.15.32:0/39753728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a4100f20 0x7fa6a4102300 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa69800c9b0 tx=0x7fa69800cd70 comp rx=0 tx=0).stop 2023-09-30T14:38:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.000+0000 7fa6ab287700 1 -- 172.21.15.32:0/39753728 shutdown_connections 2023-09-30T14:38:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.000+0000 7fa6ab287700 1 --2- 172.21.15.32:0/39753728 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa6900a10b0 0x7fa6900a3570 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.000+0000 7fa6ab287700 1 --2- 172.21.15.32:0/39753728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6a41018c0 0x7fa6a4102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:05.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.000+0000 7fa6ab287700 1 --2- 172.21.15.32:0/39753728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a40ffd30 0x7fa6a4103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:05.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.000+0000 7fa6ab287700 1 --2- 172.21.15.32:0/39753728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6a4100f20 0x7fa6a4102300 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:05.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.000+0000 7fa6ab287700 1 -- 172.21.15.32:0/39753728 >> 172.21.15.32:0/39753728 conn(0x7fa6a40fb2a0 msgr2=0x7fa6a4067ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:05.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.000+0000 7fa6ab287700 1 -- 172.21.15.32:0/39753728 shutdown_connections 2023-09-30T14:38:05.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.000+0000 7fa6ab287700 1 -- 172.21.15.32:0/39753728 wait complete. 2023-09-30T14:38:05.012 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T14:38:05.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.141+0000 7ff3af6ed700 1 -- 172.21.15.32:0/2334524180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a80ffd30 msgr2=0x7ff3a8100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:05.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.141+0000 7ff3af6ed700 1 --2- 172.21.15.32:0/2334524180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a80ffd30 0x7ff3a8100150 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40099c0 tx=0x7ff3a4009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:05.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.141+0000 7ff3af6ed700 1 -- 172.21.15.32:0/2334524180 shutdown_connections 2023-09-30T14:38:05.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.142+0000 7ff3af6ed700 1 --2- 172.21.15.32:0/2334524180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3a81018c0 0x7ff3a8108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:05.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.142+0000 7ff3af6ed700 1 --2- 172.21.15.32:0/2334524180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a80ffd30 0x7ff3a8100150 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:05.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.142+0000 7ff3af6ed700 1 --2- 172.21.15.32:0/2334524180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3a8100f20 0x7ff3a8101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:05.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.142+0000 7ff3af6ed700 1 -- 172.21.15.32:0/2334524180 >> 172.21.15.32:0/2334524180 conn(0x7ff3a80fb2a0 msgr2=0x7ff3a80fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:05.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.142+0000 7ff3af6ed700 1 -- 172.21.15.32:0/2334524180 shutdown_connections 2023-09-30T14:38:05.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.142+0000 7ff3af6ed700 1 -- 172.21.15.32:0/2334524180 wait complete. 2023-09-30T14:38:05.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.142+0000 7ff3af6ed700 1 Processor -- start 2023-09-30T14:38:05.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.142+0000 7ff3af6ed700 1 -- start start 2023-09-30T14:38:05.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.142+0000 7ff3af6ed700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3a80ffd30 0x7ff3a8195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.142+0000 7ff3af6ed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a8100f20 0x7ff3a8195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.142+0000 7ff3af6ed700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3a81018c0 0x7ff3a819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.142+0000 7ff3af6ed700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a819d5b0 con 0x7ff3a8100f20 2023-09-30T14:38:05.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.142+0000 7ff3af6ed700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a819d6f0 con 0x7ff3a81018c0 2023-09-30T14:38:05.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.142+0000 7ff3af6ed700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a819d880 con 0x7ff3a80ffd30 2023-09-30T14:38:05.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff3acc88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a8100f20 0x7ff3a8195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:05.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff3acc88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a8100f20 0x7ff3a8195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34450/0 (socket says 172.21.15.32:34450) 2023-09-30T14:38:05.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff3acc88700 1 -- 172.21.15.32:0/4153619460 learned_addr learned my addr 172.21.15.32:0/4153619460 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:05.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff3ad489700 1 --2- 172.21.15.32:0/4153619460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3a80ffd30 0x7ff3a8195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:05.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff3adc8a700 1 --2- 172.21.15.32:0/4153619460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3a81018c0 0x7ff3a819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:05.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff3acc88700 1 -- 172.21.15.32:0/4153619460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3a80ffd30 msgr2=0x7ff3a8195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:05.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff3acc88700 1 --2- 172.21.15.32:0/4153619460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3a80ffd30 0x7ff3a8195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:05.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff3acc88700 1 -- 172.21.15.32:0/4153619460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3a81018c0 msgr2=0x7ff3a819cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:05.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff3acc88700 1 --2- 172.21.15.32:0/4153619460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3a81018c0 0x7ff3a819cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:05.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff3acc88700 1 -- 172.21.15.32:0/4153619460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a40095f0 con 0x7ff3a8100f20 2023-09-30T14:38:05.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff3ad489700 1 --2- 172.21.15.32:0/4153619460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3a80ffd30 0x7ff3a8195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:05.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff3adc8a700 1 --2- 172.21.15.32:0/4153619460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3a81018c0 0x7ff3a819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:05.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff3acc88700 1 --2- 172.21.15.32:0/4153619460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a8100f20 0x7ff3a8195f70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff39800e9a0 tx=0x7ff39800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff39e7fc700 1 -- 172.21.15.32:0/4153619460 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398009790 con 0x7ff3a8100f20 2023-09-30T14:38:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff39e7fc700 1 -- 172.21.15.32:0/4153619460 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff39800cbd0 con 0x7ff3a8100f20 2023-09-30T14:38:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff3af6ed700 1 -- 172.21.15.32:0/4153619460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a819db60 con 0x7ff3a8100f20 2023-09-30T14:38:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff39e7fc700 1 -- 172.21.15.32:0/4153619460 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398018680 con 0x7ff3a8100f20 2023-09-30T14:38:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.143+0000 7ff3af6ed700 1 -- 172.21.15.32:0/4153619460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a819e070 con 0x7ff3a8100f20 2023-09-30T14:38:05.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.144+0000 7ff3af6ed700 1 -- 172.21.15.32:0/4153619460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a810b540 con 0x7ff3a8100f20 2023-09-30T14:38:05.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.146+0000 7ff39e7fc700 1 -- 172.21.15.32:0/4153619460 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7ff39800cd40 con 0x7ff3a8100f20 2023-09-30T14:38:05.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.147+0000 7ff39e7fc700 1 --2- 172.21.15.32:0/4153619460 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff3940a1030 0x7ff3940a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:05.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.147+0000 7ff3ad489700 1 --2- 172.21.15.32:0/4153619460 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff3940a1030 0x7ff3940a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:05.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.147+0000 7ff39e7fc700 1 -- 172.21.15.32:0/4153619460 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff398014070 con 0x7ff3a8100f20 2023-09-30T14:38:05.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.147+0000 7ff3ad489700 1 --2- 172.21.15.32:0/4153619460 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff3940a1030 0x7ff3940a34f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009990 tx=0x7ff3a401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:05.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.150+0000 7ff39e7fc700 1 -- 172.21.15.32:0/4153619460 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7ff3980966d0 con 0x7ff3a8100f20 2023-09-30T14:38:05.916 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:05 smithi148 ceph-mon[102355]: pgmap v279: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:05.916 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:05 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3138445913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:05.916 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:05 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1469101877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:05.917 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:05 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/39753728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:05.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:05 smithi156 ceph-mon[103579]: pgmap v279: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:05.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:05 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3138445913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:05.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:05 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1469101877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:05.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:05 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/39753728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:05.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.353+0000 7ff3af6ed700 1 -- 172.21.15.32:0/4153619460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3a8106c80 con 0x7ff3a8100f20 2023-09-30T14:38:05.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.353+0000 7ff39e7fc700 1 -- 172.21.15.32:0/4153619460 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff398099cf0 con 0x7ff3a8100f20 2023-09-30T14:38:05.919 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-09-30T14:38:05.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.356+0000 7ff3af6ed700 1 -- 172.21.15.32:0/4153619460 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff3940a1030 msgr2=0x7ff3940a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:05.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.356+0000 7ff3af6ed700 1 --2- 172.21.15.32:0/4153619460 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff3940a1030 0x7ff3940a34f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009990 tx=0x7ff3a401a040 comp rx=0 tx=0).stop 2023-09-30T14:38:05.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.356+0000 7ff3af6ed700 1 -- 172.21.15.32:0/4153619460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a8100f20 msgr2=0x7ff3a8195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:05.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.356+0000 7ff3af6ed700 1 --2- 172.21.15.32:0/4153619460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a8100f20 0x7ff3a8195f70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff39800e9a0 tx=0x7ff39800ed60 comp rx=0 tx=0).stop 2023-09-30T14:38:05.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.356+0000 7ff3af6ed700 1 -- 172.21.15.32:0/4153619460 shutdown_connections 2023-09-30T14:38:05.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.356+0000 7ff3af6ed700 1 --2- 172.21.15.32:0/4153619460 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff3940a1030 0x7ff3940a34f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:05.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.356+0000 7ff3af6ed700 1 --2- 172.21.15.32:0/4153619460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3a81018c0 0x7ff3a819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:05.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.356+0000 7ff3af6ed700 1 --2- 172.21.15.32:0/4153619460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3a80ffd30 0x7ff3a8195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:05.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.356+0000 7ff3af6ed700 1 --2- 172.21.15.32:0/4153619460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3a8100f20 0x7ff3a8195f70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:05.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.356+0000 7ff3af6ed700 1 -- 172.21.15.32:0/4153619460 >> 172.21.15.32:0/4153619460 conn(0x7ff3a80fb2a0 msgr2=0x7ff3a80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:05.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.356+0000 7ff3af6ed700 1 -- 172.21.15.32:0/4153619460 shutdown_connections 2023-09-30T14:38:05.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:05.356+0000 7ff3af6ed700 1 -- 172.21.15.32:0/4153619460 wait complete. 2023-09-30T14:38:05.921 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:05 smithi032 ceph-mon[98592]: pgmap v279: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:05.921 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:05 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3138445913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:05.921 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:05 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1469101877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:05.921 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:05 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/39753728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:05.922 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:38:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.055+0000 7fa406816700 1 -- 172.21.15.32:0/3774763399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa400100340 msgr2=0x7fa4001081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.055+0000 7fa406816700 1 --2- 172.21.15.32:0/3774763399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa400100340 0x7fa4001081d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800b0a0 tx=0x7fa3f800b3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.055+0000 7fa406816700 1 -- 172.21.15.32:0/3774763399 shutdown_connections 2023-09-30T14:38:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.055+0000 7fa406816700 1 --2- 172.21.15.32:0/3774763399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa400100340 0x7fa4001081d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.055+0000 7fa406816700 1 --2- 172.21.15.32:0/3774763399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa400101470 0x7fa400101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.055+0000 7fa406816700 1 --2- 172.21.15.32:0/3774763399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4000ff9a0 0x7fa4000ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.055+0000 7fa406816700 1 -- 172.21.15.32:0/3774763399 >> 172.21.15.32:0/3774763399 conn(0x7fa4000fb2e0 msgr2=0x7fa4000fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.055+0000 7fa406816700 1 -- 172.21.15.32:0/3774763399 shutdown_connections 2023-09-30T14:38:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.055+0000 7fa406816700 1 -- 172.21.15.32:0/3774763399 wait complete. 2023-09-30T14:38:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.055+0000 7fa406816700 1 Processor -- start 2023-09-30T14:38:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.055+0000 7fa406816700 1 -- start start 2023-09-30T14:38:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa406816700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4000ff9a0 0x7fa400198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa406816700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa400100340 0x7fa400199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa406816700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa400101470 0x7fa4001a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa406816700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4001a0aa0 con 0x7fa4000ff9a0 2023-09-30T14:38:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa406816700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4001a0be0 con 0x7fa400101470 2023-09-30T14:38:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa406816700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4001a0d70 con 0x7fa400100340 2023-09-30T14:38:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa3fffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4000ff9a0 0x7fa400198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa3fffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4000ff9a0 0x7fa400198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34460/0 (socket says 172.21.15.32:34460) 2023-09-30T14:38:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa3fffff700 1 -- 172.21.15.32:0/797786188 learned_addr learned my addr 172.21.15.32:0/797786188 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa404db3700 1 --2- 172.21.15.32:0/797786188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa400101470 0x7fa4001a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa3ff7fe700 1 --2- 172.21.15.32:0/797786188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa400100340 0x7fa400199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa3fffff700 1 -- 172.21.15.32:0/797786188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa400100340 msgr2=0x7fa400199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa3fffff700 1 --2- 172.21.15.32:0/797786188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa400100340 0x7fa400199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa3fffff700 1 -- 172.21.15.32:0/797786188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa400101470 msgr2=0x7fa4001a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa3fffff700 1 --2- 172.21.15.32:0/797786188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa400101470 0x7fa4001a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa3fffff700 1 -- 172.21.15.32:0/797786188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f8009d00 con 0x7fa4000ff9a0 2023-09-30T14:38:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa3ff7fe700 1 --2- 172.21.15.32:0/797786188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa400100340 0x7fa400199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa3fffff700 1 --2- 172.21.15.32:0/797786188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4000ff9a0 0x7fa400198f20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400e970 tx=0x7fa3f400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa3fd7fa700 1 -- 172.21.15.32:0/797786188 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4009790 con 0x7fa4000ff9a0 2023-09-30T14:38:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa3fd7fa700 1 -- 172.21.15.32:0/797786188 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3f400eec0 con 0x7fa4000ff9a0 2023-09-30T14:38:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa3fd7fa700 1 -- 172.21.15.32:0/797786188 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4003cf0 con 0x7fa4000ff9a0 2023-09-30T14:38:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa406816700 1 -- 172.21.15.32:0/797786188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4001a0ff0 con 0x7fa4000ff9a0 2023-09-30T14:38:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.056+0000 7fa406816700 1 -- 172.21.15.32:0/797786188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4001a1470 con 0x7fa4000ff9a0 2023-09-30T14:38:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.057+0000 7fa406816700 1 -- 172.21.15.32:0/797786188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40010b6e0 con 0x7fa4000ff9a0 2023-09-30T14:38:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.059+0000 7fa3fd7fa700 1 -- 172.21.15.32:0/797786188 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fa3f400ccd0 con 0x7fa4000ff9a0 2023-09-30T14:38:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.060+0000 7fa3fd7fa700 1 --2- 172.21.15.32:0/797786188 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa3e80a10f0 0x7fa3e80a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.060+0000 7fa3ff7fe700 1 --2- 172.21.15.32:0/797786188 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa3e80a10f0 0x7fa3e80a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.060+0000 7fa3fd7fa700 1 -- 172.21.15.32:0/797786188 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa3f4014070 con 0x7fa4000ff9a0 2023-09-30T14:38:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.060+0000 7fa3ff7fe700 1 --2- 172.21.15.32:0/797786188 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa3e80a10f0 0x7fa3e80a35b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009be0 tx=0x7fa3f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.063+0000 7fa3fd7fa700 1 -- 172.21.15.32:0/797786188 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fa3f4096250 con 0x7fa4000ff9a0 2023-09-30T14:38:06.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.313+0000 7fa406816700 1 -- 172.21.15.32:0/797786188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa40010b920 con 0x7fa4000ff9a0 2023-09-30T14:38:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.314+0000 7fa3fd7fa700 1 -- 172.21.15.32:0/797786188 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11321 (secure 0 0 0) 0x7fa3f4099870 con 0x7fa4000ff9a0 2023-09-30T14:38:06.326 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:06.327 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3505064841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3505064841},{"type":"v1","addr":"172.21.15.32:6839","nonce":3505064841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2262846038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2262846038},{"type":"v1","addr":"172.21.15.32:6841","nonce":2262846038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-30T14:37:11.444993+0000","modified":"2023-09-30T14:37:16.971534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6839/1445176203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1445176203},{"type":"v1","addr":"172.21.15.156:6839","nonce":1445176203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/494821686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":494821686},{"type":"v1","addr":"172.21.15.148:6839","nonce":494821686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T14:38:06.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.316+0000 7fa406816700 1 -- 172.21.15.32:0/797786188 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa3e80a10f0 msgr2=0x7fa3e80a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:06.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.316+0000 7fa406816700 1 --2- 172.21.15.32:0/797786188 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa3e80a10f0 0x7fa3e80a35b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009be0 tx=0x7fa3f0009260 comp rx=0 tx=0).stop 2023-09-30T14:38:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.316+0000 7fa406816700 1 -- 172.21.15.32:0/797786188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4000ff9a0 msgr2=0x7fa400198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.316+0000 7fa406816700 1 --2- 172.21.15.32:0/797786188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4000ff9a0 0x7fa400198f20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400e970 tx=0x7fa3f400ec80 comp rx=0 tx=0).stop 2023-09-30T14:38:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.316+0000 7fa406816700 1 -- 172.21.15.32:0/797786188 shutdown_connections 2023-09-30T14:38:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.316+0000 7fa406816700 1 --2- 172.21.15.32:0/797786188 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa3e80a10f0 0x7fa3e80a35b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.316+0000 7fa406816700 1 --2- 172.21.15.32:0/797786188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa400101470 0x7fa4001a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.316+0000 7fa406816700 1 --2- 172.21.15.32:0/797786188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4000ff9a0 0x7fa400198f20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.316+0000 7fa406816700 1 --2- 172.21.15.32:0/797786188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa400100340 0x7fa400199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.316+0000 7fa406816700 1 -- 172.21.15.32:0/797786188 >> 172.21.15.32:0/797786188 conn(0x7fa4000fb2e0 msgr2=0x7fa400068fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.316+0000 7fa406816700 1 -- 172.21.15.32:0/797786188 shutdown_connections 2023-09-30T14:38:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.316+0000 7fa406816700 1 -- 172.21.15.32:0/797786188 wait complete. 2023-09-30T14:38:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 22 2023-09-30T14:38:06.330 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:06 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/4153619460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:06.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T14:38:06.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.457+0000 7f622174a700 1 -- 172.21.15.32:0/2791072576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621c1018c0 msgr2=0x7f621c108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:06.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.457+0000 7f622174a700 1 --2- 172.21.15.32:0/2791072576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621c1018c0 0x7f621c108170 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f621400b0d0 tx=0x7f621400b3e0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.457+0000 7f622174a700 1 -- 172.21.15.32:0/2791072576 shutdown_connections 2023-09-30T14:38:06.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.457+0000 7f622174a700 1 --2- 172.21.15.32:0/2791072576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621c1018c0 0x7f621c108170 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.457+0000 7f622174a700 1 --2- 172.21.15.32:0/2791072576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f621c0ffd30 0x7f621c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.457+0000 7f622174a700 1 --2- 172.21.15.32:0/2791072576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f621c100f20 0x7f621c101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.457+0000 7f622174a700 1 -- 172.21.15.32:0/2791072576 >> 172.21.15.32:0/2791072576 conn(0x7f621c0fb280 msgr2=0x7f621c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:06.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.457+0000 7f622174a700 1 -- 172.21.15.32:0/2791072576 shutdown_connections 2023-09-30T14:38:06.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.457+0000 7f622174a700 1 -- 172.21.15.32:0/2791072576 wait complete. 2023-09-30T14:38:06.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.457+0000 7f622174a700 1 Processor -- start 2023-09-30T14:38:06.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.457+0000 7f622174a700 1 -- start start 2023-09-30T14:38:06.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f622174a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621c0ffd30 0x7f621c069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:06.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f622174a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f621c100f20 0x7f621c19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:06.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f622174a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f621c1018c0 0x7f621c19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:06.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f622174a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c19e6e0 con 0x7f621c0ffd30 2023-09-30T14:38:06.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f622174a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c19e850 con 0x7f621c1018c0 2023-09-30T14:38:06.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f622174a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c19ea10 con 0x7f621c100f20 2023-09-30T14:38:06.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f621affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621c0ffd30 0x7f621c069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:06.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f621affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621c0ffd30 0x7f621c069e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34474/0 (socket says 172.21.15.32:34474) 2023-09-30T14:38:06.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f621affd700 1 -- 172.21.15.32:0/1699487212 learned_addr learned my addr 172.21.15.32:0/1699487212 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:06.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f621a7fc700 1 --2- 172.21.15.32:0/1699487212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f621c100f20 0x7f621c19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:06.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f621b7fe700 1 --2- 172.21.15.32:0/1699487212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f621c1018c0 0x7f621c19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:06.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f621affd700 1 -- 172.21.15.32:0/1699487212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f621c100f20 msgr2=0x7f621c19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:06.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f621affd700 1 --2- 172.21.15.32:0/1699487212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f621c100f20 0x7f621c19b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f621affd700 1 -- 172.21.15.32:0/1699487212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f621c1018c0 msgr2=0x7f621c19df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:06.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f621affd700 1 --2- 172.21.15.32:0/1699487212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f621c1018c0 0x7f621c19df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f621affd700 1 -- 172.21.15.32:0/1699487212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6214009d00 con 0x7f621c0ffd30 2023-09-30T14:38:06.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f621a7fc700 1 --2- 172.21.15.32:0/1699487212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f621c100f20 0x7f621c19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:06.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f621affd700 1 --2- 172.21.15.32:0/1699487212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621c0ffd30 0x7f621c069e00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f621000d980 tx=0x7f621000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:06.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f6203fff700 1 -- 172.21.15.32:0/1699487212 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621000de60 con 0x7f621c0ffd30 2023-09-30T14:38:06.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f6203fff700 1 -- 172.21.15.32:0/1699487212 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62100044e0 con 0x7f621c0ffd30 2023-09-30T14:38:06.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f6203fff700 1 -- 172.21.15.32:0/1699487212 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210003be0 con 0x7f621c0ffd30 2023-09-30T14:38:06.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.458+0000 7f622174a700 1 -- 172.21.15.32:0/1699487212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621c19ecf0 con 0x7f621c0ffd30 2023-09-30T14:38:06.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.459+0000 7f622174a700 1 -- 172.21.15.32:0/1699487212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621c19f1b0 con 0x7f621c0ffd30 2023-09-30T14:38:06.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.459+0000 7f622174a700 1 -- 172.21.15.32:0/1699487212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621c10b5a0 con 0x7f621c0ffd30 2023-09-30T14:38:06.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.461+0000 7f6203fff700 1 -- 172.21.15.32:0/1699487212 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f6210009bb0 con 0x7f621c0ffd30 2023-09-30T14:38:06.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.461+0000 7f6203fff700 1 --2- 172.21.15.32:0/1699487212 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f62040a10f0 0x7f62040a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:06.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.461+0000 7f621a7fc700 1 --2- 172.21.15.32:0/1699487212 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f62040a10f0 0x7f62040a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:06.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.461+0000 7f6203fff700 1 -- 172.21.15.32:0/1699487212 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f62100cc740 con 0x7f621c0ffd30 2023-09-30T14:38:06.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.462+0000 7f621a7fc700 1 --2- 172.21.15.32:0/1699487212 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f62040a10f0 0x7f62040a35b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f620c005fd0 tx=0x7f620c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:06.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.465+0000 7f6203fff700 1 -- 172.21.15.32:0/1699487212 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f6210096010 con 0x7f621c0ffd30 2023-09-30T14:38:06.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:06 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/4153619460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:06.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:06 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/4153619460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:06.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.667+0000 7f622174a700 1 -- 172.21.15.32:0/1699487212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f621c10b810 con 0x7f621c0ffd30 2023-09-30T14:38:06.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.669+0000 7f6203fff700 1 -- 172.21.15.32:0/1699487212 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22133 (secure 0 0 0) 0x7f6210009940 con 0x7f621c0ffd30 2023-09-30T14:38:06.732 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:06.732 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":83,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:37:12.972217+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:38:06.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.671+0000 7f622174a700 1 -- 172.21.15.32:0/1699487212 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f62040a10f0 msgr2=0x7f62040a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:06.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.671+0000 7f622174a700 1 --2- 172.21.15.32:0/1699487212 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f62040a10f0 0x7f62040a35b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f620c005fd0 tx=0x7f620c005e30 comp rx=0 tx=0).stop 2023-09-30T14:38:06.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.671+0000 7f622174a700 1 -- 172.21.15.32:0/1699487212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621c0ffd30 msgr2=0x7f621c069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:06.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.671+0000 7f622174a700 1 --2- 172.21.15.32:0/1699487212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621c0ffd30 0x7f621c069e00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f621000d980 tx=0x7f621000dd40 comp rx=0 tx=0).stop 2023-09-30T14:38:06.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.671+0000 7f622174a700 1 -- 172.21.15.32:0/1699487212 shutdown_connections 2023-09-30T14:38:06.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.671+0000 7f622174a700 1 --2- 172.21.15.32:0/1699487212 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f62040a10f0 0x7f62040a35b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.671+0000 7f622174a700 1 --2- 172.21.15.32:0/1699487212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f621c1018c0 0x7f621c19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.671+0000 7f622174a700 1 --2- 172.21.15.32:0/1699487212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f621c0ffd30 0x7f621c069e00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.671+0000 7f622174a700 1 --2- 172.21.15.32:0/1699487212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f621c100f20 0x7f621c19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.671+0000 7f622174a700 1 -- 172.21.15.32:0/1699487212 >> 172.21.15.32:0/1699487212 conn(0x7f621c0fb280 msgr2=0x7f621c0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.671+0000 7f622174a700 1 -- 172.21.15.32:0/1699487212 shutdown_connections 2023-09-30T14:38:06.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:06.671+0000 7f622174a700 1 -- 172.21.15.32:0/1699487212 wait complete. 2023-09-30T14:38:06.888 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T14:33:20.224529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T14:37:09.573219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T14:37:10.844458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T14:38:06.889 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T14:38:07.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.021+0000 7eff1391d700 1 -- 172.21.15.32:0/1358114064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c101fc0 msgr2=0x7eff0c102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:07.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.021+0000 7eff1391d700 1 --2- 172.21.15.32:0/1358114064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c101fc0 0x7eff0c102440 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efefc009990 tx=0x7efefc009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.021+0000 7eff1391d700 1 -- 172.21.15.32:0/1358114064 shutdown_connections 2023-09-30T14:38:07.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.021+0000 7eff1391d700 1 --2- 172.21.15.32:0/1358114064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff0c102980 0x7eff0c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.021+0000 7eff1391d700 1 --2- 172.21.15.32:0/1358114064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0c100e70 0x7eff0c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.021+0000 7eff1391d700 1 --2- 172.21.15.32:0/1358114064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c101fc0 0x7eff0c102440 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.021+0000 7eff1391d700 1 -- 172.21.15.32:0/1358114064 >> 172.21.15.32:0/1358114064 conn(0x7eff0c0fc3e0 msgr2=0x7eff0c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:07.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.021+0000 7eff1391d700 1 -- 172.21.15.32:0/1358114064 shutdown_connections 2023-09-30T14:38:07.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.021+0000 7eff1391d700 1 -- 172.21.15.32:0/1358114064 wait complete. 2023-09-30T14:38:07.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.022+0000 7eff1391d700 1 Processor -- start 2023-09-30T14:38:07.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.022+0000 7eff1391d700 1 -- start start 2023-09-30T14:38:07.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.022+0000 7eff1391d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c100e70 0x7eff0c198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:07.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.022+0000 7eff1391d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0c101fc0 0x7eff0c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:07.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.022+0000 7eff1391d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff0c102980 0x7eff0c1a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:07.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.022+0000 7eff1391d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c1a08d0 con 0x7eff0c100e70 2023-09-30T14:38:07.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.022+0000 7eff1391d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c1a0a10 con 0x7eff0c101fc0 2023-09-30T14:38:07.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.022+0000 7eff1391d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c1a0ba0 con 0x7eff0c102980 2023-09-30T14:38:07.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.022+0000 7eff116b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c100e70 0x7eff0c198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:07.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.022+0000 7eff116b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c100e70 0x7eff0c198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34488/0 (socket says 172.21.15.32:34488) 2023-09-30T14:38:07.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.022+0000 7eff116b9700 1 -- 172.21.15.32:0/1228889247 learned_addr learned my addr 172.21.15.32:0/1228889247 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:07.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.022+0000 7eff11eba700 1 --2- 172.21.15.32:0/1228889247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff0c102980 0x7eff0c1a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:07.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.022+0000 7eff10eb8700 1 --2- 172.21.15.32:0/1228889247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0c101fc0 0x7eff0c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:07.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.023+0000 7eff116b9700 1 -- 172.21.15.32:0/1228889247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff0c102980 msgr2=0x7eff0c1a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:07.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.023+0000 7eff116b9700 1 --2- 172.21.15.32:0/1228889247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff0c102980 0x7eff0c1a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.023+0000 7eff116b9700 1 -- 172.21.15.32:0/1228889247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0c101fc0 msgr2=0x7eff0c199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:07.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.023+0000 7eff116b9700 1 --2- 172.21.15.32:0/1228889247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0c101fc0 0x7eff0c199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.023+0000 7eff116b9700 1 -- 172.21.15.32:0/1228889247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc0095f0 con 0x7eff0c100e70 2023-09-30T14:38:07.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.023+0000 7eff11eba700 1 --2- 172.21.15.32:0/1228889247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff0c102980 0x7eff0c1a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:07.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.023+0000 7eff116b9700 1 --2- 172.21.15.32:0/1228889247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c100e70 0x7eff0c198d50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7eff0800d950 tx=0x7eff0800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:07.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.023+0000 7eff027fc700 1 -- 172.21.15.32:0/1228889247 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08009b40 con 0x7eff0c100e70 2023-09-30T14:38:07.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.023+0000 7eff027fc700 1 -- 172.21.15.32:0/1228889247 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff08009ca0 con 0x7eff0c100e70 2023-09-30T14:38:07.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.023+0000 7eff1391d700 1 -- 172.21.15.32:0/1228889247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0c1a0e80 con 0x7eff0c100e70 2023-09-30T14:38:07.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.023+0000 7eff027fc700 1 -- 172.21.15.32:0/1228889247 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0800f620 con 0x7eff0c100e70 2023-09-30T14:38:07.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.023+0000 7eff1391d700 1 -- 172.21.15.32:0/1228889247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0c1a12a0 con 0x7eff0c100e70 2023-09-30T14:38:07.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.024+0000 7eff1391d700 1 -- 172.21.15.32:0/1228889247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0c10c670 con 0x7eff0c100e70 2023-09-30T14:38:07.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.026+0000 7eff027fc700 1 -- 172.21.15.32:0/1228889247 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7eff08010460 con 0x7eff0c100e70 2023-09-30T14:38:07.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.027+0000 7eff027fc700 1 --2- 172.21.15.32:0/1228889247 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7efef80a1030 0x7efef80a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:07.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.027+0000 7eff027fc700 1 -- 172.21.15.32:0/1228889247 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7eff080cca20 con 0x7eff0c100e70 2023-09-30T14:38:07.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.027+0000 7eff10eb8700 1 --2- 172.21.15.32:0/1228889247 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7efef80a1030 0x7efef80a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:07.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.027+0000 7eff10eb8700 1 --2- 172.21.15.32:0/1228889247 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7efef80a1030 0x7efef80a34f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efefc006010 tx=0x7efefc00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:07.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.030+0000 7eff027fc700 1 -- 172.21.15.32:0/1228889247 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7eff08096270 con 0x7eff0c100e70 2023-09-30T14:38:07.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.235+0000 7eff1391d700 1 -- 172.21.15.32:0/1228889247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7eff0c1a1490 con 0x7eff0c100e70 2023-09-30T14:38:07.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.236+0000 7eff027fc700 1 -- 172.21.15.32:0/1228889247 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7eff080200a0 con 0x7eff0c100e70 2023-09-30T14:38:07.236 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T14:38:07.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.238+0000 7eff1391d700 1 -- 172.21.15.32:0/1228889247 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7efef80a1030 msgr2=0x7efef80a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:07.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.238+0000 7eff1391d700 1 --2- 172.21.15.32:0/1228889247 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7efef80a1030 0x7efef80a34f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efefc006010 tx=0x7efefc00b540 comp rx=0 tx=0).stop 2023-09-30T14:38:07.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.238+0000 7eff1391d700 1 -- 172.21.15.32:0/1228889247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c100e70 msgr2=0x7eff0c198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.238+0000 7eff1391d700 1 --2- 172.21.15.32:0/1228889247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c100e70 0x7eff0c198d50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7eff0800d950 tx=0x7eff0800dd10 comp rx=0 tx=0).stop 2023-09-30T14:38:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.238+0000 7eff1391d700 1 -- 172.21.15.32:0/1228889247 shutdown_connections 2023-09-30T14:38:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.238+0000 7eff1391d700 1 --2- 172.21.15.32:0/1228889247 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7efef80a1030 0x7efef80a34f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.238+0000 7eff1391d700 1 --2- 172.21.15.32:0/1228889247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff0c102980 0x7eff0c1a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.238+0000 7eff1391d700 1 --2- 172.21.15.32:0/1228889247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff0c100e70 0x7eff0c198d50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.238+0000 7eff1391d700 1 --2- 172.21.15.32:0/1228889247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff0c101fc0 0x7eff0c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.238+0000 7eff1391d700 1 -- 172.21.15.32:0/1228889247 >> 172.21.15.32:0/1228889247 conn(0x7eff0c0fc3e0 msgr2=0x7eff0c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.238+0000 7eff1391d700 1 -- 172.21.15.32:0/1228889247 shutdown_connections 2023-09-30T14:38:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.238+0000 7eff1391d700 1 -- 172.21.15.32:0/1228889247 wait complete. 2023-09-30T14:38:07.250 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T14:38:07.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.379+0000 7fc2da710700 1 -- 172.21.15.32:0/4076787359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d4100f20 msgr2=0x7fc2d4101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:07.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.379+0000 7fc2da710700 1 --2- 172.21.15.32:0/4076787359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d4100f20 0x7fc2d4101380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80099c0 tx=0x7fc2c8009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.379+0000 7fc2da710700 1 -- 172.21.15.32:0/4076787359 shutdown_connections 2023-09-30T14:38:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.379+0000 7fc2da710700 1 --2- 172.21.15.32:0/4076787359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d41018c0 0x7fc2d4108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.379+0000 7fc2da710700 1 --2- 172.21.15.32:0/4076787359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d40ffd30 0x7fc2d4100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.379+0000 7fc2da710700 1 --2- 172.21.15.32:0/4076787359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d4100f20 0x7fc2d4101380 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.379+0000 7fc2da710700 1 -- 172.21.15.32:0/4076787359 >> 172.21.15.32:0/4076787359 conn(0x7fc2d40fb2a0 msgr2=0x7fc2d40fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.379+0000 7fc2da710700 1 -- 172.21.15.32:0/4076787359 shutdown_connections 2023-09-30T14:38:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.379+0000 7fc2da710700 1 -- 172.21.15.32:0/4076787359 wait complete. 2023-09-30T14:38:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2da710700 1 Processor -- start 2023-09-30T14:38:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2da710700 1 -- start start 2023-09-30T14:38:07.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2da710700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d40ffd30 0x7fc2d4195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:07.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2da710700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d4100f20 0x7fc2d4195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:07.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2da710700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d41018c0 0x7fc2d419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:07.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2da710700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d419d5c0 con 0x7fc2d40ffd30 2023-09-30T14:38:07.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2da710700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d419d700 con 0x7fc2d41018c0 2023-09-30T14:38:07.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2da710700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d419d890 con 0x7fc2d4100f20 2023-09-30T14:38:07.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2d3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d40ffd30 0x7fc2d4195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:07.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2d3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d40ffd30 0x7fc2d4195a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34502/0 (socket says 172.21.15.32:34502) 2023-09-30T14:38:07.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2d3fff700 1 -- 172.21.15.32:0/1349900363 learned_addr learned my addr 172.21.15.32:0/1349900363 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:07.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2d3fff700 1 -- 172.21.15.32:0/1349900363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d4100f20 msgr2=0x7fc2d4195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:07.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2d3fff700 1 --2- 172.21.15.32:0/1349900363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d4100f20 0x7fc2d4195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2d8cad700 1 --2- 172.21.15.32:0/1349900363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d41018c0 0x7fc2d419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:07.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2d3fff700 1 -- 172.21.15.32:0/1349900363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d41018c0 msgr2=0x7fc2d419cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:07.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2d3fff700 1 --2- 172.21.15.32:0/1349900363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d41018c0 0x7fc2d419cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2d3fff700 1 -- 172.21.15.32:0/1349900363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c80095f0 con 0x7fc2d40ffd30 2023-09-30T14:38:07.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.380+0000 7fc2d8cad700 1 --2- 172.21.15.32:0/1349900363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d41018c0 0x7fc2d419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:07.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.381+0000 7fc2d3fff700 1 --2- 172.21.15.32:0/1349900363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d40ffd30 0x7fc2d4195a40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400d980 tx=0x7fc2c400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:07.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.381+0000 7fc2d17fa700 1 -- 172.21.15.32:0/1349900363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c400bb60 con 0x7fc2d40ffd30 2023-09-30T14:38:07.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.381+0000 7fc2da710700 1 -- 172.21.15.32:0/1349900363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d419db70 con 0x7fc2d40ffd30 2023-09-30T14:38:07.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.381+0000 7fc2d17fa700 1 -- 172.21.15.32:0/1349900363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2c4007620 con 0x7fc2d40ffd30 2023-09-30T14:38:07.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.381+0000 7fc2d17fa700 1 -- 172.21.15.32:0/1349900363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c400f620 con 0x7fc2d40ffd30 2023-09-30T14:38:07.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.381+0000 7fc2da710700 1 -- 172.21.15.32:0/1349900363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d419e0d0 con 0x7fc2d40ffd30 2023-09-30T14:38:07.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.382+0000 7fc2da710700 1 -- 172.21.15.32:0/1349900363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d410b540 con 0x7fc2d40ffd30 2023-09-30T14:38:07.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.383+0000 7fc2d17fa700 1 -- 172.21.15.32:0/1349900363 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fc2c4010460 con 0x7fc2d40ffd30 2023-09-30T14:38:07.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.384+0000 7fc2d17fa700 1 --2- 172.21.15.32:0/1349900363 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2bc0a1030 0x7fc2bc0a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:07.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.384+0000 7fc2d17fa700 1 -- 172.21.15.32:0/1349900363 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc2c40cd0e0 con 0x7fc2d40ffd30 2023-09-30T14:38:07.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.384+0000 7fc2d37fe700 1 --2- 172.21.15.32:0/1349900363 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2bc0a1030 0x7fc2bc0a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:07.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.384+0000 7fc2d37fe700 1 --2- 172.21.15.32:0/1349900363 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2bc0a1030 0x7fc2bc0a34f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80036f0 tx=0x7fc2c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.387+0000 7fc2d17fa700 1 -- 172.21.15.32:0/1349900363 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fc2c4096830 con 0x7fc2d40ffd30 2023-09-30T14:38:07.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:07 smithi156 ceph-mon[103579]: pgmap v280: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:38:07.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:07 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/797786188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:07.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:07 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1699487212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.588+0000 7fc2da710700 1 -- 172.21.15.32:0/1349900363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc2d4106cd0 con 0x7fc2d40ffd30 2023-09-30T14:38:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.588+0000 7fc2d17fa700 1 -- 172.21.15.32:0/1349900363 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2c400de60 con 0x7fc2d40ffd30 2023-09-30T14:38:07.588 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-09-30T14:38:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.590+0000 7fc2da710700 1 -- 172.21.15.32:0/1349900363 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2bc0a1030 msgr2=0x7fc2bc0a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:07.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.590+0000 7fc2da710700 1 --2- 172.21.15.32:0/1349900363 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2bc0a1030 0x7fc2bc0a34f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80036f0 tx=0x7fc2c8000bc0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.590+0000 7fc2da710700 1 -- 172.21.15.32:0/1349900363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d40ffd30 msgr2=0x7fc2d4195a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:07.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.590+0000 7fc2da710700 1 --2- 172.21.15.32:0/1349900363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d40ffd30 0x7fc2d4195a40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400d980 tx=0x7fc2c400dd40 comp rx=0 tx=0).stop 2023-09-30T14:38:07.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.591+0000 7fc2da710700 1 -- 172.21.15.32:0/1349900363 shutdown_connections 2023-09-30T14:38:07.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.591+0000 7fc2da710700 1 --2- 172.21.15.32:0/1349900363 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2bc0a1030 0x7fc2bc0a34f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.591+0000 7fc2da710700 1 --2- 172.21.15.32:0/1349900363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d41018c0 0x7fc2d419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.591+0000 7fc2da710700 1 --2- 172.21.15.32:0/1349900363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2d40ffd30 0x7fc2d4195a40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.591+0000 7fc2da710700 1 --2- 172.21.15.32:0/1349900363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d4100f20 0x7fc2d4195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.591+0000 7fc2da710700 1 -- 172.21.15.32:0/1349900363 >> 172.21.15.32:0/1349900363 conn(0x7fc2d40fb2a0 msgr2=0x7fc2d40fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.591+0000 7fc2da710700 1 -- 172.21.15.32:0/1349900363 shutdown_connections 2023-09-30T14:38:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.591+0000 7fc2da710700 1 -- 172.21.15.32:0/1349900363 wait complete. 2023-09-30T14:38:07.602 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:07 smithi032 ceph-mon[98592]: pgmap v280: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:38:07.602 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:07 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/797786188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:07.602 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:07 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1699487212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:07.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T14:38:07.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:07 smithi148 ceph-mon[102355]: pgmap v280: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:38:07.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:07 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/797786188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:07.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:07 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1699487212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:07.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.732+0000 7f7dc384b700 1 -- 172.21.15.32:0/563696916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc1018c0 msgr2=0x7f7dbc108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:07.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.732+0000 7f7dc384b700 1 --2- 172.21.15.32:0/563696916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc1018c0 0x7f7dbc108170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7db400b0d0 tx=0x7f7db400b3e0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.732+0000 7f7dc384b700 1 -- 172.21.15.32:0/563696916 shutdown_connections 2023-09-30T14:38:07.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.732+0000 7f7dc384b700 1 --2- 172.21.15.32:0/563696916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc1018c0 0x7f7dbc108170 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.732+0000 7f7dc384b700 1 --2- 172.21.15.32:0/563696916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dbc0ffd30 0x7f7dbc100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.732+0000 7f7dc384b700 1 --2- 172.21.15.32:0/563696916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dbc100f20 0x7f7dbc101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.732+0000 7f7dc384b700 1 -- 172.21.15.32:0/563696916 >> 172.21.15.32:0/563696916 conn(0x7f7dbc0fb2a0 msgr2=0x7f7dbc0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:07.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.733+0000 7f7dc384b700 1 -- 172.21.15.32:0/563696916 shutdown_connections 2023-09-30T14:38:07.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.733+0000 7f7dc384b700 1 -- 172.21.15.32:0/563696916 wait complete. 2023-09-30T14:38:07.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.733+0000 7f7dc384b700 1 Processor -- start 2023-09-30T14:38:07.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.733+0000 7f7dc384b700 1 -- start start 2023-09-30T14:38:07.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.733+0000 7f7dc384b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dbc0ffd30 0x7f7dbc1949a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:07.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.733+0000 7f7dc384b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dbc100f20 0x7f7dbc194ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:07.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.733+0000 7f7dc384b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc1018c0 0x7f7dbc19bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:07.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.733+0000 7f7dc384b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc19c520 con 0x7f7dbc1018c0 2023-09-30T14:38:07.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.733+0000 7f7dc384b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc19c660 con 0x7f7dbc100f20 2023-09-30T14:38:07.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.733+0000 7f7dc384b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc19c7f0 con 0x7f7dbc0ffd30 2023-09-30T14:38:07.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.733+0000 7f7dc1de8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc1018c0 0x7f7dbc19bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:07.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.733+0000 7f7dc1de8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc1018c0 0x7f7dbc19bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34520/0 (socket says 172.21.15.32:34520) 2023-09-30T14:38:07.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.733+0000 7f7dc1de8700 1 -- 172.21.15.32:0/1122850559 learned_addr learned my addr 172.21.15.32:0/1122850559 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:07.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.734+0000 7f7dc0de6700 1 --2- 172.21.15.32:0/1122850559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dbc100f20 0x7f7dbc194ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:07.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.734+0000 7f7dc1de8700 1 -- 172.21.15.32:0/1122850559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dbc0ffd30 msgr2=0x7f7dbc1949a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:07.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.734+0000 7f7dc1de8700 1 --2- 172.21.15.32:0/1122850559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dbc0ffd30 0x7f7dbc1949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.734+0000 7f7dc1de8700 1 -- 172.21.15.32:0/1122850559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dbc100f20 msgr2=0x7f7dbc194ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:07.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.734+0000 7f7dc1de8700 1 --2- 172.21.15.32:0/1122850559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dbc100f20 0x7f7dbc194ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.734+0000 7f7dc1de8700 1 -- 172.21.15.32:0/1122850559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db4009d00 con 0x7f7dbc1018c0 2023-09-30T14:38:07.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.734+0000 7f7dc1de8700 1 --2- 172.21.15.32:0/1122850559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc1018c0 0x7f7dbc19bf50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7db4007a00 tx=0x7f7db40126d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:07.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.734+0000 7f7dae7fc700 1 -- 172.21.15.32:0/1122850559 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db400e040 con 0x7f7dbc1018c0 2023-09-30T14:38:07.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.734+0000 7f7dae7fc700 1 -- 172.21.15.32:0/1122850559 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7db400ceb0 con 0x7f7dbc1018c0 2023-09-30T14:38:07.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.734+0000 7f7dc384b700 1 -- 172.21.15.32:0/1122850559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dbc19ca70 con 0x7f7dbc1018c0 2023-09-30T14:38:07.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.734+0000 7f7dae7fc700 1 -- 172.21.15.32:0/1122850559 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db401bb90 con 0x7f7dbc1018c0 2023-09-30T14:38:07.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.734+0000 7f7dc384b700 1 -- 172.21.15.32:0/1122850559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dbc19cef0 con 0x7f7dbc1018c0 2023-09-30T14:38:07.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.735+0000 7f7dc384b700 1 -- 172.21.15.32:0/1122850559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dbc10a540 con 0x7f7dbc1018c0 2023-09-30T14:38:07.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.736+0000 7f7dae7fc700 1 -- 172.21.15.32:0/1122850559 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f7db401bd10 con 0x7f7dbc1018c0 2023-09-30T14:38:07.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.737+0000 7f7dae7fc700 1 --2- 172.21.15.32:0/1122850559 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7da80a10b0 0x7f7da80a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:07.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.737+0000 7f7dc15e7700 1 --2- 172.21.15.32:0/1122850559 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7da80a10b0 0x7f7da80a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:07.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.737+0000 7f7dae7fc700 1 -- 172.21.15.32:0/1122850559 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7db40ce400 con 0x7f7dbc1018c0 2023-09-30T14:38:07.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.737+0000 7f7dc15e7700 1 --2- 172.21.15.32:0/1122850559 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7da80a10b0 0x7f7da80a3570 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc100d80 tx=0x7f7db0005ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.741+0000 7f7dae7fc700 1 -- 172.21.15.32:0/1122850559 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f7db4097ce0 con 0x7f7dbc1018c0 2023-09-30T14:38:07.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.943+0000 7f7dc384b700 1 -- 172.21.15.32:0/1122850559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7dbc106b20 con 0x7f7dbc1018c0 2023-09-30T14:38:07.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.944+0000 7f7dae7fc700 1 -- 172.21.15.32:0/1122850559 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7db409b300 con 0x7f7dbc1018c0 2023-09-30T14:38:07.944 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-09-30T14:38:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.946+0000 7f7dc384b700 1 -- 172.21.15.32:0/1122850559 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7da80a10b0 msgr2=0x7f7da80a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.946+0000 7f7dc384b700 1 --2- 172.21.15.32:0/1122850559 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7da80a10b0 0x7f7da80a3570 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc100d80 tx=0x7f7db0005ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.946+0000 7f7dc384b700 1 -- 172.21.15.32:0/1122850559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc1018c0 msgr2=0x7f7dbc19bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.946+0000 7f7dc384b700 1 --2- 172.21.15.32:0/1122850559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc1018c0 0x7f7dbc19bf50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7db4007a00 tx=0x7f7db40126d0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.946+0000 7f7dc384b700 1 -- 172.21.15.32:0/1122850559 shutdown_connections 2023-09-30T14:38:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.946+0000 7f7dc384b700 1 --2- 172.21.15.32:0/1122850559 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7da80a10b0 0x7f7da80a3570 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.946+0000 7f7dc384b700 1 --2- 172.21.15.32:0/1122850559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc1018c0 0x7f7dbc19bf50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.946+0000 7f7dc384b700 1 --2- 172.21.15.32:0/1122850559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dbc0ffd30 0x7f7dbc1949a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.946+0000 7f7dc384b700 1 --2- 172.21.15.32:0/1122850559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dbc100f20 0x7f7dbc194ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.946+0000 7f7dc384b700 1 -- 172.21.15.32:0/1122850559 >> 172.21.15.32:0/1122850559 conn(0x7f7dbc0fb2a0 msgr2=0x7f7dbc0fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:07.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.946+0000 7f7dc384b700 1 -- 172.21.15.32:0/1122850559 shutdown_connections 2023-09-30T14:38:07.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:07.946+0000 7f7dc384b700 1 -- 172.21.15.32:0/1122850559 wait complete. 2023-09-30T14:38:07.958 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:38:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.087+0000 7fdbf0098700 1 -- 172.21.15.32:0/2325199736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbe81018c0 msgr2=0x7fdbe8108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.087+0000 7fdbf0098700 1 --2- 172.21.15.32:0/2325199736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbe81018c0 0x7fdbe8108170 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000b0d0 tx=0x7fdbe000b3e0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.087+0000 7fdbf0098700 1 -- 172.21.15.32:0/2325199736 shutdown_connections 2023-09-30T14:38:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.087+0000 7fdbf0098700 1 --2- 172.21.15.32:0/2325199736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbe81018c0 0x7fdbe8108170 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.087+0000 7fdbf0098700 1 --2- 172.21.15.32:0/2325199736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbe80ffd30 0x7fdbe8100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.087+0000 7fdbf0098700 1 --2- 172.21.15.32:0/2325199736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8100f20 0x7fdbe8101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.087+0000 7fdbf0098700 1 -- 172.21.15.32:0/2325199736 >> 172.21.15.32:0/2325199736 conn(0x7fdbe80fb280 msgr2=0x7fdbe80fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.087+0000 7fdbf0098700 1 -- 172.21.15.32:0/2325199736 shutdown_connections 2023-09-30T14:38:08.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.087+0000 7fdbf0098700 1 -- 172.21.15.32:0/2325199736 wait complete. 2023-09-30T14:38:08.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbf0098700 1 Processor -- start 2023-09-30T14:38:08.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbf0098700 1 -- start start 2023-09-30T14:38:08.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbf0098700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbe80ffd30 0x7fdbe8198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:08.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbf0098700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8100f20 0x7fdbe8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:08.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbf0098700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbe81018c0 0x7fdbe81a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:08.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbf0098700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe81a0950 con 0x7fdbe81018c0 2023-09-30T14:38:08.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbf0098700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe81a0a90 con 0x7fdbe8100f20 2023-09-30T14:38:08.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbf0098700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe81a0c20 con 0x7fdbe80ffd30 2023-09-30T14:38:08.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbee635700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbe81018c0 0x7fdbe81a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:08.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbee635700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbe81018c0 0x7fdbe81a0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34534/0 (socket says 172.21.15.32:34534) 2023-09-30T14:38:08.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbee635700 1 -- 172.21.15.32:0/235389316 learned_addr learned my addr 172.21.15.32:0/235389316 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbed633700 1 --2- 172.21.15.32:0/235389316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8100f20 0x7fdbe8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbede34700 1 --2- 172.21.15.32:0/235389316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbe80ffd30 0x7fdbe8198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbee635700 1 -- 172.21.15.32:0/235389316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbe80ffd30 msgr2=0x7fdbe8198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbee635700 1 --2- 172.21.15.32:0/235389316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbe80ffd30 0x7fdbe8198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbee635700 1 -- 172.21.15.32:0/235389316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8100f20 msgr2=0x7fdbe8199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbee635700 1 --2- 172.21.15.32:0/235389316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8100f20 0x7fdbe8199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbee635700 1 -- 172.21.15.32:0/235389316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe0009d00 con 0x7fdbe81018c0 2023-09-30T14:38:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbed633700 1 --2- 172.21.15.32:0/235389316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8100f20 0x7fdbe8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.088+0000 7fdbee635700 1 --2- 172.21.15.32:0/235389316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbe81018c0 0x7fdbe81a0380 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0007a00 tx=0x7fdbe0009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.089+0000 7fdbdeffd700 1 -- 172.21.15.32:0/235389316 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000e040 con 0x7fdbe81018c0 2023-09-30T14:38:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.089+0000 7fdbdeffd700 1 -- 172.21.15.32:0/235389316 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbe0003e10 con 0x7fdbe81018c0 2023-09-30T14:38:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.089+0000 7fdbf0098700 1 -- 172.21.15.32:0/235389316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe81a0ea0 con 0x7fdbe81018c0 2023-09-30T14:38:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.089+0000 7fdbdeffd700 1 -- 172.21.15.32:0/235389316 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0005610 con 0x7fdbe81018c0 2023-09-30T14:38:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.089+0000 7fdbf0098700 1 -- 172.21.15.32:0/235389316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe81a1210 con 0x7fdbe81018c0 2023-09-30T14:38:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.090+0000 7fdbf0098700 1 -- 172.21.15.32:0/235389316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe810b550 con 0x7fdbe81018c0 2023-09-30T14:38:08.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.091+0000 7fdbdeffd700 1 -- 172.21.15.32:0/235389316 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fdbe0005770 con 0x7fdbe81018c0 2023-09-30T14:38:08.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.091+0000 7fdbdeffd700 1 --2- 172.21.15.32:0/235389316 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdbd40a1070 0x7fdbd40a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:08.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.092+0000 7fdbede34700 1 --2- 172.21.15.32:0/235389316 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdbd40a1070 0x7fdbd40a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:08.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.092+0000 7fdbdeffd700 1 -- 172.21.15.32:0/235389316 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdbe00ce430 con 0x7fdbe81018c0 2023-09-30T14:38:08.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.092+0000 7fdbede34700 1 --2- 172.21.15.32:0/235389316 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdbd40a1070 0x7fdbd40a3530 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4005fd0 tx=0x7fdbe4005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.095+0000 7fdbdeffd700 1 -- 172.21.15.32:0/235389316 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fdbe0097c10 con 0x7fdbe81018c0 2023-09-30T14:38:08.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.343+0000 7fdbf0098700 1 -- 172.21.15.32:0/235389316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbe81a1400 con 0x7fdbe81018c0 2023-09-30T14:38:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.344+0000 7fdbdeffd700 1 -- 172.21.15.32:0/235389316 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11321 (secure 0 0 0) 0x7fdbe009b230 con 0x7fdbe81018c0 2023-09-30T14:38:08.346 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:08.347 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3505064841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3505064841},{"type":"v1","addr":"172.21.15.32:6839","nonce":3505064841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2262846038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2262846038},{"type":"v1","addr":"172.21.15.32:6841","nonce":2262846038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-30T14:37:11.444993+0000","modified":"2023-09-30T14:37:16.971534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6839/1445176203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1445176203},{"type":"v1","addr":"172.21.15.156:6839","nonce":1445176203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/494821686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":494821686},{"type":"v1","addr":"172.21.15.148:6839","nonce":494821686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T14:38:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.346+0000 7fdbf0098700 1 -- 172.21.15.32:0/235389316 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdbd40a1070 msgr2=0x7fdbd40a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.346+0000 7fdbf0098700 1 --2- 172.21.15.32:0/235389316 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdbd40a1070 0x7fdbd40a3530 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4005fd0 tx=0x7fdbe4005dd0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.346+0000 7fdbf0098700 1 -- 172.21.15.32:0/235389316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbe81018c0 msgr2=0x7fdbe81a0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.346+0000 7fdbf0098700 1 --2- 172.21.15.32:0/235389316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbe81018c0 0x7fdbe81a0380 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0007a00 tx=0x7fdbe0009570 comp rx=0 tx=0).stop 2023-09-30T14:38:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.346+0000 7fdbf0098700 1 -- 172.21.15.32:0/235389316 shutdown_connections 2023-09-30T14:38:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.346+0000 7fdbf0098700 1 --2- 172.21.15.32:0/235389316 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdbd40a1070 0x7fdbd40a3530 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.346+0000 7fdbf0098700 1 --2- 172.21.15.32:0/235389316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbe81018c0 0x7fdbe81a0380 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.346+0000 7fdbf0098700 1 --2- 172.21.15.32:0/235389316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbe80ffd30 0x7fdbe8198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.346+0000 7fdbf0098700 1 --2- 172.21.15.32:0/235389316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8100f20 0x7fdbe8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.346+0000 7fdbf0098700 1 -- 172.21.15.32:0/235389316 >> 172.21.15.32:0/235389316 conn(0x7fdbe80fb280 msgr2=0x7fdbe8067fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.346+0000 7fdbf0098700 1 -- 172.21.15.32:0/235389316 shutdown_connections 2023-09-30T14:38:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.346+0000 7fdbf0098700 1 -- 172.21.15.32:0/235389316 wait complete. 2023-09-30T14:38:08.348 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 22 2023-09-30T14:38:08.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:08 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1228889247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:08.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:08 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1349900363' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:08.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:08 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1122850559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:08.358 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T14:38:08.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.483+0000 7f162d504700 1 -- 172.21.15.32:0/4127438678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1628100f20 msgr2=0x7f1628101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:08.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.483+0000 7f162d504700 1 --2- 172.21.15.32:0/4127438678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1628100f20 0x7f1628101380 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f161c0099c0 tx=0x7f161c009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.483+0000 7f162d504700 1 -- 172.21.15.32:0/4127438678 shutdown_connections 2023-09-30T14:38:08.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.483+0000 7f162d504700 1 --2- 172.21.15.32:0/4127438678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16281018c0 0x7f1628108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.483+0000 7f162d504700 1 --2- 172.21.15.32:0/4127438678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16280ffd30 0x7f1628100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.483+0000 7f162d504700 1 --2- 172.21.15.32:0/4127438678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1628100f20 0x7f1628101380 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.483+0000 7f162d504700 1 -- 172.21.15.32:0/4127438678 >> 172.21.15.32:0/4127438678 conn(0x7f16280fb280 msgr2=0x7f16280fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:08.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.483+0000 7f162d504700 1 -- 172.21.15.32:0/4127438678 shutdown_connections 2023-09-30T14:38:08.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.483+0000 7f162d504700 1 -- 172.21.15.32:0/4127438678 wait complete. 2023-09-30T14:38:08.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.484+0000 7f162d504700 1 Processor -- start 2023-09-30T14:38:08.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.484+0000 7f162d504700 1 -- start start 2023-09-30T14:38:08.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.484+0000 7f162d504700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16280ffd30 0x7f1628068b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:08.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.484+0000 7f162d504700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1628100f20 0x7f162819b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:08.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.484+0000 7f162d504700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16281018c0 0x7f162819ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.484+0000 7f162d504700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162819e5c0 con 0x7f16281018c0 2023-09-30T14:38:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.484+0000 7f162d504700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162819e730 con 0x7f1628100f20 2023-09-30T14:38:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.484+0000 7f162d504700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162819e8f0 con 0x7f16280ffd30 2023-09-30T14:38:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.484+0000 7f16277fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16281018c0 0x7f162819ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.484+0000 7f16277fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16281018c0 0x7f162819ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34548/0 (socket says 172.21.15.32:34548) 2023-09-30T14:38:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.484+0000 7f16277fe700 1 -- 172.21.15.32:0/3857636922 learned_addr learned my addr 172.21.15.32:0/3857636922 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.484+0000 7f16267fc700 1 --2- 172.21.15.32:0/3857636922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1628100f20 0x7f162819b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.484+0000 7f1626ffd700 1 --2- 172.21.15.32:0/3857636922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16280ffd30 0x7f1628068b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.485+0000 7f16277fe700 1 -- 172.21.15.32:0/3857636922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16280ffd30 msgr2=0x7f1628068b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.485+0000 7f16277fe700 1 --2- 172.21.15.32:0/3857636922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16280ffd30 0x7f1628068b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.485+0000 7f16277fe700 1 -- 172.21.15.32:0/3857636922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1628100f20 msgr2=0x7f162819b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:08.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.485+0000 7f16277fe700 1 --2- 172.21.15.32:0/3857636922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1628100f20 0x7f162819b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.485+0000 7f16277fe700 1 -- 172.21.15.32:0/3857636922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c0095f0 con 0x7f16281018c0 2023-09-30T14:38:08.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.485+0000 7f1626ffd700 1 --2- 172.21.15.32:0/3857636922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16280ffd30 0x7f1628068b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:08.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.485+0000 7f16267fc700 1 --2- 172.21.15.32:0/3857636922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1628100f20 0x7f162819b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:08.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.485+0000 7f16277fe700 1 --2- 172.21.15.32:0/3857636922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16281018c0 0x7f162819ddf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f162000b970 tx=0x7f162000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:08.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.485+0000 7f160ffff700 1 -- 172.21.15.32:0/3857636922 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162000c9b0 con 0x7f16281018c0 2023-09-30T14:38:08.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.485+0000 7f160ffff700 1 -- 172.21.15.32:0/3857636922 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1620014440 con 0x7f16281018c0 2023-09-30T14:38:08.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.485+0000 7f162d504700 1 -- 172.21.15.32:0/3857636922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162819ebd0 con 0x7f16281018c0 2023-09-30T14:38:08.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.485+0000 7f160ffff700 1 -- 172.21.15.32:0/3857636922 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16200075d0 con 0x7f16281018c0 2023-09-30T14:38:08.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.485+0000 7f162d504700 1 -- 172.21.15.32:0/3857636922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162819f0b0 con 0x7f16281018c0 2023-09-30T14:38:08.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.486+0000 7f162d504700 1 -- 172.21.15.32:0/3857636922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1628107200 con 0x7f16281018c0 2023-09-30T14:38:08.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.487+0000 7f160ffff700 1 -- 172.21.15.32:0/3857636922 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f16200145b0 con 0x7f16281018c0 2023-09-30T14:38:08.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.488+0000 7f160ffff700 1 --2- 172.21.15.32:0/3857636922 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f16100a10f0 0x7f16100a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:08.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.488+0000 7f160ffff700 1 -- 172.21.15.32:0/3857636922 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1620006e40 con 0x7f16281018c0 2023-09-30T14:38:08.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.488+0000 7f1626ffd700 1 --2- 172.21.15.32:0/3857636922 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f16100a10f0 0x7f16100a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:08.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.488+0000 7f1626ffd700 1 --2- 172.21.15.32:0/3857636922 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f16100a10f0 0x7f16100a35b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1618005fd0 tx=0x7f1618005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:08.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.492+0000 7f160ffff700 1 -- 172.21.15.32:0/3857636922 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f162004ef30 con 0x7f16281018c0 2023-09-30T14:38:08.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:08 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1228889247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:08.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:08 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1349900363' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:08.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:08 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1122850559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:08.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:08 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1228889247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:08.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:08 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1349900363' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:08.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:08 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1122850559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:08.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.698+0000 7f162d504700 1 -- 172.21.15.32:0/3857636922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f162810b5c0 con 0x7f16281018c0 2023-09-30T14:38:08.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.699+0000 7f160ffff700 1 -- 172.21.15.32:0/3857636922 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22133 (secure 0 0 0) 0x7f1620095620 con 0x7f16281018c0 2023-09-30T14:38:08.700 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:08.700 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":83,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:37:12.972217+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:38:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.702+0000 7f162d504700 1 -- 172.21.15.32:0/3857636922 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f16100a10f0 msgr2=0x7f16100a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.702+0000 7f162d504700 1 --2- 172.21.15.32:0/3857636922 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f16100a10f0 0x7f16100a35b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1618005fd0 tx=0x7f1618005e40 comp rx=0 tx=0).stop 2023-09-30T14:38:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.702+0000 7f162d504700 1 -- 172.21.15.32:0/3857636922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16281018c0 msgr2=0x7f162819ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.702+0000 7f162d504700 1 --2- 172.21.15.32:0/3857636922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16281018c0 0x7f162819ddf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f162000b970 tx=0x7f162000bd30 comp rx=0 tx=0).stop 2023-09-30T14:38:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.702+0000 7f162d504700 1 -- 172.21.15.32:0/3857636922 shutdown_connections 2023-09-30T14:38:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.702+0000 7f162d504700 1 --2- 172.21.15.32:0/3857636922 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f16100a10f0 0x7f16100a35b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.702+0000 7f162d504700 1 --2- 172.21.15.32:0/3857636922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16281018c0 0x7f162819ddf0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.702+0000 7f162d504700 1 --2- 172.21.15.32:0/3857636922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16280ffd30 0x7f1628068b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.702+0000 7f162d504700 1 --2- 172.21.15.32:0/3857636922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1628100f20 0x7f162819b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.702+0000 7f162d504700 1 -- 172.21.15.32:0/3857636922 >> 172.21.15.32:0/3857636922 conn(0x7f16280fb280 msgr2=0x7f1628104340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:08.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.702+0000 7f162d504700 1 -- 172.21.15.32:0/3857636922 shutdown_connections 2023-09-30T14:38:08.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.702+0000 7f162d504700 1 -- 172.21.15.32:0/3857636922 wait complete. 2023-09-30T14:38:08.714 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:38:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.839+0000 7ff1944ca700 1 -- 172.21.15.32:0/518157555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18c0ffd30 msgr2=0x7ff18c100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.839+0000 7ff1944ca700 1 --2- 172.21.15.32:0/518157555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18c0ffd30 0x7ff18c100150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff1880099c0 tx=0x7ff188009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.840+0000 7ff1944ca700 1 -- 172.21.15.32:0/518157555 shutdown_connections 2023-09-30T14:38:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.840+0000 7ff1944ca700 1 --2- 172.21.15.32:0/518157555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff18c1018c0 0x7ff18c108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.840+0000 7ff1944ca700 1 --2- 172.21.15.32:0/518157555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18c0ffd30 0x7ff18c100150 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.840+0000 7ff1944ca700 1 --2- 172.21.15.32:0/518157555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff18c100f20 0x7ff18c101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.840+0000 7ff1944ca700 1 -- 172.21.15.32:0/518157555 >> 172.21.15.32:0/518157555 conn(0x7ff18c0fb280 msgr2=0x7ff18c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:08.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.840+0000 7ff1944ca700 1 -- 172.21.15.32:0/518157555 shutdown_connections 2023-09-30T14:38:08.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.840+0000 7ff1944ca700 1 -- 172.21.15.32:0/518157555 wait complete. 2023-09-30T14:38:08.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.840+0000 7ff1944ca700 1 Processor -- start 2023-09-30T14:38:08.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.840+0000 7ff1944ca700 1 -- start start 2023-09-30T14:38:08.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.840+0000 7ff1944ca700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff18c0ffd30 0x7ff18c196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:08.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.840+0000 7ff1944ca700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff18c100f20 0x7ff18c1970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:08.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.840+0000 7ff1944ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18c1018c0 0x7ff18c19e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:08.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.840+0000 7ff1944ca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c19e700 con 0x7ff18c1018c0 2023-09-30T14:38:08.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.840+0000 7ff1944ca700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c19e840 con 0x7ff18c0ffd30 2023-09-30T14:38:08.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.840+0000 7ff1944ca700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c19e9d0 con 0x7ff18c100f20 2023-09-30T14:38:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.840+0000 7ff192a67700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18c1018c0 0x7ff18c19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.841+0000 7ff192a67700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18c1018c0 0x7ff18c19e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34562/0 (socket says 172.21.15.32:34562) 2023-09-30T14:38:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.841+0000 7ff192a67700 1 -- 172.21.15.32:0/2315821472 learned_addr learned my addr 172.21.15.32:0/2315821472 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.841+0000 7ff192266700 1 --2- 172.21.15.32:0/2315821472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff18c0ffd30 0x7ff18c196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.841+0000 7ff191a65700 1 --2- 172.21.15.32:0/2315821472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff18c100f20 0x7ff18c1970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.841+0000 7ff192a67700 1 -- 172.21.15.32:0/2315821472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff18c100f20 msgr2=0x7ff18c1970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.841+0000 7ff192a67700 1 --2- 172.21.15.32:0/2315821472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff18c100f20 0x7ff18c1970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.841+0000 7ff192a67700 1 -- 172.21.15.32:0/2315821472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff18c0ffd30 msgr2=0x7ff18c196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.841+0000 7ff192a67700 1 --2- 172.21.15.32:0/2315821472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff18c0ffd30 0x7ff18c196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.841+0000 7ff192a67700 1 -- 172.21.15.32:0/2315821472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1880095f0 con 0x7ff18c1018c0 2023-09-30T14:38:08.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.841+0000 7ff192266700 1 --2- 172.21.15.32:0/2315821472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff18c0ffd30 0x7ff18c196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:08.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.841+0000 7ff192a67700 1 --2- 172.21.15.32:0/2315821472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18c1018c0 0x7ff18c19e130 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff18400ada0 tx=0x7ff18400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:08.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.841+0000 7ff1837fe700 1 -- 172.21.15.32:0/2315821472 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18400f070 con 0x7ff18c1018c0 2023-09-30T14:38:08.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.841+0000 7ff1837fe700 1 -- 172.21.15.32:0/2315821472 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff184007b30 con 0x7ff18c1018c0 2023-09-30T14:38:08.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.841+0000 7ff1944ca700 1 -- 172.21.15.32:0/2315821472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18c19ecb0 con 0x7ff18c1018c0 2023-09-30T14:38:08.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.841+0000 7ff1837fe700 1 -- 172.21.15.32:0/2315821472 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff184005440 con 0x7ff18c1018c0 2023-09-30T14:38:08.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.841+0000 7ff1944ca700 1 -- 172.21.15.32:0/2315821472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18c19f1c0 con 0x7ff18c1018c0 2023-09-30T14:38:08.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.842+0000 7ff1944ca700 1 -- 172.21.15.32:0/2315821472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18c107310 con 0x7ff18c1018c0 2023-09-30T14:38:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.844+0000 7ff1837fe700 1 -- 172.21.15.32:0/2315821472 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7ff184004ad0 con 0x7ff18c1018c0 2023-09-30T14:38:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.844+0000 7ff1837fe700 1 --2- 172.21.15.32:0/2315821472 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff1780a10f0 0x7ff1780a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.844+0000 7ff1837fe700 1 -- 172.21.15.32:0/2315821472 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff1840cc1f0 con 0x7ff18c1018c0 2023-09-30T14:38:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.844+0000 7ff192266700 1 --2- 172.21.15.32:0/2315821472 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff1780a10f0 0x7ff1780a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.845+0000 7ff192266700 1 --2- 172.21.15.32:0/2315821472 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff1780a10f0 0x7ff1780a35b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff18800b580 tx=0x7ff188005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:08.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:08.848+0000 7ff1837fe700 1 -- 172.21.15.32:0/2315821472 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7ff184095ad0 con 0x7ff18c1018c0 2023-09-30T14:38:09.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.100+0000 7ff1944ca700 1 -- 172.21.15.32:0/2315821472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff18c10c8a0 con 0x7ff18c1018c0 2023-09-30T14:38:09.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.100+0000 7ff1837fe700 1 -- 172.21.15.32:0/2315821472 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11321 (secure 0 0 0) 0x7ff1840990f0 con 0x7ff18c1018c0 2023-09-30T14:38:09.100 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:09.100 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3505064841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3505064841},{"type":"v1","addr":"172.21.15.32:6839","nonce":3505064841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2262846038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2262846038},{"type":"v1","addr":"172.21.15.32:6841","nonce":2262846038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-30T14:37:11.444993+0000","modified":"2023-09-30T14:37:16.971534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6839/1445176203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1445176203},{"type":"v1","addr":"172.21.15.156:6839","nonce":1445176203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/494821686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":494821686},{"type":"v1","addr":"172.21.15.148:6839","nonce":494821686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T14:38:09.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.102+0000 7ff1944ca700 1 -- 172.21.15.32:0/2315821472 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff1780a10f0 msgr2=0x7ff1780a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.102+0000 7ff1944ca700 1 --2- 172.21.15.32:0/2315821472 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff1780a10f0 0x7ff1780a35b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff18800b580 tx=0x7ff188005fb0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.102+0000 7ff1944ca700 1 -- 172.21.15.32:0/2315821472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18c1018c0 msgr2=0x7ff18c19e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.102+0000 7ff1944ca700 1 --2- 172.21.15.32:0/2315821472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18c1018c0 0x7ff18c19e130 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff18400ada0 tx=0x7ff18400c330 comp rx=0 tx=0).stop 2023-09-30T14:38:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.102+0000 7ff1944ca700 1 -- 172.21.15.32:0/2315821472 shutdown_connections 2023-09-30T14:38:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.102+0000 7ff1944ca700 1 --2- 172.21.15.32:0/2315821472 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff1780a10f0 0x7ff1780a35b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.102+0000 7ff1944ca700 1 --2- 172.21.15.32:0/2315821472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18c1018c0 0x7ff18c19e130 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.102+0000 7ff1944ca700 1 --2- 172.21.15.32:0/2315821472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff18c0ffd30 0x7ff18c196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.102+0000 7ff1944ca700 1 --2- 172.21.15.32:0/2315821472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff18c100f20 0x7ff18c1970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.102+0000 7ff1944ca700 1 -- 172.21.15.32:0/2315821472 >> 172.21.15.32:0/2315821472 conn(0x7ff18c0fb280 msgr2=0x7ff18c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.103+0000 7ff1944ca700 1 -- 172.21.15.32:0/2315821472 shutdown_connections 2023-09-30T14:38:09.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.103+0000 7ff1944ca700 1 -- 172.21.15.32:0/2315821472 wait complete. 2023-09-30T14:38:09.104 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 22 2023-09-30T14:38:09.115 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-30T14:37:11.444993+0000', 'modified': '2023-09-30T14:37:16.971534+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24515, 'mds_1': 24493, 'mds_2': 24485}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6835/4151174061', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 4151174061}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 4151174061}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6835/4180205116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 4180205116}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 4180205116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6837/445372881', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 445372881}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 445372881}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6837/3666119034', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 3666119034}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 3666119034}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.156:6839/1445176203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 1445176203}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 1445176203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24515': {'gid': 24515, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6839/494821686', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 494821686}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 494821686}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-30T14:38:09.116 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-09-30T14:38:09.116 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-09-30T14:38:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.244+0000 7f8936132700 1 -- 172.21.15.32:0/2115363208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89301018c0 msgr2=0x7f8930108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.244+0000 7f8936132700 1 --2- 172.21.15.32:0/2115363208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89301018c0 0x7f8930108170 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f892800b0d0 tx=0x7f892800b3e0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.245+0000 7f8936132700 1 -- 172.21.15.32:0/2115363208 shutdown_connections 2023-09-30T14:38:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.245+0000 7f8936132700 1 --2- 172.21.15.32:0/2115363208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89301018c0 0x7f8930108170 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.245+0000 7f8936132700 1 --2- 172.21.15.32:0/2115363208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89300ffd30 0x7f8930100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.245+0000 7f8936132700 1 --2- 172.21.15.32:0/2115363208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8930100f20 0x7f8930101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.245+0000 7f8936132700 1 -- 172.21.15.32:0/2115363208 >> 172.21.15.32:0/2115363208 conn(0x7f89300fb2a0 msgr2=0x7f89300fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.245+0000 7f8936132700 1 -- 172.21.15.32:0/2115363208 shutdown_connections 2023-09-30T14:38:09.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.245+0000 7f8936132700 1 -- 172.21.15.32:0/2115363208 wait complete. 2023-09-30T14:38:09.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.245+0000 7f8936132700 1 Processor -- start 2023-09-30T14:38:09.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.245+0000 7f8936132700 1 -- start start 2023-09-30T14:38:09.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f8936132700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89300ffd30 0x7f8930196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f8936132700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8930100f20 0x7f8930197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f8936132700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89301018c0 0x7f893019e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f8936132700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893019e770 con 0x7f89301018c0 2023-09-30T14:38:09.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f8936132700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893019e8e0 con 0x7f89300ffd30 2023-09-30T14:38:09.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f8936132700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893019eaa0 con 0x7f8930100f20 2023-09-30T14:38:09.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f892ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89301018c0 0x7f893019e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f892ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89301018c0 0x7f893019e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34576/0 (socket says 172.21.15.32:34576) 2023-09-30T14:38:09.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f892ffff700 1 -- 172.21.15.32:0/13484605 learned_addr learned my addr 172.21.15.32:0/13484605 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:09.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f892effd700 1 --2- 172.21.15.32:0/13484605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8930100f20 0x7f8930197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f892f7fe700 1 --2- 172.21.15.32:0/13484605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89300ffd30 0x7f8930196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f892ffff700 1 -- 172.21.15.32:0/13484605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8930100f20 msgr2=0x7f8930197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f892ffff700 1 --2- 172.21.15.32:0/13484605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8930100f20 0x7f8930197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f892ffff700 1 -- 172.21.15.32:0/13484605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89300ffd30 msgr2=0x7f8930196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f892ffff700 1 --2- 172.21.15.32:0/13484605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89300ffd30 0x7f8930196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f892ffff700 1 -- 172.21.15.32:0/13484605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8928009d00 con 0x7f89301018c0 2023-09-30T14:38:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f892ffff700 1 --2- 172.21.15.32:0/13484605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89301018c0 0x7f893019e1a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8928007a00 tx=0x7f8928009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f892cff9700 1 -- 172.21.15.32:0/13484605 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892800e040 con 0x7f89301018c0 2023-09-30T14:38:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f892cff9700 1 -- 172.21.15.32:0/13484605 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8928003e10 con 0x7f89301018c0 2023-09-30T14:38:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f8936132700 1 -- 172.21.15.32:0/13484605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893019ed20 con 0x7f89301018c0 2023-09-30T14:38:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f892cff9700 1 -- 172.21.15.32:0/13484605 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8928005610 con 0x7f89301018c0 2023-09-30T14:38:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.246+0000 7f8936132700 1 -- 172.21.15.32:0/13484605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893019f140 con 0x7f89301018c0 2023-09-30T14:38:09.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.248+0000 7f892cff9700 1 -- 172.21.15.32:0/13484605 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f8928005770 con 0x7f89301018c0 2023-09-30T14:38:09.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.249+0000 7f892cff9700 1 --2- 172.21.15.32:0/13484605 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89180a1030 0x7f89180a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.249+0000 7f892f7fe700 1 --2- 172.21.15.32:0/13484605 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89180a1030 0x7f89180a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.249+0000 7f892cff9700 1 -- 172.21.15.32:0/13484605 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f89280cde70 con 0x7f89301018c0 2023-09-30T14:38:09.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.250+0000 7f892f7fe700 1 --2- 172.21.15.32:0/13484605 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89180a1030 0x7f89180a34f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8920005950 tx=0x7f89200058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.250+0000 7f894231ab80 1 Processor -- start 2023-09-30T14:38:09.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.250+0000 7f894231ab80 1 -- start start 2023-09-30T14:38:09.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.250+0000 7f894231ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf51550 0x563eeaf51970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.250+0000 7f894231ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf51eb0 0x563eeaf52770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.250+0000 7f894231ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf52d50 0x563eeaf531d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f894231ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eeaeb9410 con 0x563eeaf52d50 2023-09-30T14:38:09.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f894231ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eeaeaabd0 con 0x563eeaf51eb0 2023-09-30T14:38:09.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f894231ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eeaea84f0 con 0x563eeaf51550 2023-09-30T14:38:09.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f892ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf52d50 0x563eeaf531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f892ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf52d50 0x563eeaf531d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34584/0 (socket says 172.21.15.32:34584) 2023-09-30T14:38:09.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f892ffff700 1 -- 172.21.15.32:0/2932334900 learned_addr learned my addr 172.21.15.32:0/2932334900 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:09.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f892effd700 1 --2- 172.21.15.32:0/2932334900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf51eb0 0x563eeaf52770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f892f7fe700 1 --2- 172.21.15.32:0/2932334900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf51550 0x563eeaf51970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f892ffff700 1 -- 172.21.15.32:0/2932334900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf51550 msgr2=0x563eeaf51970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f892ffff700 1 --2- 172.21.15.32:0/2932334900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf51550 0x563eeaf51970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f892ffff700 1 -- 172.21.15.32:0/2932334900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf51eb0 msgr2=0x563eeaf52770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f892ffff700 1 --2- 172.21.15.32:0/2932334900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf51eb0 0x563eeaf52770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f892ffff700 1 -- 172.21.15.32:0/2932334900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563eeaf06880 con 0x563eeaf52d50 2023-09-30T14:38:09.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f892ffff700 1 --2- 172.21.15.32:0/2932334900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf52d50 0x563eeaf531d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8928097e80 tx=0x7f8928097100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2610970d7e1901c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f8915ffb700 1 -- 172.21.15.32:0/2932334900 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892809ad70 con 0x563eeaf52d50 2023-09-30T14:38:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f8915ffb700 1 -- 172.21.15.32:0/2932334900 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f892809bb10 con 0x563eeaf52d50 2023-09-30T14:38:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.251+0000 7f8915ffb700 1 -- 172.21.15.32:0/2932334900 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892802b550 con 0x563eeaf52d50 2023-09-30T14:38:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.252+0000 7f894231ab80 1 -- 172.21.15.32:0/2932334900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf52d50 msgr2=0x563eeaf531d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.252+0000 7f894231ab80 1 --2- 172.21.15.32:0/2932334900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf52d50 0x563eeaf531d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8928097e80 tx=0x7f8928097100 comp rx=0 tx=0).stop 2023-09-30T14:38:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.252+0000 7f894231ab80 1 -- 172.21.15.32:0/2932334900 shutdown_connections 2023-09-30T14:38:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.252+0000 7f894231ab80 1 --2- 172.21.15.32:0/2932334900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf52d50 0x563eeaf531d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.252+0000 7f894231ab80 1 --2- 172.21.15.32:0/2932334900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf51550 0x563eeaf51970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.252+0000 7f894231ab80 1 --2- 172.21.15.32:0/2932334900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf51eb0 0x563eeaf52770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.252+0000 7f894231ab80 1 -- 172.21.15.32:0/2932334900 >> 172.21.15.32:0/2932334900 conn(0x563eeaec5210 msgr2=0x563eeaf4c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.252+0000 7f894231ab80 1 -- 172.21.15.32:0/2932334900 shutdown_connections 2023-09-30T14:38:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.252+0000 7f894231ab80 1 -- 172.21.15.32:0/2932334900 wait complete. 2023-09-30T14:38:09.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.252+0000 7f894231ab80 1 -- start start 2023-09-30T14:38:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f894231ab80 1 Processor -- start 2023-09-30T14:38:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f894231ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf51550 0x563eeaf5b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f894231ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf51eb0 0x563eeaf5b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f894231ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf52d50 0x563eeaf62710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f894231ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eeaea3de0 con 0x563eeaf51eb0 2023-09-30T14:38:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f894231ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eeae406c0 con 0x563eeaf52d50 2023-09-30T14:38:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f894231ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eeae40360 con 0x563eeaf51550 2023-09-30T14:38:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f894231ab80 10 client.0 _ll_register_callbacks cb 0x563eeac9d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f892effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf51eb0 0x563eeaf5b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f892effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf51eb0 0x563eeaf5b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34588/0 (socket says 172.21.15.32:34588) 2023-09-30T14:38:09.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f892effd700 1 -- 172.21.15.32:0/2385680023 learned_addr learned my addr 172.21.15.32:0/2385680023 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f892ffff700 1 --2- 172.21.15.32:0/2385680023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf52d50 0x563eeaf62710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f892f7fe700 1 --2- 172.21.15.32:0/2385680023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf51550 0x563eeaf5b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f892effd700 1 -- 172.21.15.32:0/2385680023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf51550 msgr2=0x563eeaf5b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f892effd700 1 --2- 172.21.15.32:0/2385680023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf51550 0x563eeaf5b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f892effd700 1 -- 172.21.15.32:0/2385680023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf52d50 msgr2=0x563eeaf62710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f892effd700 1 --2- 172.21.15.32:0/2385680023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf52d50 0x563eeaf62710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f892effd700 1 -- 172.21.15.32:0/2385680023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893019f140 con 0x563eeaf51eb0 2023-09-30T14:38:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f892effd700 1 --2- 172.21.15.32:0/2385680023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf51eb0 0x563eeaf5b6d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f892400d960 tx=0x7f892400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67905bd74ebb439f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.253+0000 7f890effd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.254+0000 7f890effd700 1 -- 172.21.15.32:0/2385680023 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892401b440 con 0x563eeaf51eb0 2023-09-30T14:38:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.254+0000 7f894231ab80 1 -- 172.21.15.32:0/2385680023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563eeaf44200 con 0x563eeaf51eb0 2023-09-30T14:38:09.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.254+0000 7f890effd700 1 -- 172.21.15.32:0/2385680023 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8924011440 con 0x563eeaf51eb0 2023-09-30T14:38:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.254+0000 7f890effd700 1 -- 172.21.15.32:0/2385680023 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8924011600 con 0x563eeaf51eb0 2023-09-30T14:38:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.254+0000 7f894231ab80 10 client.15039 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.254+0000 7f894231ab80 1 -- 172.21.15.32:0/2385680023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x563eeacced90 con 0x563eeaf51eb0 2023-09-30T14:38:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.254+0000 7f890effd700 1 -- 172.21.15.32:0/2385680023 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563eeaf44200 con 0x563eeaf51eb0 2023-09-30T14:38:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.254+0000 7f890effd700 1 -- 172.21.15.32:0/2385680023 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f892400f7b0 con 0x563eeaf51eb0 2023-09-30T14:38:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.254+0000 7f894231ab80 10 client.15039 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.254+0000 7f894231ab80 10 client.15039 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-30T14:38:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.254+0000 7f894231ab80 20 client.15039 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.254+0000 7f894231ab80 1 --2- 172.21.15.32:0/2385680023 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x563eeaf66d90 0x563eeaf65860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.254+0000 7f894231ab80 4 client.15039 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:38:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.254+0000 7f894231ab80 1 -- 172.21.15.32:0/2385680023 --> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563eeae9cdc0 con 0x563eeaf66d90 2023-09-30T14:38:09.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.255+0000 7f892f7fe700 1 --2- 172.21.15.32:0/2385680023 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x563eeaf66d90 0x563eeaf65860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.255+0000 7f890effd700 10 client.15039 ms_handle_connect on v2:172.21.15.148:6836/3666119034 2023-09-30T14:38:09.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.255+0000 7f892f7fe700 1 --2- 172.21.15.32:0/2385680023 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x563eeaf66d90 0x563eeaf65860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2892f8b86fc2fbc0 server_cookie=4800ca7537d638a in_seq=0 out_seq=0 2023-09-30T14:38:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.256+0000 7f890effd700 1 -- 172.21.15.32:0/2385680023 <== mds.2 v2:172.21.15.148:6836/3666119034 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563eeae9cdc0 con 0x563eeaf66d90 2023-09-30T14:38:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.256+0000 7f890effd700 10 client.15039 handle_command_reply: tid=0 2023-09-30T14:38:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.256+0000 7f894231ab80 1 client.15039 shutdown 2023-09-30T14:38:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.256+0000 7f894231ab80 1 -- 172.21.15.32:0/2385680023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf51eb0 msgr2=0x563eeaf5b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.256+0000 7f894231ab80 1 --2- 172.21.15.32:0/2385680023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf51eb0 0x563eeaf5b6d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f892400d960 tx=0x7f892400dd20 comp rx=0 tx=0).stop 2023-09-30T14:38:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.256+0000 7f894231ab80 1 -- 172.21.15.32:0/2385680023 shutdown_connections 2023-09-30T14:38:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.256+0000 7f894231ab80 1 --2- 172.21.15.32:0/2385680023 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x563eeaf66d90 0x563eeaf65860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.256+0000 7f894231ab80 1 --2- 172.21.15.32:0/2385680023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf52d50 0x563eeaf62710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.256+0000 7f894231ab80 1 --2- 172.21.15.32:0/2385680023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf51550 0x563eeaf5b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.256+0000 7f894231ab80 1 --2- 172.21.15.32:0/2385680023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf51eb0 0x563eeaf5b6d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.256+0000 7f890effd700 0 client.15039 ms_handle_reset on v2:172.21.15.148:6836/3666119034 2023-09-30T14:38:09.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.256+0000 7f894231ab80 1 -- 172.21.15.32:0/2385680023 >> 172.21.15.32:0/2385680023 conn(0x563eeaec5210 msgr2=0x563eeaf4c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.256+0000 7f894231ab80 1 -- 172.21.15.32:0/2385680023 shutdown_connections 2023-09-30T14:38:09.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.256+0000 7f894231ab80 1 -- 172.21.15.32:0/2385680023 wait complete. 2023-09-30T14:38:09.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.256+0000 7f894231ab80 20 client.15039 trim_cache size 0 max 16384 2023-09-30T14:38:09.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.274+0000 7f894231ab80 1 Processor -- start 2023-09-30T14:38:09.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.274+0000 7f894231ab80 1 -- start start 2023-09-30T14:38:09.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.274+0000 7f894231ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf51550 0x563eeaf59dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.274+0000 7f894231ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf51eb0 0x563eeaf55cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.274+0000 7f894231ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf52d50 0x563eeaf561f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.274+0000 7f894231ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eead248d0 con 0x563eeaf52d50 2023-09-30T14:38:09.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.274+0000 7f894231ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eeaf64400 con 0x563eeaf51eb0 2023-09-30T14:38:09.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.274+0000 7f894231ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eeaf5c380 con 0x563eeaf51550 2023-09-30T14:38:09.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.274+0000 7f892ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf52d50 0x563eeaf561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.274+0000 7f892ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf52d50 0x563eeaf561f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34598/0 (socket says 172.21.15.32:34598) 2023-09-30T14:38:09.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.274+0000 7f892ffff700 1 -- 172.21.15.32:0/1931741748 learned_addr learned my addr 172.21.15.32:0/1931741748 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:09.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.275+0000 7f892f7fe700 1 --2- 172.21.15.32:0/1931741748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf51550 0x563eeaf59dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.275+0000 7f892effd700 1 --2- 172.21.15.32:0/1931741748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf51eb0 0x563eeaf55cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.275+0000 7f892ffff700 1 -- 172.21.15.32:0/1931741748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf51550 msgr2=0x563eeaf59dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.275+0000 7f892ffff700 1 --2- 172.21.15.32:0/1931741748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf51550 0x563eeaf59dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.275+0000 7f892ffff700 1 -- 172.21.15.32:0/1931741748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf51eb0 msgr2=0x563eeaf55cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.275+0000 7f892ffff700 1 --2- 172.21.15.32:0/1931741748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf51eb0 0x563eeaf55cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.275+0000 7f892ffff700 1 -- 172.21.15.32:0/1931741748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563eeacb7c10 con 0x563eeaf52d50 2023-09-30T14:38:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.275+0000 7f892ffff700 1 --2- 172.21.15.32:0/1931741748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf52d50 0x563eeaf561f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f892809b3a0 tx=0x7f892809bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d304f4ef88c8c4ac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.275+0000 7f89167fc700 1 -- 172.21.15.32:0/1931741748 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892809ad70 con 0x563eeaf52d50 2023-09-30T14:38:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.275+0000 7f89167fc700 1 -- 172.21.15.32:0/1931741748 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f892809b410 con 0x563eeaf52d50 2023-09-30T14:38:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.275+0000 7f89167fc700 1 -- 172.21.15.32:0/1931741748 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892802b2f0 con 0x563eeaf52d50 2023-09-30T14:38:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 -- 172.21.15.32:0/1931741748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf52d50 msgr2=0x563eeaf561f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 --2- 172.21.15.32:0/1931741748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf52d50 0x563eeaf561f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f892809b3a0 tx=0x7f892809bc80 comp rx=0 tx=0).stop 2023-09-30T14:38:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 -- 172.21.15.32:0/1931741748 shutdown_connections 2023-09-30T14:38:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 --2- 172.21.15.32:0/1931741748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf52d50 0x563eeaf561f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 --2- 172.21.15.32:0/1931741748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf51550 0x563eeaf59dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 --2- 172.21.15.32:0/1931741748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf51eb0 0x563eeaf55cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 -- 172.21.15.32:0/1931741748 >> 172.21.15.32:0/1931741748 conn(0x563eeaec5210 msgr2=0x563eeaf4aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 -- 172.21.15.32:0/1931741748 shutdown_connections 2023-09-30T14:38:09.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 -- 172.21.15.32:0/1931741748 wait complete. 2023-09-30T14:38:09.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 -- start start 2023-09-30T14:38:09.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 Processor -- start 2023-09-30T14:38:09.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf51550 0x563eeafb69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf51eb0 0x563eeafb6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf52d50 0x563eeafbdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eeafbe1c0 con 0x563eeaf51550 2023-09-30T14:38:09.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eeafbe300 con 0x563eeaf51eb0 2023-09-30T14:38:09.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563eeafbe4c0 con 0x563eeaf52d50 2023-09-30T14:38:09.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f894231ab80 10 client.0 _ll_register_callbacks cb 0x563eeac9d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:09.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f892f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf51550 0x563eeafb69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.276+0000 7f892f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf51550 0x563eeafb69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34600/0 (socket says 172.21.15.32:34600) 2023-09-30T14:38:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f892f7fe700 1 -- 172.21.15.32:0/776325545 learned_addr learned my addr 172.21.15.32:0/776325545 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f892effd700 1 --2- 172.21.15.32:0/776325545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf51eb0 0x563eeafb6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f892ffff700 1 --2- 172.21.15.32:0/776325545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf52d50 0x563eeafbdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f892f7fe700 1 -- 172.21.15.32:0/776325545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf52d50 msgr2=0x563eeafbdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f892f7fe700 1 --2- 172.21.15.32:0/776325545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf52d50 0x563eeafbdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f892f7fe700 1 -- 172.21.15.32:0/776325545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf51eb0 msgr2=0x563eeafb6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f892f7fe700 1 --2- 172.21.15.32:0/776325545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf51eb0 0x563eeafb6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f892f7fe700 1 -- 172.21.15.32:0/776325545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563eeaf06880 con 0x563eeaf51550 2023-09-30T14:38:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f892effd700 1 --2- 172.21.15.32:0/776325545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf51eb0 0x563eeafb6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f892f7fe700 1 --2- 172.21.15.32:0/776325545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf51550 0x563eeafb69b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f892000ba40 tx=0x7f89200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81db9a8de9936625 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f890effd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:09.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f890effd700 1 -- 172.21.15.32:0/776325545 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8920025040 con 0x563eeaf51550 2023-09-30T14:38:09.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f890effd700 1 -- 172.21.15.32:0/776325545 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8920011a20 con 0x563eeaf51550 2023-09-30T14:38:09.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f894231ab80 1 -- 172.21.15.32:0/776325545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f892809bb10 con 0x563eeaf51550 2023-09-30T14:38:09.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f890effd700 1 -- 172.21.15.32:0/776325545 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89200275b0 con 0x563eeaf51550 2023-09-30T14:38:09.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f894231ab80 10 client.15045 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:09.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f894231ab80 1 -- 172.21.15.32:0/776325545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x563eeacf5200 con 0x563eeaf51550 2023-09-30T14:38:09.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.277+0000 7f890effd700 1 -- 172.21.15.32:0/776325545 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f892809bb10 con 0x563eeaf51550 2023-09-30T14:38:09.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.278+0000 7f890effd700 1 -- 172.21.15.32:0/776325545 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f892003a070 con 0x563eeaf51550 2023-09-30T14:38:09.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.278+0000 7f894231ab80 10 client.15045 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:09.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.278+0000 7f894231ab80 10 client.15045 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-30T14:38:09.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.278+0000 7f894231ab80 20 client.15045 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.278+0000 7f894231ab80 1 --2- 172.21.15.32:0/776325545 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x563eeafc37a0 0x563eeafbfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.278+0000 7f894231ab80 4 client.15045 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2023-09-30T14:38:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.278+0000 7f894231ab80 1 -- 172.21.15.32:0/776325545 --> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563eeae9cae0 con 0x563eeafc37a0 2023-09-30T14:38:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.278+0000 7f892effd700 1 --2- 172.21.15.32:0/776325545 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x563eeafc37a0 0x563eeafbfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.278+0000 7f890effd700 10 client.15045 ms_handle_connect on v2:172.21.15.148:6836/3666119034 2023-09-30T14:38:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.278+0000 7f892effd700 1 --2- 172.21.15.32:0/776325545 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x563eeafc37a0 0x563eeafbfc20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8dcf5e799b4bd8a0 server_cookie=382e429d697696f4 in_seq=0 out_seq=0 2023-09-30T14:38:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f890effd700 1 -- 172.21.15.32:0/776325545 <== mds.2 v2:172.21.15.148:6836/3666119034 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x563eeae9cae0 con 0x563eeafc37a0 2023-09-30T14:38:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f890effd700 10 client.15045 handle_command_reply: tid=0 2023-09-30T14:38:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f894231ab80 1 client.15045 shutdown 2023-09-30T14:38:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f894231ab80 1 -- 172.21.15.32:0/776325545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf51550 msgr2=0x563eeafb69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f894231ab80 1 --2- 172.21.15.32:0/776325545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf51550 0x563eeafb69b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f892000ba40 tx=0x7f89200047c0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f894231ab80 1 -- 172.21.15.32:0/776325545 shutdown_connections 2023-09-30T14:38:09.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f894231ab80 1 --2- 172.21.15.32:0/776325545 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x563eeafc37a0 0x563eeafbfc20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f894231ab80 1 --2- 172.21.15.32:0/776325545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563eeaf52d50 0x563eeafbdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f894231ab80 1 --2- 172.21.15.32:0/776325545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563eeaf51550 0x563eeafb69b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f894231ab80 1 --2- 172.21.15.32:0/776325545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eeaf51eb0 0x563eeafb6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f890effd700 0 client.15045 ms_handle_reset on v2:172.21.15.148:6836/3666119034 2023-09-30T14:38:09.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f894231ab80 1 -- 172.21.15.32:0/776325545 >> 172.21.15.32:0/776325545 conn(0x563eeaec5210 msgr2=0x563eeaf59dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f894231ab80 1 -- 172.21.15.32:0/776325545 shutdown_connections 2023-09-30T14:38:09.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f894231ab80 1 -- 172.21.15.32:0/776325545 wait complete. 2023-09-30T14:38:09.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f894231ab80 20 client.15045 trim_cache size 0 max 16384 2023-09-30T14:38:09.286 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T14:38:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "bca3f5c6-5f9d-11ee-8db5-212e2dc638e7", 2023-09-30T14:38:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 2, 2023-09-30T14:38:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24485, 2023-09-30T14:38:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-09-30T14:38:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-09-30T14:38:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-09-30T14:38:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 53.214399374999999, 2023-09-30T14:38:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 22, 2023-09-30T14:38:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 83, 2023-09-30T14:38:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 83, 2023-09-30T14:38:09.287 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 72.660516801 2023-09-30T14:38:09.287 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T14:38:09.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f8936132700 1 -- 172.21.15.32:0/13484605 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89180a1030 msgr2=0x7f89180a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f8936132700 1 --2- 172.21.15.32:0/13484605 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89180a1030 0x7f89180a34f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8920005950 tx=0x7f89200058e0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f8936132700 1 -- 172.21.15.32:0/13484605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89301018c0 msgr2=0x7f893019e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f8936132700 1 --2- 172.21.15.32:0/13484605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89301018c0 0x7f893019e1a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8928007a00 tx=0x7f8928009570 comp rx=0 tx=0).stop 2023-09-30T14:38:09.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f8936132700 1 -- 172.21.15.32:0/13484605 shutdown_connections 2023-09-30T14:38:09.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f8936132700 1 --2- 172.21.15.32:0/13484605 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89180a1030 0x7f89180a34f0 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8920005950 tx=0x7f89200058e0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f8936132700 1 --2- 172.21.15.32:0/13484605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89301018c0 0x7f893019e1a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f8936132700 1 --2- 172.21.15.32:0/13484605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89300ffd30 0x7f8930196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f8936132700 1 --2- 172.21.15.32:0/13484605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8930100f20 0x7f8930197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.279+0000 7f8936132700 1 -- 172.21.15.32:0/13484605 >> 172.21.15.32:0/13484605 conn(0x7f89300fb2a0 msgr2=0x7f89300fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.280+0000 7f8936132700 1 -- 172.21.15.32:0/13484605 shutdown_connections 2023-09-30T14:38:09.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.280+0000 7f8936132700 1 -- 172.21.15.32:0/13484605 wait complete. 2023-09-30T14:38:09.291 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-09-30T14:38:09.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:09 smithi032 ceph-mon[98592]: pgmap v281: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:09.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:09 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/235389316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:09.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:09 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3857636922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:09.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:09 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2315821472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.418+0000 7fc71ac8a700 1 -- 172.21.15.32:0/821661810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc714101940 msgr2=0x7fc7141081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.418+0000 7fc71ac8a700 1 --2- 172.21.15.32:0/821661810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc714101940 0x7fc7141081f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00b0d0 tx=0x7fc70c00b3e0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.418+0000 7fc71ac8a700 1 -- 172.21.15.32:0/821661810 shutdown_connections 2023-09-30T14:38:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.418+0000 7fc71ac8a700 1 --2- 172.21.15.32:0/821661810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc714101940 0x7fc7141081f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.418+0000 7fc71ac8a700 1 --2- 172.21.15.32:0/821661810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7140ffdb0 0x7fc7141001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.418+0000 7fc71ac8a700 1 --2- 172.21.15.32:0/821661810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc714100fa0 0x7fc714101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.418+0000 7fc71ac8a700 1 -- 172.21.15.32:0/821661810 >> 172.21.15.32:0/821661810 conn(0x7fc7140fb340 msgr2=0x7fc7140fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.419+0000 7fc71ac8a700 1 -- 172.21.15.32:0/821661810 shutdown_connections 2023-09-30T14:38:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.419+0000 7fc71ac8a700 1 -- 172.21.15.32:0/821661810 wait complete. 2023-09-30T14:38:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.419+0000 7fc71ac8a700 1 Processor -- start 2023-09-30T14:38:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.419+0000 7fc71ac8a700 1 -- start start 2023-09-30T14:38:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.419+0000 7fc71ac8a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7140ffdb0 0x7fc714069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.419+0000 7fc71ac8a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc714100fa0 0x7fc71419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.419+0000 7fc71ac8a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc714101940 0x7fc71419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.419+0000 7fc71ac8a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71419e760 con 0x7fc714100fa0 2023-09-30T14:38:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.419+0000 7fc71ac8a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71419e8d0 con 0x7fc714101940 2023-09-30T14:38:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.419+0000 7fc71ac8a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71419ea90 con 0x7fc7140ffdb0 2023-09-30T14:38:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.419+0000 7fc713fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc714100fa0 0x7fc71419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.419+0000 7fc713fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc714100fa0 0x7fc71419b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34616/0 (socket says 172.21.15.32:34616) 2023-09-30T14:38:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.419+0000 7fc713fff700 1 -- 172.21.15.32:0/2366375943 learned_addr learned my addr 172.21.15.32:0/2366375943 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.420+0000 7fc718a26700 1 --2- 172.21.15.32:0/2366375943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7140ffdb0 0x7fc714069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.420+0000 7fc719227700 1 --2- 172.21.15.32:0/2366375943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc714101940 0x7fc71419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.420+0000 7fc713fff700 1 -- 172.21.15.32:0/2366375943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7140ffdb0 msgr2=0x7fc714069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.420+0000 7fc713fff700 1 --2- 172.21.15.32:0/2366375943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7140ffdb0 0x7fc714069d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.420+0000 7fc713fff700 1 -- 172.21.15.32:0/2366375943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc714101940 msgr2=0x7fc71419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.420+0000 7fc713fff700 1 --2- 172.21.15.32:0/2366375943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc714101940 0x7fc71419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.420+0000 7fc713fff700 1 -- 172.21.15.32:0/2366375943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c009d00 con 0x7fc714100fa0 2023-09-30T14:38:09.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.420+0000 7fc718a26700 1 --2- 172.21.15.32:0/2366375943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7140ffdb0 0x7fc714069d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:09.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.420+0000 7fc713fff700 1 --2- 172.21.15.32:0/2366375943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc714100fa0 0x7fc71419b980 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc70000d950 tx=0x7fc70000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.420+0000 7fc711ffb700 1 -- 172.21.15.32:0/2366375943 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700009b40 con 0x7fc714100fa0 2023-09-30T14:38:09.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.420+0000 7fc71ac8a700 1 -- 172.21.15.32:0/2366375943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71419ed70 con 0x7fc714100fa0 2023-09-30T14:38:09.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.420+0000 7fc711ffb700 1 -- 172.21.15.32:0/2366375943 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc700009ca0 con 0x7fc714100fa0 2023-09-30T14:38:09.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.420+0000 7fc711ffb700 1 -- 172.21.15.32:0/2366375943 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70000f620 con 0x7fc714100fa0 2023-09-30T14:38:09.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.420+0000 7fc71ac8a700 1 -- 172.21.15.32:0/2366375943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71419f220 con 0x7fc714100fa0 2023-09-30T14:38:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.423+0000 7fc711ffb700 1 -- 172.21.15.32:0/2366375943 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fc700010460 con 0x7fc714100fa0 2023-09-30T14:38:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.423+0000 7fc711ffb700 1 --2- 172.21.15.32:0/2366375943 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc7040a1030 0x7fc7040a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.424+0000 7fc711ffb700 1 -- 172.21.15.32:0/2366375943 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc7000ccad0 con 0x7fc714100fa0 2023-09-30T14:38:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.424+0000 7fc718a26700 1 --2- 172.21.15.32:0/2366375943 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc7040a1030 0x7fc7040a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.424+0000 7fc718a26700 1 --2- 172.21.15.32:0/2366375943 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc7040a1030 0x7fc7040a34f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc714100e00 tx=0x7fc70800b670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.424+0000 7fc726e72b80 1 Processor -- start 2023-09-30T14:38:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.424+0000 7fc726e72b80 1 -- start start 2023-09-30T14:38:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.424+0000 7fc726e72b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5b960 0x55c89ad5bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.424+0000 7fc726e72b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5c2c0 0x55c89ad5cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.424+0000 7fc726e72b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5d160 0x55c89ad5d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc726e72b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89acbf410 con 0x55c89ad5b960 2023-09-30T14:38:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc726e72b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89acb0bd0 con 0x55c89ad5d160 2023-09-30T14:38:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc726e72b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89acae4f0 con 0x55c89ad5c2c0 2023-09-30T14:38:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc718a26700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5b960 0x55c89ad5bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc718a26700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5b960 0x55c89ad5bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34622/0 (socket says 172.21.15.32:34622) 2023-09-30T14:38:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc718a26700 1 -- 172.21.15.32:0/2415123917 learned_addr learned my addr 172.21.15.32:0/2415123917 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc719227700 1 --2- 172.21.15.32:0/2415123917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5d160 0x55c89ad5d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc718a26700 1 -- 172.21.15.32:0/2415123917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5c2c0 msgr2=0x55c89ad5cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc713fff700 1 --2- 172.21.15.32:0/2415123917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5c2c0 0x55c89ad5cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc718a26700 1 --2- 172.21.15.32:0/2415123917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5c2c0 0x55c89ad5cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc718a26700 1 -- 172.21.15.32:0/2415123917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5d160 msgr2=0x55c89ad5d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc718a26700 1 --2- 172.21.15.32:0/2415123917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5d160 0x55c89ad5d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc718a26700 1 -- 172.21.15.32:0/2415123917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c89ad0c880 con 0x55c89ad5b960 2023-09-30T14:38:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc713fff700 1 --2- 172.21.15.32:0/2415123917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5c2c0 0x55c89ad5cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc718a26700 1 --2- 172.21.15.32:0/2415123917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5b960 0x55c89ad5bd80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc70800fda0 tx=0x7fc708012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eab089e22db637e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc6feffd700 1 -- 172.21.15.32:0/2415123917 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708011c60 con 0x55c89ad5b960 2023-09-30T14:38:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc6feffd700 1 -- 172.21.15.32:0/2415123917 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc708011dc0 con 0x55c89ad5b960 2023-09-30T14:38:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.425+0000 7fc6feffd700 1 -- 172.21.15.32:0/2415123917 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70801d710 con 0x55c89ad5b960 2023-09-30T14:38:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.426+0000 7fc726e72b80 1 -- 172.21.15.32:0/2415123917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5b960 msgr2=0x55c89ad5bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.426+0000 7fc726e72b80 1 --2- 172.21.15.32:0/2415123917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5b960 0x55c89ad5bd80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc70800fda0 tx=0x7fc708012000 comp rx=0 tx=0).stop 2023-09-30T14:38:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.426+0000 7fc726e72b80 1 -- 172.21.15.32:0/2415123917 shutdown_connections 2023-09-30T14:38:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.426+0000 7fc726e72b80 1 --2- 172.21.15.32:0/2415123917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5d160 0x55c89ad5d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.426+0000 7fc726e72b80 1 --2- 172.21.15.32:0/2415123917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5b960 0x55c89ad5bd80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.426+0000 7fc726e72b80 1 --2- 172.21.15.32:0/2415123917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5c2c0 0x55c89ad5cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.426+0000 7fc726e72b80 1 -- 172.21.15.32:0/2415123917 >> 172.21.15.32:0/2415123917 conn(0x55c89accb210 msgr2=0x55c89ad52430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.426+0000 7fc726e72b80 1 -- 172.21.15.32:0/2415123917 shutdown_connections 2023-09-30T14:38:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.426+0000 7fc726e72b80 1 -- 172.21.15.32:0/2415123917 wait complete. 2023-09-30T14:38:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.426+0000 7fc726e72b80 1 -- start start 2023-09-30T14:38:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.426+0000 7fc726e72b80 1 Processor -- start 2023-09-30T14:38:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc726e72b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5b960 0x55c89ad65770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc726e72b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5c2c0 0x55c89ad65cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc726e72b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5d160 0x55c89ad6ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc726e72b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89aca9de0 con 0x55c89ad5d160 2023-09-30T14:38:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc726e72b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89ac466c0 con 0x55c89ad5b960 2023-09-30T14:38:09.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc726e72b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89ac46360 con 0x55c89ad5c2c0 2023-09-30T14:38:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc726e72b80 10 client.0 _ll_register_callbacks cb 0x55c89aaa37b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc719227700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5d160 0x55c89ad6ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc719227700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5d160 0x55c89ad6ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34632/0 (socket says 172.21.15.32:34632) 2023-09-30T14:38:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc719227700 1 -- 172.21.15.32:0/2387219830 learned_addr learned my addr 172.21.15.32:0/2387219830 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc718a26700 1 --2- 172.21.15.32:0/2387219830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5b960 0x55c89ad65770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc713fff700 1 --2- 172.21.15.32:0/2387219830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5c2c0 0x55c89ad65cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc719227700 1 -- 172.21.15.32:0/2387219830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5c2c0 msgr2=0x55c89ad65cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc719227700 1 --2- 172.21.15.32:0/2387219830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5c2c0 0x55c89ad65cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc719227700 1 -- 172.21.15.32:0/2387219830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5b960 msgr2=0x55c89ad65770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc719227700 1 --2- 172.21.15.32:0/2387219830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5b960 0x55c89ad65770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc719227700 1 -- 172.21.15.32:0/2387219830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c89aad4d90 con 0x55c89ad5d160 2023-09-30T14:38:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc719227700 1 --2- 172.21.15.32:0/2387219830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5d160 0x55c89ad6ccf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc70c015000 tx=0x7fc70c0079c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b0cfb35f68607b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.427+0000 7fc6f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.428+0000 7fc6f3fff700 1 -- 172.21.15.32:0/2387219830 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c00e070 con 0x55c89ad5d160 2023-09-30T14:38:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.428+0000 7fc6f3fff700 1 -- 172.21.15.32:0/2387219830 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc70c005040 con 0x55c89ad5d160 2023-09-30T14:38:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.428+0000 7fc726e72b80 1 -- 172.21.15.32:0/2387219830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c89ad4a200 con 0x55c89ad5d160 2023-09-30T14:38:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.428+0000 7fc6f3fff700 1 -- 172.21.15.32:0/2387219830 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc70c009b80 con 0x55c89ad5d160 2023-09-30T14:38:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.428+0000 7fc726e72b80 10 client.15057 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.428+0000 7fc726e72b80 1 -- 172.21.15.32:0/2387219830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55c89aabdc10 con 0x55c89ad5d160 2023-09-30T14:38:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.428+0000 7fc6f3fff700 1 -- 172.21.15.32:0/2387219830 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c89ad4a200 con 0x55c89ad5d160 2023-09-30T14:38:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.428+0000 7fc6f3fff700 1 -- 172.21.15.32:0/2387219830 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc70c022020 con 0x55c89ad5d160 2023-09-30T14:38:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.428+0000 7fc726e72b80 10 client.15057 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:09.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.428+0000 7fc726e72b80 10 client.15057 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-30T14:38:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.428+0000 7fc726e72b80 20 client.15057 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.428+0000 7fc726e72b80 1 --2- 172.21.15.32:0/2387219830 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x55c89ad71390 0x55c89ad6fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.428+0000 7fc726e72b80 4 client.15057 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:38:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.428+0000 7fc726e72b80 1 -- 172.21.15.32:0/2387219830 --> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c89aca2dc0 con 0x55c89ad71390 2023-09-30T14:38:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.428+0000 7fc718a26700 1 --2- 172.21.15.32:0/2387219830 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x55c89ad71390 0x55c89ad6fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.429+0000 7fc718a26700 1 --2- 172.21.15.32:0/2387219830 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x55c89ad71390 0x55c89ad6fde0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67a0a30d27e0bd0 server_cookie=44b5d17566beb7e5 in_seq=0 out_seq=0 2023-09-30T14:38:09.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.429+0000 7fc6f3fff700 10 client.15057 ms_handle_connect on v2:172.21.15.156:6838/1445176203 2023-09-30T14:38:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.430+0000 7fc6f3fff700 1 -- 172.21.15.32:0/2387219830 <== mds.1 v2:172.21.15.156:6838/1445176203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c89aca2dc0 con 0x55c89ad71390 2023-09-30T14:38:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.430+0000 7fc6f3fff700 10 client.15057 handle_command_reply: tid=0 2023-09-30T14:38:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.430+0000 7fc726e72b80 1 client.15057 shutdown 2023-09-30T14:38:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.430+0000 7fc726e72b80 1 -- 172.21.15.32:0/2387219830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5d160 msgr2=0x55c89ad6ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.430+0000 7fc726e72b80 1 --2- 172.21.15.32:0/2387219830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5d160 0x55c89ad6ccf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc70c015000 tx=0x7fc70c0079c0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.430+0000 7fc726e72b80 1 -- 172.21.15.32:0/2387219830 shutdown_connections 2023-09-30T14:38:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.430+0000 7fc726e72b80 1 --2- 172.21.15.32:0/2387219830 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x55c89ad71390 0x55c89ad6fde0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.430+0000 7fc726e72b80 1 --2- 172.21.15.32:0/2387219830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5d160 0x55c89ad6ccf0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.430+0000 7fc726e72b80 1 --2- 172.21.15.32:0/2387219830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5b960 0x55c89ad65770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.430+0000 7fc6f3fff700 0 client.15057 ms_handle_reset on v2:172.21.15.156:6838/1445176203 2023-09-30T14:38:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.430+0000 7fc726e72b80 1 --2- 172.21.15.32:0/2387219830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5c2c0 0x55c89ad65cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.430+0000 7fc726e72b80 1 -- 172.21.15.32:0/2387219830 >> 172.21.15.32:0/2387219830 conn(0x55c89accb210 msgr2=0x55c89ad52240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.430+0000 7fc726e72b80 1 -- 172.21.15.32:0/2387219830 shutdown_connections 2023-09-30T14:38:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.430+0000 7fc726e72b80 1 -- 172.21.15.32:0/2387219830 wait complete. 2023-09-30T14:38:09.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.430+0000 7fc726e72b80 20 client.15057 trim_cache size 0 max 16384 2023-09-30T14:38:09.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc726e72b80 1 Processor -- start 2023-09-30T14:38:09.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc726e72b80 1 -- start start 2023-09-30T14:38:09.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc726e72b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5b960 0x55c89ad60030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc726e72b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5c2c0 0x55c89ad6ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc726e72b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5d160 0x55c89ad60570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc726e72b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89ab2a8d0 con 0x55c89ad5c2c0 2023-09-30T14:38:09.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc726e72b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89aca5ed0 con 0x55c89ad5d160 2023-09-30T14:38:09.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc726e72b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89ac504d0 con 0x55c89ad5b960 2023-09-30T14:38:09.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc713fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5c2c0 0x55c89ad6ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc713fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5c2c0 0x55c89ad6ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34638/0 (socket says 172.21.15.32:34638) 2023-09-30T14:38:09.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc713fff700 1 -- 172.21.15.32:0/995235443 learned_addr learned my addr 172.21.15.32:0/995235443 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:09.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc718a26700 1 --2- 172.21.15.32:0/995235443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5b960 0x55c89ad60030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc719227700 1 --2- 172.21.15.32:0/995235443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5d160 0x55c89ad60570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc713fff700 1 -- 172.21.15.32:0/995235443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5b960 msgr2=0x55c89ad60030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc713fff700 1 --2- 172.21.15.32:0/995235443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5b960 0x55c89ad60030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc713fff700 1 -- 172.21.15.32:0/995235443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5d160 msgr2=0x55c89ad60570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc713fff700 1 --2- 172.21.15.32:0/995235443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5d160 0x55c89ad60570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc713fff700 1 -- 172.21.15.32:0/995235443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c89aafb200 con 0x55c89ad5c2c0 2023-09-30T14:38:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc718a26700 1 --2- 172.21.15.32:0/995235443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5b960 0x55c89ad60030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc713fff700 1 --2- 172.21.15.32:0/995235443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5c2c0 0x55c89ad6ccb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc700099e70 tx=0x7fc700099aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2efb719c400b4c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.448+0000 7fc6ff7fe700 1 -- 172.21.15.32:0/995235443 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700014020 con 0x55c89ad5c2c0 2023-09-30T14:38:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.449+0000 7fc6ff7fe700 1 -- 172.21.15.32:0/995235443 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc700099ee0 con 0x55c89ad5c2c0 2023-09-30T14:38:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.449+0000 7fc6ff7fe700 1 -- 172.21.15.32:0/995235443 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700010970 con 0x55c89ad5c2c0 2023-09-30T14:38:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.449+0000 7fc726e72b80 1 -- 172.21.15.32:0/995235443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5c2c0 msgr2=0x55c89ad6ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.449+0000 7fc726e72b80 1 --2- 172.21.15.32:0/995235443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5c2c0 0x55c89ad6ccb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc700099e70 tx=0x7fc700099aa0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.449+0000 7fc726e72b80 1 -- 172.21.15.32:0/995235443 shutdown_connections 2023-09-30T14:38:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.449+0000 7fc726e72b80 1 --2- 172.21.15.32:0/995235443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5d160 0x55c89ad60570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.449+0000 7fc726e72b80 1 --2- 172.21.15.32:0/995235443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5b960 0x55c89ad60030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.449+0000 7fc726e72b80 1 --2- 172.21.15.32:0/995235443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5c2c0 0x55c89ad6ccb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.449+0000 7fc726e72b80 1 -- 172.21.15.32:0/995235443 >> 172.21.15.32:0/995235443 conn(0x55c89accb210 msgr2=0x55c89ad64460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.449+0000 7fc726e72b80 1 -- 172.21.15.32:0/995235443 shutdown_connections 2023-09-30T14:38:09.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.449+0000 7fc726e72b80 1 -- 172.21.15.32:0/995235443 wait complete. 2023-09-30T14:38:09.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.449+0000 7fc726e72b80 1 -- start start 2023-09-30T14:38:09.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc726e72b80 1 Processor -- start 2023-09-30T14:38:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc726e72b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5b960 0x55c89ad670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc726e72b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5c2c0 0x55c89ad67630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc726e72b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5d160 0x55c89ad6c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc726e72b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89ad6cb70 con 0x55c89ad5b960 2023-09-30T14:38:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc726e72b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89ad6cce0 con 0x55c89ad5d160 2023-09-30T14:38:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc726e72b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89ad6cea0 con 0x55c89ad5c2c0 2023-09-30T14:38:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc726e72b80 10 client.0 _ll_register_callbacks cb 0x7fc70c009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc718a26700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5b960 0x55c89ad670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc718a26700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5b960 0x55c89ad670f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34642/0 (socket says 172.21.15.32:34642) 2023-09-30T14:38:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc718a26700 1 -- 172.21.15.32:0/2745975266 learned_addr learned my addr 172.21.15.32:0/2745975266 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc713fff700 1 --2- 172.21.15.32:0/2745975266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5c2c0 0x55c89ad67630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc719227700 1 --2- 172.21.15.32:0/2745975266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5d160 0x55c89ad6c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc718a26700 1 -- 172.21.15.32:0/2745975266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5c2c0 msgr2=0x55c89ad67630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc718a26700 1 --2- 172.21.15.32:0/2745975266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5c2c0 0x55c89ad67630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc718a26700 1 -- 172.21.15.32:0/2745975266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5d160 msgr2=0x55c89ad6c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc718a26700 1 --2- 172.21.15.32:0/2745975266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5d160 0x55c89ad6c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc718a26700 1 -- 172.21.15.32:0/2745975266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71419f220 con 0x55c89ad5b960 2023-09-30T14:38:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.450+0000 7fc719227700 1 --2- 172.21.15.32:0/2745975266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5d160 0x55c89ad6c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.451+0000 7fc718a26700 1 --2- 172.21.15.32:0/2745975266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5b960 0x55c89ad670f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc708011ab0 tx=0x7fc708011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1696bb4c296aa270 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.451+0000 7fc6f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.451+0000 7fc6f3fff700 1 -- 172.21.15.32:0/2745975266 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70802e040 con 0x55c89ad5b960 2023-09-30T14:38:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.451+0000 7fc6f3fff700 1 -- 172.21.15.32:0/2745975266 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc708035980 con 0x55c89ad5b960 2023-09-30T14:38:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.451+0000 7fc726e72b80 1 -- 172.21.15.32:0/2745975266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc708011dc0 con 0x55c89ad5b960 2023-09-30T14:38:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.451+0000 7fc6f3fff700 1 -- 172.21.15.32:0/2745975266 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc708035590 con 0x55c89ad5b960 2023-09-30T14:38:09.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.451+0000 7fc726e72b80 10 client.15063 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.451+0000 7fc726e72b80 1 -- 172.21.15.32:0/2745975266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55c89aad3260 con 0x55c89ad5b960 2023-09-30T14:38:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.451+0000 7fc6f3fff700 1 -- 172.21.15.32:0/2745975266 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc708011dc0 con 0x55c89ad5b960 2023-09-30T14:38:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.451+0000 7fc6f3fff700 1 -- 172.21.15.32:0/2745975266 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc708011440 con 0x55c89ad5b960 2023-09-30T14:38:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.451+0000 7fc726e72b80 10 client.15063 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.451+0000 7fc726e72b80 10 client.15063 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-30T14:38:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.451+0000 7fc726e72b80 20 client.15063 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.451+0000 7fc726e72b80 1 --2- 172.21.15.32:0/2745975266 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x55c89ad71390 0x55c89adc5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.451+0000 7fc726e72b80 4 client.15063 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2023-09-30T14:38:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.451+0000 7fc726e72b80 1 -- 172.21.15.32:0/2745975266 --> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c89aca2ae0 con 0x55c89ad71390 2023-09-30T14:38:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.452+0000 7fc713fff700 1 --2- 172.21.15.32:0/2745975266 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x55c89ad71390 0x55c89adc5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.452+0000 7fc713fff700 1 --2- 172.21.15.32:0/2745975266 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x55c89ad71390 0x55c89adc5f80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5edd7bc7425e4613 server_cookie=f6d677ab993f2967 in_seq=0 out_seq=0 2023-09-30T14:38:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.452+0000 7fc6f3fff700 10 client.15063 ms_handle_connect on v2:172.21.15.156:6838/1445176203 2023-09-30T14:38:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.452+0000 7fc6f3fff700 1 -- 172.21.15.32:0/2745975266 <== mds.1 v2:172.21.15.156:6838/1445176203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55c89aca2ae0 con 0x55c89ad71390 2023-09-30T14:38:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.452+0000 7fc6f3fff700 10 client.15063 handle_command_reply: tid=0 2023-09-30T14:38:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.452+0000 7fc726e72b80 1 client.15063 shutdown 2023-09-30T14:38:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.452+0000 7fc726e72b80 1 -- 172.21.15.32:0/2745975266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5b960 msgr2=0x55c89ad670f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.452+0000 7fc726e72b80 1 --2- 172.21.15.32:0/2745975266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5b960 0x55c89ad670f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc708011ab0 tx=0x7fc708011f30 comp rx=0 tx=0).stop 2023-09-30T14:38:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.452+0000 7fc726e72b80 1 -- 172.21.15.32:0/2745975266 shutdown_connections 2023-09-30T14:38:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.452+0000 7fc726e72b80 1 --2- 172.21.15.32:0/2745975266 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x55c89ad71390 0x55c89adc5f80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.452+0000 7fc726e72b80 1 --2- 172.21.15.32:0/2745975266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c89ad5d160 0x55c89ad6c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.452+0000 7fc6f3fff700 0 client.15063 ms_handle_reset on v2:172.21.15.156:6838/1445176203 2023-09-30T14:38:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.452+0000 7fc726e72b80 1 --2- 172.21.15.32:0/2745975266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c89ad5b960 0x55c89ad670f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.452+0000 7fc726e72b80 1 --2- 172.21.15.32:0/2745975266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c89ad5c2c0 0x55c89ad67630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.452+0000 7fc726e72b80 1 -- 172.21.15.32:0/2745975266 >> 172.21.15.32:0/2745975266 conn(0x55c89accb210 msgr2=0x55c89ad60030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.453+0000 7fc726e72b80 1 -- 172.21.15.32:0/2745975266 shutdown_connections 2023-09-30T14:38:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.453+0000 7fc726e72b80 1 -- 172.21.15.32:0/2745975266 wait complete. 2023-09-30T14:38:09.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.453+0000 7fc726e72b80 20 client.15063 trim_cache size 0 max 16384 2023-09-30T14:38:09.460 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T14:38:09.460 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "bca3f5c6-5f9d-11ee-8db5-212e2dc638e7", 2023-09-30T14:38:09.460 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 1, 2023-09-30T14:38:09.460 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24493, 2023-09-30T14:38:09.460 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-09-30T14:38:09.460 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-09-30T14:38:09.461 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-09-30T14:38:09.461 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 54.399734838000001, 2023-09-30T14:38:09.461 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 22, 2023-09-30T14:38:09.461 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 83, 2023-09-30T14:38:09.461 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 83, 2023-09-30T14:38:09.461 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 70.470285249 2023-09-30T14:38:09.461 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T14:38:09.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.453+0000 7fc71ac8a700 1 -- 172.21.15.32:0/2366375943 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc7040a1030 msgr2=0x7fc7040a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.453+0000 7fc71ac8a700 1 --2- 172.21.15.32:0/2366375943 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc7040a1030 0x7fc7040a34f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc714100e00 tx=0x7fc70800b670 comp rx=0 tx=0).stop 2023-09-30T14:38:09.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.453+0000 7fc71ac8a700 1 -- 172.21.15.32:0/2366375943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc714100fa0 msgr2=0x7fc71419b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.453+0000 7fc71ac8a700 1 --2- 172.21.15.32:0/2366375943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc714100fa0 0x7fc71419b980 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc70000d950 tx=0x7fc70000dd10 comp rx=0 tx=0).stop 2023-09-30T14:38:09.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.453+0000 7fc71ac8a700 1 -- 172.21.15.32:0/2366375943 shutdown_connections 2023-09-30T14:38:09.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.453+0000 7fc71ac8a700 1 --2- 172.21.15.32:0/2366375943 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc7040a1030 0x7fc7040a34f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.453+0000 7fc71ac8a700 1 --2- 172.21.15.32:0/2366375943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc714101940 0x7fc71419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.453+0000 7fc71ac8a700 1 --2- 172.21.15.32:0/2366375943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7140ffdb0 0x7fc714069d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.453+0000 7fc71ac8a700 1 --2- 172.21.15.32:0/2366375943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc714100fa0 0x7fc71419b980 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.453+0000 7fc71ac8a700 1 -- 172.21.15.32:0/2366375943 >> 172.21.15.32:0/2366375943 conn(0x7fc7140fb340 msgr2=0x7fc7140fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.453+0000 7fc71ac8a700 1 -- 172.21.15.32:0/2366375943 shutdown_connections 2023-09-30T14:38:09.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.453+0000 7fc71ac8a700 1 -- 172.21.15.32:0/2366375943 wait complete. 2023-09-30T14:38:09.465 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-30T14:38:09.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:09 smithi156 ceph-mon[103579]: pgmap v281: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:09.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:09 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/235389316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:09.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:09 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3857636922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:09.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:09 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2315821472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:09.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.592+0000 7ff5378c3700 1 -- 172.21.15.32:0/901852013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff530100ea0 msgr2=0x7ff5301012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.592+0000 7ff5378c3700 1 --2- 172.21.15.32:0/901852013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff530100ea0 0x7ff5301012c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff520009990 tx=0x7ff520009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.592+0000 7ff5378c3700 1 -- 172.21.15.32:0/901852013 shutdown_connections 2023-09-30T14:38:09.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.592+0000 7ff5378c3700 1 --2- 172.21.15.32:0/901852013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5301029b0 0x7ff5301092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.592+0000 7ff5378c3700 1 --2- 172.21.15.32:0/901852013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff530100ea0 0x7ff5301012c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.592+0000 7ff5378c3700 1 --2- 172.21.15.32:0/901852013 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff530101ff0 0x7ff530102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.592+0000 7ff5378c3700 1 -- 172.21.15.32:0/901852013 >> 172.21.15.32:0/901852013 conn(0x7ff5300fc430 msgr2=0x7ff5300fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.592+0000 7ff5378c3700 1 -- 172.21.15.32:0/901852013 shutdown_connections 2023-09-30T14:38:09.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.592+0000 7ff5378c3700 1 -- 172.21.15.32:0/901852013 wait complete. 2023-09-30T14:38:09.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.592+0000 7ff5378c3700 1 Processor -- start 2023-09-30T14:38:09.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff5378c3700 1 -- start start 2023-09-30T14:38:09.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff5378c3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff530100ea0 0x7ff530196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff5378c3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff530101ff0 0x7ff530197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff5378c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5301029b0 0x7ff53019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff5378c3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53019e690 con 0x7ff5301029b0 2023-09-30T14:38:09.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff5378c3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53019e7d0 con 0x7ff530100ea0 2023-09-30T14:38:09.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff5378c3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53019e960 con 0x7ff530101ff0 2023-09-30T14:38:09.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff535e60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5301029b0 0x7ff53019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff535e60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5301029b0 0x7ff53019e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34660/0 (socket says 172.21.15.32:34660) 2023-09-30T14:38:09.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff535e60700 1 -- 172.21.15.32:0/3410823222 learned_addr learned my addr 172.21.15.32:0/3410823222 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:09.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff53565f700 1 --2- 172.21.15.32:0/3410823222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff530100ea0 0x7ff530196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff534e5e700 1 --2- 172.21.15.32:0/3410823222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff530101ff0 0x7ff530197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff535e60700 1 -- 172.21.15.32:0/3410823222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff530101ff0 msgr2=0x7ff530197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff535e60700 1 --2- 172.21.15.32:0/3410823222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff530101ff0 0x7ff530197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff535e60700 1 -- 172.21.15.32:0/3410823222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff530100ea0 msgr2=0x7ff530196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff535e60700 1 --2- 172.21.15.32:0/3410823222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff530100ea0 0x7ff530196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff535e60700 1 -- 172.21.15.32:0/3410823222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5200095f0 con 0x7ff5301029b0 2023-09-30T14:38:09.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff53565f700 1 --2- 172.21.15.32:0/3410823222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff530100ea0 0x7ff530196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:09.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff534e5e700 1 --2- 172.21.15.32:0/3410823222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff530101ff0 0x7ff530197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:09.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.593+0000 7ff535e60700 1 --2- 172.21.15.32:0/3410823222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5301029b0 0x7ff53019e0c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff528007d70 tx=0x7ff52800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.594+0000 7ff5267fc700 1 -- 172.21.15.32:0/3410823222 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52800ad40 con 0x7ff5301029b0 2023-09-30T14:38:09.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.594+0000 7ff5267fc700 1 -- 172.21.15.32:0/3410823222 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5280128e0 con 0x7ff5301029b0 2023-09-30T14:38:09.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.594+0000 7ff5378c3700 1 -- 172.21.15.32:0/3410823222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53019ec40 con 0x7ff5301029b0 2023-09-30T14:38:09.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.594+0000 7ff5267fc700 1 -- 172.21.15.32:0/3410823222 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528011780 con 0x7ff5301029b0 2023-09-30T14:38:09.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.594+0000 7ff5378c3700 1 -- 172.21.15.32:0/3410823222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53019f150 con 0x7ff5301029b0 2023-09-30T14:38:09.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.596+0000 7ff5267fc700 1 -- 172.21.15.32:0/3410823222 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7ff528012a50 con 0x7ff5301029b0 2023-09-30T14:38:09.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.596+0000 7ff5267fc700 1 --2- 172.21.15.32:0/3410823222 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff51c0a10b0 0x7ff51c0a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.596+0000 7ff5267fc700 1 -- 172.21.15.32:0/3410823222 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff5280cc8d0 con 0x7ff5301029b0 2023-09-30T14:38:09.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.596+0000 7ff53565f700 1 --2- 172.21.15.32:0/3410823222 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff51c0a10b0 0x7ff51c0a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.597+0000 7ff53565f700 1 --2- 172.21.15.32:0/3410823222 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff51c0a10b0 0x7ff51c0a3570 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff520006010 tx=0x7ff52000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.597+0000 7ff543aabb80 1 Processor -- start 2023-09-30T14:38:09.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.597+0000 7ff543aabb80 1 -- start start 2023-09-30T14:38:09.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff543aabb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126016960 0x563126016d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff543aabb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5631260172c0 0x563126017b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff543aabb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563126018160 0x5631260185e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff543aabb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563125f7a410 con 0x563126016960 2023-09-30T14:38:09.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff543aabb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563125f6bbd0 con 0x5631260172c0 2023-09-30T14:38:09.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff543aabb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563125f694f0 con 0x563126018160 2023-09-30T14:38:09.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff53565f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126016960 0x563126016d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff53565f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126016960 0x563126016d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34664/0 (socket says 172.21.15.32:34664) 2023-09-30T14:38:09.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff53565f700 1 -- 172.21.15.32:0/2693818838 learned_addr learned my addr 172.21.15.32:0/2693818838 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:09.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff534e5e700 1 --2- 172.21.15.32:0/2693818838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5631260172c0 0x563126017b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff535e60700 1 --2- 172.21.15.32:0/2693818838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563126018160 0x5631260185e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff53565f700 1 -- 172.21.15.32:0/2693818838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563126018160 msgr2=0x5631260185e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff53565f700 1 --2- 172.21.15.32:0/2693818838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563126018160 0x5631260185e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff53565f700 1 -- 172.21.15.32:0/2693818838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5631260172c0 msgr2=0x563126017b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff53565f700 1 --2- 172.21.15.32:0/2693818838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5631260172c0 0x563126017b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff53565f700 1 -- 172.21.15.32:0/2693818838 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563125fc7880 con 0x563126016960 2023-09-30T14:38:09.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff53565f700 1 --2- 172.21.15.32:0/2693818838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126016960 0x563126016d80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff520005850 tx=0x7ff52001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c1d2401a9ddb64 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff51b7fe700 1 -- 172.21.15.32:0/2693818838 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52002b070 con 0x563126016960 2023-09-30T14:38:09.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff51b7fe700 1 -- 172.21.15.32:0/2693818838 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff52001e040 con 0x563126016960 2023-09-30T14:38:09.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.598+0000 7ff51b7fe700 1 -- 172.21.15.32:0/2693818838 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52000f6c0 con 0x563126016960 2023-09-30T14:38:09.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.599+0000 7ff543aabb80 1 -- 172.21.15.32:0/2693818838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126016960 msgr2=0x563126016d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.599+0000 7ff543aabb80 1 --2- 172.21.15.32:0/2693818838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126016960 0x563126016d80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff520005850 tx=0x7ff52001f000 comp rx=0 tx=0).stop 2023-09-30T14:38:09.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.599+0000 7ff543aabb80 1 -- 172.21.15.32:0/2693818838 shutdown_connections 2023-09-30T14:38:09.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.599+0000 7ff543aabb80 1 --2- 172.21.15.32:0/2693818838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563126018160 0x5631260185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.599+0000 7ff543aabb80 1 --2- 172.21.15.32:0/2693818838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126016960 0x563126016d80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.599+0000 7ff543aabb80 1 --2- 172.21.15.32:0/2693818838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5631260172c0 0x563126017b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.599+0000 7ff543aabb80 1 -- 172.21.15.32:0/2693818838 >> 172.21.15.32:0/2693818838 conn(0x563125f86210 msgr2=0x56312600d430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.599+0000 7ff543aabb80 1 -- 172.21.15.32:0/2693818838 shutdown_connections 2023-09-30T14:38:09.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.599+0000 7ff543aabb80 1 -- 172.21.15.32:0/2693818838 wait complete. 2023-09-30T14:38:09.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff543aabb80 1 -- start start 2023-09-30T14:38:09.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff543aabb80 1 Processor -- start 2023-09-30T14:38:09.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff543aabb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563126016960 0x563126020650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff543aabb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631260172c0 0x563126020b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff543aabb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563126018160 0x563126027bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff543aabb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563125f64de0 con 0x5631260172c0 2023-09-30T14:38:09.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff543aabb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563125f016c0 con 0x563126016960 2023-09-30T14:38:09.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff543aabb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563125f01360 con 0x563126018160 2023-09-30T14:38:09.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff543aabb80 10 client.0 _ll_register_callbacks cb 0x563125d5e7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:09.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff534e5e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631260172c0 0x563126020b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff534e5e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631260172c0 0x563126020b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34672/0 (socket says 172.21.15.32:34672) 2023-09-30T14:38:09.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff534e5e700 1 -- 172.21.15.32:0/1573681954 learned_addr learned my addr 172.21.15.32:0/1573681954 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:09.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff53565f700 1 --2- 172.21.15.32:0/1573681954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563126016960 0x563126020650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff535e60700 1 --2- 172.21.15.32:0/1573681954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563126018160 0x563126027bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff534e5e700 1 -- 172.21.15.32:0/1573681954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563126018160 msgr2=0x563126027bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff534e5e700 1 --2- 172.21.15.32:0/1573681954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563126018160 0x563126027bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff534e5e700 1 -- 172.21.15.32:0/1573681954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563126016960 msgr2=0x563126020650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff534e5e700 1 --2- 172.21.15.32:0/1573681954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563126016960 0x563126020650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.600+0000 7ff534e5e700 1 -- 172.21.15.32:0/1573681954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff530109c00 con 0x5631260172c0 2023-09-30T14:38:09.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.601+0000 7ff534e5e700 1 --2- 172.21.15.32:0/1573681954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631260172c0 0x563126020b90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff52c010a20 tx=0x7ff52c010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47571dd833d7f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.601+0000 7ff518ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:09.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.601+0000 7ff518ff9700 1 -- 172.21.15.32:0/1573681954 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c00b890 con 0x5631260172c0 2023-09-30T14:38:09.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.601+0000 7ff518ff9700 1 -- 172.21.15.32:0/1573681954 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff52c00ecb0 con 0x5631260172c0 2023-09-30T14:38:09.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.601+0000 7ff543aabb80 1 -- 172.21.15.32:0/1573681954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563126005200 con 0x5631260172c0 2023-09-30T14:38:09.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.601+0000 7ff518ff9700 1 -- 172.21.15.32:0/1573681954 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff52c00ee70 con 0x5631260172c0 2023-09-30T14:38:09.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.601+0000 7ff543aabb80 10 client.15075 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:09.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.601+0000 7ff543aabb80 1 -- 172.21.15.32:0/1573681954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x563125d8fd90 con 0x5631260172c0 2023-09-30T14:38:09.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.601+0000 7ff518ff9700 1 -- 172.21.15.32:0/1573681954 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563126005200 con 0x5631260172c0 2023-09-30T14:38:09.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.601+0000 7ff518ff9700 1 -- 172.21.15.32:0/1573681954 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff52c00d650 con 0x5631260172c0 2023-09-30T14:38:09.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.601+0000 7ff543aabb80 10 client.15075 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:09.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.601+0000 7ff543aabb80 10 client.15075 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-30T14:38:09.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.601+0000 7ff543aabb80 20 client.15075 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:09.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.601+0000 7ff543aabb80 1 --2- 172.21.15.32:0/1573681954 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56312602c2a0 0x56312602ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.601+0000 7ff543aabb80 4 client.15075 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:38:09.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.601+0000 7ff543aabb80 1 -- 172.21.15.32:0/1573681954 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563125f5ddc0 con 0x56312602c2a0 2023-09-30T14:38:09.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.602+0000 7ff53565f700 1 --2- 172.21.15.32:0/1573681954 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56312602c2a0 0x56312602ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.602+0000 7ff518ff9700 10 client.15075 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:09.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.602+0000 7ff53565f700 1 --2- 172.21.15.32:0/1573681954 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56312602c2a0 0x56312602ad30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9d656c6c65f6447 server_cookie=ad50cdee72bf5169 in_seq=0 out_seq=0 2023-09-30T14:38:09.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.603+0000 7ff518ff9700 1 -- 172.21.15.32:0/1573681954 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563125f5ddc0 con 0x56312602c2a0 2023-09-30T14:38:09.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.603+0000 7ff518ff9700 10 client.15075 handle_command_reply: tid=0 2023-09-30T14:38:09.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.603+0000 7ff543aabb80 1 client.15075 shutdown 2023-09-30T14:38:09.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.603+0000 7ff543aabb80 1 -- 172.21.15.32:0/1573681954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631260172c0 msgr2=0x563126020b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.603+0000 7ff543aabb80 1 --2- 172.21.15.32:0/1573681954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631260172c0 0x563126020b90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff52c010a20 tx=0x7ff52c010de0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.603+0000 7ff543aabb80 1 -- 172.21.15.32:0/1573681954 shutdown_connections 2023-09-30T14:38:09.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.603+0000 7ff543aabb80 1 --2- 172.21.15.32:0/1573681954 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56312602c2a0 0x56312602ad30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.603+0000 7ff543aabb80 1 --2- 172.21.15.32:0/1573681954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563126018160 0x563126027bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.603+0000 7ff543aabb80 1 --2- 172.21.15.32:0/1573681954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563126016960 0x563126020650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.603+0000 7ff543aabb80 1 --2- 172.21.15.32:0/1573681954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5631260172c0 0x563126020b90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.603+0000 7ff518ff9700 0 client.15075 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:09.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.603+0000 7ff543aabb80 1 -- 172.21.15.32:0/1573681954 >> 172.21.15.32:0/1573681954 conn(0x563125f86210 msgr2=0x56312600d240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.603+0000 7ff543aabb80 1 -- 172.21.15.32:0/1573681954 shutdown_connections 2023-09-30T14:38:09.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.603+0000 7ff543aabb80 1 -- 172.21.15.32:0/1573681954 wait complete. 2023-09-30T14:38:09.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.603+0000 7ff543aabb80 20 client.15075 trim_cache size 0 max 16384 2023-09-30T14:38:09.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:09 smithi148 ceph-mon[102355]: pgmap v281: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:09.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:09 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/235389316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:09.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:09 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3857636922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:09.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:09 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2315821472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:09.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff543aabb80 1 Processor -- start 2023-09-30T14:38:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff543aabb80 1 -- start start 2023-09-30T14:38:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff543aabb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563126016960 0x56312601f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff543aabb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5631260172c0 0x563126027b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff543aabb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126018160 0x56312601b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff543aabb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563125de58d0 con 0x563126018160 2023-09-30T14:38:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff543aabb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563125f60ed0 con 0x5631260172c0 2023-09-30T14:38:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff543aabb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563125f0b4d0 con 0x563126016960 2023-09-30T14:38:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff535e60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126018160 0x56312601b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff535e60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126018160 0x56312601b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34682/0 (socket says 172.21.15.32:34682) 2023-09-30T14:38:09.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff535e60700 1 -- 172.21.15.32:0/753868824 learned_addr learned my addr 172.21.15.32:0/753868824 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff53565f700 1 --2- 172.21.15.32:0/753868824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563126016960 0x56312601f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff534e5e700 1 --2- 172.21.15.32:0/753868824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5631260172c0 0x563126027b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff535e60700 1 -- 172.21.15.32:0/753868824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563126016960 msgr2=0x56312601f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff535e60700 1 --2- 172.21.15.32:0/753868824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563126016960 0x56312601f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff535e60700 1 -- 172.21.15.32:0/753868824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5631260172c0 msgr2=0x563126027b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff535e60700 1 --2- 172.21.15.32:0/753868824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5631260172c0 0x563126027b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff535e60700 1 -- 172.21.15.32:0/753868824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563125d78c10 con 0x563126018160 2023-09-30T14:38:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.621+0000 7ff535e60700 1 --2- 172.21.15.32:0/753868824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126018160 0x56312601b0d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff528099e80 tx=0x7ff528007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa678ddb5f8a26f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.622+0000 7ff51bfff700 1 -- 172.21.15.32:0/753868824 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52809e040 con 0x563126018160 2023-09-30T14:38:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.622+0000 7ff51bfff700 1 -- 172.21.15.32:0/753868824 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5280a2280 con 0x563126018160 2023-09-30T14:38:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.622+0000 7ff51bfff700 1 -- 172.21.15.32:0/753868824 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5280adb30 con 0x563126018160 2023-09-30T14:38:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.622+0000 7ff543aabb80 1 -- 172.21.15.32:0/753868824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126018160 msgr2=0x56312601b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.622+0000 7ff543aabb80 1 --2- 172.21.15.32:0/753868824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126018160 0x56312601b0d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff528099e80 tx=0x7ff528007f70 comp rx=0 tx=0).stop 2023-09-30T14:38:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.622+0000 7ff543aabb80 1 -- 172.21.15.32:0/753868824 shutdown_connections 2023-09-30T14:38:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.622+0000 7ff543aabb80 1 --2- 172.21.15.32:0/753868824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126018160 0x56312601b0d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.622+0000 7ff543aabb80 1 --2- 172.21.15.32:0/753868824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563126016960 0x56312601f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.622+0000 7ff543aabb80 1 --2- 172.21.15.32:0/753868824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5631260172c0 0x563126027b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.622+0000 7ff543aabb80 1 -- 172.21.15.32:0/753868824 >> 172.21.15.32:0/753868824 conn(0x563125f86210 msgr2=0x563126034c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.622+0000 7ff543aabb80 1 -- 172.21.15.32:0/753868824 shutdown_connections 2023-09-30T14:38:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.622+0000 7ff543aabb80 1 -- 172.21.15.32:0/753868824 wait complete. 2023-09-30T14:38:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff543aabb80 1 -- start start 2023-09-30T14:38:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff543aabb80 1 Processor -- start 2023-09-30T14:38:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff543aabb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563126016960 0x5631260247d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff543aabb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631260172c0 0x563126024d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff543aabb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126018160 0x56312607fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff543aabb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563126027d30 con 0x563126018160 2023-09-30T14:38:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff543aabb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563126027ea0 con 0x563126016960 2023-09-30T14:38:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff543aabb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631260801d0 con 0x5631260172c0 2023-09-30T14:38:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff543aabb80 10 client.0 _ll_register_callbacks cb 0x7ff52c00bd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff535e60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126018160 0x56312607fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff535e60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126018160 0x56312607fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34688/0 (socket says 172.21.15.32:34688) 2023-09-30T14:38:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff535e60700 1 -- 172.21.15.32:0/2297387951 learned_addr learned my addr 172.21.15.32:0/2297387951 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff534e5e700 1 --2- 172.21.15.32:0/2297387951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631260172c0 0x563126024d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff53565f700 1 --2- 172.21.15.32:0/2297387951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563126016960 0x5631260247d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff535e60700 1 -- 172.21.15.32:0/2297387951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631260172c0 msgr2=0x563126024d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff535e60700 1 --2- 172.21.15.32:0/2297387951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631260172c0 0x563126024d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff535e60700 1 -- 172.21.15.32:0/2297387951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563126016960 msgr2=0x5631260247d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff535e60700 1 --2- 172.21.15.32:0/2297387951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563126016960 0x5631260247d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.623+0000 7ff535e60700 1 -- 172.21.15.32:0/2297387951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53019f150 con 0x563126018160 2023-09-30T14:38:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.624+0000 7ff535e60700 1 --2- 172.21.15.32:0/2297387951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126018160 0x56312607fc90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff5280a1d50 tx=0x7ff5280a2100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fe52606dac3903 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.624+0000 7ff518ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.624+0000 7ff518ff9700 1 -- 172.21.15.32:0/2297387951 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5280995f0 con 0x563126018160 2023-09-30T14:38:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.624+0000 7ff518ff9700 1 -- 172.21.15.32:0/2297387951 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff528098290 con 0x563126018160 2023-09-30T14:38:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.624+0000 7ff543aabb80 1 -- 172.21.15.32:0/2297387951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff52001e040 con 0x563126018160 2023-09-30T14:38:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.624+0000 7ff518ff9700 1 -- 172.21.15.32:0/2297387951 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff528012420 con 0x563126018160 2023-09-30T14:38:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.624+0000 7ff543aabb80 10 client.15081 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.624+0000 7ff543aabb80 1 -- 172.21.15.32:0/2297387951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x563125db6200 con 0x563126018160 2023-09-30T14:38:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.624+0000 7ff518ff9700 1 -- 172.21.15.32:0/2297387951 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5280a6080 con 0x563126018160 2023-09-30T14:38:09.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.624+0000 7ff518ff9700 1 -- 172.21.15.32:0/2297387951 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff528095ce0 con 0x563126018160 2023-09-30T14:38:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.624+0000 7ff543aabb80 10 client.15081 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.624+0000 7ff543aabb80 10 client.15081 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-30T14:38:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.624+0000 7ff543aabb80 20 client.15081 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.624+0000 7ff543aabb80 1 --2- 172.21.15.32:0/2297387951 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5631260850a0 0x563126081620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.624+0000 7ff543aabb80 4 client.15081 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2023-09-30T14:38:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.624+0000 7ff543aabb80 1 -- 172.21.15.32:0/2297387951 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563125f5dae0 con 0x5631260850a0 2023-09-30T14:38:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff53565f700 1 --2- 172.21.15.32:0/2297387951 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5631260850a0 0x563126081620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff518ff9700 10 client.15081 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff53565f700 1 --2- 172.21.15.32:0/2297387951 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5631260850a0 0x563126081620 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8cc29e78478f1e0 server_cookie=9f2c4e912f059960 in_seq=0 out_seq=0 2023-09-30T14:38:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff518ff9700 1 -- 172.21.15.32:0/2297387951 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563125f5dae0 con 0x5631260850a0 2023-09-30T14:38:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff518ff9700 10 client.15081 handle_command_reply: tid=0 2023-09-30T14:38:09.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff543aabb80 1 client.15081 shutdown 2023-09-30T14:38:09.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff543aabb80 1 -- 172.21.15.32:0/2297387951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126018160 msgr2=0x56312607fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff543aabb80 1 --2- 172.21.15.32:0/2297387951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126018160 0x56312607fc90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff5280a1d50 tx=0x7ff5280a2100 comp rx=0 tx=0).stop 2023-09-30T14:38:09.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff543aabb80 1 -- 172.21.15.32:0/2297387951 shutdown_connections 2023-09-30T14:38:09.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff543aabb80 1 --2- 172.21.15.32:0/2297387951 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5631260850a0 0x563126081620 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff543aabb80 1 --2- 172.21.15.32:0/2297387951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563126018160 0x56312607fc90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff518ff9700 0 client.15081 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:09.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff543aabb80 1 --2- 172.21.15.32:0/2297387951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563126016960 0x5631260247d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff543aabb80 1 --2- 172.21.15.32:0/2297387951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5631260172c0 0x563126024d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff543aabb80 1 -- 172.21.15.32:0/2297387951 >> 172.21.15.32:0/2297387951 conn(0x563125f86210 msgr2=0x56312601f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff543aabb80 1 -- 172.21.15.32:0/2297387951 shutdown_connections 2023-09-30T14:38:09.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.625+0000 7ff543aabb80 1 -- 172.21.15.32:0/2297387951 wait complete. 2023-09-30T14:38:09.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.626+0000 7ff543aabb80 20 client.15081 trim_cache size 0 max 16384 2023-09-30T14:38:09.633 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T14:38:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "bca3f5c6-5f9d-11ee-8db5-212e2dc638e7", 2023-09-30T14:38:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-09-30T14:38:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24515, 2023-09-30T14:38:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-09-30T14:38:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-09-30T14:38:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-09-30T14:38:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 57.591993735000003, 2023-09-30T14:38:09.633 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 22, 2023-09-30T14:38:09.634 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 83, 2023-09-30T14:38:09.634 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 83, 2023-09-30T14:38:09.634 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 63.800344377999998 2023-09-30T14:38:09.634 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T14:38:09.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.626+0000 7ff5378c3700 1 -- 172.21.15.32:0/3410823222 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff51c0a10b0 msgr2=0x7ff51c0a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.626+0000 7ff5378c3700 1 --2- 172.21.15.32:0/3410823222 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff51c0a10b0 0x7ff51c0a3570 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff520006010 tx=0x7ff52000b540 comp rx=0 tx=0).stop 2023-09-30T14:38:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.626+0000 7ff5378c3700 1 -- 172.21.15.32:0/3410823222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5301029b0 msgr2=0x7ff53019e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.626+0000 7ff5378c3700 1 --2- 172.21.15.32:0/3410823222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5301029b0 0x7ff53019e0c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff528007d70 tx=0x7ff52800a340 comp rx=0 tx=0).stop 2023-09-30T14:38:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.626+0000 7ff5378c3700 1 -- 172.21.15.32:0/3410823222 shutdown_connections 2023-09-30T14:38:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.626+0000 7ff5378c3700 1 --2- 172.21.15.32:0/3410823222 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff51c0a10b0 0x7ff51c0a3570 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.626+0000 7ff5378c3700 1 --2- 172.21.15.32:0/3410823222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5301029b0 0x7ff53019e0c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.626+0000 7ff5378c3700 1 --2- 172.21.15.32:0/3410823222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff530100ea0 0x7ff530196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.626+0000 7ff5378c3700 1 --2- 172.21.15.32:0/3410823222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff530101ff0 0x7ff530197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.626+0000 7ff5378c3700 1 -- 172.21.15.32:0/3410823222 >> 172.21.15.32:0/3410823222 conn(0x7ff5300fc430 msgr2=0x7ff5300fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:09.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.626+0000 7ff5378c3700 1 -- 172.21.15.32:0/3410823222 shutdown_connections 2023-09-30T14:38:09.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:09.626+0000 7ff5378c3700 1 -- 172.21.15.32:0/3410823222 wait complete. 2023-09-30T14:38:09.638 INFO:tasks.cephadm:Setup complete, yielding 2023-09-30T14:38:09.638 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-30T14:38:09.649 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T14:38:09.649 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch status' 2023-09-30T14:38:10.209 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:38:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.418+0000 7fe1717ae700 1 -- 172.21.15.32:0/2195815529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c0ffe10 msgr2=0x7fe16c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.418+0000 7fe1717ae700 1 --2- 172.21.15.32:0/2195815529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c0ffe10 0x7fe16c100230 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009990 tx=0x7fe15c009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.418+0000 7fe1717ae700 1 -- 172.21.15.32:0/2195815529 shutdown_connections 2023-09-30T14:38:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.418+0000 7fe1717ae700 1 --2- 172.21.15.32:0/2195815529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe16c101930 0x7fe16c108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.418+0000 7fe1717ae700 1 --2- 172.21.15.32:0/2195815529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c0ffe10 0x7fe16c100230 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.418+0000 7fe1717ae700 1 --2- 172.21.15.32:0/2195815529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe16c100f70 0x7fe16c1013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.418+0000 7fe1717ae700 1 -- 172.21.15.32:0/2195815529 >> 172.21.15.32:0/2195815529 conn(0x7fe16c0fb3b0 msgr2=0x7fe16c0fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.418+0000 7fe1717ae700 1 -- 172.21.15.32:0/2195815529 shutdown_connections 2023-09-30T14:38:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.418+0000 7fe1717ae700 1 -- 172.21.15.32:0/2195815529 wait complete. 2023-09-30T14:38:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.418+0000 7fe1717ae700 1 Processor -- start 2023-09-30T14:38:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe1717ae700 1 -- start start 2023-09-30T14:38:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe1717ae700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe16c0ffe10 0x7fe16c196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe1717ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c100f70 0x7fe16c197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe1717ae700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe16c101930 0x7fe16c19e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe1717ae700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c19e750 con 0x7fe16c100f70 2023-09-30T14:38:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe1717ae700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c19e890 con 0x7fe16c0ffe10 2023-09-30T14:38:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe1717ae700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c19ea20 con 0x7fe16c101930 2023-09-30T14:38:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe16a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c100f70 0x7fe16c197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe16a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c100f70 0x7fe16c197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34698/0 (socket says 172.21.15.32:34698) 2023-09-30T14:38:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe16a7fc700 1 -- 172.21.15.32:0/1535126766 learned_addr learned my addr 172.21.15.32:0/1535126766 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe16b7fe700 1 --2- 172.21.15.32:0/1535126766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe16c101930 0x7fe16c19e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe16affd700 1 --2- 172.21.15.32:0/1535126766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe16c0ffe10 0x7fe16c196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe16a7fc700 1 -- 172.21.15.32:0/1535126766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe16c101930 msgr2=0x7fe16c19e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe16a7fc700 1 --2- 172.21.15.32:0/1535126766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe16c101930 0x7fe16c19e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe16a7fc700 1 -- 172.21.15.32:0/1535126766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe16c0ffe10 msgr2=0x7fe16c196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe16a7fc700 1 --2- 172.21.15.32:0/1535126766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe16c0ffe10 0x7fe16c196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe16a7fc700 1 -- 172.21.15.32:0/1535126766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c0095f0 con 0x7fe16c100f70 2023-09-30T14:38:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.419+0000 7fe16affd700 1 --2- 172.21.15.32:0/1535126766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe16c0ffe10 0x7fe16c196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.420+0000 7fe16a7fc700 1 --2- 172.21.15.32:0/1535126766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c100f70 0x7fe16c197100 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe16000d980 tx=0x7fe16000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.420+0000 7fe153fff700 1 -- 172.21.15.32:0/1535126766 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe160009b40 con 0x7fe16c100f70 2023-09-30T14:38:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.420+0000 7fe153fff700 1 -- 172.21.15.32:0/1535126766 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe16000b6e0 con 0x7fe16c100f70 2023-09-30T14:38:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.420+0000 7fe153fff700 1 -- 172.21.15.32:0/1535126766 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16000f620 con 0x7fe16c100f70 2023-09-30T14:38:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.420+0000 7fe1717ae700 1 -- 172.21.15.32:0/1535126766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16c19ed00 con 0x7fe16c100f70 2023-09-30T14:38:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.420+0000 7fe1717ae700 1 -- 172.21.15.32:0/1535126766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16c19f1b0 con 0x7fe16c100f70 2023-09-30T14:38:11.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.421+0000 7fe1717ae700 1 -- 172.21.15.32:0/1535126766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16c10b620 con 0x7fe16c100f70 2023-09-30T14:38:11.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.422+0000 7fe153fff700 1 -- 172.21.15.32:0/1535126766 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fe160010460 con 0x7fe16c100f70 2023-09-30T14:38:11.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.422+0000 7fe153fff700 1 --2- 172.21.15.32:0/1535126766 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1540a10b0 0x7fe1540a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:11.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.423+0000 7fe16affd700 1 --2- 172.21.15.32:0/1535126766 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1540a10b0 0x7fe1540a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:11.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.423+0000 7fe153fff700 1 -- 172.21.15.32:0/1535126766 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe1600cc9e0 con 0x7fe16c100f70 2023-09-30T14:38:11.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.423+0000 7fe16affd700 1 --2- 172.21.15.32:0/1535126766 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1540a10b0 0x7fe1540a3570 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe15c006010 tx=0x7fe15c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:11.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.427+0000 7fe153fff700 1 -- 172.21.15.32:0/1535126766 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fe160096130 con 0x7fe16c100f70 2023-09-30T14:38:11.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:11 smithi156 ceph-mon[103579]: pgmap v282: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:12.203 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:11 smithi032 ceph-mon[98592]: pgmap v282: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:12.203 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:11 smithi148 ceph-mon[102355]: pgmap v282: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:12.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.644+0000 7fe1717ae700 1 -- 172.21.15.32:0/1535126766 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe16c069a60 con 0x7fe1540a10b0 2023-09-30T14:38:12.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.646+0000 7fe153fff700 1 -- 172.21.15.32:0/1535126766 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe16c069a60 con 0x7fe1540a10b0 2023-09-30T14:38:12.204 INFO:teuthology.orchestra.run.smithi032.stdout:Backend: cephadm 2023-09-30T14:38:12.204 INFO:teuthology.orchestra.run.smithi032.stdout:Available: Yes 2023-09-30T14:38:12.204 INFO:teuthology.orchestra.run.smithi032.stdout:Paused: No 2023-09-30T14:38:12.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.648+0000 7fe1717ae700 1 -- 172.21.15.32:0/1535126766 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1540a10b0 msgr2=0x7fe1540a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:12.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.648+0000 7fe1717ae700 1 --2- 172.21.15.32:0/1535126766 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1540a10b0 0x7fe1540a3570 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe15c006010 tx=0x7fe15c00b540 comp rx=0 tx=0).stop 2023-09-30T14:38:12.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.648+0000 7fe1717ae700 1 -- 172.21.15.32:0/1535126766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c100f70 msgr2=0x7fe16c197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:12.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.648+0000 7fe1717ae700 1 --2- 172.21.15.32:0/1535126766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c100f70 0x7fe16c197100 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe16000d980 tx=0x7fe16000dc90 comp rx=0 tx=0).stop 2023-09-30T14:38:12.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.648+0000 7fe1717ae700 1 -- 172.21.15.32:0/1535126766 shutdown_connections 2023-09-30T14:38:12.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.648+0000 7fe1717ae700 1 --2- 172.21.15.32:0/1535126766 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1540a10b0 0x7fe1540a3570 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:12.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.648+0000 7fe1717ae700 1 --2- 172.21.15.32:0/1535126766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe16c101930 0x7fe16c19e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:12.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.648+0000 7fe1717ae700 1 --2- 172.21.15.32:0/1535126766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe16c0ffe10 0x7fe16c196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:12.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.648+0000 7fe1717ae700 1 --2- 172.21.15.32:0/1535126766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16c100f70 0x7fe16c197100 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:12.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.648+0000 7fe1717ae700 1 -- 172.21.15.32:0/1535126766 >> 172.21.15.32:0/1535126766 conn(0x7fe16c0fb3b0 msgr2=0x7fe16c069350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:12.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.648+0000 7fe1717ae700 1 -- 172.21.15.32:0/1535126766 shutdown_connections 2023-09-30T14:38:12.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:11.648+0000 7fe1717ae700 1 -- 172.21.15.32:0/1535126766 wait complete. 2023-09-30T14:38:12.239 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-09-30T14:38:12.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-09-30T14:38:12.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-09-30T14:38:12.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2023-09-30T14:38:12.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2023-09-30T14:38:12.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2023-09-30T14:38:12.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-09-30T14:38:12.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-09-30T14:38:12.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-09-30T14:38:12.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:12 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-09-30T14:38:12.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:12 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-09-30T14:38:12.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:12 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2023-09-30T14:38:12.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:12 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2023-09-30T14:38:12.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:12 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2023-09-30T14:38:12.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:12 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-09-30T14:38:12.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:12 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-09-30T14:38:12.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:12 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-09-30T14:38:12.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:12 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-09-30T14:38:12.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:12 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-09-30T14:38:12.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:12 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2023-09-30T14:38:12.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:12 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2023-09-30T14:38:12.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:12 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2023-09-30T14:38:12.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:12 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-09-30T14:38:12.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:12 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-09-30T14:38:12.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:12 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-09-30T14:38:12.809 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:38:13.365 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:13 smithi032 ceph-mon[98592]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:38:13.365 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:13 smithi032 ceph-mon[98592]: pgmap v283: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:38:13.365 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-09-30T14:38:13.365 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-09-30T14:38:13.365 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2023-09-30T14:38:13.365 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2023-09-30T14:38:13.365 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2023-09-30T14:38:13.365 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-09-30T14:38:13.365 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-09-30T14:38:13.366 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-09-30T14:38:13.366 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:13 smithi032 ceph-mon[98592]: osdmap e84: 12 total, 12 up, 12 in 2023-09-30T14:38:13.366 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:13 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:38:13.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:13 smithi156 ceph-mon[103579]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:38:13.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:13 smithi156 ceph-mon[103579]: pgmap v283: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:38:13.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-09-30T14:38:13.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-09-30T14:38:13.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2023-09-30T14:38:13.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2023-09-30T14:38:13.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2023-09-30T14:38:13.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-09-30T14:38:13.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-09-30T14:38:13.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-09-30T14:38:13.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:13 smithi156 ceph-mon[103579]: osdmap e84: 12 total, 12 up, 12 in 2023-09-30T14:38:13.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:13 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:38:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:13 smithi148 ceph-mon[102355]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:38:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:13 smithi148 ceph-mon[102355]: pgmap v283: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:38:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-09-30T14:38:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-09-30T14:38:13.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2023-09-30T14:38:13.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2023-09-30T14:38:13.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2023-09-30T14:38:13.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-09-30T14:38:13.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-09-30T14:38:13.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-09-30T14:38:13.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:13 smithi148 ceph-mon[102355]: osdmap e84: 12 total, 12 up, 12 in 2023-09-30T14:38:13.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:13 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:38:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.059+0000 7f4450501700 1 -- 172.21.15.32:0/284626883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44480ffe20 msgr2=0x7f4448100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.059+0000 7f4450501700 1 --2- 172.21.15.32:0/284626883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44480ffe20 0x7f4448100240 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4444009990 tx=0x7f4444009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.059+0000 7f4450501700 1 -- 172.21.15.32:0/284626883 shutdown_connections 2023-09-30T14:38:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.059+0000 7f4450501700 1 --2- 172.21.15.32:0/284626883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4448101940 0x7f4448108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.059+0000 7f4450501700 1 --2- 172.21.15.32:0/284626883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44480ffe20 0x7f4448100240 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.059+0000 7f4450501700 1 --2- 172.21.15.32:0/284626883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4448100f80 0x7f4448101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.059+0000 7f4450501700 1 -- 172.21.15.32:0/284626883 >> 172.21.15.32:0/284626883 conn(0x7f44480fb3c0 msgr2=0x7f44480fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.059+0000 7f4450501700 1 -- 172.21.15.32:0/284626883 shutdown_connections 2023-09-30T14:38:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.059+0000 7f4450501700 1 -- 172.21.15.32:0/284626883 wait complete. 2023-09-30T14:38:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.059+0000 7f4450501700 1 Processor -- start 2023-09-30T14:38:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.059+0000 7f4450501700 1 -- start start 2023-09-30T14:38:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f4450501700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44480ffe20 0x7f4448103da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f4450501700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4448100f80 0x7f44481023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f4450501700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4448101940 0x7f4448102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f4450501700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44481030b0 con 0x7f4448100f80 2023-09-30T14:38:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f4450501700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4448103220 con 0x7f44480ffe20 2023-09-30T14:38:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f4450501700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44481a0d00 con 0x7f4448101940 2023-09-30T14:38:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f444da9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4448100f80 0x7f44481023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f444da9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4448100f80 0x7f44481023f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34714/0 (socket says 172.21.15.32:34714) 2023-09-30T14:38:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f444da9c700 1 -- 172.21.15.32:0/2267975771 learned_addr learned my addr 172.21.15.32:0/2267975771 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f444e29d700 1 --2- 172.21.15.32:0/2267975771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44480ffe20 0x7f4448103da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f444ea9e700 1 --2- 172.21.15.32:0/2267975771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4448101940 0x7f4448102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f444da9c700 1 -- 172.21.15.32:0/2267975771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4448101940 msgr2=0x7f4448102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f444da9c700 1 --2- 172.21.15.32:0/2267975771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4448101940 0x7f4448102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f444da9c700 1 -- 172.21.15.32:0/2267975771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44480ffe20 msgr2=0x7f4448103da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f444da9c700 1 --2- 172.21.15.32:0/2267975771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44480ffe20 0x7f4448103da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f444da9c700 1 -- 172.21.15.32:0/2267975771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44440095f0 con 0x7f4448100f80 2023-09-30T14:38:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f444da9c700 1 --2- 172.21.15.32:0/2267975771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4448100f80 0x7f44481023f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f443800b530 tx=0x7f443800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.060+0000 7f443f7fe700 1 -- 172.21.15.32:0/2267975771 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438011830 con 0x7f4448100f80 2023-09-30T14:38:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.061+0000 7f4450501700 1 -- 172.21.15.32:0/2267975771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44481a0fe0 con 0x7f4448100f80 2023-09-30T14:38:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.061+0000 7f443f7fe700 1 -- 172.21.15.32:0/2267975771 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4438011e50 con 0x7f4448100f80 2023-09-30T14:38:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.061+0000 7f443f7fe700 1 -- 172.21.15.32:0/2267975771 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443800f2e0 con 0x7f4448100f80 2023-09-30T14:38:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.061+0000 7f4450501700 1 -- 172.21.15.32:0/2267975771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44481a14a0 con 0x7f4448100f80 2023-09-30T14:38:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.061+0000 7f4450501700 1 -- 172.21.15.32:0/2267975771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444810b6b0 con 0x7f4448100f80 2023-09-30T14:38:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.063+0000 7f443f7fe700 1 -- 172.21.15.32:0/2267975771 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f44380105a0 con 0x7f4448100f80 2023-09-30T14:38:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.063+0000 7f443f7fe700 1 --2- 172.21.15.32:0/2267975771 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44340a10b0 0x7f44340a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.063+0000 7f444e29d700 1 --2- 172.21.15.32:0/2267975771 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44340a10b0 0x7f44340a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.063+0000 7f443f7fe700 1 -- 172.21.15.32:0/2267975771 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f44380cc630 con 0x7f4448100f80 2023-09-30T14:38:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.064+0000 7f444e29d700 1 --2- 172.21.15.32:0/2267975771 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44340a10b0 0x7f44340a3570 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f444400b5c0 tx=0x7f4444005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.067+0000 7f443f7fe700 1 -- 172.21.15.32:0/2267975771 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f4438095df0 con 0x7f4448100f80 2023-09-30T14:38:14.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.289+0000 7f4450501700 1 -- 172.21.15.32:0/2267975771 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f44480699c0 con 0x7f44340a10b0 2023-09-30T14:38:14.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.309+0000 7f443f7fe700 1 -- 172.21.15.32:0/2267975771 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f44480699c0 con 0x7f44340a10b0 2023-09-30T14:38:14.309 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-09-30T14:38:14.309 INFO:teuthology.orchestra.run.smithi032.stdout:mds.a smithi032 running (71s) - 70s 16.0M - 18.0.0-6447-g1d10b717 dda7fab67785 8dedb073a3bf 2023-09-30T14:38:14.309 INFO:teuthology.orchestra.run.smithi032.stdout:mds.b smithi148 running (68s) - 68s 15.6M - 18.0.0-6447-g1d10b717 dda7fab67785 c1e52d114429 2023-09-30T14:38:14.309 INFO:teuthology.orchestra.run.smithi032.stdout:mds.c smithi156 running (80s) - 79s 14.4M - 18.0.0-6447-g1d10b717 dda7fab67785 d48c672e0ab4 2023-09-30T14:38:14.309 INFO:teuthology.orchestra.run.smithi032.stdout:mds.d smithi032 running (86s) - 86s 13.4M - 18.0.0-6447-g1d10b717 dda7fab67785 4c655f88641f 2023-09-30T14:38:14.309 INFO:teuthology.orchestra.run.smithi032.stdout:mds.e smithi148 running (84s) - 84s 15.7M - 18.0.0-6447-g1d10b717 dda7fab67785 a261c55a3699 2023-09-30T14:38:14.309 INFO:teuthology.orchestra.run.smithi032.stdout:mds.f smithi156 running (82s) - 82s 15.7M - 18.0.0-6447-g1d10b717 dda7fab67785 41da5db45176 2023-09-30T14:38:14.310 INFO:teuthology.orchestra.run.smithi032.stdout:mds.g smithi032 running (73s) - 72s 15.7M - 18.0.0-6447-g1d10b717 dda7fab67785 d730eb659aee 2023-09-30T14:38:14.310 INFO:teuthology.orchestra.run.smithi032.stdout:mds.h smithi148 running (78s) - 77s 14.1M - 18.0.0-6447-g1d10b717 dda7fab67785 b7af7cda2c9a 2023-09-30T14:38:14.310 INFO:teuthology.orchestra.run.smithi032.stdout:mds.i smithi156 running (75s) - 75s 15.7M - 18.0.0-6447-g1d10b717 dda7fab67785 86be7bd051c3 2023-09-30T14:38:14.310 INFO:teuthology.orchestra.run.smithi032.stdout:mds.j smithi032 running (88s) - 88s 15.7M - 18.0.0-6447-g1d10b717 dda7fab67785 5eabcc2d9d9d 2023-09-30T14:38:14.310 INFO:teuthology.orchestra.run.smithi032.stdout:mds.k smithi148 running (91s) - 90s 13.5M - 18.0.0-6447-g1d10b717 dda7fab67785 1e5c0d5465e9 2023-09-30T14:38:14.310 INFO:teuthology.orchestra.run.smithi032.stdout:mds.l smithi156 running (93s) - 92s 15.8M - 18.0.0-6447-g1d10b717 dda7fab67785 1c6092d3b3d5 2023-09-30T14:38:14.310 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.x smithi032 *:9283,8765 running (7m) - 7m 515M - 18.0.0-6447-g1d10b717 dda7fab67785 2a8fe76dc9c3 2023-09-30T14:38:14.310 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.y smithi148 *:8443,8765 running (5m) - 5m 449M - 18.0.0-6447-g1d10b717 dda7fab67785 06a5340551b0 2023-09-30T14:38:14.310 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.z smithi156 *:8443,8765 running (6m) - 6m 449M - 18.0.0-6447-g1d10b717 dda7fab67785 189058b0f1d2 2023-09-30T14:38:14.310 INFO:teuthology.orchestra.run.smithi032.stdout:mon.a smithi032 running (7m) - 8m 83.0M 2048M 18.0.0-6447-g1d10b717 dda7fab67785 539a940bd728 2023-09-30T14:38:14.310 INFO:teuthology.orchestra.run.smithi032.stdout:mon.b smithi148 running (6m) - 6m 57.5M 2048M 18.0.0-6447-g1d10b717 dda7fab67785 7815dedee190 2023-09-30T14:38:14.310 INFO:teuthology.orchestra.run.smithi032.stdout:mon.c smithi156 running (6m) - 6m 58.8M 2048M 18.0.0-6447-g1d10b717 dda7fab67785 c4a4135d3eba 2023-09-30T14:38:14.311 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (5m) - 5m 55.9M 4293M 18.0.0-6447-g1d10b717 dda7fab67785 8f627fa4e036 2023-09-30T14:38:14.311 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi148 running (5m) - 5m 61.0M 4292M 18.0.0-6447-g1d10b717 dda7fab67785 56d59a735d64 2023-09-30T14:38:14.311 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi156 running (5m) - 5m 55.3M 4292M 18.0.0-6447-g1d10b717 dda7fab67785 adb6ca1178d5 2023-09-30T14:38:14.311 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi032 running (4m) - 4m 52.6M 4293M 18.0.0-6447-g1d10b717 dda7fab67785 86ac14abaade 2023-09-30T14:38:14.311 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi148 running (4m) - 4m 48.6M 4292M 18.0.0-6447-g1d10b717 dda7fab67785 0cc7ce4c8c01 2023-09-30T14:38:14.311 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi156 running (3m) - 3m 56.4M 4292M 18.0.0-6447-g1d10b717 dda7fab67785 9bd836ed0b59 2023-09-30T14:38:14.311 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6 smithi032 running (3m) - 3m 47.3M 4293M 18.0.0-6447-g1d10b717 dda7fab67785 a368a122d4c9 2023-09-30T14:38:14.311 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7 smithi148 running (3m) - 3m 52.2M 4292M 18.0.0-6447-g1d10b717 dda7fab67785 4d7d260af2a2 2023-09-30T14:38:14.311 INFO:teuthology.orchestra.run.smithi032.stdout:osd.8 smithi156 running (2m) - 2m 53.2M 4292M 18.0.0-6447-g1d10b717 dda7fab67785 1840bbad36f8 2023-09-30T14:38:14.311 INFO:teuthology.orchestra.run.smithi032.stdout:osd.9 smithi032 running (2m) - 2m 46.7M 4293M 18.0.0-6447-g1d10b717 dda7fab67785 4a5ed33b8563 2023-09-30T14:38:14.311 INFO:teuthology.orchestra.run.smithi032.stdout:osd.10 smithi148 running (2m) - 2m 49.0M 4292M 18.0.0-6447-g1d10b717 dda7fab67785 73cfb445117c 2023-09-30T14:38:14.311 INFO:teuthology.orchestra.run.smithi032.stdout:osd.11 smithi156 running (111s) - 111s 45.4M 4292M 18.0.0-6447-g1d10b717 dda7fab67785 bdce43cadd8b 2023-09-30T14:38:14.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.311+0000 7f4450501700 1 -- 172.21.15.32:0/2267975771 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44340a10b0 msgr2=0x7f44340a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:14.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.311+0000 7f4450501700 1 --2- 172.21.15.32:0/2267975771 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44340a10b0 0x7f44340a3570 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f444400b5c0 tx=0x7f4444005960 comp rx=0 tx=0).stop 2023-09-30T14:38:14.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.311+0000 7f4450501700 1 -- 172.21.15.32:0/2267975771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4448100f80 msgr2=0x7f44481023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:14.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.311+0000 7f4450501700 1 --2- 172.21.15.32:0/2267975771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4448100f80 0x7f44481023f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f443800b530 tx=0x7f443800b840 comp rx=0 tx=0).stop 2023-09-30T14:38:14.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.311+0000 7f4450501700 1 -- 172.21.15.32:0/2267975771 shutdown_connections 2023-09-30T14:38:14.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.311+0000 7f4450501700 1 --2- 172.21.15.32:0/2267975771 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44340a10b0 0x7f44340a3570 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:14.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.311+0000 7f4450501700 1 --2- 172.21.15.32:0/2267975771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4448101940 0x7f4448102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:14.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.311+0000 7f4450501700 1 --2- 172.21.15.32:0/2267975771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44480ffe20 0x7f4448103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:14.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.311+0000 7f4450501700 1 --2- 172.21.15.32:0/2267975771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4448100f80 0x7f44481023f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:14.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.311+0000 7f4450501700 1 -- 172.21.15.32:0/2267975771 >> 172.21.15.32:0/2267975771 conn(0x7f44480fb3c0 msgr2=0x7f44480692b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:14.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.311+0000 7f4450501700 1 -- 172.21.15.32:0/2267975771 shutdown_connections 2023-09-30T14:38:14.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:14.311+0000 7f4450501700 1 -- 172.21.15.32:0/2267975771 wait complete. 2023-09-30T14:38:14.508 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:14 smithi032 ceph-mon[98592]: osdmap e85: 12 total, 12 up, 12 in 2023-09-30T14:38:14.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:14 smithi156 ceph-mon[103579]: osdmap e85: 12 total, 12 up, 12 in 2023-09-30T14:38:14.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:14 smithi148 ceph-mon[102355]: osdmap e85: 12 total, 12 up, 12 in 2023-09-30T14:38:14.772 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-09-30T14:38:15.395 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:38:15.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:15 smithi156 ceph-mon[103579]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2023-09-30T14:38:15.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:15 smithi156 ceph-mon[103579]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-30T14:38:15.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:15 smithi156 ceph-mon[103579]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:38:15.588 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:15 smithi032 ceph-mon[98592]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2023-09-30T14:38:15.588 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:15 smithi032 ceph-mon[98592]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-30T14:38:15.588 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:15 smithi032 ceph-mon[98592]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:38:15.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:15 smithi148 ceph-mon[102355]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2023-09-30T14:38:15.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:15 smithi148 ceph-mon[102355]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-30T14:38:15.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:15 smithi148 ceph-mon[102355]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:38:16.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:16 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:16.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:16 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:16.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:16 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:16.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:16 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.615+0000 7f201febf700 1 -- 172.21.15.32:0/2485968288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2018100a80 msgr2=0x7f2018100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.615+0000 7f201febf700 1 --2- 172.21.15.32:0/2485968288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2018100a80 0x7f2018100f00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f201000b0a0 tx=0x7f201000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.615+0000 7f201febf700 1 -- 172.21.15.32:0/2485968288 shutdown_connections 2023-09-30T14:38:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.615+0000 7f201febf700 1 --2- 172.21.15.32:0/2485968288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2018100a80 0x7f2018100f00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.615+0000 7f201febf700 1 --2- 172.21.15.32:0/2485968288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20180ff760 0x7f20180ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.615+0000 7f201febf700 1 --2- 172.21.15.32:0/2485968288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20181000c0 0x7f2018100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.615+0000 7f201febf700 1 -- 172.21.15.32:0/2485968288 >> 172.21.15.32:0/2485968288 conn(0x7f20180fb360 msgr2=0x7f20180fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.615+0000 7f201febf700 1 -- 172.21.15.32:0/2485968288 shutdown_connections 2023-09-30T14:38:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.615+0000 7f201febf700 1 -- 172.21.15.32:0/2485968288 wait complete. 2023-09-30T14:38:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.616+0000 7f201febf700 1 Processor -- start 2023-09-30T14:38:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.616+0000 7f201febf700 1 -- start start 2023-09-30T14:38:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.616+0000 7f201febf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20180ff760 0x7f2018198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.616+0000 7f201febf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20181000c0 0x7f20181993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.616+0000 7f201febf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2018100a80 0x7f20181a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.616+0000 7f201febf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20181a0a30 con 0x7f2018100a80 2023-09-30T14:38:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.616+0000 7f201febf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20181a0b70 con 0x7f20181000c0 2023-09-30T14:38:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.616+0000 7f201febf700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20181a0d00 con 0x7f20180ff760 2023-09-30T14:38:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.616+0000 7f201e45c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2018100a80 0x7f20181a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.616+0000 7f201e45c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2018100a80 0x7f20181a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34730/0 (socket says 172.21.15.32:34730) 2023-09-30T14:38:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.616+0000 7f201e45c700 1 -- 172.21.15.32:0/1606479742 learned_addr learned my addr 172.21.15.32:0/1606479742 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.616+0000 7f201dc5b700 1 --2- 172.21.15.32:0/1606479742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20180ff760 0x7f2018198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.616+0000 7f201d45a700 1 --2- 172.21.15.32:0/1606479742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20181000c0 0x7f20181993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.617+0000 7f201e45c700 1 -- 172.21.15.32:0/1606479742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20180ff760 msgr2=0x7f2018198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.617+0000 7f201e45c700 1 --2- 172.21.15.32:0/1606479742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20180ff760 0x7f2018198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.617+0000 7f201e45c700 1 -- 172.21.15.32:0/1606479742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20181000c0 msgr2=0x7f20181993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.617+0000 7f201e45c700 1 --2- 172.21.15.32:0/1606479742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20181000c0 0x7f20181993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.617+0000 7f201e45c700 1 -- 172.21.15.32:0/1606479742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2010009d00 con 0x7f2018100a80 2023-09-30T14:38:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.617+0000 7f201e45c700 1 --2- 172.21.15.32:0/1606479742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2018100a80 0x7f20181a0460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f201000b070 tx=0x7f201000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.617+0000 7f200effd700 1 -- 172.21.15.32:0/1606479742 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201000e040 con 0x7f2018100a80 2023-09-30T14:38:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.617+0000 7f200effd700 1 -- 172.21.15.32:0/1606479742 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2010003e10 con 0x7f2018100a80 2023-09-30T14:38:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.617+0000 7f201febf700 1 -- 172.21.15.32:0/1606479742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20181a0f80 con 0x7f2018100a80 2023-09-30T14:38:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.617+0000 7f200effd700 1 -- 172.21.15.32:0/1606479742 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201001b6c0 con 0x7f2018100a80 2023-09-30T14:38:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.617+0000 7f201febf700 1 -- 172.21.15.32:0/1606479742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20181a12e0 con 0x7f2018100a80 2023-09-30T14:38:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.618+0000 7f201febf700 1 -- 172.21.15.32:0/1606479742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f201810a540 con 0x7f2018100a80 2023-09-30T14:38:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.620+0000 7f200effd700 1 -- 172.21.15.32:0/1606479742 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f201001b820 con 0x7f2018100a80 2023-09-30T14:38:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.621+0000 7f200effd700 1 --2- 172.21.15.32:0/1606479742 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f20040a1030 0x7f20040a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.621+0000 7f201dc5b700 1 --2- 172.21.15.32:0/1606479742 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f20040a1030 0x7f20040a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.621+0000 7f200effd700 1 -- 172.21.15.32:0/1606479742 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f20100cdd30 con 0x7f2018100a80 2023-09-30T14:38:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.622+0000 7f201dc5b700 1 --2- 172.21.15.32:0/1606479742 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f20040a1030 0x7f20040a34f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2014009c60 tx=0x7f2014009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.624+0000 7f200effd700 1 -- 172.21.15.32:0/1606479742 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f2010097470 con 0x7f2018100a80 2023-09-30T14:38:16.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:16 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:16.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:16 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:16.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:16 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:16.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:16 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:16.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:16 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:16.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:16 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:16.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:16 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:16.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:16 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:16.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.855+0000 7f201febf700 1 -- 172.21.15.32:0/1606479742 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f201802d1b0 con 0x7f20040a1030 2023-09-30T14:38:16.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.862+0000 7f200effd700 1 -- 172.21.15.32:0/1606479742 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f201802d1b0 con 0x7f20040a1030 2023-09-30T14:38:16.862 INFO:teuthology.orchestra.run.smithi032.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-09-30T14:38:16.862 INFO:teuthology.orchestra.run.smithi032.stdout:mds.all 12/12 - 97s smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l;count:12 2023-09-30T14:38:16.863 INFO:teuthology.orchestra.run.smithi032.stdout:mgr 3/3 - 6m smithi032=x;smithi148=y;smithi156=z;count:3 2023-09-30T14:38:16.863 INFO:teuthology.orchestra.run.smithi032.stdout:mon 3/3 - 6m smithi032:172.21.15.32=a;smithi148:172.21.15.148=b;smithi156:172.21.15.156=c;count:3 2023-09-30T14:38:16.863 INFO:teuthology.orchestra.run.smithi032.stdout:osd 12 - - 2023-09-30T14:38:16.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.864+0000 7f201febf700 1 -- 172.21.15.32:0/1606479742 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f20040a1030 msgr2=0x7f20040a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:16.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.864+0000 7f201febf700 1 --2- 172.21.15.32:0/1606479742 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f20040a1030 0x7f20040a34f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2014009c60 tx=0x7f2014009260 comp rx=0 tx=0).stop 2023-09-30T14:38:16.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.864+0000 7f201febf700 1 -- 172.21.15.32:0/1606479742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2018100a80 msgr2=0x7f20181a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:16.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.864+0000 7f201febf700 1 --2- 172.21.15.32:0/1606479742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2018100a80 0x7f20181a0460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f201000b070 tx=0x7f201000bce0 comp rx=0 tx=0).stop 2023-09-30T14:38:16.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.864+0000 7f201febf700 1 -- 172.21.15.32:0/1606479742 shutdown_connections 2023-09-30T14:38:16.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.864+0000 7f201febf700 1 --2- 172.21.15.32:0/1606479742 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f20040a1030 0x7f20040a34f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:16.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.864+0000 7f201febf700 1 --2- 172.21.15.32:0/1606479742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2018100a80 0x7f20181a0460 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:16.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.864+0000 7f201febf700 1 --2- 172.21.15.32:0/1606479742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20180ff760 0x7f2018198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:16.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.864+0000 7f201febf700 1 --2- 172.21.15.32:0/1606479742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20181000c0 0x7f20181993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:16.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.864+0000 7f201febf700 1 -- 172.21.15.32:0/1606479742 >> 172.21.15.32:0/1606479742 conn(0x7f20180fb360 msgr2=0x7f20180fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:16.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.864+0000 7f201febf700 1 -- 172.21.15.32:0/1606479742 shutdown_connections 2023-09-30T14:38:16.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:16.864+0000 7f201febf700 1 -- 172.21.15.32:0/1606479742 wait complete. 2023-09-30T14:38:17.257 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-09-30T14:38:17.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:17 smithi032 ceph-mon[98592]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2023-09-30T14:38:17.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:17.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:17.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:38:17.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:38:17.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:17 smithi156 ceph-mon[103579]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2023-09-30T14:38:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:38:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:38:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:17.822 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:38:17.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:17 smithi148 ceph-mon[102355]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2023-09-30T14:38:17.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:17.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:17.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:38:17.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:38:17.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:38:18.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:18 smithi032 ceph-mon[98592]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:38:18.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:18 smithi156 ceph-mon[103579]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:38:18.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:18 smithi148 ceph-mon[102355]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:38:19.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.100+0000 7ff28adfa700 1 -- 172.21.15.32:0/3406348788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284101980 msgr2=0x7ff284108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:19.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.100+0000 7ff28adfa700 1 --2- 172.21.15.32:0/3406348788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284101980 0x7ff284108230 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00b0d0 tx=0x7ff27c00b3e0 comp rx=0 tx=0).stop 2023-09-30T14:38:19.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.101+0000 7ff28adfa700 1 -- 172.21.15.32:0/3406348788 shutdown_connections 2023-09-30T14:38:19.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.101+0000 7ff28adfa700 1 --2- 172.21.15.32:0/3406348788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284101980 0x7ff284108230 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:19.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.101+0000 7ff28adfa700 1 --2- 172.21.15.32:0/3406348788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2840ffde0 0x7ff284100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:19.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.101+0000 7ff28adfa700 1 --2- 172.21.15.32:0/3406348788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff284100fe0 0x7ff284101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:19.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.101+0000 7ff28adfa700 1 -- 172.21.15.32:0/3406348788 >> 172.21.15.32:0/3406348788 conn(0x7ff2840fb360 msgr2=0x7ff2840fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:19.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.101+0000 7ff28adfa700 1 -- 172.21.15.32:0/3406348788 shutdown_connections 2023-09-30T14:38:19.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.101+0000 7ff28adfa700 1 -- 172.21.15.32:0/3406348788 wait complete. 2023-09-30T14:38:19.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.101+0000 7ff28adfa700 1 Processor -- start 2023-09-30T14:38:19.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.101+0000 7ff28adfa700 1 -- start start 2023-09-30T14:38:19.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.101+0000 7ff28adfa700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2840ffde0 0x7ff284195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:19.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.101+0000 7ff28adfa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff284100fe0 0x7ff2841960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:19.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.101+0000 7ff28adfa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284101980 0x7ff28419d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:19.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.101+0000 7ff28adfa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28419d6f0 con 0x7ff284101980 2023-09-30T14:38:19.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.101+0000 7ff28adfa700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28419d830 con 0x7ff284100fe0 2023-09-30T14:38:19.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.101+0000 7ff28adfa700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28419d9c0 con 0x7ff2840ffde0 2023-09-30T14:38:19.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.102+0000 7ff289397700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284101980 0x7ff28419d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:19.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.102+0000 7ff289397700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284101980 0x7ff28419d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34742/0 (socket says 172.21.15.32:34742) 2023-09-30T14:38:19.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.102+0000 7ff289397700 1 -- 172.21.15.32:0/2681020505 learned_addr learned my addr 172.21.15.32:0/2681020505 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:19.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.102+0000 7ff283fff700 1 --2- 172.21.15.32:0/2681020505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff284100fe0 0x7ff2841960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:19.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.102+0000 7ff288b96700 1 --2- 172.21.15.32:0/2681020505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2840ffde0 0x7ff284195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:19.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.102+0000 7ff289397700 1 -- 172.21.15.32:0/2681020505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2840ffde0 msgr2=0x7ff284195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:19.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.102+0000 7ff289397700 1 --2- 172.21.15.32:0/2681020505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2840ffde0 0x7ff284195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:19.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.102+0000 7ff289397700 1 -- 172.21.15.32:0/2681020505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff284100fe0 msgr2=0x7ff2841960a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:19.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.102+0000 7ff289397700 1 --2- 172.21.15.32:0/2681020505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff284100fe0 0x7ff2841960a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:19.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.102+0000 7ff289397700 1 -- 172.21.15.32:0/2681020505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c009d00 con 0x7ff284101980 2023-09-30T14:38:19.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.102+0000 7ff289397700 1 --2- 172.21.15.32:0/2681020505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284101980 0x7ff28419d120 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff27c0094c0 tx=0x7ff27c0094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:19.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.102+0000 7ff281ffb700 1 -- 172.21.15.32:0/2681020505 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c00e040 con 0x7ff284101980 2023-09-30T14:38:19.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.102+0000 7ff281ffb700 1 -- 172.21.15.32:0/2681020505 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff27c003e10 con 0x7ff284101980 2023-09-30T14:38:19.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.102+0000 7ff28adfa700 1 -- 172.21.15.32:0/2681020505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28419dc40 con 0x7ff284101980 2023-09-30T14:38:19.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.102+0000 7ff281ffb700 1 -- 172.21.15.32:0/2681020505 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c012ad0 con 0x7ff284101980 2023-09-30T14:38:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.103+0000 7ff28adfa700 1 -- 172.21.15.32:0/2681020505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28419e050 con 0x7ff284101980 2023-09-30T14:38:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.104+0000 7ff28adfa700 1 -- 172.21.15.32:0/2681020505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff284107380 con 0x7ff284101980 2023-09-30T14:38:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.105+0000 7ff281ffb700 1 -- 172.21.15.32:0/2681020505 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7ff27c012c30 con 0x7ff284101980 2023-09-30T14:38:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.105+0000 7ff281ffb700 1 --2- 172.21.15.32:0/2681020505 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff2740a1030 0x7ff2740a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.105+0000 7ff288b96700 1 --2- 172.21.15.32:0/2681020505 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff2740a1030 0x7ff2740a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.105+0000 7ff281ffb700 1 -- 172.21.15.32:0/2681020505 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff27c0ce750 con 0x7ff284101980 2023-09-30T14:38:19.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.106+0000 7ff288b96700 1 --2- 172.21.15.32:0/2681020505 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff2740a1030 0x7ff2740a34f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff278005950 tx=0x7ff2780058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:19.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.109+0000 7ff281ffb700 1 -- 172.21.15.32:0/2681020505 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7ff27c097e90 con 0x7ff284101980 2023-09-30T14:38:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.314+0000 7ff28adfa700 1 -- 172.21.15.32:0/2681020505 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2840008d0 con 0x7ff2740a1030 2023-09-30T14:38:19.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.316+0000 7ff281ffb700 1 -- 172.21.15.32:0/2681020505 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7ff2840008d0 con 0x7ff2740a1030 2023-09-30T14:38:19.317 INFO:teuthology.orchestra.run.smithi032.stdout:HOST ADDR LABELS STATUS 2023-09-30T14:38:19.317 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032 172.21.15.32 2023-09-30T14:38:19.317 INFO:teuthology.orchestra.run.smithi032.stdout:smithi148 172.21.15.148 2023-09-30T14:38:19.317 INFO:teuthology.orchestra.run.smithi032.stdout:smithi156 172.21.15.156 2023-09-30T14:38:19.317 INFO:teuthology.orchestra.run.smithi032.stdout:3 hosts in cluster 2023-09-30T14:38:19.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.318+0000 7ff28adfa700 1 -- 172.21.15.32:0/2681020505 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff2740a1030 msgr2=0x7ff2740a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:19.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.318+0000 7ff28adfa700 1 --2- 172.21.15.32:0/2681020505 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff2740a1030 0x7ff2740a34f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff278005950 tx=0x7ff2780058e0 comp rx=0 tx=0).stop 2023-09-30T14:38:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.318+0000 7ff28adfa700 1 -- 172.21.15.32:0/2681020505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284101980 msgr2=0x7ff28419d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.318+0000 7ff28adfa700 1 --2- 172.21.15.32:0/2681020505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284101980 0x7ff28419d120 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff27c0094c0 tx=0x7ff27c0094f0 comp rx=0 tx=0).stop 2023-09-30T14:38:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.318+0000 7ff28adfa700 1 -- 172.21.15.32:0/2681020505 shutdown_connections 2023-09-30T14:38:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.318+0000 7ff28adfa700 1 --2- 172.21.15.32:0/2681020505 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff2740a1030 0x7ff2740a34f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.318+0000 7ff28adfa700 1 --2- 172.21.15.32:0/2681020505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284101980 0x7ff28419d120 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.318+0000 7ff28adfa700 1 --2- 172.21.15.32:0/2681020505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2840ffde0 0x7ff284195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.318+0000 7ff28adfa700 1 --2- 172.21.15.32:0/2681020505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff284100fe0 0x7ff2841960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.318+0000 7ff28adfa700 1 -- 172.21.15.32:0/2681020505 >> 172.21.15.32:0/2681020505 conn(0x7ff2840fb360 msgr2=0x7ff2841044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.319+0000 7ff28adfa700 1 -- 172.21.15.32:0/2681020505 shutdown_connections 2023-09-30T14:38:19.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:19.319+0000 7ff28adfa700 1 -- 172.21.15.32:0/2681020505 wait complete. 2023-09-30T14:38:19.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:19 smithi032 ceph-mon[98592]: pgmap v288: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s; 164 B/s, 1 objects/s recovering 2023-09-30T14:38:19.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:19 smithi032 ceph-mon[98592]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-09-30T14:38:19.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:19 smithi032 ceph-mon[98592]: Cluster is now healthy 2023-09-30T14:38:19.672 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-09-30T14:38:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:19 smithi156 ceph-mon[103579]: pgmap v288: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s; 164 B/s, 1 objects/s recovering 2023-09-30T14:38:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:19 smithi156 ceph-mon[103579]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-09-30T14:38:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:19 smithi156 ceph-mon[103579]: Cluster is now healthy 2023-09-30T14:38:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:19 smithi148 ceph-mon[102355]: pgmap v288: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s; 164 B/s, 1 objects/s recovering 2023-09-30T14:38:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:19 smithi148 ceph-mon[102355]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-09-30T14:38:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:19 smithi148 ceph-mon[102355]: Cluster is now healthy 2023-09-30T14:38:20.235 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:38:20.747 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:20 smithi032 ceph-mon[98592]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:38:20.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:20 smithi156 ceph-mon[103579]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:38:20.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:20 smithi148 ceph-mon[102355]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:38:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.434+0000 7f1fd2c96700 1 -- 172.21.15.32:0/2144498457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fcc1003f0 msgr2=0x7f1fcc108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.434+0000 7f1fd2c96700 1 --2- 172.21.15.32:0/2144498457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fcc1003f0 0x7f1fcc108250 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400b0a0 tx=0x7f1fc400b3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.434+0000 7f1fd2c96700 1 -- 172.21.15.32:0/2144498457 shutdown_connections 2023-09-30T14:38:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.434+0000 7f1fd2c96700 1 --2- 172.21.15.32:0/2144498457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fcc1003f0 0x7f1fcc108250 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.434+0000 7f1fd2c96700 1 --2- 172.21.15.32:0/2144498457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fcc101450 0x7f1fcc101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.434+0000 7f1fd2c96700 1 --2- 172.21.15.32:0/2144498457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fcc0ffa30 0x7f1fcc0ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.434+0000 7f1fd2c96700 1 -- 172.21.15.32:0/2144498457 >> 172.21.15.32:0/2144498457 conn(0x7f1fcc0fb390 msgr2=0x7f1fcc0fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.434+0000 7f1fd2c96700 1 -- 172.21.15.32:0/2144498457 shutdown_connections 2023-09-30T14:38:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.434+0000 7f1fd2c96700 1 -- 172.21.15.32:0/2144498457 wait complete. 2023-09-30T14:38:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fd2c96700 1 Processor -- start 2023-09-30T14:38:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fd2c96700 1 -- start start 2023-09-30T14:38:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fd2c96700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fcc0ffa30 0x7f1fcc195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fd2c96700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fcc1003f0 0x7f1fcc196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fd2c96700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fcc101450 0x7f1fcc19d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fd2c96700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc19d970 con 0x7f1fcc1003f0 2023-09-30T14:38:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fd2c96700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc19dab0 con 0x7f1fcc0ffa30 2023-09-30T14:38:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fd2c96700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc19dc40 con 0x7f1fcc101450 2023-09-30T14:38:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fcbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fcc1003f0 0x7f1fcc196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fcbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fcc1003f0 0x7f1fcc196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34754/0 (socket says 172.21.15.32:34754) 2023-09-30T14:38:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fd0a32700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fcc0ffa30 0x7f1fcc195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fcbfff700 1 -- 172.21.15.32:0/4278181261 learned_addr learned my addr 172.21.15.32:0/4278181261 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fd1233700 1 --2- 172.21.15.32:0/4278181261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fcc101450 0x7f1fcc19d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fcbfff700 1 -- 172.21.15.32:0/4278181261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fcc101450 msgr2=0x7f1fcc19d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fcbfff700 1 --2- 172.21.15.32:0/4278181261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fcc101450 0x7f1fcc19d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fcbfff700 1 -- 172.21.15.32:0/4278181261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fcc0ffa30 msgr2=0x7f1fcc195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fcbfff700 1 --2- 172.21.15.32:0/4278181261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fcc0ffa30 0x7f1fcc195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fcbfff700 1 -- 172.21.15.32:0/4278181261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc4009d00 con 0x7f1fcc1003f0 2023-09-30T14:38:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.435+0000 7f1fcbfff700 1 --2- 172.21.15.32:0/4278181261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fcc1003f0 0x7f1fcc196320 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400da10 tx=0x7f1fb400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.436+0000 7f1fc9ffb700 1 -- 172.21.15.32:0/4278181261 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb400eb70 con 0x7f1fcc1003f0 2023-09-30T14:38:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.436+0000 7f1fc9ffb700 1 -- 172.21.15.32:0/4278181261 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fb400ecd0 con 0x7f1fcc1003f0 2023-09-30T14:38:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.436+0000 7f1fd2c96700 1 -- 172.21.15.32:0/4278181261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc069330 con 0x7f1fcc1003f0 2023-09-30T14:38:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.436+0000 7f1fc9ffb700 1 -- 172.21.15.32:0/4278181261 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40145a0 con 0x7f1fcc1003f0 2023-09-30T14:38:21.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.436+0000 7f1fd2c96700 1 -- 172.21.15.32:0/4278181261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc069830 con 0x7f1fcc1003f0 2023-09-30T14:38:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.437+0000 7f1fd2c96700 1 -- 172.21.15.32:0/4278181261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc04eb20 con 0x7f1fcc1003f0 2023-09-30T14:38:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.438+0000 7f1fc9ffb700 1 -- 172.21.15.32:0/4278181261 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f1fb4016440 con 0x7f1fcc1003f0 2023-09-30T14:38:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.438+0000 7f1fc9ffb700 1 --2- 172.21.15.32:0/4278181261 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1fb80a1030 0x7f1fb80a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.438+0000 7f1fc9ffb700 1 -- 172.21.15.32:0/4278181261 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1fb40cdf00 con 0x7f1fcc1003f0 2023-09-30T14:38:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.438+0000 7f1fd0a32700 1 --2- 172.21.15.32:0/4278181261 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1fb80a1030 0x7f1fb80a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.439+0000 7f1fd0a32700 1 --2- 172.21.15.32:0/4278181261 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1fb80a1030 0x7f1fb80a34f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc005950 tx=0x7f1fbc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.443+0000 7f1fc9ffb700 1 -- 172.21.15.32:0/4278181261 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f1fb4050f30 con 0x7f1fcc1003f0 2023-09-30T14:38:21.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:21 smithi032 ceph-mon[98592]: pgmap v289: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 164 B/s, 1 objects/s recovering 2023-09-30T14:38:21.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.651+0000 7f1fd2c96700 1 -- 172.21.15.32:0/4278181261 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fcc106ea0 con 0x7f1fb80a1030 2023-09-30T14:38:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.654+0000 7f1fc9ffb700 1 -- 172.21.15.32:0/4278181261 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f1fcc106ea0 con 0x7f1fb80a1030 2023-09-30T14:38:21.655 INFO:teuthology.orchestra.run.smithi032.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-09-30T14:38:21.655 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747402DN480DGN 447G No 2m ago Has a FileSystem, LVM detected 2023-09-30T14:38:21.655 INFO:teuthology.orchestra.run.smithi032.stdout:smithi148 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005G400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-09-30T14:38:21.655 INFO:teuthology.orchestra.run.smithi032.stdout:smithi156 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760040480DGN 447G No 112s ago Has a FileSystem, LVM detected 2023-09-30T14:38:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.656+0000 7f1fd2c96700 1 -- 172.21.15.32:0/4278181261 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1fb80a1030 msgr2=0x7f1fb80a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.656+0000 7f1fd2c96700 1 --2- 172.21.15.32:0/4278181261 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1fb80a1030 0x7f1fb80a34f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc005950 tx=0x7f1fbc00a300 comp rx=0 tx=0).stop 2023-09-30T14:38:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.656+0000 7f1fd2c96700 1 -- 172.21.15.32:0/4278181261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fcc1003f0 msgr2=0x7f1fcc196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.656+0000 7f1fd2c96700 1 --2- 172.21.15.32:0/4278181261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fcc1003f0 0x7f1fcc196320 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400da10 tx=0x7f1fb400dd20 comp rx=0 tx=0).stop 2023-09-30T14:38:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.656+0000 7f1fd2c96700 1 -- 172.21.15.32:0/4278181261 shutdown_connections 2023-09-30T14:38:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.656+0000 7f1fd2c96700 1 --2- 172.21.15.32:0/4278181261 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1fb80a1030 0x7f1fb80a34f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.656+0000 7f1fd2c96700 1 --2- 172.21.15.32:0/4278181261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1fcc101450 0x7f1fcc19d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.656+0000 7f1fd2c96700 1 --2- 172.21.15.32:0/4278181261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fcc0ffa30 0x7f1fcc195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.656+0000 7f1fd2c96700 1 --2- 172.21.15.32:0/4278181261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fcc1003f0 0x7f1fcc196320 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.656+0000 7f1fd2c96700 1 -- 172.21.15.32:0/4278181261 >> 172.21.15.32:0/4278181261 conn(0x7f1fcc0fb390 msgr2=0x7f1fcc1044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.656+0000 7f1fd2c96700 1 -- 172.21.15.32:0/4278181261 shutdown_connections 2023-09-30T14:38:21.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:21.656+0000 7f1fd2c96700 1 -- 172.21.15.32:0/4278181261 wait complete. 2023-09-30T14:38:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:21 smithi156 ceph-mon[103579]: pgmap v289: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 164 B/s, 1 objects/s recovering 2023-09-30T14:38:21.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:21 smithi148 ceph-mon[102355]: pgmap v289: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 164 B/s, 1 objects/s recovering 2023-09-30T14:38:22.085 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-30T14:38:22.095 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T14:38:22.095 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-09-30T14:38:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:38:23.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:23 smithi032 ceph-mon[98592]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:38:23.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:23 smithi032 ceph-mon[98592]: pgmap v290: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 137 B/s, 1 objects/s recovering 2023-09-30T14:38:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:23 smithi156 ceph-mon[103579]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:38:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:23 smithi156 ceph-mon[103579]: pgmap v290: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 137 B/s, 1 objects/s recovering 2023-09-30T14:38:23.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.836+0000 7f724da40700 1 -- 172.21.15.32:0/237746203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72480ffde0 msgr2=0x7f7248100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:23.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.836+0000 7f724da40700 1 --2- 172.21.15.32:0/237746203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72480ffde0 0x7f7248100200 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f723c0099c0 tx=0x7f723c009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:23.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.836+0000 7f724da40700 1 -- 172.21.15.32:0/237746203 shutdown_connections 2023-09-30T14:38:23.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.836+0000 7f724da40700 1 --2- 172.21.15.32:0/237746203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7248101980 0x7f7248108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:23.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.836+0000 7f724da40700 1 --2- 172.21.15.32:0/237746203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72480ffde0 0x7f7248100200 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:23.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.836+0000 7f724da40700 1 --2- 172.21.15.32:0/237746203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7248100fe0 0x7f7248101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:23.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.836+0000 7f724da40700 1 -- 172.21.15.32:0/237746203 >> 172.21.15.32:0/237746203 conn(0x7f72480fb360 msgr2=0x7f72480fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:23.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.836+0000 7f724da40700 1 -- 172.21.15.32:0/237746203 shutdown_connections 2023-09-30T14:38:23.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.836+0000 7f724da40700 1 -- 172.21.15.32:0/237746203 wait complete. 2023-09-30T14:38:23.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.837+0000 7f724da40700 1 Processor -- start 2023-09-30T14:38:23.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.837+0000 7f724da40700 1 -- start start 2023-09-30T14:38:23.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.837+0000 7f724da40700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72480ffde0 0x7f7248196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:23.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.837+0000 7f724da40700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7248100fe0 0x7f72481970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.837+0000 7f724da40700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7248101980 0x7f724819e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.837+0000 7f724da40700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724810ada0 con 0x7f7248101980 2023-09-30T14:38:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.837+0000 7f724da40700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724810af10 con 0x7f72480ffde0 2023-09-30T14:38:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.837+0000 7f724da40700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724810b0d0 con 0x7f7248100fe0 2023-09-30T14:38:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f72477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7248101980 0x7f724819e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f72477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7248101980 0x7f724819e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34768/0 (socket says 172.21.15.32:34768) 2023-09-30T14:38:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f72477fe700 1 -- 172.21.15.32:0/1589108413 learned_addr learned my addr 172.21.15.32:0/1589108413 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f7246ffd700 1 --2- 172.21.15.32:0/1589108413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72480ffde0 0x7f7248196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f72467fc700 1 --2- 172.21.15.32:0/1589108413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7248100fe0 0x7f72481970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f72477fe700 1 -- 172.21.15.32:0/1589108413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7248100fe0 msgr2=0x7f72481970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f72477fe700 1 --2- 172.21.15.32:0/1589108413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7248100fe0 0x7f72481970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f72477fe700 1 -- 172.21.15.32:0/1589108413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72480ffde0 msgr2=0x7f7248196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f72477fe700 1 --2- 172.21.15.32:0/1589108413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72480ffde0 0x7f7248196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f72477fe700 1 -- 172.21.15.32:0/1589108413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c0095f0 con 0x7f7248101980 2023-09-30T14:38:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f7246ffd700 1 --2- 172.21.15.32:0/1589108413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72480ffde0 0x7f7248196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f72477fe700 1 --2- 172.21.15.32:0/1589108413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7248101980 0x7f724819e170 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f724000ef30 tx=0x7f724000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f724ca3e700 1 -- 172.21.15.32:0/1589108413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240018070 con 0x7f7248101980 2023-09-30T14:38:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f724ca3e700 1 -- 172.21.15.32:0/1589108413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f724000f040 con 0x7f7248101980 2023-09-30T14:38:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f724da40700 1 -- 172.21.15.32:0/1589108413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f724810b350 con 0x7f7248101980 2023-09-30T14:38:23.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f724da40700 1 -- 172.21.15.32:0/1589108413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f724810b800 con 0x7f7248101980 2023-09-30T14:38:23.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.838+0000 7f724ca3e700 1 -- 172.21.15.32:0/1589108413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240003dc0 con 0x7f7248101980 2023-09-30T14:38:23.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.839+0000 7f724da40700 1 -- 172.21.15.32:0/1589108413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7234005320 con 0x7f7248101980 2023-09-30T14:38:23.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.840+0000 7f724ca3e700 1 -- 172.21.15.32:0/1589108413 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f7240004ad0 con 0x7f7248101980 2023-09-30T14:38:23.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.841+0000 7f724ca3e700 1 --2- 172.21.15.32:0/1589108413 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f72300a1030 0x7f72300a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:23.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.841+0000 7f7246ffd700 1 --2- 172.21.15.32:0/1589108413 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f72300a1030 0x7f72300a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:23.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.841+0000 7f724ca3e700 1 -- 172.21.15.32:0/1589108413 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f72400cc420 con 0x7f7248101980 2023-09-30T14:38:23.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.842+0000 7f7246ffd700 1 --2- 172.21.15.32:0/1589108413 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f72300a1030 0x7f72300a34f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f723c009990 tx=0x7f723c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:23.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:23.846+0000 7f724ca3e700 1 -- 172.21.15.32:0/1589108413 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f7240095b60 con 0x7f7248101980 2023-09-30T14:38:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:23 smithi148 ceph-mon[102355]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-30T14:38:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:23 smithi148 ceph-mon[102355]: pgmap v290: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 137 B/s, 1 objects/s recovering 2023-09-30T14:38:24.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:24.091+0000 7f724da40700 1 -- 172.21.15.32:0/1589108413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f72340058a0 con 0x7f7248101980 2023-09-30T14:38:24.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:24.092+0000 7f724ca3e700 1 -- 172.21.15.32:0/1589108413 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+2943 (secure 0 0 0) 0x7f7240099180 con 0x7f7248101980 2023-09-30T14:38:24.093 INFO:teuthology.orchestra.run.smithi032.stdout:e22 2023-09-30T14:38:24.093 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-09-30T14:38:24.093 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-30T14:38:24.093 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2023-09-30T14:38:24.093 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:24.093 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2023-09-30T14:38:24.093 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2023-09-30T14:38:24.093 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 22 2023-09-30T14:38:24.093 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-09-30T14:38:24.093 INFO:teuthology.orchestra.run.smithi032.stdout:created 2023-09-30T14:37:11.444993+0000 2023-09-30T14:38:24.093 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2023-09-30T14:37:16.971534+0000 2023-09-30T14:38:24.094 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2023-09-30T14:38:24.094 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2023-09-30T14:38:24.094 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 300 2023-09-30T14:38:24.094 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2023-09-30T14:38:24.094 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2023-09-30T14:38:24.094 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2023-09-30T14:38:24.094 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2023-09-30T14:38:24.094 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2023-09-30T14:38:24.094 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 0 2023-09-30T14:38:24.094 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-30T14:38:24.094 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 3 2023-09-30T14:38:24.094 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1,2 2023-09-30T14:38:24.095 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=24515,1=24493,2=24485} 2023-09-30T14:38:24.095 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2023-09-30T14:38:24.095 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2023-09-30T14:38:24.095 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2023-09-30T14:38:24.095 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2023-09-30T14:38:24.095 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2023-09-30T14:38:24.095 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data disabled 2023-09-30T14:38:24.095 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2023-09-30T14:38:24.095 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2023-09-30T14:38:24.095 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 0 2023-09-30T14:38:24.096 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b{0:24515} state up:active seq 3 addr [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] compat {c=[1],r=[1],i=[fff]}] 2023-09-30T14:38:24.096 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c{0:24478} state up:standby-replay seq 1 addr [v2:172.21.15.156:6836/445372881,v1:172.21.15.156:6837/445372881] compat {c=[1],r=[1],i=[fff]}] 2023-09-30T14:38:24.096 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.i{1:24493} state up:active seq 6 addr [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] compat {c=[1],r=[1],i=[fff]}] 2023-09-30T14:38:24.096 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.f{1:24466} state up:standby-replay seq 1 addr [v2:172.21.15.156:6834/4180205116,v1:172.21.15.156:6835/4180205116] compat {c=[1],r=[1],i=[fff]}] 2023-09-30T14:38:24.096 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.h{2:24485} state up:active seq 6 addr [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] compat {c=[1],r=[1],i=[fff]}] 2023-09-30T14:38:24.096 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.e{2:24455} state up:standby-replay seq 1 addr [v2:172.21.15.148:6834/4151174061,v1:172.21.15.148:6835/4151174061] compat {c=[1],r=[1],i=[fff]}] 2023-09-30T14:38:24.096 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:24.096 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:24.096 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2023-09-30T14:38:24.096 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:24.096 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.j{-1:14553} state up:standby seq 1 addr [v2:172.21.15.32:6834/3687429810,v1:172.21.15.32:6835/3687429810] compat {c=[1],r=[1],i=[fff]}] 2023-09-30T14:38:24.097 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d{-1:14559} state up:standby seq 1 addr [v2:172.21.15.32:6836/2728444148,v1:172.21.15.32:6837/2728444148] compat {c=[1],r=[1],i=[fff]}] 2023-09-30T14:38:24.097 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.g{-1:14616} state up:standby seq 1 addr [v2:172.21.15.32:6838/3505064841,v1:172.21.15.32:6839/3505064841] compat {c=[1],r=[1],i=[fff]}] 2023-09-30T14:38:24.097 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a{-1:14628} state up:standby seq 1 addr [v2:172.21.15.32:6840/2262846038,v1:172.21.15.32:6841/2262846038] compat {c=[1],r=[1],i=[fff]}] 2023-09-30T14:38:24.097 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.l{-1:24427} state up:standby seq 1 addr [v2:172.21.15.156:6832/2184509909,v1:172.21.15.156:6833/2184509909] compat {c=[1],r=[1],i=[fff]}] 2023-09-30T14:38:24.097 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.k{-1:24428} state up:standby seq 1 addr [v2:172.21.15.148:6832/636626698,v1:172.21.15.148:6833/636626698] compat {c=[1],r=[1],i=[fff]}] 2023-09-30T14:38:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:24.093+0000 7f724da40700 1 -- 172.21.15.32:0/1589108413 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f72300a1030 msgr2=0x7f72300a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:24.093+0000 7f724da40700 1 --2- 172.21.15.32:0/1589108413 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f72300a1030 0x7f72300a34f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f723c009990 tx=0x7f723c01a040 comp rx=0 tx=0).stop 2023-09-30T14:38:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:24.093+0000 7f724da40700 1 -- 172.21.15.32:0/1589108413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7248101980 msgr2=0x7f724819e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:24.093+0000 7f724da40700 1 --2- 172.21.15.32:0/1589108413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7248101980 0x7f724819e170 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f724000ef30 tx=0x7f724000c450 comp rx=0 tx=0).stop 2023-09-30T14:38:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:24.094+0000 7f724da40700 1 -- 172.21.15.32:0/1589108413 shutdown_connections 2023-09-30T14:38:24.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:24.094+0000 7f724da40700 1 --2- 172.21.15.32:0/1589108413 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f72300a1030 0x7f72300a34f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:24.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:24.094+0000 7f724da40700 1 --2- 172.21.15.32:0/1589108413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7248101980 0x7f724819e170 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:24.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:24.094+0000 7f724da40700 1 --2- 172.21.15.32:0/1589108413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72480ffde0 0x7f7248196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:24.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:24.094+0000 7f724da40700 1 --2- 172.21.15.32:0/1589108413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7248100fe0 0x7f72481970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:24.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:24.094+0000 7f724da40700 1 -- 172.21.15.32:0/1589108413 >> 172.21.15.32:0/1589108413 conn(0x7f72480fb360 msgr2=0x7f7248104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:24.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:24.094+0000 7f724da40700 1 -- 172.21.15.32:0/1589108413 shutdown_connections 2023-09-30T14:38:24.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:24.094+0000 7f724da40700 1 -- 172.21.15.32:0/1589108413 wait complete. 2023-09-30T14:38:24.099 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 22 2023-09-30T14:38:24.521 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-09-30T14:38:24.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:24 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1589108413' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-30T14:38:24.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:24 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1589108413' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-30T14:38:24.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:24 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1589108413' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-30T14:38:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:38:25.605 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:25 smithi032 ceph-mon[98592]: pgmap v291: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 124 B/s, 1 objects/s recovering 2023-09-30T14:38:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:25 smithi156 ceph-mon[103579]: pgmap v291: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 124 B/s, 1 objects/s recovering 2023-09-30T14:38:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:25 smithi148 ceph-mon[102355]: pgmap v291: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 124 B/s, 1 objects/s recovering 2023-09-30T14:38:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.280+0000 7f262919c700 1 -- 172.21.15.32:0/277895652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624101940 msgr2=0x7f2624108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.280+0000 7f262919c700 1 --2- 172.21.15.32:0/277895652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624101940 0x7f2624108260 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f261c00b0a0 tx=0x7f261c00b3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.281+0000 7f262919c700 1 -- 172.21.15.32:0/277895652 shutdown_connections 2023-09-30T14:38:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.281+0000 7f262919c700 1 --2- 172.21.15.32:0/277895652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624101940 0x7f2624108260 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.281+0000 7f262919c700 1 --2- 172.21.15.32:0/277895652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26240ffe20 0x7f2624100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.281+0000 7f262919c700 1 --2- 172.21.15.32:0/277895652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2624100f80 0x7f2624101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.281+0000 7f262919c700 1 -- 172.21.15.32:0/277895652 >> 172.21.15.32:0/277895652 conn(0x7f26240fb3c0 msgr2=0x7f26240fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.281+0000 7f262919c700 1 -- 172.21.15.32:0/277895652 shutdown_connections 2023-09-30T14:38:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.281+0000 7f262919c700 1 -- 172.21.15.32:0/277895652 wait complete. 2023-09-30T14:38:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.281+0000 7f262919c700 1 Processor -- start 2023-09-30T14:38:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262919c700 1 -- start start 2023-09-30T14:38:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262919c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26240ffe20 0x7f2624195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262919c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624100f80 0x7f26241960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262919c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2624101940 0x7f262419d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262919c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262419d730 con 0x7f2624100f80 2023-09-30T14:38:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262919c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262419d870 con 0x7f26240ffe20 2023-09-30T14:38:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262919c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262419da00 con 0x7f2624101940 2023-09-30T14:38:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624100f80 0x7f26241960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624100f80 0x7f26241960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34780/0 (socket says 172.21.15.32:34780) 2023-09-30T14:38:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262259c700 1 -- 172.21.15.32:0/4006368980 learned_addr learned my addr 172.21.15.32:0/4006368980 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f2622d9d700 1 --2- 172.21.15.32:0/4006368980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26240ffe20 0x7f2624195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262359e700 1 --2- 172.21.15.32:0/4006368980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2624101940 0x7f262419d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262259c700 1 -- 172.21.15.32:0/4006368980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2624101940 msgr2=0x7f262419d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262259c700 1 --2- 172.21.15.32:0/4006368980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2624101940 0x7f262419d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262259c700 1 -- 172.21.15.32:0/4006368980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26240ffe20 msgr2=0x7f2624195ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262259c700 1 --2- 172.21.15.32:0/4006368980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26240ffe20 0x7f2624195ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262259c700 1 -- 172.21.15.32:0/4006368980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c009d00 con 0x7f2624100f80 2023-09-30T14:38:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262359e700 1 --2- 172.21.15.32:0/4006368980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2624101940 0x7f262419d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.282+0000 7f262259c700 1 --2- 172.21.15.32:0/4006368980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624100f80 0x7f26241960e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f261800e9d0 tx=0x7f261800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.283+0000 7f260bfff700 1 -- 172.21.15.32:0/4006368980 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618009790 con 0x7f2624100f80 2023-09-30T14:38:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.283+0000 7f260bfff700 1 -- 172.21.15.32:0/4006368980 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f261800cbd0 con 0x7f2624100f80 2023-09-30T14:38:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.283+0000 7f262919c700 1 -- 172.21.15.32:0/4006368980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262419dce0 con 0x7f2624100f80 2023-09-30T14:38:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.283+0000 7f260bfff700 1 -- 172.21.15.32:0/4006368980 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618018680 con 0x7f2624100f80 2023-09-30T14:38:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.283+0000 7f262919c700 1 -- 172.21.15.32:0/4006368980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262419e1d0 con 0x7f2624100f80 2023-09-30T14:38:26.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.284+0000 7f262919c700 1 -- 172.21.15.32:0/4006368980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2624107390 con 0x7f2624100f80 2023-09-30T14:38:26.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.285+0000 7f260bfff700 1 -- 172.21.15.32:0/4006368980 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f261800cd40 con 0x7f2624100f80 2023-09-30T14:38:26.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.286+0000 7f260bfff700 1 --2- 172.21.15.32:0/4006368980 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f260c0a10b0 0x7f260c0a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:26.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.286+0000 7f2622d9d700 1 --2- 172.21.15.32:0/4006368980 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f260c0a10b0 0x7f260c0a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:26.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.286+0000 7f260bfff700 1 -- 172.21.15.32:0/4006368980 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2618014070 con 0x7f2624100f80 2023-09-30T14:38:26.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.286+0000 7f2622d9d700 1 --2- 172.21.15.32:0/4006368980 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f260c0a10b0 0x7f260c0a3570 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2614007c60 tx=0x7f26140073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:26.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.289+0000 7f260bfff700 1 -- 172.21.15.32:0/4006368980 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f26180963e0 con 0x7f2624100f80 2023-09-30T14:38:26.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.492+0000 7f262919c700 1 -- 172.21.15.32:0/4006368980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f262410b880 con 0x7f2624100f80 2023-09-30T14:38:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.493+0000 7f260bfff700 1 -- 172.21.15.32:0/4006368980 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) v1 ==== 56+0+5291 (secure 0 0 0) 0x7f2618026070 con 0x7f2624100f80 2023-09-30T14:38:26.494 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 85 2023-09-30T14:38:26.494 INFO:teuthology.orchestra.run.smithi032.stdout:fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T14:38:26.494 INFO:teuthology.orchestra.run.smithi032.stdout:created 2023-09-30T14:30:12.351176+0000 2023-09-30T14:38:26.494 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2023-09-30T14:38:13.192089+0000 2023-09-30T14:38:26.494 INFO:teuthology.orchestra.run.smithi032.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-09-30T14:38:26.495 INFO:teuthology.orchestra.run.smithi032.stdout:crush_version 26 2023-09-30T14:38:26.495 INFO:teuthology.orchestra.run.smithi032.stdout:full_ratio 0.95 2023-09-30T14:38:26.495 INFO:teuthology.orchestra.run.smithi032.stdout:backfillfull_ratio 0.9 2023-09-30T14:38:26.495 INFO:teuthology.orchestra.run.smithi032.stdout:nearfull_ratio 0.85 2023-09-30T14:38:26.495 INFO:teuthology.orchestra.run.smithi032.stdout:require_min_compat_client luminous 2023-09-30T14:38:26.495 INFO:teuthology.orchestra.run.smithi032.stdout:min_compat_client luminous 2023-09-30T14:38:26.495 INFO:teuthology.orchestra.run.smithi032.stdout:require_osd_release reef 2023-09-30T14:38:26.495 INFO:teuthology.orchestra.run.smithi032.stdout:stretch_mode_enabled false 2023-09-30T14:38:26.495 INFO:teuthology.orchestra.run.smithi032.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-09-30T14:38:26.495 INFO:teuthology.orchestra.run.smithi032.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-09-30T14:38:26.495 INFO:teuthology.orchestra.run.smithi032.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-09-30T14:38:26.496 INFO:teuthology.orchestra.run.smithi032.stdout:max_osd 12 2023-09-30T14:38:26.496 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6802/3314475588,v1:172.21.15.32:6803/3314475588] [v2:172.21.15.32:6804/3314475588,v1:172.21.15.32:6805/3314475588] exists,up cf7887c5-207a-419c-8828-18179f511646 2023-09-30T14:38:26.496 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6800/2368718565,v1:172.21.15.148:6801/2368718565] [v2:172.21.15.148:6802/2368718565,v1:172.21.15.148:6803/2368718565] exists,up 35766d2e-c839-4c77-839a-535465f18cfe 2023-09-30T14:38:26.496 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6800/2843236580,v1:172.21.15.156:6801/2843236580] [v2:172.21.15.156:6802/2843236580,v1:172.21.15.156:6803/2843236580] exists,up 94cc95d0-0fef-4e49-aefc-2de4626add70 2023-09-30T14:38:26.496 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 up in weight 1 up_from 26 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6810/626048583,v1:172.21.15.32:6811/626048583] [v2:172.21.15.32:6812/626048583,v1:172.21.15.32:6813/626048583] exists,up 9e9c85a2-5720-4256-9285-f6dd719f0ce5 2023-09-30T14:38:26.496 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 up in weight 1 up_from 33 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6808/1987304559,v1:172.21.15.148:6809/1987304559] [v2:172.21.15.148:6810/1987304559,v1:172.21.15.148:6811/1987304559] exists,up d538f510-0f4b-477a-a7e3-99f203c349cf 2023-09-30T14:38:26.496 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 up in weight 1 up_from 38 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6808/2465554180,v1:172.21.15.156:6809/2465554180] [v2:172.21.15.156:6810/2465554180,v1:172.21.15.156:6811/2465554180] exists,up b7e7db08-601c-416b-be4a-e5b72a9c00a2 2023-09-30T14:38:26.496 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6 up in weight 1 up_from 46 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6818/4025142433,v1:172.21.15.32:6819/4025142433] [v2:172.21.15.32:6820/4025142433,v1:172.21.15.32:6821/4025142433] exists,up 14fb18d1-a89b-45da-ac5a-deab8fad9681 2023-09-30T14:38:26.496 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7 up in weight 1 up_from 51 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6816/3841614147,v1:172.21.15.148:6817/3841614147] [v2:172.21.15.148:6818/3841614147,v1:172.21.15.148:6819/3841614147] exists,up 15059921-bebf-46fc-87ba-415e7241af36 2023-09-30T14:38:26.496 INFO:teuthology.orchestra.run.smithi032.stdout:osd.8 up in weight 1 up_from 58 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6816/2338383463,v1:172.21.15.156:6817/2338383463] [v2:172.21.15.156:6818/2338383463,v1:172.21.15.156:6819/2338383463] exists,up 6a853c86-fce1-450d-b902-80989d4a5f21 2023-09-30T14:38:26.496 INFO:teuthology.orchestra.run.smithi032.stdout:osd.9 up in weight 1 up_from 66 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6826/496139479,v1:172.21.15.32:6827/496139479] [v2:172.21.15.32:6828/496139479,v1:172.21.15.32:6829/496139479] exists,up 47eafd4b-4c81-4a0c-9496-272c10e2fb23 2023-09-30T14:38:26.496 INFO:teuthology.orchestra.run.smithi032.stdout:osd.10 up in weight 1 up_from 71 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6824/2093868673,v1:172.21.15.148:6825/2093868673] [v2:172.21.15.148:6826/2093868673,v1:172.21.15.148:6827/2093868673] exists,up 2c7d56d1-96b3-4d0b-8d6e-dc4587579089 2023-09-30T14:38:26.497 INFO:teuthology.orchestra.run.smithi032.stdout:osd.11 up in weight 1 up_from 78 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6824/2619594507,v1:172.21.15.156:6825/2619594507] [v2:172.21.15.156:6826/2619594507,v1:172.21.15.156:6827/2619594507] exists,up e2d6db2a-bd3d-4ed8-95fa-fe8918457114 2023-09-30T14:38:26.497 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 2.10 [8,11] 2023-09-30T14:38:26.497 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 2.16 [2,4] 2023-09-30T14:38:26.497 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 2.1b [2,1] 2023-09-30T14:38:26.497 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 2.1f [7,11] 2023-09-30T14:38:26.497 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.1d [4,8] 2023-09-30T14:38:26.497 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.20 [5,9] 2023-09-30T14:38:26.497 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.24 [4,9] 2023-09-30T14:38:26.497 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.36 [4,9] 2023-09-30T14:38:26.497 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6801/1639474065 expires 2023-10-01T14:31:11.693545+0000 2023-09-30T14:38:26.497 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/1334723693 expires 2023-10-01T14:31:11.693545+0000 2023-09-30T14:38:26.497 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/1412452188 expires 2023-10-01T14:31:11.693545+0000 2023-09-30T14:38:26.498 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6801/2564784233 expires 2023-10-01T14:30:46.025499+0000 2023-09-30T14:38:26.498 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6800/2564784233 expires 2023-10-01T14:30:46.025499+0000 2023-09-30T14:38:26.498 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/3422990986 expires 2023-10-01T14:30:46.025499+0000 2023-09-30T14:38:26.498 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/2379235225 expires 2023-10-01T14:30:46.025499+0000 2023-09-30T14:38:26.498 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/2971157038 expires 2023-10-01T14:31:11.693545+0000 2023-09-30T14:38:26.498 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6800/1639474065 expires 2023-10-01T14:31:11.693545+0000 2023-09-30T14:38:26.498 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/4045220015 expires 2023-10-01T14:30:46.025499+0000 2023-09-30T14:38:26.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.495+0000 7f262919c700 1 -- 172.21.15.32:0/4006368980 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f260c0a10b0 msgr2=0x7f260c0a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:26.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.495+0000 7f262919c700 1 --2- 172.21.15.32:0/4006368980 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f260c0a10b0 0x7f260c0a3570 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2614007c60 tx=0x7f26140073d0 comp rx=0 tx=0).stop 2023-09-30T14:38:26.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.495+0000 7f262919c700 1 -- 172.21.15.32:0/4006368980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624100f80 msgr2=0x7f26241960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:26.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.495+0000 7f262919c700 1 --2- 172.21.15.32:0/4006368980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624100f80 0x7f26241960e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f261800e9d0 tx=0x7f261800ece0 comp rx=0 tx=0).stop 2023-09-30T14:38:26.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.495+0000 7f262919c700 1 -- 172.21.15.32:0/4006368980 shutdown_connections 2023-09-30T14:38:26.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.495+0000 7f262919c700 1 --2- 172.21.15.32:0/4006368980 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f260c0a10b0 0x7f260c0a3570 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:26.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.495+0000 7f262919c700 1 --2- 172.21.15.32:0/4006368980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2624101940 0x7f262419d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:26.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.495+0000 7f262919c700 1 --2- 172.21.15.32:0/4006368980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26240ffe20 0x7f2624195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:26.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.495+0000 7f262919c700 1 --2- 172.21.15.32:0/4006368980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2624100f80 0x7f26241960e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.496+0000 7f262919c700 1 -- 172.21.15.32:0/4006368980 >> 172.21.15.32:0/4006368980 conn(0x7f26240fb3c0 msgr2=0x7f26241044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.496+0000 7f262919c700 1 -- 172.21.15.32:0/4006368980 shutdown_connections 2023-09-30T14:38:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:26.496+0000 7f262919c700 1 -- 172.21.15.32:0/4006368980 wait complete. 2023-09-30T14:38:26.934 INFO:teuthology.run_tasks:Running task fs.ready... 2023-09-30T14:38:26.944 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T14:38:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.075+0000 7f0492b4b700 1 -- 172.21.15.32:0/668058921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f048c1018c0 msgr2=0x7f048c108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.075+0000 7f0492b4b700 1 --2- 172.21.15.32:0/668058921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f048c1018c0 0x7f048c108170 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f048400b0d0 tx=0x7f048400b3e0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.075+0000 7f0492b4b700 1 -- 172.21.15.32:0/668058921 shutdown_connections 2023-09-30T14:38:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.075+0000 7f0492b4b700 1 --2- 172.21.15.32:0/668058921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f048c1018c0 0x7f048c108170 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.075+0000 7f0492b4b700 1 --2- 172.21.15.32:0/668058921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f048c0ffd30 0x7f048c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.075+0000 7f0492b4b700 1 --2- 172.21.15.32:0/668058921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048c100f20 0x7f048c101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.075+0000 7f0492b4b700 1 -- 172.21.15.32:0/668058921 >> 172.21.15.32:0/668058921 conn(0x7f048c0fb280 msgr2=0x7f048c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.075+0000 7f0492b4b700 1 -- 172.21.15.32:0/668058921 shutdown_connections 2023-09-30T14:38:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.075+0000 7f0492b4b700 1 -- 172.21.15.32:0/668058921 wait complete. 2023-09-30T14:38:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.075+0000 7f0492b4b700 1 Processor -- start 2023-09-30T14:38:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.075+0000 7f0492b4b700 1 -- start start 2023-09-30T14:38:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.075+0000 7f0492b4b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f048c0ffd30 0x7f048c195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f0492b4b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f048c100f20 0x7f048c196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f0492b4b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048c1018c0 0x7f048c19d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f0492b4b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c19d670 con 0x7f048c100f20 2023-09-30T14:38:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f0492b4b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c19d7b0 con 0x7f048c1018c0 2023-09-30T14:38:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f0492b4b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c19d940 con 0x7f048c0ffd30 2023-09-30T14:38:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f048bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f048c100f20 0x7f048c196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f048bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f048c100f20 0x7f048c196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34794/0 (socket says 172.21.15.32:34794) 2023-09-30T14:38:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f048bfff700 1 -- 172.21.15.32:0/508862075 learned_addr learned my addr 172.21.15.32:0/508862075 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f04910e8700 1 --2- 172.21.15.32:0/508862075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048c1018c0 0x7f048c19d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f04908e7700 1 --2- 172.21.15.32:0/508862075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f048c0ffd30 0x7f048c195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f048bfff700 1 -- 172.21.15.32:0/508862075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f048c0ffd30 msgr2=0x7f048c195af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f048bfff700 1 --2- 172.21.15.32:0/508862075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f048c0ffd30 0x7f048c195af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f048bfff700 1 -- 172.21.15.32:0/508862075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048c1018c0 msgr2=0x7f048c19d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f048bfff700 1 --2- 172.21.15.32:0/508862075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048c1018c0 0x7f048c19d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f048bfff700 1 -- 172.21.15.32:0/508862075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0484009d00 con 0x7f048c100f20 2023-09-30T14:38:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f04910e8700 1 --2- 172.21.15.32:0/508862075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048c1018c0 0x7f048c19d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:27.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f048bfff700 1 --2- 172.21.15.32:0/508862075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f048c100f20 0x7f048c196030 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f047800c920 tx=0x7f047800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:27.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f0489ffb700 1 -- 172.21.15.32:0/508862075 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478007b00 con 0x7f048c100f20 2023-09-30T14:38:27.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f0489ffb700 1 -- 172.21.15.32:0/508862075 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0478004cf0 con 0x7f048c100f20 2023-09-30T14:38:27.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f0489ffb700 1 -- 172.21.15.32:0/508862075 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478005610 con 0x7f048c100f20 2023-09-30T14:38:27.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.076+0000 7f0492b4b700 1 -- 172.21.15.32:0/508862075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048c19dc20 con 0x7f048c100f20 2023-09-30T14:38:27.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.077+0000 7f0492b4b700 1 -- 172.21.15.32:0/508862075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048c19e130 con 0x7f048c100f20 2023-09-30T14:38:27.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.077+0000 7f0492b4b700 1 -- 172.21.15.32:0/508862075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048c107270 con 0x7f048c100f20 2023-09-30T14:38:27.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.078+0000 7f0489ffb700 1 -- 172.21.15.32:0/508862075 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f0478004750 con 0x7f048c100f20 2023-09-30T14:38:27.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.079+0000 7f0489ffb700 1 --2- 172.21.15.32:0/508862075 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f047c0a10f0 0x7f047c0a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:27.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.079+0000 7f04908e7700 1 --2- 172.21.15.32:0/508862075 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f047c0a10f0 0x7f047c0a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:27.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.079+0000 7f0489ffb700 1 -- 172.21.15.32:0/508862075 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f04780cc7d0 con 0x7f048c100f20 2023-09-30T14:38:27.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.080+0000 7f04908e7700 1 --2- 172.21.15.32:0/508862075 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f047c0a10f0 0x7f047c0a35b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0480006fd0 tx=0x7f0480008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.084+0000 7f0489ffb700 1 -- 172.21.15.32:0/508862075 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f0478095e80 con 0x7f048c100f20 2023-09-30T14:38:27.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.288+0000 7f0492b4b700 1 -- 172.21.15.32:0/508862075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f048c10b630 con 0x7f048c100f20 2023-09-30T14:38:27.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.289+0000 7f0489ffb700 1 -- 172.21.15.32:0/508862075 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22569 (secure 0 0 0) 0x7f0478084090 con 0x7f048c100f20 2023-09-30T14:38:27.290 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:27.290 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":85,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:38:13.192089+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:38:27.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.291+0000 7f0492b4b700 1 -- 172.21.15.32:0/508862075 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f047c0a10f0 msgr2=0x7f047c0a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:27.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.291+0000 7f0492b4b700 1 --2- 172.21.15.32:0/508862075 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f047c0a10f0 0x7f047c0a35b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0480006fd0 tx=0x7f0480008040 comp rx=0 tx=0).stop 2023-09-30T14:38:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.291+0000 7f0492b4b700 1 -- 172.21.15.32:0/508862075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f048c100f20 msgr2=0x7f048c196030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.291+0000 7f0492b4b700 1 --2- 172.21.15.32:0/508862075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f048c100f20 0x7f048c196030 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f047800c920 tx=0x7f047800cce0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.291+0000 7f0492b4b700 1 -- 172.21.15.32:0/508862075 shutdown_connections 2023-09-30T14:38:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.291+0000 7f0492b4b700 1 --2- 172.21.15.32:0/508862075 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f047c0a10f0 0x7f047c0a35b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.291+0000 7f0492b4b700 1 --2- 172.21.15.32:0/508862075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048c1018c0 0x7f048c19d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.291+0000 7f0492b4b700 1 --2- 172.21.15.32:0/508862075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f048c0ffd30 0x7f048c195af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.291+0000 7f0492b4b700 1 --2- 172.21.15.32:0/508862075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f048c100f20 0x7f048c196030 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.291+0000 7f0492b4b700 1 -- 172.21.15.32:0/508862075 >> 172.21.15.32:0/508862075 conn(0x7f048c0fb280 msgr2=0x7f048c1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.291+0000 7f0492b4b700 1 -- 172.21.15.32:0/508862075 shutdown_connections 2023-09-30T14:38:27.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.291+0000 7f0492b4b700 1 -- 172.21.15.32:0/508862075 wait complete. 2023-09-30T14:38:27.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T14:33:20.224529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T14:37:09.573219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T14:37:10.844458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T14:38:27.305 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T14:38:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.437+0000 7fb7250eb700 1 -- 172.21.15.32:0/4209046026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7200ffd30 msgr2=0x7fb720100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.437+0000 7fb7250eb700 1 --2- 172.21.15.32:0/4209046026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7200ffd30 0x7fb720100150 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb7100099c0 tx=0x7fb710009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.437+0000 7fb7250eb700 1 -- 172.21.15.32:0/4209046026 shutdown_connections 2023-09-30T14:38:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.437+0000 7fb7250eb700 1 --2- 172.21.15.32:0/4209046026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7201018c0 0x7fb720108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.437+0000 7fb7250eb700 1 --2- 172.21.15.32:0/4209046026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7200ffd30 0x7fb720100150 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.437+0000 7fb7250eb700 1 --2- 172.21.15.32:0/4209046026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb720100f20 0x7fb720101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.437+0000 7fb7250eb700 1 -- 172.21.15.32:0/4209046026 >> 172.21.15.32:0/4209046026 conn(0x7fb7200fb2a0 msgr2=0x7fb7200fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.437+0000 7fb7250eb700 1 -- 172.21.15.32:0/4209046026 shutdown_connections 2023-09-30T14:38:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.437+0000 7fb7250eb700 1 -- 172.21.15.32:0/4209046026 wait complete. 2023-09-30T14:38:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.438+0000 7fb7250eb700 1 Processor -- start 2023-09-30T14:38:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.438+0000 7fb7250eb700 1 -- start start 2023-09-30T14:38:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.438+0000 7fb7250eb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7200ffd30 0x7fb72010a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.438+0000 7fb7250eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb720100f20 0x7fb720108c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.438+0000 7fb7250eb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7201018c0 0x7fb7201091c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.438+0000 7fb7250eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb720109960 con 0x7fb720100f20 2023-09-30T14:38:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.438+0000 7fb7250eb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb720109ad0 con 0x7fb7200ffd30 2023-09-30T14:38:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.438+0000 7fb7250eb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7201a0c50 con 0x7fb7201018c0 2023-09-30T14:38:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.438+0000 7fb71e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb720100f20 0x7fb720108c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.438+0000 7fb71e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb720100f20 0x7fb720108c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34806/0 (socket says 172.21.15.32:34806) 2023-09-30T14:38:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.438+0000 7fb71ed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7200ffd30 0x7fb72010a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.438+0000 7fb71e59c700 1 -- 172.21.15.32:0/3263679183 learned_addr learned my addr 172.21.15.32:0/3263679183 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.438+0000 7fb71f59e700 1 --2- 172.21.15.32:0/3263679183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7201018c0 0x7fb7201091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.439+0000 7fb71e59c700 1 -- 172.21.15.32:0/3263679183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7201018c0 msgr2=0x7fb7201091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.439+0000 7fb71e59c700 1 --2- 172.21.15.32:0/3263679183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7201018c0 0x7fb7201091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.439+0000 7fb71e59c700 1 -- 172.21.15.32:0/3263679183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7200ffd30 msgr2=0x7fb72010a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.439+0000 7fb71e59c700 1 --2- 172.21.15.32:0/3263679183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7200ffd30 0x7fb72010a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.439+0000 7fb71e59c700 1 -- 172.21.15.32:0/3263679183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7100095f0 con 0x7fb720100f20 2023-09-30T14:38:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.439+0000 7fb71e59c700 1 --2- 172.21.15.32:0/3263679183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb720100f20 0x7fb720108c80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb71400e9a0 tx=0x7fb71400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.439+0000 7fb707fff700 1 -- 172.21.15.32:0/3263679183 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714009790 con 0x7fb720100f20 2023-09-30T14:38:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.439+0000 7fb707fff700 1 -- 172.21.15.32:0/3263679183 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb714010430 con 0x7fb720100f20 2023-09-30T14:38:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.439+0000 7fb7250eb700 1 -- 172.21.15.32:0/3263679183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7201a0f30 con 0x7fb720100f20 2023-09-30T14:38:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.439+0000 7fb707fff700 1 -- 172.21.15.32:0/3263679183 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714018680 con 0x7fb720100f20 2023-09-30T14:38:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.439+0000 7fb7250eb700 1 -- 172.21.15.32:0/3263679183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7201a13b0 con 0x7fb720100f20 2023-09-30T14:38:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.440+0000 7fb7250eb700 1 -- 172.21.15.32:0/3263679183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72010c6a0 con 0x7fb720100f20 2023-09-30T14:38:27.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.441+0000 7fb707fff700 1 -- 172.21.15.32:0/3263679183 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fb7140105a0 con 0x7fb720100f20 2023-09-30T14:38:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.442+0000 7fb707fff700 1 --2- 172.21.15.32:0/3263679183 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb7080a1030 0x7fb7080a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.442+0000 7fb707fff700 1 -- 172.21.15.32:0/3263679183 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb714014070 con 0x7fb720100f20 2023-09-30T14:38:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.442+0000 7fb71ed9d700 1 --2- 172.21.15.32:0/3263679183 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb7080a1030 0x7fb7080a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:27.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.442+0000 7fb71ed9d700 1 --2- 172.21.15.32:0/3263679183 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb7080a1030 0x7fb7080a34f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb710009990 tx=0x7fb710005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:27.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.445+0000 7fb707fff700 1 -- 172.21.15.32:0/3263679183 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fb7140962d0 con 0x7fb720100f20 2023-09-30T14:38:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:27 smithi032 ceph-mon[98592]: pgmap v292: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 109 B/s, 1 objects/s recovering 2023-09-30T14:38:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:27 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/4006368980' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-30T14:38:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:27 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/508862075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:27.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.651+0000 7fb7250eb700 1 -- 172.21.15.32:0/3263679183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb720106cf0 con 0x7fb720100f20 2023-09-30T14:38:27.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.651+0000 7fb707fff700 1 -- 172.21.15.32:0/3263679183 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7140260a0 con 0x7fb720100f20 2023-09-30T14:38:27.652 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T14:38:27.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.653+0000 7fb7250eb700 1 -- 172.21.15.32:0/3263679183 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb7080a1030 msgr2=0x7fb7080a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:27.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.653+0000 7fb7250eb700 1 --2- 172.21.15.32:0/3263679183 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb7080a1030 0x7fb7080a34f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb710009990 tx=0x7fb710005fb0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.653+0000 7fb7250eb700 1 -- 172.21.15.32:0/3263679183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb720100f20 msgr2=0x7fb720108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:27.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.653+0000 7fb7250eb700 1 --2- 172.21.15.32:0/3263679183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb720100f20 0x7fb720108c80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb71400e9a0 tx=0x7fb71400ecb0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.653+0000 7fb7250eb700 1 -- 172.21.15.32:0/3263679183 shutdown_connections 2023-09-30T14:38:27.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.653+0000 7fb7250eb700 1 --2- 172.21.15.32:0/3263679183 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb7080a1030 0x7fb7080a34f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.653+0000 7fb7250eb700 1 --2- 172.21.15.32:0/3263679183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7201018c0 0x7fb7201091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.653+0000 7fb7250eb700 1 --2- 172.21.15.32:0/3263679183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7200ffd30 0x7fb72010a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.653+0000 7fb7250eb700 1 --2- 172.21.15.32:0/3263679183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb720100f20 0x7fb720108c80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.653+0000 7fb7250eb700 1 -- 172.21.15.32:0/3263679183 >> 172.21.15.32:0/3263679183 conn(0x7fb7200fb2a0 msgr2=0x7fb7200fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:27.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.654+0000 7fb7250eb700 1 -- 172.21.15.32:0/3263679183 shutdown_connections 2023-09-30T14:38:27.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.654+0000 7fb7250eb700 1 -- 172.21.15.32:0/3263679183 wait complete. 2023-09-30T14:38:27.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T14:38:27.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.795+0000 7ff11a5a2700 1 -- 172.21.15.32:0/1690165372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1140ffbb0 msgr2=0x7ff1140fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:27.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.795+0000 7ff11a5a2700 1 --2- 172.21.15.32:0/1690165372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1140ffbb0 0x7ff1140fffd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff1080099c0 tx=0x7ff108009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.795+0000 7ff11a5a2700 1 -- 172.21.15.32:0/1690165372 shutdown_connections 2023-09-30T14:38:27.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.795+0000 7ff11a5a2700 1 --2- 172.21.15.32:0/1690165372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff114101740 0x7ff114107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.795+0000 7ff11a5a2700 1 --2- 172.21.15.32:0/1690165372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1140ffbb0 0x7ff1140fffd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.795+0000 7ff11a5a2700 1 --2- 172.21.15.32:0/1690165372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff114100da0 0x7ff114101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.795+0000 7ff11a5a2700 1 -- 172.21.15.32:0/1690165372 >> 172.21.15.32:0/1690165372 conn(0x7ff1140fb100 msgr2=0x7ff1140fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:27.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.795+0000 7ff11a5a2700 1 -- 172.21.15.32:0/1690165372 shutdown_connections 2023-09-30T14:38:27.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.795+0000 7ff11a5a2700 1 -- 172.21.15.32:0/1690165372 wait complete. 2023-09-30T14:38:27.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff11a5a2700 1 Processor -- start 2023-09-30T14:38:27.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff11a5a2700 1 -- start start 2023-09-30T14:38:27.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff11a5a2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1140ffbb0 0x7ff114196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:27.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff11a5a2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff114100da0 0x7ff114196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:27.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff11a5a2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff114101740 0x7ff11419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:27.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff11a5a2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11419e400 con 0x7ff114100da0 2023-09-30T14:38:27.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff11a5a2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11419e540 con 0x7ff114101740 2023-09-30T14:38:27.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff11a5a2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11419e6d0 con 0x7ff1140ffbb0 2023-09-30T14:38:27.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff118d9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff114100da0 0x7ff114196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff118d9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff114100da0 0x7ff114196dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34822/0 (socket says 172.21.15.32:34822) 2023-09-30T14:38:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff118d9f700 1 -- 172.21.15.32:0/1465612352 learned_addr learned my addr 172.21.15.32:0/1465612352 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff119da1700 1 --2- 172.21.15.32:0/1465612352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff114101740 0x7ff11419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff1195a0700 1 --2- 172.21.15.32:0/1465612352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1140ffbb0 0x7ff114196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff118d9f700 1 -- 172.21.15.32:0/1465612352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1140ffbb0 msgr2=0x7ff114196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff118d9f700 1 --2- 172.21.15.32:0/1465612352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1140ffbb0 0x7ff114196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff118d9f700 1 -- 172.21.15.32:0/1465612352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff114101740 msgr2=0x7ff11419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff118d9f700 1 --2- 172.21.15.32:0/1465612352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff114101740 0x7ff11419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff118d9f700 1 -- 172.21.15.32:0/1465612352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1080095f0 con 0x7ff114100da0 2023-09-30T14:38:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.796+0000 7ff1195a0700 1 --2- 172.21.15.32:0/1465612352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1140ffbb0 0x7ff114196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.797+0000 7ff118d9f700 1 --2- 172.21.15.32:0/1465612352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff114100da0 0x7ff114196dc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff11000c9b0 tx=0x7ff11000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:27.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.797+0000 7ff1067fc700 1 -- 172.21.15.32:0/1465612352 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110007b00 con 0x7ff114100da0 2023-09-30T14:38:27.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.797+0000 7ff1067fc700 1 -- 172.21.15.32:0/1465612352 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff110007c60 con 0x7ff114100da0 2023-09-30T14:38:27.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.797+0000 7ff11a5a2700 1 -- 172.21.15.32:0/1465612352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11419e9b0 con 0x7ff114100da0 2023-09-30T14:38:27.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.797+0000 7ff1067fc700 1 -- 172.21.15.32:0/1465612352 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110018660 con 0x7ff114100da0 2023-09-30T14:38:27.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.797+0000 7ff11a5a2700 1 -- 172.21.15.32:0/1465612352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11419ef10 con 0x7ff114100da0 2023-09-30T14:38:27.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.798+0000 7ff11a5a2700 1 -- 172.21.15.32:0/1465612352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11410b410 con 0x7ff114100da0 2023-09-30T14:38:27.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.799+0000 7ff1067fc700 1 -- 172.21.15.32:0/1465612352 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7ff11000f450 con 0x7ff114100da0 2023-09-30T14:38:27.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.799+0000 7ff1067fc700 1 --2- 172.21.15.32:0/1465612352 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff1000a1160 0x7ff1000a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:27.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.800+0000 7ff1067fc700 1 -- 172.21.15.32:0/1465612352 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff1100cc810 con 0x7ff114100da0 2023-09-30T14:38:27.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.800+0000 7ff1195a0700 1 --2- 172.21.15.32:0/1465612352 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff1000a1160 0x7ff1000a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:27.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.800+0000 7ff1195a0700 1 --2- 172.21.15.32:0/1465612352 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff1000a1160 0x7ff1000a3620 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff10800b580 tx=0x7ff108005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:27.804+0000 7ff1067fc700 1 -- 172.21.15.32:0/1465612352 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7ff110095ec0 con 0x7ff114100da0 2023-09-30T14:38:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:27 smithi156 ceph-mon[103579]: pgmap v292: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 109 B/s, 1 objects/s recovering 2023-09-30T14:38:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:27 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/4006368980' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-30T14:38:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:27 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/508862075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:27 smithi148 ceph-mon[102355]: pgmap v292: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 109 B/s, 1 objects/s recovering 2023-09-30T14:38:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:27 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/4006368980' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-30T14:38:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:27 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/508862075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.006+0000 7ff11a5a2700 1 -- 172.21.15.32:0/1465612352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff11419eb40 con 0x7ff114100da0 2023-09-30T14:38:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.007+0000 7ff1067fc700 1 -- 172.21.15.32:0/1465612352 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff11419eb40 con 0x7ff114100da0 2023-09-30T14:38:28.008 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-09-30T14:38:28.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.009+0000 7ff11a5a2700 1 -- 172.21.15.32:0/1465612352 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff1000a1160 msgr2=0x7ff1000a3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.009+0000 7ff11a5a2700 1 --2- 172.21.15.32:0/1465612352 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff1000a1160 0x7ff1000a3620 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff10800b580 tx=0x7ff108005fb0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.009+0000 7ff11a5a2700 1 -- 172.21.15.32:0/1465612352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff114100da0 msgr2=0x7ff114196dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.009+0000 7ff11a5a2700 1 --2- 172.21.15.32:0/1465612352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff114100da0 0x7ff114196dc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff11000c9b0 tx=0x7ff11000ccc0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.009+0000 7ff11a5a2700 1 -- 172.21.15.32:0/1465612352 shutdown_connections 2023-09-30T14:38:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.009+0000 7ff11a5a2700 1 --2- 172.21.15.32:0/1465612352 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff1000a1160 0x7ff1000a3620 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.009+0000 7ff11a5a2700 1 --2- 172.21.15.32:0/1465612352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff114101740 0x7ff11419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.009+0000 7ff11a5a2700 1 --2- 172.21.15.32:0/1465612352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1140ffbb0 0x7ff114196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.009+0000 7ff11a5a2700 1 --2- 172.21.15.32:0/1465612352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff114100da0 0x7ff114196dc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.009+0000 7ff11a5a2700 1 -- 172.21.15.32:0/1465612352 >> 172.21.15.32:0/1465612352 conn(0x7ff1140fb100 msgr2=0x7ff1140fb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.009+0000 7ff11a5a2700 1 -- 172.21.15.32:0/1465612352 shutdown_connections 2023-09-30T14:38:28.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.009+0000 7ff11a5a2700 1 -- 172.21.15.32:0/1465612352 wait complete. 2023-09-30T14:38:28.022 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T14:38:28.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.150+0000 7f036e14e700 1 -- 172.21.15.32:0/838157225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03681018b0 msgr2=0x7f0368103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:28.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.150+0000 7f036e14e700 1 --2- 172.21.15.32:0/838157225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03681018b0 0x7f0368103ca0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f035c009990 tx=0x7f035c009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.150+0000 7f036e14e700 1 -- 172.21.15.32:0/838157225 shutdown_connections 2023-09-30T14:38:28.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.150+0000 7f036e14e700 1 --2- 172.21.15.32:0/838157225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0368106b10 0x7f0368108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.150+0000 7f036e14e700 1 --2- 172.21.15.32:0/838157225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03681018b0 0x7f0368103ca0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.150+0000 7f036e14e700 1 --2- 172.21.15.32:0/838157225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03681041e0 0x7f03681065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.150+0000 7f036e14e700 1 -- 172.21.15.32:0/838157225 >> 172.21.15.32:0/838157225 conn(0x7f03680fb2a0 msgr2=0x7f03680fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:28.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.150+0000 7f036e14e700 1 -- 172.21.15.32:0/838157225 shutdown_connections 2023-09-30T14:38:28.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.150+0000 7f036e14e700 1 -- 172.21.15.32:0/838157225 wait complete. 2023-09-30T14:38:28.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.151+0000 7f036e14e700 1 Processor -- start 2023-09-30T14:38:28.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.151+0000 7f036e14e700 1 -- start start 2023-09-30T14:38:28.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.151+0000 7f036e14e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03681018b0 0x7f0368101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:28.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.151+0000 7f036e14e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03681041e0 0x7f03680ff860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:28.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.151+0000 7f036e14e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0368106b10 0x7f03680ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:28.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.151+0000 7f036e14e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0368100540 con 0x7f0368106b10 2023-09-30T14:38:28.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.151+0000 7f036e14e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03681006b0 con 0x7f03681018b0 2023-09-30T14:38:28.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.151+0000 7f036e14e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03681a0bb0 con 0x7f03681041e0 2023-09-30T14:38:28.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.151+0000 7f0367fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0368106b10 0x7f03680ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:28.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.151+0000 7f0367fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0368106b10 0x7f03680ffda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34838/0 (socket says 172.21.15.32:34838) 2023-09-30T14:38:28.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.151+0000 7f0367fff700 1 -- 172.21.15.32:0/3151959354 learned_addr learned my addr 172.21.15.32:0/3151959354 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:28.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.151+0000 7f03677fe700 1 --2- 172.21.15.32:0/3151959354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03681018b0 0x7f0368101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:28.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.151+0000 7f0366ffd700 1 --2- 172.21.15.32:0/3151959354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03681041e0 0x7f03680ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:28.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.152+0000 7f0367fff700 1 -- 172.21.15.32:0/3151959354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03681041e0 msgr2=0x7f03680ff860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.152+0000 7f0367fff700 1 --2- 172.21.15.32:0/3151959354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03681041e0 0x7f03680ff860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.152+0000 7f0367fff700 1 -- 172.21.15.32:0/3151959354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03681018b0 msgr2=0x7f0368101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.152+0000 7f0367fff700 1 --2- 172.21.15.32:0/3151959354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03681018b0 0x7f0368101220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.152+0000 7f0367fff700 1 -- 172.21.15.32:0/3151959354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c0095f0 con 0x7f0368106b10 2023-09-30T14:38:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.152+0000 7f0367fff700 1 --2- 172.21.15.32:0/3151959354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0368106b10 0x7f03680ffda0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0360007ee0 tx=0x7f036000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.152+0000 7f0364ff9700 1 -- 172.21.15.32:0/3151959354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360017070 con 0x7f0368106b10 2023-09-30T14:38:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.152+0000 7f036e14e700 1 -- 172.21.15.32:0/3151959354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03681a0e90 con 0x7f0368106b10 2023-09-30T14:38:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.152+0000 7f0364ff9700 1 -- 172.21.15.32:0/3151959354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f036000f040 con 0x7f0368106b10 2023-09-30T14:38:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.152+0000 7f0364ff9700 1 -- 172.21.15.32:0/3151959354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360005740 con 0x7f0368106b10 2023-09-30T14:38:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.152+0000 7f036e14e700 1 -- 172.21.15.32:0/3151959354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03681a12c0 con 0x7f0368106b10 2023-09-30T14:38:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.153+0000 7f036e14e700 1 -- 172.21.15.32:0/3151959354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03680fce20 con 0x7f0368106b10 2023-09-30T14:38:28.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.154+0000 7f0364ff9700 1 -- 172.21.15.32:0/3151959354 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f036000dc60 con 0x7f0368106b10 2023-09-30T14:38:28.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.155+0000 7f0364ff9700 1 --2- 172.21.15.32:0/3151959354 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f03500a1030 0x7f03500a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:28.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.155+0000 7f03677fe700 1 --2- 172.21.15.32:0/3151959354 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f03500a1030 0x7f03500a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:28.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.155+0000 7f0364ff9700 1 -- 172.21.15.32:0/3151959354 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f03600cc6b0 con 0x7f0368106b10 2023-09-30T14:38:28.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.156+0000 7f03677fe700 1 --2- 172.21.15.32:0/3151959354 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f03500a1030 0x7f03500a34f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f035c00b2e0 tx=0x7f035c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:28.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.159+0000 7f0364ff9700 1 -- 172.21.15.32:0/3151959354 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f0360095d60 con 0x7f0368106b10 2023-09-30T14:38:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.362+0000 7f036e14e700 1 -- 172.21.15.32:0/3151959354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f036810c7f0 con 0x7f0368106b10 2023-09-30T14:38:28.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.363+0000 7f0364ff9700 1 -- 172.21.15.32:0/3151959354 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0360099380 con 0x7f0368106b10 2023-09-30T14:38:28.364 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-09-30T14:38:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.365+0000 7f036e14e700 1 -- 172.21.15.32:0/3151959354 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f03500a1030 msgr2=0x7f03500a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.365+0000 7f036e14e700 1 --2- 172.21.15.32:0/3151959354 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f03500a1030 0x7f03500a34f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f035c00b2e0 tx=0x7f035c005fb0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.365+0000 7f036e14e700 1 -- 172.21.15.32:0/3151959354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0368106b10 msgr2=0x7f03680ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.365+0000 7f036e14e700 1 --2- 172.21.15.32:0/3151959354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0368106b10 0x7f03680ffda0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0360007ee0 tx=0x7f036000d3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.365+0000 7f036e14e700 1 -- 172.21.15.32:0/3151959354 shutdown_connections 2023-09-30T14:38:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.365+0000 7f036e14e700 1 --2- 172.21.15.32:0/3151959354 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f03500a1030 0x7f03500a34f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.365+0000 7f036e14e700 1 --2- 172.21.15.32:0/3151959354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0368106b10 0x7f03680ffda0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.365+0000 7f036e14e700 1 --2- 172.21.15.32:0/3151959354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03681018b0 0x7f0368101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.365+0000 7f036e14e700 1 --2- 172.21.15.32:0/3151959354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03681041e0 0x7f03680ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.365+0000 7f036e14e700 1 -- 172.21.15.32:0/3151959354 >> 172.21.15.32:0/3151959354 conn(0x7f03680fb2a0 msgr2=0x7f0368102520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.365+0000 7f036e14e700 1 -- 172.21.15.32:0/3151959354 shutdown_connections 2023-09-30T14:38:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.365+0000 7f036e14e700 1 -- 172.21.15.32:0/3151959354 wait complete. 2023-09-30T14:38:28.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:38:28.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.506+0000 7f8531b6f700 1 -- 172.21.15.32:0/721075439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f852c100960 msgr2=0x7f852c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:28.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.506+0000 7f8531b6f700 1 --2- 172.21.15.32:0/721075439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f852c100960 0x7f852c108200 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f852400b0a0 tx=0x7f852400b3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.507+0000 7f8531b6f700 1 -- 172.21.15.32:0/721075439 shutdown_connections 2023-09-30T14:38:28.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.507+0000 7f8531b6f700 1 --2- 172.21.15.32:0/721075439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f852c100960 0x7f852c108200 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.507+0000 7f8531b6f700 1 --2- 172.21.15.32:0/721075439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f852c101350 0x7f852c101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.507+0000 7f8531b6f700 1 --2- 172.21.15.32:0/721075439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f852c0fffa0 0x7f852c100420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.507+0000 7f8531b6f700 1 -- 172.21.15.32:0/721075439 >> 172.21.15.32:0/721075439 conn(0x7f852c0faa30 msgr2=0x7f852c0fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:28.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.507+0000 7f8531b6f700 1 -- 172.21.15.32:0/721075439 shutdown_connections 2023-09-30T14:38:28.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.507+0000 7f8531b6f700 1 -- 172.21.15.32:0/721075439 wait complete. 2023-09-30T14:38:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.507+0000 7f8531b6f700 1 Processor -- start 2023-09-30T14:38:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.507+0000 7f8531b6f700 1 -- start start 2023-09-30T14:38:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f8531b6f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f852c0fffa0 0x7f852c195b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f8531b6f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f852c100960 0x7f852c196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f8531b6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f852c101350 0x7f852c19d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f8531b6f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c19d690 con 0x7f852c101350 2023-09-30T14:38:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f8531b6f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c19d7d0 con 0x7f852c100960 2023-09-30T14:38:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f8531b6f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c19d960 con 0x7f852c0fffa0 2023-09-30T14:38:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f852bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f852c101350 0x7f852c19d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f852bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f852c101350 0x7f852c19d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34852/0 (socket says 172.21.15.32:34852) 2023-09-30T14:38:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f852bfff700 1 -- 172.21.15.32:0/3558197250 learned_addr learned my addr 172.21.15.32:0/3558197250 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f852affd700 1 --2- 172.21.15.32:0/3558197250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f852c100960 0x7f852c196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f852bfff700 1 -- 172.21.15.32:0/3558197250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f852c0fffa0 msgr2=0x7f852c195b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f852b7fe700 1 --2- 172.21.15.32:0/3558197250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f852c0fffa0 0x7f852c195b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f852bfff700 1 --2- 172.21.15.32:0/3558197250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f852c0fffa0 0x7f852c195b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f852bfff700 1 -- 172.21.15.32:0/3558197250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f852c100960 msgr2=0x7f852c196050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f852bfff700 1 --2- 172.21.15.32:0/3558197250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f852c100960 0x7f852c196050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f852bfff700 1 -- 172.21.15.32:0/3558197250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8524009d00 con 0x7f852c101350 2023-09-30T14:38:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f852affd700 1 --2- 172.21.15.32:0/3558197250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f852c100960 0x7f852c196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f852bfff700 1 --2- 172.21.15.32:0/3558197250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f852c101350 0x7f852c19d0c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f852400b070 tx=0x7f8524004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f8528ff9700 1 -- 172.21.15.32:0/3558197250 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852400e040 con 0x7f852c101350 2023-09-30T14:38:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f8528ff9700 1 -- 172.21.15.32:0/3558197250 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8524004020 con 0x7f852c101350 2023-09-30T14:38:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f8531b6f700 1 -- 172.21.15.32:0/3558197250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852c19dbe0 con 0x7f852c101350 2023-09-30T14:38:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f8528ff9700 1 -- 172.21.15.32:0/3558197250 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852401b6b0 con 0x7f852c101350 2023-09-30T14:38:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.508+0000 7f8531b6f700 1 -- 172.21.15.32:0/3558197250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c19e060 con 0x7f852c101350 2023-09-30T14:38:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.510+0000 7f8531b6f700 1 -- 172.21.15.32:0/3558197250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852c10b5b0 con 0x7f852c101350 2023-09-30T14:38:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.511+0000 7f8528ff9700 1 -- 172.21.15.32:0/3558197250 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f8524004190 con 0x7f852c101350 2023-09-30T14:38:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.512+0000 7f8528ff9700 1 --2- 172.21.15.32:0/3558197250 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f85140a1070 0x7f85140a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.512+0000 7f8528ff9700 1 -- 172.21.15.32:0/3558197250 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f85240cde70 con 0x7f852c101350 2023-09-30T14:38:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.512+0000 7f852b7fe700 1 --2- 172.21.15.32:0/3558197250 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f85140a1070 0x7f85140a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:28.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.512+0000 7f852b7fe700 1 --2- 172.21.15.32:0/3558197250 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f85140a1070 0x7f85140a3530 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f852c067d90 tx=0x7f851c009eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:28.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.515+0000 7f8528ff9700 1 -- 172.21.15.32:0/3558197250 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f85240975b0 con 0x7f852c101350 2023-09-30T14:38:28.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:28 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3263679183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:28.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:28 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1465612352' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:28.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:28 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3151959354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:28.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.767+0000 7f8531b6f700 1 -- 172.21.15.32:0/3558197250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f852c10b7d0 con 0x7f852c101350 2023-09-30T14:38:28.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.767+0000 7f8528ff9700 1 -- 172.21.15.32:0/3558197250 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11321 (secure 0 0 0) 0x7f852409abd0 con 0x7f852c101350 2023-09-30T14:38:28.768 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:28.768 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3505064841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3505064841},{"type":"v1","addr":"172.21.15.32:6839","nonce":3505064841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2262846038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2262846038},{"type":"v1","addr":"172.21.15.32:6841","nonce":2262846038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-30T14:37:11.444993+0000","modified":"2023-09-30T14:37:16.971534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6839/1445176203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1445176203},{"type":"v1","addr":"172.21.15.156:6839","nonce":1445176203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/494821686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":494821686},{"type":"v1","addr":"172.21.15.148:6839","nonce":494821686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T14:38:28.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.770+0000 7f8531b6f700 1 -- 172.21.15.32:0/3558197250 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f85140a1070 msgr2=0x7f85140a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:28.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.770+0000 7f8531b6f700 1 --2- 172.21.15.32:0/3558197250 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f85140a1070 0x7f85140a3530 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f852c067d90 tx=0x7f851c009eb0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.770+0000 7f8531b6f700 1 -- 172.21.15.32:0/3558197250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f852c101350 msgr2=0x7f852c19d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:28.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.770+0000 7f8531b6f700 1 --2- 172.21.15.32:0/3558197250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f852c101350 0x7f852c19d0c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f852400b070 tx=0x7f8524004b00 comp rx=0 tx=0).stop 2023-09-30T14:38:28.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.770+0000 7f8531b6f700 1 -- 172.21.15.32:0/3558197250 shutdown_connections 2023-09-30T14:38:28.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.770+0000 7f8531b6f700 1 --2- 172.21.15.32:0/3558197250 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f85140a1070 0x7f85140a3530 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.770+0000 7f8531b6f700 1 --2- 172.21.15.32:0/3558197250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f852c101350 0x7f852c19d0c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.770+0000 7f8531b6f700 1 --2- 172.21.15.32:0/3558197250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f852c0fffa0 0x7f852c195b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.770+0000 7f8531b6f700 1 --2- 172.21.15.32:0/3558197250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f852c100960 0x7f852c196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.770+0000 7f8531b6f700 1 -- 172.21.15.32:0/3558197250 >> 172.21.15.32:0/3558197250 conn(0x7f852c0faa30 msgr2=0x7f852c0fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.770+0000 7f8531b6f700 1 -- 172.21.15.32:0/3558197250 shutdown_connections 2023-09-30T14:38:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.770+0000 7f8531b6f700 1 -- 172.21.15.32:0/3558197250 wait complete. 2023-09-30T14:38:28.772 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 22 2023-09-30T14:38:28.783 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T14:38:28.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:28 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3263679183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:28.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:28 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1465612352' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:28.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:28 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3151959354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:28.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:28 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3263679183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:28.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:28 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1465612352' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:28.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:28 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3151959354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.908+0000 7fd281c62700 1 -- 172.21.15.32:0/4010495466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd27c1018c0 msgr2=0x7fd27c108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.908+0000 7fd281c62700 1 --2- 172.21.15.32:0/4010495466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd27c1018c0 0x7fd27c108170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd27400b0d0 tx=0x7fd27400b3e0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.908+0000 7fd281c62700 1 -- 172.21.15.32:0/4010495466 shutdown_connections 2023-09-30T14:38:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.908+0000 7fd281c62700 1 --2- 172.21.15.32:0/4010495466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd27c1018c0 0x7fd27c108170 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.908+0000 7fd281c62700 1 --2- 172.21.15.32:0/4010495466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0ffd30 0x7fd27c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.908+0000 7fd281c62700 1 --2- 172.21.15.32:0/4010495466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd27c100f20 0x7fd27c101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.908+0000 7fd281c62700 1 -- 172.21.15.32:0/4010495466 >> 172.21.15.32:0/4010495466 conn(0x7fd27c0fb280 msgr2=0x7fd27c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:28.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.908+0000 7fd281c62700 1 -- 172.21.15.32:0/4010495466 shutdown_connections 2023-09-30T14:38:28.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.908+0000 7fd281c62700 1 -- 172.21.15.32:0/4010495466 wait complete. 2023-09-30T14:38:28.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.909+0000 7fd281c62700 1 Processor -- start 2023-09-30T14:38:28.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.909+0000 7fd281c62700 1 -- start start 2023-09-30T14:38:28.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.909+0000 7fd281c62700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0ffd30 0x7fd27c069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:28.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.909+0000 7fd281c62700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd27c100f20 0x7fd27c19b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:28.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.909+0000 7fd281c62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd27c1018c0 0x7fd27c19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:28.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.909+0000 7fd281c62700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c19e780 con 0x7fd27c1018c0 2023-09-30T14:38:28.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.909+0000 7fd281c62700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c19e8f0 con 0x7fd27c0ffd30 2023-09-30T14:38:28.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.909+0000 7fd281c62700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c19eab0 con 0x7fd27c100f20 2023-09-30T14:38:28.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.909+0000 7fd27bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd27c1018c0 0x7fd27c19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:28.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.909+0000 7fd27bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd27c1018c0 0x7fd27c19dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34864/0 (socket says 172.21.15.32:34864) 2023-09-30T14:38:28.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.909+0000 7fd27bfff700 1 -- 172.21.15.32:0/1415116082 learned_addr learned my addr 172.21.15.32:0/1415116082 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:28.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.909+0000 7fd27b7fe700 1 --2- 172.21.15.32:0/1415116082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0ffd30 0x7fd27c069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:28.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.910+0000 7fd27affd700 1 --2- 172.21.15.32:0/1415116082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd27c100f20 0x7fd27c19b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:28.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.910+0000 7fd27bfff700 1 -- 172.21.15.32:0/1415116082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd27c100f20 msgr2=0x7fd27c19b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:28.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.910+0000 7fd27bfff700 1 --2- 172.21.15.32:0/1415116082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd27c100f20 0x7fd27c19b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.910+0000 7fd27bfff700 1 -- 172.21.15.32:0/1415116082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0ffd30 msgr2=0x7fd27c069e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:28.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.910+0000 7fd27bfff700 1 --2- 172.21.15.32:0/1415116082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0ffd30 0x7fd27c069e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:28.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.910+0000 7fd27bfff700 1 -- 172.21.15.32:0/1415116082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd274009d00 con 0x7fd27c1018c0 2023-09-30T14:38:28.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.910+0000 7fd27bfff700 1 --2- 172.21.15.32:0/1415116082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd27c1018c0 0x7fd27c19dfb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd274007990 tx=0x7fd274008eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:28.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.910+0000 7fd278ff9700 1 -- 172.21.15.32:0/1415116082 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27400e040 con 0x7fd27c1018c0 2023-09-30T14:38:28.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.910+0000 7fd278ff9700 1 -- 172.21.15.32:0/1415116082 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd27400be30 con 0x7fd27c1018c0 2023-09-30T14:38:28.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.910+0000 7fd281c62700 1 -- 172.21.15.32:0/1415116082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27c19ed30 con 0x7fd27c1018c0 2023-09-30T14:38:28.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.910+0000 7fd278ff9700 1 -- 172.21.15.32:0/1415116082 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27401d6d0 con 0x7fd27c1018c0 2023-09-30T14:38:28.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.910+0000 7fd281c62700 1 -- 172.21.15.32:0/1415116082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27c19f1b0 con 0x7fd27c1018c0 2023-09-30T14:38:28.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.911+0000 7fd281c62700 1 -- 172.21.15.32:0/1415116082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd27c10b5c0 con 0x7fd27c1018c0 2023-09-30T14:38:28.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.912+0000 7fd278ff9700 1 -- 172.21.15.32:0/1415116082 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fd2740051b0 con 0x7fd27c1018c0 2023-09-30T14:38:28.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.913+0000 7fd278ff9700 1 --2- 172.21.15.32:0/1415116082 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd2640a10f0 0x7fd2640a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:28.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.913+0000 7fd278ff9700 1 -- 172.21.15.32:0/1415116082 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd2740cdf40 con 0x7fd27c1018c0 2023-09-30T14:38:28.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.913+0000 7fd27b7fe700 1 --2- 172.21.15.32:0/1415116082 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd2640a10f0 0x7fd2640a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:28.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.913+0000 7fd27b7fe700 1 --2- 172.21.15.32:0/1415116082 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd2640a10f0 0x7fd2640a35b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd27c100d80 tx=0x7fd270009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:28.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:28.916+0000 7fd278ff9700 1 -- 172.21.15.32:0/1415116082 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fd274097620 con 0x7fd27c1018c0 2023-09-30T14:38:29.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.124+0000 7fd281c62700 1 -- 172.21.15.32:0/1415116082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd27c10b7e0 con 0x7fd27c1018c0 2023-09-30T14:38:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.125+0000 7fd278ff9700 1 -- 172.21.15.32:0/1415116082 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22569 (secure 0 0 0) 0x7fd2740a1070 con 0x7fd27c1018c0 2023-09-30T14:38:29.127 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:29.127 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":85,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:38:13.192089+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:38:29.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.128+0000 7fd281c62700 1 -- 172.21.15.32:0/1415116082 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd2640a10f0 msgr2=0x7fd2640a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:29.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.128+0000 7fd281c62700 1 --2- 172.21.15.32:0/1415116082 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd2640a10f0 0x7fd2640a35b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd27c100d80 tx=0x7fd270009260 comp rx=0 tx=0).stop 2023-09-30T14:38:29.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.128+0000 7fd281c62700 1 -- 172.21.15.32:0/1415116082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd27c1018c0 msgr2=0x7fd27c19dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:29.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.128+0000 7fd281c62700 1 --2- 172.21.15.32:0/1415116082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd27c1018c0 0x7fd27c19dfb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd274007990 tx=0x7fd274008eb0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.128+0000 7fd281c62700 1 -- 172.21.15.32:0/1415116082 shutdown_connections 2023-09-30T14:38:29.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.128+0000 7fd281c62700 1 --2- 172.21.15.32:0/1415116082 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd2640a10f0 0x7fd2640a35b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.128+0000 7fd281c62700 1 --2- 172.21.15.32:0/1415116082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd27c1018c0 0x7fd27c19dfb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.128+0000 7fd281c62700 1 --2- 172.21.15.32:0/1415116082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0ffd30 0x7fd27c069e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.128+0000 7fd281c62700 1 --2- 172.21.15.32:0/1415116082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd27c100f20 0x7fd27c19b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.128+0000 7fd281c62700 1 -- 172.21.15.32:0/1415116082 >> 172.21.15.32:0/1415116082 conn(0x7fd27c0fb280 msgr2=0x7fd27c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:29.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.128+0000 7fd281c62700 1 -- 172.21.15.32:0/1415116082 shutdown_connections 2023-09-30T14:38:29.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.128+0000 7fd281c62700 1 -- 172.21.15.32:0/1415116082 wait complete. 2023-09-30T14:38:29.141 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T14:33:20.224529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T14:37:09.573219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T14:37:10.844458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T14:38:29.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T14:38:29.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.269+0000 7fc094cdd700 1 -- 172.21.15.32:0/730303286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0901006a0 msgr2=0x7fc090100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:29.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.269+0000 7fc094cdd700 1 --2- 172.21.15.32:0/730303286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0901006a0 0x7fc090100b00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc08400aad0 tx=0x7fc08400ade0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.269+0000 7fc094cdd700 1 -- 172.21.15.32:0/730303286 shutdown_connections 2023-09-30T14:38:29.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.269+0000 7fc094cdd700 1 --2- 172.21.15.32:0/730303286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc090101040 0x7fc0901078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.269+0000 7fc094cdd700 1 --2- 172.21.15.32:0/730303286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0900ff4b0 0x7fc0900ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.269+0000 7fc094cdd700 1 --2- 172.21.15.32:0/730303286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0901006a0 0x7fc090100b00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.269+0000 7fc094cdd700 1 -- 172.21.15.32:0/730303286 >> 172.21.15.32:0/730303286 conn(0x7fc0900faa20 msgr2=0x7fc0900fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:29.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.269+0000 7fc094cdd700 1 -- 172.21.15.32:0/730303286 shutdown_connections 2023-09-30T14:38:29.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.270+0000 7fc094cdd700 1 -- 172.21.15.32:0/730303286 wait complete. 2023-09-30T14:38:29.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.270+0000 7fc094cdd700 1 Processor -- start 2023-09-30T14:38:29.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.270+0000 7fc094cdd700 1 -- start start 2023-09-30T14:38:29.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc094cdd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0900ff4b0 0x7fc090196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:29.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc094cdd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0901006a0 0x7fc090197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:29.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc094cdd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc090101040 0x7fc09019e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:29.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc094cdd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09019e6c0 con 0x7fc0900ff4b0 2023-09-30T14:38:29.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc094cdd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09019e800 con 0x7fc090101040 2023-09-30T14:38:29.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc094cdd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09019e990 con 0x7fc0901006a0 2023-09-30T14:38:29.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc08e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0900ff4b0 0x7fc090196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:29.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc08e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0900ff4b0 0x7fc090196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34876/0 (socket says 172.21.15.32:34876) 2023-09-30T14:38:29.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc08e59c700 1 -- 172.21.15.32:0/969171257 learned_addr learned my addr 172.21.15.32:0/969171257 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:29.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc08dd9b700 1 --2- 172.21.15.32:0/969171257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0901006a0 0x7fc090197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:29.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc08ed9d700 1 --2- 172.21.15.32:0/969171257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc090101040 0x7fc09019e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:29.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc08e59c700 1 -- 172.21.15.32:0/969171257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0901006a0 msgr2=0x7fc090197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:29.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc08e59c700 1 --2- 172.21.15.32:0/969171257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0901006a0 0x7fc090197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc08e59c700 1 -- 172.21.15.32:0/969171257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc090101040 msgr2=0x7fc09019e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:29.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc08e59c700 1 --2- 172.21.15.32:0/969171257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc090101040 0x7fc09019e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc08e59c700 1 -- 172.21.15.32:0/969171257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08400a700 con 0x7fc0900ff4b0 2023-09-30T14:38:29.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc08dd9b700 1 --2- 172.21.15.32:0/969171257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0901006a0 0x7fc090197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:29.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc08e59c700 1 --2- 172.21.15.32:0/969171257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0900ff4b0 0x7fc090196b40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc08000d980 tx=0x7fc08000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:29.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc07f7fe700 1 -- 172.21.15.32:0/969171257 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08000de60 con 0x7fc0900ff4b0 2023-09-30T14:38:29.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.271+0000 7fc094cdd700 1 -- 172.21.15.32:0/969171257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09019ec70 con 0x7fc0900ff4b0 2023-09-30T14:38:29.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.272+0000 7fc07f7fe700 1 -- 172.21.15.32:0/969171257 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0800044e0 con 0x7fc0900ff4b0 2023-09-30T14:38:29.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.272+0000 7fc07f7fe700 1 -- 172.21.15.32:0/969171257 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080003be0 con 0x7fc0900ff4b0 2023-09-30T14:38:29.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.272+0000 7fc094cdd700 1 -- 172.21.15.32:0/969171257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09019f1d0 con 0x7fc0900ff4b0 2023-09-30T14:38:29.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.273+0000 7fc094cdd700 1 -- 172.21.15.32:0/969171257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0901063e0 con 0x7fc0900ff4b0 2023-09-30T14:38:29.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.274+0000 7fc07f7fe700 1 -- 172.21.15.32:0/969171257 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fc080009bb0 con 0x7fc0900ff4b0 2023-09-30T14:38:29.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.274+0000 7fc07f7fe700 1 --2- 172.21.15.32:0/969171257 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc0780a1030 0x7fc0780a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:29.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.275+0000 7fc08dd9b700 1 --2- 172.21.15.32:0/969171257 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc0780a1030 0x7fc0780a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:29.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.275+0000 7fc07f7fe700 1 -- 172.21.15.32:0/969171257 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc0800cc970 con 0x7fc0900ff4b0 2023-09-30T14:38:29.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.275+0000 7fc08dd9b700 1 --2- 172.21.15.32:0/969171257 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc0780a1030 0x7fc0780a34f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc084000c00 tx=0x7fc0840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:29.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.278+0000 7fc07f7fe700 1 -- 172.21.15.32:0/969171257 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fc080096020 con 0x7fc0900ff4b0 2023-09-30T14:38:29.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.479+0000 7fc094cdd700 1 -- 172.21.15.32:0/969171257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc09002cfe0 con 0x7fc0900ff4b0 2023-09-30T14:38:29.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.480+0000 7fc07f7fe700 1 -- 172.21.15.32:0/969171257 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc080009940 con 0x7fc0900ff4b0 2023-09-30T14:38:29.481 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T14:38:29.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.482+0000 7fc094cdd700 1 -- 172.21.15.32:0/969171257 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc0780a1030 msgr2=0x7fc0780a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:29.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.482+0000 7fc094cdd700 1 --2- 172.21.15.32:0/969171257 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc0780a1030 0x7fc0780a34f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc084000c00 tx=0x7fc0840058e0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.482+0000 7fc094cdd700 1 -- 172.21.15.32:0/969171257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0900ff4b0 msgr2=0x7fc090196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:29.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.482+0000 7fc094cdd700 1 --2- 172.21.15.32:0/969171257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0900ff4b0 0x7fc090196b40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc08000d980 tx=0x7fc08000dd40 comp rx=0 tx=0).stop 2023-09-30T14:38:29.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.482+0000 7fc094cdd700 1 -- 172.21.15.32:0/969171257 shutdown_connections 2023-09-30T14:38:29.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.482+0000 7fc094cdd700 1 --2- 172.21.15.32:0/969171257 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc0780a1030 0x7fc0780a34f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.482+0000 7fc094cdd700 1 --2- 172.21.15.32:0/969171257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc090101040 0x7fc09019e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.482+0000 7fc094cdd700 1 --2- 172.21.15.32:0/969171257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0900ff4b0 0x7fc090196b40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.482+0000 7fc094cdd700 1 --2- 172.21.15.32:0/969171257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0901006a0 0x7fc090197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.482+0000 7fc094cdd700 1 -- 172.21.15.32:0/969171257 >> 172.21.15.32:0/969171257 conn(0x7fc0900faa20 msgr2=0x7fc0900fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:29.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.482+0000 7fc094cdd700 1 -- 172.21.15.32:0/969171257 shutdown_connections 2023-09-30T14:38:29.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.482+0000 7fc094cdd700 1 -- 172.21.15.32:0/969171257 wait complete. 2023-09-30T14:38:29.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T14:38:29.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.623+0000 7f2875521700 1 -- 172.21.15.32:0/3092082153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2870101ff0 msgr2=0x7f2870102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:29.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.623+0000 7f2875521700 1 --2- 172.21.15.32:0/3092082153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2870101ff0 0x7f2870102470 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2864009990 tx=0x7f2864009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.623+0000 7f2875521700 1 -- 172.21.15.32:0/3092082153 shutdown_connections 2023-09-30T14:38:29.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.623+0000 7f2875521700 1 --2- 172.21.15.32:0/3092082153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28701029b0 0x7f28701092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.623+0000 7f2875521700 1 --2- 172.21.15.32:0/3092082153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2870100ea0 0x7f28701012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.623+0000 7f2875521700 1 --2- 172.21.15.32:0/3092082153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2870101ff0 0x7f2870102470 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.623+0000 7f2875521700 1 -- 172.21.15.32:0/3092082153 >> 172.21.15.32:0/3092082153 conn(0x7f28700fc430 msgr2=0x7f28700fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.623+0000 7f2875521700 1 -- 172.21.15.32:0/3092082153 shutdown_connections 2023-09-30T14:38:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.623+0000 7f2875521700 1 -- 172.21.15.32:0/3092082153 wait complete. 2023-09-30T14:38:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.624+0000 7f2875521700 1 Processor -- start 2023-09-30T14:38:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.624+0000 7f2875521700 1 -- start start 2023-09-30T14:38:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.624+0000 7f2875521700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2870100ea0 0x7f2870196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.624+0000 7f2875521700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2870101ff0 0x7f2870197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.624+0000 7f2875521700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28701029b0 0x7f287019e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.624+0000 7f2875521700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287019e6a0 con 0x7f2870101ff0 2023-09-30T14:38:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.624+0000 7f2875521700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287019e7e0 con 0x7f28701029b0 2023-09-30T14:38:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.624+0000 7f2875521700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287019e970 con 0x7f2870100ea0 2023-09-30T14:38:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.624+0000 7f286e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2870101ff0 0x7f2870197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.624+0000 7f286e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2870101ff0 0x7f2870197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34892/0 (socket says 172.21.15.32:34892) 2023-09-30T14:38:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.624+0000 7f286e7fc700 1 -- 172.21.15.32:0/3091835033 learned_addr learned my addr 172.21.15.32:0/3091835033 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.625+0000 7f286f7fe700 1 --2- 172.21.15.32:0/3091835033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28701029b0 0x7f287019e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.625+0000 7f286effd700 1 --2- 172.21.15.32:0/3091835033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2870100ea0 0x7f2870196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.625+0000 7f286e7fc700 1 -- 172.21.15.32:0/3091835033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2870100ea0 msgr2=0x7f2870196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.625+0000 7f286e7fc700 1 --2- 172.21.15.32:0/3091835033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2870100ea0 0x7f2870196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.625+0000 7f286e7fc700 1 -- 172.21.15.32:0/3091835033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28701029b0 msgr2=0x7f287019e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.625+0000 7f286e7fc700 1 --2- 172.21.15.32:0/3091835033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28701029b0 0x7f287019e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.625+0000 7f286e7fc700 1 -- 172.21.15.32:0/3091835033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28640095f0 con 0x7f2870101ff0 2023-09-30T14:38:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.625+0000 7f286f7fe700 1 --2- 172.21.15.32:0/3091835033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28701029b0 0x7f287019e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.625+0000 7f286e7fc700 1 --2- 172.21.15.32:0/3091835033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2870101ff0 0x7f2870197060 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2864004760 tx=0x7f2864009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.625+0000 7f2857fff700 1 -- 172.21.15.32:0/3091835033 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286401d070 con 0x7f2870101ff0 2023-09-30T14:38:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.625+0000 7f2857fff700 1 -- 172.21.15.32:0/3091835033 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2864004200 con 0x7f2870101ff0 2023-09-30T14:38:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.625+0000 7f2875521700 1 -- 172.21.15.32:0/3091835033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287019ebf0 con 0x7f2870101ff0 2023-09-30T14:38:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.625+0000 7f2857fff700 1 -- 172.21.15.32:0/3091835033 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28640049f0 con 0x7f2870101ff0 2023-09-30T14:38:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.625+0000 7f2875521700 1 -- 172.21.15.32:0/3091835033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287019f0a0 con 0x7f2870101ff0 2023-09-30T14:38:29.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.626+0000 7f2875521700 1 -- 172.21.15.32:0/3091835033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287010c630 con 0x7f2870101ff0 2023-09-30T14:38:29.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.627+0000 7f2857fff700 1 -- 172.21.15.32:0/3091835033 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f2864003d40 con 0x7f2870101ff0 2023-09-30T14:38:29.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.628+0000 7f2857fff700 1 --2- 172.21.15.32:0/3091835033 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f28580a1030 0x7f28580a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.628+0000 7f286effd700 1 --2- 172.21.15.32:0/3091835033 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f28580a1030 0x7f28580a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.628+0000 7f2857fff700 1 -- 172.21.15.32:0/3091835033 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f28640cdc70 con 0x7f2870101ff0 2023-09-30T14:38:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.628+0000 7f286effd700 1 --2- 172.21.15.32:0/3091835033 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f28580a1030 0x7f28580a34f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2860005fd0 tx=0x7f2860005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.631+0000 7f2857fff700 1 -- 172.21.15.32:0/3091835033 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f2864097350 con 0x7f2870101ff0 2023-09-30T14:38:29.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:29 smithi156 ceph-mon[103579]: pgmap v293: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 61 B/s, 0 objects/s recovering 2023-09-30T14:38:29.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:29 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3558197250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:29.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:29 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1415116082' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.834+0000 7f2875521700 1 -- 172.21.15.32:0/3091835033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2870107e10 con 0x7f2870101ff0 2023-09-30T14:38:29.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.834+0000 7f2857fff700 1 -- 172.21.15.32:0/3091835033 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f286409a970 con 0x7f2870101ff0 2023-09-30T14:38:29.835 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-09-30T14:38:29.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.836+0000 7f2875521700 1 -- 172.21.15.32:0/3091835033 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f28580a1030 msgr2=0x7f28580a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:29.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.836+0000 7f2875521700 1 --2- 172.21.15.32:0/3091835033 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f28580a1030 0x7f28580a34f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2860005fd0 tx=0x7f2860005f60 comp rx=0 tx=0).stop 2023-09-30T14:38:29.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.836+0000 7f2875521700 1 -- 172.21.15.32:0/3091835033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2870101ff0 msgr2=0x7f2870197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:29.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.836+0000 7f2875521700 1 --2- 172.21.15.32:0/3091835033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2870101ff0 0x7f2870197060 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2864004760 tx=0x7f2864009f90 comp rx=0 tx=0).stop 2023-09-30T14:38:29.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.836+0000 7f2875521700 1 -- 172.21.15.32:0/3091835033 shutdown_connections 2023-09-30T14:38:29.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.836+0000 7f2875521700 1 --2- 172.21.15.32:0/3091835033 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f28580a1030 0x7f28580a34f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.836+0000 7f2875521700 1 --2- 172.21.15.32:0/3091835033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28701029b0 0x7f287019e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.836+0000 7f2875521700 1 --2- 172.21.15.32:0/3091835033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2870100ea0 0x7f2870196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.836+0000 7f2875521700 1 --2- 172.21.15.32:0/3091835033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2870101ff0 0x7f2870197060 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.836+0000 7f2875521700 1 -- 172.21.15.32:0/3091835033 >> 172.21.15.32:0/3091835033 conn(0x7f28700fc430 msgr2=0x7f28700fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:29.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.836+0000 7f2875521700 1 -- 172.21.15.32:0/3091835033 shutdown_connections 2023-09-30T14:38:29.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.836+0000 7f2875521700 1 -- 172.21.15.32:0/3091835033 wait complete. 2023-09-30T14:38:29.848 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:29 smithi032 ceph-mon[98592]: pgmap v293: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 61 B/s, 0 objects/s recovering 2023-09-30T14:38:29.848 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:29 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3558197250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:29.849 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:29 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1415116082' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:29.849 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T14:38:29.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:29 smithi148 ceph-mon[102355]: pgmap v293: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 61 B/s, 0 objects/s recovering 2023-09-30T14:38:29.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:29 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3558197250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:29.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:29 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1415116082' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:29.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.978+0000 7f44001da700 1 -- 172.21.15.32:0/3659588409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f8100f20 msgr2=0x7f43f8101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:29.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.978+0000 7f44001da700 1 --2- 172.21.15.32:0/3659588409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f8100f20 0x7f43f8101380 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0099c0 tx=0x7f43ec009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.978+0000 7f44001da700 1 -- 172.21.15.32:0/3659588409 shutdown_connections 2023-09-30T14:38:29.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.978+0000 7f44001da700 1 --2- 172.21.15.32:0/3659588409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f81018c0 0x7f43f8108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.978+0000 7f44001da700 1 --2- 172.21.15.32:0/3659588409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43f80ffd30 0x7f43f8100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.978+0000 7f44001da700 1 --2- 172.21.15.32:0/3659588409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f8100f20 0x7f43f8101380 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.978+0000 7f44001da700 1 -- 172.21.15.32:0/3659588409 >> 172.21.15.32:0/3659588409 conn(0x7f43f80fb2a0 msgr2=0x7f43f80fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:29.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.978+0000 7f44001da700 1 -- 172.21.15.32:0/3659588409 shutdown_connections 2023-09-30T14:38:29.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.978+0000 7f44001da700 1 -- 172.21.15.32:0/3659588409 wait complete. 2023-09-30T14:38:29.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f44001da700 1 Processor -- start 2023-09-30T14:38:29.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f44001da700 1 -- start start 2023-09-30T14:38:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f44001da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f80ffd30 0x7f43f8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f44001da700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43f8100f20 0x7f43f819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f44001da700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f81018c0 0x7f43f819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f44001da700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f819e640 con 0x7f43f80ffd30 2023-09-30T14:38:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f44001da700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f819e7b0 con 0x7f43f81018c0 2023-09-30T14:38:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f44001da700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f819e970 con 0x7f43f8100f20 2023-09-30T14:38:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f43fdf76700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f80ffd30 0x7f43f8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f43fdf76700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f80ffd30 0x7f43f8068b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34904/0 (socket says 172.21.15.32:34904) 2023-09-30T14:38:29.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f43fdf76700 1 -- 172.21.15.32:0/3852518846 learned_addr learned my addr 172.21.15.32:0/3852518846 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:29.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f43fd775700 1 --2- 172.21.15.32:0/3852518846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43f8100f20 0x7f43f819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:29.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f43fe777700 1 --2- 172.21.15.32:0/3852518846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f81018c0 0x7f43f819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:29.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f43fdf76700 1 -- 172.21.15.32:0/3852518846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43f8100f20 msgr2=0x7f43f819b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:29.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f43fdf76700 1 --2- 172.21.15.32:0/3852518846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43f8100f20 0x7f43f819b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f43fdf76700 1 -- 172.21.15.32:0/3852518846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f81018c0 msgr2=0x7f43f819dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:29.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f43fdf76700 1 --2- 172.21.15.32:0/3852518846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f81018c0 0x7f43f819dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:29.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f43fdf76700 1 -- 172.21.15.32:0/3852518846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec0095f0 con 0x7f43f80ffd30 2023-09-30T14:38:29.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f43fe777700 1 --2- 172.21.15.32:0/3852518846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f81018c0 0x7f43f819dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:29.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.979+0000 7f43fdf76700 1 --2- 172.21.15.32:0/3852518846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f80ffd30 0x7f43f8068b80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f43f400d980 tx=0x7f43f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:29.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.980+0000 7f43eaffd700 1 -- 172.21.15.32:0/3852518846 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4004020 con 0x7f43f80ffd30 2023-09-30T14:38:29.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.980+0000 7f43eaffd700 1 -- 172.21.15.32:0/3852518846 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43f4009730 con 0x7f43f80ffd30 2023-09-30T14:38:29.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.980+0000 7f44001da700 1 -- 172.21.15.32:0/3852518846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f819ec50 con 0x7f43f80ffd30 2023-09-30T14:38:29.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.980+0000 7f43eaffd700 1 -- 172.21.15.32:0/3852518846 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4003a80 con 0x7f43f80ffd30 2023-09-30T14:38:29.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.980+0000 7f44001da700 1 -- 172.21.15.32:0/3852518846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f819f1b0 con 0x7f43f80ffd30 2023-09-30T14:38:29.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.981+0000 7f44001da700 1 -- 172.21.15.32:0/3852518846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f8106cb0 con 0x7f43f80ffd30 2023-09-30T14:38:29.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.982+0000 7f43eaffd700 1 -- 172.21.15.32:0/3852518846 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f43f40098a0 con 0x7f43f80ffd30 2023-09-30T14:38:29.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.982+0000 7f43eaffd700 1 --2- 172.21.15.32:0/3852518846 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f43e40a10b0 0x7f43e40a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:29.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.982+0000 7f43fd775700 1 --2- 172.21.15.32:0/3852518846 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f43e40a10b0 0x7f43e40a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:29.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.982+0000 7f43eaffd700 1 -- 172.21.15.32:0/3852518846 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f43f40cc850 con 0x7f43f80ffd30 2023-09-30T14:38:29.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.983+0000 7f43fd775700 1 --2- 172.21.15.32:0/3852518846 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f43e40a10b0 0x7f43e40a3570 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00b580 tx=0x7f43ec005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:29.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:29.986+0000 7f43eaffd700 1 -- 172.21.15.32:0/3852518846 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f43f4095f80 con 0x7f43f80ffd30 2023-09-30T14:38:30.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.190+0000 7f44001da700 1 -- 172.21.15.32:0/3852518846 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43f802cf60 con 0x7f43f80ffd30 2023-09-30T14:38:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.190+0000 7f43eaffd700 1 -- 172.21.15.32:0/3852518846 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43f40995a0 con 0x7f43f80ffd30 2023-09-30T14:38:30.191 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-09-30T14:38:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.192+0000 7f44001da700 1 -- 172.21.15.32:0/3852518846 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f43e40a10b0 msgr2=0x7f43e40a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.192+0000 7f44001da700 1 --2- 172.21.15.32:0/3852518846 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f43e40a10b0 0x7f43e40a3570 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00b580 tx=0x7f43ec005bc0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.192+0000 7f44001da700 1 -- 172.21.15.32:0/3852518846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f80ffd30 msgr2=0x7f43f8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.192+0000 7f44001da700 1 --2- 172.21.15.32:0/3852518846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f80ffd30 0x7f43f8068b80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f43f400d980 tx=0x7f43f400dc90 comp rx=0 tx=0).stop 2023-09-30T14:38:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.192+0000 7f44001da700 1 -- 172.21.15.32:0/3852518846 shutdown_connections 2023-09-30T14:38:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.192+0000 7f44001da700 1 --2- 172.21.15.32:0/3852518846 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f43e40a10b0 0x7f43e40a3570 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.192+0000 7f44001da700 1 --2- 172.21.15.32:0/3852518846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f81018c0 0x7f43f819dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.192+0000 7f44001da700 1 --2- 172.21.15.32:0/3852518846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43f80ffd30 0x7f43f8068b80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.192+0000 7f44001da700 1 --2- 172.21.15.32:0/3852518846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43f8100f20 0x7f43f819b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.192+0000 7f44001da700 1 -- 172.21.15.32:0/3852518846 >> 172.21.15.32:0/3852518846 conn(0x7f43f80fb2a0 msgr2=0x7f43f80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.192+0000 7f44001da700 1 -- 172.21.15.32:0/3852518846 shutdown_connections 2023-09-30T14:38:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.192+0000 7f44001da700 1 -- 172.21.15.32:0/3852518846 wait complete. 2023-09-30T14:38:30.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:38:30.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.330+0000 7f3de3612700 1 -- 172.21.15.32:0/2214470890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc106b90 msgr2=0x7f3ddc108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.330+0000 7f3de3612700 1 --2- 172.21.15.32:0/2214470890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc106b90 0x7f3ddc108f80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400b0a0 tx=0x7f3dd400b3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.330+0000 7f3de3612700 1 -- 172.21.15.32:0/2214470890 shutdown_connections 2023-09-30T14:38:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.330+0000 7f3de3612700 1 --2- 172.21.15.32:0/2214470890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc106b90 0x7f3ddc108f80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.330+0000 7f3de3612700 1 --2- 172.21.15.32:0/2214470890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc101930 0x7f3ddc103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.330+0000 7f3de3612700 1 --2- 172.21.15.32:0/2214470890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ddc104260 0x7f3ddc106650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.330+0000 7f3de3612700 1 -- 172.21.15.32:0/2214470890 >> 172.21.15.32:0/2214470890 conn(0x7f3ddc0fb280 msgr2=0x7f3ddc0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.330+0000 7f3de3612700 1 -- 172.21.15.32:0/2214470890 shutdown_connections 2023-09-30T14:38:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.330+0000 7f3de3612700 1 -- 172.21.15.32:0/2214470890 wait complete. 2023-09-30T14:38:30.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.331+0000 7f3de3612700 1 Processor -- start 2023-09-30T14:38:30.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.331+0000 7f3de3612700 1 -- start start 2023-09-30T14:38:30.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.331+0000 7f3de3612700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc101930 0x7f3ddc196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:30.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.331+0000 7f3de3612700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc104260 0x7f3ddc1970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.331+0000 7f3de3612700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ddc106b90 0x7f3ddc19e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3de3612700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc19e720 con 0x7f3ddc104260 2023-09-30T14:38:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3de3612700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc19e860 con 0x7f3ddc101930 2023-09-30T14:38:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3de3612700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc19e9f0 con 0x7f3ddc106b90 2023-09-30T14:38:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3de0bad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc104260 0x7f3ddc1970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3de0bad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc104260 0x7f3ddc1970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34920/0 (socket says 172.21.15.32:34920) 2023-09-30T14:38:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3de0bad700 1 -- 172.21.15.32:0/3857610899 learned_addr learned my addr 172.21.15.32:0/3857610899 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3de13ae700 1 --2- 172.21.15.32:0/3857610899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc101930 0x7f3ddc196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3de1baf700 1 --2- 172.21.15.32:0/3857610899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ddc106b90 0x7f3ddc19e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3de0bad700 1 -- 172.21.15.32:0/3857610899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ddc106b90 msgr2=0x7f3ddc19e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3de0bad700 1 --2- 172.21.15.32:0/3857610899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ddc106b90 0x7f3ddc19e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3de0bad700 1 -- 172.21.15.32:0/3857610899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc101930 msgr2=0x7f3ddc196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:30.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3de0bad700 1 --2- 172.21.15.32:0/3857610899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc101930 0x7f3ddc196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3de0bad700 1 -- 172.21.15.32:0/3857610899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd4009d00 con 0x7f3ddc104260 2023-09-30T14:38:30.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3de0bad700 1 --2- 172.21.15.32:0/3857610899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc104260 0x7f3ddc1970e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00c920 tx=0x7f3dcc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:30.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3dd27fc700 1 -- 172.21.15.32:0/3857610899 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc007b00 con 0x7f3ddc104260 2023-09-30T14:38:30.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3dd27fc700 1 -- 172.21.15.32:0/3857610899 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dcc004cf0 con 0x7f3ddc104260 2023-09-30T14:38:30.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3de3612700 1 -- 172.21.15.32:0/3857610899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ddc19ecd0 con 0x7f3ddc104260 2023-09-30T14:38:30.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3dd27fc700 1 -- 172.21.15.32:0/3857610899 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc005610 con 0x7f3ddc104260 2023-09-30T14:38:30.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.332+0000 7f3de3612700 1 -- 172.21.15.32:0/3857610899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ddc19f1d0 con 0x7f3ddc104260 2023-09-30T14:38:30.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.333+0000 7f3de3612700 1 -- 172.21.15.32:0/3857610899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ddc0fce00 con 0x7f3ddc104260 2023-09-30T14:38:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.334+0000 7f3dd27fc700 1 -- 172.21.15.32:0/3857610899 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f3dcc004750 con 0x7f3ddc104260 2023-09-30T14:38:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.335+0000 7f3dd27fc700 1 --2- 172.21.15.32:0/3857610899 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3dc80a10f0 0x7f3dc80a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.335+0000 7f3de13ae700 1 --2- 172.21.15.32:0/3857610899 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3dc80a10f0 0x7f3dc80a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.335+0000 7f3dd27fc700 1 -- 172.21.15.32:0/3857610899 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3dcc0cc7e0 con 0x7f3ddc104260 2023-09-30T14:38:30.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.336+0000 7f3de13ae700 1 --2- 172.21.15.32:0/3857610899 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3dc80a10f0 0x7f3dc80a35b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009bf0 tx=0x7f3dd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.339+0000 7f3dd27fc700 1 -- 172.21.15.32:0/3857610899 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f3dcc095f10 con 0x7f3ddc104260 2023-09-30T14:38:30.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.589+0000 7f3de3612700 1 -- 172.21.15.32:0/3857610899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ddc10c860 con 0x7f3ddc104260 2023-09-30T14:38:30.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.590+0000 7f3dd27fc700 1 -- 172.21.15.32:0/3857610899 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11321 (secure 0 0 0) 0x7f3dcc01d070 con 0x7f3ddc104260 2023-09-30T14:38:30.591 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:30.591 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3505064841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3505064841},{"type":"v1","addr":"172.21.15.32:6839","nonce":3505064841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2262846038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2262846038},{"type":"v1","addr":"172.21.15.32:6841","nonce":2262846038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-30T14:37:11.444993+0000","modified":"2023-09-30T14:37:16.971534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6839/1445176203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1445176203},{"type":"v1","addr":"172.21.15.156:6839","nonce":1445176203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/494821686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":494821686},{"type":"v1","addr":"172.21.15.148:6839","nonce":494821686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T14:38:30.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.592+0000 7f3de3612700 1 -- 172.21.15.32:0/3857610899 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3dc80a10f0 msgr2=0x7f3dc80a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:30.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.592+0000 7f3de3612700 1 --2- 172.21.15.32:0/3857610899 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3dc80a10f0 0x7f3dc80a35b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009bf0 tx=0x7f3dd8009260 comp rx=0 tx=0).stop 2023-09-30T14:38:30.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.592+0000 7f3de3612700 1 -- 172.21.15.32:0/3857610899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc104260 msgr2=0x7f3ddc1970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:30.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.592+0000 7f3de3612700 1 --2- 172.21.15.32:0/3857610899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc104260 0x7f3ddc1970e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00c920 tx=0x7f3dcc00cce0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.592+0000 7f3de3612700 1 -- 172.21.15.32:0/3857610899 shutdown_connections 2023-09-30T14:38:30.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.592+0000 7f3de3612700 1 --2- 172.21.15.32:0/3857610899 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3dc80a10f0 0x7f3dc80a35b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.592+0000 7f3de3612700 1 --2- 172.21.15.32:0/3857610899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ddc106b90 0x7f3ddc19e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.592+0000 7f3de3612700 1 --2- 172.21.15.32:0/3857610899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ddc101930 0x7f3ddc196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.592+0000 7f3de3612700 1 --2- 172.21.15.32:0/3857610899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc104260 0x7f3ddc1970e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.592+0000 7f3de3612700 1 -- 172.21.15.32:0/3857610899 >> 172.21.15.32:0/3857610899 conn(0x7f3ddc0fb280 msgr2=0x7f3ddc1025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:30.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.592+0000 7f3de3612700 1 -- 172.21.15.32:0/3857610899 shutdown_connections 2023-09-30T14:38:30.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.592+0000 7f3de3612700 1 -- 172.21.15.32:0/3857610899 wait complete. 2023-09-30T14:38:30.594 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 22 2023-09-30T14:38:30.604 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:30 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/969171257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:30.604 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:30 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3091835033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:30.604 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:30 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3852518846' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:30.606 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T14:38:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.729+0000 7f1345d5f700 1 -- 172.21.15.32:0/1915310752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13400ffd40 msgr2=0x7f1340100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.729+0000 7f1345d5f700 1 --2- 172.21.15.32:0/1915310752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13400ffd40 0x7f1340100160 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f13300099c0 tx=0x7f1330009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.729+0000 7f1345d5f700 1 -- 172.21.15.32:0/1915310752 shutdown_connections 2023-09-30T14:38:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.729+0000 7f1345d5f700 1 --2- 172.21.15.32:0/1915310752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13401018d0 0x7f1340108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.729+0000 7f1345d5f700 1 --2- 172.21.15.32:0/1915310752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13400ffd40 0x7f1340100160 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.729+0000 7f1345d5f700 1 --2- 172.21.15.32:0/1915310752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1340100f30 0x7f1340101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.729+0000 7f1345d5f700 1 -- 172.21.15.32:0/1915310752 >> 172.21.15.32:0/1915310752 conn(0x7f13400fb270 msgr2=0x7f13400fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.729+0000 7f1345d5f700 1 -- 172.21.15.32:0/1915310752 shutdown_connections 2023-09-30T14:38:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.729+0000 7f1345d5f700 1 -- 172.21.15.32:0/1915310752 wait complete. 2023-09-30T14:38:30.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.730+0000 7f1345d5f700 1 Processor -- start 2023-09-30T14:38:30.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.730+0000 7f1345d5f700 1 -- start start 2023-09-30T14:38:30.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f1345d5f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13400ffd40 0x7f1340069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:30.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f1345d5f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1340100f30 0x7f134019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:30.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f1345d5f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13401018d0 0x7f134019de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:30.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f1345d5f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134019e650 con 0x7f1340100f30 2023-09-30T14:38:30.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f1345d5f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134019e7c0 con 0x7f13401018d0 2023-09-30T14:38:30.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f1345d5f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134019e980 con 0x7f13400ffd40 2023-09-30T14:38:30.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f133effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1340100f30 0x7f134019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:30.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f133effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1340100f30 0x7f134019b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34934/0 (socket says 172.21.15.32:34934) 2023-09-30T14:38:30.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f133effd700 1 -- 172.21.15.32:0/1147811873 learned_addr learned my addr 172.21.15.32:0/1147811873 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:30.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f133ffff700 1 --2- 172.21.15.32:0/1147811873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13401018d0 0x7f134019de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f133f7fe700 1 --2- 172.21.15.32:0/1147811873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13400ffd40 0x7f1340069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f133effd700 1 -- 172.21.15.32:0/1147811873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13400ffd40 msgr2=0x7f1340069b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f133effd700 1 --2- 172.21.15.32:0/1147811873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13400ffd40 0x7f1340069b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f133effd700 1 -- 172.21.15.32:0/1147811873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13401018d0 msgr2=0x7f134019de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f133effd700 1 --2- 172.21.15.32:0/1147811873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13401018d0 0x7f134019de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f133effd700 1 -- 172.21.15.32:0/1147811873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13300095f0 con 0x7f1340100f30 2023-09-30T14:38:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f133ffff700 1 --2- 172.21.15.32:0/1147811873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13401018d0 0x7f134019de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f133f7fe700 1 --2- 172.21.15.32:0/1147811873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13400ffd40 0x7f1340069b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f133effd700 1 --2- 172.21.15.32:0/1147811873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1340100f30 0x7f134019b830 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f133400c920 tx=0x7f133400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f133cff9700 1 -- 172.21.15.32:0/1147811873 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1334007b00 con 0x7f1340100f30 2023-09-30T14:38:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f133cff9700 1 -- 172.21.15.32:0/1147811873 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1334004cf0 con 0x7f1340100f30 2023-09-30T14:38:30.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f133cff9700 1 -- 172.21.15.32:0/1147811873 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1334005610 con 0x7f1340100f30 2023-09-30T14:38:30.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.731+0000 7f1345d5f700 1 -- 172.21.15.32:0/1147811873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f134019ec60 con 0x7f1340100f30 2023-09-30T14:38:30.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.732+0000 7f1345d5f700 1 -- 172.21.15.32:0/1147811873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134019f170 con 0x7f1340100f30 2023-09-30T14:38:30.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.732+0000 7f1345d5f700 1 -- 172.21.15.32:0/1147811873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f134010b5a0 con 0x7f1340100f30 2023-09-30T14:38:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.733+0000 7f133cff9700 1 -- 172.21.15.32:0/1147811873 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f1334004750 con 0x7f1340100f30 2023-09-30T14:38:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.734+0000 7f133cff9700 1 --2- 172.21.15.32:0/1147811873 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f13280a10f0 0x7f13280a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.734+0000 7f133f7fe700 1 --2- 172.21.15.32:0/1147811873 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f13280a10f0 0x7f13280a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.734+0000 7f133cff9700 1 -- 172.21.15.32:0/1147811873 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f13340cc7d0 con 0x7f1340100f30 2023-09-30T14:38:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.735+0000 7f133f7fe700 1 --2- 172.21.15.32:0/1147811873 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f13280a10f0 0x7f13280a35b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f133000b580 tx=0x7f1330005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.739+0000 7f133cff9700 1 -- 172.21.15.32:0/1147811873 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f1334095e80 con 0x7f1340100f30 2023-09-30T14:38:30.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:30 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/969171257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:30.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:30 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3091835033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:30.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:30 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3852518846' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:30.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:30 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/969171257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:30.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:30 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3091835033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:30.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:30 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3852518846' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:30.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.942+0000 7f1345d5f700 1 -- 172.21.15.32:0/1147811873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f134010b7e0 con 0x7f1340100f30 2023-09-30T14:38:30.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.944+0000 7f133cff9700 1 -- 172.21.15.32:0/1147811873 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22569 (secure 0 0 0) 0x7f1334084090 con 0x7f1340100f30 2023-09-30T14:38:30.945 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:30.945 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":85,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:38:13.192089+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:38:30.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.946+0000 7f1345d5f700 1 -- 172.21.15.32:0/1147811873 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f13280a10f0 msgr2=0x7f13280a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:30.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.946+0000 7f1345d5f700 1 --2- 172.21.15.32:0/1147811873 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f13280a10f0 0x7f13280a35b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f133000b580 tx=0x7f1330005fb0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.946+0000 7f1345d5f700 1 -- 172.21.15.32:0/1147811873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1340100f30 msgr2=0x7f134019b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:30.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.946+0000 7f1345d5f700 1 --2- 172.21.15.32:0/1147811873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1340100f30 0x7f134019b830 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f133400c920 tx=0x7f133400cce0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.946+0000 7f1345d5f700 1 -- 172.21.15.32:0/1147811873 shutdown_connections 2023-09-30T14:38:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.946+0000 7f1345d5f700 1 --2- 172.21.15.32:0/1147811873 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f13280a10f0 0x7f13280a35b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.946+0000 7f1345d5f700 1 --2- 172.21.15.32:0/1147811873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13401018d0 0x7f134019de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.946+0000 7f1345d5f700 1 --2- 172.21.15.32:0/1147811873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13400ffd40 0x7f1340069b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.946+0000 7f1345d5f700 1 --2- 172.21.15.32:0/1147811873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1340100f30 0x7f134019b830 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.946+0000 7f1345d5f700 1 -- 172.21.15.32:0/1147811873 >> 172.21.15.32:0/1147811873 conn(0x7f13400fb270 msgr2=0x7f13400fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.946+0000 7f1345d5f700 1 -- 172.21.15.32:0/1147811873 shutdown_connections 2023-09-30T14:38:30.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:30.946+0000 7f1345d5f700 1 -- 172.21.15.32:0/1147811873 wait complete. 2023-09-30T14:38:30.960 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-30T14:37:11.444993+0000', 'modified': '2023-09-30T14:37:16.971534+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24515, 'mds_1': 24493, 'mds_2': 24485}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6835/4151174061', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 4151174061}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 4151174061}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6835/4180205116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 4180205116}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 4180205116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6837/445372881', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 445372881}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 445372881}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6837/3666119034', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 3666119034}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 3666119034}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.156:6839/1445176203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 1445176203}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 1445176203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24515': {'gid': 24515, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6839/494821686', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 494821686}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 494821686}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-30T14:38:30.960 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-09-30T14:38:30.960 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-09-30T14:38:31.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.088+0000 7f2d9ab5e700 1 -- 172.21.15.32:0/3581806115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d94101910 msgr2=0x7f2d94103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.088+0000 7f2d9ab5e700 1 --2- 172.21.15.32:0/3581806115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d94101910 0x7f2d94103d00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009990 tx=0x7f2d88009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.088+0000 7f2d9ab5e700 1 -- 172.21.15.32:0/3581806115 shutdown_connections 2023-09-30T14:38:31.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.088+0000 7f2d9ab5e700 1 --2- 172.21.15.32:0/3581806115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d94106b70 0x7f2d94108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.088+0000 7f2d9ab5e700 1 --2- 172.21.15.32:0/3581806115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d94101910 0x7f2d94103d00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.088+0000 7f2d9ab5e700 1 --2- 172.21.15.32:0/3581806115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d94104240 0x7f2d94106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.088+0000 7f2d9ab5e700 1 -- 172.21.15.32:0/3581806115 >> 172.21.15.32:0/3581806115 conn(0x7f2d940fb2a0 msgr2=0x7f2d940fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.088+0000 7f2d9ab5e700 1 -- 172.21.15.32:0/3581806115 shutdown_connections 2023-09-30T14:38:31.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.088+0000 7f2d9ab5e700 1 -- 172.21.15.32:0/3581806115 wait complete. 2023-09-30T14:38:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.088+0000 7f2d9ab5e700 1 Processor -- start 2023-09-30T14:38:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.088+0000 7f2d9ab5e700 1 -- start start 2023-09-30T14:38:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d9ab5e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d94101910 0x7f2d94196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d9ab5e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d94104240 0x7f2d941970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d9ab5e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d94106b70 0x7f2d9419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d9ab5e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9419e700 con 0x7f2d94101910 2023-09-30T14:38:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d9ab5e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9419e840 con 0x7f2d94106b70 2023-09-30T14:38:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d9ab5e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9419e9d0 con 0x7f2d94104240 2023-09-30T14:38:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d988fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d94101910 0x7f2d94196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d988fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d94101910 0x7f2d94196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34944/0 (socket says 172.21.15.32:34944) 2023-09-30T14:38:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d988fa700 1 -- 172.21.15.32:0/114900240 learned_addr learned my addr 172.21.15.32:0/114900240 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d93fff700 1 --2- 172.21.15.32:0/114900240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d94104240 0x7f2d941970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d990fb700 1 --2- 172.21.15.32:0/114900240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d94106b70 0x7f2d9419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d988fa700 1 -- 172.21.15.32:0/114900240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d94104240 msgr2=0x7f2d941970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d988fa700 1 --2- 172.21.15.32:0/114900240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d94104240 0x7f2d941970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d988fa700 1 -- 172.21.15.32:0/114900240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d94106b70 msgr2=0x7f2d9419e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d988fa700 1 --2- 172.21.15.32:0/114900240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d94106b70 0x7f2d9419e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d988fa700 1 -- 172.21.15.32:0/114900240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d880095f0 con 0x7f2d94101910 2023-09-30T14:38:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d988fa700 1 --2- 172.21.15.32:0/114900240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d94101910 0x7f2d94196b80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2d88005910 tx=0x7f2d8800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d91ffb700 1 -- 172.21.15.32:0/114900240 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8801d070 con 0x7f2d94101910 2023-09-30T14:38:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d9ab5e700 1 -- 172.21.15.32:0/114900240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9419ec50 con 0x7f2d94101910 2023-09-30T14:38:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d91ffb700 1 -- 172.21.15.32:0/114900240 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d8800fc30 con 0x7f2d94101910 2023-09-30T14:38:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.089+0000 7f2d91ffb700 1 -- 172.21.15.32:0/114900240 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88017650 con 0x7f2d94101910 2023-09-30T14:38:31.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.090+0000 7f2d9ab5e700 1 -- 172.21.15.32:0/114900240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9419f0d0 con 0x7f2d94101910 2023-09-30T14:38:31.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.092+0000 7f2d91ffb700 1 -- 172.21.15.32:0/114900240 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f2d88022470 con 0x7f2d94101910 2023-09-30T14:38:31.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.092+0000 7f2d91ffb700 1 --2- 172.21.15.32:0/114900240 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2d840a1030 0x7f2d840a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.093+0000 7f2d91ffb700 1 -- 172.21.15.32:0/114900240 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2d880cdd10 con 0x7f2d94101910 2023-09-30T14:38:31.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.093+0000 7f2d93fff700 1 --2- 172.21.15.32:0/114900240 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2d840a1030 0x7f2d840a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.093+0000 7f2d93fff700 1 --2- 172.21.15.32:0/114900240 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2d840a1030 0x7f2d840a34f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2d80005950 tx=0x7f2d800058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.093+0000 7f2da6d46b80 1 Processor -- start 2023-09-30T14:38:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.093+0000 7f2da6d46b80 1 -- start start 2023-09-30T14:38:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.093+0000 7f2da6d46b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4c550 0x556b84e4c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.093+0000 7f2da6d46b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4ceb0 0x556b84e4d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.093+0000 7f2da6d46b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4dd50 0x556b84e4e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.093+0000 7f2da6d46b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b84db4410 con 0x556b84e4dd50 2023-09-30T14:38:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2da6d46b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b84da5bd0 con 0x556b84e4ceb0 2023-09-30T14:38:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2da6d46b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b84da34f0 con 0x556b84e4c550 2023-09-30T14:38:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2d990fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4dd50 0x556b84e4e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2d990fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4dd50 0x556b84e4e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34956/0 (socket says 172.21.15.32:34956) 2023-09-30T14:38:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2d990fb700 1 -- 172.21.15.32:0/1763071502 learned_addr learned my addr 172.21.15.32:0/1763071502 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2d93fff700 1 --2- 172.21.15.32:0/1763071502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4ceb0 0x556b84e4d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2d988fa700 1 --2- 172.21.15.32:0/1763071502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4c550 0x556b84e4c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2d990fb700 1 -- 172.21.15.32:0/1763071502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4c550 msgr2=0x556b84e4c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2d990fb700 1 --2- 172.21.15.32:0/1763071502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4c550 0x556b84e4c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2d990fb700 1 -- 172.21.15.32:0/1763071502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4ceb0 msgr2=0x556b84e4d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2d990fb700 1 --2- 172.21.15.32:0/1763071502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4ceb0 0x556b84e4d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2d990fb700 1 -- 172.21.15.32:0/1763071502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b84e01880 con 0x556b84e4dd50 2023-09-30T14:38:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2d988fa700 1 --2- 172.21.15.32:0/1763071502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4c550 0x556b84e4c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2d990fb700 1 --2- 172.21.15.32:0/1763071502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4dd50 0x556b84e4e1d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009dc0 tx=0x7f2d8c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be0844752992e31 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2d7effd700 1 -- 172.21.15.32:0/1763071502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c014610 con 0x556b84e4dd50 2023-09-30T14:38:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2d7effd700 1 -- 172.21.15.32:0/1763071502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d8c00f050 con 0x556b84e4dd50 2023-09-30T14:38:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.094+0000 7f2d7effd700 1 -- 172.21.15.32:0/1763071502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c013860 con 0x556b84e4dd50 2023-09-30T14:38:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.095+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1763071502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4dd50 msgr2=0x556b84e4e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.095+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/1763071502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4dd50 0x556b84e4e1d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009dc0 tx=0x7f2d8c0077e0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.095+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1763071502 shutdown_connections 2023-09-30T14:38:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.095+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/1763071502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4dd50 0x556b84e4e1d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.095+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/1763071502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4c550 0x556b84e4c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.095+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/1763071502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4ceb0 0x556b84e4d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.095+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1763071502 >> 172.21.15.32:0/1763071502 conn(0x556b84dc0210 msgr2=0x556b84e47430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.095+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1763071502 shutdown_connections 2023-09-30T14:38:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.095+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1763071502 wait complete. 2023-09-30T14:38:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.095+0000 7f2da6d46b80 1 -- start start 2023-09-30T14:38:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.095+0000 7f2da6d46b80 1 Processor -- start 2023-09-30T14:38:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2da6d46b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4c550 0x556b84e56340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2da6d46b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4ceb0 0x556b84e56880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2da6d46b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4dd50 0x556b84e5d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2da6d46b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b84d9ede0 con 0x556b84e4dd50 2023-09-30T14:38:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2da6d46b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b84d3b6c0 con 0x556b84e4ceb0 2023-09-30T14:38:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2da6d46b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b84d3b360 con 0x556b84e4c550 2023-09-30T14:38:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2da6d46b80 10 client.0 _ll_register_callbacks cb 0x556b84b987b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2d990fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4dd50 0x556b84e5d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2d990fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4dd50 0x556b84e5d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34962/0 (socket says 172.21.15.32:34962) 2023-09-30T14:38:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2d990fb700 1 -- 172.21.15.32:0/1692129584 learned_addr learned my addr 172.21.15.32:0/1692129584 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2d988fa700 1 --2- 172.21.15.32:0/1692129584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4c550 0x556b84e56340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2d93fff700 1 --2- 172.21.15.32:0/1692129584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4ceb0 0x556b84e56880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2d990fb700 1 -- 172.21.15.32:0/1692129584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4c550 msgr2=0x556b84e56340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2d990fb700 1 --2- 172.21.15.32:0/1692129584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4c550 0x556b84e56340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2d990fb700 1 -- 172.21.15.32:0/1692129584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4ceb0 msgr2=0x556b84e56880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2d990fb700 1 --2- 172.21.15.32:0/1692129584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4ceb0 0x556b84e56880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2d990fb700 1 -- 172.21.15.32:0/1692129584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c009a20 con 0x556b84e4dd50 2023-09-30T14:38:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2d990fb700 1 --2- 172.21.15.32:0/1692129584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4dd50 0x556b84e5d8c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009f50 tx=0x7f2d8c014b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6471d320c7f2582b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.096+0000 7f2d73fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.097+0000 7f2d73fff700 1 -- 172.21.15.32:0/1692129584 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c006000 con 0x556b84e4dd50 2023-09-30T14:38:31.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.097+0000 7f2d73fff700 1 -- 172.21.15.32:0/1692129584 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2d8c0135e0 con 0x556b84e4dd50 2023-09-30T14:38:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.097+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1692129584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b84e3f200 con 0x556b84e4dd50 2023-09-30T14:38:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.097+0000 7f2d73fff700 1 -- 172.21.15.32:0/1692129584 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d8c02c030 con 0x556b84e4dd50 2023-09-30T14:38:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.097+0000 7f2da6d46b80 10 client.15201 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.097+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1692129584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x556b84bc9d90 con 0x556b84e4dd50 2023-09-30T14:38:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.097+0000 7f2d73fff700 1 -- 172.21.15.32:0/1692129584 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556b84e3f200 con 0x556b84e4dd50 2023-09-30T14:38:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.097+0000 7f2d73fff700 1 -- 172.21.15.32:0/1692129584 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2d8c0268e0 con 0x556b84e4dd50 2023-09-30T14:38:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.097+0000 7f2da6d46b80 10 client.15201 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.097+0000 7f2da6d46b80 10 client.15201 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-30T14:38:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.097+0000 7f2da6d46b80 20 client.15201 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.097+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/1692129584 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x556b84e61d40 0x556b84e60870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.097+0000 7f2da6d46b80 4 client.15201 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:38:31.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.097+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1692129584 --> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b84d97dc0 con 0x556b84e61d40 2023-09-30T14:38:31.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.098+0000 7f2d988fa700 1 --2- 172.21.15.32:0/1692129584 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x556b84e61d40 0x556b84e60870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.098+0000 7f2d988fa700 1 --2- 172.21.15.32:0/1692129584 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x556b84e61d40 0x556b84e60870 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2010375591016367 server_cookie=93f3da4a8d339c57 in_seq=0 out_seq=0 2023-09-30T14:38:31.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.098+0000 7f2d73fff700 10 client.15201 ms_handle_connect on v2:172.21.15.148:6836/3666119034 2023-09-30T14:38:31.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.099+0000 7f2d73fff700 1 -- 172.21.15.32:0/1692129584 <== mds.2 v2:172.21.15.148:6836/3666119034 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556b84d97dc0 con 0x556b84e61d40 2023-09-30T14:38:31.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.099+0000 7f2d73fff700 10 client.15201 handle_command_reply: tid=0 2023-09-30T14:38:31.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.099+0000 7f2da6d46b80 1 client.15201 shutdown 2023-09-30T14:38:31.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.099+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1692129584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4dd50 msgr2=0x556b84e5d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.099+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/1692129584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4dd50 0x556b84e5d8c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009f50 tx=0x7f2d8c014b40 comp rx=0 tx=0).stop 2023-09-30T14:38:31.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.099+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1692129584 shutdown_connections 2023-09-30T14:38:31.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.099+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/1692129584 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x556b84e61d40 0x556b84e60870 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.099+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/1692129584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4dd50 0x556b84e5d8c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.099+0000 7f2d73fff700 0 client.15201 ms_handle_reset on v2:172.21.15.148:6836/3666119034 2023-09-30T14:38:31.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.099+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/1692129584 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4c550 0x556b84e56340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.099+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/1692129584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4ceb0 0x556b84e56880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.099+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1692129584 >> 172.21.15.32:0/1692129584 conn(0x556b84dc0210 msgr2=0x556b84e47240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.099+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1692129584 shutdown_connections 2023-09-30T14:38:31.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.099+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1692129584 wait complete. 2023-09-30T14:38:31.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.099+0000 7f2da6d46b80 20 client.15201 trim_cache size 0 max 16384 2023-09-30T14:38:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2da6d46b80 1 Processor -- start 2023-09-30T14:38:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2da6d46b80 1 -- start start 2023-09-30T14:38:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2da6d46b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4c550 0x556b84e47240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2da6d46b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4ceb0 0x556b84e5d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2da6d46b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4dd50 0x556b84e54fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2da6d46b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b84c1f8d0 con 0x556b84e4c550 2023-09-30T14:38:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2da6d46b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b84d454d0 con 0x556b84e4ceb0 2023-09-30T14:38:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2da6d46b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b84d5d590 con 0x556b84e4dd50 2023-09-30T14:38:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2d988fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4c550 0x556b84e47240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2d988fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4c550 0x556b84e47240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34966/0 (socket says 172.21.15.32:34966) 2023-09-30T14:38:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2d988fa700 1 -- 172.21.15.32:0/4188975086 learned_addr learned my addr 172.21.15.32:0/4188975086 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2d990fb700 1 --2- 172.21.15.32:0/4188975086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4dd50 0x556b84e54fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2d93fff700 1 --2- 172.21.15.32:0/4188975086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4ceb0 0x556b84e5d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2d988fa700 1 -- 172.21.15.32:0/4188975086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4dd50 msgr2=0x556b84e54fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2d988fa700 1 --2- 172.21.15.32:0/4188975086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4dd50 0x556b84e54fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2d988fa700 1 -- 172.21.15.32:0/4188975086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4ceb0 msgr2=0x556b84e5d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2d988fa700 1 --2- 172.21.15.32:0/4188975086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4ceb0 0x556b84e5d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2d988fa700 1 -- 172.21.15.32:0/4188975086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b84bb2c10 con 0x556b84e4c550 2023-09-30T14:38:31.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.117+0000 7f2d93fff700 1 --2- 172.21.15.32:0/4188975086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4ceb0 0x556b84e5d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:31.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.118+0000 7f2d988fa700 1 --2- 172.21.15.32:0/4188975086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4c550 0x556b84e47240 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2d880cb790 tx=0x7f2d8801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d7283e3fca0502 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.118+0000 7f2d7f7fe700 1 -- 172.21.15.32:0/4188975086 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8809f2f0 con 0x556b84e4c550 2023-09-30T14:38:31.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.118+0000 7f2d7f7fe700 1 -- 172.21.15.32:0/4188975086 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d880a34a0 con 0x556b84e4c550 2023-09-30T14:38:31.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.118+0000 7f2d7f7fe700 1 -- 172.21.15.32:0/4188975086 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8809a690 con 0x556b84e4c550 2023-09-30T14:38:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.118+0000 7f2da6d46b80 1 -- 172.21.15.32:0/4188975086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4c550 msgr2=0x556b84e47240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.118+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/4188975086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4c550 0x556b84e47240 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2d880cb790 tx=0x7f2d8801a040 comp rx=0 tx=0).stop 2023-09-30T14:38:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.118+0000 7f2da6d46b80 1 -- 172.21.15.32:0/4188975086 shutdown_connections 2023-09-30T14:38:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.118+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/4188975086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4dd50 0x556b84e54fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.118+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/4188975086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4c550 0x556b84e47240 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.118+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/4188975086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4ceb0 0x556b84e5d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.118+0000 7f2da6d46b80 1 -- 172.21.15.32:0/4188975086 >> 172.21.15.32:0/4188975086 conn(0x556b84dc0210 msgr2=0x556b84e46950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.118+0000 7f2da6d46b80 1 -- 172.21.15.32:0/4188975086 shutdown_connections 2023-09-30T14:38:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.118+0000 7f2da6d46b80 1 -- 172.21.15.32:0/4188975086 wait complete. 2023-09-30T14:38:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.119+0000 7f2da6d46b80 1 -- start start 2023-09-30T14:38:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.119+0000 7f2da6d46b80 1 Processor -- start 2023-09-30T14:38:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.119+0000 7f2da6d46b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4c550 0x556b84e57a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.119+0000 7f2da6d46b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4ceb0 0x556b84e57f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.119+0000 7f2da6d46b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4dd50 0x556b84e5cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.119+0000 7f2da6d46b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b84e5d290 con 0x556b84e4c550 2023-09-30T14:38:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.119+0000 7f2da6d46b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b84e5d400 con 0x556b84e4dd50 2023-09-30T14:38:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.119+0000 7f2da6d46b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b84e5d5c0 con 0x556b84e4ceb0 2023-09-30T14:38:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.119+0000 7f2da6d46b80 10 client.0 _ll_register_callbacks cb 0x7f2d80015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.119+0000 7f2d988fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4c550 0x556b84e57a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.119+0000 7f2d988fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4c550 0x556b84e57a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34972/0 (socket says 172.21.15.32:34972) 2023-09-30T14:38:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.119+0000 7f2d988fa700 1 -- 172.21.15.32:0/1028976335 learned_addr learned my addr 172.21.15.32:0/1028976335 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.119+0000 7f2d93fff700 1 --2- 172.21.15.32:0/1028976335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4ceb0 0x556b84e57f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2d990fb700 1 --2- 172.21.15.32:0/1028976335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4dd50 0x556b84e5cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2d988fa700 1 -- 172.21.15.32:0/1028976335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4ceb0 msgr2=0x556b84e57f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2d988fa700 1 --2- 172.21.15.32:0/1028976335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4ceb0 0x556b84e57f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2d988fa700 1 -- 172.21.15.32:0/1028976335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4dd50 msgr2=0x556b84e5cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2d988fa700 1 --2- 172.21.15.32:0/1028976335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4dd50 0x556b84e5cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2d988fa700 1 -- 172.21.15.32:0/1028976335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9419f0d0 con 0x556b84e4c550 2023-09-30T14:38:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2d990fb700 1 --2- 172.21.15.32:0/1028976335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4dd50 0x556b84e5cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2d988fa700 1 --2- 172.21.15.32:0/1028976335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4c550 0x556b84e57a00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2d880a2f70 tx=0x7f2d88009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de82112ee06a8d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2d73fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:31.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2d73fff700 1 -- 172.21.15.32:0/1028976335 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d880992e0 con 0x556b84e4c550 2023-09-30T14:38:31.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2d73fff700 1 -- 172.21.15.32:0/1028976335 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2d8809f2f0 con 0x556b84e4c550 2023-09-30T14:38:31.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1028976335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d8c00f050 con 0x556b84e4c550 2023-09-30T14:38:31.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2d73fff700 1 -- 172.21.15.32:0/1028976335 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d8809a690 con 0x556b84e4c550 2023-09-30T14:38:31.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2da6d46b80 10 client.15207 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:31.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1028976335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x556b84bf0200 con 0x556b84e4c550 2023-09-30T14:38:31.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2d73fff700 1 -- 172.21.15.32:0/1028976335 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d8c00f050 con 0x556b84e4c550 2023-09-30T14:38:31.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2d73fff700 1 -- 172.21.15.32:0/1028976335 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2d8807f070 con 0x556b84e4c550 2023-09-30T14:38:31.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2da6d46b80 10 client.15207 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:31.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2da6d46b80 10 client.15207 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-30T14:38:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2da6d46b80 20 client.15207 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/1028976335 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x556b84eb9c70 0x556b84eb60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2da6d46b80 4 client.15207 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2023-09-30T14:38:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.120+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1028976335 --> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556b84d97ae0 con 0x556b84eb9c70 2023-09-30T14:38:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.121+0000 7f2d93fff700 1 --2- 172.21.15.32:0/1028976335 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x556b84eb9c70 0x556b84eb60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.121+0000 7f2d93fff700 1 --2- 172.21.15.32:0/1028976335 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x556b84eb9c70 0x556b84eb60f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab921aed5c0963d server_cookie=375350ad205e950e in_seq=0 out_seq=0 2023-09-30T14:38:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.121+0000 7f2d73fff700 10 client.15207 ms_handle_connect on v2:172.21.15.148:6836/3666119034 2023-09-30T14:38:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.121+0000 7f2d73fff700 1 -- 172.21.15.32:0/1028976335 <== mds.2 v2:172.21.15.148:6836/3666119034 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x556b84d97ae0 con 0x556b84eb9c70 2023-09-30T14:38:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.121+0000 7f2d73fff700 10 client.15207 handle_command_reply: tid=0 2023-09-30T14:38:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.121+0000 7f2da6d46b80 1 client.15207 shutdown 2023-09-30T14:38:31.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.121+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1028976335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4c550 msgr2=0x556b84e57a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.121+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/1028976335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4c550 0x556b84e57a00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2d880a2f70 tx=0x7f2d88009310 comp rx=0 tx=0).stop 2023-09-30T14:38:31.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.121+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1028976335 shutdown_connections 2023-09-30T14:38:31.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.121+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/1028976335 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x556b84eb9c70 0x556b84eb60f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/1028976335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b84e4dd50 0x556b84e5cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2d73fff700 0 client.15207 ms_handle_reset on v2:172.21.15.148:6836/3666119034 2023-09-30T14:38:31.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/1028976335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556b84e4c550 0x556b84e57a00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2da6d46b80 1 --2- 172.21.15.32:0/1028976335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556b84e4ceb0 0x556b84e57f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1028976335 >> 172.21.15.32:0/1028976335 conn(0x556b84dc0210 msgr2=0x556b84e54fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1028976335 shutdown_connections 2023-09-30T14:38:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2da6d46b80 1 -- 172.21.15.32:0/1028976335 wait complete. 2023-09-30T14:38:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2da6d46b80 20 client.15207 trim_cache size 0 max 16384 2023-09-30T14:38:31.132 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T14:38:31.132 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "bca3f5c6-5f9d-11ee-8db5-212e2dc638e7", 2023-09-30T14:38:31.132 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 2, 2023-09-30T14:38:31.132 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24485, 2023-09-30T14:38:31.132 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-09-30T14:38:31.132 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-09-30T14:38:31.132 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-09-30T14:38:31.133 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 75.058175246999994, 2023-09-30T14:38:31.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 22, 2023-09-30T14:38:31.133 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 85, 2023-09-30T14:38:31.133 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 83, 2023-09-30T14:38:31.133 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 94.504292105000005 2023-09-30T14:38:31.133 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T14:38:31.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2d9ab5e700 1 -- 172.21.15.32:0/114900240 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2d840a1030 msgr2=0x7f2d840a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2d9ab5e700 1 --2- 172.21.15.32:0/114900240 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2d840a1030 0x7f2d840a34f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2d80005950 tx=0x7f2d800058e0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2d9ab5e700 1 -- 172.21.15.32:0/114900240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d94101910 msgr2=0x7f2d94196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2d9ab5e700 1 --2- 172.21.15.32:0/114900240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d94101910 0x7f2d94196b80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2d88005910 tx=0x7f2d8800f740 comp rx=0 tx=0).stop 2023-09-30T14:38:31.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2d9ab5e700 1 -- 172.21.15.32:0/114900240 shutdown_connections 2023-09-30T14:38:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2d9ab5e700 1 --2- 172.21.15.32:0/114900240 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2d840a1030 0x7f2d840a34f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2d9ab5e700 1 --2- 172.21.15.32:0/114900240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d94106b70 0x7f2d9419e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2d9ab5e700 1 --2- 172.21.15.32:0/114900240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d94101910 0x7f2d94196b80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2d9ab5e700 1 --2- 172.21.15.32:0/114900240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d94104240 0x7f2d941970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2d9ab5e700 1 -- 172.21.15.32:0/114900240 >> 172.21.15.32:0/114900240 conn(0x7f2d940fb2a0 msgr2=0x7f2d94102580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2d9ab5e700 1 -- 172.21.15.32:0/114900240 shutdown_connections 2023-09-30T14:38:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.122+0000 7f2d9ab5e700 1 -- 172.21.15.32:0/114900240 wait complete. 2023-09-30T14:38:31.136 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-09-30T14:38:31.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.262+0000 7fb659c23700 1 -- 172.21.15.32:0/1825034815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb654106b50 msgr2=0x7fb654108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.262+0000 7fb659c23700 1 --2- 172.21.15.32:0/1825034815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb654106b50 0x7fb654108fc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00b0a0 tx=0x7fb64c00b3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.262+0000 7fb659c23700 1 -- 172.21.15.32:0/1825034815 shutdown_connections 2023-09-30T14:38:31.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.262+0000 7fb659c23700 1 --2- 172.21.15.32:0/1825034815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb654106b50 0x7fb654108fc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.262+0000 7fb659c23700 1 --2- 172.21.15.32:0/1825034815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6541018f0 0x7fb654103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.262+0000 7fb659c23700 1 --2- 172.21.15.32:0/1825034815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb654104220 0x7fb654106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.262+0000 7fb659c23700 1 -- 172.21.15.32:0/1825034815 >> 172.21.15.32:0/1825034815 conn(0x7fb6540fb320 msgr2=0x7fb6540fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.262+0000 7fb659c23700 1 -- 172.21.15.32:0/1825034815 shutdown_connections 2023-09-30T14:38:31.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.262+0000 7fb659c23700 1 -- 172.21.15.32:0/1825034815 wait complete. 2023-09-30T14:38:31.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb659c23700 1 Processor -- start 2023-09-30T14:38:31.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb659c23700 1 -- start start 2023-09-30T14:38:31.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb659c23700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6541018f0 0x7fb654196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb659c23700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb654104220 0x7fb654197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb659c23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb654106b50 0x7fb65419e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb659c23700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65419e780 con 0x7fb654106b50 2023-09-30T14:38:31.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb659c23700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65419e8f0 con 0x7fb654104220 2023-09-30T14:38:31.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb659c23700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65419eab0 con 0x7fb6541018f0 2023-09-30T14:38:31.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb653fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb654106b50 0x7fb65419e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb653fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb654106b50 0x7fb65419e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34990/0 (socket says 172.21.15.32:34990) 2023-09-30T14:38:31.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb653fff700 1 -- 172.21.15.32:0/249087935 learned_addr learned my addr 172.21.15.32:0/249087935 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:31.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb652ffd700 1 --2- 172.21.15.32:0/249087935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb654104220 0x7fb654197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb6537fe700 1 --2- 172.21.15.32:0/249087935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6541018f0 0x7fb654196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb653fff700 1 -- 172.21.15.32:0/249087935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6541018f0 msgr2=0x7fb654196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb653fff700 1 --2- 172.21.15.32:0/249087935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6541018f0 0x7fb654196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb653fff700 1 -- 172.21.15.32:0/249087935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb654104220 msgr2=0x7fb654197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb653fff700 1 --2- 172.21.15.32:0/249087935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb654104220 0x7fb654197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb653fff700 1 -- 172.21.15.32:0/249087935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c009d00 con 0x7fb654106b50 2023-09-30T14:38:31.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb652ffd700 1 --2- 172.21.15.32:0/249087935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb654104220 0x7fb654197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:31.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.263+0000 7fb6537fe700 1 --2- 172.21.15.32:0/249087935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6541018f0 0x7fb654196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:31.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.264+0000 7fb653fff700 1 --2- 172.21.15.32:0/249087935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb654106b50 0x7fb65419e1b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0062a0 tx=0x7fb64c004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.264+0000 7fb650ff9700 1 -- 172.21.15.32:0/249087935 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c00e040 con 0x7fb654106b50 2023-09-30T14:38:31.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.264+0000 7fb659c23700 1 -- 172.21.15.32:0/249087935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65419ed30 con 0x7fb654106b50 2023-09-30T14:38:31.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.264+0000 7fb650ff9700 1 -- 172.21.15.32:0/249087935 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb64c01db50 con 0x7fb654106b50 2023-09-30T14:38:31.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.264+0000 7fb650ff9700 1 -- 172.21.15.32:0/249087935 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c01bc40 con 0x7fb654106b50 2023-09-30T14:38:31.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.264+0000 7fb659c23700 1 -- 172.21.15.32:0/249087935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65419f1b0 con 0x7fb654106b50 2023-09-30T14:38:31.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.266+0000 7fb650ff9700 1 -- 172.21.15.32:0/249087935 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fb64c01d690 con 0x7fb654106b50 2023-09-30T14:38:31.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.267+0000 7fb650ff9700 1 --2- 172.21.15.32:0/249087935 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb63c0a1030 0x7fb63c0a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.267+0000 7fb650ff9700 1 -- 172.21.15.32:0/249087935 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb64c0ce510 con 0x7fb654106b50 2023-09-30T14:38:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.267+0000 7fb6537fe700 1 --2- 172.21.15.32:0/249087935 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb63c0a1030 0x7fb63c0a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.267+0000 7fb6537fe700 1 --2- 172.21.15.32:0/249087935 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb63c0a1030 0x7fb63c0a34f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb644005950 tx=0x7fb6440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb665e0bb80 1 Processor -- start 2023-09-30T14:38:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb665e0bb80 1 -- start start 2023-09-30T14:38:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb665e0bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bd550 0x5589169bd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb665e0bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bdeb0 0x5589169be770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb665e0bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bed50 0x5589169bf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb665e0bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558916925410 con 0x5589169bd550 2023-09-30T14:38:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb665e0bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558916916bd0 con 0x5589169bdeb0 2023-09-30T14:38:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb665e0bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589169144f0 con 0x5589169bed50 2023-09-30T14:38:31.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb6537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bd550 0x5589169bd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb6537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bd550 0x5589169bd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34994/0 (socket says 172.21.15.32:34994) 2023-09-30T14:38:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb6537fe700 1 -- 172.21.15.32:0/709886746 learned_addr learned my addr 172.21.15.32:0/709886746 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb653fff700 1 --2- 172.21.15.32:0/709886746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bed50 0x5589169bf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb652ffd700 1 --2- 172.21.15.32:0/709886746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bdeb0 0x5589169be770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb6537fe700 1 -- 172.21.15.32:0/709886746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bed50 msgr2=0x5589169bf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb6537fe700 1 --2- 172.21.15.32:0/709886746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bed50 0x5589169bf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb6537fe700 1 -- 172.21.15.32:0/709886746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bdeb0 msgr2=0x5589169be770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb6537fe700 1 --2- 172.21.15.32:0/709886746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bdeb0 0x5589169be770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb6537fe700 1 -- 172.21.15.32:0/709886746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558916972880 con 0x5589169bd550 2023-09-30T14:38:31.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.268+0000 7fb6537fe700 1 --2- 172.21.15.32:0/709886746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bd550 0x5589169bd970 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb6440046c0 tx=0x7fb644012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7ead3c3b44c84a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.269+0000 7fb639ffb700 1 -- 172.21.15.32:0/709886746 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64400fed0 con 0x5589169bd550 2023-09-30T14:38:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.269+0000 7fb639ffb700 1 -- 172.21.15.32:0/709886746 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb644011c60 con 0x5589169bd550 2023-09-30T14:38:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.269+0000 7fb639ffb700 1 -- 172.21.15.32:0/709886746 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64401d610 con 0x5589169bd550 2023-09-30T14:38:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.269+0000 7fb665e0bb80 1 -- 172.21.15.32:0/709886746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bd550 msgr2=0x5589169bd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.269+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/709886746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bd550 0x5589169bd970 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb6440046c0 tx=0x7fb644012000 comp rx=0 tx=0).stop 2023-09-30T14:38:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.269+0000 7fb665e0bb80 1 -- 172.21.15.32:0/709886746 shutdown_connections 2023-09-30T14:38:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.269+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/709886746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bed50 0x5589169bf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.269+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/709886746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bd550 0x5589169bd970 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.269+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/709886746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bdeb0 0x5589169be770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.269+0000 7fb665e0bb80 1 -- 172.21.15.32:0/709886746 >> 172.21.15.32:0/709886746 conn(0x558916931210 msgr2=0x5589169b8430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.269+0000 7fb665e0bb80 1 -- 172.21.15.32:0/709886746 shutdown_connections 2023-09-30T14:38:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.269+0000 7fb665e0bb80 1 -- 172.21.15.32:0/709886746 wait complete. 2023-09-30T14:38:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.270+0000 7fb665e0bb80 1 -- start start 2023-09-30T14:38:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.270+0000 7fb665e0bb80 1 Processor -- start 2023-09-30T14:38:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.270+0000 7fb665e0bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bd550 0x5589169c7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.270+0000 7fb665e0bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bdeb0 0x5589169c7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.270+0000 7fb665e0bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bed50 0x5589169ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.270+0000 7fb665e0bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891690fde0 con 0x5589169bdeb0 2023-09-30T14:38:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.270+0000 7fb665e0bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589168ac6c0 con 0x5589169bed50 2023-09-30T14:38:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.270+0000 7fb665e0bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589168ac360 con 0x5589169bd550 2023-09-30T14:38:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.270+0000 7fb665e0bb80 10 client.0 _ll_register_callbacks cb 0x5589167097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:31.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.270+0000 7fb652ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bdeb0 0x5589169c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.270+0000 7fb652ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bdeb0 0x5589169c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35002/0 (socket says 172.21.15.32:35002) 2023-09-30T14:38:31.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb652ffd700 1 -- 172.21.15.32:0/3170544418 learned_addr learned my addr 172.21.15.32:0/3170544418 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:31.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb6537fe700 1 --2- 172.21.15.32:0/3170544418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bd550 0x5589169c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb653fff700 1 --2- 172.21.15.32:0/3170544418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bed50 0x5589169ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb652ffd700 1 -- 172.21.15.32:0/3170544418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bd550 msgr2=0x5589169c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb652ffd700 1 --2- 172.21.15.32:0/3170544418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bd550 0x5589169c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb652ffd700 1 -- 172.21.15.32:0/3170544418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bed50 msgr2=0x5589169ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb652ffd700 1 --2- 172.21.15.32:0/3170544418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bed50 0x5589169ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb652ffd700 1 -- 172.21.15.32:0/3170544418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64400fbb0 con 0x5589169bdeb0 2023-09-30T14:38:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb653fff700 1 --2- 172.21.15.32:0/3170544418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bed50 0x5589169ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb652ffd700 1 --2- 172.21.15.32:0/3170544418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bdeb0 0x5589169c7840 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb64800bfc0 tx=0x7fb6480116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1d78ec3368377 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb632ffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb632ffd700 1 -- 172.21.15.32:0/3170544418 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb648004830 con 0x5589169bdeb0 2023-09-30T14:38:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb632ffd700 1 -- 172.21.15.32:0/3170544418 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb648004990 con 0x5589169bdeb0 2023-09-30T14:38:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb665e0bb80 1 -- 172.21.15.32:0/3170544418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589169b0200 con 0x5589169bdeb0 2023-09-30T14:38:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb632ffd700 1 -- 172.21.15.32:0/3170544418 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb648021030 con 0x5589169bdeb0 2023-09-30T14:38:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb665e0bb80 10 client.15219 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:31.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb665e0bb80 1 -- 172.21.15.32:0/3170544418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55891673ad90 con 0x5589169bdeb0 2023-09-30T14:38:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.271+0000 7fb632ffd700 1 -- 172.21.15.32:0/3170544418 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5589169b0200 con 0x5589169bdeb0 2023-09-30T14:38:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.272+0000 7fb632ffd700 1 -- 172.21.15.32:0/3170544418 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb6480127b0 con 0x5589169bdeb0 2023-09-30T14:38:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.272+0000 7fb665e0bb80 10 client.15219 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.272+0000 7fb665e0bb80 10 client.15219 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-30T14:38:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.272+0000 7fb665e0bb80 20 client.15219 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.272+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/3170544418 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x5589169d2f30 0x5589169d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.272+0000 7fb665e0bb80 4 client.15219 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:38:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.272+0000 7fb665e0bb80 1 -- 172.21.15.32:0/3170544418 --> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558916908dc0 con 0x5589169d2f30 2023-09-30T14:38:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.272+0000 7fb6537fe700 1 --2- 172.21.15.32:0/3170544418 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x5589169d2f30 0x5589169d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.272+0000 7fb632ffd700 10 client.15219 ms_handle_connect on v2:172.21.15.156:6838/1445176203 2023-09-30T14:38:31.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.272+0000 7fb6537fe700 1 --2- 172.21.15.32:0/3170544418 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x5589169d2f30 0x5589169d19e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b61cad1750f3964 server_cookie=836af0c73621c0de in_seq=0 out_seq=0 2023-09-30T14:38:31.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.273+0000 7fb632ffd700 1 -- 172.21.15.32:0/3170544418 <== mds.1 v2:172.21.15.156:6838/1445176203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558916908dc0 con 0x5589169d2f30 2023-09-30T14:38:31.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.273+0000 7fb632ffd700 10 client.15219 handle_command_reply: tid=0 2023-09-30T14:38:31.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.273+0000 7fb665e0bb80 1 client.15219 shutdown 2023-09-30T14:38:31.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.273+0000 7fb665e0bb80 1 -- 172.21.15.32:0/3170544418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bdeb0 msgr2=0x5589169c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.273+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/3170544418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bdeb0 0x5589169c7840 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb64800bfc0 tx=0x7fb6480116c0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.273+0000 7fb665e0bb80 1 -- 172.21.15.32:0/3170544418 shutdown_connections 2023-09-30T14:38:31.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.273+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/3170544418 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x5589169d2f30 0x5589169d19e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.273+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/3170544418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bed50 0x5589169ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.273+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/3170544418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bd550 0x5589169c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.273+0000 7fb632ffd700 0 client.15219 ms_handle_reset on v2:172.21.15.156:6838/1445176203 2023-09-30T14:38:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.273+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/3170544418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bdeb0 0x5589169c7840 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.273+0000 7fb665e0bb80 1 -- 172.21.15.32:0/3170544418 >> 172.21.15.32:0/3170544418 conn(0x558916931210 msgr2=0x5589169b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.273+0000 7fb665e0bb80 1 -- 172.21.15.32:0/3170544418 shutdown_connections 2023-09-30T14:38:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.273+0000 7fb665e0bb80 1 -- 172.21.15.32:0/3170544418 wait complete. 2023-09-30T14:38:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.274+0000 7fb665e0bb80 20 client.15219 trim_cache size 0 max 16384 2023-09-30T14:38:31.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.291+0000 7fb665e0bb80 1 Processor -- start 2023-09-30T14:38:31.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.291+0000 7fb665e0bb80 1 -- start start 2023-09-30T14:38:31.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.291+0000 7fb665e0bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bd550 0x5589169cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.291+0000 7fb665e0bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bdeb0 0x5589169c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.291+0000 7fb665e0bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bed50 0x5589169c5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.291+0000 7fb665e0bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589167908d0 con 0x5589169bed50 2023-09-30T14:38:31.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.291+0000 7fb665e0bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589168ce590 con 0x5589169bd550 2023-09-30T14:38:31.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.291+0000 7fb665e0bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589169d0570 con 0x5589169bdeb0 2023-09-30T14:38:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.292+0000 7fb653fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bed50 0x5589169c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.292+0000 7fb653fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bed50 0x5589169c5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35012/0 (socket says 172.21.15.32:35012) 2023-09-30T14:38:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.292+0000 7fb653fff700 1 -- 172.21.15.32:0/3165676124 learned_addr learned my addr 172.21.15.32:0/3165676124 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.292+0000 7fb652ffd700 1 --2- 172.21.15.32:0/3165676124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bdeb0 0x5589169c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.292+0000 7fb6537fe700 1 --2- 172.21.15.32:0/3165676124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bd550 0x5589169cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.292+0000 7fb653fff700 1 -- 172.21.15.32:0/3165676124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bdeb0 msgr2=0x5589169c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.292+0000 7fb653fff700 1 --2- 172.21.15.32:0/3165676124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bdeb0 0x5589169c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.292+0000 7fb653fff700 1 -- 172.21.15.32:0/3165676124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bd550 msgr2=0x5589169cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.292+0000 7fb653fff700 1 --2- 172.21.15.32:0/3165676124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bd550 0x5589169cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.292+0000 7fb653fff700 1 -- 172.21.15.32:0/3165676124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558916723c10 con 0x5589169bed50 2023-09-30T14:38:31.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.292+0000 7fb652ffd700 1 --2- 172.21.15.32:0/3165676124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bdeb0 0x5589169c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:31.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.292+0000 7fb653fff700 1 --2- 172.21.15.32:0/3165676124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bed50 0x5589169c5190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb64c09b5b0 tx=0x7fb64c0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba36b0ffd067037 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.292+0000 7fb63a7fc700 1 -- 172.21.15.32:0/3165676124 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c09b8b0 con 0x5589169bed50 2023-09-30T14:38:31.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.292+0000 7fb63a7fc700 1 -- 172.21.15.32:0/3165676124 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb64c09fa00 con 0x5589169bed50 2023-09-30T14:38:31.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.292+0000 7fb63a7fc700 1 -- 172.21.15.32:0/3165676124 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c01dd50 con 0x5589169bed50 2023-09-30T14:38:31.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 -- 172.21.15.32:0/3165676124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bed50 msgr2=0x5589169c5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/3165676124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bed50 0x5589169c5190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb64c09b5b0 tx=0x7fb64c0bb040 comp rx=0 tx=0).stop 2023-09-30T14:38:31.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 -- 172.21.15.32:0/3165676124 shutdown_connections 2023-09-30T14:38:31.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/3165676124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bed50 0x5589169c5190 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/3165676124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bd550 0x5589169cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/3165676124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bdeb0 0x5589169c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 -- 172.21.15.32:0/3165676124 >> 172.21.15.32:0/3165676124 conn(0x558916931210 msgr2=0x5589169b6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 -- 172.21.15.32:0/3165676124 shutdown_connections 2023-09-30T14:38:31.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 -- 172.21.15.32:0/3165676124 wait complete. 2023-09-30T14:38:31.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 -- start start 2023-09-30T14:38:31.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 Processor -- start 2023-09-30T14:38:31.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bd550 0x558916a25960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bdeb0 0x558916a27eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bed50 0x558916a2a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589169c7d80 con 0x5589169bd550 2023-09-30T14:38:31.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558916a2a970 con 0x5589169bed50 2023-09-30T14:38:31.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.293+0000 7fb665e0bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558916a2ab30 con 0x5589169bdeb0 2023-09-30T14:38:31.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb665e0bb80 10 client.0 _ll_register_callbacks cb 0x7fb64800ba10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:31.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb6537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bd550 0x558916a25960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb6537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bd550 0x558916a25960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35014/0 (socket says 172.21.15.32:35014) 2023-09-30T14:38:31.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb6537fe700 1 -- 172.21.15.32:0/228732835 learned_addr learned my addr 172.21.15.32:0/228732835 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:31.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb652ffd700 1 --2- 172.21.15.32:0/228732835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bdeb0 0x558916a27eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb653fff700 1 --2- 172.21.15.32:0/228732835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bed50 0x558916a2a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb6537fe700 1 -- 172.21.15.32:0/228732835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bdeb0 msgr2=0x558916a27eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb6537fe700 1 --2- 172.21.15.32:0/228732835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bdeb0 0x558916a27eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb6537fe700 1 -- 172.21.15.32:0/228732835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bed50 msgr2=0x558916a2a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb6537fe700 1 --2- 172.21.15.32:0/228732835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bed50 0x558916a2a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb6537fe700 1 -- 172.21.15.32:0/228732835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65419f1b0 con 0x5589169bd550 2023-09-30T14:38:31.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb653fff700 1 --2- 172.21.15.32:0/228732835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bed50 0x558916a2a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:31.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb6537fe700 1 --2- 172.21.15.32:0/228732835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bd550 0x558916a25960 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb6440046c0 tx=0x7fb64400bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156365b60bc7e033 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb632ffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:31.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb632ffd700 1 -- 172.21.15.32:0/228732835 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64402e040 con 0x5589169bd550 2023-09-30T14:38:31.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb632ffd700 1 -- 172.21.15.32:0/228732835 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb6440369f0 con 0x5589169bd550 2023-09-30T14:38:31.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb665e0bb80 1 -- 172.21.15.32:0/228732835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb644011c60 con 0x5589169bd550 2023-09-30T14:38:31.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb632ffd700 1 -- 172.21.15.32:0/228732835 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb644036bb0 con 0x5589169bd550 2023-09-30T14:38:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb665e0bb80 10 client.15225 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.294+0000 7fb665e0bb80 1 -- 172.21.15.32:0/228732835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x558916761200 con 0x5589169bd550 2023-09-30T14:38:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.295+0000 7fb632ffd700 1 -- 172.21.15.32:0/228732835 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb644011c60 con 0x5589169bd550 2023-09-30T14:38:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.295+0000 7fb632ffd700 1 -- 172.21.15.32:0/228732835 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb644004910 con 0x5589169bd550 2023-09-30T14:38:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.295+0000 7fb665e0bb80 10 client.15225 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.295+0000 7fb665e0bb80 10 client.15225 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-30T14:38:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.295+0000 7fb665e0bb80 20 client.15225 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.295+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/228732835 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x558916a2fb80 0x558916a2c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.295+0000 7fb665e0bb80 4 client.15225 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2023-09-30T14:38:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.295+0000 7fb665e0bb80 1 -- 172.21.15.32:0/228732835 --> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558916908ae0 con 0x558916a2fb80 2023-09-30T14:38:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.295+0000 7fb652ffd700 1 --2- 172.21.15.32:0/228732835 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x558916a2fb80 0x558916a2c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.295+0000 7fb632ffd700 10 client.15225 ms_handle_connect on v2:172.21.15.156:6838/1445176203 2023-09-30T14:38:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.295+0000 7fb652ffd700 1 --2- 172.21.15.32:0/228732835 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x558916a2fb80 0x558916a2c000 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=29f8dca9fadb757e server_cookie=d5356392f3666cae in_seq=0 out_seq=0 2023-09-30T14:38:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb632ffd700 1 -- 172.21.15.32:0/228732835 <== mds.1 v2:172.21.15.156:6838/1445176203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558916908ae0 con 0x558916a2fb80 2023-09-30T14:38:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb632ffd700 10 client.15225 handle_command_reply: tid=0 2023-09-30T14:38:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb665e0bb80 1 client.15225 shutdown 2023-09-30T14:38:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb665e0bb80 1 -- 172.21.15.32:0/228732835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bd550 msgr2=0x558916a25960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/228732835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bd550 0x558916a25960 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb6440046c0 tx=0x7fb64400bb10 comp rx=0 tx=0).stop 2023-09-30T14:38:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb665e0bb80 1 -- 172.21.15.32:0/228732835 shutdown_connections 2023-09-30T14:38:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/228732835 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x558916a2fb80 0x558916a2c000 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/228732835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589169bed50 0x558916a2a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb632ffd700 0 client.15225 ms_handle_reset on v2:172.21.15.156:6838/1445176203 2023-09-30T14:38:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/228732835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5589169bd550 0x558916a25960 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb665e0bb80 1 --2- 172.21.15.32:0/228732835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5589169bdeb0 0x558916a27eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb665e0bb80 1 -- 172.21.15.32:0/228732835 >> 172.21.15.32:0/228732835 conn(0x558916931210 msgr2=0x5589169b8200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb665e0bb80 1 -- 172.21.15.32:0/228732835 shutdown_connections 2023-09-30T14:38:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb665e0bb80 1 -- 172.21.15.32:0/228732835 wait complete. 2023-09-30T14:38:31.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb665e0bb80 20 client.15225 trim_cache size 0 max 16384 2023-09-30T14:38:31.304 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T14:38:31.304 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "bca3f5c6-5f9d-11ee-8db5-212e2dc638e7", 2023-09-30T14:38:31.304 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 1, 2023-09-30T14:38:31.304 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24493, 2023-09-30T14:38:31.304 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-09-30T14:38:31.304 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-09-30T14:38:31.305 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-09-30T14:38:31.305 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 76.244220161000001, 2023-09-30T14:38:31.305 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 22, 2023-09-30T14:38:31.305 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 85, 2023-09-30T14:38:31.305 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 83, 2023-09-30T14:38:31.305 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 92.314769869000003 2023-09-30T14:38:31.305 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T14:38:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb659c23700 1 -- 172.21.15.32:0/249087935 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb63c0a1030 msgr2=0x7fb63c0a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb659c23700 1 --2- 172.21.15.32:0/249087935 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb63c0a1030 0x7fb63c0a34f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb644005950 tx=0x7fb6440058e0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb659c23700 1 -- 172.21.15.32:0/249087935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb654106b50 msgr2=0x7fb65419e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.296+0000 7fb659c23700 1 --2- 172.21.15.32:0/249087935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb654106b50 0x7fb65419e1b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0062a0 tx=0x7fb64c004db0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.297+0000 7fb659c23700 1 -- 172.21.15.32:0/249087935 shutdown_connections 2023-09-30T14:38:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.297+0000 7fb659c23700 1 --2- 172.21.15.32:0/249087935 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb63c0a1030 0x7fb63c0a34f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.297+0000 7fb659c23700 1 --2- 172.21.15.32:0/249087935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb654106b50 0x7fb65419e1b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.297+0000 7fb659c23700 1 --2- 172.21.15.32:0/249087935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6541018f0 0x7fb654196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.297+0000 7fb659c23700 1 --2- 172.21.15.32:0/249087935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb654104220 0x7fb654197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.297+0000 7fb659c23700 1 -- 172.21.15.32:0/249087935 >> 172.21.15.32:0/249087935 conn(0x7fb6540fb320 msgr2=0x7fb654102600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.297+0000 7fb659c23700 1 -- 172.21.15.32:0/249087935 shutdown_connections 2023-09-30T14:38:31.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.297+0000 7fb659c23700 1 -- 172.21.15.32:0/249087935 wait complete. 2023-09-30T14:38:31.310 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-30T14:38:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.436+0000 7fb62f2ba700 1 -- 172.21.15.32:0/2106714817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6280ffd30 msgr2=0x7fb628100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.436+0000 7fb62f2ba700 1 --2- 172.21.15.32:0/2106714817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6280ffd30 0x7fb628100150 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb6180099c0 tx=0x7fb618009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.436+0000 7fb62f2ba700 1 -- 172.21.15.32:0/2106714817 shutdown_connections 2023-09-30T14:38:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.436+0000 7fb62f2ba700 1 --2- 172.21.15.32:0/2106714817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6281018c0 0x7fb628108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.436+0000 7fb62f2ba700 1 --2- 172.21.15.32:0/2106714817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6280ffd30 0x7fb628100150 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.436+0000 7fb62f2ba700 1 --2- 172.21.15.32:0/2106714817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb628100f20 0x7fb628101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.436+0000 7fb62f2ba700 1 -- 172.21.15.32:0/2106714817 >> 172.21.15.32:0/2106714817 conn(0x7fb6280fb2a0 msgr2=0x7fb6280fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.436+0000 7fb62f2ba700 1 -- 172.21.15.32:0/2106714817 shutdown_connections 2023-09-30T14:38:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.436+0000 7fb62f2ba700 1 -- 172.21.15.32:0/2106714817 wait complete. 2023-09-30T14:38:31.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62f2ba700 1 Processor -- start 2023-09-30T14:38:31.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62f2ba700 1 -- start start 2023-09-30T14:38:31.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62f2ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6280ffd30 0x7fb628068b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62f2ba700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb628100f20 0x7fb62819b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62f2ba700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6281018c0 0x7fb62819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62f2ba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62819e6a0 con 0x7fb6280ffd30 2023-09-30T14:38:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62f2ba700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62819e810 con 0x7fb6281018c0 2023-09-30T14:38:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62f2ba700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62819e9d0 con 0x7fb628100f20 2023-09-30T14:38:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62d056700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6280ffd30 0x7fb628068b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62d056700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6280ffd30 0x7fb628068b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35026/0 (socket says 172.21.15.32:35026) 2023-09-30T14:38:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62d056700 1 -- 172.21.15.32:0/1633541254 learned_addr learned my addr 172.21.15.32:0/1633541254 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62c855700 1 --2- 172.21.15.32:0/1633541254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb628100f20 0x7fb62819b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62d857700 1 --2- 172.21.15.32:0/1633541254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6281018c0 0x7fb62819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62d056700 1 -- 172.21.15.32:0/1633541254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb628100f20 msgr2=0x7fb62819b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62d056700 1 --2- 172.21.15.32:0/1633541254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb628100f20 0x7fb62819b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62d056700 1 -- 172.21.15.32:0/1633541254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6281018c0 msgr2=0x7fb62819ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62d056700 1 --2- 172.21.15.32:0/1633541254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6281018c0 0x7fb62819ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62d056700 1 -- 172.21.15.32:0/1633541254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6180095f0 con 0x7fb6280ffd30 2023-09-30T14:38:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.437+0000 7fb62d056700 1 --2- 172.21.15.32:0/1633541254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6280ffd30 0x7fb628068b40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb618009fd0 tx=0x7fb618009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.438+0000 7fb61e7fc700 1 -- 172.21.15.32:0/1633541254 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61801d070 con 0x7fb6280ffd30 2023-09-30T14:38:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.438+0000 7fb61e7fc700 1 -- 172.21.15.32:0/1633541254 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb618005030 con 0x7fb6280ffd30 2023-09-30T14:38:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.438+0000 7fb62f2ba700 1 -- 172.21.15.32:0/1633541254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62819ec50 con 0x7fb6280ffd30 2023-09-30T14:38:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.438+0000 7fb61e7fc700 1 -- 172.21.15.32:0/1633541254 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618004b30 con 0x7fb6280ffd30 2023-09-30T14:38:31.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.438+0000 7fb62f2ba700 1 -- 172.21.15.32:0/1633541254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62819f0d0 con 0x7fb6280ffd30 2023-09-30T14:38:31.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.440+0000 7fb61e7fc700 1 -- 172.21.15.32:0/1633541254 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fb6180051a0 con 0x7fb6280ffd30 2023-09-30T14:38:31.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.441+0000 7fb61e7fc700 1 --2- 172.21.15.32:0/1633541254 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb6140a10b0 0x7fb6140a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.441+0000 7fb61e7fc700 1 -- 172.21.15.32:0/1633541254 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb6180cd690 con 0x7fb6280ffd30 2023-09-30T14:38:31.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.441+0000 7fb62c855700 1 --2- 172.21.15.32:0/1633541254 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb6140a10b0 0x7fb6140a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.442+0000 7fb62c855700 1 --2- 172.21.15.32:0/1633541254 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb6140a10b0 0x7fb6140a3570 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb6280688b0 tx=0x7fb624009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.442+0000 7fb63b4a2b80 1 Processor -- start 2023-09-30T14:38:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.442+0000 7fb63b4a2b80 1 -- start start 2023-09-30T14:38:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.442+0000 7fb63b4a2b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa53293960 0x55aa53293d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.442+0000 7fb63b4a2b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa532942c0 0x55aa53294b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.442+0000 7fb63b4a2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa53295160 0x55aa532955e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.442+0000 7fb63b4a2b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa531f7410 con 0x55aa53293960 2023-09-30T14:38:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.442+0000 7fb63b4a2b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa531e8bd0 con 0x55aa532942c0 2023-09-30T14:38:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.442+0000 7fb63b4a2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa531e64f0 con 0x55aa53295160 2023-09-30T14:38:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.442+0000 7fb62d056700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa53293960 0x55aa53293d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.442+0000 7fb62d056700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa53293960 0x55aa53293d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35034/0 (socket says 172.21.15.32:35034) 2023-09-30T14:38:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.442+0000 7fb62d056700 1 -- 172.21.15.32:0/2414497220 learned_addr learned my addr 172.21.15.32:0/2414497220 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.442+0000 7fb62c855700 1 --2- 172.21.15.32:0/2414497220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa532942c0 0x55aa53294b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.443+0000 7fb62d857700 1 --2- 172.21.15.32:0/2414497220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa53295160 0x55aa532955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.443+0000 7fb62d056700 1 -- 172.21.15.32:0/2414497220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa53295160 msgr2=0x55aa532955e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.443+0000 7fb62d056700 1 --2- 172.21.15.32:0/2414497220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa53295160 0x55aa532955e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.443+0000 7fb62d056700 1 -- 172.21.15.32:0/2414497220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa532942c0 msgr2=0x55aa53294b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.443+0000 7fb62d056700 1 --2- 172.21.15.32:0/2414497220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa532942c0 0x55aa53294b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.443+0000 7fb62d056700 1 -- 172.21.15.32:0/2414497220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa53244880 con 0x55aa53293960 2023-09-30T14:38:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.443+0000 7fb62c855700 1 --2- 172.21.15.32:0/2414497220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa532942c0 0x55aa53294b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.443+0000 7fb62d056700 1 --2- 172.21.15.32:0/2414497220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa53293960 0x55aa53293d80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb6180974a0 tx=0x7fb61809b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3555529b13efebcf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.443+0000 7fb6137fe700 1 -- 172.21.15.32:0/2414497220 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6180a2d30 con 0x55aa53293960 2023-09-30T14:38:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.443+0000 7fb6137fe700 1 -- 172.21.15.32:0/2414497220 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6180cb240 con 0x55aa53293960 2023-09-30T14:38:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.443+0000 7fb6137fe700 1 -- 172.21.15.32:0/2414497220 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61809ac90 con 0x55aa53293960 2023-09-30T14:38:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.444+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/2414497220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa53293960 msgr2=0x55aa53293d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.444+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/2414497220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa53293960 0x55aa53293d80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb6180974a0 tx=0x7fb61809b1b0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.444+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/2414497220 shutdown_connections 2023-09-30T14:38:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.444+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/2414497220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa53295160 0x55aa532955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.444+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/2414497220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa53293960 0x55aa53293d80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.444+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/2414497220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa532942c0 0x55aa53294b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.444+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/2414497220 >> 172.21.15.32:0/2414497220 conn(0x55aa53203210 msgr2=0x55aa5328a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.444+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/2414497220 shutdown_connections 2023-09-30T14:38:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.444+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/2414497220 wait complete. 2023-09-30T14:38:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.444+0000 7fb63b4a2b80 1 -- start start 2023-09-30T14:38:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.444+0000 7fb63b4a2b80 1 Processor -- start 2023-09-30T14:38:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb63b4a2b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa53293960 0x55aa5329d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb63b4a2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa532942c0 0x55aa5329d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb63b4a2b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa53295160 0x55aa532a4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb63b4a2b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa531e1de0 con 0x55aa53293960 2023-09-30T14:38:31.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb63b4a2b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa5317e6c0 con 0x55aa53295160 2023-09-30T14:38:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb63b4a2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa5317e360 con 0x55aa532942c0 2023-09-30T14:38:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb63b4a2b80 10 client.0 _ll_register_callbacks cb 0x55aa52fdb7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb62d056700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa53293960 0x55aa5329d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb62d056700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa53293960 0x55aa5329d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35040/0 (socket says 172.21.15.32:35040) 2023-09-30T14:38:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb62d056700 1 -- 172.21.15.32:0/694668208 learned_addr learned my addr 172.21.15.32:0/694668208 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb62c855700 1 --2- 172.21.15.32:0/694668208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa532942c0 0x55aa5329d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb62d857700 1 --2- 172.21.15.32:0/694668208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa53295160 0x55aa532a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb62d056700 1 -- 172.21.15.32:0/694668208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa532942c0 msgr2=0x55aa5329d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb62d056700 1 --2- 172.21.15.32:0/694668208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa532942c0 0x55aa5329d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb62d056700 1 -- 172.21.15.32:0/694668208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa53295160 msgr2=0x55aa532a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb62d056700 1 --2- 172.21.15.32:0/694668208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa53295160 0x55aa532a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb62d056700 1 -- 172.21.15.32:0/694668208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62819f0d0 con 0x55aa53293960 2023-09-30T14:38:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb62c855700 1 --2- 172.21.15.32:0/694668208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa532942c0 0x55aa5329d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb62d056700 1 --2- 172.21.15.32:0/694668208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa53293960 0x55aa5329d3f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb61800b580 tx=0x7fb61809ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de56e713bd617c56 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb610ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb610ff9700 1 -- 172.21.15.32:0/694668208 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61809ac90 con 0x55aa53293960 2023-09-30T14:38:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb610ff9700 1 -- 172.21.15.32:0/694668208 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb61809adf0 con 0x55aa53293960 2023-09-30T14:38:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/694668208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa53282200 con 0x55aa53293960 2023-09-30T14:38:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb610ff9700 1 -- 172.21.15.32:0/694668208 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6180a28a0 con 0x55aa53293960 2023-09-30T14:38:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb63b4a2b80 10 client.15237 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.445+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/694668208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55aa5300cd90 con 0x55aa53293960 2023-09-30T14:38:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.446+0000 7fb610ff9700 1 -- 172.21.15.32:0/694668208 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb61807f080 con 0x55aa53293960 2023-09-30T14:38:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.446+0000 7fb610ff9700 1 -- 172.21.15.32:0/694668208 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb618096940 con 0x55aa53293960 2023-09-30T14:38:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.446+0000 7fb63b4a2b80 10 client.15237 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:31.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.446+0000 7fb63b4a2b80 10 client.15237 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-30T14:38:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.446+0000 7fb63b4a2b80 20 client.15237 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.446+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/694668208 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55aa532a8ac0 0x55aa532a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.446+0000 7fb63b4a2b80 4 client.15237 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:38:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.446+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/694668208 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa531dadc0 con 0x55aa532a8ac0 2023-09-30T14:38:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.446+0000 7fb62c855700 1 --2- 172.21.15.32:0/694668208 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55aa532a8ac0 0x55aa532a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.447+0000 7fb62c855700 1 --2- 172.21.15.32:0/694668208 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55aa532a8ac0 0x55aa532a78a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e57aa89e19c15532 server_cookie=33fec79167c5dd31 in_seq=0 out_seq=0 2023-09-30T14:38:31.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.447+0000 7fb610ff9700 10 client.15237 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.447+0000 7fb610ff9700 1 -- 172.21.15.32:0/694668208 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aa531dadc0 con 0x55aa532a8ac0 2023-09-30T14:38:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.447+0000 7fb610ff9700 10 client.15237 handle_command_reply: tid=0 2023-09-30T14:38:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.447+0000 7fb63b4a2b80 1 client.15237 shutdown 2023-09-30T14:38:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.448+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/694668208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa53293960 msgr2=0x55aa5329d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.448+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/694668208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa53293960 0x55aa5329d3f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb61800b580 tx=0x7fb61809ab30 comp rx=0 tx=0).stop 2023-09-30T14:38:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.448+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/694668208 shutdown_connections 2023-09-30T14:38:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.448+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/694668208 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55aa532a8ac0 0x55aa532a78a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.448+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/694668208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa53295160 0x55aa532a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.448+0000 7fb610ff9700 0 client.15237 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.448+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/694668208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa53293960 0x55aa5329d3f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.448+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/694668208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa532942c0 0x55aa5329d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.448+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/694668208 >> 172.21.15.32:0/694668208 conn(0x55aa53203210 msgr2=0x55aa5328a240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.448+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/694668208 shutdown_connections 2023-09-30T14:38:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.448+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/694668208 wait complete. 2023-09-30T14:38:31.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.448+0000 7fb63b4a2b80 20 client.15237 trim_cache size 0 max 16384 2023-09-30T14:38:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.466+0000 7fb63b4a2b80 1 Processor -- start 2023-09-30T14:38:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.466+0000 7fb63b4a2b80 1 -- start start 2023-09-30T14:38:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.466+0000 7fb63b4a2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa53293960 0x55aa5329b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.466+0000 7fb63b4a2b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa532942c0 0x55aa5329ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.466+0000 7fb63b4a2b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa53295160 0x55aa532a4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.466+0000 7fb63b4a2b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa530628d0 con 0x55aa532942c0 2023-09-30T14:38:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.466+0000 7fb63b4a2b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa531dadf0 con 0x55aa53295160 2023-09-30T14:38:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.466+0000 7fb63b4a2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa531dded0 con 0x55aa53293960 2023-09-30T14:38:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.466+0000 7fb62c855700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa532942c0 0x55aa5329ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.466+0000 7fb62c855700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa532942c0 0x55aa5329ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35050/0 (socket says 172.21.15.32:35050) 2023-09-30T14:38:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.466+0000 7fb62c855700 1 -- 172.21.15.32:0/792165998 learned_addr learned my addr 172.21.15.32:0/792165998 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.467+0000 7fb62d056700 1 --2- 172.21.15.32:0/792165998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa53293960 0x55aa5329b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.467+0000 7fb62d857700 1 --2- 172.21.15.32:0/792165998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa53295160 0x55aa532a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.467+0000 7fb62c855700 1 -- 172.21.15.32:0/792165998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa53293960 msgr2=0x55aa5329b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.467+0000 7fb62c855700 1 --2- 172.21.15.32:0/792165998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa53293960 0x55aa5329b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.467+0000 7fb62c855700 1 -- 172.21.15.32:0/792165998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa53295160 msgr2=0x55aa532a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.467+0000 7fb62c855700 1 --2- 172.21.15.32:0/792165998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa53295160 0x55aa532a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.467+0000 7fb62c855700 1 -- 172.21.15.32:0/792165998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa52ff5c10 con 0x55aa532942c0 2023-09-30T14:38:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.467+0000 7fb62c855700 1 --2- 172.21.15.32:0/792165998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa532942c0 0x55aa5329ba60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55aa5329df80 tx=0x7fb624014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8383199719fffd17 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.467+0000 7fb613fff700 1 -- 172.21.15.32:0/792165998 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624023040 con 0x55aa532942c0 2023-09-30T14:38:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.467+0000 7fb613fff700 1 -- 172.21.15.32:0/792165998 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb624028dc0 con 0x55aa532942c0 2023-09-30T14:38:31.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.467+0000 7fb613fff700 1 -- 172.21.15.32:0/792165998 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62401d670 con 0x55aa532942c0 2023-09-30T14:38:31.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/792165998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa532942c0 msgr2=0x55aa5329ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/792165998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa532942c0 0x55aa5329ba60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55aa5329df80 tx=0x7fb624014000 comp rx=0 tx=0).stop 2023-09-30T14:38:31.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/792165998 shutdown_connections 2023-09-30T14:38:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/792165998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa53295160 0x55aa532a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/792165998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa53293960 0x55aa5329b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/792165998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa532942c0 0x55aa5329ba60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/792165998 >> 172.21.15.32:0/792165998 conn(0x55aa53203210 msgr2=0x55aa5329c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/792165998 shutdown_connections 2023-09-30T14:38:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/792165998 wait complete. 2023-09-30T14:38:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 -- start start 2023-09-30T14:38:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 Processor -- start 2023-09-30T14:38:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa53293960 0x55aa5329f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa532942c0 0x55aa532a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa53295160 0x55aa5329fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa532fc4b0 con 0x55aa532942c0 2023-09-30T14:38:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa532fc5f0 con 0x55aa53293960 2023-09-30T14:38:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.468+0000 7fb63b4a2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa532fc730 con 0x55aa53295160 2023-09-30T14:38:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb63b4a2b80 10 client.0 _ll_register_callbacks cb 0x7fb61809a3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb62c855700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa532942c0 0x55aa532a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb62c855700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa532942c0 0x55aa532a49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35056/0 (socket says 172.21.15.32:35056) 2023-09-30T14:38:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb62c855700 1 -- 172.21.15.32:0/800396012 learned_addr learned my addr 172.21.15.32:0/800396012 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb62d857700 1 --2- 172.21.15.32:0/800396012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa53295160 0x55aa5329fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb62d056700 1 --2- 172.21.15.32:0/800396012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa53293960 0x55aa5329f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb62c855700 1 -- 172.21.15.32:0/800396012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa53295160 msgr2=0x55aa5329fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb62c855700 1 --2- 172.21.15.32:0/800396012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa53295160 0x55aa5329fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb62c855700 1 -- 172.21.15.32:0/800396012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa53293960 msgr2=0x55aa5329f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:31.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb62c855700 1 --2- 172.21.15.32:0/800396012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa53293960 0x55aa5329f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb62c855700 1 -- 172.21.15.32:0/800396012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb624017070 con 0x55aa532942c0 2023-09-30T14:38:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb62d056700 1 --2- 172.21.15.32:0/800396012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa53293960 0x55aa5329f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb62c855700 1 --2- 172.21.15.32:0/800396012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa532942c0 0x55aa532a49d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb624003a20 tx=0x7fb6240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f75bdcb982a0bf5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb610ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb610ff9700 1 -- 172.21.15.32:0/800396012 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624023050 con 0x55aa532942c0 2023-09-30T14:38:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb610ff9700 1 -- 172.21.15.32:0/800396012 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb624028460 con 0x55aa532942c0 2023-09-30T14:38:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/800396012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6180cb240 con 0x55aa532942c0 2023-09-30T14:38:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb610ff9700 1 -- 172.21.15.32:0/800396012 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb624028900 con 0x55aa532942c0 2023-09-30T14:38:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb63b4a2b80 10 client.15243 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.469+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/800396012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55aa53033200 con 0x55aa532942c0 2023-09-30T14:38:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.470+0000 7fb610ff9700 1 -- 172.21.15.32:0/800396012 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6180cb240 con 0x55aa532942c0 2023-09-30T14:38:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.470+0000 7fb610ff9700 1 -- 172.21.15.32:0/800396012 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb624031c40 con 0x55aa532942c0 2023-09-30T14:38:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.470+0000 7fb63b4a2b80 10 client.15243 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:31.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.470+0000 7fb63b4a2b80 10 client.15243 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-30T14:38:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.470+0000 7fb63b4a2b80 20 client.15243 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.470+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/800396012 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55aa53301830 0x55aa532fde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.470+0000 7fb63b4a2b80 4 client.15243 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2023-09-30T14:38:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.470+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/800396012 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55aa531daae0 con 0x55aa53301830 2023-09-30T14:38:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.470+0000 7fb62d056700 1 --2- 172.21.15.32:0/800396012 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55aa53301830 0x55aa532fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.470+0000 7fb610ff9700 10 client.15243 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.470+0000 7fb62d056700 1 --2- 172.21.15.32:0/800396012 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55aa53301830 0x55aa532fde10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95cde9fc2c443db2 server_cookie=c210cb4e30e70619 in_seq=0 out_seq=0 2023-09-30T14:38:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb610ff9700 1 -- 172.21.15.32:0/800396012 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55aa531daae0 con 0x55aa53301830 2023-09-30T14:38:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb610ff9700 10 client.15243 handle_command_reply: tid=0 2023-09-30T14:38:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb63b4a2b80 1 client.15243 shutdown 2023-09-30T14:38:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/800396012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa532942c0 msgr2=0x55aa532a49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/800396012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa532942c0 0x55aa532a49d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb624003a20 tx=0x7fb6240057f0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/800396012 shutdown_connections 2023-09-30T14:38:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/800396012 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55aa53301830 0x55aa532fde10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/800396012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa53295160 0x55aa5329fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/800396012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aa53293960 0x55aa5329f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb63b4a2b80 1 --2- 172.21.15.32:0/800396012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aa532942c0 0x55aa532a49d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb610ff9700 0 client.15243 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/800396012 >> 172.21.15.32:0/800396012 conn(0x55aa53203210 msgr2=0x55aa5329b520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/800396012 shutdown_connections 2023-09-30T14:38:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb63b4a2b80 1 -- 172.21.15.32:0/800396012 wait complete. 2023-09-30T14:38:31.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb63b4a2b80 20 client.15243 trim_cache size 0 max 16384 2023-09-30T14:38:31.478 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T14:38:31.479 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "bca3f5c6-5f9d-11ee-8db5-212e2dc638e7", 2023-09-30T14:38:31.479 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-09-30T14:38:31.479 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24515, 2023-09-30T14:38:31.479 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-09-30T14:38:31.479 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-09-30T14:38:31.479 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-09-30T14:38:31.479 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 79.438427086999994, 2023-09-30T14:38:31.479 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 22, 2023-09-30T14:38:31.479 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 85, 2023-09-30T14:38:31.479 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 83, 2023-09-30T14:38:31.479 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 85.646777882999999 2023-09-30T14:38:31.479 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T14:38:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb62f2ba700 1 -- 172.21.15.32:0/1633541254 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb6140a10b0 msgr2=0x7fb6140a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb62f2ba700 1 --2- 172.21.15.32:0/1633541254 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb6140a10b0 0x7fb6140a3570 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb6280688b0 tx=0x7fb624009260 comp rx=0 tx=0).stop 2023-09-30T14:38:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb62f2ba700 1 -- 172.21.15.32:0/1633541254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6280ffd30 msgr2=0x7fb628068b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:31.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.471+0000 7fb62f2ba700 1 --2- 172.21.15.32:0/1633541254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6280ffd30 0x7fb628068b40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb618009fd0 tx=0x7fb618009580 comp rx=0 tx=0).stop 2023-09-30T14:38:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.472+0000 7fb62f2ba700 1 -- 172.21.15.32:0/1633541254 shutdown_connections 2023-09-30T14:38:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.472+0000 7fb62f2ba700 1 --2- 172.21.15.32:0/1633541254 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb6140a10b0 0x7fb6140a3570 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.472+0000 7fb62f2ba700 1 --2- 172.21.15.32:0/1633541254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6281018c0 0x7fb62819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.472+0000 7fb62f2ba700 1 --2- 172.21.15.32:0/1633541254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6280ffd30 0x7fb628068b40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.472+0000 7fb62f2ba700 1 --2- 172.21.15.32:0/1633541254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb628100f20 0x7fb62819b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.472+0000 7fb62f2ba700 1 -- 172.21.15.32:0/1633541254 >> 172.21.15.32:0/1633541254 conn(0x7fb6280fb2a0 msgr2=0x7fb628104450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.472+0000 7fb62f2ba700 1 -- 172.21.15.32:0/1633541254 shutdown_connections 2023-09-30T14:38:31.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:31.472+0000 7fb62f2ba700 1 -- 172.21.15.32:0/1633541254 wait complete. 2023-09-30T14:38:31.484 INFO:teuthology.run_tasks:Running task kclient... 2023-09-30T14:38:31.506 INFO:tasks.kclient:Mounting kernel clients... 2023-09-30T14:38:31.506 INFO:tasks.kclient:config is {'client.0': {}} 2023-09-30T14:38:31.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:31.507 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-09-30T14:38:31.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:31.524 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-09-30T14:38:31.587 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:31 smithi032 ceph-mon[98592]: pgmap v294: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:31.587 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:31 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3857610899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:31.587 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:31 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1147811873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:31.592 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2023-09-30T14:38:31.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T14:38:31.594 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-09-30T14:38:31.594 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-30T14:38:31.595 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-30T14:38:31.595 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph fs ls 2023-09-30T14:38:31.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:31 smithi156 ceph-mon[103579]: pgmap v294: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:31.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:31 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3857610899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:31.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:31 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1147811873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:31.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:31 smithi148 ceph-mon[102355]: pgmap v294: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:31.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:31 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3857610899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:31.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:31 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1147811873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:32.195 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T14:38:33.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.384+0000 7fa5d4571700 1 -- 172.21.15.32:0/176563157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc1010c0 msgr2=0x7fa5cc104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:33.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.384+0000 7fa5d4571700 1 --2- 172.21.15.32:0/176563157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc1010c0 0x7fa5cc104bf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009990 tx=0x7fa5c8009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:33.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.384+0000 7fa5d4571700 1 -- 172.21.15.32:0/176563157 shutdown_connections 2023-09-30T14:38:33.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.384+0000 7fa5d4571700 1 --2- 172.21.15.32:0/176563157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5cc1052d0 0x7fa5cc10cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:33.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.384+0000 7fa5d4571700 1 --2- 172.21.15.32:0/176563157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc100710 0x7fa5cc100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:33.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.384+0000 7fa5d4571700 1 --2- 172.21.15.32:0/176563157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc1010c0 0x7fa5cc104bf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:33.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.384+0000 7fa5d4571700 1 -- 172.21.15.32:0/176563157 >> 172.21.15.32:0/176563157 conn(0x7fa5cc0762d0 msgr2=0x7fa5cc0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:33.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.384+0000 7fa5d4571700 1 -- 172.21.15.32:0/176563157 shutdown_connections 2023-09-30T14:38:33.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.384+0000 7fa5d4571700 1 -- 172.21.15.32:0/176563157 wait complete. 2023-09-30T14:38:33.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.385+0000 7fa5d4571700 1 Processor -- start 2023-09-30T14:38:33.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.385+0000 7fa5d4571700 1 -- start start 2023-09-30T14:38:33.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.385+0000 7fa5d4571700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5cc100710 0x7fa5cc196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:33.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.385+0000 7fa5d4571700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc1010c0 0x7fa5cc196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:33.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d4571700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc1052d0 0x7fa5cc19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:33.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d4571700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc19b010 con 0x7fa5cc1010c0 2023-09-30T14:38:33.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d4571700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc19b180 con 0x7fa5cc1052d0 2023-09-30T14:38:33.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d4571700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc19b340 con 0x7fa5cc100710 2023-09-30T14:38:33.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d1b0c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc1010c0 0x7fa5cc196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:33.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d1b0c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc1010c0 0x7fa5cc196590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35070/0 (socket says 172.21.15.32:35070) 2023-09-30T14:38:33.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d1b0c700 1 -- 172.21.15.32:0/665479522 learned_addr learned my addr 172.21.15.32:0/665479522 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:33.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d230d700 1 --2- 172.21.15.32:0/665479522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5cc100710 0x7fa5cc196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:33.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d2b0e700 1 --2- 172.21.15.32:0/665479522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc1052d0 0x7fa5cc19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:33.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d1b0c700 1 -- 172.21.15.32:0/665479522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5cc100710 msgr2=0x7fa5cc196050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:33.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d1b0c700 1 --2- 172.21.15.32:0/665479522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5cc100710 0x7fa5cc196050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:33.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d1b0c700 1 -- 172.21.15.32:0/665479522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc1052d0 msgr2=0x7fa5cc19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:33.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d1b0c700 1 --2- 172.21.15.32:0/665479522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc1052d0 0x7fa5cc19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:33.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d1b0c700 1 -- 172.21.15.32:0/665479522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c80095f0 con 0x7fa5cc1010c0 2023-09-30T14:38:33.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d1b0c700 1 --2- 172.21.15.32:0/665479522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc1010c0 0x7fa5cc196590 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8003730 tx=0x7fa5c801a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:33.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5c37fe700 1 -- 172.21.15.32:0/665479522 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c801f070 con 0x7fa5cc1010c0 2023-09-30T14:38:33.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5c37fe700 1 -- 172.21.15.32:0/665479522 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5c800f460 con 0x7fa5cc1010c0 2023-09-30T14:38:33.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d4571700 1 -- 172.21.15.32:0/665479522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5cc19b5c0 con 0x7fa5cc1010c0 2023-09-30T14:38:33.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5c37fe700 1 -- 172.21.15.32:0/665479522 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8025680 con 0x7fa5cc1010c0 2023-09-30T14:38:33.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.386+0000 7fa5d4571700 1 -- 172.21.15.32:0/665479522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5cc0ff570 con 0x7fa5cc1010c0 2023-09-30T14:38:33.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.387+0000 7fa5d4571700 1 -- 172.21.15.32:0/665479522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5cc04eb20 con 0x7fa5cc1010c0 2023-09-30T14:38:33.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.388+0000 7fa5c37fe700 1 -- 172.21.15.32:0/665479522 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fa5c8023070 con 0x7fa5cc1010c0 2023-09-30T14:38:33.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.389+0000 7fa5c37fe700 1 --2- 172.21.15.32:0/665479522 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa5b80a1030 0x7fa5b80a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:33.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.389+0000 7fa5d230d700 1 --2- 172.21.15.32:0/665479522 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa5b80a1030 0x7fa5b80a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:33.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.389+0000 7fa5c37fe700 1 -- 172.21.15.32:0/665479522 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa5c80cfa60 con 0x7fa5cc1010c0 2023-09-30T14:38:33.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.390+0000 7fa5d230d700 1 --2- 172.21.15.32:0/665479522 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa5b80a1030 0x7fa5b80a34f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc005950 tx=0x7fa5bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.394+0000 7fa5c37fe700 1 -- 172.21.15.32:0/665479522 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fa5c80992c0 con 0x7fa5cc1010c0 2023-09-30T14:38:33.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.637+0000 7fa5d4571700 1 -- 172.21.15.32:0/665479522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa5cc1973b0 con 0x7fa5cc1010c0 2023-09-30T14:38:33.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.637+0000 7fa5c37fe700 1 -- 172.21.15.32:0/665479522 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa5c809c8e0 con 0x7fa5cc1010c0 2023-09-30T14:38:33.639 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-30T14:38:33.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.640+0000 7fa5d4571700 1 -- 172.21.15.32:0/665479522 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa5b80a1030 msgr2=0x7fa5b80a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:33.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.640+0000 7fa5d4571700 1 --2- 172.21.15.32:0/665479522 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa5b80a1030 0x7fa5b80a34f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc005950 tx=0x7fa5bc00b410 comp rx=0 tx=0).stop 2023-09-30T14:38:33.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.640+0000 7fa5d4571700 1 -- 172.21.15.32:0/665479522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc1010c0 msgr2=0x7fa5cc196590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:33.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.640+0000 7fa5d4571700 1 --2- 172.21.15.32:0/665479522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc1010c0 0x7fa5cc196590 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8003730 tx=0x7fa5c801a840 comp rx=0 tx=0).stop 2023-09-30T14:38:33.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.640+0000 7fa5d4571700 1 -- 172.21.15.32:0/665479522 shutdown_connections 2023-09-30T14:38:33.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.640+0000 7fa5d4571700 1 --2- 172.21.15.32:0/665479522 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa5b80a1030 0x7fa5b80a34f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:33.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.640+0000 7fa5d4571700 1 --2- 172.21.15.32:0/665479522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc1052d0 0x7fa5cc19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:33.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.640+0000 7fa5d4571700 1 --2- 172.21.15.32:0/665479522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5cc100710 0x7fa5cc196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:33.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.640+0000 7fa5d4571700 1 --2- 172.21.15.32:0/665479522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5cc1010c0 0x7fa5cc196590 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:33.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.640+0000 7fa5d4571700 1 -- 172.21.15.32:0/665479522 >> 172.21.15.32:0/665479522 conn(0x7fa5cc0762d0 msgr2=0x7fa5cc103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:33.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.640+0000 7fa5d4571700 1 -- 172.21.15.32:0/665479522 shutdown_connections 2023-09-30T14:38:33.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:33.640+0000 7fa5d4571700 1 -- 172.21.15.32:0/665479522 wait complete. 2023-09-30T14:38:33.787 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:33 smithi032 ceph-mon[98592]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:38:33.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:33 smithi156 ceph-mon[103579]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:38:33.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:33 smithi148 ceph-mon[102355]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:38:34.045 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-30T14:38:34.045 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-30T14:38:34.045 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-09-30T14:38:34.045 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-30T14:38:34.045 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-30T14:38:34.045 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-30T14:38:34.045 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-30T14:38:34.045 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-30T14:38:34.045 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-30T14:38:34.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:34.046 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-09-30T14:38:34.064 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-30T14:38:34.064 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-30T14:38:34.064 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-09-30T14:38:34.064 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-09-30T14:38:34.064 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-09-30T14:38:34.064 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-09-30T14:38:34.065 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-30T14:38:34.065 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-09-30T14:38:34.065 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-30T14:38:34.065 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-09-30T14:38:34.065 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-30T14:38:34.065 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-09-30T14:38:34.065 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-09-30T14:38:34.065 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41837sec preferred_lft 41837sec 2023-09-30T14:38:34.065 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-09-30T14:38:34.065 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-09-30T14:38:34.065 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-30T14:38:34.066 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-09-30T14:38:34.066 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-30T14:38:34.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:34.066 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-09-30T14:38:34.066 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-09-30T14:38:34.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2023-09-30T14:38:34.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2023-09-30T14:38:34.067 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2023-09-30T14:38:34.067 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-30T14:38:34.067 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-09-30T14:38:34.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:34.228 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-30T14:38:34.257 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-09-30T14:38:34.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:34.259 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-09-30T14:38:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-09-30T14:38:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-30T14:38:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-09-30T14:38:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-09-30T14:38:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-30T14:38:34.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:34.333 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-09-30T14:38:34.333 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-09-30T14:38:34.334 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-09-30T14:38:34.334 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-09-30T14:38:34.334 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-09-30T14:38:34.334 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-09-30T14:38:34.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:34.599 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-09-30T14:38:34.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:34.617 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2023-09-30T14:38:34.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:34.675 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-09-30T14:38:34.675 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-09-30T14:38:34.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T14:38:34.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-09-30T14:38:34.675 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-09-30T14:38:34.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:34 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/665479522' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-30T14:38:34.771 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-09-30T14:38:34.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:34.771 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-09-30T14:38:34.772 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-09-30T14:38:34.772 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-09-30T14:38:34.772 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-30T14:38:34.772 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-09-30T14:38:34.772 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-09-30T14:38:34.772 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-09-30T14:38:34.772 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-09-30T14:38:34.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:34 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/665479522' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-30T14:38:34.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:34 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/665479522' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-30T14:38:35.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:35.007 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-09-30T14:38:35.007 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-09-30T14:38:35.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-09-30T14:38:35.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2023-09-30T14:38:35.008 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-09-30T14:38:35.107 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-30T14:38:35.107 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-30T14:38:35.124 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-30T14:38:35.124 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-30T14:38:35.124 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-30T14:38:35.181 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-09-30T14:38:35.182 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-09-30T14:38:35.182 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/b25fcf16-a734-410f-849e-f8fb3a417cdf 2023-09-30T14:38:35.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:35.182 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/b25fcf16-a734-410f-849e-f8fb3a417cdf /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2023-09-30T14:38:35.289 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2023-09-30T14:38:35.290 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2023-09-30T14:38:35.408 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2023-09-30T14:38:35.408 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2023-09-30T14:38:35.408 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.148:3300,172.21.15.156:3300" 2023-09-30T14:38:35.408 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@bca3f5c6-5f9d-11ee-8db5-212e2dc638e7.cephfs=/volumes/_nogroup/sv_1/b25fcf16-a734-410f-849e-f8fb3a417cdf 2023-09-30T14:38:35.408 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.32:3300/172.21.15.148:3300/172.21.15.156:3300" will pass to kernel 2023-09-30T14:38:35.408 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: v2 mount support check returned 2 2023-09-30T14:38:35.409 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: kernel does not support v2 syntax 2023-09-30T14:38:35.409 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:3300,172.21.15.148:3300,172.21.15.156:3300:/volumes/_nogroup/sv_1/b25fcf16-a734-410f-849e-f8fb3a417cdf 2023-09-30T14:38:35.409 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mds_namespace=cephfs,fsid=bca3f5c6-5f9d-11ee-8db5-212e2dc638e7" will pass to kernel 2023-09-30T14:38:35.417 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-09-30T14:38:35.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:35.418 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-30T14:38:35.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:35.476 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-30T14:38:35.529 INFO:teuthology.orchestra.run.smithi032.stdout:client25001 2023-09-30T14:38:35.530 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:38:35.530 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-30T14:38:35.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:35.548 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-30T14:38:35.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:35 smithi032 ceph-mon[98592]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:35.640 INFO:teuthology.orchestra.run.smithi032.stdout:client25001 2023-09-30T14:38:35.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:35.640 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-30T14:38:35.691 INFO:teuthology.orchestra.run.smithi032.stdout:client25001 2023-09-30T14:38:35.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T14:38:35.692 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7.client25001/status') 2023-09-30T14:38:35.750 INFO:teuthology.orchestra.run.smithi032.stdout:instance: client.25001 (3)172.21.15.32:0/2409784599 2023-09-30T14:38:35.750 INFO:teuthology.orchestra.run.smithi032.stdout:blocklisted: false 2023-09-30T14:38:35.750 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2023-09-30T14:38:35.750 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2023-09-30T14:38:35.751 INFO:teuthology.orchestra.run.smithi032.stderr:71 bytes copied, 2.4964e-05 s, 2.8 MB/s 2023-09-30T14:38:35.752 INFO:teuthology.run_tasks:Running task check-counter... 2023-09-30T14:38:35.762 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-09-30T14:38:35.773 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T14:38:35.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:35 smithi156 ceph-mon[103579]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:35.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:35 smithi148 ceph-mon[102355]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-09-30T14:38:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.899+0000 7f5fc9093700 1 -- 172.21.15.32:0/3381675298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc4101910 msgr2=0x7f5fc4103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.899+0000 7f5fc9093700 1 --2- 172.21.15.32:0/3381675298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc4101910 0x7f5fc4103d00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40098f0 tx=0x7f5fb4009c00 comp rx=0 tx=0).stop 2023-09-30T14:38:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.899+0000 7f5fc9093700 1 -- 172.21.15.32:0/3381675298 shutdown_connections 2023-09-30T14:38:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.899+0000 7f5fc9093700 1 --2- 172.21.15.32:0/3381675298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fc4106b70 0x7f5fc4108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.899+0000 7f5fc9093700 1 --2- 172.21.15.32:0/3381675298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc4101910 0x7f5fc4103d00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.899+0000 7f5fc9093700 1 --2- 172.21.15.32:0/3381675298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fc4104240 0x7f5fc4106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.899+0000 7f5fc9093700 1 -- 172.21.15.32:0/3381675298 >> 172.21.15.32:0/3381675298 conn(0x7f5fc40fb2e0 msgr2=0x7f5fc40fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.899+0000 7f5fc9093700 1 -- 172.21.15.32:0/3381675298 shutdown_connections 2023-09-30T14:38:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.899+0000 7f5fc9093700 1 -- 172.21.15.32:0/3381675298 wait complete. 2023-09-30T14:38:35.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc9093700 1 Processor -- start 2023-09-30T14:38:35.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc9093700 1 -- start start 2023-09-30T14:38:35.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc9093700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fc4101910 0x7f5fc4198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:35.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc9093700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc4104240 0x7f5fc4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:35.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc9093700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fc4106b70 0x7f5fc41a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:35.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc9093700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc41a0840 con 0x7f5fc4104240 2023-09-30T14:38:35.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc9093700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc41a09b0 con 0x7f5fc4101910 2023-09-30T14:38:35.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc9093700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc41a0b70 con 0x7f5fc4106b70 2023-09-30T14:38:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc4104240 0x7f5fc4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc4104240 0x7f5fc4199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35088/0 (socket says 172.21.15.32:35088) 2023-09-30T14:38:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc259c700 1 -- 172.21.15.32:0/1653981534 learned_addr learned my addr 172.21.15.32:0/1653981534 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc2d9d700 1 --2- 172.21.15.32:0/1653981534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fc4101910 0x7f5fc4198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc359e700 1 --2- 172.21.15.32:0/1653981534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fc4106b70 0x7f5fc41a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc259c700 1 -- 172.21.15.32:0/1653981534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fc4106b70 msgr2=0x7f5fc41a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc259c700 1 --2- 172.21.15.32:0/1653981534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fc4106b70 0x7f5fc41a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc259c700 1 -- 172.21.15.32:0/1653981534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fc4101910 msgr2=0x7f5fc4198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc259c700 1 --2- 172.21.15.32:0/1653981534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fc4101910 0x7f5fc4198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc259c700 1 -- 172.21.15.32:0/1653981534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb8009520 con 0x7f5fc4104240 2023-09-30T14:38:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc359e700 1 --2- 172.21.15.32:0/1653981534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fc4106b70 0x7f5fc41a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.900+0000 7f5fc2d9d700 1 --2- 172.21.15.32:0/1653981534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fc4101910 0x7f5fc4198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:35.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.901+0000 7f5fc259c700 1 --2- 172.21.15.32:0/1653981534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc4104240 0x7f5fc4199240 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800eb30 tx=0x7f5fb800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:35.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.901+0000 7f5fabfff700 1 -- 172.21.15.32:0/1653981534 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8009880 con 0x7f5fc4104240 2023-09-30T14:38:35.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.901+0000 7f5fabfff700 1 -- 172.21.15.32:0/1653981534 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fb80044e0 con 0x7f5fc4104240 2023-09-30T14:38:35.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.901+0000 7f5fabfff700 1 -- 172.21.15.32:0/1653981534 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8010430 con 0x7f5fc4104240 2023-09-30T14:38:35.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.901+0000 7f5fc9093700 1 -- 172.21.15.32:0/1653981534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb4009520 con 0x7f5fc4104240 2023-09-30T14:38:35.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.901+0000 7f5fc9093700 1 -- 172.21.15.32:0/1653981534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc41a1180 con 0x7f5fc4104240 2023-09-30T14:38:35.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.902+0000 7f5fc9093700 1 -- 172.21.15.32:0/1653981534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc4108740 con 0x7f5fc4104240 2023-09-30T14:38:35.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.903+0000 7f5fabfff700 1 -- 172.21.15.32:0/1653981534 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f5fb8003ee0 con 0x7f5fc4104240 2023-09-30T14:38:35.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.904+0000 7f5fabfff700 1 --2- 172.21.15.32:0/1653981534 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5fac0a1070 0x7f5fac0a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:35.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.904+0000 7f5fabfff700 1 -- 172.21.15.32:0/1653981534 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5fb8014070 con 0x7f5fc4104240 2023-09-30T14:38:35.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.904+0000 7f5fc2d9d700 1 --2- 172.21.15.32:0/1653981534 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5fac0a1070 0x7f5fac0a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:35.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.904+0000 7f5fc2d9d700 1 --2- 172.21.15.32:0/1653981534 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5fac0a1070 0x7f5fac0a3530 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40098c0 tx=0x7f5fb4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:35.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:35.907+0000 7f5fabfff700 1 -- 172.21.15.32:0/1653981534 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f5fb80965c0 con 0x7f5fc4104240 2023-09-30T14:38:36.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.112+0000 7f5fc9093700 1 -- 172.21.15.32:0/1653981534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fc410c670 con 0x7f5fc4104240 2023-09-30T14:38:36.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.113+0000 7f5fabfff700 1 -- 172.21.15.32:0/1653981534 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22569 (secure 0 0 0) 0x7f5fb8004190 con 0x7f5fc4104240 2023-09-30T14:38:36.114 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:36.115 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":85,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:38:13.192089+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:38:36.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.115+0000 7f5fc9093700 1 -- 172.21.15.32:0/1653981534 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5fac0a1070 msgr2=0x7f5fac0a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:36.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.115+0000 7f5fc9093700 1 --2- 172.21.15.32:0/1653981534 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5fac0a1070 0x7f5fac0a3530 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40098c0 tx=0x7f5fb4019040 comp rx=0 tx=0).stop 2023-09-30T14:38:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.115+0000 7f5fc9093700 1 -- 172.21.15.32:0/1653981534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc4104240 msgr2=0x7f5fc4199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.115+0000 7f5fc9093700 1 --2- 172.21.15.32:0/1653981534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc4104240 0x7f5fc4199240 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800eb30 tx=0x7f5fb800eef0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.116+0000 7f5fc9093700 1 -- 172.21.15.32:0/1653981534 shutdown_connections 2023-09-30T14:38:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.116+0000 7f5fc9093700 1 --2- 172.21.15.32:0/1653981534 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5fac0a1070 0x7f5fac0a3530 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.116+0000 7f5fc9093700 1 --2- 172.21.15.32:0/1653981534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fc4106b70 0x7f5fc41a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.116+0000 7f5fc9093700 1 --2- 172.21.15.32:0/1653981534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fc4101910 0x7f5fc4198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.116+0000 7f5fc9093700 1 --2- 172.21.15.32:0/1653981534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fc4104240 0x7f5fc4199240 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.116+0000 7f5fc9093700 1 -- 172.21.15.32:0/1653981534 >> 172.21.15.32:0/1653981534 conn(0x7f5fc40fb2e0 msgr2=0x7f5fc40fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.116+0000 7f5fc9093700 1 -- 172.21.15.32:0/1653981534 shutdown_connections 2023-09-30T14:38:36.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.116+0000 7f5fc9093700 1 -- 172.21.15.32:0/1653981534 wait complete. 2023-09-30T14:38:36.129 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T14:33:20.224529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T14:37:09.573219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T14:37:10.844458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T14:38:36.130 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T14:38:36.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.258+0000 7fa99d615700 1 -- 172.21.15.32:0/760812015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa998101900 msgr2=0x7fa9981081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:36.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.258+0000 7fa99d615700 1 --2- 172.21.15.32:0/760812015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa998101900 0x7fa9981081b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa99000b0d0 tx=0x7fa99000b3e0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.258+0000 7fa99d615700 1 -- 172.21.15.32:0/760812015 shutdown_connections 2023-09-30T14:38:36.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.258+0000 7fa99d615700 1 --2- 172.21.15.32:0/760812015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa998101900 0x7fa9981081b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.258+0000 7fa99d615700 1 --2- 172.21.15.32:0/760812015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa9980ffd70 0x7fa998100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.258+0000 7fa99d615700 1 --2- 172.21.15.32:0/760812015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa998100f60 0x7fa9981013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.258+0000 7fa99d615700 1 -- 172.21.15.32:0/760812015 >> 172.21.15.32:0/760812015 conn(0x7fa9980fb2e0 msgr2=0x7fa9980fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:36.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.258+0000 7fa99d615700 1 -- 172.21.15.32:0/760812015 shutdown_connections 2023-09-30T14:38:36.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.258+0000 7fa99d615700 1 -- 172.21.15.32:0/760812015 wait complete. 2023-09-30T14:38:36.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.259+0000 7fa99d615700 1 Processor -- start 2023-09-30T14:38:36.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.259+0000 7fa99d615700 1 -- start start 2023-09-30T14:38:36.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.259+0000 7fa99d615700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9980ffd70 0x7fa998198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:36.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.259+0000 7fa99d615700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa998100f60 0x7fa998199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:36.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.259+0000 7fa99d615700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa998101900 0x7fa9981a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:36.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.259+0000 7fa99d615700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981a0980 con 0x7fa998101900 2023-09-30T14:38:36.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.259+0000 7fa99d615700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981a0ac0 con 0x7fa9980ffd70 2023-09-30T14:38:36.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.259+0000 7fa99d615700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981a0c50 con 0x7fa998100f60 2023-09-30T14:38:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.259+0000 7fa9977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa998101900 0x7fa9981a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.259+0000 7fa9977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa998101900 0x7fa9981a03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35104/0 (socket says 172.21.15.32:35104) 2023-09-30T14:38:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.259+0000 7fa9977fe700 1 -- 172.21.15.32:0/837191398 learned_addr learned my addr 172.21.15.32:0/837191398 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.259+0000 7fa996ffd700 1 --2- 172.21.15.32:0/837191398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9980ffd70 0x7fa998198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.259+0000 7fa9967fc700 1 --2- 172.21.15.32:0/837191398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa998100f60 0x7fa998199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.260+0000 7fa9977fe700 1 -- 172.21.15.32:0/837191398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa998100f60 msgr2=0x7fa998199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.260+0000 7fa9977fe700 1 --2- 172.21.15.32:0/837191398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa998100f60 0x7fa998199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.260+0000 7fa9977fe700 1 -- 172.21.15.32:0/837191398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9980ffd70 msgr2=0x7fa998198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.260+0000 7fa9977fe700 1 --2- 172.21.15.32:0/837191398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9980ffd70 0x7fa998198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.260+0000 7fa9977fe700 1 -- 172.21.15.32:0/837191398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa990009d00 con 0x7fa998101900 2023-09-30T14:38:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.260+0000 7fa9977fe700 1 --2- 172.21.15.32:0/837191398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa998101900 0x7fa9981a03b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa990007a00 tx=0x7fa9900094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:36.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.260+0000 7fa97ffff700 1 -- 172.21.15.32:0/837191398 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99000e040 con 0x7fa998101900 2023-09-30T14:38:36.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.260+0000 7fa97ffff700 1 -- 172.21.15.32:0/837191398 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa990003e10 con 0x7fa998101900 2023-09-30T14:38:36.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.260+0000 7fa97ffff700 1 -- 172.21.15.32:0/837191398 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99001b9e0 con 0x7fa998101900 2023-09-30T14:38:36.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.260+0000 7fa99d615700 1 -- 172.21.15.32:0/837191398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9981a0ed0 con 0x7fa998101900 2023-09-30T14:38:36.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.260+0000 7fa99d615700 1 -- 172.21.15.32:0/837191398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9981a1240 con 0x7fa998101900 2023-09-30T14:38:36.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.261+0000 7fa99d615700 1 -- 172.21.15.32:0/837191398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99810b600 con 0x7fa998101900 2023-09-30T14:38:36.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.262+0000 7fa97ffff700 1 -- 172.21.15.32:0/837191398 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fa990005630 con 0x7fa998101900 2023-09-30T14:38:36.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.263+0000 7fa97ffff700 1 --2- 172.21.15.32:0/837191398 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa9800a1030 0x7fa9800a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:36.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.263+0000 7fa97ffff700 1 -- 172.21.15.32:0/837191398 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa9900cded0 con 0x7fa998101900 2023-09-30T14:38:36.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.263+0000 7fa996ffd700 1 --2- 172.21.15.32:0/837191398 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa9800a1030 0x7fa9800a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:36.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.263+0000 7fa996ffd700 1 --2- 172.21.15.32:0/837191398 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa9800a1030 0x7fa9800a34f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa998100dc0 tx=0x7fa98c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:36.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.267+0000 7fa97ffff700 1 -- 172.21.15.32:0/837191398 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fa990097610 con 0x7fa998101900 2023-09-30T14:38:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.470+0000 7fa99d615700 1 -- 172.21.15.32:0/837191398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9981a13d0 con 0x7fa998101900 2023-09-30T14:38:36.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.470+0000 7fa97ffff700 1 -- 172.21.15.32:0/837191398 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa99009ac30 con 0x7fa998101900 2023-09-30T14:38:36.471 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T14:38:36.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.472+0000 7fa99d615700 1 -- 172.21.15.32:0/837191398 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa9800a1030 msgr2=0x7fa9800a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:36.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.473+0000 7fa99d615700 1 --2- 172.21.15.32:0/837191398 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa9800a1030 0x7fa9800a34f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa998100dc0 tx=0x7fa98c00b410 comp rx=0 tx=0).stop 2023-09-30T14:38:36.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.473+0000 7fa99d615700 1 -- 172.21.15.32:0/837191398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa998101900 msgr2=0x7fa9981a03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:36.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.473+0000 7fa99d615700 1 --2- 172.21.15.32:0/837191398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa998101900 0x7fa9981a03b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa990007a00 tx=0x7fa9900094c0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.473+0000 7fa99d615700 1 -- 172.21.15.32:0/837191398 shutdown_connections 2023-09-30T14:38:36.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.473+0000 7fa99d615700 1 --2- 172.21.15.32:0/837191398 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa9800a1030 0x7fa9800a34f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.473+0000 7fa99d615700 1 --2- 172.21.15.32:0/837191398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa998101900 0x7fa9981a03b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.473+0000 7fa99d615700 1 --2- 172.21.15.32:0/837191398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9980ffd70 0x7fa998198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.473+0000 7fa99d615700 1 --2- 172.21.15.32:0/837191398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa998100f60 0x7fa998199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.473+0000 7fa99d615700 1 -- 172.21.15.32:0/837191398 >> 172.21.15.32:0/837191398 conn(0x7fa9980fb2e0 msgr2=0x7fa998069020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.473+0000 7fa99d615700 1 -- 172.21.15.32:0/837191398 shutdown_connections 2023-09-30T14:38:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.473+0000 7fa99d615700 1 -- 172.21.15.32:0/837191398 wait complete. 2023-09-30T14:38:36.486 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T14:38:36.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.614+0000 7fd0f682b700 1 -- 172.21.15.32:0/244424922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f01018b0 msgr2=0x7fd0f0103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.614+0000 7fd0f682b700 1 --2- 172.21.15.32:0/244424922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f01018b0 0x7fd0f0103ca0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009990 tx=0x7fd0e0009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.614+0000 7fd0f682b700 1 -- 172.21.15.32:0/244424922 shutdown_connections 2023-09-30T14:38:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.614+0000 7fd0f682b700 1 --2- 172.21.15.32:0/244424922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0f0106b10 0x7fd0f0108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.614+0000 7fd0f682b700 1 --2- 172.21.15.32:0/244424922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f01018b0 0x7fd0f0103ca0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.614+0000 7fd0f682b700 1 --2- 172.21.15.32:0/244424922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0f01041e0 0x7fd0f01065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.614+0000 7fd0f682b700 1 -- 172.21.15.32:0/244424922 >> 172.21.15.32:0/244424922 conn(0x7fd0f00fb2a0 msgr2=0x7fd0f00fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.615+0000 7fd0f682b700 1 -- 172.21.15.32:0/244424922 shutdown_connections 2023-09-30T14:38:36.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.615+0000 7fd0f682b700 1 -- 172.21.15.32:0/244424922 wait complete. 2023-09-30T14:38:36.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.615+0000 7fd0f682b700 1 Processor -- start 2023-09-30T14:38:36.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.615+0000 7fd0f682b700 1 -- start start 2023-09-30T14:38:36.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0f682b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0f01018b0 0x7fd0f0196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:36.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0f682b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f01041e0 0x7fd0f0197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:36.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0f682b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0f0106b10 0x7fd0f019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:36.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0f682b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f019e690 con 0x7fd0f01041e0 2023-09-30T14:38:36.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0f682b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f019e7d0 con 0x7fd0f01018b0 2023-09-30T14:38:36.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0f682b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f019e960 con 0x7fd0f0106b10 2023-09-30T14:38:36.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0ef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f01041e0 0x7fd0f0197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:36.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0ef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f01041e0 0x7fd0f0197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35116/0 (socket says 172.21.15.32:35116) 2023-09-30T14:38:36.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0ef7fe700 1 -- 172.21.15.32:0/921474977 learned_addr learned my addr 172.21.15.32:0/921474977 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:36.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0f4dc8700 1 --2- 172.21.15.32:0/921474977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0f0106b10 0x7fd0f019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:36.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0effff700 1 --2- 172.21.15.32:0/921474977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0f01018b0 0x7fd0f0196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:36.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0ef7fe700 1 -- 172.21.15.32:0/921474977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0f0106b10 msgr2=0x7fd0f019e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0ef7fe700 1 --2- 172.21.15.32:0/921474977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0f0106b10 0x7fd0f019e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0ef7fe700 1 -- 172.21.15.32:0/921474977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0f01018b0 msgr2=0x7fd0f0196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0ef7fe700 1 --2- 172.21.15.32:0/921474977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0f01018b0 0x7fd0f0196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0ef7fe700 1 -- 172.21.15.32:0/921474977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e00095f0 con 0x7fd0f01041e0 2023-09-30T14:38:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0effff700 1 --2- 172.21.15.32:0/921474977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0f01018b0 0x7fd0f0196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.616+0000 7fd0ef7fe700 1 --2- 172.21.15.32:0/921474977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f01041e0 0x7fd0f0197050 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400d980 tx=0x7fd0e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.617+0000 7fd0ed7fa700 1 -- 172.21.15.32:0/921474977 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4004220 con 0x7fd0f01041e0 2023-09-30T14:38:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.617+0000 7fd0f682b700 1 -- 172.21.15.32:0/921474977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f019ec40 con 0x7fd0f01041e0 2023-09-30T14:38:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.617+0000 7fd0ed7fa700 1 -- 172.21.15.32:0/921474977 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0e4007640 con 0x7fd0f01041e0 2023-09-30T14:38:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.617+0000 7fd0ed7fa700 1 -- 172.21.15.32:0/921474977 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4005610 con 0x7fd0f01041e0 2023-09-30T14:38:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.617+0000 7fd0f682b700 1 -- 172.21.15.32:0/921474977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f019f150 con 0x7fd0f01041e0 2023-09-30T14:38:36.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.618+0000 7fd0f682b700 1 -- 172.21.15.32:0/921474977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f00fce20 con 0x7fd0f01041e0 2023-09-30T14:38:36.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.619+0000 7fd0ed7fa700 1 -- 172.21.15.32:0/921474977 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fd0e4004750 con 0x7fd0f01041e0 2023-09-30T14:38:36.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.619+0000 7fd0ed7fa700 1 --2- 172.21.15.32:0/921474977 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd0d80a1030 0x7fd0d80a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:36.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.619+0000 7fd0effff700 1 --2- 172.21.15.32:0/921474977 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd0d80a1030 0x7fd0d80a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:36.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.619+0000 7fd0ed7fa700 1 -- 172.21.15.32:0/921474977 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd0e40cc770 con 0x7fd0f01041e0 2023-09-30T14:38:36.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.620+0000 7fd0effff700 1 --2- 172.21.15.32:0/921474977 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd0d80a1030 0x7fd0d80a34f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000b5c0 tx=0x7fd0e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:36.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.624+0000 7fd0ed7fa700 1 -- 172.21.15.32:0/921474977 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fd0e4095e20 con 0x7fd0f01041e0 2023-09-30T14:38:36.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:36 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1653981534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:36.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.828+0000 7fd0f682b700 1 -- 172.21.15.32:0/921474977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd0f010c860 con 0x7fd0f01041e0 2023-09-30T14:38:36.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.828+0000 7fd0ed7fa700 1 -- 172.21.15.32:0/921474977 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd0e4099440 con 0x7fd0f01041e0 2023-09-30T14:38:36.829 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-09-30T14:38:36.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.831+0000 7fd0f682b700 1 -- 172.21.15.32:0/921474977 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd0d80a1030 msgr2=0x7fd0d80a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:36.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.831+0000 7fd0f682b700 1 --2- 172.21.15.32:0/921474977 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd0d80a1030 0x7fd0d80a34f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000b5c0 tx=0x7fd0e0005fb0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.831+0000 7fd0f682b700 1 -- 172.21.15.32:0/921474977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f01041e0 msgr2=0x7fd0f0197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:36.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.831+0000 7fd0f682b700 1 --2- 172.21.15.32:0/921474977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f01041e0 0x7fd0f0197050 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400d980 tx=0x7fd0e400dc90 comp rx=0 tx=0).stop 2023-09-30T14:38:36.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.831+0000 7fd0f682b700 1 -- 172.21.15.32:0/921474977 shutdown_connections 2023-09-30T14:38:36.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.831+0000 7fd0f682b700 1 --2- 172.21.15.32:0/921474977 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd0d80a1030 0x7fd0d80a34f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.831+0000 7fd0f682b700 1 --2- 172.21.15.32:0/921474977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0f0106b10 0x7fd0f019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.831+0000 7fd0f682b700 1 --2- 172.21.15.32:0/921474977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0f01018b0 0x7fd0f0196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.831+0000 7fd0f682b700 1 --2- 172.21.15.32:0/921474977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0f01041e0 0x7fd0f0197050 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.831+0000 7fd0f682b700 1 -- 172.21.15.32:0/921474977 >> 172.21.15.32:0/921474977 conn(0x7fd0f00fb2a0 msgr2=0x7fd0f01025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:36.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.831+0000 7fd0f682b700 1 -- 172.21.15.32:0/921474977 shutdown_connections 2023-09-30T14:38:36.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.831+0000 7fd0f682b700 1 -- 172.21.15.32:0/921474977 wait complete. 2023-09-30T14:38:36.843 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:36 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1653981534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:36.844 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T14:38:36.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:36 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1653981534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:36.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.972+0000 7f9f5c381700 1 -- 172.21.15.32:0/2278711982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f541018c0 msgr2=0x7f9f54108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.972+0000 7f9f5c381700 1 --2- 172.21.15.32:0/2278711982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f541018c0 0x7f9f54108170 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00b0d0 tx=0x7f9f4c00b3e0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.972+0000 7f9f5c381700 1 -- 172.21.15.32:0/2278711982 shutdown_connections 2023-09-30T14:38:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.972+0000 7f9f5c381700 1 --2- 172.21.15.32:0/2278711982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f541018c0 0x7f9f54108170 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.972+0000 7f9f5c381700 1 --2- 172.21.15.32:0/2278711982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f540ffd30 0x7f9f54100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.972+0000 7f9f5c381700 1 --2- 172.21.15.32:0/2278711982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54100f20 0x7f9f54101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.972+0000 7f9f5c381700 1 -- 172.21.15.32:0/2278711982 >> 172.21.15.32:0/2278711982 conn(0x7f9f540fb2a0 msgr2=0x7f9f540fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.972+0000 7f9f5c381700 1 -- 172.21.15.32:0/2278711982 shutdown_connections 2023-09-30T14:38:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.972+0000 7f9f5c381700 1 -- 172.21.15.32:0/2278711982 wait complete. 2023-09-30T14:38:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.972+0000 7f9f5c381700 1 Processor -- start 2023-09-30T14:38:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.972+0000 7f9f5c381700 1 -- start start 2023-09-30T14:38:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5c381700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f540ffd30 0x7f9f54103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5c381700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54100f20 0x7f9f541025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5c381700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f541018c0 0x7f9f54102af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5c381700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f54103290 con 0x7f9f540ffd30 2023-09-30T14:38:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5c381700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f54103400 con 0x7f9f541018c0 2023-09-30T14:38:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5c381700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541a0ce0 con 0x7f9f54100f20 2023-09-30T14:38:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5a11d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f540ffd30 0x7f9f54103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5a11d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f540ffd30 0x7f9f54103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35128/0 (socket says 172.21.15.32:35128) 2023-09-30T14:38:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5a11d700 1 -- 172.21.15.32:0/3355339660 learned_addr learned my addr 172.21.15.32:0/3355339660 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5a91e700 1 --2- 172.21.15.32:0/3355339660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f541018c0 0x7f9f54102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5991c700 1 --2- 172.21.15.32:0/3355339660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54100f20 0x7f9f541025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5a11d700 1 -- 172.21.15.32:0/3355339660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54100f20 msgr2=0x7f9f541025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5a11d700 1 --2- 172.21.15.32:0/3355339660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54100f20 0x7f9f541025b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5a11d700 1 -- 172.21.15.32:0/3355339660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f541018c0 msgr2=0x7f9f54102af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5a11d700 1 --2- 172.21.15.32:0/3355339660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f541018c0 0x7f9f54102af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5a11d700 1 -- 172.21.15.32:0/3355339660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c009d00 con 0x7f9f540ffd30 2023-09-30T14:38:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5991c700 1 --2- 172.21.15.32:0/3355339660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54100f20 0x7f9f541025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f5a11d700 1 --2- 172.21.15.32:0/3355339660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f540ffd30 0x7f9f54103f70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400d980 tx=0x7f9f4400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.973+0000 7f9f4b7fe700 1 -- 172.21.15.32:0/3355339660 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400b950 con 0x7f9f540ffd30 2023-09-30T14:38:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.974+0000 7f9f4b7fe700 1 -- 172.21.15.32:0/3355339660 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f44007620 con 0x7f9f540ffd30 2023-09-30T14:38:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.974+0000 7f9f5c381700 1 -- 172.21.15.32:0/3355339660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f541a0fc0 con 0x7f9f540ffd30 2023-09-30T14:38:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.974+0000 7f9f4b7fe700 1 -- 172.21.15.32:0/3355339660 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400f3c0 con 0x7f9f540ffd30 2023-09-30T14:38:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.974+0000 7f9f5c381700 1 -- 172.21.15.32:0/3355339660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f541a13e0 con 0x7f9f540ffd30 2023-09-30T14:38:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.974+0000 7f9f5c381700 1 -- 172.21.15.32:0/3355339660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f54107db0 con 0x7f9f540ffd30 2023-09-30T14:38:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.976+0000 7f9f4b7fe700 1 -- 172.21.15.32:0/3355339660 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f9f44010640 con 0x7f9f540ffd30 2023-09-30T14:38:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.976+0000 7f9f4b7fe700 1 --2- 172.21.15.32:0/3355339660 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9f400a1030 0x7f9f400a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.976+0000 7f9f5991c700 1 --2- 172.21.15.32:0/3355339660 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9f400a1030 0x7f9f400a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.976+0000 7f9f4b7fe700 1 -- 172.21.15.32:0/3355339660 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9f440ccf90 con 0x7f9f540ffd30 2023-09-30T14:38:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.977+0000 7f9f5991c700 1 --2- 172.21.15.32:0/3355339660 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9f400a1030 0x7f9f400a34f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9f50005e10 tx=0x7f9f50005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:36.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:36.980+0000 7f9f4b7fe700 1 -- 172.21.15.32:0/3355339660 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f9f440966e0 con 0x7f9f540ffd30 2023-09-30T14:38:37.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.186+0000 7f9f5c381700 1 -- 172.21.15.32:0/3355339660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9f5402cfe0 con 0x7f9f540ffd30 2023-09-30T14:38:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.187+0000 7f9f4b7fe700 1 -- 172.21.15.32:0/3355339660 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9f44099d00 con 0x7f9f540ffd30 2023-09-30T14:38:37.188 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-09-30T14:38:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.189+0000 7f9f5c381700 1 -- 172.21.15.32:0/3355339660 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9f400a1030 msgr2=0x7f9f400a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.189+0000 7f9f5c381700 1 --2- 172.21.15.32:0/3355339660 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9f400a1030 0x7f9f400a34f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9f50005e10 tx=0x7f9f50005d60 comp rx=0 tx=0).stop 2023-09-30T14:38:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.189+0000 7f9f5c381700 1 -- 172.21.15.32:0/3355339660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f540ffd30 msgr2=0x7f9f54103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.189+0000 7f9f5c381700 1 --2- 172.21.15.32:0/3355339660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f540ffd30 0x7f9f54103f70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400d980 tx=0x7f9f4400dc90 comp rx=0 tx=0).stop 2023-09-30T14:38:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.189+0000 7f9f5c381700 1 -- 172.21.15.32:0/3355339660 shutdown_connections 2023-09-30T14:38:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.189+0000 7f9f5c381700 1 --2- 172.21.15.32:0/3355339660 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9f400a1030 0x7f9f400a34f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.189+0000 7f9f5c381700 1 --2- 172.21.15.32:0/3355339660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f541018c0 0x7f9f54102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.189+0000 7f9f5c381700 1 --2- 172.21.15.32:0/3355339660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f540ffd30 0x7f9f54103f70 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.189+0000 7f9f5c381700 1 --2- 172.21.15.32:0/3355339660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f54100f20 0x7f9f541025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.189+0000 7f9f5c381700 1 -- 172.21.15.32:0/3355339660 >> 172.21.15.32:0/3355339660 conn(0x7f9f540fb2a0 msgr2=0x7f9f540fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.189+0000 7f9f5c381700 1 -- 172.21.15.32:0/3355339660 shutdown_connections 2023-09-30T14:38:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.189+0000 7f9f5c381700 1 -- 172.21.15.32:0/3355339660 wait complete. 2023-09-30T14:38:37.202 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T14:38:37.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.327+0000 7febed08a700 1 -- 172.21.15.32:0/644354873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febe81018d0 msgr2=0x7febe8103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:37.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.327+0000 7febed08a700 1 --2- 172.21.15.32:0/644354873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febe81018d0 0x7febe8103cc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7febd8009990 tx=0x7febd8009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.327+0000 7febed08a700 1 -- 172.21.15.32:0/644354873 shutdown_connections 2023-09-30T14:38:37.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.327+0000 7febed08a700 1 --2- 172.21.15.32:0/644354873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe8106b30 0x7febe8108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.327+0000 7febed08a700 1 --2- 172.21.15.32:0/644354873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febe81018d0 0x7febe8103cc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.327+0000 7febed08a700 1 --2- 172.21.15.32:0/644354873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febe8104200 0x7febe81065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.327+0000 7febed08a700 1 -- 172.21.15.32:0/644354873 >> 172.21.15.32:0/644354873 conn(0x7febe80fb280 msgr2=0x7febe80fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:37.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.327+0000 7febed08a700 1 -- 172.21.15.32:0/644354873 shutdown_connections 2023-09-30T14:38:37.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.327+0000 7febed08a700 1 -- 172.21.15.32:0/644354873 wait complete. 2023-09-30T14:38:37.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.328+0000 7febed08a700 1 Processor -- start 2023-09-30T14:38:37.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.328+0000 7febed08a700 1 -- start start 2023-09-30T14:38:37.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.328+0000 7febed08a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febe81018d0 0x7febe8101240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:37.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.328+0000 7febed08a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febe8104200 0x7febe80ff880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:37.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.328+0000 7febed08a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe8106b30 0x7febe80ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.328+0000 7febed08a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe8100560 con 0x7febe8104200 2023-09-30T14:38:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.328+0000 7febed08a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe81006d0 con 0x7febe81018d0 2023-09-30T14:38:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.328+0000 7febed08a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe81a0b00 con 0x7febe8106b30 2023-09-30T14:38:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.329+0000 7febe659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febe8104200 0x7febe80ff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.329+0000 7febe659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febe8104200 0x7febe80ff880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35144/0 (socket says 172.21.15.32:35144) 2023-09-30T14:38:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.329+0000 7febe659c700 1 -- 172.21.15.32:0/3480334017 learned_addr learned my addr 172.21.15.32:0/3480334017 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.329+0000 7febe759e700 1 --2- 172.21.15.32:0/3480334017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe8106b30 0x7febe80ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.329+0000 7febe6d9d700 1 --2- 172.21.15.32:0/3480334017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febe81018d0 0x7febe8101240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.329+0000 7febe659c700 1 -- 172.21.15.32:0/3480334017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe8106b30 msgr2=0x7febe80ffdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.329+0000 7febe659c700 1 --2- 172.21.15.32:0/3480334017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe8106b30 0x7febe80ffdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.329+0000 7febe659c700 1 -- 172.21.15.32:0/3480334017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febe81018d0 msgr2=0x7febe8101240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.329+0000 7febe659c700 1 --2- 172.21.15.32:0/3480334017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febe81018d0 0x7febe8101240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.329+0000 7febe659c700 1 -- 172.21.15.32:0/3480334017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd80095f0 con 0x7febe8104200 2023-09-30T14:38:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.329+0000 7febe659c700 1 --2- 172.21.15.32:0/3480334017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febe8104200 0x7febe80ff880 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7febdc00e9a0 tx=0x7febdc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.329+0000 7febcffff700 1 -- 172.21.15.32:0/3480334017 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc009790 con 0x7febe8104200 2023-09-30T14:38:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.329+0000 7febcffff700 1 -- 172.21.15.32:0/3480334017 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febdc010430 con 0x7febe8104200 2023-09-30T14:38:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.329+0000 7febcffff700 1 -- 172.21.15.32:0/3480334017 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc018680 con 0x7febe8104200 2023-09-30T14:38:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.329+0000 7febed08a700 1 -- 172.21.15.32:0/3480334017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe81a0de0 con 0x7febe8104200 2023-09-30T14:38:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.329+0000 7febed08a700 1 -- 172.21.15.32:0/3480334017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe81a1210 con 0x7febe8104200 2023-09-30T14:38:37.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.330+0000 7febed08a700 1 -- 172.21.15.32:0/3480334017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe80fce00 con 0x7febe8104200 2023-09-30T14:38:37.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.332+0000 7febcffff700 1 -- 172.21.15.32:0/3480334017 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7febdc0105a0 con 0x7febe8104200 2023-09-30T14:38:37.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.332+0000 7febcffff700 1 --2- 172.21.15.32:0/3480334017 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7febd00a1070 0x7febd00a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:37.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.332+0000 7febcffff700 1 -- 172.21.15.32:0/3480334017 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7febdc014070 con 0x7febe8104200 2023-09-30T14:38:37.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.332+0000 7febe6d9d700 1 --2- 172.21.15.32:0/3480334017 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7febd00a1070 0x7febd00a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:37.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.333+0000 7febe6d9d700 1 --2- 172.21.15.32:0/3480334017 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7febd00a1070 0x7febd00a3530 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7febd8006010 tx=0x7febd800b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:37.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.336+0000 7febcffff700 1 -- 172.21.15.32:0/3480334017 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7febdc0962d0 con 0x7febe8104200 2023-09-30T14:38:37.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.541+0000 7febed08a700 1 -- 172.21.15.32:0/3480334017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febe810c860 con 0x7febe8104200 2023-09-30T14:38:37.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.543+0000 7febcffff700 1 -- 172.21.15.32:0/3480334017 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22569 (secure 0 0 0) 0x7febdc026090 con 0x7febe8104200 2023-09-30T14:38:37.544 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:37.544 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":85,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:38:13.192089+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:38:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.545+0000 7febed08a700 1 -- 172.21.15.32:0/3480334017 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7febd00a1070 msgr2=0x7febd00a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.545+0000 7febed08a700 1 --2- 172.21.15.32:0/3480334017 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7febd00a1070 0x7febd00a3530 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7febd8006010 tx=0x7febd800b280 comp rx=0 tx=0).stop 2023-09-30T14:38:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.545+0000 7febed08a700 1 -- 172.21.15.32:0/3480334017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febe8104200 msgr2=0x7febe80ff880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.545+0000 7febed08a700 1 --2- 172.21.15.32:0/3480334017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febe8104200 0x7febe80ff880 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7febdc00e9a0 tx=0x7febdc00ecb0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.545+0000 7febed08a700 1 -- 172.21.15.32:0/3480334017 shutdown_connections 2023-09-30T14:38:37.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.545+0000 7febed08a700 1 --2- 172.21.15.32:0/3480334017 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7febd00a1070 0x7febd00a3530 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.545+0000 7febed08a700 1 --2- 172.21.15.32:0/3480334017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe8106b30 0x7febe80ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.545+0000 7febed08a700 1 --2- 172.21.15.32:0/3480334017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febe81018d0 0x7febe8101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.545+0000 7febed08a700 1 --2- 172.21.15.32:0/3480334017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febe8104200 0x7febe80ff880 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.545+0000 7febed08a700 1 -- 172.21.15.32:0/3480334017 >> 172.21.15.32:0/3480334017 conn(0x7febe80fb280 msgr2=0x7febe8102540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:37.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.545+0000 7febed08a700 1 -- 172.21.15.32:0/3480334017 shutdown_connections 2023-09-30T14:38:37.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.545+0000 7febed08a700 1 -- 172.21.15.32:0/3480334017 wait complete. 2023-09-30T14:38:37.559 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T14:33:20.224529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T14:37:09.573219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T14:37:10.844458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T14:38:37.559 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T14:38:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.687+0000 7f847fcba700 1 -- 172.21.15.32:0/3337422553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84781018b0 msgr2=0x7f8478103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.687+0000 7f847fcba700 1 --2- 172.21.15.32:0/3337422553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84781018b0 0x7f8478103ca0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8468009990 tx=0x7f8468009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.687+0000 7f847fcba700 1 -- 172.21.15.32:0/3337422553 shutdown_connections 2023-09-30T14:38:37.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.687+0000 7f847fcba700 1 --2- 172.21.15.32:0/3337422553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8478106b10 0x7f8478108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.687+0000 7f847fcba700 1 --2- 172.21.15.32:0/3337422553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84781018b0 0x7f8478103ca0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.687+0000 7f847fcba700 1 --2- 172.21.15.32:0/3337422553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84781041e0 0x7f84781065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.687+0000 7f847fcba700 1 -- 172.21.15.32:0/3337422553 >> 172.21.15.32:0/3337422553 conn(0x7f84780fb2a0 msgr2=0x7f84780fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:37.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.687+0000 7f847fcba700 1 -- 172.21.15.32:0/3337422553 shutdown_connections 2023-09-30T14:38:37.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.687+0000 7f847fcba700 1 -- 172.21.15.32:0/3337422553 wait complete. 2023-09-30T14:38:37.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847fcba700 1 Processor -- start 2023-09-30T14:38:37.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847fcba700 1 -- start start 2023-09-30T14:38:37.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847fcba700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84781018b0 0x7f8478196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:37.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847fcba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84781041e0 0x7f8478196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:37.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847fcba700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8478106b10 0x7f847819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:37.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847fcba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847819e5f0 con 0x7f84781041e0 2023-09-30T14:38:37.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847fcba700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847819e730 con 0x7f8478106b10 2023-09-30T14:38:37.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847fcba700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847819e8c0 con 0x7f84781018b0 2023-09-30T14:38:37.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847d255700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84781041e0 0x7f8478196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:37.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847d255700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84781041e0 0x7f8478196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35158/0 (socket says 172.21.15.32:35158) 2023-09-30T14:38:37.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847d255700 1 -- 172.21.15.32:0/416009232 learned_addr learned my addr 172.21.15.32:0/416009232 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:37.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847da56700 1 --2- 172.21.15.32:0/416009232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84781018b0 0x7f8478196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:37.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847e257700 1 --2- 172.21.15.32:0/416009232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8478106b10 0x7f847819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:37.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847d255700 1 -- 172.21.15.32:0/416009232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84781018b0 msgr2=0x7f8478196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:37.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847d255700 1 --2- 172.21.15.32:0/416009232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84781018b0 0x7f8478196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847d255700 1 -- 172.21.15.32:0/416009232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8478106b10 msgr2=0x7f847819e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:37.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847d255700 1 --2- 172.21.15.32:0/416009232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8478106b10 0x7f847819e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.688+0000 7f847d255700 1 -- 172.21.15.32:0/416009232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84680095f0 con 0x7f84781041e0 2023-09-30T14:38:37.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.689+0000 7f847d255700 1 --2- 172.21.15.32:0/416009232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84781041e0 0x7f8478196fb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f847400d950 tx=0x7f847400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.689+0000 7f846effd700 1 -- 172.21.15.32:0/416009232 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847400f890 con 0x7f84781041e0 2023-09-30T14:38:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.689+0000 7f846effd700 1 -- 172.21.15.32:0/416009232 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8474010460 con 0x7f84781041e0 2023-09-30T14:38:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.689+0000 7f847fcba700 1 -- 172.21.15.32:0/416009232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847819eba0 con 0x7f84781041e0 2023-09-30T14:38:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.689+0000 7f846effd700 1 -- 172.21.15.32:0/416009232 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847400e610 con 0x7f84781041e0 2023-09-30T14:38:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.689+0000 7f847fcba700 1 -- 172.21.15.32:0/416009232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847819f0b0 con 0x7f84781041e0 2023-09-30T14:38:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.690+0000 7f847fcba700 1 -- 172.21.15.32:0/416009232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84780fce20 con 0x7f84781041e0 2023-09-30T14:38:37.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.691+0000 7f846effd700 1 -- 172.21.15.32:0/416009232 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f84740108d0 con 0x7f84781041e0 2023-09-30T14:38:37.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.691+0000 7f846effd700 1 --2- 172.21.15.32:0/416009232 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f84640a1030 0x7f84640a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:37.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.691+0000 7f847da56700 1 --2- 172.21.15.32:0/416009232 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f84640a1030 0x7f84640a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:37.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.691+0000 7f846effd700 1 -- 172.21.15.32:0/416009232 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f84740cc600 con 0x7f84781041e0 2023-09-30T14:38:37.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.692+0000 7f847da56700 1 --2- 172.21.15.32:0/416009232 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f84640a1030 0x7f84640a34f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f846800b5c0 tx=0x7f8468005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:37.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.695+0000 7f846effd700 1 -- 172.21.15.32:0/416009232 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f8474095cb0 con 0x7f84781041e0 2023-09-30T14:38:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:37 smithi156 ceph-mon[103579]: pgmap v297: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:38:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:37 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/837191398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:37 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/921474977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:37 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3355339660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:37.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:37 smithi148 ceph-mon[102355]: pgmap v297: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:38:37.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:37 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/837191398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:37.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:37 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/921474977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:37.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:37 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3355339660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:37.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:37 smithi032 ceph-mon[98592]: pgmap v297: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-09-30T14:38:37.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:37 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/837191398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:37.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:37 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/921474977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:37.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:37 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3355339660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:37.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.897+0000 7f847fcba700 1 -- 172.21.15.32:0/416009232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f847810c7a0 con 0x7f84781041e0 2023-09-30T14:38:37.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.897+0000 7f846effd700 1 -- 172.21.15.32:0/416009232 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f84740992d0 con 0x7f84781041e0 2023-09-30T14:38:37.898 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T14:38:37.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.899+0000 7f847fcba700 1 -- 172.21.15.32:0/416009232 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f84640a1030 msgr2=0x7f84640a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:37.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.899+0000 7f847fcba700 1 --2- 172.21.15.32:0/416009232 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f84640a1030 0x7f84640a34f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f846800b5c0 tx=0x7f8468005960 comp rx=0 tx=0).stop 2023-09-30T14:38:37.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.899+0000 7f847fcba700 1 -- 172.21.15.32:0/416009232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84781041e0 msgr2=0x7f8478196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.899+0000 7f847fcba700 1 --2- 172.21.15.32:0/416009232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84781041e0 0x7f8478196fb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f847400d950 tx=0x7f847400dc60 comp rx=0 tx=0).stop 2023-09-30T14:38:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.899+0000 7f847fcba700 1 -- 172.21.15.32:0/416009232 shutdown_connections 2023-09-30T14:38:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.899+0000 7f847fcba700 1 --2- 172.21.15.32:0/416009232 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f84640a1030 0x7f84640a34f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.899+0000 7f847fcba700 1 --2- 172.21.15.32:0/416009232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8478106b10 0x7f847819e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.899+0000 7f847fcba700 1 --2- 172.21.15.32:0/416009232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84781018b0 0x7f8478196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.899+0000 7f847fcba700 1 --2- 172.21.15.32:0/416009232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84781041e0 0x7f8478196fb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.899+0000 7f847fcba700 1 -- 172.21.15.32:0/416009232 >> 172.21.15.32:0/416009232 conn(0x7f84780fb2a0 msgr2=0x7f8478102520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.899+0000 7f847fcba700 1 -- 172.21.15.32:0/416009232 shutdown_connections 2023-09-30T14:38:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:37.899+0000 7f847fcba700 1 -- 172.21.15.32:0/416009232 wait complete. 2023-09-30T14:38:37.912 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T14:38:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.041+0000 7fc042a60700 1 -- 172.21.15.32:0/2856948419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03c101fb0 msgr2=0x7fc03c102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.041+0000 7fc042a60700 1 --2- 172.21.15.32:0/2856948419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03c101fb0 0x7fc03c102430 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc030009990 tx=0x7fc030009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.041+0000 7fc042a60700 1 -- 172.21.15.32:0/2856948419 shutdown_connections 2023-09-30T14:38:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.041+0000 7fc042a60700 1 --2- 172.21.15.32:0/2856948419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc03c102970 0x7fc03c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.041+0000 7fc042a60700 1 --2- 172.21.15.32:0/2856948419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc03c100e60 0x7fc03c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.041+0000 7fc042a60700 1 --2- 172.21.15.32:0/2856948419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03c101fb0 0x7fc03c102430 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.041+0000 7fc042a60700 1 -- 172.21.15.32:0/2856948419 >> 172.21.15.32:0/2856948419 conn(0x7fc03c0fc3d0 msgr2=0x7fc03c0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.041+0000 7fc042a60700 1 -- 172.21.15.32:0/2856948419 shutdown_connections 2023-09-30T14:38:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.041+0000 7fc042a60700 1 -- 172.21.15.32:0/2856948419 wait complete. 2023-09-30T14:38:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.041+0000 7fc042a60700 1 Processor -- start 2023-09-30T14:38:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.041+0000 7fc042a60700 1 -- start start 2023-09-30T14:38:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc042a60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03c100e60 0x7fc03c196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc042a60700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc03c101fb0 0x7fc03c1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc042a60700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc03c102970 0x7fc03c19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc042a60700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c19e6e0 con 0x7fc03c100e60 2023-09-30T14:38:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc042a60700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c19e820 con 0x7fc03c102970 2023-09-30T14:38:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc042a60700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c19e9b0 con 0x7fc03c101fb0 2023-09-30T14:38:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc03bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03c100e60 0x7fc03c196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc03bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03c100e60 0x7fc03c196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35170/0 (socket says 172.21.15.32:35170) 2023-09-30T14:38:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc03bfff700 1 -- 172.21.15.32:0/2837743408 learned_addr learned my addr 172.21.15.32:0/2837743408 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc040ffd700 1 --2- 172.21.15.32:0/2837743408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc03c102970 0x7fc03c19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc03b7fe700 1 --2- 172.21.15.32:0/2837743408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc03c101fb0 0x7fc03c1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc03bfff700 1 -- 172.21.15.32:0/2837743408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc03c101fb0 msgr2=0x7fc03c1970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc03bfff700 1 --2- 172.21.15.32:0/2837743408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc03c101fb0 0x7fc03c1970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc03bfff700 1 -- 172.21.15.32:0/2837743408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc03c102970 msgr2=0x7fc03c19e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc03bfff700 1 --2- 172.21.15.32:0/2837743408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc03c102970 0x7fc03c19e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc03bfff700 1 -- 172.21.15.32:0/2837743408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0300095f0 con 0x7fc03c100e60 2023-09-30T14:38:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc03b7fe700 1 --2- 172.21.15.32:0/2837743408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc03c101fb0 0x7fc03c1970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc03bfff700 1 --2- 172.21.15.32:0/2837743408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03c100e60 0x7fc03c196b60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00d980 tx=0x7fc02c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc0397fa700 1 -- 172.21.15.32:0/2837743408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c004220 con 0x7fc03c100e60 2023-09-30T14:38:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc0397fa700 1 -- 172.21.15.32:0/2837743408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc02c007640 con 0x7fc03c100e60 2023-09-30T14:38:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc042a60700 1 -- 172.21.15.32:0/2837743408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03c19ec90 con 0x7fc03c100e60 2023-09-30T14:38:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.042+0000 7fc0397fa700 1 -- 172.21.15.32:0/2837743408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c005610 con 0x7fc03c100e60 2023-09-30T14:38:38.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.043+0000 7fc042a60700 1 -- 172.21.15.32:0/2837743408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03c19f1a0 con 0x7fc03c100e60 2023-09-30T14:38:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.044+0000 7fc042a60700 1 -- 172.21.15.32:0/2837743408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03c1083e0 con 0x7fc03c100e60 2023-09-30T14:38:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.045+0000 7fc0397fa700 1 -- 172.21.15.32:0/2837743408 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fc02c004750 con 0x7fc03c100e60 2023-09-30T14:38:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.045+0000 7fc0397fa700 1 --2- 172.21.15.32:0/2837743408 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc0240a1030 0x7fc0240a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.045+0000 7fc03b7fe700 1 --2- 172.21.15.32:0/2837743408 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc0240a1030 0x7fc0240a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.045+0000 7fc0397fa700 1 -- 172.21.15.32:0/2837743408 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc02c0cc780 con 0x7fc03c100e60 2023-09-30T14:38:38.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.046+0000 7fc03b7fe700 1 --2- 172.21.15.32:0/2837743408 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc0240a1030 0x7fc0240a34f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc030006010 tx=0x7fc030005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.050+0000 7fc0397fa700 1 -- 172.21.15.32:0/2837743408 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fc02c095e30 con 0x7fc03c100e60 2023-09-30T14:38:38.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.255+0000 7fc042a60700 1 -- 172.21.15.32:0/2837743408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc03c10c910 con 0x7fc03c100e60 2023-09-30T14:38:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.255+0000 7fc0397fa700 1 -- 172.21.15.32:0/2837743408 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc02c099450 con 0x7fc03c100e60 2023-09-30T14:38:38.256 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-09-30T14:38:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.257+0000 7fc042a60700 1 -- 172.21.15.32:0/2837743408 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc0240a1030 msgr2=0x7fc0240a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.257+0000 7fc042a60700 1 --2- 172.21.15.32:0/2837743408 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc0240a1030 0x7fc0240a34f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc030006010 tx=0x7fc030005c20 comp rx=0 tx=0).stop 2023-09-30T14:38:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.257+0000 7fc042a60700 1 -- 172.21.15.32:0/2837743408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03c100e60 msgr2=0x7fc03c196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.257+0000 7fc042a60700 1 --2- 172.21.15.32:0/2837743408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03c100e60 0x7fc03c196b60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00d980 tx=0x7fc02c00dc90 comp rx=0 tx=0).stop 2023-09-30T14:38:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.257+0000 7fc042a60700 1 -- 172.21.15.32:0/2837743408 shutdown_connections 2023-09-30T14:38:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.257+0000 7fc042a60700 1 --2- 172.21.15.32:0/2837743408 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc0240a1030 0x7fc0240a34f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.257+0000 7fc042a60700 1 --2- 172.21.15.32:0/2837743408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc03c102970 0x7fc03c19e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.257+0000 7fc042a60700 1 --2- 172.21.15.32:0/2837743408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03c100e60 0x7fc03c196b60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.257+0000 7fc042a60700 1 --2- 172.21.15.32:0/2837743408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc03c101fb0 0x7fc03c1970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.257+0000 7fc042a60700 1 -- 172.21.15.32:0/2837743408 >> 172.21.15.32:0/2837743408 conn(0x7fc03c0fc3d0 msgr2=0x7fc03c105520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.257+0000 7fc042a60700 1 -- 172.21.15.32:0/2837743408 shutdown_connections 2023-09-30T14:38:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.257+0000 7fc042a60700 1 -- 172.21.15.32:0/2837743408 wait complete. 2023-09-30T14:38:38.270 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T14:38:38.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.399+0000 7f1b8628b700 1 -- 172.21.15.32:0/664447834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80101930 msgr2=0x7f1b80103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:38.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.399+0000 7f1b8628b700 1 --2- 172.21.15.32:0/664447834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80101930 0x7f1b80103d20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1b70009990 tx=0x7f1b70009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.399+0000 7f1b8628b700 1 -- 172.21.15.32:0/664447834 shutdown_connections 2023-09-30T14:38:38.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.399+0000 7f1b8628b700 1 --2- 172.21.15.32:0/664447834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b80106b90 0x7f1b80108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.399+0000 7f1b8628b700 1 --2- 172.21.15.32:0/664447834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80101930 0x7f1b80103d20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.399+0000 7f1b8628b700 1 --2- 172.21.15.32:0/664447834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b80104260 0x7f1b80106650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.399+0000 7f1b8628b700 1 -- 172.21.15.32:0/664447834 >> 172.21.15.32:0/664447834 conn(0x7f1b800fb2a0 msgr2=0x7f1b800fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:38.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.399+0000 7f1b8628b700 1 -- 172.21.15.32:0/664447834 shutdown_connections 2023-09-30T14:38:38.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.399+0000 7f1b8628b700 1 -- 172.21.15.32:0/664447834 wait complete. 2023-09-30T14:38:38.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.399+0000 7f1b8628b700 1 Processor -- start 2023-09-30T14:38:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b8628b700 1 -- start start 2023-09-30T14:38:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b8628b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80101930 0x7f1b80196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b8628b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b80104260 0x7f1b801970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b8628b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b80106b90 0x7f1b8019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b8628b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8019e720 con 0x7f1b80101930 2023-09-30T14:38:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b8628b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8019e860 con 0x7f1b80106b90 2023-09-30T14:38:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b8628b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8019e9f0 con 0x7f1b80104260 2023-09-30T14:38:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b7ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80101930 0x7f1b80196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b7ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80101930 0x7f1b80196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35184/0 (socket says 172.21.15.32:35184) 2023-09-30T14:38:38.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b7ffff700 1 -- 172.21.15.32:0/1961372723 learned_addr learned my addr 172.21.15.32:0/1961372723 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b7f7fe700 1 --2- 172.21.15.32:0/1961372723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b80104260 0x7f1b801970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b84828700 1 --2- 172.21.15.32:0/1961372723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b80106b90 0x7f1b8019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b7ffff700 1 -- 172.21.15.32:0/1961372723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b80104260 msgr2=0x7f1b801970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b7ffff700 1 --2- 172.21.15.32:0/1961372723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b80104260 0x7f1b801970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b7ffff700 1 -- 172.21.15.32:0/1961372723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b80106b90 msgr2=0x7f1b8019e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b7ffff700 1 --2- 172.21.15.32:0/1961372723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b80106b90 0x7f1b8019e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b7ffff700 1 -- 172.21.15.32:0/1961372723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b700095f0 con 0x7f1b80101930 2023-09-30T14:38:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.400+0000 7f1b84828700 1 --2- 172.21.15.32:0/1961372723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b80106b90 0x7f1b8019e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.401+0000 7f1b7ffff700 1 --2- 172.21.15.32:0/1961372723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80101930 0x7f1b80196ba0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1b70005910 tx=0x7f1b7000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.401+0000 7f1b7d7fa700 1 -- 172.21.15.32:0/1961372723 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7001d070 con 0x7f1b80101930 2023-09-30T14:38:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.401+0000 7f1b7d7fa700 1 -- 172.21.15.32:0/1961372723 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b7000fb80 con 0x7f1b80101930 2023-09-30T14:38:38.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.401+0000 7f1b8628b700 1 -- 172.21.15.32:0/1961372723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8019ec70 con 0x7f1b80101930 2023-09-30T14:38:38.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.401+0000 7f1b7d7fa700 1 -- 172.21.15.32:0/1961372723 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b70017650 con 0x7f1b80101930 2023-09-30T14:38:38.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.401+0000 7f1b8628b700 1 -- 172.21.15.32:0/1961372723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8019f090 con 0x7f1b80101930 2023-09-30T14:38:38.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.402+0000 7f1b8628b700 1 -- 172.21.15.32:0/1961372723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b800fce20 con 0x7f1b80101930 2023-09-30T14:38:38.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.403+0000 7f1b7d7fa700 1 -- 172.21.15.32:0/1961372723 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f1b70022470 con 0x7f1b80101930 2023-09-30T14:38:38.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.403+0000 7f1b7d7fa700 1 --2- 172.21.15.32:0/1961372723 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1b6c0a1030 0x7f1b6c0a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:38.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.404+0000 7f1b7f7fe700 1 --2- 172.21.15.32:0/1961372723 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1b6c0a1030 0x7f1b6c0a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:38.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.404+0000 7f1b7d7fa700 1 -- 172.21.15.32:0/1961372723 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1b700cdca0 con 0x7f1b80101930 2023-09-30T14:38:38.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.404+0000 7f1b7f7fe700 1 --2- 172.21.15.32:0/1961372723 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1b6c0a1030 0x7f1b6c0a34f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1b68007c60 tx=0x7f1b680073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:38.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.407+0000 7f1b7d7fa700 1 -- 172.21.15.32:0/1961372723 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f1b70097410 con 0x7f1b80101930 2023-09-30T14:38:38.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.609+0000 7f1b8628b700 1 -- 172.21.15.32:0/1961372723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b8002cfe0 con 0x7f1b80101930 2023-09-30T14:38:38.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.609+0000 7f1b7d7fa700 1 -- 172.21.15.32:0/1961372723 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b7009aa30 con 0x7f1b80101930 2023-09-30T14:38:38.611 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-09-30T14:38:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.612+0000 7f1b8628b700 1 -- 172.21.15.32:0/1961372723 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1b6c0a1030 msgr2=0x7f1b6c0a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.612+0000 7f1b8628b700 1 --2- 172.21.15.32:0/1961372723 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1b6c0a1030 0x7f1b6c0a34f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1b68007c60 tx=0x7f1b680073d0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.612+0000 7f1b8628b700 1 -- 172.21.15.32:0/1961372723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80101930 msgr2=0x7f1b80196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.612+0000 7f1b8628b700 1 --2- 172.21.15.32:0/1961372723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80101930 0x7f1b80196ba0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1b70005910 tx=0x7f1b7000f690 comp rx=0 tx=0).stop 2023-09-30T14:38:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.612+0000 7f1b8628b700 1 -- 172.21.15.32:0/1961372723 shutdown_connections 2023-09-30T14:38:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.612+0000 7f1b8628b700 1 --2- 172.21.15.32:0/1961372723 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1b6c0a1030 0x7f1b6c0a34f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.612+0000 7f1b8628b700 1 --2- 172.21.15.32:0/1961372723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b80106b90 0x7f1b8019e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.612+0000 7f1b8628b700 1 --2- 172.21.15.32:0/1961372723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b80101930 0x7f1b80196ba0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.612+0000 7f1b8628b700 1 --2- 172.21.15.32:0/1961372723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b80104260 0x7f1b801970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.612+0000 7f1b8628b700 1 -- 172.21.15.32:0/1961372723 >> 172.21.15.32:0/1961372723 conn(0x7f1b800fb2a0 msgr2=0x7f1b801025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.612+0000 7f1b8628b700 1 -- 172.21.15.32:0/1961372723 shutdown_connections 2023-09-30T14:38:38.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.612+0000 7f1b8628b700 1 -- 172.21.15.32:0/1961372723 wait complete. 2023-09-30T14:38:38.624 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:38 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3480334017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:38.624 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:38 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/416009232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:38.624 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:38 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2837743408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:38.625 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-09-30T14:38:38.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:38:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.751+0000 7f2c2b0e4700 1 -- 172.21.15.32:0/390664554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c240ffd70 msgr2=0x7f2c24100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.751+0000 7f2c2b0e4700 1 --2- 172.21.15.32:0/390664554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c240ffd70 0x7f2c24100190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2c180099c0 tx=0x7f2c18009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.751+0000 7f2c2b0e4700 1 -- 172.21.15.32:0/390664554 shutdown_connections 2023-09-30T14:38:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.751+0000 7f2c2b0e4700 1 --2- 172.21.15.32:0/390664554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c24101900 0x7f2c241081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.751+0000 7f2c2b0e4700 1 --2- 172.21.15.32:0/390664554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c240ffd70 0x7f2c24100190 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.751+0000 7f2c2b0e4700 1 --2- 172.21.15.32:0/390664554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c24100f60 0x7f2c241013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.751+0000 7f2c2b0e4700 1 -- 172.21.15.32:0/390664554 >> 172.21.15.32:0/390664554 conn(0x7f2c240fb2e0 msgr2=0x7f2c240fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.751+0000 7f2c2b0e4700 1 -- 172.21.15.32:0/390664554 shutdown_connections 2023-09-30T14:38:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.751+0000 7f2c2b0e4700 1 -- 172.21.15.32:0/390664554 wait complete. 2023-09-30T14:38:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.751+0000 7f2c2b0e4700 1 Processor -- start 2023-09-30T14:38:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.751+0000 7f2c2b0e4700 1 -- start start 2023-09-30T14:38:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c2b0e4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c240ffd70 0x7f2c24196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c2b0e4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c24100f60 0x7f2c24197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c2b0e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c24101900 0x7f2c2419e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c2b0e4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2419e6d0 con 0x7f2c24101900 2023-09-30T14:38:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c2b0e4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2419e810 con 0x7f2c24100f60 2023-09-30T14:38:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c2b0e4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2419e9a0 con 0x7f2c240ffd70 2023-09-30T14:38:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c29681700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c24101900 0x7f2c2419e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c29681700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c24101900 0x7f2c2419e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35202/0 (socket says 172.21.15.32:35202) 2023-09-30T14:38:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c29681700 1 -- 172.21.15.32:0/3520360785 learned_addr learned my addr 172.21.15.32:0/3520360785 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c23fff700 1 --2- 172.21.15.32:0/3520360785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c24100f60 0x7f2c24197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c28e80700 1 --2- 172.21.15.32:0/3520360785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c240ffd70 0x7f2c24196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c29681700 1 -- 172.21.15.32:0/3520360785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c240ffd70 msgr2=0x7f2c24196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c29681700 1 --2- 172.21.15.32:0/3520360785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c240ffd70 0x7f2c24196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c29681700 1 -- 172.21.15.32:0/3520360785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c24100f60 msgr2=0x7f2c24197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c29681700 1 --2- 172.21.15.32:0/3520360785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c24100f60 0x7f2c24197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c29681700 1 -- 172.21.15.32:0/3520360785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c180095f0 con 0x7f2c24101900 2023-09-30T14:38:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c29681700 1 --2- 172.21.15.32:0/3520360785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c24101900 0x7f2c2419e100 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c007ee0 tx=0x7f2c1c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c21ffb700 1 -- 172.21.15.32:0/3520360785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c017070 con 0x7f2c24101900 2023-09-30T14:38:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c21ffb700 1 -- 172.21.15.32:0/3520360785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c1c00f040 con 0x7f2c24101900 2023-09-30T14:38:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.752+0000 7f2c2b0e4700 1 -- 172.21.15.32:0/3520360785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2419ec80 con 0x7f2c24101900 2023-09-30T14:38:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.753+0000 7f2c21ffb700 1 -- 172.21.15.32:0/3520360785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c005580 con 0x7f2c24101900 2023-09-30T14:38:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.753+0000 7f2c2b0e4700 1 -- 172.21.15.32:0/3520360785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2419f140 con 0x7f2c24101900 2023-09-30T14:38:38.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.754+0000 7f2c2b0e4700 1 -- 172.21.15.32:0/3520360785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2410b5d0 con 0x7f2c24101900 2023-09-30T14:38:38.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.755+0000 7f2c21ffb700 1 -- 172.21.15.32:0/3520360785 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f2c1c00db20 con 0x7f2c24101900 2023-09-30T14:38:38.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.755+0000 7f2c21ffb700 1 --2- 172.21.15.32:0/3520360785 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2c140a1070 0x7f2c140a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:38.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.756+0000 7f2c21ffb700 1 -- 172.21.15.32:0/3520360785 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2c1c0cc630 con 0x7f2c24101900 2023-09-30T14:38:38.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.756+0000 7f2c28e80700 1 --2- 172.21.15.32:0/3520360785 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2c140a1070 0x7f2c140a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:38.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.756+0000 7f2c28e80700 1 --2- 172.21.15.32:0/3520360785 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2c140a1070 0x7f2c140a3530 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2c1800b580 tx=0x7f2c18009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:38.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:38.759+0000 7f2c21ffb700 1 -- 172.21.15.32:0/3520360785 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f2c1c095ce0 con 0x7f2c24101900 2023-09-30T14:38:38.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:38 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3480334017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:38.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:38 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/416009232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:38.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:38 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2837743408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:38.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:38 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3480334017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:38.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:38 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/416009232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:38.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:38 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2837743408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.009+0000 7f2c2b0e4700 1 -- 172.21.15.32:0/3520360785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c2410b7c0 con 0x7f2c24101900 2023-09-30T14:38:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.010+0000 7f2c21ffb700 1 -- 172.21.15.32:0/3520360785 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11321 (secure 0 0 0) 0x7f2c1c099300 con 0x7f2c24101900 2023-09-30T14:38:39.011 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:39.011 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3505064841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3505064841},{"type":"v1","addr":"172.21.15.32:6839","nonce":3505064841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2262846038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2262846038},{"type":"v1","addr":"172.21.15.32:6841","nonce":2262846038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-30T14:37:11.444993+0000","modified":"2023-09-30T14:37:16.971534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6839/1445176203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1445176203},{"type":"v1","addr":"172.21.15.156:6839","nonce":1445176203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/494821686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":494821686},{"type":"v1","addr":"172.21.15.148:6839","nonce":494821686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T14:38:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.012+0000 7f2c2b0e4700 1 -- 172.21.15.32:0/3520360785 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2c140a1070 msgr2=0x7f2c140a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.012+0000 7f2c2b0e4700 1 --2- 172.21.15.32:0/3520360785 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2c140a1070 0x7f2c140a3530 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2c1800b580 tx=0x7f2c18009f10 comp rx=0 tx=0).stop 2023-09-30T14:38:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.012+0000 7f2c2b0e4700 1 -- 172.21.15.32:0/3520360785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c24101900 msgr2=0x7f2c2419e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.012+0000 7f2c2b0e4700 1 --2- 172.21.15.32:0/3520360785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c24101900 0x7f2c2419e100 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c007ee0 tx=0x7f2c1c00d3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.012+0000 7f2c2b0e4700 1 -- 172.21.15.32:0/3520360785 shutdown_connections 2023-09-30T14:38:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.012+0000 7f2c2b0e4700 1 --2- 172.21.15.32:0/3520360785 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2c140a1070 0x7f2c140a3530 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.012+0000 7f2c2b0e4700 1 --2- 172.21.15.32:0/3520360785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c24101900 0x7f2c2419e100 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.012+0000 7f2c2b0e4700 1 --2- 172.21.15.32:0/3520360785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c240ffd70 0x7f2c24196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.012+0000 7f2c2b0e4700 1 --2- 172.21.15.32:0/3520360785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c24100f60 0x7f2c24197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.012+0000 7f2c2b0e4700 1 -- 172.21.15.32:0/3520360785 >> 172.21.15.32:0/3520360785 conn(0x7f2c240fb2e0 msgr2=0x7f2c240691f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.012+0000 7f2c2b0e4700 1 -- 172.21.15.32:0/3520360785 shutdown_connections 2023-09-30T14:38:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.012+0000 7f2c2b0e4700 1 -- 172.21.15.32:0/3520360785 wait complete. 2023-09-30T14:38:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 22 2023-09-30T14:38:39.026 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-09-30T14:38:39.026 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-09-30T14:38:39.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T14:38:39.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.151+0000 7fb2bd7e9700 1 -- 172.21.15.32:0/1402296511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2b8100900 msgr2=0x7fb2b81081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:39.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.151+0000 7fb2bd7e9700 1 --2- 172.21.15.32:0/1402296511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2b8100900 0x7fb2b81081a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000b0a0 tx=0x7fb2b000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.151+0000 7fb2bd7e9700 1 -- 172.21.15.32:0/1402296511 shutdown_connections 2023-09-30T14:38:39.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.151+0000 7fb2bd7e9700 1 --2- 172.21.15.32:0/1402296511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2b8100900 0x7fb2b81081a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.151+0000 7fb2bd7e9700 1 --2- 172.21.15.32:0/1402296511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2b81012f0 0x7fb2b8101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.151+0000 7fb2bd7e9700 1 --2- 172.21.15.32:0/1402296511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2b80fff40 0x7fb2b81003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.151+0000 7fb2bd7e9700 1 -- 172.21.15.32:0/1402296511 >> 172.21.15.32:0/1402296511 conn(0x7fb2b80faa30 msgr2=0x7fb2b80fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.151+0000 7fb2bd7e9700 1 -- 172.21.15.32:0/1402296511 shutdown_connections 2023-09-30T14:38:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.151+0000 7fb2bd7e9700 1 -- 172.21.15.32:0/1402296511 wait complete. 2023-09-30T14:38:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.152+0000 7fb2bd7e9700 1 Processor -- start 2023-09-30T14:38:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.152+0000 7fb2bd7e9700 1 -- start start 2023-09-30T14:38:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.152+0000 7fb2bd7e9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2b80fff40 0x7fb2b8196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.152+0000 7fb2bd7e9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2b8100900 0x7fb2b81970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.152+0000 7fb2bd7e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2b81012f0 0x7fb2b819e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.152+0000 7fb2bd7e9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b819e710 con 0x7fb2b81012f0 2023-09-30T14:38:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.152+0000 7fb2bd7e9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b819e850 con 0x7fb2b80fff40 2023-09-30T14:38:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.152+0000 7fb2bd7e9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b819e9e0 con 0x7fb2b8100900 2023-09-30T14:38:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.152+0000 7fb2b77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2b81012f0 0x7fb2b819e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.152+0000 7fb2b67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2b8100900 0x7fb2b81970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.152+0000 7fb2b77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2b81012f0 0x7fb2b819e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35226/0 (socket says 172.21.15.32:35226) 2023-09-30T14:38:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.152+0000 7fb2b77fe700 1 -- 172.21.15.32:0/1016272334 learned_addr learned my addr 172.21.15.32:0/1016272334 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.152+0000 7fb2b6ffd700 1 --2- 172.21.15.32:0/1016272334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2b80fff40 0x7fb2b8196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.153+0000 7fb2b77fe700 1 -- 172.21.15.32:0/1016272334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2b8100900 msgr2=0x7fb2b81970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.153+0000 7fb2b77fe700 1 --2- 172.21.15.32:0/1016272334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2b8100900 0x7fb2b81970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.153+0000 7fb2b77fe700 1 -- 172.21.15.32:0/1016272334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2b80fff40 msgr2=0x7fb2b8196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.153+0000 7fb2b77fe700 1 --2- 172.21.15.32:0/1016272334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2b80fff40 0x7fb2b8196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.153+0000 7fb2b77fe700 1 -- 172.21.15.32:0/1016272334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b0009d00 con 0x7fb2b81012f0 2023-09-30T14:38:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.153+0000 7fb2b77fe700 1 --2- 172.21.15.32:0/1016272334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2b81012f0 0x7fb2b819e140 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000b780 tx=0x7fb2b000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.153+0000 7fb2affff700 1 -- 172.21.15.32:0/1016272334 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b000e040 con 0x7fb2b81012f0 2023-09-30T14:38:39.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.153+0000 7fb2affff700 1 -- 172.21.15.32:0/1016272334 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2b0004e60 con 0x7fb2b81012f0 2023-09-30T14:38:39.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.153+0000 7fb2bd7e9700 1 -- 172.21.15.32:0/1016272334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b819ec60 con 0x7fb2b81012f0 2023-09-30T14:38:39.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.153+0000 7fb2affff700 1 -- 172.21.15.32:0/1016272334 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b001b6b0 con 0x7fb2b81012f0 2023-09-30T14:38:39.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.153+0000 7fb2bd7e9700 1 -- 172.21.15.32:0/1016272334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b819f130 con 0x7fb2b81012f0 2023-09-30T14:38:39.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.154+0000 7fb2bd7e9700 1 -- 172.21.15.32:0/1016272334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b8107d70 con 0x7fb2b81012f0 2023-09-30T14:38:39.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.156+0000 7fb2affff700 1 -- 172.21.15.32:0/1016272334 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fb2b001b810 con 0x7fb2b81012f0 2023-09-30T14:38:39.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.156+0000 7fb2affff700 1 --2- 172.21.15.32:0/1016272334 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb2a40a10f0 0x7fb2a40a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:39.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.156+0000 7fb2affff700 1 -- 172.21.15.32:0/1016272334 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb2b00ce3d0 con 0x7fb2b81012f0 2023-09-30T14:38:39.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.157+0000 7fb2b6ffd700 1 --2- 172.21.15.32:0/1016272334 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb2a40a10f0 0x7fb2a40a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:39.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.157+0000 7fb2b6ffd700 1 --2- 172.21.15.32:0/1016272334 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb2a40a10f0 0x7fb2a40a35b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8005950 tx=0x7fb2a800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:39.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.160+0000 7fb2affff700 1 -- 172.21.15.32:0/1016272334 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fb2b0097bb0 con 0x7fb2b81012f0 2023-09-30T14:38:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.359+0000 7fb2bd7e9700 1 -- 172.21.15.32:0/1016272334 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb2b802d270 con 0x7fb2a40a10f0 2023-09-30T14:38:39.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.371+0000 7fb2affff700 1 -- 172.21.15.32:0/1016272334 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344458 (secure 0 0 0) 0x7fb2b802d270 con 0x7fb2a40a10f0 2023-09-30T14:38:39.376 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:39.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.375+0000 7fb2bd7e9700 1 -- 172.21.15.32:0/1016272334 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb2a40a10f0 msgr2=0x7fb2a40a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:39.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.375+0000 7fb2bd7e9700 1 --2- 172.21.15.32:0/1016272334 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb2a40a10f0 0x7fb2a40a35b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8005950 tx=0x7fb2a800b410 comp rx=0 tx=0).stop 2023-09-30T14:38:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.375+0000 7fb2bd7e9700 1 -- 172.21.15.32:0/1016272334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2b81012f0 msgr2=0x7fb2b819e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.375+0000 7fb2bd7e9700 1 --2- 172.21.15.32:0/1016272334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2b81012f0 0x7fb2b819e140 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000b780 tx=0x7fb2b000bd00 comp rx=0 tx=0).stop 2023-09-30T14:38:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.375+0000 7fb2bd7e9700 1 -- 172.21.15.32:0/1016272334 shutdown_connections 2023-09-30T14:38:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.375+0000 7fb2bd7e9700 1 --2- 172.21.15.32:0/1016272334 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb2a40a10f0 0x7fb2a40a35b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.375+0000 7fb2bd7e9700 1 --2- 172.21.15.32:0/1016272334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2b81012f0 0x7fb2b819e140 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.375+0000 7fb2bd7e9700 1 --2- 172.21.15.32:0/1016272334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2b80fff40 0x7fb2b8196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.375+0000 7fb2bd7e9700 1 --2- 172.21.15.32:0/1016272334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2b8100900 0x7fb2b81970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.375+0000 7fb2bd7e9700 1 -- 172.21.15.32:0/1016272334 >> 172.21.15.32:0/1016272334 conn(0x7fb2b80faa30 msgr2=0x7fb2b80fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.375+0000 7fb2bd7e9700 1 -- 172.21.15.32:0/1016272334 shutdown_connections 2023-09-30T14:38:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.375+0000 7fb2bd7e9700 1 -- 172.21.15.32:0/1016272334 wait complete. 2023-09-30T14:38:39.379 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-09-30T14:38:39.387 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2023-09-30T14:38:37.793844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648932,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":617,"num_read_kb":419,"num_write":332,"num_write_kb":3039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1182052,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337096,"kb_used_data":12832,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604728,"statfs":{"total":1151940427776,"available":1151595241472,"internally_reserved":0,"allocated":13139968,"data_stored":7252214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003027"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197844+0000","last_change":"2023-09-30T14:37:11.988122+0000","last_active":"2023-09-30T14:38:12.197844+0000","last_peered":"2023-09-30T14:38:12.197844+0000","last_clean":"2023-09-30T14:38:12.197844+0000","last_became_active":"2023-09-30T14:37:11.986770+0000","last_became_peered":"2023-09-30T14:37:11.986770+0000","last_unstale":"2023-09-30T14:38:12.197844+0000","last_undegraded":"2023-09-30T14:38:12.197844+0000","last_fullsized":"2023-09-30T14:38:12.197844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:46:29.655309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197069+0000","last_change":"2023-09-30T14:37:11.987953+0000","last_active":"2023-09-30T14:38:12.197069+0000","last_peered":"2023-09-30T14:38:12.197069+0000","last_clean":"2023-09-30T14:38:12.197069+0000","last_became_active":"2023-09-30T14:37:11.987248+0000","last_became_peered":"2023-09-30T14:37:11.987248+0000","last_unstale":"2023-09-30T14:38:12.197069+0000","last_undegraded":"2023-09-30T14:38:12.197069+0000","last_fullsized":"2023-09-30T14:38:12.197069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:12:59.780089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197196+0000","last_change":"2023-09-30T14:37:11.987570+0000","last_active":"2023-09-30T14:38:12.197196+0000","last_peered":"2023-09-30T14:38:12.197196+0000","last_clean":"2023-09-30T14:38:12.197196+0000","last_became_active":"2023-09-30T14:37:11.987064+0000","last_became_peered":"2023-09-30T14:37:11.987064+0000","last_unstale":"2023-09-30T14:38:12.197196+0000","last_undegraded":"2023-09-30T14:38:12.197196+0000","last_fullsized":"2023-09-30T14:38:12.197196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:26:16.674605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197162+0000","last_change":"2023-09-30T14:37:11.990161+0000","last_active":"2023-09-30T14:38:12.197162+0000","last_peered":"2023-09-30T14:38:12.197162+0000","last_clean":"2023-09-30T14:38:12.197162+0000","last_became_active":"2023-09-30T14:37:11.989580+0000","last_became_peered":"2023-09-30T14:37:11.989580+0000","last_unstale":"2023-09-30T14:38:12.197162+0000","last_undegraded":"2023-09-30T14:38:12.197162+0000","last_fullsized":"2023-09-30T14:38:12.197162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:13:06.986864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199326+0000","last_change":"2023-09-30T14:37:11.993902+0000","last_active":"2023-09-30T14:38:12.199326+0000","last_peered":"2023-09-30T14:38:12.199326+0000","last_clean":"2023-09-30T14:38:12.199326+0000","last_became_active":"2023-09-30T14:37:11.993622+0000","last_became_peered":"2023-09-30T14:37:11.993622+0000","last_unstale":"2023-09-30T14:38:12.199326+0000","last_undegraded":"2023-09-30T14:38:12.199326+0000","last_fullsized":"2023-09-30T14:38:12.199326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:09:49.252823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195466+0000","last_change":"2023-09-30T14:37:11.997282+0000","last_active":"2023-09-30T14:38:12.195466+0000","last_peered":"2023-09-30T14:38:12.195466+0000","last_clean":"2023-09-30T14:38:12.195466+0000","last_became_active":"2023-09-30T14:37:11.996937+0000","last_became_peered":"2023-09-30T14:37:11.996937+0000","last_unstale":"2023-09-30T14:38:12.195466+0000","last_undegraded":"2023-09-30T14:38:12.195466+0000","last_fullsized":"2023-09-30T14:38:12.195466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:39:27.351772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199172+0000","last_change":"2023-09-30T14:37:11.997408+0000","last_active":"2023-09-30T14:38:12.199172+0000","last_peered":"2023-09-30T14:38:12.199172+0000","last_clean":"2023-09-30T14:38:12.199172+0000","last_became_active":"2023-09-30T14:37:11.997147+0000","last_became_peered":"2023-09-30T14:37:11.997147+0000","last_unstale":"2023-09-30T14:38:12.199172+0000","last_undegraded":"2023-09-30T14:38:12.199172+0000","last_fullsized":"2023-09-30T14:38:12.199172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:21:24.721417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.823924+0000","last_change":"2023-09-30T14:37:11.986172+0000","last_active":"2023-09-30T14:38:12.823924+0000","last_peered":"2023-09-30T14:38:12.823924+0000","last_clean":"2023-09-30T14:38:12.823924+0000","last_became_active":"2023-09-30T14:37:11.985836+0000","last_became_peered":"2023-09-30T14:37:11.985836+0000","last_unstale":"2023-09-30T14:38:12.823924+0000","last_undegraded":"2023-09-30T14:38:12.823924+0000","last_fullsized":"2023-09-30T14:38:12.823924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:35:57.464679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198360+0000","last_change":"2023-09-30T14:37:11.990853+0000","last_active":"2023-09-30T14:38:12.198360+0000","last_peered":"2023-09-30T14:38:12.198360+0000","last_clean":"2023-09-30T14:38:12.198360+0000","last_became_active":"2023-09-30T14:37:11.990321+0000","last_became_peered":"2023-09-30T14:37:11.990321+0000","last_unstale":"2023-09-30T14:38:12.198360+0000","last_undegraded":"2023-09-30T14:38:12.198360+0000","last_fullsized":"2023-09-30T14:38:12.198360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:32:11.409927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.202453+0000","last_change":"2023-09-30T14:38:13.202453+0000","last_active":"2023-09-30T14:38:13.202453+0000","last_peered":"2023-09-30T14:38:13.202453+0000","last_clean":"2023-09-30T14:38:13.202453+0000","last_became_active":"2023-09-30T14:38:13.202023+0000","last_became_peered":"2023-09-30T14:38:13.202023+0000","last_unstale":"2023-09-30T14:38:13.202453+0000","last_undegraded":"2023-09-30T14:38:13.202453+0000","last_fullsized":"2023-09-30T14:38:13.202453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:50:40.737940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198974+0000","last_change":"2023-09-30T14:37:11.992213+0000","last_active":"2023-09-30T14:38:12.198974+0000","last_peered":"2023-09-30T14:38:12.198974+0000","last_clean":"2023-09-30T14:38:12.198974+0000","last_became_active":"2023-09-30T14:37:11.991868+0000","last_became_peered":"2023-09-30T14:37:11.991868+0000","last_unstale":"2023-09-30T14:38:12.198974+0000","last_undegraded":"2023-09-30T14:38:12.198974+0000","last_fullsized":"2023-09-30T14:38:12.198974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:20:06.061978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197227+0000","last_change":"2023-09-30T14:37:11.990043+0000","last_active":"2023-09-30T14:38:12.197227+0000","last_peered":"2023-09-30T14:38:12.197227+0000","last_clean":"2023-09-30T14:38:12.197227+0000","last_became_active":"2023-09-30T14:37:11.988243+0000","last_became_peered":"2023-09-30T14:37:11.988243+0000","last_unstale":"2023-09-30T14:38:12.197227+0000","last_undegraded":"2023-09-30T14:38:12.197227+0000","last_fullsized":"2023-09-30T14:38:12.197227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:17:58.946159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194482+0000","last_change":"2023-09-30T14:37:11.990361+0000","last_active":"2023-09-30T14:38:12.194482+0000","last_peered":"2023-09-30T14:38:12.194482+0000","last_clean":"2023-09-30T14:38:12.194482+0000","last_became_active":"2023-09-30T14:37:11.990080+0000","last_became_peered":"2023-09-30T14:37:11.990080+0000","last_unstale":"2023-09-30T14:38:12.194482+0000","last_undegraded":"2023-09-30T14:38:12.194482+0000","last_fullsized":"2023-09-30T14:38:12.194482+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:27:08.830419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197518+0000","last_change":"2023-09-30T14:37:11.988119+0000","last_active":"2023-09-30T14:38:12.197518+0000","last_peered":"2023-09-30T14:38:12.197518+0000","last_clean":"2023-09-30T14:38:12.197518+0000","last_became_active":"2023-09-30T14:37:11.987704+0000","last_became_peered":"2023-09-30T14:37:11.987704+0000","last_unstale":"2023-09-30T14:38:12.197518+0000","last_undegraded":"2023-09-30T14:38:12.197518+0000","last_fullsized":"2023-09-30T14:38:12.197518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:47:23.318982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196560+0000","last_change":"2023-09-30T14:37:11.985403+0000","last_active":"2023-09-30T14:38:12.196560+0000","last_peered":"2023-09-30T14:38:12.196560+0000","last_clean":"2023-09-30T14:38:12.196560+0000","last_became_active":"2023-09-30T14:37:11.984818+0000","last_became_peered":"2023-09-30T14:37:11.984818+0000","last_unstale":"2023-09-30T14:38:12.196560+0000","last_undegraded":"2023-09-30T14:38:12.196560+0000","last_fullsized":"2023-09-30T14:38:12.196560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:10:43.592505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198212+0000","last_change":"2023-09-30T14:37:11.989500+0000","last_active":"2023-09-30T14:38:12.198212+0000","last_peered":"2023-09-30T14:38:12.198212+0000","last_clean":"2023-09-30T14:38:12.198212+0000","last_became_active":"2023-09-30T14:37:11.989186+0000","last_became_peered":"2023-09-30T14:37:11.989186+0000","last_unstale":"2023-09-30T14:38:12.198212+0000","last_undegraded":"2023-09-30T14:38:12.198212+0000","last_fullsized":"2023-09-30T14:38:12.198212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:14:35.497319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197968+0000","last_change":"2023-09-30T14:37:11.987217+0000","last_active":"2023-09-30T14:38:12.197968+0000","last_peered":"2023-09-30T14:38:12.197968+0000","last_clean":"2023-09-30T14:38:12.197968+0000","last_became_active":"2023-09-30T14:37:11.986835+0000","last_became_peered":"2023-09-30T14:37:11.986835+0000","last_unstale":"2023-09-30T14:38:12.197968+0000","last_undegraded":"2023-09-30T14:38:12.197968+0000","last_fullsized":"2023-09-30T14:38:12.197968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:11:19.234768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196774+0000","last_change":"2023-09-30T14:37:11.995224+0000","last_active":"2023-09-30T14:38:12.196774+0000","last_peered":"2023-09-30T14:38:12.196774+0000","last_clean":"2023-09-30T14:38:12.196774+0000","last_became_active":"2023-09-30T14:37:11.994925+0000","last_became_peered":"2023-09-30T14:37:11.994925+0000","last_unstale":"2023-09-30T14:38:12.196774+0000","last_undegraded":"2023-09-30T14:38:12.196774+0000","last_fullsized":"2023-09-30T14:38:12.196774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:08:07.453499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198775+0000","last_change":"2023-09-30T14:37:11.990182+0000","last_active":"2023-09-30T14:38:12.198775+0000","last_peered":"2023-09-30T14:38:12.198775+0000","last_clean":"2023-09-30T14:38:12.198775+0000","last_became_active":"2023-09-30T14:37:11.989862+0000","last_became_peered":"2023-09-30T14:37:11.989862+0000","last_unstale":"2023-09-30T14:38:12.198775+0000","last_undegraded":"2023-09-30T14:38:12.198775+0000","last_fullsized":"2023-09-30T14:38:12.198775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:15:31.713002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.209988+0000","last_change":"2023-09-30T14:38:13.209988+0000","last_active":"2023-09-30T14:38:13.209988+0000","last_peered":"2023-09-30T14:38:13.209988+0000","last_clean":"2023-09-30T14:38:13.209988+0000","last_became_active":"2023-09-30T14:38:13.202561+0000","last_became_peered":"2023-09-30T14:38:13.202561+0000","last_unstale":"2023-09-30T14:38:13.209988+0000","last_undegraded":"2023-09-30T14:38:13.209988+0000","last_fullsized":"2023-09-30T14:38:13.209988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:57:36.214608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824419+0000","last_change":"2023-09-30T14:37:11.994584+0000","last_active":"2023-09-30T14:38:12.824419+0000","last_peered":"2023-09-30T14:38:12.824419+0000","last_clean":"2023-09-30T14:38:12.824419+0000","last_became_active":"2023-09-30T14:37:11.992991+0000","last_became_peered":"2023-09-30T14:37:11.992991+0000","last_unstale":"2023-09-30T14:38:12.824419+0000","last_undegraded":"2023-09-30T14:38:12.824419+0000","last_fullsized":"2023-09-30T14:38:12.824419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:39:50.995717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824532+0000","last_change":"2023-09-30T14:37:11.020653+0000","last_active":"2023-09-30T14:38:12.824532+0000","last_peered":"2023-09-30T14:38:12.824532+0000","last_clean":"2023-09-30T14:38:12.824532+0000","last_became_active":"2023-09-30T14:37:11.017621+0000","last_became_peered":"2023-09-30T14:37:11.017621+0000","last_unstale":"2023-09-30T14:38:12.824532+0000","last_undegraded":"2023-09-30T14:38:12.824532+0000","last_fullsized":"2023-09-30T14:38:12.824532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:26:59.395278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196664+0000","last_change":"2023-09-30T14:37:11.999881+0000","last_active":"2023-09-30T14:38:12.196664+0000","last_peered":"2023-09-30T14:38:12.196664+0000","last_clean":"2023-09-30T14:38:12.196664+0000","last_became_active":"2023-09-30T14:37:11.999614+0000","last_became_peered":"2023-09-30T14:37:11.999614+0000","last_unstale":"2023-09-30T14:38:12.196664+0000","last_undegraded":"2023-09-30T14:38:12.196664+0000","last_fullsized":"2023-09-30T14:38:12.196664+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:59:44.198361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200580+0000","last_change":"2023-09-30T14:37:11.027187+0000","last_active":"2023-09-30T14:38:12.200580+0000","last_peered":"2023-09-30T14:38:12.200580+0000","last_clean":"2023-09-30T14:38:12.200580+0000","last_became_active":"2023-09-30T14:37:11.026882+0000","last_became_peered":"2023-09-30T14:37:11.026882+0000","last_unstale":"2023-09-30T14:38:12.200580+0000","last_undegraded":"2023-09-30T14:38:12.200580+0000","last_fullsized":"2023-09-30T14:38:12.200580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:17:11.824002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198427+0000","last_change":"2023-09-30T14:37:11.986979+0000","last_active":"2023-09-30T14:38:12.198427+0000","last_peered":"2023-09-30T14:38:12.198427+0000","last_clean":"2023-09-30T14:38:12.198427+0000","last_became_active":"2023-09-30T14:37:11.986705+0000","last_became_peered":"2023-09-30T14:37:11.986705+0000","last_unstale":"2023-09-30T14:38:12.198427+0000","last_undegraded":"2023-09-30T14:38:12.198427+0000","last_fullsized":"2023-09-30T14:38:12.198427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:31:40.319747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199932+0000","last_change":"2023-09-30T14:37:11.022313+0000","last_active":"2023-09-30T14:38:12.199932+0000","last_peered":"2023-09-30T14:38:12.199932+0000","last_clean":"2023-09-30T14:38:12.199932+0000","last_became_active":"2023-09-30T14:37:11.021949+0000","last_became_peered":"2023-09-30T14:37:11.021949+0000","last_unstale":"2023-09-30T14:38:12.199932+0000","last_undegraded":"2023-09-30T14:38:12.199932+0000","last_fullsized":"2023-09-30T14:38:12.199932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:30:26.913666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.202706+0000","last_change":"2023-09-30T14:38:13.202706+0000","last_active":"2023-09-30T14:38:13.202706+0000","last_peered":"2023-09-30T14:38:13.202706+0000","last_clean":"2023-09-30T14:38:13.202706+0000","last_became_active":"2023-09-30T14:38:13.202370+0000","last_became_peered":"2023-09-30T14:38:13.202370+0000","last_unstale":"2023-09-30T14:38:13.202706+0000","last_undegraded":"2023-09-30T14:38:13.202706+0000","last_fullsized":"2023-09-30T14:38:13.202706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:35:21.244531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":117,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:36.239380+0000","last_change":"2023-09-30T14:38:13.206650+0000","last_active":"2023-09-30T14:38:36.239380+0000","last_peered":"2023-09-30T14:38:36.239380+0000","last_clean":"2023-09-30T14:38:36.239380+0000","last_became_active":"2023-09-30T14:38:13.203337+0000","last_became_peered":"2023-09-30T14:38:13.203337+0000","last_unstale":"2023-09-30T14:38:36.239380+0000","last_undegraded":"2023-09-30T14:38:36.239380+0000","last_fullsized":"2023-09-30T14:38:36.239380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:08:37.996166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199571+0000","last_change":"2023-09-30T14:37:11.993074+0000","last_active":"2023-09-30T14:38:12.199571+0000","last_peered":"2023-09-30T14:38:12.199571+0000","last_clean":"2023-09-30T14:38:12.199571+0000","last_became_active":"2023-09-30T14:37:11.992784+0000","last_became_peered":"2023-09-30T14:37:11.992784+0000","last_unstale":"2023-09-30T14:38:12.199571+0000","last_undegraded":"2023-09-30T14:38:12.199571+0000","last_fullsized":"2023-09-30T14:38:12.199571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:28:18.454389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195465+0000","last_change":"2023-09-30T14:37:11.022677+0000","last_active":"2023-09-30T14:38:12.195465+0000","last_peered":"2023-09-30T14:38:12.195465+0000","last_clean":"2023-09-30T14:38:12.195465+0000","last_became_active":"2023-09-30T14:37:11.022357+0000","last_became_peered":"2023-09-30T14:37:11.022357+0000","last_unstale":"2023-09-30T14:38:12.195465+0000","last_undegraded":"2023-09-30T14:38:12.195465+0000","last_fullsized":"2023-09-30T14:38:12.195465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:00:15.500968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194862+0000","last_change":"2023-09-30T14:37:11.992358+0000","last_active":"2023-09-30T14:38:12.194862+0000","last_peered":"2023-09-30T14:38:12.194862+0000","last_clean":"2023-09-30T14:38:12.194862+0000","last_became_active":"2023-09-30T14:37:11.992094+0000","last_became_peered":"2023-09-30T14:37:11.992094+0000","last_unstale":"2023-09-30T14:38:12.194862+0000","last_undegraded":"2023-09-30T14:38:12.194862+0000","last_fullsized":"2023-09-30T14:38:12.194862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:59:36.817138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194237+0000","last_change":"2023-09-30T14:37:11.030961+0000","last_active":"2023-09-30T14:38:12.194237+0000","last_peered":"2023-09-30T14:38:12.194237+0000","last_clean":"2023-09-30T14:38:12.194237+0000","last_became_active":"2023-09-30T14:37:11.030662+0000","last_became_peered":"2023-09-30T14:37:11.030662+0000","last_unstale":"2023-09-30T14:38:12.194237+0000","last_undegraded":"2023-09-30T14:38:12.194237+0000","last_fullsized":"2023-09-30T14:38:12.194237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:35:13.313382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824002+0000","last_change":"2023-09-30T14:37:11.989039+0000","last_active":"2023-09-30T14:38:12.824002+0000","last_peered":"2023-09-30T14:38:12.824002+0000","last_clean":"2023-09-30T14:38:12.824002+0000","last_became_active":"2023-09-30T14:37:11.988594+0000","last_became_peered":"2023-09-30T14:37:11.988594+0000","last_unstale":"2023-09-30T14:38:12.824002+0000","last_undegraded":"2023-09-30T14:38:12.824002+0000","last_fullsized":"2023-09-30T14:38:12.824002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:52:58.146681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194495+0000","last_change":"2023-09-30T14:37:11.026545+0000","last_active":"2023-09-30T14:38:12.194495+0000","last_peered":"2023-09-30T14:38:12.194495+0000","last_clean":"2023-09-30T14:38:12.194495+0000","last_became_active":"2023-09-30T14:37:11.026063+0000","last_became_peered":"2023-09-30T14:37:11.026063+0000","last_unstale":"2023-09-30T14:38:12.194495+0000","last_undegraded":"2023-09-30T14:38:12.194495+0000","last_fullsized":"2023-09-30T14:38:12.194495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:15:34.020336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196143+0000","last_change":"2023-09-30T14:37:11.991845+0000","last_active":"2023-09-30T14:38:12.196143+0000","last_peered":"2023-09-30T14:38:12.196143+0000","last_clean":"2023-09-30T14:38:12.196143+0000","last_became_active":"2023-09-30T14:37:11.991560+0000","last_became_peered":"2023-09-30T14:37:11.991560+0000","last_unstale":"2023-09-30T14:38:12.196143+0000","last_undegraded":"2023-09-30T14:38:12.196143+0000","last_fullsized":"2023-09-30T14:38:12.196143+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:27:39.653882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":97,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:34.235519+0000","last_change":"2023-09-30T14:37:11.028192+0000","last_active":"2023-09-30T14:38:34.235519+0000","last_peered":"2023-09-30T14:38:34.235519+0000","last_clean":"2023-09-30T14:38:34.235519+0000","last_became_active":"2023-09-30T14:37:11.027902+0000","last_became_peered":"2023-09-30T14:37:11.027902+0000","last_unstale":"2023-09-30T14:38:34.235519+0000","last_undegraded":"2023-09-30T14:38:34.235519+0000","last_fullsized":"2023-09-30T14:38:34.235519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:21:30.267559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198607+0000","last_change":"2023-09-30T14:37:11.997318+0000","last_active":"2023-09-30T14:38:12.198607+0000","last_peered":"2023-09-30T14:38:12.198607+0000","last_clean":"2023-09-30T14:38:12.198607+0000","last_became_active":"2023-09-30T14:37:11.996988+0000","last_became_peered":"2023-09-30T14:37:11.996988+0000","last_unstale":"2023-09-30T14:38:12.198607+0000","last_undegraded":"2023-09-30T14:38:12.198607+0000","last_fullsized":"2023-09-30T14:38:12.198607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:57:21.510952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.259803+0000","last_change":"2023-09-30T14:38:13.259803+0000","last_active":"2023-09-30T14:38:13.259803+0000","last_peered":"2023-09-30T14:38:13.259803+0000","last_clean":"2023-09-30T14:38:13.259803+0000","last_became_active":"2023-09-30T14:38:13.257187+0000","last_became_peered":"2023-09-30T14:38:13.257187+0000","last_unstale":"2023-09-30T14:38:13.259803+0000","last_undegraded":"2023-09-30T14:38:13.259803+0000","last_fullsized":"2023-09-30T14:38:13.259803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:37:41.453519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196748+0000","last_change":"2023-09-30T14:37:11.989338+0000","last_active":"2023-09-30T14:38:12.196748+0000","last_peered":"2023-09-30T14:38:12.196748+0000","last_clean":"2023-09-30T14:38:12.196748+0000","last_became_active":"2023-09-30T14:37:11.989045+0000","last_became_peered":"2023-09-30T14:37:11.989045+0000","last_unstale":"2023-09-30T14:38:12.196748+0000","last_undegraded":"2023-09-30T14:38:12.196748+0000","last_fullsized":"2023-09-30T14:38:12.196748+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:58:59.359684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195371+0000","last_change":"2023-09-30T14:37:11.024709+0000","last_active":"2023-09-30T14:38:12.195371+0000","last_peered":"2023-09-30T14:38:12.195371+0000","last_clean":"2023-09-30T14:38:12.195371+0000","last_became_active":"2023-09-30T14:37:11.024344+0000","last_became_peered":"2023-09-30T14:37:11.024344+0000","last_unstale":"2023-09-30T14:38:12.195371+0000","last_undegraded":"2023-09-30T14:38:12.195371+0000","last_fullsized":"2023-09-30T14:38:12.195371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:17:49.508396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198298+0000","last_change":"2023-09-30T14:37:11.989797+0000","last_active":"2023-09-30T14:38:12.198298+0000","last_peered":"2023-09-30T14:38:12.198298+0000","last_clean":"2023-09-30T14:38:12.198298+0000","last_became_active":"2023-09-30T14:37:11.989482+0000","last_became_peered":"2023-09-30T14:37:11.989482+0000","last_unstale":"2023-09-30T14:38:12.198298+0000","last_undegraded":"2023-09-30T14:38:12.198298+0000","last_fullsized":"2023-09-30T14:38:12.198298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:33:20.437706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'26","reported_seq":206,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:35.237410+0000","last_change":"2023-09-30T14:37:11.027322+0000","last_active":"2023-09-30T14:38:35.237410+0000","last_peered":"2023-09-30T14:38:35.237410+0000","last_clean":"2023-09-30T14:38:35.237410+0000","last_became_active":"2023-09-30T14:37:11.027007+0000","last_became_peered":"2023-09-30T14:37:11.027007+0000","last_unstale":"2023-09-30T14:38:35.237410+0000","last_undegraded":"2023-09-30T14:38:35.237410+0000","last_fullsized":"2023-09-30T14:38:35.237410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:47:22.919317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54708,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":165,"num_read_kb":135,"num_write":27,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200357+0000","last_change":"2023-09-30T14:37:11.988469+0000","last_active":"2023-09-30T14:38:12.200357+0000","last_peered":"2023-09-30T14:38:12.200357+0000","last_clean":"2023-09-30T14:38:12.200357+0000","last_became_active":"2023-09-30T14:37:11.988118+0000","last_became_peered":"2023-09-30T14:37:11.988118+0000","last_unstale":"2023-09-30T14:38:12.200357+0000","last_undegraded":"2023-09-30T14:38:12.200357+0000","last_fullsized":"2023-09-30T14:38:12.200357+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:11:56.750690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198520+0000","last_change":"2023-09-30T14:37:11.024345+0000","last_active":"2023-09-30T14:38:12.198520+0000","last_peered":"2023-09-30T14:38:12.198520+0000","last_clean":"2023-09-30T14:38:12.198520+0000","last_became_active":"2023-09-30T14:37:11.024039+0000","last_became_peered":"2023-09-30T14:37:11.024039+0000","last_unstale":"2023-09-30T14:38:12.198520+0000","last_undegraded":"2023-09-30T14:38:12.198520+0000","last_fullsized":"2023-09-30T14:38:12.198520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:48:32.872814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194273+0000","last_change":"2023-09-30T14:37:11.994068+0000","last_active":"2023-09-30T14:38:12.194273+0000","last_peered":"2023-09-30T14:38:12.194273+0000","last_clean":"2023-09-30T14:38:12.194273+0000","last_became_active":"2023-09-30T14:37:11.993793+0000","last_became_peered":"2023-09-30T14:37:11.993793+0000","last_unstale":"2023-09-30T14:38:12.194273+0000","last_undegraded":"2023-09-30T14:38:12.194273+0000","last_fullsized":"2023-09-30T14:38:12.194273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:51:28.071414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194699+0000","last_change":"2023-09-30T14:37:11.025622+0000","last_active":"2023-09-30T14:38:12.194699+0000","last_peered":"2023-09-30T14:38:12.194699+0000","last_clean":"2023-09-30T14:38:12.194699+0000","last_became_active":"2023-09-30T14:37:11.025338+0000","last_became_peered":"2023-09-30T14:37:11.025338+0000","last_unstale":"2023-09-30T14:38:12.194699+0000","last_undegraded":"2023-09-30T14:38:12.194699+0000","last_fullsized":"2023-09-30T14:38:12.194699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:32:03.166296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200744+0000","last_change":"2023-09-30T14:37:11.990678+0000","last_active":"2023-09-30T14:38:12.200744+0000","last_peered":"2023-09-30T14:38:12.200744+0000","last_clean":"2023-09-30T14:38:12.200744+0000","last_became_active":"2023-09-30T14:37:11.990331+0000","last_became_peered":"2023-09-30T14:37:11.990331+0000","last_unstale":"2023-09-30T14:38:12.200744+0000","last_undegraded":"2023-09-30T14:38:12.200744+0000","last_fullsized":"2023-09-30T14:38:12.200744+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:57:06.296415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":96,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:31.092801+0000","last_change":"2023-09-30T14:37:11.029501+0000","last_active":"2023-09-30T14:38:31.092801+0000","last_peered":"2023-09-30T14:38:31.092801+0000","last_clean":"2023-09-30T14:38:31.092801+0000","last_became_active":"2023-09-30T14:37:11.029207+0000","last_became_peered":"2023-09-30T14:37:11.029207+0000","last_unstale":"2023-09-30T14:38:31.092801+0000","last_undegraded":"2023-09-30T14:38:31.092801+0000","last_fullsized":"2023-09-30T14:38:31.092801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:48:53.145952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198725+0000","last_change":"2023-09-30T14:37:11.993052+0000","last_active":"2023-09-30T14:38:12.198725+0000","last_peered":"2023-09-30T14:38:12.198725+0000","last_clean":"2023-09-30T14:38:12.198725+0000","last_became_active":"2023-09-30T14:37:11.992762+0000","last_became_peered":"2023-09-30T14:37:11.992762+0000","last_unstale":"2023-09-30T14:38:12.198725+0000","last_undegraded":"2023-09-30T14:38:12.198725+0000","last_fullsized":"2023-09-30T14:38:12.198725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:09:16.440026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.214212+0000","last_change":"2023-09-30T14:38:13.213859+0000","last_active":"2023-09-30T14:38:13.214212+0000","last_peered":"2023-09-30T14:38:13.214212+0000","last_clean":"2023-09-30T14:38:13.214212+0000","last_became_active":"2023-09-30T14:38:13.200010+0000","last_became_peered":"2023-09-30T14:38:13.200010+0000","last_unstale":"2023-09-30T14:38:13.214212+0000","last_undegraded":"2023-09-30T14:38:13.214212+0000","last_fullsized":"2023-09-30T14:38:13.214212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:08:48.668186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194503+0000","last_change":"2023-09-30T14:37:11.989592+0000","last_active":"2023-09-30T14:38:12.194503+0000","last_peered":"2023-09-30T14:38:12.194503+0000","last_clean":"2023-09-30T14:38:12.194503+0000","last_became_active":"2023-09-30T14:37:11.989290+0000","last_became_peered":"2023-09-30T14:37:11.989290+0000","last_unstale":"2023-09-30T14:38:12.194503+0000","last_undegraded":"2023-09-30T14:38:12.194503+0000","last_fullsized":"2023-09-30T14:38:12.194503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:11:33.974823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.201778+0000","last_change":"2023-09-30T14:37:11.028919+0000","last_active":"2023-09-30T14:38:13.201778+0000","last_peered":"2023-09-30T14:38:13.201778+0000","last_clean":"2023-09-30T14:38:13.201778+0000","last_became_active":"2023-09-30T14:37:11.028532+0000","last_became_peered":"2023-09-30T14:37:11.028532+0000","last_unstale":"2023-09-30T14:38:13.201778+0000","last_undegraded":"2023-09-30T14:38:13.201778+0000","last_fullsized":"2023-09-30T14:38:13.201778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:15:42.675409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197404+0000","last_change":"2023-09-30T14:37:11.980611+0000","last_active":"2023-09-30T14:38:12.197404+0000","last_peered":"2023-09-30T14:38:12.197404+0000","last_clean":"2023-09-30T14:38:12.197404+0000","last_became_active":"2023-09-30T14:37:11.980288+0000","last_became_peered":"2023-09-30T14:37:11.980288+0000","last_unstale":"2023-09-30T14:38:12.197404+0000","last_undegraded":"2023-09-30T14:38:12.197404+0000","last_fullsized":"2023-09-30T14:38:12.197404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:38:49.270976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196478+0000","last_change":"2023-09-30T14:37:11.025133+0000","last_active":"2023-09-30T14:38:12.196478+0000","last_peered":"2023-09-30T14:38:12.196478+0000","last_clean":"2023-09-30T14:38:12.196478+0000","last_became_active":"2023-09-30T14:37:11.024914+0000","last_became_peered":"2023-09-30T14:37:11.024914+0000","last_unstale":"2023-09-30T14:38:12.196478+0000","last_undegraded":"2023-09-30T14:38:12.196478+0000","last_fullsized":"2023-09-30T14:38:12.196478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:23:39.328773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195743+0000","last_change":"2023-09-30T14:37:11.988434+0000","last_active":"2023-09-30T14:38:12.195743+0000","last_peered":"2023-09-30T14:38:12.195743+0000","last_clean":"2023-09-30T14:38:12.195743+0000","last_became_active":"2023-09-30T14:37:11.988156+0000","last_became_peered":"2023-09-30T14:37:11.988156+0000","last_unstale":"2023-09-30T14:38:12.195743+0000","last_undegraded":"2023-09-30T14:38:12.195743+0000","last_fullsized":"2023-09-30T14:38:12.195743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:20:49.420026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198842+0000","last_change":"2023-09-30T14:37:11.026290+0000","last_active":"2023-09-30T14:38:12.198842+0000","last_peered":"2023-09-30T14:38:12.198842+0000","last_clean":"2023-09-30T14:38:12.198842+0000","last_became_active":"2023-09-30T14:37:11.024684+0000","last_became_peered":"2023-09-30T14:37:11.024684+0000","last_unstale":"2023-09-30T14:38:12.198842+0000","last_undegraded":"2023-09-30T14:38:12.198842+0000","last_fullsized":"2023-09-30T14:38:12.198842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:43:06.463243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195851+0000","last_change":"2023-09-30T14:37:11.987989+0000","last_active":"2023-09-30T14:38:12.195851+0000","last_peered":"2023-09-30T14:38:12.195851+0000","last_clean":"2023-09-30T14:38:12.195851+0000","last_became_active":"2023-09-30T14:37:11.987690+0000","last_became_peered":"2023-09-30T14:37:11.987690+0000","last_unstale":"2023-09-30T14:38:12.195851+0000","last_undegraded":"2023-09-30T14:38:12.195851+0000","last_fullsized":"2023-09-30T14:38:12.195851+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:51:25.825871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194553+0000","last_change":"2023-09-30T14:37:11.996852+0000","last_active":"2023-09-30T14:38:12.194553+0000","last_peered":"2023-09-30T14:38:12.194553+0000","last_clean":"2023-09-30T14:38:12.194553+0000","last_became_active":"2023-09-30T14:37:11.996596+0000","last_became_peered":"2023-09-30T14:37:11.996596+0000","last_unstale":"2023-09-30T14:38:12.194553+0000","last_undegraded":"2023-09-30T14:38:12.194553+0000","last_fullsized":"2023-09-30T14:38:12.194553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:25:04.531236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":98,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:34.096608+0000","last_change":"2023-09-30T14:37:11.025458+0000","last_active":"2023-09-30T14:38:34.096608+0000","last_peered":"2023-09-30T14:38:34.096608+0000","last_clean":"2023-09-30T14:38:34.096608+0000","last_became_active":"2023-09-30T14:37:11.025157+0000","last_became_peered":"2023-09-30T14:37:11.025157+0000","last_unstale":"2023-09-30T14:38:34.096608+0000","last_undegraded":"2023-09-30T14:38:34.096608+0000","last_fullsized":"2023-09-30T14:38:34.096608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:31:24.436125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824345+0000","last_change":"2023-09-30T14:37:11.986840+0000","last_active":"2023-09-30T14:38:12.824345+0000","last_peered":"2023-09-30T14:38:12.824345+0000","last_clean":"2023-09-30T14:38:12.824345+0000","last_became_active":"2023-09-30T14:37:11.986483+0000","last_became_peered":"2023-09-30T14:37:11.986483+0000","last_unstale":"2023-09-30T14:38:12.824345+0000","last_undegraded":"2023-09-30T14:38:12.824345+0000","last_fullsized":"2023-09-30T14:38:12.824345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:14:32.399685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196845+0000","last_change":"2023-09-30T14:37:11.025706+0000","last_active":"2023-09-30T14:38:12.196845+0000","last_peered":"2023-09-30T14:38:12.196845+0000","last_clean":"2023-09-30T14:38:12.196845+0000","last_became_active":"2023-09-30T14:37:11.025376+0000","last_became_peered":"2023-09-30T14:37:11.025376+0000","last_unstale":"2023-09-30T14:38:12.196845+0000","last_undegraded":"2023-09-30T14:38:12.196845+0000","last_fullsized":"2023-09-30T14:38:12.196845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:10:36.025273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":310,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198525+0000","last_change":"2023-09-30T14:36:12.344762+0000","last_active":"2023-09-30T14:38:12.198525+0000","last_peered":"2023-09-30T14:38:12.198525+0000","last_clean":"2023-09-30T14:38:12.198525+0000","last_became_active":"2023-09-30T14:36:12.344268+0000","last_became_peered":"2023-09-30T14:36:12.344268+0000","last_unstale":"2023-09-30T14:38:12.198525+0000","last_undegraded":"2023-09-30T14:38:12.198525+0000","last_fullsized":"2023-09-30T14:38:12.198525+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:33:20.434992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:33:20.434992+0000","last_clean_scrub_stamp":"2023-09-30T14:33:20.434992+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:00:05.598961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194122+0000","last_change":"2023-09-30T14:37:11.996886+0000","last_active":"2023-09-30T14:38:12.194122+0000","last_peered":"2023-09-30T14:38:12.194122+0000","last_clean":"2023-09-30T14:38:12.194122+0000","last_became_active":"2023-09-30T14:37:11.996667+0000","last_became_peered":"2023-09-30T14:37:11.996667+0000","last_unstale":"2023-09-30T14:38:12.194122+0000","last_undegraded":"2023-09-30T14:38:12.194122+0000","last_fullsized":"2023-09-30T14:38:12.194122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:27:00.546356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198414+0000","last_change":"2023-09-30T14:37:11.027012+0000","last_active":"2023-09-30T14:38:12.198414+0000","last_peered":"2023-09-30T14:38:12.198414+0000","last_clean":"2023-09-30T14:38:12.198414+0000","last_became_active":"2023-09-30T14:37:11.026714+0000","last_became_peered":"2023-09-30T14:37:11.026714+0000","last_unstale":"2023-09-30T14:38:12.198414+0000","last_undegraded":"2023-09-30T14:38:12.198414+0000","last_fullsized":"2023-09-30T14:38:12.198414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:17:57.860438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200462+0000","last_change":"2023-09-30T14:37:11.027389+0000","last_active":"2023-09-30T14:38:12.200462+0000","last_peered":"2023-09-30T14:38:12.200462+0000","last_clean":"2023-09-30T14:38:12.200462+0000","last_became_active":"2023-09-30T14:37:11.027011+0000","last_became_peered":"2023-09-30T14:37:11.027011+0000","last_unstale":"2023-09-30T14:38:12.200462+0000","last_undegraded":"2023-09-30T14:38:12.200462+0000","last_fullsized":"2023-09-30T14:38:12.200462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:38:22.953936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194384+0000","last_change":"2023-09-30T14:37:11.997675+0000","last_active":"2023-09-30T14:38:12.194384+0000","last_peered":"2023-09-30T14:38:12.194384+0000","last_clean":"2023-09-30T14:38:12.194384+0000","last_became_active":"2023-09-30T14:37:11.997330+0000","last_became_peered":"2023-09-30T14:37:11.997330+0000","last_unstale":"2023-09-30T14:38:12.194384+0000","last_undegraded":"2023-09-30T14:38:12.194384+0000","last_fullsized":"2023-09-30T14:38:12.194384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:54:52.802138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196687+0000","last_change":"2023-09-30T14:37:11.023836+0000","last_active":"2023-09-30T14:38:12.196687+0000","last_peered":"2023-09-30T14:38:12.196687+0000","last_clean":"2023-09-30T14:38:12.196687+0000","last_became_active":"2023-09-30T14:37:11.023362+0000","last_became_peered":"2023-09-30T14:37:11.023362+0000","last_unstale":"2023-09-30T14:38:12.196687+0000","last_undegraded":"2023-09-30T14:38:12.196687+0000","last_fullsized":"2023-09-30T14:38:12.196687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:58:22.772546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824591+0000","last_change":"2023-09-30T14:37:11.985281+0000","last_active":"2023-09-30T14:38:12.824591+0000","last_peered":"2023-09-30T14:38:12.824591+0000","last_clean":"2023-09-30T14:38:12.824591+0000","last_became_active":"2023-09-30T14:37:11.984944+0000","last_became_peered":"2023-09-30T14:37:11.984944+0000","last_unstale":"2023-09-30T14:38:12.824591+0000","last_undegraded":"2023-09-30T14:38:12.824591+0000","last_fullsized":"2023-09-30T14:38:12.824591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:30:19.829727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197438+0000","last_change":"2023-09-30T14:37:11.028070+0000","last_active":"2023-09-30T14:38:12.197438+0000","last_peered":"2023-09-30T14:38:12.197438+0000","last_clean":"2023-09-30T14:38:12.197438+0000","last_became_active":"2023-09-30T14:37:11.027703+0000","last_became_peered":"2023-09-30T14:37:11.027703+0000","last_unstale":"2023-09-30T14:38:12.197438+0000","last_undegraded":"2023-09-30T14:38:12.197438+0000","last_fullsized":"2023-09-30T14:38:12.197438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:23:15.071946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197147+0000","last_change":"2023-09-30T14:37:11.989897+0000","last_active":"2023-09-30T14:38:12.197147+0000","last_peered":"2023-09-30T14:38:12.197147+0000","last_clean":"2023-09-30T14:38:12.197147+0000","last_became_active":"2023-09-30T14:37:11.989613+0000","last_became_peered":"2023-09-30T14:37:11.989613+0000","last_unstale":"2023-09-30T14:38:12.197147+0000","last_undegraded":"2023-09-30T14:38:12.197147+0000","last_fullsized":"2023-09-30T14:38:12.197147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:43:17.887766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198152+0000","last_change":"2023-09-30T14:37:11.026491+0000","last_active":"2023-09-30T14:38:12.198152+0000","last_peered":"2023-09-30T14:38:12.198152+0000","last_clean":"2023-09-30T14:38:12.198152+0000","last_became_active":"2023-09-30T14:37:11.026196+0000","last_became_peered":"2023-09-30T14:37:11.026196+0000","last_unstale":"2023-09-30T14:38:12.198152+0000","last_undegraded":"2023-09-30T14:38:12.198152+0000","last_fullsized":"2023-09-30T14:38:12.198152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:17:22.343377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199319+0000","last_change":"2023-09-30T14:37:11.988570+0000","last_active":"2023-09-30T14:38:12.199319+0000","last_peered":"2023-09-30T14:38:12.199319+0000","last_clean":"2023-09-30T14:38:12.199319+0000","last_became_active":"2023-09-30T14:37:11.988230+0000","last_became_peered":"2023-09-30T14:37:11.988230+0000","last_unstale":"2023-09-30T14:38:12.199319+0000","last_undegraded":"2023-09-30T14:38:12.199319+0000","last_fullsized":"2023-09-30T14:38:12.199319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:58:38.302343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824045+0000","last_change":"2023-09-30T14:37:11.024587+0000","last_active":"2023-09-30T14:38:12.824045+0000","last_peered":"2023-09-30T14:38:12.824045+0000","last_clean":"2023-09-30T14:38:12.824045+0000","last_became_active":"2023-09-30T14:37:11.024253+0000","last_became_peered":"2023-09-30T14:37:11.024253+0000","last_unstale":"2023-09-30T14:38:12.824045+0000","last_undegraded":"2023-09-30T14:38:12.824045+0000","last_fullsized":"2023-09-30T14:38:12.824045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:31:07.293657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197186+0000","last_change":"2023-09-30T14:37:11.987818+0000","last_active":"2023-09-30T14:38:12.197186+0000","last_peered":"2023-09-30T14:38:12.197186+0000","last_clean":"2023-09-30T14:38:12.197186+0000","last_became_active":"2023-09-30T14:37:11.987476+0000","last_became_peered":"2023-09-30T14:37:11.987476+0000","last_unstale":"2023-09-30T14:38:12.197186+0000","last_undegraded":"2023-09-30T14:38:12.197186+0000","last_fullsized":"2023-09-30T14:38:12.197186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:01:07.802959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199043+0000","last_change":"2023-09-30T14:37:11.023223+0000","last_active":"2023-09-30T14:38:12.199043+0000","last_peered":"2023-09-30T14:38:12.199043+0000","last_clean":"2023-09-30T14:38:12.199043+0000","last_became_active":"2023-09-30T14:37:11.022932+0000","last_became_peered":"2023-09-30T14:37:11.022932+0000","last_unstale":"2023-09-30T14:38:12.199043+0000","last_undegraded":"2023-09-30T14:38:12.199043+0000","last_fullsized":"2023-09-30T14:38:12.199043+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:01:04.315669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824324+0000","last_change":"2023-09-30T14:37:11.990323+0000","last_active":"2023-09-30T14:38:12.824324+0000","last_peered":"2023-09-30T14:38:12.824324+0000","last_clean":"2023-09-30T14:38:12.824324+0000","last_became_active":"2023-09-30T14:37:11.989981+0000","last_became_peered":"2023-09-30T14:37:11.989981+0000","last_unstale":"2023-09-30T14:38:12.824324+0000","last_undegraded":"2023-09-30T14:38:12.824324+0000","last_fullsized":"2023-09-30T14:38:12.824324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:30:44.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199908+0000","last_change":"2023-09-30T14:37:11.025658+0000","last_active":"2023-09-30T14:38:12.199908+0000","last_peered":"2023-09-30T14:38:12.199908+0000","last_clean":"2023-09-30T14:38:12.199908+0000","last_became_active":"2023-09-30T14:37:11.025334+0000","last_became_peered":"2023-09-30T14:37:11.025334+0000","last_unstale":"2023-09-30T14:38:12.199908+0000","last_undegraded":"2023-09-30T14:38:12.199908+0000","last_fullsized":"2023-09-30T14:38:12.199908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:23:41.933634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198716+0000","last_change":"2023-09-30T14:37:11.989989+0000","last_active":"2023-09-30T14:38:12.198716+0000","last_peered":"2023-09-30T14:38:12.198716+0000","last_clean":"2023-09-30T14:38:12.198716+0000","last_became_active":"2023-09-30T14:37:11.989666+0000","last_became_peered":"2023-09-30T14:37:11.989666+0000","last_unstale":"2023-09-30T14:38:12.198716+0000","last_undegraded":"2023-09-30T14:38:12.198716+0000","last_fullsized":"2023-09-30T14:38:12.198716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:41:09.835210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198863+0000","last_change":"2023-09-30T14:37:11.025027+0000","last_active":"2023-09-30T14:38:12.198863+0000","last_peered":"2023-09-30T14:38:12.198863+0000","last_clean":"2023-09-30T14:38:12.198863+0000","last_became_active":"2023-09-30T14:37:11.024696+0000","last_became_peered":"2023-09-30T14:37:11.024696+0000","last_unstale":"2023-09-30T14:38:12.198863+0000","last_undegraded":"2023-09-30T14:38:12.198863+0000","last_fullsized":"2023-09-30T14:38:12.198863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:34:28.384535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199073+0000","last_change":"2023-09-30T14:37:11.992674+0000","last_active":"2023-09-30T14:38:12.199073+0000","last_peered":"2023-09-30T14:38:12.199073+0000","last_clean":"2023-09-30T14:38:12.199073+0000","last_became_active":"2023-09-30T14:37:11.992243+0000","last_became_peered":"2023-09-30T14:37:11.992243+0000","last_unstale":"2023-09-30T14:38:12.199073+0000","last_undegraded":"2023-09-30T14:38:12.199073+0000","last_fullsized":"2023-09-30T14:38:12.199073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:33:19.759637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194984+0000","last_change":"2023-09-30T14:37:11.024525+0000","last_active":"2023-09-30T14:38:12.194984+0000","last_peered":"2023-09-30T14:38:12.194984+0000","last_clean":"2023-09-30T14:38:12.194984+0000","last_became_active":"2023-09-30T14:37:11.024231+0000","last_became_peered":"2023-09-30T14:37:11.024231+0000","last_unstale":"2023-09-30T14:38:12.194984+0000","last_undegraded":"2023-09-30T14:38:12.194984+0000","last_fullsized":"2023-09-30T14:38:12.194984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:53:47.142798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198957+0000","last_change":"2023-09-30T14:37:11.991401+0000","last_active":"2023-09-30T14:38:12.198957+0000","last_peered":"2023-09-30T14:38:12.198957+0000","last_clean":"2023-09-30T14:38:12.198957+0000","last_became_active":"2023-09-30T14:37:11.991012+0000","last_became_peered":"2023-09-30T14:37:11.991012+0000","last_unstale":"2023-09-30T14:38:12.198957+0000","last_undegraded":"2023-09-30T14:38:12.198957+0000","last_fullsized":"2023-09-30T14:38:12.198957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:55:00.562030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195033+0000","last_change":"2023-09-30T14:37:11.022893+0000","last_active":"2023-09-30T14:38:12.195033+0000","last_peered":"2023-09-30T14:38:12.195033+0000","last_clean":"2023-09-30T14:38:12.195033+0000","last_became_active":"2023-09-30T14:37:11.022505+0000","last_became_peered":"2023-09-30T14:37:11.022505+0000","last_unstale":"2023-09-30T14:38:12.195033+0000","last_undegraded":"2023-09-30T14:38:12.195033+0000","last_fullsized":"2023-09-30T14:38:12.195033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:00:17.721645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200355+0000","last_change":"2023-09-30T14:37:11.992942+0000","last_active":"2023-09-30T14:38:12.200355+0000","last_peered":"2023-09-30T14:38:12.200355+0000","last_clean":"2023-09-30T14:38:12.200355+0000","last_became_active":"2023-09-30T14:37:11.992698+0000","last_became_peered":"2023-09-30T14:37:11.992698+0000","last_unstale":"2023-09-30T14:38:12.200355+0000","last_undegraded":"2023-09-30T14:38:12.200355+0000","last_fullsized":"2023-09-30T14:38:12.200355+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:15:30.239423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.201641+0000","last_change":"2023-09-30T14:38:13.201641+0000","last_active":"2023-09-30T14:38:13.201641+0000","last_peered":"2023-09-30T14:38:13.201641+0000","last_clean":"2023-09-30T14:38:13.201641+0000","last_became_active":"2023-09-30T14:38:13.201255+0000","last_became_peered":"2023-09-30T14:38:13.201255+0000","last_unstale":"2023-09-30T14:38:13.201641+0000","last_undegraded":"2023-09-30T14:38:13.201641+0000","last_fullsized":"2023-09-30T14:38:13.201641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:06:02.731345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199114+0000","last_change":"2023-09-30T14:37:11.984481+0000","last_active":"2023-09-30T14:38:12.199114+0000","last_peered":"2023-09-30T14:38:12.199114+0000","last_clean":"2023-09-30T14:38:12.199114+0000","last_became_active":"2023-09-30T14:37:11.984021+0000","last_became_peered":"2023-09-30T14:37:11.984021+0000","last_unstale":"2023-09-30T14:38:12.199114+0000","last_undegraded":"2023-09-30T14:38:12.199114+0000","last_fullsized":"2023-09-30T14:38:12.199114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:36:33.184379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198978+0000","last_change":"2023-09-30T14:37:11.990779+0000","last_active":"2023-09-30T14:38:12.198978+0000","last_peered":"2023-09-30T14:38:12.198978+0000","last_clean":"2023-09-30T14:38:12.198978+0000","last_became_active":"2023-09-30T14:37:11.989960+0000","last_became_peered":"2023-09-30T14:37:11.989960+0000","last_unstale":"2023-09-30T14:38:12.198978+0000","last_undegraded":"2023-09-30T14:38:12.198978+0000","last_fullsized":"2023-09-30T14:38:12.198978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:54:36.813191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196893+0000","last_change":"2023-09-30T14:37:11.995750+0000","last_active":"2023-09-30T14:38:12.196893+0000","last_peered":"2023-09-30T14:38:12.196893+0000","last_clean":"2023-09-30T14:38:12.196893+0000","last_became_active":"2023-09-30T14:37:11.995434+0000","last_became_peered":"2023-09-30T14:37:11.995434+0000","last_unstale":"2023-09-30T14:38:12.196893+0000","last_undegraded":"2023-09-30T14:38:12.196893+0000","last_fullsized":"2023-09-30T14:38:12.196893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:49:51.234024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.201564+0000","last_change":"2023-09-30T14:38:13.201564+0000","last_active":"2023-09-30T14:38:13.201564+0000","last_peered":"2023-09-30T14:38:13.201564+0000","last_clean":"2023-09-30T14:38:13.201564+0000","last_became_active":"2023-09-30T14:38:13.201175+0000","last_became_peered":"2023-09-30T14:38:13.201175+0000","last_unstale":"2023-09-30T14:38:13.201564+0000","last_undegraded":"2023-09-30T14:38:13.201564+0000","last_fullsized":"2023-09-30T14:38:13.201564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:46:23.763202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197032+0000","last_change":"2023-09-30T14:37:11.983495+0000","last_active":"2023-09-30T14:38:12.197032+0000","last_peered":"2023-09-30T14:38:12.197032+0000","last_clean":"2023-09-30T14:38:12.197032+0000","last_became_active":"2023-09-30T14:37:11.983241+0000","last_became_peered":"2023-09-30T14:37:11.983241+0000","last_unstale":"2023-09-30T14:38:12.197032+0000","last_undegraded":"2023-09-30T14:38:12.197032+0000","last_fullsized":"2023-09-30T14:38:12.197032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:41:46.604467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200166+0000","last_change":"2023-09-30T14:37:11.996058+0000","last_active":"2023-09-30T14:38:12.200166+0000","last_peered":"2023-09-30T14:38:12.200166+0000","last_clean":"2023-09-30T14:38:12.200166+0000","last_became_active":"2023-09-30T14:37:11.995763+0000","last_became_peered":"2023-09-30T14:37:11.995763+0000","last_unstale":"2023-09-30T14:38:12.200166+0000","last_undegraded":"2023-09-30T14:38:12.200166+0000","last_fullsized":"2023-09-30T14:38:12.200166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:42:21.306561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824505+0000","last_change":"2023-09-30T14:37:11.992080+0000","last_active":"2023-09-30T14:38:12.824505+0000","last_peered":"2023-09-30T14:38:12.824505+0000","last_clean":"2023-09-30T14:38:12.824505+0000","last_became_active":"2023-09-30T14:37:11.991747+0000","last_became_peered":"2023-09-30T14:37:11.991747+0000","last_unstale":"2023-09-30T14:38:12.824505+0000","last_undegraded":"2023-09-30T14:38:12.824505+0000","last_fullsized":"2023-09-30T14:38:12.824505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:07:25.985911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194496+0000","last_change":"2023-09-30T14:37:11.997109+0000","last_active":"2023-09-30T14:38:12.194496+0000","last_peered":"2023-09-30T14:38:12.194496+0000","last_clean":"2023-09-30T14:38:12.194496+0000","last_became_active":"2023-09-30T14:37:11.996793+0000","last_became_peered":"2023-09-30T14:37:11.996793+0000","last_unstale":"2023-09-30T14:38:12.194496+0000","last_undegraded":"2023-09-30T14:38:12.194496+0000","last_fullsized":"2023-09-30T14:38:12.194496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:13:59.838638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.823992+0000","last_change":"2023-09-30T14:37:11.984083+0000","last_active":"2023-09-30T14:38:12.823992+0000","last_peered":"2023-09-30T14:38:12.823992+0000","last_clean":"2023-09-30T14:38:12.823992+0000","last_became_active":"2023-09-30T14:37:11.983352+0000","last_became_peered":"2023-09-30T14:37:11.983352+0000","last_unstale":"2023-09-30T14:38:12.823992+0000","last_undegraded":"2023-09-30T14:38:12.823992+0000","last_fullsized":"2023-09-30T14:38:12.823992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:02:03.668484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198484+0000","last_change":"2023-09-30T14:37:11.992212+0000","last_active":"2023-09-30T14:38:12.198484+0000","last_peered":"2023-09-30T14:38:12.198484+0000","last_clean":"2023-09-30T14:38:12.198484+0000","last_became_active":"2023-09-30T14:37:11.990158+0000","last_became_peered":"2023-09-30T14:37:11.990158+0000","last_unstale":"2023-09-30T14:38:12.198484+0000","last_undegraded":"2023-09-30T14:38:12.198484+0000","last_fullsized":"2023-09-30T14:38:12.198484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:14:27.976110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194553+0000","last_change":"2023-09-30T14:37:11.995884+0000","last_active":"2023-09-30T14:38:12.194553+0000","last_peered":"2023-09-30T14:38:12.194553+0000","last_clean":"2023-09-30T14:38:12.194553+0000","last_became_active":"2023-09-30T14:37:11.995551+0000","last_became_peered":"2023-09-30T14:37:11.995551+0000","last_unstale":"2023-09-30T14:38:12.194553+0000","last_undegraded":"2023-09-30T14:38:12.194553+0000","last_fullsized":"2023-09-30T14:38:12.194553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:28:26.051188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197182+0000","last_change":"2023-09-30T14:37:11.985853+0000","last_active":"2023-09-30T14:38:12.197182+0000","last_peered":"2023-09-30T14:38:12.197182+0000","last_clean":"2023-09-30T14:38:12.197182+0000","last_became_active":"2023-09-30T14:37:11.985525+0000","last_became_peered":"2023-09-30T14:37:11.985525+0000","last_unstale":"2023-09-30T14:38:12.197182+0000","last_undegraded":"2023-09-30T14:38:12.197182+0000","last_fullsized":"2023-09-30T14:38:12.197182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:07:24.392538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58318,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":491,"num_read_kb":310,"num_write":149,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1316,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":175272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449116,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":443610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.56599999999999995,"15min":0.57799999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.60199999999999998,"5min":1.056,"15min":1.056},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.57999999999999996,"15min":0.59799999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.621,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.51300000000000001}]},{"osd":1,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.51500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.80300000000000005,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.78700000000000003,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.63}]},{"osd":2,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.499,"15min":0.497},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.76100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68200000000000005,"15min":0.67600000000000005},"min":{"1min":0.46000000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.012,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":1.012,"15min":1.012},"last":0.68100000000000005}]},{"osd":5,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":1.226,"15min":1.226},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.624,"15min":0.628},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94999999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.90300000000000002}]},{"osd":6,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70899999999999996,"15min":0.71699999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.73899999999999999}]},{"osd":7,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79100000000000004,"15min":0.79600000000000004},"min":{"1min":0.53100000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.0369999999999999,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.79400000000000004,"15min":0.80300000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.032,"5min":1.319,"15min":1.319},"last":0.92400000000000004}]},{"osd":8,"last update":"Sat Sep 30 14:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.95699999999999996}]},{"osd":9,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.77700000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97099999999999997,"5min":1.597,"15min":1.597},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.84499999999999997,"15min":0.84599999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":1.012}]},{"osd":10,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.82499999999999996,"15min":0.83399999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0609999999999999,"5min":1.417,"15min":1.417},"last":0.873},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80800000000000005,"15min":0.81200000000000006},"min":{"1min":0.54100000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0469999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.93700000000000006}]}]},{"osd":10,"up_from":71,"seq":304942678048,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1605632,"data_stored":1088780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47599999999999998,"15min":0.47999999999999998},"min":{"1min":0.28100000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64600000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.33200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.67500000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48299999999999998}]},{"osd":2,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76000000000000001,"5min":0.879,"15min":0.879},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.59199999999999997}]},{"osd":3,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.65800000000000003,"15min":0.67600000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.75600000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.65200000000000002,"15min":0.67500000000000004},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79700000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.50800000000000001}]},{"osd":4,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67100000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.60199999999999998,"15min":0.61499999999999999},"min":{"1min":0.374,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71999999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57799999999999996}]},{"osd":5,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.70299999999999996,"15min":0.71599999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77500000000000002,"5min":1.244,"15min":1.244},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.505,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.80500000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73799999999999999}]},{"osd":6,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.752,"15min":0.77200000000000002},"min":{"1min":0.44600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77800000000000002,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.57999999999999996,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85299999999999998,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.70399999999999996}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.41699999999999998}]},{"osd":8,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.82099999999999995,"15min":0.83499999999999996},"min":{"1min":0.53200000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.84199999999999997,"5min":1.335,"15min":1.335},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.82799999999999996,"15min":0.84499999999999997},"min":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.85799999999999998,"5min":1.218,"15min":1.218},"last":0.78200000000000003}]},{"osd":9,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.83899999999999997,"15min":0.85899999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.91000000000000003,"5min":1.389,"15min":1.389},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.84599999999999997,"15min":0.86799999999999999},"min":{"1min":0.501,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.872,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.69199999999999995}]},{"osd":11,"last update":"Sat Sep 30 14:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79900000000000004},"min":{"1min":0.63200000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90800000000000003,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.85499999999999998,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.82799999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841571,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":442821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.46500000000000002,"15min":0.47499999999999998},"min":{"1min":0.28499999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60299999999999998,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.316},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.503,"15min":0.51800000000000002},"min":{"1min":0.32600000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65700000000000003,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.34200000000000003}]},{"osd":1,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77400000000000002,"5min":1.069,"15min":1.069},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.64500000000000002,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.749,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.45900000000000002}]},{"osd":2,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.61799999999999999,"15min":0.628},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.38,"5min":0.376,"15min":0.376},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.57499999999999996}]},{"osd":3,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.54900000000000004,"15min":0.56599999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69899999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.86199999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.40899999999999997}]},{"osd":4,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.69299999999999995,"15min":0.70999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80500000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.71399999999999997,"15min":0.72699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86399999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.503}]},{"osd":5,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.70499999999999996,"15min":0.72199999999999998},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":1.137,"15min":1.137},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.73299999999999998,"15min":0.749},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0860000000000001,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.59099999999999997}]},{"osd":6,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.69499999999999995,"15min":0.71799999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.046,"5min":1.21,"15min":1.21},"last":0.436},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.71599999999999997,"15min":0.73599999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96299999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.50900000000000001}]},{"osd":7,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.79700000000000004,"15min":0.81599999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94099999999999995,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.81200000000000006,"15min":0.82799999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.66800000000000004}]},{"osd":8,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81200000000000006,"15min":0.82599999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.81599999999999995,"15min":0.83599999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.028,"5min":1.466,"15min":1.466},"last":0.70299999999999996}]},{"osd":10,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.80900000000000005,"15min":0.82499999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90400000000000003,"5min":1.391,"15min":1.391},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79600000000000004,"15min":0.80800000000000005},"min":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.91200000000000003,"5min":1.121,"15min":1.121},"last":0.67900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":8,"up_from":58,"seq":249108103209,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.42199999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.437,"5min":0.316,"15min":0.316},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.54200000000000004}]},{"osd":1,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90300000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.60999999999999999,"15min":0.59499999999999997},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86099999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.748}]},{"osd":2,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.47299999999999998,"15min":0.45500000000000002},"min":{"1min":0.38800000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.48499999999999999,"15min":0.46600000000000003},"min":{"1min":0.35999999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.63200000000000001}]},{"osd":3,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60699999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.435,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.57499999999999996}]},{"osd":4,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.63500000000000001,"15min":0.60899999999999999},"min":{"1min":0.40699999999999997,"5min":0.378,"15min":0.378},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.66100000000000003,"15min":0.63700000000000001},"min":{"1min":0.60999999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.88500000000000001}]},{"osd":5,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86599999999999999,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.622,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.83799999999999997}]},{"osd":6,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.68500000000000005,"15min":0.66800000000000004},"min":{"1min":0.51200000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.46700000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98499999999999999,"5min":1.133,"15min":1.133},"last":0.86199999999999999}]},{"osd":7,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.73999999999999999,"15min":0.72299999999999998},"min":{"1min":0.58999999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.012,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.74199999999999999,"15min":0.72399999999999998},"min":{"1min":0.53600000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.95299999999999996}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.749,"15min":0.747},"min":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95299999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.82099999999999995}]},{"osd":10,"last update":"Sat Sep 30 14:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.76800000000000002,"15min":0.74199999999999999},"min":{"1min":0.71799999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.018},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.76200000000000001,"15min":0.73699999999999999},"min":{"1min":0.76200000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.94499999999999995}]},{"osd":11,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.91000000000000003}]}]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1034035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42999999999999999}]},{"osd":1,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.35799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.60999999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.45700000000000002,"15min":0.439},"min":{"1min":0.39300000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.49199999999999999}]},{"osd":2,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60399999999999998,"15min":0.58499999999999996},"min":{"1min":0.496,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83999999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.42799999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74299999999999999,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55800000000000005}]},{"osd":3,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.43099999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81299999999999994,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65100000000000002,"15min":0.66700000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81200000000000006,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.67900000000000005}]},{"osd":4,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.5},"min":{"1min":0.38600000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69799999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54000000000000004,"15min":0.52200000000000002},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76300000000000001,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60299999999999998}]},{"osd":5,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.63600000000000001,"15min":0.61899999999999999},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78900000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.67900000000000005,"15min":0.65900000000000003},"min":{"1min":0.45700000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.89500000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.748}]},{"osd":6,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.55700000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95799999999999996,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.747,"15min":0.73499999999999999},"min":{"1min":0.47099999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.9340000000000002,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":0.79400000000000004}]},{"osd":8,"last update":"Sat Sep 30 14:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.47999999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.872,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.73199999999999998,"15min":0.70799999999999996},"min":{"1min":0.629,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.84099999999999997}]},{"osd":9,"last update":"Sat Sep 30 14:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77600000000000002,"15min":0.77900000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.57299999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96499999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.82599999999999996}]},{"osd":10,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.77000000000000002,"15min":0.76400000000000001},"min":{"1min":0.64700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.79400000000000004,"15min":0.79600000000000004},"min":{"1min":0.52000000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.65600000000000003}]},{"osd":11,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.88700000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":442216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38100000000000001,"15min":0.371},"min":{"1min":0.27700000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.40799999999999997,"15min":0.39000000000000001},"min":{"1min":0.28100000000000003,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.26700000000000002}]},{"osd":1,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.75700000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.626},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.48599999999999999}]},{"osd":2,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.628,"15min":0.627},"min":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.75700000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.433}]},{"osd":3,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.502,"15min":0.48599999999999999},"min":{"1min":0.35299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.504},{"interface":"front","average":{"1min":0.504,"5min":0.499,"15min":0.497},"min":{"1min":0.35799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.249}]},{"osd":4,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.46800000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67300000000000004,"15min":0.65100000000000002},"min":{"1min":0.46899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.89600000000000002,"5min":1.056,"15min":1.056},"last":0.58599999999999997}]},{"osd":5,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64600000000000002,"15min":0.64700000000000002},"min":{"1min":0.499,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.48099999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1060000000000001,"5min":1.228,"15min":1.228},"last":0.36699999999999999}]},{"osd":7,"last update":"Sat Sep 30 14:38:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70999999999999996,"15min":0.68799999999999994},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1679999999999999,"5min":1.272,"15min":1.272},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.56399999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.83999999999999997,"5min":1.323,"15min":1.323},"last":0.69799999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.55000000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.81599999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.55100000000000005}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.748,"15min":0.76000000000000001},"min":{"1min":0.45500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72499999999999998,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93200000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.66300000000000003}]},{"osd":10,"last update":"Sat Sep 30 14:38:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.83699999999999997,"15min":0.84499999999999997},"min":{"1min":0.627,"5min":0.62,"15min":0.62},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.82499999999999996,"15min":0.84199999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.89500000000000002,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":497333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55600000000000005,"15min":0.50600000000000001},"min":{"1min":0.36099999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.56999999999999995,"15min":0.52300000000000002},"min":{"1min":0.45200000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.76800000000000002}]},{"osd":2,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.61299999999999999,"15min":0.52900000000000003},"min":{"1min":0.53100000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98499999999999999,"5min":0.998,"15min":0.998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.59799999999999998,"15min":0.53000000000000003},"min":{"1min":0.39200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.53200000000000003}]},{"osd":3,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57899999999999996,"15min":0.54300000000000004},"min":{"1min":0.436,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60599999999999998,"15min":0.57499999999999996},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92200000000000004,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.68799999999999994}]},{"osd":4,"last update":"Sat Sep 30 14:38:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.48099999999999998,"15min":0.443},"min":{"1min":0.28999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.503,"15min":0.46300000000000002},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.81699999999999995}]},{"osd":5,"last update":"Sat Sep 30 14:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.82799999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.68200000000000005,"15min":0.628},"min":{"1min":0.57899999999999996,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.85899999999999999}]},{"osd":6,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88300000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69499999999999995,"15min":0.67500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.88200000000000001}]},{"osd":7,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.61399999999999999,"15min":0.55900000000000005},"min":{"1min":0.58099999999999996,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.434,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.57299999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.77800000000000002,"15min":0.748},"min":{"1min":0.55200000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.751,"15min":0.71699999999999997},"min":{"1min":0.56000000000000005,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.98599999999999999}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.748},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.81799999999999995,"15min":0.81399999999999995},"min":{"1min":0.56799999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.96499999999999997}]},{"osd":10,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.81699999999999995,"15min":0.79500000000000004},"min":{"1min":0.75800000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.78100000000000003,"15min":0.76000000000000001},"min":{"1min":0.70199999999999996,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.90900000000000003}]},{"osd":11,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.94599999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":443481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 14:38:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55600000000000005,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71799999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57099999999999995,"15min":0.53400000000000003},"min":{"1min":0.42799999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.84599999999999997,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.53500000000000003}]},{"osd":2,"last update":"Sat Sep 30 14:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.47499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.58999999999999997,"15min":0.56799999999999995},"min":{"1min":0.48999999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90700000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.76800000000000002}]},{"osd":3,"last update":"Sat Sep 30 14:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.38300000000000001}]},{"osd":4,"last update":"Sat Sep 30 14:38:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60299999999999998,"15min":0.58599999999999997},"min":{"1min":0.41299999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82699999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60799999999999998,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83099999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.496}]},{"osd":5,"last update":"Sat Sep 30 14:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.47299999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.45100000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.79400000000000004,"5min":0.996,"15min":0.996},"last":0.73299999999999998}]},{"osd":6,"last update":"Sat Sep 30 14:37:47 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69199999999999995,"5min":1.03,"15min":1.03},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.317,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80600000000000005,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.51600000000000001}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69699999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.014,"5min":1.147,"15min":1.147},"last":0.442},{"interface":"front","average":{"1min":0.748,"5min":0.67800000000000005,"15min":0.66600000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56799999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.52300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85199999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.745,"15min":0.753},"min":{"1min":0.51700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85299999999999998,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.65700000000000003}]},{"osd":9,"last update":"Sat Sep 30 14:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.71999999999999997,"15min":0.73899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80200000000000005,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.78200000000000003,"15min":0.80400000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.79900000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.71699999999999997}]},{"osd":10,"last update":"Sat Sep 30 14:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.88400000000000001,"5min":1.127,"15min":1.127},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.79300000000000004,"15min":0.81100000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.873,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.66900000000000004}]},{"osd":11,"last update":"Sat Sep 30 14:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.753},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.69699999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":497497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52000000000000002,"15min":0.51100000000000001},"min":{"1min":0.372,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.83199999999999996,"5min":0.876,"15min":0.876},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.505,"15min":0.496},"min":{"1min":0.34799999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67500000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.73699999999999999}]},{"osd":1,"last update":"Sat Sep 30 14:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56799999999999995,"15min":0.52600000000000002},"min":{"1min":0.40300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.79200000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.626,"15min":0.55100000000000005},"min":{"1min":0.52700000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.68799999999999994}]},{"osd":3,"last update":"Sat Sep 30 14:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.35599999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.56499999999999995},"min":{"1min":0.42699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.89000000000000001}]},{"osd":4,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":1.018}]},{"osd":5,"last update":"Sat Sep 30 14:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.50700000000000001,"15min":0.46899999999999997},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.53000000000000003,"15min":0.49399999999999999},"min":{"1min":0.43099999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.65200000000000002}]},{"osd":6,"last update":"Sat Sep 30 14:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66600000000000004,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1000000000000001,"5min":1.405,"15min":1.405},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.94999999999999996}]},{"osd":7,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.45500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.80100000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70699999999999996,"15min":0.71299999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.86399999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.59199999999999997}]},{"osd":8,"last update":"Sat Sep 30 14:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.65100000000000002,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.627},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.46999999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75900000000000001}]},{"osd":9,"last update":"Sat Sep 30 14:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.877},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.79600000000000004}]},{"osd":10,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.55900000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.69199999999999995,"15min":0.67100000000000004},"min":{"1min":0.56200000000000006,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.98399999999999999}]},{"osd":11,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":1.0449999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149759,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":442533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.35799999999999998},"min":{"1min":0.27200000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.432,"15min":0.41199999999999998},"min":{"1min":0.316,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54800000000000004,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.59599999999999997}]},{"osd":1,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64200000000000002,"15min":0.58599999999999997},"min":{"1min":0.442,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.002,"5min":5.798,"15min":5.798},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.71799999999999997}]},{"osd":2,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.39100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.70999999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58399999999999996,"15min":0.55100000000000005},"min":{"1min":0.442,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.73199999999999998}]},{"osd":4,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.66100000000000003,"15min":0.58399999999999996},"min":{"1min":0.40999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.873,"5min":7.1619999999999999,"15min":7.1619999999999999},"last":0.624},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.621,"15min":0.57699999999999996},"min":{"1min":0.44800000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.075,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.70199999999999996}]},{"osd":5,"last update":"Sat Sep 30 14:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.42599999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.46000000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72399999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.79400000000000004}]},{"osd":7,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.71599999999999997,"15min":0.70499999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99199999999999999,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81699999999999995,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.74299999999999999}]},{"osd":8,"last update":"Sat Sep 30 14:36:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.46100000000000002}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.69499999999999995,"15min":0.71399999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81299999999999994,"5min":1.161,"15min":1.161},"last":0.82499999999999996}]},{"osd":10,"last update":"Sat Sep 30 14:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":1.0249999999999999,"15min":1.069},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.3640000000000001,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.84699999999999998,"15min":0.86899999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.304,"5min":2.597,"15min":2.597},"last":0.84199999999999997}]},{"osd":11,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.88},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.91600000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67100000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.65000000000000002}]},{"osd":1,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.372},"min":{"1min":0.32200000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.53400000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39000000000000001,"15min":0.36799999999999999},"min":{"1min":0.34200000000000003,"5min":0.224,"15min":0.224},"max":{"1min":0.56499999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.44400000000000001}]},{"osd":2,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.58799999999999997,"15min":0.53000000000000003},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.871,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57299999999999995,"15min":0.54900000000000004},"min":{"1min":0.47099999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69099999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.69999999999999996}]},{"osd":3,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.47499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72299999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.58099999999999996},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90900000000000003,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.48499999999999999}]},{"osd":5,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.621,"15min":0.57499999999999996},"min":{"1min":0.48699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84899999999999998,"5min":1.484,"15min":1.484},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.56100000000000005},"min":{"1min":0.45900000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83999999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54900000000000004}]},{"osd":6,"last update":"Sat Sep 30 14:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.626,"15min":0.626},"min":{"1min":0.501,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78300000000000003,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.439,"5min":0.435,"15min":0.435},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56000000000000005,"15min":0.53900000000000003},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.79800000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61199999999999999,"15min":0.58299999999999996},"min":{"1min":0.47099999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97299999999999998,"5min":1.714,"15min":1.714},"last":0.65700000000000003}]},{"osd":8,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67500000000000004,"15min":0.66000000000000003},"min":{"1min":0.53700000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67600000000000005,"15min":0.66300000000000003},"min":{"1min":0.48099999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.95199999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.753}]},{"osd":9,"last update":"Sat Sep 30 14:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.55100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.85299999999999998,"5min":1.038,"15min":1.038},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.58599999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.95199999999999996,"5min":1.677,"15min":1.677},"last":0.76400000000000001}]},{"osd":10,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.72099999999999997},"min":{"1min":0.41799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.92800000000000005,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.747,"15min":0.751},"min":{"1min":0.52100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0009999999999999,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.69099999999999995}]},{"osd":11,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.85199999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1536000,"data_stored":1033970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.496},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77800000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":3.4380000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53300000000000003,"15min":0.51500000000000001},"min":{"1min":0.35399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77100000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.44500000000000001}]},{"osd":1,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.56399999999999995,"15min":0.50800000000000001},"min":{"1min":0.38600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54000000000000004,"15min":0.499},"min":{"1min":0.46100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77600000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56799999999999995}]},{"osd":2,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.443,"15min":0.41899999999999998},"min":{"1min":0.33500000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.42499999999999999},"min":{"1min":0.32400000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47099999999999997}]},{"osd":3,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.60599999999999998,"15min":0.55900000000000005},"min":{"1min":0.376,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.1050000000000004,"5min":4.1050000000000004,"15min":4.1050000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.65400000000000003,"15min":0.60399999999999998},"min":{"1min":0.53200000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.1630000000000003,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":0.51000000000000001}]},{"osd":4,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.59099999999999997,"15min":0.54700000000000004},"min":{"1min":0.52700000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83799999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60799999999999998,"15min":0.57299999999999995},"min":{"1min":0.439,"5min":0.37,"15min":0.37},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.79800000000000004}]},{"osd":6,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.69499999999999995,"15min":0.66000000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":4.125,"5min":4.125,"15min":4.125},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.1970000000000001,"5min":0.73499999999999999,"15min":0.64700000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":12.24,"5min":12.24,"15min":12.24},"last":3.4660000000000002}]},{"osd":7,"last update":"Sat Sep 30 14:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.66400000000000003,"15min":0.65700000000000003},"min":{"1min":0.52600000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.56499999999999995,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.78500000000000003}]},{"osd":8,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.42599999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90400000000000003,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.754},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64200000000000002,"15min":0.625},"min":{"1min":0.52300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.87,"5min":0.874,"15min":0.874},"last":0.76900000000000002}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.78400000000000003,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.503,"15min":0.503},"max":{"1min":5.1520000000000001,"5min":5.1520000000000001,"15min":5.1520000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":0.874,"15min":0.79300000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":12.198,"5min":12.198,"15min":12.198},"last":0.71499999999999997}]},{"osd":10,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.66900000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.59599999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.81399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":55330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":55436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T14:38:39.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T14:38:39.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.553+0000 7fce3bcce700 1 -- 172.21.15.32:0/4290332332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce34102120 msgr2=0x7fce34102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:39.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.553+0000 7fce3bcce700 1 --2- 172.21.15.32:0/4290332332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce34102120 0x7fce34102580 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fce28009990 tx=0x7fce28009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.553+0000 7fce3bcce700 1 -- 172.21.15.32:0/4290332332 shutdown_connections 2023-09-30T14:38:39.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.553+0000 7fce3bcce700 1 --2- 172.21.15.32:0/4290332332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce34102ac0 0x7fce34109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.553+0000 7fce3bcce700 1 --2- 172.21.15.32:0/4290332332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce340ff150 0x7fce340ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.553+0000 7fce3bcce700 1 --2- 172.21.15.32:0/4290332332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce34102120 0x7fce34102580 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.553+0000 7fce3bcce700 1 -- 172.21.15.32:0/4290332332 >> 172.21.15.32:0/4290332332 conn(0x7fce340faa60 msgr2=0x7fce340fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:39.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.553+0000 7fce3bcce700 1 -- 172.21.15.32:0/4290332332 shutdown_connections 2023-09-30T14:38:39.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.553+0000 7fce3bcce700 1 -- 172.21.15.32:0/4290332332 wait complete. 2023-09-30T14:38:39.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.554+0000 7fce3bcce700 1 Processor -- start 2023-09-30T14:38:39.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.554+0000 7fce3bcce700 1 -- start start 2023-09-30T14:38:39.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.554+0000 7fce3bcce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce340ff150 0x7fce34101a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:39.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.554+0000 7fce3bcce700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce34102120 0x7fce341000d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:39.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.554+0000 7fce3bcce700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce34102ac0 0x7fce34100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:39.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.554+0000 7fce3bcce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce34100db0 con 0x7fce340ff150 2023-09-30T14:38:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.554+0000 7fce3bcce700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce34100f20 con 0x7fce34102ac0 2023-09-30T14:38:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.554+0000 7fce3bcce700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce341a0bd0 con 0x7fce34102120 2023-09-30T14:38:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.554+0000 7fce39a6a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce340ff150 0x7fce34101a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.554+0000 7fce39a6a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce340ff150 0x7fce34101a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35236/0 (socket says 172.21.15.32:35236) 2023-09-30T14:38:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.554+0000 7fce39a6a700 1 -- 172.21.15.32:0/1574040502 learned_addr learned my addr 172.21.15.32:0/1574040502 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.554+0000 7fce39269700 1 --2- 172.21.15.32:0/1574040502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce34102120 0x7fce341000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.555+0000 7fce39a6a700 1 -- 172.21.15.32:0/1574040502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce34102120 msgr2=0x7fce341000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.555+0000 7fce39a6a700 1 --2- 172.21.15.32:0/1574040502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce34102120 0x7fce341000d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.555+0000 7fce39a6a700 1 -- 172.21.15.32:0/1574040502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce34102ac0 msgr2=0x7fce34100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.555+0000 7fce3a26b700 1 --2- 172.21.15.32:0/1574040502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce34102ac0 0x7fce34100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.555+0000 7fce39a6a700 1 --2- 172.21.15.32:0/1574040502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce34102ac0 0x7fce34100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.555+0000 7fce39a6a700 1 -- 172.21.15.32:0/1574040502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce280095f0 con 0x7fce340ff150 2023-09-30T14:38:39.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.555+0000 7fce39269700 1 --2- 172.21.15.32:0/1574040502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce34102120 0x7fce341000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:39.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.555+0000 7fce39a6a700 1 --2- 172.21.15.32:0/1574040502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce340ff150 0x7fce34101a90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fce3000d980 tx=0x7fce3000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:39.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.555+0000 7fce26ffd700 1 -- 172.21.15.32:0/1574040502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3000de60 con 0x7fce340ff150 2023-09-30T14:38:39.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.555+0000 7fce26ffd700 1 -- 172.21.15.32:0/1574040502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce300044e0 con 0x7fce340ff150 2023-09-30T14:38:39.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.555+0000 7fce3bcce700 1 -- 172.21.15.32:0/1574040502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce341a0eb0 con 0x7fce340ff150 2023-09-30T14:38:39.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.555+0000 7fce26ffd700 1 -- 172.21.15.32:0/1574040502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce30003be0 con 0x7fce340ff150 2023-09-30T14:38:39.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.555+0000 7fce3bcce700 1 -- 172.21.15.32:0/1574040502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce341a12d0 con 0x7fce340ff150 2023-09-30T14:38:39.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.556+0000 7fce3bcce700 1 -- 172.21.15.32:0/1574040502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce34108420 con 0x7fce340ff150 2023-09-30T14:38:39.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.558+0000 7fce26ffd700 1 -- 172.21.15.32:0/1574040502 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fce30009bb0 con 0x7fce340ff150 2023-09-30T14:38:39.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.558+0000 7fce26ffd700 1 --2- 172.21.15.32:0/1574040502 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fce200a1070 0x7fce200a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:39.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.558+0000 7fce26ffd700 1 -- 172.21.15.32:0/1574040502 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fce300cc970 con 0x7fce340ff150 2023-09-30T14:38:39.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.558+0000 7fce39269700 1 --2- 172.21.15.32:0/1574040502 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fce200a1070 0x7fce200a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:39.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.559+0000 7fce39269700 1 --2- 172.21.15.32:0/1574040502 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fce200a1070 0x7fce200a3530 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fce2800b5c0 tx=0x7fce28005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:39.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.562+0000 7fce26ffd700 1 -- 172.21.15.32:0/1574040502 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fce30096020 con 0x7fce340ff150 2023-09-30T14:38:39.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:39 smithi032 ceph-mon[98592]: pgmap v298: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-09-30T14:38:39.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:39 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1961372723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:39.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:39 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3520360785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:39.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.758+0000 7fce3bcce700 1 -- 172.21.15.32:0/1574040502 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce3402d350 con 0x7fce200a1070 2023-09-30T14:38:39.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.770+0000 7fce26ffd700 1 -- 172.21.15.32:0/1574040502 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344458 (secure 0 0 0) 0x7fce3402d350 con 0x7fce200a1070 2023-09-30T14:38:39.776 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.774+0000 7fce3bcce700 1 -- 172.21.15.32:0/1574040502 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fce200a1070 msgr2=0x7fce200a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.774+0000 7fce3bcce700 1 --2- 172.21.15.32:0/1574040502 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fce200a1070 0x7fce200a3530 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fce2800b5c0 tx=0x7fce28005fb0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.774+0000 7fce3bcce700 1 -- 172.21.15.32:0/1574040502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce340ff150 msgr2=0x7fce34101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.774+0000 7fce3bcce700 1 --2- 172.21.15.32:0/1574040502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce340ff150 0x7fce34101a90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fce3000d980 tx=0x7fce3000dd40 comp rx=0 tx=0).stop 2023-09-30T14:38:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.774+0000 7fce3bcce700 1 -- 172.21.15.32:0/1574040502 shutdown_connections 2023-09-30T14:38:39.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.774+0000 7fce3bcce700 1 --2- 172.21.15.32:0/1574040502 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fce200a1070 0x7fce200a3530 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.774+0000 7fce3bcce700 1 --2- 172.21.15.32:0/1574040502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce34102ac0 0x7fce34100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.774+0000 7fce3bcce700 1 --2- 172.21.15.32:0/1574040502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce340ff150 0x7fce34101a90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.774+0000 7fce3bcce700 1 --2- 172.21.15.32:0/1574040502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce34102120 0x7fce341000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.774+0000 7fce3bcce700 1 -- 172.21.15.32:0/1574040502 >> 172.21.15.32:0/1574040502 conn(0x7fce340faa60 msgr2=0x7fce34105560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:39.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.774+0000 7fce3bcce700 1 -- 172.21.15.32:0/1574040502 shutdown_connections 2023-09-30T14:38:39.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.774+0000 7fce3bcce700 1 -- 172.21.15.32:0/1574040502 wait complete. 2023-09-30T14:38:39.780 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-09-30T14:38:39.787 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2023-09-30T14:38:37.793844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648932,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":617,"num_read_kb":419,"num_write":332,"num_write_kb":3039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1182052,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337096,"kb_used_data":12832,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604728,"statfs":{"total":1151940427776,"available":1151595241472,"internally_reserved":0,"allocated":13139968,"data_stored":7252214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003027"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197844+0000","last_change":"2023-09-30T14:37:11.988122+0000","last_active":"2023-09-30T14:38:12.197844+0000","last_peered":"2023-09-30T14:38:12.197844+0000","last_clean":"2023-09-30T14:38:12.197844+0000","last_became_active":"2023-09-30T14:37:11.986770+0000","last_became_peered":"2023-09-30T14:37:11.986770+0000","last_unstale":"2023-09-30T14:38:12.197844+0000","last_undegraded":"2023-09-30T14:38:12.197844+0000","last_fullsized":"2023-09-30T14:38:12.197844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:46:29.655309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197069+0000","last_change":"2023-09-30T14:37:11.987953+0000","last_active":"2023-09-30T14:38:12.197069+0000","last_peered":"2023-09-30T14:38:12.197069+0000","last_clean":"2023-09-30T14:38:12.197069+0000","last_became_active":"2023-09-30T14:37:11.987248+0000","last_became_peered":"2023-09-30T14:37:11.987248+0000","last_unstale":"2023-09-30T14:38:12.197069+0000","last_undegraded":"2023-09-30T14:38:12.197069+0000","last_fullsized":"2023-09-30T14:38:12.197069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:12:59.780089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197196+0000","last_change":"2023-09-30T14:37:11.987570+0000","last_active":"2023-09-30T14:38:12.197196+0000","last_peered":"2023-09-30T14:38:12.197196+0000","last_clean":"2023-09-30T14:38:12.197196+0000","last_became_active":"2023-09-30T14:37:11.987064+0000","last_became_peered":"2023-09-30T14:37:11.987064+0000","last_unstale":"2023-09-30T14:38:12.197196+0000","last_undegraded":"2023-09-30T14:38:12.197196+0000","last_fullsized":"2023-09-30T14:38:12.197196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:26:16.674605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197162+0000","last_change":"2023-09-30T14:37:11.990161+0000","last_active":"2023-09-30T14:38:12.197162+0000","last_peered":"2023-09-30T14:38:12.197162+0000","last_clean":"2023-09-30T14:38:12.197162+0000","last_became_active":"2023-09-30T14:37:11.989580+0000","last_became_peered":"2023-09-30T14:37:11.989580+0000","last_unstale":"2023-09-30T14:38:12.197162+0000","last_undegraded":"2023-09-30T14:38:12.197162+0000","last_fullsized":"2023-09-30T14:38:12.197162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:13:06.986864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199326+0000","last_change":"2023-09-30T14:37:11.993902+0000","last_active":"2023-09-30T14:38:12.199326+0000","last_peered":"2023-09-30T14:38:12.199326+0000","last_clean":"2023-09-30T14:38:12.199326+0000","last_became_active":"2023-09-30T14:37:11.993622+0000","last_became_peered":"2023-09-30T14:37:11.993622+0000","last_unstale":"2023-09-30T14:38:12.199326+0000","last_undegraded":"2023-09-30T14:38:12.199326+0000","last_fullsized":"2023-09-30T14:38:12.199326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:09:49.252823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195466+0000","last_change":"2023-09-30T14:37:11.997282+0000","last_active":"2023-09-30T14:38:12.195466+0000","last_peered":"2023-09-30T14:38:12.195466+0000","last_clean":"2023-09-30T14:38:12.195466+0000","last_became_active":"2023-09-30T14:37:11.996937+0000","last_became_peered":"2023-09-30T14:37:11.996937+0000","last_unstale":"2023-09-30T14:38:12.195466+0000","last_undegraded":"2023-09-30T14:38:12.195466+0000","last_fullsized":"2023-09-30T14:38:12.195466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:39:27.351772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199172+0000","last_change":"2023-09-30T14:37:11.997408+0000","last_active":"2023-09-30T14:38:12.199172+0000","last_peered":"2023-09-30T14:38:12.199172+0000","last_clean":"2023-09-30T14:38:12.199172+0000","last_became_active":"2023-09-30T14:37:11.997147+0000","last_became_peered":"2023-09-30T14:37:11.997147+0000","last_unstale":"2023-09-30T14:38:12.199172+0000","last_undegraded":"2023-09-30T14:38:12.199172+0000","last_fullsized":"2023-09-30T14:38:12.199172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:21:24.721417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.823924+0000","last_change":"2023-09-30T14:37:11.986172+0000","last_active":"2023-09-30T14:38:12.823924+0000","last_peered":"2023-09-30T14:38:12.823924+0000","last_clean":"2023-09-30T14:38:12.823924+0000","last_became_active":"2023-09-30T14:37:11.985836+0000","last_became_peered":"2023-09-30T14:37:11.985836+0000","last_unstale":"2023-09-30T14:38:12.823924+0000","last_undegraded":"2023-09-30T14:38:12.823924+0000","last_fullsized":"2023-09-30T14:38:12.823924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:35:57.464679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198360+0000","last_change":"2023-09-30T14:37:11.990853+0000","last_active":"2023-09-30T14:38:12.198360+0000","last_peered":"2023-09-30T14:38:12.198360+0000","last_clean":"2023-09-30T14:38:12.198360+0000","last_became_active":"2023-09-30T14:37:11.990321+0000","last_became_peered":"2023-09-30T14:37:11.990321+0000","last_unstale":"2023-09-30T14:38:12.198360+0000","last_undegraded":"2023-09-30T14:38:12.198360+0000","last_fullsized":"2023-09-30T14:38:12.198360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:32:11.409927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.202453+0000","last_change":"2023-09-30T14:38:13.202453+0000","last_active":"2023-09-30T14:38:13.202453+0000","last_peered":"2023-09-30T14:38:13.202453+0000","last_clean":"2023-09-30T14:38:13.202453+0000","last_became_active":"2023-09-30T14:38:13.202023+0000","last_became_peered":"2023-09-30T14:38:13.202023+0000","last_unstale":"2023-09-30T14:38:13.202453+0000","last_undegraded":"2023-09-30T14:38:13.202453+0000","last_fullsized":"2023-09-30T14:38:13.202453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:50:40.737940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198974+0000","last_change":"2023-09-30T14:37:11.992213+0000","last_active":"2023-09-30T14:38:12.198974+0000","last_peered":"2023-09-30T14:38:12.198974+0000","last_clean":"2023-09-30T14:38:12.198974+0000","last_became_active":"2023-09-30T14:37:11.991868+0000","last_became_peered":"2023-09-30T14:37:11.991868+0000","last_unstale":"2023-09-30T14:38:12.198974+0000","last_undegraded":"2023-09-30T14:38:12.198974+0000","last_fullsized":"2023-09-30T14:38:12.198974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:20:06.061978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197227+0000","last_change":"2023-09-30T14:37:11.990043+0000","last_active":"2023-09-30T14:38:12.197227+0000","last_peered":"2023-09-30T14:38:12.197227+0000","last_clean":"2023-09-30T14:38:12.197227+0000","last_became_active":"2023-09-30T14:37:11.988243+0000","last_became_peered":"2023-09-30T14:37:11.988243+0000","last_unstale":"2023-09-30T14:38:12.197227+0000","last_undegraded":"2023-09-30T14:38:12.197227+0000","last_fullsized":"2023-09-30T14:38:12.197227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:17:58.946159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194482+0000","last_change":"2023-09-30T14:37:11.990361+0000","last_active":"2023-09-30T14:38:12.194482+0000","last_peered":"2023-09-30T14:38:12.194482+0000","last_clean":"2023-09-30T14:38:12.194482+0000","last_became_active":"2023-09-30T14:37:11.990080+0000","last_became_peered":"2023-09-30T14:37:11.990080+0000","last_unstale":"2023-09-30T14:38:12.194482+0000","last_undegraded":"2023-09-30T14:38:12.194482+0000","last_fullsized":"2023-09-30T14:38:12.194482+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:27:08.830419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197518+0000","last_change":"2023-09-30T14:37:11.988119+0000","last_active":"2023-09-30T14:38:12.197518+0000","last_peered":"2023-09-30T14:38:12.197518+0000","last_clean":"2023-09-30T14:38:12.197518+0000","last_became_active":"2023-09-30T14:37:11.987704+0000","last_became_peered":"2023-09-30T14:37:11.987704+0000","last_unstale":"2023-09-30T14:38:12.197518+0000","last_undegraded":"2023-09-30T14:38:12.197518+0000","last_fullsized":"2023-09-30T14:38:12.197518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:47:23.318982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196560+0000","last_change":"2023-09-30T14:37:11.985403+0000","last_active":"2023-09-30T14:38:12.196560+0000","last_peered":"2023-09-30T14:38:12.196560+0000","last_clean":"2023-09-30T14:38:12.196560+0000","last_became_active":"2023-09-30T14:37:11.984818+0000","last_became_peered":"2023-09-30T14:37:11.984818+0000","last_unstale":"2023-09-30T14:38:12.196560+0000","last_undegraded":"2023-09-30T14:38:12.196560+0000","last_fullsized":"2023-09-30T14:38:12.196560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:10:43.592505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198212+0000","last_change":"2023-09-30T14:37:11.989500+0000","last_active":"2023-09-30T14:38:12.198212+0000","last_peered":"2023-09-30T14:38:12.198212+0000","last_clean":"2023-09-30T14:38:12.198212+0000","last_became_active":"2023-09-30T14:37:11.989186+0000","last_became_peered":"2023-09-30T14:37:11.989186+0000","last_unstale":"2023-09-30T14:38:12.198212+0000","last_undegraded":"2023-09-30T14:38:12.198212+0000","last_fullsized":"2023-09-30T14:38:12.198212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:14:35.497319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197968+0000","last_change":"2023-09-30T14:37:11.987217+0000","last_active":"2023-09-30T14:38:12.197968+0000","last_peered":"2023-09-30T14:38:12.197968+0000","last_clean":"2023-09-30T14:38:12.197968+0000","last_became_active":"2023-09-30T14:37:11.986835+0000","last_became_peered":"2023-09-30T14:37:11.986835+0000","last_unstale":"2023-09-30T14:38:12.197968+0000","last_undegraded":"2023-09-30T14:38:12.197968+0000","last_fullsized":"2023-09-30T14:38:12.197968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:11:19.234768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196774+0000","last_change":"2023-09-30T14:37:11.995224+0000","last_active":"2023-09-30T14:38:12.196774+0000","last_peered":"2023-09-30T14:38:12.196774+0000","last_clean":"2023-09-30T14:38:12.196774+0000","last_became_active":"2023-09-30T14:37:11.994925+0000","last_became_peered":"2023-09-30T14:37:11.994925+0000","last_unstale":"2023-09-30T14:38:12.196774+0000","last_undegraded":"2023-09-30T14:38:12.196774+0000","last_fullsized":"2023-09-30T14:38:12.196774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:08:07.453499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198775+0000","last_change":"2023-09-30T14:37:11.990182+0000","last_active":"2023-09-30T14:38:12.198775+0000","last_peered":"2023-09-30T14:38:12.198775+0000","last_clean":"2023-09-30T14:38:12.198775+0000","last_became_active":"2023-09-30T14:37:11.989862+0000","last_became_peered":"2023-09-30T14:37:11.989862+0000","last_unstale":"2023-09-30T14:38:12.198775+0000","last_undegraded":"2023-09-30T14:38:12.198775+0000","last_fullsized":"2023-09-30T14:38:12.198775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:15:31.713002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.209988+0000","last_change":"2023-09-30T14:38:13.209988+0000","last_active":"2023-09-30T14:38:13.209988+0000","last_peered":"2023-09-30T14:38:13.209988+0000","last_clean":"2023-09-30T14:38:13.209988+0000","last_became_active":"2023-09-30T14:38:13.202561+0000","last_became_peered":"2023-09-30T14:38:13.202561+0000","last_unstale":"2023-09-30T14:38:13.209988+0000","last_undegraded":"2023-09-30T14:38:13.209988+0000","last_fullsized":"2023-09-30T14:38:13.209988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:57:36.214608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824419+0000","last_change":"2023-09-30T14:37:11.994584+0000","last_active":"2023-09-30T14:38:12.824419+0000","last_peered":"2023-09-30T14:38:12.824419+0000","last_clean":"2023-09-30T14:38:12.824419+0000","last_became_active":"2023-09-30T14:37:11.992991+0000","last_became_peered":"2023-09-30T14:37:11.992991+0000","last_unstale":"2023-09-30T14:38:12.824419+0000","last_undegraded":"2023-09-30T14:38:12.824419+0000","last_fullsized":"2023-09-30T14:38:12.824419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:39:50.995717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824532+0000","last_change":"2023-09-30T14:37:11.020653+0000","last_active":"2023-09-30T14:38:12.824532+0000","last_peered":"2023-09-30T14:38:12.824532+0000","last_clean":"2023-09-30T14:38:12.824532+0000","last_became_active":"2023-09-30T14:37:11.017621+0000","last_became_peered":"2023-09-30T14:37:11.017621+0000","last_unstale":"2023-09-30T14:38:12.824532+0000","last_undegraded":"2023-09-30T14:38:12.824532+0000","last_fullsized":"2023-09-30T14:38:12.824532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:26:59.395278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196664+0000","last_change":"2023-09-30T14:37:11.999881+0000","last_active":"2023-09-30T14:38:12.196664+0000","last_peered":"2023-09-30T14:38:12.196664+0000","last_clean":"2023-09-30T14:38:12.196664+0000","last_became_active":"2023-09-30T14:37:11.999614+0000","last_became_peered":"2023-09-30T14:37:11.999614+0000","last_unstale":"2023-09-30T14:38:12.196664+0000","last_undegraded":"2023-09-30T14:38:12.196664+0000","last_fullsized":"2023-09-30T14:38:12.196664+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:59:44.198361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200580+0000","last_change":"2023-09-30T14:37:11.027187+0000","last_active":"2023-09-30T14:38:12.200580+0000","last_peered":"2023-09-30T14:38:12.200580+0000","last_clean":"2023-09-30T14:38:12.200580+0000","last_became_active":"2023-09-30T14:37:11.026882+0000","last_became_peered":"2023-09-30T14:37:11.026882+0000","last_unstale":"2023-09-30T14:38:12.200580+0000","last_undegraded":"2023-09-30T14:38:12.200580+0000","last_fullsized":"2023-09-30T14:38:12.200580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:17:11.824002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198427+0000","last_change":"2023-09-30T14:37:11.986979+0000","last_active":"2023-09-30T14:38:12.198427+0000","last_peered":"2023-09-30T14:38:12.198427+0000","last_clean":"2023-09-30T14:38:12.198427+0000","last_became_active":"2023-09-30T14:37:11.986705+0000","last_became_peered":"2023-09-30T14:37:11.986705+0000","last_unstale":"2023-09-30T14:38:12.198427+0000","last_undegraded":"2023-09-30T14:38:12.198427+0000","last_fullsized":"2023-09-30T14:38:12.198427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:31:40.319747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199932+0000","last_change":"2023-09-30T14:37:11.022313+0000","last_active":"2023-09-30T14:38:12.199932+0000","last_peered":"2023-09-30T14:38:12.199932+0000","last_clean":"2023-09-30T14:38:12.199932+0000","last_became_active":"2023-09-30T14:37:11.021949+0000","last_became_peered":"2023-09-30T14:37:11.021949+0000","last_unstale":"2023-09-30T14:38:12.199932+0000","last_undegraded":"2023-09-30T14:38:12.199932+0000","last_fullsized":"2023-09-30T14:38:12.199932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:30:26.913666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.202706+0000","last_change":"2023-09-30T14:38:13.202706+0000","last_active":"2023-09-30T14:38:13.202706+0000","last_peered":"2023-09-30T14:38:13.202706+0000","last_clean":"2023-09-30T14:38:13.202706+0000","last_became_active":"2023-09-30T14:38:13.202370+0000","last_became_peered":"2023-09-30T14:38:13.202370+0000","last_unstale":"2023-09-30T14:38:13.202706+0000","last_undegraded":"2023-09-30T14:38:13.202706+0000","last_fullsized":"2023-09-30T14:38:13.202706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:35:21.244531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":117,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:36.239380+0000","last_change":"2023-09-30T14:38:13.206650+0000","last_active":"2023-09-30T14:38:36.239380+0000","last_peered":"2023-09-30T14:38:36.239380+0000","last_clean":"2023-09-30T14:38:36.239380+0000","last_became_active":"2023-09-30T14:38:13.203337+0000","last_became_peered":"2023-09-30T14:38:13.203337+0000","last_unstale":"2023-09-30T14:38:36.239380+0000","last_undegraded":"2023-09-30T14:38:36.239380+0000","last_fullsized":"2023-09-30T14:38:36.239380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:08:37.996166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199571+0000","last_change":"2023-09-30T14:37:11.993074+0000","last_active":"2023-09-30T14:38:12.199571+0000","last_peered":"2023-09-30T14:38:12.199571+0000","last_clean":"2023-09-30T14:38:12.199571+0000","last_became_active":"2023-09-30T14:37:11.992784+0000","last_became_peered":"2023-09-30T14:37:11.992784+0000","last_unstale":"2023-09-30T14:38:12.199571+0000","last_undegraded":"2023-09-30T14:38:12.199571+0000","last_fullsized":"2023-09-30T14:38:12.199571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:28:18.454389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195465+0000","last_change":"2023-09-30T14:37:11.022677+0000","last_active":"2023-09-30T14:38:12.195465+0000","last_peered":"2023-09-30T14:38:12.195465+0000","last_clean":"2023-09-30T14:38:12.195465+0000","last_became_active":"2023-09-30T14:37:11.022357+0000","last_became_peered":"2023-09-30T14:37:11.022357+0000","last_unstale":"2023-09-30T14:38:12.195465+0000","last_undegraded":"2023-09-30T14:38:12.195465+0000","last_fullsized":"2023-09-30T14:38:12.195465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:00:15.500968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194862+0000","last_change":"2023-09-30T14:37:11.992358+0000","last_active":"2023-09-30T14:38:12.194862+0000","last_peered":"2023-09-30T14:38:12.194862+0000","last_clean":"2023-09-30T14:38:12.194862+0000","last_became_active":"2023-09-30T14:37:11.992094+0000","last_became_peered":"2023-09-30T14:37:11.992094+0000","last_unstale":"2023-09-30T14:38:12.194862+0000","last_undegraded":"2023-09-30T14:38:12.194862+0000","last_fullsized":"2023-09-30T14:38:12.194862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:59:36.817138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194237+0000","last_change":"2023-09-30T14:37:11.030961+0000","last_active":"2023-09-30T14:38:12.194237+0000","last_peered":"2023-09-30T14:38:12.194237+0000","last_clean":"2023-09-30T14:38:12.194237+0000","last_became_active":"2023-09-30T14:37:11.030662+0000","last_became_peered":"2023-09-30T14:37:11.030662+0000","last_unstale":"2023-09-30T14:38:12.194237+0000","last_undegraded":"2023-09-30T14:38:12.194237+0000","last_fullsized":"2023-09-30T14:38:12.194237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:35:13.313382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824002+0000","last_change":"2023-09-30T14:37:11.989039+0000","last_active":"2023-09-30T14:38:12.824002+0000","last_peered":"2023-09-30T14:38:12.824002+0000","last_clean":"2023-09-30T14:38:12.824002+0000","last_became_active":"2023-09-30T14:37:11.988594+0000","last_became_peered":"2023-09-30T14:37:11.988594+0000","last_unstale":"2023-09-30T14:38:12.824002+0000","last_undegraded":"2023-09-30T14:38:12.824002+0000","last_fullsized":"2023-09-30T14:38:12.824002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:52:58.146681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194495+0000","last_change":"2023-09-30T14:37:11.026545+0000","last_active":"2023-09-30T14:38:12.194495+0000","last_peered":"2023-09-30T14:38:12.194495+0000","last_clean":"2023-09-30T14:38:12.194495+0000","last_became_active":"2023-09-30T14:37:11.026063+0000","last_became_peered":"2023-09-30T14:37:11.026063+0000","last_unstale":"2023-09-30T14:38:12.194495+0000","last_undegraded":"2023-09-30T14:38:12.194495+0000","last_fullsized":"2023-09-30T14:38:12.194495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:15:34.020336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196143+0000","last_change":"2023-09-30T14:37:11.991845+0000","last_active":"2023-09-30T14:38:12.196143+0000","last_peered":"2023-09-30T14:38:12.196143+0000","last_clean":"2023-09-30T14:38:12.196143+0000","last_became_active":"2023-09-30T14:37:11.991560+0000","last_became_peered":"2023-09-30T14:37:11.991560+0000","last_unstale":"2023-09-30T14:38:12.196143+0000","last_undegraded":"2023-09-30T14:38:12.196143+0000","last_fullsized":"2023-09-30T14:38:12.196143+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:27:39.653882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":97,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:34.235519+0000","last_change":"2023-09-30T14:37:11.028192+0000","last_active":"2023-09-30T14:38:34.235519+0000","last_peered":"2023-09-30T14:38:34.235519+0000","last_clean":"2023-09-30T14:38:34.235519+0000","last_became_active":"2023-09-30T14:37:11.027902+0000","last_became_peered":"2023-09-30T14:37:11.027902+0000","last_unstale":"2023-09-30T14:38:34.235519+0000","last_undegraded":"2023-09-30T14:38:34.235519+0000","last_fullsized":"2023-09-30T14:38:34.235519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:21:30.267559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198607+0000","last_change":"2023-09-30T14:37:11.997318+0000","last_active":"2023-09-30T14:38:12.198607+0000","last_peered":"2023-09-30T14:38:12.198607+0000","last_clean":"2023-09-30T14:38:12.198607+0000","last_became_active":"2023-09-30T14:37:11.996988+0000","last_became_peered":"2023-09-30T14:37:11.996988+0000","last_unstale":"2023-09-30T14:38:12.198607+0000","last_undegraded":"2023-09-30T14:38:12.198607+0000","last_fullsized":"2023-09-30T14:38:12.198607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:57:21.510952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.259803+0000","last_change":"2023-09-30T14:38:13.259803+0000","last_active":"2023-09-30T14:38:13.259803+0000","last_peered":"2023-09-30T14:38:13.259803+0000","last_clean":"2023-09-30T14:38:13.259803+0000","last_became_active":"2023-09-30T14:38:13.257187+0000","last_became_peered":"2023-09-30T14:38:13.257187+0000","last_unstale":"2023-09-30T14:38:13.259803+0000","last_undegraded":"2023-09-30T14:38:13.259803+0000","last_fullsized":"2023-09-30T14:38:13.259803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:37:41.453519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196748+0000","last_change":"2023-09-30T14:37:11.989338+0000","last_active":"2023-09-30T14:38:12.196748+0000","last_peered":"2023-09-30T14:38:12.196748+0000","last_clean":"2023-09-30T14:38:12.196748+0000","last_became_active":"2023-09-30T14:37:11.989045+0000","last_became_peered":"2023-09-30T14:37:11.989045+0000","last_unstale":"2023-09-30T14:38:12.196748+0000","last_undegraded":"2023-09-30T14:38:12.196748+0000","last_fullsized":"2023-09-30T14:38:12.196748+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:58:59.359684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195371+0000","last_change":"2023-09-30T14:37:11.024709+0000","last_active":"2023-09-30T14:38:12.195371+0000","last_peered":"2023-09-30T14:38:12.195371+0000","last_clean":"2023-09-30T14:38:12.195371+0000","last_became_active":"2023-09-30T14:37:11.024344+0000","last_became_peered":"2023-09-30T14:37:11.024344+0000","last_unstale":"2023-09-30T14:38:12.195371+0000","last_undegraded":"2023-09-30T14:38:12.195371+0000","last_fullsized":"2023-09-30T14:38:12.195371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:17:49.508396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198298+0000","last_change":"2023-09-30T14:37:11.989797+0000","last_active":"2023-09-30T14:38:12.198298+0000","last_peered":"2023-09-30T14:38:12.198298+0000","last_clean":"2023-09-30T14:38:12.198298+0000","last_became_active":"2023-09-30T14:37:11.989482+0000","last_became_peered":"2023-09-30T14:37:11.989482+0000","last_unstale":"2023-09-30T14:38:12.198298+0000","last_undegraded":"2023-09-30T14:38:12.198298+0000","last_fullsized":"2023-09-30T14:38:12.198298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:33:20.437706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'26","reported_seq":206,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:35.237410+0000","last_change":"2023-09-30T14:37:11.027322+0000","last_active":"2023-09-30T14:38:35.237410+0000","last_peered":"2023-09-30T14:38:35.237410+0000","last_clean":"2023-09-30T14:38:35.237410+0000","last_became_active":"2023-09-30T14:37:11.027007+0000","last_became_peered":"2023-09-30T14:37:11.027007+0000","last_unstale":"2023-09-30T14:38:35.237410+0000","last_undegraded":"2023-09-30T14:38:35.237410+0000","last_fullsized":"2023-09-30T14:38:35.237410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:47:22.919317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54708,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":165,"num_read_kb":135,"num_write":27,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200357+0000","last_change":"2023-09-30T14:37:11.988469+0000","last_active":"2023-09-30T14:38:12.200357+0000","last_peered":"2023-09-30T14:38:12.200357+0000","last_clean":"2023-09-30T14:38:12.200357+0000","last_became_active":"2023-09-30T14:37:11.988118+0000","last_became_peered":"2023-09-30T14:37:11.988118+0000","last_unstale":"2023-09-30T14:38:12.200357+0000","last_undegraded":"2023-09-30T14:38:12.200357+0000","last_fullsized":"2023-09-30T14:38:12.200357+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:11:56.750690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198520+0000","last_change":"2023-09-30T14:37:11.024345+0000","last_active":"2023-09-30T14:38:12.198520+0000","last_peered":"2023-09-30T14:38:12.198520+0000","last_clean":"2023-09-30T14:38:12.198520+0000","last_became_active":"2023-09-30T14:37:11.024039+0000","last_became_peered":"2023-09-30T14:37:11.024039+0000","last_unstale":"2023-09-30T14:38:12.198520+0000","last_undegraded":"2023-09-30T14:38:12.198520+0000","last_fullsized":"2023-09-30T14:38:12.198520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:48:32.872814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194273+0000","last_change":"2023-09-30T14:37:11.994068+0000","last_active":"2023-09-30T14:38:12.194273+0000","last_peered":"2023-09-30T14:38:12.194273+0000","last_clean":"2023-09-30T14:38:12.194273+0000","last_became_active":"2023-09-30T14:37:11.993793+0000","last_became_peered":"2023-09-30T14:37:11.993793+0000","last_unstale":"2023-09-30T14:38:12.194273+0000","last_undegraded":"2023-09-30T14:38:12.194273+0000","last_fullsized":"2023-09-30T14:38:12.194273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:51:28.071414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194699+0000","last_change":"2023-09-30T14:37:11.025622+0000","last_active":"2023-09-30T14:38:12.194699+0000","last_peered":"2023-09-30T14:38:12.194699+0000","last_clean":"2023-09-30T14:38:12.194699+0000","last_became_active":"2023-09-30T14:37:11.025338+0000","last_became_peered":"2023-09-30T14:37:11.025338+0000","last_unstale":"2023-09-30T14:38:12.194699+0000","last_undegraded":"2023-09-30T14:38:12.194699+0000","last_fullsized":"2023-09-30T14:38:12.194699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:32:03.166296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200744+0000","last_change":"2023-09-30T14:37:11.990678+0000","last_active":"2023-09-30T14:38:12.200744+0000","last_peered":"2023-09-30T14:38:12.200744+0000","last_clean":"2023-09-30T14:38:12.200744+0000","last_became_active":"2023-09-30T14:37:11.990331+0000","last_became_peered":"2023-09-30T14:37:11.990331+0000","last_unstale":"2023-09-30T14:38:12.200744+0000","last_undegraded":"2023-09-30T14:38:12.200744+0000","last_fullsized":"2023-09-30T14:38:12.200744+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:57:06.296415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":96,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:31.092801+0000","last_change":"2023-09-30T14:37:11.029501+0000","last_active":"2023-09-30T14:38:31.092801+0000","last_peered":"2023-09-30T14:38:31.092801+0000","last_clean":"2023-09-30T14:38:31.092801+0000","last_became_active":"2023-09-30T14:37:11.029207+0000","last_became_peered":"2023-09-30T14:37:11.029207+0000","last_unstale":"2023-09-30T14:38:31.092801+0000","last_undegraded":"2023-09-30T14:38:31.092801+0000","last_fullsized":"2023-09-30T14:38:31.092801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:48:53.145952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198725+0000","last_change":"2023-09-30T14:37:11.993052+0000","last_active":"2023-09-30T14:38:12.198725+0000","last_peered":"2023-09-30T14:38:12.198725+0000","last_clean":"2023-09-30T14:38:12.198725+0000","last_became_active":"2023-09-30T14:37:11.992762+0000","last_became_peered":"2023-09-30T14:37:11.992762+0000","last_unstale":"2023-09-30T14:38:12.198725+0000","last_undegraded":"2023-09-30T14:38:12.198725+0000","last_fullsized":"2023-09-30T14:38:12.198725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:09:16.440026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.214212+0000","last_change":"2023-09-30T14:38:13.213859+0000","last_active":"2023-09-30T14:38:13.214212+0000","last_peered":"2023-09-30T14:38:13.214212+0000","last_clean":"2023-09-30T14:38:13.214212+0000","last_became_active":"2023-09-30T14:38:13.200010+0000","last_became_peered":"2023-09-30T14:38:13.200010+0000","last_unstale":"2023-09-30T14:38:13.214212+0000","last_undegraded":"2023-09-30T14:38:13.214212+0000","last_fullsized":"2023-09-30T14:38:13.214212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:08:48.668186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194503+0000","last_change":"2023-09-30T14:37:11.989592+0000","last_active":"2023-09-30T14:38:12.194503+0000","last_peered":"2023-09-30T14:38:12.194503+0000","last_clean":"2023-09-30T14:38:12.194503+0000","last_became_active":"2023-09-30T14:37:11.989290+0000","last_became_peered":"2023-09-30T14:37:11.989290+0000","last_unstale":"2023-09-30T14:38:12.194503+0000","last_undegraded":"2023-09-30T14:38:12.194503+0000","last_fullsized":"2023-09-30T14:38:12.194503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:11:33.974823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.201778+0000","last_change":"2023-09-30T14:37:11.028919+0000","last_active":"2023-09-30T14:38:13.201778+0000","last_peered":"2023-09-30T14:38:13.201778+0000","last_clean":"2023-09-30T14:38:13.201778+0000","last_became_active":"2023-09-30T14:37:11.028532+0000","last_became_peered":"2023-09-30T14:37:11.028532+0000","last_unstale":"2023-09-30T14:38:13.201778+0000","last_undegraded":"2023-09-30T14:38:13.201778+0000","last_fullsized":"2023-09-30T14:38:13.201778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:15:42.675409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197404+0000","last_change":"2023-09-30T14:37:11.980611+0000","last_active":"2023-09-30T14:38:12.197404+0000","last_peered":"2023-09-30T14:38:12.197404+0000","last_clean":"2023-09-30T14:38:12.197404+0000","last_became_active":"2023-09-30T14:37:11.980288+0000","last_became_peered":"2023-09-30T14:37:11.980288+0000","last_unstale":"2023-09-30T14:38:12.197404+0000","last_undegraded":"2023-09-30T14:38:12.197404+0000","last_fullsized":"2023-09-30T14:38:12.197404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:38:49.270976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196478+0000","last_change":"2023-09-30T14:37:11.025133+0000","last_active":"2023-09-30T14:38:12.196478+0000","last_peered":"2023-09-30T14:38:12.196478+0000","last_clean":"2023-09-30T14:38:12.196478+0000","last_became_active":"2023-09-30T14:37:11.024914+0000","last_became_peered":"2023-09-30T14:37:11.024914+0000","last_unstale":"2023-09-30T14:38:12.196478+0000","last_undegraded":"2023-09-30T14:38:12.196478+0000","last_fullsized":"2023-09-30T14:38:12.196478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:23:39.328773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195743+0000","last_change":"2023-09-30T14:37:11.988434+0000","last_active":"2023-09-30T14:38:12.195743+0000","last_peered":"2023-09-30T14:38:12.195743+0000","last_clean":"2023-09-30T14:38:12.195743+0000","last_became_active":"2023-09-30T14:37:11.988156+0000","last_became_peered":"2023-09-30T14:37:11.988156+0000","last_unstale":"2023-09-30T14:38:12.195743+0000","last_undegraded":"2023-09-30T14:38:12.195743+0000","last_fullsized":"2023-09-30T14:38:12.195743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:20:49.420026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198842+0000","last_change":"2023-09-30T14:37:11.026290+0000","last_active":"2023-09-30T14:38:12.198842+0000","last_peered":"2023-09-30T14:38:12.198842+0000","last_clean":"2023-09-30T14:38:12.198842+0000","last_became_active":"2023-09-30T14:37:11.024684+0000","last_became_peered":"2023-09-30T14:37:11.024684+0000","last_unstale":"2023-09-30T14:38:12.198842+0000","last_undegraded":"2023-09-30T14:38:12.198842+0000","last_fullsized":"2023-09-30T14:38:12.198842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:43:06.463243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195851+0000","last_change":"2023-09-30T14:37:11.987989+0000","last_active":"2023-09-30T14:38:12.195851+0000","last_peered":"2023-09-30T14:38:12.195851+0000","last_clean":"2023-09-30T14:38:12.195851+0000","last_became_active":"2023-09-30T14:37:11.987690+0000","last_became_peered":"2023-09-30T14:37:11.987690+0000","last_unstale":"2023-09-30T14:38:12.195851+0000","last_undegraded":"2023-09-30T14:38:12.195851+0000","last_fullsized":"2023-09-30T14:38:12.195851+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:51:25.825871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194553+0000","last_change":"2023-09-30T14:37:11.996852+0000","last_active":"2023-09-30T14:38:12.194553+0000","last_peered":"2023-09-30T14:38:12.194553+0000","last_clean":"2023-09-30T14:38:12.194553+0000","last_became_active":"2023-09-30T14:37:11.996596+0000","last_became_peered":"2023-09-30T14:37:11.996596+0000","last_unstale":"2023-09-30T14:38:12.194553+0000","last_undegraded":"2023-09-30T14:38:12.194553+0000","last_fullsized":"2023-09-30T14:38:12.194553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:25:04.531236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":98,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:34.096608+0000","last_change":"2023-09-30T14:37:11.025458+0000","last_active":"2023-09-30T14:38:34.096608+0000","last_peered":"2023-09-30T14:38:34.096608+0000","last_clean":"2023-09-30T14:38:34.096608+0000","last_became_active":"2023-09-30T14:37:11.025157+0000","last_became_peered":"2023-09-30T14:37:11.025157+0000","last_unstale":"2023-09-30T14:38:34.096608+0000","last_undegraded":"2023-09-30T14:38:34.096608+0000","last_fullsized":"2023-09-30T14:38:34.096608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:31:24.436125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824345+0000","last_change":"2023-09-30T14:37:11.986840+0000","last_active":"2023-09-30T14:38:12.824345+0000","last_peered":"2023-09-30T14:38:12.824345+0000","last_clean":"2023-09-30T14:38:12.824345+0000","last_became_active":"2023-09-30T14:37:11.986483+0000","last_became_peered":"2023-09-30T14:37:11.986483+0000","last_unstale":"2023-09-30T14:38:12.824345+0000","last_undegraded":"2023-09-30T14:38:12.824345+0000","last_fullsized":"2023-09-30T14:38:12.824345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:14:32.399685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196845+0000","last_change":"2023-09-30T14:37:11.025706+0000","last_active":"2023-09-30T14:38:12.196845+0000","last_peered":"2023-09-30T14:38:12.196845+0000","last_clean":"2023-09-30T14:38:12.196845+0000","last_became_active":"2023-09-30T14:37:11.025376+0000","last_became_peered":"2023-09-30T14:37:11.025376+0000","last_unstale":"2023-09-30T14:38:12.196845+0000","last_undegraded":"2023-09-30T14:38:12.196845+0000","last_fullsized":"2023-09-30T14:38:12.196845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:10:36.025273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":310,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198525+0000","last_change":"2023-09-30T14:36:12.344762+0000","last_active":"2023-09-30T14:38:12.198525+0000","last_peered":"2023-09-30T14:38:12.198525+0000","last_clean":"2023-09-30T14:38:12.198525+0000","last_became_active":"2023-09-30T14:36:12.344268+0000","last_became_peered":"2023-09-30T14:36:12.344268+0000","last_unstale":"2023-09-30T14:38:12.198525+0000","last_undegraded":"2023-09-30T14:38:12.198525+0000","last_fullsized":"2023-09-30T14:38:12.198525+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:33:20.434992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:33:20.434992+0000","last_clean_scrub_stamp":"2023-09-30T14:33:20.434992+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:00:05.598961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194122+0000","last_change":"2023-09-30T14:37:11.996886+0000","last_active":"2023-09-30T14:38:12.194122+0000","last_peered":"2023-09-30T14:38:12.194122+0000","last_clean":"2023-09-30T14:38:12.194122+0000","last_became_active":"2023-09-30T14:37:11.996667+0000","last_became_peered":"2023-09-30T14:37:11.996667+0000","last_unstale":"2023-09-30T14:38:12.194122+0000","last_undegraded":"2023-09-30T14:38:12.194122+0000","last_fullsized":"2023-09-30T14:38:12.194122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:27:00.546356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198414+0000","last_change":"2023-09-30T14:37:11.027012+0000","last_active":"2023-09-30T14:38:12.198414+0000","last_peered":"2023-09-30T14:38:12.198414+0000","last_clean":"2023-09-30T14:38:12.198414+0000","last_became_active":"2023-09-30T14:37:11.026714+0000","last_became_peered":"2023-09-30T14:37:11.026714+0000","last_unstale":"2023-09-30T14:38:12.198414+0000","last_undegraded":"2023-09-30T14:38:12.198414+0000","last_fullsized":"2023-09-30T14:38:12.198414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:17:57.860438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200462+0000","last_change":"2023-09-30T14:37:11.027389+0000","last_active":"2023-09-30T14:38:12.200462+0000","last_peered":"2023-09-30T14:38:12.200462+0000","last_clean":"2023-09-30T14:38:12.200462+0000","last_became_active":"2023-09-30T14:37:11.027011+0000","last_became_peered":"2023-09-30T14:37:11.027011+0000","last_unstale":"2023-09-30T14:38:12.200462+0000","last_undegraded":"2023-09-30T14:38:12.200462+0000","last_fullsized":"2023-09-30T14:38:12.200462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:38:22.953936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194384+0000","last_change":"2023-09-30T14:37:11.997675+0000","last_active":"2023-09-30T14:38:12.194384+0000","last_peered":"2023-09-30T14:38:12.194384+0000","last_clean":"2023-09-30T14:38:12.194384+0000","last_became_active":"2023-09-30T14:37:11.997330+0000","last_became_peered":"2023-09-30T14:37:11.997330+0000","last_unstale":"2023-09-30T14:38:12.194384+0000","last_undegraded":"2023-09-30T14:38:12.194384+0000","last_fullsized":"2023-09-30T14:38:12.194384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:54:52.802138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196687+0000","last_change":"2023-09-30T14:37:11.023836+0000","last_active":"2023-09-30T14:38:12.196687+0000","last_peered":"2023-09-30T14:38:12.196687+0000","last_clean":"2023-09-30T14:38:12.196687+0000","last_became_active":"2023-09-30T14:37:11.023362+0000","last_became_peered":"2023-09-30T14:37:11.023362+0000","last_unstale":"2023-09-30T14:38:12.196687+0000","last_undegraded":"2023-09-30T14:38:12.196687+0000","last_fullsized":"2023-09-30T14:38:12.196687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:58:22.772546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824591+0000","last_change":"2023-09-30T14:37:11.985281+0000","last_active":"2023-09-30T14:38:12.824591+0000","last_peered":"2023-09-30T14:38:12.824591+0000","last_clean":"2023-09-30T14:38:12.824591+0000","last_became_active":"2023-09-30T14:37:11.984944+0000","last_became_peered":"2023-09-30T14:37:11.984944+0000","last_unstale":"2023-09-30T14:38:12.824591+0000","last_undegraded":"2023-09-30T14:38:12.824591+0000","last_fullsized":"2023-09-30T14:38:12.824591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:30:19.829727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197438+0000","last_change":"2023-09-30T14:37:11.028070+0000","last_active":"2023-09-30T14:38:12.197438+0000","last_peered":"2023-09-30T14:38:12.197438+0000","last_clean":"2023-09-30T14:38:12.197438+0000","last_became_active":"2023-09-30T14:37:11.027703+0000","last_became_peered":"2023-09-30T14:37:11.027703+0000","last_unstale":"2023-09-30T14:38:12.197438+0000","last_undegraded":"2023-09-30T14:38:12.197438+0000","last_fullsized":"2023-09-30T14:38:12.197438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:23:15.071946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197147+0000","last_change":"2023-09-30T14:37:11.989897+0000","last_active":"2023-09-30T14:38:12.197147+0000","last_peered":"2023-09-30T14:38:12.197147+0000","last_clean":"2023-09-30T14:38:12.197147+0000","last_became_active":"2023-09-30T14:37:11.989613+0000","last_became_peered":"2023-09-30T14:37:11.989613+0000","last_unstale":"2023-09-30T14:38:12.197147+0000","last_undegraded":"2023-09-30T14:38:12.197147+0000","last_fullsized":"2023-09-30T14:38:12.197147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:43:17.887766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198152+0000","last_change":"2023-09-30T14:37:11.026491+0000","last_active":"2023-09-30T14:38:12.198152+0000","last_peered":"2023-09-30T14:38:12.198152+0000","last_clean":"2023-09-30T14:38:12.198152+0000","last_became_active":"2023-09-30T14:37:11.026196+0000","last_became_peered":"2023-09-30T14:37:11.026196+0000","last_unstale":"2023-09-30T14:38:12.198152+0000","last_undegraded":"2023-09-30T14:38:12.198152+0000","last_fullsized":"2023-09-30T14:38:12.198152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:17:22.343377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199319+0000","last_change":"2023-09-30T14:37:11.988570+0000","last_active":"2023-09-30T14:38:12.199319+0000","last_peered":"2023-09-30T14:38:12.199319+0000","last_clean":"2023-09-30T14:38:12.199319+0000","last_became_active":"2023-09-30T14:37:11.988230+0000","last_became_peered":"2023-09-30T14:37:11.988230+0000","last_unstale":"2023-09-30T14:38:12.199319+0000","last_undegraded":"2023-09-30T14:38:12.199319+0000","last_fullsized":"2023-09-30T14:38:12.199319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:58:38.302343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824045+0000","last_change":"2023-09-30T14:37:11.024587+0000","last_active":"2023-09-30T14:38:12.824045+0000","last_peered":"2023-09-30T14:38:12.824045+0000","last_clean":"2023-09-30T14:38:12.824045+0000","last_became_active":"2023-09-30T14:37:11.024253+0000","last_became_peered":"2023-09-30T14:37:11.024253+0000","last_unstale":"2023-09-30T14:38:12.824045+0000","last_undegraded":"2023-09-30T14:38:12.824045+0000","last_fullsized":"2023-09-30T14:38:12.824045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:31:07.293657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197186+0000","last_change":"2023-09-30T14:37:11.987818+0000","last_active":"2023-09-30T14:38:12.197186+0000","last_peered":"2023-09-30T14:38:12.197186+0000","last_clean":"2023-09-30T14:38:12.197186+0000","last_became_active":"2023-09-30T14:37:11.987476+0000","last_became_peered":"2023-09-30T14:37:11.987476+0000","last_unstale":"2023-09-30T14:38:12.197186+0000","last_undegraded":"2023-09-30T14:38:12.197186+0000","last_fullsized":"2023-09-30T14:38:12.197186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:01:07.802959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199043+0000","last_change":"2023-09-30T14:37:11.023223+0000","last_active":"2023-09-30T14:38:12.199043+0000","last_peered":"2023-09-30T14:38:12.199043+0000","last_clean":"2023-09-30T14:38:12.199043+0000","last_became_active":"2023-09-30T14:37:11.022932+0000","last_became_peered":"2023-09-30T14:37:11.022932+0000","last_unstale":"2023-09-30T14:38:12.199043+0000","last_undegraded":"2023-09-30T14:38:12.199043+0000","last_fullsized":"2023-09-30T14:38:12.199043+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:01:04.315669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824324+0000","last_change":"2023-09-30T14:37:11.990323+0000","last_active":"2023-09-30T14:38:12.824324+0000","last_peered":"2023-09-30T14:38:12.824324+0000","last_clean":"2023-09-30T14:38:12.824324+0000","last_became_active":"2023-09-30T14:37:11.989981+0000","last_became_peered":"2023-09-30T14:37:11.989981+0000","last_unstale":"2023-09-30T14:38:12.824324+0000","last_undegraded":"2023-09-30T14:38:12.824324+0000","last_fullsized":"2023-09-30T14:38:12.824324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:30:44.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199908+0000","last_change":"2023-09-30T14:37:11.025658+0000","last_active":"2023-09-30T14:38:12.199908+0000","last_peered":"2023-09-30T14:38:12.199908+0000","last_clean":"2023-09-30T14:38:12.199908+0000","last_became_active":"2023-09-30T14:37:11.025334+0000","last_became_peered":"2023-09-30T14:37:11.025334+0000","last_unstale":"2023-09-30T14:38:12.199908+0000","last_undegraded":"2023-09-30T14:38:12.199908+0000","last_fullsized":"2023-09-30T14:38:12.199908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:23:41.933634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198716+0000","last_change":"2023-09-30T14:37:11.989989+0000","last_active":"2023-09-30T14:38:12.198716+0000","last_peered":"2023-09-30T14:38:12.198716+0000","last_clean":"2023-09-30T14:38:12.198716+0000","last_became_active":"2023-09-30T14:37:11.989666+0000","last_became_peered":"2023-09-30T14:37:11.989666+0000","last_unstale":"2023-09-30T14:38:12.198716+0000","last_undegraded":"2023-09-30T14:38:12.198716+0000","last_fullsized":"2023-09-30T14:38:12.198716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:41:09.835210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198863+0000","last_change":"2023-09-30T14:37:11.025027+0000","last_active":"2023-09-30T14:38:12.198863+0000","last_peered":"2023-09-30T14:38:12.198863+0000","last_clean":"2023-09-30T14:38:12.198863+0000","last_became_active":"2023-09-30T14:37:11.024696+0000","last_became_peered":"2023-09-30T14:37:11.024696+0000","last_unstale":"2023-09-30T14:38:12.198863+0000","last_undegraded":"2023-09-30T14:38:12.198863+0000","last_fullsized":"2023-09-30T14:38:12.198863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:34:28.384535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199073+0000","last_change":"2023-09-30T14:37:11.992674+0000","last_active":"2023-09-30T14:38:12.199073+0000","last_peered":"2023-09-30T14:38:12.199073+0000","last_clean":"2023-09-30T14:38:12.199073+0000","last_became_active":"2023-09-30T14:37:11.992243+0000","last_became_peered":"2023-09-30T14:37:11.992243+0000","last_unstale":"2023-09-30T14:38:12.199073+0000","last_undegraded":"2023-09-30T14:38:12.199073+0000","last_fullsized":"2023-09-30T14:38:12.199073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:33:19.759637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194984+0000","last_change":"2023-09-30T14:37:11.024525+0000","last_active":"2023-09-30T14:38:12.194984+0000","last_peered":"2023-09-30T14:38:12.194984+0000","last_clean":"2023-09-30T14:38:12.194984+0000","last_became_active":"2023-09-30T14:37:11.024231+0000","last_became_peered":"2023-09-30T14:37:11.024231+0000","last_unstale":"2023-09-30T14:38:12.194984+0000","last_undegraded":"2023-09-30T14:38:12.194984+0000","last_fullsized":"2023-09-30T14:38:12.194984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:53:47.142798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198957+0000","last_change":"2023-09-30T14:37:11.991401+0000","last_active":"2023-09-30T14:38:12.198957+0000","last_peered":"2023-09-30T14:38:12.198957+0000","last_clean":"2023-09-30T14:38:12.198957+0000","last_became_active":"2023-09-30T14:37:11.991012+0000","last_became_peered":"2023-09-30T14:37:11.991012+0000","last_unstale":"2023-09-30T14:38:12.198957+0000","last_undegraded":"2023-09-30T14:38:12.198957+0000","last_fullsized":"2023-09-30T14:38:12.198957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:55:00.562030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195033+0000","last_change":"2023-09-30T14:37:11.022893+0000","last_active":"2023-09-30T14:38:12.195033+0000","last_peered":"2023-09-30T14:38:12.195033+0000","last_clean":"2023-09-30T14:38:12.195033+0000","last_became_active":"2023-09-30T14:37:11.022505+0000","last_became_peered":"2023-09-30T14:37:11.022505+0000","last_unstale":"2023-09-30T14:38:12.195033+0000","last_undegraded":"2023-09-30T14:38:12.195033+0000","last_fullsized":"2023-09-30T14:38:12.195033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:00:17.721645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200355+0000","last_change":"2023-09-30T14:37:11.992942+0000","last_active":"2023-09-30T14:38:12.200355+0000","last_peered":"2023-09-30T14:38:12.200355+0000","last_clean":"2023-09-30T14:38:12.200355+0000","last_became_active":"2023-09-30T14:37:11.992698+0000","last_became_peered":"2023-09-30T14:37:11.992698+0000","last_unstale":"2023-09-30T14:38:12.200355+0000","last_undegraded":"2023-09-30T14:38:12.200355+0000","last_fullsized":"2023-09-30T14:38:12.200355+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:15:30.239423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.201641+0000","last_change":"2023-09-30T14:38:13.201641+0000","last_active":"2023-09-30T14:38:13.201641+0000","last_peered":"2023-09-30T14:38:13.201641+0000","last_clean":"2023-09-30T14:38:13.201641+0000","last_became_active":"2023-09-30T14:38:13.201255+0000","last_became_peered":"2023-09-30T14:38:13.201255+0000","last_unstale":"2023-09-30T14:38:13.201641+0000","last_undegraded":"2023-09-30T14:38:13.201641+0000","last_fullsized":"2023-09-30T14:38:13.201641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:06:02.731345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199114+0000","last_change":"2023-09-30T14:37:11.984481+0000","last_active":"2023-09-30T14:38:12.199114+0000","last_peered":"2023-09-30T14:38:12.199114+0000","last_clean":"2023-09-30T14:38:12.199114+0000","last_became_active":"2023-09-30T14:37:11.984021+0000","last_became_peered":"2023-09-30T14:37:11.984021+0000","last_unstale":"2023-09-30T14:38:12.199114+0000","last_undegraded":"2023-09-30T14:38:12.199114+0000","last_fullsized":"2023-09-30T14:38:12.199114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:36:33.184379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198978+0000","last_change":"2023-09-30T14:37:11.990779+0000","last_active":"2023-09-30T14:38:12.198978+0000","last_peered":"2023-09-30T14:38:12.198978+0000","last_clean":"2023-09-30T14:38:12.198978+0000","last_became_active":"2023-09-30T14:37:11.989960+0000","last_became_peered":"2023-09-30T14:37:11.989960+0000","last_unstale":"2023-09-30T14:38:12.198978+0000","last_undegraded":"2023-09-30T14:38:12.198978+0000","last_fullsized":"2023-09-30T14:38:12.198978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:54:36.813191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196893+0000","last_change":"2023-09-30T14:37:11.995750+0000","last_active":"2023-09-30T14:38:12.196893+0000","last_peered":"2023-09-30T14:38:12.196893+0000","last_clean":"2023-09-30T14:38:12.196893+0000","last_became_active":"2023-09-30T14:37:11.995434+0000","last_became_peered":"2023-09-30T14:37:11.995434+0000","last_unstale":"2023-09-30T14:38:12.196893+0000","last_undegraded":"2023-09-30T14:38:12.196893+0000","last_fullsized":"2023-09-30T14:38:12.196893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:49:51.234024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.201564+0000","last_change":"2023-09-30T14:38:13.201564+0000","last_active":"2023-09-30T14:38:13.201564+0000","last_peered":"2023-09-30T14:38:13.201564+0000","last_clean":"2023-09-30T14:38:13.201564+0000","last_became_active":"2023-09-30T14:38:13.201175+0000","last_became_peered":"2023-09-30T14:38:13.201175+0000","last_unstale":"2023-09-30T14:38:13.201564+0000","last_undegraded":"2023-09-30T14:38:13.201564+0000","last_fullsized":"2023-09-30T14:38:13.201564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:46:23.763202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197032+0000","last_change":"2023-09-30T14:37:11.983495+0000","last_active":"2023-09-30T14:38:12.197032+0000","last_peered":"2023-09-30T14:38:12.197032+0000","last_clean":"2023-09-30T14:38:12.197032+0000","last_became_active":"2023-09-30T14:37:11.983241+0000","last_became_peered":"2023-09-30T14:37:11.983241+0000","last_unstale":"2023-09-30T14:38:12.197032+0000","last_undegraded":"2023-09-30T14:38:12.197032+0000","last_fullsized":"2023-09-30T14:38:12.197032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:41:46.604467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200166+0000","last_change":"2023-09-30T14:37:11.996058+0000","last_active":"2023-09-30T14:38:12.200166+0000","last_peered":"2023-09-30T14:38:12.200166+0000","last_clean":"2023-09-30T14:38:12.200166+0000","last_became_active":"2023-09-30T14:37:11.995763+0000","last_became_peered":"2023-09-30T14:37:11.995763+0000","last_unstale":"2023-09-30T14:38:12.200166+0000","last_undegraded":"2023-09-30T14:38:12.200166+0000","last_fullsized":"2023-09-30T14:38:12.200166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:42:21.306561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824505+0000","last_change":"2023-09-30T14:37:11.992080+0000","last_active":"2023-09-30T14:38:12.824505+0000","last_peered":"2023-09-30T14:38:12.824505+0000","last_clean":"2023-09-30T14:38:12.824505+0000","last_became_active":"2023-09-30T14:37:11.991747+0000","last_became_peered":"2023-09-30T14:37:11.991747+0000","last_unstale":"2023-09-30T14:38:12.824505+0000","last_undegraded":"2023-09-30T14:38:12.824505+0000","last_fullsized":"2023-09-30T14:38:12.824505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:07:25.985911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194496+0000","last_change":"2023-09-30T14:37:11.997109+0000","last_active":"2023-09-30T14:38:12.194496+0000","last_peered":"2023-09-30T14:38:12.194496+0000","last_clean":"2023-09-30T14:38:12.194496+0000","last_became_active":"2023-09-30T14:37:11.996793+0000","last_became_peered":"2023-09-30T14:37:11.996793+0000","last_unstale":"2023-09-30T14:38:12.194496+0000","last_undegraded":"2023-09-30T14:38:12.194496+0000","last_fullsized":"2023-09-30T14:38:12.194496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:13:59.838638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.823992+0000","last_change":"2023-09-30T14:37:11.984083+0000","last_active":"2023-09-30T14:38:12.823992+0000","last_peered":"2023-09-30T14:38:12.823992+0000","last_clean":"2023-09-30T14:38:12.823992+0000","last_became_active":"2023-09-30T14:37:11.983352+0000","last_became_peered":"2023-09-30T14:37:11.983352+0000","last_unstale":"2023-09-30T14:38:12.823992+0000","last_undegraded":"2023-09-30T14:38:12.823992+0000","last_fullsized":"2023-09-30T14:38:12.823992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:02:03.668484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198484+0000","last_change":"2023-09-30T14:37:11.992212+0000","last_active":"2023-09-30T14:38:12.198484+0000","last_peered":"2023-09-30T14:38:12.198484+0000","last_clean":"2023-09-30T14:38:12.198484+0000","last_became_active":"2023-09-30T14:37:11.990158+0000","last_became_peered":"2023-09-30T14:37:11.990158+0000","last_unstale":"2023-09-30T14:38:12.198484+0000","last_undegraded":"2023-09-30T14:38:12.198484+0000","last_fullsized":"2023-09-30T14:38:12.198484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:14:27.976110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194553+0000","last_change":"2023-09-30T14:37:11.995884+0000","last_active":"2023-09-30T14:38:12.194553+0000","last_peered":"2023-09-30T14:38:12.194553+0000","last_clean":"2023-09-30T14:38:12.194553+0000","last_became_active":"2023-09-30T14:37:11.995551+0000","last_became_peered":"2023-09-30T14:37:11.995551+0000","last_unstale":"2023-09-30T14:38:12.194553+0000","last_undegraded":"2023-09-30T14:38:12.194553+0000","last_fullsized":"2023-09-30T14:38:12.194553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:28:26.051188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197182+0000","last_change":"2023-09-30T14:37:11.985853+0000","last_active":"2023-09-30T14:38:12.197182+0000","last_peered":"2023-09-30T14:38:12.197182+0000","last_clean":"2023-09-30T14:38:12.197182+0000","last_became_active":"2023-09-30T14:37:11.985525+0000","last_became_peered":"2023-09-30T14:37:11.985525+0000","last_unstale":"2023-09-30T14:38:12.197182+0000","last_undegraded":"2023-09-30T14:38:12.197182+0000","last_fullsized":"2023-09-30T14:38:12.197182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:07:24.392538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58318,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":491,"num_read_kb":310,"num_write":149,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1316,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":175272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449116,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":443610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.56599999999999995,"15min":0.57799999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.60199999999999998,"5min":1.056,"15min":1.056},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.57999999999999996,"15min":0.59799999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.621,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.51300000000000001}]},{"osd":1,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.51500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.80300000000000005,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.78700000000000003,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.63}]},{"osd":2,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.499,"15min":0.497},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.76100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68200000000000005,"15min":0.67600000000000005},"min":{"1min":0.46000000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.012,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":1.012,"15min":1.012},"last":0.68100000000000005}]},{"osd":5,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":1.226,"15min":1.226},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.624,"15min":0.628},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94999999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.90300000000000002}]},{"osd":6,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70899999999999996,"15min":0.71699999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.73899999999999999}]},{"osd":7,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79100000000000004,"15min":0.79600000000000004},"min":{"1min":0.53100000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.0369999999999999,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.79400000000000004,"15min":0.80300000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.032,"5min":1.319,"15min":1.319},"last":0.92400000000000004}]},{"osd":8,"last update":"Sat Sep 30 14:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.95699999999999996}]},{"osd":9,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.77700000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97099999999999997,"5min":1.597,"15min":1.597},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.84499999999999997,"15min":0.84599999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":1.012}]},{"osd":10,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.82499999999999996,"15min":0.83399999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0609999999999999,"5min":1.417,"15min":1.417},"last":0.873},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80800000000000005,"15min":0.81200000000000006},"min":{"1min":0.54100000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0469999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.93700000000000006}]}]},{"osd":10,"up_from":71,"seq":304942678048,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1605632,"data_stored":1088780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47599999999999998,"15min":0.47999999999999998},"min":{"1min":0.28100000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64600000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.33200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.67500000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48299999999999998}]},{"osd":2,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76000000000000001,"5min":0.879,"15min":0.879},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.59199999999999997}]},{"osd":3,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.65800000000000003,"15min":0.67600000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.75600000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.65200000000000002,"15min":0.67500000000000004},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79700000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.50800000000000001}]},{"osd":4,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67100000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.60199999999999998,"15min":0.61499999999999999},"min":{"1min":0.374,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71999999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57799999999999996}]},{"osd":5,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.70299999999999996,"15min":0.71599999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77500000000000002,"5min":1.244,"15min":1.244},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.505,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.80500000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73799999999999999}]},{"osd":6,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.752,"15min":0.77200000000000002},"min":{"1min":0.44600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77800000000000002,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.57999999999999996,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85299999999999998,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.70399999999999996}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.41699999999999998}]},{"osd":8,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.82099999999999995,"15min":0.83499999999999996},"min":{"1min":0.53200000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.84199999999999997,"5min":1.335,"15min":1.335},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.82799999999999996,"15min":0.84499999999999997},"min":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.85799999999999998,"5min":1.218,"15min":1.218},"last":0.78200000000000003}]},{"osd":9,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.83899999999999997,"15min":0.85899999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.91000000000000003,"5min":1.389,"15min":1.389},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.84599999999999997,"15min":0.86799999999999999},"min":{"1min":0.501,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.872,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.69199999999999995}]},{"osd":11,"last update":"Sat Sep 30 14:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79900000000000004},"min":{"1min":0.63200000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90800000000000003,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.85499999999999998,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.82799999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841571,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":442821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.46500000000000002,"15min":0.47499999999999998},"min":{"1min":0.28499999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60299999999999998,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.316},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.503,"15min":0.51800000000000002},"min":{"1min":0.32600000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65700000000000003,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.34200000000000003}]},{"osd":1,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77400000000000002,"5min":1.069,"15min":1.069},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.64500000000000002,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.749,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.45900000000000002}]},{"osd":2,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.61799999999999999,"15min":0.628},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.38,"5min":0.376,"15min":0.376},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.57499999999999996}]},{"osd":3,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.54900000000000004,"15min":0.56599999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69899999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.86199999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.40899999999999997}]},{"osd":4,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.69299999999999995,"15min":0.70999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80500000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.71399999999999997,"15min":0.72699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86399999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.503}]},{"osd":5,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.70499999999999996,"15min":0.72199999999999998},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":1.137,"15min":1.137},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.73299999999999998,"15min":0.749},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0860000000000001,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.59099999999999997}]},{"osd":6,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.69499999999999995,"15min":0.71799999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.046,"5min":1.21,"15min":1.21},"last":0.436},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.71599999999999997,"15min":0.73599999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96299999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.50900000000000001}]},{"osd":7,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.79700000000000004,"15min":0.81599999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94099999999999995,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.81200000000000006,"15min":0.82799999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.66800000000000004}]},{"osd":8,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81200000000000006,"15min":0.82599999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.81599999999999995,"15min":0.83599999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.028,"5min":1.466,"15min":1.466},"last":0.70299999999999996}]},{"osd":10,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.80900000000000005,"15min":0.82499999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90400000000000003,"5min":1.391,"15min":1.391},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79600000000000004,"15min":0.80800000000000005},"min":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.91200000000000003,"5min":1.121,"15min":1.121},"last":0.67900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":8,"up_from":58,"seq":249108103209,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.42199999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.437,"5min":0.316,"15min":0.316},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.54200000000000004}]},{"osd":1,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90300000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.60999999999999999,"15min":0.59499999999999997},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86099999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.748}]},{"osd":2,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.47299999999999998,"15min":0.45500000000000002},"min":{"1min":0.38800000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.48499999999999999,"15min":0.46600000000000003},"min":{"1min":0.35999999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.63200000000000001}]},{"osd":3,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60699999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.435,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.57499999999999996}]},{"osd":4,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.63500000000000001,"15min":0.60899999999999999},"min":{"1min":0.40699999999999997,"5min":0.378,"15min":0.378},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.66100000000000003,"15min":0.63700000000000001},"min":{"1min":0.60999999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.88500000000000001}]},{"osd":5,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86599999999999999,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.622,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.83799999999999997}]},{"osd":6,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.68500000000000005,"15min":0.66800000000000004},"min":{"1min":0.51200000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.46700000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98499999999999999,"5min":1.133,"15min":1.133},"last":0.86199999999999999}]},{"osd":7,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.73999999999999999,"15min":0.72299999999999998},"min":{"1min":0.58999999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.012,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.74199999999999999,"15min":0.72399999999999998},"min":{"1min":0.53600000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.95299999999999996}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.749,"15min":0.747},"min":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95299999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.82099999999999995}]},{"osd":10,"last update":"Sat Sep 30 14:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.76800000000000002,"15min":0.74199999999999999},"min":{"1min":0.71799999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.018},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.76200000000000001,"15min":0.73699999999999999},"min":{"1min":0.76200000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.94499999999999995}]},{"osd":11,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.91000000000000003}]}]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1034035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42999999999999999}]},{"osd":1,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.35799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.60999999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.45700000000000002,"15min":0.439},"min":{"1min":0.39300000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.49199999999999999}]},{"osd":2,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60399999999999998,"15min":0.58499999999999996},"min":{"1min":0.496,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83999999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.42799999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74299999999999999,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55800000000000005}]},{"osd":3,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.43099999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81299999999999994,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65100000000000002,"15min":0.66700000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81200000000000006,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.67900000000000005}]},{"osd":4,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.5},"min":{"1min":0.38600000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69799999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54000000000000004,"15min":0.52200000000000002},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76300000000000001,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60299999999999998}]},{"osd":5,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.63600000000000001,"15min":0.61899999999999999},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78900000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.67900000000000005,"15min":0.65900000000000003},"min":{"1min":0.45700000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.89500000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.748}]},{"osd":6,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.55700000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95799999999999996,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.747,"15min":0.73499999999999999},"min":{"1min":0.47099999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.9340000000000002,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":0.79400000000000004}]},{"osd":8,"last update":"Sat Sep 30 14:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.47999999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.872,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.73199999999999998,"15min":0.70799999999999996},"min":{"1min":0.629,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.84099999999999997}]},{"osd":9,"last update":"Sat Sep 30 14:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77600000000000002,"15min":0.77900000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.57299999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96499999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.82599999999999996}]},{"osd":10,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.77000000000000002,"15min":0.76400000000000001},"min":{"1min":0.64700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.79400000000000004,"15min":0.79600000000000004},"min":{"1min":0.52000000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.65600000000000003}]},{"osd":11,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.88700000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":442216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38100000000000001,"15min":0.371},"min":{"1min":0.27700000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.40799999999999997,"15min":0.39000000000000001},"min":{"1min":0.28100000000000003,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.26700000000000002}]},{"osd":1,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.75700000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.626},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.48599999999999999}]},{"osd":2,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.628,"15min":0.627},"min":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.75700000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.433}]},{"osd":3,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.502,"15min":0.48599999999999999},"min":{"1min":0.35299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.504},{"interface":"front","average":{"1min":0.504,"5min":0.499,"15min":0.497},"min":{"1min":0.35799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.249}]},{"osd":4,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.46800000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67300000000000004,"15min":0.65100000000000002},"min":{"1min":0.46899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.89600000000000002,"5min":1.056,"15min":1.056},"last":0.58599999999999997}]},{"osd":5,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64600000000000002,"15min":0.64700000000000002},"min":{"1min":0.499,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.48099999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1060000000000001,"5min":1.228,"15min":1.228},"last":0.36699999999999999}]},{"osd":7,"last update":"Sat Sep 30 14:38:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70999999999999996,"15min":0.68799999999999994},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1679999999999999,"5min":1.272,"15min":1.272},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.56399999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.83999999999999997,"5min":1.323,"15min":1.323},"last":0.69799999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.55000000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.81599999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.55100000000000005}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.748,"15min":0.76000000000000001},"min":{"1min":0.45500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72499999999999998,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93200000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.66300000000000003}]},{"osd":10,"last update":"Sat Sep 30 14:38:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.83699999999999997,"15min":0.84499999999999997},"min":{"1min":0.627,"5min":0.62,"15min":0.62},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.82499999999999996,"15min":0.84199999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.89500000000000002,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":497333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55600000000000005,"15min":0.50600000000000001},"min":{"1min":0.36099999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.56999999999999995,"15min":0.52300000000000002},"min":{"1min":0.45200000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.76800000000000002}]},{"osd":2,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.61299999999999999,"15min":0.52900000000000003},"min":{"1min":0.53100000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98499999999999999,"5min":0.998,"15min":0.998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.59799999999999998,"15min":0.53000000000000003},"min":{"1min":0.39200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.53200000000000003}]},{"osd":3,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57899999999999996,"15min":0.54300000000000004},"min":{"1min":0.436,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60599999999999998,"15min":0.57499999999999996},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92200000000000004,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.68799999999999994}]},{"osd":4,"last update":"Sat Sep 30 14:38:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.48099999999999998,"15min":0.443},"min":{"1min":0.28999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.503,"15min":0.46300000000000002},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.81699999999999995}]},{"osd":5,"last update":"Sat Sep 30 14:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.82799999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.68200000000000005,"15min":0.628},"min":{"1min":0.57899999999999996,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.85899999999999999}]},{"osd":6,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88300000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69499999999999995,"15min":0.67500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.88200000000000001}]},{"osd":7,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.61399999999999999,"15min":0.55900000000000005},"min":{"1min":0.58099999999999996,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.434,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.57299999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.77800000000000002,"15min":0.748},"min":{"1min":0.55200000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.751,"15min":0.71699999999999997},"min":{"1min":0.56000000000000005,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.98599999999999999}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.748},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.81799999999999995,"15min":0.81399999999999995},"min":{"1min":0.56799999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.96499999999999997}]},{"osd":10,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.81699999999999995,"15min":0.79500000000000004},"min":{"1min":0.75800000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.78100000000000003,"15min":0.76000000000000001},"min":{"1min":0.70199999999999996,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.90900000000000003}]},{"osd":11,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.94599999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":443481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 14:38:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55600000000000005,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71799999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57099999999999995,"15min":0.53400000000000003},"min":{"1min":0.42799999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.84599999999999997,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.53500000000000003}]},{"osd":2,"last update":"Sat Sep 30 14:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.47499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.58999999999999997,"15min":0.56799999999999995},"min":{"1min":0.48999999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90700000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.76800000000000002}]},{"osd":3,"last update":"Sat Sep 30 14:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.38300000000000001}]},{"osd":4,"last update":"Sat Sep 30 14:38:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60299999999999998,"15min":0.58599999999999997},"min":{"1min":0.41299999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82699999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60799999999999998,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83099999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.496}]},{"osd":5,"last update":"Sat Sep 30 14:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.47299999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.45100000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.79400000000000004,"5min":0.996,"15min":0.996},"last":0.73299999999999998}]},{"osd":6,"last update":"Sat Sep 30 14:37:47 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69199999999999995,"5min":1.03,"15min":1.03},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.317,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80600000000000005,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.51600000000000001}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69699999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.014,"5min":1.147,"15min":1.147},"last":0.442},{"interface":"front","average":{"1min":0.748,"5min":0.67800000000000005,"15min":0.66600000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56799999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.52300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85199999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.745,"15min":0.753},"min":{"1min":0.51700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85299999999999998,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.65700000000000003}]},{"osd":9,"last update":"Sat Sep 30 14:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.71999999999999997,"15min":0.73899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80200000000000005,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.78200000000000003,"15min":0.80400000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.79900000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.71699999999999997}]},{"osd":10,"last update":"Sat Sep 30 14:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.88400000000000001,"5min":1.127,"15min":1.127},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.79300000000000004,"15min":0.81100000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.873,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.66900000000000004}]},{"osd":11,"last update":"Sat Sep 30 14:37:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.753},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.69699999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":497497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52000000000000002,"15min":0.51100000000000001},"min":{"1min":0.372,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.83199999999999996,"5min":0.876,"15min":0.876},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.505,"15min":0.496},"min":{"1min":0.34799999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67500000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.73699999999999999}]},{"osd":1,"last update":"Sat Sep 30 14:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56799999999999995,"15min":0.52600000000000002},"min":{"1min":0.40300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.79200000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.626,"15min":0.55100000000000005},"min":{"1min":0.52700000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.68799999999999994}]},{"osd":3,"last update":"Sat Sep 30 14:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.35599999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.56499999999999995},"min":{"1min":0.42699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.89000000000000001}]},{"osd":4,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":1.018}]},{"osd":5,"last update":"Sat Sep 30 14:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.50700000000000001,"15min":0.46899999999999997},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.53000000000000003,"15min":0.49399999999999999},"min":{"1min":0.43099999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.65200000000000002}]},{"osd":6,"last update":"Sat Sep 30 14:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66600000000000004,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1000000000000001,"5min":1.405,"15min":1.405},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.94999999999999996}]},{"osd":7,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.45500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.80100000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70699999999999996,"15min":0.71299999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.86399999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.59199999999999997}]},{"osd":8,"last update":"Sat Sep 30 14:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.65100000000000002,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.627},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.46999999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75900000000000001}]},{"osd":9,"last update":"Sat Sep 30 14:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.877},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.79600000000000004}]},{"osd":10,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.55900000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.69199999999999995,"15min":0.67100000000000004},"min":{"1min":0.56200000000000006,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.98399999999999999}]},{"osd":11,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":1.0449999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149759,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":442533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.35799999999999998},"min":{"1min":0.27200000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.432,"15min":0.41199999999999998},"min":{"1min":0.316,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54800000000000004,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.59599999999999997}]},{"osd":1,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64200000000000002,"15min":0.58599999999999997},"min":{"1min":0.442,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.002,"5min":5.798,"15min":5.798},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.71799999999999997}]},{"osd":2,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.39100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.70999999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58399999999999996,"15min":0.55100000000000005},"min":{"1min":0.442,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.73199999999999998}]},{"osd":4,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.66100000000000003,"15min":0.58399999999999996},"min":{"1min":0.40999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.873,"5min":7.1619999999999999,"15min":7.1619999999999999},"last":0.624},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.621,"15min":0.57699999999999996},"min":{"1min":0.44800000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.075,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.70199999999999996}]},{"osd":5,"last update":"Sat Sep 30 14:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.42599999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.46000000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72399999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.79400000000000004}]},{"osd":7,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.71599999999999997,"15min":0.70499999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99199999999999999,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81699999999999995,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.74299999999999999}]},{"osd":8,"last update":"Sat Sep 30 14:36:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.46100000000000002}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.69499999999999995,"15min":0.71399999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81299999999999994,"5min":1.161,"15min":1.161},"last":0.82499999999999996}]},{"osd":10,"last update":"Sat Sep 30 14:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":1.0249999999999999,"15min":1.069},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.3640000000000001,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.84699999999999998,"15min":0.86899999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.304,"5min":2.597,"15min":2.597},"last":0.84199999999999997}]},{"osd":11,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.88},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.91600000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67100000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.65000000000000002}]},{"osd":1,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.372},"min":{"1min":0.32200000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.53400000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39000000000000001,"15min":0.36799999999999999},"min":{"1min":0.34200000000000003,"5min":0.224,"15min":0.224},"max":{"1min":0.56499999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.44400000000000001}]},{"osd":2,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.58799999999999997,"15min":0.53000000000000003},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.871,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57299999999999995,"15min":0.54900000000000004},"min":{"1min":0.47099999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69099999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.69999999999999996}]},{"osd":3,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.47499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72299999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.58099999999999996},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90900000000000003,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.48499999999999999}]},{"osd":5,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.621,"15min":0.57499999999999996},"min":{"1min":0.48699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84899999999999998,"5min":1.484,"15min":1.484},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.56100000000000005},"min":{"1min":0.45900000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83999999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54900000000000004}]},{"osd":6,"last update":"Sat Sep 30 14:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.626,"15min":0.626},"min":{"1min":0.501,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78300000000000003,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.439,"5min":0.435,"15min":0.435},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56000000000000005,"15min":0.53900000000000003},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.79800000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61199999999999999,"15min":0.58299999999999996},"min":{"1min":0.47099999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97299999999999998,"5min":1.714,"15min":1.714},"last":0.65700000000000003}]},{"osd":8,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67500000000000004,"15min":0.66000000000000003},"min":{"1min":0.53700000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67600000000000005,"15min":0.66300000000000003},"min":{"1min":0.48099999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.95199999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.753}]},{"osd":9,"last update":"Sat Sep 30 14:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.55100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.85299999999999998,"5min":1.038,"15min":1.038},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.58599999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.95199999999999996,"5min":1.677,"15min":1.677},"last":0.76400000000000001}]},{"osd":10,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.72099999999999997},"min":{"1min":0.41799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.92800000000000005,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.747,"15min":0.751},"min":{"1min":0.52100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0009999999999999,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.69099999999999995}]},{"osd":11,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.85199999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1536000,"data_stored":1033970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.496},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77800000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":3.4380000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53300000000000003,"15min":0.51500000000000001},"min":{"1min":0.35399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77100000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.44500000000000001}]},{"osd":1,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.56399999999999995,"15min":0.50800000000000001},"min":{"1min":0.38600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54000000000000004,"15min":0.499},"min":{"1min":0.46100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77600000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56799999999999995}]},{"osd":2,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.443,"15min":0.41899999999999998},"min":{"1min":0.33500000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.42499999999999999},"min":{"1min":0.32400000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47099999999999997}]},{"osd":3,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.60599999999999998,"15min":0.55900000000000005},"min":{"1min":0.376,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.1050000000000004,"5min":4.1050000000000004,"15min":4.1050000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.65400000000000003,"15min":0.60399999999999998},"min":{"1min":0.53200000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.1630000000000003,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":0.51000000000000001}]},{"osd":4,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.59099999999999997,"15min":0.54700000000000004},"min":{"1min":0.52700000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83799999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60799999999999998,"15min":0.57299999999999995},"min":{"1min":0.439,"5min":0.37,"15min":0.37},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.79800000000000004}]},{"osd":6,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.69499999999999995,"15min":0.66000000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":4.125,"5min":4.125,"15min":4.125},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.1970000000000001,"5min":0.73499999999999999,"15min":0.64700000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":12.24,"5min":12.24,"15min":12.24},"last":3.4660000000000002}]},{"osd":7,"last update":"Sat Sep 30 14:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.66400000000000003,"15min":0.65700000000000003},"min":{"1min":0.52600000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.56499999999999995,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.78500000000000003}]},{"osd":8,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.42599999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90400000000000003,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.754},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64200000000000002,"15min":0.625},"min":{"1min":0.52300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.87,"5min":0.874,"15min":0.874},"last":0.76900000000000002}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.78400000000000003,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.503,"15min":0.503},"max":{"1min":5.1520000000000001,"5min":5.1520000000000001,"15min":5.1520000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":0.874,"15min":0.79300000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":12.198,"5min":12.198,"15min":12.198},"last":0.71499999999999997}]},{"osd":10,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.66900000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.59599999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.81399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":55330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":55436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T14:38:39.794 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-09-30T14:38:39.794 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T14:38:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:39 smithi156 ceph-mon[103579]: pgmap v298: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-09-30T14:38:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:39 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1961372723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:39 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3520360785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:39.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:39 smithi148 ceph-mon[102355]: pgmap v298: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-09-30T14:38:39.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:39 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1961372723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:39.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:39 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3520360785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:39.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.952+0000 7f9d709ef700 1 -- 172.21.15.32:0/2984398392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d6c101900 msgr2=0x7f9d6c1081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:39.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.952+0000 7f9d709ef700 1 --2- 172.21.15.32:0/2984398392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d6c101900 0x7f9d6c1081b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400b0d0 tx=0x7f9d6400b3e0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.952+0000 7f9d709ef700 1 -- 172.21.15.32:0/2984398392 shutdown_connections 2023-09-30T14:38:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.952+0000 7f9d709ef700 1 --2- 172.21.15.32:0/2984398392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d6c101900 0x7f9d6c1081b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.952+0000 7f9d709ef700 1 --2- 172.21.15.32:0/2984398392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d6c0ffd70 0x7f9d6c100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.952+0000 7f9d709ef700 1 --2- 172.21.15.32:0/2984398392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c100f60 0x7f9d6c1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.952+0000 7f9d709ef700 1 -- 172.21.15.32:0/2984398392 >> 172.21.15.32:0/2984398392 conn(0x7f9d6c0fb2e0 msgr2=0x7f9d6c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.952+0000 7f9d709ef700 1 -- 172.21.15.32:0/2984398392 shutdown_connections 2023-09-30T14:38:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.952+0000 7f9d709ef700 1 -- 172.21.15.32:0/2984398392 wait complete. 2023-09-30T14:38:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d709ef700 1 Processor -- start 2023-09-30T14:38:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d709ef700 1 -- start start 2023-09-30T14:38:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d709ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c0ffd70 0x7f9d6c196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d709ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d6c100f60 0x7f9d6c197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d709ef700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d6c101900 0x7f9d6c19e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d709ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c19e660 con 0x7f9d6c100f60 2023-09-30T14:38:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d709ef700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c19e7a0 con 0x7f9d6c101900 2023-09-30T14:38:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d709ef700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c19e930 con 0x7f9d6c0ffd70 2023-09-30T14:38:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d69d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d6c100f60 0x7f9d6c197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d69d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d6c100f60 0x7f9d6c197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35252/0 (socket says 172.21.15.32:35252) 2023-09-30T14:38:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d69d9b700 1 -- 172.21.15.32:0/2076669445 learned_addr learned my addr 172.21.15.32:0/2076669445 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d6a59c700 1 --2- 172.21.15.32:0/2076669445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c0ffd70 0x7f9d6c196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d6ad9d700 1 --2- 172.21.15.32:0/2076669445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d6c101900 0x7f9d6c19e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d69d9b700 1 -- 172.21.15.32:0/2076669445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c0ffd70 msgr2=0x7f9d6c196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d69d9b700 1 --2- 172.21.15.32:0/2076669445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c0ffd70 0x7f9d6c196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d69d9b700 1 -- 172.21.15.32:0/2076669445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d6c101900 msgr2=0x7f9d6c19e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d69d9b700 1 --2- 172.21.15.32:0/2076669445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d6c101900 0x7f9d6c19e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:39.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d69d9b700 1 -- 172.21.15.32:0/2076669445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d64009d00 con 0x7f9d6c100f60 2023-09-30T14:38:39.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d6ad9d700 1 --2- 172.21.15.32:0/2076669445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d6c101900 0x7f9d6c19e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:39.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.953+0000 7f9d6a59c700 1 --2- 172.21.15.32:0/2076669445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c0ffd70 0x7f9d6c196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:39.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.954+0000 7f9d69d9b700 1 --2- 172.21.15.32:0/2076669445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d6c100f60 0x7f9d6c197020 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000c9b0 tx=0x7f9d6000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:39.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.954+0000 7f9d5b7fe700 1 -- 172.21.15.32:0/2076669445 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d60007b00 con 0x7f9d6c100f60 2023-09-30T14:38:39.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.954+0000 7f9d709ef700 1 -- 172.21.15.32:0/2076669445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6c19ec10 con 0x7f9d6c100f60 2023-09-30T14:38:39.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.954+0000 7f9d5b7fe700 1 -- 172.21.15.32:0/2076669445 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d60007c60 con 0x7f9d6c100f60 2023-09-30T14:38:39.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.954+0000 7f9d5b7fe700 1 -- 172.21.15.32:0/2076669445 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d60018660 con 0x7f9d6c100f60 2023-09-30T14:38:39.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.954+0000 7f9d709ef700 1 -- 172.21.15.32:0/2076669445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c19f120 con 0x7f9d6c100f60 2023-09-30T14:38:39.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.955+0000 7f9d709ef700 1 -- 172.21.15.32:0/2076669445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6c107c10 con 0x7f9d6c100f60 2023-09-30T14:38:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.956+0000 7f9d5b7fe700 1 -- 172.21.15.32:0/2076669445 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f9d60018880 con 0x7f9d6c100f60 2023-09-30T14:38:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.956+0000 7f9d5b7fe700 1 --2- 172.21.15.32:0/2076669445 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9d540a1070 0x7f9d540a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.957+0000 7f9d6a59c700 1 --2- 172.21.15.32:0/2076669445 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9d540a1070 0x7f9d540a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.957+0000 7f9d5b7fe700 1 -- 172.21.15.32:0/2076669445 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9d600ccbd0 con 0x7f9d6c100f60 2023-09-30T14:38:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.957+0000 7f9d6a59c700 1 --2- 172.21.15.32:0/2076669445 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9d540a1070 0x7f9d540a3530 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c100dc0 tx=0x7f9d5c00a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:39.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:39.960+0000 7f9d5b7fe700 1 -- 172.21.15.32:0/2076669445 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f9d600cce60 con 0x7f9d6c100f60 2023-09-30T14:38:40.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.160+0000 7f9d709ef700 1 -- 172.21.15.32:0/2076669445 --> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9d6c02d220 con 0x7f9d540a1070 2023-09-30T14:38:40.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.173+0000 7f9d5b7fe700 1 -- 172.21.15.32:0/2076669445 <== mgr.14152 v2:172.21.15.32:6800/3273633815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344350 (secure 0 0 0) 0x7f9d6c02d220 con 0x7f9d540a1070 2023-09-30T14:38:40.177 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:40.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.177+0000 7f9d709ef700 1 -- 172.21.15.32:0/2076669445 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9d540a1070 msgr2=0x7f9d540a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:40.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.177+0000 7f9d709ef700 1 --2- 172.21.15.32:0/2076669445 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9d540a1070 0x7f9d540a3530 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c100dc0 tx=0x7f9d5c00a680 comp rx=0 tx=0).stop 2023-09-30T14:38:40.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.177+0000 7f9d709ef700 1 -- 172.21.15.32:0/2076669445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d6c100f60 msgr2=0x7f9d6c197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:40.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.177+0000 7f9d709ef700 1 --2- 172.21.15.32:0/2076669445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d6c100f60 0x7f9d6c197020 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000c9b0 tx=0x7f9d6000cd70 comp rx=0 tx=0).stop 2023-09-30T14:38:40.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.177+0000 7f9d709ef700 1 -- 172.21.15.32:0/2076669445 shutdown_connections 2023-09-30T14:38:40.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.177+0000 7f9d709ef700 1 --2- 172.21.15.32:0/2076669445 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9d540a1070 0x7f9d540a3530 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.177+0000 7f9d709ef700 1 --2- 172.21.15.32:0/2076669445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d6c101900 0x7f9d6c19e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.177+0000 7f9d709ef700 1 --2- 172.21.15.32:0/2076669445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c0ffd70 0x7f9d6c196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.177+0000 7f9d709ef700 1 --2- 172.21.15.32:0/2076669445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d6c100f60 0x7f9d6c197020 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.177+0000 7f9d709ef700 1 -- 172.21.15.32:0/2076669445 >> 172.21.15.32:0/2076669445 conn(0x7f9d6c0fb2e0 msgr2=0x7f9d6c0fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.177+0000 7f9d709ef700 1 -- 172.21.15.32:0/2076669445 shutdown_connections 2023-09-30T14:38:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.177+0000 7f9d709ef700 1 -- 172.21.15.32:0/2076669445 wait complete. 2023-09-30T14:38:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-09-30T14:38:40.190 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2023-09-30T14:38:39.794432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649250,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":623,"num_read_kb":420,"num_write":333,"num_write_kb":3040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1182052,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337084,"kb_used_data":12840,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604740,"statfs":{"total":1151940427776,"available":1151595253760,"internally_reserved":0,"allocated":13148160,"data_stored":7261808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":318,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":31,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002986"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197844+0000","last_change":"2023-09-30T14:37:11.988122+0000","last_active":"2023-09-30T14:38:12.197844+0000","last_peered":"2023-09-30T14:38:12.197844+0000","last_clean":"2023-09-30T14:38:12.197844+0000","last_became_active":"2023-09-30T14:37:11.986770+0000","last_became_peered":"2023-09-30T14:37:11.986770+0000","last_unstale":"2023-09-30T14:38:12.197844+0000","last_undegraded":"2023-09-30T14:38:12.197844+0000","last_fullsized":"2023-09-30T14:38:12.197844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:46:29.655309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197069+0000","last_change":"2023-09-30T14:37:11.987953+0000","last_active":"2023-09-30T14:38:12.197069+0000","last_peered":"2023-09-30T14:38:12.197069+0000","last_clean":"2023-09-30T14:38:12.197069+0000","last_became_active":"2023-09-30T14:37:11.987248+0000","last_became_peered":"2023-09-30T14:37:11.987248+0000","last_unstale":"2023-09-30T14:38:12.197069+0000","last_undegraded":"2023-09-30T14:38:12.197069+0000","last_fullsized":"2023-09-30T14:38:12.197069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:12:59.780089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197196+0000","last_change":"2023-09-30T14:37:11.987570+0000","last_active":"2023-09-30T14:38:12.197196+0000","last_peered":"2023-09-30T14:38:12.197196+0000","last_clean":"2023-09-30T14:38:12.197196+0000","last_became_active":"2023-09-30T14:37:11.987064+0000","last_became_peered":"2023-09-30T14:37:11.987064+0000","last_unstale":"2023-09-30T14:38:12.197196+0000","last_undegraded":"2023-09-30T14:38:12.197196+0000","last_fullsized":"2023-09-30T14:38:12.197196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:26:16.674605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197162+0000","last_change":"2023-09-30T14:37:11.990161+0000","last_active":"2023-09-30T14:38:12.197162+0000","last_peered":"2023-09-30T14:38:12.197162+0000","last_clean":"2023-09-30T14:38:12.197162+0000","last_became_active":"2023-09-30T14:37:11.989580+0000","last_became_peered":"2023-09-30T14:37:11.989580+0000","last_unstale":"2023-09-30T14:38:12.197162+0000","last_undegraded":"2023-09-30T14:38:12.197162+0000","last_fullsized":"2023-09-30T14:38:12.197162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:13:06.986864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199326+0000","last_change":"2023-09-30T14:37:11.993902+0000","last_active":"2023-09-30T14:38:12.199326+0000","last_peered":"2023-09-30T14:38:12.199326+0000","last_clean":"2023-09-30T14:38:12.199326+0000","last_became_active":"2023-09-30T14:37:11.993622+0000","last_became_peered":"2023-09-30T14:37:11.993622+0000","last_unstale":"2023-09-30T14:38:12.199326+0000","last_undegraded":"2023-09-30T14:38:12.199326+0000","last_fullsized":"2023-09-30T14:38:12.199326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:09:49.252823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195466+0000","last_change":"2023-09-30T14:37:11.997282+0000","last_active":"2023-09-30T14:38:12.195466+0000","last_peered":"2023-09-30T14:38:12.195466+0000","last_clean":"2023-09-30T14:38:12.195466+0000","last_became_active":"2023-09-30T14:37:11.996937+0000","last_became_peered":"2023-09-30T14:37:11.996937+0000","last_unstale":"2023-09-30T14:38:12.195466+0000","last_undegraded":"2023-09-30T14:38:12.195466+0000","last_fullsized":"2023-09-30T14:38:12.195466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:39:27.351772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199172+0000","last_change":"2023-09-30T14:37:11.997408+0000","last_active":"2023-09-30T14:38:12.199172+0000","last_peered":"2023-09-30T14:38:12.199172+0000","last_clean":"2023-09-30T14:38:12.199172+0000","last_became_active":"2023-09-30T14:37:11.997147+0000","last_became_peered":"2023-09-30T14:37:11.997147+0000","last_unstale":"2023-09-30T14:38:12.199172+0000","last_undegraded":"2023-09-30T14:38:12.199172+0000","last_fullsized":"2023-09-30T14:38:12.199172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:21:24.721417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.823924+0000","last_change":"2023-09-30T14:37:11.986172+0000","last_active":"2023-09-30T14:38:12.823924+0000","last_peered":"2023-09-30T14:38:12.823924+0000","last_clean":"2023-09-30T14:38:12.823924+0000","last_became_active":"2023-09-30T14:37:11.985836+0000","last_became_peered":"2023-09-30T14:37:11.985836+0000","last_unstale":"2023-09-30T14:38:12.823924+0000","last_undegraded":"2023-09-30T14:38:12.823924+0000","last_fullsized":"2023-09-30T14:38:12.823924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:35:57.464679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198360+0000","last_change":"2023-09-30T14:37:11.990853+0000","last_active":"2023-09-30T14:38:12.198360+0000","last_peered":"2023-09-30T14:38:12.198360+0000","last_clean":"2023-09-30T14:38:12.198360+0000","last_became_active":"2023-09-30T14:37:11.990321+0000","last_became_peered":"2023-09-30T14:37:11.990321+0000","last_unstale":"2023-09-30T14:38:12.198360+0000","last_undegraded":"2023-09-30T14:38:12.198360+0000","last_fullsized":"2023-09-30T14:38:12.198360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:32:11.409927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.202453+0000","last_change":"2023-09-30T14:38:13.202453+0000","last_active":"2023-09-30T14:38:13.202453+0000","last_peered":"2023-09-30T14:38:13.202453+0000","last_clean":"2023-09-30T14:38:13.202453+0000","last_became_active":"2023-09-30T14:38:13.202023+0000","last_became_peered":"2023-09-30T14:38:13.202023+0000","last_unstale":"2023-09-30T14:38:13.202453+0000","last_undegraded":"2023-09-30T14:38:13.202453+0000","last_fullsized":"2023-09-30T14:38:13.202453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:50:40.737940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198974+0000","last_change":"2023-09-30T14:37:11.992213+0000","last_active":"2023-09-30T14:38:12.198974+0000","last_peered":"2023-09-30T14:38:12.198974+0000","last_clean":"2023-09-30T14:38:12.198974+0000","last_became_active":"2023-09-30T14:37:11.991868+0000","last_became_peered":"2023-09-30T14:37:11.991868+0000","last_unstale":"2023-09-30T14:38:12.198974+0000","last_undegraded":"2023-09-30T14:38:12.198974+0000","last_fullsized":"2023-09-30T14:38:12.198974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:20:06.061978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197227+0000","last_change":"2023-09-30T14:37:11.990043+0000","last_active":"2023-09-30T14:38:12.197227+0000","last_peered":"2023-09-30T14:38:12.197227+0000","last_clean":"2023-09-30T14:38:12.197227+0000","last_became_active":"2023-09-30T14:37:11.988243+0000","last_became_peered":"2023-09-30T14:37:11.988243+0000","last_unstale":"2023-09-30T14:38:12.197227+0000","last_undegraded":"2023-09-30T14:38:12.197227+0000","last_fullsized":"2023-09-30T14:38:12.197227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:17:58.946159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194482+0000","last_change":"2023-09-30T14:37:11.990361+0000","last_active":"2023-09-30T14:38:12.194482+0000","last_peered":"2023-09-30T14:38:12.194482+0000","last_clean":"2023-09-30T14:38:12.194482+0000","last_became_active":"2023-09-30T14:37:11.990080+0000","last_became_peered":"2023-09-30T14:37:11.990080+0000","last_unstale":"2023-09-30T14:38:12.194482+0000","last_undegraded":"2023-09-30T14:38:12.194482+0000","last_fullsized":"2023-09-30T14:38:12.194482+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:27:08.830419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197518+0000","last_change":"2023-09-30T14:37:11.988119+0000","last_active":"2023-09-30T14:38:12.197518+0000","last_peered":"2023-09-30T14:38:12.197518+0000","last_clean":"2023-09-30T14:38:12.197518+0000","last_became_active":"2023-09-30T14:37:11.987704+0000","last_became_peered":"2023-09-30T14:37:11.987704+0000","last_unstale":"2023-09-30T14:38:12.197518+0000","last_undegraded":"2023-09-30T14:38:12.197518+0000","last_fullsized":"2023-09-30T14:38:12.197518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:47:23.318982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196560+0000","last_change":"2023-09-30T14:37:11.985403+0000","last_active":"2023-09-30T14:38:12.196560+0000","last_peered":"2023-09-30T14:38:12.196560+0000","last_clean":"2023-09-30T14:38:12.196560+0000","last_became_active":"2023-09-30T14:37:11.984818+0000","last_became_peered":"2023-09-30T14:37:11.984818+0000","last_unstale":"2023-09-30T14:38:12.196560+0000","last_undegraded":"2023-09-30T14:38:12.196560+0000","last_fullsized":"2023-09-30T14:38:12.196560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:10:43.592505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198212+0000","last_change":"2023-09-30T14:37:11.989500+0000","last_active":"2023-09-30T14:38:12.198212+0000","last_peered":"2023-09-30T14:38:12.198212+0000","last_clean":"2023-09-30T14:38:12.198212+0000","last_became_active":"2023-09-30T14:37:11.989186+0000","last_became_peered":"2023-09-30T14:37:11.989186+0000","last_unstale":"2023-09-30T14:38:12.198212+0000","last_undegraded":"2023-09-30T14:38:12.198212+0000","last_fullsized":"2023-09-30T14:38:12.198212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:14:35.497319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197968+0000","last_change":"2023-09-30T14:37:11.987217+0000","last_active":"2023-09-30T14:38:12.197968+0000","last_peered":"2023-09-30T14:38:12.197968+0000","last_clean":"2023-09-30T14:38:12.197968+0000","last_became_active":"2023-09-30T14:37:11.986835+0000","last_became_peered":"2023-09-30T14:37:11.986835+0000","last_unstale":"2023-09-30T14:38:12.197968+0000","last_undegraded":"2023-09-30T14:38:12.197968+0000","last_fullsized":"2023-09-30T14:38:12.197968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:11:19.234768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196774+0000","last_change":"2023-09-30T14:37:11.995224+0000","last_active":"2023-09-30T14:38:12.196774+0000","last_peered":"2023-09-30T14:38:12.196774+0000","last_clean":"2023-09-30T14:38:12.196774+0000","last_became_active":"2023-09-30T14:37:11.994925+0000","last_became_peered":"2023-09-30T14:37:11.994925+0000","last_unstale":"2023-09-30T14:38:12.196774+0000","last_undegraded":"2023-09-30T14:38:12.196774+0000","last_fullsized":"2023-09-30T14:38:12.196774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:08:07.453499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198775+0000","last_change":"2023-09-30T14:37:11.990182+0000","last_active":"2023-09-30T14:38:12.198775+0000","last_peered":"2023-09-30T14:38:12.198775+0000","last_clean":"2023-09-30T14:38:12.198775+0000","last_became_active":"2023-09-30T14:37:11.989862+0000","last_became_peered":"2023-09-30T14:37:11.989862+0000","last_unstale":"2023-09-30T14:38:12.198775+0000","last_undegraded":"2023-09-30T14:38:12.198775+0000","last_fullsized":"2023-09-30T14:38:12.198775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:15:31.713002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.209988+0000","last_change":"2023-09-30T14:38:13.209988+0000","last_active":"2023-09-30T14:38:13.209988+0000","last_peered":"2023-09-30T14:38:13.209988+0000","last_clean":"2023-09-30T14:38:13.209988+0000","last_became_active":"2023-09-30T14:38:13.202561+0000","last_became_peered":"2023-09-30T14:38:13.202561+0000","last_unstale":"2023-09-30T14:38:13.209988+0000","last_undegraded":"2023-09-30T14:38:13.209988+0000","last_fullsized":"2023-09-30T14:38:13.209988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:57:36.214608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824419+0000","last_change":"2023-09-30T14:37:11.994584+0000","last_active":"2023-09-30T14:38:12.824419+0000","last_peered":"2023-09-30T14:38:12.824419+0000","last_clean":"2023-09-30T14:38:12.824419+0000","last_became_active":"2023-09-30T14:37:11.992991+0000","last_became_peered":"2023-09-30T14:37:11.992991+0000","last_unstale":"2023-09-30T14:38:12.824419+0000","last_undegraded":"2023-09-30T14:38:12.824419+0000","last_fullsized":"2023-09-30T14:38:12.824419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:39:50.995717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824532+0000","last_change":"2023-09-30T14:37:11.020653+0000","last_active":"2023-09-30T14:38:12.824532+0000","last_peered":"2023-09-30T14:38:12.824532+0000","last_clean":"2023-09-30T14:38:12.824532+0000","last_became_active":"2023-09-30T14:37:11.017621+0000","last_became_peered":"2023-09-30T14:37:11.017621+0000","last_unstale":"2023-09-30T14:38:12.824532+0000","last_undegraded":"2023-09-30T14:38:12.824532+0000","last_fullsized":"2023-09-30T14:38:12.824532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:26:59.395278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196664+0000","last_change":"2023-09-30T14:37:11.999881+0000","last_active":"2023-09-30T14:38:12.196664+0000","last_peered":"2023-09-30T14:38:12.196664+0000","last_clean":"2023-09-30T14:38:12.196664+0000","last_became_active":"2023-09-30T14:37:11.999614+0000","last_became_peered":"2023-09-30T14:37:11.999614+0000","last_unstale":"2023-09-30T14:38:12.196664+0000","last_undegraded":"2023-09-30T14:38:12.196664+0000","last_fullsized":"2023-09-30T14:38:12.196664+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:59:44.198361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200580+0000","last_change":"2023-09-30T14:37:11.027187+0000","last_active":"2023-09-30T14:38:12.200580+0000","last_peered":"2023-09-30T14:38:12.200580+0000","last_clean":"2023-09-30T14:38:12.200580+0000","last_became_active":"2023-09-30T14:37:11.026882+0000","last_became_peered":"2023-09-30T14:37:11.026882+0000","last_unstale":"2023-09-30T14:38:12.200580+0000","last_undegraded":"2023-09-30T14:38:12.200580+0000","last_fullsized":"2023-09-30T14:38:12.200580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:17:11.824002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198427+0000","last_change":"2023-09-30T14:37:11.986979+0000","last_active":"2023-09-30T14:38:12.198427+0000","last_peered":"2023-09-30T14:38:12.198427+0000","last_clean":"2023-09-30T14:38:12.198427+0000","last_became_active":"2023-09-30T14:37:11.986705+0000","last_became_peered":"2023-09-30T14:37:11.986705+0000","last_unstale":"2023-09-30T14:38:12.198427+0000","last_undegraded":"2023-09-30T14:38:12.198427+0000","last_fullsized":"2023-09-30T14:38:12.198427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:31:40.319747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199932+0000","last_change":"2023-09-30T14:37:11.022313+0000","last_active":"2023-09-30T14:38:12.199932+0000","last_peered":"2023-09-30T14:38:12.199932+0000","last_clean":"2023-09-30T14:38:12.199932+0000","last_became_active":"2023-09-30T14:37:11.021949+0000","last_became_peered":"2023-09-30T14:37:11.021949+0000","last_unstale":"2023-09-30T14:38:12.199932+0000","last_undegraded":"2023-09-30T14:38:12.199932+0000","last_fullsized":"2023-09-30T14:38:12.199932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:30:26.913666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.202706+0000","last_change":"2023-09-30T14:38:13.202706+0000","last_active":"2023-09-30T14:38:13.202706+0000","last_peered":"2023-09-30T14:38:13.202706+0000","last_clean":"2023-09-30T14:38:13.202706+0000","last_became_active":"2023-09-30T14:38:13.202370+0000","last_became_peered":"2023-09-30T14:38:13.202370+0000","last_unstale":"2023-09-30T14:38:13.202706+0000","last_undegraded":"2023-09-30T14:38:13.202706+0000","last_fullsized":"2023-09-30T14:38:13.202706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:35:21.244531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":117,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:36.239380+0000","last_change":"2023-09-30T14:38:13.206650+0000","last_active":"2023-09-30T14:38:36.239380+0000","last_peered":"2023-09-30T14:38:36.239380+0000","last_clean":"2023-09-30T14:38:36.239380+0000","last_became_active":"2023-09-30T14:38:13.203337+0000","last_became_peered":"2023-09-30T14:38:13.203337+0000","last_unstale":"2023-09-30T14:38:36.239380+0000","last_undegraded":"2023-09-30T14:38:36.239380+0000","last_fullsized":"2023-09-30T14:38:36.239380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:08:37.996166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199571+0000","last_change":"2023-09-30T14:37:11.993074+0000","last_active":"2023-09-30T14:38:12.199571+0000","last_peered":"2023-09-30T14:38:12.199571+0000","last_clean":"2023-09-30T14:38:12.199571+0000","last_became_active":"2023-09-30T14:37:11.992784+0000","last_became_peered":"2023-09-30T14:37:11.992784+0000","last_unstale":"2023-09-30T14:38:12.199571+0000","last_undegraded":"2023-09-30T14:38:12.199571+0000","last_fullsized":"2023-09-30T14:38:12.199571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:28:18.454389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195465+0000","last_change":"2023-09-30T14:37:11.022677+0000","last_active":"2023-09-30T14:38:12.195465+0000","last_peered":"2023-09-30T14:38:12.195465+0000","last_clean":"2023-09-30T14:38:12.195465+0000","last_became_active":"2023-09-30T14:37:11.022357+0000","last_became_peered":"2023-09-30T14:37:11.022357+0000","last_unstale":"2023-09-30T14:38:12.195465+0000","last_undegraded":"2023-09-30T14:38:12.195465+0000","last_fullsized":"2023-09-30T14:38:12.195465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:00:15.500968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194862+0000","last_change":"2023-09-30T14:37:11.992358+0000","last_active":"2023-09-30T14:38:12.194862+0000","last_peered":"2023-09-30T14:38:12.194862+0000","last_clean":"2023-09-30T14:38:12.194862+0000","last_became_active":"2023-09-30T14:37:11.992094+0000","last_became_peered":"2023-09-30T14:37:11.992094+0000","last_unstale":"2023-09-30T14:38:12.194862+0000","last_undegraded":"2023-09-30T14:38:12.194862+0000","last_fullsized":"2023-09-30T14:38:12.194862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:59:36.817138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194237+0000","last_change":"2023-09-30T14:37:11.030961+0000","last_active":"2023-09-30T14:38:12.194237+0000","last_peered":"2023-09-30T14:38:12.194237+0000","last_clean":"2023-09-30T14:38:12.194237+0000","last_became_active":"2023-09-30T14:37:11.030662+0000","last_became_peered":"2023-09-30T14:37:11.030662+0000","last_unstale":"2023-09-30T14:38:12.194237+0000","last_undegraded":"2023-09-30T14:38:12.194237+0000","last_fullsized":"2023-09-30T14:38:12.194237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:35:13.313382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824002+0000","last_change":"2023-09-30T14:37:11.989039+0000","last_active":"2023-09-30T14:38:12.824002+0000","last_peered":"2023-09-30T14:38:12.824002+0000","last_clean":"2023-09-30T14:38:12.824002+0000","last_became_active":"2023-09-30T14:37:11.988594+0000","last_became_peered":"2023-09-30T14:37:11.988594+0000","last_unstale":"2023-09-30T14:38:12.824002+0000","last_undegraded":"2023-09-30T14:38:12.824002+0000","last_fullsized":"2023-09-30T14:38:12.824002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:52:58.146681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194495+0000","last_change":"2023-09-30T14:37:11.026545+0000","last_active":"2023-09-30T14:38:12.194495+0000","last_peered":"2023-09-30T14:38:12.194495+0000","last_clean":"2023-09-30T14:38:12.194495+0000","last_became_active":"2023-09-30T14:37:11.026063+0000","last_became_peered":"2023-09-30T14:37:11.026063+0000","last_unstale":"2023-09-30T14:38:12.194495+0000","last_undegraded":"2023-09-30T14:38:12.194495+0000","last_fullsized":"2023-09-30T14:38:12.194495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:15:34.020336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196143+0000","last_change":"2023-09-30T14:37:11.991845+0000","last_active":"2023-09-30T14:38:12.196143+0000","last_peered":"2023-09-30T14:38:12.196143+0000","last_clean":"2023-09-30T14:38:12.196143+0000","last_became_active":"2023-09-30T14:37:11.991560+0000","last_became_peered":"2023-09-30T14:37:11.991560+0000","last_unstale":"2023-09-30T14:38:12.196143+0000","last_undegraded":"2023-09-30T14:38:12.196143+0000","last_fullsized":"2023-09-30T14:38:12.196143+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:27:39.653882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":97,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:34.235519+0000","last_change":"2023-09-30T14:37:11.028192+0000","last_active":"2023-09-30T14:38:34.235519+0000","last_peered":"2023-09-30T14:38:34.235519+0000","last_clean":"2023-09-30T14:38:34.235519+0000","last_became_active":"2023-09-30T14:37:11.027902+0000","last_became_peered":"2023-09-30T14:37:11.027902+0000","last_unstale":"2023-09-30T14:38:34.235519+0000","last_undegraded":"2023-09-30T14:38:34.235519+0000","last_fullsized":"2023-09-30T14:38:34.235519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:21:30.267559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198607+0000","last_change":"2023-09-30T14:37:11.997318+0000","last_active":"2023-09-30T14:38:12.198607+0000","last_peered":"2023-09-30T14:38:12.198607+0000","last_clean":"2023-09-30T14:38:12.198607+0000","last_became_active":"2023-09-30T14:37:11.996988+0000","last_became_peered":"2023-09-30T14:37:11.996988+0000","last_unstale":"2023-09-30T14:38:12.198607+0000","last_undegraded":"2023-09-30T14:38:12.198607+0000","last_fullsized":"2023-09-30T14:38:12.198607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:57:21.510952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.259803+0000","last_change":"2023-09-30T14:38:13.259803+0000","last_active":"2023-09-30T14:38:13.259803+0000","last_peered":"2023-09-30T14:38:13.259803+0000","last_clean":"2023-09-30T14:38:13.259803+0000","last_became_active":"2023-09-30T14:38:13.257187+0000","last_became_peered":"2023-09-30T14:38:13.257187+0000","last_unstale":"2023-09-30T14:38:13.259803+0000","last_undegraded":"2023-09-30T14:38:13.259803+0000","last_fullsized":"2023-09-30T14:38:13.259803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:37:41.453519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196748+0000","last_change":"2023-09-30T14:37:11.989338+0000","last_active":"2023-09-30T14:38:12.196748+0000","last_peered":"2023-09-30T14:38:12.196748+0000","last_clean":"2023-09-30T14:38:12.196748+0000","last_became_active":"2023-09-30T14:37:11.989045+0000","last_became_peered":"2023-09-30T14:37:11.989045+0000","last_unstale":"2023-09-30T14:38:12.196748+0000","last_undegraded":"2023-09-30T14:38:12.196748+0000","last_fullsized":"2023-09-30T14:38:12.196748+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:58:59.359684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195371+0000","last_change":"2023-09-30T14:37:11.024709+0000","last_active":"2023-09-30T14:38:12.195371+0000","last_peered":"2023-09-30T14:38:12.195371+0000","last_clean":"2023-09-30T14:38:12.195371+0000","last_became_active":"2023-09-30T14:37:11.024344+0000","last_became_peered":"2023-09-30T14:37:11.024344+0000","last_unstale":"2023-09-30T14:38:12.195371+0000","last_undegraded":"2023-09-30T14:38:12.195371+0000","last_fullsized":"2023-09-30T14:38:12.195371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:17:49.508396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198298+0000","last_change":"2023-09-30T14:37:11.989797+0000","last_active":"2023-09-30T14:38:12.198298+0000","last_peered":"2023-09-30T14:38:12.198298+0000","last_clean":"2023-09-30T14:38:12.198298+0000","last_became_active":"2023-09-30T14:37:11.989482+0000","last_became_peered":"2023-09-30T14:37:11.989482+0000","last_unstale":"2023-09-30T14:38:12.198298+0000","last_undegraded":"2023-09-30T14:38:12.198298+0000","last_fullsized":"2023-09-30T14:38:12.198298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:33:20.437706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'26","reported_seq":206,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:35.237410+0000","last_change":"2023-09-30T14:37:11.027322+0000","last_active":"2023-09-30T14:38:35.237410+0000","last_peered":"2023-09-30T14:38:35.237410+0000","last_clean":"2023-09-30T14:38:35.237410+0000","last_became_active":"2023-09-30T14:37:11.027007+0000","last_became_peered":"2023-09-30T14:37:11.027007+0000","last_unstale":"2023-09-30T14:38:35.237410+0000","last_undegraded":"2023-09-30T14:38:35.237410+0000","last_fullsized":"2023-09-30T14:38:35.237410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:47:22.919317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54708,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":165,"num_read_kb":135,"num_write":27,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200357+0000","last_change":"2023-09-30T14:37:11.988469+0000","last_active":"2023-09-30T14:38:12.200357+0000","last_peered":"2023-09-30T14:38:12.200357+0000","last_clean":"2023-09-30T14:38:12.200357+0000","last_became_active":"2023-09-30T14:37:11.988118+0000","last_became_peered":"2023-09-30T14:37:11.988118+0000","last_unstale":"2023-09-30T14:38:12.200357+0000","last_undegraded":"2023-09-30T14:38:12.200357+0000","last_fullsized":"2023-09-30T14:38:12.200357+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:11:56.750690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198520+0000","last_change":"2023-09-30T14:37:11.024345+0000","last_active":"2023-09-30T14:38:12.198520+0000","last_peered":"2023-09-30T14:38:12.198520+0000","last_clean":"2023-09-30T14:38:12.198520+0000","last_became_active":"2023-09-30T14:37:11.024039+0000","last_became_peered":"2023-09-30T14:37:11.024039+0000","last_unstale":"2023-09-30T14:38:12.198520+0000","last_undegraded":"2023-09-30T14:38:12.198520+0000","last_fullsized":"2023-09-30T14:38:12.198520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:48:32.872814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194273+0000","last_change":"2023-09-30T14:37:11.994068+0000","last_active":"2023-09-30T14:38:12.194273+0000","last_peered":"2023-09-30T14:38:12.194273+0000","last_clean":"2023-09-30T14:38:12.194273+0000","last_became_active":"2023-09-30T14:37:11.993793+0000","last_became_peered":"2023-09-30T14:37:11.993793+0000","last_unstale":"2023-09-30T14:38:12.194273+0000","last_undegraded":"2023-09-30T14:38:12.194273+0000","last_fullsized":"2023-09-30T14:38:12.194273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:51:28.071414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194699+0000","last_change":"2023-09-30T14:37:11.025622+0000","last_active":"2023-09-30T14:38:12.194699+0000","last_peered":"2023-09-30T14:38:12.194699+0000","last_clean":"2023-09-30T14:38:12.194699+0000","last_became_active":"2023-09-30T14:37:11.025338+0000","last_became_peered":"2023-09-30T14:37:11.025338+0000","last_unstale":"2023-09-30T14:38:12.194699+0000","last_undegraded":"2023-09-30T14:38:12.194699+0000","last_fullsized":"2023-09-30T14:38:12.194699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:32:03.166296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200744+0000","last_change":"2023-09-30T14:37:11.990678+0000","last_active":"2023-09-30T14:38:12.200744+0000","last_peered":"2023-09-30T14:38:12.200744+0000","last_clean":"2023-09-30T14:38:12.200744+0000","last_became_active":"2023-09-30T14:37:11.990331+0000","last_became_peered":"2023-09-30T14:37:11.990331+0000","last_unstale":"2023-09-30T14:38:12.200744+0000","last_undegraded":"2023-09-30T14:38:12.200744+0000","last_fullsized":"2023-09-30T14:38:12.200744+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:57:06.296415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":103,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:36.101657+0000","last_change":"2023-09-30T14:37:11.029501+0000","last_active":"2023-09-30T14:38:36.101657+0000","last_peered":"2023-09-30T14:38:36.101657+0000","last_clean":"2023-09-30T14:38:36.101657+0000","last_became_active":"2023-09-30T14:37:11.029207+0000","last_became_peered":"2023-09-30T14:37:11.029207+0000","last_unstale":"2023-09-30T14:38:36.101657+0000","last_undegraded":"2023-09-30T14:38:36.101657+0000","last_fullsized":"2023-09-30T14:38:36.101657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:48:53.145952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":906,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198725+0000","last_change":"2023-09-30T14:37:11.993052+0000","last_active":"2023-09-30T14:38:12.198725+0000","last_peered":"2023-09-30T14:38:12.198725+0000","last_clean":"2023-09-30T14:38:12.198725+0000","last_became_active":"2023-09-30T14:37:11.992762+0000","last_became_peered":"2023-09-30T14:37:11.992762+0000","last_unstale":"2023-09-30T14:38:12.198725+0000","last_undegraded":"2023-09-30T14:38:12.198725+0000","last_fullsized":"2023-09-30T14:38:12.198725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:09:16.440026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.214212+0000","last_change":"2023-09-30T14:38:13.213859+0000","last_active":"2023-09-30T14:38:13.214212+0000","last_peered":"2023-09-30T14:38:13.214212+0000","last_clean":"2023-09-30T14:38:13.214212+0000","last_became_active":"2023-09-30T14:38:13.200010+0000","last_became_peered":"2023-09-30T14:38:13.200010+0000","last_unstale":"2023-09-30T14:38:13.214212+0000","last_undegraded":"2023-09-30T14:38:13.214212+0000","last_fullsized":"2023-09-30T14:38:13.214212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:08:48.668186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194503+0000","last_change":"2023-09-30T14:37:11.989592+0000","last_active":"2023-09-30T14:38:12.194503+0000","last_peered":"2023-09-30T14:38:12.194503+0000","last_clean":"2023-09-30T14:38:12.194503+0000","last_became_active":"2023-09-30T14:37:11.989290+0000","last_became_peered":"2023-09-30T14:37:11.989290+0000","last_unstale":"2023-09-30T14:38:12.194503+0000","last_undegraded":"2023-09-30T14:38:12.194503+0000","last_fullsized":"2023-09-30T14:38:12.194503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:11:33.974823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.201778+0000","last_change":"2023-09-30T14:37:11.028919+0000","last_active":"2023-09-30T14:38:13.201778+0000","last_peered":"2023-09-30T14:38:13.201778+0000","last_clean":"2023-09-30T14:38:13.201778+0000","last_became_active":"2023-09-30T14:37:11.028532+0000","last_became_peered":"2023-09-30T14:37:11.028532+0000","last_unstale":"2023-09-30T14:38:13.201778+0000","last_undegraded":"2023-09-30T14:38:13.201778+0000","last_fullsized":"2023-09-30T14:38:13.201778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:15:42.675409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197404+0000","last_change":"2023-09-30T14:37:11.980611+0000","last_active":"2023-09-30T14:38:12.197404+0000","last_peered":"2023-09-30T14:38:12.197404+0000","last_clean":"2023-09-30T14:38:12.197404+0000","last_became_active":"2023-09-30T14:37:11.980288+0000","last_became_peered":"2023-09-30T14:37:11.980288+0000","last_unstale":"2023-09-30T14:38:12.197404+0000","last_undegraded":"2023-09-30T14:38:12.197404+0000","last_fullsized":"2023-09-30T14:38:12.197404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:38:49.270976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196478+0000","last_change":"2023-09-30T14:37:11.025133+0000","last_active":"2023-09-30T14:38:12.196478+0000","last_peered":"2023-09-30T14:38:12.196478+0000","last_clean":"2023-09-30T14:38:12.196478+0000","last_became_active":"2023-09-30T14:37:11.024914+0000","last_became_peered":"2023-09-30T14:37:11.024914+0000","last_unstale":"2023-09-30T14:38:12.196478+0000","last_undegraded":"2023-09-30T14:38:12.196478+0000","last_fullsized":"2023-09-30T14:38:12.196478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:23:39.328773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195743+0000","last_change":"2023-09-30T14:37:11.988434+0000","last_active":"2023-09-30T14:38:12.195743+0000","last_peered":"2023-09-30T14:38:12.195743+0000","last_clean":"2023-09-30T14:38:12.195743+0000","last_became_active":"2023-09-30T14:37:11.988156+0000","last_became_peered":"2023-09-30T14:37:11.988156+0000","last_unstale":"2023-09-30T14:38:12.195743+0000","last_undegraded":"2023-09-30T14:38:12.195743+0000","last_fullsized":"2023-09-30T14:38:12.195743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:20:49.420026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198842+0000","last_change":"2023-09-30T14:37:11.026290+0000","last_active":"2023-09-30T14:38:12.198842+0000","last_peered":"2023-09-30T14:38:12.198842+0000","last_clean":"2023-09-30T14:38:12.198842+0000","last_became_active":"2023-09-30T14:37:11.024684+0000","last_became_peered":"2023-09-30T14:37:11.024684+0000","last_unstale":"2023-09-30T14:38:12.198842+0000","last_undegraded":"2023-09-30T14:38:12.198842+0000","last_fullsized":"2023-09-30T14:38:12.198842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:43:06.463243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195851+0000","last_change":"2023-09-30T14:37:11.987989+0000","last_active":"2023-09-30T14:38:12.195851+0000","last_peered":"2023-09-30T14:38:12.195851+0000","last_clean":"2023-09-30T14:38:12.195851+0000","last_became_active":"2023-09-30T14:37:11.987690+0000","last_became_peered":"2023-09-30T14:37:11.987690+0000","last_unstale":"2023-09-30T14:38:12.195851+0000","last_undegraded":"2023-09-30T14:38:12.195851+0000","last_fullsized":"2023-09-30T14:38:12.195851+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:51:25.825871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194553+0000","last_change":"2023-09-30T14:37:11.996852+0000","last_active":"2023-09-30T14:38:12.194553+0000","last_peered":"2023-09-30T14:38:12.194553+0000","last_clean":"2023-09-30T14:38:12.194553+0000","last_became_active":"2023-09-30T14:37:11.996596+0000","last_became_peered":"2023-09-30T14:37:11.996596+0000","last_unstale":"2023-09-30T14:38:12.194553+0000","last_undegraded":"2023-09-30T14:38:12.194553+0000","last_fullsized":"2023-09-30T14:38:12.194553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:25:04.531236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":98,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:34.096608+0000","last_change":"2023-09-30T14:37:11.025458+0000","last_active":"2023-09-30T14:38:34.096608+0000","last_peered":"2023-09-30T14:38:34.096608+0000","last_clean":"2023-09-30T14:38:34.096608+0000","last_became_active":"2023-09-30T14:37:11.025157+0000","last_became_peered":"2023-09-30T14:37:11.025157+0000","last_unstale":"2023-09-30T14:38:34.096608+0000","last_undegraded":"2023-09-30T14:38:34.096608+0000","last_fullsized":"2023-09-30T14:38:34.096608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:31:24.436125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824345+0000","last_change":"2023-09-30T14:37:11.986840+0000","last_active":"2023-09-30T14:38:12.824345+0000","last_peered":"2023-09-30T14:38:12.824345+0000","last_clean":"2023-09-30T14:38:12.824345+0000","last_became_active":"2023-09-30T14:37:11.986483+0000","last_became_peered":"2023-09-30T14:37:11.986483+0000","last_unstale":"2023-09-30T14:38:12.824345+0000","last_undegraded":"2023-09-30T14:38:12.824345+0000","last_fullsized":"2023-09-30T14:38:12.824345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:14:32.399685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196845+0000","last_change":"2023-09-30T14:37:11.025706+0000","last_active":"2023-09-30T14:38:12.196845+0000","last_peered":"2023-09-30T14:38:12.196845+0000","last_clean":"2023-09-30T14:38:12.196845+0000","last_became_active":"2023-09-30T14:37:11.025376+0000","last_became_peered":"2023-09-30T14:37:11.025376+0000","last_unstale":"2023-09-30T14:38:12.196845+0000","last_undegraded":"2023-09-30T14:38:12.196845+0000","last_fullsized":"2023-09-30T14:38:12.196845+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:10:36.025273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":310,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198525+0000","last_change":"2023-09-30T14:36:12.344762+0000","last_active":"2023-09-30T14:38:12.198525+0000","last_peered":"2023-09-30T14:38:12.198525+0000","last_clean":"2023-09-30T14:38:12.198525+0000","last_became_active":"2023-09-30T14:36:12.344268+0000","last_became_peered":"2023-09-30T14:36:12.344268+0000","last_unstale":"2023-09-30T14:38:12.198525+0000","last_undegraded":"2023-09-30T14:38:12.198525+0000","last_fullsized":"2023-09-30T14:38:12.198525+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:33:20.434992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:33:20.434992+0000","last_clean_scrub_stamp":"2023-09-30T14:33:20.434992+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:00:05.598961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194122+0000","last_change":"2023-09-30T14:37:11.996886+0000","last_active":"2023-09-30T14:38:12.194122+0000","last_peered":"2023-09-30T14:38:12.194122+0000","last_clean":"2023-09-30T14:38:12.194122+0000","last_became_active":"2023-09-30T14:37:11.996667+0000","last_became_peered":"2023-09-30T14:37:11.996667+0000","last_unstale":"2023-09-30T14:38:12.194122+0000","last_undegraded":"2023-09-30T14:38:12.194122+0000","last_fullsized":"2023-09-30T14:38:12.194122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:27:00.546356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198414+0000","last_change":"2023-09-30T14:37:11.027012+0000","last_active":"2023-09-30T14:38:12.198414+0000","last_peered":"2023-09-30T14:38:12.198414+0000","last_clean":"2023-09-30T14:38:12.198414+0000","last_became_active":"2023-09-30T14:37:11.026714+0000","last_became_peered":"2023-09-30T14:37:11.026714+0000","last_unstale":"2023-09-30T14:38:12.198414+0000","last_undegraded":"2023-09-30T14:38:12.198414+0000","last_fullsized":"2023-09-30T14:38:12.198414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:17:57.860438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200462+0000","last_change":"2023-09-30T14:37:11.027389+0000","last_active":"2023-09-30T14:38:12.200462+0000","last_peered":"2023-09-30T14:38:12.200462+0000","last_clean":"2023-09-30T14:38:12.200462+0000","last_became_active":"2023-09-30T14:37:11.027011+0000","last_became_peered":"2023-09-30T14:37:11.027011+0000","last_unstale":"2023-09-30T14:38:12.200462+0000","last_undegraded":"2023-09-30T14:38:12.200462+0000","last_fullsized":"2023-09-30T14:38:12.200462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:38:22.953936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194384+0000","last_change":"2023-09-30T14:37:11.997675+0000","last_active":"2023-09-30T14:38:12.194384+0000","last_peered":"2023-09-30T14:38:12.194384+0000","last_clean":"2023-09-30T14:38:12.194384+0000","last_became_active":"2023-09-30T14:37:11.997330+0000","last_became_peered":"2023-09-30T14:37:11.997330+0000","last_unstale":"2023-09-30T14:38:12.194384+0000","last_undegraded":"2023-09-30T14:38:12.194384+0000","last_fullsized":"2023-09-30T14:38:12.194384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:54:52.802138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196687+0000","last_change":"2023-09-30T14:37:11.023836+0000","last_active":"2023-09-30T14:38:12.196687+0000","last_peered":"2023-09-30T14:38:12.196687+0000","last_clean":"2023-09-30T14:38:12.196687+0000","last_became_active":"2023-09-30T14:37:11.023362+0000","last_became_peered":"2023-09-30T14:37:11.023362+0000","last_unstale":"2023-09-30T14:38:12.196687+0000","last_undegraded":"2023-09-30T14:38:12.196687+0000","last_fullsized":"2023-09-30T14:38:12.196687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:58:22.772546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824591+0000","last_change":"2023-09-30T14:37:11.985281+0000","last_active":"2023-09-30T14:38:12.824591+0000","last_peered":"2023-09-30T14:38:12.824591+0000","last_clean":"2023-09-30T14:38:12.824591+0000","last_became_active":"2023-09-30T14:37:11.984944+0000","last_became_peered":"2023-09-30T14:37:11.984944+0000","last_unstale":"2023-09-30T14:38:12.824591+0000","last_undegraded":"2023-09-30T14:38:12.824591+0000","last_fullsized":"2023-09-30T14:38:12.824591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:30:19.829727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197438+0000","last_change":"2023-09-30T14:37:11.028070+0000","last_active":"2023-09-30T14:38:12.197438+0000","last_peered":"2023-09-30T14:38:12.197438+0000","last_clean":"2023-09-30T14:38:12.197438+0000","last_became_active":"2023-09-30T14:37:11.027703+0000","last_became_peered":"2023-09-30T14:37:11.027703+0000","last_unstale":"2023-09-30T14:38:12.197438+0000","last_undegraded":"2023-09-30T14:38:12.197438+0000","last_fullsized":"2023-09-30T14:38:12.197438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:23:15.071946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197147+0000","last_change":"2023-09-30T14:37:11.989897+0000","last_active":"2023-09-30T14:38:12.197147+0000","last_peered":"2023-09-30T14:38:12.197147+0000","last_clean":"2023-09-30T14:38:12.197147+0000","last_became_active":"2023-09-30T14:37:11.989613+0000","last_became_peered":"2023-09-30T14:37:11.989613+0000","last_unstale":"2023-09-30T14:38:12.197147+0000","last_undegraded":"2023-09-30T14:38:12.197147+0000","last_fullsized":"2023-09-30T14:38:12.197147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:43:17.887766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198152+0000","last_change":"2023-09-30T14:37:11.026491+0000","last_active":"2023-09-30T14:38:12.198152+0000","last_peered":"2023-09-30T14:38:12.198152+0000","last_clean":"2023-09-30T14:38:12.198152+0000","last_became_active":"2023-09-30T14:37:11.026196+0000","last_became_peered":"2023-09-30T14:37:11.026196+0000","last_unstale":"2023-09-30T14:38:12.198152+0000","last_undegraded":"2023-09-30T14:38:12.198152+0000","last_fullsized":"2023-09-30T14:38:12.198152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:17:22.343377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199319+0000","last_change":"2023-09-30T14:37:11.988570+0000","last_active":"2023-09-30T14:38:12.199319+0000","last_peered":"2023-09-30T14:38:12.199319+0000","last_clean":"2023-09-30T14:38:12.199319+0000","last_became_active":"2023-09-30T14:37:11.988230+0000","last_became_peered":"2023-09-30T14:37:11.988230+0000","last_unstale":"2023-09-30T14:38:12.199319+0000","last_undegraded":"2023-09-30T14:38:12.199319+0000","last_fullsized":"2023-09-30T14:38:12.199319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T17:58:38.302343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824045+0000","last_change":"2023-09-30T14:37:11.024587+0000","last_active":"2023-09-30T14:38:12.824045+0000","last_peered":"2023-09-30T14:38:12.824045+0000","last_clean":"2023-09-30T14:38:12.824045+0000","last_became_active":"2023-09-30T14:37:11.024253+0000","last_became_peered":"2023-09-30T14:37:11.024253+0000","last_unstale":"2023-09-30T14:38:12.824045+0000","last_undegraded":"2023-09-30T14:38:12.824045+0000","last_fullsized":"2023-09-30T14:38:12.824045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:31:07.293657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197186+0000","last_change":"2023-09-30T14:37:11.987818+0000","last_active":"2023-09-30T14:38:12.197186+0000","last_peered":"2023-09-30T14:38:12.197186+0000","last_clean":"2023-09-30T14:38:12.197186+0000","last_became_active":"2023-09-30T14:37:11.987476+0000","last_became_peered":"2023-09-30T14:37:11.987476+0000","last_unstale":"2023-09-30T14:38:12.197186+0000","last_undegraded":"2023-09-30T14:38:12.197186+0000","last_fullsized":"2023-09-30T14:38:12.197186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:01:07.802959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199043+0000","last_change":"2023-09-30T14:37:11.023223+0000","last_active":"2023-09-30T14:38:12.199043+0000","last_peered":"2023-09-30T14:38:12.199043+0000","last_clean":"2023-09-30T14:38:12.199043+0000","last_became_active":"2023-09-30T14:37:11.022932+0000","last_became_peered":"2023-09-30T14:37:11.022932+0000","last_unstale":"2023-09-30T14:38:12.199043+0000","last_undegraded":"2023-09-30T14:38:12.199043+0000","last_fullsized":"2023-09-30T14:38:12.199043+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:01:04.315669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824324+0000","last_change":"2023-09-30T14:37:11.990323+0000","last_active":"2023-09-30T14:38:12.824324+0000","last_peered":"2023-09-30T14:38:12.824324+0000","last_clean":"2023-09-30T14:38:12.824324+0000","last_became_active":"2023-09-30T14:37:11.989981+0000","last_became_peered":"2023-09-30T14:37:11.989981+0000","last_unstale":"2023-09-30T14:38:12.824324+0000","last_undegraded":"2023-09-30T14:38:12.824324+0000","last_fullsized":"2023-09-30T14:38:12.824324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:30:44.385419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199908+0000","last_change":"2023-09-30T14:37:11.025658+0000","last_active":"2023-09-30T14:38:12.199908+0000","last_peered":"2023-09-30T14:38:12.199908+0000","last_clean":"2023-09-30T14:38:12.199908+0000","last_became_active":"2023-09-30T14:37:11.025334+0000","last_became_peered":"2023-09-30T14:37:11.025334+0000","last_unstale":"2023-09-30T14:38:12.199908+0000","last_undegraded":"2023-09-30T14:38:12.199908+0000","last_fullsized":"2023-09-30T14:38:12.199908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:23:41.933634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198716+0000","last_change":"2023-09-30T14:37:11.989989+0000","last_active":"2023-09-30T14:38:12.198716+0000","last_peered":"2023-09-30T14:38:12.198716+0000","last_clean":"2023-09-30T14:38:12.198716+0000","last_became_active":"2023-09-30T14:37:11.989666+0000","last_became_peered":"2023-09-30T14:37:11.989666+0000","last_unstale":"2023-09-30T14:38:12.198716+0000","last_undegraded":"2023-09-30T14:38:12.198716+0000","last_fullsized":"2023-09-30T14:38:12.198716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:41:09.835210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198863+0000","last_change":"2023-09-30T14:37:11.025027+0000","last_active":"2023-09-30T14:38:12.198863+0000","last_peered":"2023-09-30T14:38:12.198863+0000","last_clean":"2023-09-30T14:38:12.198863+0000","last_became_active":"2023-09-30T14:37:11.024696+0000","last_became_peered":"2023-09-30T14:37:11.024696+0000","last_unstale":"2023-09-30T14:38:12.198863+0000","last_undegraded":"2023-09-30T14:38:12.198863+0000","last_fullsized":"2023-09-30T14:38:12.198863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:34:28.384535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199073+0000","last_change":"2023-09-30T14:37:11.992674+0000","last_active":"2023-09-30T14:38:12.199073+0000","last_peered":"2023-09-30T14:38:12.199073+0000","last_clean":"2023-09-30T14:38:12.199073+0000","last_became_active":"2023-09-30T14:37:11.992243+0000","last_became_peered":"2023-09-30T14:37:11.992243+0000","last_unstale":"2023-09-30T14:38:12.199073+0000","last_undegraded":"2023-09-30T14:38:12.199073+0000","last_fullsized":"2023-09-30T14:38:12.199073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:33:19.759637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194984+0000","last_change":"2023-09-30T14:37:11.024525+0000","last_active":"2023-09-30T14:38:12.194984+0000","last_peered":"2023-09-30T14:38:12.194984+0000","last_clean":"2023-09-30T14:38:12.194984+0000","last_became_active":"2023-09-30T14:37:11.024231+0000","last_became_peered":"2023-09-30T14:37:11.024231+0000","last_unstale":"2023-09-30T14:38:12.194984+0000","last_undegraded":"2023-09-30T14:38:12.194984+0000","last_fullsized":"2023-09-30T14:38:12.194984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T16:53:47.142798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198957+0000","last_change":"2023-09-30T14:37:11.991401+0000","last_active":"2023-09-30T14:38:12.198957+0000","last_peered":"2023-09-30T14:38:12.198957+0000","last_clean":"2023-09-30T14:38:12.198957+0000","last_became_active":"2023-09-30T14:37:11.991012+0000","last_became_peered":"2023-09-30T14:37:11.991012+0000","last_unstale":"2023-09-30T14:38:12.198957+0000","last_undegraded":"2023-09-30T14:38:12.198957+0000","last_fullsized":"2023-09-30T14:38:12.198957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T01:55:00.562030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.195033+0000","last_change":"2023-09-30T14:37:11.022893+0000","last_active":"2023-09-30T14:38:12.195033+0000","last_peered":"2023-09-30T14:38:12.195033+0000","last_clean":"2023-09-30T14:38:12.195033+0000","last_became_active":"2023-09-30T14:37:11.022505+0000","last_became_peered":"2023-09-30T14:37:11.022505+0000","last_unstale":"2023-09-30T14:38:12.195033+0000","last_undegraded":"2023-09-30T14:38:12.195033+0000","last_fullsized":"2023-09-30T14:38:12.195033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:09.999125+0000","last_clean_scrub_stamp":"2023-09-30T14:37:09.999125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:00:17.721645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200355+0000","last_change":"2023-09-30T14:37:11.992942+0000","last_active":"2023-09-30T14:38:12.200355+0000","last_peered":"2023-09-30T14:38:12.200355+0000","last_clean":"2023-09-30T14:38:12.200355+0000","last_became_active":"2023-09-30T14:37:11.992698+0000","last_became_peered":"2023-09-30T14:37:11.992698+0000","last_unstale":"2023-09-30T14:38:12.200355+0000","last_undegraded":"2023-09-30T14:38:12.200355+0000","last_fullsized":"2023-09-30T14:38:12.200355+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:15:30.239423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.201641+0000","last_change":"2023-09-30T14:38:13.201641+0000","last_active":"2023-09-30T14:38:13.201641+0000","last_peered":"2023-09-30T14:38:13.201641+0000","last_clean":"2023-09-30T14:38:13.201641+0000","last_became_active":"2023-09-30T14:38:13.201255+0000","last_became_peered":"2023-09-30T14:38:13.201255+0000","last_unstale":"2023-09-30T14:38:13.201641+0000","last_undegraded":"2023-09-30T14:38:13.201641+0000","last_fullsized":"2023-09-30T14:38:13.201641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T00:06:02.731345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.199114+0000","last_change":"2023-09-30T14:37:11.984481+0000","last_active":"2023-09-30T14:38:12.199114+0000","last_peered":"2023-09-30T14:38:12.199114+0000","last_clean":"2023-09-30T14:38:12.199114+0000","last_became_active":"2023-09-30T14:37:11.984021+0000","last_became_peered":"2023-09-30T14:37:11.984021+0000","last_unstale":"2023-09-30T14:38:12.199114+0000","last_undegraded":"2023-09-30T14:38:12.199114+0000","last_fullsized":"2023-09-30T14:38:12.199114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-02T02:36:33.184379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198978+0000","last_change":"2023-09-30T14:37:11.990779+0000","last_active":"2023-09-30T14:38:12.198978+0000","last_peered":"2023-09-30T14:38:12.198978+0000","last_clean":"2023-09-30T14:38:12.198978+0000","last_became_active":"2023-09-30T14:37:11.989960+0000","last_became_peered":"2023-09-30T14:37:11.989960+0000","last_unstale":"2023-09-30T14:38:12.198978+0000","last_undegraded":"2023-09-30T14:38:12.198978+0000","last_fullsized":"2023-09-30T14:38:12.198978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T15:54:36.813191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.196893+0000","last_change":"2023-09-30T14:37:11.995750+0000","last_active":"2023-09-30T14:38:12.196893+0000","last_peered":"2023-09-30T14:38:12.196893+0000","last_clean":"2023-09-30T14:38:12.196893+0000","last_became_active":"2023-09-30T14:37:11.995434+0000","last_became_peered":"2023-09-30T14:37:11.995434+0000","last_unstale":"2023-09-30T14:38:12.196893+0000","last_undegraded":"2023-09-30T14:38:12.196893+0000","last_fullsized":"2023-09-30T14:38:12.196893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T14:49:51.234024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-09-30T14:38:13.201564+0000","last_change":"2023-09-30T14:38:13.201564+0000","last_active":"2023-09-30T14:38:13.201564+0000","last_peered":"2023-09-30T14:38:13.201564+0000","last_clean":"2023-09-30T14:38:13.201564+0000","last_became_active":"2023-09-30T14:38:13.201175+0000","last_became_peered":"2023-09-30T14:38:13.201175+0000","last_unstale":"2023-09-30T14:38:13.201564+0000","last_undegraded":"2023-09-30T14:38:13.201564+0000","last_fullsized":"2023-09-30T14:38:13.201564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:46:23.763202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197032+0000","last_change":"2023-09-30T14:37:11.983495+0000","last_active":"2023-09-30T14:38:12.197032+0000","last_peered":"2023-09-30T14:38:12.197032+0000","last_clean":"2023-09-30T14:38:12.197032+0000","last_became_active":"2023-09-30T14:37:11.983241+0000","last_became_peered":"2023-09-30T14:37:11.983241+0000","last_unstale":"2023-09-30T14:38:12.197032+0000","last_undegraded":"2023-09-30T14:38:12.197032+0000","last_fullsized":"2023-09-30T14:38:12.197032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:41:46.604467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.200166+0000","last_change":"2023-09-30T14:37:11.996058+0000","last_active":"2023-09-30T14:38:12.200166+0000","last_peered":"2023-09-30T14:38:12.200166+0000","last_clean":"2023-09-30T14:38:12.200166+0000","last_became_active":"2023-09-30T14:37:11.995763+0000","last_became_peered":"2023-09-30T14:37:11.995763+0000","last_unstale":"2023-09-30T14:38:12.200166+0000","last_undegraded":"2023-09-30T14:38:12.200166+0000","last_fullsized":"2023-09-30T14:38:12.200166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T20:42:21.306561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.824505+0000","last_change":"2023-09-30T14:37:11.992080+0000","last_active":"2023-09-30T14:38:12.824505+0000","last_peered":"2023-09-30T14:38:12.824505+0000","last_clean":"2023-09-30T14:38:12.824505+0000","last_became_active":"2023-09-30T14:37:11.991747+0000","last_became_peered":"2023-09-30T14:37:11.991747+0000","last_unstale":"2023-09-30T14:38:12.824505+0000","last_undegraded":"2023-09-30T14:38:12.824505+0000","last_fullsized":"2023-09-30T14:38:12.824505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:07:25.985911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194496+0000","last_change":"2023-09-30T14:37:11.997109+0000","last_active":"2023-09-30T14:38:12.194496+0000","last_peered":"2023-09-30T14:38:12.194496+0000","last_clean":"2023-09-30T14:38:12.194496+0000","last_became_active":"2023-09-30T14:37:11.996793+0000","last_became_peered":"2023-09-30T14:37:11.996793+0000","last_unstale":"2023-09-30T14:38:12.194496+0000","last_undegraded":"2023-09-30T14:38:12.194496+0000","last_fullsized":"2023-09-30T14:38:12.194496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T22:13:59.838638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.823992+0000","last_change":"2023-09-30T14:37:11.984083+0000","last_active":"2023-09-30T14:38:12.823992+0000","last_peered":"2023-09-30T14:38:12.823992+0000","last_clean":"2023-09-30T14:38:12.823992+0000","last_became_active":"2023-09-30T14:37:11.983352+0000","last_became_peered":"2023-09-30T14:37:11.983352+0000","last_unstale":"2023-09-30T14:38:12.823992+0000","last_undegraded":"2023-09-30T14:38:12.823992+0000","last_fullsized":"2023-09-30T14:38:12.823992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T21:02:03.668484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.198484+0000","last_change":"2023-09-30T14:37:11.992212+0000","last_active":"2023-09-30T14:38:12.198484+0000","last_peered":"2023-09-30T14:38:12.198484+0000","last_clean":"2023-09-30T14:38:12.198484+0000","last_became_active":"2023-09-30T14:37:11.990158+0000","last_became_peered":"2023-09-30T14:37:11.990158+0000","last_unstale":"2023-09-30T14:38:12.198484+0000","last_undegraded":"2023-09-30T14:38:12.198484+0000","last_fullsized":"2023-09-30T14:38:12.198484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T18:14:27.976110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.194553+0000","last_change":"2023-09-30T14:37:11.995884+0000","last_active":"2023-09-30T14:38:12.194553+0000","last_peered":"2023-09-30T14:38:12.194553+0000","last_clean":"2023-09-30T14:38:12.194553+0000","last_became_active":"2023-09-30T14:37:11.995551+0000","last_became_peered":"2023-09-30T14:37:11.995551+0000","last_unstale":"2023-09-30T14:38:12.194553+0000","last_undegraded":"2023-09-30T14:38:12.194553+0000","last_fullsized":"2023-09-30T14:38:12.194553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T23:28:26.051188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-09-30T14:38:12.197182+0000","last_change":"2023-09-30T14:37:11.985853+0000","last_active":"2023-09-30T14:38:12.197182+0000","last_peered":"2023-09-30T14:38:12.197182+0000","last_clean":"2023-09-30T14:38:12.197182+0000","last_became_active":"2023-09-30T14:37:11.985525+0000","last_became_peered":"2023-09-30T14:37:11.985525+0000","last_unstale":"2023-09-30T14:38:12.197182+0000","last_undegraded":"2023-09-30T14:38:12.197182+0000","last_fullsized":"2023-09-30T14:38:12.197182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T14:37:11.006182+0000","last_clean_scrub_stamp":"2023-09-30T14:37:11.006182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T19:07:24.392538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58636,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":497,"num_read_kb":311,"num_write":150,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1316,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":184866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449116,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":443610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.56599999999999995,"15min":0.57799999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.60199999999999998,"5min":1.056,"15min":1.056},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.57999999999999996,"15min":0.59799999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.621,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.51300000000000001}]},{"osd":1,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.51500000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.80300000000000005,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.78700000000000003,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.63}]},{"osd":2,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.499,"15min":0.497},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.76100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68200000000000005,"15min":0.67600000000000005},"min":{"1min":0.46000000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.012,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":1.012,"15min":1.012},"last":0.68100000000000005}]},{"osd":5,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":1.226,"15min":1.226},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.624,"15min":0.628},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94999999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.90300000000000002}]},{"osd":6,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70899999999999996,"15min":0.71699999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.73899999999999999}]},{"osd":7,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79100000000000004,"15min":0.79600000000000004},"min":{"1min":0.53100000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.0369999999999999,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.79400000000000004,"15min":0.80300000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.032,"5min":1.319,"15min":1.319},"last":0.92400000000000004}]},{"osd":8,"last update":"Sat Sep 30 14:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.95699999999999996}]},{"osd":9,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.77700000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97099999999999997,"5min":1.597,"15min":1.597},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.84499999999999997,"15min":0.84599999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":1.012}]},{"osd":10,"last update":"Sat Sep 30 14:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.82499999999999996,"15min":0.83399999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0609999999999999,"5min":1.417,"15min":1.417},"last":0.873},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80800000000000005,"15min":0.81200000000000006},"min":{"1min":0.54100000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0469999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.93700000000000006}]}]},{"osd":10,"up_from":71,"seq":304942678048,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1605632,"data_stored":1088780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47599999999999998,"15min":0.47999999999999998},"min":{"1min":0.28100000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64600000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.33200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.67500000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48299999999999998}]},{"osd":2,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76000000000000001,"5min":0.879,"15min":0.879},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.59199999999999997}]},{"osd":3,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.65800000000000003,"15min":0.67600000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.75600000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.65200000000000002,"15min":0.67500000000000004},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79700000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.50800000000000001}]},{"osd":4,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67100000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.60199999999999998,"15min":0.61499999999999999},"min":{"1min":0.374,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71999999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57799999999999996}]},{"osd":5,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.70299999999999996,"15min":0.71599999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77500000000000002,"5min":1.244,"15min":1.244},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.505,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.80500000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73799999999999999}]},{"osd":6,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.752,"15min":0.77200000000000002},"min":{"1min":0.44600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77800000000000002,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.57999999999999996,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85299999999999998,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.70399999999999996}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.41699999999999998}]},{"osd":8,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.82099999999999995,"15min":0.83499999999999996},"min":{"1min":0.53200000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.84199999999999997,"5min":1.335,"15min":1.335},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.82799999999999996,"15min":0.84499999999999997},"min":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.85799999999999998,"5min":1.218,"15min":1.218},"last":0.78200000000000003}]},{"osd":9,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.83899999999999997,"15min":0.85899999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.91000000000000003,"5min":1.389,"15min":1.389},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.84599999999999997,"15min":0.86799999999999999},"min":{"1min":0.501,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.872,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.69199999999999995}]},{"osd":11,"last update":"Sat Sep 30 14:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79700000000000004,"15min":0.79900000000000004},"min":{"1min":0.63200000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90800000000000003,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.85499999999999998,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.82799999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841572,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":442821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.46500000000000002,"15min":0.47499999999999998},"min":{"1min":0.28499999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60299999999999998,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.378},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.503,"15min":0.51800000000000002},"min":{"1min":0.32600000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65700000000000003,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.32500000000000001}]},{"osd":1,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77400000000000002,"5min":1.069,"15min":1.069},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.64500000000000002,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.749,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.60099999999999998}]},{"osd":2,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.61799999999999999,"15min":0.628},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.38,"5min":0.376,"15min":0.376},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.45600000000000002}]},{"osd":3,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.54900000000000004,"15min":0.56599999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69899999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.86199999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.42899999999999999}]},{"osd":4,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.69299999999999995,"15min":0.70999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80500000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.71399999999999997,"15min":0.72699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86399999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.503}]},{"osd":5,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.70499999999999996,"15min":0.72199999999999998},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":1.137,"15min":1.137},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.73299999999999998,"15min":0.749},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0860000000000001,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.57999999999999996}]},{"osd":6,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.69499999999999995,"15min":0.71799999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.046,"5min":1.21,"15min":1.21},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.71599999999999997,"15min":0.73599999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96299999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.495}]},{"osd":7,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.79700000000000004,"15min":0.81599999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94099999999999995,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.751},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.81200000000000006,"15min":0.82799999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.65800000000000003}]},{"osd":8,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81200000000000006,"15min":0.82599999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.81599999999999995,"15min":0.83599999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.028,"5min":1.466,"15min":1.466},"last":0.56100000000000005}]},{"osd":10,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.80900000000000005,"15min":0.82499999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90400000000000003,"5min":1.391,"15min":1.391},"last":0.628},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79600000000000004,"15min":0.80800000000000005},"min":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.91200000000000003,"5min":1.121,"15min":1.121},"last":0.73199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]}]},{"osd":8,"up_from":58,"seq":249108103209,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.42199999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.437,"5min":0.316,"15min":0.316},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.54200000000000004}]},{"osd":1,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.58799999999999997,"15min":0.56399999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90300000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.60999999999999999,"15min":0.59499999999999997},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86099999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.748}]},{"osd":2,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.47299999999999998,"15min":0.45500000000000002},"min":{"1min":0.38800000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.48499999999999999,"15min":0.46600000000000003},"min":{"1min":0.35999999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.63200000000000001}]},{"osd":3,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60699999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.435,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.57499999999999996}]},{"osd":4,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.63500000000000001,"15min":0.60899999999999999},"min":{"1min":0.40699999999999997,"5min":0.378,"15min":0.378},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.66100000000000003,"15min":0.63700000000000001},"min":{"1min":0.60999999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.88500000000000001}]},{"osd":5,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86599999999999999,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.622,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.83799999999999997}]},{"osd":6,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.68500000000000005,"15min":0.66800000000000004},"min":{"1min":0.51200000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.46700000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98499999999999999,"5min":1.133,"15min":1.133},"last":0.86199999999999999}]},{"osd":7,"last update":"Sat Sep 30 14:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.73999999999999999,"15min":0.72299999999999998},"min":{"1min":0.58999999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.012,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.74199999999999999,"15min":0.72399999999999998},"min":{"1min":0.53600000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.95299999999999996}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.749,"15min":0.747},"min":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95299999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.82099999999999995}]},{"osd":10,"last update":"Sat Sep 30 14:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.76800000000000002,"15min":0.74199999999999999},"min":{"1min":0.71799999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.018},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.76200000000000001,"15min":0.73699999999999999},"min":{"1min":0.76200000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.94499999999999995}]},{"osd":11,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.91000000000000003}]}]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1034035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42999999999999999}]},{"osd":1,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.35799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.60999999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.45700000000000002,"15min":0.439},"min":{"1min":0.39300000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.49199999999999999}]},{"osd":2,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60399999999999998,"15min":0.58499999999999996},"min":{"1min":0.496,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83999999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.42799999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74299999999999999,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55800000000000005}]},{"osd":3,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.43099999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81299999999999994,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65100000000000002,"15min":0.66700000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81200000000000006,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.67900000000000005}]},{"osd":4,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.5},"min":{"1min":0.38600000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69799999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54000000000000004,"15min":0.52200000000000002},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76300000000000001,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60299999999999998}]},{"osd":5,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.63600000000000001,"15min":0.61899999999999999},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78900000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.67900000000000005,"15min":0.65900000000000003},"min":{"1min":0.45700000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.89500000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.748}]},{"osd":6,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.55700000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95799999999999996,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.747,"15min":0.73499999999999999},"min":{"1min":0.47099999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.9340000000000002,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":0.79400000000000004}]},{"osd":8,"last update":"Sat Sep 30 14:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.47999999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.872,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.73199999999999998,"15min":0.70799999999999996},"min":{"1min":0.629,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.84099999999999997}]},{"osd":9,"last update":"Sat Sep 30 14:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77600000000000002,"15min":0.77900000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.57299999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96499999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.82599999999999996}]},{"osd":10,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.77000000000000002,"15min":0.76400000000000001},"min":{"1min":0.64700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.79400000000000004,"15min":0.79600000000000004},"min":{"1min":0.52000000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.65600000000000003}]},{"osd":11,"last update":"Sat Sep 30 14:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.88700000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":442216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38100000000000001,"15min":0.371},"min":{"1min":0.27700000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.40799999999999997,"15min":0.39000000000000001},"min":{"1min":0.28100000000000003,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.26700000000000002}]},{"osd":1,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.75700000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.626},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.48599999999999999}]},{"osd":2,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.628,"15min":0.627},"min":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.75700000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.433}]},{"osd":3,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.502,"15min":0.48599999999999999},"min":{"1min":0.35299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.504},{"interface":"front","average":{"1min":0.504,"5min":0.499,"15min":0.497},"min":{"1min":0.35799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.249}]},{"osd":4,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.46800000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67300000000000004,"15min":0.65100000000000002},"min":{"1min":0.46899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.89600000000000002,"5min":1.056,"15min":1.056},"last":0.58599999999999997}]},{"osd":5,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64600000000000002,"15min":0.64700000000000002},"min":{"1min":0.499,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.48099999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1060000000000001,"5min":1.228,"15min":1.228},"last":0.36699999999999999}]},{"osd":7,"last update":"Sat Sep 30 14:38:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70999999999999996,"15min":0.68799999999999994},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1679999999999999,"5min":1.272,"15min":1.272},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.56399999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.83999999999999997,"5min":1.323,"15min":1.323},"last":0.69799999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.55000000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.81599999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.55100000000000005}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.748,"15min":0.76000000000000001},"min":{"1min":0.45500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72499999999999998,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93200000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.66300000000000003}]},{"osd":10,"last update":"Sat Sep 30 14:38:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.83699999999999997,"15min":0.84499999999999997},"min":{"1min":0.627,"5min":0.62,"15min":0.62},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.82499999999999996,"15min":0.84199999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.89500000000000002,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":501971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55600000000000005,"15min":0.50600000000000001},"min":{"1min":0.36099999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.56999999999999995,"15min":0.52300000000000002},"min":{"1min":0.45200000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.63500000000000001}]},{"osd":2,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.61299999999999999,"15min":0.52900000000000003},"min":{"1min":0.53100000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98499999999999999,"5min":0.998,"15min":0.998},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.59799999999999998,"15min":0.53000000000000003},"min":{"1min":0.39200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.75}]},{"osd":3,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57899999999999996,"15min":0.54300000000000004},"min":{"1min":0.436,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60599999999999998,"15min":0.57499999999999996},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92200000000000004,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.68999999999999995}]},{"osd":4,"last update":"Sat Sep 30 14:38:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.48099999999999998,"15min":0.443},"min":{"1min":0.28999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.503,"15min":0.46300000000000002},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.50800000000000001}]},{"osd":5,"last update":"Sat Sep 30 14:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.82799999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.68200000000000005,"15min":0.628},"min":{"1min":0.57899999999999996,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.80300000000000005}]},{"osd":6,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88300000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69499999999999995,"15min":0.67500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.88900000000000001}]},{"osd":7,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.61399999999999999,"15min":0.55900000000000005},"min":{"1min":0.58099999999999996,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.434,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.70599999999999996}]},{"osd":8,"last update":"Sat Sep 30 14:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.77800000000000002,"15min":0.748},"min":{"1min":0.55200000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.751,"15min":0.71699999999999997},"min":{"1min":0.56000000000000005,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.85399999999999998}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.81799999999999995,"15min":0.81399999999999995},"min":{"1min":0.56799999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.96999999999999997}]},{"osd":10,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.81699999999999995,"15min":0.79500000000000004},"min":{"1min":0.75800000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.78100000000000003,"15min":0.76000000000000001},"min":{"1min":0.70199999999999996,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.90300000000000002}]},{"osd":11,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.93300000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":443481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 14:38:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55600000000000005,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71799999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57099999999999995,"15min":0.53400000000000003},"min":{"1min":0.42799999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.84599999999999997,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56499999999999995}]},{"osd":2,"last update":"Sat Sep 30 14:38:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.47499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.502},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.58999999999999997,"15min":0.56799999999999995},"min":{"1min":0.48999999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90700000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.81499999999999995}]},{"osd":3,"last update":"Sat Sep 30 14:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.45700000000000002}]},{"osd":4,"last update":"Sat Sep 30 14:38:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60299999999999998,"15min":0.58599999999999997},"min":{"1min":0.41299999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82699999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.752},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60799999999999998,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83099999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.73699999999999999}]},{"osd":5,"last update":"Sat Sep 30 14:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59699999999999998,"15min":0.56000000000000005},"min":{"1min":0.496,"5min":0.372,"15min":0.372},"max":{"1min":0.80800000000000005,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.625},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.51300000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89300000000000002,"5min":0.996,"15min":0.996},"last":0.65000000000000002}]},{"osd":6,"last update":"Sat Sep 30 14:37:47 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69199999999999995,"5min":1.03,"15min":1.03},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.317,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80600000000000005,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.33400000000000002}]},{"osd":7,"last update":"Sat Sep 30 14:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69699999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.014,"5min":1.147,"15min":1.147},"last":0.442},{"interface":"front","average":{"1min":0.748,"5min":0.67800000000000005,"15min":0.66600000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56799999999999995}]},{"osd":8,"last update":"Sat Sep 30 14:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69999999999999996,"15min":0.71299999999999997},"min":{"1min":0.52400000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82799999999999996,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.54100000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97199999999999998,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.72299999999999998}]},{"osd":9,"last update":"Sat Sep 30 14:37:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.71999999999999997,"15min":0.73899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80200000000000005,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.78200000000000003,"15min":0.80400000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.79900000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.37}]},{"osd":10,"last update":"Sat Sep 30 14:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.88400000000000001,"5min":1.127,"15min":1.127},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.79300000000000004,"15min":0.81100000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.873,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58599999999999997}]},{"osd":11,"last update":"Sat Sep 30 14:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.75900000000000001,"15min":0.76400000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.90100000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77900000000000003,"15min":0.78000000000000003},"min":{"1min":0.64200000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.77400000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":502135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52000000000000002,"15min":0.51100000000000001},"min":{"1min":0.372,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.83199999999999996,"5min":0.876,"15min":0.876},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.505,"15min":0.496},"min":{"1min":0.34799999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67500000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.51200000000000001}]},{"osd":1,"last update":"Sat Sep 30 14:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56799999999999995,"15min":0.52600000000000002},"min":{"1min":0.40300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.79200000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.626,"15min":0.55100000000000005},"min":{"1min":0.52700000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.69499999999999995}]},{"osd":3,"last update":"Sat Sep 30 14:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.35599999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58999999999999997,"15min":0.56499999999999995},"min":{"1min":0.42699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.57199999999999995}]},{"osd":4,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.67800000000000005}]},{"osd":5,"last update":"Sat Sep 30 14:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.50700000000000001,"15min":0.46899999999999997},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.53000000000000003,"15min":0.49399999999999999},"min":{"1min":0.43099999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.59199999999999997}]},{"osd":6,"last update":"Sat Sep 30 14:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66600000000000004,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1000000000000001,"5min":1.405,"15min":1.405},"last":0.628},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.61199999999999999}]},{"osd":7,"last update":"Sat Sep 30 14:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.45500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.80100000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70699999999999996,"15min":0.71299999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.86399999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.59199999999999997}]},{"osd":8,"last update":"Sat Sep 30 14:38:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.65100000000000002,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.46999999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.78900000000000003}]},{"osd":9,"last update":"Sat Sep 30 14:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.65000000000000002}]},{"osd":10,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.55900000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.69199999999999995,"15min":0.67100000000000004},"min":{"1min":0.56200000000000006,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.76400000000000001}]},{"osd":11,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.746},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.81599999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149759,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":442533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.35799999999999998},"min":{"1min":0.27200000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.432,"15min":0.41199999999999998},"min":{"1min":0.316,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54800000000000004,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.59599999999999997}]},{"osd":1,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64200000000000002,"15min":0.58599999999999997},"min":{"1min":0.442,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.002,"5min":5.798,"15min":5.798},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.71799999999999997}]},{"osd":2,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56699999999999995,"15min":0.53500000000000003},"min":{"1min":0.39100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.70999999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58399999999999996,"15min":0.55100000000000005},"min":{"1min":0.442,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.73199999999999998}]},{"osd":4,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.66100000000000003,"15min":0.58399999999999996},"min":{"1min":0.40999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.873,"5min":7.1619999999999999,"15min":7.1619999999999999},"last":0.624},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.621,"15min":0.57699999999999996},"min":{"1min":0.44800000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.075,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.70199999999999996}]},{"osd":5,"last update":"Sat Sep 30 14:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.42599999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66300000000000003,"15min":0.65400000000000003},"min":{"1min":0.46000000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72399999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.79400000000000004}]},{"osd":7,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.71599999999999997,"15min":0.70499999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.99199999999999999,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81699999999999995,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.74299999999999999}]},{"osd":8,"last update":"Sat Sep 30 14:36:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.46100000000000002}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.69499999999999995,"15min":0.71399999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81299999999999994,"5min":1.161,"15min":1.161},"last":0.82499999999999996}]},{"osd":10,"last update":"Sat Sep 30 14:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":1.0249999999999999,"15min":1.069},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.3640000000000001,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.84699999999999998,"15min":0.86899999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.304,"5min":2.597,"15min":2.597},"last":0.84199999999999997}]},{"osd":11,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.88},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.91600000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920826,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":443296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67100000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.65000000000000002}]},{"osd":1,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.372},"min":{"1min":0.32200000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.53400000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39000000000000001,"15min":0.36799999999999999},"min":{"1min":0.34200000000000003,"5min":0.224,"15min":0.224},"max":{"1min":0.56499999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.38300000000000001}]},{"osd":2,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.58799999999999997,"15min":0.53000000000000003},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.871,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57299999999999995,"15min":0.54900000000000004},"min":{"1min":0.47099999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69099999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.754}]},{"osd":3,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.47499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72299999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.58099999999999996},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90900000000000003,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.44900000000000001}]},{"osd":5,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.621,"15min":0.57499999999999996},"min":{"1min":0.48699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84899999999999998,"5min":1.484,"15min":1.484},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.56100000000000005},"min":{"1min":0.45900000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83999999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66900000000000004}]},{"osd":6,"last update":"Sat Sep 30 14:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.626,"15min":0.626},"min":{"1min":0.501,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78300000000000003,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.439,"5min":0.435,"15min":0.435},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.54400000000000004}]},{"osd":7,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56000000000000005,"15min":0.53900000000000003},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.79800000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61199999999999999,"15min":0.58299999999999996},"min":{"1min":0.47099999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97299999999999998,"5min":1.714,"15min":1.714},"last":0.73699999999999999}]},{"osd":8,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67500000000000004,"15min":0.66000000000000003},"min":{"1min":0.53700000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67600000000000005,"15min":0.66300000000000003},"min":{"1min":0.48099999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.95199999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.83399999999999996}]},{"osd":9,"last update":"Sat Sep 30 14:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.55100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.85299999999999998,"5min":1.038,"15min":1.038},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.58599999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.95199999999999996,"5min":1.677,"15min":1.677},"last":0.79300000000000004}]},{"osd":10,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.72099999999999997},"min":{"1min":0.41799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.92800000000000005,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.747,"15min":0.751},"min":{"1min":0.52100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0009999999999999,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.81999999999999995}]},{"osd":11,"last update":"Sat Sep 30 14:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.875},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.89700000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1536000,"data_stored":1033970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.496},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77800000000000002,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":3.4380000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53300000000000003,"15min":0.51500000000000001},"min":{"1min":0.35399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77100000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.44500000000000001}]},{"osd":1,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.56399999999999995,"15min":0.50800000000000001},"min":{"1min":0.38600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54000000000000004,"15min":0.499},"min":{"1min":0.46100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77600000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56799999999999995}]},{"osd":2,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.443,"15min":0.41899999999999998},"min":{"1min":0.33500000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.42499999999999999},"min":{"1min":0.32400000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47099999999999997}]},{"osd":3,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.60599999999999998,"15min":0.55900000000000005},"min":{"1min":0.376,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.1050000000000004,"5min":4.1050000000000004,"15min":4.1050000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.65400000000000003,"15min":0.60399999999999998},"min":{"1min":0.53200000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.1630000000000003,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":0.51000000000000001}]},{"osd":4,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.59099999999999997,"15min":0.54700000000000004},"min":{"1min":0.52700000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83799999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60799999999999998,"15min":0.57299999999999995},"min":{"1min":0.439,"5min":0.37,"15min":0.37},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.79800000000000004}]},{"osd":6,"last update":"Sat Sep 30 14:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.69499999999999995,"15min":0.66000000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":4.125,"5min":4.125,"15min":4.125},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.1970000000000001,"5min":0.73499999999999999,"15min":0.64700000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":12.24,"5min":12.24,"15min":12.24},"last":3.4660000000000002}]},{"osd":7,"last update":"Sat Sep 30 14:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.66400000000000003,"15min":0.65700000000000003},"min":{"1min":0.52600000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.56499999999999995,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.78500000000000003}]},{"osd":8,"last update":"Sat Sep 30 14:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.42599999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90400000000000003,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.754},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64200000000000002,"15min":0.625},"min":{"1min":0.52300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.87,"5min":0.874,"15min":0.874},"last":0.76900000000000002}]},{"osd":9,"last update":"Sat Sep 30 14:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.78400000000000003,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.503,"15min":0.503},"max":{"1min":5.1520000000000001,"5min":5.1520000000000001,"15min":5.1520000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":0.874,"15min":0.79300000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":12.198,"5min":12.198,"15min":12.198},"last":0.71499999999999997}]},{"osd":10,"last update":"Sat Sep 30 14:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.66900000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.59599999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.81399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":60074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T14:38:40.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T14:38:40.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.356+0000 7fea13410700 1 -- 172.21.15.32:0/1359923776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c100f20 msgr2=0x7fea0c101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:40.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.356+0000 7fea13410700 1 --2- 172.21.15.32:0/1359923776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c100f20 0x7fea0c101380 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0099c0 tx=0x7fe9fc009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.356+0000 7fea13410700 1 -- 172.21.15.32:0/1359923776 shutdown_connections 2023-09-30T14:38:40.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.356+0000 7fea13410700 1 --2- 172.21.15.32:0/1359923776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea0c1018c0 0x7fea0c108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.356+0000 7fea13410700 1 --2- 172.21.15.32:0/1359923776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea0c0ffd30 0x7fea0c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.356+0000 7fea13410700 1 --2- 172.21.15.32:0/1359923776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c100f20 0x7fea0c101380 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.356+0000 7fea13410700 1 -- 172.21.15.32:0/1359923776 >> 172.21.15.32:0/1359923776 conn(0x7fea0c0fb280 msgr2=0x7fea0c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:40.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.356+0000 7fea13410700 1 -- 172.21.15.32:0/1359923776 shutdown_connections 2023-09-30T14:38:40.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.356+0000 7fea13410700 1 -- 172.21.15.32:0/1359923776 wait complete. 2023-09-30T14:38:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.356+0000 7fea13410700 1 Processor -- start 2023-09-30T14:38:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.356+0000 7fea13410700 1 -- start start 2023-09-30T14:38:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.356+0000 7fea13410700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c0ffd30 0x7fea0c195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.356+0000 7fea13410700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea0c100f20 0x7fea0c195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea13410700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea0c1018c0 0x7fea0c19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea13410700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c19d5a0 con 0x7fea0c0ffd30 2023-09-30T14:38:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea13410700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c19d6e0 con 0x7fea0c1018c0 2023-09-30T14:38:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea13410700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c19d870 con 0x7fea0c100f20 2023-09-30T14:38:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea111ac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c0ffd30 0x7fea0c195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea111ac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c0ffd30 0x7fea0c195a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35268/0 (socket says 172.21.15.32:35268) 2023-09-30T14:38:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea111ac700 1 -- 172.21.15.32:0/3561280011 learned_addr learned my addr 172.21.15.32:0/3561280011 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea119ad700 1 --2- 172.21.15.32:0/3561280011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea0c1018c0 0x7fea0c19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea109ab700 1 --2- 172.21.15.32:0/3561280011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea0c100f20 0x7fea0c195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea111ac700 1 -- 172.21.15.32:0/3561280011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea0c100f20 msgr2=0x7fea0c195f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea111ac700 1 --2- 172.21.15.32:0/3561280011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea0c100f20 0x7fea0c195f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea111ac700 1 -- 172.21.15.32:0/3561280011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea0c1018c0 msgr2=0x7fea0c19cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea111ac700 1 --2- 172.21.15.32:0/3561280011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea0c1018c0 0x7fea0c19cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea111ac700 1 -- 172.21.15.32:0/3561280011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc0095f0 con 0x7fea0c0ffd30 2023-09-30T14:38:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea109ab700 1 --2- 172.21.15.32:0/3561280011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea0c100f20 0x7fea0c195f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea111ac700 1 --2- 172.21.15.32:0/3561280011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c0ffd30 0x7fea0c195a20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fea0800e970 tx=0x7fea0800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea027fc700 1 -- 172.21.15.32:0/3561280011 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08009790 con 0x7fea0c0ffd30 2023-09-30T14:38:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea027fc700 1 -- 172.21.15.32:0/3561280011 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea0800eec0 con 0x7fea0c0ffd30 2023-09-30T14:38:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea13410700 1 -- 172.21.15.32:0/3561280011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0c19db50 con 0x7fea0c0ffd30 2023-09-30T14:38:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea027fc700 1 -- 172.21.15.32:0/3561280011 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08003cf0 con 0x7fea0c0ffd30 2023-09-30T14:38:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.357+0000 7fea13410700 1 -- 172.21.15.32:0/3561280011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0c19e060 con 0x7fea0c0ffd30 2023-09-30T14:38:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.358+0000 7fea13410700 1 -- 172.21.15.32:0/3561280011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0c107310 con 0x7fea0c0ffd30 2023-09-30T14:38:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.359+0000 7fea027fc700 1 -- 172.21.15.32:0/3561280011 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fea0800ccd0 con 0x7fea0c0ffd30 2023-09-30T14:38:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.360+0000 7fea027fc700 1 --2- 172.21.15.32:0/3561280011 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe9f80a1070 0x7fe9f80a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.360+0000 7fea109ab700 1 --2- 172.21.15.32:0/3561280011 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe9f80a1070 0x7fe9f80a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.360+0000 7fea027fc700 1 -- 172.21.15.32:0/3561280011 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fea08014070 con 0x7fea0c0ffd30 2023-09-30T14:38:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.361+0000 7fea109ab700 1 --2- 172.21.15.32:0/3561280011 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe9f80a1070 0x7fe9f80a3530 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0098e0 tx=0x7fe9fc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:40.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.365+0000 7fea027fc700 1 -- 172.21.15.32:0/3561280011 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fea080961c0 con 0x7fea0c0ffd30 2023-09-30T14:38:40.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.568+0000 7fea13410700 1 -- 172.21.15.32:0/3561280011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea0c10b590 con 0x7fea0c0ffd30 2023-09-30T14:38:40.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.569+0000 7fea027fc700 1 -- 172.21.15.32:0/3561280011 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22569 (secure 0 0 0) 0x7fea080997e0 con 0x7fea0c0ffd30 2023-09-30T14:38:40.571 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:40.571 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":85,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:38:13.192089+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:38:40.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.572+0000 7fea13410700 1 -- 172.21.15.32:0/3561280011 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe9f80a1070 msgr2=0x7fe9f80a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:40.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.572+0000 7fea13410700 1 --2- 172.21.15.32:0/3561280011 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe9f80a1070 0x7fe9f80a3530 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0098e0 tx=0x7fe9fc019040 comp rx=0 tx=0).stop 2023-09-30T14:38:40.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.572+0000 7fea13410700 1 -- 172.21.15.32:0/3561280011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c0ffd30 msgr2=0x7fea0c195a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:40.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.572+0000 7fea13410700 1 --2- 172.21.15.32:0/3561280011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c0ffd30 0x7fea0c195a20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fea0800e970 tx=0x7fea0800ec80 comp rx=0 tx=0).stop 2023-09-30T14:38:40.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.572+0000 7fea13410700 1 -- 172.21.15.32:0/3561280011 shutdown_connections 2023-09-30T14:38:40.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.572+0000 7fea13410700 1 --2- 172.21.15.32:0/3561280011 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe9f80a1070 0x7fe9f80a3530 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.572+0000 7fea13410700 1 --2- 172.21.15.32:0/3561280011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea0c1018c0 0x7fea0c19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.572+0000 7fea13410700 1 --2- 172.21.15.32:0/3561280011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c0ffd30 0x7fea0c195a20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.572+0000 7fea13410700 1 --2- 172.21.15.32:0/3561280011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea0c100f20 0x7fea0c195f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.572+0000 7fea13410700 1 -- 172.21.15.32:0/3561280011 >> 172.21.15.32:0/3561280011 conn(0x7fea0c0fb280 msgr2=0x7fea0c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:40.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.572+0000 7fea13410700 1 -- 172.21.15.32:0/3561280011 shutdown_connections 2023-09-30T14:38:40.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.572+0000 7fea13410700 1 -- 172.21.15.32:0/3561280011 wait complete. 2023-09-30T14:38:40.585 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T14:33:20.224529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T14:37:09.573219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T14:37:10.844458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T14:38:40.586 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T14:38:40.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.713+0000 7fe377b34700 1 -- 172.21.15.32:0/4015190127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370100f60 msgr2=0x7fe3701013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:40.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.713+0000 7fe377b34700 1 --2- 172.21.15.32:0/4015190127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370100f60 0x7fe3701013c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe3600099c0 tx=0x7fe360009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.714+0000 7fe377b34700 1 -- 172.21.15.32:0/4015190127 shutdown_connections 2023-09-30T14:38:40.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.714+0000 7fe377b34700 1 --2- 172.21.15.32:0/4015190127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe370101900 0x7fe3701081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.714+0000 7fe377b34700 1 --2- 172.21.15.32:0/4015190127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3700ffd70 0x7fe370100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.714+0000 7fe377b34700 1 --2- 172.21.15.32:0/4015190127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370100f60 0x7fe3701013c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.714+0000 7fe377b34700 1 -- 172.21.15.32:0/4015190127 >> 172.21.15.32:0/4015190127 conn(0x7fe3700fb320 msgr2=0x7fe3700fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.714+0000 7fe377b34700 1 -- 172.21.15.32:0/4015190127 shutdown_connections 2023-09-30T14:38:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.714+0000 7fe377b34700 1 -- 172.21.15.32:0/4015190127 wait complete. 2023-09-30T14:38:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.714+0000 7fe377b34700 1 Processor -- start 2023-09-30T14:38:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.714+0000 7fe377b34700 1 -- start start 2023-09-30T14:38:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.714+0000 7fe377b34700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3700ffd70 0x7fe370196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.714+0000 7fe377b34700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe370100f60 0x7fe370197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.714+0000 7fe377b34700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370101900 0x7fe37019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.714+0000 7fe377b34700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37019e6b0 con 0x7fe370101900 2023-09-30T14:38:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.714+0000 7fe377b34700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37019e7f0 con 0x7fe370100f60 2023-09-30T14:38:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.714+0000 7fe377b34700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37019e980 con 0x7fe3700ffd70 2023-09-30T14:38:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe3760d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370101900 0x7fe37019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe3760d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370101900 0x7fe37019e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35286/0 (socket says 172.21.15.32:35286) 2023-09-30T14:38:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe3760d1700 1 -- 172.21.15.32:0/1114302468 learned_addr learned my addr 172.21.15.32:0/1114302468 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe3750cf700 1 --2- 172.21.15.32:0/1114302468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe370100f60 0x7fe370197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe3758d0700 1 --2- 172.21.15.32:0/1114302468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3700ffd70 0x7fe370196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe3760d1700 1 -- 172.21.15.32:0/1114302468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3700ffd70 msgr2=0x7fe370196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe3760d1700 1 --2- 172.21.15.32:0/1114302468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3700ffd70 0x7fe370196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe3760d1700 1 -- 172.21.15.32:0/1114302468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe370100f60 msgr2=0x7fe370197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe3760d1700 1 --2- 172.21.15.32:0/1114302468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe370100f60 0x7fe370197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe3760d1700 1 -- 172.21.15.32:0/1114302468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3600095f0 con 0x7fe370101900 2023-09-30T14:38:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe3750cf700 1 --2- 172.21.15.32:0/1114302468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe370100f60 0x7fe370197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe3760d1700 1 --2- 172.21.15.32:0/1114302468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370101900 0x7fe37019e0e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe368007ee0 tx=0x7fe36800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe366ffd700 1 -- 172.21.15.32:0/1114302468 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368017070 con 0x7fe370101900 2023-09-30T14:38:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe366ffd700 1 -- 172.21.15.32:0/1114302468 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe36800f040 con 0x7fe370101900 2023-09-30T14:38:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe377b34700 1 -- 172.21.15.32:0/1114302468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37019ec60 con 0x7fe370101900 2023-09-30T14:38:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe366ffd700 1 -- 172.21.15.32:0/1114302468 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368005740 con 0x7fe370101900 2023-09-30T14:38:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.715+0000 7fe377b34700 1 -- 172.21.15.32:0/1114302468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37019f170 con 0x7fe370101900 2023-09-30T14:38:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.716+0000 7fe377b34700 1 -- 172.21.15.32:0/1114302468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37010b580 con 0x7fe370101900 2023-09-30T14:38:40.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.717+0000 7fe366ffd700 1 -- 172.21.15.32:0/1114302468 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7fe3680058a0 con 0x7fe370101900 2023-09-30T14:38:40.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.718+0000 7fe366ffd700 1 --2- 172.21.15.32:0/1114302468 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe35c0a10b0 0x7fe35c0a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:40.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.718+0000 7fe366ffd700 1 -- 172.21.15.32:0/1114302468 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe3680cc750 con 0x7fe370101900 2023-09-30T14:38:40.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.718+0000 7fe3758d0700 1 --2- 172.21.15.32:0/1114302468 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe35c0a10b0 0x7fe35c0a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:40.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.718+0000 7fe3758d0700 1 --2- 172.21.15.32:0/1114302468 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe35c0a10b0 0x7fe35c0a3570 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe36c005950 tx=0x7fe36c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:40.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.722+0000 7fe366ffd700 1 -- 172.21.15.32:0/1114302468 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fe368095e00 con 0x7fe370101900 2023-09-30T14:38:40.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:40 smithi156 ceph-mon[103579]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:38:40.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:40 smithi148 ceph-mon[102355]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:38:40.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:40 smithi032 ceph-mon[98592]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:38:40.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.924+0000 7fe377b34700 1 -- 172.21.15.32:0/1114302468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe37019f3c0 con 0x7fe370101900 2023-09-30T14:38:40.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.924+0000 7fe366ffd700 1 -- 172.21.15.32:0/1114302468 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe368099420 con 0x7fe370101900 2023-09-30T14:38:40.925 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T14:38:40.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.926+0000 7fe377b34700 1 -- 172.21.15.32:0/1114302468 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe35c0a10b0 msgr2=0x7fe35c0a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:40.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.926+0000 7fe377b34700 1 --2- 172.21.15.32:0/1114302468 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe35c0a10b0 0x7fe35c0a3570 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe36c005950 tx=0x7fe36c0058e0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.926+0000 7fe377b34700 1 -- 172.21.15.32:0/1114302468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370101900 msgr2=0x7fe37019e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:40.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.926+0000 7fe377b34700 1 --2- 172.21.15.32:0/1114302468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370101900 0x7fe37019e0e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe368007ee0 tx=0x7fe36800d3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.926+0000 7fe377b34700 1 -- 172.21.15.32:0/1114302468 shutdown_connections 2023-09-30T14:38:40.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.926+0000 7fe377b34700 1 --2- 172.21.15.32:0/1114302468 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe35c0a10b0 0x7fe35c0a3570 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.926+0000 7fe377b34700 1 --2- 172.21.15.32:0/1114302468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe370101900 0x7fe37019e0e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.926+0000 7fe377b34700 1 --2- 172.21.15.32:0/1114302468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3700ffd70 0x7fe370196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.926+0000 7fe377b34700 1 --2- 172.21.15.32:0/1114302468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe370100f60 0x7fe370197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:40.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.926+0000 7fe377b34700 1 -- 172.21.15.32:0/1114302468 >> 172.21.15.32:0/1114302468 conn(0x7fe3700fb320 msgr2=0x7fe370068f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:40.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.927+0000 7fe377b34700 1 -- 172.21.15.32:0/1114302468 shutdown_connections 2023-09-30T14:38:40.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:40.927+0000 7fe377b34700 1 -- 172.21.15.32:0/1114302468 wait complete. 2023-09-30T14:38:40.940 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T14:38:41.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.067+0000 7ff8f0586700 1 -- 172.21.15.32:0/623752923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e8100f40 msgr2=0x7ff8e81013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.067+0000 7ff8f0586700 1 --2- 172.21.15.32:0/623752923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e8100f40 0x7ff8e81013a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0099c0 tx=0x7ff8dc009cd0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.068+0000 7ff8f0586700 1 -- 172.21.15.32:0/623752923 shutdown_connections 2023-09-30T14:38:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.068+0000 7ff8f0586700 1 --2- 172.21.15.32:0/623752923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8e81018e0 0x7ff8e8108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.068+0000 7ff8f0586700 1 --2- 172.21.15.32:0/623752923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8e80ffd50 0x7ff8e8100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.068+0000 7ff8f0586700 1 --2- 172.21.15.32:0/623752923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e8100f40 0x7ff8e81013a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.068+0000 7ff8f0586700 1 -- 172.21.15.32:0/623752923 >> 172.21.15.32:0/623752923 conn(0x7ff8e80fb2a0 msgr2=0x7ff8e80fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.068+0000 7ff8f0586700 1 -- 172.21.15.32:0/623752923 shutdown_connections 2023-09-30T14:38:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.068+0000 7ff8f0586700 1 -- 172.21.15.32:0/623752923 wait complete. 2023-09-30T14:38:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.068+0000 7ff8f0586700 1 Processor -- start 2023-09-30T14:38:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.068+0000 7ff8f0586700 1 -- start start 2023-09-30T14:38:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.068+0000 7ff8f0586700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8e80ffd50 0x7ff8e8069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.068+0000 7ff8f0586700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8e8100f40 0x7ff8e819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8f0586700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e81018e0 0x7ff8e819dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8f0586700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e819e690 con 0x7ff8e81018e0 2023-09-30T14:38:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8f0586700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e819e800 con 0x7ff8e8100f40 2023-09-30T14:38:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8f0586700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e819e9c0 con 0x7ff8e80ffd50 2023-09-30T14:38:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8eeb23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e81018e0 0x7ff8e819dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8eeb23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e81018e0 0x7ff8e819dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35298/0 (socket says 172.21.15.32:35298) 2023-09-30T14:38:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8ee322700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8e80ffd50 0x7ff8e8069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8eeb23700 1 -- 172.21.15.32:0/1808821426 learned_addr learned my addr 172.21.15.32:0/1808821426 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8edb21700 1 --2- 172.21.15.32:0/1808821426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8e8100f40 0x7ff8e819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8eeb23700 1 -- 172.21.15.32:0/1808821426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8e80ffd50 msgr2=0x7ff8e8069b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8eeb23700 1 --2- 172.21.15.32:0/1808821426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8e80ffd50 0x7ff8e8069b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8eeb23700 1 -- 172.21.15.32:0/1808821426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8e8100f40 msgr2=0x7ff8e819b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8eeb23700 1 --2- 172.21.15.32:0/1808821426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8e8100f40 0x7ff8e819b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8eeb23700 1 -- 172.21.15.32:0/1808821426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc0095f0 con 0x7ff8e81018e0 2023-09-30T14:38:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8edb21700 1 --2- 172.21.15.32:0/1808821426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8e8100f40 0x7ff8e819b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8eeb23700 1 --2- 172.21.15.32:0/1808821426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e81018e0 0x7ff8e819dec0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0007d70 tx=0x7ff8e000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8db7fe700 1 -- 172.21.15.32:0/1808821426 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000ad40 con 0x7ff8e81018e0 2023-09-30T14:38:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8db7fe700 1 -- 172.21.15.32:0/1808821426 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8e00128e0 con 0x7ff8e81018e0 2023-09-30T14:38:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8f0586700 1 -- 172.21.15.32:0/1808821426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e819eca0 con 0x7ff8e81018e0 2023-09-30T14:38:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8db7fe700 1 -- 172.21.15.32:0/1808821426 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00116b0 con 0x7ff8e81018e0 2023-09-30T14:38:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.069+0000 7ff8f0586700 1 -- 172.21.15.32:0/1808821426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e819f1d0 con 0x7ff8e81018e0 2023-09-30T14:38:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.070+0000 7ff8f0586700 1 -- 172.21.15.32:0/1808821426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e8107330 con 0x7ff8e81018e0 2023-09-30T14:38:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.071+0000 7ff8db7fe700 1 -- 172.21.15.32:0/1808821426 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7ff8e0012420 con 0x7ff8e81018e0 2023-09-30T14:38:41.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.072+0000 7ff8db7fe700 1 --2- 172.21.15.32:0/1808821426 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff8d40a10b0 0x7ff8d40a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:41.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.072+0000 7ff8ee322700 1 --2- 172.21.15.32:0/1808821426 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff8d40a10b0 0x7ff8d40a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:41.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.072+0000 7ff8db7fe700 1 -- 172.21.15.32:0/1808821426 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff8e00cc960 con 0x7ff8e81018e0 2023-09-30T14:38:41.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.072+0000 7ff8ee322700 1 --2- 172.21.15.32:0/1808821426 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff8d40a10b0 0x7ff8d40a3570 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8100da0 tx=0x7ff8e400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.076+0000 7ff8db7fe700 1 -- 172.21.15.32:0/1808821426 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7ff8e0096120 con 0x7ff8e81018e0 2023-09-30T14:38:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.279+0000 7ff8f0586700 1 -- 172.21.15.32:0/1808821426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8e810b820 con 0x7ff8e81018e0 2023-09-30T14:38:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.279+0000 7ff8db7fe700 1 -- 172.21.15.32:0/1808821426 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff8e0099740 con 0x7ff8e81018e0 2023-09-30T14:38:41.282 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-09-30T14:38:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.281+0000 7ff8f0586700 1 -- 172.21.15.32:0/1808821426 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff8d40a10b0 msgr2=0x7ff8d40a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.281+0000 7ff8f0586700 1 --2- 172.21.15.32:0/1808821426 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff8d40a10b0 0x7ff8d40a3570 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8100da0 tx=0x7ff8e400b410 comp rx=0 tx=0).stop 2023-09-30T14:38:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.282+0000 7ff8f0586700 1 -- 172.21.15.32:0/1808821426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e81018e0 msgr2=0x7ff8e819dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.282+0000 7ff8f0586700 1 --2- 172.21.15.32:0/1808821426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e81018e0 0x7ff8e819dec0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0007d70 tx=0x7ff8e000a340 comp rx=0 tx=0).stop 2023-09-30T14:38:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.282+0000 7ff8f0586700 1 -- 172.21.15.32:0/1808821426 shutdown_connections 2023-09-30T14:38:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.282+0000 7ff8f0586700 1 --2- 172.21.15.32:0/1808821426 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff8d40a10b0 0x7ff8d40a3570 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.282+0000 7ff8f0586700 1 --2- 172.21.15.32:0/1808821426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8e81018e0 0x7ff8e819dec0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.282+0000 7ff8f0586700 1 --2- 172.21.15.32:0/1808821426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8e80ffd50 0x7ff8e8069b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.282+0000 7ff8f0586700 1 --2- 172.21.15.32:0/1808821426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8e8100f40 0x7ff8e819b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.282+0000 7ff8f0586700 1 -- 172.21.15.32:0/1808821426 >> 172.21.15.32:0/1808821426 conn(0x7ff8e80fb2a0 msgr2=0x7ff8e8104470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.282+0000 7ff8f0586700 1 -- 172.21.15.32:0/1808821426 shutdown_connections 2023-09-30T14:38:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.282+0000 7ff8f0586700 1 -- 172.21.15.32:0/1808821426 wait complete. 2023-09-30T14:38:41.295 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T14:38:41.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.423+0000 7f5c7c828700 1 -- 172.21.15.32:0/1599121244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c78102970 msgr2=0x7f5c78109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.423+0000 7f5c7c828700 1 --2- 172.21.15.32:0/1599121244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c78102970 0x7f5c78109290 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000b0a0 tx=0x7f5c7000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.423+0000 7f5c7c828700 1 -- 172.21.15.32:0/1599121244 shutdown_connections 2023-09-30T14:38:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.423+0000 7f5c7c828700 1 --2- 172.21.15.32:0/1599121244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c78102970 0x7f5c78109290 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.423+0000 7f5c7c828700 1 --2- 172.21.15.32:0/1599121244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c78100e60 0x7f5c78101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.423+0000 7f5c7c828700 1 --2- 172.21.15.32:0/1599121244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c78101fb0 0x7f5c78102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.423+0000 7f5c7c828700 1 -- 172.21.15.32:0/1599121244 >> 172.21.15.32:0/1599121244 conn(0x7f5c780fc3d0 msgr2=0x7f5c780fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.423+0000 7f5c7c828700 1 -- 172.21.15.32:0/1599121244 shutdown_connections 2023-09-30T14:38:41.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.423+0000 7f5c7c828700 1 -- 172.21.15.32:0/1599121244 wait complete. 2023-09-30T14:38:41.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.424+0000 7f5c7c828700 1 Processor -- start 2023-09-30T14:38:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.424+0000 7f5c7c828700 1 -- start start 2023-09-30T14:38:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.424+0000 7f5c7c828700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c78100e60 0x7f5c78074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.424+0000 7f5c7c828700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c78101fb0 0x7f5c78073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.424+0000 7f5c7c828700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c78102970 0x7f5c78073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.424+0000 7f5c7c828700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78073ef0 con 0x7f5c78101fb0 2023-09-30T14:38:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.424+0000 7f5c7c828700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78074060 con 0x7f5c78102970 2023-09-30T14:38:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.424+0000 7f5c7c828700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c781a0c70 con 0x7f5c78100e60 2023-09-30T14:38:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c75d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c78101fb0 0x7f5c78073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c75d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c78101fb0 0x7f5c78073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35310/0 (socket says 172.21.15.32:35310) 2023-09-30T14:38:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c75d9b700 1 -- 172.21.15.32:0/2698072613 learned_addr learned my addr 172.21.15.32:0/2698072613 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:41.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c7659c700 1 --2- 172.21.15.32:0/2698072613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c78100e60 0x7f5c78074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c76d9d700 1 --2- 172.21.15.32:0/2698072613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c78102970 0x7f5c78073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c75d9b700 1 -- 172.21.15.32:0/2698072613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c78100e60 msgr2=0x7f5c78074bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c75d9b700 1 --2- 172.21.15.32:0/2698072613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c78100e60 0x7f5c78074bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c75d9b700 1 -- 172.21.15.32:0/2698072613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c78102970 msgr2=0x7f5c78073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c75d9b700 1 --2- 172.21.15.32:0/2698072613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c78102970 0x7f5c78073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c75d9b700 1 -- 172.21.15.32:0/2698072613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c70009d00 con 0x7f5c78101fb0 2023-09-30T14:38:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c76d9d700 1 --2- 172.21.15.32:0/2698072613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c78102970 0x7f5c78073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c75d9b700 1 --2- 172.21.15.32:0/2698072613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c78101fb0 0x7f5c78073210 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000d950 tx=0x7f5c6000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c6f7fe700 1 -- 172.21.15.32:0/2698072613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60009940 con 0x7f5c78101fb0 2023-09-30T14:38:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c6f7fe700 1 -- 172.21.15.32:0/2698072613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c60010460 con 0x7f5c78101fb0 2023-09-30T14:38:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c6f7fe700 1 -- 172.21.15.32:0/2698072613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6000f620 con 0x7f5c78101fb0 2023-09-30T14:38:41.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c7c828700 1 -- 172.21.15.32:0/2698072613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c781a0f50 con 0x7f5c78101fb0 2023-09-30T14:38:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.425+0000 7f5c7c828700 1 -- 172.21.15.32:0/2698072613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c781a1380 con 0x7f5c78101fb0 2023-09-30T14:38:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.426+0000 7f5c7c828700 1 -- 172.21.15.32:0/2698072613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7810c660 con 0x7f5c78101fb0 2023-09-30T14:38:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.427+0000 7f5c6f7fe700 1 -- 172.21.15.32:0/2698072613 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f5c600108d0 con 0x7f5c78101fb0 2023-09-30T14:38:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.428+0000 7f5c6f7fe700 1 --2- 172.21.15.32:0/2698072613 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5c640a1030 0x7f5c640a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:41.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.428+0000 7f5c7659c700 1 --2- 172.21.15.32:0/2698072613 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5c640a1030 0x7f5c640a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:41.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.428+0000 7f5c6f7fe700 1 -- 172.21.15.32:0/2698072613 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5c600cc6c0 con 0x7f5c78101fb0 2023-09-30T14:38:41.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.429+0000 7f5c7659c700 1 --2- 172.21.15.32:0/2698072613 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5c640a1030 0x7f5c640a34f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009c60 tx=0x7f5c68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:41.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.432+0000 7f5c6f7fe700 1 -- 172.21.15.32:0/2698072613 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f5c60095d70 con 0x7f5c78101fb0 2023-09-30T14:38:41.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:41 smithi032 ceph-mon[98592]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:38:41.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:41 smithi032 ceph-mon[98592]: pgmap v299: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-09-30T14:38:41.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:41 smithi032 ceph-mon[98592]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:38:41.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:41 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3561280011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:41.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:41 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1114302468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:41.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:41 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1808821426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:41.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.636+0000 7f5c7c828700 1 -- 172.21.15.32:0/2698072613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c78107d30 con 0x7f5c78101fb0 2023-09-30T14:38:41.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.637+0000 7f5c6f7fe700 1 -- 172.21.15.32:0/2698072613 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c60099390 con 0x7f5c78101fb0 2023-09-30T14:38:41.638 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-09-30T14:38:41.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.639+0000 7f5c7c828700 1 -- 172.21.15.32:0/2698072613 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5c640a1030 msgr2=0x7f5c640a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:41.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.639+0000 7f5c7c828700 1 --2- 172.21.15.32:0/2698072613 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5c640a1030 0x7f5c640a34f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009c60 tx=0x7f5c68009260 comp rx=0 tx=0).stop 2023-09-30T14:38:41.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.639+0000 7f5c7c828700 1 -- 172.21.15.32:0/2698072613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c78101fb0 msgr2=0x7f5c78073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:41.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.639+0000 7f5c7c828700 1 --2- 172.21.15.32:0/2698072613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c78101fb0 0x7f5c78073210 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000d950 tx=0x7f5c6000dc60 comp rx=0 tx=0).stop 2023-09-30T14:38:41.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.639+0000 7f5c7c828700 1 -- 172.21.15.32:0/2698072613 shutdown_connections 2023-09-30T14:38:41.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.639+0000 7f5c7c828700 1 --2- 172.21.15.32:0/2698072613 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5c640a1030 0x7f5c640a34f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.639+0000 7f5c7c828700 1 --2- 172.21.15.32:0/2698072613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c78102970 0x7f5c78073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.639+0000 7f5c7c828700 1 --2- 172.21.15.32:0/2698072613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c78100e60 0x7f5c78074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.639+0000 7f5c7c828700 1 --2- 172.21.15.32:0/2698072613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c78101fb0 0x7f5c78073210 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.639+0000 7f5c7c828700 1 -- 172.21.15.32:0/2698072613 >> 172.21.15.32:0/2698072613 conn(0x7f5c780fc3d0 msgr2=0x7f5c780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:41.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.639+0000 7f5c7c828700 1 -- 172.21.15.32:0/2698072613 shutdown_connections 2023-09-30T14:38:41.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.639+0000 7f5c7c828700 1 -- 172.21.15.32:0/2698072613 wait complete. 2023-09-30T14:38:41.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:38:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.777+0000 7f2dd3b3c700 1 -- 172.21.15.32:0/974559428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dcc100e40 msgr2=0x7f2dcc101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.777+0000 7f2dd3b3c700 1 --2- 172.21.15.32:0/974559428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dcc100e40 0x7f2dcc101260 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009990 tx=0x7f2dc0009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.777+0000 7f2dd3b3c700 1 -- 172.21.15.32:0/974559428 shutdown_connections 2023-09-30T14:38:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.777+0000 7f2dd3b3c700 1 --2- 172.21.15.32:0/974559428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dcc102950 0x7f2dcc109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.777+0000 7f2dd3b3c700 1 --2- 172.21.15.32:0/974559428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dcc100e40 0x7f2dcc101260 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.777+0000 7f2dd3b3c700 1 --2- 172.21.15.32:0/974559428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dcc101f90 0x7f2dcc102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.777+0000 7f2dd3b3c700 1 -- 172.21.15.32:0/974559428 >> 172.21.15.32:0/974559428 conn(0x7f2dcc0fc390 msgr2=0x7f2dcc0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.777+0000 7f2dd3b3c700 1 -- 172.21.15.32:0/974559428 shutdown_connections 2023-09-30T14:38:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.777+0000 7f2dd3b3c700 1 -- 172.21.15.32:0/974559428 wait complete. 2023-09-30T14:38:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd3b3c700 1 Processor -- start 2023-09-30T14:38:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd3b3c700 1 -- start start 2023-09-30T14:38:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd3b3c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dcc100e40 0x7f2dcc196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd3b3c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dcc101f90 0x7f2dcc197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd3b3c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dcc102950 0x7f2dcc19e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd3b3c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc19e660 con 0x7f2dcc100e40 2023-09-30T14:38:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd3b3c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc19e7a0 con 0x7f2dcc101f90 2023-09-30T14:38:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd3b3c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc19e930 con 0x7f2dcc102950 2023-09-30T14:38:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd18d8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dcc100e40 0x7f2dcc196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd18d8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dcc100e40 0x7f2dcc196ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35322/0 (socket says 172.21.15.32:35322) 2023-09-30T14:38:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd18d8700 1 -- 172.21.15.32:0/650494351 learned_addr learned my addr 172.21.15.32:0/650494351 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd10d7700 1 --2- 172.21.15.32:0/650494351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dcc101f90 0x7f2dcc197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd20d9700 1 --2- 172.21.15.32:0/650494351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dcc102950 0x7f2dcc19e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd18d8700 1 -- 172.21.15.32:0/650494351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dcc102950 msgr2=0x7f2dcc19e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd18d8700 1 --2- 172.21.15.32:0/650494351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dcc102950 0x7f2dcc19e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd18d8700 1 -- 172.21.15.32:0/650494351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dcc101f90 msgr2=0x7f2dcc197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd18d8700 1 --2- 172.21.15.32:0/650494351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dcc101f90 0x7f2dcc197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd18d8700 1 -- 172.21.15.32:0/650494351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc00095f0 con 0x7f2dcc100e40 2023-09-30T14:38:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.778+0000 7f2dd20d9700 1 --2- 172.21.15.32:0/650494351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dcc102950 0x7f2dcc19e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.779+0000 7f2dd18d8700 1 --2- 172.21.15.32:0/650494351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dcc100e40 0x7f2dcc196ae0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0005770 tx=0x7f2dc00057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.779+0000 7f2dbeffd700 1 -- 172.21.15.32:0/650494351 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc001d070 con 0x7f2dcc100e40 2023-09-30T14:38:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.779+0000 7f2dbeffd700 1 -- 172.21.15.32:0/650494351 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dc0004060 con 0x7f2dcc100e40 2023-09-30T14:38:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.779+0000 7f2dd3b3c700 1 -- 172.21.15.32:0/650494351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dcc073050 con 0x7f2dcc100e40 2023-09-30T14:38:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.779+0000 7f2dbeffd700 1 -- 172.21.15.32:0/650494351 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0022d70 con 0x7f2dcc100e40 2023-09-30T14:38:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.779+0000 7f2dd3b3c700 1 -- 172.21.15.32:0/650494351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc073500 con 0x7f2dcc100e40 2023-09-30T14:38:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.780+0000 7f2dd3b3c700 1 -- 172.21.15.32:0/650494351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dcc108420 con 0x7f2dcc100e40 2023-09-30T14:38:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.781+0000 7f2dbeffd700 1 -- 172.21.15.32:0/650494351 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f2dc0022470 con 0x7f2dcc100e40 2023-09-30T14:38:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.781+0000 7f2dbeffd700 1 --2- 172.21.15.32:0/650494351 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2db80a1070 0x7f2db80a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.782+0000 7f2dbeffd700 1 -- 172.21.15.32:0/650494351 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2dc00ce440 con 0x7f2dcc100e40 2023-09-30T14:38:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.782+0000 7f2dd10d7700 1 --2- 172.21.15.32:0/650494351 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2db80a1070 0x7f2db80a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.782+0000 7f2dd10d7700 1 --2- 172.21.15.32:0/650494351 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2db80a1070 0x7f2db80a3530 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8005950 tx=0x7f2dc800f820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:41.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:41.804+0000 7f2dbeffd700 1 -- 172.21.15.32:0/650494351 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f2dc0097b80 con 0x7f2dcc100e40 2023-09-30T14:38:41.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:41 smithi156 ceph-mon[103579]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:38:41.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:41 smithi156 ceph-mon[103579]: pgmap v299: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-09-30T14:38:41.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:41 smithi156 ceph-mon[103579]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:38:41.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:41 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3561280011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:41.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:41 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1114302468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:41.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:41 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1808821426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:41 smithi148 ceph-mon[102355]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:38:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:41 smithi148 ceph-mon[102355]: pgmap v299: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-09-30T14:38:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:41 smithi148 ceph-mon[102355]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-30T14:38:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:41 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3561280011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:41 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1114302468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T14:38:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:41 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1808821426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T14:38:42.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.068+0000 7f2dd3b3c700 1 -- 172.21.15.32:0/650494351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2dcc10c890 con 0x7f2dcc100e40 2023-09-30T14:38:42.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.069+0000 7f2dbeffd700 1 -- 172.21.15.32:0/650494351 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11321 (secure 0 0 0) 0x7f2dc009b1a0 con 0x7f2dcc100e40 2023-09-30T14:38:42.069 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:42.070 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3505064841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3505064841},{"type":"v1","addr":"172.21.15.32:6839","nonce":3505064841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2262846038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2262846038},{"type":"v1","addr":"172.21.15.32:6841","nonce":2262846038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-30T14:37:11.444993+0000","modified":"2023-09-30T14:37:16.971534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6839/1445176203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1445176203},{"type":"v1","addr":"172.21.15.156:6839","nonce":1445176203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/494821686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":494821686},{"type":"v1","addr":"172.21.15.148:6839","nonce":494821686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T14:38:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.071+0000 7f2dd3b3c700 1 -- 172.21.15.32:0/650494351 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2db80a1070 msgr2=0x7f2db80a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.071+0000 7f2dd3b3c700 1 --2- 172.21.15.32:0/650494351 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2db80a1070 0x7f2db80a3530 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8005950 tx=0x7f2dc800f820 comp rx=0 tx=0).stop 2023-09-30T14:38:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.071+0000 7f2dd3b3c700 1 -- 172.21.15.32:0/650494351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dcc100e40 msgr2=0x7f2dcc196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.071+0000 7f2dd3b3c700 1 --2- 172.21.15.32:0/650494351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dcc100e40 0x7f2dcc196ae0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0005770 tx=0x7f2dc00057a0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.071+0000 7f2dd3b3c700 1 -- 172.21.15.32:0/650494351 shutdown_connections 2023-09-30T14:38:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.071+0000 7f2dd3b3c700 1 --2- 172.21.15.32:0/650494351 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2db80a1070 0x7f2db80a3530 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.071+0000 7f2dd3b3c700 1 --2- 172.21.15.32:0/650494351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2dcc102950 0x7f2dcc19e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.071+0000 7f2dd3b3c700 1 --2- 172.21.15.32:0/650494351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dcc100e40 0x7f2dcc196ae0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.071+0000 7f2dd3b3c700 1 --2- 172.21.15.32:0/650494351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dcc101f90 0x7f2dcc197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.071+0000 7f2dd3b3c700 1 -- 172.21.15.32:0/650494351 >> 172.21.15.32:0/650494351 conn(0x7f2dcc0fc390 msgr2=0x7f2dcc105560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.071+0000 7f2dd3b3c700 1 -- 172.21.15.32:0/650494351 shutdown_connections 2023-09-30T14:38:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.071+0000 7f2dd3b3c700 1 -- 172.21.15.32:0/650494351 wait complete. 2023-09-30T14:38:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 22 2023-09-30T14:38:42.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T14:38:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.209+0000 7f602b7ca700 1 -- 172.21.15.32:0/1966990075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60241018c0 msgr2=0x7f6024108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.209+0000 7f602b7ca700 1 --2- 172.21.15.32:0/1966990075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60241018c0 0x7f6024108170 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f601c00b0d0 tx=0x7f601c00b3e0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.209+0000 7f602b7ca700 1 -- 172.21.15.32:0/1966990075 shutdown_connections 2023-09-30T14:38:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.209+0000 7f602b7ca700 1 --2- 172.21.15.32:0/1966990075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60241018c0 0x7f6024108170 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.209+0000 7f602b7ca700 1 --2- 172.21.15.32:0/1966990075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60240ffd30 0x7f6024100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.209+0000 7f602b7ca700 1 --2- 172.21.15.32:0/1966990075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6024100f20 0x7f6024101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.209+0000 7f602b7ca700 1 -- 172.21.15.32:0/1966990075 >> 172.21.15.32:0/1966990075 conn(0x7f60240fb280 msgr2=0x7f60240fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.209+0000 7f602b7ca700 1 -- 172.21.15.32:0/1966990075 shutdown_connections 2023-09-30T14:38:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.209+0000 7f602b7ca700 1 -- 172.21.15.32:0/1966990075 wait complete. 2023-09-30T14:38:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.210+0000 7f602b7ca700 1 Processor -- start 2023-09-30T14:38:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.210+0000 7f602b7ca700 1 -- start start 2023-09-30T14:38:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.210+0000 7f602b7ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60240ffd30 0x7f6024195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.210+0000 7f602b7ca700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6024100f20 0x7f6024195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.210+0000 7f602b7ca700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60241018c0 0x7f602419d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.210+0000 7f602b7ca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602419d620 con 0x7f60240ffd30 2023-09-30T14:38:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.210+0000 7f602b7ca700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602419d760 con 0x7f60241018c0 2023-09-30T14:38:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.210+0000 7f602b7ca700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602419d8f0 con 0x7f6024100f20 2023-09-30T14:38:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.210+0000 7f6029566700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60240ffd30 0x7f6024195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.210+0000 7f6029566700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60240ffd30 0x7f6024195aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35336/0 (socket says 172.21.15.32:35336) 2023-09-30T14:38:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.210+0000 7f6029566700 1 -- 172.21.15.32:0/3857595877 learned_addr learned my addr 172.21.15.32:0/3857595877 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.210+0000 7f6028d65700 1 --2- 172.21.15.32:0/3857595877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6024100f20 0x7f6024195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.210+0000 7f6029d67700 1 --2- 172.21.15.32:0/3857595877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60241018c0 0x7f602419d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.211+0000 7f6029566700 1 -- 172.21.15.32:0/3857595877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6024100f20 msgr2=0x7f6024195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.211+0000 7f6029566700 1 --2- 172.21.15.32:0/3857595877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6024100f20 0x7f6024195fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.211+0000 7f6029566700 1 -- 172.21.15.32:0/3857595877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60241018c0 msgr2=0x7f602419d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.211+0000 7f6029566700 1 --2- 172.21.15.32:0/3857595877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60241018c0 0x7f602419d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.211+0000 7f6029566700 1 -- 172.21.15.32:0/3857595877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c009d00 con 0x7f60240ffd30 2023-09-30T14:38:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.211+0000 7f6029d67700 1 --2- 172.21.15.32:0/3857595877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60241018c0 0x7f602419d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.211+0000 7f6029566700 1 --2- 172.21.15.32:0/3857595877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60240ffd30 0x7f6024195aa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f601800c8f0 tx=0x7f601800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.211+0000 7f60167fc700 1 -- 172.21.15.32:0/3857595877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6018004220 con 0x7f60240ffd30 2023-09-30T14:38:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.211+0000 7f60167fc700 1 -- 172.21.15.32:0/3857595877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6018004380 con 0x7f60240ffd30 2023-09-30T14:38:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.211+0000 7f602b7ca700 1 -- 172.21.15.32:0/3857595877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602419dbd0 con 0x7f60240ffd30 2023-09-30T14:38:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.211+0000 7f60167fc700 1 -- 172.21.15.32:0/3857595877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6018003c40 con 0x7f60240ffd30 2023-09-30T14:38:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.211+0000 7f602b7ca700 1 -- 172.21.15.32:0/3857595877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602419e090 con 0x7f60240ffd30 2023-09-30T14:38:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.212+0000 7f602b7ca700 1 -- 172.21.15.32:0/3857595877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f602410b570 con 0x7f60240ffd30 2023-09-30T14:38:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.214+0000 7f60167fc700 1 -- 172.21.15.32:0/3857595877 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 135793+0+0 (secure 0 0 0) 0x7f6018007740 con 0x7f60240ffd30 2023-09-30T14:38:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.214+0000 7f60167fc700 1 --2- 172.21.15.32:0/3857595877 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f60100a1070 0x7f60100a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.214+0000 7f60167fc700 1 -- 172.21.15.32:0/3857595877 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f60180cc890 con 0x7f60240ffd30 2023-09-30T14:38:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.214+0000 7f6028d65700 1 --2- 172.21.15.32:0/3857595877 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f60100a1070 0x7f60100a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.215+0000 7f6028d65700 1 --2- 172.21.15.32:0/3857595877 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f60100a1070 0x7f60100a3530 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6020005950 tx=0x7f602000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:42.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.218+0000 7f60167fc700 1 -- 172.21.15.32:0/3857595877 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f6018095f40 con 0x7f60240ffd30 2023-09-30T14:38:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.420+0000 7f602b7ca700 1 -- 172.21.15.32:0/3857595877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f602410b7b0 con 0x7f60240ffd30 2023-09-30T14:38:42.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.422+0000 7f60167fc700 1 -- 172.21.15.32:0/3857595877 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22569 (secure 0 0 0) 0x7f6018099560 con 0x7f60240ffd30 2023-09-30T14:38:42.423 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:38:42.423 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":85,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:38:13.192089+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T14:38:42.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.423+0000 7f602b7ca700 1 -- 172.21.15.32:0/3857595877 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f60100a1070 msgr2=0x7f60100a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.424+0000 7f602b7ca700 1 --2- 172.21.15.32:0/3857595877 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f60100a1070 0x7f60100a3530 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6020005950 tx=0x7f602000b410 comp rx=0 tx=0).stop 2023-09-30T14:38:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.424+0000 7f602b7ca700 1 -- 172.21.15.32:0/3857595877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60240ffd30 msgr2=0x7f6024195aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.424+0000 7f602b7ca700 1 --2- 172.21.15.32:0/3857595877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60240ffd30 0x7f6024195aa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f601800c8f0 tx=0x7f601800cc00 comp rx=0 tx=0).stop 2023-09-30T14:38:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.424+0000 7f602b7ca700 1 -- 172.21.15.32:0/3857595877 shutdown_connections 2023-09-30T14:38:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.424+0000 7f602b7ca700 1 --2- 172.21.15.32:0/3857595877 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f60100a1070 0x7f60100a3530 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.424+0000 7f602b7ca700 1 --2- 172.21.15.32:0/3857595877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60241018c0 0x7f602419d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.424+0000 7f602b7ca700 1 --2- 172.21.15.32:0/3857595877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60240ffd30 0x7f6024195aa0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.424+0000 7f602b7ca700 1 --2- 172.21.15.32:0/3857595877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6024100f20 0x7f6024195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.424+0000 7f602b7ca700 1 -- 172.21.15.32:0/3857595877 >> 172.21.15.32:0/3857595877 conn(0x7f60240fb280 msgr2=0x7f60240fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.424+0000 7f602b7ca700 1 -- 172.21.15.32:0/3857595877 shutdown_connections 2023-09-30T14:38:42.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.424+0000 7f602b7ca700 1 -- 172.21.15.32:0/3857595877 wait complete. 2023-09-30T14:38:42.437 DEBUG:tasks.fwd_scrub:Yielding 2023-09-30T14:38:42.438 INFO:teuthology.run_tasks:Running task background_exec... 2023-09-30T14:38:42.448 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T14:38:42.448 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-09-30T14:38:42.448 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-09-30T14:38:42.449 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-30T14:38:42.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-30T14:38:42.450 INFO:teuthology.run_tasks:Running task workunit... 2023-09-30T14:38:42.462 INFO:tasks.workunit:Pulling workunits from ref 1d10b71792f3be8887a7631e69851ac2df3585af 2023-09-30T14:38:42.462 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-09-30T14:38:42.462 INFO:tasks.workunit:timeout=3h 2023-09-30T14:38:42.462 INFO:tasks.workunit:cleanup=True 2023-09-30T14:38:42.463 DEBUG:teuthology.orchestra.run.smithi032:> stat -- /home/ubuntu/cephtest/mnt.0 2023-09-30T14:38:42.482 INFO:teuthology.orchestra.run.smithi032.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-09-30T14:38:42.482 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-09-30T14:38:42.482 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2023-09-30T14:38:42.482 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T14:38:42.482 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:cephfs_t:s0 2023-09-30T14:38:42.482 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-09-30 14:37:18.209261991 +0000 2023-09-30T14:38:42.482 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-09-30 14:37:18.209261991 +0000 2023-09-30T14:38:42.483 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-09-30 14:38:35.449689965 +0000 2023-09-30T14:38:42.483 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2023-09-30 14:37:18.209261991 +0000 2023-09-30T14:38:42.483 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-09-30T14:38:42.483 DEBUG:teuthology.orchestra.run.smithi032:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-09-30T14:38:42.556 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1d10b71792f3be8887a7631e69851ac2df3585af 2023-09-30T14:38:42.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.586+0000 7fdf9dfe6700 1 -- 172.21.15.32:0/1668386460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf98104200 msgr2=0x7fdf981065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.586+0000 7fdf9dfe6700 1 --2- 172.21.15.32:0/1668386460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf98104200 0x7fdf981065f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009990 tx=0x7fdf80009ca0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.587+0000 7fdf9dfe6700 1 -- 172.21.15.32:0/1668386460 shutdown_connections 2023-09-30T14:38:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.587+0000 7fdf9dfe6700 1 --2- 172.21.15.32:0/1668386460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf98106b30 0x7fdf98108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.587+0000 7fdf9dfe6700 1 --2- 172.21.15.32:0/1668386460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf981018d0 0x7fdf98103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.587+0000 7fdf9dfe6700 1 --2- 172.21.15.32:0/1668386460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf98104200 0x7fdf981065f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.587+0000 7fdf9dfe6700 1 -- 172.21.15.32:0/1668386460 >> 172.21.15.32:0/1668386460 conn(0x7fdf980fb2a0 msgr2=0x7fdf980fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.587+0000 7fdf9dfe6700 1 -- 172.21.15.32:0/1668386460 shutdown_connections 2023-09-30T14:38:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.587+0000 7fdf9dfe6700 1 -- 172.21.15.32:0/1668386460 wait complete. 2023-09-30T14:38:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.587+0000 7fdf9dfe6700 1 Processor -- start 2023-09-30T14:38:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.587+0000 7fdf9dfe6700 1 -- start start 2023-09-30T14:38:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.587+0000 7fdf9dfe6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf981018d0 0x7fdf98101240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.587+0000 7fdf9dfe6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf98104200 0x7fdf980ff880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.587+0000 7fdf9dfe6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf98106b30 0x7fdf980ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.587+0000 7fdf9dfe6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf98100560 con 0x7fdf981018d0 2023-09-30T14:38:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.587+0000 7fdf9dfe6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf981006d0 con 0x7fdf98106b30 2023-09-30T14:38:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.587+0000 7fdf9dfe6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf981a0a60 con 0x7fdf98104200 2023-09-30T14:38:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.588+0000 7fdf977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf981018d0 0x7fdf98101240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.588+0000 7fdf977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf981018d0 0x7fdf98101240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35350/0 (socket says 172.21.15.32:35350) 2023-09-30T14:38:42.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.588+0000 7fdf977fe700 1 -- 172.21.15.32:0/162470893 learned_addr learned my addr 172.21.15.32:0/162470893 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:42.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.588+0000 7fdf97fff700 1 --2- 172.21.15.32:0/162470893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf98106b30 0x7fdf980ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.588+0000 7fdf8ffff700 1 --2- 172.21.15.32:0/162470893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf98104200 0x7fdf980ff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.588+0000 7fdf977fe700 1 -- 172.21.15.32:0/162470893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf98104200 msgr2=0x7fdf980ff880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.588+0000 7fdf977fe700 1 --2- 172.21.15.32:0/162470893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf98104200 0x7fdf980ff880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.588+0000 7fdf977fe700 1 -- 172.21.15.32:0/162470893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf98106b30 msgr2=0x7fdf980ffdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.588+0000 7fdf977fe700 1 --2- 172.21.15.32:0/162470893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf98106b30 0x7fdf980ffdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.588+0000 7fdf977fe700 1 -- 172.21.15.32:0/162470893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf800095f0 con 0x7fdf981018d0 2023-09-30T14:38:42.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.588+0000 7fdf977fe700 1 --2- 172.21.15.32:0/162470893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf981018d0 0x7fdf98101240 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800c9b0 tx=0x7fdf8800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:42.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.588+0000 7fdf957fa700 1 -- 172.21.15.32:0/162470893 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88007b00 con 0x7fdf981018d0 2023-09-30T14:38:42.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.588+0000 7fdf957fa700 1 -- 172.21.15.32:0/162470893 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf88007c60 con 0x7fdf981018d0 2023-09-30T14:38:42.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.588+0000 7fdf9dfe6700 1 -- 172.21.15.32:0/162470893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf981a0d40 con 0x7fdf981018d0 2023-09-30T14:38:42.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.588+0000 7fdf957fa700 1 -- 172.21.15.32:0/162470893 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88018660 con 0x7fdf981018d0 2023-09-30T14:38:42.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.588+0000 7fdf9dfe6700 1 -- 172.21.15.32:0/162470893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf981a11e0 con 0x7fdf981018d0 2023-09-30T14:38:42.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.591+0000 7fdf957fa700 1 -- 172.21.15.32:0/162470893 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fdf88018880 con 0x7fdf981018d0 2023-09-30T14:38:42.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.591+0000 7fdf957fa700 1 --2- 172.21.15.32:0/162470893 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdf840a1020 0x7fdf840a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.592+0000 7fdf957fa700 1 -- 172.21.15.32:0/162470893 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdf880cc700 con 0x7fdf981018d0 2023-09-30T14:38:42.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.592+0000 7fdf8ffff700 1 --2- 172.21.15.32:0/162470893 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdf840a1020 0x7fdf840a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.592+0000 7fdf8ffff700 1 --2- 172.21.15.32:0/162470893 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdf840a1020 0x7fdf840a34e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdf80006010 tx=0x7fdf8000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:42.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.592+0000 7fdfaa1ceb80 1 Processor -- start 2023-09-30T14:38:42.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.592+0000 7fdfaa1ceb80 1 -- start start 2023-09-30T14:38:42.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.592+0000 7fdfaa1ceb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4620 0x55e2a42b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdfaa1ceb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b4f80 0x55e2a42b5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdfaa1ceb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b5e20 0x55e2a42b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdfaa1ceb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a421c4e0 con 0x55e2a42b4620 2023-09-30T14:38:42.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdfaa1ceb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a420dca0 con 0x55e2a42b5e20 2023-09-30T14:38:42.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdfaa1ceb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a420b5c0 con 0x55e2a42b4f80 2023-09-30T14:38:42.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdf977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4620 0x55e2a42b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdf977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4620 0x55e2a42b4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35358/0 (socket says 172.21.15.32:35358) 2023-09-30T14:38:42.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdf977fe700 1 -- 172.21.15.32:0/3119573008 learned_addr learned my addr 172.21.15.32:0/3119573008 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:42.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdf8ffff700 1 --2- 172.21.15.32:0/3119573008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b4f80 0x55e2a42b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdf97fff700 1 --2- 172.21.15.32:0/3119573008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b5e20 0x55e2a42b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdf977fe700 1 -- 172.21.15.32:0/3119573008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b4f80 msgr2=0x55e2a42b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:42.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdf977fe700 1 --2- 172.21.15.32:0/3119573008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b4f80 0x55e2a42b5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdf977fe700 1 -- 172.21.15.32:0/3119573008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b5e20 msgr2=0x55e2a42b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:42.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdf977fe700 1 --2- 172.21.15.32:0/3119573008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b5e20 0x55e2a42b62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdf977fe700 1 -- 172.21.15.32:0/3119573008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2a4269950 con 0x55e2a42b4620 2023-09-30T14:38:42.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdf97fff700 1 --2- 172.21.15.32:0/3119573008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b5e20 0x55e2a42b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:42.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdf977fe700 1 --2- 172.21.15.32:0/3119573008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4620 0x55e2a42b4a40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdf880960d0 tx=0x7fdf88007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72da3a5d5ef47595 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:42.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdf8dffb700 1 -- 172.21.15.32:0/3119573008 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88099bd0 con 0x55e2a42b4620 2023-09-30T14:38:42.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdf8dffb700 1 -- 172.21.15.32:0/3119573008 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf8809dd20 con 0x55e2a42b4620 2023-09-30T14:38:42.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.593+0000 7fdf8dffb700 1 -- 172.21.15.32:0/3119573008 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8809a090 con 0x55e2a42b4620 2023-09-30T14:38:42.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.594+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/3119573008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4620 msgr2=0x55e2a42b4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.594+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/3119573008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4620 0x55e2a42b4a40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdf880960d0 tx=0x7fdf88007dd0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.594+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/3119573008 shutdown_connections 2023-09-30T14:38:42.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.594+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/3119573008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b5e20 0x55e2a42b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.594+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/3119573008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4620 0x55e2a42b4a40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.594+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/3119573008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b4f80 0x55e2a42b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.594+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/3119573008 >> 172.21.15.32:0/3119573008 conn(0x55e2a42282e0 msgr2=0x55e2a42af500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:42.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.594+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/3119573008 shutdown_connections 2023-09-30T14:38:42.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.594+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/3119573008 wait complete. 2023-09-30T14:38:42.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.595+0000 7fdfaa1ceb80 1 -- start start 2023-09-30T14:38:42.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.595+0000 7fdfaa1ceb80 1 Processor -- start 2023-09-30T14:38:42.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.595+0000 7fdfaa1ceb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b4620 0x55e2a42be2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.595+0000 7fdfaa1ceb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b4f80 0x55e2a42be830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.595+0000 7fdfaa1ceb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b5e20 0x55e2a42c5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.595+0000 7fdfaa1ceb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a4206eb0 con 0x55e2a42b5e20 2023-09-30T14:38:42.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.595+0000 7fdfaa1ceb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a41a3790 con 0x55e2a42b4f80 2023-09-30T14:38:42.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.595+0000 7fdfaa1ceb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a41a3430 con 0x55e2a42b4620 2023-09-30T14:38:42.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.595+0000 7fdfaa1ceb80 10 client.0 _ll_register_callbacks cb 0x55e2a4000880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.595+0000 7fdf97fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b5e20 0x55e2a42c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.595+0000 7fdf97fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b5e20 0x55e2a42c5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35368/0 (socket says 172.21.15.32:35368) 2023-09-30T14:38:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.595+0000 7fdf97fff700 1 -- 172.21.15.32:0/1714438595 learned_addr learned my addr 172.21.15.32:0/1714438595 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.595+0000 7fdf977fe700 1 --2- 172.21.15.32:0/1714438595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b4620 0x55e2a42be2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.595+0000 7fdf8ffff700 1 --2- 172.21.15.32:0/1714438595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b4f80 0x55e2a42be830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.596+0000 7fdf97fff700 1 -- 172.21.15.32:0/1714438595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b4620 msgr2=0x55e2a42be2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.596+0000 7fdf97fff700 1 --2- 172.21.15.32:0/1714438595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b4620 0x55e2a42be2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.596+0000 7fdf97fff700 1 -- 172.21.15.32:0/1714438595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b4f80 msgr2=0x55e2a42be830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.596+0000 7fdf97fff700 1 --2- 172.21.15.32:0/1714438595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b4f80 0x55e2a42be830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.596+0000 7fdf97fff700 1 -- 172.21.15.32:0/1714438595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf981a11e0 con 0x55e2a42b5e20 2023-09-30T14:38:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.596+0000 7fdf8ffff700 1 --2- 172.21.15.32:0/1714438595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b4f80 0x55e2a42be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:42.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.596+0000 7fdf97fff700 1 --2- 172.21.15.32:0/1714438595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b5e20 0x55e2a42c5870 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000f850 tx=0x7fdf9000fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4455b54fa079d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.596+0000 7fdf76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.596+0000 7fdf76ffd700 1 -- 172.21.15.32:0/1714438595 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9000d070 con 0x55e2a42b5e20 2023-09-30T14:38:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.596+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/1714438595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2a42a72d0 con 0x55e2a42b5e20 2023-09-30T14:38:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.596+0000 7fdf76ffd700 1 -- 172.21.15.32:0/1714438595 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdf90004bb0 con 0x55e2a42b5e20 2023-09-30T14:38:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.596+0000 7fdf76ffd700 1 -- 172.21.15.32:0/1714438595 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf90004d70 con 0x55e2a42b5e20 2023-09-30T14:38:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.596+0000 7fdfaa1ceb80 10 client.15369 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.596+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/1714438595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55e2a4031e60 con 0x55e2a42b5e20 2023-09-30T14:38:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.596+0000 7fdf76ffd700 1 -- 172.21.15.32:0/1714438595 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf90023020 con 0x55e2a42b5e20 2023-09-30T14:38:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.597+0000 7fdf76ffd700 1 -- 172.21.15.32:0/1714438595 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdf90014640 con 0x55e2a42b5e20 2023-09-30T14:38:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.597+0000 7fdfaa1ceb80 10 client.15369 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:42.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.597+0000 7fdfaa1ceb80 10 client.15369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-30T14:38:42.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.597+0000 7fdfaa1ceb80 20 client.15369 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:42.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.597+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/1714438595 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e2a42ca190 0x55e2a42cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.597+0000 7fdfaa1ceb80 4 client.15369 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:38:42.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.597+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/1714438595 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2a41ffe90 con 0x55e2a42ca190 2023-09-30T14:38:42.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.597+0000 7fdf977fe700 1 --2- 172.21.15.32:0/1714438595 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e2a42ca190 0x55e2a42cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.597+0000 7fdf977fe700 1 --2- 172.21.15.32:0/1714438595 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e2a42ca190 0x55e2a42cc5b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a66b6dd79445beb0 server_cookie=b1e448613d099459 in_seq=0 out_seq=0 2023-09-30T14:38:42.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.597+0000 7fdf76ffd700 10 client.15369 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:42.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.598+0000 7fdf76ffd700 1 -- 172.21.15.32:0/1714438595 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e2a41ffe90 con 0x55e2a42ca190 2023-09-30T14:38:42.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.598+0000 7fdf76ffd700 10 client.15369 handle_command_reply: tid=0 2023-09-30T14:38:42.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.598+0000 7fdfaa1ceb80 1 client.15369 shutdown 2023-09-30T14:38:42.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.598+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/1714438595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b5e20 msgr2=0x55e2a42c5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.598+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/1714438595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b5e20 0x55e2a42c5870 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000f850 tx=0x7fdf9000fb60 comp rx=0 tx=0).stop 2023-09-30T14:38:42.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.598+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/1714438595 shutdown_connections 2023-09-30T14:38:42.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.598+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/1714438595 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e2a42ca190 0x55e2a42cc5b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.598+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/1714438595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b5e20 0x55e2a42c5870 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.598+0000 7fdf76ffd700 0 client.15369 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:42.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.598+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/1714438595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b4620 0x55e2a42be2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.598+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/1714438595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b4f80 0x55e2a42be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.598+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/1714438595 >> 172.21.15.32:0/1714438595 conn(0x55e2a42282e0 msgr2=0x55e2a42af310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:42.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.598+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/1714438595 shutdown_connections 2023-09-30T14:38:42.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.598+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/1714438595 wait complete. 2023-09-30T14:38:42.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.599+0000 7fdfaa1ceb80 20 client.15369 trim_cache size 0 max 16384 2023-09-30T14:38:42.609 INFO:tasks.workunit.client.0.smithi032.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-09-30T14:38:42.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.616+0000 7fdfaa1ceb80 1 Processor -- start 2023-09-30T14:38:42.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.616+0000 7fdfaa1ceb80 1 -- start start 2023-09-30T14:38:42.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdfaa1ceb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b4620 0x55e2a42be2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdfaa1ceb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4f80 0x55e2a42c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdfaa1ceb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b5e20 0x55e2a42be830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdfaa1ceb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a40879a0 con 0x55e2a42b4f80 2023-09-30T14:38:42.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdfaa1ceb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a42bde60 con 0x55e2a42b4620 2023-09-30T14:38:42.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdfaa1ceb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a42abde0 con 0x55e2a42b5e20 2023-09-30T14:38:42.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdf8ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4f80 0x55e2a42c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdf8ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4f80 0x55e2a42c5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35376/0 (socket says 172.21.15.32:35376) 2023-09-30T14:38:42.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdf8ffff700 1 -- 172.21.15.32:0/1389242700 learned_addr learned my addr 172.21.15.32:0/1389242700 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:42.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdf977fe700 1 --2- 172.21.15.32:0/1389242700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b4620 0x55e2a42be2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdf97fff700 1 --2- 172.21.15.32:0/1389242700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b5e20 0x55e2a42be830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdf8ffff700 1 -- 172.21.15.32:0/1389242700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b5e20 msgr2=0x55e2a42be830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:42.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdf8ffff700 1 --2- 172.21.15.32:0/1389242700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b5e20 0x55e2a42be830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdf8ffff700 1 -- 172.21.15.32:0/1389242700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b4620 msgr2=0x55e2a42be2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:42.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdf8ffff700 1 --2- 172.21.15.32:0/1389242700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b4620 0x55e2a42be2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdf8ffff700 1 -- 172.21.15.32:0/1389242700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2a42ae250 con 0x55e2a42b4f80 2023-09-30T14:38:42.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdf8ffff700 1 --2- 172.21.15.32:0/1389242700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4f80 0x55e2a42c5790 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000bb50 tx=0x7fdf80022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabfa713ede885d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:42.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.617+0000 7fdf8e7fc700 1 -- 172.21.15.32:0/1389242700 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80010b00 con 0x55e2a42b4f80 2023-09-30T14:38:42.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.618+0000 7fdf8e7fc700 1 -- 172.21.15.32:0/1389242700 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf8002c050 con 0x55e2a42b4f80 2023-09-30T14:38:42.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.618+0000 7fdf8e7fc700 1 -- 172.21.15.32:0/1389242700 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80024410 con 0x55e2a42b4f80 2023-09-30T14:38:42.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.618+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/1389242700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4f80 msgr2=0x55e2a42c5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.618+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/1389242700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4f80 0x55e2a42c5790 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000bb50 tx=0x7fdf80022000 comp rx=0 tx=0).stop 2023-09-30T14:38:42.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.618+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/1389242700 shutdown_connections 2023-09-30T14:38:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.618+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/1389242700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b5e20 0x55e2a42be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.618+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/1389242700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b4620 0x55e2a42be2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.618+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/1389242700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4f80 0x55e2a42c5790 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.618+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/1389242700 >> 172.21.15.32:0/1389242700 conn(0x55e2a42282e0 msgr2=0x55e2a42b8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.618+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/1389242700 shutdown_connections 2023-09-30T14:38:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.618+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/1389242700 wait complete. 2023-09-30T14:38:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.619+0000 7fdfaa1ceb80 1 -- start start 2023-09-30T14:38:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.619+0000 7fdfaa1ceb80 1 Processor -- start 2023-09-30T14:38:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.619+0000 7fdfaa1ceb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4620 0x55e2a42c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.619+0000 7fdfaa1ceb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b4f80 0x55e2a42c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.619+0000 7fdfaa1ceb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b5e20 0x55e2a431e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.619+0000 7fdfaa1ceb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a431ea40 con 0x55e2a42b4620 2023-09-30T14:38:42.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.619+0000 7fdfaa1ceb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a431eb80 con 0x55e2a42b4f80 2023-09-30T14:38:42.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.619+0000 7fdfaa1ceb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a431ed10 con 0x55e2a42b5e20 2023-09-30T14:38:42.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.619+0000 7fdfaa1ceb80 10 client.0 _ll_register_callbacks cb 0x7fdf90007f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:42.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.619+0000 7fdf977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4620 0x55e2a42c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.619+0000 7fdf977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4620 0x55e2a42c4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35380/0 (socket says 172.21.15.32:35380) 2023-09-30T14:38:42.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.619+0000 7fdf977fe700 1 -- 172.21.15.32:0/2129556150 learned_addr learned my addr 172.21.15.32:0/2129556150 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:42.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.619+0000 7fdf8ffff700 1 --2- 172.21.15.32:0/2129556150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b4f80 0x55e2a42c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.619+0000 7fdf97fff700 1 --2- 172.21.15.32:0/2129556150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b5e20 0x55e2a431e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdf977fe700 1 -- 172.21.15.32:0/2129556150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b5e20 msgr2=0x55e2a431e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:42.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdf977fe700 1 --2- 172.21.15.32:0/2129556150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b5e20 0x55e2a431e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdf977fe700 1 -- 172.21.15.32:0/2129556150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b4f80 msgr2=0x55e2a42c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdf977fe700 1 --2- 172.21.15.32:0/2129556150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b4f80 0x55e2a42c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdf977fe700 1 -- 172.21.15.32:0/2129556150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf98109b90 con 0x55e2a42b4620 2023-09-30T14:38:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdf977fe700 1 --2- 172.21.15.32:0/2129556150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4620 0x55e2a42c4d20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdf880074a0 tx=0x7fdf88096040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c72e471643172f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdf76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdf76ffd700 1 -- 172.21.15.32:0/2129556150 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf880999a0 con 0x55e2a42b4620 2023-09-30T14:38:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdf76ffd700 1 -- 172.21.15.32:0/2129556150 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdf880990c0 con 0x55e2a42b4620 2023-09-30T14:38:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/2129556150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf8809dd20 con 0x55e2a42b4620 2023-09-30T14:38:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdf76ffd700 1 -- 172.21.15.32:0/2129556150 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf88099280 con 0x55e2a42b4620 2023-09-30T14:38:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdfaa1ceb80 10 client.15375 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/2129556150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55e2a401ace0 con 0x55e2a42b4620 2023-09-30T14:38:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdf76ffd700 1 -- 172.21.15.32:0/2129556150 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf8809dd20 con 0x55e2a42b4620 2023-09-30T14:38:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdf76ffd700 1 -- 172.21.15.32:0/2129556150 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdf880a1ad0 con 0x55e2a42b4620 2023-09-30T14:38:42.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdfaa1ceb80 10 client.15375 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:42.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdfaa1ceb80 10 client.15375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-30T14:38:42.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdfaa1ceb80 20 client.15375 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:42.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.620+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/2129556150 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e2a4320360 0x55e2a4322780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.621+0000 7fdfaa1ceb80 4 client.15375 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-30T14:38:42.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.621+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/2129556150 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e2a41ffbb0 con 0x55e2a4320360 2023-09-30T14:38:42.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.621+0000 7fdf8ffff700 1 --2- 172.21.15.32:0/2129556150 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e2a4320360 0x55e2a4322780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.621+0000 7fdf76ffd700 10 client.15375 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:42.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.621+0000 7fdf8ffff700 1 --2- 172.21.15.32:0/2129556150 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e2a4320360 0x55e2a4322780 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cedc731e9ab7ff6 server_cookie=821e8aea3dec36a7 in_seq=0 out_seq=0 2023-09-30T14:38:42.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.622+0000 7fdf76ffd700 1 -- 172.21.15.32:0/2129556150 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e2a41ffbb0 con 0x55e2a4320360 2023-09-30T14:38:42.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.622+0000 7fdf76ffd700 10 client.15375 handle_command_reply: tid=0 2023-09-30T14:38:42.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.622+0000 7fdfaa1ceb80 1 client.15375 shutdown 2023-09-30T14:38:42.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/2129556150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4620 msgr2=0x55e2a42c4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/2129556150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4620 0x55e2a42c4d20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdf880074a0 tx=0x7fdf88096040 comp rx=0 tx=0).stop 2023-09-30T14:38:42.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/2129556150 shutdown_connections 2023-09-30T14:38:42.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/2129556150 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e2a4320360 0x55e2a4322780 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/2129556150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e2a42b5e20 0x55e2a431e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdf76ffd700 0 client.15375 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:42.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/2129556150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e2a42b4620 0x55e2a42c4d20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdfaa1ceb80 1 --2- 172.21.15.32:0/2129556150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2a42b4f80 0x55e2a42c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/2129556150 >> 172.21.15.32:0/2129556150 conn(0x55e2a42282e0 msgr2=0x55e2a42b8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:42.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/2129556150 shutdown_connections 2023-09-30T14:38:42.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdfaa1ceb80 1 -- 172.21.15.32:0/2129556150 wait complete. 2023-09-30T14:38:42.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdfaa1ceb80 20 client.15375 trim_cache size 0 max 16384 2023-09-30T14:38:42.630 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T14:38:42.630 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2023-09-30T14:38:42.630 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "6c05820b-f0a7-4d0a-82bd-5e669c9e81a6", 2023-09-30T14:38:42.630 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2023-09-30T14:38:42.630 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T14:38:42.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdf9dfe6700 1 -- 172.21.15.32:0/162470893 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdf840a1020 msgr2=0x7fdf840a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdf9dfe6700 1 --2- 172.21.15.32:0/162470893 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdf840a1020 0x7fdf840a34e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdf80006010 tx=0x7fdf8000b540 comp rx=0 tx=0).stop 2023-09-30T14:38:42.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdf9dfe6700 1 -- 172.21.15.32:0/162470893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf981018d0 msgr2=0x7fdf98101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdf9dfe6700 1 --2- 172.21.15.32:0/162470893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf981018d0 0x7fdf98101240 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800c9b0 tx=0x7fdf8800cd70 comp rx=0 tx=0).stop 2023-09-30T14:38:42.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdf9dfe6700 1 -- 172.21.15.32:0/162470893 shutdown_connections 2023-09-30T14:38:42.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdf9dfe6700 1 --2- 172.21.15.32:0/162470893 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdf840a1020 0x7fdf840a34e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdf9dfe6700 1 --2- 172.21.15.32:0/162470893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf98106b30 0x7fdf980ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdf9dfe6700 1 --2- 172.21.15.32:0/162470893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf981018d0 0x7fdf98101240 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdf9dfe6700 1 --2- 172.21.15.32:0/162470893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf98104200 0x7fdf980ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.623+0000 7fdf9dfe6700 1 -- 172.21.15.32:0/162470893 >> 172.21.15.32:0/162470893 conn(0x7fdf980fb2a0 msgr2=0x7fdf98102540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:42.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.624+0000 7fdf9dfe6700 1 -- 172.21.15.32:0/162470893 shutdown_connections 2023-09-30T14:38:42.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.624+0000 7fdf9dfe6700 1 -- 172.21.15.32:0/162470893 wait complete. 2023-09-30T14:38:42.637 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-30T14:38:42.701 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:42 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2698072613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:42.701 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:42 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/650494351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:42.701 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:42 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3857595877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:42.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:42 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2698072613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:42.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:42 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/650494351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:42.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:42 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3857595877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:42.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.816+0000 7fdb90850700 1 -- 172.21.15.32:0/2729840537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb8c106b10 msgr2=0x7fdb8c108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.816+0000 7fdb90850700 1 --2- 172.21.15.32:0/2729840537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb8c106b10 0x7fdb8c108f80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400b0a0 tx=0x7fdb8400b3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.816+0000 7fdb90850700 1 -- 172.21.15.32:0/2729840537 shutdown_connections 2023-09-30T14:38:42.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.816+0000 7fdb90850700 1 --2- 172.21.15.32:0/2729840537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb8c106b10 0x7fdb8c108f80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.816+0000 7fdb90850700 1 --2- 172.21.15.32:0/2729840537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb8c1018b0 0x7fdb8c103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.816+0000 7fdb90850700 1 --2- 172.21.15.32:0/2729840537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb8c1041e0 0x7fdb8c1065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.816+0000 7fdb90850700 1 -- 172.21.15.32:0/2729840537 >> 172.21.15.32:0/2729840537 conn(0x7fdb8c0fb2a0 msgr2=0x7fdb8c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:42.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.816+0000 7fdb90850700 1 -- 172.21.15.32:0/2729840537 shutdown_connections 2023-09-30T14:38:42.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.816+0000 7fdb90850700 1 -- 172.21.15.32:0/2729840537 wait complete. 2023-09-30T14:38:42.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.817+0000 7fdb90850700 1 Processor -- start 2023-09-30T14:38:42.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.817+0000 7fdb90850700 1 -- start start 2023-09-30T14:38:42.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.817+0000 7fdb90850700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb8c1018b0 0x7fdb8c196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.817+0000 7fdb90850700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb8c1041e0 0x7fdb8c197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb90850700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb8c106b10 0x7fdb8c19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb90850700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c19e770 con 0x7fdb8c1041e0 2023-09-30T14:38:42.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb90850700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c19e8e0 con 0x7fdb8c1018b0 2023-09-30T14:38:42.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb90850700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c19eaa0 con 0x7fdb8c106b10 2023-09-30T14:38:42.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb89d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb8c1041e0 0x7fdb8c197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb89d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb8c1041e0 0x7fdb8c197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35396/0 (socket says 172.21.15.32:35396) 2023-09-30T14:38:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb89d9b700 1 -- 172.21.15.32:0/406494999 learned_addr learned my addr 172.21.15.32:0/406494999 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb8a59c700 1 --2- 172.21.15.32:0/406494999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb8c1018b0 0x7fdb8c196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb8ad9d700 1 --2- 172.21.15.32:0/406494999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb8c106b10 0x7fdb8c19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb89d9b700 1 -- 172.21.15.32:0/406494999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb8c106b10 msgr2=0x7fdb8c19e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb89d9b700 1 --2- 172.21.15.32:0/406494999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb8c106b10 0x7fdb8c19e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb89d9b700 1 -- 172.21.15.32:0/406494999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb8c1018b0 msgr2=0x7fdb8c196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb89d9b700 1 --2- 172.21.15.32:0/406494999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb8c1018b0 0x7fdb8c196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb89d9b700 1 -- 172.21.15.32:0/406494999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c009520 con 0x7fdb8c1041e0 2023-09-30T14:38:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb89d9b700 1 --2- 172.21.15.32:0/406494999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb8c1041e0 0x7fdb8c197170 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000e9a0 tx=0x7fdb8000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb7b7fe700 1 -- 172.21.15.32:0/406494999 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb80009790 con 0x7fdb8c1041e0 2023-09-30T14:38:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb7b7fe700 1 -- 172.21.15.32:0/406494999 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb8000cbd0 con 0x7fdb8c1041e0 2023-09-30T14:38:42.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb90850700 1 -- 172.21.15.32:0/406494999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb84009d00 con 0x7fdb8c1041e0 2023-09-30T14:38:42.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb7b7fe700 1 -- 172.21.15.32:0/406494999 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb80018680 con 0x7fdb8c1041e0 2023-09-30T14:38:42.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.818+0000 7fdb90850700 1 -- 172.21.15.32:0/406494999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c19f100 con 0x7fdb8c1041e0 2023-09-30T14:38:42.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.822+0000 7fdb9ca38b80 1 Processor -- start 2023-09-30T14:38:42.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.822+0000 7fdb9ca38b80 1 -- start start 2023-09-30T14:38:42.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.822+0000 7fdb9ca38b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365bb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.822+0000 7fdb9ca38b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365bbf00 0x5640365bc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.822+0000 7fdb9ca38b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365bcda0 0x5640365bd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.822+0000 7fdb9ca38b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564036523460 con 0x5640365bb5a0 2023-09-30T14:38:42.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.822+0000 7fdb9ca38b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564036514c20 con 0x5640365bbf00 2023-09-30T14:38:42.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.822+0000 7fdb9ca38b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564036512540 con 0x5640365bcda0 2023-09-30T14:38:42.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.822+0000 7fdb8ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365bb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.822+0000 7fdb8ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365bb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35400/0 (socket says 172.21.15.32:35400) 2023-09-30T14:38:42.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.822+0000 7fdb8ad9d700 1 -- 172.21.15.32:0/4124121660 learned_addr learned my addr 172.21.15.32:0/4124121660 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:42.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.823+0000 7fdb8a59c700 1 --2- 172.21.15.32:0/4124121660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365bbf00 0x5640365bc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.823+0000 7fdb8ad9d700 1 -- 172.21.15.32:0/4124121660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365bcda0 msgr2=0x5640365bd220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T14:38:42.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.823+0000 7fdb8ad9d700 1 --2- 172.21.15.32:0/4124121660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365bcda0 0x5640365bd220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.823+0000 7fdb8ad9d700 1 -- 172.21.15.32:0/4124121660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365bbf00 msgr2=0x5640365bc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:42.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.823+0000 7fdb8ad9d700 1 --2- 172.21.15.32:0/4124121660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365bbf00 0x5640365bc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.823+0000 7fdb8ad9d700 1 -- 172.21.15.32:0/4124121660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640365708d0 con 0x5640365bb5a0 2023-09-30T14:38:42.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.823+0000 7fdb8a59c700 1 --2- 172.21.15.32:0/4124121660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365bbf00 0x5640365bc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:42.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.823+0000 7fdb8ad9d700 1 --2- 172.21.15.32:0/4124121660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365bb9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdb84015000 tx=0x7fdb84009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445b94dd1fa76b99 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:42.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.823+0000 7fdb78ff9700 1 -- 172.21.15.32:0/4124121660 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8400e050 con 0x5640365bb5a0 2023-09-30T14:38:42.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.823+0000 7fdb78ff9700 1 -- 172.21.15.32:0/4124121660 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb84009460 con 0x5640365bb5a0 2023-09-30T14:38:42.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.823+0000 7fdb78ff9700 1 -- 172.21.15.32:0/4124121660 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84005240 con 0x5640365bb5a0 2023-09-30T14:38:42.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.824+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/4124121660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 msgr2=0x5640365bb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.824+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/4124121660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365bb9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdb84015000 tx=0x7fdb84009c40 comp rx=0 tx=0).stop 2023-09-30T14:38:42.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.824+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/4124121660 shutdown_connections 2023-09-30T14:38:42.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.824+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/4124121660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365bcda0 0x5640365bd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.824+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/4124121660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365bb9c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.824+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/4124121660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365bbf00 0x5640365bc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.824+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/4124121660 >> 172.21.15.32:0/4124121660 conn(0x56403652f260 msgr2=0x5640365b6480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:42.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.824+0000 7fdb7b7fe700 1 -- 172.21.15.32:0/406494999 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fdb8000cd40 con 0x7fdb8c1041e0 2023-09-30T14:38:42.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.824+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/4124121660 shutdown_connections 2023-09-30T14:38:42.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.824+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/4124121660 wait complete. 2023-09-30T14:38:42.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.825+0000 7fdb9ca38b80 1 -- start start 2023-09-30T14:38:42.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.825+0000 7fdb9ca38b80 1 Processor -- start 2023-09-30T14:38:42.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.825+0000 7fdb7b7fe700 1 --2- 172.21.15.32:0/406494999 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdb740a0eb0 0x7fdb740a3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.825+0000 7fdb8ad9d700 1 --2- 172.21.15.32:0/406494999 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdb740a0eb0 0x7fdb740a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.825+0000 7fdb9ca38b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.825+0000 7fdb9ca38b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365bbf00 0x5640365c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.825+0000 7fdb7b7fe700 1 -- 172.21.15.32:0/406494999 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdb80014070 con 0x7fdb8c1041e0 2023-09-30T14:38:42.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.825+0000 7fdb9ca38b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365caa20 0x5640365cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.825+0000 7fdb9ca38b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403650de30 con 0x5640365bb5a0 2023-09-30T14:38:42.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.825+0000 7fdb9ca38b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640364aa710 con 0x5640365caa20 2023-09-30T14:38:42.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.825+0000 7fdb9ca38b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640364aa3b0 con 0x5640365bbf00 2023-09-30T14:38:42.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.825+0000 7fdb9ca38b80 10 client.0 _ll_register_callbacks cb 0x564036307800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:42.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.825+0000 7fdb8a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.826+0000 7fdb8a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365c54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35406/0 (socket says 172.21.15.32:35406) 2023-09-30T14:38:42.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.826+0000 7fdb8a59c700 1 -- 172.21.15.32:0/1258704865 learned_addr learned my addr 172.21.15.32:0/1258704865 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:42.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.826+0000 7fdb89d9b700 1 --2- 172.21.15.32:0/1258704865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365bbf00 0x5640365c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.826+0000 7fdb8a59c700 1 -- 172.21.15.32:0/1258704865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365bbf00 msgr2=0x5640365c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:42.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.826+0000 7fdb8ad9d700 1 --2- 172.21.15.32:0/406494999 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdb740a0eb0 0x7fdb740a3370 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdb84004ca0 tx=0x7fdb8400b3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:42.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.826+0000 7fdb8a59c700 1 --2- 172.21.15.32:0/1258704865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365bbf00 0x5640365c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.826+0000 7fdb8ad9d700 1 --2- 172.21.15.32:0/1258704865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365caa20 0x5640365cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.826+0000 7fdb8a59c700 1 -- 172.21.15.32:0/1258704865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365caa20 msgr2=0x5640365cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:42.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.826+0000 7fdb8a59c700 1 --2- 172.21.15.32:0/1258704865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365caa20 0x5640365cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.826+0000 7fdb8a59c700 1 -- 172.21.15.32:0/1258704865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c109b70 con 0x5640365bb5a0 2023-09-30T14:38:42.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.826+0000 7fdb8a59c700 1 --2- 172.21.15.32:0/1258704865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365c54b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0116f0 tx=0x7fdb7c011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f9b8386375eb17 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:42.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.826+0000 7fdb71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:42.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.826+0000 7fdb71ffb700 1 -- 172.21.15.32:0/1258704865 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c00f740 con 0x5640365bb5a0 2023-09-30T14:38:42.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.826+0000 7fdb71ffb700 1 -- 172.21.15.32:0/1258704865 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdb7c00f8a0 con 0x5640365bb5a0 2023-09-30T14:38:42.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.826+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/1258704865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640365ae250 con 0x5640365bb5a0 2023-09-30T14:38:42.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.827+0000 7fdb71ffb700 1 -- 172.21.15.32:0/1258704865 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb7c00fa60 con 0x5640365bb5a0 2023-09-30T14:38:42.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.827+0000 7fdb9ca38b80 10 client.15387 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:42.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.827+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/1258704865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x564036338de0 con 0x5640365bb5a0 2023-09-30T14:38:42.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.827+0000 7fdb71ffb700 1 -- 172.21.15.32:0/1258704865 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5640365ae250 con 0x5640365bb5a0 2023-09-30T14:38:42.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.827+0000 7fdb71ffb700 1 -- 172.21.15.32:0/1258704865 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdb7c01bd20 con 0x5640365bb5a0 2023-09-30T14:38:42.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.827+0000 7fdb9ca38b80 10 client.15387 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:42.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.827+0000 7fdb9ca38b80 10 client.15387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-30T14:38:42.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.827+0000 7fdb9ca38b80 20 client.15387 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:42.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.827+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/1258704865 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5640365d1640 0x5640365d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.827+0000 7fdb9ca38b80 4 client.15387 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:38:42.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.827+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/1258704865 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564036506e10 con 0x5640365d1640 2023-09-30T14:38:42.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.827+0000 7fdb89d9b700 1 --2- 172.21.15.32:0/1258704865 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5640365d1640 0x5640365d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.828+0000 7fdb71ffb700 10 client.15387 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:42.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.828+0000 7fdb89d9b700 1 --2- 172.21.15.32:0/1258704865 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5640365d1640 0x5640365d3a60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aedfc5850ee016c server_cookie=1b9a9725960880f4 in_seq=0 out_seq=0 2023-09-30T14:38:42.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.829+0000 7fdb71ffb700 1 -- 172.21.15.32:0/1258704865 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564036506e10 con 0x5640365d1640 2023-09-30T14:38:42.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.829+0000 7fdb71ffb700 10 client.15387 handle_command_reply: tid=0 2023-09-30T14:38:42.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.829+0000 7fdb9ca38b80 1 client.15387 shutdown 2023-09-30T14:38:42.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.829+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/1258704865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 msgr2=0x5640365c54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.829+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/1258704865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365c54b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0116f0 tx=0x7fdb7c011a00 comp rx=0 tx=0).stop 2023-09-30T14:38:42.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.829+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/1258704865 shutdown_connections 2023-09-30T14:38:42.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.829+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/1258704865 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5640365d1640 0x5640365d3a60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.829+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/1258704865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365caa20 0x5640365cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.829+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/1258704865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365c54b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.829+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/1258704865 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365bbf00 0x5640365c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.829+0000 7fdb71ffb700 0 client.15387 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:42.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.829+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/1258704865 >> 172.21.15.32:0/1258704865 conn(0x56403652f260 msgr2=0x5640365b6290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:42.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.829+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/1258704865 shutdown_connections 2023-09-30T14:38:42.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.829+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/1258704865 wait complete. 2023-09-30T14:38:42.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.829+0000 7fdb9ca38b80 20 client.15387 trim_cache size 0 max 16384 2023-09-30T14:38:42.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.846+0000 7fdb9ca38b80 1 Processor -- start 2023-09-30T14:38:42.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.846+0000 7fdb9ca38b80 1 -- start start 2023-09-30T14:38:42.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.846+0000 7fdb9ca38b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.846+0000 7fdb9ca38b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365bbf00 0x5640365b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.846+0000 7fdb9ca38b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365caa20 0x5640365e1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.846+0000 7fdb9ca38b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403638e920 con 0x5640365bb5a0 2023-09-30T14:38:42.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.846+0000 7fdb9ca38b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640365b0890 con 0x5640365bbf00 2023-09-30T14:38:42.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.846+0000 7fdb9ca38b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640365e17b0 con 0x5640365caa20 2023-09-30T14:38:42.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.846+0000 7fdb8a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.846+0000 7fdb8a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35414/0 (socket says 172.21.15.32:35414) 2023-09-30T14:38:42.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.847+0000 7fdb8a59c700 1 -- 172.21.15.32:0/1048629786 learned_addr learned my addr 172.21.15.32:0/1048629786 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:42.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.847+0000 7fdb89d9b700 1 --2- 172.21.15.32:0/1048629786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365bbf00 0x5640365b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.847+0000 7fdb8ad9d700 1 --2- 172.21.15.32:0/1048629786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365caa20 0x5640365e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.847+0000 7fdb8a59c700 1 -- 172.21.15.32:0/1048629786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365caa20 msgr2=0x5640365e1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:42.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.847+0000 7fdb8a59c700 1 --2- 172.21.15.32:0/1048629786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365caa20 0x5640365e1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.847+0000 7fdb8a59c700 1 -- 172.21.15.32:0/1048629786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365bbf00 msgr2=0x5640365b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.847+0000 7fdb8a59c700 1 --2- 172.21.15.32:0/1048629786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365bbf00 0x5640365b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.847+0000 7fdb8a59c700 1 -- 172.21.15.32:0/1048629786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564036321c60 con 0x5640365bb5a0 2023-09-30T14:38:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.847+0000 7fdb8a59c700 1 --2- 172.21.15.32:0/1048629786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365b21f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0116c0 tx=0x7fdb7c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1311db9da6b88b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.847+0000 7fdb797fa700 1 -- 172.21.15.32:0/1048629786 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c004020 con 0x5640365bb5a0 2023-09-30T14:38:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.847+0000 7fdb797fa700 1 -- 172.21.15.32:0/1048629786 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb7c0257f0 con 0x5640365bb5a0 2023-09-30T14:38:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.847+0000 7fdb797fa700 1 -- 172.21.15.32:0/1048629786 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c0053a0 con 0x5640365bb5a0 2023-09-30T14:38:42.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.848+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/1048629786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 msgr2=0x5640365b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.848+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/1048629786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365b21f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0116c0 tx=0x7fdb7c009900 comp rx=0 tx=0).stop 2023-09-30T14:38:42.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.848+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/1048629786 shutdown_connections 2023-09-30T14:38:42.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.848+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/1048629786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365caa20 0x5640365e1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.848+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/1048629786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365bb5a0 0x5640365b21f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.848+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/1048629786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365bbf00 0x5640365b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.848+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/1048629786 >> 172.21.15.32:0/1048629786 conn(0x56403652f260 msgr2=0x5640365d83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:42.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.848+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/1048629786 shutdown_connections 2023-09-30T14:38:42.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.848+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/1048629786 wait complete. 2023-09-30T14:38:42.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb9ca38b80 1 -- start start 2023-09-30T14:38:42.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb9ca38b80 1 Processor -- start 2023-09-30T14:38:42.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb9ca38b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365bb5a0 0x5640365deca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb9ca38b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365bbf00 0x5640365df1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb9ca38b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365caa20 0x5640365e01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb9ca38b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640365e0730 con 0x5640365caa20 2023-09-30T14:38:42.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb9ca38b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640365e08a0 con 0x5640365bbf00 2023-09-30T14:38:42.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb9ca38b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640365e0a60 con 0x5640365bb5a0 2023-09-30T14:38:42.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb9ca38b80 10 client.0 _ll_register_callbacks cb 0x7fdb7c009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:42.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb8ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365caa20 0x5640365e01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb8ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365caa20 0x5640365e01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35424/0 (socket says 172.21.15.32:35424) 2023-09-30T14:38:42.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb8ad9d700 1 -- 172.21.15.32:0/4681948 learned_addr learned my addr 172.21.15.32:0/4681948 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:42.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb8a59c700 1 --2- 172.21.15.32:0/4681948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365bb5a0 0x5640365deca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb89d9b700 1 --2- 172.21.15.32:0/4681948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365bbf00 0x5640365df1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb8ad9d700 1 -- 172.21.15.32:0/4681948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365bb5a0 msgr2=0x5640365deca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:42.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb8ad9d700 1 --2- 172.21.15.32:0/4681948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365bb5a0 0x5640365deca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb8ad9d700 1 -- 172.21.15.32:0/4681948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365bbf00 msgr2=0x5640365df1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:42.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb8ad9d700 1 --2- 172.21.15.32:0/4681948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365bbf00 0x5640365df1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.849+0000 7fdb8ad9d700 1 -- 172.21.15.32:0/4681948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564036338de0 con 0x5640365caa20 2023-09-30T14:38:42.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.850+0000 7fdb8ad9d700 1 --2- 172.21.15.32:0/4681948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365caa20 0x5640365e01f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdb84015000 tx=0x7fdb840095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c694e7a199ac44d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:42.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.850+0000 7fdb71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:42.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.850+0000 7fdb71ffb700 1 -- 172.21.15.32:0/4681948 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84031620 con 0x5640365caa20 2023-09-30T14:38:42.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.850+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/4681948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb84009460 con 0x5640365caa20 2023-09-30T14:38:42.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.850+0000 7fdb71ffb700 1 -- 172.21.15.32:0/4681948 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdb84004cd0 con 0x5640365caa20 2023-09-30T14:38:42.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.850+0000 7fdb71ffb700 1 -- 172.21.15.32:0/4681948 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb8403f030 con 0x5640365caa20 2023-09-30T14:38:42.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.850+0000 7fdb9ca38b80 10 client.15393 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:42.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.850+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/4681948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x56403635f250 con 0x5640365caa20 2023-09-30T14:38:42.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.850+0000 7fdb71ffb700 1 -- 172.21.15.32:0/4681948 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb84009460 con 0x5640365caa20 2023-09-30T14:38:42.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.850+0000 7fdb71ffb700 1 -- 172.21.15.32:0/4681948 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdb84037020 con 0x5640365caa20 2023-09-30T14:38:42.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.850+0000 7fdb9ca38b80 10 client.15393 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:42.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.850+0000 7fdb9ca38b80 10 client.15393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-30T14:38:42.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.850+0000 7fdb9ca38b80 20 client.15393 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:42.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.850+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/4681948 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564036626370 0x564036628790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:42.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.850+0000 7fdb9ca38b80 4 client.15393 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-30T14:38:42.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.850+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/4681948 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564036506b30 con 0x564036626370 2023-09-30T14:38:42.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.851+0000 7fdb8a59c700 1 --2- 172.21.15.32:0/4681948 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564036626370 0x564036628790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:42.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.851+0000 7fdb8a59c700 1 --2- 172.21.15.32:0/4681948 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564036626370 0x564036628790 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b52f7da9ad17675 server_cookie=14f90b4980c436e9 in_seq=0 out_seq=0 2023-09-30T14:38:42.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.851+0000 7fdb71ffb700 10 client.15393 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:42.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb71ffb700 1 -- 172.21.15.32:0/4681948 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564036506b30 con 0x564036626370 2023-09-30T14:38:42.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb71ffb700 10 client.15393 handle_command_reply: tid=0 2023-09-30T14:38:42.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb9ca38b80 1 client.15393 shutdown 2023-09-30T14:38:42.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/4681948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365caa20 msgr2=0x5640365e01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/4681948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365caa20 0x5640365e01f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdb84015000 tx=0x7fdb840095d0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/4681948 shutdown_connections 2023-09-30T14:38:42.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/4681948 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564036626370 0x564036628790 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/4681948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5640365caa20 0x5640365e01f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/4681948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640365bb5a0 0x5640365deca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb71ffb700 0 client.15393 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:42.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb9ca38b80 1 --2- 172.21.15.32:0/4681948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640365bbf00 0x5640365df1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/4681948 >> 172.21.15.32:0/4681948 conn(0x56403652f260 msgr2=0x5640365c3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:42.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/4681948 shutdown_connections 2023-09-30T14:38:42.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb9ca38b80 1 -- 172.21.15.32:0/4681948 wait complete. 2023-09-30T14:38:42.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb9ca38b80 20 client.15393 trim_cache size 0 max 16384 2023-09-30T14:38:42.860 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T14:38:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-30T14:38:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2023-09-30T14:38:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: "6c05820b-f0a7-4d0a-82bd-5e669c9e81a6": { 2023-09-30T14:38:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2023-09-30T14:38:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "6c05820b-f0a7-4d0a-82bd-5e669c9e81a6", 2023-09-30T14:38:42.860 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2023-09-30T14:38:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T14:38:42.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T14:38:42.861 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T14:38:42.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb90850700 1 -- 172.21.15.32:0/406494999 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdb740a0eb0 msgr2=0x7fdb740a3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb90850700 1 --2- 172.21.15.32:0/406494999 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdb740a0eb0 0x7fdb740a3370 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdb84004ca0 tx=0x7fdb8400b3b0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb90850700 1 -- 172.21.15.32:0/406494999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb8c1041e0 msgr2=0x7fdb8c197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:42.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.852+0000 7fdb90850700 1 --2- 172.21.15.32:0/406494999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb8c1041e0 0x7fdb8c197170 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000e9a0 tx=0x7fdb8000ed60 comp rx=0 tx=0).stop 2023-09-30T14:38:42.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.853+0000 7fdb90850700 1 -- 172.21.15.32:0/406494999 shutdown_connections 2023-09-30T14:38:42.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.853+0000 7fdb90850700 1 --2- 172.21.15.32:0/406494999 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdb740a0eb0 0x7fdb740a3370 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.853+0000 7fdb90850700 1 --2- 172.21.15.32:0/406494999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb8c106b10 0x7fdb8c19e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.853+0000 7fdb90850700 1 --2- 172.21.15.32:0/406494999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb8c1018b0 0x7fdb8c196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.853+0000 7fdb90850700 1 --2- 172.21.15.32:0/406494999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb8c1041e0 0x7fdb8c197170 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:42.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.853+0000 7fdb90850700 1 -- 172.21.15.32:0/406494999 >> 172.21.15.32:0/406494999 conn(0x7fdb8c0fb2a0 msgr2=0x7fdb8c102520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:42.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.853+0000 7fdb90850700 1 -- 172.21.15.32:0/406494999 shutdown_connections 2023-09-30T14:38:42.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:38:42.853+0000 7fdb90850700 1 -- 172.21.15.32:0/406494999 wait complete. 2023-09-30T14:38:42.863 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:42 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2698072613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T14:38:42.863 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:42 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/650494351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:38:42.863 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:42 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3857595877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T14:38:42.866 INFO:tasks.cephfs.filesystem:scrub status for tag:6c05820b-f0a7-4d0a-82bd-5e669c9e81a6 - {'path': '/', 'tag': '6c05820b-f0a7-4d0a-82bd-5e669c9e81a6', 'options': 'recursive,force'} 2023-09-30T14:38:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:43 smithi156 ceph-mon[103579]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2023-09-30T14:38:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:43 smithi156 ceph-mon[103579]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-09-30T14:38:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:43 smithi156 ceph-mon[103579]: scrub queued for path: / 2023-09-30T14:38:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:43 smithi156 ceph-mon[103579]: scrub summary: idle+waiting paths [/] 2023-09-30T14:38:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:43 smithi156 ceph-mon[103579]: scrub summary: active paths [/] 2023-09-30T14:38:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:43 smithi156 ceph-mon[103579]: scrub summary: idle+waiting paths [/] 2023-09-30T14:38:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:43 smithi148 ceph-mon[102355]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2023-09-30T14:38:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:43 smithi148 ceph-mon[102355]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-09-30T14:38:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:43 smithi148 ceph-mon[102355]: scrub queued for path: / 2023-09-30T14:38:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:43 smithi148 ceph-mon[102355]: scrub summary: idle+waiting paths [/] 2023-09-30T14:38:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:43 smithi148 ceph-mon[102355]: scrub summary: active paths [/] 2023-09-30T14:38:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:43 smithi148 ceph-mon[102355]: scrub summary: idle+waiting paths [/] 2023-09-30T14:38:43.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:43 smithi032 ceph-mon[98592]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2023-09-30T14:38:43.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:43 smithi032 ceph-mon[98592]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-09-30T14:38:43.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:43 smithi032 ceph-mon[98592]: scrub queued for path: / 2023-09-30T14:38:43.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:43 smithi032 ceph-mon[98592]: scrub summary: idle+waiting paths [/] 2023-09-30T14:38:43.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:43 smithi032 ceph-mon[98592]: scrub summary: active paths [/] 2023-09-30T14:38:43.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:43 smithi032 ceph-mon[98592]: scrub summary: idle+waiting paths [/] 2023-09-30T14:38:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:45 smithi156 ceph-mon[103579]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 6 op/s 2023-09-30T14:38:45.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:45 smithi148 ceph-mon[102355]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 6 op/s 2023-09-30T14:38:45.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:45 smithi032 ceph-mon[98592]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 6 op/s 2023-09-30T14:38:46.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:46 smithi156 ceph-mon[103579]: scrub summary: idle 2023-09-30T14:38:46.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:46 smithi148 ceph-mon[102355]: scrub summary: idle 2023-09-30T14:38:46.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:46 smithi032 ceph-mon[98592]: scrub summary: idle 2023-09-30T14:38:47.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:47 smithi156 ceph-mon[103579]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-30T14:38:47.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:47 smithi148 ceph-mon[102355]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-30T14:38:47.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:47 smithi032 ceph-mon[98592]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-30T14:38:49.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:49 smithi156 ceph-mon[103579]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-09-30T14:38:49.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:49 smithi148 ceph-mon[102355]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-09-30T14:38:49.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:49 smithi032 ceph-mon[98592]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-09-30T14:38:51.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:51 smithi156 ceph-mon[103579]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-09-30T14:38:51.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:51 smithi148 ceph-mon[102355]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-09-30T14:38:51.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:51 smithi032 ceph-mon[98592]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-09-30T14:38:53.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:53 smithi156 ceph-mon[103579]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-09-30T14:38:53.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:53 smithi148 ceph-mon[102355]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-09-30T14:38:53.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:53 smithi032 ceph-mon[98592]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-09-30T14:38:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.617+0000 7f9fe7e1f700 1 -- 172.21.15.32:0/3188236720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe01018f0 msgr2=0x7f9fe01081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.617+0000 7f9fe7e1f700 1 --2- 172.21.15.32:0/3188236720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe01018f0 0x7f9fe01081a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800b0d0 tx=0x7f9fd800b3e0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.617+0000 7f9fe7e1f700 1 -- 172.21.15.32:0/3188236720 shutdown_connections 2023-09-30T14:38:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.617+0000 7f9fe7e1f700 1 --2- 172.21.15.32:0/3188236720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe01018f0 0x7f9fe01081a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.617+0000 7f9fe7e1f700 1 --2- 172.21.15.32:0/3188236720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fe00ffd60 0x7f9fe0100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.617+0000 7f9fe7e1f700 1 --2- 172.21.15.32:0/3188236720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fe0100f50 0x7f9fe01013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.617+0000 7f9fe7e1f700 1 -- 172.21.15.32:0/3188236720 >> 172.21.15.32:0/3188236720 conn(0x7f9fe00fb2b0 msgr2=0x7f9fe00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.617+0000 7f9fe7e1f700 1 -- 172.21.15.32:0/3188236720 shutdown_connections 2023-09-30T14:38:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.617+0000 7f9fe7e1f700 1 -- 172.21.15.32:0/3188236720 wait complete. 2023-09-30T14:38:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.618+0000 7f9fe7e1f700 1 Processor -- start 2023-09-30T14:38:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.618+0000 7f9fe7e1f700 1 -- start start 2023-09-30T14:38:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.618+0000 7f9fe7e1f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe00ffd60 0x7f9fe0196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.618+0000 7f9fe7e1f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fe0100f50 0x7f9fe01972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.618+0000 7f9fe7e1f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fe01018f0 0x7f9fe01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.618+0000 7f9fe7e1f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe01987a0 con 0x7f9fe00ffd60 2023-09-30T14:38:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.618+0000 7f9fe7e1f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe0198910 con 0x7f9fe0100f50 2023-09-30T14:38:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.618+0000 7f9fe7e1f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe0198ad0 con 0x7f9fe01018f0 2023-09-30T14:38:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.618+0000 7f9fe5bbb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe00ffd60 0x7f9fe0196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.618+0000 7f9fe5bbb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe00ffd60 0x7f9fe0196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35434/0 (socket says 172.21.15.32:35434) 2023-09-30T14:38:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.618+0000 7f9fe5bbb700 1 -- 172.21.15.32:0/1584794385 learned_addr learned my addr 172.21.15.32:0/1584794385 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.619+0000 7f9fe63bc700 1 --2- 172.21.15.32:0/1584794385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fe01018f0 0x7f9fe01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.619+0000 7f9fe53ba700 1 --2- 172.21.15.32:0/1584794385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fe0100f50 0x7f9fe01972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.619+0000 7f9fe5bbb700 1 -- 172.21.15.32:0/1584794385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fe01018f0 msgr2=0x7f9fe01980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.619+0000 7f9fe5bbb700 1 --2- 172.21.15.32:0/1584794385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fe01018f0 0x7f9fe01980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.619+0000 7f9fe5bbb700 1 -- 172.21.15.32:0/1584794385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fe0100f50 msgr2=0x7f9fe01972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.619+0000 7f9fe5bbb700 1 --2- 172.21.15.32:0/1584794385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fe0100f50 0x7f9fe01972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.619+0000 7f9fe5bbb700 1 -- 172.21.15.32:0/1584794385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd8009d00 con 0x7f9fe00ffd60 2023-09-30T14:38:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.619+0000 7f9fe5bbb700 1 --2- 172.21.15.32:0/1584794385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe00ffd60 0x7f9fe0196db0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00d980 tx=0x7f9fdc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.619+0000 7f9fd6ffd700 1 -- 172.21.15.32:0/1584794385 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc00de60 con 0x7f9fe00ffd60 2023-09-30T14:38:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.619+0000 7f9fd6ffd700 1 -- 172.21.15.32:0/1584794385 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fdc0044e0 con 0x7f9fe00ffd60 2023-09-30T14:38:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.619+0000 7f9fe7e1f700 1 -- 172.21.15.32:0/1584794385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe0198db0 con 0x7f9fe00ffd60 2023-09-30T14:38:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.619+0000 7f9fd6ffd700 1 -- 172.21.15.32:0/1584794385 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc003be0 con 0x7f9fe00ffd60 2023-09-30T14:38:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.619+0000 7f9fe7e1f700 1 -- 172.21.15.32:0/1584794385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe019f550 con 0x7f9fe00ffd60 2023-09-30T14:38:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.622+0000 7f9fd6ffd700 1 -- 172.21.15.32:0/1584794385 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f9fdc009bb0 con 0x7f9fe00ffd60 2023-09-30T14:38:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.622+0000 7f9fd6ffd700 1 --2- 172.21.15.32:0/1584794385 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9fcc0a1020 0x7f9fcc0a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.622+0000 7f9fe53ba700 1 --2- 172.21.15.32:0/1584794385 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9fcc0a1020 0x7f9fcc0a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.622+0000 7f9fd6ffd700 1 -- 172.21.15.32:0/1584794385 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9fdc0cc410 con 0x7f9fe00ffd60 2023-09-30T14:38:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.623+0000 7f9fe53ba700 1 --2- 172.21.15.32:0/1584794385 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9fcc0a1020 0x7f9fcc0a34e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0196b20 tx=0x7f9fd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.624+0000 7f9ff4007b80 1 Processor -- start 2023-09-30T14:38:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.624+0000 7f9ff4007b80 1 -- start start 2023-09-30T14:38:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.624+0000 7f9ff4007b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c92200 0x55eda7c92620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.624+0000 7f9ff4007b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c93310 0x55eda7c93790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.624+0000 7f9ff4007b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c93cd0 0x55eda7c9a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.624+0000 7f9ff4007b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda7bfa400 con 0x55eda7c92200 2023-09-30T14:38:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.624+0000 7f9ff4007b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda7bebbc0 con 0x55eda7c93cd0 2023-09-30T14:38:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.624+0000 7f9ff4007b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda7be94e0 con 0x55eda7c93310 2023-09-30T14:38:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.624+0000 7f9fe5bbb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c92200 0x55eda7c92620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.625+0000 7f9fe5bbb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c92200 0x55eda7c92620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35442/0 (socket says 172.21.15.32:35442) 2023-09-30T14:38:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.625+0000 7f9fe5bbb700 1 -- 172.21.15.32:0/2270201173 learned_addr learned my addr 172.21.15.32:0/2270201173 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.625+0000 7f9fe63bc700 1 --2- 172.21.15.32:0/2270201173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c93cd0 0x55eda7c9a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.625+0000 7f9fe53ba700 1 --2- 172.21.15.32:0/2270201173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c93310 0x55eda7c93790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.625+0000 7f9fe5bbb700 1 -- 172.21.15.32:0/2270201173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c93310 msgr2=0x55eda7c93790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.625+0000 7f9fe5bbb700 1 --2- 172.21.15.32:0/2270201173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c93310 0x55eda7c93790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.625+0000 7f9fe5bbb700 1 -- 172.21.15.32:0/2270201173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c93cd0 msgr2=0x55eda7c9a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.625+0000 7f9fe5bbb700 1 --2- 172.21.15.32:0/2270201173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c93cd0 0x55eda7c9a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.625+0000 7f9fe5bbb700 1 -- 172.21.15.32:0/2270201173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eda7c47870 con 0x55eda7c92200 2023-09-30T14:38:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.625+0000 7f9fe63bc700 1 --2- 172.21.15.32:0/2270201173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c93cd0 0x55eda7c9a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:38:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.625+0000 7f9fe53ba700 1 --2- 172.21.15.32:0/2270201173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c93310 0x55eda7c93790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:38:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.625+0000 7f9fe5bbb700 1 --2- 172.21.15.32:0/2270201173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c92200 0x55eda7c92620 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc095de0 tx=0x7f9fdc0997c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b2ee9eadd6764f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.625+0000 7f9fcbfff700 1 -- 172.21.15.32:0/2270201173 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0b90c0 con 0x55eda7c92200 2023-09-30T14:38:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.625+0000 7f9fcbfff700 1 -- 172.21.15.32:0/2270201173 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fdc0998e0 con 0x55eda7c92200 2023-09-30T14:38:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.625+0000 7f9fcbfff700 1 -- 172.21.15.32:0/2270201173 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc098fd0 con 0x55eda7c92200 2023-09-30T14:38:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.626+0000 7f9ff4007b80 1 -- 172.21.15.32:0/2270201173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c92200 msgr2=0x55eda7c92620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.626+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/2270201173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c92200 0x55eda7c92620 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc095de0 tx=0x7f9fdc0997c0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.626+0000 7f9ff4007b80 1 -- 172.21.15.32:0/2270201173 shutdown_connections 2023-09-30T14:38:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.626+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/2270201173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c93cd0 0x55eda7c9a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.626+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/2270201173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c92200 0x55eda7c92620 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.626+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/2270201173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c93310 0x55eda7c93790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.626+0000 7f9ff4007b80 1 -- 172.21.15.32:0/2270201173 >> 172.21.15.32:0/2270201173 conn(0x55eda7c06200 msgr2=0x55eda7c8cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.626+0000 7f9ff4007b80 1 -- 172.21.15.32:0/2270201173 shutdown_connections 2023-09-30T14:38:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.626+0000 7f9ff4007b80 1 -- 172.21.15.32:0/2270201173 wait complete. 2023-09-30T14:38:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.626+0000 7f9ff4007b80 1 -- start start 2023-09-30T14:38:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.627+0000 7f9ff4007b80 1 Processor -- start 2023-09-30T14:38:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.627+0000 7f9ff4007b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c92200 0x55eda7c9bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.627+0000 7f9ff4007b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c93310 0x55eda7c9c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.627+0000 7f9ff4007b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93cd0 0x55eda7ca3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.627+0000 7f9ff4007b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda7be4dd0 con 0x55eda7c93cd0 2023-09-30T14:38:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.627+0000 7f9ff4007b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda7b816b0 con 0x55eda7c93310 2023-09-30T14:38:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.627+0000 7f9ff4007b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda7b81350 con 0x55eda7c92200 2023-09-30T14:38:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.627+0000 7f9ff4007b80 10 client.0 _ll_register_callbacks cb 0x55eda79de7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.627+0000 7f9fe5bbb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c92200 0x55eda7c9bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.627+0000 7f9fe63bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93cd0 0x55eda7ca3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.627+0000 7f9fe5bbb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c92200 0x55eda7c9bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:49830/0 (socket says 172.21.15.32:49830) 2023-09-30T14:38:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.627+0000 7f9fe5bbb700 1 -- 172.21.15.32:0/3136504719 learned_addr learned my addr 172.21.15.32:0/3136504719 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.627+0000 7f9fe53ba700 1 --2- 172.21.15.32:0/3136504719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c93310 0x55eda7c9c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.628+0000 7f9fe63bc700 1 -- 172.21.15.32:0/3136504719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c92200 msgr2=0x55eda7c9bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.628+0000 7f9fe63bc700 1 --2- 172.21.15.32:0/3136504719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c92200 0x55eda7c9bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.628+0000 7f9fe63bc700 1 -- 172.21.15.32:0/3136504719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c93310 msgr2=0x55eda7c9c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.628+0000 7f9fe63bc700 1 --2- 172.21.15.32:0/3136504719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c93310 0x55eda7c9c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.628+0000 7f9fe63bc700 1 -- 172.21.15.32:0/3136504719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe019f550 con 0x55eda7c93cd0 2023-09-30T14:38:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.628+0000 7f9fe63bc700 1 --2- 172.21.15.32:0/3136504719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93cd0 0x55eda7ca3470 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800b770 tx=0x7f9fd80078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7feaaa120f219d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.628+0000 7f9fc97fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.628+0000 7f9fc97fa700 1 -- 172.21.15.32:0/3136504719 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd800e040 con 0x55eda7c93cd0 2023-09-30T14:38:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.628+0000 7f9fc97fa700 1 -- 172.21.15.32:0/3136504719 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9fd8004c50 con 0x55eda7c93cd0 2023-09-30T14:38:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.628+0000 7f9ff4007b80 1 -- 172.21.15.32:0/3136504719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eda7c85090 con 0x55eda7c93cd0 2023-09-30T14:38:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.628+0000 7f9fc97fa700 1 -- 172.21.15.32:0/3136504719 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fd800be60 con 0x55eda7c93cd0 2023-09-30T14:38:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.628+0000 7f9ff4007b80 10 client.15405 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.628+0000 7f9ff4007b80 1 -- 172.21.15.32:0/3136504719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55eda7a0fd80 con 0x55eda7c93cd0 2023-09-30T14:38:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.628+0000 7f9fc97fa700 1 -- 172.21.15.32:0/3136504719 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eda7c85090 con 0x55eda7c93cd0 2023-09-30T14:38:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.629+0000 7f9fc97fa700 1 -- 172.21.15.32:0/3136504719 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9fd802e070 con 0x55eda7c93cd0 2023-09-30T14:38:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.629+0000 7f9ff4007b80 10 client.15405 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.629+0000 7f9ff4007b80 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:38:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.629+0000 7f9ff4007b80 20 client.15405 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.629+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/3136504719 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55eda7cac090 0x55eda7cae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.629+0000 7f9ff4007b80 4 client.15405 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:38:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.629+0000 7f9ff4007b80 1 -- 172.21.15.32:0/3136504719 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eda7bdddb0 con 0x55eda7cac090 2023-09-30T14:38:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.629+0000 7f9fe5bbb700 1 --2- 172.21.15.32:0/3136504719 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55eda7cac090 0x55eda7cae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.629+0000 7f9fc97fa700 10 client.15405 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.629+0000 7f9fe5bbb700 1 --2- 172.21.15.32:0/3136504719 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55eda7cac090 0x55eda7cae4b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2b354284bea863c server_cookie=89e014f685828bb9 in_seq=0 out_seq=0 2023-09-30T14:38:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.630+0000 7f9fc97fa700 1 -- 172.21.15.32:0/3136504719 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55eda7bdddb0 con 0x55eda7cac090 2023-09-30T14:38:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.630+0000 7f9fc97fa700 10 client.15405 handle_command_reply: tid=0 2023-09-30T14:38:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.630+0000 7f9ff4007b80 1 client.15405 shutdown 2023-09-30T14:38:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.630+0000 7f9ff4007b80 1 -- 172.21.15.32:0/3136504719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93cd0 msgr2=0x55eda7ca3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.630+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/3136504719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93cd0 0x55eda7ca3470 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800b770 tx=0x7f9fd80078d0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.630+0000 7f9ff4007b80 1 -- 172.21.15.32:0/3136504719 shutdown_connections 2023-09-30T14:38:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.630+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/3136504719 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55eda7cac090 0x55eda7cae4b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.630+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/3136504719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93cd0 0x55eda7ca3470 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.630+0000 7f9fc97fa700 0 client.15405 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.630+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/3136504719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c92200 0x55eda7c9bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.630+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/3136504719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c93310 0x55eda7c9c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.630+0000 7f9ff4007b80 1 -- 172.21.15.32:0/3136504719 >> 172.21.15.32:0/3136504719 conn(0x55eda7c06200 msgr2=0x55eda7c8c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.630+0000 7f9ff4007b80 1 -- 172.21.15.32:0/3136504719 shutdown_connections 2023-09-30T14:38:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.630+0000 7f9ff4007b80 1 -- 172.21.15.32:0/3136504719 wait complete. 2023-09-30T14:38:55.638 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.631+0000 7f9ff4007b80 20 client.15405 trim_cache size 0 max 16384 2023-09-30T14:38:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9ff4007b80 1 Processor -- start 2023-09-30T14:38:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9ff4007b80 1 -- start start 2023-09-30T14:38:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9ff4007b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c92200 0x55eda7c96cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9ff4007b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93310 0x55eda7c97200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9ff4007b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c93cd0 0x55eda7ca3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9ff4007b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda7c9b230 con 0x55eda7c93310 2023-09-30T14:38:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9ff4007b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda7a658c0 con 0x55eda7c92200 2023-09-30T14:38:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9ff4007b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda7c9b3a0 con 0x55eda7c93cd0 2023-09-30T14:38:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9fe53ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93310 0x55eda7c97200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9fe53ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93310 0x55eda7c97200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35458/0 (socket says 172.21.15.32:35458) 2023-09-30T14:38:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9fe53ba700 1 -- 172.21.15.32:0/2646999574 learned_addr learned my addr 172.21.15.32:0/2646999574 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9fe5bbb700 1 --2- 172.21.15.32:0/2646999574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c92200 0x55eda7c96cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9fe63bc700 1 --2- 172.21.15.32:0/2646999574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c93cd0 0x55eda7ca3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9fe53ba700 1 -- 172.21.15.32:0/2646999574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c93cd0 msgr2=0x55eda7ca3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9fe53ba700 1 --2- 172.21.15.32:0/2646999574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c93cd0 0x55eda7ca3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9fe53ba700 1 -- 172.21.15.32:0/2646999574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c92200 msgr2=0x55eda7c96cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9fe53ba700 1 --2- 172.21.15.32:0/2646999574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c92200 0x55eda7c96cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.646+0000 7f9fe53ba700 1 -- 172.21.15.32:0/2646999574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eda79f8c00 con 0x55eda7c93310 2023-09-30T14:38:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.647+0000 7f9fe53ba700 1 --2- 172.21.15.32:0/2646999574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93310 0x55eda7c97200 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0010d60 tx=0x7f9fd0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5b1ea8ff091d9a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.647+0000 7f9fd4ff9700 1 -- 172.21.15.32:0/2646999574 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0012bc0 con 0x55eda7c93310 2023-09-30T14:38:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.647+0000 7f9fd4ff9700 1 -- 172.21.15.32:0/2646999574 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fd0012d20 con 0x55eda7c93310 2023-09-30T14:38:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.647+0000 7f9fd4ff9700 1 -- 172.21.15.32:0/2646999574 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd00206b0 con 0x55eda7c93310 2023-09-30T14:38:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.647+0000 7f9ff4007b80 1 -- 172.21.15.32:0/2646999574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93310 msgr2=0x55eda7c97200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.647+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/2646999574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93310 0x55eda7c97200 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0010d60 tx=0x7f9fd0013000 comp rx=0 tx=0).stop 2023-09-30T14:38:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.648+0000 7f9ff4007b80 1 -- 172.21.15.32:0/2646999574 shutdown_connections 2023-09-30T14:38:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.648+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/2646999574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c93cd0 0x55eda7ca3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.648+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/2646999574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c92200 0x55eda7c96cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.648+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/2646999574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93310 0x55eda7c97200 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.648+0000 7f9ff4007b80 1 -- 172.21.15.32:0/2646999574 >> 172.21.15.32:0/2646999574 conn(0x55eda7c06200 msgr2=0x55eda7c9a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.648+0000 7f9ff4007b80 1 -- 172.21.15.32:0/2646999574 shutdown_connections 2023-09-30T14:38:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.648+0000 7f9ff4007b80 1 -- 172.21.15.32:0/2646999574 wait complete. 2023-09-30T14:38:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.648+0000 7f9ff4007b80 1 -- start start 2023-09-30T14:38:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.648+0000 7f9ff4007b80 1 Processor -- start 2023-09-30T14:38:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.648+0000 7f9ff4007b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c92200 0x55eda7ca2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.648+0000 7f9ff4007b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c93310 0x55eda7ca3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.648+0000 7f9ff4007b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93cd0 0x55eda7ca1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.648+0000 7f9ff4007b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda7ca1c70 con 0x55eda7c93cd0 2023-09-30T14:38:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.648+0000 7f9ff4007b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda7ca1de0 con 0x55eda7c92200 2023-09-30T14:38:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.648+0000 7f9ff4007b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda7ca1fa0 con 0x55eda7c93310 2023-09-30T14:38:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.648+0000 7f9ff4007b80 10 client.0 _ll_register_callbacks cb 0x7f9fd0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:38:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9fe63bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93cd0 0x55eda7ca1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9fe63bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93cd0 0x55eda7ca1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35466/0 (socket says 172.21.15.32:35466) 2023-09-30T14:38:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9fe63bc700 1 -- 172.21.15.32:0/4021593800 learned_addr learned my addr 172.21.15.32:0/4021593800 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:38:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9fe53ba700 1 --2- 172.21.15.32:0/4021593800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c93310 0x55eda7ca3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9fe5bbb700 1 --2- 172.21.15.32:0/4021593800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c92200 0x55eda7ca2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9fe63bc700 1 -- 172.21.15.32:0/4021593800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c93310 msgr2=0x55eda7ca3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9fe63bc700 1 --2- 172.21.15.32:0/4021593800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c93310 0x55eda7ca3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9fe63bc700 1 -- 172.21.15.32:0/4021593800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c92200 msgr2=0x55eda7ca2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:38:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9fe63bc700 1 --2- 172.21.15.32:0/4021593800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c92200 0x55eda7ca2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9fe63bc700 1 -- 172.21.15.32:0/4021593800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd0010a10 con 0x55eda7c93cd0 2023-09-30T14:38:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9fe63bc700 1 --2- 172.21.15.32:0/4021593800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93cd0 0x55eda7ca1560 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55eda7c931c0 tx=0x7f9fd80055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a802ce4c0068608 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:38:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9fc97fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:38:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9fc97fa700 1 -- 172.21.15.32:0/4021593800 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd800e040 con 0x55eda7c93cd0 2023-09-30T14:38:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9fc97fa700 1 -- 172.21.15.32:0/4021593800 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9fd8004a50 con 0x55eda7c93cd0 2023-09-30T14:38:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9ff4007b80 1 -- 172.21.15.32:0/4021593800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fdc0998e0 con 0x55eda7c93cd0 2023-09-30T14:38:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9fc97fa700 1 -- 172.21.15.32:0/4021593800 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fd8005aa0 con 0x55eda7c93cd0 2023-09-30T14:38:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9ff4007b80 10 client.15411 fetch_fsmap learned FSMap version 22 2023-09-30T14:38:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.649+0000 7f9ff4007b80 1 -- 172.21.15.32:0/4021593800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55eda7a361f0 con 0x55eda7c93cd0 2023-09-30T14:38:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.650+0000 7f9fc97fa700 1 -- 172.21.15.32:0/4021593800 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fd803f080 con 0x55eda7c93cd0 2023-09-30T14:38:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.650+0000 7f9fc97fa700 1 -- 172.21.15.32:0/4021593800 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9fd802e070 con 0x55eda7c93cd0 2023-09-30T14:38:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.650+0000 7f9ff4007b80 10 client.15411 fetch_fsmap finished waiting for FSMap version 22 2023-09-30T14:38:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.650+0000 7f9ff4007b80 10 client.15411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:38:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.650+0000 7f9ff4007b80 20 client.15411 populate_metadata read hostname 'smithi032' 2023-09-30T14:38:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.650+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/4021593800 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55eda7d01150 0x55eda7d03570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:38:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.650+0000 7f9ff4007b80 4 client.15411 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:38:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.650+0000 7f9ff4007b80 1 -- 172.21.15.32:0/4021593800 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eda7bddad0 con 0x55eda7d01150 2023-09-30T14:38:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.650+0000 7f9fe5bbb700 1 --2- 172.21.15.32:0/4021593800 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55eda7d01150 0x55eda7d03570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:38:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.651+0000 7f9fc97fa700 10 client.15411 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.651+0000 7f9fe5bbb700 1 --2- 172.21.15.32:0/4021593800 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55eda7d01150 0x55eda7d03570 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cceaa4d94e590f0 server_cookie=13eda8325fc4cc31 in_seq=0 out_seq=0 2023-09-30T14:38:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.663+0000 7f9fc97fa700 1 -- 172.21.15.32:0/4021593800 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eda7bddad0 con 0x55eda7d01150 2023-09-30T14:38:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.663+0000 7f9fc97fa700 10 client.15411 handle_command_reply: tid=0 2023-09-30T14:38:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.663+0000 7f9ff4007b80 1 client.15411 shutdown 2023-09-30T14:38:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.663+0000 7f9ff4007b80 1 -- 172.21.15.32:0/4021593800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93cd0 msgr2=0x55eda7ca1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.663+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/4021593800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93cd0 0x55eda7ca1560 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55eda7c931c0 tx=0x7f9fd80055b0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.663+0000 7f9ff4007b80 1 -- 172.21.15.32:0/4021593800 shutdown_connections 2023-09-30T14:38:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.663+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/4021593800 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55eda7d01150 0x55eda7d03570 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.663+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/4021593800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eda7c93cd0 0x55eda7ca1560 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.663+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/4021593800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eda7c92200 0x55eda7ca2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.663+0000 7f9fc97fa700 0 client.15411 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:38:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.663+0000 7f9ff4007b80 1 --2- 172.21.15.32:0/4021593800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eda7c93310 0x55eda7ca3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.663+0000 7f9ff4007b80 1 -- 172.21.15.32:0/4021593800 >> 172.21.15.32:0/4021593800 conn(0x55eda7c06200 msgr2=0x55eda7c96cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.663+0000 7f9ff4007b80 1 -- 172.21.15.32:0/4021593800 shutdown_connections 2023-09-30T14:38:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.663+0000 7f9ff4007b80 1 -- 172.21.15.32:0/4021593800 wait complete. 2023-09-30T14:38:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.663+0000 7f9ff4007b80 20 client.15411 trim_cache size 0 max 16384 2023-09-30T14:38:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:38:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:38:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:38:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:38:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.664+0000 7f9fe7e1f700 1 -- 172.21.15.32:0/1584794385 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9fcc0a1020 msgr2=0x7f9fcc0a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.664+0000 7f9fe7e1f700 1 --2- 172.21.15.32:0/1584794385 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9fcc0a1020 0x7f9fcc0a34e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0196b20 tx=0x7f9fd0009260 comp rx=0 tx=0).stop 2023-09-30T14:38:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.664+0000 7f9fe7e1f700 1 -- 172.21.15.32:0/1584794385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe00ffd60 msgr2=0x7f9fe0196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:38:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.664+0000 7f9fe7e1f700 1 --2- 172.21.15.32:0/1584794385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe00ffd60 0x7f9fe0196db0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00d980 tx=0x7f9fdc00dd40 comp rx=0 tx=0).stop 2023-09-30T14:38:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.664+0000 7f9fe7e1f700 1 -- 172.21.15.32:0/1584794385 shutdown_connections 2023-09-30T14:38:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.664+0000 7f9fe7e1f700 1 --2- 172.21.15.32:0/1584794385 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9fcc0a1020 0x7f9fcc0a34e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.664+0000 7f9fe7e1f700 1 --2- 172.21.15.32:0/1584794385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fe01018f0 0x7f9fe01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.664+0000 7f9fe7e1f700 1 --2- 172.21.15.32:0/1584794385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fe00ffd60 0x7f9fe0196db0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.664+0000 7f9fe7e1f700 1 --2- 172.21.15.32:0/1584794385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fe0100f50 0x7f9fe01972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:38:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.664+0000 7f9fe7e1f700 1 -- 172.21.15.32:0/1584794385 >> 172.21.15.32:0/1584794385 conn(0x7f9fe00fb2b0 msgr2=0x7f9fe00682a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:38:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.664+0000 7f9fe7e1f700 1 -- 172.21.15.32:0/1584794385 shutdown_connections 2023-09-30T14:38:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:38:55.664+0000 7f9fe7e1f700 1 -- 172.21.15.32:0/1584794385 wait complete. 2023-09-30T14:38:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:55 smithi156 ceph-mon[103579]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2023-09-30T14:38:55.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:55 smithi148 ceph-mon[102355]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2023-09-30T14:38:55.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:55 smithi032 ceph-mon[98592]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2023-09-30T14:38:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:57 smithi156 ceph-mon[103579]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-30T14:38:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:57 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:38:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:57 smithi148 ceph-mon[102355]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-30T14:38:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:57 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:38:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:57 smithi032 ceph-mon[98592]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-30T14:38:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:57 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:38:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:38:59 smithi156 ceph-mon[103579]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 597 B/s wr, 7 op/s 2023-09-30T14:38:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:38:59 smithi148 ceph-mon[102355]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 597 B/s wr, 7 op/s 2023-09-30T14:38:59.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:38:59 smithi032 ceph-mon[98592]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 597 B/s wr, 7 op/s 2023-09-30T14:39:01.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:01 smithi156 ceph-mon[103579]: pgmap v309: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2023-09-30T14:39:01.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:01 smithi148 ceph-mon[102355]: pgmap v309: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2023-09-30T14:39:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:01 smithi032 ceph-mon[98592]: pgmap v309: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2023-09-30T14:39:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:03 smithi156 ceph-mon[103579]: pgmap v310: 97 pgs: 97 active+clean; 657 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2023-09-30T14:39:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:03 smithi148 ceph-mon[102355]: pgmap v310: 97 pgs: 97 active+clean; 657 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2023-09-30T14:39:03.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:03 smithi032 ceph-mon[98592]: pgmap v310: 97 pgs: 97 active+clean; 657 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2023-09-30T14:39:05.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:05 smithi156 ceph-mon[103579]: pgmap v311: 97 pgs: 97 active+clean; 657 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2023-09-30T14:39:05.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:05 smithi148 ceph-mon[102355]: pgmap v311: 97 pgs: 97 active+clean; 657 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2023-09-30T14:39:05.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:05 smithi032 ceph-mon[98592]: pgmap v311: 97 pgs: 97 active+clean; 657 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2023-09-30T14:39:07.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:07 smithi156 ceph-mon[103579]: pgmap v312: 97 pgs: 97 active+clean; 661 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2023-09-30T14:39:07.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:07 smithi148 ceph-mon[102355]: pgmap v312: 97 pgs: 97 active+clean; 661 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2023-09-30T14:39:07.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:07 smithi032 ceph-mon[98592]: pgmap v312: 97 pgs: 97 active+clean; 661 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2023-09-30T14:39:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.794+0000 7f02482c6700 1 -- 172.21.15.32:0/3111064797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0240104220 msgr2=0x7f0240106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.794+0000 7f02482c6700 1 --2- 172.21.15.32:0/3111064797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0240104220 0x7f0240106610 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0230009990 tx=0x7f0230009ca0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.794+0000 7f02482c6700 1 -- 172.21.15.32:0/3111064797 shutdown_connections 2023-09-30T14:39:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.794+0000 7f02482c6700 1 --2- 172.21.15.32:0/3111064797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0240106b50 0x7f0240108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.794+0000 7f02482c6700 1 --2- 172.21.15.32:0/3111064797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02401018f0 0x7f0240103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.795+0000 7f02482c6700 1 --2- 172.21.15.32:0/3111064797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0240104220 0x7f0240106610 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.795+0000 7f02482c6700 1 -- 172.21.15.32:0/3111064797 >> 172.21.15.32:0/3111064797 conn(0x7f02400fb2e0 msgr2=0x7f02400fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.795+0000 7f02482c6700 1 -- 172.21.15.32:0/3111064797 shutdown_connections 2023-09-30T14:39:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.795+0000 7f02482c6700 1 -- 172.21.15.32:0/3111064797 wait complete. 2023-09-30T14:39:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.795+0000 7f02482c6700 1 Processor -- start 2023-09-30T14:39:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.795+0000 7f02482c6700 1 -- start start 2023-09-30T14:39:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.795+0000 7f02482c6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02401018f0 0x7f02400ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.795+0000 7f02482c6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0240104220 0x7f02400ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f02482c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0240106b50 0x7f0240100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f02482c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02401a0b60 con 0x7f0240106b50 2023-09-30T14:39:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f02482c6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02401a0ca0 con 0x7f02401018f0 2023-09-30T14:39:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f02482c6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02401a0de0 con 0x7f0240104220 2023-09-30T14:39:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f0246863700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0240106b50 0x7f0240100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f0246863700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0240106b50 0x7f0240100320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35480/0 (socket says 172.21.15.32:35480) 2023-09-30T14:39:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f0246863700 1 -- 172.21.15.32:0/2758205142 learned_addr learned my addr 172.21.15.32:0/2758205142 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f0245861700 1 --2- 172.21.15.32:0/2758205142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0240104220 0x7f02400ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f0246062700 1 --2- 172.21.15.32:0/2758205142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02401018f0 0x7f02400ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f0246863700 1 -- 172.21.15.32:0/2758205142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0240104220 msgr2=0x7f02400ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f0246863700 1 --2- 172.21.15.32:0/2758205142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0240104220 0x7f02400ffde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f0246863700 1 -- 172.21.15.32:0/2758205142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02401018f0 msgr2=0x7f02400ff8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f0246863700 1 --2- 172.21.15.32:0/2758205142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02401018f0 0x7f02400ff8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f0246863700 1 -- 172.21.15.32:0/2758205142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02300095f0 con 0x7f0240106b50 2023-09-30T14:39:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f0245861700 1 --2- 172.21.15.32:0/2758205142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0240104220 0x7f02400ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f0246863700 1 --2- 172.21.15.32:0/2758205142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0240106b50 0x7f0240100320 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f023800ef30 tx=0x7f023800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.796+0000 7f02377fe700 1 -- 172.21.15.32:0/2758205142 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238018070 con 0x7f0240106b50 2023-09-30T14:39:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.797+0000 7f02482c6700 1 -- 172.21.15.32:0/2758205142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02401a1010 con 0x7f0240106b50 2023-09-30T14:39:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.797+0000 7f02377fe700 1 -- 172.21.15.32:0/2758205142 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f023800f040 con 0x7f0240106b50 2023-09-30T14:39:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.797+0000 7f02482c6700 1 -- 172.21.15.32:0/2758205142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02401a1570 con 0x7f0240106b50 2023-09-30T14:39:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.797+0000 7f02377fe700 1 -- 172.21.15.32:0/2758205142 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023800cd00 con 0x7f0240106b50 2023-09-30T14:39:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.800+0000 7f02377fe700 1 -- 172.21.15.32:0/2758205142 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f0238003a00 con 0x7f0240106b50 2023-09-30T14:39:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.800+0000 7f02544aeb80 1 Processor -- start 2023-09-30T14:39:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.801+0000 7f02544aeb80 1 -- start start 2023-09-30T14:39:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.801+0000 7f02377fe700 1 --2- 172.21.15.32:0/2758205142 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f022c0a1020 0x7f022c0a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.801+0000 7f02544aeb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7f8a10 0x563c2d7f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.801+0000 7f02544aeb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f9b20 0x563c2d7f9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.801+0000 7f02377fe700 1 -- 172.21.15.32:0/2758205142 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f02380cc110 con 0x7f0240106b50 2023-09-30T14:39:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.801+0000 7f02544aeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7fa4e0 0x563c2d800d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.801+0000 7f0245861700 1 --2- 172.21.15.32:0/2758205142 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f022c0a1020 0x7f022c0a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.801+0000 7f02544aeb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d758400 con 0x563c2d7f8a10 2023-09-30T14:39:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.801+0000 7f02544aeb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d749bc0 con 0x563c2d7f9b20 2023-09-30T14:39:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.801+0000 7f02544aeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d7474e0 con 0x563c2d7fa4e0 2023-09-30T14:39:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.801+0000 7f0246062700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7f8a10 0x563c2d7f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.801+0000 7f0246062700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7f8a10 0x563c2d7f8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35484/0 (socket says 172.21.15.32:35484) 2023-09-30T14:39:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.801+0000 7f0246062700 1 -- 172.21.15.32:0/83822252 learned_addr learned my addr 172.21.15.32:0/83822252 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.801+0000 7f0246863700 1 --2- 172.21.15.32:0/83822252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7fa4e0 0x563c2d800d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.802+0000 7f0246062700 1 -- 172.21.15.32:0/83822252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7fa4e0 msgr2=0x563c2d800d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.802+0000 7f0246062700 1 --2- 172.21.15.32:0/83822252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7fa4e0 0x563c2d800d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.802+0000 7f0245861700 1 --2- 172.21.15.32:0/83822252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f9b20 0x563c2d7f9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.802+0000 7f0246062700 1 -- 172.21.15.32:0/83822252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f9b20 msgr2=0x563c2d7f9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.802+0000 7f0246062700 1 --2- 172.21.15.32:0/83822252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f9b20 0x563c2d7f9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.802+0000 7f0246062700 1 -- 172.21.15.32:0/83822252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c2d7a5870 con 0x563c2d7f8a10 2023-09-30T14:39:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.802+0000 7f0245861700 1 --2- 172.21.15.32:0/83822252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f9b20 0x563c2d7f9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:39:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.802+0000 7f0245861700 1 --2- 172.21.15.32:0/2758205142 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f022c0a1020 0x7f022c0a34e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f023000b2e0 tx=0x7f0230004760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.802+0000 7f0246062700 1 --2- 172.21.15.32:0/83822252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7f8a10 0x563c2d7f8e30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f023c00bd30 tx=0x7f023c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6004704d06c187fe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.802+0000 7f0234d82700 1 -- 172.21.15.32:0/83822252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c010e60 con 0x563c2d7f8a10 2023-09-30T14:39:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.802+0000 7f0234d82700 1 -- 172.21.15.32:0/83822252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f023c015070 con 0x563c2d7f8a10 2023-09-30T14:39:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.802+0000 7f0234d82700 1 -- 172.21.15.32:0/83822252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c0116a0 con 0x563c2d7f8a10 2023-09-30T14:39:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.803+0000 7f02544aeb80 1 -- 172.21.15.32:0/83822252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7f8a10 msgr2=0x563c2d7f8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.803+0000 7f02544aeb80 1 --2- 172.21.15.32:0/83822252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7f8a10 0x563c2d7f8e30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f023c00bd30 tx=0x7f023c0106b0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.803+0000 7f02544aeb80 1 -- 172.21.15.32:0/83822252 shutdown_connections 2023-09-30T14:39:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.803+0000 7f02544aeb80 1 --2- 172.21.15.32:0/83822252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7fa4e0 0x563c2d800d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.803+0000 7f02544aeb80 1 --2- 172.21.15.32:0/83822252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7f8a10 0x563c2d7f8e30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.803+0000 7f02544aeb80 1 --2- 172.21.15.32:0/83822252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f9b20 0x563c2d7f9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.803+0000 7f02544aeb80 1 -- 172.21.15.32:0/83822252 >> 172.21.15.32:0/83822252 conn(0x563c2d764200 msgr2=0x563c2d7eaef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.803+0000 7f02544aeb80 1 -- 172.21.15.32:0/83822252 shutdown_connections 2023-09-30T14:39:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.803+0000 7f02544aeb80 1 -- 172.21.15.32:0/83822252 wait complete. 2023-09-30T14:39:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.804+0000 7f02544aeb80 1 -- start start 2023-09-30T14:39:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.804+0000 7f02544aeb80 1 Processor -- start 2023-09-30T14:39:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.804+0000 7f02544aeb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7f8a10 0x563c2d8026e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.804+0000 7f02544aeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7f9b20 0x563c2d802c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.804+0000 7f02544aeb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7fa4e0 0x563c2d809c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.804+0000 7f02544aeb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d742dd0 con 0x563c2d7f8a10 2023-09-30T14:39:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.804+0000 7f02544aeb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d6df6b0 con 0x563c2d7fa4e0 2023-09-30T14:39:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.804+0000 7f02544aeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d6df350 con 0x563c2d7f9b20 2023-09-30T14:39:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.804+0000 7f02544aeb80 10 client.0 _ll_register_callbacks cb 0x563c2d53c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.804+0000 7f0246062700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7f8a10 0x563c2d8026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.804+0000 7f0246062700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7f8a10 0x563c2d8026e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35490/0 (socket says 172.21.15.32:35490) 2023-09-30T14:39:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.804+0000 7f0246062700 1 -- 172.21.15.32:0/2091580248 learned_addr learned my addr 172.21.15.32:0/2091580248 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.805+0000 7f0245861700 1 --2- 172.21.15.32:0/2091580248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7f9b20 0x563c2d802c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.805+0000 7f0246863700 1 --2- 172.21.15.32:0/2091580248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7fa4e0 0x563c2d809c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.805+0000 7f0246062700 1 -- 172.21.15.32:0/2091580248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7f9b20 msgr2=0x563c2d802c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.805+0000 7f0246062700 1 --2- 172.21.15.32:0/2091580248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7f9b20 0x563c2d802c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.805+0000 7f0246062700 1 -- 172.21.15.32:0/2091580248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7fa4e0 msgr2=0x563c2d809c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.805+0000 7f0246062700 1 --2- 172.21.15.32:0/2091580248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7fa4e0 0x563c2d809c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.805+0000 7f0246062700 1 -- 172.21.15.32:0/2091580248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f023c00b960 con 0x563c2d7f8a10 2023-09-30T14:39:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.805+0000 7f0246062700 1 --2- 172.21.15.32:0/2091580248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7f8a10 0x563c2d8026e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f023c000c00 tx=0x7f023c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a68543a055b437 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.805+0000 7f0229ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.805+0000 7f0229ffb700 1 -- 172.21.15.32:0/2091580248 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c01cb30 con 0x563c2d7f8a10 2023-09-30T14:39:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.805+0000 7f0229ffb700 1 -- 172.21.15.32:0/2091580248 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f023c017070 con 0x563c2d7f8a10 2023-09-30T14:39:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.805+0000 7f02544aeb80 1 -- 172.21.15.32:0/2091580248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c2d7e3090 con 0x563c2d7f8a10 2023-09-30T14:39:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.805+0000 7f0229ffb700 1 -- 172.21.15.32:0/2091580248 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f023c011440 con 0x563c2d7f8a10 2023-09-30T14:39:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.805+0000 7f02544aeb80 10 client.15423 fetch_fsmap learned FSMap version 23 2023-09-30T14:39:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.805+0000 7f02544aeb80 1 -- 172.21.15.32:0/2091580248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563c2d56dd80 con 0x563c2d7f8a10 2023-09-30T14:39:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.806+0000 7f0229ffb700 1 -- 172.21.15.32:0/2091580248 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563c2d7e3090 con 0x563c2d7f8a10 2023-09-30T14:39:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.806+0000 7f0229ffb700 1 -- 172.21.15.32:0/2091580248 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f023c0276c0 con 0x563c2d7f8a10 2023-09-30T14:39:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.806+0000 7f02544aeb80 10 client.15423 fetch_fsmap finished waiting for FSMap version 23 2023-09-30T14:39:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.806+0000 7f02544aeb80 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.806+0000 7f02544aeb80 20 client.15423 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.806+0000 7f02544aeb80 1 --2- 172.21.15.32:0/2091580248 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563c2d80e410 0x563c2d810830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.806+0000 7f02544aeb80 4 client.15423 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:39:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.806+0000 7f02544aeb80 1 -- 172.21.15.32:0/2091580248 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c2d73bdb0 con 0x563c2d80e410 2023-09-30T14:39:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.806+0000 7f0245861700 1 --2- 172.21.15.32:0/2091580248 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563c2d80e410 0x563c2d810830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.807+0000 7f0229ffb700 10 client.15423 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.807+0000 7f0245861700 1 --2- 172.21.15.32:0/2091580248 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563c2d80e410 0x563c2d810830 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a582be123f10de2a server_cookie=fcf9ae09643f00df in_seq=0 out_seq=0 2023-09-30T14:39:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.807+0000 7f0229ffb700 1 -- 172.21.15.32:0/2091580248 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563c2d73bdb0 con 0x563c2d80e410 2023-09-30T14:39:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.807+0000 7f0229ffb700 10 client.15423 handle_command_reply: tid=0 2023-09-30T14:39:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.807+0000 7f02544aeb80 1 client.15423 shutdown 2023-09-30T14:39:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.807+0000 7f02544aeb80 1 -- 172.21.15.32:0/2091580248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7f8a10 msgr2=0x563c2d8026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.807+0000 7f02544aeb80 1 --2- 172.21.15.32:0/2091580248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7f8a10 0x563c2d8026e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f023c000c00 tx=0x7f023c01c480 comp rx=0 tx=0).stop 2023-09-30T14:39:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.807+0000 7f02544aeb80 1 -- 172.21.15.32:0/2091580248 shutdown_connections 2023-09-30T14:39:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.807+0000 7f02544aeb80 1 --2- 172.21.15.32:0/2091580248 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563c2d80e410 0x563c2d810830 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.808+0000 7f02544aeb80 1 --2- 172.21.15.32:0/2091580248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7fa4e0 0x563c2d809c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.808+0000 7f02544aeb80 1 --2- 172.21.15.32:0/2091580248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7f8a10 0x563c2d8026e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.808+0000 7f02544aeb80 1 --2- 172.21.15.32:0/2091580248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7f9b20 0x563c2d802c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.808+0000 7f02544aeb80 1 -- 172.21.15.32:0/2091580248 >> 172.21.15.32:0/2091580248 conn(0x563c2d764200 msgr2=0x563c2d7ea590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.808+0000 7f0229ffb700 0 client.15423 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.808+0000 7f02544aeb80 1 -- 172.21.15.32:0/2091580248 shutdown_connections 2023-09-30T14:39:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.808+0000 7f02544aeb80 1 -- 172.21.15.32:0/2091580248 wait complete. 2023-09-30T14:39:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.808+0000 7f02544aeb80 20 client.15423 trim_cache size 0 max 16384 2023-09-30T14:39:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.822+0000 7f02544aeb80 1 Processor -- start 2023-09-30T14:39:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f02544aeb80 1 -- start start 2023-09-30T14:39:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f02544aeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7f8a10 0x563c2d810420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f02544aeb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f9b20 0x563c2d809b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f02544aeb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7fa4e0 0x563c2d8008a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f02544aeb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d5c38c0 con 0x563c2d7fa4e0 2023-09-30T14:39:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f02544aeb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d7ec9e0 con 0x563c2d7f9b20 2023-09-30T14:39:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f02544aeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d800650 con 0x563c2d7f8a10 2023-09-30T14:39:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f0246062700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7f8a10 0x563c2d810420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f0245861700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f9b20 0x563c2d809b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f0245861700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f9b20 0x563c2d809b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:46976/0 (socket says 172.21.15.32:46976) 2023-09-30T14:39:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f0245861700 1 -- 172.21.15.32:0/4026851738 learned_addr learned my addr 172.21.15.32:0/4026851738 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f0246863700 1 --2- 172.21.15.32:0/4026851738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7fa4e0 0x563c2d8008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f0246062700 1 -- 172.21.15.32:0/4026851738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f9b20 msgr2=0x563c2d809b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f0246062700 1 --2- 172.21.15.32:0/4026851738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f9b20 0x563c2d809b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f0246062700 1 -- 172.21.15.32:0/4026851738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7fa4e0 msgr2=0x563c2d8008a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f0246062700 1 --2- 172.21.15.32:0/4026851738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7fa4e0 0x563c2d8008a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f0246062700 1 -- 172.21.15.32:0/4026851738 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c2d556c00 con 0x563c2d7f8a10 2023-09-30T14:39:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.823+0000 7f0246863700 1 --2- 172.21.15.32:0/4026851738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7fa4e0 0x563c2d8008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:39:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.824+0000 7f0246062700 1 --2- 172.21.15.32:0/4026851738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7f8a10 0x563c2d810420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f023c027a70 tx=0x7f023c005890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=661fde8fd41e88d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.824+0000 7f0235583700 1 -- 172.21.15.32:0/4026851738 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c031220 con 0x563c2d7f8a10 2023-09-30T14:39:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.824+0000 7f0235583700 1 -- 172.21.15.32:0/4026851738 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f023c0278b0 con 0x563c2d7f8a10 2023-09-30T14:39:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.824+0000 7f0235583700 1 -- 172.21.15.32:0/4026851738 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c005550 con 0x563c2d7f8a10 2023-09-30T14:39:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.825+0000 7f02544aeb80 1 -- 172.21.15.32:0/4026851738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7f8a10 msgr2=0x563c2d810420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.825+0000 7f02544aeb80 1 --2- 172.21.15.32:0/4026851738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7f8a10 0x563c2d810420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f023c027a70 tx=0x7f023c005890 comp rx=0 tx=0).stop 2023-09-30T14:39:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.825+0000 7f02544aeb80 1 -- 172.21.15.32:0/4026851738 shutdown_connections 2023-09-30T14:39:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.825+0000 7f02544aeb80 1 --2- 172.21.15.32:0/4026851738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7fa4e0 0x563c2d8008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.825+0000 7f02544aeb80 1 --2- 172.21.15.32:0/4026851738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7f8a10 0x563c2d810420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.825+0000 7f02544aeb80 1 --2- 172.21.15.32:0/4026851738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f9b20 0x563c2d809b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.825+0000 7f02544aeb80 1 -- 172.21.15.32:0/4026851738 >> 172.21.15.32:0/4026851738 conn(0x563c2d764200 msgr2=0x563c2d815190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.825+0000 7f02544aeb80 1 -- 172.21.15.32:0/4026851738 shutdown_connections 2023-09-30T14:39:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.825+0000 7f02544aeb80 1 -- 172.21.15.32:0/4026851738 wait complete. 2023-09-30T14:39:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.825+0000 7f02544aeb80 1 -- start start 2023-09-30T14:39:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.825+0000 7f02544aeb80 1 Processor -- start 2023-09-30T14:39:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.825+0000 7f02544aeb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f8a10 0x563c2d807480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.825+0000 7f02544aeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7f9b20 0x563c2d8079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.826+0000 7f02544aeb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7fa4e0 0x563c2d8089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.826+0000 7f02544aeb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d808f10 con 0x563c2d7fa4e0 2023-09-30T14:39:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.826+0000 7f02544aeb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d809080 con 0x563c2d7f8a10 2023-09-30T14:39:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.826+0000 7f02544aeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d809240 con 0x563c2d7f9b20 2023-09-30T14:39:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.826+0000 7f02544aeb80 10 client.0 _ll_register_callbacks cb 0x7f023c00bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.826+0000 7f0246062700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f8a10 0x563c2d807480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.826+0000 7f0246062700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f8a10 0x563c2d807480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:46980/0 (socket says 172.21.15.32:46980) 2023-09-30T14:39:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.826+0000 7f0246062700 1 -- 172.21.15.32:0/1789786696 learned_addr learned my addr 172.21.15.32:0/1789786696 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.826+0000 7f0246863700 1 --2- 172.21.15.32:0/1789786696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7fa4e0 0x563c2d8089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.826+0000 7f0246062700 1 -- 172.21.15.32:0/1789786696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7f9b20 msgr2=0x563c2d8079c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.826+0000 7f0246062700 1 --2- 172.21.15.32:0/1789786696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7f9b20 0x563c2d8079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.826+0000 7f0246062700 1 -- 172.21.15.32:0/1789786696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7fa4e0 msgr2=0x563c2d8089d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.826+0000 7f0246062700 1 --2- 172.21.15.32:0/1789786696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7fa4e0 0x563c2d8089d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.826+0000 7f0246062700 1 -- 172.21.15.32:0/1789786696 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c2d56dd80 con 0x563c2d7f8a10 2023-09-30T14:39:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.826+0000 7f0246062700 1 --2- 172.21.15.32:0/1789786696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f8a10 0x563c2d807480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f023c01def0 tx=0x7f023c010ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eda5fc70a22c4fec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.826+0000 7f0229ffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:39:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.827+0000 7f0229ffb700 1 -- 172.21.15.32:0/1789786696 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c03e030 con 0x563c2d7f8a10 2023-09-30T14:39:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.827+0000 7f02544aeb80 1 -- 172.21.15.32:0/1789786696 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f023c015070 con 0x563c2d7f8a10 2023-09-30T14:39:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.827+0000 7f0229ffb700 1 -- 172.21.15.32:0/1789786696 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f023c03f070 con 0x563c2d7f8a10 2023-09-30T14:39:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.827+0000 7f0229ffb700 1 -- 172.21.15.32:0/1789786696 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f023c01cb30 con 0x563c2d7f8a10 2023-09-30T14:39:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.827+0000 7f02544aeb80 10 client.25139 fetch_fsmap learned FSMap version 23 2023-09-30T14:39:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.827+0000 7f02544aeb80 1 -- 172.21.15.32:0/1789786696 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563c2d5941f0 con 0x563c2d7f8a10 2023-09-30T14:39:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.827+0000 7f0229ffb700 1 -- 172.21.15.32:0/1789786696 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f023c049020 con 0x563c2d7f8a10 2023-09-30T14:39:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.827+0000 7f0229ffb700 1 -- 172.21.15.32:0/1789786696 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f023c011440 con 0x563c2d7f8a10 2023-09-30T14:39:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.827+0000 7f02544aeb80 10 client.25139 fetch_fsmap finished waiting for FSMap version 23 2023-09-30T14:39:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.827+0000 7f02544aeb80 10 client.25139 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.827+0000 7f02544aeb80 20 client.25139 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.827+0000 7f02544aeb80 1 --2- 172.21.15.32:0/1789786696 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563c2d8628f0 0x563c2d864d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.827+0000 7f02544aeb80 4 client.25139 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:39:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.827+0000 7f02544aeb80 1 -- 172.21.15.32:0/1789786696 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563c2d73bad0 con 0x563c2d8628f0 2023-09-30T14:39:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.828+0000 7f0245861700 1 --2- 172.21.15.32:0/1789786696 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563c2d8628f0 0x563c2d864d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.828+0000 7f0229ffb700 10 client.25139 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.828+0000 7f0245861700 1 --2- 172.21.15.32:0/1789786696 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563c2d8628f0 0x563c2d864d10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c4b3c848dcff6b server_cookie=3c75e3d54daadd88 in_seq=0 out_seq=0 2023-09-30T14:39:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.839+0000 7f0229ffb700 1 -- 172.21.15.32:0/1789786696 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563c2d73bad0 con 0x563c2d8628f0 2023-09-30T14:39:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.839+0000 7f0229ffb700 10 client.25139 handle_command_reply: tid=0 2023-09-30T14:39:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.839+0000 7f02544aeb80 1 client.25139 shutdown 2023-09-30T14:39:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.839+0000 7f02544aeb80 1 -- 172.21.15.32:0/1789786696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f8a10 msgr2=0x563c2d807480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.839+0000 7f02544aeb80 1 --2- 172.21.15.32:0/1789786696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f8a10 0x563c2d807480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f023c01def0 tx=0x7f023c010ee0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.839+0000 7f02544aeb80 1 -- 172.21.15.32:0/1789786696 shutdown_connections 2023-09-30T14:39:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.839+0000 7f02544aeb80 1 --2- 172.21.15.32:0/1789786696 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563c2d8628f0 0x563c2d864d10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.839+0000 7f02544aeb80 1 --2- 172.21.15.32:0/1789786696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c2d7fa4e0 0x563c2d8089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.839+0000 7f02544aeb80 1 --2- 172.21.15.32:0/1789786696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c2d7f8a10 0x563c2d807480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.839+0000 7f02544aeb80 1 --2- 172.21.15.32:0/1789786696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c2d7f9b20 0x563c2d8079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.840+0000 7f02544aeb80 1 -- 172.21.15.32:0/1789786696 >> 172.21.15.32:0/1789786696 conn(0x563c2d764200 msgr2=0x563c2d810420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.840+0000 7f0229ffb700 0 client.25139 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.840+0000 7f02544aeb80 1 -- 172.21.15.32:0/1789786696 shutdown_connections 2023-09-30T14:39:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.840+0000 7f02544aeb80 1 -- 172.21.15.32:0/1789786696 wait complete. 2023-09-30T14:39:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.840+0000 7f02544aeb80 20 client.25139 trim_cache size 0 max 16384 2023-09-30T14:39:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:39:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:39:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:39:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:39:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.840+0000 7f02482c6700 1 -- 172.21.15.32:0/2758205142 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f022c0a1020 msgr2=0x7f022c0a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.840+0000 7f02482c6700 1 --2- 172.21.15.32:0/2758205142 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f022c0a1020 0x7f022c0a34e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f023000b2e0 tx=0x7f0230004760 comp rx=0 tx=0).stop 2023-09-30T14:39:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.840+0000 7f02482c6700 1 -- 172.21.15.32:0/2758205142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0240106b50 msgr2=0x7f0240100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.840+0000 7f02482c6700 1 --2- 172.21.15.32:0/2758205142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0240106b50 0x7f0240100320 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f023800ef30 tx=0x7f023800c3a0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.841+0000 7f02482c6700 1 -- 172.21.15.32:0/2758205142 shutdown_connections 2023-09-30T14:39:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.841+0000 7f02482c6700 1 --2- 172.21.15.32:0/2758205142 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f022c0a1020 0x7f022c0a34e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.841+0000 7f02482c6700 1 --2- 172.21.15.32:0/2758205142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0240106b50 0x7f0240100320 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.841+0000 7f02482c6700 1 --2- 172.21.15.32:0/2758205142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02401018f0 0x7f02400ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.841+0000 7f02482c6700 1 --2- 172.21.15.32:0/2758205142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0240104220 0x7f02400ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.841+0000 7f02482c6700 1 -- 172.21.15.32:0/2758205142 >> 172.21.15.32:0/2758205142 conn(0x7f02400fb2e0 msgr2=0x7f0240102560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.841+0000 7f02482c6700 1 -- 172.21.15.32:0/2758205142 shutdown_connections 2023-09-30T14:39:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:08.841+0000 7f02482c6700 1 -- 172.21.15.32:0/2758205142 wait complete. 2023-09-30T14:39:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:09 smithi156 ceph-mon[103579]: pgmap v313: 97 pgs: 97 active+clean; 661 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2023-09-30T14:39:09.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:09 smithi148 ceph-mon[102355]: pgmap v313: 97 pgs: 97 active+clean; 661 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2023-09-30T14:39:09.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:09 smithi032 ceph-mon[98592]: pgmap v313: 97 pgs: 97 active+clean; 661 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2023-09-30T14:39:11.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:11 smithi156 ceph-mon[103579]: pgmap v314: 97 pgs: 97 active+clean; 662 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2023-09-30T14:39:11.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:11 smithi148 ceph-mon[102355]: pgmap v314: 97 pgs: 97 active+clean; 662 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2023-09-30T14:39:11.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:11 smithi032 ceph-mon[98592]: pgmap v314: 97 pgs: 97 active+clean; 662 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2023-09-30T14:39:12.867 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-30T14:39:13.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.046+0000 7fc5c9170700 1 -- 172.21.15.32:0/3120538874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c4100e70 msgr2=0x7fc5c4101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.046+0000 7fc5c9170700 1 --2- 172.21.15.32:0/3120538874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c4100e70 0x7fc5c4101290 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009990 tx=0x7fc5b4009ca0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.046+0000 7fc5c9170700 1 -- 172.21.15.32:0/3120538874 shutdown_connections 2023-09-30T14:39:13.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.046+0000 7fc5c9170700 1 --2- 172.21.15.32:0/3120538874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5c4102980 0x7fc5c41092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.046+0000 7fc5c9170700 1 --2- 172.21.15.32:0/3120538874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c4100e70 0x7fc5c4101290 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.046+0000 7fc5c9170700 1 --2- 172.21.15.32:0/3120538874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5c4101fc0 0x7fc5c4102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.046+0000 7fc5c9170700 1 -- 172.21.15.32:0/3120538874 >> 172.21.15.32:0/3120538874 conn(0x7fc5c40fc3e0 msgr2=0x7fc5c40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.046+0000 7fc5c9170700 1 -- 172.21.15.32:0/3120538874 shutdown_connections 2023-09-30T14:39:13.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.046+0000 7fc5c9170700 1 -- 172.21.15.32:0/3120538874 wait complete. 2023-09-30T14:39:13.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.047+0000 7fc5c9170700 1 Processor -- start 2023-09-30T14:39:13.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.047+0000 7fc5c9170700 1 -- start start 2023-09-30T14:39:13.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.047+0000 7fc5c9170700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5c4100e70 0x7fc5c4196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.047+0000 7fc5c9170700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5c4101fc0 0x7fc5c4197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.047+0000 7fc5c9170700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c4102980 0x7fc5c419e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.047+0000 7fc5c9170700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c419e6d0 con 0x7fc5c4102980 2023-09-30T14:39:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.047+0000 7fc5c9170700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c419e810 con 0x7fc5c4100e70 2023-09-30T14:39:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.047+0000 7fc5c9170700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c419e9a0 con 0x7fc5c4101fc0 2023-09-30T14:39:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.048+0000 7fc5c2d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5c4100e70 0x7fc5c4196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.048+0000 7fc5c2d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5c4100e70 0x7fc5c4196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:46994/0 (socket says 172.21.15.32:46994) 2023-09-30T14:39:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.048+0000 7fc5c2d9d700 1 -- 172.21.15.32:0/94661580 learned_addr learned my addr 172.21.15.32:0/94661580 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.048+0000 7fc5c359e700 1 --2- 172.21.15.32:0/94661580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c4102980 0x7fc5c419e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.048+0000 7fc5c2d9d700 1 -- 172.21.15.32:0/94661580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5c4101fc0 msgr2=0x7fc5c4197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.048+0000 7fc5c2d9d700 1 --2- 172.21.15.32:0/94661580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5c4101fc0 0x7fc5c4197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.048+0000 7fc5c2d9d700 1 -- 172.21.15.32:0/94661580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c4102980 msgr2=0x7fc5c419e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.048+0000 7fc5c2d9d700 1 --2- 172.21.15.32:0/94661580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c4102980 0x7fc5c419e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.048+0000 7fc5c2d9d700 1 -- 172.21.15.32:0/94661580 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b40095f0 con 0x7fc5c4100e70 2023-09-30T14:39:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.048+0000 7fc5c2d9d700 1 --2- 172.21.15.32:0/94661580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5c4100e70 0x7fc5c4196b50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4005910 tx=0x7fc5b4005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.048+0000 7fc5abfff700 1 -- 172.21.15.32:0/94661580 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4003a90 con 0x7fc5c4100e70 2023-09-30T14:39:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.048+0000 7fc5abfff700 1 -- 172.21.15.32:0/94661580 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5b4023070 con 0x7fc5c4100e70 2023-09-30T14:39:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.049+0000 7fc5c9170700 1 -- 172.21.15.32:0/94661580 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c4073050 con 0x7fc5c4100e70 2023-09-30T14:39:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.049+0000 7fc5abfff700 1 -- 172.21.15.32:0/94661580 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b401fd90 con 0x7fc5c4100e70 2023-09-30T14:39:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.049+0000 7fc5c9170700 1 -- 172.21.15.32:0/94661580 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c4073500 con 0x7fc5c4100e70 2023-09-30T14:39:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.051+0000 7fc5abfff700 1 -- 172.21.15.32:0/94661580 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fc5b4025030 con 0x7fc5c4100e70 2023-09-30T14:39:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.052+0000 7fc5abfff700 1 --2- 172.21.15.32:0/94661580 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5ac0a0fa0 0x7fc5ac0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.052+0000 7fc5abfff700 1 -- 172.21.15.32:0/94661580 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc5b40d1b00 con 0x7fc5c4100e70 2023-09-30T14:39:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.052+0000 7fc5c259c700 1 --2- 172.21.15.32:0/94661580 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5ac0a0fa0 0x7fc5ac0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.053+0000 7fc5c259c700 1 --2- 172.21.15.32:0/94661580 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5ac0a0fa0 0x7fc5ac0a3460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8005950 tx=0x7fc5b80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.053+0000 7fc5d5358b80 1 Processor -- start 2023-09-30T14:39:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.053+0000 7fc5d5358b80 1 -- start start 2023-09-30T14:39:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.053+0000 7fc5d5358b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623cae99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.053+0000 7fc5d5358b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623cae9f00 0x55623caea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.053+0000 7fc5d5358b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623caeada0 0x55623caeb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.053+0000 7fc5d5358b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623ca51460 con 0x55623caeada0 2023-09-30T14:39:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.053+0000 7fc5d5358b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623ca42c20 con 0x55623cae9f00 2023-09-30T14:39:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.053+0000 7fc5d5358b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623ca40540 con 0x55623cae95a0 2023-09-30T14:39:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.053+0000 7fc5c359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623caeada0 0x55623caeb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.053+0000 7fc5c359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623caeada0 0x55623caeb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35528/0 (socket says 172.21.15.32:35528) 2023-09-30T14:39:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.053+0000 7fc5c359e700 1 -- 172.21.15.32:0/254010956 learned_addr learned my addr 172.21.15.32:0/254010956 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.053+0000 7fc5c2d9d700 1 --2- 172.21.15.32:0/254010956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623cae99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.054+0000 7fc5c259c700 1 --2- 172.21.15.32:0/254010956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623cae9f00 0x55623caea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.054+0000 7fc5c359e700 1 -- 172.21.15.32:0/254010956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 msgr2=0x55623cae99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.054+0000 7fc5c359e700 1 --2- 172.21.15.32:0/254010956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623cae99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.054+0000 7fc5c359e700 1 -- 172.21.15.32:0/254010956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623cae9f00 msgr2=0x55623caea7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.054+0000 7fc5c359e700 1 --2- 172.21.15.32:0/254010956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623cae9f00 0x55623caea7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.054+0000 7fc5c359e700 1 -- 172.21.15.32:0/254010956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55623ca9e8d0 con 0x55623caeada0 2023-09-30T14:39:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.054+0000 7fc5c2d9d700 1 --2- 172.21.15.32:0/254010956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623cae99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.054+0000 7fc5c359e700 1 --2- 172.21.15.32:0/254010956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623caeada0 0x55623caeb220 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0098f0 tx=0x7fc5bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8dfefa34885111 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.054+0000 7fc5a97fa700 1 -- 172.21.15.32:0/254010956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc019070 con 0x55623caeada0 2023-09-30T14:39:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.054+0000 7fc5a97fa700 1 -- 172.21.15.32:0/254010956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5bc00f040 con 0x55623caeada0 2023-09-30T14:39:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.054+0000 7fc5a97fa700 1 -- 172.21.15.32:0/254010956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc007aa0 con 0x55623caeada0 2023-09-30T14:39:13.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.055+0000 7fc5d5358b80 1 -- 172.21.15.32:0/254010956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623caeada0 msgr2=0x55623caeb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.055+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/254010956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623caeada0 0x55623caeb220 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0098f0 tx=0x7fc5bc009c00 comp rx=0 tx=0).stop 2023-09-30T14:39:13.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.055+0000 7fc5d5358b80 1 -- 172.21.15.32:0/254010956 shutdown_connections 2023-09-30T14:39:13.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.055+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/254010956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623caeada0 0x55623caeb220 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.055+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/254010956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623cae99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.055+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/254010956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623cae9f00 0x55623caea7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.055+0000 7fc5d5358b80 1 -- 172.21.15.32:0/254010956 >> 172.21.15.32:0/254010956 conn(0x55623ca5d260 msgr2=0x55623cae4480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.055+0000 7fc5d5358b80 1 -- 172.21.15.32:0/254010956 shutdown_connections 2023-09-30T14:39:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.055+0000 7fc5d5358b80 1 -- 172.21.15.32:0/254010956 wait complete. 2023-09-30T14:39:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.056+0000 7fc5d5358b80 1 -- start start 2023-09-30T14:39:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.056+0000 7fc5d5358b80 1 Processor -- start 2023-09-30T14:39:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.056+0000 7fc5d5358b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623caf3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.056+0000 7fc5d5358b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623cae9f00 0x55623caf38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.056+0000 7fc5d5358b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623caeada0 0x55623cafa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.056+0000 7fc5d5358b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623ca3be30 con 0x55623cae9f00 2023-09-30T14:39:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.056+0000 7fc5d5358b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623c9d8710 con 0x55623caeada0 2023-09-30T14:39:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.056+0000 7fc5d5358b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623c9d83b0 con 0x55623cae95a0 2023-09-30T14:39:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.056+0000 7fc5d5358b80 10 client.0 _ll_register_callbacks cb 0x55623c835800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.056+0000 7fc5c2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623caf3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.057+0000 7fc5c259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623cae9f00 0x55623caf38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.057+0000 7fc5c2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623caf3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:49914/0 (socket says 172.21.15.32:49914) 2023-09-30T14:39:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.057+0000 7fc5c2d9d700 1 -- 172.21.15.32:0/3732837180 learned_addr learned my addr 172.21.15.32:0/3732837180 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.057+0000 7fc5c359e700 1 --2- 172.21.15.32:0/3732837180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623caeada0 0x55623cafa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.057+0000 7fc5c2d9d700 1 -- 172.21.15.32:0/3732837180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623caeada0 msgr2=0x55623cafa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.057+0000 7fc5c2d9d700 1 --2- 172.21.15.32:0/3732837180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623caeada0 0x55623cafa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.057+0000 7fc5c2d9d700 1 -- 172.21.15.32:0/3732837180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623cae9f00 msgr2=0x55623caf38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.057+0000 7fc5c2d9d700 1 --2- 172.21.15.32:0/3732837180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623cae9f00 0x55623caf38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.057+0000 7fc5c2d9d700 1 -- 172.21.15.32:0/3732837180 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c4073500 con 0x55623cae95a0 2023-09-30T14:39:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.057+0000 7fc5c2d9d700 1 --2- 172.21.15.32:0/3732837180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623caf3360 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40cf660 tx=0x7fc5b4031000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4a57e4fa77c9c17 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.057+0000 7fc5a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:39:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.057+0000 7fc5a6ff5700 1 -- 172.21.15.32:0/3732837180 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b40a52b0 con 0x55623cae95a0 2023-09-30T14:39:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.057+0000 7fc5a6ff5700 1 -- 172.21.15.32:0/3732837180 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc5b409f060 con 0x55623cae95a0 2023-09-30T14:39:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.058+0000 7fc5d5358b80 1 -- 172.21.15.32:0/3732837180 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55623cadc250 con 0x55623cae95a0 2023-09-30T14:39:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.058+0000 7fc5a6ff5700 1 -- 172.21.15.32:0/3732837180 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5b40a31b0 con 0x55623cae95a0 2023-09-30T14:39:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.058+0000 7fc5d5358b80 10 client.25129 fetch_fsmap learned FSMap version 23 2023-09-30T14:39:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.058+0000 7fc5d5358b80 1 -- 172.21.15.32:0/3732837180 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fc5bc003c50 con 0x55623cae95a0 2023-09-30T14:39:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.058+0000 7fc5a6ff5700 1 -- 172.21.15.32:0/3732837180 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5b40bb030 con 0x55623cae95a0 2023-09-30T14:39:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.059+0000 7fc5a6ff5700 1 -- 172.21.15.32:0/3732837180 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc5b409e670 con 0x55623cae95a0 2023-09-30T14:39:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.059+0000 7fc5d5358b80 10 client.25129 fetch_fsmap finished waiting for FSMap version 23 2023-09-30T14:39:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.059+0000 7fc5d5358b80 10 client.25129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.059+0000 7fc5d5358b80 20 client.25129 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.059+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/3732837180 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55623cb07610 0x55623cb09a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.059+0000 7fc5d5358b80 4 client.25129 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:39:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.059+0000 7fc5d5358b80 1 -- 172.21.15.32:0/3732837180 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55623ca34e10 con 0x55623cb07610 2023-09-30T14:39:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.060+0000 7fc5c259c700 1 --2- 172.21.15.32:0/3732837180 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55623cb07610 0x55623cb09a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.060+0000 7fc5c259c700 1 --2- 172.21.15.32:0/3732837180 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55623cb07610 0x55623cb09a30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e0e4695de41869e server_cookie=3106ebe336b60e34 in_seq=0 out_seq=0 2023-09-30T14:39:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.060+0000 7fc5a6ff5700 10 client.25129 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.061+0000 7fc5a6ff5700 1 -- 172.21.15.32:0/3732837180 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55623ca34e10 con 0x55623cb07610 2023-09-30T14:39:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.061+0000 7fc5a6ff5700 10 client.25129 handle_command_reply: tid=0 2023-09-30T14:39:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.061+0000 7fc5d5358b80 1 client.25129 shutdown 2023-09-30T14:39:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.061+0000 7fc5d5358b80 1 -- 172.21.15.32:0/3732837180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 msgr2=0x55623caf3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.061+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/3732837180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623caf3360 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40cf660 tx=0x7fc5b4031000 comp rx=0 tx=0).stop 2023-09-30T14:39:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.061+0000 7fc5d5358b80 1 -- 172.21.15.32:0/3732837180 shutdown_connections 2023-09-30T14:39:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.061+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/3732837180 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55623cb07610 0x55623cb09a30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.061+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/3732837180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623caeada0 0x55623cafa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.061+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/3732837180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623caf3360 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.061+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/3732837180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623cae9f00 0x55623caf38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.061+0000 7fc5d5358b80 1 -- 172.21.15.32:0/3732837180 >> 172.21.15.32:0/3732837180 conn(0x55623ca5d260 msgr2=0x55623cae4290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.061+0000 7fc5a6ff5700 0 client.25129 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.061+0000 7fc5d5358b80 1 -- 172.21.15.32:0/3732837180 shutdown_connections 2023-09-30T14:39:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.061+0000 7fc5d5358b80 1 -- 172.21.15.32:0/3732837180 wait complete. 2023-09-30T14:39:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.061+0000 7fc5d5358b80 20 client.25129 trim_cache size 0 max 16384 2023-09-30T14:39:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.079+0000 7fc5d5358b80 1 Processor -- start 2023-09-30T14:39:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.079+0000 7fc5d5358b80 1 -- start start 2023-09-30T14:39:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.079+0000 7fc5d5358b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623cae4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.079+0000 7fc5d5358b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623cae9f00 0x55623cafa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.079+0000 7fc5d5358b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623caeada0 0x55623caf1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.079+0000 7fc5d5358b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623c8bc920 con 0x55623cae9f00 2023-09-30T14:39:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.079+0000 7fc5d5358b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623cae0d60 con 0x55623caeada0 2023-09-30T14:39:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.079+0000 7fc5d5358b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623caeac10 con 0x55623cae95a0 2023-09-30T14:39:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.079+0000 7fc5c2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623cae4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.079+0000 7fc5c2d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623cae4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:49922/0 (socket says 172.21.15.32:49922) 2023-09-30T14:39:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.079+0000 7fc5c2d9d700 1 -- 172.21.15.32:0/2566140637 learned_addr learned my addr 172.21.15.32:0/2566140637 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.079+0000 7fc5c259c700 1 --2- 172.21.15.32:0/2566140637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623cae9f00 0x55623cafa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.079+0000 7fc5c359e700 1 --2- 172.21.15.32:0/2566140637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623caeada0 0x55623caf1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.080+0000 7fc5c2d9d700 1 -- 172.21.15.32:0/2566140637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623caeada0 msgr2=0x55623caf1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.080+0000 7fc5c2d9d700 1 --2- 172.21.15.32:0/2566140637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623caeada0 0x55623caf1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.080+0000 7fc5c2d9d700 1 -- 172.21.15.32:0/2566140637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623cae9f00 msgr2=0x55623cafa8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.080+0000 7fc5c2d9d700 1 --2- 172.21.15.32:0/2566140637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623cae9f00 0x55623cafa8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.080+0000 7fc5c2d9d700 1 -- 172.21.15.32:0/2566140637 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55623c866de0 con 0x55623cae95a0 2023-09-30T14:39:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.080+0000 7fc5c359e700 1 --2- 172.21.15.32:0/2566140637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623caeada0 0x55623caf1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:39:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.080+0000 7fc5c2d9d700 1 --2- 172.21.15.32:0/2566140637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623cae4290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc5b401ffd0 tx=0x7fc5b4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1693dd9a314f08a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.080+0000 7fc5a9ffb700 1 -- 172.21.15.32:0/2566140637 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b40a52b0 con 0x55623cae95a0 2023-09-30T14:39:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.080+0000 7fc5a9ffb700 1 -- 172.21.15.32:0/2566140637 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5b40bb030 con 0x55623cae95a0 2023-09-30T14:39:13.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.080+0000 7fc5a9ffb700 1 -- 172.21.15.32:0/2566140637 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b409e7b0 con 0x55623cae95a0 2023-09-30T14:39:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.081+0000 7fc5d5358b80 1 -- 172.21.15.32:0/2566140637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 msgr2=0x55623cae4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.081+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/2566140637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623cae4290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc5b401ffd0 tx=0x7fc5b4009ca0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.081+0000 7fc5d5358b80 1 -- 172.21.15.32:0/2566140637 shutdown_connections 2023-09-30T14:39:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.081+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/2566140637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623caeada0 0x55623caf1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.081+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/2566140637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae95a0 0x55623cae4290 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.081+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/2566140637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623cae9f00 0x55623cafa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.081+0000 7fc5d5358b80 1 -- 172.21.15.32:0/2566140637 >> 172.21.15.32:0/2566140637 conn(0x55623ca5d260 msgr2=0x55623cb09620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.081+0000 7fc5d5358b80 1 -- 172.21.15.32:0/2566140637 shutdown_connections 2023-09-30T14:39:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.081+0000 7fc5d5358b80 1 -- 172.21.15.32:0/2566140637 wait complete. 2023-09-30T14:39:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.081+0000 7fc5d5358b80 1 -- start start 2023-09-30T14:39:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.081+0000 7fc5d5358b80 1 Processor -- start 2023-09-30T14:39:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5d5358b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623cae95a0 0x55623caf6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5d5358b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae9f00 0x55623caf6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5d5358b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623caeada0 0x55623caf9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5d5358b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623cafa1a0 con 0x55623caeada0 2023-09-30T14:39:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5d5358b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623cafa310 con 0x55623cae95a0 2023-09-30T14:39:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5d5358b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623cafa4d0 con 0x55623cae9f00 2023-09-30T14:39:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5d5358b80 10 client.0 _ll_register_callbacks cb 0x55623c835800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5c359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623caeada0 0x55623caf9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5c2d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623cae95a0 0x55623caf6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5c359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623caeada0 0x55623caf9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35548/0 (socket says 172.21.15.32:35548) 2023-09-30T14:39:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5c359e700 1 -- 172.21.15.32:0/659153277 learned_addr learned my addr 172.21.15.32:0/659153277 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5c259c700 1 --2- 172.21.15.32:0/659153277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae9f00 0x55623caf6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5c2d9d700 1 -- 172.21.15.32:0/659153277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae9f00 msgr2=0x55623caf6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5c2d9d700 1 --2- 172.21.15.32:0/659153277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae9f00 0x55623caf6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5c2d9d700 1 -- 172.21.15.32:0/659153277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623caeada0 msgr2=0x55623caf9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5c2d9d700 1 --2- 172.21.15.32:0/659153277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623caeada0 0x55623caf9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.082+0000 7fc5c2d9d700 1 -- 172.21.15.32:0/659153277 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5bc003c50 con 0x55623cae95a0 2023-09-30T14:39:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.083+0000 7fc5c2d9d700 1 --2- 172.21.15.32:0/659153277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623cae95a0 0x55623caf6700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55623cadeb40 tx=0x7fc5b401f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cf079bbcbd07ecf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.083+0000 7fc5a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:39:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.083+0000 7fc5a6ff5700 1 -- 172.21.15.32:0/659153277 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b40a7360 con 0x55623cae95a0 2023-09-30T14:39:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.083+0000 7fc5d5358b80 1 -- 172.21.15.32:0/659153277 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5bc00f040 con 0x55623cae95a0 2023-09-30T14:39:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.083+0000 7fc5a6ff5700 1 -- 172.21.15.32:0/659153277 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc5b409f2c0 con 0x55623cae95a0 2023-09-30T14:39:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.083+0000 7fc5a6ff5700 1 -- 172.21.15.32:0/659153277 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5b40a31b0 con 0x55623cae95a0 2023-09-30T14:39:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.083+0000 7fc5d5358b80 10 client.25157 fetch_fsmap learned FSMap version 23 2023-09-30T14:39:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.083+0000 7fc5d5358b80 1 -- 172.21.15.32:0/659153277 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55623c84fc60 con 0x55623cae95a0 2023-09-30T14:39:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.083+0000 7fc5a6ff5700 1 -- 172.21.15.32:0/659153277 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5bc00f040 con 0x55623cae95a0 2023-09-30T14:39:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.083+0000 7fc5a6ff5700 1 -- 172.21.15.32:0/659153277 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc5b406a020 con 0x55623cae95a0 2023-09-30T14:39:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.083+0000 7fc5d5358b80 10 client.25157 fetch_fsmap finished waiting for FSMap version 23 2023-09-30T14:39:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.083+0000 7fc5d5358b80 10 client.25157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.083+0000 7fc5d5358b80 20 client.25157 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.083+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/659153277 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55623cb5bef0 0x55623cb5e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.083+0000 7fc5d5358b80 4 client.25157 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-30T14:39:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.083+0000 7fc5d5358b80 1 -- 172.21.15.32:0/659153277 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55623ca34b30 con 0x55623cb5bef0 2023-09-30T14:39:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.084+0000 7fc5c259c700 1 --2- 172.21.15.32:0/659153277 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55623cb5bef0 0x55623cb5e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.084+0000 7fc5c259c700 1 --2- 172.21.15.32:0/659153277 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55623cb5bef0 0x55623cb5e2e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3f999cfbfdfb285 server_cookie=fa62af4160390067 in_seq=0 out_seq=0 2023-09-30T14:39:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.084+0000 7fc5a6ff5700 10 client.25157 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.084+0000 7fc5a6ff5700 1 -- 172.21.15.32:0/659153277 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55623ca34b30 con 0x55623cb5bef0 2023-09-30T14:39:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.084+0000 7fc5a6ff5700 10 client.25157 handle_command_reply: tid=0 2023-09-30T14:39:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.084+0000 7fc5d5358b80 1 client.25157 shutdown 2023-09-30T14:39:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.085+0000 7fc5d5358b80 1 -- 172.21.15.32:0/659153277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623cae95a0 msgr2=0x55623caf6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.085+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/659153277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623cae95a0 0x55623caf6700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55623cadeb40 tx=0x7fc5b401f7f0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.085+0000 7fc5d5358b80 1 -- 172.21.15.32:0/659153277 shutdown_connections 2023-09-30T14:39:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.085+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/659153277 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55623cb5bef0 0x55623cb5e2e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.085+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/659153277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55623caeada0 0x55623caf9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.085+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/659153277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55623cae95a0 0x55623caf6700 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.085+0000 7fc5d5358b80 1 --2- 172.21.15.32:0/659153277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55623cae9f00 0x55623caf6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.085+0000 7fc5d5358b80 1 -- 172.21.15.32:0/659153277 >> 172.21.15.32:0/659153277 conn(0x55623ca5d260 msgr2=0x55623cafb110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.085+0000 7fc5a6ff5700 0 client.25157 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:13.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.085+0000 7fc5d5358b80 1 -- 172.21.15.32:0/659153277 shutdown_connections 2023-09-30T14:39:13.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.085+0000 7fc5d5358b80 1 -- 172.21.15.32:0/659153277 wait complete. 2023-09-30T14:39:13.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.085+0000 7fc5d5358b80 20 client.25157 trim_cache size 0 max 16384 2023-09-30T14:39:13.091 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T14:39:13.091 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2023-09-30T14:39:13.091 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2023-09-30T14:39:13.091 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T14:39:13.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.085+0000 7fc5c9170700 1 -- 172.21.15.32:0/94661580 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5ac0a0fa0 msgr2=0x7fc5ac0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.085+0000 7fc5c9170700 1 --2- 172.21.15.32:0/94661580 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5ac0a0fa0 0x7fc5ac0a3460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8005950 tx=0x7fc5b80058e0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.085+0000 7fc5c9170700 1 -- 172.21.15.32:0/94661580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5c4100e70 msgr2=0x7fc5c4196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.085+0000 7fc5c9170700 1 --2- 172.21.15.32:0/94661580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5c4100e70 0x7fc5c4196b50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4005910 tx=0x7fc5b4005e80 comp rx=0 tx=0).stop 2023-09-30T14:39:13.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.086+0000 7fc5c9170700 1 -- 172.21.15.32:0/94661580 shutdown_connections 2023-09-30T14:39:13.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.086+0000 7fc5c9170700 1 --2- 172.21.15.32:0/94661580 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5ac0a0fa0 0x7fc5ac0a3460 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.086+0000 7fc5c9170700 1 --2- 172.21.15.32:0/94661580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5c4102980 0x7fc5c419e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.086+0000 7fc5c9170700 1 --2- 172.21.15.32:0/94661580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5c4100e70 0x7fc5c4196b50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.086+0000 7fc5c9170700 1 --2- 172.21.15.32:0/94661580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5c4101fc0 0x7fc5c4197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.086+0000 7fc5c9170700 1 -- 172.21.15.32:0/94661580 >> 172.21.15.32:0/94661580 conn(0x7fc5c40fc3e0 msgr2=0x7fc5c40fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.086+0000 7fc5c9170700 1 -- 172.21.15.32:0/94661580 shutdown_connections 2023-09-30T14:39:13.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.086+0000 7fc5c9170700 1 -- 172.21.15.32:0/94661580 wait complete. 2023-09-30T14:39:13.103 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-30T14:39:13.103 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:39:13.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.235+0000 7fcdc645b700 1 -- 172.21.15.32:0/889255841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc0100e80 msgr2=0x7fcdc01012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.235+0000 7fcdc645b700 1 --2- 172.21.15.32:0/889255841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc0100e80 0x7fcdc01012a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009990 tx=0x7fcdb0009ca0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.235+0000 7fcdc645b700 1 -- 172.21.15.32:0/889255841 shutdown_connections 2023-09-30T14:39:13.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.235+0000 7fcdc645b700 1 --2- 172.21.15.32:0/889255841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdc0102990 0x7fcdc01092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.235+0000 7fcdc645b700 1 --2- 172.21.15.32:0/889255841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc0100e80 0x7fcdc01012a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.235+0000 7fcdc645b700 1 --2- 172.21.15.32:0/889255841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdc0101fd0 0x7fcdc0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.235+0000 7fcdc645b700 1 -- 172.21.15.32:0/889255841 >> 172.21.15.32:0/889255841 conn(0x7fcdc00fc3d0 msgr2=0x7fcdc00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.235+0000 7fcdc645b700 1 -- 172.21.15.32:0/889255841 shutdown_connections 2023-09-30T14:39:13.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.236+0000 7fcdc645b700 1 -- 172.21.15.32:0/889255841 wait complete. 2023-09-30T14:39:13.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.236+0000 7fcdc645b700 1 Processor -- start 2023-09-30T14:39:13.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.236+0000 7fcdc645b700 1 -- start start 2023-09-30T14:39:13.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.237+0000 7fcdc645b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc0100e80 0x7fcdc0074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.237+0000 7fcdc645b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdc0101fd0 0x7fcdc0073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.237+0000 7fcdc645b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdc0102990 0x7fcdc0073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.237+0000 7fcdc645b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc0073ef0 con 0x7fcdc0100e80 2023-09-30T14:39:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.237+0000 7fcdc645b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc0074060 con 0x7fcdc0102990 2023-09-30T14:39:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.237+0000 7fcdc645b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc01a0b40 con 0x7fcdc0101fd0 2023-09-30T14:39:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.237+0000 7fcdbffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc0100e80 0x7fcdc0074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.237+0000 7fcdbffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc0100e80 0x7fcdc0074bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35560/0 (socket says 172.21.15.32:35560) 2023-09-30T14:39:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.237+0000 7fcdbffff700 1 -- 172.21.15.32:0/3070367010 learned_addr learned my addr 172.21.15.32:0/3070367010 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.237+0000 7fcdbf7fe700 1 --2- 172.21.15.32:0/3070367010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdc0101fd0 0x7fcdc0073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.237+0000 7fcdc49f8700 1 --2- 172.21.15.32:0/3070367010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdc0102990 0x7fcdc0073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.237+0000 7fcdbffff700 1 -- 172.21.15.32:0/3070367010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdc0101fd0 msgr2=0x7fcdc0073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.237+0000 7fcdbffff700 1 --2- 172.21.15.32:0/3070367010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdc0101fd0 0x7fcdc0073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.237+0000 7fcdbffff700 1 -- 172.21.15.32:0/3070367010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdc0102990 msgr2=0x7fcdc0073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.237+0000 7fcdbffff700 1 --2- 172.21.15.32:0/3070367010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdc0102990 0x7fcdc0073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.237+0000 7fcdbffff700 1 -- 172.21.15.32:0/3070367010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb00095f0 con 0x7fcdc0100e80 2023-09-30T14:39:13.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.238+0000 7fcdbffff700 1 --2- 172.21.15.32:0/3070367010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc0100e80 0x7fcdc0074bd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0004760 tx=0x7fcdb0004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.238+0000 7fcdbd7fa700 1 -- 172.21.15.32:0/3070367010 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb001d070 con 0x7fcdc0100e80 2023-09-30T14:39:13.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.238+0000 7fcdc645b700 1 -- 172.21.15.32:0/3070367010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc01a0dc0 con 0x7fcdc0100e80 2023-09-30T14:39:13.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.238+0000 7fcdbd7fa700 1 -- 172.21.15.32:0/3070367010 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdb0004cf0 con 0x7fcdc0100e80 2023-09-30T14:39:13.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.238+0000 7fcdbd7fa700 1 -- 172.21.15.32:0/3070367010 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0003d00 con 0x7fcdc0100e80 2023-09-30T14:39:13.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.238+0000 7fcdc645b700 1 -- 172.21.15.32:0/3070367010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc01a11b0 con 0x7fcdc0100e80 2023-09-30T14:39:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.239+0000 7fcdc645b700 1 -- 172.21.15.32:0/3070367010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc0107d50 con 0x7fcdc0100e80 2023-09-30T14:39:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.240+0000 7fcdbd7fa700 1 -- 172.21.15.32:0/3070367010 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fcdb0005000 con 0x7fcdc0100e80 2023-09-30T14:39:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.241+0000 7fcdbd7fa700 1 --2- 172.21.15.32:0/3070367010 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcda80a0ff0 0x7fcda80a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.241+0000 7fcdbf7fe700 1 --2- 172.21.15.32:0/3070367010 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcda80a0ff0 0x7fcda80a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.241+0000 7fcdbd7fa700 1 -- 172.21.15.32:0/3070367010 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcdb00cdbc0 con 0x7fcdc0100e80 2023-09-30T14:39:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.241+0000 7fcdbf7fe700 1 --2- 172.21.15.32:0/3070367010 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcda80a0ff0 0x7fcda80a34b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4005950 tx=0x7fcdb400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.245+0000 7fcdbd7fa700 1 -- 172.21.15.32:0/3070367010 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fcdb0097460 con 0x7fcdc0100e80 2023-09-30T14:39:13.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.513+0000 7fcdc645b700 1 -- 172.21.15.32:0/3070367010 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdc002cf30 con 0x7fcdc0100e80 2023-09-30T14:39:13.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.514+0000 7fcdbd7fa700 1 -- 172.21.15.32:0/3070367010 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11322 (secure 0 0 0) 0x7fcdb0004e60 con 0x7fcdc0100e80 2023-09-30T14:39:13.515 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:39:13.515 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3505064841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3505064841},{"type":"v1","addr":"172.21.15.32:6839","nonce":3505064841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2262846038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2262846038},{"type":"v1","addr":"172.21.15.32:6841","nonce":2262846038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-30T14:37:11.444993+0000","modified":"2023-09-30T14:38:55.830142+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6839/1445176203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1445176203},{"type":"v1","addr":"172.21.15.156:6839","nonce":1445176203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/494821686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":494821686},{"type":"v1","addr":"172.21.15.148:6839","nonce":494821686}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T14:39:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.516+0000 7fcdc645b700 1 -- 172.21.15.32:0/3070367010 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcda80a0ff0 msgr2=0x7fcda80a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.516+0000 7fcdc645b700 1 --2- 172.21.15.32:0/3070367010 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcda80a0ff0 0x7fcda80a34b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4005950 tx=0x7fcdb400b410 comp rx=0 tx=0).stop 2023-09-30T14:39:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.516+0000 7fcdc645b700 1 -- 172.21.15.32:0/3070367010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc0100e80 msgr2=0x7fcdc0074bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.516+0000 7fcdc645b700 1 --2- 172.21.15.32:0/3070367010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc0100e80 0x7fcdc0074bd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0004760 tx=0x7fcdb0004790 comp rx=0 tx=0).stop 2023-09-30T14:39:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.516+0000 7fcdc645b700 1 -- 172.21.15.32:0/3070367010 shutdown_connections 2023-09-30T14:39:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.516+0000 7fcdc645b700 1 --2- 172.21.15.32:0/3070367010 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcda80a0ff0 0x7fcda80a34b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.516+0000 7fcdc645b700 1 --2- 172.21.15.32:0/3070367010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdc0102990 0x7fcdc0073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.516+0000 7fcdc645b700 1 --2- 172.21.15.32:0/3070367010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc0100e80 0x7fcdc0074bd0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.516+0000 7fcdc645b700 1 --2- 172.21.15.32:0/3070367010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdc0101fd0 0x7fcdc0073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.516+0000 7fcdc645b700 1 -- 172.21.15.32:0/3070367010 >> 172.21.15.32:0/3070367010 conn(0x7fcdc00fc3d0 msgr2=0x7fcdc00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.517+0000 7fcdc645b700 1 -- 172.21.15.32:0/3070367010 shutdown_connections 2023-09-30T14:39:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.517+0000 7fcdc645b700 1 -- 172.21.15.32:0/3070367010 wait complete. 2023-09-30T14:39:13.518 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2023-09-30T14:39:13.531 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-30T14:39:13.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.670+0000 7fd9b0361700 1 -- 172.21.15.32:0/1772965270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a81018c0 msgr2=0x7fd9a8108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.670+0000 7fd9b0361700 1 --2- 172.21.15.32:0/1772965270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a81018c0 0x7fd9a8108170 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000b0d0 tx=0x7fd9a000b3e0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.670+0000 7fd9b0361700 1 -- 172.21.15.32:0/1772965270 shutdown_connections 2023-09-30T14:39:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.670+0000 7fd9b0361700 1 --2- 172.21.15.32:0/1772965270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a81018c0 0x7fd9a8108170 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.670+0000 7fd9b0361700 1 --2- 172.21.15.32:0/1772965270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a80ffd30 0x7fd9a8100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.670+0000 7fd9b0361700 1 --2- 172.21.15.32:0/1772965270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a8100f20 0x7fd9a8101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.670+0000 7fd9b0361700 1 -- 172.21.15.32:0/1772965270 >> 172.21.15.32:0/1772965270 conn(0x7fd9a80fb2a0 msgr2=0x7fd9a80fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.670+0000 7fd9b0361700 1 -- 172.21.15.32:0/1772965270 shutdown_connections 2023-09-30T14:39:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.670+0000 7fd9b0361700 1 -- 172.21.15.32:0/1772965270 wait complete. 2023-09-30T14:39:13.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.671+0000 7fd9b0361700 1 Processor -- start 2023-09-30T14:39:13.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.671+0000 7fd9b0361700 1 -- start start 2023-09-30T14:39:13.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.671+0000 7fd9b0361700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a80ffd30 0x7fd9a8068b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.671+0000 7fd9b0361700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a8100f20 0x7fd9a819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.671+0000 7fd9b0361700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a81018c0 0x7fd9a819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.672+0000 7fd9b0361700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a819e7a0 con 0x7fd9a8100f20 2023-09-30T14:39:13.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.672+0000 7fd9b0361700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a819e910 con 0x7fd9a80ffd30 2023-09-30T14:39:13.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.672+0000 7fd9b0361700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a819ead0 con 0x7fd9a81018c0 2023-09-30T14:39:13.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.672+0000 7fd9ad8fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a8100f20 0x7fd9a819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.672+0000 7fd9ae0fd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a80ffd30 0x7fd9a8068b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.672+0000 7fd9ad8fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a8100f20 0x7fd9a819b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35578/0 (socket says 172.21.15.32:35578) 2023-09-30T14:39:13.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.672+0000 7fd9ad8fc700 1 -- 172.21.15.32:0/3366703399 learned_addr learned my addr 172.21.15.32:0/3366703399 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:13.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.672+0000 7fd9ae8fe700 1 --2- 172.21.15.32:0/3366703399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a81018c0 0x7fd9a819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.672+0000 7fd9ad8fc700 1 -- 172.21.15.32:0/3366703399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a81018c0 msgr2=0x7fd9a819dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.672+0000 7fd9ad8fc700 1 --2- 172.21.15.32:0/3366703399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a81018c0 0x7fd9a819dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.672+0000 7fd9ad8fc700 1 -- 172.21.15.32:0/3366703399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a80ffd30 msgr2=0x7fd9a8068b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.672+0000 7fd9ad8fc700 1 --2- 172.21.15.32:0/3366703399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a80ffd30 0x7fd9a8068b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.672+0000 7fd9ad8fc700 1 -- 172.21.15.32:0/3366703399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a0009d00 con 0x7fd9a8100f20 2023-09-30T14:39:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.672+0000 7fd9ae8fe700 1 --2- 172.21.15.32:0/3366703399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a81018c0 0x7fd9a819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:39:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.672+0000 7fd9ad8fc700 1 --2- 172.21.15.32:0/3366703399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a8100f20 0x7fd9a819b980 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd99800e9a0 tx=0x7fd99800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.673+0000 7fd99f7fe700 1 -- 172.21.15.32:0/3366703399 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998009790 con 0x7fd9a8100f20 2023-09-30T14:39:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.673+0000 7fd9b0361700 1 -- 172.21.15.32:0/3366703399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a819edb0 con 0x7fd9a8100f20 2023-09-30T14:39:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.673+0000 7fd99f7fe700 1 -- 172.21.15.32:0/3366703399 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd99800cbd0 con 0x7fd9a8100f20 2023-09-30T14:39:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.673+0000 7fd99f7fe700 1 -- 172.21.15.32:0/3366703399 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998018680 con 0x7fd9a8100f20 2023-09-30T14:39:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.673+0000 7fd9b0361700 1 -- 172.21.15.32:0/3366703399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a819f230 con 0x7fd9a8100f20 2023-09-30T14:39:13.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.675+0000 7fd99f7fe700 1 -- 172.21.15.32:0/3366703399 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fd99800cd40 con 0x7fd9a8100f20 2023-09-30T14:39:13.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.676+0000 7fd99f7fe700 1 --2- 172.21.15.32:0/3366703399 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd9940a0fa0 0x7fd9940a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.676+0000 7fd9ae0fd700 1 --2- 172.21.15.32:0/3366703399 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd9940a0fa0 0x7fd9940a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.676+0000 7fd99f7fe700 1 -- 172.21.15.32:0/3366703399 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd998014070 con 0x7fd9a8100f20 2023-09-30T14:39:13.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.677+0000 7fd9ae0fd700 1 --2- 172.21.15.32:0/3366703399 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd9940a0fa0 0x7fd9940a3460 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8100d80 tx=0x7fd9a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.678+0000 7fd9bc549b80 1 Processor -- start 2023-09-30T14:39:13.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.678+0000 7fd9bc549b80 1 -- start start 2023-09-30T14:39:13.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9bc549b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee215a0 0x55d71ee219c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9bc549b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee21f00 0x55d71ee227c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9bc549b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee22da0 0x55d71ee23220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9bc549b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d71ed89460 con 0x55d71ee21f00 2023-09-30T14:39:13.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9bc549b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d71ed7ac20 con 0x55d71ee215a0 2023-09-30T14:39:13.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9bc549b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d71ed78540 con 0x55d71ee22da0 2023-09-30T14:39:13.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9ad8fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee21f00 0x55d71ee227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9ad8fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee21f00 0x55d71ee227c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35586/0 (socket says 172.21.15.32:35586) 2023-09-30T14:39:13.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9ad8fc700 1 -- 172.21.15.32:0/2146467026 learned_addr learned my addr 172.21.15.32:0/2146467026 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:13.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9ae0fd700 1 --2- 172.21.15.32:0/2146467026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee215a0 0x55d71ee219c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9ae8fe700 1 --2- 172.21.15.32:0/2146467026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee22da0 0x55d71ee23220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9ad8fc700 1 -- 172.21.15.32:0/2146467026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee22da0 msgr2=0x55d71ee23220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9ad8fc700 1 --2- 172.21.15.32:0/2146467026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee22da0 0x55d71ee23220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9ad8fc700 1 -- 172.21.15.32:0/2146467026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee215a0 msgr2=0x55d71ee219c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9ad8fc700 1 --2- 172.21.15.32:0/2146467026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee215a0 0x55d71ee219c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9ad8fc700 1 -- 172.21.15.32:0/2146467026 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d71edd68d0 con 0x55d71ee21f00 2023-09-30T14:39:13.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.679+0000 7fd9ae0fd700 1 --2- 172.21.15.32:0/2146467026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee215a0 0x55d71ee219c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:39:13.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.680+0000 7fd9ad8fc700 1 --2- 172.21.15.32:0/2146467026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee21f00 0x55d71ee227c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd9980962c0 tx=0x7fd998099720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7b611ebc6065b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.680+0000 7fd99cd82700 1 -- 172.21.15.32:0/2146467026 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9980b1070 con 0x55d71ee21f00 2023-09-30T14:39:13.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.680+0000 7fd99cd82700 1 -- 172.21.15.32:0/2146467026 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd998099dc0 con 0x55d71ee21f00 2023-09-30T14:39:13.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.680+0000 7fd99cd82700 1 -- 172.21.15.32:0/2146467026 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9980b24d0 con 0x55d71ee21f00 2023-09-30T14:39:13.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.681+0000 7fd9bc549b80 1 -- 172.21.15.32:0/2146467026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee21f00 msgr2=0x55d71ee227c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.681+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/2146467026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee21f00 0x55d71ee227c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd9980962c0 tx=0x7fd998099720 comp rx=0 tx=0).stop 2023-09-30T14:39:13.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.681+0000 7fd9bc549b80 1 -- 172.21.15.32:0/2146467026 shutdown_connections 2023-09-30T14:39:13.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.681+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/2146467026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee22da0 0x55d71ee23220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.681+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/2146467026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee215a0 0x55d71ee219c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.681+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/2146467026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee21f00 0x55d71ee227c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.681+0000 7fd9bc549b80 1 -- 172.21.15.32:0/2146467026 >> 172.21.15.32:0/2146467026 conn(0x55d71ed95260 msgr2=0x55d71ee1c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.681+0000 7fd9bc549b80 1 -- 172.21.15.32:0/2146467026 shutdown_connections 2023-09-30T14:39:13.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.681+0000 7fd9bc549b80 1 -- 172.21.15.32:0/2146467026 wait complete. 2023-09-30T14:39:13.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.681+0000 7fd9bc549b80 1 -- start start 2023-09-30T14:39:13.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.681+0000 7fd9bc549b80 1 Processor -- start 2023-09-30T14:39:13.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9bc549b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee215a0 0x55d71ee2b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9bc549b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee21f00 0x55d71ee2b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9bc549b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee22da0 0x55d71ee32990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9bc549b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d71ed73e30 con 0x55d71ee22da0 2023-09-30T14:39:13.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9bc549b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d71ed10710 con 0x55d71ee21f00 2023-09-30T14:39:13.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9bc549b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d71ed103b0 con 0x55d71ee215a0 2023-09-30T14:39:13.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9bc549b80 10 client.0 _ll_register_callbacks cb 0x55d71eb6d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:13.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9ae0fd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee215a0 0x55d71ee2b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9ae0fd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee215a0 0x55d71ee2b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:49970/0 (socket says 172.21.15.32:49970) 2023-09-30T14:39:13.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9ae0fd700 1 -- 172.21.15.32:0/918403450 learned_addr learned my addr 172.21.15.32:0/918403450 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:13.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9ae8fe700 1 --2- 172.21.15.32:0/918403450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee22da0 0x55d71ee32990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9ad8fc700 1 --2- 172.21.15.32:0/918403450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee21f00 0x55d71ee2b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9ae0fd700 1 -- 172.21.15.32:0/918403450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee21f00 msgr2=0x55d71ee2b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9ae0fd700 1 --2- 172.21.15.32:0/918403450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee21f00 0x55d71ee2b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9ae0fd700 1 -- 172.21.15.32:0/918403450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee22da0 msgr2=0x55d71ee32990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9ae0fd700 1 --2- 172.21.15.32:0/918403450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee22da0 0x55d71ee32990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.682+0000 7fd9ae0fd700 1 -- 172.21.15.32:0/918403450 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a819f230 con 0x55d71ee215a0 2023-09-30T14:39:13.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.683+0000 7fd9ae0fd700 1 --2- 172.21.15.32:0/918403450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee215a0 0x55d71ee2b410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009b20 tx=0x7fd9a40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=291e2a735271cbb7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.683+0000 7fd991ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:39:13.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.683+0000 7fd991ffb700 1 -- 172.21.15.32:0/918403450 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a401c820 con 0x55d71ee215a0 2023-09-30T14:39:13.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.683+0000 7fd9bc549b80 1 -- 172.21.15.32:0/918403450 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d71ee14250 con 0x55d71ee215a0 2023-09-30T14:39:13.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.683+0000 7fd991ffb700 1 -- 172.21.15.32:0/918403450 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd9a401c980 con 0x55d71ee215a0 2023-09-30T14:39:13.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.683+0000 7fd991ffb700 1 -- 172.21.15.32:0/918403450 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9a401cb40 con 0x55d71ee215a0 2023-09-30T14:39:13.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.683+0000 7fd9bc549b80 10 client.25153 fetch_fsmap learned FSMap version 23 2023-09-30T14:39:13.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.683+0000 7fd9bc549b80 1 -- 172.21.15.32:0/918403450 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d71eb9ede0 con 0x55d71ee215a0 2023-09-30T14:39:13.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.683+0000 7fd991ffb700 1 -- 172.21.15.32:0/918403450 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d71ee14250 con 0x55d71ee215a0 2023-09-30T14:39:13.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.684+0000 7fd991ffb700 1 -- 172.21.15.32:0/918403450 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd9a400e3b0 con 0x55d71ee215a0 2023-09-30T14:39:13.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.684+0000 7fd9bc549b80 10 client.25153 fetch_fsmap finished waiting for FSMap version 23 2023-09-30T14:39:13.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.684+0000 7fd9bc549b80 10 client.25153 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-30T14:39:13.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.684+0000 7fd9bc549b80 20 client.25153 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:13.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.684+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/918403450 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x55d71ee35c10 0x55d71ee36020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.684+0000 7fd9bc549b80 4 client.25153 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:39:13.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.684+0000 7fd9bc549b80 1 -- 172.21.15.32:0/918403450 --> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d71ed6ce10 con 0x55d71ee35c10 2023-09-30T14:39:13.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.684+0000 7fd9ad8fc700 1 --2- 172.21.15.32:0/918403450 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x55d71ee35c10 0x55d71ee36020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.684+0000 7fd991ffb700 10 client.25153 ms_handle_connect on v2:172.21.15.148:6836/3666119034 2023-09-30T14:39:13.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.684+0000 7fd9ad8fc700 1 --2- 172.21.15.32:0/918403450 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x55d71ee35c10 0x55d71ee36020 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee72c086a5b4411a server_cookie=6a57e7fcb920f9d0 in_seq=0 out_seq=0 2023-09-30T14:39:13.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.685+0000 7fd991ffb700 1 -- 172.21.15.32:0/918403450 <== mds.2 v2:172.21.15.148:6836/3666119034 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d71ed6ce10 con 0x55d71ee35c10 2023-09-30T14:39:13.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.685+0000 7fd991ffb700 10 client.25153 handle_command_reply: tid=0 2023-09-30T14:39:13.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.685+0000 7fd9bc549b80 1 client.25153 shutdown 2023-09-30T14:39:13.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.685+0000 7fd9bc549b80 1 -- 172.21.15.32:0/918403450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee215a0 msgr2=0x55d71ee2b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.685+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/918403450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee215a0 0x55d71ee2b410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009b20 tx=0x7fd9a40047c0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.685+0000 7fd9bc549b80 1 -- 172.21.15.32:0/918403450 shutdown_connections 2023-09-30T14:39:13.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.685+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/918403450 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x55d71ee35c10 0x55d71ee36020 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.685+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/918403450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee22da0 0x55d71ee32990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.685+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/918403450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee215a0 0x55d71ee2b410 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.685+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/918403450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee21f00 0x55d71ee2b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.685+0000 7fd9bc549b80 1 -- 172.21.15.32:0/918403450 >> 172.21.15.32:0/918403450 conn(0x55d71ed95260 msgr2=0x55d71ee1c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.685+0000 7fd991ffb700 0 client.25153 ms_handle_reset on v2:172.21.15.148:6836/3666119034 2023-09-30T14:39:13.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.685+0000 7fd9bc549b80 1 -- 172.21.15.32:0/918403450 shutdown_connections 2023-09-30T14:39:13.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.685+0000 7fd9bc549b80 1 -- 172.21.15.32:0/918403450 wait complete. 2023-09-30T14:39:13.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.686+0000 7fd9bc549b80 20 client.25153 trim_cache size 0 max 16384 2023-09-30T14:39:13.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.711+0000 7fd9bc549b80 1 Processor -- start 2023-09-30T14:39:13.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.711+0000 7fd9bc549b80 1 -- start start 2023-09-30T14:39:13.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.711+0000 7fd9bc549b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee215a0 0x55d71ee2a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.711+0000 7fd9bc549b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee21f00 0x55d71ee32920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.711+0000 7fd9bc549b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee22da0 0x55d71ee35c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.711+0000 7fd9bc549b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d71ebf4920 con 0x55d71ee22da0 2023-09-30T14:39:13.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.711+0000 7fd9bc549b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d71ee2a8c0 con 0x55d71ee215a0 2023-09-30T14:39:13.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.711+0000 7fd9bc549b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d71ee2af80 con 0x55d71ee21f00 2023-09-30T14:39:13.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.711+0000 7fd9ae8fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee22da0 0x55d71ee35c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.711+0000 7fd9ae8fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee22da0 0x55d71ee35c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35600/0 (socket says 172.21.15.32:35600) 2023-09-30T14:39:13.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.711+0000 7fd9ae8fe700 1 -- 172.21.15.32:0/448068512 learned_addr learned my addr 172.21.15.32:0/448068512 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:13.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.712+0000 7fd9ad8fc700 1 --2- 172.21.15.32:0/448068512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee21f00 0x55d71ee32920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.712+0000 7fd9ae0fd700 1 --2- 172.21.15.32:0/448068512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee215a0 0x55d71ee2a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.712+0000 7fd9ae8fe700 1 -- 172.21.15.32:0/448068512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee21f00 msgr2=0x55d71ee32920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.712+0000 7fd9ae8fe700 1 --2- 172.21.15.32:0/448068512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee21f00 0x55d71ee32920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.712+0000 7fd9ae8fe700 1 -- 172.21.15.32:0/448068512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee215a0 msgr2=0x55d71ee2a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.712+0000 7fd9ae8fe700 1 --2- 172.21.15.32:0/448068512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee215a0 0x55d71ee2a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.712+0000 7fd9ae8fe700 1 -- 172.21.15.32:0/448068512 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d71eb87c60 con 0x55d71ee22da0 2023-09-30T14:39:13.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.712+0000 7fd9ae0fd700 1 --2- 172.21.15.32:0/448068512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee215a0 0x55d71ee2a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:39:13.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.712+0000 7fd9ae8fe700 1 --2- 172.21.15.32:0/448068512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee22da0 0x55d71ee35c10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000b0d0 tx=0x7fd9a0009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48c4e979cc99864 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.712+0000 7fd99d583700 1 -- 172.21.15.32:0/448068512 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000e040 con 0x55d71ee22da0 2023-09-30T14:39:13.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.712+0000 7fd99d583700 1 -- 172.21.15.32:0/448068512 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9a00076e0 con 0x55d71ee22da0 2023-09-30T14:39:13.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.712+0000 7fd99d583700 1 -- 172.21.15.32:0/448068512 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0027860 con 0x55d71ee22da0 2023-09-30T14:39:13.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.713+0000 7fd9bc549b80 1 -- 172.21.15.32:0/448068512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee22da0 msgr2=0x55d71ee35c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.713+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/448068512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee22da0 0x55d71ee35c10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000b0d0 tx=0x7fd9a0009450 comp rx=0 tx=0).stop 2023-09-30T14:39:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.713+0000 7fd9bc549b80 1 -- 172.21.15.32:0/448068512 shutdown_connections 2023-09-30T14:39:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.713+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/448068512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee22da0 0x55d71ee35c10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.713+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/448068512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee215a0 0x55d71ee2a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.713+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/448068512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee21f00 0x55d71ee32920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.713+0000 7fd9bc549b80 1 -- 172.21.15.32:0/448068512 >> 172.21.15.32:0/448068512 conn(0x55d71ed95260 msgr2=0x55d71ee29230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.713+0000 7fd9bc549b80 1 -- 172.21.15.32:0/448068512 shutdown_connections 2023-09-30T14:39:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.713+0000 7fd9bc549b80 1 -- 172.21.15.32:0/448068512 wait complete. 2023-09-30T14:39:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.714+0000 7fd9bc549b80 1 -- start start 2023-09-30T14:39:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.714+0000 7fd9bc549b80 1 Processor -- start 2023-09-30T14:39:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.714+0000 7fd9bc549b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee215a0 0x55d71ee2fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.714+0000 7fd9bc549b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee21f00 0x55d71ee30290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.714+0000 7fd9bc549b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee22da0 0x55d71ee312a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.714+0000 7fd9bc549b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d71ee317e0 con 0x55d71ee215a0 2023-09-30T14:39:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.714+0000 7fd9bc549b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d71ee31950 con 0x55d71ee21f00 2023-09-30T14:39:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.714+0000 7fd9bc549b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d71ee31b10 con 0x55d71ee22da0 2023-09-30T14:39:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.714+0000 7fd9bc549b80 10 client.0 _ll_register_callbacks cb 0x7fd9a4011ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.714+0000 7fd9ae0fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee215a0 0x55d71ee2fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.715+0000 7fd9ae0fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee215a0 0x55d71ee2fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35602/0 (socket says 172.21.15.32:35602) 2023-09-30T14:39:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.715+0000 7fd9ae0fd700 1 -- 172.21.15.32:0/1759123891 learned_addr learned my addr 172.21.15.32:0/1759123891 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.715+0000 7fd9ae8fe700 1 --2- 172.21.15.32:0/1759123891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee22da0 0x55d71ee312a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.715+0000 7fd9ad8fc700 1 --2- 172.21.15.32:0/1759123891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee21f00 0x55d71ee30290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.715+0000 7fd9ae8fe700 1 -- 172.21.15.32:0/1759123891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee21f00 msgr2=0x55d71ee30290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.715+0000 7fd9ae8fe700 1 --2- 172.21.15.32:0/1759123891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee21f00 0x55d71ee30290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.715+0000 7fd9ae8fe700 1 -- 172.21.15.32:0/1759123891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee215a0 msgr2=0x55d71ee2fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.715+0000 7fd9ae8fe700 1 --2- 172.21.15.32:0/1759123891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee215a0 0x55d71ee2fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.715+0000 7fd9ae8fe700 1 -- 172.21.15.32:0/1759123891 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a8108ac0 con 0x55d71ee22da0 2023-09-30T14:39:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.715+0000 7fd9ae8fe700 1 --2- 172.21.15.32:0/1759123891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee22da0 0x55d71ee312a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0007a30 tx=0x7fd9a0009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a942e6f72989e335 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.715+0000 7fd991ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:39:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.715+0000 7fd991ffb700 1 -- 172.21.15.32:0/1759123891 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000e040 con 0x55d71ee22da0 2023-09-30T14:39:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.715+0000 7fd9bc549b80 1 -- 172.21.15.32:0/1759123891 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd998099dc0 con 0x55d71ee22da0 2023-09-30T14:39:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.716+0000 7fd991ffb700 1 -- 172.21.15.32:0/1759123891 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd9a0004a50 con 0x55d71ee22da0 2023-09-30T14:39:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.716+0000 7fd991ffb700 1 -- 172.21.15.32:0/1759123891 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9a0036030 con 0x55d71ee22da0 2023-09-30T14:39:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.716+0000 7fd9bc549b80 10 client.25159 fetch_fsmap learned FSMap version 23 2023-09-30T14:39:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.716+0000 7fd9bc549b80 1 -- 172.21.15.32:0/1759123891 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d71ebc5250 con 0x55d71ee22da0 2023-09-30T14:39:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.716+0000 7fd991ffb700 1 -- 172.21.15.32:0/1759123891 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd998099dc0 con 0x55d71ee22da0 2023-09-30T14:39:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.716+0000 7fd991ffb700 1 -- 172.21.15.32:0/1759123891 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd9a000b100 con 0x55d71ee22da0 2023-09-30T14:39:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.716+0000 7fd9bc549b80 10 client.25159 fetch_fsmap finished waiting for FSMap version 23 2023-09-30T14:39:13.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.716+0000 7fd9bc549b80 10 client.25159 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-30T14:39:13.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.716+0000 7fd9bc549b80 20 client.25159 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:13.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.716+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/1759123891 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x55d71ee94700 0x55d71ee96b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.716+0000 7fd9bc549b80 4 client.25159 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-30T14:39:13.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.716+0000 7fd9bc549b80 1 -- 172.21.15.32:0/1759123891 --> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d71ed6cb30 con 0x55d71ee94700 2023-09-30T14:39:13.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.717+0000 7fd9ae0fd700 1 --2- 172.21.15.32:0/1759123891 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x55d71ee94700 0x55d71ee96b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.717+0000 7fd991ffb700 10 client.25159 ms_handle_connect on v2:172.21.15.148:6836/3666119034 2023-09-30T14:39:13.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.717+0000 7fd9ae0fd700 1 --2- 172.21.15.32:0/1759123891 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x55d71ee94700 0x55d71ee96b20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db7b505d8bc88526 server_cookie=73f219d6e7363c53 in_seq=0 out_seq=0 2023-09-30T14:39:13.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.717+0000 7fd991ffb700 1 -- 172.21.15.32:0/1759123891 <== mds.2 v2:172.21.15.148:6836/3666119034 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d71ed6cb30 con 0x55d71ee94700 2023-09-30T14:39:13.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.717+0000 7fd991ffb700 10 client.25159 handle_command_reply: tid=0 2023-09-30T14:39:13.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.717+0000 7fd9bc549b80 1 client.25159 shutdown 2023-09-30T14:39:13.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.717+0000 7fd9bc549b80 1 -- 172.21.15.32:0/1759123891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee22da0 msgr2=0x55d71ee312a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.717+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/1759123891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee22da0 0x55d71ee312a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0007a30 tx=0x7fd9a0009800 comp rx=0 tx=0).stop 2023-09-30T14:39:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.717+0000 7fd9bc549b80 1 -- 172.21.15.32:0/1759123891 shutdown_connections 2023-09-30T14:39:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.717+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/1759123891 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x55d71ee94700 0x55d71ee96b20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.717+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/1759123891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d71ee22da0 0x55d71ee312a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.717+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/1759123891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d71ee215a0 0x55d71ee2fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.717+0000 7fd9bc549b80 1 --2- 172.21.15.32:0/1759123891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d71ee21f00 0x55d71ee30290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.717+0000 7fd9bc549b80 1 -- 172.21.15.32:0/1759123891 >> 172.21.15.32:0/1759123891 conn(0x55d71ed95260 msgr2=0x55d71ee35c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.717+0000 7fd991ffb700 0 client.25159 ms_handle_reset on v2:172.21.15.148:6836/3666119034 2023-09-30T14:39:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.717+0000 7fd9bc549b80 1 -- 172.21.15.32:0/1759123891 shutdown_connections 2023-09-30T14:39:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.718+0000 7fd9bc549b80 1 -- 172.21.15.32:0/1759123891 wait complete. 2023-09-30T14:39:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.718+0000 7fd9bc549b80 20 client.25159 trim_cache size 0 max 16384 2023-09-30T14:39:13.724 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2023-09-30T14:39:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.718+0000 7fd9b0361700 1 -- 172.21.15.32:0/3366703399 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd9940a0fa0 msgr2=0x7fd9940a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.718+0000 7fd9b0361700 1 --2- 172.21.15.32:0/3366703399 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd9940a0fa0 0x7fd9940a3460 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8100d80 tx=0x7fd9a4009260 comp rx=0 tx=0).stop 2023-09-30T14:39:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.718+0000 7fd9b0361700 1 -- 172.21.15.32:0/3366703399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a8100f20 msgr2=0x7fd9a819b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.718+0000 7fd9b0361700 1 --2- 172.21.15.32:0/3366703399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a8100f20 0x7fd9a819b980 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd99800e9a0 tx=0x7fd99800ecb0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.718+0000 7fd9b0361700 1 -- 172.21.15.32:0/3366703399 shutdown_connections 2023-09-30T14:39:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.718+0000 7fd9b0361700 1 --2- 172.21.15.32:0/3366703399 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd9940a0fa0 0x7fd9940a3460 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.718+0000 7fd9b0361700 1 --2- 172.21.15.32:0/3366703399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9a81018c0 0x7fd9a819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.718+0000 7fd9b0361700 1 --2- 172.21.15.32:0/3366703399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a80ffd30 0x7fd9a8068b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.718+0000 7fd9b0361700 1 --2- 172.21.15.32:0/3366703399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a8100f20 0x7fd9a819b980 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.718+0000 7fd9b0361700 1 -- 172.21.15.32:0/3366703399 >> 172.21.15.32:0/3366703399 conn(0x7fd9a80fb2a0 msgr2=0x7fd9a81043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.718+0000 7fd9b0361700 1 -- 172.21.15.32:0/3366703399 shutdown_connections 2023-09-30T14:39:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.718+0000 7fd9b0361700 1 -- 172.21.15.32:0/3366703399 wait complete. 2023-09-30T14:39:13.732 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-30T14:39:13.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:13 smithi156 ceph-mon[103579]: pgmap v315: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2023-09-30T14:39:13.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:13 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3070367010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:39:13.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:13 smithi148 ceph-mon[102355]: pgmap v315: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2023-09-30T14:39:13.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:13 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3070367010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:39:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.864+0000 7fe18f2dd700 1 -- 172.21.15.32:0/1433052655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe188101920 msgr2=0x7fe1881081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.864+0000 7fe18f2dd700 1 --2- 172.21.15.32:0/1433052655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe188101920 0x7fe1881081d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe180009f90 tx=0x7fe18000b330 comp rx=0 tx=0).stop 2023-09-30T14:39:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.864+0000 7fe18f2dd700 1 -- 172.21.15.32:0/1433052655 shutdown_connections 2023-09-30T14:39:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.864+0000 7fe18f2dd700 1 --2- 172.21.15.32:0/1433052655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe188101920 0x7fe1881081d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.864+0000 7fe18f2dd700 1 --2- 172.21.15.32:0/1433052655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1880ffd90 0x7fe1881001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.864+0000 7fe18f2dd700 1 --2- 172.21.15.32:0/1433052655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe188100f80 0x7fe1881013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.864+0000 7fe18f2dd700 1 -- 172.21.15.32:0/1433052655 >> 172.21.15.32:0/1433052655 conn(0x7fe1880fb2e0 msgr2=0x7fe1880fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.864+0000 7fe18f2dd700 1 -- 172.21.15.32:0/1433052655 shutdown_connections 2023-09-30T14:39:13.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.864+0000 7fe18f2dd700 1 -- 172.21.15.32:0/1433052655 wait complete. 2023-09-30T14:39:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.865+0000 7fe18f2dd700 1 Processor -- start 2023-09-30T14:39:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.865+0000 7fe18f2dd700 1 -- start start 2023-09-30T14:39:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.865+0000 7fe18f2dd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1880ffd90 0x7fe188196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.865+0000 7fe18f2dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe188100f80 0x7fe1881971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.865+0000 7fe18f2dd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe188101920 0x7fe18819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.865+0000 7fe18f2dd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18819e7f0 con 0x7fe188100f80 2023-09-30T14:39:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.865+0000 7fe18f2dd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18819e960 con 0x7fe188101920 2023-09-30T14:39:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.865+0000 7fe18f2dd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18819eb20 con 0x7fe1880ffd90 2023-09-30T14:39:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.865+0000 7fe18d079700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1880ffd90 0x7fe188196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.865+0000 7fe18d87a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe188101920 0x7fe18819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.865+0000 7fe18d87a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe188101920 0x7fe18819e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47096/0 (socket says 172.21.15.32:47096) 2023-09-30T14:39:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.866+0000 7fe18d87a700 1 -- 172.21.15.32:0/2846910876 learned_addr learned my addr 172.21.15.32:0/2846910876 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.866+0000 7fe18c878700 1 --2- 172.21.15.32:0/2846910876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe188100f80 0x7fe1881971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.866+0000 7fe18d87a700 1 -- 172.21.15.32:0/2846910876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1880ffd90 msgr2=0x7fe188196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.866+0000 7fe18d87a700 1 --2- 172.21.15.32:0/2846910876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1880ffd90 0x7fe188196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.866+0000 7fe18d87a700 1 -- 172.21.15.32:0/2846910876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe188100f80 msgr2=0x7fe1881971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.866+0000 7fe18d87a700 1 --2- 172.21.15.32:0/2846910876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe188100f80 0x7fe1881971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.866+0000 7fe18d87a700 1 -- 172.21.15.32:0/2846910876 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe180009c40 con 0x7fe188101920 2023-09-30T14:39:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.866+0000 7fe18c878700 1 --2- 172.21.15.32:0/2846910876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe188100f80 0x7fe1881971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:39:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.866+0000 7fe18d87a700 1 --2- 172.21.15.32:0/2846910876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe188101920 0x7fe18819e220 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe180001220 tx=0x7fe180004d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.866+0000 7fe17e7fc700 1 -- 172.21.15.32:0/2846910876 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18000e040 con 0x7fe188101920 2023-09-30T14:39:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.866+0000 7fe18f2dd700 1 -- 172.21.15.32:0/2846910876 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18819eda0 con 0x7fe188101920 2023-09-30T14:39:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.866+0000 7fe17e7fc700 1 -- 172.21.15.32:0/2846910876 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe18001d470 con 0x7fe188101920 2023-09-30T14:39:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.866+0000 7fe17e7fc700 1 -- 172.21.15.32:0/2846910876 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18001b680 con 0x7fe188101920 2023-09-30T14:39:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.867+0000 7fe18f2dd700 1 -- 172.21.15.32:0/2846910876 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18819f210 con 0x7fe188101920 2023-09-30T14:39:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.869+0000 7fe17e7fc700 1 -- 172.21.15.32:0/2846910876 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fe18001d5e0 con 0x7fe188101920 2023-09-30T14:39:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.870+0000 7fe17e7fc700 1 --2- 172.21.15.32:0/2846910876 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1740a0fa0 0x7fe1740a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.870+0000 7fe17e7fc700 1 -- 172.21.15.32:0/2846910876 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe1800cd940 con 0x7fe188101920 2023-09-30T14:39:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.870+0000 7fe18d079700 1 --2- 172.21.15.32:0/2846910876 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1740a0fa0 0x7fe1740a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.870+0000 7fe18d079700 1 --2- 172.21.15.32:0/2846910876 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1740a0fa0 0x7fe1740a3460 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe188197b50 tx=0x7fe18400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.871+0000 7fe19b4c5b80 1 Processor -- start 2023-09-30T14:39:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe19b4c5b80 1 -- start start 2023-09-30T14:39:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe19b4c5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e269b0 0x55eed8e26dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe19b4c5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e27310 0x55eed8e27bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe19b4c5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e281b0 0x55eed8e28630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe19b4c5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed8d8a460 con 0x55eed8e281b0 2023-09-30T14:39:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe19b4c5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed8d7bc20 con 0x55eed8e269b0 2023-09-30T14:39:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe19b4c5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed8d79540 con 0x55eed8e27310 2023-09-30T14:39:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe18d079700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e269b0 0x55eed8e26dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe18d87a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e281b0 0x55eed8e28630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe18d079700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e269b0 0x55eed8e26dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47100/0 (socket says 172.21.15.32:47100) 2023-09-30T14:39:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe18d079700 1 -- 172.21.15.32:0/2850737032 learned_addr learned my addr 172.21.15.32:0/2850737032 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe18c878700 1 --2- 172.21.15.32:0/2850737032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e27310 0x55eed8e27bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe18d079700 1 -- 172.21.15.32:0/2850737032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e27310 msgr2=0x55eed8e27bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe18d079700 1 --2- 172.21.15.32:0/2850737032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e27310 0x55eed8e27bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe18d079700 1 -- 172.21.15.32:0/2850737032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e281b0 msgr2=0x55eed8e28630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe18d079700 1 --2- 172.21.15.32:0/2850737032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e281b0 0x55eed8e28630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.872+0000 7fe18d079700 1 -- 172.21.15.32:0/2850737032 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eed8dd78d0 con 0x55eed8e269b0 2023-09-30T14:39:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.873+0000 7fe18d079700 1 --2- 172.21.15.32:0/2850737032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e269b0 0x55eed8e26dd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe184010f70 tx=0x7fe184013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f3ab292105b04b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.873+0000 7fe1737fe700 1 -- 172.21.15.32:0/2850737032 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184011db0 con 0x55eed8e269b0 2023-09-30T14:39:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.873+0000 7fe1737fe700 1 -- 172.21.15.32:0/2850737032 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe184015070 con 0x55eed8e269b0 2023-09-30T14:39:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.873+0000 7fe1737fe700 1 -- 172.21.15.32:0/2850737032 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184003da0 con 0x55eed8e269b0 2023-09-30T14:39:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.874+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/2850737032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e269b0 msgr2=0x55eed8e26dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.874+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/2850737032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e269b0 0x55eed8e26dd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe184010f70 tx=0x7fe184013000 comp rx=0 tx=0).stop 2023-09-30T14:39:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.874+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/2850737032 shutdown_connections 2023-09-30T14:39:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.874+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/2850737032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e281b0 0x55eed8e28630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.874+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/2850737032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e269b0 0x55eed8e26dd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.874+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/2850737032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e27310 0x55eed8e27bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.874+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/2850737032 >> 172.21.15.32:0/2850737032 conn(0x55eed8d96260 msgr2=0x55eed8e1d480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.874+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/2850737032 shutdown_connections 2023-09-30T14:39:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.874+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/2850737032 wait complete. 2023-09-30T14:39:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.874+0000 7fe19b4c5b80 1 -- start start 2023-09-30T14:39:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.875+0000 7fe19b4c5b80 1 Processor -- start 2023-09-30T14:39:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.875+0000 7fe19b4c5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e269b0 0x55eed8e30720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.875+0000 7fe19b4c5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e27310 0x55eed8e30c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.875+0000 7fe19b4c5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e281b0 0x55eed8e37ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.875+0000 7fe19b4c5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed8d74e30 con 0x55eed8e27310 2023-09-30T14:39:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.875+0000 7fe19b4c5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed8d11710 con 0x55eed8e281b0 2023-09-30T14:39:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.875+0000 7fe19b4c5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed8d113b0 con 0x55eed8e269b0 2023-09-30T14:39:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.875+0000 7fe19b4c5b80 10 client.0 _ll_register_callbacks cb 0x55eed8b6e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:13.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.875+0000 7fe18d079700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e269b0 0x55eed8e30720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.875+0000 7fe18c878700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e27310 0x55eed8e30c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.875+0000 7fe18d079700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e269b0 0x55eed8e30720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:50012/0 (socket says 172.21.15.32:50012) 2023-09-30T14:39:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.875+0000 7fe18c878700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e27310 0x55eed8e30c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35632/0 (socket says 172.21.15.32:35632) 2023-09-30T14:39:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.875+0000 7fe18d079700 1 -- 172.21.15.32:0/3130450075 learned_addr learned my addr 172.21.15.32:0/3130450075 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.876+0000 7fe18d87a700 1 --2- 172.21.15.32:0/3130450075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e281b0 0x55eed8e37ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.876+0000 7fe18d079700 1 -- 172.21.15.32:0/3130450075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e281b0 msgr2=0x55eed8e37ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.876+0000 7fe18d079700 1 --2- 172.21.15.32:0/3130450075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e281b0 0x55eed8e37ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.876+0000 7fe18d079700 1 -- 172.21.15.32:0/3130450075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e27310 msgr2=0x55eed8e30c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.876+0000 7fe18d079700 1 --2- 172.21.15.32:0/3130450075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e27310 0x55eed8e30c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.876+0000 7fe18d079700 1 -- 172.21.15.32:0/3130450075 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe184010c20 con 0x55eed8e269b0 2023-09-30T14:39:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.876+0000 7fe18c878700 1 --2- 172.21.15.32:0/3130450075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e27310 0x55eed8e30c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.876+0000 7fe18d079700 1 --2- 172.21.15.32:0/3130450075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e269b0 0x55eed8e30720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe184011e10 tx=0x7fe184011e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc509f2370c09d3d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.876+0000 7fe170ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:39:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.876+0000 7fe170ff9700 1 -- 172.21.15.32:0/3130450075 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184004250 con 0x55eed8e269b0 2023-09-30T14:39:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.876+0000 7fe170ff9700 1 -- 172.21.15.32:0/3130450075 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe184017070 con 0x55eed8e269b0 2023-09-30T14:39:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.876+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/3130450075 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eed8e15250 con 0x55eed8e269b0 2023-09-30T14:39:13.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.876+0000 7fe170ff9700 1 -- 172.21.15.32:0/3130450075 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1840043b0 con 0x55eed8e269b0 2023-09-30T14:39:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.876+0000 7fe19b4c5b80 10 client.25171 fetch_fsmap learned FSMap version 23 2023-09-30T14:39:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.877+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/3130450075 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55eed8b9fde0 con 0x55eed8e269b0 2023-09-30T14:39:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.877+0000 7fe170ff9700 1 -- 172.21.15.32:0/3130450075 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eed8e15250 con 0x55eed8e269b0 2023-09-30T14:39:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.877+0000 7fe170ff9700 1 -- 172.21.15.32:0/3130450075 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe18401ec40 con 0x55eed8e269b0 2023-09-30T14:39:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.877+0000 7fe19b4c5b80 10 client.25171 fetch_fsmap finished waiting for FSMap version 23 2023-09-30T14:39:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.877+0000 7fe19b4c5b80 10 client.25171 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-30T14:39:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.877+0000 7fe19b4c5b80 20 client.25171 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.877+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/3130450075 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x55eed8e3c4c0 0x55eed8e3e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.877+0000 7fe19b4c5b80 4 client.25171 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:39:13.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.877+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/3130450075 --> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eed8d6de10 con 0x55eed8e3c4c0 2023-09-30T14:39:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.877+0000 7fe18c878700 1 --2- 172.21.15.32:0/3130450075 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x55eed8e3c4c0 0x55eed8e3e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.878+0000 7fe170ff9700 10 client.25171 ms_handle_connect on v2:172.21.15.156:6838/1445176203 2023-09-30T14:39:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.878+0000 7fe18c878700 1 --2- 172.21.15.32:0/3130450075 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x55eed8e3c4c0 0x55eed8e3e8e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c34895ff17c03183 server_cookie=ebed88c1bfa85092 in_seq=0 out_seq=0 2023-09-30T14:39:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.878+0000 7fe170ff9700 1 -- 172.21.15.32:0/3130450075 <== mds.1 v2:172.21.15.156:6838/1445176203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55eed8d6de10 con 0x55eed8e3c4c0 2023-09-30T14:39:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.878+0000 7fe170ff9700 10 client.25171 handle_command_reply: tid=0 2023-09-30T14:39:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.878+0000 7fe19b4c5b80 1 client.25171 shutdown 2023-09-30T14:39:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.879+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/3130450075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e269b0 msgr2=0x55eed8e30720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.879+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/3130450075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e269b0 0x55eed8e30720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe184011e10 tx=0x7fe184011e40 comp rx=0 tx=0).stop 2023-09-30T14:39:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.879+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/3130450075 shutdown_connections 2023-09-30T14:39:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.879+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/3130450075 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x55eed8e3c4c0 0x55eed8e3e8e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.879+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/3130450075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e281b0 0x55eed8e37ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.879+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/3130450075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e269b0 0x55eed8e30720 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.879+0000 7fe170ff9700 0 client.25171 ms_handle_reset on v2:172.21.15.156:6838/1445176203 2023-09-30T14:39:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.879+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/3130450075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e27310 0x55eed8e30c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.879+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/3130450075 >> 172.21.15.32:0/3130450075 conn(0x55eed8d96260 msgr2=0x55eed8e1d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.879+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/3130450075 shutdown_connections 2023-09-30T14:39:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.879+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/3130450075 wait complete. 2023-09-30T14:39:13.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.879+0000 7fe19b4c5b80 20 client.25171 trim_cache size 0 max 16384 2023-09-30T14:39:13.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:13 smithi032 ceph-mon[98592]: pgmap v315: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2023-09-30T14:39:13.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:13 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3070367010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:39:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.893+0000 7fe19b4c5b80 1 Processor -- start 2023-09-30T14:39:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.893+0000 7fe19b4c5b80 1 -- start start 2023-09-30T14:39:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.893+0000 7fe19b4c5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e269b0 0x55eed8e28590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.893+0000 7fe19b4c5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e27310 0x55eed8e37c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.893+0000 7fe19b4c5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e281b0 0x55eed8e1bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.893+0000 7fe19b4c5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed8bf5920 con 0x55eed8e269b0 2023-09-30T14:39:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.893+0000 7fe19b4c5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed8e30290 con 0x55eed8e27310 2023-09-30T14:39:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.893+0000 7fe19b4c5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed8e19d60 con 0x55eed8e281b0 2023-09-30T14:39:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.893+0000 7fe18d079700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e269b0 0x55eed8e28590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.893+0000 7fe18d079700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e269b0 0x55eed8e28590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35638/0 (socket says 172.21.15.32:35638) 2023-09-30T14:39:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.893+0000 7fe18d079700 1 -- 172.21.15.32:0/2841033644 learned_addr learned my addr 172.21.15.32:0/2841033644 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.894+0000 7fe18c878700 1 --2- 172.21.15.32:0/2841033644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e27310 0x55eed8e37c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.894+0000 7fe18d87a700 1 --2- 172.21.15.32:0/2841033644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e281b0 0x55eed8e1bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.894+0000 7fe18d079700 1 -- 172.21.15.32:0/2841033644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e281b0 msgr2=0x55eed8e1bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.894+0000 7fe18d079700 1 --2- 172.21.15.32:0/2841033644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e281b0 0x55eed8e1bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.894+0000 7fe18d079700 1 -- 172.21.15.32:0/2841033644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e27310 msgr2=0x55eed8e37c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.894+0000 7fe18d079700 1 --2- 172.21.15.32:0/2841033644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e27310 0x55eed8e37c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.894+0000 7fe18d079700 1 -- 172.21.15.32:0/2841033644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eed8b88c60 con 0x55eed8e269b0 2023-09-30T14:39:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.894+0000 7fe18d87a700 1 --2- 172.21.15.32:0/2841033644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e281b0 0x55eed8e1bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:39:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.894+0000 7fe18d079700 1 --2- 172.21.15.32:0/2841033644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e269b0 0x55eed8e28590 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe18400b7a0 tx=0x7fe18400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb543c76c182420 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.894+0000 7fe173fff700 1 -- 172.21.15.32:0/2841033644 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1840315f0 con 0x55eed8e269b0 2023-09-30T14:39:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.894+0000 7fe173fff700 1 -- 172.21.15.32:0/2841033644 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe184031750 con 0x55eed8e269b0 2023-09-30T14:39:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.894+0000 7fe173fff700 1 -- 172.21.15.32:0/2841033644 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184004540 con 0x55eed8e269b0 2023-09-30T14:39:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.895+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/2841033644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e269b0 msgr2=0x55eed8e28590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.895+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/2841033644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e269b0 0x55eed8e28590 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe18400b7a0 tx=0x7fe18400b730 comp rx=0 tx=0).stop 2023-09-30T14:39:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.895+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/2841033644 shutdown_connections 2023-09-30T14:39:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.895+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/2841033644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e281b0 0x55eed8e1bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.895+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/2841033644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e269b0 0x55eed8e28590 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.895+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/2841033644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e27310 0x55eed8e37c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.895+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/2841033644 >> 172.21.15.32:0/2841033644 conn(0x55eed8d96260 msgr2=0x55eed8e2f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.895+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/2841033644 shutdown_connections 2023-09-30T14:39:13.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.896+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/2841033644 wait complete. 2023-09-30T14:39:13.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.896+0000 7fe19b4c5b80 1 -- start start 2023-09-30T14:39:13.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.896+0000 7fe19b4c5b80 1 Processor -- start 2023-09-30T14:39:13.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.896+0000 7fe19b4c5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e269b0 0x55eed8e33c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.896+0000 7fe19b4c5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e27310 0x55eed8e36190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe19b4c5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e281b0 0x55eed8e366d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe19b4c5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed8e2deb0 con 0x55eed8e281b0 2023-09-30T14:39:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe19b4c5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed8e36c10 con 0x55eed8e269b0 2023-09-30T14:39:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe19b4c5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed8e36dd0 con 0x55eed8e27310 2023-09-30T14:39:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe19b4c5b80 10 client.0 _ll_register_callbacks cb 0x7fe180007420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe18d079700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e269b0 0x55eed8e33c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe18d079700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e269b0 0x55eed8e33c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47120/0 (socket says 172.21.15.32:47120) 2023-09-30T14:39:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe18d079700 1 -- 172.21.15.32:0/1400829522 learned_addr learned my addr 172.21.15.32:0/1400829522 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe18c878700 1 --2- 172.21.15.32:0/1400829522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e27310 0x55eed8e36190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe18d87a700 1 --2- 172.21.15.32:0/1400829522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e281b0 0x55eed8e366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe18d079700 1 -- 172.21.15.32:0/1400829522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e27310 msgr2=0x55eed8e36190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe18d079700 1 --2- 172.21.15.32:0/1400829522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e27310 0x55eed8e36190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe18d079700 1 -- 172.21.15.32:0/1400829522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e281b0 msgr2=0x55eed8e366d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:13.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe18d079700 1 --2- 172.21.15.32:0/1400829522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e281b0 0x55eed8e366d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe18d079700 1 -- 172.21.15.32:0/1400829522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eed8b9fde0 con 0x55eed8e269b0 2023-09-30T14:39:13.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe18d079700 1 --2- 172.21.15.32:0/1400829522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e269b0 0x55eed8e33c40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe18400bf80 tx=0x7fe184031af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75408e79ed95296 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:13.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.897+0000 7fe170ff9700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:39:13.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.898+0000 7fe170ff9700 1 -- 172.21.15.32:0/1400829522 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1840315f0 con 0x55eed8e269b0 2023-09-30T14:39:13.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.898+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/1400829522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe184015070 con 0x55eed8e269b0 2023-09-30T14:39:13.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.898+0000 7fe170ff9700 1 -- 172.21.15.32:0/1400829522 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe184017070 con 0x55eed8e269b0 2023-09-30T14:39:13.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.898+0000 7fe170ff9700 1 -- 172.21.15.32:0/1400829522 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe184003ef0 con 0x55eed8e269b0 2023-09-30T14:39:13.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.898+0000 7fe19b4c5b80 10 client.25193 fetch_fsmap learned FSMap version 23 2023-09-30T14:39:13.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.898+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/1400829522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55eed8bc6250 con 0x55eed8e269b0 2023-09-30T14:39:13.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.898+0000 7fe170ff9700 1 -- 172.21.15.32:0/1400829522 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe184015070 con 0x55eed8e269b0 2023-09-30T14:39:13.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.898+0000 7fe170ff9700 1 -- 172.21.15.32:0/1400829522 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe18403c400 con 0x55eed8e269b0 2023-09-30T14:39:13.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.898+0000 7fe19b4c5b80 10 client.25193 fetch_fsmap finished waiting for FSMap version 23 2023-09-30T14:39:13.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.898+0000 7fe19b4c5b80 10 client.25193 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-30T14:39:13.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.898+0000 7fe19b4c5b80 20 client.25193 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:13.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.899+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/1400829522 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x55eed8e22330 0x55eed8e24750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:13.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.899+0000 7fe19b4c5b80 4 client.25193 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-30T14:39:13.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.899+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/1400829522 --> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eed8d6db30 con 0x55eed8e22330 2023-09-30T14:39:13.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.899+0000 7fe18c878700 1 --2- 172.21.15.32:0/1400829522 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x55eed8e22330 0x55eed8e24750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.899+0000 7fe170ff9700 10 client.25193 ms_handle_connect on v2:172.21.15.156:6838/1445176203 2023-09-30T14:39:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.899+0000 7fe18c878700 1 --2- 172.21.15.32:0/1400829522 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x55eed8e22330 0x55eed8e24750 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f67b51465a499c3 server_cookie=6cc45772e982d80e in_seq=0 out_seq=0 2023-09-30T14:39:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.900+0000 7fe170ff9700 1 -- 172.21.15.32:0/1400829522 <== mds.1 v2:172.21.15.156:6838/1445176203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eed8d6db30 con 0x55eed8e22330 2023-09-30T14:39:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.900+0000 7fe170ff9700 10 client.25193 handle_command_reply: tid=0 2023-09-30T14:39:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.900+0000 7fe19b4c5b80 1 client.25193 shutdown 2023-09-30T14:39:13.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.900+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/1400829522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e269b0 msgr2=0x55eed8e33c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.900+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/1400829522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e269b0 0x55eed8e33c40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe18400bf80 tx=0x7fe184031af0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.900+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/1400829522 shutdown_connections 2023-09-30T14:39:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.900+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/1400829522 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x55eed8e22330 0x55eed8e24750 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.900+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/1400829522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55eed8e281b0 0x55eed8e366d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.900+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/1400829522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eed8e269b0 0x55eed8e33c40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.900+0000 7fe19b4c5b80 1 --2- 172.21.15.32:0/1400829522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eed8e27310 0x55eed8e36190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.900+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/1400829522 >> 172.21.15.32:0/1400829522 conn(0x55eed8d96260 msgr2=0x55eed8e2e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.900+0000 7fe170ff9700 0 client.25193 ms_handle_reset on v2:172.21.15.156:6838/1445176203 2023-09-30T14:39:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.900+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/1400829522 shutdown_connections 2023-09-30T14:39:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.900+0000 7fe19b4c5b80 1 -- 172.21.15.32:0/1400829522 wait complete. 2023-09-30T14:39:13.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.900+0000 7fe19b4c5b80 20 client.25193 trim_cache size 0 max 16384 2023-09-30T14:39:13.908 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2023-09-30T14:39:13.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.901+0000 7fe18f2dd700 1 -- 172.21.15.32:0/2846910876 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1740a0fa0 msgr2=0x7fe1740a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.901+0000 7fe18f2dd700 1 --2- 172.21.15.32:0/2846910876 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1740a0fa0 0x7fe1740a3460 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe188197b50 tx=0x7fe18400b410 comp rx=0 tx=0).stop 2023-09-30T14:39:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.901+0000 7fe18f2dd700 1 -- 172.21.15.32:0/2846910876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe188101920 msgr2=0x7fe18819e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.901+0000 7fe18f2dd700 1 --2- 172.21.15.32:0/2846910876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe188101920 0x7fe18819e220 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe180001220 tx=0x7fe180004d20 comp rx=0 tx=0).stop 2023-09-30T14:39:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.901+0000 7fe18f2dd700 1 -- 172.21.15.32:0/2846910876 shutdown_connections 2023-09-30T14:39:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.901+0000 7fe18f2dd700 1 --2- 172.21.15.32:0/2846910876 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe1740a0fa0 0x7fe1740a3460 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.901+0000 7fe18f2dd700 1 --2- 172.21.15.32:0/2846910876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe188101920 0x7fe18819e220 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.901+0000 7fe18f2dd700 1 --2- 172.21.15.32:0/2846910876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1880ffd90 0x7fe188196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.901+0000 7fe18f2dd700 1 --2- 172.21.15.32:0/2846910876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe188100f80 0x7fe1881971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.901+0000 7fe18f2dd700 1 -- 172.21.15.32:0/2846910876 >> 172.21.15.32:0/2846910876 conn(0x7fe1880fb2e0 msgr2=0x7fe188104400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.901+0000 7fe18f2dd700 1 -- 172.21.15.32:0/2846910876 shutdown_connections 2023-09-30T14:39:13.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:13.901+0000 7fe18f2dd700 1 -- 172.21.15.32:0/2846910876 wait complete. 2023-09-30T14:39:13.915 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-30T14:39:14.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.050+0000 7f9e2efe7700 1 -- 172.21.15.32:0/129271594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e28102a60 msgr2=0x7f9e28109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:14.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.050+0000 7f9e2efe7700 1 --2- 172.21.15.32:0/129271594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e28102a60 0x7f9e28109310 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000b0a0 tx=0x7f9e2000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.050+0000 7f9e2efe7700 1 -- 172.21.15.32:0/129271594 shutdown_connections 2023-09-30T14:39:14.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.050+0000 7f9e2efe7700 1 --2- 172.21.15.32:0/129271594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e28102a60 0x7f9e28109310 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.050+0000 7f9e2efe7700 1 --2- 172.21.15.32:0/129271594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e280ff0f0 0x7f9e280ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.050+0000 7f9e2efe7700 1 --2- 172.21.15.32:0/129271594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e281020c0 0x7f9e28102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.050+0000 7f9e2efe7700 1 -- 172.21.15.32:0/129271594 >> 172.21.15.32:0/129271594 conn(0x7f9e280faa20 msgr2=0x7f9e280fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:14.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.050+0000 7f9e2efe7700 1 -- 172.21.15.32:0/129271594 shutdown_connections 2023-09-30T14:39:14.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.050+0000 7f9e2efe7700 1 -- 172.21.15.32:0/129271594 wait complete. 2023-09-30T14:39:14.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.051+0000 7f9e2efe7700 1 Processor -- start 2023-09-30T14:39:14.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.051+0000 7f9e2efe7700 1 -- start start 2023-09-30T14:39:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.051+0000 7f9e2efe7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e280ff0f0 0x7f9e28196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.051+0000 7f9e2efe7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e281020c0 0x7f9e281971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.051+0000 7f9e2efe7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e28102a60 0x7f9e2819e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.051+0000 7f9e2efe7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2819e800 con 0x7f9e280ff0f0 2023-09-30T14:39:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.051+0000 7f9e2efe7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2819e970 con 0x7f9e28102a60 2023-09-30T14:39:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.051+0000 7f9e2efe7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2819eb30 con 0x7f9e281020c0 2023-09-30T14:39:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.051+0000 7f9e2cd83700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e280ff0f0 0x7f9e28196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.051+0000 7f9e2cd83700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e280ff0f0 0x7f9e28196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35656/0 (socket says 172.21.15.32:35656) 2023-09-30T14:39:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.051+0000 7f9e2cd83700 1 -- 172.21.15.32:0/1935749482 learned_addr learned my addr 172.21.15.32:0/1935749482 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:14.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.051+0000 7f9e2d584700 1 --2- 172.21.15.32:0/1935749482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e28102a60 0x7f9e2819e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.052+0000 7f9e27fff700 1 --2- 172.21.15.32:0/1935749482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e281020c0 0x7f9e281971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.052+0000 7f9e2cd83700 1 -- 172.21.15.32:0/1935749482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e281020c0 msgr2=0x7f9e281971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.052+0000 7f9e2cd83700 1 --2- 172.21.15.32:0/1935749482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e281020c0 0x7f9e281971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.052+0000 7f9e2cd83700 1 -- 172.21.15.32:0/1935749482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e28102a60 msgr2=0x7f9e2819e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.052+0000 7f9e2cd83700 1 --2- 172.21.15.32:0/1935749482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e28102a60 0x7f9e2819e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.052+0000 7f9e2cd83700 1 -- 172.21.15.32:0/1935749482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e14009520 con 0x7f9e280ff0f0 2023-09-30T14:39:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.052+0000 7f9e2cd83700 1 --2- 172.21.15.32:0/1935749482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e280ff0f0 0x7f9e28196c90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00bf30 tx=0x7f9e1c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.052+0000 7f9e25ffb700 1 -- 172.21.15.32:0/1935749482 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c004220 con 0x7f9e280ff0f0 2023-09-30T14:39:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.052+0000 7f9e2efe7700 1 -- 172.21.15.32:0/1935749482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e20009d00 con 0x7f9e280ff0f0 2023-09-30T14:39:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.052+0000 7f9e25ffb700 1 -- 172.21.15.32:0/1935749482 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e1c016e90 con 0x7f9e280ff0f0 2023-09-30T14:39:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.052+0000 7f9e25ffb700 1 -- 172.21.15.32:0/1935749482 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c0207f0 con 0x7f9e280ff0f0 2023-09-30T14:39:14.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.052+0000 7f9e2efe7700 1 -- 172.21.15.32:0/1935749482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e281002e0 con 0x7f9e280ff0f0 2023-09-30T14:39:14.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.056+0000 7f9e25ffb700 1 -- 172.21.15.32:0/1935749482 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f9e1c004380 con 0x7f9e280ff0f0 2023-09-30T14:39:14.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.056+0000 7f9e3b1cfb80 1 Processor -- start 2023-09-30T14:39:14.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.056+0000 7f9e3b1cfb80 1 -- start start 2023-09-30T14:39:14.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.056+0000 7f9e3b1cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815e5a0 0x5557a815e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.056+0000 7f9e3b1cfb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815ef00 0x5557a815f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.057+0000 7f9e3b1cfb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a8160220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.057+0000 7f9e3b1cfb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a80c6460 con 0x5557a815fda0 2023-09-30T14:39:14.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.057+0000 7f9e3b1cfb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a80b7c20 con 0x5557a815ef00 2023-09-30T14:39:14.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.057+0000 7f9e3b1cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a80b5540 con 0x5557a815e5a0 2023-09-30T14:39:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.057+0000 7f9e25ffb700 1 --2- 172.21.15.32:0/1935749482 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9e180a1020 0x7f9e180a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.057+0000 7f9e25ffb700 1 -- 172.21.15.32:0/1935749482 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9e1c0d46c0 con 0x7f9e280ff0f0 2023-09-30T14:39:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.057+0000 7f9e2d584700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815e5a0 0x5557a815e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.057+0000 7f9e2d584700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815e5a0 0x5557a815e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:50046/0 (socket says 172.21.15.32:50046) 2023-09-30T14:39:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.057+0000 7f9e2d584700 1 -- 172.21.15.32:0/3689912340 learned_addr learned my addr 172.21.15.32:0/3689912340 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.057+0000 7f9e27fff700 1 --2- 172.21.15.32:0/3689912340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a8160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.057+0000 7f9e2d584700 1 --2- 172.21.15.32:0/1935749482 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9e180a1020 0x7f9e180a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.058+0000 7f9e2d584700 1 -- 172.21.15.32:0/3689912340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815ef00 msgr2=0x5557a815f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.058+0000 7f9e2d584700 1 --2- 172.21.15.32:0/3689912340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815ef00 0x5557a815f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.058+0000 7f9e2d584700 1 -- 172.21.15.32:0/3689912340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 msgr2=0x5557a8160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.058+0000 7f9e2d584700 1 --2- 172.21.15.32:0/3689912340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a8160220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.058+0000 7f9e2d584700 1 -- 172.21.15.32:0/3689912340 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557a81138d0 con 0x5557a815e5a0 2023-09-30T14:39:14.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.058+0000 7f9e27fff700 1 --2- 172.21.15.32:0/3689912340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a8160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.058+0000 7f9e2d584700 1 --2- 172.21.15.32:0/3689912340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815e5a0 0x5557a815e9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009cd0 tx=0x7f9e20008f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3812e353dfca6f19 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.058+0000 7f9e2d584700 1 --2- 172.21.15.32:0/1935749482 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9e180a1020 0x7f9e180a34e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9e2001a000 tx=0x7f9e20026000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.058+0000 7f9e12ffd700 1 -- 172.21.15.32:0/3689912340 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000e040 con 0x5557a815e5a0 2023-09-30T14:39:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.058+0000 7f9e12ffd700 1 -- 172.21.15.32:0/3689912340 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e20020450 con 0x5557a815e5a0 2023-09-30T14:39:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.058+0000 7f9e12ffd700 1 -- 172.21.15.32:0/3689912340 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e200052a0 con 0x5557a815e5a0 2023-09-30T14:39:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.059+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/3689912340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815e5a0 msgr2=0x5557a815e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.059+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/3689912340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815e5a0 0x5557a815e9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009cd0 tx=0x7f9e20008f30 comp rx=0 tx=0).stop 2023-09-30T14:39:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.059+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/3689912340 shutdown_connections 2023-09-30T14:39:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.059+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/3689912340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a8160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.059+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/3689912340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815e5a0 0x5557a815e9c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.059+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/3689912340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815ef00 0x5557a815f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.059+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/3689912340 >> 172.21.15.32:0/3689912340 conn(0x5557a80d2260 msgr2=0x5557a8159480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.059+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/3689912340 shutdown_connections 2023-09-30T14:39:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.059+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/3689912340 wait complete. 2023-09-30T14:39:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.059+0000 7f9e3b1cfb80 1 -- start start 2023-09-30T14:39:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.060+0000 7f9e3b1cfb80 1 Processor -- start 2023-09-30T14:39:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.060+0000 7f9e3b1cfb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815e5a0 0x5557a81683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.060+0000 7f9e3b1cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815ef00 0x5557a8168900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.060+0000 7f9e3b1cfb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a816f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.060+0000 7f9e3b1cfb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a80b0e30 con 0x5557a815fda0 2023-09-30T14:39:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.060+0000 7f9e3b1cfb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a804d710 con 0x5557a815e5a0 2023-09-30T14:39:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.060+0000 7f9e3b1cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a804d3b0 con 0x5557a815ef00 2023-09-30T14:39:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.060+0000 7f9e3b1cfb80 10 client.0 _ll_register_callbacks cb 0x5557a7eaa800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.060+0000 7f9e2d584700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a816f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.060+0000 7f9e2d584700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a816f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35672/0 (socket says 172.21.15.32:35672) 2023-09-30T14:39:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.060+0000 7f9e2d584700 1 -- 172.21.15.32:0/2154993537 learned_addr learned my addr 172.21.15.32:0/2154993537 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.060+0000 7f9e2cd83700 1 --2- 172.21.15.32:0/2154993537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815e5a0 0x5557a81683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.061+0000 7f9e27fff700 1 --2- 172.21.15.32:0/2154993537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815ef00 0x5557a8168900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.061+0000 7f9e2d584700 1 -- 172.21.15.32:0/2154993537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815ef00 msgr2=0x5557a8168900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.061+0000 7f9e2d584700 1 --2- 172.21.15.32:0/2154993537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815ef00 0x5557a8168900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.061+0000 7f9e2d584700 1 -- 172.21.15.32:0/2154993537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815e5a0 msgr2=0x5557a81683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.061+0000 7f9e2d584700 1 --2- 172.21.15.32:0/2154993537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815e5a0 0x5557a81683c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.061+0000 7f9e2d584700 1 -- 172.21.15.32:0/2154993537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e140099c0 con 0x5557a815fda0 2023-09-30T14:39:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.061+0000 7f9e27fff700 1 --2- 172.21.15.32:0/2154993537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815ef00 0x5557a8168900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:39:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.061+0000 7f9e2d584700 1 --2- 172.21.15.32:0/2154993537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a816f940 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009bb0 tx=0x7f9e20009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09d1ed6b43613fa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.061+0000 7f9e07fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.061+0000 7f9e07fff700 1 -- 172.21.15.32:0/2154993537 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e200052f0 con 0x5557a815fda0 2023-09-30T14:39:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.061+0000 7f9e07fff700 1 -- 172.21.15.32:0/2154993537 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9e20003a00 con 0x5557a815fda0 2023-09-30T14:39:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.061+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/2154993537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557a8151250 con 0x5557a815fda0 2023-09-30T14:39:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.061+0000 7f9e07fff700 1 -- 172.21.15.32:0/2154993537 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e20003bc0 con 0x5557a815fda0 2023-09-30T14:39:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.062+0000 7f9e3b1cfb80 10 client.15498 fetch_fsmap learned FSMap version 23 2023-09-30T14:39:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.062+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/2154993537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f9e28109c10 con 0x5557a815fda0 2023-09-30T14:39:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.062+0000 7f9e07fff700 1 -- 172.21.15.32:0/2154993537 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5557a8151250 con 0x5557a815fda0 2023-09-30T14:39:14.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.062+0000 7f9e07fff700 1 -- 172.21.15.32:0/2154993537 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f9e20032780 con 0x5557a815fda0 2023-09-30T14:39:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.062+0000 7f9e3b1cfb80 10 client.15498 fetch_fsmap finished waiting for FSMap version 23 2023-09-30T14:39:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.062+0000 7f9e3b1cfb80 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.062+0000 7f9e3b1cfb80 20 client.15498 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.062+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/2154993537 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5557a8173f80 0x5557a81763a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.062+0000 7f9e3b1cfb80 4 client.15498 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:39:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.062+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/2154993537 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557a80a9e10 con 0x5557a8173f80 2023-09-30T14:39:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.062+0000 7f9e2cd83700 1 --2- 172.21.15.32:0/2154993537 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5557a8173f80 0x5557a81763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.063+0000 7f9e07fff700 10 client.15498 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.063+0000 7f9e2cd83700 1 --2- 172.21.15.32:0/2154993537 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5557a8173f80 0x5557a81763a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ff5aa99f35080f3 server_cookie=f45c95d20371917e in_seq=0 out_seq=0 2023-09-30T14:39:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.064+0000 7f9e07fff700 1 -- 172.21.15.32:0/2154993537 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5557a80a9e10 con 0x5557a8173f80 2023-09-30T14:39:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.064+0000 7f9e07fff700 10 client.15498 handle_command_reply: tid=0 2023-09-30T14:39:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.064+0000 7f9e3b1cfb80 1 client.15498 shutdown 2023-09-30T14:39:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.064+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/2154993537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 msgr2=0x5557a816f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.064+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/2154993537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a816f940 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009bb0 tx=0x7f9e20009be0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.064+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/2154993537 shutdown_connections 2023-09-30T14:39:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.064+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/2154993537 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5557a8173f80 0x5557a81763a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.064+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/2154993537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a816f940 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.064+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/2154993537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815e5a0 0x5557a81683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.064+0000 7f9e07fff700 0 client.15498 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.064+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/2154993537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815ef00 0x5557a8168900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.064+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/2154993537 >> 172.21.15.32:0/2154993537 conn(0x5557a80d2260 msgr2=0x5557a8159290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:14.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.064+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/2154993537 shutdown_connections 2023-09-30T14:39:14.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.064+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/2154993537 wait complete. 2023-09-30T14:39:14.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.064+0000 7f9e3b1cfb80 20 client.15498 trim_cache size 0 max 16384 2023-09-30T14:39:14.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.078+0000 7f9e3b1cfb80 1 Processor -- start 2023-09-30T14:39:14.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.078+0000 7f9e3b1cfb80 1 -- start start 2023-09-30T14:39:14.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.078+0000 7f9e3b1cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815e5a0 0x5557a81683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.078+0000 7f9e3b1cfb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815ef00 0x5557a816f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.078+0000 7f9e3b1cfb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a8168900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.078+0000 7f9e3b1cfb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a8167f90 con 0x5557a815fda0 2023-09-30T14:39:14.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.078+0000 7f9e3b1cfb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a7f31920 con 0x5557a815ef00 2023-09-30T14:39:14.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.078+0000 7f9e3b1cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a8167870 con 0x5557a815e5a0 2023-09-30T14:39:14.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.079+0000 7f9e2d584700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a8168900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.079+0000 7f9e2d584700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a8168900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35680/0 (socket says 172.21.15.32:35680) 2023-09-30T14:39:14.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.079+0000 7f9e2d584700 1 -- 172.21.15.32:0/589752156 learned_addr learned my addr 172.21.15.32:0/589752156 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:14.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.079+0000 7f9e2cd83700 1 --2- 172.21.15.32:0/589752156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815e5a0 0x5557a81683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.079+0000 7f9e27fff700 1 --2- 172.21.15.32:0/589752156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815ef00 0x5557a816f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.079+0000 7f9e2d584700 1 -- 172.21.15.32:0/589752156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815e5a0 msgr2=0x5557a81683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:14.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.079+0000 7f9e2d584700 1 --2- 172.21.15.32:0/589752156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815e5a0 0x5557a81683c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.079+0000 7f9e2d584700 1 -- 172.21.15.32:0/589752156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815ef00 msgr2=0x5557a816f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:14.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.079+0000 7f9e2d584700 1 --2- 172.21.15.32:0/589752156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815ef00 0x5557a816f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.079+0000 7f9e2d584700 1 -- 172.21.15.32:0/589752156 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557a7edbde0 con 0x5557a815fda0 2023-09-30T14:39:14.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.079+0000 7f9e27fff700 1 --2- 172.21.15.32:0/589752156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815ef00 0x5557a816f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:39:14.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.079+0000 7f9e2d584700 1 --2- 172.21.15.32:0/589752156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a8168900 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009440 tx=0x7f9e20006210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd991262a2d288d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:14.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.079+0000 7f9e137fe700 1 -- 172.21.15.32:0/589752156 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20024070 con 0x5557a815fda0 2023-09-30T14:39:14.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.079+0000 7f9e137fe700 1 -- 172.21.15.32:0/589752156 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e20020e90 con 0x5557a815fda0 2023-09-30T14:39:14.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.079+0000 7f9e137fe700 1 -- 172.21.15.32:0/589752156 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e200053e0 con 0x5557a815fda0 2023-09-30T14:39:14.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.080+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/589752156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 msgr2=0x5557a8168900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:14.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.080+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/589752156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a8168900 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009440 tx=0x7f9e20006210 comp rx=0 tx=0).stop 2023-09-30T14:39:14.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.080+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/589752156 shutdown_connections 2023-09-30T14:39:14.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.080+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/589752156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815fda0 0x5557a8168900 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.080+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/589752156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815e5a0 0x5557a81683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.080+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/589752156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815ef00 0x5557a816f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.080+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/589752156 >> 172.21.15.32:0/589752156 conn(0x5557a80d2260 msgr2=0x5557a8170170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:14.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.080+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/589752156 shutdown_connections 2023-09-30T14:39:14.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.080+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/589752156 wait complete. 2023-09-30T14:39:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.081+0000 7f9e3b1cfb80 1 -- start start 2023-09-30T14:39:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.081+0000 7f9e3b1cfb80 1 Processor -- start 2023-09-30T14:39:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.081+0000 7f9e3b1cfb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815e5a0 0x5557a816d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.081+0000 7f9e3b1cfb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815ef00 0x5557a816d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.081+0000 7f9e3b1cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815fda0 0x5557a816e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.081+0000 7f9e3b1cfb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a816edd0 con 0x5557a815e5a0 2023-09-30T14:39:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.081+0000 7f9e3b1cfb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a816ef40 con 0x5557a815ef00 2023-09-30T14:39:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.081+0000 7f9e3b1cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557a816f100 con 0x5557a815fda0 2023-09-30T14:39:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.081+0000 7f9e3b1cfb80 10 client.0 _ll_register_callbacks cb 0x7f9e140097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.081+0000 7f9e2cd83700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815e5a0 0x5557a816d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.081+0000 7f9e2cd83700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815e5a0 0x5557a816d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35684/0 (socket says 172.21.15.32:35684) 2023-09-30T14:39:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.082+0000 7f9e2cd83700 1 -- 172.21.15.32:0/726675953 learned_addr learned my addr 172.21.15.32:0/726675953 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.082+0000 7f9e2d584700 1 --2- 172.21.15.32:0/726675953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815fda0 0x5557a816e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.082+0000 7f9e27fff700 1 --2- 172.21.15.32:0/726675953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815ef00 0x5557a816d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.082+0000 7f9e2cd83700 1 -- 172.21.15.32:0/726675953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815fda0 msgr2=0x5557a816e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.082+0000 7f9e2cd83700 1 --2- 172.21.15.32:0/726675953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815fda0 0x5557a816e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.082+0000 7f9e2cd83700 1 -- 172.21.15.32:0/726675953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815ef00 msgr2=0x5557a816d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.082+0000 7f9e2cd83700 1 --2- 172.21.15.32:0/726675953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815ef00 0x5557a816d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.082+0000 7f9e2cd83700 1 -- 172.21.15.32:0/726675953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e28109c10 con 0x5557a815e5a0 2023-09-30T14:39:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.082+0000 7f9e2cd83700 1 --2- 172.21.15.32:0/726675953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815e5a0 0x5557a816d340 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c09e5a0 tx=0x7f9e1c003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fb4ab747e2fae9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.082+0000 7f9e07fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:14.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.082+0000 7f9e07fff700 1 -- 172.21.15.32:0/726675953 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c0a9ec0 con 0x5557a815e5a0 2023-09-30T14:39:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.082+0000 7f9e07fff700 1 -- 172.21.15.32:0/726675953 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9e1c0d2280 con 0x5557a815e5a0 2023-09-30T14:39:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.082+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/726675953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e20020450 con 0x5557a815e5a0 2023-09-30T14:39:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.082+0000 7f9e07fff700 1 -- 172.21.15.32:0/726675953 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e1c0a1020 con 0x5557a815e5a0 2023-09-30T14:39:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.082+0000 7f9e3b1cfb80 10 client.15504 fetch_fsmap learned FSMap version 23 2023-09-30T14:39:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.083+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/726675953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5557a7ec4c60 con 0x5557a815e5a0 2023-09-30T14:39:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.083+0000 7f9e07fff700 1 -- 172.21.15.32:0/726675953 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e1c084020 con 0x5557a815e5a0 2023-09-30T14:39:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.083+0000 7f9e07fff700 1 -- 172.21.15.32:0/726675953 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f9e1c0c10b0 con 0x5557a815e5a0 2023-09-30T14:39:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.083+0000 7f9e3b1cfb80 10 client.15504 fetch_fsmap finished waiting for FSMap version 23 2023-09-30T14:39:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.083+0000 7f9e3b1cfb80 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.083+0000 7f9e3b1cfb80 20 client.15504 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.083+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/726675953 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5557a81c8a70 0x5557a81cae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.083+0000 7f9e3b1cfb80 4 client.15504 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-30T14:39:14.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.083+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/726675953 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5557a80a9b30 con 0x5557a81c8a70 2023-09-30T14:39:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.083+0000 7f9e27fff700 1 --2- 172.21.15.32:0/726675953 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5557a81c8a70 0x5557a81cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.084+0000 7f9e07fff700 10 client.15504 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.084+0000 7f9e27fff700 1 --2- 172.21.15.32:0/726675953 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5557a81c8a70 0x5557a81cae90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ffe65ec2db34766 server_cookie=bcd86461d1edf93a in_seq=0 out_seq=0 2023-09-30T14:39:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.084+0000 7f9e07fff700 1 -- 172.21.15.32:0/726675953 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5557a80a9b30 con 0x5557a81c8a70 2023-09-30T14:39:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.084+0000 7f9e07fff700 10 client.15504 handle_command_reply: tid=0 2023-09-30T14:39:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.084+0000 7f9e3b1cfb80 1 client.15504 shutdown 2023-09-30T14:39:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.084+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/726675953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815e5a0 msgr2=0x5557a816d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.084+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/726675953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815e5a0 0x5557a816d340 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c09e5a0 tx=0x7f9e1c003ed0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.084+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/726675953 shutdown_connections 2023-09-30T14:39:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.084+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/726675953 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5557a81c8a70 0x5557a81cae90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.084+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/726675953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5557a815fda0 0x5557a816e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.084+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/726675953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5557a815e5a0 0x5557a816d340 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.084+0000 7f9e3b1cfb80 1 --2- 172.21.15.32:0/726675953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557a815ef00 0x5557a816d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.084+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/726675953 >> 172.21.15.32:0/726675953 conn(0x5557a80d2260 msgr2=0x5557a8162d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.084+0000 7f9e07fff700 0 client.15504 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.084+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/726675953 shutdown_connections 2023-09-30T14:39:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.085+0000 7f9e3b1cfb80 1 -- 172.21.15.32:0/726675953 wait complete. 2023-09-30T14:39:14.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.085+0000 7f9e3b1cfb80 20 client.15504 trim_cache size 0 max 16384 2023-09-30T14:39:14.090 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2023-09-30T14:39:14.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.085+0000 7f9e2efe7700 1 -- 172.21.15.32:0/1935749482 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9e180a1020 msgr2=0x7f9e180a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:14.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.085+0000 7f9e2efe7700 1 --2- 172.21.15.32:0/1935749482 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9e180a1020 0x7f9e180a34e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9e2001a000 tx=0x7f9e20026000 comp rx=0 tx=0).stop 2023-09-30T14:39:14.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.085+0000 7f9e2efe7700 1 -- 172.21.15.32:0/1935749482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e280ff0f0 msgr2=0x7f9e28196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:14.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.085+0000 7f9e2efe7700 1 --2- 172.21.15.32:0/1935749482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e280ff0f0 0x7f9e28196c90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00bf30 tx=0x7f9e1c00dbb0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.085+0000 7f9e2efe7700 1 -- 172.21.15.32:0/1935749482 shutdown_connections 2023-09-30T14:39:14.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.085+0000 7f9e2efe7700 1 --2- 172.21.15.32:0/1935749482 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9e180a1020 0x7f9e180a34e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.085+0000 7f9e2efe7700 1 --2- 172.21.15.32:0/1935749482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e28102a60 0x7f9e2819e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.085+0000 7f9e2efe7700 1 --2- 172.21.15.32:0/1935749482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e280ff0f0 0x7f9e28196c90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.085+0000 7f9e2efe7700 1 --2- 172.21.15.32:0/1935749482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e281020c0 0x7f9e281971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:14.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.085+0000 7f9e2efe7700 1 -- 172.21.15.32:0/1935749482 >> 172.21.15.32:0/1935749482 conn(0x7f9e280faa20 msgr2=0x7f9e280fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:14.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.086+0000 7f9e2efe7700 1 -- 172.21.15.32:0/1935749482 shutdown_connections 2023-09-30T14:39:14.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:14.086+0000 7f9e2efe7700 1 -- 172.21.15.32:0/1935749482 wait complete. 2023-09-30T14:39:15.100 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-30T14:39:15.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-30T14:39:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.248+0000 7fe700725700 1 -- 172.21.15.32:0/2476915040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f81042a0 msgr2=0x7fe6f8106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.248+0000 7fe700725700 1 --2- 172.21.15.32:0/2476915040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f81042a0 0x7fe6f8106690 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009990 tx=0x7fe6f4009ca0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.248+0000 7fe700725700 1 -- 172.21.15.32:0/2476915040 shutdown_connections 2023-09-30T14:39:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.248+0000 7fe700725700 1 --2- 172.21.15.32:0/2476915040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f8106bd0 0x7fe6f8109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.248+0000 7fe700725700 1 --2- 172.21.15.32:0/2476915040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f8101970 0x7fe6f8103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.248+0000 7fe700725700 1 --2- 172.21.15.32:0/2476915040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f81042a0 0x7fe6f8106690 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.248+0000 7fe700725700 1 -- 172.21.15.32:0/2476915040 >> 172.21.15.32:0/2476915040 conn(0x7fe6f80fb320 msgr2=0x7fe6f80fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.248+0000 7fe700725700 1 -- 172.21.15.32:0/2476915040 shutdown_connections 2023-09-30T14:39:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.248+0000 7fe700725700 1 -- 172.21.15.32:0/2476915040 wait complete. 2023-09-30T14:39:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.249+0000 7fe700725700 1 Processor -- start 2023-09-30T14:39:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.249+0000 7fe700725700 1 -- start start 2023-09-30T14:39:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.249+0000 7fe700725700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f8101970 0x7fe6f8196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.249+0000 7fe700725700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f81042a0 0x7fe6f8197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.249+0000 7fe700725700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f8106bd0 0x7fe6f819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.249+0000 7fe700725700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f819e680 con 0x7fe6f81042a0 2023-09-30T14:39:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.249+0000 7fe700725700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f819e7c0 con 0x7fe6f8106bd0 2023-09-30T14:39:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.249+0000 7fe700725700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f819e950 con 0x7fe6f8101970 2023-09-30T14:39:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.249+0000 7fe6fdcc0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f81042a0 0x7fe6f8197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.250+0000 7fe6fdcc0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f81042a0 0x7fe6f8197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35700/0 (socket says 172.21.15.32:35700) 2023-09-30T14:39:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.250+0000 7fe6fdcc0700 1 -- 172.21.15.32:0/1201917305 learned_addr learned my addr 172.21.15.32:0/1201917305 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.250+0000 7fe6fe4c1700 1 --2- 172.21.15.32:0/1201917305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f8101970 0x7fe6f8196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.250+0000 7fe6fecc2700 1 --2- 172.21.15.32:0/1201917305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f8106bd0 0x7fe6f819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.250+0000 7fe6fdcc0700 1 -- 172.21.15.32:0/1201917305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f8101970 msgr2=0x7fe6f8196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.250+0000 7fe6fdcc0700 1 --2- 172.21.15.32:0/1201917305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f8101970 0x7fe6f8196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.250+0000 7fe6fdcc0700 1 -- 172.21.15.32:0/1201917305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f8106bd0 msgr2=0x7fe6f819e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.250+0000 7fe6fdcc0700 1 --2- 172.21.15.32:0/1201917305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f8106bd0 0x7fe6f819e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.250+0000 7fe6fdcc0700 1 -- 172.21.15.32:0/1201917305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f40095f0 con 0x7fe6f81042a0 2023-09-30T14:39:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.250+0000 7fe6fecc2700 1 --2- 172.21.15.32:0/1201917305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f8106bd0 0x7fe6f819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:39:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.250+0000 7fe6fdcc0700 1 --2- 172.21.15.32:0/1201917305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f81042a0 0x7fe6f8197040 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40059d0 tx=0x7fe6f400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.250+0000 7fe6ef7fe700 1 -- 172.21.15.32:0/1201917305 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f400fe50 con 0x7fe6f81042a0 2023-09-30T14:39:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.250+0000 7fe6ef7fe700 1 -- 172.21.15.32:0/1201917305 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6f4023070 con 0x7fe6f81042a0 2023-09-30T14:39:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.250+0000 7fe6ef7fe700 1 -- 172.21.15.32:0/1201917305 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4017650 con 0x7fe6f81042a0 2023-09-30T14:39:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.251+0000 7fe700725700 1 -- 172.21.15.32:0/1201917305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f80ff760 con 0x7fe6f81042a0 2023-09-30T14:39:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.251+0000 7fe700725700 1 -- 172.21.15.32:0/1201917305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f80ffbc0 con 0x7fe6f81042a0 2023-09-30T14:39:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.254+0000 7fe6ef7fe700 1 -- 172.21.15.32:0/1201917305 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fe6f4025030 con 0x7fe6f81042a0 2023-09-30T14:39:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.255+0000 7fe6ef7fe700 1 --2- 172.21.15.32:0/1201917305 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe6e40a1020 0x7fe6e40a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.255+0000 7fe6ef7fe700 1 -- 172.21.15.32:0/1201917305 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe6f40d1e50 con 0x7fe6f81042a0 2023-09-30T14:39:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.255+0000 7fe6fe4c1700 1 --2- 172.21.15.32:0/1201917305 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe6e40a1020 0x7fe6e40a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.256+0000 7fe70c90db80 1 Processor -- start 2023-09-30T14:39:15.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.256+0000 7fe70c90db80 1 -- start start 2023-09-30T14:39:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.256+0000 7fe70c90db80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9620 0x55e7977c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.256+0000 7fe70c90db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977c9f80 0x55e7977ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.256+0000 7fe70c90db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977cae20 0x55e7977cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.256+0000 7fe70c90db80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7977314e0 con 0x55e7977c9620 2023-09-30T14:39:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.256+0000 7fe70c90db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e797722ca0 con 0x55e7977cae20 2023-09-30T14:39:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.256+0000 7fe70c90db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7977205c0 con 0x55e7977c9f80 2023-09-30T14:39:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.256+0000 7fe6fecc2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977cae20 0x55e7977cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.257+0000 7fe6fecc2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977cae20 0x55e7977cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47182/0 (socket says 172.21.15.32:47182) 2023-09-30T14:39:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.257+0000 7fe6fecc2700 1 -- 172.21.15.32:0/2303409421 learned_addr learned my addr 172.21.15.32:0/2303409421 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.257+0000 7fe6fdcc0700 1 --2- 172.21.15.32:0/2303409421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977c9f80 0x55e7977ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.257+0000 7fe6fe4c1700 1 --2- 172.21.15.32:0/1201917305 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe6e40a1020 0x7fe6e40a34e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009c60 tx=0x7fe6e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.257+0000 7fe6fdcc0700 1 -- 172.21.15.32:0/2303409421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977cae20 msgr2=0x55e7977cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.257+0000 7fe6fdcc0700 1 --2- 172.21.15.32:0/2303409421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977cae20 0x55e7977cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.257+0000 7fe6fdcc0700 1 -- 172.21.15.32:0/2303409421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9620 msgr2=0x55e7977c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.257+0000 7fe6fdcc0700 1 --2- 172.21.15.32:0/2303409421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9620 0x55e7977c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.257+0000 7fe6fdcc0700 1 -- 172.21.15.32:0/2303409421 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e79777e950 con 0x55e7977c9f80 2023-09-30T14:39:15.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.257+0000 7fe6fecc2700 1 --2- 172.21.15.32:0/2303409421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977cae20 0x55e7977cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.257+0000 7fe6fdcc0700 1 --2- 172.21.15.32:0/2303409421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977c9f80 0x55e7977ca840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40cf9b0 tx=0x7fe6f4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f30db3ec4516154 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.257+0000 7fe6ecff9700 1 -- 172.21.15.32:0/2303409421 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f409fab0 con 0x55e7977c9f80 2023-09-30T14:39:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.257+0000 7fe6ecff9700 1 -- 172.21.15.32:0/2303409421 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6f409f3b0 con 0x55e7977c9f80 2023-09-30T14:39:15.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.257+0000 7fe6ecff9700 1 -- 172.21.15.32:0/2303409421 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f409f830 con 0x55e7977c9f80 2023-09-30T14:39:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.258+0000 7fe70c90db80 1 -- 172.21.15.32:0/2303409421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977c9f80 msgr2=0x55e7977ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.258+0000 7fe70c90db80 1 --2- 172.21.15.32:0/2303409421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977c9f80 0x55e7977ca840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40cf9b0 tx=0x7fe6f4009ca0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.258+0000 7fe70c90db80 1 -- 172.21.15.32:0/2303409421 shutdown_connections 2023-09-30T14:39:15.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.258+0000 7fe70c90db80 1 --2- 172.21.15.32:0/2303409421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977cae20 0x55e7977cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.258+0000 7fe70c90db80 1 --2- 172.21.15.32:0/2303409421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9620 0x55e7977c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.258+0000 7fe70c90db80 1 --2- 172.21.15.32:0/2303409421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977c9f80 0x55e7977ca840 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.258+0000 7fe70c90db80 1 -- 172.21.15.32:0/2303409421 >> 172.21.15.32:0/2303409421 conn(0x55e79773d2e0 msgr2=0x55e7977c4500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:15.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.258+0000 7fe70c90db80 1 -- 172.21.15.32:0/2303409421 shutdown_connections 2023-09-30T14:39:15.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.259+0000 7fe70c90db80 1 -- 172.21.15.32:0/2303409421 wait complete. 2023-09-30T14:39:15.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.259+0000 7fe70c90db80 1 -- start start 2023-09-30T14:39:15.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.259+0000 7fe70c90db80 1 Processor -- start 2023-09-30T14:39:15.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.259+0000 7fe70c90db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977c9620 0x55e7977d3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.259+0000 7fe70c90db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977c9f80 0x55e7977d3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.259+0000 7fe70c90db80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977cae20 0x55e7977da6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.259+0000 7fe70c90db80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79771beb0 con 0x55e7977cae20 2023-09-30T14:39:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.259+0000 7fe70c90db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7976b8790 con 0x55e7977c9f80 2023-09-30T14:39:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.259+0000 7fe70c90db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7976b8430 con 0x55e7977c9620 2023-09-30T14:39:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.259+0000 7fe70c90db80 10 client.0 _ll_register_callbacks cb 0x55e797515880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.260+0000 7fe6fecc2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977cae20 0x55e7977da6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.260+0000 7fe6fecc2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977cae20 0x55e7977da6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35714/0 (socket says 172.21.15.32:35714) 2023-09-30T14:39:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.260+0000 7fe6fecc2700 1 -- 172.21.15.32:0/2985262964 learned_addr learned my addr 172.21.15.32:0/2985262964 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.260+0000 7fe6fe4c1700 1 --2- 172.21.15.32:0/2985262964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977c9620 0x55e7977d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.260+0000 7fe6fdcc0700 1 --2- 172.21.15.32:0/2985262964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977c9f80 0x55e7977d3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.260+0000 7fe6fecc2700 1 -- 172.21.15.32:0/2985262964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977c9620 msgr2=0x55e7977d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:15.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.260+0000 7fe6fecc2700 1 --2- 172.21.15.32:0/2985262964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977c9620 0x55e7977d3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.260+0000 7fe6fecc2700 1 -- 172.21.15.32:0/2985262964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977c9f80 msgr2=0x55e7977d3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.260+0000 7fe6fecc2700 1 --2- 172.21.15.32:0/2985262964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977c9f80 0x55e7977d3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.260+0000 7fe6fecc2700 1 -- 172.21.15.32:0/2985262964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f0009a90 con 0x55e7977cae20 2023-09-30T14:39:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.260+0000 7fe6fe4c1700 1 --2- 172.21.15.32:0/2985262964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977c9620 0x55e7977d3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.260+0000 7fe6fdcc0700 1 --2- 172.21.15.32:0/2985262964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977c9f80 0x55e7977d3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.260+0000 7fe6fecc2700 1 --2- 172.21.15.32:0/2985262964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977cae20 0x55e7977da6c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009fd0 tx=0x7fe6f0007f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1327727b38fc9f80 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.260+0000 7fe6e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.261+0000 7fe6e1ffb700 1 -- 172.21.15.32:0/2985262964 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0004db0 con 0x55e7977cae20 2023-09-30T14:39:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.261+0000 7fe6e1ffb700 1 -- 172.21.15.32:0/2985262964 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe6f0012b20 con 0x55e7977cae20 2023-09-30T14:39:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.261+0000 7fe70c90db80 1 -- 172.21.15.32:0/2985262964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7977bc2d0 con 0x55e7977cae20 2023-09-30T14:39:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.261+0000 7fe6e1ffb700 1 -- 172.21.15.32:0/2985262964 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6f0012ce0 con 0x55e7977cae20 2023-09-30T14:39:15.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.261+0000 7fe70c90db80 10 client.15513 fetch_fsmap learned FSMap version 23 2023-09-30T14:39:15.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.261+0000 7fe70c90db80 1 -- 172.21.15.32:0/2985262964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fe6f80ffbc0 con 0x55e7977cae20 2023-09-30T14:39:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.261+0000 7fe6e1ffb700 1 -- 172.21.15.32:0/2985262964 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e7977bc2d0 con 0x55e7977cae20 2023-09-30T14:39:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.261+0000 7fe6e1ffb700 1 -- 172.21.15.32:0/2985262964 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe6f0014040 con 0x55e7977cae20 2023-09-30T14:39:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.261+0000 7fe70c90db80 10 client.15513 fetch_fsmap finished waiting for FSMap version 23 2023-09-30T14:39:15.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.261+0000 7fe70c90db80 10 client.15513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.261+0000 7fe70c90db80 20 client.15513 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.261+0000 7fe70c90db80 1 --2- 172.21.15.32:0/2985262964 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e7977e77d0 0x55e7977e9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.261+0000 7fe70c90db80 4 client.15513 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:39:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.261+0000 7fe70c90db80 1 -- 172.21.15.32:0/2985262964 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e797714e90 con 0x55e7977e77d0 2023-09-30T14:39:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.262+0000 7fe6fe4c1700 1 --2- 172.21.15.32:0/2985262964 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e7977e77d0 0x55e7977e9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.262+0000 7fe6e1ffb700 10 client.15513 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.262+0000 7fe6fe4c1700 1 --2- 172.21.15.32:0/2985262964 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e7977e77d0 0x55e7977e9bf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6058f20c5d93c68a server_cookie=49ca34b44d6caa51 in_seq=0 out_seq=0 2023-09-30T14:39:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.263+0000 7fe6e1ffb700 1 -- 172.21.15.32:0/2985262964 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e797714e90 con 0x55e7977e77d0 2023-09-30T14:39:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.263+0000 7fe6e1ffb700 10 client.15513 handle_command_reply: tid=0 2023-09-30T14:39:15.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.263+0000 7fe70c90db80 1 client.15513 shutdown 2023-09-30T14:39:15.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.263+0000 7fe70c90db80 1 -- 172.21.15.32:0/2985262964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977cae20 msgr2=0x55e7977da6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.263+0000 7fe70c90db80 1 --2- 172.21.15.32:0/2985262964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977cae20 0x55e7977da6c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009fd0 tx=0x7fe6f0007f80 comp rx=0 tx=0).stop 2023-09-30T14:39:15.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.263+0000 7fe70c90db80 1 -- 172.21.15.32:0/2985262964 shutdown_connections 2023-09-30T14:39:15.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.263+0000 7fe70c90db80 1 --2- 172.21.15.32:0/2985262964 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e7977e77d0 0x55e7977e9bf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.263+0000 7fe70c90db80 1 --2- 172.21.15.32:0/2985262964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977cae20 0x55e7977da6c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.263+0000 7fe70c90db80 1 --2- 172.21.15.32:0/2985262964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977c9620 0x55e7977d3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.263+0000 7fe70c90db80 1 --2- 172.21.15.32:0/2985262964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977c9f80 0x55e7977d3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.263+0000 7fe70c90db80 1 -- 172.21.15.32:0/2985262964 >> 172.21.15.32:0/2985262964 conn(0x55e79773d2e0 msgr2=0x55e7977c4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:15.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.263+0000 7fe6e1ffb700 0 client.15513 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:15.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.263+0000 7fe70c90db80 1 -- 172.21.15.32:0/2985262964 shutdown_connections 2023-09-30T14:39:15.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.263+0000 7fe70c90db80 1 -- 172.21.15.32:0/2985262964 wait complete. 2023-09-30T14:39:15.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.263+0000 7fe70c90db80 20 client.15513 trim_cache size 0 max 16384 2023-09-30T14:39:15.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.287+0000 7fe70c90db80 1 Processor -- start 2023-09-30T14:39:15.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.287+0000 7fe70c90db80 1 -- start start 2023-09-30T14:39:15.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.287+0000 7fe70c90db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977c9620 0x55e7977cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.287+0000 7fe70c90db80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9f80 0x55e7977da680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.287+0000 7fe70c90db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977cae20 0x55e7977ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.287+0000 7fe70c90db80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79759c9a0 con 0x55e7977c9f80 2023-09-30T14:39:15.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.287+0000 7fe70c90db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7977d2430 con 0x55e7977c9620 2023-09-30T14:39:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.287+0000 7fe70c90db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7977d2cb0 con 0x55e7977cae20 2023-09-30T14:39:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.287+0000 7fe6fdcc0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9f80 0x55e7977da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.288+0000 7fe6fdcc0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9f80 0x55e7977da680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35720/0 (socket says 172.21.15.32:35720) 2023-09-30T14:39:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.288+0000 7fe6fdcc0700 1 -- 172.21.15.32:0/416754976 learned_addr learned my addr 172.21.15.32:0/416754976 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.288+0000 7fe6fdcc0700 1 -- 172.21.15.32:0/416754976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977cae20 msgr2=0x55e7977ce300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-30T14:39:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.288+0000 7fe6fdcc0700 1 --2- 172.21.15.32:0/416754976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977cae20 0x55e7977ce300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.288+0000 7fe6fdcc0700 1 -- 172.21.15.32:0/416754976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977c9620 msgr2=0x55e7977cddc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-30T14:39:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.288+0000 7fe6fdcc0700 1 --2- 172.21.15.32:0/416754976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977c9620 0x55e7977cddc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.288+0000 7fe6fdcc0700 1 -- 172.21.15.32:0/416754976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7977ee820 con 0x55e7977c9f80 2023-09-30T14:39:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.288+0000 7fe6fdcc0700 1 --2- 172.21.15.32:0/416754976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9f80 0x55e7977da680 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe6f409be20 tx=0x7fe6f409d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e4cf37c82de960 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:15.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.288+0000 7fe6ed7fa700 1 -- 172.21.15.32:0/416754976 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40a3500 con 0x55e7977c9f80 2023-09-30T14:39:15.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.288+0000 7fe6ed7fa700 1 -- 172.21.15.32:0/416754976 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6f40a76b0 con 0x55e7977c9f80 2023-09-30T14:39:15.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.288+0000 7fe6ed7fa700 1 -- 172.21.15.32:0/416754976 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f409f870 con 0x55e7977c9f80 2023-09-30T14:39:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.289+0000 7fe70c90db80 1 -- 172.21.15.32:0/416754976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9f80 msgr2=0x55e7977da680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.289+0000 7fe70c90db80 1 --2- 172.21.15.32:0/416754976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9f80 0x55e7977da680 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe6f409be20 tx=0x7fe6f409d750 comp rx=0 tx=0).stop 2023-09-30T14:39:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.289+0000 7fe70c90db80 1 -- 172.21.15.32:0/416754976 shutdown_connections 2023-09-30T14:39:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.289+0000 7fe70c90db80 1 --2- 172.21.15.32:0/416754976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977cae20 0x55e7977ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.289+0000 7fe70c90db80 1 --2- 172.21.15.32:0/416754976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977c9620 0x55e7977cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.289+0000 7fe70c90db80 1 --2- 172.21.15.32:0/416754976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9f80 0x55e7977da680 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.289+0000 7fe70c90db80 1 -- 172.21.15.32:0/416754976 >> 172.21.15.32:0/416754976 conn(0x55e79773d2e0 msgr2=0x55e7977e97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.289+0000 7fe70c90db80 1 -- 172.21.15.32:0/416754976 shutdown_connections 2023-09-30T14:39:15.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.289+0000 7fe70c90db80 1 -- 172.21.15.32:0/416754976 wait complete. 2023-09-30T14:39:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.290+0000 7fe70c90db80 1 -- start start 2023-09-30T14:39:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.290+0000 7fe70c90db80 1 Processor -- start 2023-09-30T14:39:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.290+0000 7fe70c90db80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9620 0x55e7977d8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.290+0000 7fe70c90db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977c9f80 0x55e7977d87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.290+0000 7fe70c90db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977cae20 0x55e7977d97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.290+0000 7fe70c90db80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7977d9d20 con 0x55e7977c9620 2023-09-30T14:39:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.290+0000 7fe70c90db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7977d9e90 con 0x55e7977cae20 2023-09-30T14:39:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.290+0000 7fe70c90db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7977da050 con 0x55e7977c9f80 2023-09-30T14:39:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.290+0000 7fe70c90db80 10 client.0 _ll_register_callbacks cb 0x7fe6f0007590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.290+0000 7fe6fe4c1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9620 0x55e7977d8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.290+0000 7fe6fe4c1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9620 0x55e7977d8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35724/0 (socket says 172.21.15.32:35724) 2023-09-30T14:39:15.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.290+0000 7fe6fe4c1700 1 -- 172.21.15.32:0/822983180 learned_addr learned my addr 172.21.15.32:0/822983180 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.291+0000 7fe6fecc2700 1 --2- 172.21.15.32:0/822983180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977cae20 0x55e7977d97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.291+0000 7fe6fe4c1700 1 -- 172.21.15.32:0/822983180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977c9f80 msgr2=0x55e7977d87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.291+0000 7fe6fe4c1700 1 --2- 172.21.15.32:0/822983180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977c9f80 0x55e7977d87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.291+0000 7fe6fe4c1700 1 -- 172.21.15.32:0/822983180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977cae20 msgr2=0x55e7977d97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.291+0000 7fe6fe4c1700 1 --2- 172.21.15.32:0/822983180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977cae20 0x55e7977d97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.291+0000 7fe6fe4c1700 1 -- 172.21.15.32:0/822983180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e79777e950 con 0x55e7977c9620 2023-09-30T14:39:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.291+0000 7fe6fe4c1700 1 --2- 172.21.15.32:0/822983180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9620 0x55e7977d8290 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800a010 tx=0x7fe6e80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0d121c020dfaae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.291+0000 7fe6e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.291+0000 7fe6e1ffb700 1 -- 172.21.15.32:0/822983180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8024070 con 0x55e7977c9620 2023-09-30T14:39:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.291+0000 7fe6e1ffb700 1 -- 172.21.15.32:0/822983180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe6e8003a90 con 0x55e7977c9620 2023-09-30T14:39:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.291+0000 7fe70c90db80 1 -- 172.21.15.32:0/822983180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6f409f3b0 con 0x55e7977c9620 2023-09-30T14:39:15.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.291+0000 7fe6e1ffb700 1 -- 172.21.15.32:0/822983180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6e8003c50 con 0x55e7977c9620 2023-09-30T14:39:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.291+0000 7fe70c90db80 10 client.15519 fetch_fsmap learned FSMap version 23 2023-09-30T14:39:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.291+0000 7fe70c90db80 1 -- 172.21.15.32:0/822983180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e797546e60 con 0x55e7977c9620 2023-09-30T14:39:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.292+0000 7fe6e1ffb700 1 -- 172.21.15.32:0/822983180 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6f409f3b0 con 0x55e7977c9620 2023-09-30T14:39:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.292+0000 7fe6e1ffb700 1 -- 172.21.15.32:0/822983180 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe6e8026ad0 con 0x55e7977c9620 2023-09-30T14:39:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.292+0000 7fe70c90db80 10 client.15519 fetch_fsmap finished waiting for FSMap version 23 2023-09-30T14:39:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.292+0000 7fe70c90db80 10 client.15519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.292+0000 7fe70c90db80 20 client.15519 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.292+0000 7fe70c90db80 1 --2- 172.21.15.32:0/822983180 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e79783c620 0x55e79783ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.292+0000 7fe70c90db80 4 client.15519 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-30T14:39:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.292+0000 7fe70c90db80 1 -- 172.21.15.32:0/822983180 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e797714bb0 con 0x55e79783c620 2023-09-30T14:39:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.292+0000 7fe6fdcc0700 1 --2- 172.21.15.32:0/822983180 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e79783c620 0x55e79783ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.293+0000 7fe6fdcc0700 1 --2- 172.21.15.32:0/822983180 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e79783c620 0x55e79783ea40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3960710b6fba4dfa server_cookie=dac808ab9a45736 in_seq=0 out_seq=0 2023-09-30T14:39:15.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.293+0000 7fe6e1ffb700 10 client.15519 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.294+0000 7fe6e1ffb700 1 -- 172.21.15.32:0/822983180 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e797714bb0 con 0x55e79783c620 2023-09-30T14:39:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.294+0000 7fe6e1ffb700 10 client.15519 handle_command_reply: tid=0 2023-09-30T14:39:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.294+0000 7fe70c90db80 1 client.15519 shutdown 2023-09-30T14:39:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.294+0000 7fe70c90db80 1 -- 172.21.15.32:0/822983180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9620 msgr2=0x55e7977d8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.294+0000 7fe70c90db80 1 --2- 172.21.15.32:0/822983180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9620 0x55e7977d8290 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800a010 tx=0x7fe6e80047c0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.294+0000 7fe70c90db80 1 -- 172.21.15.32:0/822983180 shutdown_connections 2023-09-30T14:39:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.294+0000 7fe70c90db80 1 --2- 172.21.15.32:0/822983180 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e79783c620 0x55e79783ea40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.294+0000 7fe70c90db80 1 --2- 172.21.15.32:0/822983180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e7977cae20 0x55e7977d97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.294+0000 7fe70c90db80 1 --2- 172.21.15.32:0/822983180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e7977c9620 0x55e7977d8290 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.294+0000 7fe70c90db80 1 --2- 172.21.15.32:0/822983180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e7977c9f80 0x55e7977d87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.294+0000 7fe70c90db80 1 -- 172.21.15.32:0/822983180 >> 172.21.15.32:0/822983180 conn(0x55e79773d2e0 msgr2=0x55e7977cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.294+0000 7fe6e1ffb700 0 client.15519 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.294+0000 7fe70c90db80 1 -- 172.21.15.32:0/822983180 shutdown_connections 2023-09-30T14:39:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.294+0000 7fe70c90db80 1 -- 172.21.15.32:0/822983180 wait complete. 2023-09-30T14:39:15.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.295+0000 7fe70c90db80 20 client.15519 trim_cache size 0 max 16384 2023-09-30T14:39:15.300 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T14:39:15.300 INFO:teuthology.orchestra.run.smithi032.stdout: "return_code": 0, 2023-09-30T14:39:15.300 INFO:teuthology.orchestra.run.smithi032.stdout: "scrub_tag": "618aa650-1927-4e5c-beb0-5e741455d032", 2023-09-30T14:39:15.301 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": "asynchronous" 2023-09-30T14:39:15.301 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T14:39:15.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.295+0000 7fe700725700 1 -- 172.21.15.32:0/1201917305 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe6e40a1020 msgr2=0x7fe6e40a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.295+0000 7fe700725700 1 --2- 172.21.15.32:0/1201917305 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe6e40a1020 0x7fe6e40a34e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009c60 tx=0x7fe6e8009260 comp rx=0 tx=0).stop 2023-09-30T14:39:15.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.295+0000 7fe700725700 1 -- 172.21.15.32:0/1201917305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f81042a0 msgr2=0x7fe6f8197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.295+0000 7fe700725700 1 --2- 172.21.15.32:0/1201917305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f81042a0 0x7fe6f8197040 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40059d0 tx=0x7fe6f400f690 comp rx=0 tx=0).stop 2023-09-30T14:39:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.295+0000 7fe700725700 1 -- 172.21.15.32:0/1201917305 shutdown_connections 2023-09-30T14:39:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.295+0000 7fe700725700 1 --2- 172.21.15.32:0/1201917305 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe6e40a1020 0x7fe6e40a34e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.295+0000 7fe700725700 1 --2- 172.21.15.32:0/1201917305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f8106bd0 0x7fe6f819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.295+0000 7fe700725700 1 --2- 172.21.15.32:0/1201917305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6f8101970 0x7fe6f8196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.295+0000 7fe700725700 1 --2- 172.21.15.32:0/1201917305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f81042a0 0x7fe6f8197040 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.295+0000 7fe700725700 1 -- 172.21.15.32:0/1201917305 >> 172.21.15.32:0/1201917305 conn(0x7fe6f80fb320 msgr2=0x7fe6f81025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.295+0000 7fe700725700 1 -- 172.21.15.32:0/1201917305 shutdown_connections 2023-09-30T14:39:15.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.295+0000 7fe700725700 1 -- 172.21.15.32:0/1201917305 wait complete. 2023-09-30T14:39:15.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-30T14:39:15.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.454+0000 7fe4757f1700 1 -- 172.21.15.32:0/476610108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe470100f20 msgr2=0x7fe470101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.454+0000 7fe4757f1700 1 --2- 172.21.15.32:0/476610108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe470100f20 0x7fe470101380 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe4640099c0 tx=0x7fe464009cd0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.454+0000 7fe4757f1700 1 -- 172.21.15.32:0/476610108 shutdown_connections 2023-09-30T14:39:15.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.454+0000 7fe4757f1700 1 --2- 172.21.15.32:0/476610108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4701018c0 0x7fe470108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.454+0000 7fe4757f1700 1 --2- 172.21.15.32:0/476610108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4700ffd30 0x7fe470100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.454+0000 7fe4757f1700 1 --2- 172.21.15.32:0/476610108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe470100f20 0x7fe470101380 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.454+0000 7fe4757f1700 1 -- 172.21.15.32:0/476610108 >> 172.21.15.32:0/476610108 conn(0x7fe4700fb2a0 msgr2=0x7fe4700fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.454+0000 7fe4757f1700 1 -- 172.21.15.32:0/476610108 shutdown_connections 2023-09-30T14:39:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.454+0000 7fe4757f1700 1 -- 172.21.15.32:0/476610108 wait complete. 2023-09-30T14:39:15.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.455+0000 7fe4757f1700 1 Processor -- start 2023-09-30T14:39:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.455+0000 7fe4757f1700 1 -- start start 2023-09-30T14:39:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.455+0000 7fe4757f1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4700ffd30 0x7fe47010a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.455+0000 7fe4757f1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe470100f20 0x7fe470108c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.455+0000 7fe4757f1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4701018c0 0x7fe4701091c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.455+0000 7fe4757f1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe470109960 con 0x7fe470100f20 2023-09-30T14:39:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.455+0000 7fe4757f1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe470109ad0 con 0x7fe4700ffd30 2023-09-30T14:39:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.455+0000 7fe4757f1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4701a0b50 con 0x7fe4701018c0 2023-09-30T14:39:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.456+0000 7fe46f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4701018c0 0x7fe4701091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.456+0000 7fe46f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4701018c0 0x7fe4701091c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:50122/0 (socket says 172.21.15.32:50122) 2023-09-30T14:39:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.456+0000 7fe46effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4700ffd30 0x7fe47010a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.456+0000 7fe46f7fe700 1 -- 172.21.15.32:0/3089634068 learned_addr learned my addr 172.21.15.32:0/3089634068 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:15.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.456+0000 7fe46e7fc700 1 --2- 172.21.15.32:0/3089634068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe470100f20 0x7fe470108c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.456+0000 7fe46effd700 1 -- 172.21.15.32:0/3089634068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4701018c0 msgr2=0x7fe4701091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.456+0000 7fe46effd700 1 --2- 172.21.15.32:0/3089634068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4701018c0 0x7fe4701091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.456+0000 7fe46effd700 1 -- 172.21.15.32:0/3089634068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe470100f20 msgr2=0x7fe470108c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.456+0000 7fe46effd700 1 --2- 172.21.15.32:0/3089634068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe470100f20 0x7fe470108c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.456+0000 7fe46effd700 1 -- 172.21.15.32:0/3089634068 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4640095f0 con 0x7fe4700ffd30 2023-09-30T14:39:15.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.456+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/3089634068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4701018c0 0x7fe4701091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:15.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.456+0000 7fe46effd700 1 --2- 172.21.15.32:0/3089634068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4700ffd30 0x7fe47010a640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe46000c940 tx=0x7fe46000cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:15.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.456+0000 7fe457fff700 1 -- 172.21.15.32:0/3089634068 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460004220 con 0x7fe4700ffd30 2023-09-30T14:39:15.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.457+0000 7fe4757f1700 1 -- 172.21.15.32:0/3089634068 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4701a0e30 con 0x7fe4700ffd30 2023-09-30T14:39:15.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.457+0000 7fe457fff700 1 -- 172.21.15.32:0/3089634068 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe46000ce30 con 0x7fe4700ffd30 2023-09-30T14:39:15.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.457+0000 7fe457fff700 1 -- 172.21.15.32:0/3089634068 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460003bd0 con 0x7fe4700ffd30 2023-09-30T14:39:15.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.457+0000 7fe4757f1700 1 -- 172.21.15.32:0/3089634068 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4701a12b0 con 0x7fe4700ffd30 2023-09-30T14:39:15.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.459+0000 7fe457fff700 1 -- 172.21.15.32:0/3089634068 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fe460003d30 con 0x7fe4700ffd30 2023-09-30T14:39:15.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.460+0000 7fe457fff700 1 --2- 172.21.15.32:0/3089634068 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe4580a0fa0 0x7fe4580a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.460+0000 7fe46e7fc700 1 --2- 172.21.15.32:0/3089634068 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe4580a0fa0 0x7fe4580a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.460+0000 7fe457fff700 1 -- 172.21.15.32:0/3089634068 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe4600cc260 con 0x7fe4700ffd30 2023-09-30T14:39:15.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.461+0000 7fe4819d9b80 1 Processor -- start 2023-09-30T14:39:15.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.461+0000 7fe46e7fc700 1 --2- 172.21.15.32:0/3089634068 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe4580a0fa0 0x7fe4580a3460 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe46400b580 tx=0x7fe46401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:15.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.461+0000 7fe4819d9b80 1 -- start start 2023-09-30T14:39:15.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.461+0000 7fe4819d9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11e69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.461+0000 7fe4819d9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 0x55fae11e77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.461+0000 7fe4819d9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e7da0 0x55fae11e8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.461+0000 7fe4819d9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fae114e460 con 0x55fae11e65a0 2023-09-30T14:39:15.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.461+0000 7fe4819d9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fae113fc20 con 0x55fae11e6f00 2023-09-30T14:39:15.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.461+0000 7fe4819d9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fae113d540 con 0x55fae11e7da0 2023-09-30T14:39:15.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.462+0000 7fe46effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11e69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.462+0000 7fe46effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11e69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35746/0 (socket says 172.21.15.32:35746) 2023-09-30T14:39:15.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.462+0000 7fe46effd700 1 -- 172.21.15.32:0/1062641634 learned_addr learned my addr 172.21.15.32:0/1062641634 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:15.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.462+0000 7fe46e7fc700 1 --2- 172.21.15.32:0/1062641634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 0x55fae11e77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.462+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/1062641634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e7da0 0x55fae11e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.462+0000 7fe46effd700 1 -- 172.21.15.32:0/1062641634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e7da0 msgr2=0x55fae11e8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:15.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.462+0000 7fe46effd700 1 --2- 172.21.15.32:0/1062641634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e7da0 0x55fae11e8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.462+0000 7fe46effd700 1 -- 172.21.15.32:0/1062641634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 msgr2=0x55fae11e77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:15.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.462+0000 7fe46effd700 1 --2- 172.21.15.32:0/1062641634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 0x55fae11e77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.462+0000 7fe46effd700 1 -- 172.21.15.32:0/1062641634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fae119b8d0 con 0x55fae11e65a0 2023-09-30T14:39:15.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.462+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/1062641634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e7da0 0x55fae11e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:39:15.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.462+0000 7fe46effd700 1 --2- 172.21.15.32:0/1062641634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11e69c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe460096140 tx=0x7fe4600992e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e2fd33232ee798 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:15.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.463+0000 7fe4557fa700 1 -- 172.21.15.32:0/1062641634 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4600996d0 con 0x55fae11e65a0 2023-09-30T14:39:15.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.463+0000 7fe4557fa700 1 -- 172.21.15.32:0/1062641634 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe46009d820 con 0x55fae11e65a0 2023-09-30T14:39:15.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.463+0000 7fe4557fa700 1 -- 172.21.15.32:0/1062641634 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460099930 con 0x55fae11e65a0 2023-09-30T14:39:15.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.463+0000 7fe4819d9b80 1 -- 172.21.15.32:0/1062641634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 msgr2=0x55fae11e69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.463+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/1062641634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11e69c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe460096140 tx=0x7fe4600992e0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.464+0000 7fe4819d9b80 1 -- 172.21.15.32:0/1062641634 shutdown_connections 2023-09-30T14:39:15.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.464+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/1062641634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e7da0 0x55fae11e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.464+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/1062641634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11e69c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.464+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/1062641634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 0x55fae11e77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.464+0000 7fe4819d9b80 1 -- 172.21.15.32:0/1062641634 >> 172.21.15.32:0/1062641634 conn(0x55fae115a260 msgr2=0x55fae11e1480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:15.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.464+0000 7fe4819d9b80 1 -- 172.21.15.32:0/1062641634 shutdown_connections 2023-09-30T14:39:15.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.464+0000 7fe4819d9b80 1 -- 172.21.15.32:0/1062641634 wait complete. 2023-09-30T14:39:15.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.464+0000 7fe4819d9b80 1 -- start start 2023-09-30T14:39:15.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.464+0000 7fe4819d9b80 1 Processor -- start 2023-09-30T14:39:15.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe4819d9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11f0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe4819d9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e6f00 0x55fae11f0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe4819d9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e7da0 0x55fae11f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe4819d9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fae1138e30 con 0x55fae11e65a0 2023-09-30T14:39:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe4819d9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fae10d5710 con 0x55fae11e7da0 2023-09-30T14:39:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe4819d9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fae10d53b0 con 0x55fae11e6f00 2023-09-30T14:39:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe4819d9b80 10 client.0 _ll_register_callbacks cb 0x55fae0f32800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:15.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe46effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe46effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11f0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35752/0 (socket says 172.21.15.32:35752) 2023-09-30T14:39:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe46effd700 1 -- 172.21.15.32:0/1937774876 learned_addr learned my addr 172.21.15.32:0/1937774876 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/1937774876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e7da0 0x55fae11f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe46e7fc700 1 --2- 172.21.15.32:0/1937774876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e6f00 0x55fae11f0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe46effd700 1 -- 172.21.15.32:0/1937774876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e6f00 msgr2=0x55fae11f0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe46effd700 1 --2- 172.21.15.32:0/1937774876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e6f00 0x55fae11f0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe46effd700 1 -- 172.21.15.32:0/1937774876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e7da0 msgr2=0x55fae11f79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe46effd700 1 --2- 172.21.15.32:0/1937774876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e7da0 0x55fae11f79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe46effd700 1 -- 172.21.15.32:0/1937774876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4701a12b0 con 0x55fae11e65a0 2023-09-30T14:39:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/1937774876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e7da0 0x55fae11f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:39:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe46effd700 1 --2- 172.21.15.32:0/1937774876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11f0450 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe4600a1530 tx=0x7fe4600a1a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebd2bbb1010ac6b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:15.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.465+0000 7fe452ff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.466+0000 7fe452ff5700 1 -- 172.21.15.32:0/1937774876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4600b5070 con 0x55fae11e65a0 2023-09-30T14:39:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.466+0000 7fe452ff5700 1 -- 172.21.15.32:0/1937774876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe460004380 con 0x55fae11e65a0 2023-09-30T14:39:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.466+0000 7fe4819d9b80 1 -- 172.21.15.32:0/1937774876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fae11d9250 con 0x55fae11e65a0 2023-09-30T14:39:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.466+0000 7fe452ff5700 1 -- 172.21.15.32:0/1937774876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4600996d0 con 0x55fae11e65a0 2023-09-30T14:39:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.466+0000 7fe4819d9b80 10 client.15531 fetch_fsmap learned FSMap version 23 2023-09-30T14:39:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.466+0000 7fe4819d9b80 1 -- 172.21.15.32:0/1937774876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fae0f63de0 con 0x55fae11e65a0 2023-09-30T14:39:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.466+0000 7fe452ff5700 1 -- 172.21.15.32:0/1937774876 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fae11d9250 con 0x55fae11e65a0 2023-09-30T14:39:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.466+0000 7fe452ff5700 1 -- 172.21.15.32:0/1937774876 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe460098630 con 0x55fae11e65a0 2023-09-30T14:39:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.466+0000 7fe4819d9b80 10 client.15531 fetch_fsmap finished waiting for FSMap version 23 2023-09-30T14:39:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.466+0000 7fe4819d9b80 10 client.15531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.466+0000 7fe4819d9b80 20 client.15531 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.466+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/1937774876 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55fae11fbd20 0x55fae11fe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.466+0000 7fe4819d9b80 4 client.15531 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:39:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.466+0000 7fe4819d9b80 1 -- 172.21.15.32:0/1937774876 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fae1131e10 con 0x55fae11fbd20 2023-09-30T14:39:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.467+0000 7fe46e7fc700 1 --2- 172.21.15.32:0/1937774876 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55fae11fbd20 0x55fae11fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.467+0000 7fe46e7fc700 1 --2- 172.21.15.32:0/1937774876 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55fae11fbd20 0x55fae11fe140 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b94634607b2343c server_cookie=8b48594dc89ed350 in_seq=0 out_seq=0 2023-09-30T14:39:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.467+0000 7fe452ff5700 10 client.15531 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.468+0000 7fe452ff5700 1 -- 172.21.15.32:0/1937774876 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fae1131e10 con 0x55fae11fbd20 2023-09-30T14:39:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.468+0000 7fe452ff5700 10 client.15531 handle_command_reply: tid=0 2023-09-30T14:39:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.468+0000 7fe4819d9b80 1 client.15531 shutdown 2023-09-30T14:39:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.468+0000 7fe4819d9b80 1 -- 172.21.15.32:0/1937774876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 msgr2=0x55fae11f0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.468+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/1937774876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11f0450 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe4600a1530 tx=0x7fe4600a1a60 comp rx=0 tx=0).stop 2023-09-30T14:39:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.468+0000 7fe4819d9b80 1 -- 172.21.15.32:0/1937774876 shutdown_connections 2023-09-30T14:39:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.468+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/1937774876 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55fae11fbd20 0x55fae11fe140 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.468+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/1937774876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e7da0 0x55fae11f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.468+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/1937774876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11f0450 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.468+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/1937774876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e6f00 0x55fae11f0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.468+0000 7fe4819d9b80 1 -- 172.21.15.32:0/1937774876 >> 172.21.15.32:0/1937774876 conn(0x55fae115a260 msgr2=0x55fae11e1290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:15.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.468+0000 7fe452ff5700 0 client.15531 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:15.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.468+0000 7fe4819d9b80 1 -- 172.21.15.32:0/1937774876 shutdown_connections 2023-09-30T14:39:15.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.468+0000 7fe4819d9b80 1 -- 172.21.15.32:0/1937774876 wait complete. 2023-09-30T14:39:15.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.469+0000 7fe4819d9b80 20 client.15531 trim_cache size 0 max 16384 2023-09-30T14:39:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.487+0000 7fe4819d9b80 1 Processor -- start 2023-09-30T14:39:15.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.487+0000 7fe4819d9b80 1 -- start start 2023-09-30T14:39:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.487+0000 7fe4819d9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11e1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.487+0000 7fe4819d9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 0x55fae11f7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.487+0000 7fe4819d9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e7da0 0x55fae11ee250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.487+0000 7fe4819d9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fae0fb9920 con 0x55fae11e65a0 2023-09-30T14:39:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.487+0000 7fe4819d9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fae11ef900 con 0x55fae11e6f00 2023-09-30T14:39:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.487+0000 7fe4819d9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fae11effc0 con 0x55fae11e7da0 2023-09-30T14:39:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.487+0000 7fe46e7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 0x55fae11f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.487+0000 7fe46e7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 0x55fae11f7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47236/0 (socket says 172.21.15.32:47236) 2023-09-30T14:39:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.487+0000 7fe46e7fc700 1 -- 172.21.15.32:0/546759213 learned_addr learned my addr 172.21.15.32:0/546759213 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.488+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/546759213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e7da0 0x55fae11ee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.488+0000 7fe46effd700 1 --2- 172.21.15.32:0/546759213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11e1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.488+0000 7fe46e7fc700 1 -- 172.21.15.32:0/546759213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e7da0 msgr2=0x55fae11ee250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.488+0000 7fe46e7fc700 1 --2- 172.21.15.32:0/546759213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e7da0 0x55fae11ee250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.488+0000 7fe46e7fc700 1 -- 172.21.15.32:0/546759213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 msgr2=0x55fae11e1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.488+0000 7fe46e7fc700 1 --2- 172.21.15.32:0/546759213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11e1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.488+0000 7fe46e7fc700 1 -- 172.21.15.32:0/546759213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fae11e0590 con 0x55fae11e6f00 2023-09-30T14:39:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.488+0000 7fe46effd700 1 --2- 172.21.15.32:0/546759213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11e1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:39:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.488+0000 7fe46e7fc700 1 --2- 172.21.15.32:0/546759213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 0x55fae11f7990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe4640095c0 tx=0x7fe46401e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9acd09abab6bb84 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.488+0000 7fe455ffb700 1 -- 172.21.15.32:0/546759213 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464034070 con 0x55fae11e6f00 2023-09-30T14:39:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.488+0000 7fe455ffb700 1 -- 172.21.15.32:0/546759213 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe464029040 con 0x55fae11e6f00 2023-09-30T14:39:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.488+0000 7fe455ffb700 1 -- 172.21.15.32:0/546759213 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46401d040 con 0x55fae11e6f00 2023-09-30T14:39:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.489+0000 7fe4819d9b80 1 -- 172.21.15.32:0/546759213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 msgr2=0x55fae11f7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.489+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/546759213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 0x55fae11f7990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe4640095c0 tx=0x7fe46401e000 comp rx=0 tx=0).stop 2023-09-30T14:39:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.489+0000 7fe4819d9b80 1 -- 172.21.15.32:0/546759213 shutdown_connections 2023-09-30T14:39:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.489+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/546759213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e7da0 0x55fae11ee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.489+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/546759213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e65a0 0x55fae11e1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.489+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/546759213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 0x55fae11f7990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.489+0000 7fe4819d9b80 1 -- 172.21.15.32:0/546759213 >> 172.21.15.32:0/546759213 conn(0x55fae115a260 msgr2=0x55fae11fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.489+0000 7fe4819d9b80 1 -- 172.21.15.32:0/546759213 shutdown_connections 2023-09-30T14:39:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.489+0000 7fe4819d9b80 1 -- 172.21.15.32:0/546759213 wait complete. 2023-09-30T14:39:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.490+0000 7fe4819d9b80 1 -- start start 2023-09-30T14:39:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.490+0000 7fe4819d9b80 1 Processor -- start 2023-09-30T14:39:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.490+0000 7fe4819d9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e65a0 0x55fae11f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.490+0000 7fe4819d9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 0x55fae11f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.490+0000 7fe4819d9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e7da0 0x55fae11f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.490+0000 7fe4819d9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fae11f7490 con 0x55fae11e7da0 2023-09-30T14:39:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.490+0000 7fe4819d9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fae11f7600 con 0x55fae11e6f00 2023-09-30T14:39:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.490+0000 7fe4819d9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fae11f77c0 con 0x55fae11e65a0 2023-09-30T14:39:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.490+0000 7fe4819d9b80 10 client.0 _ll_register_callbacks cb 0x7fe464009510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.490+0000 7fe46f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e7da0 0x55fae11f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.490+0000 7fe46f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e7da0 0x55fae11f6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35768/0 (socket says 172.21.15.32:35768) 2023-09-30T14:39:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.490+0000 7fe46f7fe700 1 -- 172.21.15.32:0/3176340092 learned_addr learned my addr 172.21.15.32:0/3176340092 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.490+0000 7fe46effd700 1 --2- 172.21.15.32:0/3176340092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e65a0 0x55fae11f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.490+0000 7fe46e7fc700 1 --2- 172.21.15.32:0/3176340092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 0x55fae11f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.491+0000 7fe46effd700 1 -- 172.21.15.32:0/3176340092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 msgr2=0x55fae11f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.491+0000 7fe46effd700 1 --2- 172.21.15.32:0/3176340092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 0x55fae11f3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.491+0000 7fe46effd700 1 -- 172.21.15.32:0/3176340092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e7da0 msgr2=0x55fae11f6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.491+0000 7fe46effd700 1 --2- 172.21.15.32:0/3176340092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e7da0 0x55fae11f6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.491+0000 7fe46effd700 1 -- 172.21.15.32:0/3176340092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe470067fb0 con 0x55fae11e65a0 2023-09-30T14:39:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.491+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/3176340092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e7da0 0x55fae11f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.491+0000 7fe46effd700 1 --2- 172.21.15.32:0/3176340092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e65a0 0x55fae11f39f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe460096140 tx=0x7fe460095a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9c0087835db8c2e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.491+0000 7fe452ff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:39:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.491+0000 7fe452ff5700 1 -- 172.21.15.32:0/3176340092 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4600b5070 con 0x55fae11e65a0 2023-09-30T14:39:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.491+0000 7fe452ff5700 1 -- 172.21.15.32:0/3176340092 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe4600a15d0 con 0x55fae11e65a0 2023-09-30T14:39:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.491+0000 7fe4819d9b80 1 -- 172.21.15.32:0/3176340092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe46009d820 con 0x55fae11e65a0 2023-09-30T14:39:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.491+0000 7fe452ff5700 1 -- 172.21.15.32:0/3176340092 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4600a1880 con 0x55fae11e65a0 2023-09-30T14:39:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.491+0000 7fe4819d9b80 10 client.25222 fetch_fsmap learned FSMap version 23 2023-09-30T14:39:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.491+0000 7fe4819d9b80 1 -- 172.21.15.32:0/3176340092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fae0f4cc60 con 0x55fae11e65a0 2023-09-30T14:39:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.492+0000 7fe452ff5700 1 -- 172.21.15.32:0/3176340092 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe46009d820 con 0x55fae11e65a0 2023-09-30T14:39:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.492+0000 7fe452ff5700 1 -- 172.21.15.32:0/3176340092 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe46009e560 con 0x55fae11e65a0 2023-09-30T14:39:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.492+0000 7fe4819d9b80 10 client.25222 fetch_fsmap finished waiting for FSMap version 23 2023-09-30T14:39:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.492+0000 7fe4819d9b80 10 client.25222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.492+0000 7fe4819d9b80 20 client.25222 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.492+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/3176340092 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55fae1261920 0x55fae1263d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.492+0000 7fe4819d9b80 4 client.25222 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-30T14:39:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.492+0000 7fe4819d9b80 1 -- 172.21.15.32:0/3176340092 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fae1131b30 con 0x55fae1261920 2023-09-30T14:39:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.493+0000 7fe46e7fc700 1 --2- 172.21.15.32:0/3176340092 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55fae1261920 0x55fae1263d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.493+0000 7fe452ff5700 10 client.25222 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.493+0000 7fe46e7fc700 1 --2- 172.21.15.32:0/3176340092 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55fae1261920 0x55fae1263d40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eadd999749aa6db2 server_cookie=4e7b4229427d71f3 in_seq=0 out_seq=0 2023-09-30T14:39:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.493+0000 7fe452ff5700 1 -- 172.21.15.32:0/3176340092 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fae1131b30 con 0x55fae1261920 2023-09-30T14:39:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.493+0000 7fe452ff5700 10 client.25222 handle_command_reply: tid=0 2023-09-30T14:39:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.493+0000 7fe4819d9b80 1 client.25222 shutdown 2023-09-30T14:39:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.493+0000 7fe4819d9b80 1 -- 172.21.15.32:0/3176340092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e65a0 msgr2=0x55fae11f39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.493+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/3176340092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e65a0 0x55fae11f39f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe460096140 tx=0x7fe460095a10 comp rx=0 tx=0).stop 2023-09-30T14:39:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4819d9b80 1 -- 172.21.15.32:0/3176340092 shutdown_connections 2023-09-30T14:39:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/3176340092 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55fae1261920 0x55fae1263d40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/3176340092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55fae11e7da0 0x55fae11f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe452ff5700 0 client.25222 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/3176340092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fae11e65a0 0x55fae11f39f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4819d9b80 1 --2- 172.21.15.32:0/3176340092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fae11e6f00 0x55fae11f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4819d9b80 1 -- 172.21.15.32:0/3176340092 >> 172.21.15.32:0/3176340092 conn(0x55fae115a260 msgr2=0x55fae11ead80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4819d9b80 1 -- 172.21.15.32:0/3176340092 shutdown_connections 2023-09-30T14:39:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4819d9b80 1 -- 172.21.15.32:0/3176340092 wait complete. 2023-09-30T14:39:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4819d9b80 20 client.25222 trim_cache size 0 max 16384 2023-09-30T14:39:15.500 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T14:39:15.500 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-30T14:39:15.500 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": { 2023-09-30T14:39:15.500 INFO:teuthology.orchestra.run.smithi032.stdout: "618aa650-1927-4e5c-beb0-5e741455d032": { 2023-09-30T14:39:15.500 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/", 2023-09-30T14:39:15.501 INFO:teuthology.orchestra.run.smithi032.stdout: "tag": "618aa650-1927-4e5c-beb0-5e741455d032", 2023-09-30T14:39:15.501 INFO:teuthology.orchestra.run.smithi032.stdout: "options": "recursive,force" 2023-09-30T14:39:15.501 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T14:39:15.501 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T14:39:15.501 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T14:39:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4757f1700 1 -- 172.21.15.32:0/3089634068 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe4580a0fa0 msgr2=0x7fe4580a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4757f1700 1 --2- 172.21.15.32:0/3089634068 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe4580a0fa0 0x7fe4580a3460 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe46400b580 tx=0x7fe46401a040 comp rx=0 tx=0).stop 2023-09-30T14:39:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4757f1700 1 -- 172.21.15.32:0/3089634068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4700ffd30 msgr2=0x7fe47010a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4757f1700 1 --2- 172.21.15.32:0/3089634068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4700ffd30 0x7fe47010a640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe46000c940 tx=0x7fe46000cc50 comp rx=0 tx=0).stop 2023-09-30T14:39:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4757f1700 1 -- 172.21.15.32:0/3089634068 shutdown_connections 2023-09-30T14:39:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4757f1700 1 --2- 172.21.15.32:0/3089634068 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe4580a0fa0 0x7fe4580a3460 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4757f1700 1 --2- 172.21.15.32:0/3089634068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4701018c0 0x7fe4701091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4757f1700 1 --2- 172.21.15.32:0/3089634068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4700ffd30 0x7fe47010a640 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4757f1700 1 --2- 172.21.15.32:0/3089634068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe470100f20 0x7fe470108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.494+0000 7fe4757f1700 1 -- 172.21.15.32:0/3089634068 >> 172.21.15.32:0/3089634068 conn(0x7fe4700fb2a0 msgr2=0x7fe4700fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.495+0000 7fe4757f1700 1 -- 172.21.15.32:0/3089634068 shutdown_connections 2023-09-30T14:39:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:15.495+0000 7fe4757f1700 1 -- 172.21.15.32:0/3089634068 wait complete. 2023-09-30T14:39:15.514 INFO:tasks.cephfs.filesystem:scrub status for tag:618aa650-1927-4e5c-beb0-5e741455d032 - {'path': '/', 'tag': '618aa650-1927-4e5c-beb0-5e741455d032', 'options': 'recursive,force'} 2023-09-30T14:39:15.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:15 smithi148 ceph-mon[102355]: pgmap v316: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-09-30T14:39:15.609 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 30 14:39:15 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-h[138164]: 2023-09-30T14:39:15.296+0000 7f0fc4f15700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /volumes/_nogroup/sv_1/b25fcf16-a734-410f-849e-f8fb3a417cdf/client.0/ auth v4 ap=1 f() n(v0 rc2023-09-30T14:38:42.553557+0000 1=0+1) (inest lock) (iversion lock) | ptrwaiter=0 lock=0 importingcaps=0 dirfrag=0 caps=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561fa970c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000005:[<0x100000001fb/client.0 v2>,<0x100000001fa/b25fcf16-a734-410f-849e-f8fb3a417cdf v18>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v25>,<0x1/volumes v20>]//[]","memoryvalue":"(2)0x10000000005:[<0x100000001fb/client.0 v4>,<0x100000001fa/b25fcf16-a734-410f-849e-f8fb3a417cdf v20>,<0x10000000001/sv_1 v39>,<0x10000000000/_nogroup v23>,<0x1/volumes v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 rc2023-09-30T14:38:42.553557+0000 1=0+1)","error_str":""},"return_code":0} 2023-09-30T14:39:15.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:15 smithi156 ceph-mon[103579]: pgmap v316: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-09-30T14:39:15.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:15 smithi032 ceph-mon[98592]: pgmap v316: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-09-30T14:39:16.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:16 smithi156 ceph-mon[103579]: scrub queued for path: / 2023-09-30T14:39:16.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:16 smithi156 ceph-mon[103579]: scrub summary: idle+waiting paths [/] 2023-09-30T14:39:16.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:16 smithi156 ceph-mon[103579]: scrub summary: active paths [/] 2023-09-30T14:39:16.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:16 smithi156 ceph-mon[103579]: scrub summary: active paths [/] 2023-09-30T14:39:16.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:16 smithi156 ceph-mon[103579]: Scrub error on inode 0x10000000005 (/volumes/_nogroup/sv_1/b25fcf16-a734-410f-849e-f8fb3a417cdf/client.0) see mds.h log and `damage ls` output for details 2023-09-30T14:39:16.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:16 smithi156 ceph-mon[103579]: scrub summary: idle+waiting paths [/] 2023-09-30T14:39:16.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:16 smithi156 ceph-mon[103579]: scrub summary: idle+waiting paths [/] 2023-09-30T14:39:16.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:16 smithi148 ceph-mon[102355]: scrub queued for path: / 2023-09-30T14:39:16.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:16 smithi148 ceph-mon[102355]: scrub summary: idle+waiting paths [/] 2023-09-30T14:39:16.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:16 smithi148 ceph-mon[102355]: scrub summary: active paths [/] 2023-09-30T14:39:16.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:16 smithi148 ceph-mon[102355]: scrub summary: active paths [/] 2023-09-30T14:39:16.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:16 smithi148 ceph-mon[102355]: Scrub error on inode 0x10000000005 (/volumes/_nogroup/sv_1/b25fcf16-a734-410f-849e-f8fb3a417cdf/client.0) see mds.h log and `damage ls` output for details 2023-09-30T14:39:16.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:16 smithi148 ceph-mon[102355]: scrub summary: idle+waiting paths [/] 2023-09-30T14:39:16.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:16 smithi148 ceph-mon[102355]: scrub summary: idle+waiting paths [/] 2023-09-30T14:39:16.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:16 smithi032 ceph-mon[98592]: scrub queued for path: / 2023-09-30T14:39:16.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:16 smithi032 ceph-mon[98592]: scrub summary: idle+waiting paths [/] 2023-09-30T14:39:16.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:16 smithi032 ceph-mon[98592]: scrub summary: active paths [/] 2023-09-30T14:39:16.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:16 smithi032 ceph-mon[98592]: scrub summary: active paths [/] 2023-09-30T14:39:16.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:16 smithi032 ceph-mon[98592]: Scrub error on inode 0x10000000005 (/volumes/_nogroup/sv_1/b25fcf16-a734-410f-849e-f8fb3a417cdf/client.0) see mds.h log and `damage ls` output for details 2023-09-30T14:39:16.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:16 smithi032 ceph-mon[98592]: scrub summary: idle+waiting paths [/] 2023-09-30T14:39:16.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:16 smithi032 ceph-mon[98592]: scrub summary: idle+waiting paths [/] 2023-09-30T14:39:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:17 smithi156 ceph-mon[103579]: pgmap v317: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2023-09-30T14:39:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:17 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:39:17.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:17 smithi148 ceph-mon[102355]: pgmap v317: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2023-09-30T14:39:17.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:17 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:39:17.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:17 smithi032 ceph-mon[98592]: pgmap v317: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2023-09-30T14:39:17.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:17 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:39:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:19 smithi156 ceph-mon[103579]: pgmap v318: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-09-30T14:39:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:19 smithi148 ceph-mon[102355]: pgmap v318: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-09-30T14:39:19.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:19 smithi032 ceph-mon[98592]: pgmap v318: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-09-30T14:39:21.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:20 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:20 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:20 smithi156 ceph-mon[103579]: pgmap v319: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-09-30T14:39:21.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:20 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:20 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:20 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:20 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:20 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:20 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:20 smithi148 ceph-mon[102355]: pgmap v319: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-09-30T14:39:21.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:20 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:20 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:20 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:20 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:20 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:20 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:20 smithi032 ceph-mon[98592]: pgmap v319: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-09-30T14:39:21.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:20 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:20 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:20 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:20 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:21.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:20 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-a[98568]: 2023-09-30T14:39:20.824+0000 7f49afbf4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-09-30T14:39:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.984+0000 7f463663c700 1 -- 172.21.15.32:0/3939213443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4630101070 msgr2=0x7f4630107920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.984+0000 7f463663c700 1 --2- 172.21.15.32:0/3939213443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4630101070 0x7f4630107920 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f462800c240 tx=0x7f462800c550 comp rx=0 tx=0).stop 2023-09-30T14:39:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.984+0000 7f463663c700 1 -- 172.21.15.32:0/3939213443 shutdown_connections 2023-09-30T14:39:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.984+0000 7f463663c700 1 --2- 172.21.15.32:0/3939213443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4630101070 0x7f4630107920 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.984+0000 7f463663c700 1 --2- 172.21.15.32:0/3939213443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46300ff4e0 0x7f46300ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.984+0000 7f463663c700 1 --2- 172.21.15.32:0/3939213443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46301006d0 0x7f4630100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.984+0000 7f463663c700 1 -- 172.21.15.32:0/3939213443 >> 172.21.15.32:0/3939213443 conn(0x7f46300faa30 msgr2=0x7f46300fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.984+0000 7f463663c700 1 -- 172.21.15.32:0/3939213443 shutdown_connections 2023-09-30T14:39:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.984+0000 7f463663c700 1 -- 172.21.15.32:0/3939213443 wait complete. 2023-09-30T14:39:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.984+0000 7f463663c700 1 Processor -- start 2023-09-30T14:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.984+0000 7f463663c700 1 -- start start 2023-09-30T14:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f463663c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46300ff4e0 0x7f4630196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f463663c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46301006d0 0x7f46301972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f463663c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4630101070 0x7f4630198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f463663c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4630198770 con 0x7f46300ff4e0 2023-09-30T14:39:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f463663c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46301988e0 con 0x7f4630101070 2023-09-30T14:39:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f463663c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4630198aa0 con 0x7f46301006d0 2023-09-30T14:39:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f462ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46300ff4e0 0x7f4630196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f462ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46300ff4e0 0x7f4630196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35776/0 (socket says 172.21.15.32:35776) 2023-09-30T14:39:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f462ffff700 1 -- 172.21.15.32:0/3754184913 learned_addr learned my addr 172.21.15.32:0/3754184913 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f4634bd9700 1 --2- 172.21.15.32:0/3754184913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4630101070 0x7f4630198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f462f7fe700 1 --2- 172.21.15.32:0/3754184913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46301006d0 0x7f46301972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f462ffff700 1 -- 172.21.15.32:0/3754184913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46301006d0 msgr2=0x7f46301972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f462ffff700 1 --2- 172.21.15.32:0/3754184913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46301006d0 0x7f46301972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f462ffff700 1 -- 172.21.15.32:0/3754184913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4630101070 msgr2=0x7f4630198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f462ffff700 1 --2- 172.21.15.32:0/3754184913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4630101070 0x7f4630198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f462ffff700 1 -- 172.21.15.32:0/3754184913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462800ae10 con 0x7f46300ff4e0 2023-09-30T14:39:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f4634bd9700 1 --2- 172.21.15.32:0/3754184913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4630101070 0x7f4630198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:39:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f462ffff700 1 --2- 172.21.15.32:0/3754184913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46300ff4e0 0x7f4630196d80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f462000d980 tx=0x7f462000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f462d7fa700 1 -- 172.21.15.32:0/3754184913 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462000de60 con 0x7f46300ff4e0 2023-09-30T14:39:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.985+0000 7f462d7fa700 1 -- 172.21.15.32:0/3754184913 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46200044e0 con 0x7f46300ff4e0 2023-09-30T14:39:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.986+0000 7f462d7fa700 1 -- 172.21.15.32:0/3754184913 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620003be0 con 0x7f46300ff4e0 2023-09-30T14:39:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.986+0000 7f463663c700 1 -- 172.21.15.32:0/3754184913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4630198d80 con 0x7f46300ff4e0 2023-09-30T14:39:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.986+0000 7f463663c700 1 -- 172.21.15.32:0/3754184913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463019f4e0 con 0x7f46300ff4e0 2023-09-30T14:39:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.988+0000 7f462d7fa700 1 -- 172.21.15.32:0/3754184913 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f4620009bb0 con 0x7f46300ff4e0 2023-09-30T14:39:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.988+0000 7f462d7fa700 1 --2- 172.21.15.32:0/3754184913 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f46180a1020 0x7f46180a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.988+0000 7f462d7fa700 1 -- 172.21.15.32:0/3754184913 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f46200cc410 con 0x7f46300ff4e0 2023-09-30T14:39:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.988+0000 7f462f7fe700 1 --2- 172.21.15.32:0/3754184913 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f46180a1020 0x7f46180a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.989+0000 7f462f7fe700 1 --2- 172.21.15.32:0/3754184913 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f46180a1020 0x7f46180a34e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4624009bf0 tx=0x7f4624009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.989+0000 7f4642824b80 1 Processor -- start 2023-09-30T14:39:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.989+0000 7f4642824b80 1 -- start start 2023-09-30T14:39:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.989+0000 7f4642824b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e60200 0x562bc2e60620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.989+0000 7f4642824b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e61310 0x562bc2e61790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.989+0000 7f4642824b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e61cd0 0x562bc2e68560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.989+0000 7f4642824b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc2dc8400 con 0x562bc2e60200 2023-09-30T14:39:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.989+0000 7f4642824b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc2db9bc0 con 0x562bc2e61310 2023-09-30T14:39:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.989+0000 7f4642824b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc2db74e0 con 0x562bc2e61cd0 2023-09-30T14:39:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.989+0000 7f462ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e60200 0x562bc2e60620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.989+0000 7f462ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e60200 0x562bc2e60620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35784/0 (socket says 172.21.15.32:35784) 2023-09-30T14:39:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.990+0000 7f462ffff700 1 -- 172.21.15.32:0/3711883029 learned_addr learned my addr 172.21.15.32:0/3711883029 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.990+0000 7f462f7fe700 1 --2- 172.21.15.32:0/3711883029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e61310 0x562bc2e61790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.990+0000 7f462ffff700 1 -- 172.21.15.32:0/3711883029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e61cd0 msgr2=0x562bc2e68560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.990+0000 7f462ffff700 1 --2- 172.21.15.32:0/3711883029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e61cd0 0x562bc2e68560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.990+0000 7f462ffff700 1 -- 172.21.15.32:0/3711883029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e61310 msgr2=0x562bc2e61790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.990+0000 7f462ffff700 1 --2- 172.21.15.32:0/3711883029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e61310 0x562bc2e61790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.990+0000 7f462ffff700 1 -- 172.21.15.32:0/3711883029 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bc2e15870 con 0x562bc2e60200 2023-09-30T14:39:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.990+0000 7f462ffff700 1 --2- 172.21.15.32:0/3711883029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e60200 0x562bc2e60620 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4620095de0 tx=0x7f46200997c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5713613f965efac5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.990+0000 7f46167fc700 1 -- 172.21.15.32:0/3711883029 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46200b90c0 con 0x562bc2e60200 2023-09-30T14:39:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.990+0000 7f46167fc700 1 -- 172.21.15.32:0/3711883029 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46200998e0 con 0x562bc2e60200 2023-09-30T14:39:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.990+0000 7f46167fc700 1 -- 172.21.15.32:0/3711883029 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620099030 con 0x562bc2e60200 2023-09-30T14:39:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.991+0000 7f4642824b80 1 -- 172.21.15.32:0/3711883029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e60200 msgr2=0x562bc2e60620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.991+0000 7f4642824b80 1 --2- 172.21.15.32:0/3711883029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e60200 0x562bc2e60620 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4620095de0 tx=0x7f46200997c0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.991+0000 7f4642824b80 1 -- 172.21.15.32:0/3711883029 shutdown_connections 2023-09-30T14:39:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.991+0000 7f4642824b80 1 --2- 172.21.15.32:0/3711883029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e61cd0 0x562bc2e68560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.991+0000 7f4642824b80 1 --2- 172.21.15.32:0/3711883029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e60200 0x562bc2e60620 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.991+0000 7f4642824b80 1 --2- 172.21.15.32:0/3711883029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e61310 0x562bc2e61790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.991+0000 7f4642824b80 1 -- 172.21.15.32:0/3711883029 >> 172.21.15.32:0/3711883029 conn(0x562bc2dd4200 msgr2=0x562bc2e5aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.991+0000 7f4642824b80 1 -- 172.21.15.32:0/3711883029 shutdown_connections 2023-09-30T14:39:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.991+0000 7f4642824b80 1 -- 172.21.15.32:0/3711883029 wait complete. 2023-09-30T14:39:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.991+0000 7f4642824b80 1 -- start start 2023-09-30T14:39:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f4642824b80 1 Processor -- start 2023-09-30T14:39:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f4642824b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e60200 0x562bc2e69f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f4642824b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e61310 0x562bc2e6a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f4642824b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e61cd0 0x562bc2e71500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f4642824b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc2db2dd0 con 0x562bc2e60200 2023-09-30T14:39:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f4642824b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc2d4f6b0 con 0x562bc2e61310 2023-09-30T14:39:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f4642824b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc2d4f350 con 0x562bc2e61cd0 2023-09-30T14:39:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f4642824b80 10 client.0 _ll_register_callbacks cb 0x562bc2bac7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f462ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e60200 0x562bc2e69f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f462ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e60200 0x562bc2e69f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35790/0 (socket says 172.21.15.32:35790) 2023-09-30T14:39:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f462ffff700 1 -- 172.21.15.32:0/1741619765 learned_addr learned my addr 172.21.15.32:0/1741619765 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f4634bd9700 1 --2- 172.21.15.32:0/1741619765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e61cd0 0x562bc2e71500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f462f7fe700 1 --2- 172.21.15.32:0/1741619765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e61310 0x562bc2e6a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f462ffff700 1 -- 172.21.15.32:0/1741619765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e61cd0 msgr2=0x562bc2e71500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f462ffff700 1 --2- 172.21.15.32:0/1741619765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e61cd0 0x562bc2e71500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f462ffff700 1 -- 172.21.15.32:0/1741619765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e61310 msgr2=0x562bc2e6a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f462ffff700 1 --2- 172.21.15.32:0/1741619765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e61310 0x562bc2e6a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f462ffff700 1 -- 172.21.15.32:0/1741619765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463019f4e0 con 0x562bc2e60200 2023-09-30T14:39:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f462ffff700 1 --2- 172.21.15.32:0/1741619765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e60200 0x562bc2e69f80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f462000d950 tx=0x7f4620095600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fa92fa51a84c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.992+0000 7f460f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.993+0000 7f460f7fe700 1 -- 172.21.15.32:0/1741619765 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46200a1c70 con 0x562bc2e60200 2023-09-30T14:39:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.993+0000 7f4642824b80 1 -- 172.21.15.32:0/1741619765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bc2e53090 con 0x562bc2e60200 2023-09-30T14:39:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.993+0000 7f460f7fe700 1 -- 172.21.15.32:0/1741619765 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f46200b9050 con 0x562bc2e60200 2023-09-30T14:39:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.993+0000 7f460f7fe700 1 -- 172.21.15.32:0/1741619765 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46200a17e0 con 0x562bc2e60200 2023-09-30T14:39:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.993+0000 7f4642824b80 10 client.15546 fetch_fsmap learned FSMap version 24 2023-09-30T14:39:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.993+0000 7f4642824b80 1 -- 172.21.15.32:0/1741619765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562bc2bddd80 con 0x562bc2e60200 2023-09-30T14:39:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.993+0000 7f460f7fe700 1 -- 172.21.15.32:0/1741619765 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562bc2e53090 con 0x562bc2e60200 2023-09-30T14:39:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.993+0000 7f460f7fe700 1 -- 172.21.15.32:0/1741619765 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f4620081020 con 0x562bc2e60200 2023-09-30T14:39:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.993+0000 7f4642824b80 10 client.15546 fetch_fsmap finished waiting for FSMap version 24 2023-09-30T14:39:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.993+0000 7f4642824b80 10 client.15546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.993+0000 7f4642824b80 20 client.15546 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.993+0000 7f4642824b80 1 --2- 172.21.15.32:0/1741619765 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562bc2e75b10 0x562bc2e77f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.993+0000 7f4642824b80 4 client.15546 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:39:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.993+0000 7f4642824b80 1 -- 172.21.15.32:0/1741619765 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bc2dabdb0 con 0x562bc2e75b10 2023-09-30T14:39:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.994+0000 7f462f7fe700 1 --2- 172.21.15.32:0/1741619765 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562bc2e75b10 0x562bc2e77f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.994+0000 7f460f7fe700 10 client.15546 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.994+0000 7f462f7fe700 1 --2- 172.21.15.32:0/1741619765 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562bc2e75b10 0x562bc2e77f30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2adb25d687b662c3 server_cookie=7efa0a4327894c81 in_seq=0 out_seq=0 2023-09-30T14:39:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.995+0000 7f460f7fe700 1 -- 172.21.15.32:0/1741619765 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562bc2dabdb0 con 0x562bc2e75b10 2023-09-30T14:39:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.995+0000 7f460f7fe700 10 client.15546 handle_command_reply: tid=0 2023-09-30T14:39:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.995+0000 7f4642824b80 1 client.15546 shutdown 2023-09-30T14:39:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.995+0000 7f4642824b80 1 -- 172.21.15.32:0/1741619765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e60200 msgr2=0x562bc2e69f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.995+0000 7f4642824b80 1 --2- 172.21.15.32:0/1741619765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e60200 0x562bc2e69f80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f462000d950 tx=0x7f4620095600 comp rx=0 tx=0).stop 2023-09-30T14:39:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.995+0000 7f4642824b80 1 -- 172.21.15.32:0/1741619765 shutdown_connections 2023-09-30T14:39:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.995+0000 7f4642824b80 1 --2- 172.21.15.32:0/1741619765 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562bc2e75b10 0x562bc2e77f30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.995+0000 7f4642824b80 1 --2- 172.21.15.32:0/1741619765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e61cd0 0x562bc2e71500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.995+0000 7f460f7fe700 0 client.15546 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.995+0000 7f4642824b80 1 --2- 172.21.15.32:0/1741619765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e60200 0x562bc2e69f80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.995+0000 7f4642824b80 1 --2- 172.21.15.32:0/1741619765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e61310 0x562bc2e6a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.995+0000 7f4642824b80 1 -- 172.21.15.32:0/1741619765 >> 172.21.15.32:0/1741619765 conn(0x562bc2dd4200 msgr2=0x562bc2e5a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.995+0000 7f4642824b80 1 -- 172.21.15.32:0/1741619765 shutdown_connections 2023-09-30T14:39:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.995+0000 7f4642824b80 1 -- 172.21.15.32:0/1741619765 wait complete. 2023-09-30T14:39:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:21.995+0000 7f4642824b80 20 client.15546 trim_cache size 0 max 16384 2023-09-30T14:39:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.010+0000 7f4642824b80 1 Processor -- start 2023-09-30T14:39:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.010+0000 7f4642824b80 1 -- start start 2023-09-30T14:39:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f4642824b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e60200 0x562bc2e64cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f4642824b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e61310 0x562bc2e714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f4642824b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e61cd0 0x562bc2e65200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f4642824b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc2e60ec0 con 0x562bc2e61cd0 2023-09-30T14:39:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f4642824b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc2c338c0 con 0x562bc2e60200 2023-09-30T14:39:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f4642824b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc2e69450 con 0x562bc2e61310 2023-09-30T14:39:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f4634bd9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e61cd0 0x562bc2e65200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f462f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e61310 0x562bc2e714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f4634bd9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e61cd0 0x562bc2e65200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35802/0 (socket says 172.21.15.32:35802) 2023-09-30T14:39:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f4634bd9700 1 -- 172.21.15.32:0/3207930390 learned_addr learned my addr 172.21.15.32:0/3207930390 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f462ffff700 1 --2- 172.21.15.32:0/3207930390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e60200 0x562bc2e64cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f4634bd9700 1 -- 172.21.15.32:0/3207930390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e61310 msgr2=0x562bc2e714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f4634bd9700 1 --2- 172.21.15.32:0/3207930390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e61310 0x562bc2e714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f4634bd9700 1 -- 172.21.15.32:0/3207930390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e60200 msgr2=0x562bc2e64cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f4634bd9700 1 --2- 172.21.15.32:0/3207930390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e60200 0x562bc2e64cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f4634bd9700 1 -- 172.21.15.32:0/3207930390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bc2bc6c00 con 0x562bc2e61cd0 2023-09-30T14:39:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.011+0000 7f462f7fe700 1 --2- 172.21.15.32:0/3207930390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e61310 0x562bc2e714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.012+0000 7f4634bd9700 1 --2- 172.21.15.32:0/3207930390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e61cd0 0x562bc2e65200 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4628016000 tx=0x7f4628006210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaaf34de7eb5e016 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.012+0000 7f4616ffd700 1 -- 172.21.15.32:0/3207930390 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462800f040 con 0x562bc2e61cd0 2023-09-30T14:39:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.012+0000 7f4616ffd700 1 -- 172.21.15.32:0/3207930390 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f462800a590 con 0x562bc2e61cd0 2023-09-30T14:39:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.012+0000 7f4616ffd700 1 -- 172.21.15.32:0/3207930390 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46280074a0 con 0x562bc2e61cd0 2023-09-30T14:39:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.012+0000 7f4642824b80 1 -- 172.21.15.32:0/3207930390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e61cd0 msgr2=0x562bc2e65200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.012+0000 7f4642824b80 1 --2- 172.21.15.32:0/3207930390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e61cd0 0x562bc2e65200 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4628016000 tx=0x7f4628006210 comp rx=0 tx=0).stop 2023-09-30T14:39:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.013+0000 7f4642824b80 1 -- 172.21.15.32:0/3207930390 shutdown_connections 2023-09-30T14:39:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.013+0000 7f4642824b80 1 --2- 172.21.15.32:0/3207930390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e61cd0 0x562bc2e65200 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.013+0000 7f4642824b80 1 --2- 172.21.15.32:0/3207930390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e60200 0x562bc2e64cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.013+0000 7f4642824b80 1 --2- 172.21.15.32:0/3207930390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e61310 0x562bc2e714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.013+0000 7f4642824b80 1 -- 172.21.15.32:0/3207930390 >> 172.21.15.32:0/3207930390 conn(0x562bc2dd4200 msgr2=0x562bc2e67fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.013+0000 7f4642824b80 1 -- 172.21.15.32:0/3207930390 shutdown_connections 2023-09-30T14:39:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.013+0000 7f4642824b80 1 -- 172.21.15.32:0/3207930390 wait complete. 2023-09-30T14:39:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.013+0000 7f4642824b80 1 -- start start 2023-09-30T14:39:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.013+0000 7f4642824b80 1 Processor -- start 2023-09-30T14:39:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f4642824b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e60200 0x562bc2e6f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f4642824b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e61310 0x562bc2e6f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f4642824b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e61cd0 0x562bc2e70720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f4642824b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc2e70c60 con 0x562bc2e61310 2023-09-30T14:39:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f4642824b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc2e70dd0 con 0x562bc2e61cd0 2023-09-30T14:39:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f4642824b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc2e70f90 con 0x562bc2e60200 2023-09-30T14:39:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f4642824b80 10 client.0 _ll_register_callbacks cb 0x7f4620005b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f462f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e61310 0x562bc2e6f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f462f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e61310 0x562bc2e6f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35806/0 (socket says 172.21.15.32:35806) 2023-09-30T14:39:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f462f7fe700 1 -- 172.21.15.32:0/1869053850 learned_addr learned my addr 172.21.15.32:0/1869053850 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f462ffff700 1 --2- 172.21.15.32:0/1869053850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e60200 0x562bc2e6f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f4634bd9700 1 --2- 172.21.15.32:0/1869053850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e61cd0 0x562bc2e70720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f462f7fe700 1 -- 172.21.15.32:0/1869053850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e60200 msgr2=0x562bc2e6f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f462f7fe700 1 --2- 172.21.15.32:0/1869053850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e60200 0x562bc2e6f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f462f7fe700 1 -- 172.21.15.32:0/1869053850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e61cd0 msgr2=0x562bc2e70720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f462f7fe700 1 --2- 172.21.15.32:0/1869053850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e61cd0 0x562bc2e70720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f462f7fe700 1 -- 172.21.15.32:0/1869053850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46240119e0 con 0x562bc2e61310 2023-09-30T14:39:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f462ffff700 1 --2- 172.21.15.32:0/1869053850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e60200 0x562bc2e6f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:39:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f462f7fe700 1 --2- 172.21.15.32:0/1869053850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e61310 0x562bc2e6f710 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f46240094f0 tx=0x7f46240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10ae12d1e61d063 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.014+0000 7f460f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.015+0000 7f460f7fe700 1 -- 172.21.15.32:0/1869053850 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624025070 con 0x562bc2e61310 2023-09-30T14:39:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.015+0000 7f460f7fe700 1 -- 172.21.15.32:0/1869053850 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4624018070 con 0x562bc2e61310 2023-09-30T14:39:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.015+0000 7f4642824b80 1 -- 172.21.15.32:0/1869053850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46200998e0 con 0x562bc2e61310 2023-09-30T14:39:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.015+0000 7f460f7fe700 1 -- 172.21.15.32:0/1869053850 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46240277a0 con 0x562bc2e61310 2023-09-30T14:39:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.015+0000 7f4642824b80 10 client.15552 fetch_fsmap learned FSMap version 24 2023-09-30T14:39:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.015+0000 7f4642824b80 1 -- 172.21.15.32:0/1869053850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f4630108270 con 0x562bc2e61310 2023-09-30T14:39:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.015+0000 7f460f7fe700 1 -- 172.21.15.32:0/1869053850 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46200998e0 con 0x562bc2e61310 2023-09-30T14:39:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.015+0000 7f460f7fe700 1 -- 172.21.15.32:0/1869053850 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f4624038070 con 0x562bc2e61310 2023-09-30T14:39:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.015+0000 7f4642824b80 10 client.15552 fetch_fsmap finished waiting for FSMap version 24 2023-09-30T14:39:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.015+0000 7f4642824b80 10 client.15552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.015+0000 7f4642824b80 20 client.15552 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.015+0000 7f4642824b80 1 --2- 172.21.15.32:0/1869053850 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562bc2eca920 0x562bc2eccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.015+0000 7f4642824b80 4 client.15552 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:39:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.015+0000 7f4642824b80 1 -- 172.21.15.32:0/1869053850 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562bc2dabad0 con 0x562bc2eca920 2023-09-30T14:39:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.016+0000 7f462ffff700 1 --2- 172.21.15.32:0/1869053850 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562bc2eca920 0x562bc2eccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.016+0000 7f462ffff700 1 --2- 172.21.15.32:0/1869053850 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562bc2eca920 0x562bc2eccd40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4584b30f003290e server_cookie=cb5d68bd959ca05e in_seq=0 out_seq=0 2023-09-30T14:39:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.016+0000 7f460f7fe700 10 client.15552 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.020+0000 7f460f7fe700 1 -- 172.21.15.32:0/1869053850 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562bc2dabad0 con 0x562bc2eca920 2023-09-30T14:39:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.020+0000 7f460f7fe700 10 client.15552 handle_command_reply: tid=0 2023-09-30T14:39:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.020+0000 7f4642824b80 1 client.15552 shutdown 2023-09-30T14:39:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.020+0000 7f4642824b80 1 -- 172.21.15.32:0/1869053850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e61310 msgr2=0x562bc2e6f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.020+0000 7f4642824b80 1 --2- 172.21.15.32:0/1869053850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e61310 0x562bc2e6f710 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f46240094f0 tx=0x7f46240047c0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.020+0000 7f4642824b80 1 -- 172.21.15.32:0/1869053850 shutdown_connections 2023-09-30T14:39:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.020+0000 7f4642824b80 1 --2- 172.21.15.32:0/1869053850 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562bc2eca920 0x562bc2eccd40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.020+0000 7f4642824b80 1 --2- 172.21.15.32:0/1869053850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bc2e61cd0 0x562bc2e70720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.020+0000 7f4642824b80 1 --2- 172.21.15.32:0/1869053850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bc2e60200 0x562bc2e6f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.020+0000 7f4642824b80 1 --2- 172.21.15.32:0/1869053850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562bc2e61310 0x562bc2e6f710 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f4642824b80 1 -- 172.21.15.32:0/1869053850 >> 172.21.15.32:0/1869053850 conn(0x562bc2dd4200 msgr2=0x562bc2e64cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f460f7fe700 0 client.15552 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f4642824b80 1 -- 172.21.15.32:0/1869053850 shutdown_connections 2023-09-30T14:39:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f4642824b80 1 -- 172.21.15.32:0/1869053850 wait complete. 2023-09-30T14:39:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f4642824b80 20 client.15552 trim_cache size 0 max 16384 2023-09-30T14:39:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:39:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:39:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:39:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:39:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f463663c700 1 -- 172.21.15.32:0/3754184913 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f46180a1020 msgr2=0x7f46180a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f463663c700 1 --2- 172.21.15.32:0/3754184913 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f46180a1020 0x7f46180a34e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4624009bf0 tx=0x7f4624009260 comp rx=0 tx=0).stop 2023-09-30T14:39:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f463663c700 1 -- 172.21.15.32:0/3754184913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46300ff4e0 msgr2=0x7f4630196d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f463663c700 1 --2- 172.21.15.32:0/3754184913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46300ff4e0 0x7f4630196d80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f462000d980 tx=0x7f462000dd40 comp rx=0 tx=0).stop 2023-09-30T14:39:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f463663c700 1 -- 172.21.15.32:0/3754184913 shutdown_connections 2023-09-30T14:39:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f463663c700 1 --2- 172.21.15.32:0/3754184913 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f46180a1020 0x7f46180a34e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f463663c700 1 --2- 172.21.15.32:0/3754184913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4630101070 0x7f4630198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f463663c700 1 --2- 172.21.15.32:0/3754184913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46300ff4e0 0x7f4630196d80 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f463663c700 1 --2- 172.21.15.32:0/3754184913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46301006d0 0x7f46301972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f463663c700 1 -- 172.21.15.32:0/3754184913 >> 172.21.15.32:0/3754184913 conn(0x7f46300faa30 msgr2=0x7f46300fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f463663c700 1 -- 172.21.15.32:0/3754184913 shutdown_connections 2023-09-30T14:39:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:22.021+0000 7f463663c700 1 -- 172.21.15.32:0/3754184913 wait complete. 2023-09-30T14:39:22.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:21 smithi156 ceph-mon[103579]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-09-30T14:39:22.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:21 smithi156 ceph-mon[103579]: scrub summary: idle 2023-09-30T14:39:22.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:21 smithi156 ceph-mon[103579]: scrub summary: idle 2023-09-30T14:39:22.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:21 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] up:active 2023-09-30T14:39:22.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:21 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:39:22.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:21 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:39:22.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:21 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:39:22.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:21 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:22.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:21 smithi148 ceph-mon[102355]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-09-30T14:39:22.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:21 smithi148 ceph-mon[102355]: scrub summary: idle 2023-09-30T14:39:22.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:21 smithi148 ceph-mon[102355]: scrub summary: idle 2023-09-30T14:39:22.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:21 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] up:active 2023-09-30T14:39:22.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:21 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:39:22.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:21 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:39:22.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:21 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:39:22.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:21 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:22.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:21 smithi032 ceph-mon[98592]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-09-30T14:39:22.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:21 smithi032 ceph-mon[98592]: scrub summary: idle 2023-09-30T14:39:22.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:21 smithi032 ceph-mon[98592]: scrub summary: idle 2023-09-30T14:39:22.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:21 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] up:active 2023-09-30T14:39:22.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:21 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:39:22.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:21 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:39:22.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:21 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:39:22.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:21 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:39:23.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:23 smithi156 ceph-mon[103579]: pgmap v320: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 8 op/s 2023-09-30T14:39:23.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:23 smithi148 ceph-mon[102355]: pgmap v320: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 8 op/s 2023-09-30T14:39:23.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:23 smithi032 ceph-mon[98592]: pgmap v320: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 8 op/s 2023-09-30T14:39:25.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:25 smithi156 ceph-mon[103579]: pgmap v321: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2023-09-30T14:39:25.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:25 smithi148 ceph-mon[102355]: pgmap v321: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2023-09-30T14:39:25.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:25 smithi032 ceph-mon[98592]: pgmap v321: 97 pgs: 97 active+clean; 666 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2023-09-30T14:39:25.692 INFO:tasks.workunit.client.0.smithi032.stderr:Note: switching to '1d10b71792f3be8887a7631e69851ac2df3585af'. 2023-09-30T14:39:25.692 INFO:tasks.workunit.client.0.smithi032.stderr: 2023-09-30T14:39:25.692 INFO:tasks.workunit.client.0.smithi032.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-09-30T14:39:25.693 INFO:tasks.workunit.client.0.smithi032.stderr:changes and commit them, and you can discard any commits you make in this 2023-09-30T14:39:25.693 INFO:tasks.workunit.client.0.smithi032.stderr:state without impacting any branches by switching back to a branch. 2023-09-30T14:39:25.693 INFO:tasks.workunit.client.0.smithi032.stderr: 2023-09-30T14:39:25.693 INFO:tasks.workunit.client.0.smithi032.stderr:If you want to create a new branch to retain commits you create, you may 2023-09-30T14:39:25.693 INFO:tasks.workunit.client.0.smithi032.stderr:do so (now or later) by using -c with the switch command. Example: 2023-09-30T14:39:25.693 INFO:tasks.workunit.client.0.smithi032.stderr: 2023-09-30T14:39:25.693 INFO:tasks.workunit.client.0.smithi032.stderr: git switch -c 2023-09-30T14:39:25.693 INFO:tasks.workunit.client.0.smithi032.stderr: 2023-09-30T14:39:25.693 INFO:tasks.workunit.client.0.smithi032.stderr:Or undo this operation with: 2023-09-30T14:39:25.693 INFO:tasks.workunit.client.0.smithi032.stderr: 2023-09-30T14:39:25.693 INFO:tasks.workunit.client.0.smithi032.stderr: git switch - 2023-09-30T14:39:25.693 INFO:tasks.workunit.client.0.smithi032.stderr: 2023-09-30T14:39:25.694 INFO:tasks.workunit.client.0.smithi032.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-09-30T14:39:25.694 INFO:tasks.workunit.client.0.smithi032.stderr: 2023-09-30T14:39:25.694 INFO:tasks.workunit.client.0.smithi032.stderr:HEAD is now at 1d10b71792f Merge pull request #53723 from zdover23/wip-doc-2023-09-29-architecture-13-of-x 2023-09-30T14:39:25.697 DEBUG:teuthology.orchestra.run.smithi032:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-09-30T14:39:25.822 INFO:tasks.workunit.client.0.smithi032.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-09-30T14:39:25.825 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-09-30T14:39:25.825 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-09-30T14:39:26.350 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-09-30T14:39:26.404 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-09-30T14:39:26.443 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-09-30T14:39:26.445 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-09-30T14:39:26.445 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-09-30T14:39:26.482 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-09-30T14:39:26.486 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T14:39:26.486 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-09-30T14:39:26.503 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2023-09-30T14:39:26.504 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2023-09-30T14:39:26.504 DEBUG:teuthology.orchestra.run.smithi032:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1d10b71792f3be8887a7631e69851ac2df3585af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2023-09-30T14:39:26.569 INFO:tasks.workunit.client.0.smithi032.stderr:+ dbench 1 2023-09-30T14:39:26.577 INFO:tasks.workunit.client.0.smithi032.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-09-30T14:39:26.578 INFO:tasks.workunit.client.0.smithi032.stdout: 2023-09-30T14:39:26.578 INFO:tasks.workunit.client.0.smithi032.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-09-30T14:39:26.588 INFO:tasks.workunit.client.0.smithi032.stdout:0 of 1 processes prepared for launch 0 sec 2023-09-30T14:39:26.688 INFO:tasks.workunit.client.0.smithi032.stdout:1 of 1 processes prepared for launch 0 sec 2023-09-30T14:39:26.688 INFO:tasks.workunit.client.0.smithi032.stdout:releasing clients 2023-09-30T14:39:27.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:27 smithi156 ceph-mon[103579]: pgmap v322: 97 pgs: 97 active+clean; 671 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 7 op/s 2023-09-30T14:39:27.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:27 smithi148 ceph-mon[102355]: pgmap v322: 97 pgs: 97 active+clean; 671 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 7 op/s 2023-09-30T14:39:27.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:27 smithi032 ceph-mon[98592]: pgmap v322: 97 pgs: 97 active+clean; 671 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 7 op/s 2023-09-30T14:39:27.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 3513 38.37 MB/sec warmup 1 sec latency 5.637 ms 2023-09-30T14:39:28.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 6752 26.70 MB/sec warmup 2 sec latency 5.834 ms 2023-09-30T14:39:29.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:29 smithi156 ceph-mon[103579]: pgmap v323: 97 pgs: 97 active+clean; 746 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7.3 KiB/s wr, 6 op/s 2023-09-30T14:39:29.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:29 smithi148 ceph-mon[102355]: pgmap v323: 97 pgs: 97 active+clean; 746 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7.3 KiB/s wr, 6 op/s 2023-09-30T14:39:29.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:29 smithi032 ceph-mon[98592]: pgmap v323: 97 pgs: 97 active+clean; 746 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7.3 KiB/s wr, 6 op/s 2023-09-30T14:39:29.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 9995 23.69 MB/sec warmup 3 sec latency 5.904 ms 2023-09-30T14:39:30.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 13030 21.33 MB/sec warmup 4 sec latency 17.942 ms 2023-09-30T14:39:31.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:31 smithi156 ceph-mon[103579]: pgmap v324: 97 pgs: 97 active+clean; 2.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 139 KiB/s wr, 28 op/s 2023-09-30T14:39:31.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:31 smithi148 ceph-mon[102355]: pgmap v324: 97 pgs: 97 active+clean; 2.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 139 KiB/s wr, 28 op/s 2023-09-30T14:39:31.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:31 smithi032 ceph-mon[98592]: pgmap v324: 97 pgs: 97 active+clean; 2.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 139 KiB/s wr, 28 op/s 2023-09-30T14:39:31.688 INFO:tasks.workunit.client.0.smithi032.stdout: 1 15691 20.51 MB/sec warmup 5 sec latency 24.580 ms 2023-09-30T14:39:32.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 18733 19.58 MB/sec warmup 6 sec latency 6.795 ms 2023-09-30T14:39:33.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:33 smithi156 ceph-mon[103579]: pgmap v325: 97 pgs: 97 active+clean; 3.8 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 318 KiB/s wr, 37 op/s 2023-09-30T14:39:33.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:33 smithi148 ceph-mon[102355]: pgmap v325: 97 pgs: 97 active+clean; 3.8 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 318 KiB/s wr, 37 op/s 2023-09-30T14:39:33.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:33 smithi032 ceph-mon[98592]: pgmap v325: 97 pgs: 97 active+clean; 3.8 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 318 KiB/s wr, 37 op/s 2023-09-30T14:39:33.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 22001 19.42 MB/sec warmup 7 sec latency 12.607 ms 2023-09-30T14:39:34.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 25254 19.36 MB/sec warmup 8 sec latency 18.334 ms 2023-09-30T14:39:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.139+0000 7f51325ab700 1 -- 172.21.15.32:0/4222826581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f512c1018f0 msgr2=0x7f512c1081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.139+0000 7f51325ab700 1 --2- 172.21.15.32:0/4222826581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f512c1018f0 0x7f512c1081a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f512400b0d0 tx=0x7f512400b3e0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.139+0000 7f51325ab700 1 -- 172.21.15.32:0/4222826581 shutdown_connections 2023-09-30T14:39:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.139+0000 7f51325ab700 1 --2- 172.21.15.32:0/4222826581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f512c1018f0 0x7f512c1081a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.139+0000 7f51325ab700 1 --2- 172.21.15.32:0/4222826581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c0ffd60 0x7f512c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.139+0000 7f51325ab700 1 --2- 172.21.15.32:0/4222826581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c100f50 0x7f512c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.139+0000 7f51325ab700 1 -- 172.21.15.32:0/4222826581 >> 172.21.15.32:0/4222826581 conn(0x7f512c0fb2b0 msgr2=0x7f512c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.139+0000 7f51325ab700 1 -- 172.21.15.32:0/4222826581 shutdown_connections 2023-09-30T14:39:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.139+0000 7f51325ab700 1 -- 172.21.15.32:0/4222826581 wait complete. 2023-09-30T14:39:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.140+0000 7f51325ab700 1 Processor -- start 2023-09-30T14:39:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.140+0000 7f51325ab700 1 -- start start 2023-09-30T14:39:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.140+0000 7f51325ab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f512c0ffd60 0x7f512c195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.140+0000 7f51325ab700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c100f50 0x7f512c196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.140+0000 7f51325ab700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c1018f0 0x7f512c196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.140+0000 7f51325ab700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c197650 con 0x7f512c0ffd60 2023-09-30T14:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.140+0000 7f51325ab700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c1977c0 con 0x7f512c100f50 2023-09-30T14:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.140+0000 7f51325ab700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c197980 con 0x7f512c1018f0 2023-09-30T14:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.140+0000 7f512bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f512c0ffd60 0x7f512c195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.141+0000 7f512bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f512c0ffd60 0x7f512c195c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35844/0 (socket says 172.21.15.32:35844) 2023-09-30T14:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.141+0000 7f512b7fe700 1 --2- 172.21.15.32:0/2192620158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c100f50 0x7f512c196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.141+0000 7f512bfff700 1 -- 172.21.15.32:0/2192620158 learned_addr learned my addr 172.21.15.32:0/2192620158 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.141+0000 7f5130b48700 1 --2- 172.21.15.32:0/2192620158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c1018f0 0x7f512c196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.141+0000 7f512bfff700 1 -- 172.21.15.32:0/2192620158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c1018f0 msgr2=0x7f512c196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.141+0000 7f512bfff700 1 --2- 172.21.15.32:0/2192620158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c1018f0 0x7f512c196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.141+0000 7f512bfff700 1 -- 172.21.15.32:0/2192620158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c100f50 msgr2=0x7f512c196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.141+0000 7f512bfff700 1 --2- 172.21.15.32:0/2192620158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c100f50 0x7f512c196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.141+0000 7f512bfff700 1 -- 172.21.15.32:0/2192620158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c009520 con 0x7f512c0ffd60 2023-09-30T14:39:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.141+0000 7f512b7fe700 1 --2- 172.21.15.32:0/2192620158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c100f50 0x7f512c196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.141+0000 7f512bfff700 1 --2- 172.21.15.32:0/2192620158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f512c0ffd60 0x7f512c195c30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f511c00ea50 tx=0x7f511c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.141+0000 7f51297fa700 1 -- 172.21.15.32:0/2192620158 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c00cbd0 con 0x7f512c0ffd60 2023-09-30T14:39:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.141+0000 7f51325ab700 1 -- 172.21.15.32:0/2192620158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5120009520 con 0x7f512c0ffd60 2023-09-30T14:39:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.141+0000 7f51297fa700 1 -- 172.21.15.32:0/2192620158 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f511c00cd30 con 0x7f512c0ffd60 2023-09-30T14:39:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.141+0000 7f51297fa700 1 -- 172.21.15.32:0/2192620158 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c018680 con 0x7f512c0ffd60 2023-09-30T14:39:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.141+0000 7f51325ab700 1 -- 172.21.15.32:0/2192620158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5124009d00 con 0x7f512c0ffd60 2023-09-30T14:39:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.143+0000 7f51297fa700 1 -- 172.21.15.32:0/2192620158 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f511c0188a0 con 0x7f512c0ffd60 2023-09-30T14:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.144+0000 7f51297fa700 1 --2- 172.21.15.32:0/2192620158 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51140a0fa0 0x7f51140a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.144+0000 7f51297fa700 1 -- 172.21.15.32:0/2192620158 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f511c014070 con 0x7f512c0ffd60 2023-09-30T14:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.144+0000 7f512b7fe700 1 --2- 172.21.15.32:0/2192620158 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51140a0fa0 0x7f51140a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.145+0000 7f512b7fe700 1 --2- 172.21.15.32:0/2192620158 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51140a0fa0 0x7f51140a3460 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5120009fd0 tx=0x7f5120009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.145+0000 7f513e793b80 1 Processor -- start 2023-09-30T14:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.145+0000 7f513e793b80 1 -- start start 2023-09-30T14:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.145+0000 7f513e793b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139060200 0x55f139060620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.145+0000 7f513e793b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139061310 0x55f139061790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.145+0000 7f513e793b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061cd0 0x55f139068560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.145+0000 7f513e793b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f138fc8400 con 0x55f139061cd0 2023-09-30T14:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.145+0000 7f513e793b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f138fb9bc0 con 0x55f139061310 2023-09-30T14:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.145+0000 7f513e793b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f138fb74e0 con 0x55f139060200 2023-09-30T14:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.145+0000 7f5130b48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061cd0 0x55f139068560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.145+0000 7f512b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139061310 0x55f139061790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.145+0000 7f512bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139060200 0x55f139060620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.146+0000 7f512b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139061310 0x55f139061790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47330/0 (socket says 172.21.15.32:47330) 2023-09-30T14:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.146+0000 7f512bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139060200 0x55f139060620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:50234/0 (socket says 172.21.15.32:50234) 2023-09-30T14:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.146+0000 7f5130b48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061cd0 0x55f139068560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35856/0 (socket says 172.21.15.32:35856) 2023-09-30T14:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.146+0000 7f512b7fe700 1 -- 172.21.15.32:0/669433602 learned_addr learned my addr 172.21.15.32:0/669433602 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.146+0000 7f5130b48700 1 -- 172.21.15.32:0/669433602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139060200 msgr2=0x55f139060620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.146+0000 7f5130b48700 1 --2- 172.21.15.32:0/669433602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139060200 0x55f139060620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.146+0000 7f5130b48700 1 -- 172.21.15.32:0/669433602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139061310 msgr2=0x55f139061790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.146+0000 7f5130b48700 1 --2- 172.21.15.32:0/669433602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139061310 0x55f139061790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.146+0000 7f5130b48700 1 -- 172.21.15.32:0/669433602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f139015870 con 0x55f139061cd0 2023-09-30T14:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.146+0000 7f512b7fe700 1 --2- 172.21.15.32:0/669433602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139061310 0x55f139061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.146+0000 7f5130b48700 1 --2- 172.21.15.32:0/669433602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061cd0 0x55f139068560 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f512400b770 tx=0x7f5124009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba033d03ed2b7b26 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.146+0000 7f51127fc700 1 -- 172.21.15.32:0/669433602 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512400e040 con 0x55f139061cd0 2023-09-30T14:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.146+0000 7f51127fc700 1 -- 172.21.15.32:0/669433602 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5124009460 con 0x55f139061cd0 2023-09-30T14:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.146+0000 7f51127fc700 1 -- 172.21.15.32:0/669433602 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512401c6a0 con 0x55f139061cd0 2023-09-30T14:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.147+0000 7f513e793b80 1 -- 172.21.15.32:0/669433602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061cd0 msgr2=0x55f139068560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.147+0000 7f513e793b80 1 --2- 172.21.15.32:0/669433602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061cd0 0x55f139068560 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f512400b770 tx=0x7f5124009c20 comp rx=0 tx=0).stop 2023-09-30T14:39:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.147+0000 7f513e793b80 1 -- 172.21.15.32:0/669433602 shutdown_connections 2023-09-30T14:39:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.147+0000 7f513e793b80 1 --2- 172.21.15.32:0/669433602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061cd0 0x55f139068560 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.147+0000 7f513e793b80 1 --2- 172.21.15.32:0/669433602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139060200 0x55f139060620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.147+0000 7f513e793b80 1 --2- 172.21.15.32:0/669433602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139061310 0x55f139061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.147+0000 7f513e793b80 1 -- 172.21.15.32:0/669433602 >> 172.21.15.32:0/669433602 conn(0x55f138fd4200 msgr2=0x55f13905aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.147+0000 7f513e793b80 1 -- 172.21.15.32:0/669433602 shutdown_connections 2023-09-30T14:39:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.147+0000 7f513e793b80 1 -- 172.21.15.32:0/669433602 wait complete. 2023-09-30T14:39:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.147+0000 7f513e793b80 1 -- start start 2023-09-30T14:39:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f513e793b80 1 Processor -- start 2023-09-30T14:39:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f513e793b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139060200 0x55f139069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f513e793b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061310 0x55f13906a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f513e793b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139061cd0 0x55f139071310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f513e793b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f138fb2dd0 con 0x55f139061310 2023-09-30T14:39:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f513e793b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f138f4f6b0 con 0x55f139060200 2023-09-30T14:39:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f513e793b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f138f4f350 con 0x55f139061cd0 2023-09-30T14:39:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f513e793b80 10 client.0 _ll_register_callbacks cb 0x55f138dac7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f512bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139060200 0x55f139069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f512b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061310 0x55f13906a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f512bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139060200 0x55f139069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47334/0 (socket says 172.21.15.32:47334) 2023-09-30T14:39:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f512bfff700 1 -- 172.21.15.32:0/4238085757 learned_addr learned my addr 172.21.15.32:0/4238085757 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f5130b48700 1 --2- 172.21.15.32:0/4238085757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139061cd0 0x55f139071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f512bfff700 1 -- 172.21.15.32:0/4238085757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139061cd0 msgr2=0x55f139071310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f512bfff700 1 --2- 172.21.15.32:0/4238085757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139061cd0 0x55f139071310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f512bfff700 1 -- 172.21.15.32:0/4238085757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061310 msgr2=0x55f13906a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f512bfff700 1 --2- 172.21.15.32:0/4238085757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061310 0x55f13906a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.148+0000 7f512bfff700 1 -- 172.21.15.32:0/4238085757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5120017050 con 0x55f139060200 2023-09-30T14:39:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f512bfff700 1 --2- 172.21.15.32:0/4238085757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139060200 0x55f139069d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f511c099e20 tx=0x7f511c01e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe3e8dc180cf743 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f510b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:39:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f510b7fe700 1 -- 172.21.15.32:0/4238085757 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c0b1070 con 0x55f139060200 2023-09-30T14:39:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f513e793b80 1 -- 172.21.15.32:0/4238085757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f139053090 con 0x55f139060200 2023-09-30T14:39:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f510b7fe700 1 -- 172.21.15.32:0/4238085757 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f511c0a24b0 con 0x55f139060200 2023-09-30T14:39:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f510b7fe700 1 -- 172.21.15.32:0/4238085757 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f511c0b8170 con 0x55f139060200 2023-09-30T14:39:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f513e793b80 10 client.25262 fetch_fsmap learned FSMap version 24 2023-09-30T14:39:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f513e793b80 1 -- 172.21.15.32:0/4238085757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f512c108af0 con 0x55f139060200 2023-09-30T14:39:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f510b7fe700 1 -- 172.21.15.32:0/4238085757 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f139053090 con 0x55f139060200 2023-09-30T14:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f510b7fe700 1 -- 172.21.15.32:0/4238085757 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f511c0a2140 con 0x55f139060200 2023-09-30T14:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f513e793b80 10 client.25262 fetch_fsmap finished waiting for FSMap version 24 2023-09-30T14:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f513e793b80 10 client.25262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f513e793b80 20 client.25262 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f513e793b80 1 --2- 172.21.15.32:0/4238085757 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f139075bc0 0x55f139077fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f513e793b80 4 client.25262 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f513e793b80 1 -- 172.21.15.32:0/4238085757 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f138fabdb0 con 0x55f139075bc0 2023-09-30T14:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.149+0000 7f512b7fe700 1 --2- 172.21.15.32:0/4238085757 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f139075bc0 0x55f139077fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.150+0000 7f510b7fe700 10 client.25262 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.150+0000 7f512b7fe700 1 --2- 172.21.15.32:0/4238085757 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f139075bc0 0x55f139077fe0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a360e2484dcef596 server_cookie=d143db9536fe1f36 in_seq=0 out_seq=0 2023-09-30T14:39:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.150+0000 7f510b7fe700 1 -- 172.21.15.32:0/4238085757 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f138fabdb0 con 0x55f139075bc0 2023-09-30T14:39:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.150+0000 7f510b7fe700 10 client.25262 handle_command_reply: tid=0 2023-09-30T14:39:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.150+0000 7f513e793b80 1 client.25262 shutdown 2023-09-30T14:39:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.151+0000 7f513e793b80 1 -- 172.21.15.32:0/4238085757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139060200 msgr2=0x55f139069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.151+0000 7f513e793b80 1 --2- 172.21.15.32:0/4238085757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139060200 0x55f139069d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f511c099e20 tx=0x7f511c01e5a0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.151+0000 7f513e793b80 1 -- 172.21.15.32:0/4238085757 shutdown_connections 2023-09-30T14:39:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.151+0000 7f513e793b80 1 --2- 172.21.15.32:0/4238085757 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f139075bc0 0x55f139077fe0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.151+0000 7f513e793b80 1 --2- 172.21.15.32:0/4238085757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139061cd0 0x55f139071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.151+0000 7f510b7fe700 0 client.25262 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.151+0000 7f513e793b80 1 --2- 172.21.15.32:0/4238085757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139060200 0x55f139069d90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.151+0000 7f513e793b80 1 --2- 172.21.15.32:0/4238085757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061310 0x55f13906a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.151+0000 7f513e793b80 1 -- 172.21.15.32:0/4238085757 >> 172.21.15.32:0/4238085757 conn(0x55f138fd4200 msgr2=0x55f13905a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.151+0000 7f513e793b80 1 -- 172.21.15.32:0/4238085757 shutdown_connections 2023-09-30T14:39:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.151+0000 7f513e793b80 1 -- 172.21.15.32:0/4238085757 wait complete. 2023-09-30T14:39:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.151+0000 7f513e793b80 20 client.25262 trim_cache size 0 max 16384 2023-09-30T14:39:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.169+0000 7f513e793b80 1 Processor -- start 2023-09-30T14:39:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.169+0000 7f513e793b80 1 -- start start 2023-09-30T14:39:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.169+0000 7f513e793b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139060200 0x55f139068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.169+0000 7f513e793b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139061310 0x55f13905aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.169+0000 7f513e793b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061cd0 0x55f13905a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.169+0000 7f513e793b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f138e338c0 con 0x55f139061cd0 2023-09-30T14:39:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.169+0000 7f513e793b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f13905b400 con 0x55f139060200 2023-09-30T14:39:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.169+0000 7f513e793b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f139067e00 con 0x55f139061310 2023-09-30T14:39:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.169+0000 7f5130b48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061cd0 0x55f13905a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.169+0000 7f5130b48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061cd0 0x55f13905a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35870/0 (socket says 172.21.15.32:35870) 2023-09-30T14:39:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.169+0000 7f5130b48700 1 -- 172.21.15.32:0/1232841297 learned_addr learned my addr 172.21.15.32:0/1232841297 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.169+0000 7f512bfff700 1 --2- 172.21.15.32:0/1232841297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139060200 0x55f139068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.170+0000 7f512b7fe700 1 --2- 172.21.15.32:0/1232841297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139061310 0x55f13905aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.170+0000 7f5130b48700 1 -- 172.21.15.32:0/1232841297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139061310 msgr2=0x55f13905aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.170+0000 7f5130b48700 1 --2- 172.21.15.32:0/1232841297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139061310 0x55f13905aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.170+0000 7f5130b48700 1 -- 172.21.15.32:0/1232841297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139060200 msgr2=0x55f139068ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.170+0000 7f5130b48700 1 --2- 172.21.15.32:0/1232841297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139060200 0x55f139068ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.170+0000 7f5130b48700 1 -- 172.21.15.32:0/1232841297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f138dddd80 con 0x55f139061cd0 2023-09-30T14:39:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.170+0000 7f5130b48700 1 --2- 172.21.15.32:0/1232841297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061cd0 0x55f13905a590 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5124009fd0 tx=0x7f51240095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89e03999988995 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.170+0000 7f5112ffd700 1 -- 172.21.15.32:0/1232841297 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512400e070 con 0x55f139061cd0 2023-09-30T14:39:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.170+0000 7f5112ffd700 1 -- 172.21.15.32:0/1232841297 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f512400f040 con 0x55f139061cd0 2023-09-30T14:39:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.170+0000 7f5112ffd700 1 -- 172.21.15.32:0/1232841297 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512401cb70 con 0x55f139061cd0 2023-09-30T14:39:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.171+0000 7f513e793b80 1 -- 172.21.15.32:0/1232841297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061cd0 msgr2=0x55f13905a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.171+0000 7f513e793b80 1 --2- 172.21.15.32:0/1232841297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061cd0 0x55f13905a590 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5124009fd0 tx=0x7f51240095d0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.171+0000 7f513e793b80 1 -- 172.21.15.32:0/1232841297 shutdown_connections 2023-09-30T14:39:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.171+0000 7f513e793b80 1 --2- 172.21.15.32:0/1232841297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061cd0 0x55f13905a590 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.171+0000 7f513e793b80 1 --2- 172.21.15.32:0/1232841297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139060200 0x55f139068ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.171+0000 7f513e793b80 1 --2- 172.21.15.32:0/1232841297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139061310 0x55f13905aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.171+0000 7f513e793b80 1 -- 172.21.15.32:0/1232841297 >> 172.21.15.32:0/1232841297 conn(0x55f138fd4200 msgr2=0x55f13907c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.171+0000 7f513e793b80 1 -- 172.21.15.32:0/1232841297 shutdown_connections 2023-09-30T14:39:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.171+0000 7f513e793b80 1 -- 172.21.15.32:0/1232841297 wait complete. 2023-09-30T14:39:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.171+0000 7f513e793b80 1 -- start start 2023-09-30T14:39:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.171+0000 7f513e793b80 1 Processor -- start 2023-09-30T14:39:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f513e793b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139060200 0x55f1390c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f513e793b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061310 0x55f1390c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f513e793b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139061cd0 0x55f1390ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f513e793b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1390caea0 con 0x55f139061310 2023-09-30T14:39:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f513e793b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1390cb010 con 0x55f139061cd0 2023-09-30T14:39:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f513e793b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1390cb1d0 con 0x55f139060200 2023-09-30T14:39:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f513e793b80 10 client.0 _ll_register_callbacks cb 0x7f511c099b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f512bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139060200 0x55f1390c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f512bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139060200 0x55f1390c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:50254/0 (socket says 172.21.15.32:50254) 2023-09-30T14:39:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f512bfff700 1 -- 172.21.15.32:0/3558382640 learned_addr learned my addr 172.21.15.32:0/3558382640 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f5130b48700 1 --2- 172.21.15.32:0/3558382640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139061cd0 0x55f1390ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f512b7fe700 1 --2- 172.21.15.32:0/3558382640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061310 0x55f1390c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f512bfff700 1 -- 172.21.15.32:0/3558382640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139061cd0 msgr2=0x55f1390ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f512bfff700 1 --2- 172.21.15.32:0/3558382640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139061cd0 0x55f1390ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f512bfff700 1 -- 172.21.15.32:0/3558382640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061310 msgr2=0x55f1390c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f512bfff700 1 --2- 172.21.15.32:0/3558382640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061310 0x55f1390c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f512bfff700 1 -- 172.21.15.32:0/3558382640 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f139015870 con 0x55f139060200 2023-09-30T14:39:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.172+0000 7f5130b48700 1 --2- 172.21.15.32:0/3558382640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139061cd0 0x55f1390ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:39:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f512bfff700 1 --2- 172.21.15.32:0/3558382640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139060200 0x55f1390c5bd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f511c09a0b0 tx=0x7f511c00efa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f9b060c96ea633 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f510b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:39:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f510b7fe700 1 -- 172.21.15.32:0/3558382640 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c0b1070 con 0x55f139060200 2023-09-30T14:39:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f510b7fe700 1 -- 172.21.15.32:0/3558382640 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f511c0a24b0 con 0x55f139060200 2023-09-30T14:39:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f513e793b80 1 -- 172.21.15.32:0/3558382640 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5124009460 con 0x55f139060200 2023-09-30T14:39:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f510b7fe700 1 -- 172.21.15.32:0/3558382640 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f511c0a2020 con 0x55f139060200 2023-09-30T14:39:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f513e793b80 10 client.25255 fetch_fsmap learned FSMap version 24 2023-09-30T14:39:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f513e793b80 1 -- 172.21.15.32:0/3558382640 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f138dc6c00 con 0x55f139060200 2023-09-30T14:39:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f510b7fe700 1 -- 172.21.15.32:0/3558382640 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5124009460 con 0x55f139060200 2023-09-30T14:39:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f510b7fe700 1 -- 172.21.15.32:0/3558382640 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f511c0991a0 con 0x55f139060200 2023-09-30T14:39:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f513e793b80 10 client.25255 fetch_fsmap finished waiting for FSMap version 24 2023-09-30T14:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f513e793b80 10 client.25255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f513e793b80 20 client.25255 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f513e793b80 1 --2- 172.21.15.32:0/3558382640 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f13906d6f0 0x55f13906db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f513e793b80 4 client.25255 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f513e793b80 1 -- 172.21.15.32:0/3558382640 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f138fabad0 con 0x55f13906d6f0 2023-09-30T14:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.173+0000 7f512b7fe700 1 --2- 172.21.15.32:0/3558382640 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f13906d6f0 0x55f13906db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.174+0000 7f512b7fe700 1 --2- 172.21.15.32:0/3558382640 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f13906d6f0 0x55f13906db00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65f374a88d13a307 server_cookie=d345cbd1c97cc9e8 in_seq=0 out_seq=0 2023-09-30T14:39:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.174+0000 7f510b7fe700 10 client.25255 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.180+0000 7f510b7fe700 1 -- 172.21.15.32:0/3558382640 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f138fabad0 con 0x55f13906d6f0 2023-09-30T14:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.180+0000 7f510b7fe700 10 client.25255 handle_command_reply: tid=0 2023-09-30T14:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.180+0000 7f513e793b80 1 client.25255 shutdown 2023-09-30T14:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.180+0000 7f513e793b80 1 -- 172.21.15.32:0/3558382640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139060200 msgr2=0x55f1390c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.180+0000 7f513e793b80 1 --2- 172.21.15.32:0/3558382640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139060200 0x55f1390c5bd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f511c09a0b0 tx=0x7f511c00efa0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.180+0000 7f513e793b80 1 -- 172.21.15.32:0/3558382640 shutdown_connections 2023-09-30T14:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.180+0000 7f513e793b80 1 --2- 172.21.15.32:0/3558382640 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f13906d6f0 0x55f13906db00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.180+0000 7f513e793b80 1 --2- 172.21.15.32:0/3558382640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f139061cd0 0x55f1390ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.180+0000 7f510b7fe700 0 client.25255 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.180+0000 7f513e793b80 1 --2- 172.21.15.32:0/3558382640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f139060200 0x55f1390c5bd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.180+0000 7f513e793b80 1 --2- 172.21.15.32:0/3558382640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f139061310 0x55f1390c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.180+0000 7f513e793b80 1 -- 172.21.15.32:0/3558382640 >> 172.21.15.32:0/3558382640 conn(0x55f138fd4200 msgr2=0x55f139068050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.180+0000 7f513e793b80 1 -- 172.21.15.32:0/3558382640 shutdown_connections 2023-09-30T14:39:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.181+0000 7f513e793b80 1 -- 172.21.15.32:0/3558382640 wait complete. 2023-09-30T14:39:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.181+0000 7f513e793b80 20 client.25255 trim_cache size 0 max 16384 2023-09-30T14:39:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:39:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:39:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:39:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:39:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.181+0000 7f51325ab700 1 -- 172.21.15.32:0/2192620158 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51140a0fa0 msgr2=0x7f51140a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.181+0000 7f51325ab700 1 --2- 172.21.15.32:0/2192620158 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51140a0fa0 0x7f51140a3460 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5120009fd0 tx=0x7f5120009260 comp rx=0 tx=0).stop 2023-09-30T14:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.181+0000 7f51325ab700 1 -- 172.21.15.32:0/2192620158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f512c0ffd60 msgr2=0x7f512c195c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.181+0000 7f51325ab700 1 --2- 172.21.15.32:0/2192620158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f512c0ffd60 0x7f512c195c30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f511c00ea50 tx=0x7f511c00ed60 comp rx=0 tx=0).stop 2023-09-30T14:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.181+0000 7f51325ab700 1 -- 172.21.15.32:0/2192620158 shutdown_connections 2023-09-30T14:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.181+0000 7f51325ab700 1 --2- 172.21.15.32:0/2192620158 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51140a0fa0 0x7f51140a3460 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.181+0000 7f51325ab700 1 --2- 172.21.15.32:0/2192620158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f512c1018f0 0x7f512c196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.181+0000 7f51325ab700 1 --2- 172.21.15.32:0/2192620158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f512c0ffd60 0x7f512c195c30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.181+0000 7f51325ab700 1 --2- 172.21.15.32:0/2192620158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c100f50 0x7f512c196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.181+0000 7f51325ab700 1 -- 172.21.15.32:0/2192620158 >> 172.21.15.32:0/2192620158 conn(0x7f512c0fb2b0 msgr2=0x7f512c1043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.181+0000 7f51325ab700 1 -- 172.21.15.32:0/2192620158 shutdown_connections 2023-09-30T14:39:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:35.181+0000 7f51325ab700 1 -- 172.21.15.32:0/2192620158 wait complete. 2023-09-30T14:39:35.304 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:35 smithi156 ceph-mon[103579]: pgmap v326: 97 pgs: 97 active+clean; 7.6 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 700 KiB/s wr, 103 op/s 2023-09-30T14:39:35.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:35 smithi148 ceph-mon[102355]: pgmap v326: 97 pgs: 97 active+clean; 7.6 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 700 KiB/s wr, 103 op/s 2023-09-30T14:39:35.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:35 smithi032 ceph-mon[98592]: pgmap v326: 97 pgs: 97 active+clean; 7.6 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 700 KiB/s wr, 103 op/s 2023-09-30T14:39:35.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 28644 19.30 MB/sec warmup 9 sec latency 10.717 ms 2023-09-30T14:39:36.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 31624 18.73 MB/sec warmup 10 sec latency 21.254 ms 2023-09-30T14:39:37.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:37 smithi156 ceph-mon[103579]: pgmap v327: 97 pgs: 97 active+clean; 9.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 890 KiB/s wr, 115 op/s 2023-09-30T14:39:37.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:37 smithi148 ceph-mon[102355]: pgmap v327: 97 pgs: 97 active+clean; 9.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 890 KiB/s wr, 115 op/s 2023-09-30T14:39:37.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:37 smithi032 ceph-mon[98592]: pgmap v327: 97 pgs: 97 active+clean; 9.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 890 KiB/s wr, 115 op/s 2023-09-30T14:39:37.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 33239 17.75 MB/sec warmup 11 sec latency 209.858 ms 2023-09-30T14:39:38.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 36240 17.71 MB/sec warmup 12 sec latency 74.051 ms 2023-09-30T14:39:39.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:39 smithi156 ceph-mon[103579]: pgmap v328: 97 pgs: 97 active+clean; 14 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.3 MiB/s wr, 184 op/s 2023-09-30T14:39:39.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:39 smithi148 ceph-mon[102355]: pgmap v328: 97 pgs: 97 active+clean; 14 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.3 MiB/s wr, 184 op/s 2023-09-30T14:39:39.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:39 smithi032 ceph-mon[98592]: pgmap v328: 97 pgs: 97 active+clean; 14 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.3 MiB/s wr, 184 op/s 2023-09-30T14:39:39.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 39478 17.79 MB/sec warmup 13 sec latency 36.069 ms 2023-09-30T14:39:40.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 42822 17.87 MB/sec warmup 14 sec latency 20.230 ms 2023-09-30T14:39:41.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:41 smithi156 ceph-mon[103579]: pgmap v329: 97 pgs: 97 active+clean; 14 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2023-09-30T14:39:41.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:41 smithi148 ceph-mon[102355]: pgmap v329: 97 pgs: 97 active+clean; 14 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2023-09-30T14:39:41.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:41 smithi032 ceph-mon[98592]: pgmap v329: 97 pgs: 97 active+clean; 14 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2023-09-30T14:39:41.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 45570 17.53 MB/sec warmup 15 sec latency 43.502 ms 2023-09-30T14:39:42.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 47550 17.16 MB/sec warmup 16 sec latency 88.020 ms 2023-09-30T14:39:43.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:43 smithi156 ceph-mon[103579]: pgmap v330: 97 pgs: 97 active+clean; 19 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2023-09-30T14:39:43.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:43 smithi148 ceph-mon[102355]: pgmap v330: 97 pgs: 97 active+clean; 19 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2023-09-30T14:39:43.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:43 smithi032 ceph-mon[98592]: pgmap v330: 97 pgs: 97 active+clean; 19 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2023-09-30T14:39:43.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 50563 17.02 MB/sec warmup 17 sec latency 6.852 ms 2023-09-30T14:39:44.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 53772 17.10 MB/sec warmup 18 sec latency 8.548 ms 2023-09-30T14:39:45.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:45 smithi156 ceph-mon[103579]: pgmap v331: 97 pgs: 97 active+clean; 20 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.7 MiB/s wr, 229 op/s 2023-09-30T14:39:45.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:45 smithi148 ceph-mon[102355]: pgmap v331: 97 pgs: 97 active+clean; 20 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.7 MiB/s wr, 229 op/s 2023-09-30T14:39:45.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:45 smithi032 ceph-mon[98592]: pgmap v331: 97 pgs: 97 active+clean; 20 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.7 MiB/s wr, 229 op/s 2023-09-30T14:39:45.514 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-30T14:39:45.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.666+0000 7f8952345700 1 -- 172.21.15.32:0/4014394481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c100f40 msgr2=0x7f894c1013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:45.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.666+0000 7f8952345700 1 --2- 172.21.15.32:0/4014394481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c100f40 0x7f894c1013a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8940009990 tx=0x7f8940009ca0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.666+0000 7f8952345700 1 -- 172.21.15.32:0/4014394481 shutdown_connections 2023-09-30T14:39:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.666+0000 7f8952345700 1 --2- 172.21.15.32:0/4014394481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f894c1018e0 0x7f894c108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.666+0000 7f8952345700 1 --2- 172.21.15.32:0/4014394481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f894c0ffd50 0x7f894c100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.666+0000 7f8952345700 1 --2- 172.21.15.32:0/4014394481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c100f40 0x7f894c1013a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.666+0000 7f8952345700 1 -- 172.21.15.32:0/4014394481 >> 172.21.15.32:0/4014394481 conn(0x7f894c0fb2a0 msgr2=0x7f894c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.667+0000 7f8952345700 1 -- 172.21.15.32:0/4014394481 shutdown_connections 2023-09-30T14:39:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.667+0000 7f8952345700 1 -- 172.21.15.32:0/4014394481 wait complete. 2023-09-30T14:39:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.667+0000 7f8952345700 1 Processor -- start 2023-09-30T14:39:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f8952345700 1 -- start start 2023-09-30T14:39:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f8952345700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f894c0ffd50 0x7f894c196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f8952345700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c100f40 0x7f894c197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f8952345700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f894c1018e0 0x7f894c19e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f8952345700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c19e740 con 0x7f894c100f40 2023-09-30T14:39:45.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f8952345700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c19e880 con 0x7f894c1018e0 2023-09-30T14:39:45.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f8952345700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c19ea10 con 0x7f894c0ffd50 2023-09-30T14:39:45.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f894b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c100f40 0x7f894c197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f894b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c100f40 0x7f894c197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35888/0 (socket says 172.21.15.32:35888) 2023-09-30T14:39:45.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f894b7fe700 1 -- 172.21.15.32:0/454293524 learned_addr learned my addr 172.21.15.32:0/454293524 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:45.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f89508e2700 1 --2- 172.21.15.32:0/454293524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f894c1018e0 0x7f894c19e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f894bfff700 1 --2- 172.21.15.32:0/454293524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f894c0ffd50 0x7f894c196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f894b7fe700 1 -- 172.21.15.32:0/454293524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f894c0ffd50 msgr2=0x7f894c196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:45.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f894b7fe700 1 --2- 172.21.15.32:0/454293524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f894c0ffd50 0x7f894c196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f894b7fe700 1 -- 172.21.15.32:0/454293524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f894c1018e0 msgr2=0x7f894c19e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:45.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f894b7fe700 1 --2- 172.21.15.32:0/454293524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f894c1018e0 0x7f894c19e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.668+0000 7f894b7fe700 1 -- 172.21.15.32:0/454293524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89400095f0 con 0x7f894c100f40 2023-09-30T14:39:45.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.669+0000 7f894b7fe700 1 --2- 172.21.15.32:0/454293524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c100f40 0x7f894c197100 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f89400059d0 tx=0x7f89400047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:45.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.669+0000 7f89497fa700 1 -- 172.21.15.32:0/454293524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894001d070 con 0x7f894c100f40 2023-09-30T14:39:45.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.669+0000 7f89497fa700 1 -- 172.21.15.32:0/454293524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f894000fb90 con 0x7f894c100f40 2023-09-30T14:39:45.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.669+0000 7f8952345700 1 -- 172.21.15.32:0/454293524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894c19ec90 con 0x7f894c100f40 2023-09-30T14:39:45.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.669+0000 7f89497fa700 1 -- 172.21.15.32:0/454293524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8940017650 con 0x7f894c100f40 2023-09-30T14:39:45.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.669+0000 7f8952345700 1 -- 172.21.15.32:0/454293524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894c19f0b0 con 0x7f894c100f40 2023-09-30T14:39:45.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.671+0000 7f89497fa700 1 -- 172.21.15.32:0/454293524 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f894000fd00 con 0x7f894c100f40 2023-09-30T14:39:45.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.672+0000 7f89497fa700 1 --2- 172.21.15.32:0/454293524 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89340a0fa0 0x7f89340a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.672+0000 7f894bfff700 1 --2- 172.21.15.32:0/454293524 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89340a0fa0 0x7f89340a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.672+0000 7f89497fa700 1 -- 172.21.15.32:0/454293524 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f89400cddc0 con 0x7f894c100f40 2023-09-30T14:39:45.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.672+0000 7f894bfff700 1 --2- 172.21.15.32:0/454293524 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89340a0fa0 0x7f89340a3460 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f894c100da0 tx=0x7f893c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:45.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f895e52db80 1 Processor -- start 2023-09-30T14:39:45.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f895e52db80 1 -- start start 2023-09-30T14:39:45.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f895e52db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172eca5a0 0x561172eca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f895e52db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172ecaf00 0x561172ecb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f895e52db80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecbda0 0x561172ecc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f895e52db80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172e32460 con 0x561172ecbda0 2023-09-30T14:39:45.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f895e52db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172e23c20 con 0x561172eca5a0 2023-09-30T14:39:45.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f895e52db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172e21540 con 0x561172ecaf00 2023-09-30T14:39:45.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f89508e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecbda0 0x561172ecc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f894bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172eca5a0 0x561172eca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f89508e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecbda0 0x561172ecc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35898/0 (socket says 172.21.15.32:35898) 2023-09-30T14:39:45.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f89508e2700 1 -- 172.21.15.32:0/3162254428 learned_addr learned my addr 172.21.15.32:0/3162254428 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:45.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f894b7fe700 1 --2- 172.21.15.32:0/3162254428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172ecaf00 0x561172ecb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f894bfff700 1 -- 172.21.15.32:0/3162254428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172ecaf00 msgr2=0x561172ecb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:45.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f894bfff700 1 --2- 172.21.15.32:0/3162254428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172ecaf00 0x561172ecb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f894bfff700 1 -- 172.21.15.32:0/3162254428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecbda0 msgr2=0x561172ecc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:45.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.673+0000 7f894bfff700 1 --2- 172.21.15.32:0/3162254428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecbda0 0x561172ecc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.674+0000 7f894bfff700 1 -- 172.21.15.32:0/3162254428 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561172e7f8d0 con 0x561172eca5a0 2023-09-30T14:39:45.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.674+0000 7f89508e2700 1 --2- 172.21.15.32:0/3162254428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecbda0 0x561172ecc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:45.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.674+0000 7f894bfff700 1 --2- 172.21.15.32:0/3162254428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172eca5a0 0x561172eca9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f893c010de0 tx=0x7f893c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68358f0407ac164b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:45.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.674+0000 7f89327fc700 1 -- 172.21.15.32:0/3162254428 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c011bb0 con 0x561172eca5a0 2023-09-30T14:39:45.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.674+0000 7f89327fc700 1 -- 172.21.15.32:0/3162254428 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f893c015070 con 0x561172eca5a0 2023-09-30T14:39:45.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.674+0000 7f89327fc700 1 -- 172.21.15.32:0/3162254428 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c003cc0 con 0x561172eca5a0 2023-09-30T14:39:45.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.675+0000 7f895e52db80 1 -- 172.21.15.32:0/3162254428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172eca5a0 msgr2=0x561172eca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:45.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.675+0000 7f895e52db80 1 --2- 172.21.15.32:0/3162254428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172eca5a0 0x561172eca9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f893c010de0 tx=0x7f893c013000 comp rx=0 tx=0).stop 2023-09-30T14:39:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.675+0000 7f895e52db80 1 -- 172.21.15.32:0/3162254428 shutdown_connections 2023-09-30T14:39:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.675+0000 7f895e52db80 1 --2- 172.21.15.32:0/3162254428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecbda0 0x561172ecc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.675+0000 7f895e52db80 1 --2- 172.21.15.32:0/3162254428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172eca5a0 0x561172eca9c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.675+0000 7f895e52db80 1 --2- 172.21.15.32:0/3162254428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172ecaf00 0x561172ecb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.675+0000 7f895e52db80 1 -- 172.21.15.32:0/3162254428 >> 172.21.15.32:0/3162254428 conn(0x561172e3e260 msgr2=0x561172ec5480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.675+0000 7f895e52db80 1 -- 172.21.15.32:0/3162254428 shutdown_connections 2023-09-30T14:39:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.675+0000 7f895e52db80 1 -- 172.21.15.32:0/3162254428 wait complete. 2023-09-30T14:39:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.675+0000 7f895e52db80 1 -- start start 2023-09-30T14:39:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.675+0000 7f895e52db80 1 Processor -- start 2023-09-30T14:39:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f895e52db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172eca5a0 0x561172ed4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f895e52db80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecaf00 0x561172ed48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f895e52db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172ecbda0 0x561172edb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f895e52db80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172e1ce30 con 0x561172ecaf00 2023-09-30T14:39:45.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f895e52db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172db9710 con 0x561172ecbda0 2023-09-30T14:39:45.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f895e52db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172db93b0 con 0x561172eca5a0 2023-09-30T14:39:45.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f895e52db80 10 client.0 _ll_register_callbacks cb 0x561172c16800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:45.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f894b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecaf00 0x561172ed48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f894b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecaf00 0x561172ed48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35902/0 (socket says 172.21.15.32:35902) 2023-09-30T14:39:45.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f894b7fe700 1 -- 172.21.15.32:0/902801821 learned_addr learned my addr 172.21.15.32:0/902801821 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:45.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f89508e2700 1 --2- 172.21.15.32:0/902801821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172ecbda0 0x561172edb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f894bfff700 1 --2- 172.21.15.32:0/902801821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172eca5a0 0x561172ed4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f894b7fe700 1 -- 172.21.15.32:0/902801821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172eca5a0 msgr2=0x561172ed4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f894b7fe700 1 --2- 172.21.15.32:0/902801821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172eca5a0 0x561172ed4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f894b7fe700 1 -- 172.21.15.32:0/902801821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172ecbda0 msgr2=0x561172edb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f894b7fe700 1 --2- 172.21.15.32:0/902801821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172ecbda0 0x561172edb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f894b7fe700 1 -- 172.21.15.32:0/902801821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8944011330 con 0x561172ecaf00 2023-09-30T14:39:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f894b7fe700 1 --2- 172.21.15.32:0/902801821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecaf00 0x561172ed48d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8940005910 tx=0x7f894009b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8d6f361ecc8247 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f892b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f892b7fe700 1 -- 172.21.15.32:0/902801821 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894009f470 con 0x561172ecaf00 2023-09-30T14:39:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f892b7fe700 1 -- 172.21.15.32:0/902801821 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f89400a3620 con 0x561172ecaf00 2023-09-30T14:39:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f895e52db80 1 -- 172.21.15.32:0/902801821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561172ebd250 con 0x561172ecaf00 2023-09-30T14:39:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f892b7fe700 1 -- 172.21.15.32:0/902801821 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89400b5be0 con 0x561172ecaf00 2023-09-30T14:39:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.676+0000 7f895e52db80 10 client.15582 fetch_fsmap learned FSMap version 24 2023-09-30T14:39:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.677+0000 7f895e52db80 1 -- 172.21.15.32:0/902801821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f893c00be50 con 0x561172ecaf00 2023-09-30T14:39:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.677+0000 7f892b7fe700 1 -- 172.21.15.32:0/902801821 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89400b9080 con 0x561172ecaf00 2023-09-30T14:39:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.677+0000 7f892b7fe700 1 -- 172.21.15.32:0/902801821 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f8940097110 con 0x561172ecaf00 2023-09-30T14:39:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.677+0000 7f895e52db80 10 client.15582 fetch_fsmap finished waiting for FSMap version 24 2023-09-30T14:39:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.677+0000 7f895e52db80 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.677+0000 7f895e52db80 20 client.15582 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.677+0000 7f895e52db80 1 --2- 172.21.15.32:0/902801821 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x561172edfe40 0x561172ee2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.677+0000 7f895e52db80 4 client.15582 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:39:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.677+0000 7f895e52db80 1 -- 172.21.15.32:0/902801821 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561172e15e10 con 0x561172edfe40 2023-09-30T14:39:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.677+0000 7f894bfff700 1 --2- 172.21.15.32:0/902801821 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x561172edfe40 0x561172ee2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.678+0000 7f892b7fe700 10 client.15582 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:45.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.678+0000 7f894bfff700 1 --2- 172.21.15.32:0/902801821 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x561172edfe40 0x561172ee2260 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c9b91fd00ad5b30 server_cookie=f172c12f840af008 in_seq=0 out_seq=0 2023-09-30T14:39:45.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.678+0000 7f892b7fe700 1 -- 172.21.15.32:0/902801821 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561172e15e10 con 0x561172edfe40 2023-09-30T14:39:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.678+0000 7f892b7fe700 10 client.15582 handle_command_reply: tid=0 2023-09-30T14:39:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.678+0000 7f895e52db80 1 client.15582 shutdown 2023-09-30T14:39:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.678+0000 7f895e52db80 1 -- 172.21.15.32:0/902801821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecaf00 msgr2=0x561172ed48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.678+0000 7f895e52db80 1 --2- 172.21.15.32:0/902801821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecaf00 0x561172ed48d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8940005910 tx=0x7f894009b860 comp rx=0 tx=0).stop 2023-09-30T14:39:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.678+0000 7f895e52db80 1 -- 172.21.15.32:0/902801821 shutdown_connections 2023-09-30T14:39:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.678+0000 7f895e52db80 1 --2- 172.21.15.32:0/902801821 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x561172edfe40 0x561172ee2260 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.678+0000 7f895e52db80 1 --2- 172.21.15.32:0/902801821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172ecbda0 0x561172edb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.678+0000 7f892b7fe700 0 client.15582 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.678+0000 7f895e52db80 1 --2- 172.21.15.32:0/902801821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172eca5a0 0x561172ed4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.678+0000 7f895e52db80 1 --2- 172.21.15.32:0/902801821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecaf00 0x561172ed48d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.678+0000 7f895e52db80 1 -- 172.21.15.32:0/902801821 >> 172.21.15.32:0/902801821 conn(0x561172e3e260 msgr2=0x561172ec5290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.678+0000 7f895e52db80 1 -- 172.21.15.32:0/902801821 shutdown_connections 2023-09-30T14:39:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.678+0000 7f895e52db80 1 -- 172.21.15.32:0/902801821 wait complete. 2023-09-30T14:39:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.679+0000 7f895e52db80 20 client.15582 trim_cache size 0 max 16384 2023-09-30T14:39:45.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 57036 17.19 MB/sec warmup 19 sec latency 14.023 ms 2023-09-30T14:39:45.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.696+0000 7f895e52db80 1 Processor -- start 2023-09-30T14:39:45.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.696+0000 7f895e52db80 1 -- start start 2023-09-30T14:39:45.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.696+0000 7f895e52db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172eca5a0 0x561172ed3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.696+0000 7f895e52db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172ecaf00 0x561172edb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.696+0000 7f895e52db80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecbda0 0x561172eced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.696+0000 7f895e52db80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172c9d920 con 0x561172ecbda0 2023-09-30T14:39:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.696+0000 7f895e52db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172ecb3c0 con 0x561172eca5a0 2023-09-30T14:39:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.696+0000 7f895e52db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172edfc10 con 0x561172ecaf00 2023-09-30T14:39:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f89508e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecbda0 0x561172eced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f894bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172eca5a0 0x561172ed3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f89508e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecbda0 0x561172eced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35912/0 (socket says 172.21.15.32:35912) 2023-09-30T14:39:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f894bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172eca5a0 0x561172ed3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47384/0 (socket says 172.21.15.32:47384) 2023-09-30T14:39:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f89508e2700 1 -- 172.21.15.32:0/1228529153 learned_addr learned my addr 172.21.15.32:0/1228529153 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f894b7fe700 1 --2- 172.21.15.32:0/1228529153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172ecaf00 0x561172edb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f894bfff700 1 -- 172.21.15.32:0/1228529153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172ecaf00 msgr2=0x561172edb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f894bfff700 1 --2- 172.21.15.32:0/1228529153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172ecaf00 0x561172edb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f894bfff700 1 -- 172.21.15.32:0/1228529153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecbda0 msgr2=0x561172eced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f894bfff700 1 --2- 172.21.15.32:0/1228529153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecbda0 0x561172eced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f894bfff700 1 -- 172.21.15.32:0/1228529153 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561172c47de0 con 0x561172eca5a0 2023-09-30T14:39:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f894b7fe700 1 --2- 172.21.15.32:0/1228529153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172ecaf00 0x561172edb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f89508e2700 1 --2- 172.21.15.32:0/1228529153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecbda0 0x561172eced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f894bfff700 1 --2- 172.21.15.32:0/1228529153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172eca5a0 0x561172ed3040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x561172ee2b70 tx=0x7f893c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9194c5e4b647826c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f8932ffd700 1 -- 172.21.15.32:0/1228529153 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c039070 con 0x561172eca5a0 2023-09-30T14:39:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f8932ffd700 1 -- 172.21.15.32:0/1228529153 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f893c02d040 con 0x561172eca5a0 2023-09-30T14:39:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.697+0000 7f8932ffd700 1 -- 172.21.15.32:0/1228529153 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c02c050 con 0x561172eca5a0 2023-09-30T14:39:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.698+0000 7f895e52db80 1 -- 172.21.15.32:0/1228529153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172eca5a0 msgr2=0x561172ed3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.698+0000 7f895e52db80 1 --2- 172.21.15.32:0/1228529153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172eca5a0 0x561172ed3040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x561172ee2b70 tx=0x7f893c016000 comp rx=0 tx=0).stop 2023-09-30T14:39:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.698+0000 7f895e52db80 1 -- 172.21.15.32:0/1228529153 shutdown_connections 2023-09-30T14:39:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.698+0000 7f895e52db80 1 --2- 172.21.15.32:0/1228529153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecbda0 0x561172eced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.698+0000 7f895e52db80 1 --2- 172.21.15.32:0/1228529153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172eca5a0 0x561172ed3040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.698+0000 7f895e52db80 1 --2- 172.21.15.32:0/1228529153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172ecaf00 0x561172edb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.698+0000 7f895e52db80 1 -- 172.21.15.32:0/1228529153 >> 172.21.15.32:0/1228529153 conn(0x561172e3e260 msgr2=0x561172ee6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.698+0000 7f895e52db80 1 -- 172.21.15.32:0/1228529153 shutdown_connections 2023-09-30T14:39:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.698+0000 7f895e52db80 1 -- 172.21.15.32:0/1228529153 wait complete. 2023-09-30T14:39:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.698+0000 7f895e52db80 1 -- start start 2023-09-30T14:39:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f895e52db80 1 Processor -- start 2023-09-30T14:39:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f895e52db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172eca5a0 0x561172edb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f895e52db80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecaf00 0x561172ed9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f895e52db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172ecbda0 0x561172eda0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f895e52db80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172eda620 con 0x561172ecaf00 2023-09-30T14:39:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f895e52db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172eda790 con 0x561172ecbda0 2023-09-30T14:39:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f895e52db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172eda950 con 0x561172eca5a0 2023-09-30T14:39:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f895e52db80 10 client.0 _ll_register_callbacks cb 0x561172c16800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f894b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecaf00 0x561172ed9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f89508e2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172ecbda0 0x561172eda0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f894b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecaf00 0x561172ed9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35916/0 (socket says 172.21.15.32:35916) 2023-09-30T14:39:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f894b7fe700 1 -- 172.21.15.32:0/2234986188 learned_addr learned my addr 172.21.15.32:0/2234986188 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f894bfff700 1 --2- 172.21.15.32:0/2234986188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172eca5a0 0x561172edb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f89508e2700 1 -- 172.21.15.32:0/2234986188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172eca5a0 msgr2=0x561172edb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f89508e2700 1 --2- 172.21.15.32:0/2234986188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172eca5a0 0x561172edb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f89508e2700 1 -- 172.21.15.32:0/2234986188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecaf00 msgr2=0x561172ed9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f89508e2700 1 --2- 172.21.15.32:0/2234986188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecaf00 0x561172ed9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f89508e2700 1 -- 172.21.15.32:0/2234986188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893c00be50 con 0x561172ecbda0 2023-09-30T14:39:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.699+0000 7f894bfff700 1 --2- 172.21.15.32:0/2234986188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172eca5a0 0x561172edb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:39:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f894b7fe700 1 --2- 172.21.15.32:0/2234986188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecaf00 0x561172ed9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f89508e2700 1 --2- 172.21.15.32:0/2234986188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172ecbda0 0x561172eda0e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8944009550 tx=0x7f89440077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6ef40f52f69e4d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f892b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:39:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f892b7fe700 1 -- 172.21.15.32:0/2234986188 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894401c070 con 0x561172ecbda0 2023-09-30T14:39:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f895e52db80 1 -- 172.21.15.32:0/2234986188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f893c015070 con 0x561172ecbda0 2023-09-30T14:39:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f892b7fe700 1 -- 172.21.15.32:0/2234986188 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8944004bb0 con 0x561172ecbda0 2023-09-30T14:39:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f892b7fe700 1 -- 172.21.15.32:0/2234986188 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f894400e040 con 0x561172ecbda0 2023-09-30T14:39:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f895e52db80 10 client.25286 fetch_fsmap learned FSMap version 24 2023-09-30T14:39:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f895e52db80 1 -- 172.21.15.32:0/2234986188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f8944011760 con 0x561172ecbda0 2023-09-30T14:39:45.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f892b7fe700 1 -- 172.21.15.32:0/2234986188 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f893c015070 con 0x561172ecbda0 2023-09-30T14:39:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f892b7fe700 1 -- 172.21.15.32:0/2234986188 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f894402a070 con 0x561172ecbda0 2023-09-30T14:39:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f895e52db80 10 client.25286 fetch_fsmap finished waiting for FSMap version 24 2023-09-30T14:39:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f895e52db80 10 client.25286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f895e52db80 20 client.25286 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f895e52db80 1 --2- 172.21.15.32:0/2234986188 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x561172f354c0 0x561172f378e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f895e52db80 4 client.25286 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-09-30T14:39:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.700+0000 7f895e52db80 1 -- 172.21.15.32:0/2234986188 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561172e15b30 con 0x561172f354c0 2023-09-30T14:39:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.701+0000 7f894bfff700 1 --2- 172.21.15.32:0/2234986188 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x561172f354c0 0x561172f378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.701+0000 7f894bfff700 1 --2- 172.21.15.32:0/2234986188 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x561172f354c0 0x561172f378e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec1d602b48e266 server_cookie=342abea2f3e93ced in_seq=0 out_seq=0 2023-09-30T14:39:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.701+0000 7f892b7fe700 10 client.25286 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.701+0000 7f892b7fe700 1 -- 172.21.15.32:0/2234986188 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561172e15b30 con 0x561172f354c0 2023-09-30T14:39:45.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.701+0000 7f892b7fe700 10 client.25286 handle_command_reply: tid=0 2023-09-30T14:39:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.701+0000 7f895e52db80 1 client.25286 shutdown 2023-09-30T14:39:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.701+0000 7f895e52db80 1 -- 172.21.15.32:0/2234986188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172ecbda0 msgr2=0x561172eda0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.701+0000 7f895e52db80 1 --2- 172.21.15.32:0/2234986188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172ecbda0 0x561172eda0e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8944009550 tx=0x7f89440077e0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.701+0000 7f895e52db80 1 -- 172.21.15.32:0/2234986188 shutdown_connections 2023-09-30T14:39:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.701+0000 7f895e52db80 1 --2- 172.21.15.32:0/2234986188 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x561172f354c0 0x561172f378e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.701+0000 7f895e52db80 1 --2- 172.21.15.32:0/2234986188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561172ecbda0 0x561172eda0e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.701+0000 7f892b7fe700 0 client.25286 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.701+0000 7f895e52db80 1 --2- 172.21.15.32:0/2234986188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561172eca5a0 0x561172edb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.701+0000 7f895e52db80 1 --2- 172.21.15.32:0/2234986188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561172ecaf00 0x561172ed9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.701+0000 7f895e52db80 1 -- 172.21.15.32:0/2234986188 >> 172.21.15.32:0/2234986188 conn(0x561172e3e260 msgr2=0x561172ed3040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.701+0000 7f895e52db80 1 -- 172.21.15.32:0/2234986188 shutdown_connections 2023-09-30T14:39:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.702+0000 7f895e52db80 1 -- 172.21.15.32:0/2234986188 wait complete. 2023-09-30T14:39:45.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.702+0000 7f895e52db80 20 client.25286 trim_cache size 0 max 16384 2023-09-30T14:39:45.709 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T14:39:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: "status": "no active scrubs running", 2023-09-30T14:39:45.709 INFO:teuthology.orchestra.run.smithi032.stdout: "scrubs": {} 2023-09-30T14:39:45.709 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T14:39:45.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.702+0000 7f8952345700 1 -- 172.21.15.32:0/454293524 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89340a0fa0 msgr2=0x7f89340a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:45.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.702+0000 7f8952345700 1 --2- 172.21.15.32:0/454293524 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89340a0fa0 0x7f89340a3460 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f894c100da0 tx=0x7f893c00b410 comp rx=0 tx=0).stop 2023-09-30T14:39:45.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.702+0000 7f8952345700 1 -- 172.21.15.32:0/454293524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c100f40 msgr2=0x7f894c197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:45.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.702+0000 7f8952345700 1 --2- 172.21.15.32:0/454293524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c100f40 0x7f894c197100 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f89400059d0 tx=0x7f89400047a0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.702+0000 7f8952345700 1 -- 172.21.15.32:0/454293524 shutdown_connections 2023-09-30T14:39:45.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.702+0000 7f8952345700 1 --2- 172.21.15.32:0/454293524 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89340a0fa0 0x7f89340a3460 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.702+0000 7f8952345700 1 --2- 172.21.15.32:0/454293524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f894c1018e0 0x7f894c19e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.702+0000 7f8952345700 1 --2- 172.21.15.32:0/454293524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f894c0ffd50 0x7f894c196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.702+0000 7f8952345700 1 --2- 172.21.15.32:0/454293524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c100f40 0x7f894c197100 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.702+0000 7f8952345700 1 -- 172.21.15.32:0/454293524 >> 172.21.15.32:0/454293524 conn(0x7f894c0fb2a0 msgr2=0x7f894c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.702+0000 7f8952345700 1 -- 172.21.15.32:0/454293524 shutdown_connections 2023-09-30T14:39:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.702+0000 7f8952345700 1 -- 172.21.15.32:0/454293524 wait complete. 2023-09-30T14:39:45.714 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-30T14:39:45.714 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T14:39:45.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.845+0000 7fb941442700 1 -- 172.21.15.32:0/2021183851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93c102960 msgr2=0x7fb93c109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:45.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.845+0000 7fb941442700 1 --2- 172.21.15.32:0/2021183851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93c102960 0x7fb93c109280 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb93400b0a0 tx=0x7fb93400b3b0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.845+0000 7fb941442700 1 -- 172.21.15.32:0/2021183851 shutdown_connections 2023-09-30T14:39:45.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.845+0000 7fb941442700 1 --2- 172.21.15.32:0/2021183851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93c102960 0x7fb93c109280 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.845+0000 7fb941442700 1 --2- 172.21.15.32:0/2021183851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb93c100e50 0x7fb93c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.845+0000 7fb941442700 1 --2- 172.21.15.32:0/2021183851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb93c101fa0 0x7fb93c102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.845+0000 7fb941442700 1 -- 172.21.15.32:0/2021183851 >> 172.21.15.32:0/2021183851 conn(0x7fb93c0fc3a0 msgr2=0x7fb93c0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:45.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.845+0000 7fb941442700 1 -- 172.21.15.32:0/2021183851 shutdown_connections 2023-09-30T14:39:45.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.845+0000 7fb941442700 1 -- 172.21.15.32:0/2021183851 wait complete. 2023-09-30T14:39:45.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.846+0000 7fb941442700 1 Processor -- start 2023-09-30T14:39:45.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.846+0000 7fb941442700 1 -- start start 2023-09-30T14:39:45.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.846+0000 7fb941442700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb93c100e50 0x7fb93c196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.846+0000 7fb941442700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb93c101fa0 0x7fb93c197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.846+0000 7fb941442700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93c102960 0x7fb93c19e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.846+0000 7fb941442700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c19e750 con 0x7fb93c102960 2023-09-30T14:39:45.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.846+0000 7fb941442700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c19e890 con 0x7fb93c100e50 2023-09-30T14:39:45.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.846+0000 7fb941442700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c19ea20 con 0x7fb93c101fa0 2023-09-30T14:39:45.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.846+0000 7fb93b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93c102960 0x7fb93c19e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.847+0000 7fb93affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb93c100e50 0x7fb93c196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.847+0000 7fb93b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93c102960 0x7fb93c19e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35932/0 (socket says 172.21.15.32:35932) 2023-09-30T14:39:45.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.847+0000 7fb93a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb93c101fa0 0x7fb93c197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.847+0000 7fb93b7fe700 1 -- 172.21.15.32:0/3090320005 learned_addr learned my addr 172.21.15.32:0/3090320005 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:45.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.847+0000 7fb93b7fe700 1 -- 172.21.15.32:0/3090320005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb93c101fa0 msgr2=0x7fb93c197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:45.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.847+0000 7fb93b7fe700 1 --2- 172.21.15.32:0/3090320005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb93c101fa0 0x7fb93c197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.847+0000 7fb93b7fe700 1 -- 172.21.15.32:0/3090320005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb93c100e50 msgr2=0x7fb93c196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:45.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.847+0000 7fb93b7fe700 1 --2- 172.21.15.32:0/3090320005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb93c100e50 0x7fb93c196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:45.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.847+0000 7fb93b7fe700 1 -- 172.21.15.32:0/3090320005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb934009d00 con 0x7fb93c102960 2023-09-30T14:39:45.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.847+0000 7fb93b7fe700 1 --2- 172.21.15.32:0/3090320005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93c102960 0x7fb93c19e180 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb93400b780 tx=0x7fb93400bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:45.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.847+0000 7fb923fff700 1 -- 172.21.15.32:0/3090320005 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93400e040 con 0x7fb93c102960 2023-09-30T14:39:45.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.847+0000 7fb923fff700 1 -- 172.21.15.32:0/3090320005 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb934004c60 con 0x7fb93c102960 2023-09-30T14:39:45.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.847+0000 7fb923fff700 1 -- 172.21.15.32:0/3090320005 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93401b6b0 con 0x7fb93c102960 2023-09-30T14:39:45.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.847+0000 7fb941442700 1 -- 172.21.15.32:0/3090320005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93c19eca0 con 0x7fb93c102960 2023-09-30T14:39:45.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.847+0000 7fb941442700 1 -- 172.21.15.32:0/3090320005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93c19f0d0 con 0x7fb93c102960 2023-09-30T14:39:45.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.848+0000 7fb941442700 1 -- 172.21.15.32:0/3090320005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb93c10c650 con 0x7fb93c102960 2023-09-30T14:39:45.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.849+0000 7fb923fff700 1 -- 172.21.15.32:0/3090320005 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fb93401d480 con 0x7fb93c102960 2023-09-30T14:39:45.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.850+0000 7fb923fff700 1 --2- 172.21.15.32:0/3090320005 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9240a0ff0 0x7fb9240a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:45.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.850+0000 7fb93affd700 1 --2- 172.21.15.32:0/3090320005 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9240a0ff0 0x7fb9240a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:45.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.850+0000 7fb923fff700 1 -- 172.21.15.32:0/3090320005 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9340ce460 con 0x7fb93c102960 2023-09-30T14:39:45.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.850+0000 7fb93affd700 1 --2- 172.21.15.32:0/3090320005 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9240a0ff0 0x7fb9240a34b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009c60 tx=0x7fb92c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:45.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:45.854+0000 7fb923fff700 1 -- 172.21.15.32:0/3090320005 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fb934097d00 con 0x7fb93c102960 2023-09-30T14:39:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.119+0000 7fb941442700 1 -- 172.21.15.32:0/3090320005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb93c19f260 con 0x7fb93c102960 2023-09-30T14:39:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.120+0000 7fb923fff700 1 -- 172.21.15.32:0/3090320005 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11323 (secure 0 0 0) 0x7fb93409b320 con 0x7fb93c102960 2023-09-30T14:39:46.120 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T14:39:46.120 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3505064841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3505064841},{"type":"v1","addr":"172.21.15.32:6839","nonce":3505064841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2262846038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2262846038},{"type":"v1","addr":"172.21.15.32:6841","nonce":2262846038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-30T14:37:11.444993+0000","modified":"2023-09-30T14:39:20.088818+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":37,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6839/1445176203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1445176203},{"type":"v1","addr":"172.21.15.156:6839","nonce":1445176203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/494821686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":494821686},{"type":"v1","addr":"172.21.15.148:6839","nonce":494821686}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T14:39:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.122+0000 7fb941442700 1 -- 172.21.15.32:0/3090320005 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9240a0ff0 msgr2=0x7fb9240a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.122+0000 7fb941442700 1 --2- 172.21.15.32:0/3090320005 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9240a0ff0 0x7fb9240a34b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009c60 tx=0x7fb92c009260 comp rx=0 tx=0).stop 2023-09-30T14:39:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.122+0000 7fb941442700 1 -- 172.21.15.32:0/3090320005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93c102960 msgr2=0x7fb93c19e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.122+0000 7fb941442700 1 --2- 172.21.15.32:0/3090320005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93c102960 0x7fb93c19e180 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb93400b780 tx=0x7fb93400bd00 comp rx=0 tx=0).stop 2023-09-30T14:39:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.122+0000 7fb941442700 1 -- 172.21.15.32:0/3090320005 shutdown_connections 2023-09-30T14:39:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.122+0000 7fb941442700 1 --2- 172.21.15.32:0/3090320005 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9240a0ff0 0x7fb9240a34b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.122+0000 7fb941442700 1 --2- 172.21.15.32:0/3090320005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb93c102960 0x7fb93c19e180 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.122+0000 7fb941442700 1 --2- 172.21.15.32:0/3090320005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb93c100e50 0x7fb93c196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.122+0000 7fb941442700 1 --2- 172.21.15.32:0/3090320005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb93c101fa0 0x7fb93c197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.122+0000 7fb941442700 1 -- 172.21.15.32:0/3090320005 >> 172.21.15.32:0/3090320005 conn(0x7fb93c0fc3a0 msgr2=0x7fb93c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.122+0000 7fb941442700 1 -- 172.21.15.32:0/3090320005 shutdown_connections 2023-09-30T14:39:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.122+0000 7fb941442700 1 -- 172.21.15.32:0/3090320005 wait complete. 2023-09-30T14:39:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2023-09-30T14:39:46.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-30T14:39:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.273+0000 7fc5d1ffa700 1 -- 172.21.15.32:0/187797992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5cc0ffd70 msgr2=0x7fc5cc100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.273+0000 7fc5d1ffa700 1 --2- 172.21.15.32:0/187797992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5cc0ffd70 0x7fc5cc100190 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0099c0 tx=0x7fc5bc009cd0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.273+0000 7fc5d1ffa700 1 -- 172.21.15.32:0/187797992 shutdown_connections 2023-09-30T14:39:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.273+0000 7fc5d1ffa700 1 --2- 172.21.15.32:0/187797992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc101900 0x7fc5cc1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.273+0000 7fc5d1ffa700 1 --2- 172.21.15.32:0/187797992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5cc0ffd70 0x7fc5cc100190 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.273+0000 7fc5d1ffa700 1 --2- 172.21.15.32:0/187797992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5cc100f60 0x7fc5cc1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.273+0000 7fc5d1ffa700 1 -- 172.21.15.32:0/187797992 >> 172.21.15.32:0/187797992 conn(0x7fc5cc0fb2e0 msgr2=0x7fc5cc0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.273+0000 7fc5d1ffa700 1 -- 172.21.15.32:0/187797992 shutdown_connections 2023-09-30T14:39:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.273+0000 7fc5d1ffa700 1 -- 172.21.15.32:0/187797992 wait complete. 2023-09-30T14:39:46.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.274+0000 7fc5d1ffa700 1 Processor -- start 2023-09-30T14:39:46.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.274+0000 7fc5d1ffa700 1 -- start start 2023-09-30T14:39:46.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.274+0000 7fc5d1ffa700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5cc0ffd70 0x7fc5cc195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.274+0000 7fc5d1ffa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc100f60 0x7fc5cc196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.274+0000 7fc5d1ffa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5cc101900 0x7fc5cc19d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.274+0000 7fc5d1ffa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc19d640 con 0x7fc5cc101900 2023-09-30T14:39:46.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.274+0000 7fc5d1ffa700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc19d780 con 0x7fc5cc100f60 2023-09-30T14:39:46.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.274+0000 7fc5d1ffa700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc19d910 con 0x7fc5cc0ffd70 2023-09-30T14:39:46.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5cbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5cc101900 0x7fc5cc19d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5caffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc100f60 0x7fc5cc196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5cbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5cc101900 0x7fc5cc19d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35946/0 (socket says 172.21.15.32:35946) 2023-09-30T14:39:46.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5cbfff700 1 -- 172.21.15.32:0/923141126 learned_addr learned my addr 172.21.15.32:0/923141126 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:46.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5caffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc100f60 0x7fc5cc196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47420/0 (socket says 172.21.15.32:47420) 2023-09-30T14:39:46.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5cb7fe700 1 --2- 172.21.15.32:0/923141126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5cc0ffd70 0x7fc5cc195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5caffd700 1 -- 172.21.15.32:0/923141126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5cc0ffd70 msgr2=0x7fc5cc195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5caffd700 1 --2- 172.21.15.32:0/923141126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5cc0ffd70 0x7fc5cc195ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5caffd700 1 -- 172.21.15.32:0/923141126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5cc101900 msgr2=0x7fc5cc19d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5caffd700 1 --2- 172.21.15.32:0/923141126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5cc101900 0x7fc5cc19d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5caffd700 1 -- 172.21.15.32:0/923141126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc0095f0 con 0x7fc5cc100f60 2023-09-30T14:39:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5cbfff700 1 --2- 172.21.15.32:0/923141126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5cc101900 0x7fc5cc19d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5cb7fe700 1 --2- 172.21.15.32:0/923141126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5cc0ffd70 0x7fc5cc195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:39:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5caffd700 1 --2- 172.21.15.32:0/923141126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc100f60 0x7fc5cc196000 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000e9f0 tx=0x7fc5c000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5c8ff9700 1 -- 172.21.15.32:0/923141126 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c000eee0 con 0x7fc5cc100f60 2023-09-30T14:39:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5d1ffa700 1 -- 172.21.15.32:0/923141126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5cc19dbf0 con 0x7fc5cc100f60 2023-09-30T14:39:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5c8ff9700 1 -- 172.21.15.32:0/923141126 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5c000cbd0 con 0x7fc5cc100f60 2023-09-30T14:39:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5c8ff9700 1 -- 172.21.15.32:0/923141126 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0018680 con 0x7fc5cc100f60 2023-09-30T14:39:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.275+0000 7fc5d1ffa700 1 -- 172.21.15.32:0/923141126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5cc19e100 con 0x7fc5cc100f60 2023-09-30T14:39:46.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.278+0000 7fc5c8ff9700 1 -- 172.21.15.32:0/923141126 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fc5c000cd40 con 0x7fc5cc100f60 2023-09-30T14:39:46.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.278+0000 7fc5c8ff9700 1 --2- 172.21.15.32:0/923141126 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5b40a0fa0 0x7fc5b40a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.279+0000 7fc5cb7fe700 1 --2- 172.21.15.32:0/923141126 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5b40a0fa0 0x7fc5b40a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.279+0000 7fc5c8ff9700 1 -- 172.21.15.32:0/923141126 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc5c0014070 con 0x7fc5cc100f60 2023-09-30T14:39:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.279+0000 7fc5cb7fe700 1 --2- 172.21.15.32:0/923141126 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5b40a0fa0 0x7fc5b40a3460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009990 tx=0x7fc5bc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.279+0000 7fc5de1e2b80 1 Processor -- start 2023-09-30T14:39:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.279+0000 7fc5de1e2b80 1 -- start start 2023-09-30T14:39:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5de1e2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a157b0 0x55f089a15bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5de1e2b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a16110 0x55f089a169d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5de1e2b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16fb0 0x55f089a17430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5de1e2b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08997b460 con 0x55f089a16fb0 2023-09-30T14:39:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5de1e2b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08996cc20 con 0x55f089a16110 2023-09-30T14:39:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5de1e2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08996a540 con 0x55f089a157b0 2023-09-30T14:39:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5cbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16fb0 0x55f089a17430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5caffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a16110 0x55f089a169d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5cb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a157b0 0x55f089a15bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5cbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16fb0 0x55f089a17430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35954/0 (socket says 172.21.15.32:35954) 2023-09-30T14:39:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5cbfff700 1 -- 172.21.15.32:0/1858117398 learned_addr learned my addr 172.21.15.32:0/1858117398 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5cbfff700 1 -- 172.21.15.32:0/1858117398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a157b0 msgr2=0x55f089a15bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5cbfff700 1 --2- 172.21.15.32:0/1858117398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a157b0 0x55f089a15bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5cbfff700 1 -- 172.21.15.32:0/1858117398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a16110 msgr2=0x55f089a169d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5cbfff700 1 --2- 172.21.15.32:0/1858117398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a16110 0x55f089a169d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5cbfff700 1 -- 172.21.15.32:0/1858117398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0899c88d0 con 0x55f089a16fb0 2023-09-30T14:39:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5caffd700 1 --2- 172.21.15.32:0/1858117398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a16110 0x55f089a169d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5cb7fe700 1 --2- 172.21.15.32:0/1858117398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a157b0 0x55f089a15bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5cbfff700 1 --2- 172.21.15.32:0/1858117398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16fb0 0x55f089a17430 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009500 tx=0x7fc5c4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8aa78009cda826 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5b1ffb700 1 -- 172.21.15.32:0/1858117398 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c400e900 con 0x55f089a16fb0 2023-09-30T14:39:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5b1ffb700 1 -- 172.21.15.32:0/1858117398 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5c400f040 con 0x55f089a16fb0 2023-09-30T14:39:46.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.280+0000 7fc5b1ffb700 1 -- 172.21.15.32:0/1858117398 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4017ce0 con 0x55f089a16fb0 2023-09-30T14:39:46.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.281+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/1858117398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16fb0 msgr2=0x55f089a17430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.281+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/1858117398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16fb0 0x55f089a17430 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009500 tx=0x7fc5c4009920 comp rx=0 tx=0).stop 2023-09-30T14:39:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.281+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/1858117398 shutdown_connections 2023-09-30T14:39:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.281+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/1858117398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16fb0 0x55f089a17430 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.281+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/1858117398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a157b0 0x55f089a15bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.281+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/1858117398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a16110 0x55f089a169d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.281+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/1858117398 >> 172.21.15.32:0/1858117398 conn(0x55f089987260 msgr2=0x55f089a0e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.281+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/1858117398 shutdown_connections 2023-09-30T14:39:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.281+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/1858117398 wait complete. 2023-09-30T14:39:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5de1e2b80 1 -- start start 2023-09-30T14:39:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5de1e2b80 1 Processor -- start 2023-09-30T14:39:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5de1e2b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a157b0 0x55f089a1f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5de1e2b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16110 0x55f089a1fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5de1e2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a16fb0 0x55f089a26af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5de1e2b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f089965e30 con 0x55f089a16110 2023-09-30T14:39:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5de1e2b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f089902710 con 0x55f089a157b0 2023-09-30T14:39:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5de1e2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0899023b0 con 0x55f089a16fb0 2023-09-30T14:39:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5de1e2b80 10 client.0 _ll_register_callbacks cb 0x55f08975f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5caffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16110 0x55f089a1fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5caffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16110 0x55f089a1fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35958/0 (socket says 172.21.15.32:35958) 2023-09-30T14:39:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5caffd700 1 -- 172.21.15.32:0/3036456362 learned_addr learned my addr 172.21.15.32:0/3036456362 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5cb7fe700 1 --2- 172.21.15.32:0/3036456362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a157b0 0x55f089a1f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5cbfff700 1 --2- 172.21.15.32:0/3036456362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a16fb0 0x55f089a26af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5caffd700 1 -- 172.21.15.32:0/3036456362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a16fb0 msgr2=0x55f089a26af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5caffd700 1 --2- 172.21.15.32:0/3036456362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a16fb0 0x55f089a26af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5caffd700 1 -- 172.21.15.32:0/3036456362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a157b0 msgr2=0x55f089a1f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5caffd700 1 --2- 172.21.15.32:0/3036456362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a157b0 0x55f089a1f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.282+0000 7fc5caffd700 1 -- 172.21.15.32:0/3036456362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5cc19e100 con 0x55f089a16110 2023-09-30T14:39:46.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.283+0000 7fc5caffd700 1 --2- 172.21.15.32:0/3036456362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16110 0x55f089a1fab0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00ca390 tx=0x7fc5c009a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0882427f1e70a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.283+0000 7fc5aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:46.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.283+0000 7fc5aaffd700 1 -- 172.21.15.32:0/3036456362 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c00b2420 con 0x55f089a16110 2023-09-30T14:39:46.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.283+0000 7fc5aaffd700 1 -- 172.21.15.32:0/3036456362 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc5c009dea0 con 0x55f089a16110 2023-09-30T14:39:46.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.283+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3036456362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f089a06250 con 0x55f089a16110 2023-09-30T14:39:46.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.283+0000 7fc5aaffd700 1 -- 172.21.15.32:0/3036456362 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5c00a20e0 con 0x55f089a16110 2023-09-30T14:39:46.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.283+0000 7fc5de1e2b80 10 client.15606 fetch_fsmap learned FSMap version 24 2023-09-30T14:39:46.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.283+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3036456362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fc5c40091e0 con 0x55f089a16110 2023-09-30T14:39:46.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.283+0000 7fc5aaffd700 1 -- 172.21.15.32:0/3036456362 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f089a06250 con 0x55f089a16110 2023-09-30T14:39:46.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.283+0000 7fc5aaffd700 1 -- 172.21.15.32:0/3036456362 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc5c0098e20 con 0x55f089a16110 2023-09-30T14:39:46.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.283+0000 7fc5de1e2b80 10 client.15606 fetch_fsmap finished waiting for FSMap version 24 2023-09-30T14:39:46.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.283+0000 7fc5de1e2b80 10 client.15606 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-30T14:39:46.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.283+0000 7fc5de1e2b80 20 client.15606 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:46.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.283+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/3036456362 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x55f089a2b350 0x55f089a2d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.283+0000 7fc5de1e2b80 4 client.15606 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:39:46.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.283+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3036456362 --> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f08995ee10 con 0x55f089a2b350 2023-09-30T14:39:46.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.284+0000 7fc5cb7fe700 1 --2- 172.21.15.32:0/3036456362 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x55f089a2b350 0x55f089a2d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.284+0000 7fc5cb7fe700 1 --2- 172.21.15.32:0/3036456362 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x55f089a2b350 0x55f089a2d770 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ddb7888781ee4195 server_cookie=31409c35cc9d7f6e in_seq=0 out_seq=0 2023-09-30T14:39:46.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.284+0000 7fc5aaffd700 10 client.15606 ms_handle_connect on v2:172.21.15.148:6836/3666119034 2023-09-30T14:39:46.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.285+0000 7fc5aaffd700 1 -- 172.21.15.32:0/3036456362 <== mds.2 v2:172.21.15.148:6836/3666119034 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f08995ee10 con 0x55f089a2b350 2023-09-30T14:39:46.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.285+0000 7fc5aaffd700 10 client.15606 handle_command_reply: tid=0 2023-09-30T14:39:46.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.285+0000 7fc5de1e2b80 1 client.15606 shutdown 2023-09-30T14:39:46.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.285+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3036456362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16110 msgr2=0x55f089a1fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.285+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/3036456362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16110 0x55f089a1fab0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00ca390 tx=0x7fc5c009a290 comp rx=0 tx=0).stop 2023-09-30T14:39:46.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.285+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3036456362 shutdown_connections 2023-09-30T14:39:46.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.285+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/3036456362 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x55f089a2b350 0x55f089a2d770 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.285+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/3036456362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a16fb0 0x55f089a26af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.285+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/3036456362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a157b0 0x55f089a1f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.285+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/3036456362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16110 0x55f089a1fab0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.285+0000 7fc5aaffd700 0 client.15606 ms_handle_reset on v2:172.21.15.148:6836/3666119034 2023-09-30T14:39:46.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.285+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3036456362 >> 172.21.15.32:0/3036456362 conn(0x55f089987260 msgr2=0x55f089a0e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.285+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3036456362 shutdown_connections 2023-09-30T14:39:46.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.285+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3036456362 wait complete. 2023-09-30T14:39:46.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.285+0000 7fc5de1e2b80 20 client.15606 trim_cache size 0 max 16384 2023-09-30T14:39:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.299+0000 7fc5de1e2b80 1 Processor -- start 2023-09-30T14:39:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.299+0000 7fc5de1e2b80 1 -- start start 2023-09-30T14:39:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.299+0000 7fc5de1e2b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a157b0 0x55f089a272c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.299+0000 7fc5de1e2b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16110 0x55f089a26ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.299+0000 7fc5de1e2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a16fb0 0x55f089a1d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.299+0000 7fc5de1e2b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0897e6920 con 0x55f089a16110 2023-09-30T14:39:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.299+0000 7fc5de1e2b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f089a28810 con 0x55f089a157b0 2023-09-30T14:39:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.299+0000 7fc5de1e2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f089a164f0 con 0x55f089a16fb0 2023-09-30T14:39:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.299+0000 7fc5caffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16110 0x55f089a26ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.299+0000 7fc5cbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a16fb0 0x55f089a1d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.299+0000 7fc5caffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16110 0x55f089a26ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35968/0 (socket says 172.21.15.32:35968) 2023-09-30T14:39:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.299+0000 7fc5caffd700 1 -- 172.21.15.32:0/3164000374 learned_addr learned my addr 172.21.15.32:0/3164000374 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.299+0000 7fc5cb7fe700 1 --2- 172.21.15.32:0/3164000374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a157b0 0x55f089a272c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.300+0000 7fc5cbfff700 1 -- 172.21.15.32:0/3164000374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a157b0 msgr2=0x55f089a272c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.300+0000 7fc5cbfff700 1 --2- 172.21.15.32:0/3164000374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a157b0 0x55f089a272c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.300+0000 7fc5cbfff700 1 -- 172.21.15.32:0/3164000374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16110 msgr2=0x55f089a26ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.300+0000 7fc5cbfff700 1 --2- 172.21.15.32:0/3164000374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16110 0x55f089a26ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.300+0000 7fc5cbfff700 1 -- 172.21.15.32:0/3164000374 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f089790de0 con 0x55f089a16fb0 2023-09-30T14:39:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.300+0000 7fc5caffd700 1 --2- 172.21.15.32:0/3164000374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16110 0x55f089a26ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.300+0000 7fc5cbfff700 1 --2- 172.21.15.32:0/3164000374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a16fb0 0x55f089a1d320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4007ba0 tx=0x7fc5c4007500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58eb32d38f004d71 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.300+0000 7fc5b27fc700 1 -- 172.21.15.32:0/3164000374 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c400e900 con 0x55f089a16fb0 2023-09-30T14:39:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.300+0000 7fc5b27fc700 1 -- 172.21.15.32:0/3164000374 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5c40050c0 con 0x55f089a16fb0 2023-09-30T14:39:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.300+0000 7fc5b27fc700 1 -- 172.21.15.32:0/3164000374 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40176c0 con 0x55f089a16fb0 2023-09-30T14:39:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.301+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3164000374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a16fb0 msgr2=0x55f089a1d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.301+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/3164000374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a16fb0 0x55f089a1d320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4007ba0 tx=0x7fc5c4007500 comp rx=0 tx=0).stop 2023-09-30T14:39:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.301+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3164000374 shutdown_connections 2023-09-30T14:39:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.301+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/3164000374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a16fb0 0x55f089a1d320 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.301+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/3164000374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a157b0 0x55f089a272c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.301+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/3164000374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a16110 0x55f089a26ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.301+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3164000374 >> 172.21.15.32:0/3164000374 conn(0x55f089987260 msgr2=0x55f089a1e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.301+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3164000374 shutdown_connections 2023-09-30T14:39:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.301+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3164000374 wait complete. 2023-09-30T14:39:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.301+0000 7fc5de1e2b80 1 -- start start 2023-09-30T14:39:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.301+0000 7fc5de1e2b80 1 Processor -- start 2023-09-30T14:39:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.301+0000 7fc5de1e2b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a157b0 0x55f089a26430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5de1e2b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a16110 0x55f089a26970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5de1e2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a16fb0 0x55f089a22ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5de1e2b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f089a235c0 con 0x55f089a157b0 2023-09-30T14:39:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5de1e2b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f089a23730 con 0x55f089a16110 2023-09-30T14:39:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5de1e2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f089a238f0 con 0x55f089a16fb0 2023-09-30T14:39:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5de1e2b80 10 client.0 _ll_register_callbacks cb 0x7fc5c00996b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5cb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a157b0 0x55f089a26430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5cb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a157b0 0x55f089a26430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35970/0 (socket says 172.21.15.32:35970) 2023-09-30T14:39:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5cb7fe700 1 -- 172.21.15.32:0/3623654886 learned_addr learned my addr 172.21.15.32:0/3623654886 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5caffd700 1 --2- 172.21.15.32:0/3623654886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a16110 0x55f089a26970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5cbfff700 1 --2- 172.21.15.32:0/3623654886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a16fb0 0x55f089a22ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5cb7fe700 1 -- 172.21.15.32:0/3623654886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a16fb0 msgr2=0x55f089a22ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5cb7fe700 1 --2- 172.21.15.32:0/3623654886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a16fb0 0x55f089a22ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5cb7fe700 1 -- 172.21.15.32:0/3623654886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a16110 msgr2=0x55f089a26970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5cb7fe700 1 --2- 172.21.15.32:0/3623654886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a16110 0x55f089a26970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5cb7fe700 1 -- 172.21.15.32:0/3623654886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c40091e0 con 0x55f089a157b0 2023-09-30T14:39:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5cb7fe700 1 --2- 172.21.15.32:0/3623654886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a157b0 0x55f089a26430 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc100dc0 tx=0x7fc5bc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a6e58895b69c8c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5aaffd700 1 -- 172.21.15.32:0/3623654886 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc02a040 con 0x55f089a157b0 2023-09-30T14:39:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5aaffd700 1 -- 172.21.15.32:0/3623654886 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc5bc00f460 con 0x55f089a157b0 2023-09-30T14:39:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.302+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3623654886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5c400f040 con 0x55f089a157b0 2023-09-30T14:39:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.303+0000 7fc5aaffd700 1 -- 172.21.15.32:0/3623654886 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5bc01d040 con 0x55f089a157b0 2023-09-30T14:39:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.303+0000 7fc5de1e2b80 10 client.15612 fetch_fsmap learned FSMap version 24 2023-09-30T14:39:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.303+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3623654886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f0899c88d0 con 0x55f089a157b0 2023-09-30T14:39:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.303+0000 7fc5aaffd700 1 -- 172.21.15.32:0/3623654886 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5c400f040 con 0x55f089a157b0 2023-09-30T14:39:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.303+0000 7fc5aaffd700 1 -- 172.21.15.32:0/3623654886 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc5bc023670 con 0x55f089a157b0 2023-09-30T14:39:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.303+0000 7fc5de1e2b80 10 client.15612 fetch_fsmap finished waiting for FSMap version 24 2023-09-30T14:39:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.303+0000 7fc5de1e2b80 10 client.15612 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-30T14:39:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.303+0000 7fc5de1e2b80 20 client.15612 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.303+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/3623654886 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x55f089a80810 0x55f089a82c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.303+0000 7fc5de1e2b80 4 client.15612 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-30T14:39:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.303+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3623654886 --> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f08995eb30 con 0x55f089a80810 2023-09-30T14:39:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.303+0000 7fc5caffd700 1 --2- 172.21.15.32:0/3623654886 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x55f089a80810 0x55f089a82c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.304+0000 7fc5aaffd700 10 client.15612 ms_handle_connect on v2:172.21.15.148:6836/3666119034 2023-09-30T14:39:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.304+0000 7fc5caffd700 1 --2- 172.21.15.32:0/3623654886 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x55f089a80810 0x55f089a82c30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f1a742947179486 server_cookie=594180a751c64912 in_seq=0 out_seq=0 2023-09-30T14:39:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.304+0000 7fc5aaffd700 1 -- 172.21.15.32:0/3623654886 <== mds.2 v2:172.21.15.148:6836/3666119034 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+195 (crc 0 0 0) 0x55f08995eb30 con 0x55f089a80810 2023-09-30T14:39:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.304+0000 7fc5aaffd700 10 client.15612 handle_command_reply: tid=0 2023-09-30T14:39:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.304+0000 7fc5de1e2b80 1 client.15612 shutdown 2023-09-30T14:39:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.304+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3623654886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a157b0 msgr2=0x55f089a26430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.304+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/3623654886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a157b0 0x55f089a26430 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc100dc0 tx=0x7fc5bc01e000 comp rx=0 tx=0).stop 2023-09-30T14:39:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.304+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3623654886 shutdown_connections 2023-09-30T14:39:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.304+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/3623654886 >> [v2:172.21.15.148:6836/3666119034,v1:172.21.15.148:6837/3666119034] conn(0x55f089a80810 0x55f089a82c30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/3623654886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f089a16fb0 0x55f089a22ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/3623654886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f089a157b0 0x55f089a26430 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5aaffd700 0 client.15612 ms_handle_reset on v2:172.21.15.148:6836/3666119034 2023-09-30T14:39:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5de1e2b80 1 --2- 172.21.15.32:0/3623654886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f089a16110 0x55f089a26970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3623654886 >> 172.21.15.32:0/3623654886 conn(0x55f089987260 msgr2=0x55f089a1d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3623654886 shutdown_connections 2023-09-30T14:39:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5de1e2b80 1 -- 172.21.15.32:0/3623654886 wait complete. 2023-09-30T14:39:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5de1e2b80 20 client.15612 trim_cache size 0 max 16384 2023-09-30T14:39:46.314 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-09-30T14:39:46.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T14:39:46.315 INFO:teuthology.orchestra.run.smithi032.stdout: "damage_type": "backtrace", 2023-09-30T14:39:46.315 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 2313572090, 2023-09-30T14:39:46.315 INFO:teuthology.orchestra.run.smithi032.stdout: "ino": 1099511627781, 2023-09-30T14:39:46.315 INFO:teuthology.orchestra.run.smithi032.stdout: "path": "/volumes/_nogroup/sv_1/b25fcf16-a734-410f-849e-f8fb3a417cdf/client.0" 2023-09-30T14:39:46.315 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T14:39:46.315 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-09-30T14:39:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5d1ffa700 1 -- 172.21.15.32:0/923141126 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5b40a0fa0 msgr2=0x7fc5b40a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5d1ffa700 1 --2- 172.21.15.32:0/923141126 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5b40a0fa0 0x7fc5b40a3460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009990 tx=0x7fc5bc01a040 comp rx=0 tx=0).stop 2023-09-30T14:39:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5d1ffa700 1 -- 172.21.15.32:0/923141126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc100f60 msgr2=0x7fc5cc196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5d1ffa700 1 --2- 172.21.15.32:0/923141126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc100f60 0x7fc5cc196000 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000e9f0 tx=0x7fc5c000ed00 comp rx=0 tx=0).stop 2023-09-30T14:39:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5d1ffa700 1 -- 172.21.15.32:0/923141126 shutdown_connections 2023-09-30T14:39:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5d1ffa700 1 --2- 172.21.15.32:0/923141126 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc5b40a0fa0 0x7fc5b40a3460 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5d1ffa700 1 --2- 172.21.15.32:0/923141126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5cc101900 0x7fc5cc19d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5d1ffa700 1 --2- 172.21.15.32:0/923141126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5cc0ffd70 0x7fc5cc195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5d1ffa700 1 --2- 172.21.15.32:0/923141126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5cc100f60 0x7fc5cc196000 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5d1ffa700 1 -- 172.21.15.32:0/923141126 >> 172.21.15.32:0/923141126 conn(0x7fc5cc0fb2e0 msgr2=0x7fc5cc0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5d1ffa700 1 -- 172.21.15.32:0/923141126 shutdown_connections 2023-09-30T14:39:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.305+0000 7fc5d1ffa700 1 -- 172.21.15.32:0/923141126 wait complete. 2023-09-30T14:39:46.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-30T14:39:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.449+0000 7fca6b8ff700 1 -- 172.21.15.32:0/90624480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca641041e0 msgr2=0x7fca641065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.449+0000 7fca6b8ff700 1 --2- 172.21.15.32:0/90624480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca641041e0 0x7fca641065d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fca54009990 tx=0x7fca54009ca0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.450+0000 7fca6b8ff700 1 -- 172.21.15.32:0/90624480 shutdown_connections 2023-09-30T14:39:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.450+0000 7fca6b8ff700 1 --2- 172.21.15.32:0/90624480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca64106b10 0x7fca64108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.450+0000 7fca6b8ff700 1 --2- 172.21.15.32:0/90624480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca641018b0 0x7fca64103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.450+0000 7fca6b8ff700 1 --2- 172.21.15.32:0/90624480 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca641041e0 0x7fca641065d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.450+0000 7fca6b8ff700 1 -- 172.21.15.32:0/90624480 >> 172.21.15.32:0/90624480 conn(0x7fca640fb2a0 msgr2=0x7fca640fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.450+0000 7fca6b8ff700 1 -- 172.21.15.32:0/90624480 shutdown_connections 2023-09-30T14:39:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.450+0000 7fca6b8ff700 1 -- 172.21.15.32:0/90624480 wait complete. 2023-09-30T14:39:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.450+0000 7fca6b8ff700 1 Processor -- start 2023-09-30T14:39:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca6b8ff700 1 -- start start 2023-09-30T14:39:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca6b8ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca641018b0 0x7fca64101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca6b8ff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca641041e0 0x7fca640ff860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca6b8ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca64106b10 0x7fca640ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca6b8ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca64100540 con 0x7fca64106b10 2023-09-30T14:39:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca6b8ff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca641006b0 con 0x7fca641041e0 2023-09-30T14:39:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca6b8ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca641a0b90 con 0x7fca641018b0 2023-09-30T14:39:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca69e9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca64106b10 0x7fca640ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca69e9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca64106b10 0x7fca640ffda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35988/0 (socket says 172.21.15.32:35988) 2023-09-30T14:39:46.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca6969b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca641018b0 0x7fca64101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca69e9c700 1 -- 172.21.15.32:0/1987582654 learned_addr learned my addr 172.21.15.32:0/1987582654 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca68e9a700 1 --2- 172.21.15.32:0/1987582654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca641041e0 0x7fca640ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca68e9a700 1 -- 172.21.15.32:0/1987582654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca641018b0 msgr2=0x7fca64101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca68e9a700 1 --2- 172.21.15.32:0/1987582654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca641018b0 0x7fca64101220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca68e9a700 1 -- 172.21.15.32:0/1987582654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca64106b10 msgr2=0x7fca640ffda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca68e9a700 1 --2- 172.21.15.32:0/1987582654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca64106b10 0x7fca640ffda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca68e9a700 1 -- 172.21.15.32:0/1987582654 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca540095f0 con 0x7fca641041e0 2023-09-30T14:39:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca69e9c700 1 --2- 172.21.15.32:0/1987582654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca64106b10 0x7fca640ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.451+0000 7fca6969b700 1 --2- 172.21.15.32:0/1987582654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca641018b0 0x7fca64101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.452+0000 7fca68e9a700 1 --2- 172.21.15.32:0/1987582654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca641041e0 0x7fca640ff860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fca5400b680 tx=0x7fca54004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.452+0000 7fca5a7fc700 1 -- 172.21.15.32:0/1987582654 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5401d070 con 0x7fca641041e0 2023-09-30T14:39:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.452+0000 7fca6b8ff700 1 -- 172.21.15.32:0/1987582654 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca641a0e10 con 0x7fca641041e0 2023-09-30T14:39:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.452+0000 7fca5a7fc700 1 -- 172.21.15.32:0/1987582654 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca540054f0 con 0x7fca641041e0 2023-09-30T14:39:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.452+0000 7fca5a7fc700 1 -- 172.21.15.32:0/1987582654 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca54004b90 con 0x7fca641041e0 2023-09-30T14:39:46.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.452+0000 7fca6b8ff700 1 -- 172.21.15.32:0/1987582654 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca641a11d0 con 0x7fca641041e0 2023-09-30T14:39:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.455+0000 7fca5a7fc700 1 -- 172.21.15.32:0/1987582654 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fca5400fb90 con 0x7fca641041e0 2023-09-30T14:39:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.455+0000 7fca5a7fc700 1 --2- 172.21.15.32:0/1987582654 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fca500a0fa0 0x7fca500a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.455+0000 7fca6969b700 1 --2- 172.21.15.32:0/1987582654 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fca500a0fa0 0x7fca500a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.455+0000 7fca5a7fc700 1 -- 172.21.15.32:0/1987582654 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fca540cd840 con 0x7fca641041e0 2023-09-30T14:39:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.456+0000 7fca6969b700 1 --2- 172.21.15.32:0/1987582654 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fca500a0fa0 0x7fca500a3460 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fca60011490 tx=0x7fca60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.456+0000 7fca77ae7b80 1 Processor -- start 2023-09-30T14:39:46.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.456+0000 7fca77ae7b80 1 -- start start 2023-09-30T14:39:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.456+0000 7fca77ae7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa05a0 0x559fd5aa09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.456+0000 7fca77ae7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5aa17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.456+0000 7fca77ae7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa1da0 0x559fd5aa2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.456+0000 7fca77ae7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd5a08460 con 0x559fd5aa1da0 2023-09-30T14:39:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.456+0000 7fca77ae7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd59f9c20 con 0x559fd5aa05a0 2023-09-30T14:39:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.456+0000 7fca77ae7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd59f7540 con 0x559fd5aa0f00 2023-09-30T14:39:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.456+0000 7fca6969b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa05a0 0x559fd5aa09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.456+0000 7fca68e9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5aa17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.456+0000 7fca69e9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa1da0 0x559fd5aa2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.456+0000 7fca68e9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5aa17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:50376/0 (socket says 172.21.15.32:50376) 2023-09-30T14:39:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.456+0000 7fca68e9a700 1 -- 172.21.15.32:0/1821097739 learned_addr learned my addr 172.21.15.32:0/1821097739 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.457+0000 7fca6969b700 1 -- 172.21.15.32:0/1821097739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 msgr2=0x559fd5aa17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.457+0000 7fca6969b700 1 --2- 172.21.15.32:0/1821097739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5aa17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.457+0000 7fca6969b700 1 -- 172.21.15.32:0/1821097739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa1da0 msgr2=0x559fd5aa2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.457+0000 7fca6969b700 1 --2- 172.21.15.32:0/1821097739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa1da0 0x559fd5aa2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.457+0000 7fca6969b700 1 -- 172.21.15.32:0/1821097739 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fd5a558d0 con 0x559fd5aa05a0 2023-09-30T14:39:46.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.457+0000 7fca69e9c700 1 --2- 172.21.15.32:0/1821097739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa1da0 0x559fd5aa2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.457+0000 7fca6969b700 1 --2- 172.21.15.32:0/1821097739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa05a0 0x559fd5aa09c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fca600042f0 tx=0x7fca60014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3950478025cc2abc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.457+0000 7fca4f7fe700 1 -- 172.21.15.32:0/1821097739 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca60003d10 con 0x559fd5aa05a0 2023-09-30T14:39:46.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.457+0000 7fca4f7fe700 1 -- 172.21.15.32:0/1821097739 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca60003e70 con 0x559fd5aa05a0 2023-09-30T14:39:46.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.457+0000 7fca4f7fe700 1 -- 172.21.15.32:0/1821097739 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6001e7b0 con 0x559fd5aa05a0 2023-09-30T14:39:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.458+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1821097739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa05a0 msgr2=0x559fd5aa09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.458+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/1821097739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa05a0 0x559fd5aa09c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fca600042f0 tx=0x7fca60014000 comp rx=0 tx=0).stop 2023-09-30T14:39:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.458+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1821097739 shutdown_connections 2023-09-30T14:39:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.458+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/1821097739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa1da0 0x559fd5aa2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.458+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/1821097739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa05a0 0x559fd5aa09c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.458+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/1821097739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5aa17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.458+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1821097739 >> 172.21.15.32:0/1821097739 conn(0x559fd5a14260 msgr2=0x559fd5a9b480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.458+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1821097739 shutdown_connections 2023-09-30T14:39:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.458+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1821097739 wait complete. 2023-09-30T14:39:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.458+0000 7fca77ae7b80 1 -- start start 2023-09-30T14:39:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.458+0000 7fca77ae7b80 1 Processor -- start 2023-09-30T14:39:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca77ae7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa05a0 0x559fd5aaa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca77ae7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5aaa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca77ae7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa1da0 0x559fd5ab1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca77ae7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd59f2e30 con 0x559fd5aa05a0 2023-09-30T14:39:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca77ae7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd598f710 con 0x559fd5aa1da0 2023-09-30T14:39:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca77ae7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd598f3b0 con 0x559fd5aa0f00 2023-09-30T14:39:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca77ae7b80 10 client.0 _ll_register_callbacks cb 0x559fd57ec800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca6969b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa05a0 0x559fd5aaa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca68e9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5aaa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca6969b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa05a0 0x559fd5aaa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35998/0 (socket says 172.21.15.32:35998) 2023-09-30T14:39:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca6969b700 1 -- 172.21.15.32:0/1152134153 learned_addr learned my addr 172.21.15.32:0/1152134153 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca69e9c700 1 --2- 172.21.15.32:0/1152134153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa1da0 0x559fd5ab1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca6969b700 1 -- 172.21.15.32:0/1152134153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 msgr2=0x559fd5aaa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca6969b700 1 --2- 172.21.15.32:0/1152134153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5aaa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca6969b700 1 -- 172.21.15.32:0/1152134153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa1da0 msgr2=0x559fd5ab1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca6969b700 1 --2- 172.21.15.32:0/1152134153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa1da0 0x559fd5ab1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca6969b700 1 -- 172.21.15.32:0/1152134153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5c007500 con 0x559fd5aa05a0 2023-09-30T14:39:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca68e9a700 1 --2- 172.21.15.32:0/1152134153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5aaa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.459+0000 7fca69e9c700 1 --2- 172.21.15.32:0/1152134153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa1da0 0x559fd5ab1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.460+0000 7fca6969b700 1 --2- 172.21.15.32:0/1152134153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa05a0 0x559fd5aaa3c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fca60011be0 tx=0x7fca600266a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c731c5838c78bf53 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.460+0000 7fca4cff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.460+0000 7fca4cff9700 1 -- 172.21.15.32:0/1152134153 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6002b070 con 0x559fd5aa05a0 2023-09-30T14:39:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.460+0000 7fca4cff9700 1 -- 172.21.15.32:0/1152134153 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fca60026a20 con 0x559fd5aa05a0 2023-09-30T14:39:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.460+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1152134153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fd5a93250 con 0x559fd5aa05a0 2023-09-30T14:39:46.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.460+0000 7fca4cff9700 1 -- 172.21.15.32:0/1152134153 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca60026be0 con 0x559fd5aa05a0 2023-09-30T14:39:46.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.460+0000 7fca77ae7b80 10 client.15624 fetch_fsmap learned FSMap version 24 2023-09-30T14:39:46.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.460+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1152134153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fca60017070 con 0x559fd5aa05a0 2023-09-30T14:39:46.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.460+0000 7fca4cff9700 1 -- 172.21.15.32:0/1152134153 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559fd5a93250 con 0x559fd5aa05a0 2023-09-30T14:39:46.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.460+0000 7fca4cff9700 1 -- 172.21.15.32:0/1152134153 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fca60024020 con 0x559fd5aa05a0 2023-09-30T14:39:46.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.460+0000 7fca77ae7b80 10 client.15624 fetch_fsmap finished waiting for FSMap version 24 2023-09-30T14:39:46.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.460+0000 7fca77ae7b80 10 client.15624 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-30T14:39:46.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.460+0000 7fca77ae7b80 20 client.15624 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:46.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.460+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/1152134153 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x559fd5ab6030 0x559fd5ab8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.460+0000 7fca77ae7b80 4 client.15624 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:39:46.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.460+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1152134153 --> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fd59ebe10 con 0x559fd5ab6030 2023-09-30T14:39:46.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.461+0000 7fca68e9a700 1 --2- 172.21.15.32:0/1152134153 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x559fd5ab6030 0x559fd5ab8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.461+0000 7fca68e9a700 1 --2- 172.21.15.32:0/1152134153 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x559fd5ab6030 0x559fd5ab8450 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd5e0b46dcf766f5 server_cookie=c55eb1fe82780a02 in_seq=0 out_seq=0 2023-09-30T14:39:46.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.461+0000 7fca4cff9700 10 client.15624 ms_handle_connect on v2:172.21.15.156:6838/1445176203 2023-09-30T14:39:46.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.462+0000 7fca4cff9700 1 -- 172.21.15.32:0/1152134153 <== mds.1 v2:172.21.15.156:6838/1445176203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559fd59ebe10 con 0x559fd5ab6030 2023-09-30T14:39:46.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.462+0000 7fca4cff9700 10 client.15624 handle_command_reply: tid=0 2023-09-30T14:39:46.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.462+0000 7fca77ae7b80 1 client.15624 shutdown 2023-09-30T14:39:46.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.462+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1152134153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa05a0 msgr2=0x559fd5aaa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.462+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/1152134153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa05a0 0x559fd5aaa3c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fca60011be0 tx=0x7fca600266a0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.462+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1152134153 shutdown_connections 2023-09-30T14:39:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.462+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/1152134153 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x559fd5ab6030 0x559fd5ab8450 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.462+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/1152134153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa1da0 0x559fd5ab1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.462+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/1152134153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa05a0 0x559fd5aaa3c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.462+0000 7fca4cff9700 0 client.15624 ms_handle_reset on v2:172.21.15.156:6838/1445176203 2023-09-30T14:39:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.462+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/1152134153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5aaa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.462+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1152134153 >> 172.21.15.32:0/1152134153 conn(0x559fd5a14260 msgr2=0x559fd5a9b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.462+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1152134153 shutdown_connections 2023-09-30T14:39:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.462+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1152134153 wait complete. 2023-09-30T14:39:46.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.462+0000 7fca77ae7b80 20 client.15624 trim_cache size 0 max 16384 2023-09-30T14:39:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.475+0000 7fca77ae7b80 1 Processor -- start 2023-09-30T14:39:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.475+0000 7fca77ae7b80 1 -- start start 2023-09-30T14:39:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca77ae7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa05a0 0x559fd5aa90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca77ae7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5ab1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca77ae7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa1da0 0x559fd5ab2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca77ae7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd5aa9dd0 con 0x559fd5aa1da0 2023-09-30T14:39:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca77ae7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd5873920 con 0x559fd5aa05a0 2023-09-30T14:39:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca77ae7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd5aa15b0 con 0x559fd5aa0f00 2023-09-30T14:39:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca69e9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa1da0 0x559fd5ab2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca6969b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa05a0 0x559fd5aa90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca68e9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5ab1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca6969b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa05a0 0x559fd5aa90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47482/0 (socket says 172.21.15.32:47482) 2023-09-30T14:39:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca69e9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa1da0 0x559fd5ab2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36010/0 (socket says 172.21.15.32:36010) 2023-09-30T14:39:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca6969b700 1 -- 172.21.15.32:0/480430297 learned_addr learned my addr 172.21.15.32:0/480430297 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:46.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca6969b700 1 -- 172.21.15.32:0/480430297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 msgr2=0x559fd5ab1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca6969b700 1 --2- 172.21.15.32:0/480430297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5ab1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca6969b700 1 -- 172.21.15.32:0/480430297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa1da0 msgr2=0x559fd5ab2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca6969b700 1 --2- 172.21.15.32:0/480430297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa1da0 0x559fd5ab2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca6969b700 1 -- 172.21.15.32:0/480430297 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fd581dde0 con 0x559fd5aa05a0 2023-09-30T14:39:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca69e9c700 1 --2- 172.21.15.32:0/480430297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa1da0 0x559fd5ab2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca68e9a700 1 --2- 172.21.15.32:0/480430297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5ab1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca6969b700 1 --2- 172.21.15.32:0/480430297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa05a0 0x559fd5aa90a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fca600097c0 tx=0x7fca60009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38a8293e6db7a4fb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.476+0000 7fca4ffff700 1 -- 172.21.15.32:0/480430297 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6002b070 con 0x559fd5aa05a0 2023-09-30T14:39:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.477+0000 7fca4ffff700 1 -- 172.21.15.32:0/480430297 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca6001e530 con 0x559fd5aa05a0 2023-09-30T14:39:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.477+0000 7fca4ffff700 1 -- 172.21.15.32:0/480430297 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6001ead0 con 0x559fd5aa05a0 2023-09-30T14:39:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.477+0000 7fca77ae7b80 1 -- 172.21.15.32:0/480430297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa05a0 msgr2=0x559fd5aa90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.477+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/480430297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa05a0 0x559fd5aa90a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fca600097c0 tx=0x7fca60009370 comp rx=0 tx=0).stop 2023-09-30T14:39:46.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.477+0000 7fca77ae7b80 1 -- 172.21.15.32:0/480430297 shutdown_connections 2023-09-30T14:39:46.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.477+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/480430297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa1da0 0x559fd5ab2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.477+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/480430297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa05a0 0x559fd5aa90a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.477+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/480430297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5ab1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.477+0000 7fca77ae7b80 1 -- 172.21.15.32:0/480430297 >> 172.21.15.32:0/480430297 conn(0x559fd5a14260 msgr2=0x559fd5a99f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.477+0000 7fca77ae7b80 1 -- 172.21.15.32:0/480430297 shutdown_connections 2023-09-30T14:39:46.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.477+0000 7fca77ae7b80 1 -- 172.21.15.32:0/480430297 wait complete. 2023-09-30T14:39:46.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.477+0000 7fca77ae7b80 1 -- start start 2023-09-30T14:39:46.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca77ae7b80 1 Processor -- start 2023-09-30T14:39:46.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca77ae7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa05a0 0x559fd5aaf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca77ae7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5aafa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca77ae7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa1da0 0x559fd5ab0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca77ae7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd5ab0f90 con 0x559fd5aa05a0 2023-09-30T14:39:46.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca77ae7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd5ab1100 con 0x559fd5aa1da0 2023-09-30T14:39:46.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca77ae7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd5ab12c0 con 0x559fd5aa0f00 2023-09-30T14:39:46.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca77ae7b80 10 client.0 _ll_register_callbacks cb 0x7fca5c009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:46.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca6969b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa05a0 0x559fd5aaf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca6969b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa05a0 0x559fd5aaf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36012/0 (socket says 172.21.15.32:36012) 2023-09-30T14:39:46.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca6969b700 1 -- 172.21.15.32:0/1146909430 learned_addr learned my addr 172.21.15.32:0/1146909430 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:46.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca69e9c700 1 --2- 172.21.15.32:0/1146909430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa1da0 0x559fd5ab0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca68e9a700 1 --2- 172.21.15.32:0/1146909430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5aafa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca6969b700 1 -- 172.21.15.32:0/1146909430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 msgr2=0x559fd5aafa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca6969b700 1 --2- 172.21.15.32:0/1146909430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5aafa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca6969b700 1 -- 172.21.15.32:0/1146909430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa1da0 msgr2=0x559fd5ab0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.478+0000 7fca6969b700 1 --2- 172.21.15.32:0/1146909430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa1da0 0x559fd5ab0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca6969b700 1 -- 172.21.15.32:0/1146909430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca641a11d0 con 0x559fd5aa05a0 2023-09-30T14:39:46.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca69e9c700 1 --2- 172.21.15.32:0/1146909430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa1da0 0x559fd5ab0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca6969b700 1 --2- 172.21.15.32:0/1146909430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa05a0 0x559fd5aaf500 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fca60003e40 tx=0x7fca60004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac651e5593c3433d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca4cff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:46.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca4cff9700 1 -- 172.21.15.32:0/1146909430 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6002b070 con 0x559fd5aa05a0 2023-09-30T14:39:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1146909430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca60003e70 con 0x559fd5aa05a0 2023-09-30T14:39:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca4cff9700 1 -- 172.21.15.32:0/1146909430 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fca6001ea30 con 0x559fd5aa05a0 2023-09-30T14:39:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca4cff9700 1 -- 172.21.15.32:0/1146909430 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca6001ec30 con 0x559fd5aa05a0 2023-09-30T14:39:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca77ae7b80 10 client.15630 fetch_fsmap learned FSMap version 24 2023-09-30T14:39:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1146909430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fca5c007a70 con 0x559fd5aa05a0 2023-09-30T14:39:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca4cff9700 1 -- 172.21.15.32:0/1146909430 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca60003e70 con 0x559fd5aa05a0 2023-09-30T14:39:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca4cff9700 1 -- 172.21.15.32:0/1146909430 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fca60041070 con 0x559fd5aa05a0 2023-09-30T14:39:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca77ae7b80 10 client.15630 fetch_fsmap finished waiting for FSMap version 24 2023-09-30T14:39:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca77ae7b80 10 client.15630 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-30T14:39:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca77ae7b80 20 client.15630 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/1146909430 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x559fd5b0a850 0x559fd5b0cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca77ae7b80 4 client.15630 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-30T14:39:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.479+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1146909430 --> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559fd59ebb30 con 0x559fd5b0a850 2023-09-30T14:39:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.480+0000 7fca68e9a700 1 --2- 172.21.15.32:0/1146909430 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x559fd5b0a850 0x559fd5b0cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.480+0000 7fca4cff9700 10 client.15630 ms_handle_connect on v2:172.21.15.156:6838/1445176203 2023-09-30T14:39:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.480+0000 7fca68e9a700 1 --2- 172.21.15.32:0/1146909430 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x559fd5b0a850 0x559fd5b0cc40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1d4061e666e23ea server_cookie=9281fc5eddb9f249 in_seq=0 out_seq=0 2023-09-30T14:39:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.480+0000 7fca4cff9700 1 -- 172.21.15.32:0/1146909430 <== mds.1 v2:172.21.15.156:6838/1445176203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559fd59ebb30 con 0x559fd5b0a850 2023-09-30T14:39:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.480+0000 7fca4cff9700 10 client.15630 handle_command_reply: tid=0 2023-09-30T14:39:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.480+0000 7fca77ae7b80 1 client.15630 shutdown 2023-09-30T14:39:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.480+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1146909430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa05a0 msgr2=0x559fd5aaf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.480+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/1146909430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa05a0 0x559fd5aaf500 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fca60003e40 tx=0x7fca60004a60 comp rx=0 tx=0).stop 2023-09-30T14:39:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1146909430 shutdown_connections 2023-09-30T14:39:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/1146909430 >> [v2:172.21.15.156:6838/1445176203,v1:172.21.15.156:6839/1445176203] conn(0x559fd5b0a850 0x559fd5b0cc40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/1146909430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fd5aa1da0 0x559fd5ab0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca4cff9700 0 client.15630 ms_handle_reset on v2:172.21.15.156:6838/1445176203 2023-09-30T14:39:46.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/1146909430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559fd5aa05a0 0x559fd5aaf500 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca77ae7b80 1 --2- 172.21.15.32:0/1146909430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fd5aa0f00 0x559fd5aafa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1146909430 >> 172.21.15.32:0/1146909430 conn(0x559fd5a14260 msgr2=0x559fd5aa90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1146909430 shutdown_connections 2023-09-30T14:39:46.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca77ae7b80 1 -- 172.21.15.32:0/1146909430 wait complete. 2023-09-30T14:39:46.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca77ae7b80 20 client.15630 trim_cache size 0 max 16384 2023-09-30T14:39:46.488 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2023-09-30T14:39:46.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca6b8ff700 1 -- 172.21.15.32:0/1987582654 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fca500a0fa0 msgr2=0x7fca500a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca6b8ff700 1 --2- 172.21.15.32:0/1987582654 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fca500a0fa0 0x7fca500a3460 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fca60011490 tx=0x7fca60009260 comp rx=0 tx=0).stop 2023-09-30T14:39:46.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca6b8ff700 1 -- 172.21.15.32:0/1987582654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca641041e0 msgr2=0x7fca640ff860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca6b8ff700 1 --2- 172.21.15.32:0/1987582654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca641041e0 0x7fca640ff860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fca5400b680 tx=0x7fca54004730 comp rx=0 tx=0).stop 2023-09-30T14:39:46.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca6b8ff700 1 -- 172.21.15.32:0/1987582654 shutdown_connections 2023-09-30T14:39:46.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca6b8ff700 1 --2- 172.21.15.32:0/1987582654 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fca500a0fa0 0x7fca500a3460 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca6b8ff700 1 --2- 172.21.15.32:0/1987582654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca64106b10 0x7fca640ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca6b8ff700 1 --2- 172.21.15.32:0/1987582654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca641018b0 0x7fca64101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca6b8ff700 1 --2- 172.21.15.32:0/1987582654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca641041e0 0x7fca640ff860 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca6b8ff700 1 -- 172.21.15.32:0/1987582654 >> 172.21.15.32:0/1987582654 conn(0x7fca640fb2a0 msgr2=0x7fca64102520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca6b8ff700 1 -- 172.21.15.32:0/1987582654 shutdown_connections 2023-09-30T14:39:46.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.481+0000 7fca6b8ff700 1 -- 172.21.15.32:0/1987582654 wait complete. 2023-09-30T14:39:46.494 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-30T14:39:46.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.627+0000 7fb53789e700 1 -- 172.21.15.32:0/3138974514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5300ffd30 msgr2=0x7fb530100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.627+0000 7fb53789e700 1 --2- 172.21.15.32:0/3138974514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5300ffd30 0x7fb530100150 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0099c0 tx=0x7fb52c009cd0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.627+0000 7fb53789e700 1 -- 172.21.15.32:0/3138974514 shutdown_connections 2023-09-30T14:39:46.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.627+0000 7fb53789e700 1 --2- 172.21.15.32:0/3138974514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5301018c0 0x7fb530108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.627+0000 7fb53789e700 1 --2- 172.21.15.32:0/3138974514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5300ffd30 0x7fb530100150 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.627+0000 7fb53789e700 1 --2- 172.21.15.32:0/3138974514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb530100f20 0x7fb530101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.627+0000 7fb53789e700 1 -- 172.21.15.32:0/3138974514 >> 172.21.15.32:0/3138974514 conn(0x7fb5300fb2a0 msgr2=0x7fb5300fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.627+0000 7fb53789e700 1 -- 172.21.15.32:0/3138974514 shutdown_connections 2023-09-30T14:39:46.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.627+0000 7fb53789e700 1 -- 172.21.15.32:0/3138974514 wait complete. 2023-09-30T14:39:46.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.628+0000 7fb53789e700 1 Processor -- start 2023-09-30T14:39:46.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.628+0000 7fb53789e700 1 -- start start 2023-09-30T14:39:46.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.628+0000 7fb53789e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5300ffd30 0x7fb530068b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.628+0000 7fb53789e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb530100f20 0x7fb53019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.628+0000 7fb53789e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5301018c0 0x7fb53019df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.628+0000 7fb53789e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53019e6e0 con 0x7fb5301018c0 2023-09-30T14:39:46.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.628+0000 7fb53789e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53019e850 con 0x7fb5300ffd30 2023-09-30T14:39:46.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.628+0000 7fb53789e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53019ea10 con 0x7fb530100f20 2023-09-30T14:39:46.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.628+0000 7fb535e3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5301018c0 0x7fb53019df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.628+0000 7fb535e3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5301018c0 0x7fb53019df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36030/0 (socket says 172.21.15.32:36030) 2023-09-30T14:39:46.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.628+0000 7fb535e3b700 1 -- 172.21.15.32:0/3701683241 learned_addr learned my addr 172.21.15.32:0/3701683241 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:46.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.628+0000 7fb53563a700 1 --2- 172.21.15.32:0/3701683241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5300ffd30 0x7fb530068b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.628+0000 7fb534e39700 1 --2- 172.21.15.32:0/3701683241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb530100f20 0x7fb53019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.629+0000 7fb535e3b700 1 -- 172.21.15.32:0/3701683241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb530100f20 msgr2=0x7fb53019b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.629+0000 7fb535e3b700 1 --2- 172.21.15.32:0/3701683241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb530100f20 0x7fb53019b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.629+0000 7fb535e3b700 1 -- 172.21.15.32:0/3701683241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5300ffd30 msgr2=0x7fb530068b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.629+0000 7fb535e3b700 1 --2- 172.21.15.32:0/3701683241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5300ffd30 0x7fb530068b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.629+0000 7fb535e3b700 1 -- 172.21.15.32:0/3701683241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c0095f0 con 0x7fb5301018c0 2023-09-30T14:39:46.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.629+0000 7fb53563a700 1 --2- 172.21.15.32:0/3701683241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5300ffd30 0x7fb530068b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.629+0000 7fb535e3b700 1 --2- 172.21.15.32:0/3701683241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5301018c0 0x7fb53019df40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb52800ef30 tx=0x7fb52800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.629+0000 7fb5267fc700 1 -- 172.21.15.32:0/3701683241 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528018070 con 0x7fb5301018c0 2023-09-30T14:39:46.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.629+0000 7fb5267fc700 1 -- 172.21.15.32:0/3701683241 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb52800f040 con 0x7fb5301018c0 2023-09-30T14:39:46.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.629+0000 7fb53789e700 1 -- 172.21.15.32:0/3701683241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53019ec90 con 0x7fb5301018c0 2023-09-30T14:39:46.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.629+0000 7fb5267fc700 1 -- 172.21.15.32:0/3701683241 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52800cd00 con 0x7fb5301018c0 2023-09-30T14:39:46.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.629+0000 7fb53789e700 1 -- 172.21.15.32:0/3701683241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53019f150 con 0x7fb5301018c0 2023-09-30T14:39:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.632+0000 7fb5267fc700 1 -- 172.21.15.32:0/3701683241 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fb528003a00 con 0x7fb5301018c0 2023-09-30T14:39:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.632+0000 7fb5267fc700 1 --2- 172.21.15.32:0/3701683241 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb51c0a0fa0 0x7fb51c0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.632+0000 7fb5267fc700 1 -- 172.21.15.32:0/3701683241 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb5280cbec0 con 0x7fb5301018c0 2023-09-30T14:39:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.632+0000 7fb53563a700 1 --2- 172.21.15.32:0/3701683241 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb51c0a0fa0 0x7fb51c0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.633+0000 7fb53563a700 1 --2- 172.21.15.32:0/3701683241 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb51c0a0fa0 0x7fb51c0a3460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009990 tx=0x7fb52c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.633+0000 7fb543a86b80 1 Processor -- start 2023-09-30T14:39:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.633+0000 7fb543a86b80 1 -- start start 2023-09-30T14:39:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb543a86b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d35a0 0x55c3240d39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb543a86b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240d47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb543a86b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d4da0 0x55c3240d5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb543a86b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c32403b460 con 0x55c3240d35a0 2023-09-30T14:39:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb543a86b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c32402cc20 con 0x55c3240d3f00 2023-09-30T14:39:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb543a86b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c32402a540 con 0x55c3240d4da0 2023-09-30T14:39:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb53563a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d35a0 0x55c3240d39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb53563a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d35a0 0x55c3240d39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36034/0 (socket says 172.21.15.32:36034) 2023-09-30T14:39:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb53563a700 1 -- 172.21.15.32:0/30510800 learned_addr learned my addr 172.21.15.32:0/30510800 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb534e39700 1 --2- 172.21.15.32:0/30510800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240d47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb535e3b700 1 --2- 172.21.15.32:0/30510800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d4da0 0x55c3240d5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb53563a700 1 -- 172.21.15.32:0/30510800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d4da0 msgr2=0x55c3240d5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb53563a700 1 --2- 172.21.15.32:0/30510800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d4da0 0x55c3240d5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb53563a700 1 -- 172.21.15.32:0/30510800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 msgr2=0x55c3240d47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb53563a700 1 --2- 172.21.15.32:0/30510800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240d47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb53563a700 1 -- 172.21.15.32:0/30510800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3240888d0 con 0x55c3240d35a0 2023-09-30T14:39:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb534e39700 1 --2- 172.21.15.32:0/30510800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240d47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb53563a700 1 --2- 172.21.15.32:0/30510800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d35a0 0x55c3240d39c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb52c00b580 tx=0x7fb52c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ed4d092f079e9c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.634+0000 7fb51b7fe700 1 -- 172.21.15.32:0/30510800 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c02a070 con 0x55c3240d35a0 2023-09-30T14:39:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.635+0000 7fb51b7fe700 1 -- 172.21.15.32:0/30510800 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb52c01e040 con 0x55c3240d35a0 2023-09-30T14:39:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.635+0000 7fb51b7fe700 1 -- 172.21.15.32:0/30510800 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c00f760 con 0x55c3240d35a0 2023-09-30T14:39:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.635+0000 7fb543a86b80 1 -- 172.21.15.32:0/30510800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d35a0 msgr2=0x55c3240d39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.635+0000 7fb543a86b80 1 --2- 172.21.15.32:0/30510800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d35a0 0x55c3240d39c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb52c00b580 tx=0x7fb52c01f000 comp rx=0 tx=0).stop 2023-09-30T14:39:46.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.635+0000 7fb543a86b80 1 -- 172.21.15.32:0/30510800 shutdown_connections 2023-09-30T14:39:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.635+0000 7fb543a86b80 1 --2- 172.21.15.32:0/30510800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d4da0 0x55c3240d5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.635+0000 7fb543a86b80 1 --2- 172.21.15.32:0/30510800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d35a0 0x55c3240d39c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.635+0000 7fb543a86b80 1 --2- 172.21.15.32:0/30510800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240d47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.635+0000 7fb543a86b80 1 -- 172.21.15.32:0/30510800 >> 172.21.15.32:0/30510800 conn(0x55c324047260 msgr2=0x55c3240ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.635+0000 7fb543a86b80 1 -- 172.21.15.32:0/30510800 shutdown_connections 2023-09-30T14:39:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.635+0000 7fb543a86b80 1 -- 172.21.15.32:0/30510800 wait complete. 2023-09-30T14:39:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.636+0000 7fb543a86b80 1 -- start start 2023-09-30T14:39:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.636+0000 7fb543a86b80 1 Processor -- start 2023-09-30T14:39:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.636+0000 7fb543a86b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d35a0 0x55c3240dd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.636+0000 7fb543a86b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d3f00 0x55c3240dd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.636+0000 7fb543a86b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d4da0 0x55c3240e4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.636+0000 7fb543a86b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c324025e30 con 0x55c3240d3f00 2023-09-30T14:39:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.636+0000 7fb543a86b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c323fc2710 con 0x55c3240d35a0 2023-09-30T14:39:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.636+0000 7fb543a86b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c323fc23b0 con 0x55c3240d4da0 2023-09-30T14:39:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.636+0000 7fb543a86b80 10 client.0 _ll_register_callbacks cb 0x55c323e1f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.636+0000 7fb534e39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d3f00 0x55c3240dd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.636+0000 7fb534e39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d3f00 0x55c3240dd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36042/0 (socket says 172.21.15.32:36042) 2023-09-30T14:39:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.636+0000 7fb53563a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d35a0 0x55c3240dd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.636+0000 7fb534e39700 1 -- 172.21.15.32:0/137396692 learned_addr learned my addr 172.21.15.32:0/137396692 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.637+0000 7fb534e39700 1 -- 172.21.15.32:0/137396692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d4da0 msgr2=0x55c3240e4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.637+0000 7fb535e3b700 1 --2- 172.21.15.32:0/137396692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d4da0 0x55c3240e4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.637+0000 7fb534e39700 1 --2- 172.21.15.32:0/137396692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d4da0 0x55c3240e4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.637+0000 7fb534e39700 1 -- 172.21.15.32:0/137396692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d35a0 msgr2=0x55c3240dd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.637+0000 7fb534e39700 1 --2- 172.21.15.32:0/137396692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d35a0 0x55c3240dd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.637+0000 7fb534e39700 1 -- 172.21.15.32:0/137396692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5200098e0 con 0x55c3240d3f00 2023-09-30T14:39:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.637+0000 7fb53563a700 1 --2- 172.21.15.32:0/137396692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d35a0 0x55c3240dd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.637+0000 7fb535e3b700 1 --2- 172.21.15.32:0/137396692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d4da0 0x55c3240e4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:39:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.637+0000 7fb534e39700 1 --2- 172.21.15.32:0/137396692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d3f00 0x55c3240dd810 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb520012980 tx=0x7fb520012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a956362690a41584 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.637+0000 7fb518ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.637+0000 7fb518ff9700 1 -- 172.21.15.32:0/137396692 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520017070 con 0x55c3240d3f00 2023-09-30T14:39:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.637+0000 7fb518ff9700 1 -- 172.21.15.32:0/137396692 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb52000f7b0 con 0x55c3240d3f00 2023-09-30T14:39:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.637+0000 7fb543a86b80 1 -- 172.21.15.32:0/137396692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3240c6250 con 0x55c3240d3f00 2023-09-30T14:39:46.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.638+0000 7fb518ff9700 1 -- 172.21.15.32:0/137396692 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5200122e0 con 0x55c3240d3f00 2023-09-30T14:39:46.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.638+0000 7fb543a86b80 10 client.15642 fetch_fsmap learned FSMap version 24 2023-09-30T14:39:46.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.638+0000 7fb543a86b80 1 -- 172.21.15.32:0/137396692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fb530108ac0 con 0x55c3240d3f00 2023-09-30T14:39:46.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.638+0000 7fb518ff9700 1 -- 172.21.15.32:0/137396692 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c3240c6250 con 0x55c3240d3f00 2023-09-30T14:39:46.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.638+0000 7fb518ff9700 1 -- 172.21.15.32:0/137396692 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb520026b90 con 0x55c3240d3f00 2023-09-30T14:39:46.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.638+0000 7fb543a86b80 10 client.15642 fetch_fsmap finished waiting for FSMap version 24 2023-09-30T14:39:46.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.638+0000 7fb543a86b80 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:46.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.638+0000 7fb543a86b80 20 client.15642 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:46.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.638+0000 7fb543a86b80 1 --2- 172.21.15.32:0/137396692 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c3240e9100 0x55c3240eb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.638+0000 7fb543a86b80 4 client.15642 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:39:46.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.638+0000 7fb543a86b80 1 -- 172.21.15.32:0/137396692 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c32401ee10 con 0x55c3240e9100 2023-09-30T14:39:46.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.638+0000 7fb53563a700 1 --2- 172.21.15.32:0/137396692 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c3240e9100 0x55c3240eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.639+0000 7fb53563a700 1 --2- 172.21.15.32:0/137396692 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c3240e9100 0x55c3240eb520 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0e51d37c0308ac5 server_cookie=def8b386b7c56f3a in_seq=0 out_seq=0 2023-09-30T14:39:46.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.639+0000 7fb518ff9700 10 client.15642 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:46.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.639+0000 7fb518ff9700 1 -- 172.21.15.32:0/137396692 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c32401ee10 con 0x55c3240e9100 2023-09-30T14:39:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.639+0000 7fb518ff9700 10 client.15642 handle_command_reply: tid=0 2023-09-30T14:39:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.639+0000 7fb543a86b80 1 client.15642 shutdown 2023-09-30T14:39:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.640+0000 7fb543a86b80 1 -- 172.21.15.32:0/137396692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d3f00 msgr2=0x55c3240dd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.640+0000 7fb543a86b80 1 --2- 172.21.15.32:0/137396692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d3f00 0x55c3240dd810 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb520012980 tx=0x7fb520012c90 comp rx=0 tx=0).stop 2023-09-30T14:39:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.640+0000 7fb543a86b80 1 -- 172.21.15.32:0/137396692 shutdown_connections 2023-09-30T14:39:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.640+0000 7fb543a86b80 1 --2- 172.21.15.32:0/137396692 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c3240e9100 0x55c3240eb520 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.640+0000 7fb543a86b80 1 --2- 172.21.15.32:0/137396692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d4da0 0x55c3240e4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.640+0000 7fb543a86b80 1 --2- 172.21.15.32:0/137396692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d35a0 0x55c3240dd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.640+0000 7fb518ff9700 0 client.15642 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.640+0000 7fb543a86b80 1 --2- 172.21.15.32:0/137396692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d3f00 0x55c3240dd810 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.640+0000 7fb543a86b80 1 -- 172.21.15.32:0/137396692 >> 172.21.15.32:0/137396692 conn(0x55c324047260 msgr2=0x55c3240ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.640+0000 7fb543a86b80 1 -- 172.21.15.32:0/137396692 shutdown_connections 2023-09-30T14:39:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.640+0000 7fb543a86b80 1 -- 172.21.15.32:0/137396692 wait complete. 2023-09-30T14:39:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.640+0000 7fb543a86b80 20 client.15642 trim_cache size 0 max 16384 2023-09-30T14:39:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb543a86b80 1 Processor -- start 2023-09-30T14:39:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb543a86b80 1 -- start start 2023-09-30T14:39:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb543a86b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d35a0 0x55c3240d3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb543a86b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240e4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb543a86b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d4da0 0x55c3240ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb543a86b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c323ea6920 con 0x55c3240d35a0 2023-09-30T14:39:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb543a86b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3240cd440 con 0x55c3240d3f00 2023-09-30T14:39:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb543a86b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3240d31c0 con 0x55c3240d4da0 2023-09-30T14:39:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb53563a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d35a0 0x55c3240d3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb534e39700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240e4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb53563a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d35a0 0x55c3240d3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36048/0 (socket says 172.21.15.32:36048) 2023-09-30T14:39:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb53563a700 1 -- 172.21.15.32:0/2086465959 learned_addr learned my addr 172.21.15.32:0/2086465959 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb534e39700 1 --2- 172.21.15.32:0/2086465959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240e4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47526/0 (socket says 172.21.15.32:47526) 2023-09-30T14:39:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb535e3b700 1 --2- 172.21.15.32:0/2086465959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d4da0 0x55c3240ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb53563a700 1 -- 172.21.15.32:0/2086465959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d4da0 msgr2=0x55c3240ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb53563a700 1 --2- 172.21.15.32:0/2086465959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d4da0 0x55c3240ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb53563a700 1 -- 172.21.15.32:0/2086465959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 msgr2=0x55c3240e4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb53563a700 1 --2- 172.21.15.32:0/2086465959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240e4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb53563a700 1 -- 172.21.15.32:0/2086465959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c323e50de0 con 0x55c3240d35a0 2023-09-30T14:39:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.654+0000 7fb534e39700 1 --2- 172.21.15.32:0/2086465959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240e4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.655+0000 7fb53563a700 1 --2- 172.21.15.32:0/2086465959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d35a0 0x55c3240d3980 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb52c031040 tx=0x7fb52c005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cd6ea124ffa7a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.655+0000 7fb51bfff700 1 -- 172.21.15.32:0/2086465959 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c02a040 con 0x55c3240d35a0 2023-09-30T14:39:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.655+0000 7fb51bfff700 1 -- 172.21.15.32:0/2086465959 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb52c004be0 con 0x55c3240d35a0 2023-09-30T14:39:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.655+0000 7fb51bfff700 1 -- 172.21.15.32:0/2086465959 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c0053b0 con 0x55c3240d35a0 2023-09-30T14:39:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 -- 172.21.15.32:0/2086465959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d35a0 msgr2=0x55c3240d3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 --2- 172.21.15.32:0/2086465959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d35a0 0x55c3240d3980 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb52c031040 tx=0x7fb52c005fb0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 -- 172.21.15.32:0/2086465959 shutdown_connections 2023-09-30T14:39:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 --2- 172.21.15.32:0/2086465959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d4da0 0x55c3240ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 --2- 172.21.15.32:0/2086465959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d35a0 0x55c3240d3980 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 --2- 172.21.15.32:0/2086465959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240e4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 -- 172.21.15.32:0/2086465959 >> 172.21.15.32:0/2086465959 conn(0x55c324047260 msgr2=0x55c3240efe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 -- 172.21.15.32:0/2086465959 shutdown_connections 2023-09-30T14:39:46.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 -- 172.21.15.32:0/2086465959 wait complete. 2023-09-30T14:39:46.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 -- start start 2023-09-30T14:39:46.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 Processor -- start 2023-09-30T14:39:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d35a0 0x55c3240e1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240e20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d4da0 0x55c3240e30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3240e3620 con 0x55c3240d4da0 2023-09-30T14:39:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3240e3790 con 0x55c3240d3f00 2023-09-30T14:39:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3240e3950 con 0x55c3240d35a0 2023-09-30T14:39:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.656+0000 7fb543a86b80 10 client.0 _ll_register_callbacks cb 0x7fb520012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb535e3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d4da0 0x55c3240e30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb534e39700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240e20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb534e39700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240e20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47532/0 (socket says 172.21.15.32:47532) 2023-09-30T14:39:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb535e3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d4da0 0x55c3240e30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36058/0 (socket says 172.21.15.32:36058) 2023-09-30T14:39:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb534e39700 1 -- 172.21.15.32:0/1577023748 learned_addr learned my addr 172.21.15.32:0/1577023748 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb53563a700 1 --2- 172.21.15.32:0/1577023748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d35a0 0x55c3240e1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb535e3b700 1 -- 172.21.15.32:0/1577023748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d35a0 msgr2=0x55c3240e1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb535e3b700 1 --2- 172.21.15.32:0/1577023748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d35a0 0x55c3240e1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb535e3b700 1 -- 172.21.15.32:0/1577023748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 msgr2=0x55c3240e20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb535e3b700 1 --2- 172.21.15.32:0/1577023748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240e20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb535e3b700 1 -- 172.21.15.32:0/1577023748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb530108ac0 con 0x55c3240d4da0 2023-09-30T14:39:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb53563a700 1 --2- 172.21.15.32:0/1577023748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d35a0 0x55c3240e1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb534e39700 1 --2- 172.21.15.32:0/1577023748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240e20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb535e3b700 1 --2- 172.21.15.32:0/1577023748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d4da0 0x55c3240e30e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb5280c9d40 tx=0x7fb528004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255ab70e37e5bda9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb518ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb518ff9700 1 -- 172.21.15.32:0/1577023748 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52809d350 con 0x55c3240d4da0 2023-09-30T14:39:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb518ff9700 1 -- 172.21.15.32:0/1577023748 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb52800ce60 con 0x55c3240d4da0 2023-09-30T14:39:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb543a86b80 1 -- 172.21.15.32:0/1577023748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb52c01e040 con 0x55c3240d4da0 2023-09-30T14:39:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb518ff9700 1 -- 172.21.15.32:0/1577023748 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb528099420 con 0x55c3240d4da0 2023-09-30T14:39:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb543a86b80 10 client.15648 fetch_fsmap learned FSMap version 24 2023-09-30T14:39:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.657+0000 7fb543a86b80 1 -- 172.21.15.32:0/1577023748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c3240888d0 con 0x55c3240d4da0 2023-09-30T14:39:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.658+0000 7fb518ff9700 1 -- 172.21.15.32:0/1577023748 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5280b7020 con 0x55c3240d4da0 2023-09-30T14:39:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.658+0000 7fb518ff9700 1 -- 172.21.15.32:0/1577023748 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb528098910 con 0x55c3240d4da0 2023-09-30T14:39:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.658+0000 7fb543a86b80 10 client.15648 fetch_fsmap finished waiting for FSMap version 24 2023-09-30T14:39:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.658+0000 7fb543a86b80 10 client.15648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.658+0000 7fb543a86b80 20 client.15648 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.658+0000 7fb543a86b80 1 --2- 172.21.15.32:0/1577023748 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c32413d590 0x55c32413f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.658+0000 7fb543a86b80 4 client.15648 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-09-30T14:39:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.658+0000 7fb543a86b80 1 -- 172.21.15.32:0/1577023748 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c32401eb30 con 0x55c32413d590 2023-09-30T14:39:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.658+0000 7fb53563a700 1 --2- 172.21.15.32:0/1577023748 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c32413d590 0x55c32413f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.658+0000 7fb518ff9700 10 client.15648 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.658+0000 7fb53563a700 1 --2- 172.21.15.32:0/1577023748 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c32413d590 0x55c32413f9b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92efb3a12237b1cd server_cookie=473afb57c5abbd74 in_seq=0 out_seq=0 2023-09-30T14:39:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb518ff9700 1 -- 172.21.15.32:0/1577023748 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c32401eb30 con 0x55c32413d590 2023-09-30T14:39:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb518ff9700 10 client.15648 handle_command_reply: tid=0 2023-09-30T14:39:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb543a86b80 1 client.15648 shutdown 2023-09-30T14:39:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb543a86b80 1 -- 172.21.15.32:0/1577023748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d4da0 msgr2=0x55c3240e30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb543a86b80 1 --2- 172.21.15.32:0/1577023748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d4da0 0x55c3240e30e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb5280c9d40 tx=0x7fb528004ba0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb543a86b80 1 -- 172.21.15.32:0/1577023748 shutdown_connections 2023-09-30T14:39:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb543a86b80 1 --2- 172.21.15.32:0/1577023748 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c32413d590 0x55c32413f9b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb543a86b80 1 --2- 172.21.15.32:0/1577023748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3240d4da0 0x55c3240e30e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb543a86b80 1 --2- 172.21.15.32:0/1577023748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3240d35a0 0x55c3240e1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb543a86b80 1 --2- 172.21.15.32:0/1577023748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3240d3f00 0x55c3240e20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb543a86b80 1 -- 172.21.15.32:0/1577023748 >> 172.21.15.32:0/1577023748 conn(0x55c324047260 msgr2=0x55c3240dbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb518ff9700 0 client.15648 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:46.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb543a86b80 1 -- 172.21.15.32:0/1577023748 shutdown_connections 2023-09-30T14:39:46.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb543a86b80 1 -- 172.21.15.32:0/1577023748 wait complete. 2023-09-30T14:39:46.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb543a86b80 20 client.15648 trim_cache size 0 max 16384 2023-09-30T14:39:46.667 INFO:teuthology.orchestra.run.smithi032.stdout:[] 2023-09-30T14:39:46.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb53789e700 1 -- 172.21.15.32:0/3701683241 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb51c0a0fa0 msgr2=0x7fb51c0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb53789e700 1 --2- 172.21.15.32:0/3701683241 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb51c0a0fa0 0x7fb51c0a3460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009990 tx=0x7fb52c01a040 comp rx=0 tx=0).stop 2023-09-30T14:39:46.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb53789e700 1 -- 172.21.15.32:0/3701683241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5301018c0 msgr2=0x7fb53019df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:46.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.659+0000 7fb53789e700 1 --2- 172.21.15.32:0/3701683241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5301018c0 0x7fb53019df40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb52800ef30 tx=0x7fb52800c3a0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.660+0000 7fb53789e700 1 -- 172.21.15.32:0/3701683241 shutdown_connections 2023-09-30T14:39:46.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.660+0000 7fb53789e700 1 --2- 172.21.15.32:0/3701683241 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb51c0a0fa0 0x7fb51c0a3460 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.660+0000 7fb53789e700 1 --2- 172.21.15.32:0/3701683241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5301018c0 0x7fb53019df40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.660+0000 7fb53789e700 1 --2- 172.21.15.32:0/3701683241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5300ffd30 0x7fb530068b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.660+0000 7fb53789e700 1 --2- 172.21.15.32:0/3701683241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb530100f20 0x7fb53019b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:46.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.660+0000 7fb53789e700 1 -- 172.21.15.32:0/3701683241 >> 172.21.15.32:0/3701683241 conn(0x7fb5300fb2a0 msgr2=0x7fb5300fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:46.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.660+0000 7fb53789e700 1 -- 172.21.15.32:0/3701683241 shutdown_connections 2023-09-30T14:39:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T14:39:46.660+0000 7fb53789e700 1 -- 172.21.15.32:0/3701683241 wait complete. 2023-09-30T14:39:46.673 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 2313572090, 'ino': 1099511627781, 'path': '/volumes/_nogroup/sv_1/b25fcf16-a734-410f-849e-f8fb3a417cdf/client.0'}] 2023-09-30T14:39:46.674 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_1d10b71792f3be8887a7631e69851ac2df3585af/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_1d10b71792f3be8887a7631e69851ac2df3585af/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_1d10b71792f3be8887a7631e69851ac2df3585af/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/github.com_ceph_ceph-c_1d10b71792f3be8887a7631e69851ac2df3585af/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-09-30T14:39:46.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 59672 16.94 MB/sec warmup 20 sec latency 16.890 ms 2023-09-30T14:39:47.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:47 smithi156 ceph-mon[103579]: pgmap v332: 97 pgs: 97 active+clean; 22 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2023-09-30T14:39:47.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:47 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3090320005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:39:47.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:47 smithi148 ceph-mon[102355]: pgmap v332: 97 pgs: 97 active+clean; 22 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2023-09-30T14:39:47.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:47 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3090320005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:39:47.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:47 smithi032 ceph-mon[98592]: pgmap v332: 97 pgs: 97 active+clean; 22 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2023-09-30T14:39:47.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:47 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3090320005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T14:39:47.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 61988 16.73 MB/sec warmup 21 sec latency 15.243 ms 2023-09-30T14:39:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.298+0000 7f5a09c72700 1 -- 172.21.15.32:0/4116592117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a04101930 msgr2=0x7f5a041081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.298+0000 7f5a09c72700 1 --2- 172.21.15.32:0/4116592117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a04101930 0x7f5a041081e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00b0d0 tx=0x7f59fc00b3e0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.298+0000 7f5a09c72700 1 -- 172.21.15.32:0/4116592117 shutdown_connections 2023-09-30T14:39:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.298+0000 7f5a09c72700 1 --2- 172.21.15.32:0/4116592117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a04101930 0x7f5a041081e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.298+0000 7f5a09c72700 1 --2- 172.21.15.32:0/4116592117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a040ffda0 0x7f5a041001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.298+0000 7f5a09c72700 1 --2- 172.21.15.32:0/4116592117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a04100f90 0x7f5a041013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.298+0000 7f5a09c72700 1 -- 172.21.15.32:0/4116592117 >> 172.21.15.32:0/4116592117 conn(0x7f5a040fb2f0 msgr2=0x7f5a040fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.298+0000 7f5a09c72700 1 -- 172.21.15.32:0/4116592117 shutdown_connections 2023-09-30T14:39:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.298+0000 7f5a09c72700 1 -- 172.21.15.32:0/4116592117 wait complete. 2023-09-30T14:39:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.298+0000 7f5a09c72700 1 Processor -- start 2023-09-30T14:39:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a09c72700 1 -- start start 2023-09-30T14:39:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a09c72700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a040ffda0 0x7f5a04196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a09c72700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a04100f90 0x7f5a041972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a09c72700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a04101930 0x7f5a04198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a09c72700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a04198790 con 0x7f5a04101930 2023-09-30T14:39:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a09c72700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a04198900 con 0x7f5a04100f90 2023-09-30T14:39:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a09c72700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a04198ac0 con 0x7f5a040ffda0 2023-09-30T14:39:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a03fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a04101930 0x7f5a04198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a03fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a04101930 0x7f5a04198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36072/0 (socket says 172.21.15.32:36072) 2023-09-30T14:39:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a03fff700 1 -- 172.21.15.32:0/1335910215 learned_addr learned my addr 172.21.15.32:0/1335910215 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a037fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a040ffda0 0x7f5a04196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a02ffd700 1 --2- 172.21.15.32:0/1335910215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a04100f90 0x7f5a041972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a03fff700 1 -- 172.21.15.32:0/1335910215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a040ffda0 msgr2=0x7f5a04196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a03fff700 1 --2- 172.21.15.32:0/1335910215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a040ffda0 0x7f5a04196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a03fff700 1 -- 172.21.15.32:0/1335910215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a04100f90 msgr2=0x7f5a041972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a03fff700 1 --2- 172.21.15.32:0/1335910215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a04100f90 0x7f5a041972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.299+0000 7f5a03fff700 1 -- 172.21.15.32:0/1335910215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f8009520 con 0x7f5a04101930 2023-09-30T14:39:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.300+0000 7f5a03fff700 1 --2- 172.21.15.32:0/1335910215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a04101930 0x7f5a04198080 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f59fc008eb0 tx=0x7f59fc008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.300+0000 7f5a00ff9700 1 -- 172.21.15.32:0/1335910215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc00e040 con 0x7f5a04101930 2023-09-30T14:39:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.300+0000 7f5a00ff9700 1 -- 172.21.15.32:0/1335910215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59fc00be30 con 0x7f5a04101930 2023-09-30T14:39:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.300+0000 7f5a09c72700 1 -- 172.21.15.32:0/1335910215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59fc009d00 con 0x7f5a04101930 2023-09-30T14:39:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.300+0000 7f5a00ff9700 1 -- 172.21.15.32:0/1335910215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc01d470 con 0x7f5a04101930 2023-09-30T14:39:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.300+0000 7f5a09c72700 1 -- 172.21.15.32:0/1335910215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0419f2d0 con 0x7f5a04101930 2023-09-30T14:39:48.309 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:48 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:39:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.302+0000 7f5a00ff9700 1 -- 172.21.15.32:0/1335910215 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f59fc005190 con 0x7f5a04101930 2023-09-30T14:39:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.302+0000 7f5a00ff9700 1 --2- 172.21.15.32:0/1335910215 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f59ec0a0fa0 0x7f59ec0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.303+0000 7f5a037fe700 1 --2- 172.21.15.32:0/1335910215 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f59ec0a0fa0 0x7f59ec0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.303+0000 7f5a00ff9700 1 -- 172.21.15.32:0/1335910215 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f59fc0cd6e0 con 0x7f5a04101930 2023-09-30T14:39:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.303+0000 7f5a037fe700 1 --2- 172.21.15.32:0/1335910215 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f59ec0a0fa0 0x7f59ec0a3460 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5a04196af0 tx=0x7f59f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.303+0000 7f5a15e5ab80 1 Processor -- start 2023-09-30T14:39:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.303+0000 7f5a15e5ab80 1 -- start start 2023-09-30T14:39:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.303+0000 7f5a15e5ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647216200 0x560647216620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.303+0000 7f5a15e5ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647217310 0x560647217790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f5a15e5ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647217cd0 0x56064721e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f5a15e5ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064717e400 con 0x560647216200 2023-09-30T14:39:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f5a15e5ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064716fbc0 con 0x560647217cd0 2023-09-30T14:39:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f5a15e5ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064716d4e0 con 0x560647217310 2023-09-30T14:39:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f5a037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647216200 0x560647216620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f5a037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647216200 0x560647216620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36076/0 (socket says 172.21.15.32:36076) 2023-09-30T14:39:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f5a037fe700 1 -- 172.21.15.32:0/961836534 learned_addr learned my addr 172.21.15.32:0/961836534 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f5a02ffd700 1 --2- 172.21.15.32:0/961836534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647217310 0x560647217790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f5a03fff700 1 --2- 172.21.15.32:0/961836534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647217cd0 0x56064721e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f5a037fe700 1 -- 172.21.15.32:0/961836534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647217310 msgr2=0x560647217790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f5a037fe700 1 --2- 172.21.15.32:0/961836534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647217310 0x560647217790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f5a037fe700 1 -- 172.21.15.32:0/961836534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647217cd0 msgr2=0x56064721e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f5a037fe700 1 --2- 172.21.15.32:0/961836534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647217cd0 0x56064721e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f5a037fe700 1 -- 172.21.15.32:0/961836534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606471cb870 con 0x560647216200 2023-09-30T14:39:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f5a02ffd700 1 --2- 172.21.15.32:0/961836534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647217310 0x560647217790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:39:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f5a037fe700 1 --2- 172.21.15.32:0/961836534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647216200 0x560647216620 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f59f4011d30 tx=0x7f59f4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbeda0d22bffe82 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f59e9ffb700 1 -- 172.21.15.32:0/961836534 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f400eb20 con 0x560647216200 2023-09-30T14:39:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f59e9ffb700 1 -- 172.21.15.32:0/961836534 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59f4021070 con 0x560647216200 2023-09-30T14:39:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.304+0000 7f59e9ffb700 1 -- 172.21.15.32:0/961836534 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f401cea0 con 0x560647216200 2023-09-30T14:39:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.305+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/961836534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647216200 msgr2=0x560647216620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.305+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/961836534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647216200 0x560647216620 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f59f4011d30 tx=0x7f59f4014000 comp rx=0 tx=0).stop 2023-09-30T14:39:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.305+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/961836534 shutdown_connections 2023-09-30T14:39:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.305+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/961836534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647217cd0 0x56064721e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.305+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/961836534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647216200 0x560647216620 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.305+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/961836534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647217310 0x560647217790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.305+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/961836534 >> 172.21.15.32:0/961836534 conn(0x56064718a200 msgr2=0x560647210ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.305+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/961836534 shutdown_connections 2023-09-30T14:39:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.305+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/961836534 wait complete. 2023-09-30T14:39:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.305+0000 7f5a15e5ab80 1 -- start start 2023-09-30T14:39:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a15e5ab80 1 Processor -- start 2023-09-30T14:39:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a15e5ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647216200 0x56064721fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a15e5ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647217310 0x5606472203e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a15e5ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647217cd0 0x560647227420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a15e5ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560647168dd0 con 0x560647217310 2023-09-30T14:39:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a15e5ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606471056b0 con 0x560647216200 2023-09-30T14:39:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a15e5ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560647105350 con 0x560647217cd0 2023-09-30T14:39:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a15e5ab80 10 client.0 _ll_register_callbacks cb 0x560646f627a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a037fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647216200 0x56064721fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a02ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647217310 0x5606472203e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a037fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647216200 0x56064721fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47558/0 (socket says 172.21.15.32:47558) 2023-09-30T14:39:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a037fe700 1 -- 172.21.15.32:0/3422121547 learned_addr learned my addr 172.21.15.32:0/3422121547 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a03fff700 1 --2- 172.21.15.32:0/3422121547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647217cd0 0x560647227420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a02ffd700 1 -- 172.21.15.32:0/3422121547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647217cd0 msgr2=0x560647227420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a02ffd700 1 --2- 172.21.15.32:0/3422121547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647217cd0 0x560647227420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a02ffd700 1 -- 172.21.15.32:0/3422121547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647216200 msgr2=0x56064721fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a02ffd700 1 --2- 172.21.15.32:0/3422121547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647216200 0x56064721fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.306+0000 7f5a02ffd700 1 -- 172.21.15.32:0/3422121547 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f80099c0 con 0x560647217310 2023-09-30T14:39:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.307+0000 7f5a02ffd700 1 --2- 172.21.15.32:0/3422121547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647217310 0x5606472203e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f59f80129a0 tx=0x7f59f8012cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b3a70348687f42 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.307+0000 7f59e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.307+0000 7f59e2ffd700 1 -- 172.21.15.32:0/3422121547 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8017070 con 0x560647217310 2023-09-30T14:39:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.307+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/3422121547 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560647209090 con 0x560647217310 2023-09-30T14:39:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.307+0000 7f59e2ffd700 1 -- 172.21.15.32:0/3422121547 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f59f80119c0 con 0x560647217310 2023-09-30T14:39:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.307+0000 7f59e2ffd700 1 -- 172.21.15.32:0/3422121547 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59f8022030 con 0x560647217310 2023-09-30T14:39:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.307+0000 7f5a15e5ab80 10 client.15660 fetch_fsmap learned FSMap version 25 2023-09-30T14:39:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.307+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/3422121547 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f59f4017070 con 0x560647217310 2023-09-30T14:39:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.307+0000 7f59e2ffd700 1 -- 172.21.15.32:0/3422121547 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560647209090 con 0x560647217310 2023-09-30T14:39:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.307+0000 7f59e2ffd700 1 -- 172.21.15.32:0/3422121547 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f59f801cce0 con 0x560647217310 2023-09-30T14:39:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.307+0000 7f5a15e5ab80 10 client.15660 fetch_fsmap finished waiting for FSMap version 25 2023-09-30T14:39:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.307+0000 7f5a15e5ab80 10 client.15660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.307+0000 7f5a15e5ab80 20 client.15660 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.308+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/3422121547 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56064722bda0 0x56064722e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.308+0000 7f5a15e5ab80 4 client.15660 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:39:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.308+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/3422121547 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560647161db0 con 0x56064722bda0 2023-09-30T14:39:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.308+0000 7f5a037fe700 1 --2- 172.21.15.32:0/3422121547 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56064722bda0 0x56064722e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.308+0000 7f59e2ffd700 10 client.15660 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.308+0000 7f5a037fe700 1 --2- 172.21.15.32:0/3422121547 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56064722bda0 0x56064722e1c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c1d55aea353b014 server_cookie=f4ac23dcd02041fe in_seq=0 out_seq=0 2023-09-30T14:39:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.309+0000 7f59e2ffd700 1 -- 172.21.15.32:0/3422121547 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560647161db0 con 0x56064722bda0 2023-09-30T14:39:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.309+0000 7f59e2ffd700 10 client.15660 handle_command_reply: tid=0 2023-09-30T14:39:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.309+0000 7f5a15e5ab80 1 client.15660 shutdown 2023-09-30T14:39:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.309+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/3422121547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647217310 msgr2=0x5606472203e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.309+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/3422121547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647217310 0x5606472203e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f59f80129a0 tx=0x7f59f8012cb0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.309+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/3422121547 shutdown_connections 2023-09-30T14:39:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.309+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/3422121547 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56064722bda0 0x56064722e1c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.309+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/3422121547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647217cd0 0x560647227420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.309+0000 7f59e2ffd700 0 client.15660 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.309+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/3422121547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647216200 0x56064721fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.309+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/3422121547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647217310 0x5606472203e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.309+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/3422121547 >> 172.21.15.32:0/3422121547 conn(0x56064718a200 msgr2=0x560647210590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.309+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/3422121547 shutdown_connections 2023-09-30T14:39:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.309+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/3422121547 wait complete. 2023-09-30T14:39:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.309+0000 7f5a15e5ab80 20 client.15660 trim_cache size 0 max 16384 2023-09-30T14:39:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.324+0000 7f5a15e5ab80 1 Processor -- start 2023-09-30T14:39:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.324+0000 7f5a15e5ab80 1 -- start start 2023-09-30T14:39:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.324+0000 7f5a15e5ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647216200 0x5606472176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.324+0000 7f5a15e5ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647217310 0x5606472273e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.324+0000 7f5a15e5ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647217cd0 0x56064721e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.324+0000 7f5a15e5ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560646fe98c0 con 0x560647217310 2023-09-30T14:39:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.324+0000 7f5a15e5ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560647208c70 con 0x560647217cd0 2023-09-30T14:39:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.324+0000 7f5a15e5ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560647220ba0 con 0x560647216200 2023-09-30T14:39:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.324+0000 7f5a02ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647217310 0x5606472273e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.324+0000 7f5a02ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647217310 0x5606472273e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36092/0 (socket says 172.21.15.32:36092) 2023-09-30T14:39:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.324+0000 7f5a02ffd700 1 -- 172.21.15.32:0/2792690316 learned_addr learned my addr 172.21.15.32:0/2792690316 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.324+0000 7f5a037fe700 1 --2- 172.21.15.32:0/2792690316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647216200 0x5606472176f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.325+0000 7f5a03fff700 1 --2- 172.21.15.32:0/2792690316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647217cd0 0x56064721e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.325+0000 7f5a02ffd700 1 -- 172.21.15.32:0/2792690316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647216200 msgr2=0x5606472176f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.325+0000 7f5a02ffd700 1 --2- 172.21.15.32:0/2792690316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647216200 0x5606472176f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.325+0000 7f5a02ffd700 1 -- 172.21.15.32:0/2792690316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647217cd0 msgr2=0x56064721e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.325+0000 7f5a02ffd700 1 --2- 172.21.15.32:0/2792690316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647217cd0 0x56064721e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.325+0000 7f5a02ffd700 1 -- 172.21.15.32:0/2792690316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560646f93d80 con 0x560647217310 2023-09-30T14:39:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.325+0000 7f5a03fff700 1 --2- 172.21.15.32:0/2792690316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647217cd0 0x56064721e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:39:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.325+0000 7f5a02ffd700 1 --2- 172.21.15.32:0/2792690316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647217310 0x5606472273e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f59f8011470 tx=0x7f59f80114a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88059544ad018a5e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.325+0000 7f59ea7fc700 1 -- 172.21.15.32:0/2792690316 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8022070 con 0x560647217310 2023-09-30T14:39:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.325+0000 7f59ea7fc700 1 -- 172.21.15.32:0/2792690316 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59f80119c0 con 0x560647217310 2023-09-30T14:39:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.325+0000 7f59ea7fc700 1 -- 172.21.15.32:0/2792690316 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8017040 con 0x560647217310 2023-09-30T14:39:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.326+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/2792690316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647217310 msgr2=0x5606472273e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.326+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/2792690316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647217310 0x5606472273e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f59f8011470 tx=0x7f59f80114a0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.326+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/2792690316 shutdown_connections 2023-09-30T14:39:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.326+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/2792690316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647217cd0 0x56064721e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.326+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/2792690316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647216200 0x5606472176f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.326+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/2792690316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647217310 0x5606472273e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.326+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/2792690316 >> 172.21.15.32:0/2792690316 conn(0x56064718a200 msgr2=0x56064721ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.326+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/2792690316 shutdown_connections 2023-09-30T14:39:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.326+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/2792690316 wait complete. 2023-09-30T14:39:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.326+0000 7f5a15e5ab80 1 -- start start 2023-09-30T14:39:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.326+0000 7f5a15e5ab80 1 Processor -- start 2023-09-30T14:39:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a15e5ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647216200 0x5606472251c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a15e5ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647217310 0x560647225700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a15e5ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647217cd0 0x560647226710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a15e5ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560647226c50 con 0x560647216200 2023-09-30T14:39:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a15e5ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560647226dc0 con 0x560647217310 2023-09-30T14:39:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a15e5ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560647226f80 con 0x560647217cd0 2023-09-30T14:39:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a15e5ab80 10 client.0 _ll_register_callbacks cb 0x7f59f80124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:39:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647216200 0x5606472251c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a03fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647217cd0 0x560647226710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647216200 0x5606472251c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36096/0 (socket says 172.21.15.32:36096) 2023-09-30T14:39:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a037fe700 1 -- 172.21.15.32:0/4254161105 learned_addr learned my addr 172.21.15.32:0/4254161105 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:39:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a02ffd700 1 --2- 172.21.15.32:0/4254161105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647217310 0x560647225700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a037fe700 1 -- 172.21.15.32:0/4254161105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647217cd0 msgr2=0x560647226710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a037fe700 1 --2- 172.21.15.32:0/4254161105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647217cd0 0x560647226710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a037fe700 1 -- 172.21.15.32:0/4254161105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647217310 msgr2=0x560647225700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:39:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a037fe700 1 --2- 172.21.15.32:0/4254161105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647217310 0x560647225700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a037fe700 1 -- 172.21.15.32:0/4254161105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f4017070 con 0x560647216200 2023-09-30T14:39:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a037fe700 1 --2- 172.21.15.32:0/4254161105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647216200 0x5606472251c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f59f401c9c0 tx=0x7f59f4005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132f6f51d1ad7909 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:39:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f59e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:39:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f59e2ffd700 1 -- 172.21.15.32:0/4254161105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f401cc10 con 0x560647216200 2023-09-30T14:39:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f59e2ffd700 1 -- 172.21.15.32:0/4254161105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f59f4011ce0 con 0x560647216200 2023-09-30T14:39:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.327+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/4254161105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59f4021070 con 0x560647216200 2023-09-30T14:39:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.328+0000 7f59e2ffd700 1 -- 172.21.15.32:0/4254161105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59f4011ea0 con 0x560647216200 2023-09-30T14:39:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.328+0000 7f5a15e5ab80 10 client.15666 fetch_fsmap learned FSMap version 25 2023-09-30T14:39:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.328+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/4254161105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560646f7cc00 con 0x560647216200 2023-09-30T14:39:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.328+0000 7f59e2ffd700 1 -- 172.21.15.32:0/4254161105 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59f4021070 con 0x560647216200 2023-09-30T14:39:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.328+0000 7f59e2ffd700 1 -- 172.21.15.32:0/4254161105 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f59f400eb60 con 0x560647216200 2023-09-30T14:39:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.328+0000 7f5a15e5ab80 10 client.15666 fetch_fsmap finished waiting for FSMap version 25 2023-09-30T14:39:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.328+0000 7f5a15e5ab80 10 client.15666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:39:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.328+0000 7f5a15e5ab80 20 client.15666 populate_metadata read hostname 'smithi032' 2023-09-30T14:39:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.328+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/4254161105 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560647280c40 0x560647283060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:39:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.328+0000 7f5a15e5ab80 4 client.15666 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:39:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.328+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/4254161105 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560647161ad0 con 0x560647280c40 2023-09-30T14:39:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.328+0000 7f5a02ffd700 1 --2- 172.21.15.32:0/4254161105 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560647280c40 0x560647283060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:39:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.329+0000 7f59e2ffd700 10 client.15666 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.329+0000 7f5a02ffd700 1 --2- 172.21.15.32:0/4254161105 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560647280c40 0x560647283060 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f471a8f0d258e3 server_cookie=9dd04c663550f68d in_seq=0 out_seq=0 2023-09-30T14:39:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.348+0000 7f59e2ffd700 1 -- 172.21.15.32:0/4254161105 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560647161ad0 con 0x560647280c40 2023-09-30T14:39:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.348+0000 7f59e2ffd700 10 client.15666 handle_command_reply: tid=0 2023-09-30T14:39:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.348+0000 7f5a15e5ab80 1 client.15666 shutdown 2023-09-30T14:39:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.349+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/4254161105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647216200 msgr2=0x5606472251c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.349+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/4254161105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647216200 0x5606472251c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f59f401c9c0 tx=0x7f59f4005c40 comp rx=0 tx=0).stop 2023-09-30T14:39:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.349+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/4254161105 shutdown_connections 2023-09-30T14:39:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.349+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/4254161105 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560647280c40 0x560647283060 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.349+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/4254161105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560647217cd0 0x560647226710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.349+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/4254161105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560647216200 0x5606472251c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.349+0000 7f59e2ffd700 0 client.15666 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:39:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.349+0000 7f5a15e5ab80 1 --2- 172.21.15.32:0/4254161105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560647217310 0x560647225700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.349+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/4254161105 >> 172.21.15.32:0/4254161105 conn(0x56064718a200 msgr2=0x56064721e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.349+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/4254161105 shutdown_connections 2023-09-30T14:39:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.349+0000 7f5a15e5ab80 1 -- 172.21.15.32:0/4254161105 wait complete. 2023-09-30T14:39:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.349+0000 7f5a15e5ab80 20 client.15666 trim_cache size 0 max 16384 2023-09-30T14:39:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:39:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:39:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:39:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:39:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.349+0000 7f5a09c72700 1 -- 172.21.15.32:0/1335910215 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f59ec0a0fa0 msgr2=0x7f59ec0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.349+0000 7f5a09c72700 1 --2- 172.21.15.32:0/1335910215 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f59ec0a0fa0 0x7f59ec0a3460 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5a04196af0 tx=0x7f59f4009260 comp rx=0 tx=0).stop 2023-09-30T14:39:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.349+0000 7f5a09c72700 1 -- 172.21.15.32:0/1335910215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a04101930 msgr2=0x7f5a04198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:39:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.349+0000 7f5a09c72700 1 --2- 172.21.15.32:0/1335910215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a04101930 0x7f5a04198080 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f59fc008eb0 tx=0x7f59fc008ee0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.350+0000 7f5a09c72700 1 -- 172.21.15.32:0/1335910215 shutdown_connections 2023-09-30T14:39:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.350+0000 7f5a09c72700 1 --2- 172.21.15.32:0/1335910215 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f59ec0a0fa0 0x7f59ec0a3460 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.350+0000 7f5a09c72700 1 --2- 172.21.15.32:0/1335910215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a04101930 0x7f5a04198080 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.350+0000 7f5a09c72700 1 --2- 172.21.15.32:0/1335910215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a040ffda0 0x7f5a04196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.350+0000 7f5a09c72700 1 --2- 172.21.15.32:0/1335910215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a04100f90 0x7f5a041972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:39:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.350+0000 7f5a09c72700 1 -- 172.21.15.32:0/1335910215 >> 172.21.15.32:0/1335910215 conn(0x7f5a040fb2f0 msgr2=0x7f5a04104380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:39:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.350+0000 7f5a09c72700 1 -- 172.21.15.32:0/1335910215 shutdown_connections 2023-09-30T14:39:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:39:48.350+0000 7f5a09c72700 1 -- 172.21.15.32:0/1335910215 wait complete. 2023-09-30T14:39:48.358 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:48 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:39:48.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:48 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:39:48.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 64591 16.61 MB/sec warmup 22 sec latency 19.495 ms 2023-09-30T14:39:49.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:49 smithi156 ceph-mon[103579]: pgmap v333: 97 pgs: 97 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.8 MiB/s wr, 240 op/s 2023-09-30T14:39:49.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:49 smithi148 ceph-mon[102355]: pgmap v333: 97 pgs: 97 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.8 MiB/s wr, 240 op/s 2023-09-30T14:39:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:49 smithi032 ceph-mon[98592]: pgmap v333: 97 pgs: 97 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.8 MiB/s wr, 240 op/s 2023-09-30T14:39:49.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 67689 16.69 MB/sec warmup 23 sec latency 11.866 ms 2023-09-30T14:39:50.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 70339 16.53 MB/sec warmup 24 sec latency 37.039 ms 2023-09-30T14:39:51.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:51 smithi156 ceph-mon[103579]: pgmap v334: 97 pgs: 97 active+clean; 28 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2023-09-30T14:39:51.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:51 smithi148 ceph-mon[102355]: pgmap v334: 97 pgs: 97 active+clean; 28 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2023-09-30T14:39:51.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:51 smithi032 ceph-mon[98592]: pgmap v334: 97 pgs: 97 active+clean; 28 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2023-09-30T14:39:51.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 72656 16.35 MB/sec warmup 25 sec latency 10.871 ms 2023-09-30T14:39:52.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 75356 16.27 MB/sec warmup 26 sec latency 11.764 ms 2023-09-30T14:39:53.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:53 smithi156 ceph-mon[103579]: pgmap v335: 97 pgs: 97 active+clean; 30 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.7 MiB/s wr, 233 op/s 2023-09-30T14:39:53.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:53 smithi148 ceph-mon[102355]: pgmap v335: 97 pgs: 97 active+clean; 30 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.7 MiB/s wr, 233 op/s 2023-09-30T14:39:53.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:53 smithi032 ceph-mon[98592]: pgmap v335: 97 pgs: 97 active+clean; 30 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.7 MiB/s wr, 233 op/s 2023-09-30T14:39:53.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 78331 16.35 MB/sec warmup 27 sec latency 14.971 ms 2023-09-30T14:39:54.689 INFO:tasks.workunit.client.0.smithi032.stdout: 1 81098 16.22 MB/sec warmup 28 sec latency 11.774 ms 2023-09-30T14:39:55.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:55 smithi156 ceph-mon[103579]: pgmap v336: 97 pgs: 97 active+clean; 32 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.3 MiB/s wr, 181 op/s 2023-09-30T14:39:55.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:55 smithi148 ceph-mon[102355]: pgmap v336: 97 pgs: 97 active+clean; 32 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.3 MiB/s wr, 181 op/s 2023-09-30T14:39:55.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:55 smithi032 ceph-mon[98592]: pgmap v336: 97 pgs: 97 active+clean; 32 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.3 MiB/s wr, 181 op/s 2023-09-30T14:39:55.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 83496 16.08 MB/sec warmup 29 sec latency 10.890 ms 2023-09-30T14:39:56.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 85898 16.01 MB/sec warmup 30 sec latency 10.493 ms 2023-09-30T14:39:57.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:57 smithi156 ceph-mon[103579]: pgmap v337: 97 pgs: 97 active+clean; 36 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.7 MiB/s wr, 258 op/s 2023-09-30T14:39:57.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:57 smithi148 ceph-mon[102355]: pgmap v337: 97 pgs: 97 active+clean; 36 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.7 MiB/s wr, 258 op/s 2023-09-30T14:39:57.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:57 smithi032 ceph-mon[98592]: pgmap v337: 97 pgs: 97 active+clean; 36 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.7 MiB/s wr, 258 op/s 2023-09-30T14:39:57.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 88934 16.09 MB/sec warmup 31 sec latency 32.366 ms 2023-09-30T14:39:58.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 90418 15.76 MB/sec warmup 32 sec latency 37.527 ms 2023-09-30T14:39:59.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:39:59 smithi156 ceph-mon[103579]: pgmap v338: 97 pgs: 97 active+clean; 39 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.7 MiB/s wr, 279 op/s 2023-09-30T14:39:59.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:39:59 smithi148 ceph-mon[102355]: pgmap v338: 97 pgs: 97 active+clean; 39 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.7 MiB/s wr, 279 op/s 2023-09-30T14:39:59.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:39:59 smithi032 ceph-mon[98592]: pgmap v338: 97 pgs: 97 active+clean; 39 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.7 MiB/s wr, 279 op/s 2023-09-30T14:39:59.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 92816 15.70 MB/sec warmup 33 sec latency 17.030 ms 2023-09-30T14:40:00.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:00 smithi148 ceph-mon[102355]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-09-30T14:40:00.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:00 smithi148 ceph-mon[102355]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-09-30T14:40:00.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:00 smithi148 ceph-mon[102355]: mds.h(mds.2): Metadata damage detected 2023-09-30T14:40:00.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:00 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-a[98568]: 2023-09-30T14:39:59.999+0000 7f49afbf4700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-09-30T14:40:00.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:00 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-a[98568]: 2023-09-30T14:39:59.999+0000 7f49afbf4700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-09-30T14:40:00.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:00 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-a[98568]: 2023-09-30T14:39:59.999+0000 7f49afbf4700 -1 log_channel(cluster) log [ERR] : mds.h(mds.2): Metadata damage detected 2023-09-30T14:40:00.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:00 smithi032 ceph-mon[98592]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-09-30T14:40:00.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:00 smithi032 ceph-mon[98592]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-09-30T14:40:00.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:00 smithi032 ceph-mon[98592]: mds.h(mds.2): Metadata damage detected 2023-09-30T14:40:00.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:00 smithi156 ceph-mon[103579]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-09-30T14:40:00.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:00 smithi156 ceph-mon[103579]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-09-30T14:40:00.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:00 smithi156 ceph-mon[103579]: mds.h(mds.2): Metadata damage detected 2023-09-30T14:40:00.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 95456 15.63 MB/sec warmup 34 sec latency 99.907 ms 2023-09-30T14:40:01.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:01 smithi148 ceph-mon[102355]: pgmap v339: 97 pgs: 97 active+clean; 48 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.2 MiB/s wr, 233 op/s 2023-09-30T14:40:01.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:01 smithi032 ceph-mon[98592]: pgmap v339: 97 pgs: 97 active+clean; 48 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.2 MiB/s wr, 233 op/s 2023-09-30T14:40:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.464+0000 7fb2a315e700 1 -- 172.21.15.32:0/1830650494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c1042d0 msgr2=0x7fb29c1066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.464+0000 7fb2a315e700 1 --2- 172.21.15.32:0/1830650494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c1042d0 0x7fb29c1066c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009990 tx=0x7fb28c009ca0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.464+0000 7fb2a315e700 1 -- 172.21.15.32:0/1830650494 shutdown_connections 2023-09-30T14:40:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.464+0000 7fb2a315e700 1 --2- 172.21.15.32:0/1830650494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb29c106c00 0x7fb29c108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.464+0000 7fb2a315e700 1 --2- 172.21.15.32:0/1830650494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29c1019a0 0x7fb29c103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.464+0000 7fb2a315e700 1 --2- 172.21.15.32:0/1830650494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c1042d0 0x7fb29c1066c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.464+0000 7fb2a315e700 1 -- 172.21.15.32:0/1830650494 >> 172.21.15.32:0/1830650494 conn(0x7fb29c0fb2b0 msgr2=0x7fb29c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.464+0000 7fb2a315e700 1 -- 172.21.15.32:0/1830650494 shutdown_connections 2023-09-30T14:40:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.464+0000 7fb2a315e700 1 -- 172.21.15.32:0/1830650494 wait complete. 2023-09-30T14:40:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.465+0000 7fb2a315e700 1 Processor -- start 2023-09-30T14:40:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.465+0000 7fb2a315e700 1 -- start start 2023-09-30T14:40:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.465+0000 7fb2a315e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29c1019a0 0x7fb29c196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.465+0000 7fb2a315e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb29c1042d0 0x7fb29c197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.465+0000 7fb2a315e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c106c00 0x7fb29c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.465+0000 7fb2a315e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c198600 con 0x7fb29c106c00 2023-09-30T14:40:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.465+0000 7fb2a315e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c198770 con 0x7fb29c1019a0 2023-09-30T14:40:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.465+0000 7fb2a315e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c198930 con 0x7fb29c1042d0 2023-09-30T14:40:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.465+0000 7fb2a16fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c106c00 0x7fb29c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.465+0000 7fb2a16fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c106c00 0x7fb29c197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36114/0 (socket says 172.21.15.32:36114) 2023-09-30T14:40:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.465+0000 7fb2a0efa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29c1019a0 0x7fb29c196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.465+0000 7fb293fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb29c1042d0 0x7fb29c197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.465+0000 7fb2a16fb700 1 -- 172.21.15.32:0/3890670889 learned_addr learned my addr 172.21.15.32:0/3890670889 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.466+0000 7fb2a16fb700 1 -- 172.21.15.32:0/3890670889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb29c1042d0 msgr2=0x7fb29c197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.466+0000 7fb2a16fb700 1 --2- 172.21.15.32:0/3890670889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb29c1042d0 0x7fb29c197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.466+0000 7fb2a16fb700 1 -- 172.21.15.32:0/3890670889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29c1019a0 msgr2=0x7fb29c196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.466+0000 7fb2a16fb700 1 --2- 172.21.15.32:0/3890670889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29c1019a0 0x7fb29c196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.466+0000 7fb2a16fb700 1 -- 172.21.15.32:0/3890670889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28c0095f0 con 0x7fb29c106c00 2023-09-30T14:40:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.466+0000 7fb2a0efa700 1 --2- 172.21.15.32:0/3890670889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29c1019a0 0x7fb29c196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:40:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.466+0000 7fb2a16fb700 1 --2- 172.21.15.32:0/3890670889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c106c00 0x7fb29c197fa0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb29400ef30 tx=0x7fb29400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.466+0000 7fb291ffb700 1 -- 172.21.15.32:0/3890670889 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294018070 con 0x7fb29c106c00 2023-09-30T14:40:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.466+0000 7fb291ffb700 1 -- 172.21.15.32:0/3890670889 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb29400f040 con 0x7fb29c106c00 2023-09-30T14:40:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.466+0000 7fb291ffb700 1 -- 172.21.15.32:0/3890670889 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29400cd00 con 0x7fb29c106c00 2023-09-30T14:40:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.466+0000 7fb2a315e700 1 -- 172.21.15.32:0/3890670889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29c198bb0 con 0x7fb29c106c00 2023-09-30T14:40:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.466+0000 7fb2a315e700 1 -- 172.21.15.32:0/3890670889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c19f3d0 con 0x7fb29c106c00 2023-09-30T14:40:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.468+0000 7fb291ffb700 1 -- 172.21.15.32:0/3890670889 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fb294003a00 con 0x7fb29c106c00 2023-09-30T14:40:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.469+0000 7fb291ffb700 1 --2- 172.21.15.32:0/3890670889 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb2840a1020 0x7fb2840a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.469+0000 7fb2a0efa700 1 --2- 172.21.15.32:0/3890670889 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb2840a1020 0x7fb2840a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.469+0000 7fb291ffb700 1 -- 172.21.15.32:0/3890670889 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb2940cc1e0 con 0x7fb29c106c00 2023-09-30T14:40:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.470+0000 7fb2a0efa700 1 --2- 172.21.15.32:0/3890670889 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb2840a1020 0x7fb2840a34e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb298011490 tx=0x7fb298009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.470+0000 7fb2af346b80 1 Processor -- start 2023-09-30T14:40:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.470+0000 7fb2af346b80 1 -- start start 2023-09-30T14:40:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.470+0000 7fb2af346b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32d6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.470+0000 7fb2af346b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7310 0x556da32d7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.470+0000 7fb2af346b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7cd0 0x556da32de560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.470+0000 7fb2af346b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da323e400 con 0x556da32d6200 2023-09-30T14:40:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.470+0000 7fb2af346b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da322fbc0 con 0x556da32d7310 2023-09-30T14:40:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.470+0000 7fb2af346b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da322d4e0 con 0x556da32d7cd0 2023-09-30T14:40:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.471+0000 7fb2a0efa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32d6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.471+0000 7fb2a0efa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32d6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36118/0 (socket says 172.21.15.32:36118) 2023-09-30T14:40:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.471+0000 7fb293fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7310 0x556da32d7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.471+0000 7fb2a0efa700 1 -- 172.21.15.32:0/442400828 learned_addr learned my addr 172.21.15.32:0/442400828 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.471+0000 7fb2a16fb700 1 --2- 172.21.15.32:0/442400828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7cd0 0x556da32de560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.471+0000 7fb2a0efa700 1 -- 172.21.15.32:0/442400828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7cd0 msgr2=0x556da32de560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.471+0000 7fb2a0efa700 1 --2- 172.21.15.32:0/442400828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7cd0 0x556da32de560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.471+0000 7fb2a0efa700 1 -- 172.21.15.32:0/442400828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7310 msgr2=0x556da32d7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.471+0000 7fb2a0efa700 1 --2- 172.21.15.32:0/442400828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7310 0x556da32d7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.471+0000 7fb2a0efa700 1 -- 172.21.15.32:0/442400828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556da328b870 con 0x556da32d6200 2023-09-30T14:40:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.471+0000 7fb293fff700 1 --2- 172.21.15.32:0/442400828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7310 0x556da32d7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:40:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.471+0000 7fb2a0efa700 1 --2- 172.21.15.32:0/442400828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32d6620 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb2980043c0 tx=0x7fb298014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b611dcdbc547c4a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.471+0000 7fb282ffd700 1 -- 172.21.15.32:0/442400828 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298003d10 con 0x556da32d6200 2023-09-30T14:40:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.471+0000 7fb282ffd700 1 -- 172.21.15.32:0/442400828 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb298003e70 con 0x556da32d6200 2023-09-30T14:40:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.471+0000 7fb282ffd700 1 -- 172.21.15.32:0/442400828 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29801e7d0 con 0x556da32d6200 2023-09-30T14:40:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.472+0000 7fb2af346b80 1 -- 172.21.15.32:0/442400828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 msgr2=0x556da32d6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.472+0000 7fb2af346b80 1 --2- 172.21.15.32:0/442400828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32d6620 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb2980043c0 tx=0x7fb298014000 comp rx=0 tx=0).stop 2023-09-30T14:40:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.472+0000 7fb2af346b80 1 -- 172.21.15.32:0/442400828 shutdown_connections 2023-09-30T14:40:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.472+0000 7fb2af346b80 1 --2- 172.21.15.32:0/442400828 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7cd0 0x556da32de560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.472+0000 7fb2af346b80 1 --2- 172.21.15.32:0/442400828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32d6620 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.472+0000 7fb2af346b80 1 --2- 172.21.15.32:0/442400828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7310 0x556da32d7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.472+0000 7fb2af346b80 1 -- 172.21.15.32:0/442400828 >> 172.21.15.32:0/442400828 conn(0x556da324a200 msgr2=0x556da32d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.472+0000 7fb2af346b80 1 -- 172.21.15.32:0/442400828 shutdown_connections 2023-09-30T14:40:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.472+0000 7fb2af346b80 1 -- 172.21.15.32:0/442400828 wait complete. 2023-09-30T14:40:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.472+0000 7fb2af346b80 1 -- start start 2023-09-30T14:40:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.472+0000 7fb2af346b80 1 Processor -- start 2023-09-30T14:40:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2af346b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32dff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2af346b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 0x556da32e0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2af346b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7cd0 0x556da32e74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2af346b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da3228dd0 con 0x556da32d6200 2023-09-30T14:40:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2af346b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da31c56b0 con 0x556da32d7cd0 2023-09-30T14:40:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2af346b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da31c5350 con 0x556da32d7310 2023-09-30T14:40:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2af346b80 10 client.0 _ll_register_callbacks cb 0x556da30227a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:40:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2a0efa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32dff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2a0efa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32dff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36124/0 (socket says 172.21.15.32:36124) 2023-09-30T14:40:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2a0efa700 1 -- 172.21.15.32:0/3584319510 learned_addr learned my addr 172.21.15.32:0/3584319510 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb293fff700 1 --2- 172.21.15.32:0/3584319510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 0x556da32e0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2a16fb700 1 --2- 172.21.15.32:0/3584319510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7cd0 0x556da32e74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2a0efa700 1 -- 172.21.15.32:0/3584319510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 msgr2=0x556da32e0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2a0efa700 1 --2- 172.21.15.32:0/3584319510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 0x556da32e0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2a0efa700 1 -- 172.21.15.32:0/3584319510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7cd0 msgr2=0x556da32e74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2a0efa700 1 --2- 172.21.15.32:0/3584319510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7cd0 0x556da32e74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2a0efa700 1 -- 172.21.15.32:0/3584319510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c109bc0 con 0x556da32d6200 2023-09-30T14:40:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2a16fb700 1 --2- 172.21.15.32:0/3584319510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7cd0 0x556da32e74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:40:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb2a0efa700 1 --2- 172.21.15.32:0/3584319510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32dff30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb298003e40 tx=0x7fb298026630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a569175e99d8a5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.473+0000 7fb27bfff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:40:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.474+0000 7fb27bfff700 1 -- 172.21.15.32:0/3584319510 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29802b070 con 0x556da32d6200 2023-09-30T14:40:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.474+0000 7fb27bfff700 1 -- 172.21.15.32:0/3584319510 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb2980269e0 con 0x556da32d6200 2023-09-30T14:40:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.474+0000 7fb2af346b80 1 -- 172.21.15.32:0/3584319510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556da32c9090 con 0x556da32d6200 2023-09-30T14:40:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.474+0000 7fb27bfff700 1 -- 172.21.15.32:0/3584319510 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb298026ba0 con 0x556da32d6200 2023-09-30T14:40:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.474+0000 7fb2af346b80 10 client.15678 fetch_fsmap learned FSMap version 25 2023-09-30T14:40:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.474+0000 7fb2af346b80 1 -- 172.21.15.32:0/3584319510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fb298017070 con 0x556da32d6200 2023-09-30T14:40:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.474+0000 7fb27bfff700 1 -- 172.21.15.32:0/3584319510 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556da32c9090 con 0x556da32d6200 2023-09-30T14:40:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.474+0000 7fb27bfff700 1 -- 172.21.15.32:0/3584319510 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb298024020 con 0x556da32d6200 2023-09-30T14:40:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.474+0000 7fb2af346b80 10 client.15678 fetch_fsmap finished waiting for FSMap version 25 2023-09-30T14:40:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.474+0000 7fb2af346b80 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:40:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.474+0000 7fb2af346b80 20 client.15678 populate_metadata read hostname 'smithi032' 2023-09-30T14:40:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.474+0000 7fb2af346b80 1 --2- 172.21.15.32:0/3584319510 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556da32ebc70 0x556da32ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.474+0000 7fb2af346b80 4 client.15678 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:40:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.474+0000 7fb2af346b80 1 -- 172.21.15.32:0/3584319510 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556da3221db0 con 0x556da32ebc70 2023-09-30T14:40:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.474+0000 7fb293fff700 1 --2- 172.21.15.32:0/3584319510 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556da32ebc70 0x556da32ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.475+0000 7fb27bfff700 10 client.15678 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.475+0000 7fb293fff700 1 --2- 172.21.15.32:0/3584319510 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556da32ebc70 0x556da32ee090 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a064761bc5824ad server_cookie=4fecac10b67786d9 in_seq=0 out_seq=0 2023-09-30T14:40:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.475+0000 7fb27bfff700 1 -- 172.21.15.32:0/3584319510 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556da3221db0 con 0x556da32ebc70 2023-09-30T14:40:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.475+0000 7fb27bfff700 10 client.15678 handle_command_reply: tid=0 2023-09-30T14:40:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.475+0000 7fb2af346b80 1 client.15678 shutdown 2023-09-30T14:40:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.476+0000 7fb2af346b80 1 -- 172.21.15.32:0/3584319510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 msgr2=0x556da32dff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.476+0000 7fb2af346b80 1 --2- 172.21.15.32:0/3584319510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32dff30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb298003e40 tx=0x7fb298026630 comp rx=0 tx=0).stop 2023-09-30T14:40:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.476+0000 7fb2af346b80 1 -- 172.21.15.32:0/3584319510 shutdown_connections 2023-09-30T14:40:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.476+0000 7fb2af346b80 1 --2- 172.21.15.32:0/3584319510 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556da32ebc70 0x556da32ee090 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.476+0000 7fb2af346b80 1 --2- 172.21.15.32:0/3584319510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7cd0 0x556da32e74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.476+0000 7fb2af346b80 1 --2- 172.21.15.32:0/3584319510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32dff30 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.476+0000 7fb2af346b80 1 --2- 172.21.15.32:0/3584319510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 0x556da32e0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.476+0000 7fb27bfff700 0 client.15678 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.476+0000 7fb2af346b80 1 -- 172.21.15.32:0/3584319510 >> 172.21.15.32:0/3584319510 conn(0x556da324a200 msgr2=0x556da32d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.476+0000 7fb2af346b80 1 -- 172.21.15.32:0/3584319510 shutdown_connections 2023-09-30T14:40:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.476+0000 7fb2af346b80 1 -- 172.21.15.32:0/3584319510 wait complete. 2023-09-30T14:40:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.476+0000 7fb2af346b80 20 client.15678 trim_cache size 0 max 16384 2023-09-30T14:40:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.491+0000 7fb2af346b80 1 Processor -- start 2023-09-30T14:40:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.491+0000 7fb2af346b80 1 -- start start 2023-09-30T14:40:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.491+0000 7fb2af346b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32dec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.491+0000 7fb2af346b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 0x556da32e7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.491+0000 7fb2af346b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7cd0 0x556da32d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.491+0000 7fb2af346b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da30a98c0 con 0x556da32d6200 2023-09-30T14:40:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.491+0000 7fb2af346b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da32cdba0 con 0x556da32d7cd0 2023-09-30T14:40:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.491+0000 7fb2af346b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da32d80b0 con 0x556da32d7310 2023-09-30T14:40:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.491+0000 7fb2a0efa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32dec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.491+0000 7fb2a0efa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32dec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36132/0 (socket says 172.21.15.32:36132) 2023-09-30T14:40:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.491+0000 7fb2a0efa700 1 -- 172.21.15.32:0/1915201250 learned_addr learned my addr 172.21.15.32:0/1915201250 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.491+0000 7fb2a16fb700 1 --2- 172.21.15.32:0/1915201250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7cd0 0x556da32d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.491+0000 7fb293fff700 1 --2- 172.21.15.32:0/1915201250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 0x556da32e7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.492+0000 7fb2a16fb700 1 -- 172.21.15.32:0/1915201250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 msgr2=0x556da32e7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.492+0000 7fb2a16fb700 1 --2- 172.21.15.32:0/1915201250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 0x556da32e7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.492+0000 7fb2a16fb700 1 -- 172.21.15.32:0/1915201250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 msgr2=0x556da32dec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.492+0000 7fb2a16fb700 1 --2- 172.21.15.32:0/1915201250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32dec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.492+0000 7fb2a16fb700 1 -- 172.21.15.32:0/1915201250 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556da3053d80 con 0x556da32d7cd0 2023-09-30T14:40:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.492+0000 7fb2a0efa700 1 --2- 172.21.15.32:0/1915201250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32dec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:40:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.492+0000 7fb2a16fb700 1 --2- 172.21.15.32:0/1915201250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7cd0 0x556da32d0590 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb29400c6f0 tx=0x7fb294021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf3d33dbb93db503 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.492+0000 7fb2837fe700 1 -- 172.21.15.32:0/1915201250 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29401b320 con 0x556da32d7cd0 2023-09-30T14:40:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.492+0000 7fb2837fe700 1 -- 172.21.15.32:0/1915201250 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb294099e40 con 0x556da32d7cd0 2023-09-30T14:40:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.492+0000 7fb2837fe700 1 -- 172.21.15.32:0/1915201250 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2940999a0 con 0x556da32d7cd0 2023-09-30T14:40:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.492+0000 7fb2af346b80 1 -- 172.21.15.32:0/1915201250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7cd0 msgr2=0x556da32d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.492+0000 7fb2af346b80 1 --2- 172.21.15.32:0/1915201250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7cd0 0x556da32d0590 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb29400c6f0 tx=0x7fb294021000 comp rx=0 tx=0).stop 2023-09-30T14:40:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2af346b80 1 -- 172.21.15.32:0/1915201250 shutdown_connections 2023-09-30T14:40:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2af346b80 1 --2- 172.21.15.32:0/1915201250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d7cd0 0x556da32d0590 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2af346b80 1 --2- 172.21.15.32:0/1915201250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d6200 0x556da32dec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2af346b80 1 --2- 172.21.15.32:0/1915201250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 0x556da32e7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2af346b80 1 -- 172.21.15.32:0/1915201250 >> 172.21.15.32:0/1915201250 conn(0x556da324a200 msgr2=0x556da32f29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2af346b80 1 -- 172.21.15.32:0/1915201250 shutdown_connections 2023-09-30T14:40:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2af346b80 1 -- 172.21.15.32:0/1915201250 wait complete. 2023-09-30T14:40:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2af346b80 1 -- start start 2023-09-30T14:40:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2af346b80 1 Processor -- start 2023-09-30T14:40:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2af346b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d6200 0x556da32e4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2af346b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 0x556da32e5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2af346b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d7cd0 0x556da32e6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2af346b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da32e6990 con 0x556da32d7cd0 2023-09-30T14:40:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2af346b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da32e6b00 con 0x556da32d6200 2023-09-30T14:40:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2af346b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556da32e6cc0 con 0x556da32d7310 2023-09-30T14:40:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2af346b80 10 client.0 _ll_register_callbacks cb 0x7fb298005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:40:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2a16fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d7cd0 0x556da32e6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2a0efa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d6200 0x556da32e4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb293fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 0x556da32e5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2a0efa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d6200 0x556da32e4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47614/0 (socket says 172.21.15.32:47614) 2023-09-30T14:40:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2a16fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d7cd0 0x556da32e6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36142/0 (socket says 172.21.15.32:36142) 2023-09-30T14:40:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb2a0efa700 1 -- 172.21.15.32:0/2235419774 learned_addr learned my addr 172.21.15.32:0/2235419774 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.493+0000 7fb293fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 0x556da32e5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:50522/0 (socket says 172.21.15.32:50522) 2023-09-30T14:40:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.494+0000 7fb293fff700 1 -- 172.21.15.32:0/2235419774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d6200 msgr2=0x556da32e4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.494+0000 7fb293fff700 1 --2- 172.21.15.32:0/2235419774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d6200 0x556da32e4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.494+0000 7fb293fff700 1 -- 172.21.15.32:0/2235419774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d7cd0 msgr2=0x556da32e6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.494+0000 7fb293fff700 1 --2- 172.21.15.32:0/2235419774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d7cd0 0x556da32e6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.494+0000 7fb293fff700 1 -- 172.21.15.32:0/2235419774 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb298017070 con 0x556da32d7310 2023-09-30T14:40:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.494+0000 7fb293fff700 1 --2- 172.21.15.32:0/2235419774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 0x556da32e5440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb28c000c00 tx=0x7fb28c005600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61ecedc20c28acc5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.494+0000 7fb27bfff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:40:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.494+0000 7fb27bfff700 1 -- 172.21.15.32:0/2235419774 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c01d040 con 0x556da32d7310 2023-09-30T14:40:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.494+0000 7fb2af346b80 1 -- 172.21.15.32:0/2235419774 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb298003e70 con 0x556da32d7310 2023-09-30T14:40:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.494+0000 7fb27bfff700 1 -- 172.21.15.32:0/2235419774 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb28c029af0 con 0x556da32d7310 2023-09-30T14:40:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.494+0000 7fb27bfff700 1 -- 172.21.15.32:0/2235419774 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb28c0056f0 con 0x556da32d7310 2023-09-30T14:40:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.495+0000 7fb2af346b80 10 client.25366 fetch_fsmap learned FSMap version 25 2023-09-30T14:40:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.495+0000 7fb2af346b80 1 -- 172.21.15.32:0/2235419774 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fb29c19f3d0 con 0x556da32d7310 2023-09-30T14:40:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.495+0000 7fb27bfff700 1 -- 172.21.15.32:0/2235419774 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb298003e70 con 0x556da32d7310 2023-09-30T14:40:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.495+0000 7fb27bfff700 1 -- 172.21.15.32:0/2235419774 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb28c00f7b0 con 0x556da32d7310 2023-09-30T14:40:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.495+0000 7fb2af346b80 10 client.25366 fetch_fsmap finished waiting for FSMap version 25 2023-09-30T14:40:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.495+0000 7fb2af346b80 10 client.25366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:40:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.495+0000 7fb2af346b80 20 client.25366 populate_metadata read hostname 'smithi032' 2023-09-30T14:40:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.495+0000 7fb2af346b80 1 --2- 172.21.15.32:0/2235419774 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556da3340520 0x556da33429a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.495+0000 7fb2af346b80 4 client.25366 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:40:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.495+0000 7fb2af346b80 1 -- 172.21.15.32:0/2235419774 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556da3221ad0 con 0x556da3340520 2023-09-30T14:40:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.495+0000 7fb2a0efa700 1 --2- 172.21.15.32:0/2235419774 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556da3340520 0x556da33429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.496+0000 7fb27bfff700 10 client.25366 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.496+0000 7fb2a0efa700 1 --2- 172.21.15.32:0/2235419774 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556da3340520 0x556da33429a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1ee4b603f77a3a2 server_cookie=f0d7fb0123e3da78 in_seq=0 out_seq=0 2023-09-30T14:40:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.536+0000 7fb27bfff700 1 -- 172.21.15.32:0/2235419774 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556da3221ad0 con 0x556da3340520 2023-09-30T14:40:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.536+0000 7fb27bfff700 10 client.25366 handle_command_reply: tid=0 2023-09-30T14:40:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.536+0000 7fb2af346b80 1 client.25366 shutdown 2023-09-30T14:40:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.537+0000 7fb2af346b80 1 -- 172.21.15.32:0/2235419774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 msgr2=0x556da32e5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.537+0000 7fb2af346b80 1 --2- 172.21.15.32:0/2235419774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 0x556da32e5440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb28c000c00 tx=0x7fb28c005600 comp rx=0 tx=0).stop 2023-09-30T14:40:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.537+0000 7fb2af346b80 1 -- 172.21.15.32:0/2235419774 shutdown_connections 2023-09-30T14:40:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.537+0000 7fb2af346b80 1 --2- 172.21.15.32:0/2235419774 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556da3340520 0x556da33429a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.537+0000 7fb2af346b80 1 --2- 172.21.15.32:0/2235419774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556da32d7cd0 0x556da32e6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.537+0000 7fb27bfff700 0 client.25366 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.537+0000 7fb2af346b80 1 --2- 172.21.15.32:0/2235419774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556da32d6200 0x556da32e4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.537+0000 7fb2af346b80 1 --2- 172.21.15.32:0/2235419774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556da32d7310 0x556da32e5440 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.537+0000 7fb2af346b80 1 -- 172.21.15.32:0/2235419774 >> 172.21.15.32:0/2235419774 conn(0x556da324a200 msgr2=0x556da32dec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.537+0000 7fb2af346b80 1 -- 172.21.15.32:0/2235419774 shutdown_connections 2023-09-30T14:40:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.537+0000 7fb2af346b80 1 -- 172.21.15.32:0/2235419774 wait complete. 2023-09-30T14:40:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.537+0000 7fb2af346b80 20 client.25366 trim_cache size 0 max 16384 2023-09-30T14:40:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:40:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:40:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:40:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:40:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.537+0000 7fb2a315e700 1 -- 172.21.15.32:0/3890670889 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb2840a1020 msgr2=0x7fb2840a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.537+0000 7fb2a315e700 1 --2- 172.21.15.32:0/3890670889 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb2840a1020 0x7fb2840a34e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb298011490 tx=0x7fb298009260 comp rx=0 tx=0).stop 2023-09-30T14:40:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.537+0000 7fb2a315e700 1 -- 172.21.15.32:0/3890670889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c106c00 msgr2=0x7fb29c197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.537+0000 7fb2a315e700 1 --2- 172.21.15.32:0/3890670889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c106c00 0x7fb29c197fa0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb29400ef30 tx=0x7fb29400c3a0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.538+0000 7fb2a315e700 1 -- 172.21.15.32:0/3890670889 shutdown_connections 2023-09-30T14:40:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.538+0000 7fb2a315e700 1 --2- 172.21.15.32:0/3890670889 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb2840a1020 0x7fb2840a34e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.538+0000 7fb2a315e700 1 --2- 172.21.15.32:0/3890670889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c106c00 0x7fb29c197fa0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.538+0000 7fb2a315e700 1 --2- 172.21.15.32:0/3890670889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29c1019a0 0x7fb29c196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.538+0000 7fb2a315e700 1 --2- 172.21.15.32:0/3890670889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb29c1042d0 0x7fb29c197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.538+0000 7fb2a315e700 1 -- 172.21.15.32:0/3890670889 >> 172.21.15.32:0/3890670889 conn(0x7fb29c0fb2b0 msgr2=0x7fb29c102610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.538+0000 7fb2a315e700 1 -- 172.21.15.32:0/3890670889 shutdown_connections 2023-09-30T14:40:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:01.538+0000 7fb2a315e700 1 -- 172.21.15.32:0/3890670889 wait complete. 2023-09-30T14:40:01.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:01 smithi156 ceph-mon[103579]: pgmap v339: 97 pgs: 97 active+clean; 48 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.2 MiB/s wr, 233 op/s 2023-09-30T14:40:01.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 96983 15.41 MB/sec warmup 35 sec latency 26.116 ms 2023-09-30T14:40:02.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 99724 15.46 MB/sec warmup 36 sec latency 10.340 ms 2023-09-30T14:40:03.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:03 smithi148 ceph-mon[102355]: pgmap v340: 97 pgs: 97 active+clean; 58 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 3.0 MiB/s wr, 235 op/s 2023-09-30T14:40:03.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:03 smithi032 ceph-mon[98592]: pgmap v340: 97 pgs: 97 active+clean; 58 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 3.0 MiB/s wr, 235 op/s 2023-09-30T14:40:03.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:03 smithi156 ceph-mon[103579]: pgmap v340: 97 pgs: 97 active+clean; 58 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 3.0 MiB/s wr, 235 op/s 2023-09-30T14:40:03.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 101222 15.19 MB/sec warmup 37 sec latency 56.469 ms 2023-09-30T14:40:04.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 103385 15.15 MB/sec warmup 38 sec latency 27.183 ms 2023-09-30T14:40:05.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:05 smithi148 ceph-mon[102355]: pgmap v341: 97 pgs: 97 active+clean; 64 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.4 MiB/s wr, 283 op/s 2023-09-30T14:40:05.621 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:05 smithi156 ceph-mon[103579]: pgmap v341: 97 pgs: 97 active+clean; 64 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.4 MiB/s wr, 283 op/s 2023-09-30T14:40:05.622 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:05 smithi032 ceph-mon[98592]: pgmap v341: 97 pgs: 97 active+clean; 64 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.4 MiB/s wr, 283 op/s 2023-09-30T14:40:05.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 106317 15.14 MB/sec warmup 39 sec latency 13.466 ms 2023-09-30T14:40:06.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 107875 14.99 MB/sec warmup 40 sec latency 11.187 ms 2023-09-30T14:40:07.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:07 smithi148 ceph-mon[102355]: pgmap v342: 97 pgs: 97 active+clean; 65 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.9 MiB/s wr, 327 op/s 2023-09-30T14:40:07.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:07 smithi032 ceph-mon[98592]: pgmap v342: 97 pgs: 97 active+clean; 65 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.9 MiB/s wr, 327 op/s 2023-09-30T14:40:07.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:07 smithi156 ceph-mon[103579]: pgmap v342: 97 pgs: 97 active+clean; 65 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.9 MiB/s wr, 327 op/s 2023-09-30T14:40:07.690 INFO:tasks.workunit.client.0.smithi032.stdout: 1 110380 14.97 MB/sec warmup 41 sec latency 10.634 ms 2023-09-30T14:40:08.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 111695 14.74 MB/sec warmup 42 sec latency 70.475 ms 2023-09-30T14:40:09.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:09 smithi148 ceph-mon[102355]: pgmap v343: 97 pgs: 97 active+clean; 66 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 299 op/s 2023-09-30T14:40:09.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:09 smithi032 ceph-mon[98592]: pgmap v343: 97 pgs: 97 active+clean; 66 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 299 op/s 2023-09-30T14:40:09.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:09 smithi156 ceph-mon[103579]: pgmap v343: 97 pgs: 97 active+clean; 66 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 299 op/s 2023-09-30T14:40:09.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 113990 14.72 MB/sec warmup 43 sec latency 10.629 ms 2023-09-30T14:40:10.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 116547 14.67 MB/sec warmup 44 sec latency 27.221 ms 2023-09-30T14:40:11.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:11 smithi148 ceph-mon[102355]: pgmap v344: 97 pgs: 97 active+clean; 67 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 311 op/s 2023-09-30T14:40:11.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:11 smithi032 ceph-mon[98592]: pgmap v344: 97 pgs: 97 active+clean; 67 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 311 op/s 2023-09-30T14:40:11.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:11 smithi156 ceph-mon[103579]: pgmap v344: 97 pgs: 97 active+clean; 67 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 311 op/s 2023-09-30T14:40:11.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 118023 14.51 MB/sec warmup 45 sec latency 13.627 ms 2023-09-30T14:40:12.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 120878 14.58 MB/sec warmup 46 sec latency 11.110 ms 2023-09-30T14:40:13.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:13 smithi148 ceph-mon[102355]: pgmap v345: 97 pgs: 97 active+clean; 67 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 337 op/s 2023-09-30T14:40:13.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:13 smithi032 ceph-mon[98592]: pgmap v345: 97 pgs: 97 active+clean; 67 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 337 op/s 2023-09-30T14:40:13.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:13 smithi156 ceph-mon[103579]: pgmap v345: 97 pgs: 97 active+clean; 67 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 337 op/s 2023-09-30T14:40:13.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 122059 14.39 MB/sec warmup 47 sec latency 130.742 ms 2023-09-30T14:40:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.652+0000 7f0a13f3f700 1 -- 172.21.15.32:0/1804590073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a0c104290 msgr2=0x7f0a0c106680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.653+0000 7f0a13f3f700 1 --2- 172.21.15.32:0/1804590073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a0c104290 0x7f0a0c106680 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009990 tx=0x7f09fc009ca0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.653+0000 7f0a13f3f700 1 -- 172.21.15.32:0/1804590073 shutdown_connections 2023-09-30T14:40:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.653+0000 7f0a13f3f700 1 --2- 172.21.15.32:0/1804590073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c106bc0 0x7f0a0c108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.653+0000 7f0a13f3f700 1 --2- 172.21.15.32:0/1804590073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a0c101960 0x7f0a0c103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.653+0000 7f0a13f3f700 1 --2- 172.21.15.32:0/1804590073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a0c104290 0x7f0a0c106680 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.653+0000 7f0a13f3f700 1 -- 172.21.15.32:0/1804590073 >> 172.21.15.32:0/1804590073 conn(0x7f0a0c0fb2f0 msgr2=0x7f0a0c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.653+0000 7f0a13f3f700 1 -- 172.21.15.32:0/1804590073 shutdown_connections 2023-09-30T14:40:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.653+0000 7f0a13f3f700 1 -- 172.21.15.32:0/1804590073 wait complete. 2023-09-30T14:40:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.653+0000 7f0a13f3f700 1 Processor -- start 2023-09-30T14:40:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.653+0000 7f0a13f3f700 1 -- start start 2023-09-30T14:40:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.653+0000 7f0a13f3f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c101960 0x7f0a0c196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a13f3f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a0c104290 0x7f0a0c197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a13f3f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a0c106bc0 0x7f0a0c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a13f3f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c198660 con 0x7f0a0c106bc0 2023-09-30T14:40:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a13f3f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c1987d0 con 0x7f0a0c104290 2023-09-30T14:40:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a13f3f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c198990 con 0x7f0a0c101960 2023-09-30T14:40:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a124dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a0c106bc0 0x7f0a0c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a114da700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a0c104290 0x7f0a0c197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a11cdb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c101960 0x7f0a0c196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a124dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a0c106bc0 0x7f0a0c197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36156/0 (socket says 172.21.15.32:36156) 2023-09-30T14:40:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a124dc700 1 -- 172.21.15.32:0/2732259701 learned_addr learned my addr 172.21.15.32:0/2732259701 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a114da700 1 -- 172.21.15.32:0/2732259701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c101960 msgr2=0x7f0a0c196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a114da700 1 --2- 172.21.15.32:0/2732259701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c101960 0x7f0a0c196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a114da700 1 -- 172.21.15.32:0/2732259701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a0c106bc0 msgr2=0x7f0a0c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a114da700 1 --2- 172.21.15.32:0/2732259701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a0c106bc0 0x7f0a0c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a114da700 1 -- 172.21.15.32:0/2732259701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc0095f0 con 0x7f0a0c104290 2023-09-30T14:40:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a114da700 1 --2- 172.21.15.32:0/2732259701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a0c104290 0x7f0a0c197220 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0098e0 tx=0x7f09fc009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.654+0000 7f0a02ffd700 1 -- 172.21.15.32:0/2732259701 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc021070 con 0x7f0a0c104290 2023-09-30T14:40:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.655+0000 7f0a02ffd700 1 -- 172.21.15.32:0/2732259701 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09fc00bec0 con 0x7f0a0c104290 2023-09-30T14:40:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.655+0000 7f0a13f3f700 1 -- 172.21.15.32:0/2732259701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c198c10 con 0x7f0a0c104290 2023-09-30T14:40:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.655+0000 7f0a02ffd700 1 -- 172.21.15.32:0/2732259701 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc00f7e0 con 0x7f0a0c104290 2023-09-30T14:40:14.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.655+0000 7f0a13f3f700 1 -- 172.21.15.32:0/2732259701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c0ff930 con 0x7f0a0c104290 2023-09-30T14:40:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.657+0000 7f0a02ffd700 1 -- 172.21.15.32:0/2732259701 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f09fc026a30 con 0x7f0a0c104290 2023-09-30T14:40:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.657+0000 7f0a02ffd700 1 --2- 172.21.15.32:0/2732259701 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f09f80a0fa0 0x7f09f80a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.658+0000 7f0a11cdb700 1 --2- 172.21.15.32:0/2732259701 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f09f80a0fa0 0x7f09f80a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.658+0000 7f0a02ffd700 1 -- 172.21.15.32:0/2732259701 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f09fc0d21c0 con 0x7f0a0c104290 2023-09-30T14:40:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.658+0000 7f0a11cdb700 1 --2- 172.21.15.32:0/2732259701 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f09f80a0fa0 0x7f09f80a3460 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009c60 tx=0x7f0a08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a20127b80 1 Processor -- start 2023-09-30T14:40:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a20127b80 1 -- start start 2023-09-30T14:40:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a20127b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 0x55c3547c3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a20127b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4310 0x55c3547c4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a20127b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4cd0 0x55c3547cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a20127b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35472b400 con 0x55c3547c4cd0 2023-09-30T14:40:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a20127b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35471cbc0 con 0x55c3547c4310 2023-09-30T14:40:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a20127b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35471a4e0 con 0x55c3547c3200 2023-09-30T14:40:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a124dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4cd0 0x55c3547cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a11cdb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 0x55c3547c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a124dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4cd0 0x55c3547cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36164/0 (socket says 172.21.15.32:36164) 2023-09-30T14:40:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a124dc700 1 -- 172.21.15.32:0/175155579 learned_addr learned my addr 172.21.15.32:0/175155579 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a114da700 1 --2- 172.21.15.32:0/175155579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4310 0x55c3547c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a124dc700 1 -- 172.21.15.32:0/175155579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 msgr2=0x55c3547c3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a124dc700 1 --2- 172.21.15.32:0/175155579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 0x55c3547c3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a124dc700 1 -- 172.21.15.32:0/175155579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4310 msgr2=0x55c3547c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a124dc700 1 --2- 172.21.15.32:0/175155579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4310 0x55c3547c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a124dc700 1 -- 172.21.15.32:0/175155579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c354778870 con 0x55c3547c4cd0 2023-09-30T14:40:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a114da700 1 --2- 172.21.15.32:0/175155579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4310 0x55c3547c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:40:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.659+0000 7f0a11cdb700 1 --2- 172.21.15.32:0/175155579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 0x55c3547c3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:40:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.660+0000 7f0a124dc700 1 --2- 172.21.15.32:0/175155579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4cd0 0x55c3547cb560 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400e5e0 tx=0x7f0a04009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c39a6b7a8fbbe63 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.660+0000 7f09f7fff700 1 -- 172.21.15.32:0/175155579 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0400e040 con 0x55c3547c4cd0 2023-09-30T14:40:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.660+0000 7f09f7fff700 1 -- 172.21.15.32:0/175155579 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a0400f040 con 0x55c3547c4cd0 2023-09-30T14:40:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.660+0000 7f09f7fff700 1 -- 172.21.15.32:0/175155579 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04004660 con 0x55c3547c4cd0 2023-09-30T14:40:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.661+0000 7f0a20127b80 1 -- 172.21.15.32:0/175155579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4cd0 msgr2=0x55c3547cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.661+0000 7f0a20127b80 1 --2- 172.21.15.32:0/175155579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4cd0 0x55c3547cb560 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400e5e0 tx=0x7f0a04009920 comp rx=0 tx=0).stop 2023-09-30T14:40:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.661+0000 7f0a20127b80 1 -- 172.21.15.32:0/175155579 shutdown_connections 2023-09-30T14:40:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.661+0000 7f0a20127b80 1 --2- 172.21.15.32:0/175155579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4cd0 0x55c3547cb560 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.661+0000 7f0a20127b80 1 --2- 172.21.15.32:0/175155579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 0x55c3547c3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.661+0000 7f0a20127b80 1 --2- 172.21.15.32:0/175155579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4310 0x55c3547c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.661+0000 7f0a20127b80 1 -- 172.21.15.32:0/175155579 >> 172.21.15.32:0/175155579 conn(0x55c354737200 msgr2=0x55c3547bdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.661+0000 7f0a20127b80 1 -- 172.21.15.32:0/175155579 shutdown_connections 2023-09-30T14:40:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.661+0000 7f0a20127b80 1 -- 172.21.15.32:0/175155579 wait complete. 2023-09-30T14:40:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.661+0000 7f0a20127b80 1 -- start start 2023-09-30T14:40:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.661+0000 7f0a20127b80 1 Processor -- start 2023-09-30T14:40:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.661+0000 7f0a20127b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 0x55c3547cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.661+0000 7f0a20127b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4310 0x55c3547cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.661+0000 7f0a20127b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4cd0 0x55c3547d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a20127b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c354715dd0 con 0x55c3547c4310 2023-09-30T14:40:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a20127b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3546b26b0 con 0x55c3547c4cd0 2023-09-30T14:40:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a20127b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3546b2350 con 0x55c3547c3200 2023-09-30T14:40:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a20127b80 10 client.0 _ll_register_callbacks cb 0x55c35450f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:40:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a114da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4310 0x55c3547cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a11cdb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 0x55c3547cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a114da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4310 0x55c3547cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36168/0 (socket says 172.21.15.32:36168) 2023-09-30T14:40:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a114da700 1 -- 172.21.15.32:0/595546137 learned_addr learned my addr 172.21.15.32:0/595546137 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a124dc700 1 --2- 172.21.15.32:0/595546137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4cd0 0x55c3547d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a114da700 1 -- 172.21.15.32:0/595546137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 msgr2=0x55c3547cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a114da700 1 --2- 172.21.15.32:0/595546137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 0x55c3547cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a114da700 1 -- 172.21.15.32:0/595546137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4cd0 msgr2=0x55c3547d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a114da700 1 --2- 172.21.15.32:0/595546137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4cd0 0x55c3547d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a114da700 1 -- 172.21.15.32:0/595546137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c0ff930 con 0x55c3547c4310 2023-09-30T14:40:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a124dc700 1 --2- 172.21.15.32:0/595546137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4cd0 0x55c3547d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:40:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a114da700 1 --2- 172.21.15.32:0/595546137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4310 0x55c3547cd340 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00b680 tx=0x7f09fc09f0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac76759a78a4e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f09f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:40:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f09f57fa700 1 -- 172.21.15.32:0/595546137 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc0d1070 con 0x55c3547c4310 2023-09-30T14:40:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a20127b80 1 -- 172.21.15.32:0/595546137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3547b6090 con 0x55c3547c4310 2023-09-30T14:40:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f09f57fa700 1 -- 172.21.15.32:0/595546137 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f09fc0a3810 con 0x55c3547c4310 2023-09-30T14:40:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f09f57fa700 1 -- 172.21.15.32:0/595546137 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09fc0a79c0 con 0x55c3547c4310 2023-09-30T14:40:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a20127b80 10 client.15696 fetch_fsmap learned FSMap version 25 2023-09-30T14:40:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.662+0000 7f0a20127b80 1 -- 172.21.15.32:0/595546137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f0a0400e2c0 con 0x55c3547c4310 2023-09-30T14:40:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.663+0000 7f09f57fa700 1 -- 172.21.15.32:0/595546137 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c3547b6090 con 0x55c3547c4310 2023-09-30T14:40:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.663+0000 7f09f57fa700 1 -- 172.21.15.32:0/595546137 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f09fc0a7650 con 0x55c3547c4310 2023-09-30T14:40:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.663+0000 7f0a20127b80 10 client.15696 fetch_fsmap finished waiting for FSMap version 25 2023-09-30T14:40:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.663+0000 7f0a20127b80 10 client.15696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:40:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.663+0000 7f0a20127b80 20 client.15696 populate_metadata read hostname 'smithi032' 2023-09-30T14:40:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.663+0000 7f0a20127b80 1 --2- 172.21.15.32:0/595546137 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c3547d8860 0x55c3547dac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.663+0000 7f0a20127b80 4 client.15696 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:40:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.663+0000 7f0a20127b80 1 -- 172.21.15.32:0/595546137 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c35470edb0 con 0x55c3547d8860 2023-09-30T14:40:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.663+0000 7f0a11cdb700 1 --2- 172.21.15.32:0/595546137 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c3547d8860 0x55c3547dac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.663+0000 7f09f57fa700 10 client.15696 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.663+0000 7f0a11cdb700 1 --2- 172.21.15.32:0/595546137 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c3547d8860 0x55c3547dac80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfa31a899bbdbfc7 server_cookie=1b5d3ccb1f10130c in_seq=0 out_seq=0 2023-09-30T14:40:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.664+0000 7f09f57fa700 1 -- 172.21.15.32:0/595546137 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c35470edb0 con 0x55c3547d8860 2023-09-30T14:40:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.664+0000 7f09f57fa700 10 client.15696 handle_command_reply: tid=0 2023-09-30T14:40:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.664+0000 7f0a20127b80 1 client.15696 shutdown 2023-09-30T14:40:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.664+0000 7f0a20127b80 1 -- 172.21.15.32:0/595546137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4310 msgr2=0x55c3547cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.664+0000 7f0a20127b80 1 --2- 172.21.15.32:0/595546137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4310 0x55c3547cd340 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00b680 tx=0x7f09fc09f0a0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.664+0000 7f0a20127b80 1 -- 172.21.15.32:0/595546137 shutdown_connections 2023-09-30T14:40:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.664+0000 7f0a20127b80 1 --2- 172.21.15.32:0/595546137 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c3547d8860 0x55c3547dac80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.664+0000 7f0a20127b80 1 --2- 172.21.15.32:0/595546137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4cd0 0x55c3547d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.664+0000 7f0a20127b80 1 --2- 172.21.15.32:0/595546137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 0x55c3547cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.664+0000 7f09f57fa700 0 client.15696 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.664+0000 7f0a20127b80 1 --2- 172.21.15.32:0/595546137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4310 0x55c3547cd340 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.664+0000 7f0a20127b80 1 -- 172.21.15.32:0/595546137 >> 172.21.15.32:0/595546137 conn(0x55c354737200 msgr2=0x55c3547bd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.664+0000 7f0a20127b80 1 -- 172.21.15.32:0/595546137 shutdown_connections 2023-09-30T14:40:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.664+0000 7f0a20127b80 1 -- 172.21.15.32:0/595546137 wait complete. 2023-09-30T14:40:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.664+0000 7f0a20127b80 20 client.15696 trim_cache size 0 max 16384 2023-09-30T14:40:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.679+0000 7f0a20127b80 1 Processor -- start 2023-09-30T14:40:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.679+0000 7f0a20127b80 1 -- start start 2023-09-30T14:40:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a20127b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c3200 0x55c3547cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a20127b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c4310 0x55c3547d4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a20127b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4cd0 0x55c3547cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a20127b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3545968c0 con 0x55c3547c3200 2023-09-30T14:40:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a20127b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3547d8680 con 0x55c3547c4cd0 2023-09-30T14:40:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a20127b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3547c40e0 con 0x55c3547c4310 2023-09-30T14:40:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a11cdb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c3200 0x55c3547cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a11cdb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c3200 0x55c3547cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36174/0 (socket says 172.21.15.32:36174) 2023-09-30T14:40:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a11cdb700 1 -- 172.21.15.32:0/325167441 learned_addr learned my addr 172.21.15.32:0/325167441 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a114da700 1 --2- 172.21.15.32:0/325167441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c4310 0x55c3547d4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a124dc700 1 --2- 172.21.15.32:0/325167441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4cd0 0x55c3547cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a11cdb700 1 -- 172.21.15.32:0/325167441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c4310 msgr2=0x55c3547d4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a11cdb700 1 --2- 172.21.15.32:0/325167441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c4310 0x55c3547d4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a11cdb700 1 -- 172.21.15.32:0/325167441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4cd0 msgr2=0x55c3547cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a11cdb700 1 --2- 172.21.15.32:0/325167441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4cd0 0x55c3547cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a11cdb700 1 -- 172.21.15.32:0/325167441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c354540d80 con 0x55c3547c3200 2023-09-30T14:40:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a124dc700 1 --2- 172.21.15.32:0/325167441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4cd0 0x55c3547cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:40:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a11cdb700 1 --2- 172.21.15.32:0/325167441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c3200 0x55c3547cc4b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x55c3547db590 tx=0x7f0a08014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438021fb946ed109 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a00ff9700 1 -- 172.21.15.32:0/325167441 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08022070 con 0x55c3547c3200 2023-09-30T14:40:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a00ff9700 1 -- 172.21.15.32:0/325167441 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a08023040 con 0x55c3547c3200 2023-09-30T14:40:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.680+0000 7f0a00ff9700 1 -- 172.21.15.32:0/325167441 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08029820 con 0x55c3547c3200 2023-09-30T14:40:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.681+0000 7f0a20127b80 1 -- 172.21.15.32:0/325167441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c3200 msgr2=0x55c3547cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.681+0000 7f0a20127b80 1 --2- 172.21.15.32:0/325167441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c3200 0x55c3547cc4b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x55c3547db590 tx=0x7f0a08014000 comp rx=0 tx=0).stop 2023-09-30T14:40:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.681+0000 7f0a20127b80 1 -- 172.21.15.32:0/325167441 shutdown_connections 2023-09-30T14:40:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.681+0000 7f0a20127b80 1 --2- 172.21.15.32:0/325167441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4cd0 0x55c3547cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.681+0000 7f0a20127b80 1 --2- 172.21.15.32:0/325167441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c3200 0x55c3547cc4b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.681+0000 7f0a20127b80 1 --2- 172.21.15.32:0/325167441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c4310 0x55c3547d4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.681+0000 7f0a20127b80 1 -- 172.21.15.32:0/325167441 >> 172.21.15.32:0/325167441 conn(0x55c354737200 msgr2=0x55c3547df5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.681+0000 7f0a20127b80 1 -- 172.21.15.32:0/325167441 shutdown_connections 2023-09-30T14:40:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.681+0000 7f0a20127b80 1 -- 172.21.15.32:0/325167441 wait complete. 2023-09-30T14:40:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.681+0000 7f0a20127b80 1 -- start start 2023-09-30T14:40:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.682+0000 7f0a20127b80 1 Processor -- start 2023-09-30T14:40:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.682+0000 7f0a20127b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 0x55c3547d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.682+0000 7f0a20127b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4310 0x55c3547d2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.682+0000 7f0a20127b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4cd0 0x55c3547d3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.682+0000 7f0a20127b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3547d3ba0 con 0x55c3547c4cd0 2023-09-30T14:40:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.682+0000 7f0a20127b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3547d3d10 con 0x55c3547c4310 2023-09-30T14:40:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.682+0000 7f0a20127b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3547d3ed0 con 0x55c3547c3200 2023-09-30T14:40:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.682+0000 7f0a20127b80 10 client.0 _ll_register_callbacks cb 0x7f09fc09f4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:40:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.682+0000 7f0a11cdb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 0x55c3547d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.682+0000 7f0a11cdb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 0x55c3547d2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:50562/0 (socket says 172.21.15.32:50562) 2023-09-30T14:40:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.682+0000 7f0a11cdb700 1 -- 172.21.15.32:0/1101506253 learned_addr learned my addr 172.21.15.32:0/1101506253 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.682+0000 7f0a124dc700 1 --2- 172.21.15.32:0/1101506253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4cd0 0x55c3547d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.683+0000 7f0a114da700 1 --2- 172.21.15.32:0/1101506253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4310 0x55c3547d2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.683+0000 7f0a11cdb700 1 -- 172.21.15.32:0/1101506253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4310 msgr2=0x55c3547d2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.683+0000 7f0a11cdb700 1 --2- 172.21.15.32:0/1101506253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4310 0x55c3547d2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.683+0000 7f0a11cdb700 1 -- 172.21.15.32:0/1101506253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4cd0 msgr2=0x55c3547d3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.683+0000 7f0a11cdb700 1 --2- 172.21.15.32:0/1101506253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4cd0 0x55c3547d3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.683+0000 7f0a11cdb700 1 -- 172.21.15.32:0/1101506253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a08009e60 con 0x55c3547c3200 2023-09-30T14:40:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.683+0000 7f0a124dc700 1 --2- 172.21.15.32:0/1101506253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4cd0 0x55c3547d3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:40:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.683+0000 7f0a11cdb700 1 --2- 172.21.15.32:0/1101506253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 0x55c3547d2110 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0a080097c0 tx=0x7f0a0801dd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adc511d7931b2e3a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.683+0000 7f09f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:40:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.683+0000 7f09f57fa700 1 -- 172.21.15.32:0/1101506253 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08022050 con 0x55c3547c3200 2023-09-30T14:40:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.683+0000 7f09f57fa700 1 -- 172.21.15.32:0/1101506253 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0a08005450 con 0x55c3547c3200 2023-09-30T14:40:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.683+0000 7f0a20127b80 1 -- 172.21.15.32:0/1101506253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a0400f040 con 0x55c3547c3200 2023-09-30T14:40:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.683+0000 7f09f57fa700 1 -- 172.21.15.32:0/1101506253 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a08005610 con 0x55c3547c3200 2023-09-30T14:40:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.683+0000 7f0a20127b80 10 client.25384 fetch_fsmap learned FSMap version 25 2023-09-30T14:40:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.683+0000 7f0a20127b80 1 -- 172.21.15.32:0/1101506253 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c354529c00 con 0x55c3547c3200 2023-09-30T14:40:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.684+0000 7f09f57fa700 1 -- 172.21.15.32:0/1101506253 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a0400f040 con 0x55c3547c3200 2023-09-30T14:40:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.684+0000 7f09f57fa700 1 -- 172.21.15.32:0/1101506253 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0a08029600 con 0x55c3547c3200 2023-09-30T14:40:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.684+0000 7f0a20127b80 10 client.25384 fetch_fsmap finished waiting for FSMap version 25 2023-09-30T14:40:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.684+0000 7f0a20127b80 10 client.25384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:40:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.684+0000 7f0a20127b80 20 client.25384 populate_metadata read hostname 'smithi032' 2023-09-30T14:40:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.684+0000 7f0a20127b80 1 --2- 172.21.15.32:0/1101506253 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c35482d4e0 0x55c35482f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.684+0000 7f0a20127b80 4 client.25384 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:40:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.684+0000 7f0a20127b80 1 -- 172.21.15.32:0/1101506253 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c35470ead0 con 0x55c35482d4e0 2023-09-30T14:40:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.684+0000 7f0a114da700 1 --2- 172.21.15.32:0/1101506253 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c35482d4e0 0x55c35482f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.684+0000 7f0a114da700 1 --2- 172.21.15.32:0/1101506253 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c35482d4e0 0x55c35482f900 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87a9bfbb143073c1 server_cookie=ad6edccb5d0ab46a in_seq=0 out_seq=0 2023-09-30T14:40:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.684+0000 7f09f57fa700 10 client.25384 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:14.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 124560 14.38 MB/sec warmup 48 sec latency 15.831 ms 2023-09-30T14:40:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.721+0000 7f09f57fa700 1 -- 172.21.15.32:0/1101506253 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c35470ead0 con 0x55c35482d4e0 2023-09-30T14:40:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.721+0000 7f09f57fa700 10 client.25384 handle_command_reply: tid=0 2023-09-30T14:40:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.721+0000 7f0a20127b80 1 client.25384 shutdown 2023-09-30T14:40:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.721+0000 7f0a20127b80 1 -- 172.21.15.32:0/1101506253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 msgr2=0x55c3547d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.721+0000 7f0a20127b80 1 --2- 172.21.15.32:0/1101506253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 0x55c3547d2110 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0a080097c0 tx=0x7f0a0801dd00 comp rx=0 tx=0).stop 2023-09-30T14:40:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.721+0000 7f0a20127b80 1 -- 172.21.15.32:0/1101506253 shutdown_connections 2023-09-30T14:40:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.721+0000 7f0a20127b80 1 --2- 172.21.15.32:0/1101506253 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c35482d4e0 0x55c35482f900 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.721+0000 7f0a20127b80 1 --2- 172.21.15.32:0/1101506253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c3547c4cd0 0x55c3547d3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.721+0000 7f0a20127b80 1 --2- 172.21.15.32:0/1101506253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3547c3200 0x55c3547d2110 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.721+0000 7f0a20127b80 1 --2- 172.21.15.32:0/1101506253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c3547c4310 0x55c3547d2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.721+0000 7f09f57fa700 0 client.25384 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.721+0000 7f0a20127b80 1 -- 172.21.15.32:0/1101506253 >> 172.21.15.32:0/1101506253 conn(0x55c354737200 msgr2=0x55c3547cb050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.721+0000 7f0a20127b80 1 -- 172.21.15.32:0/1101506253 shutdown_connections 2023-09-30T14:40:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.721+0000 7f0a20127b80 1 -- 172.21.15.32:0/1101506253 wait complete. 2023-09-30T14:40:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.721+0000 7f0a20127b80 20 client.25384 trim_cache size 0 max 16384 2023-09-30T14:40:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:40:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:40:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:40:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:40:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.722+0000 7f0a13f3f700 1 -- 172.21.15.32:0/2732259701 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f09f80a0fa0 msgr2=0x7f09f80a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.722+0000 7f0a13f3f700 1 --2- 172.21.15.32:0/2732259701 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f09f80a0fa0 0x7f09f80a3460 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009c60 tx=0x7f0a08009260 comp rx=0 tx=0).stop 2023-09-30T14:40:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.722+0000 7f0a13f3f700 1 -- 172.21.15.32:0/2732259701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a0c104290 msgr2=0x7f0a0c197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.722+0000 7f0a13f3f700 1 --2- 172.21.15.32:0/2732259701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a0c104290 0x7f0a0c197220 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0098e0 tx=0x7f09fc009580 comp rx=0 tx=0).stop 2023-09-30T14:40:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.722+0000 7f0a13f3f700 1 -- 172.21.15.32:0/2732259701 shutdown_connections 2023-09-30T14:40:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.722+0000 7f0a13f3f700 1 --2- 172.21.15.32:0/2732259701 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f09f80a0fa0 0x7f09f80a3460 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.722+0000 7f0a13f3f700 1 --2- 172.21.15.32:0/2732259701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a0c106bc0 0x7f0a0c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.722+0000 7f0a13f3f700 1 --2- 172.21.15.32:0/2732259701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c101960 0x7f0a0c196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.722+0000 7f0a13f3f700 1 --2- 172.21.15.32:0/2732259701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a0c104290 0x7f0a0c197220 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.722+0000 7f0a13f3f700 1 -- 172.21.15.32:0/2732259701 >> 172.21.15.32:0/2732259701 conn(0x7f0a0c0fb2f0 msgr2=0x7f0a0c1025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.722+0000 7f0a13f3f700 1 -- 172.21.15.32:0/2732259701 shutdown_connections 2023-09-30T14:40:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:14.722+0000 7f0a13f3f700 1 -- 172.21.15.32:0/2732259701 wait complete. 2023-09-30T14:40:15.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:15 smithi148 ceph-mon[102355]: pgmap v346: 97 pgs: 97 active+clean; 70 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 384 op/s 2023-09-30T14:40:15.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:15 smithi032 ceph-mon[98592]: pgmap v346: 97 pgs: 97 active+clean; 70 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 384 op/s 2023-09-30T14:40:15.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:15 smithi156 ceph-mon[103579]: pgmap v346: 97 pgs: 97 active+clean; 70 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 384 op/s 2023-09-30T14:40:15.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 126740 14.25 MB/sec warmup 49 sec latency 33.031 ms 2023-09-30T14:40:16.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 128535 14.20 MB/sec warmup 50 sec latency 11.472 ms 2023-09-30T14:40:17.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:17 smithi148 ceph-mon[102355]: pgmap v347: 97 pgs: 97 active+clean; 71 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 360 op/s 2023-09-30T14:40:17.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:17 smithi032 ceph-mon[98592]: pgmap v347: 97 pgs: 97 active+clean; 71 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 360 op/s 2023-09-30T14:40:17.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:17 smithi156 ceph-mon[103579]: pgmap v347: 97 pgs: 97 active+clean; 71 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 360 op/s 2023-09-30T14:40:17.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 131497 14.28 MB/sec warmup 51 sec latency 13.858 ms 2023-09-30T14:40:18.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 132748 14.11 MB/sec warmup 52 sec latency 79.575 ms 2023-09-30T14:40:19.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:19 smithi148 ceph-mon[102355]: pgmap v348: 97 pgs: 97 active+clean; 72 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 356 op/s 2023-09-30T14:40:19.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:19 smithi032 ceph-mon[98592]: pgmap v348: 97 pgs: 97 active+clean; 72 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 356 op/s 2023-09-30T14:40:19.554 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:19 smithi156 ceph-mon[103579]: pgmap v348: 97 pgs: 97 active+clean; 72 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 356 op/s 2023-09-30T14:40:19.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 135194 14.11 MB/sec warmup 53 sec latency 28.732 ms 2023-09-30T14:40:20.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 137422 13.99 MB/sec warmup 54 sec latency 14.637 ms 2023-09-30T14:40:21.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:21 smithi148 ceph-mon[102355]: pgmap v349: 97 pgs: 97 active+clean; 75 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 325 op/s 2023-09-30T14:40:21.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:21 smithi032 ceph-mon[98592]: pgmap v349: 97 pgs: 97 active+clean; 75 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 325 op/s 2023-09-30T14:40:21.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:21 smithi156 ceph-mon[103579]: pgmap v349: 97 pgs: 97 active+clean; 75 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 325 op/s 2023-09-30T14:40:21.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 138823 13.94 MB/sec warmup 55 sec latency 182.343 ms 2023-09-30T14:40:22.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:22 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:40:22.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:22 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:40:22.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:22 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:40:22.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:22 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:40:22.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:22 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:40:22.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:22 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:40:22.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:22 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:40:22.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:22 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:40:22.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:22 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:40:22.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:22 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:40:22.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:22 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:40:22.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:22 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:40:22.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 141381 13.92 MB/sec warmup 56 sec latency 12.262 ms 2023-09-30T14:40:23.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:23 smithi148 ceph-mon[102355]: pgmap v350: 97 pgs: 97 active+clean; 77 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 349 op/s 2023-09-30T14:40:23.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:23 smithi032 ceph-mon[98592]: pgmap v350: 97 pgs: 97 active+clean; 77 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 349 op/s 2023-09-30T14:40:23.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:23 smithi156 ceph-mon[103579]: pgmap v350: 97 pgs: 97 active+clean; 77 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 349 op/s 2023-09-30T14:40:23.691 INFO:tasks.workunit.client.0.smithi032.stdout: 1 142785 13.80 MB/sec warmup 57 sec latency 51.672 ms 2023-09-30T14:40:24.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 145695 13.87 MB/sec warmup 58 sec latency 16.344 ms 2023-09-30T14:40:25.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:25 smithi148 ceph-mon[102355]: pgmap v351: 97 pgs: 97 active+clean; 77 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 351 op/s 2023-09-30T14:40:25.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:25 smithi032 ceph-mon[98592]: pgmap v351: 97 pgs: 97 active+clean; 77 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 351 op/s 2023-09-30T14:40:25.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:25 smithi156 ceph-mon[103579]: pgmap v351: 97 pgs: 97 active+clean; 77 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 351 op/s 2023-09-30T14:40:25.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 147602 13.74 MB/sec warmup 59 sec latency 13.930 ms 2023-09-30T14:40:26.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 149487 13.74 MB/sec warmup 60 sec latency 42.739 ms 2023-09-30T14:40:27.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:27 smithi148 ceph-mon[102355]: pgmap v352: 97 pgs: 97 active+clean; 78 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 321 op/s 2023-09-30T14:40:27.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:27 smithi032 ceph-mon[98592]: pgmap v352: 97 pgs: 97 active+clean; 78 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 321 op/s 2023-09-30T14:40:27.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:27 smithi156 ceph-mon[103579]: pgmap v352: 97 pgs: 97 active+clean; 78 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 321 op/s 2023-09-30T14:40:27.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 152490 13.80 MB/sec warmup 61 sec latency 18.462 ms 2023-09-30T14:40:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.833+0000 7f28e8379700 1 -- 172.21.15.32:0/1992464163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0100f80 msgr2=0x7f28e01013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.833+0000 7f28e8379700 1 --2- 172.21.15.32:0/1992464163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0100f80 0x7f28e01013e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f28d00099c0 tx=0x7f28d0009cd0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.833+0000 7f28e8379700 1 -- 172.21.15.32:0/1992464163 shutdown_connections 2023-09-30T14:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.833+0000 7f28e8379700 1 --2- 172.21.15.32:0/1992464163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28e0101920 0x7f28e01081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.833+0000 7f28e8379700 1 --2- 172.21.15.32:0/1992464163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28e00ffd90 0x7f28e01001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.833+0000 7f28e8379700 1 --2- 172.21.15.32:0/1992464163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0100f80 0x7f28e01013e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.833+0000 7f28e8379700 1 -- 172.21.15.32:0/1992464163 >> 172.21.15.32:0/1992464163 conn(0x7f28e00fb320 msgr2=0x7f28e00fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.833+0000 7f28e8379700 1 -- 172.21.15.32:0/1992464163 shutdown_connections 2023-09-30T14:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.833+0000 7f28e8379700 1 -- 172.21.15.32:0/1992464163 wait complete. 2023-09-30T14:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.833+0000 7f28e8379700 1 Processor -- start 2023-09-30T14:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.833+0000 7f28e8379700 1 -- start start 2023-09-30T14:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e8379700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28e00ffd90 0x7f28e0102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e8379700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0100f80 0x7f28e01028a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e8379700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28e0101920 0x7f28e0102de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e8379700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e01a0bc0 con 0x7f28e0100f80 2023-09-30T14:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e8379700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e01a0d00 con 0x7f28e0101920 2023-09-30T14:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e8379700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e01a0e40 con 0x7f28e00ffd90 2023-09-30T14:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e5914700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0100f80 0x7f28e01028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e5914700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0100f80 0x7f28e01028a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36196/0 (socket says 172.21.15.32:36196) 2023-09-30T14:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e5914700 1 -- 172.21.15.32:0/2721286334 learned_addr learned my addr 172.21.15.32:0/2721286334 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e6115700 1 --2- 172.21.15.32:0/2721286334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28e00ffd90 0x7f28e0102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e6916700 1 --2- 172.21.15.32:0/2721286334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28e0101920 0x7f28e0102de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e5914700 1 -- 172.21.15.32:0/2721286334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28e00ffd90 msgr2=0x7f28e0102360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e5914700 1 --2- 172.21.15.32:0/2721286334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28e00ffd90 0x7f28e0102360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e5914700 1 -- 172.21.15.32:0/2721286334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28e0101920 msgr2=0x7f28e0102de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e5914700 1 --2- 172.21.15.32:0/2721286334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28e0101920 0x7f28e0102de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e5914700 1 -- 172.21.15.32:0/2721286334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d00095f0 con 0x7f28e0100f80 2023-09-30T14:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e5914700 1 --2- 172.21.15.32:0/2721286334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0100f80 0x7f28e01028a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009fd0 tx=0x7f28d0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28d77fe700 1 -- 172.21.15.32:0/2721286334 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d001d070 con 0x7f28e0100f80 2023-09-30T14:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28d77fe700 1 -- 172.21.15.32:0/2721286334 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28d0005030 con 0x7f28e0100f80 2023-09-30T14:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28e8379700 1 -- 172.21.15.32:0/2721286334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e01a1070 con 0x7f28e0100f80 2023-09-30T14:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.834+0000 7f28d77fe700 1 -- 172.21.15.32:0/2721286334 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0004a90 con 0x7f28e0100f80 2023-09-30T14:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.835+0000 7f28e8379700 1 -- 172.21.15.32:0/2721286334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e01a14f0 con 0x7f28e0100f80 2023-09-30T14:40:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.836+0000 7f28d77fe700 1 -- 172.21.15.32:0/2721286334 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f28d00051a0 con 0x7f28e0100f80 2023-09-30T14:40:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.837+0000 7f28d77fe700 1 --2- 172.21.15.32:0/2721286334 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f28cc0a1020 0x7f28cc0a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.837+0000 7f28d77fe700 1 -- 172.21.15.32:0/2721286334 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f28d00cd440 con 0x7f28e0100f80 2023-09-30T14:40:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.837+0000 7f28e6115700 1 --2- 172.21.15.32:0/2721286334 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f28cc0a1020 0x7f28cc0a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.838+0000 7f28e6115700 1 --2- 172.21.15.32:0/2721286334 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f28cc0a1020 0x7f28cc0a34e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f28e0103c70 tx=0x7f28dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.838+0000 7f28f4561b80 1 Processor -- start 2023-09-30T14:40:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.838+0000 7f28f4561b80 1 -- start start 2023-09-30T14:40:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.838+0000 7f28f4561b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f3200 0x5606219f3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.838+0000 7f28f4561b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4310 0x5606219f4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.838+0000 7f28f4561b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f4cd0 0x5606219fb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.838+0000 7f28f4561b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062195b400 con 0x5606219f4310 2023-09-30T14:40:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.838+0000 7f28f4561b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062194cbc0 con 0x5606219f3200 2023-09-30T14:40:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.838+0000 7f28f4561b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062194a4e0 con 0x5606219f4cd0 2023-09-30T14:40:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.839+0000 7f28e5914700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4310 0x5606219f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.839+0000 7f28e5914700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4310 0x5606219f4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36204/0 (socket says 172.21.15.32:36204) 2023-09-30T14:40:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.839+0000 7f28e5914700 1 -- 172.21.15.32:0/3673775703 learned_addr learned my addr 172.21.15.32:0/3673775703 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.839+0000 7f28e6916700 1 --2- 172.21.15.32:0/3673775703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f4cd0 0x5606219fb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.839+0000 7f28e6115700 1 --2- 172.21.15.32:0/3673775703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f3200 0x5606219f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.839+0000 7f28e6916700 1 -- 172.21.15.32:0/3673775703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f3200 msgr2=0x5606219f3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.839+0000 7f28e6916700 1 --2- 172.21.15.32:0/3673775703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f3200 0x5606219f3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.839+0000 7f28e6916700 1 -- 172.21.15.32:0/3673775703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4310 msgr2=0x5606219f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.839+0000 7f28e6916700 1 --2- 172.21.15.32:0/3673775703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4310 0x5606219f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.839+0000 7f28e6916700 1 -- 172.21.15.32:0/3673775703 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606219a8870 con 0x5606219f4cd0 2023-09-30T14:40:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.839+0000 7f28e5914700 1 --2- 172.21.15.32:0/3673775703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4310 0x5606219f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:40:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.839+0000 7f28e6916700 1 --2- 172.21.15.32:0/3673775703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f4cd0 0x5606219fb560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f28d8011580 tx=0x7f28d8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe55299398e897cd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.839+0000 7f28d4d82700 1 -- 172.21.15.32:0/3673775703 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d800e040 con 0x5606219f4cd0 2023-09-30T14:40:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.839+0000 7f28d4d82700 1 -- 172.21.15.32:0/3673775703 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28d800f040 con 0x5606219f4cd0 2023-09-30T14:40:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.839+0000 7f28d4d82700 1 -- 172.21.15.32:0/3673775703 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d80043b0 con 0x5606219f4cd0 2023-09-30T14:40:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.840+0000 7f28f4561b80 1 -- 172.21.15.32:0/3673775703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f4cd0 msgr2=0x5606219fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.840+0000 7f28f4561b80 1 --2- 172.21.15.32:0/3673775703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f4cd0 0x5606219fb560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f28d8011580 tx=0x7f28d8009920 comp rx=0 tx=0).stop 2023-09-30T14:40:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.840+0000 7f28f4561b80 1 -- 172.21.15.32:0/3673775703 shutdown_connections 2023-09-30T14:40:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.840+0000 7f28f4561b80 1 --2- 172.21.15.32:0/3673775703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f4cd0 0x5606219fb560 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.840+0000 7f28f4561b80 1 --2- 172.21.15.32:0/3673775703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f3200 0x5606219f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.840+0000 7f28f4561b80 1 --2- 172.21.15.32:0/3673775703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4310 0x5606219f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.840+0000 7f28f4561b80 1 -- 172.21.15.32:0/3673775703 >> 172.21.15.32:0/3673775703 conn(0x560621967200 msgr2=0x5606219edef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.840+0000 7f28f4561b80 1 -- 172.21.15.32:0/3673775703 shutdown_connections 2023-09-30T14:40:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.840+0000 7f28f4561b80 1 -- 172.21.15.32:0/3673775703 wait complete. 2023-09-30T14:40:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28f4561b80 1 -- start start 2023-09-30T14:40:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28f4561b80 1 Processor -- start 2023-09-30T14:40:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28f4561b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f3200 0x5606219fce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28f4561b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f4310 0x5606219fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28f4561b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f4cd0 0x560621a043e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28f4561b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560621945dd0 con 0x5606219f3200 2023-09-30T14:40:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28f4561b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606218e26b0 con 0x5606219f4310 2023-09-30T14:40:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28f4561b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606218e2350 con 0x5606219f4cd0 2023-09-30T14:40:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28f4561b80 10 client.0 _ll_register_callbacks cb 0x56062173f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:40:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28e6115700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f3200 0x5606219fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28e6115700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f3200 0x5606219fce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36208/0 (socket says 172.21.15.32:36208) 2023-09-30T14:40:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28e5914700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f4310 0x5606219fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28e6115700 1 -- 172.21.15.32:0/962851019 learned_addr learned my addr 172.21.15.32:0/962851019 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28e6916700 1 --2- 172.21.15.32:0/962851019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f4cd0 0x560621a043e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28e6115700 1 -- 172.21.15.32:0/962851019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f4cd0 msgr2=0x560621a043e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28e6115700 1 --2- 172.21.15.32:0/962851019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f4cd0 0x560621a043e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28e6115700 1 -- 172.21.15.32:0/962851019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f4310 msgr2=0x5606219fd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28e6115700 1 --2- 172.21.15.32:0/962851019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f4310 0x5606219fd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28e6115700 1 -- 172.21.15.32:0/962851019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e01a14f0 con 0x5606219f3200 2023-09-30T14:40:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.841+0000 7f28e5914700 1 --2- 172.21.15.32:0/962851019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f4310 0x5606219fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:40:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.842+0000 7f28e6115700 1 --2- 172.21.15.32:0/962851019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f3200 0x5606219fce60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f28dc01d8f0 tx=0x7f28dc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802e182ca3b4117 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.842+0000 7f28c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:40:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.842+0000 7f28c9ffb700 1 -- 172.21.15.32:0/962851019 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc022070 con 0x5606219f3200 2023-09-30T14:40:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.842+0000 7f28c9ffb700 1 -- 172.21.15.32:0/962851019 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f28dc004240 con 0x5606219f3200 2023-09-30T14:40:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.842+0000 7f28f4561b80 1 -- 172.21.15.32:0/962851019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606219e6090 con 0x5606219f3200 2023-09-30T14:40:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.842+0000 7f28c9ffb700 1 -- 172.21.15.32:0/962851019 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28dc02d030 con 0x5606219f3200 2023-09-30T14:40:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.842+0000 7f28f4561b80 10 client.15714 fetch_fsmap learned FSMap version 25 2023-09-30T14:40:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.842+0000 7f28f4561b80 1 -- 172.21.15.32:0/962851019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f28d8011230 con 0x5606219f3200 2023-09-30T14:40:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.842+0000 7f28c9ffb700 1 -- 172.21.15.32:0/962851019 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5606219e6090 con 0x5606219f3200 2023-09-30T14:40:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.842+0000 7f28c9ffb700 1 -- 172.21.15.32:0/962851019 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f28dc0275c0 con 0x5606219f3200 2023-09-30T14:40:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.842+0000 7f28f4561b80 10 client.15714 fetch_fsmap finished waiting for FSMap version 25 2023-09-30T14:40:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.842+0000 7f28f4561b80 10 client.15714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:40:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.842+0000 7f28f4561b80 20 client.15714 populate_metadata read hostname 'smithi032' 2023-09-30T14:40:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.842+0000 7f28f4561b80 1 --2- 172.21.15.32:0/962851019 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560621a08d10 0x560621a0b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.842+0000 7f28f4561b80 4 client.15714 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:40:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.842+0000 7f28f4561b80 1 -- 172.21.15.32:0/962851019 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56062193edb0 con 0x560621a08d10 2023-09-30T14:40:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.843+0000 7f28e5914700 1 --2- 172.21.15.32:0/962851019 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560621a08d10 0x560621a0b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.843+0000 7f28c9ffb700 10 client.15714 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.843+0000 7f28e5914700 1 --2- 172.21.15.32:0/962851019 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560621a08d10 0x560621a0b130 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11b18d25788d5124 server_cookie=2ab51a3184c195ea in_seq=0 out_seq=0 2023-09-30T14:40:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.843+0000 7f28c9ffb700 1 -- 172.21.15.32:0/962851019 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56062193edb0 con 0x560621a08d10 2023-09-30T14:40:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.843+0000 7f28c9ffb700 10 client.15714 handle_command_reply: tid=0 2023-09-30T14:40:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.844+0000 7f28f4561b80 1 client.15714 shutdown 2023-09-30T14:40:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.844+0000 7f28f4561b80 1 -- 172.21.15.32:0/962851019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f3200 msgr2=0x5606219fce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.844+0000 7f28f4561b80 1 --2- 172.21.15.32:0/962851019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f3200 0x5606219fce60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f28dc01d8f0 tx=0x7f28dc0047c0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.844+0000 7f28f4561b80 1 -- 172.21.15.32:0/962851019 shutdown_connections 2023-09-30T14:40:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.844+0000 7f28f4561b80 1 --2- 172.21.15.32:0/962851019 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560621a08d10 0x560621a0b130 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.844+0000 7f28f4561b80 1 --2- 172.21.15.32:0/962851019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f4cd0 0x560621a043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.844+0000 7f28f4561b80 1 --2- 172.21.15.32:0/962851019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f3200 0x5606219fce60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.844+0000 7f28f4561b80 1 --2- 172.21.15.32:0/962851019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f4310 0x5606219fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.844+0000 7f28c9ffb700 0 client.15714 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.844+0000 7f28f4561b80 1 -- 172.21.15.32:0/962851019 >> 172.21.15.32:0/962851019 conn(0x560621967200 msgr2=0x5606219ed590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.844+0000 7f28f4561b80 1 -- 172.21.15.32:0/962851019 shutdown_connections 2023-09-30T14:40:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.844+0000 7f28f4561b80 1 -- 172.21.15.32:0/962851019 wait complete. 2023-09-30T14:40:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.844+0000 7f28f4561b80 20 client.15714 trim_cache size 0 max 16384 2023-09-30T14:40:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.859+0000 7f28f4561b80 1 Processor -- start 2023-09-30T14:40:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.859+0000 7f28f4561b80 1 -- start start 2023-09-30T14:40:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.859+0000 7f28f4561b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f3200 0x5606219f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.859+0000 7f28f4561b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f4310 0x560621a043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.859+0000 7f28f4561b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4cd0 0x5606219ecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.859+0000 7f28f4561b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606217c68c0 con 0x5606219f4cd0 2023-09-30T14:40:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.859+0000 7f28f4561b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606219ef9a0 con 0x5606219f3200 2023-09-30T14:40:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.859+0000 7f28f4561b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606219fae60 con 0x5606219f4310 2023-09-30T14:40:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.859+0000 7f28e6916700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4cd0 0x5606219ecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.859+0000 7f28e6916700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4cd0 0x5606219ecaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36220/0 (socket says 172.21.15.32:36220) 2023-09-30T14:40:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.859+0000 7f28e6916700 1 -- 172.21.15.32:0/317986078 learned_addr learned my addr 172.21.15.32:0/317986078 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.859+0000 7f28e5914700 1 --2- 172.21.15.32:0/317986078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f4310 0x560621a043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.859+0000 7f28e6115700 1 --2- 172.21.15.32:0/317986078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f3200 0x5606219f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.860+0000 7f28e6916700 1 -- 172.21.15.32:0/317986078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f4310 msgr2=0x560621a043a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.860+0000 7f28e6916700 1 --2- 172.21.15.32:0/317986078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f4310 0x560621a043a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.860+0000 7f28e6916700 1 -- 172.21.15.32:0/317986078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f3200 msgr2=0x5606219f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.860+0000 7f28e6916700 1 --2- 172.21.15.32:0/317986078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f3200 0x5606219f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.860+0000 7f28e6916700 1 -- 172.21.15.32:0/317986078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560621770d80 con 0x5606219f4cd0 2023-09-30T14:40:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.860+0000 7f28e6916700 1 --2- 172.21.15.32:0/317986078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4cd0 0x5606219ecaf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f28d800b010 tx=0x7f28d80090d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297e56e6a4fdbffd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.860+0000 7f28d5583700 1 -- 172.21.15.32:0/317986078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d800e040 con 0x5606219f4cd0 2023-09-30T14:40:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.860+0000 7f28d5583700 1 -- 172.21.15.32:0/317986078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28d8003c00 con 0x5606219f4cd0 2023-09-30T14:40:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.860+0000 7f28d5583700 1 -- 172.21.15.32:0/317986078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d80052a0 con 0x5606219f4cd0 2023-09-30T14:40:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.860+0000 7f28f4561b80 1 -- 172.21.15.32:0/317986078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4cd0 msgr2=0x5606219ecaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.860+0000 7f28f4561b80 1 --2- 172.21.15.32:0/317986078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4cd0 0x5606219ecaf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f28d800b010 tx=0x7f28d80090d0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.860+0000 7f28f4561b80 1 -- 172.21.15.32:0/317986078 shutdown_connections 2023-09-30T14:40:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.860+0000 7f28f4561b80 1 --2- 172.21.15.32:0/317986078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4cd0 0x5606219ecaf0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.860+0000 7f28f4561b80 1 --2- 172.21.15.32:0/317986078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f3200 0x5606219f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.860+0000 7f28f4561b80 1 --2- 172.21.15.32:0/317986078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f4310 0x560621a043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.860+0000 7f28f4561b80 1 -- 172.21.15.32:0/317986078 >> 172.21.15.32:0/317986078 conn(0x560621967200 msgr2=0x5606219edec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.861+0000 7f28f4561b80 1 -- 172.21.15.32:0/317986078 shutdown_connections 2023-09-30T14:40:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.861+0000 7f28f4561b80 1 -- 172.21.15.32:0/317986078 wait complete. 2023-09-30T14:40:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.861+0000 7f28f4561b80 1 -- start start 2023-09-30T14:40:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.861+0000 7f28f4561b80 1 Processor -- start 2023-09-30T14:40:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.861+0000 7f28f4561b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f3200 0x560621a020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.861+0000 7f28f4561b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f4310 0x560621a02600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.861+0000 7f28f4561b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4cd0 0x560621a03610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.861+0000 7f28f4561b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560621a03b50 con 0x5606219f4cd0 2023-09-30T14:40:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.861+0000 7f28f4561b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560621a03cc0 con 0x5606219f4310 2023-09-30T14:40:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.861+0000 7f28f4561b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560621a03e80 con 0x5606219f3200 2023-09-30T14:40:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.861+0000 7f28f4561b80 10 client.0 _ll_register_callbacks cb 0x56062173f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:40:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.861+0000 7f28e6916700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4cd0 0x560621a03610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.861+0000 7f28e5914700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f4310 0x560621a02600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.861+0000 7f28e6916700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4cd0 0x560621a03610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36226/0 (socket says 172.21.15.32:36226) 2023-09-30T14:40:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.861+0000 7f28e6916700 1 -- 172.21.15.32:0/2453926039 learned_addr learned my addr 172.21.15.32:0/2453926039 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.861+0000 7f28e6115700 1 --2- 172.21.15.32:0/2453926039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f3200 0x560621a020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.862+0000 7f28e6916700 1 -- 172.21.15.32:0/2453926039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f3200 msgr2=0x560621a020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.862+0000 7f28e6916700 1 --2- 172.21.15.32:0/2453926039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f3200 0x560621a020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.862+0000 7f28e6916700 1 -- 172.21.15.32:0/2453926039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f4310 msgr2=0x560621a02600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.862+0000 7f28e6916700 1 --2- 172.21.15.32:0/2453926039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f4310 0x560621a02600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.862+0000 7f28e6916700 1 -- 172.21.15.32:0/2453926039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606219a8870 con 0x5606219f4cd0 2023-09-30T14:40:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.862+0000 7f28e5914700 1 --2- 172.21.15.32:0/2453926039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f4310 0x560621a02600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:40:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.862+0000 7f28e6916700 1 --2- 172.21.15.32:0/2453926039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4cd0 0x560621a03610 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009960 tx=0x7f28d8007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6537e3b3dbae5c74 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.862+0000 7f28c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:40:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.862+0000 7f28c9ffb700 1 -- 172.21.15.32:0/2453926039 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d800e040 con 0x5606219f4cd0 2023-09-30T14:40:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.862+0000 7f28c9ffb700 1 -- 172.21.15.32:0/2453926039 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f28d8005040 con 0x5606219f4cd0 2023-09-30T14:40:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.862+0000 7f28f4561b80 1 -- 172.21.15.32:0/2453926039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28d800f040 con 0x5606219f4cd0 2023-09-30T14:40:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.862+0000 7f28c9ffb700 1 -- 172.21.15.32:0/2453926039 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28d8005260 con 0x5606219f4cd0 2023-09-30T14:40:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.862+0000 7f28f4561b80 10 client.15720 fetch_fsmap learned FSMap version 25 2023-09-30T14:40:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.862+0000 7f28f4561b80 1 -- 172.21.15.32:0/2453926039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560621759c00 con 0x5606219f4cd0 2023-09-30T14:40:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.862+0000 7f28c9ffb700 1 -- 172.21.15.32:0/2453926039 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28d800f040 con 0x5606219f4cd0 2023-09-30T14:40:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.862+0000 7f28c9ffb700 1 -- 172.21.15.32:0/2453926039 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f28d801b020 con 0x5606219f4cd0 2023-09-30T14:40:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.863+0000 7f28f4561b80 10 client.15720 fetch_fsmap finished waiting for FSMap version 25 2023-09-30T14:40:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.863+0000 7f28f4561b80 10 client.15720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:40:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.863+0000 7f28f4561b80 20 client.15720 populate_metadata read hostname 'smithi032' 2023-09-30T14:40:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.863+0000 7f28f4561b80 1 --2- 172.21.15.32:0/2453926039 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560621a5d610 0x560621a5fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.863+0000 7f28f4561b80 4 client.15720 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:40:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.863+0000 7f28f4561b80 1 -- 172.21.15.32:0/2453926039 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56062193ead0 con 0x560621a5d610 2023-09-30T14:40:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.863+0000 7f28e6115700 1 --2- 172.21.15.32:0/2453926039 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560621a5d610 0x560621a5fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.863+0000 7f28c9ffb700 10 client.15720 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.863+0000 7f28e6115700 1 --2- 172.21.15.32:0/2453926039 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560621a5d610 0x560621a5fa30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76f23a36d5d0bb27 server_cookie=6b7bbf0864884874 in_seq=0 out_seq=0 2023-09-30T14:40:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.904+0000 7f28c9ffb700 1 -- 172.21.15.32:0/2453926039 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56062193ead0 con 0x560621a5d610 2023-09-30T14:40:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.904+0000 7f28c9ffb700 10 client.15720 handle_command_reply: tid=0 2023-09-30T14:40:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.904+0000 7f28f4561b80 1 client.15720 shutdown 2023-09-30T14:40:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.904+0000 7f28f4561b80 1 -- 172.21.15.32:0/2453926039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4cd0 msgr2=0x560621a03610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.904+0000 7f28f4561b80 1 --2- 172.21.15.32:0/2453926039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4cd0 0x560621a03610 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009960 tx=0x7f28d8007b60 comp rx=0 tx=0).stop 2023-09-30T14:40:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.904+0000 7f28f4561b80 1 -- 172.21.15.32:0/2453926039 shutdown_connections 2023-09-30T14:40:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.904+0000 7f28f4561b80 1 --2- 172.21.15.32:0/2453926039 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560621a5d610 0x560621a5fa30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.904+0000 7f28f4561b80 1 --2- 172.21.15.32:0/2453926039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5606219f4cd0 0x560621a03610 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.904+0000 7f28c9ffb700 0 client.15720 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.904+0000 7f28f4561b80 1 --2- 172.21.15.32:0/2453926039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5606219f3200 0x560621a020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.904+0000 7f28f4561b80 1 --2- 172.21.15.32:0/2453926039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606219f4310 0x560621a02600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.904+0000 7f28f4561b80 1 -- 172.21.15.32:0/2453926039 >> 172.21.15.32:0/2453926039 conn(0x560621967200 msgr2=0x5606219fbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.904+0000 7f28f4561b80 1 -- 172.21.15.32:0/2453926039 shutdown_connections 2023-09-30T14:40:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.904+0000 7f28f4561b80 1 -- 172.21.15.32:0/2453926039 wait complete. 2023-09-30T14:40:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.904+0000 7f28f4561b80 20 client.15720 trim_cache size 0 max 16384 2023-09-30T14:40:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:40:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:40:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:40:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:40:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.905+0000 7f28e8379700 1 -- 172.21.15.32:0/2721286334 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f28cc0a1020 msgr2=0x7f28cc0a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.905+0000 7f28e8379700 1 --2- 172.21.15.32:0/2721286334 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f28cc0a1020 0x7f28cc0a34e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f28e0103c70 tx=0x7f28dc009260 comp rx=0 tx=0).stop 2023-09-30T14:40:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.905+0000 7f28e8379700 1 -- 172.21.15.32:0/2721286334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0100f80 msgr2=0x7f28e01028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.905+0000 7f28e8379700 1 --2- 172.21.15.32:0/2721286334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0100f80 0x7f28e01028a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009fd0 tx=0x7f28d0009580 comp rx=0 tx=0).stop 2023-09-30T14:40:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.905+0000 7f28e8379700 1 -- 172.21.15.32:0/2721286334 shutdown_connections 2023-09-30T14:40:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.905+0000 7f28e8379700 1 --2- 172.21.15.32:0/2721286334 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f28cc0a1020 0x7f28cc0a34e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.905+0000 7f28e8379700 1 --2- 172.21.15.32:0/2721286334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28e0101920 0x7f28e0102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.905+0000 7f28e8379700 1 --2- 172.21.15.32:0/2721286334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28e00ffd90 0x7f28e0102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.905+0000 7f28e8379700 1 --2- 172.21.15.32:0/2721286334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28e0100f80 0x7f28e01028a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.905+0000 7f28e8379700 1 -- 172.21.15.32:0/2721286334 >> 172.21.15.32:0/2721286334 conn(0x7f28e00fb320 msgr2=0x7f28e0068e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.905+0000 7f28e8379700 1 -- 172.21.15.32:0/2721286334 shutdown_connections 2023-09-30T14:40:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:27.905+0000 7f28e8379700 1 -- 172.21.15.32:0/2721286334 wait complete. 2023-09-30T14:40:28.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 153592 13.63 MB/sec warmup 62 sec latency 40.313 ms 2023-09-30T14:40:29.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:29 smithi148 ceph-mon[102355]: pgmap v353: 97 pgs: 97 active+clean; 81 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2023-09-30T14:40:29.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:29 smithi032 ceph-mon[98592]: pgmap v353: 97 pgs: 97 active+clean; 81 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2023-09-30T14:40:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:29 smithi156 ceph-mon[103579]: pgmap v353: 97 pgs: 97 active+clean; 81 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2023-09-30T14:40:29.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 156390 13.68 MB/sec warmup 63 sec latency 14.508 ms 2023-09-30T14:40:30.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 158428 13.58 MB/sec warmup 64 sec latency 13.850 ms 2023-09-30T14:40:31.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:31 smithi148 ceph-mon[102355]: pgmap v354: 97 pgs: 97 active+clean; 81 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 321 op/s 2023-09-30T14:40:31.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:31 smithi032 ceph-mon[98592]: pgmap v354: 97 pgs: 97 active+clean; 81 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 321 op/s 2023-09-30T14:40:31.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:31 smithi156 ceph-mon[103579]: pgmap v354: 97 pgs: 97 active+clean; 81 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 321 op/s 2023-09-30T14:40:31.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 160192 13.56 MB/sec warmup 65 sec latency 47.249 ms 2023-09-30T14:40:32.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 162836 13.56 MB/sec warmup 66 sec latency 12.909 ms 2023-09-30T14:40:33.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:33 smithi148 ceph-mon[102355]: pgmap v355: 97 pgs: 97 active+clean; 81 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 346 op/s 2023-09-30T14:40:33.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:33 smithi032 ceph-mon[98592]: pgmap v355: 97 pgs: 97 active+clean; 81 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 346 op/s 2023-09-30T14:40:33.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:33 smithi156 ceph-mon[103579]: pgmap v355: 97 pgs: 97 active+clean; 81 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 346 op/s 2023-09-30T14:40:33.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 164254 13.47 MB/sec warmup 67 sec latency 36.810 ms 2023-09-30T14:40:34.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 166995 13.52 MB/sec warmup 68 sec latency 50.422 ms 2023-09-30T14:40:35.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:35 smithi148 ceph-mon[102355]: pgmap v356: 97 pgs: 97 active+clean; 84 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 357 op/s 2023-09-30T14:40:35.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:35 smithi032 ceph-mon[98592]: pgmap v356: 97 pgs: 97 active+clean; 84 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 357 op/s 2023-09-30T14:40:35.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:35 smithi156 ceph-mon[103579]: pgmap v356: 97 pgs: 97 active+clean; 84 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 357 op/s 2023-09-30T14:40:35.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 168933 13.41 MB/sec warmup 69 sec latency 13.146 ms 2023-09-30T14:40:36.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 170830 13.41 MB/sec warmup 70 sec latency 49.191 ms 2023-09-30T14:40:37.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:37 smithi148 ceph-mon[102355]: pgmap v357: 97 pgs: 97 active+clean; 84 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 339 op/s 2023-09-30T14:40:37.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:37 smithi032 ceph-mon[98592]: pgmap v357: 97 pgs: 97 active+clean; 84 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 339 op/s 2023-09-30T14:40:37.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:37 smithi156 ceph-mon[103579]: pgmap v357: 97 pgs: 97 active+clean; 84 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 339 op/s 2023-09-30T14:40:37.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 173636 13.43 MB/sec warmup 71 sec latency 48.971 ms 2023-09-30T14:40:38.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 174881 13.33 MB/sec warmup 72 sec latency 28.633 ms 2023-09-30T14:40:39.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:39 smithi148 ceph-mon[102355]: pgmap v358: 97 pgs: 97 active+clean; 88 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 388 op/s 2023-09-30T14:40:39.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:39 smithi032 ceph-mon[98592]: pgmap v358: 97 pgs: 97 active+clean; 88 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 388 op/s 2023-09-30T14:40:39.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:39 smithi156 ceph-mon[103579]: pgmap v358: 97 pgs: 97 active+clean; 88 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 388 op/s 2023-09-30T14:40:39.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 177639 13.38 MB/sec warmup 73 sec latency 15.091 ms 2023-09-30T14:40:40.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 179701 13.30 MB/sec warmup 74 sec latency 13.403 ms 2023-09-30T14:40:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.020+0000 7f3a28341700 1 -- 172.21.15.32:0/3586625768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a20100f50 msgr2=0x7f3a201013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.020+0000 7f3a28341700 1 --2- 172.21.15.32:0/3586625768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a20100f50 0x7f3a201013b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3a100099c0 tx=0x7f3a10009cd0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.020+0000 7f3a28341700 1 -- 172.21.15.32:0/3586625768 shutdown_connections 2023-09-30T14:40:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.020+0000 7f3a28341700 1 --2- 172.21.15.32:0/3586625768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a201018f0 0x7f3a201081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.020+0000 7f3a28341700 1 --2- 172.21.15.32:0/3586625768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a200ffd60 0x7f3a20100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.020+0000 7f3a28341700 1 --2- 172.21.15.32:0/3586625768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a20100f50 0x7f3a201013b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.020+0000 7f3a28341700 1 -- 172.21.15.32:0/3586625768 >> 172.21.15.32:0/3586625768 conn(0x7f3a200fb2b0 msgr2=0x7f3a200fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.020+0000 7f3a28341700 1 -- 172.21.15.32:0/3586625768 shutdown_connections 2023-09-30T14:40:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.020+0000 7f3a28341700 1 -- 172.21.15.32:0/3586625768 wait complete. 2023-09-30T14:40:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.021+0000 7f3a28341700 1 Processor -- start 2023-09-30T14:40:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.021+0000 7f3a28341700 1 -- start start 2023-09-30T14:40:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.021+0000 7f3a28341700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a200ffd60 0x7f3a20195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.021+0000 7f3a28341700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a20100f50 0x7f3a20196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.021+0000 7f3a28341700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a201018f0 0x7f3a20196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.021+0000 7f3a28341700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a20197520 con 0x7f3a20100f50 2023-09-30T14:40:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.021+0000 7f3a28341700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a20197690 con 0x7f3a200ffd60 2023-09-30T14:40:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.021+0000 7f3a28341700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a20197850 con 0x7f3a201018f0 2023-09-30T14:40:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.021+0000 7f3a258dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a20100f50 0x7f3a20196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.021+0000 7f3a260dd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a200ffd60 0x7f3a20195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.021+0000 7f3a260dd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a200ffd60 0x7f3a20195b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47712/0 (socket says 172.21.15.32:47712) 2023-09-30T14:40:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.021+0000 7f3a260dd700 1 -- 172.21.15.32:0/4117994916 learned_addr learned my addr 172.21.15.32:0/4117994916 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.021+0000 7f3a268de700 1 --2- 172.21.15.32:0/4117994916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a201018f0 0x7f3a20196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.022+0000 7f3a258dc700 1 -- 172.21.15.32:0/4117994916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a201018f0 msgr2=0x7f3a20196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.022+0000 7f3a258dc700 1 --2- 172.21.15.32:0/4117994916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a201018f0 0x7f3a20196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.022+0000 7f3a258dc700 1 -- 172.21.15.32:0/4117994916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a200ffd60 msgr2=0x7f3a20195b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.022+0000 7f3a258dc700 1 --2- 172.21.15.32:0/4117994916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a200ffd60 0x7f3a20195b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.022+0000 7f3a258dc700 1 -- 172.21.15.32:0/4117994916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c009520 con 0x7f3a20100f50 2023-09-30T14:40:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.022+0000 7f3a258dc700 1 --2- 172.21.15.32:0/4117994916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a20100f50 0x7f3a20196040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009fd0 tx=0x7f3a10009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.022+0000 7f3a177fe700 1 -- 172.21.15.32:0/4117994916 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1001d070 con 0x7f3a20100f50 2023-09-30T14:40:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.022+0000 7f3a177fe700 1 -- 172.21.15.32:0/4117994916 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a10005020 con 0x7f3a20100f50 2023-09-30T14:40:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.022+0000 7f3a28341700 1 -- 172.21.15.32:0/4117994916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a100095f0 con 0x7f3a20100f50 2023-09-30T14:40:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.022+0000 7f3a177fe700 1 -- 172.21.15.32:0/4117994916 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10004a90 con 0x7f3a20100f50 2023-09-30T14:40:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.022+0000 7f3a28341700 1 -- 172.21.15.32:0/4117994916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a20197d00 con 0x7f3a20100f50 2023-09-30T14:40:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.024+0000 7f3a177fe700 1 -- 172.21.15.32:0/4117994916 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f3a10005190 con 0x7f3a20100f50 2023-09-30T14:40:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.025+0000 7f3a177fe700 1 --2- 172.21.15.32:0/4117994916 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3a0c0a0fa0 0x7f3a0c0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.025+0000 7f3a177fe700 1 -- 172.21.15.32:0/4117994916 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3a100cd110 con 0x7f3a20100f50 2023-09-30T14:40:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.025+0000 7f3a260dd700 1 --2- 172.21.15.32:0/4117994916 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3a0c0a0fa0 0x7f3a0c0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.025+0000 7f3a260dd700 1 --2- 172.21.15.32:0/4117994916 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3a0c0a0fa0 0x7f3a0c0a3460 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c005950 tx=0x7f3a1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.026+0000 7f3a34529b80 1 Processor -- start 2023-09-30T14:40:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.026+0000 7f3a34529b80 1 -- start start 2023-09-30T14:40:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.026+0000 7f3a34529b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243e200 0x562dc243e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.026+0000 7f3a34529b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243f310 0x562dc243f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.026+0000 7f3a34529b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243fcd0 0x562dc2446560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.026+0000 7f3a34529b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dc23a6400 con 0x562dc243f310 2023-09-30T14:40:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.026+0000 7f3a34529b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dc2397bc0 con 0x562dc243e200 2023-09-30T14:40:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.026+0000 7f3a34529b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dc23954e0 con 0x562dc243fcd0 2023-09-30T14:40:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.026+0000 7f3a258dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243f310 0x562dc243f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.026+0000 7f3a260dd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243e200 0x562dc243e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.026+0000 7f3a258dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243f310 0x562dc243f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36246/0 (socket says 172.21.15.32:36246) 2023-09-30T14:40:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.026+0000 7f3a258dc700 1 -- 172.21.15.32:0/573266810 learned_addr learned my addr 172.21.15.32:0/573266810 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.026+0000 7f3a268de700 1 --2- 172.21.15.32:0/573266810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243fcd0 0x562dc2446560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.027+0000 7f3a260dd700 1 -- 172.21.15.32:0/573266810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243fcd0 msgr2=0x562dc2446560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.027+0000 7f3a260dd700 1 --2- 172.21.15.32:0/573266810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243fcd0 0x562dc2446560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.027+0000 7f3a260dd700 1 -- 172.21.15.32:0/573266810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243f310 msgr2=0x562dc243f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.027+0000 7f3a260dd700 1 --2- 172.21.15.32:0/573266810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243f310 0x562dc243f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.027+0000 7f3a260dd700 1 -- 172.21.15.32:0/573266810 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dc23f3870 con 0x562dc243e200 2023-09-30T14:40:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.027+0000 7f3a260dd700 1 --2- 172.21.15.32:0/573266810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243e200 0x562dc243e620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00e600 tx=0x7f3a1c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8ca4f45d10e34ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.027+0000 7f3a14d82700 1 -- 172.21.15.32:0/573266810 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c004020 con 0x562dc243e200 2023-09-30T14:40:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.027+0000 7f3a14d82700 1 -- 172.21.15.32:0/573266810 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a1c021070 con 0x562dc243e200 2023-09-30T14:40:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.027+0000 7f3a14d82700 1 -- 172.21.15.32:0/573266810 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c0266c0 con 0x562dc243e200 2023-09-30T14:40:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.028+0000 7f3a34529b80 1 -- 172.21.15.32:0/573266810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243e200 msgr2=0x562dc243e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.028+0000 7f3a34529b80 1 --2- 172.21.15.32:0/573266810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243e200 0x562dc243e620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00e600 tx=0x7f3a1c014000 comp rx=0 tx=0).stop 2023-09-30T14:40:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.028+0000 7f3a34529b80 1 -- 172.21.15.32:0/573266810 shutdown_connections 2023-09-30T14:40:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.028+0000 7f3a34529b80 1 --2- 172.21.15.32:0/573266810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243fcd0 0x562dc2446560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.028+0000 7f3a34529b80 1 --2- 172.21.15.32:0/573266810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243e200 0x562dc243e620 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.028+0000 7f3a34529b80 1 --2- 172.21.15.32:0/573266810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243f310 0x562dc243f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.028+0000 7f3a34529b80 1 -- 172.21.15.32:0/573266810 >> 172.21.15.32:0/573266810 conn(0x562dc23b2200 msgr2=0x562dc2438ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.028+0000 7f3a34529b80 1 -- 172.21.15.32:0/573266810 shutdown_connections 2023-09-30T14:40:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.028+0000 7f3a34529b80 1 -- 172.21.15.32:0/573266810 wait complete. 2023-09-30T14:40:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.028+0000 7f3a34529b80 1 -- start start 2023-09-30T14:40:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.028+0000 7f3a34529b80 1 Processor -- start 2023-09-30T14:40:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a34529b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243e200 0x562dc244a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a34529b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243f310 0x562dc244a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a34529b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243fcd0 0x562dc2451640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a34529b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dc2390dd0 con 0x562dc243fcd0 2023-09-30T14:40:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a34529b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dc232d6b0 con 0x562dc243e200 2023-09-30T14:40:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a34529b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dc232d350 con 0x562dc243f310 2023-09-30T14:40:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a34529b80 10 client.0 _ll_register_callbacks cb 0x562dc218a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:40:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a268de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243fcd0 0x562dc2451640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a260dd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243e200 0x562dc244a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a268de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243fcd0 0x562dc2451640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36254/0 (socket says 172.21.15.32:36254) 2023-09-30T14:40:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a268de700 1 -- 172.21.15.32:0/252792486 learned_addr learned my addr 172.21.15.32:0/252792486 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a258dc700 1 --2- 172.21.15.32:0/252792486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243f310 0x562dc244a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a260dd700 1 -- 172.21.15.32:0/252792486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243f310 msgr2=0x562dc244a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a260dd700 1 --2- 172.21.15.32:0/252792486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243f310 0x562dc244a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a260dd700 1 -- 172.21.15.32:0/252792486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243fcd0 msgr2=0x562dc2451640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a260dd700 1 --2- 172.21.15.32:0/252792486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243fcd0 0x562dc2451640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a260dd700 1 -- 172.21.15.32:0/252792486 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1c017070 con 0x562dc243e200 2023-09-30T14:40:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a268de700 1 --2- 172.21.15.32:0/252792486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243fcd0 0x562dc2451640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:40:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a260dd700 1 --2- 172.21.15.32:0/252792486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243e200 0x562dc244a0c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00e600 tx=0x7f3a1c00ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b040d3bb1f18a709 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:40:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.029+0000 7f3a09ffb700 1 -- 172.21.15.32:0/252792486 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c0040b0 con 0x562dc243e200 2023-09-30T14:40:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.030+0000 7f3a34529b80 1 -- 172.21.15.32:0/252792486 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562dc2431090 con 0x562dc243e200 2023-09-30T14:40:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.030+0000 7f3a09ffb700 1 -- 172.21.15.32:0/252792486 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3a1c004210 con 0x562dc243e200 2023-09-30T14:40:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.030+0000 7f3a09ffb700 1 -- 172.21.15.32:0/252792486 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a1c0043d0 con 0x562dc243e200 2023-09-30T14:40:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.030+0000 7f3a34529b80 10 client.25427 fetch_fsmap learned FSMap version 25 2023-09-30T14:40:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.030+0000 7f3a34529b80 1 -- 172.21.15.32:0/252792486 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562dc21bbd80 con 0x562dc243e200 2023-09-30T14:40:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.030+0000 7f3a09ffb700 1 -- 172.21.15.32:0/252792486 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562dc2431090 con 0x562dc243e200 2023-09-30T14:40:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.030+0000 7f3a09ffb700 1 -- 172.21.15.32:0/252792486 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3a1c039070 con 0x562dc243e200 2023-09-30T14:40:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.030+0000 7f3a34529b80 10 client.25427 fetch_fsmap finished waiting for FSMap version 25 2023-09-30T14:40:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.030+0000 7f3a34529b80 10 client.25427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:40:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.030+0000 7f3a34529b80 20 client.25427 populate_metadata read hostname 'smithi032' 2023-09-30T14:40:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.030+0000 7f3a34529b80 1 --2- 172.21.15.32:0/252792486 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562dc2455e50 0x562dc2458270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.030+0000 7f3a34529b80 4 client.25427 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:40:41.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.030+0000 7f3a34529b80 1 -- 172.21.15.32:0/252792486 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562dc2389db0 con 0x562dc2455e50 2023-09-30T14:40:41.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.030+0000 7f3a258dc700 1 --2- 172.21.15.32:0/252792486 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562dc2455e50 0x562dc2458270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.031+0000 7f3a258dc700 1 --2- 172.21.15.32:0/252792486 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562dc2455e50 0x562dc2458270 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac3645cb0d69cb84 server_cookie=c2545fa559586c84 in_seq=0 out_seq=0 2023-09-30T14:40:41.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.031+0000 7f3a09ffb700 10 client.25427 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:41.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.031+0000 7f3a09ffb700 1 -- 172.21.15.32:0/252792486 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562dc2389db0 con 0x562dc2455e50 2023-09-30T14:40:41.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.031+0000 7f3a09ffb700 10 client.25427 handle_command_reply: tid=0 2023-09-30T14:40:41.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.031+0000 7f3a34529b80 1 client.25427 shutdown 2023-09-30T14:40:41.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.031+0000 7f3a34529b80 1 -- 172.21.15.32:0/252792486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243e200 msgr2=0x562dc244a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:41.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.031+0000 7f3a34529b80 1 --2- 172.21.15.32:0/252792486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243e200 0x562dc244a0c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00e600 tx=0x7f3a1c00ee90 comp rx=0 tx=0).stop 2023-09-30T14:40:41.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.032+0000 7f3a34529b80 1 -- 172.21.15.32:0/252792486 shutdown_connections 2023-09-30T14:40:41.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.032+0000 7f3a34529b80 1 --2- 172.21.15.32:0/252792486 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562dc2455e50 0x562dc2458270 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.032+0000 7f3a34529b80 1 --2- 172.21.15.32:0/252792486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243fcd0 0x562dc2451640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.032+0000 7f3a09ffb700 0 client.25427 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.032+0000 7f3a34529b80 1 --2- 172.21.15.32:0/252792486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243e200 0x562dc244a0c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.032+0000 7f3a34529b80 1 --2- 172.21.15.32:0/252792486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243f310 0x562dc244a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.032+0000 7f3a34529b80 1 -- 172.21.15.32:0/252792486 >> 172.21.15.32:0/252792486 conn(0x562dc23b2200 msgr2=0x562dc2438590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.032+0000 7f3a34529b80 1 -- 172.21.15.32:0/252792486 shutdown_connections 2023-09-30T14:40:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.032+0000 7f3a34529b80 1 -- 172.21.15.32:0/252792486 wait complete. 2023-09-30T14:40:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.032+0000 7f3a34529b80 20 client.25427 trim_cache size 0 max 16384 2023-09-30T14:40:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a34529b80 1 Processor -- start 2023-09-30T14:40:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a34529b80 1 -- start start 2023-09-30T14:40:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a34529b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243e200 0x562dc243f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a34529b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243f310 0x562dc2451600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a34529b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243fcd0 0x562dc2448db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a34529b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dc22118c0 con 0x562dc243e200 2023-09-30T14:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a34529b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dc2435ba0 con 0x562dc243fcd0 2023-09-30T14:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a34529b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dc24427c0 con 0x562dc243f310 2023-09-30T14:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a260dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243e200 0x562dc243f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a260dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243e200 0x562dc243f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36258/0 (socket says 172.21.15.32:36258) 2023-09-30T14:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a260dd700 1 -- 172.21.15.32:0/165158910 learned_addr learned my addr 172.21.15.32:0/165158910 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a268de700 1 --2- 172.21.15.32:0/165158910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243fcd0 0x562dc2448db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a258dc700 1 --2- 172.21.15.32:0/165158910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243f310 0x562dc2451600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a260dd700 1 -- 172.21.15.32:0/165158910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243f310 msgr2=0x562dc2451600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a260dd700 1 --2- 172.21.15.32:0/165158910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243f310 0x562dc2451600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a260dd700 1 -- 172.21.15.32:0/165158910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243fcd0 msgr2=0x562dc2448db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a260dd700 1 --2- 172.21.15.32:0/165158910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243fcd0 0x562dc2448db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a260dd700 1 -- 172.21.15.32:0/165158910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dc21a4c00 con 0x562dc243e200 2023-09-30T14:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.047+0000 7f3a260dd700 1 --2- 172.21.15.32:0/165158910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243e200 0x562dc243f6f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00e5d0 tx=0x7f3a1c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7345be8102a6d2c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.048+0000 7f3a15583700 1 -- 172.21.15.32:0/165158910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c01d780 con 0x562dc243e200 2023-09-30T14:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.048+0000 7f3a15583700 1 -- 172.21.15.32:0/165158910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a1c036070 con 0x562dc243e200 2023-09-30T14:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.048+0000 7f3a15583700 1 -- 172.21.15.32:0/165158910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c004110 con 0x562dc243e200 2023-09-30T14:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.048+0000 7f3a34529b80 1 -- 172.21.15.32:0/165158910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243e200 msgr2=0x562dc243f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.048+0000 7f3a34529b80 1 --2- 172.21.15.32:0/165158910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243e200 0x562dc243f6f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00e5d0 tx=0x7f3a1c005720 comp rx=0 tx=0).stop 2023-09-30T14:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.048+0000 7f3a34529b80 1 -- 172.21.15.32:0/165158910 shutdown_connections 2023-09-30T14:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.048+0000 7f3a34529b80 1 --2- 172.21.15.32:0/165158910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243fcd0 0x562dc2448db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.048+0000 7f3a34529b80 1 --2- 172.21.15.32:0/165158910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243e200 0x562dc243f6f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.048+0000 7f3a34529b80 1 --2- 172.21.15.32:0/165158910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243f310 0x562dc2451600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.048+0000 7f3a34529b80 1 -- 172.21.15.32:0/165158910 >> 172.21.15.32:0/165158910 conn(0x562dc23b2200 msgr2=0x562dc245cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.048+0000 7f3a34529b80 1 -- 172.21.15.32:0/165158910 shutdown_connections 2023-09-30T14:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.048+0000 7f3a34529b80 1 -- 172.21.15.32:0/165158910 wait complete. 2023-09-30T14:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.049+0000 7f3a34529b80 1 -- start start 2023-09-30T14:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.049+0000 7f3a34529b80 1 Processor -- start 2023-09-30T14:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.049+0000 7f3a34529b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243e200 0x562dc244ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.049+0000 7f3a34529b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243f310 0x562dc244f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.049+0000 7f3a34529b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243fcd0 0x562dc2450330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.049+0000 7f3a34529b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dc2450870 con 0x562dc243f310 2023-09-30T14:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.049+0000 7f3a34529b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dc24509e0 con 0x562dc243fcd0 2023-09-30T14:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.049+0000 7f3a34529b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dc2450ba0 con 0x562dc243e200 2023-09-30T14:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.049+0000 7f3a34529b80 10 client.0 _ll_register_callbacks cb 0x7f3a100962f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.049+0000 7f3a258dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243f310 0x562dc244f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.049+0000 7f3a268de700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243fcd0 0x562dc2450330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.049+0000 7f3a258dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243f310 0x562dc244f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36266/0 (socket says 172.21.15.32:36266) 2023-09-30T14:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.049+0000 7f3a258dc700 1 -- 172.21.15.32:0/3920136890 learned_addr learned my addr 172.21.15.32:0/3920136890 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.049+0000 7f3a260dd700 1 --2- 172.21.15.32:0/3920136890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243e200 0x562dc244ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.050+0000 7f3a258dc700 1 -- 172.21.15.32:0/3920136890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243e200 msgr2=0x562dc244ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.050+0000 7f3a258dc700 1 --2- 172.21.15.32:0/3920136890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243e200 0x562dc244ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.050+0000 7f3a258dc700 1 -- 172.21.15.32:0/3920136890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243fcd0 msgr2=0x562dc2450330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.050+0000 7f3a258dc700 1 --2- 172.21.15.32:0/3920136890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243fcd0 0x562dc2450330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.050+0000 7f3a258dc700 1 -- 172.21.15.32:0/3920136890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562dc21bbd80 con 0x562dc243f310 2023-09-30T14:40:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.050+0000 7f3a268de700 1 --2- 172.21.15.32:0/3920136890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243fcd0 0x562dc2450330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:40:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.050+0000 7f3a258dc700 1 --2- 172.21.15.32:0/3920136890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243f310 0x562dc244f320 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3a100caf90 tx=0x7f3a1002a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76127ccb9773825 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.050+0000 7f3a09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:40:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.050+0000 7f3a09ffb700 1 -- 172.21.15.32:0/3920136890 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a100a2970 con 0x562dc243f310 2023-09-30T14:40:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.050+0000 7f3a09ffb700 1 -- 172.21.15.32:0/3920136890 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3a10098a10 con 0x562dc243f310 2023-09-30T14:40:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.050+0000 7f3a34529b80 1 -- 172.21.15.32:0/3920136890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a1c021070 con 0x562dc243f310 2023-09-30T14:40:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.050+0000 7f3a09ffb700 1 -- 172.21.15.32:0/3920136890 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a10004bf0 con 0x562dc243f310 2023-09-30T14:40:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.050+0000 7f3a34529b80 10 client.15738 fetch_fsmap learned FSMap version 25 2023-09-30T14:40:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.050+0000 7f3a34529b80 1 -- 172.21.15.32:0/3920136890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562dc21e21f0 con 0x562dc243f310 2023-09-30T14:40:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.050+0000 7f3a09ffb700 1 -- 172.21.15.32:0/3920136890 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a1c021070 con 0x562dc243f310 2023-09-30T14:40:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.050+0000 7f3a09ffb700 1 -- 172.21.15.32:0/3920136890 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3a10099b60 con 0x562dc243f310 2023-09-30T14:40:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.051+0000 7f3a34529b80 10 client.15738 fetch_fsmap finished waiting for FSMap version 25 2023-09-30T14:40:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.051+0000 7f3a34529b80 10 client.15738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:40:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.051+0000 7f3a34529b80 20 client.15738 populate_metadata read hostname 'smithi032' 2023-09-30T14:40:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.051+0000 7f3a34529b80 1 --2- 172.21.15.32:0/3920136890 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562dc24aa210 0x562dc24ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.051+0000 7f3a34529b80 4 client.15738 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:40:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.051+0000 7f3a34529b80 1 -- 172.21.15.32:0/3920136890 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562dc2389ad0 con 0x562dc24aa210 2023-09-30T14:40:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.051+0000 7f3a260dd700 1 --2- 172.21.15.32:0/3920136890 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562dc24aa210 0x562dc24ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.051+0000 7f3a09ffb700 10 client.15738 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.051+0000 7f3a260dd700 1 --2- 172.21.15.32:0/3920136890 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562dc24aa210 0x562dc24ac630 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=250b3fb376d9edaf server_cookie=1f2771e530afbe73 in_seq=0 out_seq=0 2023-09-30T14:40:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.090+0000 7f3a09ffb700 1 -- 172.21.15.32:0/3920136890 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562dc2389ad0 con 0x562dc24aa210 2023-09-30T14:40:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.090+0000 7f3a09ffb700 10 client.15738 handle_command_reply: tid=0 2023-09-30T14:40:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.091+0000 7f3a34529b80 1 client.15738 shutdown 2023-09-30T14:40:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.091+0000 7f3a34529b80 1 -- 172.21.15.32:0/3920136890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243f310 msgr2=0x562dc244f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.091+0000 7f3a34529b80 1 --2- 172.21.15.32:0/3920136890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243f310 0x562dc244f320 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3a100caf90 tx=0x7f3a1002a000 comp rx=0 tx=0).stop 2023-09-30T14:40:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.091+0000 7f3a34529b80 1 -- 172.21.15.32:0/3920136890 shutdown_connections 2023-09-30T14:40:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.091+0000 7f3a34529b80 1 --2- 172.21.15.32:0/3920136890 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562dc24aa210 0x562dc24ac630 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.091+0000 7f3a34529b80 1 --2- 172.21.15.32:0/3920136890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dc243fcd0 0x562dc2450330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.091+0000 7f3a34529b80 1 --2- 172.21.15.32:0/3920136890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562dc243e200 0x562dc244ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.091+0000 7f3a34529b80 1 --2- 172.21.15.32:0/3920136890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562dc243f310 0x562dc244f320 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.091+0000 7f3a34529b80 1 -- 172.21.15.32:0/3920136890 >> 172.21.15.32:0/3920136890 conn(0x562dc23b2200 msgr2=0x562dc2448db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.091+0000 7f3a09ffb700 0 client.15738 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.091+0000 7f3a34529b80 1 -- 172.21.15.32:0/3920136890 shutdown_connections 2023-09-30T14:40:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.091+0000 7f3a34529b80 1 -- 172.21.15.32:0/3920136890 wait complete. 2023-09-30T14:40:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.091+0000 7f3a34529b80 20 client.15738 trim_cache size 0 max 16384 2023-09-30T14:40:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:40:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:40:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:40:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:40:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.091+0000 7f3a28341700 1 -- 172.21.15.32:0/4117994916 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3a0c0a0fa0 msgr2=0x7f3a0c0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.091+0000 7f3a28341700 1 --2- 172.21.15.32:0/4117994916 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3a0c0a0fa0 0x7f3a0c0a3460 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c005950 tx=0x7f3a1c009260 comp rx=0 tx=0).stop 2023-09-30T14:40:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.091+0000 7f3a28341700 1 -- 172.21.15.32:0/4117994916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a20100f50 msgr2=0x7f3a20196040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.092+0000 7f3a28341700 1 --2- 172.21.15.32:0/4117994916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a20100f50 0x7f3a20196040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009fd0 tx=0x7f3a10009580 comp rx=0 tx=0).stop 2023-09-30T14:40:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.092+0000 7f3a28341700 1 -- 172.21.15.32:0/4117994916 shutdown_connections 2023-09-30T14:40:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.092+0000 7f3a28341700 1 --2- 172.21.15.32:0/4117994916 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3a0c0a0fa0 0x7f3a0c0a3460 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.092+0000 7f3a28341700 1 --2- 172.21.15.32:0/4117994916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a201018f0 0x7f3a20196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.092+0000 7f3a28341700 1 --2- 172.21.15.32:0/4117994916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a200ffd60 0x7f3a20195b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.092+0000 7f3a28341700 1 --2- 172.21.15.32:0/4117994916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a20100f50 0x7f3a20196040 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.092+0000 7f3a28341700 1 -- 172.21.15.32:0/4117994916 >> 172.21.15.32:0/4117994916 conn(0x7f3a200fb2b0 msgr2=0x7f3a200fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.092+0000 7f3a28341700 1 -- 172.21.15.32:0/4117994916 shutdown_connections 2023-09-30T14:40:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:41.092+0000 7f3a28341700 1 -- 172.21.15.32:0/4117994916 wait complete. 2023-09-30T14:40:41.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:41 smithi148 ceph-mon[102355]: pgmap v359: 97 pgs: 97 active+clean; 87 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 311 op/s 2023-09-30T14:40:41.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:41 smithi032 ceph-mon[98592]: pgmap v359: 97 pgs: 97 active+clean; 87 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 311 op/s 2023-09-30T14:40:41.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:41 smithi156 ceph-mon[103579]: pgmap v359: 97 pgs: 97 active+clean; 87 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 311 op/s 2023-09-30T14:40:41.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 181394 13.28 MB/sec warmup 75 sec latency 17.050 ms 2023-09-30T14:40:42.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 184080 13.28 MB/sec warmup 76 sec latency 12.539 ms 2023-09-30T14:40:43.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:43 smithi148 ceph-mon[102355]: pgmap v360: 97 pgs: 97 active+clean; 87 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 314 op/s 2023-09-30T14:40:43.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:43 smithi032 ceph-mon[98592]: pgmap v360: 97 pgs: 97 active+clean; 87 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 314 op/s 2023-09-30T14:40:43.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:43 smithi156 ceph-mon[103579]: pgmap v360: 97 pgs: 97 active+clean; 87 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 314 op/s 2023-09-30T14:40:43.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 185236 13.19 MB/sec warmup 77 sec latency 91.318 ms 2023-09-30T14:40:44.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 187840 13.20 MB/sec warmup 78 sec latency 72.901 ms 2023-09-30T14:40:45.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:45 smithi148 ceph-mon[102355]: pgmap v361: 97 pgs: 97 active+clean; 90 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 365 op/s 2023-09-30T14:40:45.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:45 smithi032 ceph-mon[98592]: pgmap v361: 97 pgs: 97 active+clean; 90 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 365 op/s 2023-09-30T14:40:45.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:45 smithi156 ceph-mon[103579]: pgmap v361: 97 pgs: 97 active+clean; 90 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 365 op/s 2023-09-30T14:40:45.692 INFO:tasks.workunit.client.0.smithi032.stdout: 1 189440 13.16 MB/sec warmup 79 sec latency 14.188 ms 2023-09-30T14:40:46.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 191930 13.17 MB/sec warmup 80 sec latency 13.953 ms 2023-09-30T14:40:47.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:47 smithi032 ceph-mon[98592]: pgmap v362: 97 pgs: 97 active+clean; 87 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 324 op/s 2023-09-30T14:40:47.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:47 smithi156 ceph-mon[103579]: pgmap v362: 97 pgs: 97 active+clean; 87 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 324 op/s 2023-09-30T14:40:47.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:47 smithi148 ceph-mon[102355]: pgmap v362: 97 pgs: 97 active+clean; 87 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 324 op/s 2023-09-30T14:40:47.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 194046 13.10 MB/sec warmup 81 sec latency 14.486 ms 2023-09-30T14:40:48.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 195819 13.08 MB/sec warmup 82 sec latency 32.781 ms 2023-09-30T14:40:49.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:49 smithi032 ceph-mon[98592]: pgmap v363: 97 pgs: 97 active+clean; 89 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 357 op/s 2023-09-30T14:40:49.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:49 smithi156 ceph-mon[103579]: pgmap v363: 97 pgs: 97 active+clean; 89 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 357 op/s 2023-09-30T14:40:49.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:49 smithi148 ceph-mon[102355]: pgmap v363: 97 pgs: 97 active+clean; 89 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 357 op/s 2023-09-30T14:40:49.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 198613 13.13 MB/sec warmup 83 sec latency 43.178 ms 2023-09-30T14:40:50.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 200092 13.06 MB/sec warmup 84 sec latency 12.993 ms 2023-09-30T14:40:51.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:51 smithi032 ceph-mon[98592]: pgmap v364: 97 pgs: 97 active+clean; 89 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 275 op/s 2023-09-30T14:40:51.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:51 smithi156 ceph-mon[103579]: pgmap v364: 97 pgs: 97 active+clean; 89 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 275 op/s 2023-09-30T14:40:51.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:51 smithi148 ceph-mon[102355]: pgmap v364: 97 pgs: 97 active+clean; 89 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 275 op/s 2023-09-30T14:40:51.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 202566 13.07 MB/sec warmup 85 sec latency 11.224 ms 2023-09-30T14:40:52.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 204691 13.01 MB/sec warmup 86 sec latency 12.440 ms 2023-09-30T14:40:53.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:53 smithi032 ceph-mon[98592]: pgmap v365: 97 pgs: 97 active+clean; 91 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 332 op/s 2023-09-30T14:40:53.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:53 smithi156 ceph-mon[103579]: pgmap v365: 97 pgs: 97 active+clean; 91 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 332 op/s 2023-09-30T14:40:53.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:53 smithi148 ceph-mon[102355]: pgmap v365: 97 pgs: 97 active+clean; 91 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 332 op/s 2023-09-30T14:40:53.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 206471 12.99 MB/sec warmup 87 sec latency 16.467 ms 2023-09-30T14:40:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.211+0000 7fcc754eb700 1 -- 172.21.15.32:0/4206898772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc70100ea0 msgr2=0x7fcc701012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.211+0000 7fcc754eb700 1 --2- 172.21.15.32:0/4206898772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc70100ea0 0x7fcc701012c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcc60009990 tx=0x7fcc60009ca0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.212+0000 7fcc754eb700 1 -- 172.21.15.32:0/4206898772 shutdown_connections 2023-09-30T14:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.212+0000 7fcc754eb700 1 --2- 172.21.15.32:0/4206898772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc701029b0 0x7fcc701092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.212+0000 7fcc754eb700 1 --2- 172.21.15.32:0/4206898772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc70100ea0 0x7fcc701012c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.212+0000 7fcc754eb700 1 --2- 172.21.15.32:0/4206898772 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc70101ff0 0x7fcc70102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.212+0000 7fcc754eb700 1 -- 172.21.15.32:0/4206898772 >> 172.21.15.32:0/4206898772 conn(0x7fcc700fc3f0 msgr2=0x7fcc700fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.212+0000 7fcc754eb700 1 -- 172.21.15.32:0/4206898772 shutdown_connections 2023-09-30T14:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.212+0000 7fcc754eb700 1 -- 172.21.15.32:0/4206898772 wait complete. 2023-09-30T14:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.212+0000 7fcc754eb700 1 Processor -- start 2023-09-30T14:40:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.212+0000 7fcc754eb700 1 -- start start 2023-09-30T14:40:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.212+0000 7fcc754eb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc70100ea0 0x7fcc70073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.212+0000 7fcc754eb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc70101ff0 0x7fcc70073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc754eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc701029b0 0x7fcc70073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc754eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc701a0bb0 con 0x7fcc701029b0 2023-09-30T14:40:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc754eb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc701a0cf0 con 0x7fcc70101ff0 2023-09-30T14:40:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc754eb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc701a0e30 con 0x7fcc70100ea0 2023-09-30T14:40:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc6effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc70100ea0 0x7fcc70073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc6e7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc70101ff0 0x7fcc70073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc6f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc701029b0 0x7fcc70073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc6effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc70100ea0 0x7fcc70073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:50660/0 (socket says 172.21.15.32:50660) 2023-09-30T14:40:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc6effd700 1 -- 172.21.15.32:0/1802981300 learned_addr learned my addr 172.21.15.32:0/1802981300 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc6f7fe700 1 -- 172.21.15.32:0/1802981300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc70100ea0 msgr2=0x7fcc70073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc6f7fe700 1 --2- 172.21.15.32:0/1802981300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc70100ea0 0x7fcc70073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc6f7fe700 1 -- 172.21.15.32:0/1802981300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc70101ff0 msgr2=0x7fcc70073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc6f7fe700 1 --2- 172.21.15.32:0/1802981300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc70101ff0 0x7fcc70073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc6f7fe700 1 -- 172.21.15.32:0/1802981300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc600095f0 con 0x7fcc701029b0 2023-09-30T14:40:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc6effd700 1 --2- 172.21.15.32:0/1802981300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc70100ea0 0x7fcc70073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:40:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc6f7fe700 1 --2- 172.21.15.32:0/1802981300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc701029b0 0x7fcc70073c90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800efc0 tx=0x7fcc6800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc57fff700 1 -- 172.21.15.32:0/1802981300 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc68018070 con 0x7fcc701029b0 2023-09-30T14:40:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc754eb700 1 -- 172.21.15.32:0/1802981300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc701a1060 con 0x7fcc701029b0 2023-09-30T14:40:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc57fff700 1 -- 172.21.15.32:0/1802981300 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc6800f040 con 0x7fcc701029b0 2023-09-30T14:40:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.213+0000 7fcc57fff700 1 -- 172.21.15.32:0/1802981300 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6800cd40 con 0x7fcc701029b0 2023-09-30T14:40:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.214+0000 7fcc754eb700 1 -- 172.21.15.32:0/1802981300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc701a1570 con 0x7fcc701029b0 2023-09-30T14:40:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.216+0000 7fcc57fff700 1 -- 172.21.15.32:0/1802981300 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fcc68003a00 con 0x7fcc701029b0 2023-09-30T14:40:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.217+0000 7fcc57fff700 1 --2- 172.21.15.32:0/1802981300 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcc580a0fa0 0x7fcc580a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.217+0000 7fcc6effd700 1 --2- 172.21.15.32:0/1802981300 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcc580a0fa0 0x7fcc580a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.217+0000 7fcc57fff700 1 -- 172.21.15.32:0/1802981300 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcc680cc210 con 0x7fcc701029b0 2023-09-30T14:40:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.218+0000 7fcc6effd700 1 --2- 172.21.15.32:0/1802981300 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcc580a0fa0 0x7fcc580a3460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcc60006010 tx=0x7fcc6000bba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.218+0000 7fcc816d3b80 1 Processor -- start 2023-09-30T14:40:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.218+0000 7fcc816d3b80 1 -- start start 2023-09-30T14:40:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.218+0000 7fcc816d3b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c59200 0x557be9c59620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.218+0000 7fcc816d3b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c5a310 0x557be9c5a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.218+0000 7fcc816d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c5acd0 0x557be9c61560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.218+0000 7fcc816d3b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be9bc1400 con 0x557be9c59200 2023-09-30T14:40:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.218+0000 7fcc816d3b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be9bb2bc0 con 0x557be9c5a310 2023-09-30T14:40:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.218+0000 7fcc816d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be9bb04e0 con 0x557be9c5acd0 2023-09-30T14:40:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.219+0000 7fcc6effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c59200 0x557be9c59620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.219+0000 7fcc6effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c59200 0x557be9c59620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36288/0 (socket says 172.21.15.32:36288) 2023-09-30T14:40:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.219+0000 7fcc6effd700 1 -- 172.21.15.32:0/4259979543 learned_addr learned my addr 172.21.15.32:0/4259979543 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.219+0000 7fcc6f7fe700 1 --2- 172.21.15.32:0/4259979543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c5acd0 0x557be9c61560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.219+0000 7fcc6e7fc700 1 --2- 172.21.15.32:0/4259979543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c5a310 0x557be9c5a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.219+0000 7fcc6f7fe700 1 -- 172.21.15.32:0/4259979543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c5a310 msgr2=0x557be9c5a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.219+0000 7fcc6f7fe700 1 --2- 172.21.15.32:0/4259979543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c5a310 0x557be9c5a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.219+0000 7fcc6f7fe700 1 -- 172.21.15.32:0/4259979543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c59200 msgr2=0x557be9c59620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.219+0000 7fcc6f7fe700 1 --2- 172.21.15.32:0/4259979543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c59200 0x557be9c59620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.219+0000 7fcc6f7fe700 1 -- 172.21.15.32:0/4259979543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557be9c0e870 con 0x557be9c5acd0 2023-09-30T14:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.219+0000 7fcc6effd700 1 --2- 172.21.15.32:0/4259979543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c59200 0x557be9c59620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.219+0000 7fcc6f7fe700 1 --2- 172.21.15.32:0/4259979543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c5acd0 0x557be9c61560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc680c9d70 tx=0x7fcc68099540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28e812729488e8c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.219+0000 7fcc557fa700 1 -- 172.21.15.32:0/4259979543 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc68004ad0 con 0x557be9c5acd0 2023-09-30T14:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.219+0000 7fcc557fa700 1 -- 172.21.15.32:0/4259979543 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc6801b320 con 0x557be9c5acd0 2023-09-30T14:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.219+0000 7fcc557fa700 1 -- 172.21.15.32:0/4259979543 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc68097b10 con 0x557be9c5acd0 2023-09-30T14:40:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.220+0000 7fcc816d3b80 1 -- 172.21.15.32:0/4259979543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c5acd0 msgr2=0x557be9c61560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.220+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/4259979543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c5acd0 0x557be9c61560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc680c9d70 tx=0x7fcc68099540 comp rx=0 tx=0).stop 2023-09-30T14:40:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.220+0000 7fcc816d3b80 1 -- 172.21.15.32:0/4259979543 shutdown_connections 2023-09-30T14:40:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.220+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/4259979543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c5acd0 0x557be9c61560 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.220+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/4259979543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c59200 0x557be9c59620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.220+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/4259979543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c5a310 0x557be9c5a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.220+0000 7fcc816d3b80 1 -- 172.21.15.32:0/4259979543 >> 172.21.15.32:0/4259979543 conn(0x557be9bcd200 msgr2=0x557be9c53ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.220+0000 7fcc816d3b80 1 -- 172.21.15.32:0/4259979543 shutdown_connections 2023-09-30T14:40:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.221+0000 7fcc816d3b80 1 -- 172.21.15.32:0/4259979543 wait complete. 2023-09-30T14:40:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.221+0000 7fcc816d3b80 1 -- start start 2023-09-30T14:40:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.221+0000 7fcc816d3b80 1 Processor -- start 2023-09-30T14:40:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.221+0000 7fcc816d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c59200 0x557be9c6b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.221+0000 7fcc816d3b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c5a310 0x557be9c6bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.221+0000 7fcc816d3b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c5acd0 0x557be9c72cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.221+0000 7fcc816d3b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be9babdd0 con 0x557be9c5a310 2023-09-30T14:40:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.221+0000 7fcc816d3b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be9b486b0 con 0x557be9c5acd0 2023-09-30T14:40:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.221+0000 7fcc816d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be9b48350 con 0x557be9c59200 2023-09-30T14:40:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.221+0000 7fcc816d3b80 10 client.0 _ll_register_callbacks cb 0x557be99a57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:40:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.222+0000 7fcc6e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c5a310 0x557be9c6bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.222+0000 7fcc6e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c5a310 0x557be9c6bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36292/0 (socket says 172.21.15.32:36292) 2023-09-30T14:40:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.222+0000 7fcc6e7fc700 1 -- 172.21.15.32:0/3958946564 learned_addr learned my addr 172.21.15.32:0/3958946564 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.222+0000 7fcc6effd700 1 --2- 172.21.15.32:0/3958946564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c59200 0x557be9c6b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.222+0000 7fcc6f7fe700 1 --2- 172.21.15.32:0/3958946564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c5acd0 0x557be9c72cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.222+0000 7fcc6e7fc700 1 -- 172.21.15.32:0/3958946564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c59200 msgr2=0x557be9c6b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.222+0000 7fcc6e7fc700 1 --2- 172.21.15.32:0/3958946564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c59200 0x557be9c6b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.222+0000 7fcc6e7fc700 1 -- 172.21.15.32:0/3958946564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c5acd0 msgr2=0x557be9c72cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.222+0000 7fcc6e7fc700 1 --2- 172.21.15.32:0/3958946564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c5acd0 0x557be9c72cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.222+0000 7fcc6e7fc700 1 -- 172.21.15.32:0/3958946564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc70109c00 con 0x557be9c5a310 2023-09-30T14:40:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.222+0000 7fcc6e7fc700 1 --2- 172.21.15.32:0/3958946564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c5a310 0x557be9c6bc70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fcc64011490 tx=0x7fcc640117a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76afd8764a9af7be server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.222+0000 7fcc52ff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:40:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.223+0000 7fcc52ff5700 1 -- 172.21.15.32:0/3958946564 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64016070 con 0x557be9c5a310 2023-09-30T14:40:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.223+0000 7fcc52ff5700 1 -- 172.21.15.32:0/3958946564 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcc64004320 con 0x557be9c5a310 2023-09-30T14:40:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.223+0000 7fcc816d3b80 1 -- 172.21.15.32:0/3958946564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557be9c4c090 con 0x557be9c5a310 2023-09-30T14:40:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.223+0000 7fcc52ff5700 1 -- 172.21.15.32:0/3958946564 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc64021030 con 0x557be9c5a310 2023-09-30T14:40:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.223+0000 7fcc816d3b80 10 client.15750 fetch_fsmap learned FSMap version 25 2023-09-30T14:40:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.223+0000 7fcc816d3b80 1 -- 172.21.15.32:0/3958946564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fcc701a1570 con 0x557be9c5a310 2023-09-30T14:40:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.223+0000 7fcc52ff5700 1 -- 172.21.15.32:0/3958946564 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557be9c4c090 con 0x557be9c5a310 2023-09-30T14:40:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.224+0000 7fcc52ff5700 1 -- 172.21.15.32:0/3958946564 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fcc6401b6f0 con 0x557be9c5a310 2023-09-30T14:40:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.224+0000 7fcc816d3b80 10 client.15750 fetch_fsmap finished waiting for FSMap version 25 2023-09-30T14:40:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.224+0000 7fcc816d3b80 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:40:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.224+0000 7fcc816d3b80 20 client.15750 populate_metadata read hostname 'smithi032' 2023-09-30T14:40:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.224+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/3958946564 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557be9c66f40 0x557be9c69360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.224+0000 7fcc816d3b80 4 client.15750 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:40:54.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.224+0000 7fcc816d3b80 1 -- 172.21.15.32:0/3958946564 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557be9ba4db0 con 0x557be9c66f40 2023-09-30T14:40:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.225+0000 7fcc6effd700 1 --2- 172.21.15.32:0/3958946564 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557be9c66f40 0x557be9c69360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.225+0000 7fcc52ff5700 10 client.15750 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.225+0000 7fcc6effd700 1 --2- 172.21.15.32:0/3958946564 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557be9c66f40 0x557be9c69360 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8a0860ec303dc08 server_cookie=abea5dd41fc558d0 in_seq=0 out_seq=0 2023-09-30T14:40:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.226+0000 7fcc52ff5700 1 -- 172.21.15.32:0/3958946564 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557be9ba4db0 con 0x557be9c66f40 2023-09-30T14:40:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.226+0000 7fcc52ff5700 10 client.15750 handle_command_reply: tid=0 2023-09-30T14:40:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.226+0000 7fcc816d3b80 1 client.15750 shutdown 2023-09-30T14:40:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.226+0000 7fcc816d3b80 1 -- 172.21.15.32:0/3958946564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c5a310 msgr2=0x557be9c6bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.226+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/3958946564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c5a310 0x557be9c6bc70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fcc64011490 tx=0x7fcc640117a0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.227+0000 7fcc816d3b80 1 -- 172.21.15.32:0/3958946564 shutdown_connections 2023-09-30T14:40:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.227+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/3958946564 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557be9c66f40 0x557be9c69360 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.227+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/3958946564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c5acd0 0x557be9c72cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.227+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/3958946564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c59200 0x557be9c6b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.227+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/3958946564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c5a310 0x557be9c6bc70 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.227+0000 7fcc52ff5700 0 client.15750 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.227+0000 7fcc816d3b80 1 -- 172.21.15.32:0/3958946564 >> 172.21.15.32:0/3958946564 conn(0x557be9bcd200 msgr2=0x557be9c53590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.227+0000 7fcc816d3b80 1 -- 172.21.15.32:0/3958946564 shutdown_connections 2023-09-30T14:40:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.227+0000 7fcc816d3b80 1 -- 172.21.15.32:0/3958946564 wait complete. 2023-09-30T14:40:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.228+0000 7fcc816d3b80 20 client.15750 trim_cache size 0 max 16384 2023-09-30T14:40:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.251+0000 7fcc816d3b80 1 Processor -- start 2023-09-30T14:40:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.251+0000 7fcc816d3b80 1 -- start start 2023-09-30T14:40:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc816d3b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c59200 0x557be9c625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc816d3b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c5a310 0x557be9c70c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc816d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c5acd0 0x557be9c5dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc816d3b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be9a2c8c0 con 0x557be9c59200 2023-09-30T14:40:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc816d3b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be9c623d0 con 0x557be9c5a310 2023-09-30T14:40:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc816d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be9c55a00 con 0x557be9c5acd0 2023-09-30T14:40:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc6effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c59200 0x557be9c625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc6effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c59200 0x557be9c625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36300/0 (socket says 172.21.15.32:36300) 2023-09-30T14:40:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc6effd700 1 -- 172.21.15.32:0/2097824959 learned_addr learned my addr 172.21.15.32:0/2097824959 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc6e7fc700 1 --2- 172.21.15.32:0/2097824959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c5a310 0x557be9c70c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc6f7fe700 1 --2- 172.21.15.32:0/2097824959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c5acd0 0x557be9c5dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc6effd700 1 -- 172.21.15.32:0/2097824959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c5acd0 msgr2=0x557be9c5dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc6effd700 1 --2- 172.21.15.32:0/2097824959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c5acd0 0x557be9c5dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc6effd700 1 -- 172.21.15.32:0/2097824959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c5a310 msgr2=0x557be9c70c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc6effd700 1 --2- 172.21.15.32:0/2097824959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c5a310 0x557be9c70c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc6effd700 1 -- 172.21.15.32:0/2097824959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557be99d6d80 con 0x557be9c59200 2023-09-30T14:40:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc6effd700 1 --2- 172.21.15.32:0/2097824959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c59200 0x557be9c625d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcc60005fb0 tx=0x7fcc60020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceef360139cb840a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc55ffb700 1 -- 172.21.15.32:0/2097824959 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60058b10 con 0x557be9c59200 2023-09-30T14:40:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc55ffb700 1 -- 172.21.15.32:0/2097824959 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc6001e040 con 0x557be9c59200 2023-09-30T14:40:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.252+0000 7fcc55ffb700 1 -- 172.21.15.32:0/2097824959 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6000f6c0 con 0x557be9c59200 2023-09-30T14:40:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.253+0000 7fcc816d3b80 1 -- 172.21.15.32:0/2097824959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c59200 msgr2=0x557be9c625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.253+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/2097824959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c59200 0x557be9c625d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcc60005fb0 tx=0x7fcc60020000 comp rx=0 tx=0).stop 2023-09-30T14:40:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.253+0000 7fcc816d3b80 1 -- 172.21.15.32:0/2097824959 shutdown_connections 2023-09-30T14:40:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.253+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/2097824959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c5acd0 0x557be9c5dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.253+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/2097824959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c59200 0x557be9c625d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.253+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/2097824959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c5a310 0x557be9c70c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.253+0000 7fcc816d3b80 1 -- 172.21.15.32:0/2097824959 >> 172.21.15.32:0/2097824959 conn(0x557be9bcd200 msgr2=0x557be9c62e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.253+0000 7fcc816d3b80 1 -- 172.21.15.32:0/2097824959 shutdown_connections 2023-09-30T14:40:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.253+0000 7fcc816d3b80 1 -- 172.21.15.32:0/2097824959 wait complete. 2023-09-30T14:40:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc816d3b80 1 -- start start 2023-09-30T14:40:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc816d3b80 1 Processor -- start 2023-09-30T14:40:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc816d3b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c59200 0x557be9c70b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc816d3b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c5a310 0x557be9c6f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc816d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c5acd0 0x557be9c6f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc816d3b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be9c6fbb0 con 0x557be9c5a310 2023-09-30T14:40:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc816d3b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be9c6fd20 con 0x557be9c59200 2023-09-30T14:40:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc816d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be9c6fee0 con 0x557be9c5acd0 2023-09-30T14:40:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc816d3b80 10 client.0 _ll_register_callbacks cb 0x7fcc64007b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:40:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc6e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c5a310 0x557be9c6f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc6e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c5a310 0x557be9c6f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36308/0 (socket says 172.21.15.32:36308) 2023-09-30T14:40:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc6e7fc700 1 -- 172.21.15.32:0/2405799200 learned_addr learned my addr 172.21.15.32:0/2405799200 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:40:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc6effd700 1 --2- 172.21.15.32:0/2405799200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c59200 0x557be9c70b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc6f7fe700 1 --2- 172.21.15.32:0/2405799200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c5acd0 0x557be9c6f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc6e7fc700 1 -- 172.21.15.32:0/2405799200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c5acd0 msgr2=0x557be9c6f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc6e7fc700 1 --2- 172.21.15.32:0/2405799200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c5acd0 0x557be9c6f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc6e7fc700 1 -- 172.21.15.32:0/2405799200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c59200 msgr2=0x557be9c70b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:40:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc6e7fc700 1 --2- 172.21.15.32:0/2405799200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c59200 0x557be9c70b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc6e7fc700 1 -- 172.21.15.32:0/2405799200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc600046c0 con 0x557be9c5a310 2023-09-30T14:40:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.254+0000 7fcc6effd700 1 --2- 172.21.15.32:0/2405799200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c59200 0x557be9c70b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:40:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.255+0000 7fcc6e7fc700 1 --2- 172.21.15.32:0/2405799200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c5a310 0x557be9c6f130 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fcc64000c00 tx=0x7fcc6401b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e81c3f2791ad1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:40:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.255+0000 7fcc52ff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:40:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.255+0000 7fcc52ff5700 1 -- 172.21.15.32:0/2405799200 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64021070 con 0x557be9c5a310 2023-09-30T14:40:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.255+0000 7fcc52ff5700 1 -- 172.21.15.32:0/2405799200 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcc64022070 con 0x557be9c5a310 2023-09-30T14:40:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.255+0000 7fcc816d3b80 1 -- 172.21.15.32:0/2405799200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc6801b320 con 0x557be9c5a310 2023-09-30T14:40:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.255+0000 7fcc52ff5700 1 -- 172.21.15.32:0/2405799200 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc64016040 con 0x557be9c5a310 2023-09-30T14:40:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.255+0000 7fcc816d3b80 10 client.15756 fetch_fsmap learned FSMap version 25 2023-09-30T14:40:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.255+0000 7fcc816d3b80 1 -- 172.21.15.32:0/2405799200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557be99bfc00 con 0x557be9c5a310 2023-09-30T14:40:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.255+0000 7fcc52ff5700 1 -- 172.21.15.32:0/2405799200 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc6403f020 con 0x557be9c5a310 2023-09-30T14:40:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.255+0000 7fcc52ff5700 1 -- 172.21.15.32:0/2405799200 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fcc6403b790 con 0x557be9c5a310 2023-09-30T14:40:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.255+0000 7fcc816d3b80 10 client.15756 fetch_fsmap finished waiting for FSMap version 25 2023-09-30T14:40:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.255+0000 7fcc816d3b80 10 client.15756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:40:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.255+0000 7fcc816d3b80 20 client.15756 populate_metadata read hostname 'smithi032' 2023-09-30T14:40:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.255+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/2405799200 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557be9cc3a80 0x557be9cc5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:40:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.255+0000 7fcc816d3b80 4 client.15756 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:40:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.255+0000 7fcc816d3b80 1 -- 172.21.15.32:0/2405799200 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557be9ba4ad0 con 0x557be9cc3a80 2023-09-30T14:40:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.256+0000 7fcc6effd700 1 --2- 172.21.15.32:0/2405799200 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557be9cc3a80 0x557be9cc5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:40:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.256+0000 7fcc52ff5700 10 client.15756 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.256+0000 7fcc6effd700 1 --2- 172.21.15.32:0/2405799200 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557be9cc3a80 0x557be9cc5ea0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f856b96329f35222 server_cookie=80857ba974e154d in_seq=0 out_seq=0 2023-09-30T14:40:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.303+0000 7fcc52ff5700 1 -- 172.21.15.32:0/2405799200 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557be9ba4ad0 con 0x557be9cc3a80 2023-09-30T14:40:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.303+0000 7fcc52ff5700 10 client.15756 handle_command_reply: tid=0 2023-09-30T14:40:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.303+0000 7fcc816d3b80 1 client.15756 shutdown 2023-09-30T14:40:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.303+0000 7fcc816d3b80 1 -- 172.21.15.32:0/2405799200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c5a310 msgr2=0x557be9c6f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.303+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/2405799200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c5a310 0x557be9c6f130 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fcc64000c00 tx=0x7fcc6401b650 comp rx=0 tx=0).stop 2023-09-30T14:40:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.303+0000 7fcc816d3b80 1 -- 172.21.15.32:0/2405799200 shutdown_connections 2023-09-30T14:40:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.303+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/2405799200 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557be9cc3a80 0x557be9cc5ea0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.303+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/2405799200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557be9c5acd0 0x557be9c6f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.303+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/2405799200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557be9c59200 0x557be9c70b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.303+0000 7fcc52ff5700 0 client.15756 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:40:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.303+0000 7fcc816d3b80 1 --2- 172.21.15.32:0/2405799200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557be9c5a310 0x557be9c6f130 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.303+0000 7fcc816d3b80 1 -- 172.21.15.32:0/2405799200 >> 172.21.15.32:0/2405799200 conn(0x557be9bcd200 msgr2=0x557be9c5dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.303+0000 7fcc816d3b80 1 -- 172.21.15.32:0/2405799200 shutdown_connections 2023-09-30T14:40:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.303+0000 7fcc816d3b80 1 -- 172.21.15.32:0/2405799200 wait complete. 2023-09-30T14:40:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.303+0000 7fcc816d3b80 20 client.15756 trim_cache size 0 max 16384 2023-09-30T14:40:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:40:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:40:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:40:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:40:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.304+0000 7fcc754eb700 1 -- 172.21.15.32:0/1802981300 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcc580a0fa0 msgr2=0x7fcc580a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.304+0000 7fcc754eb700 1 --2- 172.21.15.32:0/1802981300 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcc580a0fa0 0x7fcc580a3460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcc60006010 tx=0x7fcc6000bba0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.304+0000 7fcc754eb700 1 -- 172.21.15.32:0/1802981300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc701029b0 msgr2=0x7fcc70073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:40:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.304+0000 7fcc754eb700 1 --2- 172.21.15.32:0/1802981300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc701029b0 0x7fcc70073c90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800efc0 tx=0x7fcc6800c3a0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.304+0000 7fcc754eb700 1 -- 172.21.15.32:0/1802981300 shutdown_connections 2023-09-30T14:40:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.304+0000 7fcc754eb700 1 --2- 172.21.15.32:0/1802981300 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcc580a0fa0 0x7fcc580a3460 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.304+0000 7fcc754eb700 1 --2- 172.21.15.32:0/1802981300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc701029b0 0x7fcc70073c90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.304+0000 7fcc754eb700 1 --2- 172.21.15.32:0/1802981300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc70100ea0 0x7fcc70073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.304+0000 7fcc754eb700 1 --2- 172.21.15.32:0/1802981300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc70101ff0 0x7fcc70073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:40:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.304+0000 7fcc754eb700 1 -- 172.21.15.32:0/1802981300 >> 172.21.15.32:0/1802981300 conn(0x7fcc700fc3f0 msgr2=0x7fcc700fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:40:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.304+0000 7fcc754eb700 1 -- 172.21.15.32:0/1802981300 shutdown_connections 2023-09-30T14:40:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:40:54.304+0000 7fcc754eb700 1 -- 172.21.15.32:0/1802981300 wait complete. 2023-09-30T14:40:54.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 208894 12.99 MB/sec warmup 88 sec latency 52.078 ms 2023-09-30T14:40:55.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:55 smithi032 ceph-mon[98592]: pgmap v366: 97 pgs: 97 active+clean; 92 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 339 op/s 2023-09-30T14:40:55.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:55 smithi156 ceph-mon[103579]: pgmap v366: 97 pgs: 97 active+clean; 92 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 339 op/s 2023-09-30T14:40:55.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:55 smithi148 ceph-mon[102355]: pgmap v366: 97 pgs: 97 active+clean; 92 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 339 op/s 2023-09-30T14:40:55.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 210351 12.93 MB/sec warmup 89 sec latency 15.451 ms 2023-09-30T14:40:56.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 213064 12.98 MB/sec warmup 90 sec latency 25.755 ms 2023-09-30T14:40:57.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:57 smithi032 ceph-mon[98592]: pgmap v367: 97 pgs: 97 active+clean; 95 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 319 op/s 2023-09-30T14:40:57.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:57 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:40:57.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:57 smithi156 ceph-mon[103579]: pgmap v367: 97 pgs: 97 active+clean; 95 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 319 op/s 2023-09-30T14:40:57.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:57 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:40:57.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:57 smithi148 ceph-mon[102355]: pgmap v367: 97 pgs: 97 active+clean; 95 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 319 op/s 2023-09-30T14:40:57.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:57 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:40:57.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 214596 12.90 MB/sec warmup 91 sec latency 14.049 ms 2023-09-30T14:40:58.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 216881 12.91 MB/sec warmup 92 sec latency 10.024 ms 2023-09-30T14:40:59.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:40:59 smithi032 ceph-mon[98592]: pgmap v368: 97 pgs: 97 active+clean; 97 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 335 op/s 2023-09-30T14:40:59.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:40:59 smithi156 ceph-mon[103579]: pgmap v368: 97 pgs: 97 active+clean; 97 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 335 op/s 2023-09-30T14:40:59.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:40:59 smithi148 ceph-mon[102355]: pgmap v368: 97 pgs: 97 active+clean; 97 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 335 op/s 2023-09-30T14:40:59.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 219180 12.89 MB/sec warmup 93 sec latency 44.469 ms 2023-09-30T14:41:00.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 220687 12.84 MB/sec warmup 94 sec latency 18.197 ms 2023-09-30T14:41:01.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:01 smithi032 ceph-mon[98592]: pgmap v369: 97 pgs: 97 active+clean; 95 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 300 op/s 2023-09-30T14:41:01.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:01 smithi156 ceph-mon[103579]: pgmap v369: 97 pgs: 97 active+clean; 95 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 300 op/s 2023-09-30T14:41:01.608 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:01 smithi148 ceph-mon[102355]: pgmap v369: 97 pgs: 97 active+clean; 95 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 300 op/s 2023-09-30T14:41:01.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 223600 12.89 MB/sec warmup 95 sec latency 11.299 ms 2023-09-30T14:41:02.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 224966 12.82 MB/sec warmup 96 sec latency 16.254 ms 2023-09-30T14:41:03.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:03 smithi156 ceph-mon[103579]: pgmap v370: 97 pgs: 97 active+clean; 97 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2023-09-30T14:41:03.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:03 smithi148 ceph-mon[102355]: pgmap v370: 97 pgs: 97 active+clean; 97 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2023-09-30T14:41:03.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:03 smithi032 ceph-mon[98592]: pgmap v370: 97 pgs: 97 active+clean; 97 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2023-09-30T14:41:03.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 227367 12.83 MB/sec warmup 97 sec latency 11.490 ms 2023-09-30T14:41:04.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 228917 12.76 MB/sec warmup 98 sec latency 68.896 ms 2023-09-30T14:41:05.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:05 smithi156 ceph-mon[103579]: pgmap v371: 97 pgs: 97 active+clean; 99 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 327 op/s 2023-09-30T14:41:05.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:05 smithi148 ceph-mon[102355]: pgmap v371: 97 pgs: 97 active+clean; 99 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 327 op/s 2023-09-30T14:41:05.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:05 smithi032 ceph-mon[98592]: pgmap v371: 97 pgs: 97 active+clean; 99 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 327 op/s 2023-09-30T14:41:05.693 INFO:tasks.workunit.client.0.smithi032.stdout: 1 230992 12.77 MB/sec warmup 99 sec latency 117.265 ms 2023-09-30T14:41:06.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 233052 12.71 MB/sec warmup 100 sec latency 11.224 ms 2023-09-30T14:41:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.422+0000 7f4a4ce86700 1 -- 172.21.15.32:0/1095258432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a48100e60 msgr2=0x7f4a48101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.422+0000 7f4a4ce86700 1 --2- 172.21.15.32:0/1095258432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a48100e60 0x7f4a48101280 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009990 tx=0x7f4a38009ca0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.422+0000 7f4a4ce86700 1 -- 172.21.15.32:0/1095258432 shutdown_connections 2023-09-30T14:41:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.422+0000 7f4a4ce86700 1 --2- 172.21.15.32:0/1095258432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a48102970 0x7f4a48109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.422+0000 7f4a4ce86700 1 --2- 172.21.15.32:0/1095258432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a48100e60 0x7f4a48101280 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.422+0000 7f4a4ce86700 1 --2- 172.21.15.32:0/1095258432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a48101fb0 0x7f4a48102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.422+0000 7f4a4ce86700 1 -- 172.21.15.32:0/1095258432 >> 172.21.15.32:0/1095258432 conn(0x7f4a480fc3d0 msgr2=0x7f4a480fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.422+0000 7f4a4ce86700 1 -- 172.21.15.32:0/1095258432 shutdown_connections 2023-09-30T14:41:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.422+0000 7f4a4ce86700 1 -- 172.21.15.32:0/1095258432 wait complete. 2023-09-30T14:41:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.423+0000 7f4a4ce86700 1 Processor -- start 2023-09-30T14:41:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.423+0000 7f4a4ce86700 1 -- start start 2023-09-30T14:41:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.423+0000 7f4a4ce86700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a48100e60 0x7f4a48196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.423+0000 7f4a4ce86700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a48101fb0 0x7f4a48197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.423+0000 7f4a4ce86700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a48102970 0x7f4a48197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.423+0000 7f4a4ce86700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a48198650 con 0x7f4a48101fb0 2023-09-30T14:41:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.423+0000 7f4a4ce86700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a481987c0 con 0x7f4a48100e60 2023-09-30T14:41:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.423+0000 7f4a4ce86700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a48198980 con 0x7f4a48102970 2023-09-30T14:41:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.423+0000 7f4a45d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a48101fb0 0x7f4a48197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.423+0000 7f4a4659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a48100e60 0x7f4a48196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.424+0000 7f4a4659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a48100e60 0x7f4a48196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47796/0 (socket says 172.21.15.32:47796) 2023-09-30T14:41:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.424+0000 7f4a4659c700 1 -- 172.21.15.32:0/1379933425 learned_addr learned my addr 172.21.15.32:0/1379933425 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.424+0000 7f4a46d9d700 1 --2- 172.21.15.32:0/1379933425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a48102970 0x7f4a48197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.424+0000 7f4a45d9b700 1 -- 172.21.15.32:0/1379933425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a48102970 msgr2=0x7f4a48197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.424+0000 7f4a45d9b700 1 --2- 172.21.15.32:0/1379933425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a48102970 0x7f4a48197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.424+0000 7f4a45d9b700 1 -- 172.21.15.32:0/1379933425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a48100e60 msgr2=0x7f4a48196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.424+0000 7f4a45d9b700 1 --2- 172.21.15.32:0/1379933425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a48100e60 0x7f4a48196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.424+0000 7f4a45d9b700 1 -- 172.21.15.32:0/1379933425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a380095f0 con 0x7f4a48101fb0 2023-09-30T14:41:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.424+0000 7f4a4659c700 1 --2- 172.21.15.32:0/1379933425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a48100e60 0x7f4a48196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:41:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.424+0000 7f4a45d9b700 1 --2- 172.21.15.32:0/1379933425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a48101fb0 0x7f4a48197170 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00e9f0 tx=0x7f4a3c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.424+0000 7f4a377fe700 1 -- 172.21.15.32:0/1379933425 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00c9c0 con 0x7f4a48101fb0 2023-09-30T14:41:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.424+0000 7f4a4ce86700 1 -- 172.21.15.32:0/1379933425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a48198c60 con 0x7f4a48101fb0 2023-09-30T14:41:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.424+0000 7f4a377fe700 1 -- 172.21.15.32:0/1379933425 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a3c010430 con 0x7f4a48101fb0 2023-09-30T14:41:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.424+0000 7f4a377fe700 1 -- 172.21.15.32:0/1379933425 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c018680 con 0x7f4a48101fb0 2023-09-30T14:41:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.425+0000 7f4a4ce86700 1 -- 172.21.15.32:0/1379933425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4819f420 con 0x7f4a48101fb0 2023-09-30T14:41:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.427+0000 7f4a377fe700 1 -- 172.21.15.32:0/1379933425 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f4a3c010860 con 0x7f4a48101fb0 2023-09-30T14:41:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.427+0000 7f4a377fe700 1 --2- 172.21.15.32:0/1379933425 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4a300a0fa0 0x7f4a300a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.428+0000 7f4a377fe700 1 -- 172.21.15.32:0/1379933425 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4a3c014070 con 0x7f4a48101fb0 2023-09-30T14:41:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.428+0000 7f4a4659c700 1 --2- 172.21.15.32:0/1379933425 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4a300a0fa0 0x7f4a300a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.428+0000 7f4a4659c700 1 --2- 172.21.15.32:0/1379933425 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4a300a0fa0 0x7f4a300a3460 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800b2e0 tx=0x7f4a38004500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:07.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.429+0000 7f4a5906eb80 1 Processor -- start 2023-09-30T14:41:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.429+0000 7f4a5906eb80 1 -- start start 2023-09-30T14:41:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a5906eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fdd200 0x560c92fdd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a5906eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fde310 0x560c92fde790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a5906eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdecd0 0x560c92fe5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a5906eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c92f45400 con 0x560c92fdd200 2023-09-30T14:41:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a5906eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c92f36bc0 con 0x560c92fde310 2023-09-30T14:41:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a5906eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c92f344e0 con 0x560c92fdecd0 2023-09-30T14:41:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a4659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fdd200 0x560c92fdd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a4659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fdd200 0x560c92fdd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36328/0 (socket says 172.21.15.32:36328) 2023-09-30T14:41:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a4659c700 1 -- 172.21.15.32:0/2967279028 learned_addr learned my addr 172.21.15.32:0/2967279028 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a45d9b700 1 --2- 172.21.15.32:0/2967279028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fde310 0x560c92fde790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a46d9d700 1 --2- 172.21.15.32:0/2967279028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdecd0 0x560c92fe5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a45d9b700 1 -- 172.21.15.32:0/2967279028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdecd0 msgr2=0x560c92fe5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a45d9b700 1 --2- 172.21.15.32:0/2967279028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdecd0 0x560c92fe5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a45d9b700 1 -- 172.21.15.32:0/2967279028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fdd200 msgr2=0x560c92fdd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a45d9b700 1 --2- 172.21.15.32:0/2967279028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fdd200 0x560c92fdd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a45d9b700 1 -- 172.21.15.32:0/2967279028 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c92f92870 con 0x560c92fde310 2023-09-30T14:41:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a45d9b700 1 --2- 172.21.15.32:0/2967279028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fde310 0x560c92fde790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0ca350 tx=0x7f4a3c099c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2d4b9258c3f808f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a34ff9700 1 -- 172.21.15.32:0/2967279028 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c099d10 con 0x560c92fde310 2023-09-30T14:41:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a34ff9700 1 -- 172.21.15.32:0/2967279028 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a3c09de60 con 0x560c92fde310 2023-09-30T14:41:07.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.430+0000 7f4a34ff9700 1 -- 172.21.15.32:0/2967279028 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c099460 con 0x560c92fde310 2023-09-30T14:41:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.431+0000 7f4a5906eb80 1 -- 172.21.15.32:0/2967279028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fde310 msgr2=0x560c92fde790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.431+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/2967279028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fde310 0x560c92fde790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0ca350 tx=0x7f4a3c099c10 comp rx=0 tx=0).stop 2023-09-30T14:41:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.431+0000 7f4a5906eb80 1 -- 172.21.15.32:0/2967279028 shutdown_connections 2023-09-30T14:41:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.431+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/2967279028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdecd0 0x560c92fe5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.431+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/2967279028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fdd200 0x560c92fdd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.431+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/2967279028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fde310 0x560c92fde790 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.431+0000 7f4a5906eb80 1 -- 172.21.15.32:0/2967279028 >> 172.21.15.32:0/2967279028 conn(0x560c92f51200 msgr2=0x560c92fd7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.431+0000 7f4a5906eb80 1 -- 172.21.15.32:0/2967279028 shutdown_connections 2023-09-30T14:41:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.431+0000 7f4a5906eb80 1 -- 172.21.15.32:0/2967279028 wait complete. 2023-09-30T14:41:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a5906eb80 1 -- start start 2023-09-30T14:41:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a5906eb80 1 Processor -- start 2023-09-30T14:41:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a5906eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fdd200 0x560c92fe6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a5906eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fde310 0x560c92fe72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a5906eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdecd0 0x560c92fee320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a5906eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c92f2fdd0 con 0x560c92fde310 2023-09-30T14:41:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a5906eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c92ecc6b0 con 0x560c92fdd200 2023-09-30T14:41:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a5906eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c92ecc350 con 0x560c92fdecd0 2023-09-30T14:41:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a5906eb80 10 client.0 _ll_register_callbacks cb 0x560c92d297a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:41:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a45d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fde310 0x560c92fe72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a45d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fde310 0x560c92fe72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36336/0 (socket says 172.21.15.32:36336) 2023-09-30T14:41:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a45d9b700 1 -- 172.21.15.32:0/2087533924 learned_addr learned my addr 172.21.15.32:0/2087533924 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a4659c700 1 --2- 172.21.15.32:0/2087533924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fdd200 0x560c92fe6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a46d9d700 1 --2- 172.21.15.32:0/2087533924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdecd0 0x560c92fee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a45d9b700 1 -- 172.21.15.32:0/2087533924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdecd0 msgr2=0x560c92fee320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a45d9b700 1 --2- 172.21.15.32:0/2087533924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdecd0 0x560c92fee320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a45d9b700 1 -- 172.21.15.32:0/2087533924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fdd200 msgr2=0x560c92fe6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a45d9b700 1 --2- 172.21.15.32:0/2087533924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fdd200 0x560c92fe6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a45d9b700 1 -- 172.21.15.32:0/2087533924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4819f420 con 0x560c92fde310 2023-09-30T14:41:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.432+0000 7f4a45d9b700 1 --2- 172.21.15.32:0/2087533924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fde310 0x560c92fe72e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0a1b70 tx=0x7f4a3c0105c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e2ab995d5693ed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.433+0000 7f4a2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:41:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.433+0000 7f4a2dffb700 1 -- 172.21.15.32:0/2087533924 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c099d90 con 0x560c92fde310 2023-09-30T14:41:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.433+0000 7f4a2dffb700 1 -- 172.21.15.32:0/2087533924 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4a3c0a1c10 con 0x560c92fde310 2023-09-30T14:41:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.433+0000 7f4a5906eb80 1 -- 172.21.15.32:0/2087533924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c92fd0090 con 0x560c92fde310 2023-09-30T14:41:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.433+0000 7f4a2dffb700 1 -- 172.21.15.32:0/2087533924 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a3c0a1dd0 con 0x560c92fde310 2023-09-30T14:41:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.433+0000 7f4a5906eb80 10 client.15768 fetch_fsmap learned FSMap version 26 2023-09-30T14:41:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.433+0000 7f4a5906eb80 1 -- 172.21.15.32:0/2087533924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560c92d5ad80 con 0x560c92fde310 2023-09-30T14:41:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.433+0000 7f4a2dffb700 1 -- 172.21.15.32:0/2087533924 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a3c07c020 con 0x560c92fde310 2023-09-30T14:41:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.433+0000 7f4a2dffb700 1 -- 172.21.15.32:0/2087533924 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4a3c098c70 con 0x560c92fde310 2023-09-30T14:41:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.433+0000 7f4a5906eb80 10 client.15768 fetch_fsmap finished waiting for FSMap version 26 2023-09-30T14:41:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.433+0000 7f4a5906eb80 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:41:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.433+0000 7f4a5906eb80 20 client.15768 populate_metadata read hostname 'smithi032' 2023-09-30T14:41:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.433+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/2087533924 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560c92ff28d0 0x560c92ff4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.433+0000 7f4a5906eb80 4 client.15768 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:41:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.433+0000 7f4a5906eb80 1 -- 172.21.15.32:0/2087533924 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c92f28db0 con 0x560c92ff28d0 2023-09-30T14:41:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.433+0000 7f4a4659c700 1 --2- 172.21.15.32:0/2087533924 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560c92ff28d0 0x560c92ff4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.434+0000 7f4a2dffb700 10 client.15768 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:41:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.434+0000 7f4a4659c700 1 --2- 172.21.15.32:0/2087533924 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560c92ff28d0 0x560c92ff4cf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c19a2687d849d0be server_cookie=8605c742b2a4f5bd in_seq=0 out_seq=0 2023-09-30T14:41:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.434+0000 7f4a2dffb700 1 -- 172.21.15.32:0/2087533924 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560c92f28db0 con 0x560c92ff28d0 2023-09-30T14:41:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.434+0000 7f4a2dffb700 10 client.15768 handle_command_reply: tid=0 2023-09-30T14:41:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.434+0000 7f4a5906eb80 1 client.15768 shutdown 2023-09-30T14:41:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.434+0000 7f4a5906eb80 1 -- 172.21.15.32:0/2087533924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fde310 msgr2=0x560c92fe72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.434+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/2087533924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fde310 0x560c92fe72e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0a1b70 tx=0x7f4a3c0105c0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.435+0000 7f4a5906eb80 1 -- 172.21.15.32:0/2087533924 shutdown_connections 2023-09-30T14:41:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.435+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/2087533924 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560c92ff28d0 0x560c92ff4cf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.435+0000 7f4a2dffb700 0 client.15768 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:41:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.435+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/2087533924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdecd0 0x560c92fee320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.435+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/2087533924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fdd200 0x560c92fe6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.435+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/2087533924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fde310 0x560c92fe72e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.435+0000 7f4a5906eb80 1 -- 172.21.15.32:0/2087533924 >> 172.21.15.32:0/2087533924 conn(0x560c92f51200 msgr2=0x560c92fd7590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.435+0000 7f4a5906eb80 1 -- 172.21.15.32:0/2087533924 shutdown_connections 2023-09-30T14:41:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.435+0000 7f4a5906eb80 1 -- 172.21.15.32:0/2087533924 wait complete. 2023-09-30T14:41:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.435+0000 7f4a5906eb80 20 client.15768 trim_cache size 0 max 16384 2023-09-30T14:41:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a5906eb80 1 Processor -- start 2023-09-30T14:41:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a5906eb80 1 -- start start 2023-09-30T14:41:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a5906eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fdd200 0x560c92fd7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a5906eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fde310 0x560c92fee2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a5906eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fdecd0 0x560c92fe6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a5906eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c92db08c0 con 0x560c92fdecd0 2023-09-30T14:41:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a5906eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c92fcfc60 con 0x560c92fdd200 2023-09-30T14:41:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a5906eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c92fdda80 con 0x560c92fde310 2023-09-30T14:41:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a46d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fdecd0 0x560c92fe6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a45d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fde310 0x560c92fee2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a46d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fdecd0 0x560c92fe6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36346/0 (socket says 172.21.15.32:36346) 2023-09-30T14:41:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a46d9d700 1 -- 172.21.15.32:0/3348831287 learned_addr learned my addr 172.21.15.32:0/3348831287 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a4659c700 1 --2- 172.21.15.32:0/3348831287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fdd200 0x560c92fd7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a45d9b700 1 -- 172.21.15.32:0/3348831287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fdd200 msgr2=0x560c92fd7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a45d9b700 1 --2- 172.21.15.32:0/3348831287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fdd200 0x560c92fd7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a45d9b700 1 -- 172.21.15.32:0/3348831287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fdecd0 msgr2=0x560c92fe6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a45d9b700 1 --2- 172.21.15.32:0/3348831287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fdecd0 0x560c92fe6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.451+0000 7f4a45d9b700 1 -- 172.21.15.32:0/3348831287 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c92d43c00 con 0x560c92fde310 2023-09-30T14:41:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.452+0000 7f4a45d9b700 1 --2- 172.21.15.32:0/3348831287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fde310 0x560c92fee2e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00e9c0 tx=0x7f4a3c0982c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acbc6b5ad96957ae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.452+0000 7f4a357fa700 1 -- 172.21.15.32:0/3348831287 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c096050 con 0x560c92fde310 2023-09-30T14:41:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.452+0000 7f4a357fa700 1 -- 172.21.15.32:0/3348831287 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a3c07c070 con 0x560c92fde310 2023-09-30T14:41:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.452+0000 7f4a357fa700 1 -- 172.21.15.32:0/3348831287 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c0a1c10 con 0x560c92fde310 2023-09-30T14:41:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 -- 172.21.15.32:0/3348831287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fde310 msgr2=0x560c92fee2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/3348831287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fde310 0x560c92fee2e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00e9c0 tx=0x7f4a3c0982c0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 -- 172.21.15.32:0/3348831287 shutdown_connections 2023-09-30T14:41:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/3348831287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fdecd0 0x560c92fe6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/3348831287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fdd200 0x560c92fd7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/3348831287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fde310 0x560c92fee2e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 -- 172.21.15.32:0/3348831287 >> 172.21.15.32:0/3348831287 conn(0x560c92f51200 msgr2=0x560c92fd7590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 -- 172.21.15.32:0/3348831287 shutdown_connections 2023-09-30T14:41:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 -- 172.21.15.32:0/3348831287 wait complete. 2023-09-30T14:41:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 -- start start 2023-09-30T14:41:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 Processor -- start 2023-09-30T14:41:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdd200 0x560c92febbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fde310 0x560c92fec110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fdecd0 0x560c92fed120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c92fed660 con 0x560c92fde310 2023-09-30T14:41:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c92fed7d0 con 0x560c92fdecd0 2023-09-30T14:41:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c92fed990 con 0x560c92fdd200 2023-09-30T14:41:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.453+0000 7f4a5906eb80 10 client.0 _ll_register_callbacks cb 0x7f4a3800bf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:41:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a4659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdd200 0x560c92febbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a45d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fde310 0x560c92fec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a4659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdd200 0x560c92febbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:50730/0 (socket says 172.21.15.32:50730) 2023-09-30T14:41:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a4659c700 1 -- 172.21.15.32:0/550535254 learned_addr learned my addr 172.21.15.32:0/550535254 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a4659c700 1 -- 172.21.15.32:0/550535254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fdecd0 msgr2=0x560c92fed120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a4659c700 1 --2- 172.21.15.32:0/550535254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fdecd0 0x560c92fed120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a4659c700 1 -- 172.21.15.32:0/550535254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fde310 msgr2=0x560c92fec110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a4659c700 1 --2- 172.21.15.32:0/550535254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fde310 0x560c92fec110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a4659c700 1 -- 172.21.15.32:0/550535254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c92d5ad80 con 0x560c92fdd200 2023-09-30T14:41:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a45d9b700 1 --2- 172.21.15.32:0/550535254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fde310 0x560c92fec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:41:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a4659c700 1 --2- 172.21.15.32:0/550535254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdd200 0x560c92febbd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4a38005850 tx=0x7f4a3801a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b93c7f7480b8405f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:41:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a2dffb700 1 -- 172.21.15.32:0/550535254 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3802d040 con 0x560c92fdd200 2023-09-30T14:41:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a2dffb700 1 -- 172.21.15.32:0/550535254 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4a3800fe80 con 0x560c92fdd200 2023-09-30T14:41:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a5906eb80 1 -- 172.21.15.32:0/550535254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a3c09de60 con 0x560c92fdd200 2023-09-30T14:41:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a2dffb700 1 -- 172.21.15.32:0/550535254 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a38035250 con 0x560c92fdd200 2023-09-30T14:41:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a5906eb80 10 client.25456 fetch_fsmap learned FSMap version 26 2023-09-30T14:41:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.454+0000 7f4a5906eb80 1 -- 172.21.15.32:0/550535254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560c92d811f0 con 0x560c92fdd200 2023-09-30T14:41:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.455+0000 7f4a2dffb700 1 -- 172.21.15.32:0/550535254 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a3c09de60 con 0x560c92fdd200 2023-09-30T14:41:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.455+0000 7f4a2dffb700 1 -- 172.21.15.32:0/550535254 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4a38034540 con 0x560c92fdd200 2023-09-30T14:41:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.455+0000 7f4a5906eb80 10 client.25456 fetch_fsmap finished waiting for FSMap version 26 2023-09-30T14:41:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.455+0000 7f4a5906eb80 10 client.25456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:41:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.455+0000 7f4a5906eb80 20 client.25456 populate_metadata read hostname 'smithi032' 2023-09-30T14:41:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.455+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/550535254 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560c92ff28d0 0x560c930491a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.455+0000 7f4a5906eb80 4 client.25456 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:41:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.455+0000 7f4a5906eb80 1 -- 172.21.15.32:0/550535254 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c92f28ad0 con 0x560c92ff28d0 2023-09-30T14:41:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.455+0000 7f4a45d9b700 1 --2- 172.21.15.32:0/550535254 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560c92ff28d0 0x560c930491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.455+0000 7f4a2dffb700 10 client.25456 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:41:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.455+0000 7f4a45d9b700 1 --2- 172.21.15.32:0/550535254 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560c92ff28d0 0x560c930491a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74af3fc5684c3f86 server_cookie=151799082c21ed3f in_seq=0 out_seq=0 2023-09-30T14:41:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.493+0000 7f4a2dffb700 1 -- 172.21.15.32:0/550535254 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c92f28ad0 con 0x560c92ff28d0 2023-09-30T14:41:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.493+0000 7f4a2dffb700 10 client.25456 handle_command_reply: tid=0 2023-09-30T14:41:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.493+0000 7f4a5906eb80 1 client.25456 shutdown 2023-09-30T14:41:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.493+0000 7f4a5906eb80 1 -- 172.21.15.32:0/550535254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdd200 msgr2=0x560c92febbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.493+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/550535254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdd200 0x560c92febbd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4a38005850 tx=0x7f4a3801a050 comp rx=0 tx=0).stop 2023-09-30T14:41:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.493+0000 7f4a5906eb80 1 -- 172.21.15.32:0/550535254 shutdown_connections 2023-09-30T14:41:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.493+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/550535254 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560c92ff28d0 0x560c930491a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.493+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/550535254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c92fdecd0 0x560c92fed120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.493+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/550535254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c92fdd200 0x560c92febbd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.493+0000 7f4a2dffb700 0 client.25456 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:41:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.493+0000 7f4a5906eb80 1 --2- 172.21.15.32:0/550535254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560c92fde310 0x560c92fec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.493+0000 7f4a5906eb80 1 -- 172.21.15.32:0/550535254 >> 172.21.15.32:0/550535254 conn(0x560c92f51200 msgr2=0x560c92fe6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.493+0000 7f4a5906eb80 1 -- 172.21.15.32:0/550535254 shutdown_connections 2023-09-30T14:41:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.493+0000 7f4a5906eb80 1 -- 172.21.15.32:0/550535254 wait complete. 2023-09-30T14:41:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.493+0000 7f4a5906eb80 20 client.25456 trim_cache size 0 max 16384 2023-09-30T14:41:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:41:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:41:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:41:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:41:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.494+0000 7f4a4ce86700 1 -- 172.21.15.32:0/1379933425 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4a300a0fa0 msgr2=0x7f4a300a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.494+0000 7f4a4ce86700 1 --2- 172.21.15.32:0/1379933425 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4a300a0fa0 0x7f4a300a3460 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800b2e0 tx=0x7f4a38004500 comp rx=0 tx=0).stop 2023-09-30T14:41:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.494+0000 7f4a4ce86700 1 -- 172.21.15.32:0/1379933425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a48101fb0 msgr2=0x7f4a48197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.494+0000 7f4a4ce86700 1 --2- 172.21.15.32:0/1379933425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a48101fb0 0x7f4a48197170 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00e9f0 tx=0x7f4a3c00ed00 comp rx=0 tx=0).stop 2023-09-30T14:41:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.494+0000 7f4a4ce86700 1 -- 172.21.15.32:0/1379933425 shutdown_connections 2023-09-30T14:41:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.494+0000 7f4a4ce86700 1 --2- 172.21.15.32:0/1379933425 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4a300a0fa0 0x7f4a300a3460 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.494+0000 7f4a4ce86700 1 --2- 172.21.15.32:0/1379933425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a48102970 0x7f4a48197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.494+0000 7f4a4ce86700 1 --2- 172.21.15.32:0/1379933425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a48100e60 0x7f4a48196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.494+0000 7f4a4ce86700 1 --2- 172.21.15.32:0/1379933425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a48101fb0 0x7f4a48197170 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.494+0000 7f4a4ce86700 1 -- 172.21.15.32:0/1379933425 >> 172.21.15.32:0/1379933425 conn(0x7f4a480fc3d0 msgr2=0x7f4a480fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.494+0000 7f4a4ce86700 1 -- 172.21.15.32:0/1379933425 shutdown_connections 2023-09-30T14:41:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:07.494+0000 7f4a4ce86700 1 -- 172.21.15.32:0/1379933425 wait complete. 2023-09-30T14:41:07.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:07 smithi156 ceph-mon[103579]: pgmap v372: 97 pgs: 97 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 332 op/s 2023-09-30T14:41:07.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:07 smithi148 ceph-mon[102355]: pgmap v372: 97 pgs: 97 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 332 op/s 2023-09-30T14:41:07.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:07 smithi032 ceph-mon[98592]: pgmap v372: 97 pgs: 97 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 332 op/s 2023-09-30T14:41:07.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 234683 12.70 MB/sec warmup 101 sec latency 19.810 ms 2023-09-30T14:41:08.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 237314 12.71 MB/sec warmup 102 sec latency 14.782 ms 2023-09-30T14:41:09.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:09 smithi156 ceph-mon[103579]: pgmap v373: 97 pgs: 97 active+clean; 102 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 320 op/s 2023-09-30T14:41:09.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:09 smithi148 ceph-mon[102355]: pgmap v373: 97 pgs: 97 active+clean; 102 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 320 op/s 2023-09-30T14:41:09.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:09 smithi032 ceph-mon[98592]: pgmap v373: 97 pgs: 97 active+clean; 102 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 320 op/s 2023-09-30T14:41:09.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 238486 12.65 MB/sec warmup 103 sec latency 48.940 ms 2023-09-30T14:41:10.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 241356 12.69 MB/sec warmup 104 sec latency 22.066 ms 2023-09-30T14:41:11.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:11 smithi156 ceph-mon[103579]: pgmap v374: 97 pgs: 97 active+clean; 103 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 2.7 MiB/s wr, 273 op/s 2023-09-30T14:41:11.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:11 smithi148 ceph-mon[102355]: pgmap v374: 97 pgs: 97 active+clean; 103 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 2.7 MiB/s wr, 273 op/s 2023-09-30T14:41:11.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:11 smithi032 ceph-mon[98592]: pgmap v374: 97 pgs: 97 active+clean; 103 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 2.7 MiB/s wr, 273 op/s 2023-09-30T14:41:11.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 242801 12.63 MB/sec warmup 105 sec latency 31.415 ms 2023-09-30T14:41:12.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 245175 12.65 MB/sec warmup 106 sec latency 9.395 ms 2023-09-30T14:41:13.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:13 smithi156 ceph-mon[103579]: pgmap v375: 97 pgs: 97 active+clean; 104 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 308 op/s 2023-09-30T14:41:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:13 smithi148 ceph-mon[102355]: pgmap v375: 97 pgs: 97 active+clean; 104 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 308 op/s 2023-09-30T14:41:13.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:13 smithi032 ceph-mon[98592]: pgmap v375: 97 pgs: 97 active+clean; 104 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 308 op/s 2023-09-30T14:41:13.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 247228 12.59 MB/sec warmup 107 sec latency 55.454 ms 2023-09-30T14:41:14.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 248922 12.59 MB/sec warmup 108 sec latency 32.387 ms 2023-09-30T14:41:15.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:15 smithi156 ceph-mon[103579]: pgmap v376: 97 pgs: 97 active+clean; 102 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 330 op/s 2023-09-30T14:41:15.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:15 smithi148 ceph-mon[102355]: pgmap v376: 97 pgs: 97 active+clean; 102 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 330 op/s 2023-09-30T14:41:15.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:15 smithi032 ceph-mon[98592]: pgmap v376: 97 pgs: 97 active+clean; 102 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 330 op/s 2023-09-30T14:41:15.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 251816 12.63 MB/sec warmup 109 sec latency 15.071 ms 2023-09-30T14:41:16.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 252494 12.53 MB/sec warmup 110 sec latency 146.402 ms 2023-09-30T14:41:17.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:17 smithi156 ceph-mon[103579]: pgmap v377: 97 pgs: 97 active+clean; 102 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 3.0 MiB/s wr, 290 op/s 2023-09-30T14:41:17.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:17 smithi148 ceph-mon[102355]: pgmap v377: 97 pgs: 97 active+clean; 102 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 3.0 MiB/s wr, 290 op/s 2023-09-30T14:41:17.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:17 smithi032 ceph-mon[98592]: pgmap v377: 97 pgs: 97 active+clean; 102 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 3.0 MiB/s wr, 290 op/s 2023-09-30T14:41:17.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 255405 12.58 MB/sec warmup 111 sec latency 53.006 ms 2023-09-30T14:41:18.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 256916 12.53 MB/sec warmup 112 sec latency 14.140 ms 2023-09-30T14:41:19.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:19 smithi156 ceph-mon[103579]: pgmap v378: 97 pgs: 97 active+clean; 105 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 323 op/s 2023-09-30T14:41:19.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:19 smithi148 ceph-mon[102355]: pgmap v378: 97 pgs: 97 active+clean; 105 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 323 op/s 2023-09-30T14:41:19.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:19 smithi032 ceph-mon[98592]: pgmap v378: 97 pgs: 97 active+clean; 105 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 323 op/s 2023-09-30T14:41:19.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 259253 12.54 MB/sec warmup 113 sec latency 51.211 ms 2023-09-30T14:41:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.609+0000 7fb875f60700 1 -- 172.21.15.32:0/3834045911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb870104270 msgr2=0x7fb870106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.609+0000 7fb875f60700 1 --2- 172.21.15.32:0/3834045911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb870104270 0x7fb870106660 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb858009990 tx=0x7fb858009ca0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.609+0000 7fb875f60700 1 -- 172.21.15.32:0/3834045911 shutdown_connections 2023-09-30T14:41:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.609+0000 7fb875f60700 1 --2- 172.21.15.32:0/3834045911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb870106ba0 0x7fb870108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.609+0000 7fb875f60700 1 --2- 172.21.15.32:0/3834045911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb870101940 0x7fb870103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.609+0000 7fb875f60700 1 --2- 172.21.15.32:0/3834045911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb870104270 0x7fb870106660 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.609+0000 7fb875f60700 1 -- 172.21.15.32:0/3834045911 >> 172.21.15.32:0/3834045911 conn(0x7fb8700fb2b0 msgr2=0x7fb8700fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.609+0000 7fb875f60700 1 -- 172.21.15.32:0/3834045911 shutdown_connections 2023-09-30T14:41:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.609+0000 7fb875f60700 1 -- 172.21.15.32:0/3834045911 wait complete. 2023-09-30T14:41:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.610+0000 7fb875f60700 1 Processor -- start 2023-09-30T14:41:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.610+0000 7fb875f60700 1 -- start start 2023-09-30T14:41:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.610+0000 7fb875f60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb870101940 0x7fb870198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.610+0000 7fb875f60700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb870104270 0x7fb870199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.610+0000 7fb875f60700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb870106ba0 0x7fb87019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.610+0000 7fb875f60700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87019a810 con 0x7fb870101940 2023-09-30T14:41:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.610+0000 7fb875f60700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87019a980 con 0x7fb870106ba0 2023-09-30T14:41:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.610+0000 7fb875f60700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87019ab40 con 0x7fb870104270 2023-09-30T14:41:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.610+0000 7fb86f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb870101940 0x7fb870198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.610+0000 7fb86f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb870101940 0x7fb870198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36362/0 (socket says 172.21.15.32:36362) 2023-09-30T14:41:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.610+0000 7fb86ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb870106ba0 0x7fb87019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.610+0000 7fb86f7fe700 1 -- 172.21.15.32:0/2412451049 learned_addr learned my addr 172.21.15.32:0/2412451049 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.610+0000 7fb86effd700 1 --2- 172.21.15.32:0/2412451049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb870104270 0x7fb870199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.611+0000 7fb86ffff700 1 -- 172.21.15.32:0/2412451049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb870104270 msgr2=0x7fb870199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.611+0000 7fb86ffff700 1 --2- 172.21.15.32:0/2412451049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb870104270 0x7fb870199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.611+0000 7fb86ffff700 1 -- 172.21.15.32:0/2412451049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb870101940 msgr2=0x7fb870198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.611+0000 7fb86ffff700 1 --2- 172.21.15.32:0/2412451049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb870101940 0x7fb870198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.611+0000 7fb86ffff700 1 -- 172.21.15.32:0/2412451049 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb860009520 con 0x7fb870106ba0 2023-09-30T14:41:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.611+0000 7fb86f7fe700 1 --2- 172.21.15.32:0/2412451049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb870101940 0x7fb870198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:41:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.611+0000 7fb86ffff700 1 --2- 172.21.15.32:0/2412451049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb870106ba0 0x7fb87019a100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb86000e900 tx=0x7fb86000ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.611+0000 7fb86cff9700 1 -- 172.21.15.32:0/2412451049 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860004220 con 0x7fb870106ba0 2023-09-30T14:41:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.611+0000 7fb86cff9700 1 -- 172.21.15.32:0/2412451049 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb860004380 con 0x7fb870106ba0 2023-09-30T14:41:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.611+0000 7fb875f60700 1 -- 172.21.15.32:0/2412451049 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8580095f0 con 0x7fb870106ba0 2023-09-30T14:41:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.611+0000 7fb86cff9700 1 -- 172.21.15.32:0/2412451049 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860003d20 con 0x7fb870106ba0 2023-09-30T14:41:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.611+0000 7fb875f60700 1 -- 172.21.15.32:0/2412451049 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87019af70 con 0x7fb870106ba0 2023-09-30T14:41:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.613+0000 7fb86cff9700 1 -- 172.21.15.32:0/2412451049 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fb86000c780 con 0x7fb870106ba0 2023-09-30T14:41:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.614+0000 7fb86cff9700 1 --2- 172.21.15.32:0/2412451049 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb85c0a0fa0 0x7fb85c0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.614+0000 7fb86f7fe700 1 --2- 172.21.15.32:0/2412451049 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb85c0a0fa0 0x7fb85c0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.614+0000 7fb86cff9700 1 -- 172.21.15.32:0/2412451049 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb860014070 con 0x7fb870106ba0 2023-09-30T14:41:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.614+0000 7fb86f7fe700 1 --2- 172.21.15.32:0/2412451049 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb85c0a0fa0 0x7fb85c0a3460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb868009c10 tx=0x7fb86800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb882148b80 1 Processor -- start 2023-09-30T14:41:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb882148b80 1 -- start start 2023-09-30T14:41:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb882148b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b472d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb882148b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472e310 0x5586b472e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb882148b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472ecd0 0x5586b4735560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb882148b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b4695400 con 0x5586b472d200 2023-09-30T14:41:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb882148b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b4686bc0 con 0x5586b472e310 2023-09-30T14:41:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb882148b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b46844e0 con 0x5586b472ecd0 2023-09-30T14:41:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb86f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b472d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb86f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b472d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36370/0 (socket says 172.21.15.32:36370) 2023-09-30T14:41:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb86f7fe700 1 -- 172.21.15.32:0/2264473849 learned_addr learned my addr 172.21.15.32:0/2264473849 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb86effd700 1 --2- 172.21.15.32:0/2264473849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472e310 0x5586b472e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb86ffff700 1 --2- 172.21.15.32:0/2264473849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472ecd0 0x5586b4735560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb86f7fe700 1 -- 172.21.15.32:0/2264473849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472ecd0 msgr2=0x5586b4735560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb86f7fe700 1 --2- 172.21.15.32:0/2264473849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472ecd0 0x5586b4735560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb86f7fe700 1 -- 172.21.15.32:0/2264473849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472e310 msgr2=0x5586b472e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb86f7fe700 1 --2- 172.21.15.32:0/2264473849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472e310 0x5586b472e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb86f7fe700 1 -- 172.21.15.32:0/2264473849 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586b46e2870 con 0x5586b472d200 2023-09-30T14:41:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.615+0000 7fb86f7fe700 1 --2- 172.21.15.32:0/2264473849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b472d620 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb86800e950 tx=0x7fb868014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906a59cf2ab54283 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.616+0000 7fb865ffb700 1 -- 172.21.15.32:0/2264473849 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868015960 con 0x5586b472d200 2023-09-30T14:41:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.616+0000 7fb865ffb700 1 -- 172.21.15.32:0/2264473849 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb86800f040 con 0x5586b472d200 2023-09-30T14:41:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.616+0000 7fb865ffb700 1 -- 172.21.15.32:0/2264473849 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868017e00 con 0x5586b472d200 2023-09-30T14:41:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.616+0000 7fb882148b80 1 -- 172.21.15.32:0/2264473849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 msgr2=0x5586b472d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.616+0000 7fb882148b80 1 --2- 172.21.15.32:0/2264473849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b472d620 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb86800e950 tx=0x7fb868014000 comp rx=0 tx=0).stop 2023-09-30T14:41:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.616+0000 7fb882148b80 1 -- 172.21.15.32:0/2264473849 shutdown_connections 2023-09-30T14:41:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.616+0000 7fb882148b80 1 --2- 172.21.15.32:0/2264473849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472ecd0 0x5586b4735560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.616+0000 7fb882148b80 1 --2- 172.21.15.32:0/2264473849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b472d620 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.616+0000 7fb882148b80 1 --2- 172.21.15.32:0/2264473849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472e310 0x5586b472e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.616+0000 7fb882148b80 1 -- 172.21.15.32:0/2264473849 >> 172.21.15.32:0/2264473849 conn(0x5586b46a1200 msgr2=0x5586b4727ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.616+0000 7fb882148b80 1 -- 172.21.15.32:0/2264473849 shutdown_connections 2023-09-30T14:41:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.616+0000 7fb882148b80 1 -- 172.21.15.32:0/2264473849 wait complete. 2023-09-30T14:41:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.617+0000 7fb882148b80 1 -- start start 2023-09-30T14:41:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.617+0000 7fb882148b80 1 Processor -- start 2023-09-30T14:41:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.617+0000 7fb882148b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b4736f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.617+0000 7fb882148b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472e310 0x5586b4737440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.617+0000 7fb882148b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472ecd0 0x5586b473e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.617+0000 7fb882148b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b467fdd0 con 0x5586b472d200 2023-09-30T14:41:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.617+0000 7fb882148b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b461c6b0 con 0x5586b472e310 2023-09-30T14:41:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.617+0000 7fb882148b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b461c350 con 0x5586b472ecd0 2023-09-30T14:41:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb882148b80 10 client.0 _ll_register_callbacks cb 0x5586b44797a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:41:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb86f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b4736f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb86f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b4736f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36376/0 (socket says 172.21.15.32:36376) 2023-09-30T14:41:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb86f7fe700 1 -- 172.21.15.32:0/3789581456 learned_addr learned my addr 172.21.15.32:0/3789581456 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb86effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472e310 0x5586b4737440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb86ffff700 1 --2- 172.21.15.32:0/3789581456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472ecd0 0x5586b473e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb86f7fe700 1 -- 172.21.15.32:0/3789581456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472ecd0 msgr2=0x5586b473e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb86f7fe700 1 --2- 172.21.15.32:0/3789581456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472ecd0 0x5586b473e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb86f7fe700 1 -- 172.21.15.32:0/3789581456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472e310 msgr2=0x5586b4737440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb86f7fe700 1 --2- 172.21.15.32:0/3789581456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472e310 0x5586b4737440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb86f7fe700 1 -- 172.21.15.32:0/3789581456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86800e600 con 0x5586b472d200 2023-09-30T14:41:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb86ffff700 1 --2- 172.21.15.32:0/3789581456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472ecd0 0x5586b473e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:41:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb86effd700 1 --2- 172.21.15.32:0/3789581456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472e310 0x5586b4737440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:41:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb86f7fe700 1 --2- 172.21.15.32:0/3789581456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b4736f00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb86800ef40 tx=0x7fb868017d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24216cec0c66e894 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb84effd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:41:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb84effd700 1 -- 172.21.15.32:0/3789581456 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868004ca0 con 0x5586b472d200 2023-09-30T14:41:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb84effd700 1 -- 172.21.15.32:0/3789581456 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb868004e00 con 0x5586b472d200 2023-09-30T14:41:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb882148b80 1 -- 172.21.15.32:0/3789581456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586b4720090 con 0x5586b472d200 2023-09-30T14:41:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.618+0000 7fb84effd700 1 -- 172.21.15.32:0/3789581456 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb868015760 con 0x5586b472d200 2023-09-30T14:41:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.619+0000 7fb882148b80 10 client.15786 fetch_fsmap learned FSMap version 26 2023-09-30T14:41:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.619+0000 7fb882148b80 1 -- 172.21.15.32:0/3789581456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5586b44aad80 con 0x5586b472d200 2023-09-30T14:41:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.619+0000 7fb84effd700 1 -- 172.21.15.32:0/3789581456 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5586b4720090 con 0x5586b472d200 2023-09-30T14:41:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.619+0000 7fb84effd700 1 -- 172.21.15.32:0/3789581456 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb86803e070 con 0x5586b472d200 2023-09-30T14:41:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.619+0000 7fb882148b80 10 client.15786 fetch_fsmap finished waiting for FSMap version 26 2023-09-30T14:41:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.619+0000 7fb882148b80 10 client.15786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:41:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.619+0000 7fb882148b80 20 client.15786 populate_metadata read hostname 'smithi032' 2023-09-30T14:41:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.619+0000 7fb882148b80 1 --2- 172.21.15.32:0/3789581456 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5586b4742ca0 0x5586b47450c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.619+0000 7fb882148b80 4 client.15786 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:41:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.619+0000 7fb882148b80 1 -- 172.21.15.32:0/3789581456 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586b4678db0 con 0x5586b4742ca0 2023-09-30T14:41:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.619+0000 7fb86effd700 1 --2- 172.21.15.32:0/3789581456 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5586b4742ca0 0x5586b47450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.620+0000 7fb86effd700 1 --2- 172.21.15.32:0/3789581456 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5586b4742ca0 0x5586b47450c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0a5031d004d7fc4 server_cookie=4e56e02219cf86c8 in_seq=0 out_seq=0 2023-09-30T14:41:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.620+0000 7fb84effd700 10 client.15786 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:41:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.620+0000 7fb84effd700 1 -- 172.21.15.32:0/3789581456 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5586b4678db0 con 0x5586b4742ca0 2023-09-30T14:41:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.620+0000 7fb84effd700 10 client.15786 handle_command_reply: tid=0 2023-09-30T14:41:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.620+0000 7fb882148b80 1 client.15786 shutdown 2023-09-30T14:41:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.620+0000 7fb882148b80 1 -- 172.21.15.32:0/3789581456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 msgr2=0x5586b4736f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.620+0000 7fb882148b80 1 --2- 172.21.15.32:0/3789581456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b4736f00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb86800ef40 tx=0x7fb868017d70 comp rx=0 tx=0).stop 2023-09-30T14:41:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.620+0000 7fb882148b80 1 -- 172.21.15.32:0/3789581456 shutdown_connections 2023-09-30T14:41:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.620+0000 7fb882148b80 1 --2- 172.21.15.32:0/3789581456 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5586b4742ca0 0x5586b47450c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.620+0000 7fb882148b80 1 --2- 172.21.15.32:0/3789581456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472ecd0 0x5586b473e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.620+0000 7fb882148b80 1 --2- 172.21.15.32:0/3789581456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b4736f00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.620+0000 7fb882148b80 1 --2- 172.21.15.32:0/3789581456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472e310 0x5586b4737440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.620+0000 7fb84effd700 0 client.15786 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:41:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.620+0000 7fb882148b80 1 -- 172.21.15.32:0/3789581456 >> 172.21.15.32:0/3789581456 conn(0x5586b46a1200 msgr2=0x5586b4727590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.620+0000 7fb882148b80 1 -- 172.21.15.32:0/3789581456 shutdown_connections 2023-09-30T14:41:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.620+0000 7fb882148b80 1 -- 172.21.15.32:0/3789581456 wait complete. 2023-09-30T14:41:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.621+0000 7fb882148b80 20 client.15786 trim_cache size 0 max 16384 2023-09-30T14:41:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb882148b80 1 Processor -- start 2023-09-30T14:41:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb882148b80 1 -- start start 2023-09-30T14:41:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb882148b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472d200 0x5586b4735c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb882148b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472e310 0x5586b473e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb882148b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472ecd0 0x5586b4735110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb882148b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b45008c0 con 0x5586b472ecd0 2023-09-30T14:41:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb882148b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b47299c0 con 0x5586b472d200 2023-09-30T14:41:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb882148b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b47327b0 con 0x5586b472e310 2023-09-30T14:41:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb86effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472e310 0x5586b473e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb86ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472ecd0 0x5586b4735110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb86f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472d200 0x5586b4735c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb86ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472ecd0 0x5586b4735110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36388/0 (socket says 172.21.15.32:36388) 2023-09-30T14:41:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb86effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472e310 0x5586b473e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:50768/0 (socket says 172.21.15.32:50768) 2023-09-30T14:41:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb86ffff700 1 -- 172.21.15.32:0/774300 learned_addr learned my addr 172.21.15.32:0/774300 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb86ffff700 1 -- 172.21.15.32:0/774300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472e310 msgr2=0x5586b473e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb86ffff700 1 --2- 172.21.15.32:0/774300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472e310 0x5586b473e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb86ffff700 1 -- 172.21.15.32:0/774300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472d200 msgr2=0x5586b4735c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb86ffff700 1 --2- 172.21.15.32:0/774300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472d200 0x5586b4735c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.637+0000 7fb86ffff700 1 -- 172.21.15.32:0/774300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586b4493c00 con 0x5586b472ecd0 2023-09-30T14:41:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.638+0000 7fb86ffff700 1 --2- 172.21.15.32:0/774300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472ecd0 0x5586b4735110 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb860099c20 tx=0x7fb86009a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806fa054bf51e536 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.638+0000 7fb8667fc700 1 -- 172.21.15.32:0/774300 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860099c90 con 0x5586b472ecd0 2023-09-30T14:41:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.638+0000 7fb8667fc700 1 -- 172.21.15.32:0/774300 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb86009dde0 con 0x5586b472ecd0 2023-09-30T14:41:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.638+0000 7fb8667fc700 1 -- 172.21.15.32:0/774300 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86000ccd0 con 0x5586b472ecd0 2023-09-30T14:41:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.639+0000 7fb882148b80 1 -- 172.21.15.32:0/774300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472ecd0 msgr2=0x5586b4735110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.639+0000 7fb882148b80 1 --2- 172.21.15.32:0/774300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472ecd0 0x5586b4735110 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb860099c20 tx=0x7fb86009a410 comp rx=0 tx=0).stop 2023-09-30T14:41:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.639+0000 7fb882148b80 1 -- 172.21.15.32:0/774300 shutdown_connections 2023-09-30T14:41:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.639+0000 7fb882148b80 1 --2- 172.21.15.32:0/774300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472ecd0 0x5586b4735110 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.639+0000 7fb882148b80 1 --2- 172.21.15.32:0/774300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472d200 0x5586b4735c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.639+0000 7fb882148b80 1 --2- 172.21.15.32:0/774300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472e310 0x5586b473e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.639+0000 7fb882148b80 1 -- 172.21.15.32:0/774300 >> 172.21.15.32:0/774300 conn(0x5586b46a1200 msgr2=0x5586b4749a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.639+0000 7fb882148b80 1 -- 172.21.15.32:0/774300 shutdown_connections 2023-09-30T14:41:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.639+0000 7fb882148b80 1 -- 172.21.15.32:0/774300 wait complete. 2023-09-30T14:41:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.639+0000 7fb882148b80 1 -- start start 2023-09-30T14:41:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.639+0000 7fb882148b80 1 Processor -- start 2023-09-30T14:41:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb882148b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b473c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb882148b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472e310 0x5586b473c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb882148b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472ecd0 0x5586b473d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb882148b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b473de90 con 0x5586b472d200 2023-09-30T14:41:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb882148b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b473e000 con 0x5586b472ecd0 2023-09-30T14:41:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb882148b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586b473e1c0 con 0x5586b472e310 2023-09-30T14:41:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb882148b80 10 client.0 _ll_register_callbacks cb 0x7fb860005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:41:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb86f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b473c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb86f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b473c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36390/0 (socket says 172.21.15.32:36390) 2023-09-30T14:41:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb86effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472e310 0x5586b473c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb86f7fe700 1 -- 172.21.15.32:0/2959461023 learned_addr learned my addr 172.21.15.32:0/2959461023 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb86ffff700 1 --2- 172.21.15.32:0/2959461023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472ecd0 0x5586b473d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb86f7fe700 1 -- 172.21.15.32:0/2959461023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472e310 msgr2=0x5586b473c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb86f7fe700 1 --2- 172.21.15.32:0/2959461023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472e310 0x5586b473c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb86f7fe700 1 -- 172.21.15.32:0/2959461023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472ecd0 msgr2=0x5586b473d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb86f7fe700 1 --2- 172.21.15.32:0/2959461023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472ecd0 0x5586b473d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb86f7fe700 1 -- 172.21.15.32:0/2959461023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8580095f0 con 0x5586b472d200 2023-09-30T14:41:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb86effd700 1 --2- 172.21.15.32:0/2959461023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472e310 0x5586b473c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:41:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb86ffff700 1 --2- 172.21.15.32:0/2959461023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472ecd0 0x5586b473d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:41:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb86f7fe700 1 --2- 172.21.15.32:0/2959461023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b473c400 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb868005610 tx=0x7fb868009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8ceaf91cfd6dd0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb84effd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:41:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb84effd700 1 -- 172.21.15.32:0/2959461023 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868005980 con 0x5586b472d200 2023-09-30T14:41:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb84effd700 1 -- 172.21.15.32:0/2959461023 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb868015760 con 0x5586b472d200 2023-09-30T14:41:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.640+0000 7fb882148b80 1 -- 172.21.15.32:0/2959461023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb86800f040 con 0x5586b472d200 2023-09-30T14:41:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.641+0000 7fb84effd700 1 -- 172.21.15.32:0/2959461023 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb868015920 con 0x5586b472d200 2023-09-30T14:41:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.641+0000 7fb882148b80 10 client.15792 fetch_fsmap learned FSMap version 26 2023-09-30T14:41:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.641+0000 7fb882148b80 1 -- 172.21.15.32:0/2959461023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5586b44d11f0 con 0x5586b472d200 2023-09-30T14:41:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.641+0000 7fb84effd700 1 -- 172.21.15.32:0/2959461023 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb86800f040 con 0x5586b472d200 2023-09-30T14:41:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.641+0000 7fb84effd700 1 -- 172.21.15.32:0/2959461023 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb86800ec60 con 0x5586b472d200 2023-09-30T14:41:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.641+0000 7fb882148b80 10 client.15792 fetch_fsmap finished waiting for FSMap version 26 2023-09-30T14:41:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.641+0000 7fb882148b80 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:41:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.641+0000 7fb882148b80 20 client.15792 populate_metadata read hostname 'smithi032' 2023-09-30T14:41:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.641+0000 7fb882148b80 1 --2- 172.21.15.32:0/2959461023 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5586b47977f0 0x5586b4799c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.641+0000 7fb882148b80 4 client.15792 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:41:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.641+0000 7fb882148b80 1 -- 172.21.15.32:0/2959461023 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5586b4678ad0 con 0x5586b47977f0 2023-09-30T14:41:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.641+0000 7fb86effd700 1 --2- 172.21.15.32:0/2959461023 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5586b47977f0 0x5586b4799c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.642+0000 7fb84effd700 10 client.15792 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:41:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.642+0000 7fb86effd700 1 --2- 172.21.15.32:0/2959461023 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5586b47977f0 0x5586b4799c10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd145934b18b0986 server_cookie=7b29c099f7a24118 in_seq=0 out_seq=0 2023-09-30T14:41:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.685+0000 7fb84effd700 1 -- 172.21.15.32:0/2959461023 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5586b4678ad0 con 0x5586b47977f0 2023-09-30T14:41:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.685+0000 7fb84effd700 10 client.15792 handle_command_reply: tid=0 2023-09-30T14:41:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.685+0000 7fb882148b80 1 client.15792 shutdown 2023-09-30T14:41:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.686+0000 7fb882148b80 1 -- 172.21.15.32:0/2959461023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 msgr2=0x5586b473c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.686+0000 7fb882148b80 1 --2- 172.21.15.32:0/2959461023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b473c400 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb868005610 tx=0x7fb868009a10 comp rx=0 tx=0).stop 2023-09-30T14:41:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.686+0000 7fb882148b80 1 -- 172.21.15.32:0/2959461023 shutdown_connections 2023-09-30T14:41:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.686+0000 7fb882148b80 1 --2- 172.21.15.32:0/2959461023 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5586b47977f0 0x5586b4799c10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.686+0000 7fb882148b80 1 --2- 172.21.15.32:0/2959461023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586b472ecd0 0x5586b473d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.686+0000 7fb882148b80 1 --2- 172.21.15.32:0/2959461023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5586b472d200 0x5586b473c400 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.686+0000 7fb84effd700 0 client.15792 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:41:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.686+0000 7fb882148b80 1 --2- 172.21.15.32:0/2959461023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5586b472e310 0x5586b473c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.686+0000 7fb882148b80 1 -- 172.21.15.32:0/2959461023 >> 172.21.15.32:0/2959461023 conn(0x5586b46a1200 msgr2=0x5586b4735c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.686+0000 7fb882148b80 1 -- 172.21.15.32:0/2959461023 shutdown_connections 2023-09-30T14:41:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.686+0000 7fb882148b80 1 -- 172.21.15.32:0/2959461023 wait complete. 2023-09-30T14:41:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.686+0000 7fb882148b80 20 client.15792 trim_cache size 0 max 16384 2023-09-30T14:41:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:41:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:41:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:41:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:41:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.687+0000 7fb875f60700 1 -- 172.21.15.32:0/2412451049 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb85c0a0fa0 msgr2=0x7fb85c0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.687+0000 7fb875f60700 1 --2- 172.21.15.32:0/2412451049 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb85c0a0fa0 0x7fb85c0a3460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb868009c10 tx=0x7fb86800b040 comp rx=0 tx=0).stop 2023-09-30T14:41:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.687+0000 7fb875f60700 1 -- 172.21.15.32:0/2412451049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb870106ba0 msgr2=0x7fb87019a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.687+0000 7fb875f60700 1 --2- 172.21.15.32:0/2412451049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb870106ba0 0x7fb87019a100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb86000e900 tx=0x7fb86000ec10 comp rx=0 tx=0).stop 2023-09-30T14:41:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.687+0000 7fb875f60700 1 -- 172.21.15.32:0/2412451049 shutdown_connections 2023-09-30T14:41:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.687+0000 7fb875f60700 1 --2- 172.21.15.32:0/2412451049 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb85c0a0fa0 0x7fb85c0a3460 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.687+0000 7fb875f60700 1 --2- 172.21.15.32:0/2412451049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb870106ba0 0x7fb87019a100 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.687+0000 7fb875f60700 1 --2- 172.21.15.32:0/2412451049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb870101940 0x7fb870198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.687+0000 7fb875f60700 1 --2- 172.21.15.32:0/2412451049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb870104270 0x7fb870199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.687+0000 7fb875f60700 1 -- 172.21.15.32:0/2412451049 >> 172.21.15.32:0/2412451049 conn(0x7fb8700fb2b0 msgr2=0x7fb870100040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.687+0000 7fb875f60700 1 -- 172.21.15.32:0/2412451049 shutdown_connections 2023-09-30T14:41:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:20.687+0000 7fb875f60700 1 -- 172.21.15.32:0/2412451049 wait complete. 2023-09-30T14:41:20.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 261032 12.48 MB/sec warmup 114 sec latency 24.042 ms 2023-09-30T14:41:21.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:21 smithi156 ceph-mon[103579]: pgmap v379: 97 pgs: 97 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.9 MiB/s wr, 295 op/s 2023-09-30T14:41:21.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:21 smithi148 ceph-mon[102355]: pgmap v379: 97 pgs: 97 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.9 MiB/s wr, 295 op/s 2023-09-30T14:41:21.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:21 smithi032 ceph-mon[98592]: pgmap v379: 97 pgs: 97 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.9 MiB/s wr, 295 op/s 2023-09-30T14:41:21.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 262899 12.49 MB/sec warmup 115 sec latency 135.153 ms 2023-09-30T14:41:22.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 264967 12.44 MB/sec warmup 116 sec latency 34.286 ms 2023-09-30T14:41:23.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:23 smithi156 ceph-mon[103579]: pgmap v380: 97 pgs: 97 active+clean; 107 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 303 op/s 2023-09-30T14:41:23.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:41:23.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:41:23.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:23 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:41:23.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:23 smithi148 ceph-mon[102355]: pgmap v380: 97 pgs: 97 active+clean; 107 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 303 op/s 2023-09-30T14:41:23.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:41:23.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:41:23.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:23 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:41:23.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:23 smithi032 ceph-mon[98592]: pgmap v380: 97 pgs: 97 active+clean; 107 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 303 op/s 2023-09-30T14:41:23.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:41:23.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:41:23.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:23 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:41:23.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 266736 12.44 MB/sec warmup 117 sec latency 16.682 ms 2023-09-30T14:41:24.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:24 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:41:24.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:24 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:41:24.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:24 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:41:24.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:24 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:41:24.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:24 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:41:24.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:24 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:41:24.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:41:24.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:41:24.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:41:24.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 269416 12.45 MB/sec warmup 118 sec latency 13.584 ms 2023-09-30T14:41:25.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:25 smithi156 ceph-mon[103579]: pgmap v381: 97 pgs: 97 active+clean; 108 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 2.9 MiB/s wr, 295 op/s 2023-09-30T14:41:25.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:25 smithi148 ceph-mon[102355]: pgmap v381: 97 pgs: 97 active+clean; 108 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 2.9 MiB/s wr, 295 op/s 2023-09-30T14:41:25.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:25 smithi032 ceph-mon[98592]: pgmap v381: 97 pgs: 97 active+clean; 108 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 2.9 MiB/s wr, 295 op/s 2023-09-30T14:41:25.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 270671 12.41 MB/sec warmup 119 sec latency 20.915 ms 2023-09-30T14:41:27.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:27 smithi156 ceph-mon[103579]: pgmap v382: 97 pgs: 97 active+clean; 110 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 297 op/s 2023-09-30T14:41:27.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:27 smithi148 ceph-mon[102355]: pgmap v382: 97 pgs: 97 active+clean; 110 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 297 op/s 2023-09-30T14:41:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:27 smithi032 ceph-mon[98592]: pgmap v382: 97 pgs: 97 active+clean; 110 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 297 op/s 2023-09-30T14:41:27.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 275606 7.47 MB/sec execute 1 sec latency 46.577 ms 2023-09-30T14:41:28.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 277559 10.03 MB/sec execute 2 sec latency 14.332 ms 2023-09-30T14:41:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:29 smithi156 ceph-mon[103579]: pgmap v383: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 317 op/s 2023-09-30T14:41:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:29 smithi148 ceph-mon[102355]: pgmap v383: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 317 op/s 2023-09-30T14:41:29.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:29 smithi032 ceph-mon[98592]: pgmap v383: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 317 op/s 2023-09-30T14:41:29.694 INFO:tasks.workunit.client.0.smithi032.stdout: 1 280435 12.70 MB/sec execute 3 sec latency 11.286 ms 2023-09-30T14:41:30.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 281864 10.94 MB/sec execute 4 sec latency 29.286 ms 2023-09-30T14:41:31.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:31 smithi156 ceph-mon[103579]: pgmap v384: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2023-09-30T14:41:31.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:31 smithi148 ceph-mon[102355]: pgmap v384: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2023-09-30T14:41:31.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:31 smithi032 ceph-mon[98592]: pgmap v384: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2023-09-30T14:41:31.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 284232 11.53 MB/sec execute 5 sec latency 10.879 ms 2023-09-30T14:41:32.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:32 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:41:32.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:32 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:41:32.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:32 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:41:32.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 286394 10.95 MB/sec execute 6 sec latency 56.938 ms 2023-09-30T14:41:33.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:33 smithi156 ceph-mon[103579]: pgmap v385: 97 pgs: 97 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 3.1 MiB/s wr, 305 op/s 2023-09-30T14:41:33.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:33 smithi148 ceph-mon[102355]: pgmap v385: 97 pgs: 97 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 3.1 MiB/s wr, 305 op/s 2023-09-30T14:41:33.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:33 smithi032 ceph-mon[98592]: pgmap v385: 97 pgs: 97 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 3.1 MiB/s wr, 305 op/s 2023-09-30T14:41:33.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 288216 11.07 MB/sec execute 7 sec latency 12.798 ms 2023-09-30T14:41:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.801+0000 7f325cf1d700 1 -- 172.21.15.32:0/3471150765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32580ff520 msgr2=0x7f32580ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.801+0000 7f325cf1d700 1 --2- 172.21.15.32:0/3471150765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32580ff520 0x7f32580ff940 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f324800aad0 tx=0x7f324800ade0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.801+0000 7f325cf1d700 1 -- 172.21.15.32:0/3471150765 shutdown_connections 2023-09-30T14:41:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.801+0000 7f325cf1d700 1 --2- 172.21.15.32:0/3471150765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32581010b0 0x7f3258107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.801+0000 7f325cf1d700 1 --2- 172.21.15.32:0/3471150765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32580ff520 0x7f32580ff940 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.801+0000 7f325cf1d700 1 --2- 172.21.15.32:0/3471150765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3258100710 0x7f3258100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.801+0000 7f325cf1d700 1 -- 172.21.15.32:0/3471150765 >> 172.21.15.32:0/3471150765 conn(0x7f32580faa90 msgr2=0x7f32580fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.801+0000 7f325cf1d700 1 -- 172.21.15.32:0/3471150765 shutdown_connections 2023-09-30T14:41:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.801+0000 7f325cf1d700 1 -- 172.21.15.32:0/3471150765 wait complete. 2023-09-30T14:41:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.802+0000 7f325cf1d700 1 Processor -- start 2023-09-30T14:41:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.802+0000 7f325cf1d700 1 -- start start 2023-09-30T14:41:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.802+0000 7f325cf1d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32580ff520 0x7f3258196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.802+0000 7f325cf1d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3258100710 0x7f3258197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.802+0000 7f325cf1d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32581010b0 0x7f3258197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.802+0000 7f325cf1d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32581986b0 con 0x7f32580ff520 2023-09-30T14:41:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.802+0000 7f325cf1d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3258198820 con 0x7f32581010b0 2023-09-30T14:41:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.802+0000 7f325cf1d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32581989e0 con 0x7f3258100710 2023-09-30T14:41:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.802+0000 7f325659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32580ff520 0x7f3258196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.802+0000 7f325659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32580ff520 0x7f3258196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36404/0 (socket says 172.21.15.32:36404) 2023-09-30T14:41:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.802+0000 7f325659c700 1 -- 172.21.15.32:0/1902936403 learned_addr learned my addr 172.21.15.32:0/1902936403 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.802+0000 7f3256d9d700 1 --2- 172.21.15.32:0/1902936403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32581010b0 0x7f3258197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.802+0000 7f3255d9b700 1 --2- 172.21.15.32:0/1902936403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3258100710 0x7f3258197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.803+0000 7f325659c700 1 -- 172.21.15.32:0/1902936403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3258100710 msgr2=0x7f3258197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.803+0000 7f325659c700 1 --2- 172.21.15.32:0/1902936403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3258100710 0x7f3258197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.803+0000 7f325659c700 1 -- 172.21.15.32:0/1902936403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32581010b0 msgr2=0x7f3258197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.803+0000 7f325659c700 1 --2- 172.21.15.32:0/1902936403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32581010b0 0x7f3258197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.803+0000 7f325659c700 1 -- 172.21.15.32:0/1902936403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324800a700 con 0x7f32580ff520 2023-09-30T14:41:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.803+0000 7f325659c700 1 --2- 172.21.15.32:0/1902936403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32580ff520 0x7f3258196d30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3248003ab0 tx=0x7f324800e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.803+0000 7f32477fe700 1 -- 172.21.15.32:0/1902936403 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324800e8d0 con 0x7f32580ff520 2023-09-30T14:41:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.803+0000 7f32477fe700 1 -- 172.21.15.32:0/1902936403 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32480050d0 con 0x7f32580ff520 2023-09-30T14:41:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.803+0000 7f325cf1d700 1 -- 172.21.15.32:0/1902936403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3258198c60 con 0x7f32580ff520 2023-09-30T14:41:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.803+0000 7f32477fe700 1 -- 172.21.15.32:0/1902936403 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248016840 con 0x7f32580ff520 2023-09-30T14:41:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.803+0000 7f325cf1d700 1 -- 172.21.15.32:0/1902936403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325819f3e0 con 0x7f32580ff520 2023-09-30T14:41:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.805+0000 7f32477fe700 1 -- 172.21.15.32:0/1902936403 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f3248005240 con 0x7f32580ff520 2023-09-30T14:41:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.806+0000 7f32477fe700 1 --2- 172.21.15.32:0/1902936403 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f32400a0fa0 0x7f32400a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.806+0000 7f32477fe700 1 -- 172.21.15.32:0/1902936403 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3248012070 con 0x7f32580ff520 2023-09-30T14:41:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.806+0000 7f3255d9b700 1 --2- 172.21.15.32:0/1902936403 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f32400a0fa0 0x7f32400a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3269105b80 1 Processor -- start 2023-09-30T14:41:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3255d9b700 1 --2- 172.21.15.32:0/1902936403 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f32400a0fa0 0x7f32400a3460 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f324c009c60 tx=0x7f324c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3269105b80 1 -- start start 2023-09-30T14:41:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3269105b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c08f200 0x55d11c08f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3269105b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c090790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3269105b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c090cd0 0x55d11c097560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3269105b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11bff7400 con 0x55d11c090310 2023-09-30T14:41:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3269105b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11bfe8bc0 con 0x55d11c08f200 2023-09-30T14:41:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3269105b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11bfe64e0 con 0x55d11c090cd0 2023-09-30T14:41:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3255d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c090790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3256d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c090cd0 0x55d11c097560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3255d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c090790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36414/0 (socket says 172.21.15.32:36414) 2023-09-30T14:41:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3255d9b700 1 -- 172.21.15.32:0/513364792 learned_addr learned my addr 172.21.15.32:0/513364792 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f325659c700 1 --2- 172.21.15.32:0/513364792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c08f200 0x55d11c08f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3256d9d700 1 -- 172.21.15.32:0/513364792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c08f200 msgr2=0x55d11c08f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3256d9d700 1 --2- 172.21.15.32:0/513364792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c08f200 0x55d11c08f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3256d9d700 1 -- 172.21.15.32:0/513364792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 msgr2=0x55d11c090790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3256d9d700 1 --2- 172.21.15.32:0/513364792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c090790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.807+0000 7f3256d9d700 1 -- 172.21.15.32:0/513364792 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d11c044870 con 0x55d11c090cd0 2023-09-30T14:41:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.808+0000 7f3256d9d700 1 --2- 172.21.15.32:0/513364792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c090cd0 0x55d11c097560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3250011580 tx=0x7f3250009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72fe2f0e4f8fc3e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.808+0000 7f3244ff9700 1 -- 172.21.15.32:0/513364792 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325000e040 con 0x55d11c090cd0 2023-09-30T14:41:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.808+0000 7f3244ff9700 1 -- 172.21.15.32:0/513364792 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f325000f040 con 0x55d11c090cd0 2023-09-30T14:41:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.808+0000 7f3244ff9700 1 -- 172.21.15.32:0/513364792 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3250004290 con 0x55d11c090cd0 2023-09-30T14:41:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.809+0000 7f3269105b80 1 -- 172.21.15.32:0/513364792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c090cd0 msgr2=0x55d11c097560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.809+0000 7f3269105b80 1 --2- 172.21.15.32:0/513364792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c090cd0 0x55d11c097560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3250011580 tx=0x7f3250009920 comp rx=0 tx=0).stop 2023-09-30T14:41:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.809+0000 7f3269105b80 1 -- 172.21.15.32:0/513364792 shutdown_connections 2023-09-30T14:41:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.809+0000 7f3269105b80 1 --2- 172.21.15.32:0/513364792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c090cd0 0x55d11c097560 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.809+0000 7f3269105b80 1 --2- 172.21.15.32:0/513364792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c08f200 0x55d11c08f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.809+0000 7f3269105b80 1 --2- 172.21.15.32:0/513364792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c090790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.809+0000 7f3269105b80 1 -- 172.21.15.32:0/513364792 >> 172.21.15.32:0/513364792 conn(0x55d11c003200 msgr2=0x55d11c089ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.809+0000 7f3269105b80 1 -- 172.21.15.32:0/513364792 shutdown_connections 2023-09-30T14:41:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.809+0000 7f3269105b80 1 -- 172.21.15.32:0/513364792 wait complete. 2023-09-30T14:41:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.809+0000 7f3269105b80 1 -- start start 2023-09-30T14:41:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.809+0000 7f3269105b80 1 Processor -- start 2023-09-30T14:41:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.809+0000 7f3269105b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c08f200 0x55d11c098e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.809+0000 7f3269105b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c0993a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.809+0000 7f3269105b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c090cd0 0x55d11c0a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.809+0000 7f3269105b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11bfe1dd0 con 0x55d11c090310 2023-09-30T14:41:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.809+0000 7f3269105b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11bf7e6b0 con 0x55d11c090cd0 2023-09-30T14:41:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f3269105b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11bf7e350 con 0x55d11c08f200 2023-09-30T14:41:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f3269105b80 10 client.0 _ll_register_callbacks cb 0x55d11bddb7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:41:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f3255d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c0993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f3255d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c0993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36420/0 (socket says 172.21.15.32:36420) 2023-09-30T14:41:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f325659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c08f200 0x55d11c098e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f3255d9b700 1 -- 172.21.15.32:0/515131999 learned_addr learned my addr 172.21.15.32:0/515131999 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f3256d9d700 1 --2- 172.21.15.32:0/515131999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c090cd0 0x55d11c0a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f3255d9b700 1 -- 172.21.15.32:0/515131999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c08f200 msgr2=0x55d11c098e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f3255d9b700 1 --2- 172.21.15.32:0/515131999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c08f200 0x55d11c098e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f3255d9b700 1 -- 172.21.15.32:0/515131999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c090cd0 msgr2=0x55d11c0a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f3255d9b700 1 --2- 172.21.15.32:0/515131999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c090cd0 0x55d11c0a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f3255d9b700 1 -- 172.21.15.32:0/515131999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3250011230 con 0x55d11c090310 2023-09-30T14:41:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f325659c700 1 --2- 172.21.15.32:0/515131999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c08f200 0x55d11c098e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:41:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f3255d9b700 1 --2- 172.21.15.32:0/515131999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c0993a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f324c003900 tx=0x7f324c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf2a24a9133214e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f323dffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:41:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f323dffb700 1 -- 172.21.15.32:0/515131999 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c022070 con 0x55d11c090310 2023-09-30T14:41:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f3269105b80 1 -- 172.21.15.32:0/515131999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d11c082090 con 0x55d11c090310 2023-09-30T14:41:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.810+0000 7f323dffb700 1 -- 172.21.15.32:0/515131999 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f324c01c4f0 con 0x55d11c090310 2023-09-30T14:41:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.811+0000 7f323dffb700 1 -- 172.21.15.32:0/515131999 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f324c02d030 con 0x55d11c090310 2023-09-30T14:41:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.811+0000 7f3269105b80 10 client.15804 fetch_fsmap learned FSMap version 27 2023-09-30T14:41:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.811+0000 7f3269105b80 1 -- 172.21.15.32:0/515131999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d11be0cd80 con 0x55d11c090310 2023-09-30T14:41:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.811+0000 7f323dffb700 1 -- 172.21.15.32:0/515131999 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d11c082090 con 0x55d11c090310 2023-09-30T14:41:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.811+0000 7f323dffb700 1 -- 172.21.15.32:0/515131999 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f324c027740 con 0x55d11c090310 2023-09-30T14:41:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.811+0000 7f3269105b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 27 2023-09-30T14:41:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.811+0000 7f3269105b80 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:41:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.811+0000 7f3269105b80 20 client.15804 populate_metadata read hostname 'smithi032' 2023-09-30T14:41:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.811+0000 7f3269105b80 1 --2- 172.21.15.32:0/515131999 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d11c0a4d10 0x55d11c0a7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.811+0000 7f3269105b80 4 client.15804 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:41:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.811+0000 7f3269105b80 1 -- 172.21.15.32:0/515131999 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d11bfdadb0 con 0x55d11c0a4d10 2023-09-30T14:41:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.811+0000 7f325659c700 1 --2- 172.21.15.32:0/515131999 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d11c0a4d10 0x55d11c0a7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.812+0000 7f323dffb700 10 client.15804 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:41:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.812+0000 7f325659c700 1 --2- 172.21.15.32:0/515131999 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d11c0a4d10 0x55d11c0a7130 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=881e5de5e2d62da7 server_cookie=30bcf71f491542e6 in_seq=0 out_seq=0 2023-09-30T14:41:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.812+0000 7f323dffb700 1 -- 172.21.15.32:0/515131999 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d11bfdadb0 con 0x55d11c0a4d10 2023-09-30T14:41:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.812+0000 7f323dffb700 10 client.15804 handle_command_reply: tid=0 2023-09-30T14:41:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.812+0000 7f3269105b80 1 client.15804 shutdown 2023-09-30T14:41:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.812+0000 7f3269105b80 1 -- 172.21.15.32:0/515131999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 msgr2=0x55d11c0993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.812+0000 7f3269105b80 1 --2- 172.21.15.32:0/515131999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c0993a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f324c003900 tx=0x7f324c0047c0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.812+0000 7f3269105b80 1 -- 172.21.15.32:0/515131999 shutdown_connections 2023-09-30T14:41:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.812+0000 7f3269105b80 1 --2- 172.21.15.32:0/515131999 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d11c0a4d10 0x55d11c0a7130 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.812+0000 7f3269105b80 1 --2- 172.21.15.32:0/515131999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c090cd0 0x55d11c0a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.812+0000 7f3269105b80 1 --2- 172.21.15.32:0/515131999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c08f200 0x55d11c098e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.812+0000 7f323dffb700 0 client.15804 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:41:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.812+0000 7f3269105b80 1 --2- 172.21.15.32:0/515131999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c0993a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.812+0000 7f3269105b80 1 -- 172.21.15.32:0/515131999 >> 172.21.15.32:0/515131999 conn(0x55d11c003200 msgr2=0x55d11c089590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.812+0000 7f3269105b80 1 -- 172.21.15.32:0/515131999 shutdown_connections 2023-09-30T14:41:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.812+0000 7f3269105b80 1 -- 172.21.15.32:0/515131999 wait complete. 2023-09-30T14:41:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.813+0000 7f3269105b80 20 client.15804 trim_cache size 0 max 16384 2023-09-30T14:41:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.828+0000 7f3269105b80 1 Processor -- start 2023-09-30T14:41:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.828+0000 7f3269105b80 1 -- start start 2023-09-30T14:41:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.828+0000 7f3269105b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c08f200 0x55d11c0906f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.828+0000 7f3269105b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c0a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.828+0000 7f3269105b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c090cd0 0x55d11c097c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.828+0000 7f3269105b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11be628c0 con 0x55d11c090310 2023-09-30T14:41:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.828+0000 7f3269105b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11c098150 con 0x55d11c090cd0 2023-09-30T14:41:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.828+0000 7f3269105b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11c098a00 con 0x55d11c08f200 2023-09-30T14:41:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.828+0000 7f3255d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c0a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.828+0000 7f3255d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c0a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36428/0 (socket says 172.21.15.32:36428) 2023-09-30T14:41:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.828+0000 7f3255d9b700 1 -- 172.21.15.32:0/2790311240 learned_addr learned my addr 172.21.15.32:0/2790311240 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.828+0000 7f325659c700 1 --2- 172.21.15.32:0/2790311240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c08f200 0x55d11c0906f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.829+0000 7f3256d9d700 1 --2- 172.21.15.32:0/2790311240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c090cd0 0x55d11c097c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.829+0000 7f3255d9b700 1 -- 172.21.15.32:0/2790311240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c08f200 msgr2=0x55d11c0906f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.829+0000 7f3255d9b700 1 --2- 172.21.15.32:0/2790311240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c08f200 0x55d11c0906f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.829+0000 7f3255d9b700 1 -- 172.21.15.32:0/2790311240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c090cd0 msgr2=0x55d11c097c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.829+0000 7f3255d9b700 1 --2- 172.21.15.32:0/2790311240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c090cd0 0x55d11c097c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.829+0000 7f3255d9b700 1 -- 172.21.15.32:0/2790311240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d11bdf5c00 con 0x55d11c090310 2023-09-30T14:41:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.829+0000 7f3255d9b700 1 --2- 172.21.15.32:0/2790311240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c0a03a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f324c0097c0 tx=0x7f324c0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974508cf0d78df8b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.829+0000 7f32457fa700 1 -- 172.21.15.32:0/2790311240 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c022050 con 0x55d11c090310 2023-09-30T14:41:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.829+0000 7f32457fa700 1 -- 172.21.15.32:0/2790311240 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f324c023040 con 0x55d11c090310 2023-09-30T14:41:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.829+0000 7f32457fa700 1 -- 172.21.15.32:0/2790311240 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c02d040 con 0x55d11c090310 2023-09-30T14:41:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.830+0000 7f3269105b80 1 -- 172.21.15.32:0/2790311240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 msgr2=0x55d11c0a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.830+0000 7f3269105b80 1 --2- 172.21.15.32:0/2790311240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c0a03a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f324c0097c0 tx=0x7f324c0092d0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.830+0000 7f3269105b80 1 -- 172.21.15.32:0/2790311240 shutdown_connections 2023-09-30T14:41:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.830+0000 7f3269105b80 1 --2- 172.21.15.32:0/2790311240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c090cd0 0x55d11c097c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.830+0000 7f3269105b80 1 --2- 172.21.15.32:0/2790311240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c08f200 0x55d11c0906f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.830+0000 7f3269105b80 1 --2- 172.21.15.32:0/2790311240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c090310 0x55d11c0a03a0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.830+0000 7f3269105b80 1 -- 172.21.15.32:0/2790311240 >> 172.21.15.32:0/2790311240 conn(0x55d11c003200 msgr2=0x55d11c0abab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.830+0000 7f3269105b80 1 -- 172.21.15.32:0/2790311240 shutdown_connections 2023-09-30T14:41:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.830+0000 7f3269105b80 1 -- 172.21.15.32:0/2790311240 wait complete. 2023-09-30T14:41:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.830+0000 7f3269105b80 1 -- start start 2023-09-30T14:41:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.830+0000 7f3269105b80 1 Processor -- start 2023-09-30T14:41:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f3269105b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c08f200 0x55d11c09dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f3269105b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c090310 0x55d11c09e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f3269105b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c090cd0 0x55d11c09f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f3269105b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11c09f630 con 0x55d11c08f200 2023-09-30T14:41:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f3269105b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11c09f7a0 con 0x55d11c090310 2023-09-30T14:41:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f3269105b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11c09f960 con 0x55d11c090cd0 2023-09-30T14:41:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f3269105b80 10 client.0 _ll_register_callbacks cb 0x7f324c0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:41:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f325659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c08f200 0x55d11c09dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f325659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c08f200 0x55d11c09dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36432/0 (socket says 172.21.15.32:36432) 2023-09-30T14:41:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f3255d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c090310 0x55d11c09e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f325659c700 1 -- 172.21.15.32:0/2701119367 learned_addr learned my addr 172.21.15.32:0/2701119367 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f3256d9d700 1 --2- 172.21.15.32:0/2701119367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c090cd0 0x55d11c09f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f325659c700 1 -- 172.21.15.32:0/2701119367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c090cd0 msgr2=0x55d11c09f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f325659c700 1 --2- 172.21.15.32:0/2701119367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c090cd0 0x55d11c09f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f325659c700 1 -- 172.21.15.32:0/2701119367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c090310 msgr2=0x55d11c09e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f325659c700 1 --2- 172.21.15.32:0/2701119367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c090310 0x55d11c09e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f325659c700 1 -- 172.21.15.32:0/2701119367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d11be0cd80 con 0x55d11c08f200 2023-09-30T14:41:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f3255d9b700 1 --2- 172.21.15.32:0/2701119367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c090310 0x55d11c09e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:41:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f325659c700 1 --2- 172.21.15.32:0/2701119367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c08f200 0x55d11c09dc80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f32480cbaa0 tx=0x7f3248005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54bf597a6a541a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f323dffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:41:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f323dffb700 1 -- 172.21.15.32:0/2701119367 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324809f640 con 0x55d11c08f200 2023-09-30T14:41:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f323dffb700 1 -- 172.21.15.32:0/2701119367 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f32480a37f0 con 0x55d11c08f200 2023-09-30T14:41:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.831+0000 7f3269105b80 1 -- 172.21.15.32:0/2701119367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f325000f040 con 0x55d11c08f200 2023-09-30T14:41:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.832+0000 7f323dffb700 1 -- 172.21.15.32:0/2701119367 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f324809b750 con 0x55d11c08f200 2023-09-30T14:41:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.832+0000 7f3269105b80 10 client.15810 fetch_fsmap learned FSMap version 27 2023-09-30T14:41:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.832+0000 7f3269105b80 1 -- 172.21.15.32:0/2701119367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d11be331f0 con 0x55d11c08f200 2023-09-30T14:41:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.832+0000 7f323dffb700 1 -- 172.21.15.32:0/2701119367 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f325000f040 con 0x55d11c08f200 2023-09-30T14:41:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.832+0000 7f323dffb700 1 -- 172.21.15.32:0/2701119367 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 27) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f32480a3360 con 0x55d11c08f200 2023-09-30T14:41:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.832+0000 7f3269105b80 10 client.15810 fetch_fsmap finished waiting for FSMap version 27 2023-09-30T14:41:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.832+0000 7f3269105b80 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:41:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.832+0000 7f3269105b80 20 client.15810 populate_metadata read hostname 'smithi032' 2023-09-30T14:41:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.832+0000 7f3269105b80 1 --2- 172.21.15.32:0/2701119367 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d11c0f9080 0x55d11c0fb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.832+0000 7f3269105b80 4 client.15810 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:41:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.832+0000 7f3269105b80 1 -- 172.21.15.32:0/2701119367 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d11bfdaad0 con 0x55d11c0f9080 2023-09-30T14:41:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.832+0000 7f3255d9b700 1 --2- 172.21.15.32:0/2701119367 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d11c0f9080 0x55d11c0fb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.832+0000 7f323dffb700 10 client.15810 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:41:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.832+0000 7f3255d9b700 1 --2- 172.21.15.32:0/2701119367 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d11c0f9080 0x55d11c0fb470 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53805929d0bba064 server_cookie=434679dc82bc5ad8 in_seq=0 out_seq=0 2023-09-30T14:41:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.871+0000 7f323dffb700 1 -- 172.21.15.32:0/2701119367 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d11bfdaad0 con 0x55d11c0f9080 2023-09-30T14:41:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.871+0000 7f323dffb700 10 client.15810 handle_command_reply: tid=0 2023-09-30T14:41:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.871+0000 7f3269105b80 1 client.15810 shutdown 2023-09-30T14:41:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.871+0000 7f3269105b80 1 -- 172.21.15.32:0/2701119367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c08f200 msgr2=0x55d11c09dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.871+0000 7f3269105b80 1 --2- 172.21.15.32:0/2701119367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c08f200 0x55d11c09dc80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f32480cbaa0 tx=0x7f3248005040 comp rx=0 tx=0).stop 2023-09-30T14:41:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.871+0000 7f3269105b80 1 -- 172.21.15.32:0/2701119367 shutdown_connections 2023-09-30T14:41:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.871+0000 7f3269105b80 1 --2- 172.21.15.32:0/2701119367 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d11c0f9080 0x55d11c0fb470 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.871+0000 7f3269105b80 1 --2- 172.21.15.32:0/2701119367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d11c090cd0 0x55d11c09f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.871+0000 7f3269105b80 1 --2- 172.21.15.32:0/2701119367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d11c08f200 0x55d11c09dc80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.871+0000 7f323dffb700 0 client.15810 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:41:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.871+0000 7f3269105b80 1 --2- 172.21.15.32:0/2701119367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d11c090310 0x55d11c09e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.871+0000 7f3269105b80 1 -- 172.21.15.32:0/2701119367 >> 172.21.15.32:0/2701119367 conn(0x55d11c003200 msgr2=0x55d11c089e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.871+0000 7f3269105b80 1 -- 172.21.15.32:0/2701119367 shutdown_connections 2023-09-30T14:41:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.871+0000 7f3269105b80 1 -- 172.21.15.32:0/2701119367 wait complete. 2023-09-30T14:41:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.871+0000 7f3269105b80 20 client.15810 trim_cache size 0 max 16384 2023-09-30T14:41:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:41:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:41:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:41:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:41:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.872+0000 7f325cf1d700 1 -- 172.21.15.32:0/1902936403 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f32400a0fa0 msgr2=0x7f32400a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.872+0000 7f325cf1d700 1 --2- 172.21.15.32:0/1902936403 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f32400a0fa0 0x7f32400a3460 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f324c009c60 tx=0x7f324c009260 comp rx=0 tx=0).stop 2023-09-30T14:41:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.872+0000 7f325cf1d700 1 -- 172.21.15.32:0/1902936403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32580ff520 msgr2=0x7f3258196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.872+0000 7f325cf1d700 1 --2- 172.21.15.32:0/1902936403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32580ff520 0x7f3258196d30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3248003ab0 tx=0x7f324800e4d0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.872+0000 7f325cf1d700 1 -- 172.21.15.32:0/1902936403 shutdown_connections 2023-09-30T14:41:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.872+0000 7f325cf1d700 1 --2- 172.21.15.32:0/1902936403 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f32400a0fa0 0x7f32400a3460 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.872+0000 7f325cf1d700 1 --2- 172.21.15.32:0/1902936403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32581010b0 0x7f3258197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.872+0000 7f325cf1d700 1 --2- 172.21.15.32:0/1902936403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32580ff520 0x7f3258196d30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.872+0000 7f325cf1d700 1 --2- 172.21.15.32:0/1902936403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3258100710 0x7f3258197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.872+0000 7f325cf1d700 1 -- 172.21.15.32:0/1902936403 >> 172.21.15.32:0/1902936403 conn(0x7f32580faa90 msgr2=0x7f3258103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.872+0000 7f325cf1d700 1 -- 172.21.15.32:0/1902936403 shutdown_connections 2023-09-30T14:41:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:33.872+0000 7f325cf1d700 1 -- 172.21.15.32:0/1902936403 wait complete. 2023-09-30T14:41:34.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 291016 11.82 MB/sec execute 8 sec latency 21.578 ms 2023-09-30T14:41:35.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:35 smithi156 ceph-mon[103579]: pgmap v386: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 3.0 MiB/s wr, 296 op/s 2023-09-30T14:41:35.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:35 smithi148 ceph-mon[102355]: pgmap v386: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 3.0 MiB/s wr, 296 op/s 2023-09-30T14:41:35.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:35 smithi032 ceph-mon[98592]: pgmap v386: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 3.0 MiB/s wr, 296 op/s 2023-09-30T14:41:35.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 292469 11.23 MB/sec execute 9 sec latency 51.579 ms 2023-09-30T14:41:36.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 294830 11.50 MB/sec execute 10 sec latency 12.540 ms 2023-09-30T14:41:37.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:37 smithi156 ceph-mon[103579]: pgmap v387: 97 pgs: 97 active+clean; 118 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 351 op/s 2023-09-30T14:41:37.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:37 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:41:37.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:37 smithi148 ceph-mon[102355]: pgmap v387: 97 pgs: 97 active+clean; 118 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 351 op/s 2023-09-30T14:41:37.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:37 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:41:37.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:37 smithi032 ceph-mon[98592]: pgmap v387: 97 pgs: 97 active+clean; 118 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 351 op/s 2023-09-30T14:41:37.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:37 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:41:37.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 296887 11.11 MB/sec execute 11 sec latency 46.702 ms 2023-09-30T14:41:38.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 298593 11.18 MB/sec execute 12 sec latency 13.911 ms 2023-09-30T14:41:39.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:39 smithi156 ceph-mon[103579]: pgmap v388: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 3.0 MiB/s wr, 307 op/s 2023-09-30T14:41:39.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:39 smithi148 ceph-mon[102355]: pgmap v388: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 3.0 MiB/s wr, 307 op/s 2023-09-30T14:41:39.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:39 smithi032 ceph-mon[98592]: pgmap v388: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 3.0 MiB/s wr, 307 op/s 2023-09-30T14:41:39.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 301532 11.66 MB/sec execute 13 sec latency 11.678 ms 2023-09-30T14:41:40.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 302677 11.09 MB/sec execute 14 sec latency 35.276 ms 2023-09-30T14:41:41.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:41 smithi156 ceph-mon[103579]: pgmap v389: 97 pgs: 97 active+clean; 119 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 2.9 MiB/s wr, 289 op/s 2023-09-30T14:41:41.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:41 smithi148 ceph-mon[102355]: pgmap v389: 97 pgs: 97 active+clean; 119 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 2.9 MiB/s wr, 289 op/s 2023-09-30T14:41:41.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:41 smithi032 ceph-mon[98592]: pgmap v389: 97 pgs: 97 active+clean; 119 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 2.9 MiB/s wr, 289 op/s 2023-09-30T14:41:41.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 305336 11.47 MB/sec execute 15 sec latency 12.495 ms 2023-09-30T14:41:42.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 306911 11.11 MB/sec execute 16 sec latency 12.206 ms 2023-09-30T14:41:43.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:43 smithi156 ceph-mon[103579]: pgmap v390: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.8 MiB/s wr, 310 op/s 2023-09-30T14:41:43.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:43 smithi148 ceph-mon[102355]: pgmap v390: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.8 MiB/s wr, 310 op/s 2023-09-30T14:41:43.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:43 smithi032 ceph-mon[98592]: pgmap v390: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.8 MiB/s wr, 310 op/s 2023-09-30T14:41:43.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 309008 11.27 MB/sec execute 17 sec latency 33.091 ms 2023-09-30T14:41:44.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 311055 11.04 MB/sec execute 18 sec latency 61.523 ms 2023-09-30T14:41:45.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:45 smithi156 ceph-mon[103579]: pgmap v391: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 2.7 MiB/s wr, 290 op/s 2023-09-30T14:41:45.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:45 smithi148 ceph-mon[102355]: pgmap v391: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 2.7 MiB/s wr, 290 op/s 2023-09-30T14:41:45.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:45 smithi032 ceph-mon[98592]: pgmap v391: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 2.7 MiB/s wr, 290 op/s 2023-09-30T14:41:45.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 312636 11.09 MB/sec execute 19 sec latency 40.331 ms 2023-09-30T14:41:46.695 INFO:tasks.workunit.client.0.smithi032.stdout: 1 315155 11.16 MB/sec execute 20 sec latency 14.435 ms 2023-09-30T14:41:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.982+0000 7fd84ffd1700 1 -- 172.21.15.32:0/3042205984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8480ffd60 msgr2=0x7fd848100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.982+0000 7fd84ffd1700 1 --2- 172.21.15.32:0/3042205984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8480ffd60 0x7fd848100180 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd8440099c0 tx=0x7fd844009cd0 comp rx=0 tx=0).stop 2023-09-30T14:41:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.982+0000 7fd84ffd1700 1 -- 172.21.15.32:0/3042205984 shutdown_connections 2023-09-30T14:41:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.982+0000 7fd84ffd1700 1 --2- 172.21.15.32:0/3042205984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8481018f0 0x7fd8481081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.982+0000 7fd84ffd1700 1 --2- 172.21.15.32:0/3042205984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8480ffd60 0x7fd848100180 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.982+0000 7fd84ffd1700 1 --2- 172.21.15.32:0/3042205984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd848100f50 0x7fd8481013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.982+0000 7fd84ffd1700 1 -- 172.21.15.32:0/3042205984 >> 172.21.15.32:0/3042205984 conn(0x7fd8480fb2b0 msgr2=0x7fd8480fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.983+0000 7fd84ffd1700 1 -- 172.21.15.32:0/3042205984 shutdown_connections 2023-09-30T14:41:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.983+0000 7fd84ffd1700 1 -- 172.21.15.32:0/3042205984 wait complete. 2023-09-30T14:41:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.983+0000 7fd84ffd1700 1 Processor -- start 2023-09-30T14:41:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.983+0000 7fd84ffd1700 1 -- start start 2023-09-30T14:41:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.983+0000 7fd84ffd1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8480ffd60 0x7fd8481998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.983+0000 7fd84ffd1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd848100f50 0x7fd84819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.983+0000 7fd84ffd1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8481018f0 0x7fd84819e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.983+0000 7fd84ffd1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84819e910 con 0x7fd8481018f0 2023-09-30T14:41:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.983+0000 7fd84ffd1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84819ea50 con 0x7fd8480ffd60 2023-09-30T14:41:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.983+0000 7fd84ffd1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84819ec10 con 0x7fd848100f50 2023-09-30T14:41:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd84e56e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8481018f0 0x7fd84819e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd84dd6d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8480ffd60 0x7fd8481998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd84d56c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd848100f50 0x7fd84819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd84e56e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8481018f0 0x7fd84819e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36450/0 (socket says 172.21.15.32:36450) 2023-09-30T14:41:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd84e56e700 1 -- 172.21.15.32:0/837300160 learned_addr learned my addr 172.21.15.32:0/837300160 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd84e56e700 1 -- 172.21.15.32:0/837300160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd848100f50 msgr2=0x7fd84819bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd84e56e700 1 --2- 172.21.15.32:0/837300160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd848100f50 0x7fd84819bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd84e56e700 1 -- 172.21.15.32:0/837300160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8480ffd60 msgr2=0x7fd8481998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd84e56e700 1 --2- 172.21.15.32:0/837300160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8480ffd60 0x7fd8481998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd84e56e700 1 -- 172.21.15.32:0/837300160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8440095f0 con 0x7fd8481018f0 2023-09-30T14:41:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd84dd6d700 1 --2- 172.21.15.32:0/837300160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8480ffd60 0x7fd8481998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:41:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd84e56e700 1 --2- 172.21.15.32:0/837300160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8481018f0 0x7fd84819e340 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd84000ef30 tx=0x7fd84000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd83effd700 1 -- 172.21.15.32:0/837300160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840018070 con 0x7fd8481018f0 2023-09-30T14:41:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd84ffd1700 1 -- 172.21.15.32:0/837300160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd84819ee90 con 0x7fd8481018f0 2023-09-30T14:41:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd83effd700 1 -- 172.21.15.32:0/837300160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd84000f040 con 0x7fd8481018f0 2023-09-30T14:41:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd84ffd1700 1 -- 172.21.15.32:0/837300160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84819f310 con 0x7fd8481018f0 2023-09-30T14:41:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.984+0000 7fd83effd700 1 -- 172.21.15.32:0/837300160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84000cd00 con 0x7fd8481018f0 2023-09-30T14:41:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.986+0000 7fd83effd700 1 -- 172.21.15.32:0/837300160 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fd840004ad0 con 0x7fd8481018f0 2023-09-30T14:41:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.987+0000 7fd83effd700 1 --2- 172.21.15.32:0/837300160 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd8340a1020 0x7fd8340a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.987+0000 7fd83effd700 1 -- 172.21.15.32:0/837300160 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd8400cc130 con 0x7fd8481018f0 2023-09-30T14:41:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.987+0000 7fd84dd6d700 1 --2- 172.21.15.32:0/837300160 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd8340a1020 0x7fd8340a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.988+0000 7fd84dd6d700 1 --2- 172.21.15.32:0/837300160 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd8340a1020 0x7fd8340a34e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd848068d00 tx=0x7fd84401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.988+0000 7fd85c1b9b80 1 Processor -- start 2023-09-30T14:41:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.988+0000 7fd85c1b9b80 1 -- start start 2023-09-30T14:41:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.988+0000 7fd85c1b9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 0x557c6949f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.988+0000 7fd85c1b9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c694a0310 0x557c694a0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.988+0000 7fd85c1b9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0cd0 0x557c694a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.988+0000 7fd85c1b9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c69407400 con 0x557c694a0cd0 2023-09-30T14:41:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.988+0000 7fd85c1b9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c693f8bc0 con 0x557c694a0310 2023-09-30T14:41:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.988+0000 7fd85c1b9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c693f64e0 con 0x557c6949f200 2023-09-30T14:41:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.988+0000 7fd84dd6d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 0x557c6949f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.988+0000 7fd84dd6d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 0x557c6949f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:50836/0 (socket says 172.21.15.32:50836) 2023-09-30T14:41:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.988+0000 7fd84dd6d700 1 -- 172.21.15.32:0/379749593 learned_addr learned my addr 172.21.15.32:0/379749593 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.988+0000 7fd84e56e700 1 --2- 172.21.15.32:0/379749593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0cd0 0x557c694a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.988+0000 7fd84d56c700 1 --2- 172.21.15.32:0/379749593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c694a0310 0x557c694a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.989+0000 7fd84dd6d700 1 -- 172.21.15.32:0/379749593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c694a0310 msgr2=0x557c694a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.989+0000 7fd84dd6d700 1 --2- 172.21.15.32:0/379749593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c694a0310 0x557c694a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.989+0000 7fd84dd6d700 1 -- 172.21.15.32:0/379749593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0cd0 msgr2=0x557c694a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.989+0000 7fd84dd6d700 1 --2- 172.21.15.32:0/379749593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0cd0 0x557c694a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.989+0000 7fd84dd6d700 1 -- 172.21.15.32:0/379749593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c69454870 con 0x557c6949f200 2023-09-30T14:41:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.989+0000 7fd84dd6d700 1 --2- 172.21.15.32:0/379749593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 0x557c6949f620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd844000c00 tx=0x7fd84401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af4f35f2b583709c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.989+0000 7fd833fff700 1 -- 172.21.15.32:0/379749593 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84402a070 con 0x557c6949f200 2023-09-30T14:41:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.989+0000 7fd833fff700 1 -- 172.21.15.32:0/379749593 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd84401e040 con 0x557c6949f200 2023-09-30T14:41:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.989+0000 7fd833fff700 1 -- 172.21.15.32:0/379749593 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84400f800 con 0x557c6949f200 2023-09-30T14:41:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.990+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/379749593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 msgr2=0x557c6949f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.990+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/379749593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 0x557c6949f620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd844000c00 tx=0x7fd84401f000 comp rx=0 tx=0).stop 2023-09-30T14:41:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.990+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/379749593 shutdown_connections 2023-09-30T14:41:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.990+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/379749593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0cd0 0x557c694a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.990+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/379749593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 0x557c6949f620 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.990+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/379749593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c694a0310 0x557c694a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.990+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/379749593 >> 172.21.15.32:0/379749593 conn(0x557c69413200 msgr2=0x557c69499ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.990+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/379749593 shutdown_connections 2023-09-30T14:41:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.990+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/379749593 wait complete. 2023-09-30T14:41:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.990+0000 7fd85c1b9b80 1 -- start start 2023-09-30T14:41:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.990+0000 7fd85c1b9b80 1 Processor -- start 2023-09-30T14:41:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd85c1b9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 0x557c694a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd85c1b9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0310 0x557c694a9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd85c1b9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c694a0cd0 0x557c694b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd85c1b9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c693f1dd0 con 0x557c694a0310 2023-09-30T14:41:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd85c1b9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6938e6b0 con 0x557c694a0cd0 2023-09-30T14:41:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd85c1b9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c6938e350 con 0x557c6949f200 2023-09-30T14:41:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd85c1b9b80 10 client.0 _ll_register_callbacks cb 0x557c691eb7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:41:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd84d56c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0310 0x557c694a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd84dd6d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 0x557c694a8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd84d56c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0310 0x557c694a9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36462/0 (socket says 172.21.15.32:36462) 2023-09-30T14:41:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd84d56c700 1 -- 172.21.15.32:0/2951506273 learned_addr learned my addr 172.21.15.32:0/2951506273 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd84e56e700 1 --2- 172.21.15.32:0/2951506273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c694a0cd0 0x557c694b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd84d56c700 1 -- 172.21.15.32:0/2951506273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 msgr2=0x557c694a8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd84d56c700 1 --2- 172.21.15.32:0/2951506273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 0x557c694a8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd84d56c700 1 -- 172.21.15.32:0/2951506273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c694a0cd0 msgr2=0x557c694b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd84d56c700 1 --2- 172.21.15.32:0/2951506273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c694a0cd0 0x557c694b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd84d56c700 1 -- 172.21.15.32:0/2951506273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd848108af0 con 0x557c694a0310 2023-09-30T14:41:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd84d56c700 1 --2- 172.21.15.32:0/2951506273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0310 0x557c694a9400 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd838012770 tx=0x7fd838012a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4b0df19fd6a6b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd8317fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:41:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd8317fa700 1 -- 172.21.15.32:0/2951506273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd838017070 con 0x557c694a0310 2023-09-30T14:41:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/2951506273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c69492090 con 0x557c694a0310 2023-09-30T14:41:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.991+0000 7fd8317fa700 1 -- 172.21.15.32:0/2951506273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd83800f480 con 0x557c694a0310 2023-09-30T14:41:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.992+0000 7fd8317fa700 1 -- 172.21.15.32:0/2951506273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd83800f640 con 0x557c694a0310 2023-09-30T14:41:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.992+0000 7fd85c1b9b80 10 client.15822 fetch_fsmap learned FSMap version 28 2023-09-30T14:41:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.992+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/2951506273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557c6921cd80 con 0x557c694a0310 2023-09-30T14:41:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.992+0000 7fd8317fa700 1 -- 172.21.15.32:0/2951506273 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c69492090 con 0x557c694a0310 2023-09-30T14:41:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.992+0000 7fd8317fa700 1 -- 172.21.15.32:0/2951506273 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd838010790 con 0x557c694a0310 2023-09-30T14:41:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.992+0000 7fd85c1b9b80 10 client.15822 fetch_fsmap finished waiting for FSMap version 28 2023-09-30T14:41:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.992+0000 7fd85c1b9b80 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:41:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.992+0000 7fd85c1b9b80 20 client.15822 populate_metadata read hostname 'smithi032' 2023-09-30T14:41:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.992+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/2951506273 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557c694b4e50 0x557c694b7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.992+0000 7fd85c1b9b80 4 client.15822 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:41:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.992+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/2951506273 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c693eadb0 con 0x557c694b4e50 2023-09-30T14:41:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.992+0000 7fd84dd6d700 1 --2- 172.21.15.32:0/2951506273 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557c694b4e50 0x557c694b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.993+0000 7fd8317fa700 10 client.15822 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:41:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.993+0000 7fd84dd6d700 1 --2- 172.21.15.32:0/2951506273 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557c694b4e50 0x557c694b7270 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49ffc8bb47a9098e server_cookie=a181b0978cdd4526 in_seq=0 out_seq=0 2023-09-30T14:41:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.993+0000 7fd8317fa700 1 -- 172.21.15.32:0/2951506273 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557c693eadb0 con 0x557c694b4e50 2023-09-30T14:41:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.993+0000 7fd8317fa700 10 client.15822 handle_command_reply: tid=0 2023-09-30T14:41:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.993+0000 7fd85c1b9b80 1 client.15822 shutdown 2023-09-30T14:41:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.993+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/2951506273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0310 msgr2=0x557c694a9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.993+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/2951506273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0310 0x557c694a9400 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd838012770 tx=0x7fd838012a80 comp rx=0 tx=0).stop 2023-09-30T14:41:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.993+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/2951506273 shutdown_connections 2023-09-30T14:41:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.993+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/2951506273 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557c694b4e50 0x557c694b7270 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.994+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/2951506273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c694a0cd0 0x557c694b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.994+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/2951506273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 0x557c694a8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.994+0000 7fd8317fa700 0 client.15822 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:41:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.994+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/2951506273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0310 0x557c694a9400 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.994+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/2951506273 >> 172.21.15.32:0/2951506273 conn(0x557c69413200 msgr2=0x557c69499590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.994+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/2951506273 shutdown_connections 2023-09-30T14:41:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.994+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/2951506273 wait complete. 2023-09-30T14:41:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:46.994+0000 7fd85c1b9b80 20 client.15822 trim_cache size 0 max 16384 2023-09-30T14:41:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.009+0000 7fd85c1b9b80 1 Processor -- start 2023-09-30T14:41:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.009+0000 7fd85c1b9b80 1 -- start start 2023-09-30T14:41:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.009+0000 7fd85c1b9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 0x557c694a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.009+0000 7fd85c1b9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0310 0x557c694b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.009+0000 7fd85c1b9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c694a0cd0 0x557c694a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.009+0000 7fd85c1b9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c692728c0 con 0x557c694a0310 2023-09-30T14:41:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.009+0000 7fd85c1b9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c694a9c00 con 0x557c694a0cd0 2023-09-30T14:41:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.009+0000 7fd85c1b9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c694a6f40 con 0x557c6949f200 2023-09-30T14:41:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.009+0000 7fd84d56c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0310 0x557c694b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.009+0000 7fd84d56c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0310 0x557c694b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36470/0 (socket says 172.21.15.32:36470) 2023-09-30T14:41:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.009+0000 7fd84d56c700 1 -- 172.21.15.32:0/3993495080 learned_addr learned my addr 172.21.15.32:0/3993495080 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.009+0000 7fd84dd6d700 1 --2- 172.21.15.32:0/3993495080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 0x557c694a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.009+0000 7fd84e56e700 1 --2- 172.21.15.32:0/3993495080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c694a0cd0 0x557c694a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.009+0000 7fd84dd6d700 1 -- 172.21.15.32:0/3993495080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c694a0cd0 msgr2=0x557c694a7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.009+0000 7fd84dd6d700 1 --2- 172.21.15.32:0/3993495080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c694a0cd0 0x557c694a7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.010+0000 7fd84dd6d700 1 -- 172.21.15.32:0/3993495080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0310 msgr2=0x557c694b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.010+0000 7fd84dd6d700 1 --2- 172.21.15.32:0/3993495080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0310 0x557c694b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.010+0000 7fd84dd6d700 1 -- 172.21.15.32:0/3993495080 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c69205c00 con 0x557c6949f200 2023-09-30T14:41:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.010+0000 7fd84d56c700 1 --2- 172.21.15.32:0/3993495080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0310 0x557c694b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:41:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.010+0000 7fd84dd6d700 1 --2- 172.21.15.32:0/3993495080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 0x557c694a06f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd8440098e0 tx=0x7fd84400f4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb2514c1f25b096a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.010+0000 7fd83cff9700 1 -- 172.21.15.32:0/3993495080 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84402a040 con 0x557c6949f200 2023-09-30T14:41:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.010+0000 7fd83cff9700 1 -- 172.21.15.32:0/3993495080 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd84403b5a0 con 0x557c6949f200 2023-09-30T14:41:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.010+0000 7fd83cff9700 1 -- 172.21.15.32:0/3993495080 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84400fba0 con 0x557c6949f200 2023-09-30T14:41:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.010+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/3993495080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 msgr2=0x557c694a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.010+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/3993495080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 0x557c694a06f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd8440098e0 tx=0x7fd84400f4c0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.010+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/3993495080 shutdown_connections 2023-09-30T14:41:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.010+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/3993495080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c694a0cd0 0x557c694a7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.010+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/3993495080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c6949f200 0x557c694a06f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.010+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/3993495080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0310 0x557c694b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/3993495080 >> 172.21.15.32:0/3993495080 conn(0x557c69413200 msgr2=0x557c69499590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/3993495080 shutdown_connections 2023-09-30T14:41:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/3993495080 wait complete. 2023-09-30T14:41:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd85c1b9b80 1 -- start start 2023-09-30T14:41:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd85c1b9b80 1 Processor -- start 2023-09-30T14:41:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd85c1b9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c6949f200 0x557c694adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd85c1b9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c694a0310 0x557c694ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd85c1b9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0cd0 0x557c694af130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd85c1b9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c694af670 con 0x557c694a0cd0 2023-09-30T14:41:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd85c1b9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c694af7e0 con 0x557c6949f200 2023-09-30T14:41:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd85c1b9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c694af9a0 con 0x557c694a0310 2023-09-30T14:41:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd85c1b9b80 10 client.0 _ll_register_callbacks cb 0x7fd838009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:41:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd84e56e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0cd0 0x557c694af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd84e56e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0cd0 0x557c694af130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36478/0 (socket says 172.21.15.32:36478) 2023-09-30T14:41:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd84e56e700 1 -- 172.21.15.32:0/3308018453 learned_addr learned my addr 172.21.15.32:0/3308018453 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:41:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd84dd6d700 1 --2- 172.21.15.32:0/3308018453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c6949f200 0x557c694adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.011+0000 7fd84d56c700 1 --2- 172.21.15.32:0/3308018453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c694a0310 0x557c694ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.012+0000 7fd84e56e700 1 -- 172.21.15.32:0/3308018453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c694a0310 msgr2=0x557c694ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.012+0000 7fd84e56e700 1 --2- 172.21.15.32:0/3308018453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c694a0310 0x557c694ae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.012+0000 7fd84e56e700 1 -- 172.21.15.32:0/3308018453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c6949f200 msgr2=0x557c694adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:41:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.012+0000 7fd84e56e700 1 --2- 172.21.15.32:0/3308018453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c6949f200 0x557c694adbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.012+0000 7fd84e56e700 1 -- 172.21.15.32:0/3308018453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c6921cd80 con 0x557c694a0cd0 2023-09-30T14:41:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.012+0000 7fd84e56e700 1 --2- 172.21.15.32:0/3308018453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0cd0 0x557c694af130 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd840099390 tx=0x7fd840099bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ad2e6304cc666d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:41:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.012+0000 7fd8317fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:41:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.012+0000 7fd8317fa700 1 -- 172.21.15.32:0/3308018453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840099690 con 0x557c694a0cd0 2023-09-30T14:41:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.012+0000 7fd8317fa700 1 -- 172.21.15.32:0/3308018453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd84009d7e0 con 0x557c694a0cd0 2023-09-30T14:41:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.012+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/3308018453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd84401e040 con 0x557c694a0cd0 2023-09-30T14:41:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.012+0000 7fd8317fa700 1 -- 172.21.15.32:0/3308018453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8400a1990 con 0x557c694a0cd0 2023-09-30T14:41:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.012+0000 7fd85c1b9b80 10 client.15828 fetch_fsmap learned FSMap version 28 2023-09-30T14:41:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.012+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/3308018453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557c69454870 con 0x557c694a0cd0 2023-09-30T14:41:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.012+0000 7fd8317fa700 1 -- 172.21.15.32:0/3308018453 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd84401e040 con 0x557c694a0cd0 2023-09-30T14:41:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.012+0000 7fd8317fa700 1 -- 172.21.15.32:0/3308018453 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd8400af070 con 0x557c694a0cd0 2023-09-30T14:41:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.012+0000 7fd85c1b9b80 10 client.15828 fetch_fsmap finished waiting for FSMap version 28 2023-09-30T14:41:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.013+0000 7fd85c1b9b80 10 client.15828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:41:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.013+0000 7fd85c1b9b80 20 client.15828 populate_metadata read hostname 'smithi032' 2023-09-30T14:41:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.013+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/3308018453 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557c69509760 0x557c6950bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:41:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.013+0000 7fd85c1b9b80 4 client.15828 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:41:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.013+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/3308018453 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c693eaad0 con 0x557c69509760 2023-09-30T14:41:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.013+0000 7fd84dd6d700 1 --2- 172.21.15.32:0/3308018453 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557c69509760 0x557c6950bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:41:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.013+0000 7fd8317fa700 10 client.15828 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:41:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.013+0000 7fd84dd6d700 1 --2- 172.21.15.32:0/3308018453 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557c69509760 0x557c6950bbe0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=579cce1810cb3b67 server_cookie=92c8b3c322032179 in_seq=0 out_seq=0 2023-09-30T14:41:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.050+0000 7fd8317fa700 1 -- 172.21.15.32:0/3308018453 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c693eaad0 con 0x557c69509760 2023-09-30T14:41:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.050+0000 7fd8317fa700 10 client.15828 handle_command_reply: tid=0 2023-09-30T14:41:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.050+0000 7fd85c1b9b80 1 client.15828 shutdown 2023-09-30T14:41:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.050+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/3308018453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0cd0 msgr2=0x557c694af130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.050+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/3308018453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0cd0 0x557c694af130 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd840099390 tx=0x7fd840099bd0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/3308018453 shutdown_connections 2023-09-30T14:41:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/3308018453 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557c69509760 0x557c6950bbe0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/3308018453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557c694a0cd0 0x557c694af130 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/3308018453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c6949f200 0x557c694adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd85c1b9b80 1 --2- 172.21.15.32:0/3308018453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c694a0310 0x557c694ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd8317fa700 0 client.15828 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:41:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/3308018453 >> 172.21.15.32:0/3308018453 conn(0x557c69413200 msgr2=0x557c694a7110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/3308018453 shutdown_connections 2023-09-30T14:41:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd85c1b9b80 1 -- 172.21.15.32:0/3308018453 wait complete. 2023-09-30T14:41:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd85c1b9b80 20 client.15828 trim_cache size 0 max 16384 2023-09-30T14:41:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:41:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:41:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:41:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:41:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd84ffd1700 1 -- 172.21.15.32:0/837300160 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd8340a1020 msgr2=0x7fd8340a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd84ffd1700 1 --2- 172.21.15.32:0/837300160 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd8340a1020 0x7fd8340a34e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd848068d00 tx=0x7fd84401a040 comp rx=0 tx=0).stop 2023-09-30T14:41:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd84ffd1700 1 -- 172.21.15.32:0/837300160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8481018f0 msgr2=0x7fd84819e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:41:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd84ffd1700 1 --2- 172.21.15.32:0/837300160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8481018f0 0x7fd84819e340 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd84000ef30 tx=0x7fd84000c3a0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd84ffd1700 1 -- 172.21.15.32:0/837300160 shutdown_connections 2023-09-30T14:41:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd84ffd1700 1 --2- 172.21.15.32:0/837300160 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd8340a1020 0x7fd8340a34e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd84ffd1700 1 --2- 172.21.15.32:0/837300160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8481018f0 0x7fd84819e340 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd84ffd1700 1 --2- 172.21.15.32:0/837300160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8480ffd60 0x7fd8481998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd84ffd1700 1 --2- 172.21.15.32:0/837300160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd848100f50 0x7fd84819bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:41:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd84ffd1700 1 -- 172.21.15.32:0/837300160 >> 172.21.15.32:0/837300160 conn(0x7fd8480fb2b0 msgr2=0x7fd8480fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:41:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd84ffd1700 1 -- 172.21.15.32:0/837300160 shutdown_connections 2023-09-30T14:41:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:41:47.051+0000 7fd84ffd1700 1 -- 172.21.15.32:0/837300160 wait complete. 2023-09-30T14:41:47.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:47 smithi156 ceph-mon[103579]: pgmap v392: 97 pgs: 97 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 352 op/s 2023-09-30T14:41:47.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:47 smithi148 ceph-mon[102355]: pgmap v392: 97 pgs: 97 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 352 op/s 2023-09-30T14:41:47.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:47 smithi032 ceph-mon[98592]: pgmap v392: 97 pgs: 97 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 352 op/s 2023-09-30T14:41:47.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 316485 10.96 MB/sec execute 21 sec latency 17.045 ms 2023-09-30T14:41:48.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 319074 11.10 MB/sec execute 22 sec latency 12.839 ms 2023-09-30T14:41:49.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:49 smithi156 ceph-mon[103579]: pgmap v393: 97 pgs: 97 active+clean; 123 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.7 MiB/s wr, 279 op/s 2023-09-30T14:41:49.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:49 smithi148 ceph-mon[102355]: pgmap v393: 97 pgs: 97 active+clean; 123 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.7 MiB/s wr, 279 op/s 2023-09-30T14:41:49.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:49 smithi032 ceph-mon[98592]: pgmap v393: 97 pgs: 97 active+clean; 123 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.7 MiB/s wr, 279 op/s 2023-09-30T14:41:49.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 320120 10.85 MB/sec execute 23 sec latency 63.392 ms 2023-09-30T14:41:50.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 322789 11.10 MB/sec execute 24 sec latency 41.578 ms 2023-09-30T14:41:51.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:51 smithi156 ceph-mon[103579]: pgmap v394: 97 pgs: 97 active+clean; 123 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 2.8 MiB/s wr, 283 op/s 2023-09-30T14:41:51.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:51 smithi148 ceph-mon[102355]: pgmap v394: 97 pgs: 97 active+clean; 123 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 2.8 MiB/s wr, 283 op/s 2023-09-30T14:41:51.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:51 smithi032 ceph-mon[98592]: pgmap v394: 97 pgs: 97 active+clean; 123 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 2.8 MiB/s wr, 283 op/s 2023-09-30T14:41:51.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 323974 10.80 MB/sec execute 25 sec latency 18.991 ms 2023-09-30T14:41:52.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 326581 11.03 MB/sec execute 26 sec latency 11.130 ms 2023-09-30T14:41:53.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:53 smithi156 ceph-mon[103579]: pgmap v395: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 2.9 MiB/s wr, 306 op/s 2023-09-30T14:41:53.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:53 smithi148 ceph-mon[102355]: pgmap v395: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 2.9 MiB/s wr, 306 op/s 2023-09-30T14:41:53.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:53 smithi032 ceph-mon[98592]: pgmap v395: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 2.9 MiB/s wr, 306 op/s 2023-09-30T14:41:53.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 327929 10.83 MB/sec execute 27 sec latency 14.758 ms 2023-09-30T14:41:54.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 330178 10.93 MB/sec execute 28 sec latency 17.245 ms 2023-09-30T14:41:55.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:55 smithi156 ceph-mon[103579]: pgmap v396: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 2.9 MiB/s wr, 290 op/s 2023-09-30T14:41:55.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:55 smithi148 ceph-mon[102355]: pgmap v396: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 2.9 MiB/s wr, 290 op/s 2023-09-30T14:41:55.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:55 smithi032 ceph-mon[98592]: pgmap v396: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 2.9 MiB/s wr, 290 op/s 2023-09-30T14:41:55.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 331553 10.74 MB/sec execute 29 sec latency 36.172 ms 2023-09-30T14:41:56.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 333803 10.84 MB/sec execute 30 sec latency 25.831 ms 2023-09-30T14:41:57.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:57 smithi156 ceph-mon[103579]: pgmap v397: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2023-09-30T14:41:57.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:57 smithi148 ceph-mon[102355]: pgmap v397: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2023-09-30T14:41:57.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:57 smithi032 ceph-mon[98592]: pgmap v397: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2023-09-30T14:41:57.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 335488 10.68 MB/sec execute 31 sec latency 13.870 ms 2023-09-30T14:41:58.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 337391 10.76 MB/sec execute 32 sec latency 12.096 ms 2023-09-30T14:41:59.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:41:59 smithi156 ceph-mon[103579]: pgmap v398: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 2.5 MiB/s wr, 264 op/s 2023-09-30T14:41:59.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:41:59 smithi148 ceph-mon[102355]: pgmap v398: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 2.5 MiB/s wr, 264 op/s 2023-09-30T14:41:59.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:41:59 smithi032 ceph-mon[98592]: pgmap v398: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 2.5 MiB/s wr, 264 op/s 2023-09-30T14:41:59.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 339536 10.68 MB/sec execute 33 sec latency 17.018 ms 2023-09-30T14:42:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.172+0000 7f02a427a700 1 -- 172.21.15.32:0/3635695163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f029c1018f0 msgr2=0x7f029c1081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.172+0000 7f02a427a700 1 --2- 172.21.15.32:0/3635695163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f029c1018f0 0x7f029c1081a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f02940099c0 tx=0x7f0294009cd0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.172+0000 7f02a427a700 1 -- 172.21.15.32:0/3635695163 shutdown_connections 2023-09-30T14:42:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.172+0000 7f02a427a700 1 --2- 172.21.15.32:0/3635695163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f029c1018f0 0x7f029c1081a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.172+0000 7f02a427a700 1 --2- 172.21.15.32:0/3635695163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c0ffd60 0x7f029c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.172+0000 7f02a427a700 1 --2- 172.21.15.32:0/3635695163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f029c100f50 0x7f029c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.172+0000 7f02a427a700 1 -- 172.21.15.32:0/3635695163 >> 172.21.15.32:0/3635695163 conn(0x7f029c0fb2b0 msgr2=0x7f029c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.172+0000 7f02a427a700 1 -- 172.21.15.32:0/3635695163 shutdown_connections 2023-09-30T14:42:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.172+0000 7f02a427a700 1 -- 172.21.15.32:0/3635695163 wait complete. 2023-09-30T14:42:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a427a700 1 Processor -- start 2023-09-30T14:42:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a427a700 1 -- start start 2023-09-30T14:42:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a427a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f029c0ffd60 0x7f029c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a427a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c100f50 0x7f029c19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a427a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f029c1018f0 0x7f029c19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a427a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c19e840 con 0x7f029c1018f0 2023-09-30T14:42:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a427a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c19e980 con 0x7f029c0ffd60 2023-09-30T14:42:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a427a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c19eb40 con 0x7f029c100f50 2023-09-30T14:42:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a2817700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f029c1018f0 0x7f029c19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a2817700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f029c1018f0 0x7f029c19e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36492/0 (socket says 172.21.15.32:36492) 2023-09-30T14:42:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a2817700 1 -- 172.21.15.32:0/1663704722 learned_addr learned my addr 172.21.15.32:0/1663704722 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a1815700 1 --2- 172.21.15.32:0/1663704722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c100f50 0x7f029c19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a2016700 1 --2- 172.21.15.32:0/1663704722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f029c0ffd60 0x7f029c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a2817700 1 -- 172.21.15.32:0/1663704722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c100f50 msgr2=0x7f029c19bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a2817700 1 --2- 172.21.15.32:0/1663704722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c100f50 0x7f029c19bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a2817700 1 -- 172.21.15.32:0/1663704722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f029c0ffd60 msgr2=0x7f029c1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a2817700 1 --2- 172.21.15.32:0/1663704722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f029c0ffd60 0x7f029c1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a2817700 1 -- 172.21.15.32:0/1663704722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02940095f0 con 0x7f029c1018f0 2023-09-30T14:42:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.173+0000 7f02a2817700 1 --2- 172.21.15.32:0/1663704722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f029c1018f0 0x7f029c19e270 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f02940098e0 tx=0x7f029400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.174+0000 7f02937fe700 1 -- 172.21.15.32:0/1663704722 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029401d070 con 0x7f029c1018f0 2023-09-30T14:42:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.174+0000 7f02937fe700 1 -- 172.21.15.32:0/1663704722 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0294005030 con 0x7f029c1018f0 2023-09-30T14:42:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.174+0000 7f02937fe700 1 -- 172.21.15.32:0/1663704722 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0294004a00 con 0x7f029c1018f0 2023-09-30T14:42:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.174+0000 7f02a427a700 1 -- 172.21.15.32:0/1663704722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029c19edc0 con 0x7f029c1018f0 2023-09-30T14:42:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.174+0000 7f02a427a700 1 -- 172.21.15.32:0/1663704722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029c19f190 con 0x7f029c1018f0 2023-09-30T14:42:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.176+0000 7f02937fe700 1 -- 172.21.15.32:0/1663704722 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f0294005480 con 0x7f029c1018f0 2023-09-30T14:42:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.177+0000 7f02937fe700 1 --2- 172.21.15.32:0/1663704722 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f02880a0fa0 0x7f02880a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.177+0000 7f02937fe700 1 -- 172.21.15.32:0/1663704722 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f02940cd4b0 con 0x7f029c1018f0 2023-09-30T14:42:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.177+0000 7f02a2016700 1 --2- 172.21.15.32:0/1663704722 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f02880a0fa0 0x7f02880a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.177+0000 7f02b0462b80 1 Processor -- start 2023-09-30T14:42:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.177+0000 7f02a2016700 1 --2- 172.21.15.32:0/1663704722 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f02880a0fa0 0x7f02880a3460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f029c068d20 tx=0x7f029800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02b0462b80 1 -- start start 2023-09-30T14:42:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02b0462b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 0x5616fd626a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02b0462b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd627ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02b0462b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 0x5616fd62e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02b0462b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616fd58a400 con 0x5616fd6280e0 2023-09-30T14:42:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02b0462b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616fd57bbc0 con 0x5616fd627720 2023-09-30T14:42:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02b0462b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616fd5794e0 con 0x5616fd626610 2023-09-30T14:42:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02a1815700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd627ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02a2016700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 0x5616fd626a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02a1815700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd627ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:47974/0 (socket says 172.21.15.32:47974) 2023-09-30T14:42:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02a1815700 1 -- 172.21.15.32:0/2018657002 learned_addr learned my addr 172.21.15.32:0/2018657002 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02a2817700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 0x5616fd62e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02a1815700 1 -- 172.21.15.32:0/2018657002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 msgr2=0x5616fd626a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02a1815700 1 --2- 172.21.15.32:0/2018657002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 0x5616fd626a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02a1815700 1 -- 172.21.15.32:0/2018657002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 msgr2=0x5616fd62e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02a1815700 1 --2- 172.21.15.32:0/2018657002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 0x5616fd62e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02a1815700 1 -- 172.21.15.32:0/2018657002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616fd5d7870 con 0x5616fd627720 2023-09-30T14:42:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.178+0000 7f02a1815700 1 --2- 172.21.15.32:0/2018657002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd627ba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f028c009d20 tx=0x7f028c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8de87c73fce5b2a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.179+0000 7f0290d82700 1 -- 172.21.15.32:0/2018657002 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c004a30 con 0x5616fd627720 2023-09-30T14:42:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.179+0000 7f0290d82700 1 -- 172.21.15.32:0/2018657002 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f028c0054e0 con 0x5616fd627720 2023-09-30T14:42:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.179+0000 7f0290d82700 1 -- 172.21.15.32:0/2018657002 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c019760 con 0x5616fd627720 2023-09-30T14:42:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.179+0000 7f02b0462b80 1 -- 172.21.15.32:0/2018657002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 msgr2=0x5616fd627ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.179+0000 7f02b0462b80 1 --2- 172.21.15.32:0/2018657002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd627ba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f028c009d20 tx=0x7f028c011720 comp rx=0 tx=0).stop 2023-09-30T14:42:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.179+0000 7f02b0462b80 1 -- 172.21.15.32:0/2018657002 shutdown_connections 2023-09-30T14:42:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.179+0000 7f02b0462b80 1 --2- 172.21.15.32:0/2018657002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 0x5616fd62e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.179+0000 7f02b0462b80 1 --2- 172.21.15.32:0/2018657002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 0x5616fd626a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.179+0000 7f02b0462b80 1 --2- 172.21.15.32:0/2018657002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd627ba0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.179+0000 7f02b0462b80 1 -- 172.21.15.32:0/2018657002 >> 172.21.15.32:0/2018657002 conn(0x5616fd596200 msgr2=0x5616fd61cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.180+0000 7f02b0462b80 1 -- 172.21.15.32:0/2018657002 shutdown_connections 2023-09-30T14:42:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.180+0000 7f02b0462b80 1 -- 172.21.15.32:0/2018657002 wait complete. 2023-09-30T14:42:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.180+0000 7f02b0462b80 1 -- start start 2023-09-30T14:42:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.180+0000 7f02b0462b80 1 Processor -- start 2023-09-30T14:42:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.180+0000 7f02b0462b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 0x5616fd630190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.180+0000 7f02b0462b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd6306d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.180+0000 7f02b0462b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 0x5616fd637710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.180+0000 7f02b0462b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616fd574dd0 con 0x5616fd6280e0 2023-09-30T14:42:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.180+0000 7f02b0462b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616fd5116b0 con 0x5616fd627720 2023-09-30T14:42:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.180+0000 7f02b0462b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616fd511350 con 0x5616fd626610 2023-09-30T14:42:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.180+0000 7f02b0462b80 10 client.0 _ll_register_callbacks cb 0x5616fd36e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:42:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.180+0000 7f02a2817700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 0x5616fd637710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.180+0000 7f02a2016700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 0x5616fd630190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.180+0000 7f02a2817700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 0x5616fd637710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36506/0 (socket says 172.21.15.32:36506) 2023-09-30T14:42:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.180+0000 7f02a2817700 1 -- 172.21.15.32:0/1282094854 learned_addr learned my addr 172.21.15.32:0/1282094854 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.180+0000 7f02a1815700 1 --2- 172.21.15.32:0/1282094854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd6306d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.181+0000 7f02a2817700 1 -- 172.21.15.32:0/1282094854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 msgr2=0x5616fd630190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.181+0000 7f02a2817700 1 --2- 172.21.15.32:0/1282094854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 0x5616fd630190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.181+0000 7f02a2817700 1 -- 172.21.15.32:0/1282094854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 msgr2=0x5616fd6306d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.181+0000 7f02a2817700 1 --2- 172.21.15.32:0/1282094854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd6306d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.181+0000 7f02a2817700 1 -- 172.21.15.32:0/1282094854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028c009980 con 0x5616fd6280e0 2023-09-30T14:42:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.181+0000 7f02a2016700 1 --2- 172.21.15.32:0/1282094854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 0x5616fd630190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.181+0000 7f02a1815700 1 --2- 172.21.15.32:0/1282094854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd6306d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.181+0000 7f02a2817700 1 --2- 172.21.15.32:0/1282094854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 0x5616fd637710 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f029400b580 tx=0x7f0294085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35863b58a92c2b48 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.181+0000 7f0285ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:42:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.181+0000 7f0285ffb700 1 -- 172.21.15.32:0/1282094854 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029409aa10 con 0x5616fd6280e0 2023-09-30T14:42:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.181+0000 7f02b0462b80 1 -- 172.21.15.32:0/1282094854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616fd615090 con 0x5616fd6280e0 2023-09-30T14:42:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.181+0000 7f0285ffb700 1 -- 172.21.15.32:0/1282094854 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f029409eb60 con 0x5616fd6280e0 2023-09-30T14:42:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.181+0000 7f0285ffb700 1 -- 172.21.15.32:0/1282094854 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02940b3020 con 0x5616fd6280e0 2023-09-30T14:42:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.181+0000 7f02b0462b80 10 client.15840 fetch_fsmap learned FSMap version 28 2023-09-30T14:42:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.181+0000 7f02b0462b80 1 -- 172.21.15.32:0/1282094854 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5616fd39fd80 con 0x5616fd6280e0 2023-09-30T14:42:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.181+0000 7f0285ffb700 1 -- 172.21.15.32:0/1282094854 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5616fd615090 con 0x5616fd6280e0 2023-09-30T14:42:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.182+0000 7f0285ffb700 1 -- 172.21.15.32:0/1282094854 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f02940a2880 con 0x5616fd6280e0 2023-09-30T14:42:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.182+0000 7f02b0462b80 10 client.15840 fetch_fsmap finished waiting for FSMap version 28 2023-09-30T14:42:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.182+0000 7f02b0462b80 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:42:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.182+0000 7f02b0462b80 20 client.15840 populate_metadata read hostname 'smithi032' 2023-09-30T14:42:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.182+0000 7f02b0462b80 1 --2- 172.21.15.32:0/1282094854 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5616fd63bcd0 0x5616fd63e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.182+0000 7f02b0462b80 4 client.15840 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:42:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.182+0000 7f02b0462b80 1 -- 172.21.15.32:0/1282094854 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616fd56ddb0 con 0x5616fd63bcd0 2023-09-30T14:42:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.182+0000 7f02a2016700 1 --2- 172.21.15.32:0/1282094854 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5616fd63bcd0 0x5616fd63e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.182+0000 7f02a2016700 1 --2- 172.21.15.32:0/1282094854 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5616fd63bcd0 0x5616fd63e0f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d7fd08e8eabe6a9 server_cookie=e6b869388f2a12a1 in_seq=0 out_seq=0 2023-09-30T14:42:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.182+0000 7f0285ffb700 10 client.15840 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.183+0000 7f0285ffb700 1 -- 172.21.15.32:0/1282094854 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5616fd56ddb0 con 0x5616fd63bcd0 2023-09-30T14:42:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.183+0000 7f0285ffb700 10 client.15840 handle_command_reply: tid=0 2023-09-30T14:42:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.183+0000 7f02b0462b80 1 client.15840 shutdown 2023-09-30T14:42:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.183+0000 7f02b0462b80 1 -- 172.21.15.32:0/1282094854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 msgr2=0x5616fd637710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.183+0000 7f02b0462b80 1 --2- 172.21.15.32:0/1282094854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 0x5616fd637710 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f029400b580 tx=0x7f0294085040 comp rx=0 tx=0).stop 2023-09-30T14:42:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.183+0000 7f02b0462b80 1 -- 172.21.15.32:0/1282094854 shutdown_connections 2023-09-30T14:42:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.183+0000 7f02b0462b80 1 --2- 172.21.15.32:0/1282094854 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5616fd63bcd0 0x5616fd63e0f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.183+0000 7f02b0462b80 1 --2- 172.21.15.32:0/1282094854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 0x5616fd637710 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.183+0000 7f0285ffb700 0 client.15840 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.183+0000 7f02b0462b80 1 --2- 172.21.15.32:0/1282094854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 0x5616fd630190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.183+0000 7f02b0462b80 1 --2- 172.21.15.32:0/1282094854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd6306d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.183+0000 7f02b0462b80 1 -- 172.21.15.32:0/1282094854 >> 172.21.15.32:0/1282094854 conn(0x5616fd596200 msgr2=0x5616fd61c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.183+0000 7f02b0462b80 1 -- 172.21.15.32:0/1282094854 shutdown_connections 2023-09-30T14:42:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.183+0000 7f02b0462b80 1 -- 172.21.15.32:0/1282094854 wait complete. 2023-09-30T14:42:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.184+0000 7f02b0462b80 20 client.15840 trim_cache size 0 max 16384 2023-09-30T14:42:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02b0462b80 1 Processor -- start 2023-09-30T14:42:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02b0462b80 1 -- start start 2023-09-30T14:42:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02b0462b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 0x5616fd627b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02b0462b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd6376d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02b0462b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 0x5616fd62e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02b0462b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616fd3f58c0 con 0x5616fd6280e0 2023-09-30T14:42:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02b0462b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616fd61dbf0 con 0x5616fd627720 2023-09-30T14:42:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02b0462b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616fd614bd0 con 0x5616fd626610 2023-09-30T14:42:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02a2817700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 0x5616fd62e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02a2817700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 0x5616fd62e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36514/0 (socket says 172.21.15.32:36514) 2023-09-30T14:42:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02a2016700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 0x5616fd627b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02a2817700 1 -- 172.21.15.32:0/1423329782 learned_addr learned my addr 172.21.15.32:0/1423329782 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02a1815700 1 --2- 172.21.15.32:0/1423329782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd6376d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02a2817700 1 -- 172.21.15.32:0/1423329782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 msgr2=0x5616fd627b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02a2817700 1 --2- 172.21.15.32:0/1423329782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 0x5616fd627b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02a2817700 1 -- 172.21.15.32:0/1423329782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 msgr2=0x5616fd6376d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02a2817700 1 --2- 172.21.15.32:0/1423329782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd6376d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02a2817700 1 -- 172.21.15.32:0/1423329782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616fd388c00 con 0x5616fd6280e0 2023-09-30T14:42:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.199+0000 7f02a2016700 1 --2- 172.21.15.32:0/1423329782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 0x5616fd627b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.200+0000 7f02a2817700 1 --2- 172.21.15.32:0/1423329782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 0x5616fd62e450 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0294097480 tx=0x7f0294098fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9939cfa9dffc6ddb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.200+0000 7f0291583700 1 -- 172.21.15.32:0/1423329782 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029409eb60 con 0x5616fd6280e0 2023-09-30T14:42:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.200+0000 7f0291583700 1 -- 172.21.15.32:0/1423329782 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f029409aa10 con 0x5616fd6280e0 2023-09-30T14:42:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.200+0000 7f0291583700 1 -- 172.21.15.32:0/1423329782 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02940ba1d0 con 0x5616fd6280e0 2023-09-30T14:42:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.200+0000 7f02b0462b80 1 -- 172.21.15.32:0/1423329782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 msgr2=0x5616fd62e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.200+0000 7f02b0462b80 1 --2- 172.21.15.32:0/1423329782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 0x5616fd62e450 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0294097480 tx=0x7f0294098fc0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.200+0000 7f02b0462b80 1 -- 172.21.15.32:0/1423329782 shutdown_connections 2023-09-30T14:42:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.200+0000 7f02b0462b80 1 --2- 172.21.15.32:0/1423329782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd6280e0 0x5616fd62e450 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.200+0000 7f02b0462b80 1 --2- 172.21.15.32:0/1423329782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd626610 0x5616fd627b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.200+0000 7f02b0462b80 1 --2- 172.21.15.32:0/1423329782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd6376d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.200+0000 7f02b0462b80 1 -- 172.21.15.32:0/1423329782 >> 172.21.15.32:0/1423329782 conn(0x5616fd596200 msgr2=0x5616fd62efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.200+0000 7f02b0462b80 1 -- 172.21.15.32:0/1423329782 shutdown_connections 2023-09-30T14:42:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.200+0000 7f02b0462b80 1 -- 172.21.15.32:0/1423329782 wait complete. 2023-09-30T14:42:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02b0462b80 1 -- start start 2023-09-30T14:42:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02b0462b80 1 Processor -- start 2023-09-30T14:42:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02b0462b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd626610 0x5616fd634e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02b0462b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd635370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02b0462b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd6280e0 0x5616fd636380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02b0462b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616fd6368c0 con 0x5616fd626610 2023-09-30T14:42:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02b0462b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616fd636a30 con 0x5616fd627720 2023-09-30T14:42:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02b0462b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616fd636bf0 con 0x5616fd6280e0 2023-09-30T14:42:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02b0462b80 10 client.0 _ll_register_callbacks cb 0x7f029409a370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:42:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02a2016700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd626610 0x5616fd634e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02a2016700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd626610 0x5616fd634e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36516/0 (socket says 172.21.15.32:36516) 2023-09-30T14:42:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02a1815700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd635370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02a2817700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd6280e0 0x5616fd636380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02a2016700 1 -- 172.21.15.32:0/1191708883 learned_addr learned my addr 172.21.15.32:0/1191708883 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02a2817700 1 -- 172.21.15.32:0/1191708883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 msgr2=0x5616fd635370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02a2817700 1 --2- 172.21.15.32:0/1191708883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd635370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02a2817700 1 -- 172.21.15.32:0/1191708883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd626610 msgr2=0x5616fd634e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02a2817700 1 --2- 172.21.15.32:0/1191708883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd626610 0x5616fd634e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.201+0000 7f02a2817700 1 -- 172.21.15.32:0/1191708883 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0298003f40 con 0x5616fd6280e0 2023-09-30T14:42:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.202+0000 7f02a2817700 1 --2- 172.21.15.32:0/1191708883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd6280e0 0x5616fd636380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0294097430 tx=0x7f02940a2a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29a1d8945a9d466e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.202+0000 7f0285ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:42:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.202+0000 7f0285ffb700 1 -- 172.21.15.32:0/1191708883 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02940ba1b0 con 0x5616fd6280e0 2023-09-30T14:42:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.202+0000 7f0285ffb700 1 -- 172.21.15.32:0/1191708883 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f029409eb60 con 0x5616fd6280e0 2023-09-30T14:42:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.202+0000 7f02b0462b80 1 -- 172.21.15.32:0/1191708883 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f028c0054e0 con 0x5616fd6280e0 2023-09-30T14:42:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.202+0000 7f0285ffb700 1 -- 172.21.15.32:0/1191708883 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02940661f0 con 0x5616fd6280e0 2023-09-30T14:42:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.202+0000 7f02b0462b80 10 client.25528 fetch_fsmap learned FSMap version 28 2023-09-30T14:42:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.202+0000 7f02b0462b80 1 -- 172.21.15.32:0/1191708883 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5616fd39fd80 con 0x5616fd6280e0 2023-09-30T14:42:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.202+0000 7f0285ffb700 1 -- 172.21.15.32:0/1191708883 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f028c0054e0 con 0x5616fd6280e0 2023-09-30T14:42:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.202+0000 7f0285ffb700 1 -- 172.21.15.32:0/1191708883 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 28) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0294096890 con 0x5616fd6280e0 2023-09-30T14:42:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.203+0000 7f02b0462b80 10 client.25528 fetch_fsmap finished waiting for FSMap version 28 2023-09-30T14:42:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.203+0000 7f02b0462b80 10 client.25528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:42:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.203+0000 7f02b0462b80 20 client.25528 populate_metadata read hostname 'smithi032' 2023-09-30T14:42:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.203+0000 7f02b0462b80 1 --2- 172.21.15.32:0/1191708883 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5616fd68fe20 0x5616fd692240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.203+0000 7f02b0462b80 4 client.25528 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:42:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.203+0000 7f02b0462b80 1 -- 172.21.15.32:0/1191708883 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5616fd56dad0 con 0x5616fd68fe20 2023-09-30T14:42:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.203+0000 7f02a2016700 1 --2- 172.21.15.32:0/1191708883 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5616fd68fe20 0x5616fd692240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.203+0000 7f0285ffb700 10 client.25528 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.203+0000 7f02a2016700 1 --2- 172.21.15.32:0/1191708883 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5616fd68fe20 0x5616fd692240 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bdcbfa0a3df362f server_cookie=94aa58bfa4aa09e4 in_seq=0 out_seq=0 2023-09-30T14:42:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.241+0000 7f0285ffb700 1 -- 172.21.15.32:0/1191708883 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5616fd56dad0 con 0x5616fd68fe20 2023-09-30T14:42:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.241+0000 7f0285ffb700 10 client.25528 handle_command_reply: tid=0 2023-09-30T14:42:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.241+0000 7f02b0462b80 1 client.25528 shutdown 2023-09-30T14:42:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02b0462b80 1 -- 172.21.15.32:0/1191708883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd6280e0 msgr2=0x5616fd636380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02b0462b80 1 --2- 172.21.15.32:0/1191708883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd6280e0 0x5616fd636380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0294097430 tx=0x7f02940a2a90 comp rx=0 tx=0).stop 2023-09-30T14:42:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02b0462b80 1 -- 172.21.15.32:0/1191708883 shutdown_connections 2023-09-30T14:42:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02b0462b80 1 --2- 172.21.15.32:0/1191708883 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5616fd68fe20 0x5616fd692240 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02b0462b80 1 --2- 172.21.15.32:0/1191708883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616fd6280e0 0x5616fd636380 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f0285ffb700 0 client.25528 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02b0462b80 1 --2- 172.21.15.32:0/1191708883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616fd626610 0x5616fd634e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02b0462b80 1 --2- 172.21.15.32:0/1191708883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616fd627720 0x5616fd635370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02b0462b80 1 -- 172.21.15.32:0/1191708883 >> 172.21.15.32:0/1191708883 conn(0x5616fd596200 msgr2=0x5616fd62e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02b0462b80 1 -- 172.21.15.32:0/1191708883 shutdown_connections 2023-09-30T14:42:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02b0462b80 1 -- 172.21.15.32:0/1191708883 wait complete. 2023-09-30T14:42:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02b0462b80 20 client.25528 trim_cache size 0 max 16384 2023-09-30T14:42:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:42:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:42:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:42:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:42:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02a427a700 1 -- 172.21.15.32:0/1663704722 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f02880a0fa0 msgr2=0x7f02880a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02a427a700 1 --2- 172.21.15.32:0/1663704722 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f02880a0fa0 0x7f02880a3460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f029c068d20 tx=0x7f029800b040 comp rx=0 tx=0).stop 2023-09-30T14:42:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02a427a700 1 -- 172.21.15.32:0/1663704722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f029c1018f0 msgr2=0x7f029c19e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02a427a700 1 --2- 172.21.15.32:0/1663704722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f029c1018f0 0x7f029c19e270 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f02940098e0 tx=0x7f029400f690 comp rx=0 tx=0).stop 2023-09-30T14:42:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02a427a700 1 -- 172.21.15.32:0/1663704722 shutdown_connections 2023-09-30T14:42:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02a427a700 1 --2- 172.21.15.32:0/1663704722 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f02880a0fa0 0x7f02880a3460 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02a427a700 1 --2- 172.21.15.32:0/1663704722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f029c1018f0 0x7f029c19e270 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02a427a700 1 --2- 172.21.15.32:0/1663704722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f029c0ffd60 0x7f029c1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02a427a700 1 --2- 172.21.15.32:0/1663704722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029c100f50 0x7f029c19bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.242+0000 7f02a427a700 1 -- 172.21.15.32:0/1663704722 >> 172.21.15.32:0/1663704722 conn(0x7f029c0fb2b0 msgr2=0x7f029c104400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.243+0000 7f02a427a700 1 -- 172.21.15.32:0/1663704722 shutdown_connections 2023-09-30T14:42:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:00.243+0000 7f02a427a700 1 -- 172.21.15.32:0/1663704722 wait complete. 2023-09-30T14:42:00.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 341023 10.69 MB/sec execute 34 sec latency 37.174 ms 2023-09-30T14:42:01.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:01 smithi156 ceph-mon[103579]: pgmap v399: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 2.6 MiB/s wr, 274 op/s 2023-09-30T14:42:01.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:01 smithi148 ceph-mon[102355]: pgmap v399: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 2.6 MiB/s wr, 274 op/s 2023-09-30T14:42:01.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:01 smithi032 ceph-mon[98592]: pgmap v399: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 2.6 MiB/s wr, 274 op/s 2023-09-30T14:42:01.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 343858 10.80 MB/sec execute 35 sec latency 18.103 ms 2023-09-30T14:42:02.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 345452 10.77 MB/sec execute 36 sec latency 13.326 ms 2023-09-30T14:42:03.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:03 smithi148 ceph-mon[102355]: pgmap v400: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 2.7 MiB/s wr, 274 op/s 2023-09-30T14:42:03.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:03 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:42:03.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:03 smithi032 ceph-mon[98592]: pgmap v400: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 2.7 MiB/s wr, 274 op/s 2023-09-30T14:42:03.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:03 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:42:03.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 347877 10.85 MB/sec execute 37 sec latency 12.670 ms 2023-09-30T14:42:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:03 smithi156 ceph-mon[103579]: pgmap v400: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 2.7 MiB/s wr, 274 op/s 2023-09-30T14:42:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:03 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:42:04.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 349302 10.71 MB/sec execute 38 sec latency 14.323 ms 2023-09-30T14:42:05.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:05 smithi148 ceph-mon[102355]: pgmap v401: 97 pgs: 97 active+clean; 131 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.5 MiB/s wr, 274 op/s 2023-09-30T14:42:05.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:05 smithi032 ceph-mon[98592]: pgmap v401: 97 pgs: 97 active+clean; 131 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.5 MiB/s wr, 274 op/s 2023-09-30T14:42:05.696 INFO:tasks.workunit.client.0.smithi032.stdout: 1 351557 10.79 MB/sec execute 39 sec latency 38.831 ms 2023-09-30T14:42:05.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:05 smithi156 ceph-mon[103579]: pgmap v401: 97 pgs: 97 active+clean; 131 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.5 MiB/s wr, 274 op/s 2023-09-30T14:42:06.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 353455 10.69 MB/sec execute 40 sec latency 92.951 ms 2023-09-30T14:42:07.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:07 smithi148 ceph-mon[102355]: pgmap v402: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 2.8 MiB/s wr, 288 op/s 2023-09-30T14:42:07.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:07 smithi032 ceph-mon[98592]: pgmap v402: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 2.8 MiB/s wr, 288 op/s 2023-09-30T14:42:07.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 355411 10.74 MB/sec execute 41 sec latency 13.002 ms 2023-09-30T14:42:07.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:07 smithi156 ceph-mon[103579]: pgmap v402: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 2.8 MiB/s wr, 288 op/s 2023-09-30T14:42:08.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 358210 10.89 MB/sec execute 42 sec latency 12.689 ms 2023-09-30T14:42:09.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:09 smithi148 ceph-mon[102355]: pgmap v403: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 2.6 MiB/s wr, 274 op/s 2023-09-30T14:42:09.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:09 smithi032 ceph-mon[98592]: pgmap v403: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 2.6 MiB/s wr, 274 op/s 2023-09-30T14:42:09.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 359702 10.80 MB/sec execute 43 sec latency 13.187 ms 2023-09-30T14:42:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:09 smithi156 ceph-mon[103579]: pgmap v403: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 2.6 MiB/s wr, 274 op/s 2023-09-30T14:42:10.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 362137 10.87 MB/sec execute 44 sec latency 32.417 ms 2023-09-30T14:42:11.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:11 smithi148 ceph-mon[102355]: pgmap v404: 97 pgs: 97 active+clean; 136 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 3.0 MiB/s wr, 307 op/s 2023-09-30T14:42:11.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:11 smithi032 ceph-mon[98592]: pgmap v404: 97 pgs: 97 active+clean; 136 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 3.0 MiB/s wr, 307 op/s 2023-09-30T14:42:11.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 363683 10.75 MB/sec execute 45 sec latency 77.989 ms 2023-09-30T14:42:11.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:11 smithi156 ceph-mon[103579]: pgmap v404: 97 pgs: 97 active+clean; 136 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 3.0 MiB/s wr, 307 op/s 2023-09-30T14:42:12.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 365873 10.81 MB/sec execute 46 sec latency 16.082 ms 2023-09-30T14:42:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.363+0000 7f071fb70700 1 -- 172.21.15.32:0/4048068030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0718107f40 msgr2=0x7f0718108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.363+0000 7f071fb70700 1 --2- 172.21.15.32:0/4048068030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0718107f40 0x7f0718108360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0714009990 tx=0x7f0714009ca0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.363+0000 7f071fb70700 1 -- 172.21.15.32:0/4048068030 shutdown_connections 2023-09-30T14:42:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.363+0000 7f071fb70700 1 --2- 172.21.15.32:0/4048068030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0718109260 0x7f07181096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.363+0000 7f071fb70700 1 --2- 172.21.15.32:0/4048068030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0718107f40 0x7f0718108360 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.363+0000 7f071fb70700 1 --2- 172.21.15.32:0/4048068030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07181088a0 0x7f0718108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.363+0000 7f071fb70700 1 -- 172.21.15.32:0/4048068030 >> 172.21.15.32:0/4048068030 conn(0x7f071806ecd0 msgr2=0x7f0718071130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.363+0000 7f071fb70700 1 -- 172.21.15.32:0/4048068030 shutdown_connections 2023-09-30T14:42:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.363+0000 7f071fb70700 1 -- 172.21.15.32:0/4048068030 wait complete. 2023-09-30T14:42:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.364+0000 7f071fb70700 1 Processor -- start 2023-09-30T14:42:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.364+0000 7f071fb70700 1 -- start start 2023-09-30T14:42:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.364+0000 7f071fb70700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0718107f40 0x7f07181a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.364+0000 7f071fb70700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07181088a0 0x7f07181a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.364+0000 7f071fb70700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0718109260 0x7f07181a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.364+0000 7f071fb70700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07181a2f30 con 0x7f0718107f40 2023-09-30T14:42:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.364+0000 7f071fb70700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07181a30a0 con 0x7f07181088a0 2023-09-30T14:42:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.364+0000 7f071fb70700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07181a3260 con 0x7f0718109260 2023-09-30T14:42:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.364+0000 7f071d10b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07181088a0 0x7f07181a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.364+0000 7f071d90c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0718107f40 0x7f07181a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.364+0000 7f071d10b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07181088a0 0x7f07181a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48008/0 (socket says 172.21.15.32:48008) 2023-09-30T14:42:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.364+0000 7f071d10b700 1 -- 172.21.15.32:0/3739807552 learned_addr learned my addr 172.21.15.32:0/3739807552 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.364+0000 7f071e10d700 1 --2- 172.21.15.32:0/3739807552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0718109260 0x7f07181a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.365+0000 7f071d90c700 1 -- 172.21.15.32:0/3739807552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0718109260 msgr2=0x7f07181a28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.365+0000 7f071d90c700 1 --2- 172.21.15.32:0/3739807552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0718109260 0x7f07181a28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.365+0000 7f071d90c700 1 -- 172.21.15.32:0/3739807552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07181088a0 msgr2=0x7f07181a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.365+0000 7f071d90c700 1 --2- 172.21.15.32:0/3739807552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07181088a0 0x7f07181a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.365+0000 7f071d90c700 1 -- 172.21.15.32:0/3739807552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07140095f0 con 0x7f0718107f40 2023-09-30T14:42:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.365+0000 7f071d90c700 1 --2- 172.21.15.32:0/3739807552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0718107f40 0x7f07181a1510 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f07140059d0 tx=0x7f0714004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.365+0000 7f070effd700 1 -- 172.21.15.32:0/3739807552 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071401d070 con 0x7f0718107f40 2023-09-30T14:42:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.365+0000 7f070effd700 1 -- 172.21.15.32:0/3739807552 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0714022470 con 0x7f0718107f40 2023-09-30T14:42:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.365+0000 7f071fb70700 1 -- 172.21.15.32:0/3739807552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07181a34e0 con 0x7f0718107f40 2023-09-30T14:42:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.365+0000 7f070effd700 1 -- 172.21.15.32:0/3739807552 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714017650 con 0x7f0718107f40 2023-09-30T14:42:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.365+0000 7f071fb70700 1 -- 172.21.15.32:0/3739807552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07181a9d00 con 0x7f0718107f40 2023-09-30T14:42:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.367+0000 7f070effd700 1 -- 172.21.15.32:0/3739807552 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f07140225e0 con 0x7f0718107f40 2023-09-30T14:42:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.368+0000 7f070effd700 1 --2- 172.21.15.32:0/3739807552 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f07040a0fa0 0x7f07040a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.368+0000 7f071d10b700 1 --2- 172.21.15.32:0/3739807552 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f07040a0fa0 0x7f07040a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.368+0000 7f070effd700 1 -- 172.21.15.32:0/3739807552 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f07140cd940 con 0x7f0718107f40 2023-09-30T14:42:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.369+0000 7f071d10b700 1 --2- 172.21.15.32:0/3739807552 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f07040a0fa0 0x7f07040a3460 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0708009d20 tx=0x7f0708009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.369+0000 7f072bd58b80 1 Processor -- start 2023-09-30T14:42:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.369+0000 7f072bd58b80 1 -- start start 2023-09-30T14:42:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.369+0000 7f072bd58b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733c200 0x56202733c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.369+0000 7f072bd58b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733d310 0x56202733d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.369+0000 7f072bd58b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733dcd0 0x562027344560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f072bd58b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620272a4400 con 0x56202733c200 2023-09-30T14:42:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f072bd58b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562027295bc0 con 0x56202733dcd0 2023-09-30T14:42:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f072bd58b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620272934e0 con 0x56202733d310 2023-09-30T14:42:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f071d90c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733c200 0x56202733c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f071d90c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733c200 0x56202733c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36538/0 (socket says 172.21.15.32:36538) 2023-09-30T14:42:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f071d90c700 1 -- 172.21.15.32:0/4126996881 learned_addr learned my addr 172.21.15.32:0/4126996881 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f071e10d700 1 --2- 172.21.15.32:0/4126996881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733dcd0 0x562027344560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f071d10b700 1 --2- 172.21.15.32:0/4126996881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733d310 0x56202733d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f071d90c700 1 -- 172.21.15.32:0/4126996881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733d310 msgr2=0x56202733d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f071d90c700 1 --2- 172.21.15.32:0/4126996881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733d310 0x56202733d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f071d90c700 1 -- 172.21.15.32:0/4126996881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733dcd0 msgr2=0x562027344560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f071d90c700 1 --2- 172.21.15.32:0/4126996881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733dcd0 0x562027344560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f071d90c700 1 -- 172.21.15.32:0/4126996881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620272f1870 con 0x56202733c200 2023-09-30T14:42:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f071d90c700 1 --2- 172.21.15.32:0/4126996881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733c200 0x56202733c620 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f071409aba0 tx=0x7f071409aad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9785ce96c567639e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f0703fff700 1 -- 172.21.15.32:0/4126996881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07140a10f0 con 0x56202733c200 2023-09-30T14:42:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f0703fff700 1 -- 172.21.15.32:0/4126996881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f071409aea0 con 0x56202733c200 2023-09-30T14:42:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.370+0000 7f0703fff700 1 -- 172.21.15.32:0/4126996881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071409b320 con 0x56202733c200 2023-09-30T14:42:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.371+0000 7f072bd58b80 1 -- 172.21.15.32:0/4126996881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733c200 msgr2=0x56202733c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.371+0000 7f072bd58b80 1 --2- 172.21.15.32:0/4126996881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733c200 0x56202733c620 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f071409aba0 tx=0x7f071409aad0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.371+0000 7f072bd58b80 1 -- 172.21.15.32:0/4126996881 shutdown_connections 2023-09-30T14:42:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.371+0000 7f072bd58b80 1 --2- 172.21.15.32:0/4126996881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733dcd0 0x562027344560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.371+0000 7f072bd58b80 1 --2- 172.21.15.32:0/4126996881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733c200 0x56202733c620 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.371+0000 7f072bd58b80 1 --2- 172.21.15.32:0/4126996881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733d310 0x56202733d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.371+0000 7f072bd58b80 1 -- 172.21.15.32:0/4126996881 >> 172.21.15.32:0/4126996881 conn(0x5620272b0200 msgr2=0x562027336ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.371+0000 7f072bd58b80 1 -- 172.21.15.32:0/4126996881 shutdown_connections 2023-09-30T14:42:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.371+0000 7f072bd58b80 1 -- 172.21.15.32:0/4126996881 wait complete. 2023-09-30T14:42:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.372+0000 7f072bd58b80 1 -- start start 2023-09-30T14:42:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.372+0000 7f072bd58b80 1 Processor -- start 2023-09-30T14:42:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.372+0000 7f072bd58b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733c200 0x562027345ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.372+0000 7f072bd58b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733d310 0x5620273460e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.372+0000 7f072bd58b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733dcd0 0x56202734d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.372+0000 7f072bd58b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202728edd0 con 0x56202733c200 2023-09-30T14:42:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.372+0000 7f072bd58b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202722b6b0 con 0x56202733dcd0 2023-09-30T14:42:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.372+0000 7f072bd58b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202722b350 con 0x56202733d310 2023-09-30T14:42:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.372+0000 7f072bd58b80 10 client.0 _ll_register_callbacks cb 0x5620270887a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:42:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.372+0000 7f071d90c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733c200 0x562027345ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.372+0000 7f071d90c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733c200 0x562027345ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36544/0 (socket says 172.21.15.32:36544) 2023-09-30T14:42:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.372+0000 7f071d90c700 1 -- 172.21.15.32:0/1404977563 learned_addr learned my addr 172.21.15.32:0/1404977563 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.372+0000 7f071d10b700 1 --2- 172.21.15.32:0/1404977563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733d310 0x5620273460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.372+0000 7f071e10d700 1 --2- 172.21.15.32:0/1404977563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733dcd0 0x56202734d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.372+0000 7f071d90c700 1 -- 172.21.15.32:0/1404977563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733d310 msgr2=0x5620273460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f071d90c700 1 --2- 172.21.15.32:0/1404977563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733d310 0x5620273460e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f071d90c700 1 -- 172.21.15.32:0/1404977563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733dcd0 msgr2=0x56202734d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f071d90c700 1 --2- 172.21.15.32:0/1404977563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733dcd0 0x56202734d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f071d90c700 1 -- 172.21.15.32:0/1404977563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07181a9d00 con 0x56202733c200 2023-09-30T14:42:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f071d10b700 1 --2- 172.21.15.32:0/1404977563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733d310 0x5620273460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f071e10d700 1 --2- 172.21.15.32:0/1404977563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733dcd0 0x56202734d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f071d90c700 1 --2- 172.21.15.32:0/1404977563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733c200 0x562027345ba0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0714006010 tx=0x7f071409b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2917f3d222b19e27 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f07017fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:42:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f07017fa700 1 -- 172.21.15.32:0/1404977563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714099320 con 0x56202733c200 2023-09-30T14:42:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f07017fa700 1 -- 172.21.15.32:0/1404977563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f07140a31a0 con 0x56202733c200 2023-09-30T14:42:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f072bd58b80 1 -- 172.21.15.32:0/1404977563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56202732f090 con 0x56202733c200 2023-09-30T14:42:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f07017fa700 1 -- 172.21.15.32:0/1404977563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07140a10c0 con 0x56202733c200 2023-09-30T14:42:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f072bd58b80 10 client.15858 fetch_fsmap learned FSMap version 29 2023-09-30T14:42:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f072bd58b80 1 -- 172.21.15.32:0/1404977563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5620270b9d80 con 0x56202733c200 2023-09-30T14:42:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f07017fa700 1 -- 172.21.15.32:0/1404977563 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f071407d080 con 0x56202733c200 2023-09-30T14:42:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f07017fa700 1 -- 172.21.15.32:0/1404977563 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0714096e10 con 0x56202733c200 2023-09-30T14:42:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f072bd58b80 10 client.15858 fetch_fsmap finished waiting for FSMap version 29 2023-09-30T14:42:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f072bd58b80 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:42:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f072bd58b80 20 client.15858 populate_metadata read hostname 'smithi032' 2023-09-30T14:42:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f072bd58b80 1 --2- 172.21.15.32:0/1404977563 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562027351510 0x562027353930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.373+0000 7f072bd58b80 4 client.15858 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:42:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.374+0000 7f072bd58b80 1 -- 172.21.15.32:0/1404977563 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562027287db0 con 0x562027351510 2023-09-30T14:42:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.374+0000 7f071d10b700 1 --2- 172.21.15.32:0/1404977563 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562027351510 0x562027353930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.374+0000 7f07017fa700 10 client.15858 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.374+0000 7f071d10b700 1 --2- 172.21.15.32:0/1404977563 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562027351510 0x562027353930 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d80ff47c9ed6bcb server_cookie=3bc25828394af5ce in_seq=0 out_seq=0 2023-09-30T14:42:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.375+0000 7f07017fa700 1 -- 172.21.15.32:0/1404977563 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562027287db0 con 0x562027351510 2023-09-30T14:42:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.375+0000 7f07017fa700 10 client.15858 handle_command_reply: tid=0 2023-09-30T14:42:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.375+0000 7f072bd58b80 1 client.15858 shutdown 2023-09-30T14:42:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.375+0000 7f072bd58b80 1 -- 172.21.15.32:0/1404977563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733c200 msgr2=0x562027345ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.375+0000 7f072bd58b80 1 --2- 172.21.15.32:0/1404977563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733c200 0x562027345ba0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0714006010 tx=0x7f071409b330 comp rx=0 tx=0).stop 2023-09-30T14:42:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.375+0000 7f072bd58b80 1 -- 172.21.15.32:0/1404977563 shutdown_connections 2023-09-30T14:42:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.375+0000 7f072bd58b80 1 --2- 172.21.15.32:0/1404977563 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562027351510 0x562027353930 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.375+0000 7f072bd58b80 1 --2- 172.21.15.32:0/1404977563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733dcd0 0x56202734d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.375+0000 7f072bd58b80 1 --2- 172.21.15.32:0/1404977563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733c200 0x562027345ba0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.375+0000 7f072bd58b80 1 --2- 172.21.15.32:0/1404977563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733d310 0x5620273460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.375+0000 7f07017fa700 0 client.15858 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.375+0000 7f072bd58b80 1 -- 172.21.15.32:0/1404977563 >> 172.21.15.32:0/1404977563 conn(0x5620272b0200 msgr2=0x562027336590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.375+0000 7f072bd58b80 1 -- 172.21.15.32:0/1404977563 shutdown_connections 2023-09-30T14:42:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.375+0000 7f072bd58b80 1 -- 172.21.15.32:0/1404977563 wait complete. 2023-09-30T14:42:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.375+0000 7f072bd58b80 20 client.15858 trim_cache size 0 max 16384 2023-09-30T14:42:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.391+0000 7f072bd58b80 1 Processor -- start 2023-09-30T14:42:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.391+0000 7f072bd58b80 1 -- start start 2023-09-30T14:42:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.391+0000 7f072bd58b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733c200 0x562027345950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.391+0000 7f072bd58b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733d310 0x56202734d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.391+0000 7f072bd58b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733dcd0 0x562027345e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.391+0000 7f072bd58b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202710f8c0 con 0x56202733dcd0 2023-09-30T14:42:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.391+0000 7f072bd58b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562027333ba0 con 0x56202733d310 2023-09-30T14:42:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.391+0000 7f072bd58b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562027346d20 con 0x56202733c200 2023-09-30T14:42:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.391+0000 7f071e10d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733dcd0 0x562027345e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.391+0000 7f071e10d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733dcd0 0x562027345e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36556/0 (socket says 172.21.15.32:36556) 2023-09-30T14:42:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.391+0000 7f071d10b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733d310 0x56202734d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.391+0000 7f071e10d700 1 -- 172.21.15.32:0/1361022882 learned_addr learned my addr 172.21.15.32:0/1361022882 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.391+0000 7f071d90c700 1 --2- 172.21.15.32:0/1361022882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733c200 0x562027345950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.392+0000 7f071e10d700 1 -- 172.21.15.32:0/1361022882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733c200 msgr2=0x562027345950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.392+0000 7f071e10d700 1 --2- 172.21.15.32:0/1361022882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733c200 0x562027345950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.392+0000 7f071e10d700 1 -- 172.21.15.32:0/1361022882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733d310 msgr2=0x56202734d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.392+0000 7f071e10d700 1 --2- 172.21.15.32:0/1361022882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733d310 0x56202734d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.392+0000 7f071e10d700 1 -- 172.21.15.32:0/1361022882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620270a2c00 con 0x56202733dcd0 2023-09-30T14:42:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.392+0000 7f071d10b700 1 --2- 172.21.15.32:0/1361022882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733d310 0x56202734d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.392+0000 7f071e10d700 1 --2- 172.21.15.32:0/1361022882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733dcd0 0x562027345e90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0710011b90 tx=0x7f0710009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b40c54f85b32246 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.392+0000 7f070cd82700 1 -- 172.21.15.32:0/1361022882 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071000e040 con 0x56202733dcd0 2023-09-30T14:42:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.392+0000 7f070cd82700 1 -- 172.21.15.32:0/1361022882 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f071000f040 con 0x56202733dcd0 2023-09-30T14:42:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.392+0000 7f070cd82700 1 -- 172.21.15.32:0/1361022882 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0710007d90 con 0x56202733dcd0 2023-09-30T14:42:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 -- 172.21.15.32:0/1361022882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733dcd0 msgr2=0x562027345e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 --2- 172.21.15.32:0/1361022882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733dcd0 0x562027345e90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0710011b90 tx=0x7f0710009920 comp rx=0 tx=0).stop 2023-09-30T14:42:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 -- 172.21.15.32:0/1361022882 shutdown_connections 2023-09-30T14:42:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 --2- 172.21.15.32:0/1361022882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733dcd0 0x562027345e90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 --2- 172.21.15.32:0/1361022882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733c200 0x562027345950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 --2- 172.21.15.32:0/1361022882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733d310 0x56202734d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 -- 172.21.15.32:0/1361022882 >> 172.21.15.32:0/1361022882 conn(0x5620272b0200 msgr2=0x562027340a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 -- 172.21.15.32:0/1361022882 shutdown_connections 2023-09-30T14:42:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 -- 172.21.15.32:0/1361022882 wait complete. 2023-09-30T14:42:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 -- start start 2023-09-30T14:42:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 Processor -- start 2023-09-30T14:42:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733c200 0x56202734af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733d310 0x56202734b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733dcd0 0x56202734c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202734c9a0 con 0x56202733d310 2023-09-30T14:42:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202734cb10 con 0x56202733c200 2023-09-30T14:42:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.393+0000 7f072bd58b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202734ccd0 con 0x56202733dcd0 2023-09-30T14:42:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f072bd58b80 10 client.0 _ll_register_callbacks cb 0x7f0710009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:42:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f071d10b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733d310 0x56202734b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f071d90c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733c200 0x56202734af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f071e10d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733dcd0 0x56202734c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f071d10b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733d310 0x56202734b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36560/0 (socket says 172.21.15.32:36560) 2023-09-30T14:42:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f071d10b700 1 -- 172.21.15.32:0/1481652433 learned_addr learned my addr 172.21.15.32:0/1481652433 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f071d90c700 1 -- 172.21.15.32:0/1481652433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733dcd0 msgr2=0x56202734c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f071d90c700 1 --2- 172.21.15.32:0/1481652433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733dcd0 0x56202734c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f071d90c700 1 -- 172.21.15.32:0/1481652433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733d310 msgr2=0x56202734b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f071d90c700 1 --2- 172.21.15.32:0/1481652433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733d310 0x56202734b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f071d90c700 1 -- 172.21.15.32:0/1481652433 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0708017070 con 0x56202733c200 2023-09-30T14:42:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f071e10d700 1 --2- 172.21.15.32:0/1481652433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733dcd0 0x56202734c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f071d10b700 1 --2- 172.21.15.32:0/1481652433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733d310 0x56202734b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f071d90c700 1 --2- 172.21.15.32:0/1481652433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733c200 0x56202734af10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0714009990 tx=0x7f071409a800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5e279b397c163db server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f07017fa700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:42:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f07017fa700 1 -- 172.21.15.32:0/1481652433 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714095da0 con 0x56202733c200 2023-09-30T14:42:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f07017fa700 1 -- 172.21.15.32:0/1481652433 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f07140a31a0 con 0x56202733c200 2023-09-30T14:42:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.394+0000 7f072bd58b80 1 -- 172.21.15.32:0/1481652433 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f071409aea0 con 0x56202733c200 2023-09-30T14:42:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.395+0000 7f07017fa700 1 -- 172.21.15.32:0/1481652433 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0714095c20 con 0x56202733c200 2023-09-30T14:42:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.395+0000 7f072bd58b80 10 client.25556 fetch_fsmap learned FSMap version 29 2023-09-30T14:42:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.395+0000 7f072bd58b80 1 -- 172.21.15.32:0/1481652433 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f0710011840 con 0x56202733c200 2023-09-30T14:42:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.395+0000 7f07017fa700 1 -- 172.21.15.32:0/1481652433 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f071409aea0 con 0x56202733c200 2023-09-30T14:42:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.395+0000 7f07017fa700 1 -- 172.21.15.32:0/1481652433 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0714099e50 con 0x56202733c200 2023-09-30T14:42:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.395+0000 7f072bd58b80 10 client.25556 fetch_fsmap finished waiting for FSMap version 29 2023-09-30T14:42:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.395+0000 7f072bd58b80 10 client.25556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:42:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.395+0000 7f072bd58b80 20 client.25556 populate_metadata read hostname 'smithi032' 2023-09-30T14:42:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.395+0000 7f072bd58b80 1 --2- 172.21.15.32:0/1481652433 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5620273a5e40 0x5620273a8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.395+0000 7f072bd58b80 4 client.25556 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:42:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.395+0000 7f072bd58b80 1 -- 172.21.15.32:0/1481652433 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562027287ad0 con 0x5620273a5e40 2023-09-30T14:42:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.395+0000 7f071d10b700 1 --2- 172.21.15.32:0/1481652433 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5620273a5e40 0x5620273a8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.396+0000 7f07017fa700 10 client.25556 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.396+0000 7f071d10b700 1 --2- 172.21.15.32:0/1481652433 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5620273a5e40 0x5620273a8230 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379a1ac06e3cf97a server_cookie=bdca9fd38c5e7f27 in_seq=0 out_seq=0 2023-09-30T14:42:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.440+0000 7f07017fa700 1 -- 172.21.15.32:0/1481652433 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562027287ad0 con 0x5620273a5e40 2023-09-30T14:42:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.440+0000 7f07017fa700 10 client.25556 handle_command_reply: tid=0 2023-09-30T14:42:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.440+0000 7f072bd58b80 1 client.25556 shutdown 2023-09-30T14:42:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.441+0000 7f072bd58b80 1 -- 172.21.15.32:0/1481652433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733c200 msgr2=0x56202734af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.441+0000 7f072bd58b80 1 --2- 172.21.15.32:0/1481652433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733c200 0x56202734af10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0714009990 tx=0x7f071409a800 comp rx=0 tx=0).stop 2023-09-30T14:42:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.441+0000 7f072bd58b80 1 -- 172.21.15.32:0/1481652433 shutdown_connections 2023-09-30T14:42:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.441+0000 7f072bd58b80 1 --2- 172.21.15.32:0/1481652433 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5620273a5e40 0x5620273a8230 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.441+0000 7f072bd58b80 1 --2- 172.21.15.32:0/1481652433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56202733dcd0 0x56202734c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.441+0000 7f072bd58b80 1 --2- 172.21.15.32:0/1481652433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56202733c200 0x56202734af10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.441+0000 7f072bd58b80 1 --2- 172.21.15.32:0/1481652433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56202733d310 0x56202734b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.441+0000 7f07017fa700 0 client.25556 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.441+0000 7f072bd58b80 1 -- 172.21.15.32:0/1481652433 >> 172.21.15.32:0/1481652433 conn(0x5620272b0200 msgr2=0x562027345950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.441+0000 7f072bd58b80 1 -- 172.21.15.32:0/1481652433 shutdown_connections 2023-09-30T14:42:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.441+0000 7f072bd58b80 1 -- 172.21.15.32:0/1481652433 wait complete. 2023-09-30T14:42:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.441+0000 7f072bd58b80 20 client.25556 trim_cache size 0 max 16384 2023-09-30T14:42:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:42:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:42:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:42:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:42:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.441+0000 7f071fb70700 1 -- 172.21.15.32:0/3739807552 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f07040a0fa0 msgr2=0x7f07040a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.441+0000 7f071fb70700 1 --2- 172.21.15.32:0/3739807552 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f07040a0fa0 0x7f07040a3460 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0708009d20 tx=0x7f0708009260 comp rx=0 tx=0).stop 2023-09-30T14:42:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.442+0000 7f071fb70700 1 -- 172.21.15.32:0/3739807552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0718107f40 msgr2=0x7f07181a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.442+0000 7f071fb70700 1 --2- 172.21.15.32:0/3739807552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0718107f40 0x7f07181a1510 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f07140059d0 tx=0x7f0714004710 comp rx=0 tx=0).stop 2023-09-30T14:42:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.442+0000 7f071fb70700 1 -- 172.21.15.32:0/3739807552 shutdown_connections 2023-09-30T14:42:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.442+0000 7f071fb70700 1 --2- 172.21.15.32:0/3739807552 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f07040a0fa0 0x7f07040a3460 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.442+0000 7f071fb70700 1 --2- 172.21.15.32:0/3739807552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0718109260 0x7f07181a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.442+0000 7f071fb70700 1 --2- 172.21.15.32:0/3739807552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0718107f40 0x7f07181a1510 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.442+0000 7f071fb70700 1 --2- 172.21.15.32:0/3739807552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07181088a0 0x7f07181a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.442+0000 7f071fb70700 1 -- 172.21.15.32:0/3739807552 >> 172.21.15.32:0/3739807552 conn(0x7f071806ecd0 msgr2=0x7f0718070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.442+0000 7f071fb70700 1 -- 172.21.15.32:0/3739807552 shutdown_connections 2023-09-30T14:42:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:13.442+0000 7f071fb70700 1 -- 172.21.15.32:0/3739807552 wait complete. 2023-09-30T14:42:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:13 smithi148 ceph-mon[102355]: pgmap v405: 97 pgs: 97 active+clean; 138 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 3.2 MiB/s wr, 318 op/s 2023-09-30T14:42:13.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:13 smithi032 ceph-mon[98592]: pgmap v405: 97 pgs: 97 active+clean; 138 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 3.2 MiB/s wr, 318 op/s 2023-09-30T14:42:13.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 367943 10.75 MB/sec execute 47 sec latency 73.040 ms 2023-09-30T14:42:13.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:13 smithi156 ceph-mon[103579]: pgmap v405: 97 pgs: 97 active+clean; 138 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 3.2 MiB/s wr, 318 op/s 2023-09-30T14:42:14.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 369750 10.77 MB/sec execute 48 sec latency 20.165 ms 2023-09-30T14:42:15.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:15 smithi148 ceph-mon[102355]: pgmap v406: 97 pgs: 97 active+clean; 138 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 2.9 MiB/s wr, 300 op/s 2023-09-30T14:42:15.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:15 smithi032 ceph-mon[98592]: pgmap v406: 97 pgs: 97 active+clean; 138 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 2.9 MiB/s wr, 300 op/s 2023-09-30T14:42:15.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 372535 10.90 MB/sec execute 49 sec latency 12.122 ms 2023-09-30T14:42:15.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:15 smithi156 ceph-mon[103579]: pgmap v406: 97 pgs: 97 active+clean; 138 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 2.9 MiB/s wr, 300 op/s 2023-09-30T14:42:16.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 373619 10.75 MB/sec execute 50 sec latency 68.047 ms 2023-09-30T14:42:17.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:17 smithi148 ceph-mon[102355]: pgmap v407: 97 pgs: 97 active+clean; 141 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 357 op/s 2023-09-30T14:42:17.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:17 smithi032 ceph-mon[98592]: pgmap v407: 97 pgs: 97 active+clean; 141 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 357 op/s 2023-09-30T14:42:17.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 376286 10.87 MB/sec execute 51 sec latency 38.732 ms 2023-09-30T14:42:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:17 smithi156 ceph-mon[103579]: pgmap v407: 97 pgs: 97 active+clean; 141 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 357 op/s 2023-09-30T14:42:18.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 378360 10.80 MB/sec execute 52 sec latency 15.520 ms 2023-09-30T14:42:19.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:19 smithi148 ceph-mon[102355]: pgmap v408: 97 pgs: 97 active+clean; 141 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 3.0 MiB/s wr, 297 op/s 2023-09-30T14:42:19.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:19 smithi032 ceph-mon[98592]: pgmap v408: 97 pgs: 97 active+clean; 141 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 3.0 MiB/s wr, 297 op/s 2023-09-30T14:42:19.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 380155 10.83 MB/sec execute 53 sec latency 12.526 ms 2023-09-30T14:42:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:19 smithi156 ceph-mon[103579]: pgmap v408: 97 pgs: 97 active+clean; 141 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 3.0 MiB/s wr, 297 op/s 2023-09-30T14:42:20.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 382749 10.87 MB/sec execute 54 sec latency 40.072 ms 2023-09-30T14:42:21.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:21 smithi148 ceph-mon[102355]: pgmap v409: 97 pgs: 97 active+clean; 142 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 304 op/s 2023-09-30T14:42:21.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:21 smithi032 ceph-mon[98592]: pgmap v409: 97 pgs: 97 active+clean; 142 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 304 op/s 2023-09-30T14:42:21.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 384007 10.80 MB/sec execute 55 sec latency 32.611 ms 2023-09-30T14:42:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:21 smithi156 ceph-mon[103579]: pgmap v409: 97 pgs: 97 active+clean; 142 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 304 op/s 2023-09-30T14:42:22.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 386897 10.93 MB/sec execute 56 sec latency 11.020 ms 2023-09-30T14:42:23.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:23 smithi148 ceph-mon[102355]: pgmap v410: 97 pgs: 97 active+clean; 143 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 306 op/s 2023-09-30T14:42:23.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:23 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:42:23.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:23 smithi032 ceph-mon[98592]: pgmap v410: 97 pgs: 97 active+clean; 143 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 306 op/s 2023-09-30T14:42:23.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:23 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:42:23.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 388325 10.83 MB/sec execute 57 sec latency 12.106 ms 2023-09-30T14:42:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:23 smithi156 ceph-mon[103579]: pgmap v410: 97 pgs: 97 active+clean; 143 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 306 op/s 2023-09-30T14:42:23.886 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:23 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:42:24.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:24 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:42:24.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:24 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:42:24.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:24 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:42:24.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:24 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:42:24.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:24 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:42:24.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:42:24.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:42:24.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:42:24.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:42:24.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:24 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:42:24.697 INFO:tasks.workunit.client.0.smithi032.stdout: 1 390601 10.89 MB/sec execute 58 sec latency 10.788 ms 2023-09-30T14:42:24.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:24 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:42:24.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:24 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:42:24.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:24 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:42:24.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:24 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:42:24.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:24 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:42:25.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:25 smithi148 ceph-mon[102355]: pgmap v411: 97 pgs: 97 active+clean; 145 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.7 MiB/s wr, 284 op/s 2023-09-30T14:42:25.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:25 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:42:25.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:25 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:42:25.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:25 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:42:25.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:25 smithi032 ceph-mon[98592]: pgmap v411: 97 pgs: 97 active+clean; 145 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.7 MiB/s wr, 284 op/s 2023-09-30T14:42:25.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:25 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:42:25.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:25 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:42:25.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:25 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:42:25.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 392259 10.80 MB/sec execute 59 sec latency 74.159 ms 2023-09-30T14:42:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:25 smithi156 ceph-mon[103579]: pgmap v411: 97 pgs: 97 active+clean; 145 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.7 MiB/s wr, 284 op/s 2023-09-30T14:42:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:25 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:42:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:25 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:42:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:25 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:42:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.570+0000 7f9686878700 1 -- 172.21.15.32:0/1958001627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96800ffd60 msgr2=0x7f9680100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.570+0000 7f9686878700 1 --2- 172.21.15.32:0/1958001627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96800ffd60 0x7f9680100180 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f96700099c0 tx=0x7f9670009cd0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.570+0000 7f9686878700 1 -- 172.21.15.32:0/1958001627 shutdown_connections 2023-09-30T14:42:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.570+0000 7f9686878700 1 --2- 172.21.15.32:0/1958001627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96801018f0 0x7f96801081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.570+0000 7f9686878700 1 --2- 172.21.15.32:0/1958001627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96800ffd60 0x7f9680100180 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.570+0000 7f9686878700 1 --2- 172.21.15.32:0/1958001627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9680100f50 0x7f96801013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.570+0000 7f9686878700 1 -- 172.21.15.32:0/1958001627 >> 172.21.15.32:0/1958001627 conn(0x7f96800fb2b0 msgr2=0x7f96800fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.570+0000 7f9686878700 1 -- 172.21.15.32:0/1958001627 shutdown_connections 2023-09-30T14:42:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.570+0000 7f9686878700 1 -- 172.21.15.32:0/1958001627 wait complete. 2023-09-30T14:42:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.570+0000 7f9686878700 1 Processor -- start 2023-09-30T14:42:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f9686878700 1 -- start start 2023-09-30T14:42:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f9686878700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96800ffd60 0x7f9680199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f9686878700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9680100f50 0x7f968019bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f9686878700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96801018f0 0x7f968019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f9686878700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968019e8e0 con 0x7f96801018f0 2023-09-30T14:42:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f9686878700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968019ea20 con 0x7f96800ffd60 2023-09-30T14:42:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f9686878700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968019ebe0 con 0x7f9680100f50 2023-09-30T14:42:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f9684e15700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96801018f0 0x7f968019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f967ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96800ffd60 0x7f9680199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f967ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96800ffd60 0x7f9680199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48048/0 (socket says 172.21.15.32:48048) 2023-09-30T14:42:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f967ffff700 1 -- 172.21.15.32:0/3072898323 learned_addr learned my addr 172.21.15.32:0/3072898323 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f967f7fe700 1 --2- 172.21.15.32:0/3072898323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9680100f50 0x7f968019bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f9684e15700 1 -- 172.21.15.32:0/3072898323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9680100f50 msgr2=0x7f968019bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f9684e15700 1 --2- 172.21.15.32:0/3072898323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9680100f50 0x7f968019bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f9684e15700 1 -- 172.21.15.32:0/3072898323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96800ffd60 msgr2=0x7f9680199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f9684e15700 1 --2- 172.21.15.32:0/3072898323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96800ffd60 0x7f9680199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f9684e15700 1 -- 172.21.15.32:0/3072898323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96700095f0 con 0x7f96801018f0 2023-09-30T14:42:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.571+0000 7f967ffff700 1 --2- 172.21.15.32:0/3072898323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96800ffd60 0x7f9680199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.572+0000 7f9684e15700 1 --2- 172.21.15.32:0/3072898323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96801018f0 0x7f968019e310 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f967800efc0 tx=0x7f967800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.572+0000 7f967d7fa700 1 -- 172.21.15.32:0/3072898323 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678018070 con 0x7f96801018f0 2023-09-30T14:42:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.572+0000 7f967d7fa700 1 -- 172.21.15.32:0/3072898323 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f967800f040 con 0x7f96801018f0 2023-09-30T14:42:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.572+0000 7f9686878700 1 -- 172.21.15.32:0/3072898323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968019ee60 con 0x7f96801018f0 2023-09-30T14:42:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.572+0000 7f967d7fa700 1 -- 172.21.15.32:0/3072898323 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967800cd40 con 0x7f96801018f0 2023-09-30T14:42:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.572+0000 7f9686878700 1 -- 172.21.15.32:0/3072898323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968019f290 con 0x7f96801018f0 2023-09-30T14:42:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.574+0000 7f967d7fa700 1 -- 172.21.15.32:0/3072898323 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f9678003a00 con 0x7f96801018f0 2023-09-30T14:42:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.575+0000 7f967d7fa700 1 --2- 172.21.15.32:0/3072898323 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f96680a0fa0 0x7f96680a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.575+0000 7f967d7fa700 1 -- 172.21.15.32:0/3072898323 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f96780cc310 con 0x7f96801018f0 2023-09-30T14:42:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.575+0000 7f967ffff700 1 --2- 172.21.15.32:0/3072898323 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f96680a0fa0 0x7f96680a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f967ffff700 1 --2- 172.21.15.32:0/3072898323 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f96680a0fa0 0x7f96680a3460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9670004800 tx=0x7f967001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f9692a60b80 1 Processor -- start 2023-09-30T14:42:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f9692a60b80 1 -- start start 2023-09-30T14:42:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f9692a60b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8453200 0x562fa8453620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f9692a60b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8454310 0x562fa8454790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f9692a60b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454cd0 0x562fa845b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f9692a60b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa83bb400 con 0x562fa8453200 2023-09-30T14:42:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f9692a60b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa83acbc0 con 0x562fa8454cd0 2023-09-30T14:42:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f9692a60b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa83aa4e0 con 0x562fa8454310 2023-09-30T14:42:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f967ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8453200 0x562fa8453620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f967ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8453200 0x562fa8453620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36580/0 (socket says 172.21.15.32:36580) 2023-09-30T14:42:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f967ffff700 1 -- 172.21.15.32:0/2326451545 learned_addr learned my addr 172.21.15.32:0/2326451545 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f967f7fe700 1 --2- 172.21.15.32:0/2326451545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8454310 0x562fa8454790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f9684e15700 1 --2- 172.21.15.32:0/2326451545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454cd0 0x562fa845b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f967ffff700 1 -- 172.21.15.32:0/2326451545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8454310 msgr2=0x562fa8454790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f967ffff700 1 --2- 172.21.15.32:0/2326451545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8454310 0x562fa8454790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f967ffff700 1 -- 172.21.15.32:0/2326451545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454cd0 msgr2=0x562fa845b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f967ffff700 1 --2- 172.21.15.32:0/2326451545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454cd0 0x562fa845b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.576+0000 7f967ffff700 1 -- 172.21.15.32:0/2326451545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fa8408870 con 0x562fa8453200 2023-09-30T14:42:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.577+0000 7f967ffff700 1 --2- 172.21.15.32:0/2326451545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8453200 0x562fa8453620 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f96700098e0 tx=0x7f967001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18834276412c1bf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.577+0000 7f96667fc700 1 -- 172.21.15.32:0/2326451545 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967002a070 con 0x562fa8453200 2023-09-30T14:42:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.577+0000 7f96667fc700 1 -- 172.21.15.32:0/2326451545 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f967001e040 con 0x562fa8453200 2023-09-30T14:42:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.577+0000 7f96667fc700 1 -- 172.21.15.32:0/2326451545 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670005240 con 0x562fa8453200 2023-09-30T14:42:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.577+0000 7f9692a60b80 1 -- 172.21.15.32:0/2326451545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8453200 msgr2=0x562fa8453620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.577+0000 7f9692a60b80 1 --2- 172.21.15.32:0/2326451545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8453200 0x562fa8453620 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f96700098e0 tx=0x7f967001f000 comp rx=0 tx=0).stop 2023-09-30T14:42:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.577+0000 7f9692a60b80 1 -- 172.21.15.32:0/2326451545 shutdown_connections 2023-09-30T14:42:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.577+0000 7f9692a60b80 1 --2- 172.21.15.32:0/2326451545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454cd0 0x562fa845b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.577+0000 7f9692a60b80 1 --2- 172.21.15.32:0/2326451545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8453200 0x562fa8453620 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.577+0000 7f9692a60b80 1 --2- 172.21.15.32:0/2326451545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8454310 0x562fa8454790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.577+0000 7f9692a60b80 1 -- 172.21.15.32:0/2326451545 >> 172.21.15.32:0/2326451545 conn(0x562fa83c7200 msgr2=0x562fa844def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.577+0000 7f9692a60b80 1 -- 172.21.15.32:0/2326451545 shutdown_connections 2023-09-30T14:42:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.578+0000 7f9692a60b80 1 -- 172.21.15.32:0/2326451545 wait complete. 2023-09-30T14:42:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.578+0000 7f9692a60b80 1 -- start start 2023-09-30T14:42:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.578+0000 7f9692a60b80 1 Processor -- start 2023-09-30T14:42:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.578+0000 7f9692a60b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8453200 0x562fa845ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f9692a60b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8454310 0x562fa845d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f9692a60b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454cd0 0x562fa8464380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f9692a60b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa83a5dd0 con 0x562fa8454310 2023-09-30T14:42:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f9692a60b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa83426b0 con 0x562fa8454cd0 2023-09-30T14:42:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f9692a60b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa8342350 con 0x562fa8453200 2023-09-30T14:42:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f9692a60b80 10 client.0 _ll_register_callbacks cb 0x562fa819f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:42:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f967f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8454310 0x562fa845d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f967f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8454310 0x562fa845d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36588/0 (socket says 172.21.15.32:36588) 2023-09-30T14:42:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f967f7fe700 1 -- 172.21.15.32:0/4121449112 learned_addr learned my addr 172.21.15.32:0/4121449112 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f9684e15700 1 --2- 172.21.15.32:0/4121449112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454cd0 0x562fa8464380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f967ffff700 1 --2- 172.21.15.32:0/4121449112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8453200 0x562fa845ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f967f7fe700 1 -- 172.21.15.32:0/4121449112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8453200 msgr2=0x562fa845ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f967f7fe700 1 --2- 172.21.15.32:0/4121449112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8453200 0x562fa845ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f967f7fe700 1 -- 172.21.15.32:0/4121449112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454cd0 msgr2=0x562fa8464380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f967f7fe700 1 --2- 172.21.15.32:0/4121449112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454cd0 0x562fa8464380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f967f7fe700 1 -- 172.21.15.32:0/4121449112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9680108af0 con 0x562fa8454310 2023-09-30T14:42:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f9684e15700 1 --2- 172.21.15.32:0/4121449112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454cd0 0x562fa8464380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f967f7fe700 1 --2- 172.21.15.32:0/4121449112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8454310 0x562fa845d340 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f96740127f0 tx=0x7f9674012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2571bd2c09f986 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f965f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:42:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f965f7fe700 1 -- 172.21.15.32:0/4121449112 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674017070 con 0x562fa8454310 2023-09-30T14:42:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f9692a60b80 1 -- 172.21.15.32:0/4121449112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562fa8446090 con 0x562fa8454310 2023-09-30T14:42:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f965f7fe700 1 -- 172.21.15.32:0/4121449112 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f967400f4e0 con 0x562fa8454310 2023-09-30T14:42:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.579+0000 7f965f7fe700 1 -- 172.21.15.32:0/4121449112 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f967400f6a0 con 0x562fa8454310 2023-09-30T14:42:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.580+0000 7f9692a60b80 10 client.15876 fetch_fsmap learned FSMap version 30 2023-09-30T14:42:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.580+0000 7f9692a60b80 1 -- 172.21.15.32:0/4121449112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562fa81d0d80 con 0x562fa8454310 2023-09-30T14:42:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.580+0000 7f965f7fe700 1 -- 172.21.15.32:0/4121449112 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562fa8446090 con 0x562fa8454310 2023-09-30T14:42:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.580+0000 7f965f7fe700 1 -- 172.21.15.32:0/4121449112 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f96740108a0 con 0x562fa8454310 2023-09-30T14:42:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.580+0000 7f9692a60b80 10 client.15876 fetch_fsmap finished waiting for FSMap version 30 2023-09-30T14:42:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.580+0000 7f9692a60b80 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:42:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.580+0000 7f9692a60b80 20 client.15876 populate_metadata read hostname 'smithi032' 2023-09-30T14:42:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.580+0000 7f9692a60b80 1 --2- 172.21.15.32:0/4121449112 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562fa8468d30 0x562fa846b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.580+0000 7f9692a60b80 4 client.15876 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:42:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.580+0000 7f9692a60b80 1 -- 172.21.15.32:0/4121449112 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562fa839edb0 con 0x562fa8468d30 2023-09-30T14:42:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.580+0000 7f967ffff700 1 --2- 172.21.15.32:0/4121449112 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562fa8468d30 0x562fa846b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.581+0000 7f965f7fe700 10 client.15876 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.581+0000 7f967ffff700 1 --2- 172.21.15.32:0/4121449112 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562fa8468d30 0x562fa846b150 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f743870587a0a7db server_cookie=16f6edf4ff258107 in_seq=0 out_seq=0 2023-09-30T14:42:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.581+0000 7f965f7fe700 1 -- 172.21.15.32:0/4121449112 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562fa839edb0 con 0x562fa8468d30 2023-09-30T14:42:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.581+0000 7f965f7fe700 10 client.15876 handle_command_reply: tid=0 2023-09-30T14:42:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.581+0000 7f9692a60b80 1 client.15876 shutdown 2023-09-30T14:42:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.581+0000 7f9692a60b80 1 -- 172.21.15.32:0/4121449112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8454310 msgr2=0x562fa845d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.581+0000 7f9692a60b80 1 --2- 172.21.15.32:0/4121449112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8454310 0x562fa845d340 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f96740127f0 tx=0x7f9674012b00 comp rx=0 tx=0).stop 2023-09-30T14:42:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.581+0000 7f9692a60b80 1 -- 172.21.15.32:0/4121449112 shutdown_connections 2023-09-30T14:42:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.581+0000 7f9692a60b80 1 --2- 172.21.15.32:0/4121449112 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562fa8468d30 0x562fa846b150 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.581+0000 7f9692a60b80 1 --2- 172.21.15.32:0/4121449112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454cd0 0x562fa8464380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.581+0000 7f9692a60b80 1 --2- 172.21.15.32:0/4121449112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8453200 0x562fa845ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.581+0000 7f965f7fe700 0 client.15876 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.581+0000 7f9692a60b80 1 --2- 172.21.15.32:0/4121449112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8454310 0x562fa845d340 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.581+0000 7f9692a60b80 1 -- 172.21.15.32:0/4121449112 >> 172.21.15.32:0/4121449112 conn(0x562fa83c7200 msgr2=0x562fa844d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.581+0000 7f9692a60b80 1 -- 172.21.15.32:0/4121449112 shutdown_connections 2023-09-30T14:42:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.581+0000 7f9692a60b80 1 -- 172.21.15.32:0/4121449112 wait complete. 2023-09-30T14:42:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.582+0000 7f9692a60b80 20 client.15876 trim_cache size 0 max 16384 2023-09-30T14:42:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.597+0000 7f9692a60b80 1 Processor -- start 2023-09-30T14:42:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.597+0000 7f9692a60b80 1 -- start start 2023-09-30T14:42:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.597+0000 7f9692a60b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8453200 0x562fa84546f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.597+0000 7f9692a60b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8454310 0x562fa8464340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.597+0000 7f9692a60b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8454cd0 0x562fa845bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.597+0000 7f9692a60b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa82268c0 con 0x562fa8454cd0 2023-09-30T14:42:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.597+0000 7f9692a60b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa8457790 con 0x562fa8453200 2023-09-30T14:42:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.597+0000 7f9692a60b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa845da80 con 0x562fa8454310 2023-09-30T14:42:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.597+0000 7f9684e15700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8454cd0 0x562fa845bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.597+0000 7f967f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8454310 0x562fa8464340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.597+0000 7f967ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8453200 0x562fa84546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.597+0000 7f9684e15700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8454cd0 0x562fa845bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36598/0 (socket says 172.21.15.32:36598) 2023-09-30T14:42:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.597+0000 7f967f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8454310 0x562fa8464340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:50978/0 (socket says 172.21.15.32:50978) 2023-09-30T14:42:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.597+0000 7f9684e15700 1 -- 172.21.15.32:0/4115389601 learned_addr learned my addr 172.21.15.32:0/4115389601 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.598+0000 7f9684e15700 1 -- 172.21.15.32:0/4115389601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8454310 msgr2=0x562fa8464340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.598+0000 7f9684e15700 1 --2- 172.21.15.32:0/4115389601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8454310 0x562fa8464340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.598+0000 7f9684e15700 1 -- 172.21.15.32:0/4115389601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8453200 msgr2=0x562fa84546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.598+0000 7f9684e15700 1 --2- 172.21.15.32:0/4115389601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8453200 0x562fa84546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.598+0000 7f9684e15700 1 -- 172.21.15.32:0/4115389601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fa81b9c00 con 0x562fa8454cd0 2023-09-30T14:42:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.598+0000 7f967ffff700 1 --2- 172.21.15.32:0/4115389601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8453200 0x562fa84546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.598+0000 7f9684e15700 1 --2- 172.21.15.32:0/4115389601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8454cd0 0x562fa845bb30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f96780090a0 tx=0x7f9678004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755ed810f5fce88d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.598+0000 7f9666ffd700 1 -- 172.21.15.32:0/4115389601 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678082ea0 con 0x562fa8454cd0 2023-09-30T14:42:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.598+0000 7f9666ffd700 1 -- 172.21.15.32:0/4115389601 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9678099f70 con 0x562fa8454cd0 2023-09-30T14:42:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.598+0000 7f9666ffd700 1 -- 172.21.15.32:0/4115389601 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967801b580 con 0x562fa8454cd0 2023-09-30T14:42:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.599+0000 7f9692a60b80 1 -- 172.21.15.32:0/4115389601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8454cd0 msgr2=0x562fa845bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.599+0000 7f9692a60b80 1 --2- 172.21.15.32:0/4115389601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8454cd0 0x562fa845bb30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f96780090a0 tx=0x7f9678004dc0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.599+0000 7f9692a60b80 1 -- 172.21.15.32:0/4115389601 shutdown_connections 2023-09-30T14:42:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.599+0000 7f9692a60b80 1 --2- 172.21.15.32:0/4115389601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8454cd0 0x562fa845bb30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.599+0000 7f9692a60b80 1 --2- 172.21.15.32:0/4115389601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8453200 0x562fa84546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.599+0000 7f9692a60b80 1 --2- 172.21.15.32:0/4115389601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8454310 0x562fa8464340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.599+0000 7f9692a60b80 1 -- 172.21.15.32:0/4115389601 >> 172.21.15.32:0/4115389601 conn(0x562fa83c7200 msgr2=0x562fa846fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.599+0000 7f9692a60b80 1 -- 172.21.15.32:0/4115389601 shutdown_connections 2023-09-30T14:42:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.599+0000 7f9692a60b80 1 -- 172.21.15.32:0/4115389601 wait complete. 2023-09-30T14:42:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.599+0000 7f9692a60b80 1 -- start start 2023-09-30T14:42:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.599+0000 7f9692a60b80 1 Processor -- start 2023-09-30T14:42:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f9692a60b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8453200 0x562fa8461c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f9692a60b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454310 0x562fa8462190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f9692a60b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8454cd0 0x562fa84631a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f9692a60b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa84636e0 con 0x562fa8453200 2023-09-30T14:42:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f9692a60b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa8463850 con 0x562fa8454310 2023-09-30T14:42:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f9692a60b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fa8463a10 con 0x562fa8454cd0 2023-09-30T14:42:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f9692a60b80 10 client.0 _ll_register_callbacks cb 0x7f9674009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:42:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f967ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8453200 0x562fa8461c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f967ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8453200 0x562fa8461c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36600/0 (socket says 172.21.15.32:36600) 2023-09-30T14:42:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f967ffff700 1 -- 172.21.15.32:0/857432133 learned_addr learned my addr 172.21.15.32:0/857432133 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f967f7fe700 1 --2- 172.21.15.32:0/857432133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454310 0x562fa8462190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f9684e15700 1 --2- 172.21.15.32:0/857432133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8454cd0 0x562fa84631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f967ffff700 1 -- 172.21.15.32:0/857432133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8454cd0 msgr2=0x562fa84631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f967ffff700 1 --2- 172.21.15.32:0/857432133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8454cd0 0x562fa84631a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f967ffff700 1 -- 172.21.15.32:0/857432133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454310 msgr2=0x562fa8462190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f967ffff700 1 --2- 172.21.15.32:0/857432133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454310 0x562fa8462190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f967ffff700 1 -- 172.21.15.32:0/857432133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fa8408870 con 0x562fa8453200 2023-09-30T14:42:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f967f7fe700 1 --2- 172.21.15.32:0/857432133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454310 0x562fa8462190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f967ffff700 1 --2- 172.21.15.32:0/857432133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8453200 0x562fa8461c50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f96700095c0 tx=0x7f96700094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5df811751acccfb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f965f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:42:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f965f7fe700 1 -- 172.21.15.32:0/857432133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967002a040 con 0x562fa8453200 2023-09-30T14:42:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.600+0000 7f965f7fe700 1 -- 172.21.15.32:0/857432133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f967003daf0 con 0x562fa8453200 2023-09-30T14:42:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.601+0000 7f9692a60b80 1 -- 172.21.15.32:0/857432133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f967001e040 con 0x562fa8453200 2023-09-30T14:42:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.601+0000 7f965f7fe700 1 -- 172.21.15.32:0/857432133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f967003dcb0 con 0x562fa8453200 2023-09-30T14:42:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.601+0000 7f9692a60b80 10 client.15882 fetch_fsmap learned FSMap version 30 2023-09-30T14:42:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.601+0000 7f9692a60b80 1 -- 172.21.15.32:0/857432133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f968019f290 con 0x562fa8453200 2023-09-30T14:42:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.601+0000 7f965f7fe700 1 -- 172.21.15.32:0/857432133 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f967001e040 con 0x562fa8453200 2023-09-30T14:42:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.601+0000 7f965f7fe700 1 -- 172.21.15.32:0/857432133 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9670021070 con 0x562fa8453200 2023-09-30T14:42:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.601+0000 7f9692a60b80 10 client.15882 fetch_fsmap finished waiting for FSMap version 30 2023-09-30T14:42:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.601+0000 7f9692a60b80 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:42:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.601+0000 7f9692a60b80 20 client.15882 populate_metadata read hostname 'smithi032' 2023-09-30T14:42:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.601+0000 7f9692a60b80 1 --2- 172.21.15.32:0/857432133 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562fa84bd350 0x562fa84bf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.601+0000 7f9692a60b80 4 client.15882 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:42:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.601+0000 7f9692a60b80 1 -- 172.21.15.32:0/857432133 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562fa839ead0 con 0x562fa84bd350 2023-09-30T14:42:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.601+0000 7f967f7fe700 1 --2- 172.21.15.32:0/857432133 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562fa84bd350 0x562fa84bf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.602+0000 7f965f7fe700 10 client.15882 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:26.602+0000 7f967f7fe700 1 --2- 172.21.15.32:0/857432133 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562fa84bd350 0x562fa84bf770 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=613b2e18e3b3888f server_cookie=5b094d9a1367a07e in_seq=0 out_seq=0 2023-09-30T14:42:26.762 INFO:tasks.workunit.client.0.smithi032.stdout: 1 394213 10.84 MB/sec execute 60 sec latency 26.399 ms 2023-09-30T14:42:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.175+0000 7f965f7fe700 1 -- 172.21.15.32:0/857432133 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562fa839ead0 con 0x562fa84bd350 2023-09-30T14:42:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.175+0000 7f965f7fe700 10 client.15882 handle_command_reply: tid=0 2023-09-30T14:42:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.175+0000 7f9692a60b80 1 client.15882 shutdown 2023-09-30T14:42:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.175+0000 7f9692a60b80 1 -- 172.21.15.32:0/857432133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8453200 msgr2=0x562fa8461c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.175+0000 7f9692a60b80 1 --2- 172.21.15.32:0/857432133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8453200 0x562fa8461c50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f96700095c0 tx=0x7f96700094f0 comp rx=0 tx=0).stop 2023-09-30T14:42:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.175+0000 7f9692a60b80 1 -- 172.21.15.32:0/857432133 shutdown_connections 2023-09-30T14:42:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.175+0000 7f9692a60b80 1 --2- 172.21.15.32:0/857432133 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562fa84bd350 0x562fa84bf770 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.175+0000 7f9692a60b80 1 --2- 172.21.15.32:0/857432133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fa8454cd0 0x562fa84631a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.175+0000 7f9692a60b80 1 --2- 172.21.15.32:0/857432133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562fa8453200 0x562fa8461c50 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.175+0000 7f965f7fe700 0 client.15882 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.175+0000 7f9692a60b80 1 --2- 172.21.15.32:0/857432133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562fa8454310 0x562fa8462190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.175+0000 7f9692a60b80 1 -- 172.21.15.32:0/857432133 >> 172.21.15.32:0/857432133 conn(0x562fa83c7200 msgr2=0x562fa845b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.175+0000 7f9692a60b80 1 -- 172.21.15.32:0/857432133 shutdown_connections 2023-09-30T14:42:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.175+0000 7f9692a60b80 1 -- 172.21.15.32:0/857432133 wait complete. 2023-09-30T14:42:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.175+0000 7f9692a60b80 20 client.15882 trim_cache size 0 max 16384 2023-09-30T14:42:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:42:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:42:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:42:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:42:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.176+0000 7f9686878700 1 -- 172.21.15.32:0/3072898323 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f96680a0fa0 msgr2=0x7f96680a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.176+0000 7f9686878700 1 --2- 172.21.15.32:0/3072898323 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f96680a0fa0 0x7f96680a3460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9670004800 tx=0x7f967001a040 comp rx=0 tx=0).stop 2023-09-30T14:42:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.176+0000 7f9686878700 1 -- 172.21.15.32:0/3072898323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96801018f0 msgr2=0x7f968019e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.176+0000 7f9686878700 1 --2- 172.21.15.32:0/3072898323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96801018f0 0x7f968019e310 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f967800efc0 tx=0x7f967800c3a0 comp rx=0 tx=0).stop 2023-09-30T14:42:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.176+0000 7f9686878700 1 -- 172.21.15.32:0/3072898323 shutdown_connections 2023-09-30T14:42:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.176+0000 7f9686878700 1 --2- 172.21.15.32:0/3072898323 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f96680a0fa0 0x7f96680a3460 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.176+0000 7f9686878700 1 --2- 172.21.15.32:0/3072898323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96801018f0 0x7f968019e310 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.176+0000 7f9686878700 1 --2- 172.21.15.32:0/3072898323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96800ffd60 0x7f9680199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.176+0000 7f9686878700 1 --2- 172.21.15.32:0/3072898323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9680100f50 0x7f968019bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.176+0000 7f9686878700 1 -- 172.21.15.32:0/3072898323 >> 172.21.15.32:0/3072898323 conn(0x7f96800fb2b0 msgr2=0x7f96800fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.176+0000 7f9686878700 1 -- 172.21.15.32:0/3072898323 shutdown_connections 2023-09-30T14:42:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:27.176+0000 7f9686878700 1 -- 172.21.15.32:0/3072898323 wait complete. 2023-09-30T14:42:27.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:27 smithi148 ceph-mon[102355]: pgmap v412: 97 pgs: 97 active+clean; 146 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 327 op/s 2023-09-30T14:42:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:27 smithi032 ceph-mon[98592]: pgmap v412: 97 pgs: 97 active+clean; 146 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 327 op/s 2023-09-30T14:42:27.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 396743 10.87 MB/sec execute 61 sec latency 31.157 ms 2023-09-30T14:42:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:27 smithi156 ceph-mon[103579]: pgmap v412: 97 pgs: 97 active+clean; 146 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 327 op/s 2023-09-30T14:42:28.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 398099 10.81 MB/sec execute 62 sec latency 95.701 ms 2023-09-30T14:42:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:29 smithi148 ceph-mon[102355]: pgmap v413: 97 pgs: 97 active+clean; 148 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 2.8 MiB/s wr, 276 op/s 2023-09-30T14:42:29.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:29 smithi032 ceph-mon[98592]: pgmap v413: 97 pgs: 97 active+clean; 148 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 2.8 MiB/s wr, 276 op/s 2023-09-30T14:42:29.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 401066 10.93 MB/sec execute 63 sec latency 26.173 ms 2023-09-30T14:42:29.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:29 smithi156 ceph-mon[103579]: pgmap v413: 97 pgs: 97 active+clean; 148 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 2.8 MiB/s wr, 276 op/s 2023-09-30T14:42:30.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 402480 10.84 MB/sec execute 64 sec latency 14.861 ms 2023-09-30T14:42:31.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:31 smithi148 ceph-mon[102355]: pgmap v414: 97 pgs: 97 active+clean; 148 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.8 MiB/s wr, 284 op/s 2023-09-30T14:42:31.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:31 smithi032 ceph-mon[98592]: pgmap v414: 97 pgs: 97 active+clean; 148 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.8 MiB/s wr, 284 op/s 2023-09-30T14:42:31.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 404759 10.89 MB/sec execute 65 sec latency 33.598 ms 2023-09-30T14:42:31.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:31 smithi156 ceph-mon[103579]: pgmap v414: 97 pgs: 97 active+clean; 148 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.8 MiB/s wr, 284 op/s 2023-09-30T14:42:32.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 406959 10.85 MB/sec execute 66 sec latency 12.779 ms 2023-09-30T14:42:33.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:33 smithi148 ceph-mon[102355]: pgmap v415: 97 pgs: 97 active+clean; 150 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 328 op/s 2023-09-30T14:42:33.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:33 smithi032 ceph-mon[98592]: pgmap v415: 97 pgs: 97 active+clean; 150 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 328 op/s 2023-09-30T14:42:33.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 408568 10.85 MB/sec execute 67 sec latency 36.008 ms 2023-09-30T14:42:33.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:33 smithi156 ceph-mon[103579]: pgmap v415: 97 pgs: 97 active+clean; 150 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 328 op/s 2023-09-30T14:42:34.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 411436 10.95 MB/sec execute 68 sec latency 10.880 ms 2023-09-30T14:42:35.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:35 smithi148 ceph-mon[102355]: pgmap v416: 97 pgs: 97 active+clean; 151 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 2.7 MiB/s wr, 287 op/s 2023-09-30T14:42:35.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:35 smithi032 ceph-mon[98592]: pgmap v416: 97 pgs: 97 active+clean; 151 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 2.7 MiB/s wr, 287 op/s 2023-09-30T14:42:35.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 412894 10.89 MB/sec execute 69 sec latency 19.375 ms 2023-09-30T14:42:35.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:35 smithi156 ceph-mon[103579]: pgmap v416: 97 pgs: 97 active+clean; 151 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 2.7 MiB/s wr, 287 op/s 2023-09-30T14:42:36.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 415289 10.93 MB/sec execute 70 sec latency 46.711 ms 2023-09-30T14:42:37.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:37 smithi148 ceph-mon[102355]: pgmap v417: 97 pgs: 97 active+clean; 151 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.7 MiB/s wr, 287 op/s 2023-09-30T14:42:37.664 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:37 smithi032 ceph-mon[98592]: pgmap v417: 97 pgs: 97 active+clean; 151 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.7 MiB/s wr, 287 op/s 2023-09-30T14:42:37.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 417075 10.86 MB/sec execute 71 sec latency 20.668 ms 2023-09-30T14:42:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:37 smithi156 ceph-mon[103579]: pgmap v417: 97 pgs: 97 active+clean; 151 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.7 MiB/s wr, 287 op/s 2023-09-30T14:42:38.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 419136 10.89 MB/sec execute 72 sec latency 12.888 ms 2023-09-30T14:42:39.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:39 smithi148 ceph-mon[102355]: pgmap v418: 97 pgs: 97 active+clean; 158 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 2.8 MiB/s wr, 307 op/s 2023-09-30T14:42:39.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:39 smithi032 ceph-mon[98592]: pgmap v418: 97 pgs: 97 active+clean; 158 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 2.8 MiB/s wr, 307 op/s 2023-09-30T14:42:39.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 422092 10.98 MB/sec execute 73 sec latency 12.479 ms 2023-09-30T14:42:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:39 smithi156 ceph-mon[103579]: pgmap v418: 97 pgs: 97 active+clean; 158 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 2.8 MiB/s wr, 307 op/s 2023-09-30T14:42:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.293+0000 7f51e6fc5700 1 -- 172.21.15.32:0/1124507668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e0100280 msgr2=0x7f51e01006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.293+0000 7f51e6fc5700 1 --2- 172.21.15.32:0/1124507668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e0100280 0x7f51e01006a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009990 tx=0x7f51d4009ca0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.293+0000 7f51e6fc5700 1 -- 172.21.15.32:0/1124507668 shutdown_connections 2023-09-30T14:42:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.293+0000 7f51e6fc5700 1 --2- 172.21.15.32:0/1124507668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51e0101950 0x7f51e01081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.293+0000 7f51e6fc5700 1 --2- 172.21.15.32:0/1124507668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e0100280 0x7f51e01006a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.293+0000 7f51e6fc5700 1 --2- 172.21.15.32:0/1124507668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e0100f90 0x7f51e0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.293+0000 7f51e6fc5700 1 -- 172.21.15.32:0/1124507668 >> 172.21.15.32:0/1124507668 conn(0x7f51e00faa30 msgr2=0x7f51e00fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.293+0000 7f51e6fc5700 1 -- 172.21.15.32:0/1124507668 shutdown_connections 2023-09-30T14:42:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.293+0000 7f51e6fc5700 1 -- 172.21.15.32:0/1124507668 wait complete. 2023-09-30T14:42:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.293+0000 7f51e6fc5700 1 Processor -- start 2023-09-30T14:42:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51e6fc5700 1 -- start start 2023-09-30T14:42:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51e6fc5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51e0100280 0x7f51e0196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51e6fc5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e0100f90 0x7f51e0197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51e6fc5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e0101950 0x7f51e0197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51e6fc5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e0198690 con 0x7f51e0101950 2023-09-30T14:42:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51e6fc5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e0198800 con 0x7f51e0100f90 2023-09-30T14:42:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51e6fc5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e01989c0 con 0x7f51e0100280 2023-09-30T14:42:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51e5562700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e0101950 0x7f51e0197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51dffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e0100f90 0x7f51e0197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51e4d61700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51e0100280 0x7f51e0196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51e5562700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e0101950 0x7f51e0197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36618/0 (socket says 172.21.15.32:36618) 2023-09-30T14:42:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51e5562700 1 -- 172.21.15.32:0/2124886873 learned_addr learned my addr 172.21.15.32:0/2124886873 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51dffff700 1 -- 172.21.15.32:0/2124886873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51e0100280 msgr2=0x7f51e0196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51dffff700 1 --2- 172.21.15.32:0/2124886873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51e0100280 0x7f51e0196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51dffff700 1 -- 172.21.15.32:0/2124886873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e0101950 msgr2=0x7f51e0197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51dffff700 1 --2- 172.21.15.32:0/2124886873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e0101950 0x7f51e0197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51dffff700 1 -- 172.21.15.32:0/2124886873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d40095f0 con 0x7f51e0100f90 2023-09-30T14:42:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51e4d61700 1 --2- 172.21.15.32:0/2124886873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51e0100280 0x7f51e0196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.294+0000 7f51e5562700 1 --2- 172.21.15.32:0/2124886873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e0101950 0x7f51e0197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.295+0000 7f51dffff700 1 --2- 172.21.15.32:0/2124886873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e0100f90 0x7f51e0197250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00d740 tx=0x7f51cc00da50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.295+0000 7f51ddffb700 1 -- 172.21.15.32:0/2124886873 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc004220 con 0x7f51e0100f90 2023-09-30T14:42:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.295+0000 7f51ddffb700 1 -- 172.21.15.32:0/2124886873 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51cc004380 con 0x7f51e0100f90 2023-09-30T14:42:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.295+0000 7f51e6fc5700 1 -- 172.21.15.32:0/2124886873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e0198ca0 con 0x7f51e0100f90 2023-09-30T14:42:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.295+0000 7f51ddffb700 1 -- 172.21.15.32:0/2124886873 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc003c20 con 0x7f51e0100f90 2023-09-30T14:42:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.295+0000 7f51e6fc5700 1 -- 172.21.15.32:0/2124886873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e01023e0 con 0x7f51e0100f90 2023-09-30T14:42:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.297+0000 7f51ddffb700 1 -- 172.21.15.32:0/2124886873 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f51cc00f690 con 0x7f51e0100f90 2023-09-30T14:42:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.298+0000 7f51ddffb700 1 --2- 172.21.15.32:0/2124886873 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51d00a0fa0 0x7f51d00a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.298+0000 7f51e4d61700 1 --2- 172.21.15.32:0/2124886873 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51d00a0fa0 0x7f51d00a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.298+0000 7f51ddffb700 1 -- 172.21.15.32:0/2124886873 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f51cc0cc3c0 con 0x7f51e0100f90 2023-09-30T14:42:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.299+0000 7f51e4d61700 1 --2- 172.21.15.32:0/2124886873 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51d00a0fa0 0x7f51d00a3460 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f51d400b2e0 tx=0x7f51d4005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.299+0000 7f51f31adb80 1 Processor -- start 2023-09-30T14:42:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.299+0000 7f51f31adb80 1 -- start start 2023-09-30T14:42:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.299+0000 7f51f31adb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d2a10 0x55dba97d2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.299+0000 7f51f31adb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d3b20 0x55dba97d3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.299+0000 7f51f31adb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d44e0 0x55dba97dad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.299+0000 7f51f31adb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba9732400 con 0x55dba97d44e0 2023-09-30T14:42:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.299+0000 7f51f31adb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba9723bc0 con 0x55dba97d3b20 2023-09-30T14:42:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.299+0000 7f51f31adb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba97214e0 con 0x55dba97d2a10 2023-09-30T14:42:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.299+0000 7f51e4d61700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d2a10 0x55dba97d2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.299+0000 7f51e5562700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d44e0 0x55dba97dad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.300+0000 7f51e4d61700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d2a10 0x55dba97d2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51004/0 (socket says 172.21.15.32:51004) 2023-09-30T14:42:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.300+0000 7f51dffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d3b20 0x55dba97d3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.300+0000 7f51e4d61700 1 -- 172.21.15.32:0/3393398492 learned_addr learned my addr 172.21.15.32:0/3393398492 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.300+0000 7f51e4d61700 1 -- 172.21.15.32:0/3393398492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d3b20 msgr2=0x55dba97d3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.300+0000 7f51e4d61700 1 --2- 172.21.15.32:0/3393398492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d3b20 0x55dba97d3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.300+0000 7f51e4d61700 1 -- 172.21.15.32:0/3393398492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d44e0 msgr2=0x55dba97dad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.300+0000 7f51e4d61700 1 --2- 172.21.15.32:0/3393398492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d44e0 0x55dba97dad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.300+0000 7f51e4d61700 1 -- 172.21.15.32:0/3393398492 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dba977f870 con 0x55dba97d2a10 2023-09-30T14:42:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.300+0000 7f51dffff700 1 --2- 172.21.15.32:0/3393398492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d3b20 0x55dba97d3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.300+0000 7f51e4d61700 1 --2- 172.21.15.32:0/3393398492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d2a10 0x55dba97d2e30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f51d4015000 tx=0x7f51d4020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60a9ce7cd5c110e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.300+0000 7f51caffd700 1 -- 172.21.15.32:0/3393398492 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d400fe50 con 0x55dba97d2a10 2023-09-30T14:42:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.300+0000 7f51caffd700 1 -- 172.21.15.32:0/3393398492 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51d401e040 con 0x55dba97d2a10 2023-09-30T14:42:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.300+0000 7f51caffd700 1 -- 172.21.15.32:0/3393398492 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d40247d0 con 0x55dba97d2a10 2023-09-30T14:42:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.301+0000 7f51f31adb80 1 -- 172.21.15.32:0/3393398492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d2a10 msgr2=0x55dba97d2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.301+0000 7f51f31adb80 1 --2- 172.21.15.32:0/3393398492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d2a10 0x55dba97d2e30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f51d4015000 tx=0x7f51d4020000 comp rx=0 tx=0).stop 2023-09-30T14:42:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.301+0000 7f51f31adb80 1 -- 172.21.15.32:0/3393398492 shutdown_connections 2023-09-30T14:42:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.301+0000 7f51f31adb80 1 --2- 172.21.15.32:0/3393398492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d44e0 0x55dba97dad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.301+0000 7f51f31adb80 1 --2- 172.21.15.32:0/3393398492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d2a10 0x55dba97d2e30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.301+0000 7f51f31adb80 1 --2- 172.21.15.32:0/3393398492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d3b20 0x55dba97d3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.301+0000 7f51f31adb80 1 -- 172.21.15.32:0/3393398492 >> 172.21.15.32:0/3393398492 conn(0x55dba973e200 msgr2=0x55dba97c4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.301+0000 7f51f31adb80 1 -- 172.21.15.32:0/3393398492 shutdown_connections 2023-09-30T14:42:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.301+0000 7f51f31adb80 1 -- 172.21.15.32:0/3393398492 wait complete. 2023-09-30T14:42:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.301+0000 7f51f31adb80 1 -- start start 2023-09-30T14:42:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51f31adb80 1 Processor -- start 2023-09-30T14:42:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51f31adb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d2a10 0x55dba97dc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51f31adb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d3b20 0x55dba97dcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51f31adb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d44e0 0x55dba97e3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51f31adb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba971cdd0 con 0x55dba97d3b20 2023-09-30T14:42:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51f31adb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba96b96b0 con 0x55dba97d44e0 2023-09-30T14:42:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51f31adb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba96b9350 con 0x55dba97d2a10 2023-09-30T14:42:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51f31adb80 10 client.0 _ll_register_callbacks cb 0x55dba95167a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:42:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51dffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d3b20 0x55dba97dcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51dffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d3b20 0x55dba97dcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36630/0 (socket says 172.21.15.32:36630) 2023-09-30T14:42:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51dffff700 1 -- 172.21.15.32:0/703830600 learned_addr learned my addr 172.21.15.32:0/703830600 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51e4d61700 1 --2- 172.21.15.32:0/703830600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d2a10 0x55dba97dc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51e5562700 1 --2- 172.21.15.32:0/703830600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d44e0 0x55dba97e3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51dffff700 1 -- 172.21.15.32:0/703830600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d2a10 msgr2=0x55dba97dc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51dffff700 1 --2- 172.21.15.32:0/703830600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d2a10 0x55dba97dc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51dffff700 1 -- 172.21.15.32:0/703830600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d44e0 msgr2=0x55dba97e3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.302+0000 7f51dffff700 1 --2- 172.21.15.32:0/703830600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d44e0 0x55dba97e3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51dffff700 1 -- 172.21.15.32:0/703830600 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e01023e0 con 0x55dba97d3b20 2023-09-30T14:42:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51e5562700 1 --2- 172.21.15.32:0/703830600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d44e0 0x55dba97e3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:42:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51dffff700 1 --2- 172.21.15.32:0/703830600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d3b20 0x55dba97dcc10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00dda0 tx=0x7f51cc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1628ed3f5cd06c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51bffff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:42:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51bffff700 1 -- 172.21.15.32:0/703830600 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0044f0 con 0x55dba97d3b20 2023-09-30T14:42:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51bffff700 1 -- 172.21.15.32:0/703830600 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f51cc099890 con 0x55dba97d3b20 2023-09-30T14:42:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51f31adb80 1 -- 172.21.15.32:0/703830600 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dba97bd090 con 0x55dba97d3b20 2023-09-30T14:42:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51bffff700 1 -- 172.21.15.32:0/703830600 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51cc09d9e0 con 0x55dba97d3b20 2023-09-30T14:42:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51f31adb80 10 client.15894 fetch_fsmap learned FSMap version 30 2023-09-30T14:42:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51f31adb80 1 -- 172.21.15.32:0/703830600 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f51e0108b10 con 0x55dba97d3b20 2023-09-30T14:42:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51bffff700 1 -- 172.21.15.32:0/703830600 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dba97bd090 con 0x55dba97d3b20 2023-09-30T14:42:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51bffff700 1 -- 172.21.15.32:0/703830600 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f51cc098d80 con 0x55dba97d3b20 2023-09-30T14:42:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51f31adb80 10 client.15894 fetch_fsmap finished waiting for FSMap version 30 2023-09-30T14:42:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51f31adb80 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:42:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51f31adb80 20 client.15894 populate_metadata read hostname 'smithi032' 2023-09-30T14:42:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51f31adb80 1 --2- 172.21.15.32:0/703830600 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dba97e8480 0x55dba97ea8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51f31adb80 4 client.15894 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:42:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.303+0000 7f51f31adb80 1 -- 172.21.15.32:0/703830600 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dba9715db0 con 0x55dba97e8480 2023-09-30T14:42:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.304+0000 7f51e4d61700 1 --2- 172.21.15.32:0/703830600 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dba97e8480 0x55dba97ea8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.304+0000 7f51e4d61700 1 --2- 172.21.15.32:0/703830600 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dba97e8480 0x55dba97ea8a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eea5cb8a53187c77 server_cookie=1c903a858fa3dbb0 in_seq=0 out_seq=0 2023-09-30T14:42:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.304+0000 7f51bffff700 10 client.15894 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.305+0000 7f51bffff700 1 -- 172.21.15.32:0/703830600 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dba9715db0 con 0x55dba97e8480 2023-09-30T14:42:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.305+0000 7f51bffff700 10 client.15894 handle_command_reply: tid=0 2023-09-30T14:42:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.305+0000 7f51f31adb80 1 client.15894 shutdown 2023-09-30T14:42:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.305+0000 7f51f31adb80 1 -- 172.21.15.32:0/703830600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d3b20 msgr2=0x55dba97dcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.305+0000 7f51f31adb80 1 --2- 172.21.15.32:0/703830600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d3b20 0x55dba97dcc10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00dda0 tx=0x7f51cc01d000 comp rx=0 tx=0).stop 2023-09-30T14:42:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.305+0000 7f51f31adb80 1 -- 172.21.15.32:0/703830600 shutdown_connections 2023-09-30T14:42:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.305+0000 7f51f31adb80 1 --2- 172.21.15.32:0/703830600 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dba97e8480 0x55dba97ea8a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.305+0000 7f51f31adb80 1 --2- 172.21.15.32:0/703830600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d44e0 0x55dba97e3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.305+0000 7f51bffff700 0 client.15894 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.305+0000 7f51f31adb80 1 --2- 172.21.15.32:0/703830600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d2a10 0x55dba97dc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.305+0000 7f51f31adb80 1 --2- 172.21.15.32:0/703830600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d3b20 0x55dba97dcc10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.305+0000 7f51f31adb80 1 -- 172.21.15.32:0/703830600 >> 172.21.15.32:0/703830600 conn(0x55dba973e200 msgr2=0x55dba97c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.305+0000 7f51f31adb80 1 -- 172.21.15.32:0/703830600 shutdown_connections 2023-09-30T14:42:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.305+0000 7f51f31adb80 1 -- 172.21.15.32:0/703830600 wait complete. 2023-09-30T14:42:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.305+0000 7f51f31adb80 20 client.15894 trim_cache size 0 max 16384 2023-09-30T14:42:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.320+0000 7f51f31adb80 1 Processor -- start 2023-09-30T14:42:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.320+0000 7f51f31adb80 1 -- start start 2023-09-30T14:42:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51f31adb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d2a10 0x55dba97dbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51f31adb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d3b20 0x55dba97e3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51f31adb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d44e0 0x55dba97db340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51f31adb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba959d8c0 con 0x55dba97d2a10 2023-09-30T14:42:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51f31adb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba97c5cb0 con 0x55dba97d3b20 2023-09-30T14:42:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51f31adb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba97d48c0 con 0x55dba97d44e0 2023-09-30T14:42:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51e4d61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d2a10 0x55dba97dbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51e4d61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d2a10 0x55dba97dbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36636/0 (socket says 172.21.15.32:36636) 2023-09-30T14:42:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51e4d61700 1 -- 172.21.15.32:0/3233459568 learned_addr learned my addr 172.21.15.32:0/3233459568 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51e5562700 1 --2- 172.21.15.32:0/3233459568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d44e0 0x55dba97db340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51dffff700 1 --2- 172.21.15.32:0/3233459568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d3b20 0x55dba97e3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51e4d61700 1 -- 172.21.15.32:0/3233459568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d44e0 msgr2=0x55dba97db340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51e4d61700 1 --2- 172.21.15.32:0/3233459568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d44e0 0x55dba97db340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51e4d61700 1 -- 172.21.15.32:0/3233459568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d3b20 msgr2=0x55dba97e3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51e4d61700 1 --2- 172.21.15.32:0/3233459568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d3b20 0x55dba97e3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51e4d61700 1 -- 172.21.15.32:0/3233459568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dba9547d80 con 0x55dba97d2a10 2023-09-30T14:42:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51dffff700 1 --2- 172.21.15.32:0/3233459568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d3b20 0x55dba97e3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:42:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51e4d61700 1 --2- 172.21.15.32:0/3233459568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d2a10 0x55dba97dbd80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f51d400fb20 tx=0x7f51d4027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da80b4ae05acb4ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51cb7fe700 1 -- 172.21.15.32:0/3233459568 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d403ed40 con 0x55dba97d2a10 2023-09-30T14:42:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51cb7fe700 1 -- 172.21.15.32:0/3233459568 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51d4004830 con 0x55dba97d2a10 2023-09-30T14:42:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.321+0000 7f51cb7fe700 1 -- 172.21.15.32:0/3233459568 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4005280 con 0x55dba97d2a10 2023-09-30T14:42:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.322+0000 7f51f31adb80 1 -- 172.21.15.32:0/3233459568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d2a10 msgr2=0x55dba97dbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.322+0000 7f51f31adb80 1 --2- 172.21.15.32:0/3233459568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d2a10 0x55dba97dbd80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f51d400fb20 tx=0x7f51d4027040 comp rx=0 tx=0).stop 2023-09-30T14:42:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.322+0000 7f51f31adb80 1 -- 172.21.15.32:0/3233459568 shutdown_connections 2023-09-30T14:42:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.322+0000 7f51f31adb80 1 --2- 172.21.15.32:0/3233459568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d44e0 0x55dba97db340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.322+0000 7f51f31adb80 1 --2- 172.21.15.32:0/3233459568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d2a10 0x55dba97dbd80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.322+0000 7f51f31adb80 1 --2- 172.21.15.32:0/3233459568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d3b20 0x55dba97e3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.322+0000 7f51f31adb80 1 -- 172.21.15.32:0/3233459568 >> 172.21.15.32:0/3233459568 conn(0x55dba973e200 msgr2=0x55dba97c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.322+0000 7f51f31adb80 1 -- 172.21.15.32:0/3233459568 shutdown_connections 2023-09-30T14:42:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.322+0000 7f51f31adb80 1 -- 172.21.15.32:0/3233459568 wait complete. 2023-09-30T14:42:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.322+0000 7f51f31adb80 1 -- start start 2023-09-30T14:42:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.322+0000 7f51f31adb80 1 Processor -- start 2023-09-30T14:42:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51f31adb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d2a10 0x55dba97e1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51f31adb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d3b20 0x55dba97e1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51f31adb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d44e0 0x55dba97e2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51f31adb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba97e2eb0 con 0x55dba97d44e0 2023-09-30T14:42:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51f31adb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba97e3020 con 0x55dba97d2a10 2023-09-30T14:42:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51f31adb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba97e31e0 con 0x55dba97d3b20 2023-09-30T14:42:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51f31adb80 10 client.0 _ll_register_callbacks cb 0x7f51cc0993d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:42:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51e5562700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d44e0 0x55dba97e2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51e4d61700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d2a10 0x55dba97e1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51dffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d3b20 0x55dba97e1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51e5562700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d44e0 0x55dba97e2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36646/0 (socket says 172.21.15.32:36646) 2023-09-30T14:42:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51e4d61700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d2a10 0x55dba97e1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48118/0 (socket says 172.21.15.32:48118) 2023-09-30T14:42:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51e5562700 1 -- 172.21.15.32:0/2993625921 learned_addr learned my addr 172.21.15.32:0/2993625921 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51dffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d3b20 0x55dba97e1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51026/0 (socket says 172.21.15.32:51026) 2023-09-30T14:42:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51e5562700 1 -- 172.21.15.32:0/2993625921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d3b20 msgr2=0x55dba97e1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51e5562700 1 --2- 172.21.15.32:0/2993625921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d3b20 0x55dba97e1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51e5562700 1 -- 172.21.15.32:0/2993625921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d2a10 msgr2=0x55dba97e1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51e5562700 1 --2- 172.21.15.32:0/2993625921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d2a10 0x55dba97e1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51e5562700 1 -- 172.21.15.32:0/2993625921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dba977f870 con 0x55dba97d44e0 2023-09-30T14:42:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51dffff700 1 --2- 172.21.15.32:0/2993625921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d3b20 0x55dba97e1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51e5562700 1 --2- 172.21.15.32:0/2993625921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d44e0 0x55dba97e2970 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009c10 tx=0x7f51d8008730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e808935af5e5e973 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.323+0000 7f51bffff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:42:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.324+0000 7f51bffff700 1 -- 172.21.15.32:0/2993625921 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8015030 con 0x55dba97d44e0 2023-09-30T14:42:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.324+0000 7f51bffff700 1 -- 172.21.15.32:0/2993625921 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f51d8008e60 con 0x55dba97d44e0 2023-09-30T14:42:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.324+0000 7f51f31adb80 1 -- 172.21.15.32:0/2993625921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51d401e040 con 0x55dba97d44e0 2023-09-30T14:42:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.324+0000 7f51bffff700 1 -- 172.21.15.32:0/2993625921 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51d800e070 con 0x55dba97d44e0 2023-09-30T14:42:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.324+0000 7f51f31adb80 10 client.15900 fetch_fsmap learned FSMap version 30 2023-09-30T14:42:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.324+0000 7f51f31adb80 1 -- 172.21.15.32:0/2993625921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55dba9530c00 con 0x55dba97d44e0 2023-09-30T14:42:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.324+0000 7f51bffff700 1 -- 172.21.15.32:0/2993625921 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51d802b020 con 0x55dba97d44e0 2023-09-30T14:42:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.324+0000 7f51bffff700 1 -- 172.21.15.32:0/2993625921 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f51d8025450 con 0x55dba97d44e0 2023-09-30T14:42:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.324+0000 7f51f31adb80 10 client.15900 fetch_fsmap finished waiting for FSMap version 30 2023-09-30T14:42:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.324+0000 7f51f31adb80 10 client.15900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:42:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.324+0000 7f51f31adb80 20 client.15900 populate_metadata read hostname 'smithi032' 2023-09-30T14:42:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.324+0000 7f51f31adb80 1 --2- 172.21.15.32:0/2993625921 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dba983cdb0 0x55dba983f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.324+0000 7f51f31adb80 4 client.15900 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:42:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.324+0000 7f51f31adb80 1 -- 172.21.15.32:0/2993625921 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dba9715ad0 con 0x55dba983cdb0 2023-09-30T14:42:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.324+0000 7f51e4d61700 1 --2- 172.21.15.32:0/2993625921 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dba983cdb0 0x55dba983f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.325+0000 7f51e4d61700 1 --2- 172.21.15.32:0/2993625921 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dba983cdb0 0x55dba983f1d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c820953e85d5f3c server_cookie=f94ea2a5cdf9c7d9 in_seq=0 out_seq=0 2023-09-30T14:42:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.325+0000 7f51bffff700 10 client.15900 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.372+0000 7f51bffff700 1 -- 172.21.15.32:0/2993625921 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dba9715ad0 con 0x55dba983cdb0 2023-09-30T14:42:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.372+0000 7f51bffff700 10 client.15900 handle_command_reply: tid=0 2023-09-30T14:42:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.372+0000 7f51f31adb80 1 client.15900 shutdown 2023-09-30T14:42:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.372+0000 7f51f31adb80 1 -- 172.21.15.32:0/2993625921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d44e0 msgr2=0x55dba97e2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.372+0000 7f51f31adb80 1 --2- 172.21.15.32:0/2993625921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d44e0 0x55dba97e2970 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009c10 tx=0x7f51d8008730 comp rx=0 tx=0).stop 2023-09-30T14:42:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.372+0000 7f51f31adb80 1 -- 172.21.15.32:0/2993625921 shutdown_connections 2023-09-30T14:42:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.372+0000 7f51f31adb80 1 --2- 172.21.15.32:0/2993625921 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dba983cdb0 0x55dba983f1d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.372+0000 7f51f31adb80 1 --2- 172.21.15.32:0/2993625921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dba97d44e0 0x55dba97e2970 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.372+0000 7f51f31adb80 1 --2- 172.21.15.32:0/2993625921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dba97d2a10 0x55dba97e1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.372+0000 7f51bffff700 0 client.15900 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.372+0000 7f51f31adb80 1 --2- 172.21.15.32:0/2993625921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dba97d3b20 0x55dba97e1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.372+0000 7f51f31adb80 1 -- 172.21.15.32:0/2993625921 >> 172.21.15.32:0/2993625921 conn(0x55dba973e200 msgr2=0x55dba97e4480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.372+0000 7f51f31adb80 1 -- 172.21.15.32:0/2993625921 shutdown_connections 2023-09-30T14:42:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.372+0000 7f51f31adb80 1 -- 172.21.15.32:0/2993625921 wait complete. 2023-09-30T14:42:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.373+0000 7f51f31adb80 20 client.15900 trim_cache size 0 max 16384 2023-09-30T14:42:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:42:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:42:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:42:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:42:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.373+0000 7f51e6fc5700 1 -- 172.21.15.32:0/2124886873 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51d00a0fa0 msgr2=0x7f51d00a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.373+0000 7f51e6fc5700 1 --2- 172.21.15.32:0/2124886873 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51d00a0fa0 0x7f51d00a3460 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f51d400b2e0 tx=0x7f51d4005a90 comp rx=0 tx=0).stop 2023-09-30T14:42:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.373+0000 7f51e6fc5700 1 -- 172.21.15.32:0/2124886873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e0100f90 msgr2=0x7f51e0197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.373+0000 7f51e6fc5700 1 --2- 172.21.15.32:0/2124886873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e0100f90 0x7f51e0197250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00d740 tx=0x7f51cc00da50 comp rx=0 tx=0).stop 2023-09-30T14:42:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.373+0000 7f51e6fc5700 1 -- 172.21.15.32:0/2124886873 shutdown_connections 2023-09-30T14:42:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.373+0000 7f51e6fc5700 1 --2- 172.21.15.32:0/2124886873 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51d00a0fa0 0x7f51d00a3460 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.373+0000 7f51e6fc5700 1 --2- 172.21.15.32:0/2124886873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51e0101950 0x7f51e0197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.373+0000 7f51e6fc5700 1 --2- 172.21.15.32:0/2124886873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51e0100280 0x7f51e0196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.373+0000 7f51e6fc5700 1 --2- 172.21.15.32:0/2124886873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e0100f90 0x7f51e0197250 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.373+0000 7f51e6fc5700 1 -- 172.21.15.32:0/2124886873 >> 172.21.15.32:0/2124886873 conn(0x7f51e00faa30 msgr2=0x7f51e00fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.373+0000 7f51e6fc5700 1 -- 172.21.15.32:0/2124886873 shutdown_connections 2023-09-30T14:42:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:40.373+0000 7f51e6fc5700 1 -- 172.21.15.32:0/2124886873 wait complete. 2023-09-30T14:42:40.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 423240 10.88 MB/sec execute 74 sec latency 15.043 ms 2023-09-30T14:42:41.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:41 smithi032 ceph-mon[98592]: pgmap v419: 97 pgs: 97 active+clean; 158 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.7 MiB/s wr, 264 op/s 2023-09-30T14:42:41.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 425833 10.95 MB/sec execute 75 sec latency 40.891 ms 2023-09-30T14:42:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:41 smithi156 ceph-mon[103579]: pgmap v419: 97 pgs: 97 active+clean; 158 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.7 MiB/s wr, 264 op/s 2023-09-30T14:42:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:41 smithi148 ceph-mon[102355]: pgmap v419: 97 pgs: 97 active+clean; 158 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.7 MiB/s wr, 264 op/s 2023-09-30T14:42:42.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 427365 10.89 MB/sec execute 76 sec latency 14.811 ms 2023-09-30T14:42:43.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:43 smithi032 ceph-mon[98592]: pgmap v420: 97 pgs: 97 active+clean; 163 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 343 op/s 2023-09-30T14:42:43.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 429681 10.93 MB/sec execute 77 sec latency 16.142 ms 2023-09-30T14:42:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:43 smithi156 ceph-mon[103579]: pgmap v420: 97 pgs: 97 active+clean; 163 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 343 op/s 2023-09-30T14:42:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:43 smithi148 ceph-mon[102355]: pgmap v420: 97 pgs: 97 active+clean; 163 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 343 op/s 2023-09-30T14:42:44.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 432416 10.96 MB/sec execute 78 sec latency 14.340 ms 2023-09-30T14:42:45.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:45 smithi032 ceph-mon[98592]: pgmap v421: 97 pgs: 97 active+clean; 160 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 2.8 MiB/s wr, 294 op/s 2023-09-30T14:42:45.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 433875 10.92 MB/sec execute 79 sec latency 12.713 ms 2023-09-30T14:42:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:45 smithi156 ceph-mon[103579]: pgmap v421: 97 pgs: 97 active+clean; 160 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 2.8 MiB/s wr, 294 op/s 2023-09-30T14:42:45.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:45 smithi148 ceph-mon[102355]: pgmap v421: 97 pgs: 97 active+clean; 160 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 2.8 MiB/s wr, 294 op/s 2023-09-30T14:42:46.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 436533 10.99 MB/sec execute 80 sec latency 35.181 ms 2023-09-30T14:42:47.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:47 smithi032 ceph-mon[98592]: pgmap v422: 97 pgs: 97 active+clean; 158 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 318 op/s 2023-09-30T14:42:48.797 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:47 smithi148 ceph-mon[102355]: pgmap v422: 97 pgs: 97 active+clean; 158 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 318 op/s 2023-09-30T14:42:48.797 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:47 smithi156 ceph-mon[103579]: pgmap v422: 97 pgs: 97 active+clean; 158 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 318 op/s 2023-09-30T14:42:48.797 INFO:tasks.workunit.client.0.smithi032.stdout: 1 437951 10.93 MB/sec execute 81 sec latency 16.334 ms 2023-09-30T14:42:48.797 INFO:tasks.workunit.client.0.smithi032.stdout: 1 440274 10.96 MB/sec execute 82 sec latency 11.342 ms 2023-09-30T14:42:49.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:49 smithi032 ceph-mon[98592]: pgmap v423: 97 pgs: 97 active+clean; 158 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 2.9 MiB/s wr, 315 op/s 2023-09-30T14:42:49.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 442457 10.93 MB/sec execute 83 sec latency 14.877 ms 2023-09-30T14:42:49.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:49 smithi156 ceph-mon[103579]: pgmap v423: 97 pgs: 97 active+clean; 158 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 2.9 MiB/s wr, 315 op/s 2023-09-30T14:42:49.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:49 smithi148 ceph-mon[102355]: pgmap v423: 97 pgs: 97 active+clean; 158 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 2.9 MiB/s wr, 315 op/s 2023-09-30T14:42:50.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 443823 10.93 MB/sec execute 84 sec latency 95.109 ms 2023-09-30T14:42:51.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:51 smithi032 ceph-mon[98592]: pgmap v424: 97 pgs: 97 active+clean; 160 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 2.6 MiB/s wr, 307 op/s 2023-09-30T14:42:51.698 INFO:tasks.workunit.client.0.smithi032.stdout: 1 446006 10.90 MB/sec execute 85 sec latency 38.518 ms 2023-09-30T14:42:51.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:51 smithi156 ceph-mon[103579]: pgmap v424: 97 pgs: 97 active+clean; 160 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 2.6 MiB/s wr, 307 op/s 2023-09-30T14:42:51.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:51 smithi148 ceph-mon[102355]: pgmap v424: 97 pgs: 97 active+clean; 160 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 2.6 MiB/s wr, 307 op/s 2023-09-30T14:42:52.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 447566 10.90 MB/sec execute 86 sec latency 23.833 ms 2023-09-30T14:42:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.490+0000 7ffa57542700 1 -- 172.21.15.32:0/2537687639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa50101940 msgr2=0x7ffa50103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.490+0000 7ffa57542700 1 --2- 172.21.15.32:0/2537687639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa50101940 0x7ffa50103d30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009990 tx=0x7ffa4c009ca0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.490+0000 7ffa57542700 1 -- 172.21.15.32:0/2537687639 shutdown_connections 2023-09-30T14:42:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.490+0000 7ffa57542700 1 --2- 172.21.15.32:0/2537687639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa50106ba0 0x7ffa50108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.490+0000 7ffa57542700 1 --2- 172.21.15.32:0/2537687639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa50101940 0x7ffa50103d30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.490+0000 7ffa57542700 1 --2- 172.21.15.32:0/2537687639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa50104270 0x7ffa50106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.490+0000 7ffa57542700 1 -- 172.21.15.32:0/2537687639 >> 172.21.15.32:0/2537687639 conn(0x7ffa500fb2b0 msgr2=0x7ffa500fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.490+0000 7ffa57542700 1 -- 172.21.15.32:0/2537687639 shutdown_connections 2023-09-30T14:42:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.490+0000 7ffa57542700 1 -- 172.21.15.32:0/2537687639 wait complete. 2023-09-30T14:42:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.491+0000 7ffa57542700 1 Processor -- start 2023-09-30T14:42:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.491+0000 7ffa57542700 1 -- start start 2023-09-30T14:42:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.491+0000 7ffa57542700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa50101940 0x7ffa50196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.491+0000 7ffa57542700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa50104270 0x7ffa50197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.491+0000 7ffa57542700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa50106ba0 0x7ffa50197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.491+0000 7ffa57542700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa50198610 con 0x7ffa50106ba0 2023-09-30T14:42:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.491+0000 7ffa57542700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa50198780 con 0x7ffa50101940 2023-09-30T14:42:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.491+0000 7ffa57542700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa50198940 con 0x7ffa50104270 2023-09-30T14:42:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.491+0000 7ffa55adf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa50106ba0 0x7ffa50197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.491+0000 7ffa55adf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa50106ba0 0x7ffa50197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36658/0 (socket says 172.21.15.32:36658) 2023-09-30T14:42:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.491+0000 7ffa55adf700 1 -- 172.21.15.32:0/1686396631 learned_addr learned my addr 172.21.15.32:0/1686396631 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.491+0000 7ffa552de700 1 --2- 172.21.15.32:0/1686396631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa50101940 0x7ffa50196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.491+0000 7ffa54add700 1 --2- 172.21.15.32:0/1686396631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa50104270 0x7ffa50197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.492+0000 7ffa55adf700 1 -- 172.21.15.32:0/1686396631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa50104270 msgr2=0x7ffa50197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.492+0000 7ffa55adf700 1 --2- 172.21.15.32:0/1686396631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa50104270 0x7ffa50197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.492+0000 7ffa55adf700 1 -- 172.21.15.32:0/1686396631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa50101940 msgr2=0x7ffa50196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.492+0000 7ffa55adf700 1 --2- 172.21.15.32:0/1686396631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa50101940 0x7ffa50196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.492+0000 7ffa55adf700 1 -- 172.21.15.32:0/1686396631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c0095f0 con 0x7ffa50106ba0 2023-09-30T14:42:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.492+0000 7ffa55adf700 1 --2- 172.21.15.32:0/1686396631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa50106ba0 0x7ffa50197fb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800ef30 tx=0x7ffa4800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.492+0000 7ffa467fc700 1 -- 172.21.15.32:0/1686396631 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa48018070 con 0x7ffa50106ba0 2023-09-30T14:42:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.492+0000 7ffa467fc700 1 -- 172.21.15.32:0/1686396631 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa4800f040 con 0x7ffa50106ba0 2023-09-30T14:42:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.492+0000 7ffa467fc700 1 -- 172.21.15.32:0/1686396631 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4800cd00 con 0x7ffa50106ba0 2023-09-30T14:42:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.492+0000 7ffa57542700 1 -- 172.21.15.32:0/1686396631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa50198bc0 con 0x7ffa50106ba0 2023-09-30T14:42:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.492+0000 7ffa57542700 1 -- 172.21.15.32:0/1686396631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5019f3e0 con 0x7ffa50106ba0 2023-09-30T14:42:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.494+0000 7ffa467fc700 1 -- 172.21.15.32:0/1686396631 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7ffa48003a00 con 0x7ffa50106ba0 2023-09-30T14:42:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.495+0000 7ffa467fc700 1 --2- 172.21.15.32:0/1686396631 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffa3c0a0fa0 0x7ffa3c0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.495+0000 7ffa552de700 1 --2- 172.21.15.32:0/1686396631 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffa3c0a0fa0 0x7ffa3c0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.495+0000 7ffa467fc700 1 -- 172.21.15.32:0/1686396631 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffa480cc280 con 0x7ffa50106ba0 2023-09-30T14:42:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.495+0000 7ffa552de700 1 --2- 172.21.15.32:0/1686396631 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffa3c0a0fa0 0x7ffa3c0a3460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00b2e0 tx=0x7ffa4c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.496+0000 7ffa6372ab80 1 Processor -- start 2023-09-30T14:42:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.496+0000 7ffa6372ab80 1 -- start start 2023-09-30T14:42:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.496+0000 7ffa6372ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadd610 0x563cfeadda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.496+0000 7ffa6372ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeade720 0x563cfeadeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.496+0000 7ffa6372ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeadf0e0 0x563cfeae5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.496+0000 7ffa6372ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfea41400 con 0x563cfeadd610 2023-09-30T14:42:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.496+0000 7ffa6372ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfea32bc0 con 0x563cfeadf0e0 2023-09-30T14:42:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.496+0000 7ffa6372ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfea304e0 con 0x563cfeade720 2023-09-30T14:42:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.496+0000 7ffa552de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadd610 0x563cfeadda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.496+0000 7ffa552de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadd610 0x563cfeadda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36662/0 (socket says 172.21.15.32:36662) 2023-09-30T14:42:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.496+0000 7ffa552de700 1 -- 172.21.15.32:0/3025634492 learned_addr learned my addr 172.21.15.32:0/3025634492 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.496+0000 7ffa55adf700 1 --2- 172.21.15.32:0/3025634492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeadf0e0 0x563cfeae5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.496+0000 7ffa54add700 1 --2- 172.21.15.32:0/3025634492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeade720 0x563cfeadeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.497+0000 7ffa55adf700 1 -- 172.21.15.32:0/3025634492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeade720 msgr2=0x563cfeadeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.497+0000 7ffa55adf700 1 --2- 172.21.15.32:0/3025634492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeade720 0x563cfeadeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.497+0000 7ffa55adf700 1 -- 172.21.15.32:0/3025634492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadd610 msgr2=0x563cfeadda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.497+0000 7ffa55adf700 1 --2- 172.21.15.32:0/3025634492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadd610 0x563cfeadda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.497+0000 7ffa55adf700 1 -- 172.21.15.32:0/3025634492 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cfea8e870 con 0x563cfeadf0e0 2023-09-30T14:42:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.497+0000 7ffa55adf700 1 --2- 172.21.15.32:0/3025634492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeadf0e0 0x563cfeae5970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800c6f0 tx=0x7ffa48009450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9be9c55bb1bfe94 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.497+0000 7ffa3b7fe700 1 -- 172.21.15.32:0/3025634492 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa48004ad0 con 0x563cfeadf0e0 2023-09-30T14:42:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.497+0000 7ffa3b7fe700 1 -- 172.21.15.32:0/3025634492 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa4801b320 con 0x563cfeadf0e0 2023-09-30T14:42:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.497+0000 7ffa3b7fe700 1 -- 172.21.15.32:0/3025634492 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa480a1b70 con 0x563cfeadf0e0 2023-09-30T14:42:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.498+0000 7ffa6372ab80 1 -- 172.21.15.32:0/3025634492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeadf0e0 msgr2=0x563cfeae5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.498+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/3025634492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeadf0e0 0x563cfeae5970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800c6f0 tx=0x7ffa48009450 comp rx=0 tx=0).stop 2023-09-30T14:42:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.498+0000 7ffa6372ab80 1 -- 172.21.15.32:0/3025634492 shutdown_connections 2023-09-30T14:42:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.498+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/3025634492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeadf0e0 0x563cfeae5970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.498+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/3025634492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadd610 0x563cfeadda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.498+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/3025634492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeade720 0x563cfeadeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.498+0000 7ffa6372ab80 1 -- 172.21.15.32:0/3025634492 >> 172.21.15.32:0/3025634492 conn(0x563cfea4d200 msgr2=0x563cfead3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.498+0000 7ffa6372ab80 1 -- 172.21.15.32:0/3025634492 shutdown_connections 2023-09-30T14:42:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.498+0000 7ffa6372ab80 1 -- 172.21.15.32:0/3025634492 wait complete. 2023-09-30T14:42:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.498+0000 7ffa6372ab80 1 -- start start 2023-09-30T14:42:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.499+0000 7ffa6372ab80 1 Processor -- start 2023-09-30T14:42:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.499+0000 7ffa6372ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeadd610 0x563cfeae7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.499+0000 7ffa6372ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeade720 0x563cfeae77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.499+0000 7ffa6372ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadf0e0 0x563cfeaee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.499+0000 7ffa6372ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfea2bdd0 con 0x563cfeadf0e0 2023-09-30T14:42:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.499+0000 7ffa6372ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfe9c86b0 con 0x563cfeade720 2023-09-30T14:42:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.499+0000 7ffa6372ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfe9c8350 con 0x563cfeadd610 2023-09-30T14:42:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.499+0000 7ffa6372ab80 10 client.0 _ll_register_callbacks cb 0x563cfe8257a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:42:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.499+0000 7ffa54add700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeade720 0x563cfeae77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.499+0000 7ffa55adf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadf0e0 0x563cfeaee7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.499+0000 7ffa54add700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeade720 0x563cfeae77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48146/0 (socket says 172.21.15.32:48146) 2023-09-30T14:42:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.499+0000 7ffa552de700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeadd610 0x563cfeae7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.499+0000 7ffa55adf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadf0e0 0x563cfeaee7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36672/0 (socket says 172.21.15.32:36672) 2023-09-30T14:42:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.499+0000 7ffa54add700 1 -- 172.21.15.32:0/2853157869 learned_addr learned my addr 172.21.15.32:0/2853157869 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa55adf700 1 -- 172.21.15.32:0/2853157869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeadd610 msgr2=0x563cfeae7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa55adf700 1 --2- 172.21.15.32:0/2853157869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeadd610 0x563cfeae7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa55adf700 1 -- 172.21.15.32:0/2853157869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeade720 msgr2=0x563cfeae77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa55adf700 1 --2- 172.21.15.32:0/2853157869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeade720 0x563cfeae77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa55adf700 1 -- 172.21.15.32:0/2853157869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5019f3e0 con 0x563cfeadf0e0 2023-09-30T14:42:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa54add700 1 --2- 172.21.15.32:0/2853157869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeade720 0x563cfeae77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa55adf700 1 --2- 172.21.15.32:0/2853157869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadf0e0 0x563cfeaee7f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ffa480999a0 tx=0x7ffa4800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9538b9d8a981e6b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:42:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa38ff9700 1 -- 172.21.15.32:0/2853157869 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa48099c40 con 0x563cfeadf0e0 2023-09-30T14:42:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa38ff9700 1 -- 172.21.15.32:0/2853157869 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffa480997e0 con 0x563cfeadf0e0 2023-09-30T14:42:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa6372ab80 1 -- 172.21.15.32:0/2853157869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cfeacc090 con 0x563cfeadf0e0 2023-09-30T14:42:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa38ff9700 1 -- 172.21.15.32:0/2853157869 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa48065250 con 0x563cfeadf0e0 2023-09-30T14:42:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa6372ab80 10 client.15912 fetch_fsmap learned FSMap version 30 2023-09-30T14:42:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa6372ab80 1 -- 172.21.15.32:0/2853157869 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563cfe856d80 con 0x563cfeadf0e0 2023-09-30T14:42:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa38ff9700 1 -- 172.21.15.32:0/2853157869 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa4807e080 con 0x563cfeadf0e0 2023-09-30T14:42:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa38ff9700 1 -- 172.21.15.32:0/2853157869 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffa480a1650 con 0x563cfeadf0e0 2023-09-30T14:42:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa6372ab80 10 client.15912 fetch_fsmap finished waiting for FSMap version 30 2023-09-30T14:42:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa6372ab80 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:42:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa6372ab80 20 client.15912 populate_metadata read hostname 'smithi032' 2023-09-30T14:42:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/2853157869 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563cfeaf2c40 0x563cfeaf5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa6372ab80 4 client.15912 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:42:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.500+0000 7ffa6372ab80 1 -- 172.21.15.32:0/2853157869 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cfea24db0 con 0x563cfeaf2c40 2023-09-30T14:42:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.501+0000 7ffa552de700 1 --2- 172.21.15.32:0/2853157869 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563cfeaf2c40 0x563cfeaf5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.501+0000 7ffa38ff9700 10 client.15912 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.501+0000 7ffa552de700 1 --2- 172.21.15.32:0/2853157869 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563cfeaf2c40 0x563cfeaf5060 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91450f5154cf829d server_cookie=7dec5672d68f9fa7 in_seq=0 out_seq=0 2023-09-30T14:42:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.502+0000 7ffa38ff9700 1 -- 172.21.15.32:0/2853157869 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563cfea24db0 con 0x563cfeaf2c40 2023-09-30T14:42:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.502+0000 7ffa38ff9700 10 client.15912 handle_command_reply: tid=0 2023-09-30T14:42:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.502+0000 7ffa6372ab80 1 client.15912 shutdown 2023-09-30T14:42:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.502+0000 7ffa6372ab80 1 -- 172.21.15.32:0/2853157869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadf0e0 msgr2=0x563cfeaee7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.502+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/2853157869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadf0e0 0x563cfeaee7f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ffa480999a0 tx=0x7ffa4800cf40 comp rx=0 tx=0).stop 2023-09-30T14:42:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.502+0000 7ffa6372ab80 1 -- 172.21.15.32:0/2853157869 shutdown_connections 2023-09-30T14:42:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.502+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/2853157869 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563cfeaf2c40 0x563cfeaf5060 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.502+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/2853157869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadf0e0 0x563cfeaee7f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.502+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/2853157869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeadd610 0x563cfeae7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.502+0000 7ffa38ff9700 0 client.15912 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.502+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/2853157869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeade720 0x563cfeae77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.502+0000 7ffa6372ab80 1 -- 172.21.15.32:0/2853157869 >> 172.21.15.32:0/2853157869 conn(0x563cfea4d200 msgr2=0x563cfead3590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.502+0000 7ffa6372ab80 1 -- 172.21.15.32:0/2853157869 shutdown_connections 2023-09-30T14:42:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.502+0000 7ffa6372ab80 1 -- 172.21.15.32:0/2853157869 wait complete. 2023-09-30T14:42:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.502+0000 7ffa6372ab80 20 client.15912 trim_cache size 0 max 16384 2023-09-30T14:42:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.517+0000 7ffa6372ab80 1 Processor -- start 2023-09-30T14:42:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.517+0000 7ffa6372ab80 1 -- start start 2023-09-30T14:42:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.517+0000 7ffa6372ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadd610 0x563cfeae6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.517+0000 7ffa6372ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeade720 0x563cfeaee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.517+0000 7ffa6372ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeadf0e0 0x563cfeae5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.517+0000 7ffa6372ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfe8ac8c0 con 0x563cfeadd610 2023-09-30T14:42:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.517+0000 7ffa6372ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfeae52d0 con 0x563cfeadf0e0 2023-09-30T14:42:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.517+0000 7ffa6372ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfeae2be0 con 0x563cfeade720 2023-09-30T14:42:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.517+0000 7ffa552de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadd610 0x563cfeae6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.517+0000 7ffa552de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadd610 0x563cfeae6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36676/0 (socket says 172.21.15.32:36676) 2023-09-30T14:42:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.517+0000 7ffa552de700 1 -- 172.21.15.32:0/1314739833 learned_addr learned my addr 172.21.15.32:0/1314739833 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.518+0000 7ffa55adf700 1 --2- 172.21.15.32:0/1314739833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeadf0e0 0x563cfeae5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.518+0000 7ffa552de700 1 -- 172.21.15.32:0/1314739833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeade720 msgr2=0x563cfeaee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.518+0000 7ffa54add700 1 --2- 172.21.15.32:0/1314739833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeade720 0x563cfeaee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.518+0000 7ffa552de700 1 --2- 172.21.15.32:0/1314739833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeade720 0x563cfeaee7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.518+0000 7ffa552de700 1 -- 172.21.15.32:0/1314739833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeadf0e0 msgr2=0x563cfeae5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.518+0000 7ffa552de700 1 --2- 172.21.15.32:0/1314739833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeadf0e0 0x563cfeae5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.518+0000 7ffa552de700 1 -- 172.21.15.32:0/1314739833 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cfe83fc00 con 0x563cfeadd610 2023-09-30T14:42:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.518+0000 7ffa55adf700 1 --2- 172.21.15.32:0/1314739833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeadf0e0 0x563cfeae5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:42:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.518+0000 7ffa54add700 1 --2- 172.21.15.32:0/1314739833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeade720 0x563cfeaee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:42:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.518+0000 7ffa552de700 1 --2- 172.21.15.32:0/1314739833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadd610 0x563cfeae6920 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00b960 tx=0x7ffa4c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b68a8103a7dc84 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.518+0000 7ffa3bfff700 1 -- 172.21.15.32:0/1314739833 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c02d050 con 0x563cfeadd610 2023-09-30T14:42:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.518+0000 7ffa3bfff700 1 -- 172.21.15.32:0/1314739833 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa4c02e040 con 0x563cfeadd610 2023-09-30T14:42:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.518+0000 7ffa3bfff700 1 -- 172.21.15.32:0/1314739833 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c004b40 con 0x563cfeadd610 2023-09-30T14:42:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 -- 172.21.15.32:0/1314739833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadd610 msgr2=0x563cfeae6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/1314739833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadd610 0x563cfeae6920 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00b960 tx=0x7ffa4c020000 comp rx=0 tx=0).stop 2023-09-30T14:42:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 -- 172.21.15.32:0/1314739833 shutdown_connections 2023-09-30T14:42:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/1314739833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeadf0e0 0x563cfeae5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/1314739833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadd610 0x563cfeae6920 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/1314739833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeade720 0x563cfeaee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 -- 172.21.15.32:0/1314739833 >> 172.21.15.32:0/1314739833 conn(0x563cfea4d200 msgr2=0x563cfead3590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 -- 172.21.15.32:0/1314739833 shutdown_connections 2023-09-30T14:42:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 -- 172.21.15.32:0/1314739833 wait complete. 2023-09-30T14:42:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 -- start start 2023-09-30T14:42:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 Processor -- start 2023-09-30T14:42:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeadd610 0x563cfeaec250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeade720 0x563cfeaec790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadf0e0 0x563cfeaed700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfeaedc40 con 0x563cfeadf0e0 2023-09-30T14:42:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfeaeddb0 con 0x563cfeade720 2023-09-30T14:42:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.519+0000 7ffa6372ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfeaedf70 con 0x563cfeadd610 2023-09-30T14:42:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa6372ab80 10 client.0 _ll_register_callbacks cb 0x7ffa4c00b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:42:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa55adf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadf0e0 0x563cfeaed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa55adf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadf0e0 0x563cfeaed700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36686/0 (socket says 172.21.15.32:36686) 2023-09-30T14:42:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa55adf700 1 -- 172.21.15.32:0/3876101986 learned_addr learned my addr 172.21.15.32:0/3876101986 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:42:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa54add700 1 --2- 172.21.15.32:0/3876101986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeade720 0x563cfeaec790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa552de700 1 --2- 172.21.15.32:0/3876101986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeadd610 0x563cfeaec250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa55adf700 1 -- 172.21.15.32:0/3876101986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeadd610 msgr2=0x563cfeaec250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa55adf700 1 --2- 172.21.15.32:0/3876101986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeadd610 0x563cfeaec250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa55adf700 1 -- 172.21.15.32:0/3876101986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeade720 msgr2=0x563cfeaec790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:42:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa55adf700 1 --2- 172.21.15.32:0/3876101986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeade720 0x563cfeaec790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa55adf700 1 -- 172.21.15.32:0/3876101986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa50109ba0 con 0x563cfeadf0e0 2023-09-30T14:42:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa54add700 1 --2- 172.21.15.32:0/3876101986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeade720 0x563cfeaec790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:42:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa552de700 1 --2- 172.21.15.32:0/3876101986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeadd610 0x563cfeaec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:42:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa55adf700 1 --2- 172.21.15.32:0/3876101986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadf0e0 0x563cfeaed700 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffa48099770 tx=0x7ffa480a1880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6af239a4e03138 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:42:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:42:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa38ff9700 1 -- 172.21.15.32:0/3876101986 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa480997e0 con 0x563cfeadf0e0 2023-09-30T14:42:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa38ff9700 1 -- 172.21.15.32:0/3876101986 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffa48004ad0 con 0x563cfeadf0e0 2023-09-30T14:42:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.520+0000 7ffa6372ab80 1 -- 172.21.15.32:0/3876101986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa4801b320 con 0x563cfeadf0e0 2023-09-30T14:42:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.521+0000 7ffa38ff9700 1 -- 172.21.15.32:0/3876101986 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa48095810 con 0x563cfeadf0e0 2023-09-30T14:42:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.521+0000 7ffa6372ab80 10 client.15918 fetch_fsmap learned FSMap version 30 2023-09-30T14:42:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.521+0000 7ffa6372ab80 1 -- 172.21.15.32:0/3876101986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563cfe87d1f0 con 0x563cfeadf0e0 2023-09-30T14:42:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.521+0000 7ffa38ff9700 1 -- 172.21.15.32:0/3876101986 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa4801b320 con 0x563cfeadf0e0 2023-09-30T14:42:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.521+0000 7ffa38ff9700 1 -- 172.21.15.32:0/3876101986 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffa48098740 con 0x563cfeadf0e0 2023-09-30T14:42:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.521+0000 7ffa6372ab80 10 client.15918 fetch_fsmap finished waiting for FSMap version 30 2023-09-30T14:42:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.521+0000 7ffa6372ab80 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:42:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.521+0000 7ffa6372ab80 20 client.15918 populate_metadata read hostname 'smithi032' 2023-09-30T14:42:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.521+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/3876101986 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563cfeb47180 0x563cfeb495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:42:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.521+0000 7ffa6372ab80 4 client.15918 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:42:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.521+0000 7ffa6372ab80 1 -- 172.21.15.32:0/3876101986 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563cfea24ad0 con 0x563cfeb47180 2023-09-30T14:42:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.521+0000 7ffa552de700 1 --2- 172.21.15.32:0/3876101986 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563cfeb47180 0x563cfeb495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:42:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.521+0000 7ffa552de700 1 --2- 172.21.15.32:0/3876101986 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563cfeb47180 0x563cfeb495a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=901fa75f5bce9f7c server_cookie=490ae60db0e0fba in_seq=0 out_seq=0 2023-09-30T14:42:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.521+0000 7ffa38ff9700 10 client.15918 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.563+0000 7ffa38ff9700 1 -- 172.21.15.32:0/3876101986 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563cfea24ad0 con 0x563cfeb47180 2023-09-30T14:42:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.563+0000 7ffa38ff9700 10 client.15918 handle_command_reply: tid=0 2023-09-30T14:42:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.563+0000 7ffa6372ab80 1 client.15918 shutdown 2023-09-30T14:42:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.563+0000 7ffa6372ab80 1 -- 172.21.15.32:0/3876101986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadf0e0 msgr2=0x563cfeaed700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.563+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/3876101986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadf0e0 0x563cfeaed700 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffa48099770 tx=0x7ffa480a1880 comp rx=0 tx=0).stop 2023-09-30T14:42:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.563+0000 7ffa6372ab80 1 -- 172.21.15.32:0/3876101986 shutdown_connections 2023-09-30T14:42:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/3876101986 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563cfeb47180 0x563cfeb495a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/3876101986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cfeadf0e0 0x563cfeaed700 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/3876101986 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cfeadd610 0x563cfeaec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa38ff9700 0 client.15918 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:42:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa6372ab80 1 --2- 172.21.15.32:0/3876101986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cfeade720 0x563cfeaec790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa6372ab80 1 -- 172.21.15.32:0/3876101986 >> 172.21.15.32:0/3876101986 conn(0x563cfea4d200 msgr2=0x563cfeae5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa6372ab80 1 -- 172.21.15.32:0/3876101986 shutdown_connections 2023-09-30T14:42:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa6372ab80 1 -- 172.21.15.32:0/3876101986 wait complete. 2023-09-30T14:42:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa6372ab80 20 client.15918 trim_cache size 0 max 16384 2023-09-30T14:42:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:42:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:42:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:42:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:42:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa57542700 1 -- 172.21.15.32:0/1686396631 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffa3c0a0fa0 msgr2=0x7ffa3c0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa57542700 1 --2- 172.21.15.32:0/1686396631 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffa3c0a0fa0 0x7ffa3c0a3460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00b2e0 tx=0x7ffa4c005fb0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa57542700 1 -- 172.21.15.32:0/1686396631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa50106ba0 msgr2=0x7ffa50197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:42:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa57542700 1 --2- 172.21.15.32:0/1686396631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa50106ba0 0x7ffa50197fb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800ef30 tx=0x7ffa4800c3a0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa57542700 1 -- 172.21.15.32:0/1686396631 shutdown_connections 2023-09-30T14:42:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa57542700 1 --2- 172.21.15.32:0/1686396631 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffa3c0a0fa0 0x7ffa3c0a3460 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa57542700 1 --2- 172.21.15.32:0/1686396631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa50106ba0 0x7ffa50197fb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa57542700 1 --2- 172.21.15.32:0/1686396631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa50101940 0x7ffa50196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa57542700 1 --2- 172.21.15.32:0/1686396631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa50104270 0x7ffa50197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:42:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa57542700 1 -- 172.21.15.32:0/1686396631 >> 172.21.15.32:0/1686396631 conn(0x7ffa500fb2b0 msgr2=0x7ffa50102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:42:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.564+0000 7ffa57542700 1 -- 172.21.15.32:0/1686396631 shutdown_connections 2023-09-30T14:42:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:42:53.565+0000 7ffa57542700 1 -- 172.21.15.32:0/1686396631 wait complete. 2023-09-30T14:42:53.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:53 smithi032 ceph-mon[98592]: pgmap v425: 97 pgs: 97 active+clean; 160 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2023-09-30T14:42:53.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 450141 10.93 MB/sec execute 87 sec latency 56.046 ms 2023-09-30T14:42:53.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:53 smithi156 ceph-mon[103579]: pgmap v425: 97 pgs: 97 active+clean; 160 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2023-09-30T14:42:53.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:53 smithi148 ceph-mon[102355]: pgmap v425: 97 pgs: 97 active+clean; 160 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2023-09-30T14:42:54.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 451678 10.89 MB/sec execute 88 sec latency 12.673 ms 2023-09-30T14:42:55.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:55 smithi032 ceph-mon[98592]: pgmap v426: 97 pgs: 97 active+clean; 162 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 2.6 MiB/s wr, 253 op/s 2023-09-30T14:42:55.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 454300 10.96 MB/sec execute 89 sec latency 27.058 ms 2023-09-30T14:42:55.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:55 smithi156 ceph-mon[103579]: pgmap v426: 97 pgs: 97 active+clean; 162 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 2.6 MiB/s wr, 253 op/s 2023-09-30T14:42:55.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:55 smithi148 ceph-mon[102355]: pgmap v426: 97 pgs: 97 active+clean; 162 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 2.6 MiB/s wr, 253 op/s 2023-09-30T14:42:56.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 455954 10.90 MB/sec execute 90 sec latency 14.061 ms 2023-09-30T14:42:57.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:57 smithi032 ceph-mon[98592]: pgmap v427: 97 pgs: 97 active+clean; 164 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 317 op/s 2023-09-30T14:42:57.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 457985 10.93 MB/sec execute 91 sec latency 59.531 ms 2023-09-30T14:42:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:57 smithi156 ceph-mon[103579]: pgmap v427: 97 pgs: 97 active+clean; 164 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 317 op/s 2023-09-30T14:42:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:57 smithi148 ceph-mon[102355]: pgmap v427: 97 pgs: 97 active+clean; 164 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 317 op/s 2023-09-30T14:42:58.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 458685 10.95 MB/sec execute 92 sec latency 252.092 ms 2023-09-30T14:42:59.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:42:59 smithi032 ceph-mon[98592]: pgmap v428: 97 pgs: 97 active+clean; 165 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 2.8 MiB/s wr, 276 op/s 2023-09-30T14:42:59.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 458685 10.83 MB/sec execute 93 sec latency 1252.192 ms 2023-09-30T14:42:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:42:59 smithi156 ceph-mon[103579]: pgmap v428: 97 pgs: 97 active+clean; 165 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 2.8 MiB/s wr, 276 op/s 2023-09-30T14:42:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:42:59 smithi148 ceph-mon[102355]: pgmap v428: 97 pgs: 97 active+clean; 165 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 2.8 MiB/s wr, 276 op/s 2023-09-30T14:43:00.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 460184 10.86 MB/sec execute 94 sec latency 1522.960 ms 2023-09-30T14:43:00.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:00 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:43:00.955 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:00 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:43:00.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:00 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:43:01.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 462141 10.89 MB/sec execute 95 sec latency 360.168 ms 2023-09-30T14:43:01.811 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:01 smithi156 ceph-mon[103579]: pgmap v429: 97 pgs: 97 active+clean; 167 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 315 op/s 2023-09-30T14:43:01.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:01 smithi148 ceph-mon[102355]: pgmap v429: 97 pgs: 97 active+clean; 167 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 315 op/s 2023-09-30T14:43:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:01 smithi032 ceph-mon[98592]: pgmap v429: 97 pgs: 97 active+clean; 167 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 315 op/s 2023-09-30T14:43:02.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 462141 10.78 MB/sec execute 96 sec latency 1017.160 ms 2023-09-30T14:43:03.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 462141 10.67 MB/sec execute 97 sec latency 2017.273 ms 2023-09-30T14:43:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:03 smithi156 ceph-mon[103579]: pgmap v430: 97 pgs: 97 active+clean; 161 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 3.0 MiB/s wr, 274 op/s 2023-09-30T14:43:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:03 smithi148 ceph-mon[102355]: pgmap v430: 97 pgs: 97 active+clean; 161 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 3.0 MiB/s wr, 274 op/s 2023-09-30T14:43:03.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:03 smithi032 ceph-mon[98592]: pgmap v430: 97 pgs: 97 active+clean; 161 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 3.0 MiB/s wr, 274 op/s 2023-09-30T14:43:04.699 INFO:tasks.workunit.client.0.smithi032.stdout: 1 462145 10.56 MB/sec execute 98 sec latency 2331.631 ms 2023-09-30T14:43:05.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 462145 10.45 MB/sec execute 99 sec latency 1683.309 ms 2023-09-30T14:43:05.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:05 smithi156 ceph-mon[103579]: pgmap v431: 97 pgs: 97 active+clean; 158 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 2.7 MiB/s wr, 258 op/s 2023-09-30T14:43:05.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:05 smithi148 ceph-mon[102355]: pgmap v431: 97 pgs: 97 active+clean; 158 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 2.7 MiB/s wr, 258 op/s 2023-09-30T14:43:05.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:05 smithi032 ceph-mon[98592]: pgmap v431: 97 pgs: 97 active+clean; 158 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 2.7 MiB/s wr, 258 op/s 2023-09-30T14:43:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.683+0000 7f3c157d6700 1 -- 172.21.15.32:0/363215547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c10102970 msgr2=0x7f3c10109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.683+0000 7f3c157d6700 1 --2- 172.21.15.32:0/363215547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c10102970 0x7f3c10109290 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009f90 tx=0x7f3c0800b330 comp rx=0 tx=0).stop 2023-09-30T14:43:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.683+0000 7f3c157d6700 1 -- 172.21.15.32:0/363215547 shutdown_connections 2023-09-30T14:43:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.683+0000 7f3c157d6700 1 --2- 172.21.15.32:0/363215547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c10102970 0x7f3c10109290 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.683+0000 7f3c157d6700 1 --2- 172.21.15.32:0/363215547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c10100e60 0x7f3c10101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.683+0000 7f3c157d6700 1 --2- 172.21.15.32:0/363215547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c10101fb0 0x7f3c10102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.683+0000 7f3c157d6700 1 -- 172.21.15.32:0/363215547 >> 172.21.15.32:0/363215547 conn(0x7f3c100fc3f0 msgr2=0x7f3c100fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.683+0000 7f3c157d6700 1 -- 172.21.15.32:0/363215547 shutdown_connections 2023-09-30T14:43:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.683+0000 7f3c157d6700 1 -- 172.21.15.32:0/363215547 wait complete. 2023-09-30T14:43:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.683+0000 7f3c157d6700 1 Processor -- start 2023-09-30T14:43:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c157d6700 1 -- start start 2023-09-30T14:43:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c157d6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c10100e60 0x7f3c10196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c157d6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c10101fb0 0x7f3c10197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c157d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c10102970 0x7f3c10197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c157d6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c10198500 con 0x7f3c10102970 2023-09-30T14:43:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c157d6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c10198670 con 0x7f3c10100e60 2023-09-30T14:43:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c157d6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c10198830 con 0x7f3c10101fb0 2023-09-30T14:43:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c0effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c10100e60 0x7f3c10196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c0effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c10100e60 0x7f3c10196ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48170/0 (socket says 172.21.15.32:48170) 2023-09-30T14:43:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c0effd700 1 -- 172.21.15.32:0/1420721315 learned_addr learned my addr 172.21.15.32:0/1420721315 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c0f7fe700 1 --2- 172.21.15.32:0/1420721315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c10102970 0x7f3c10197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c0e7fc700 1 --2- 172.21.15.32:0/1420721315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c10101fb0 0x7f3c10197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c0effd700 1 -- 172.21.15.32:0/1420721315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c10101fb0 msgr2=0x7f3c10197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c0effd700 1 --2- 172.21.15.32:0/1420721315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c10101fb0 0x7f3c10197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c0effd700 1 -- 172.21.15.32:0/1420721315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c10102970 msgr2=0x7f3c10197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c0effd700 1 --2- 172.21.15.32:0/1420721315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c10102970 0x7f3c10197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.684+0000 7f3c0effd700 1 -- 172.21.15.32:0/1420721315 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c000095f0 con 0x7f3c10100e60 2023-09-30T14:43:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.685+0000 7f3c0effd700 1 --2- 172.21.15.32:0/1420721315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c10100e60 0x7f3c10196ae0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000ec10 tx=0x7f3c0000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.685+0000 7f3bf7fff700 1 -- 172.21.15.32:0/1420721315 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0000cc90 con 0x7f3c10100e60 2023-09-30T14:43:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.685+0000 7f3c157d6700 1 -- 172.21.15.32:0/1420721315 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c08009c40 con 0x7f3c10100e60 2023-09-30T14:43:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.685+0000 7f3bf7fff700 1 -- 172.21.15.32:0/1420721315 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c000044e0 con 0x7f3c10100e60 2023-09-30T14:43:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.685+0000 7f3bf7fff700 1 -- 172.21.15.32:0/1420721315 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c00010430 con 0x7f3c10100e60 2023-09-30T14:43:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.685+0000 7f3c157d6700 1 -- 172.21.15.32:0/1420721315 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c10198d00 con 0x7f3c10100e60 2023-09-30T14:43:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.687+0000 7f3bf7fff700 1 -- 172.21.15.32:0/1420721315 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f3c00003fb0 con 0x7f3c10100e60 2023-09-30T14:43:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.688+0000 7f3bf7fff700 1 --2- 172.21.15.32:0/1420721315 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3bf80a0fa0 0x7f3bf80a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.688+0000 7f3bf7fff700 1 -- 172.21.15.32:0/1420721315 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3c00014070 con 0x7f3c10100e60 2023-09-30T14:43:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.688+0000 7f3c0e7fc700 1 --2- 172.21.15.32:0/1420721315 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3bf80a0fa0 0x7f3bf80a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c219beb80 1 Processor -- start 2023-09-30T14:43:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c0e7fc700 1 --2- 172.21.15.32:0/1420721315 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3bf80a0fa0 0x7f3bf80a3460 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009c60 tx=0x7f3c04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c219beb80 1 -- start start 2023-09-30T14:43:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c219beb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15c610 0x55caaf15ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c219beb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15d720 0x55caaf15dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c219beb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15e0e0 0x55caaf164970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c219beb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caaf0c0400 con 0x55caaf15d720 2023-09-30T14:43:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c219beb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caaf0b1bc0 con 0x55caaf15e0e0 2023-09-30T14:43:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c219beb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caaf0af4e0 con 0x55caaf15c610 2023-09-30T14:43:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c0e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15d720 0x55caaf15dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c0e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15d720 0x55caaf15dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36704/0 (socket says 172.21.15.32:36704) 2023-09-30T14:43:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c0e7fc700 1 -- 172.21.15.32:0/1046470157 learned_addr learned my addr 172.21.15.32:0/1046470157 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c0f7fe700 1 --2- 172.21.15.32:0/1046470157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15e0e0 0x55caaf164970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c0effd700 1 --2- 172.21.15.32:0/1046470157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15c610 0x55caaf15ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c0e7fc700 1 -- 172.21.15.32:0/1046470157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15c610 msgr2=0x55caaf15ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c0e7fc700 1 --2- 172.21.15.32:0/1046470157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15c610 0x55caaf15ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c0e7fc700 1 -- 172.21.15.32:0/1046470157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15e0e0 msgr2=0x55caaf164970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c0e7fc700 1 --2- 172.21.15.32:0/1046470157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15e0e0 0x55caaf164970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.689+0000 7f3c0e7fc700 1 -- 172.21.15.32:0/1046470157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caaf10d870 con 0x55caaf15d720 2023-09-30T14:43:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.690+0000 7f3c0e7fc700 1 --2- 172.21.15.32:0/1046470157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15d720 0x55caaf15dba0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400e4e0 tx=0x7f3c04014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b9fca62a0d577d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.690+0000 7f3bf57fa700 1 -- 172.21.15.32:0/1046470157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0400ee00 con 0x55caaf15d720 2023-09-30T14:43:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.690+0000 7f3bf57fa700 1 -- 172.21.15.32:0/1046470157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c04021070 con 0x55caaf15d720 2023-09-30T14:43:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.690+0000 7f3bf57fa700 1 -- 172.21.15.32:0/1046470157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0401d7b0 con 0x55caaf15d720 2023-09-30T14:43:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.690+0000 7f3c219beb80 1 -- 172.21.15.32:0/1046470157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15d720 msgr2=0x55caaf15dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.690+0000 7f3c219beb80 1 --2- 172.21.15.32:0/1046470157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15d720 0x55caaf15dba0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400e4e0 tx=0x7f3c04014000 comp rx=0 tx=0).stop 2023-09-30T14:43:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.690+0000 7f3c219beb80 1 -- 172.21.15.32:0/1046470157 shutdown_connections 2023-09-30T14:43:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.691+0000 7f3c219beb80 1 --2- 172.21.15.32:0/1046470157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15e0e0 0x55caaf164970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.691+0000 7f3c219beb80 1 --2- 172.21.15.32:0/1046470157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15c610 0x55caaf15ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.691+0000 7f3c219beb80 1 --2- 172.21.15.32:0/1046470157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15d720 0x55caaf15dba0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.691+0000 7f3c219beb80 1 -- 172.21.15.32:0/1046470157 >> 172.21.15.32:0/1046470157 conn(0x55caaf0cc200 msgr2=0x55caaf152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.691+0000 7f3c219beb80 1 -- 172.21.15.32:0/1046470157 shutdown_connections 2023-09-30T14:43:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.691+0000 7f3c219beb80 1 -- 172.21.15.32:0/1046470157 wait complete. 2023-09-30T14:43:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.691+0000 7f3c219beb80 1 -- start start 2023-09-30T14:43:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.691+0000 7f3c219beb80 1 Processor -- start 2023-09-30T14:43:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.691+0000 7f3c219beb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15c610 0x55caaf1662e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.691+0000 7f3c219beb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15d720 0x55caaf166820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.691+0000 7f3c219beb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15e0e0 0x55caaf16d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.691+0000 7f3c219beb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caaf0aadd0 con 0x55caaf15d720 2023-09-30T14:43:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.691+0000 7f3c219beb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caaf0476b0 con 0x55caaf15c610 2023-09-30T14:43:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.691+0000 7f3c219beb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caaf047350 con 0x55caaf15e0e0 2023-09-30T14:43:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.691+0000 7f3c219beb80 10 client.0 _ll_register_callbacks cb 0x55caaeea47a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:43:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3c0e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15d720 0x55caaf166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3c0effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15c610 0x55caaf1662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3c0e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15d720 0x55caaf166820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36710/0 (socket says 172.21.15.32:36710) 2023-09-30T14:43:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3c0effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15c610 0x55caaf1662e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48184/0 (socket says 172.21.15.32:48184) 2023-09-30T14:43:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3c0e7fc700 1 -- 172.21.15.32:0/1270062174 learned_addr learned my addr 172.21.15.32:0/1270062174 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3c0f7fe700 1 --2- 172.21.15.32:0/1270062174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15e0e0 0x55caaf16d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3c0e7fc700 1 -- 172.21.15.32:0/1270062174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15e0e0 msgr2=0x55caaf16d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3c0e7fc700 1 --2- 172.21.15.32:0/1270062174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15e0e0 0x55caaf16d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3c0e7fc700 1 -- 172.21.15.32:0/1270062174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15c610 msgr2=0x55caaf1662e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3c0e7fc700 1 --2- 172.21.15.32:0/1270062174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15c610 0x55caaf1662e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3c0e7fc700 1 -- 172.21.15.32:0/1270062174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c04017070 con 0x55caaf15d720 2023-09-30T14:43:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3c0effd700 1 --2- 172.21.15.32:0/1270062174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15c610 0x55caaf1662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3c0e7fc700 1 --2- 172.21.15.32:0/1270062174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15d720 0x55caaf166820 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400e4b0 tx=0x7f3c0400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f1ac0850e6d6a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3bf2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:43:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3bf2ff5700 1 -- 172.21.15.32:0/1270062174 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c040052d0 con 0x55caaf15d720 2023-09-30T14:43:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3bf2ff5700 1 -- 172.21.15.32:0/1270062174 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3c04005430 con 0x55caaf15d720 2023-09-30T14:43:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3c219beb80 1 -- 172.21.15.32:0/1270062174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55caaf14b090 con 0x55caaf15d720 2023-09-30T14:43:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3bf2ff5700 1 -- 172.21.15.32:0/1270062174 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c040055f0 con 0x55caaf15d720 2023-09-30T14:43:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3c219beb80 10 client.15930 fetch_fsmap learned FSMap version 31 2023-09-30T14:43:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.692+0000 7f3c219beb80 1 -- 172.21.15.32:0/1270062174 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55caaeed5d80 con 0x55caaf15d720 2023-09-30T14:43:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.693+0000 7f3bf2ff5700 1 -- 172.21.15.32:0/1270062174 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55caaf14b090 con 0x55caaf15d720 2023-09-30T14:43:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.693+0000 7f3bf2ff5700 1 -- 172.21.15.32:0/1270062174 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3c0401c710 con 0x55caaf15d720 2023-09-30T14:43:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.693+0000 7f3c219beb80 10 client.15930 fetch_fsmap finished waiting for FSMap version 31 2023-09-30T14:43:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.693+0000 7f3c219beb80 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:43:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.693+0000 7f3c219beb80 20 client.15930 populate_metadata read hostname 'smithi032' 2023-09-30T14:43:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.693+0000 7f3c219beb80 1 --2- 172.21.15.32:0/1270062174 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55caaf172020 0x55caaf174440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.693+0000 7f3c219beb80 4 client.15930 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:43:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.693+0000 7f3c219beb80 1 -- 172.21.15.32:0/1270062174 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55caaf0a3db0 con 0x55caaf172020 2023-09-30T14:43:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.693+0000 7f3c0effd700 1 --2- 172.21.15.32:0/1270062174 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55caaf172020 0x55caaf174440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.693+0000 7f3bf2ff5700 10 client.15930 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.693+0000 7f3c0effd700 1 --2- 172.21.15.32:0/1270062174 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55caaf172020 0x55caaf174440 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9668ad5c8afad810 server_cookie=de970bfe22d027e1 in_seq=0 out_seq=0 2023-09-30T14:43:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.694+0000 7f3bf2ff5700 1 -- 172.21.15.32:0/1270062174 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55caaf0a3db0 con 0x55caaf172020 2023-09-30T14:43:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.694+0000 7f3bf2ff5700 10 client.15930 handle_command_reply: tid=0 2023-09-30T14:43:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.694+0000 7f3c219beb80 1 client.15930 shutdown 2023-09-30T14:43:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.694+0000 7f3c219beb80 1 -- 172.21.15.32:0/1270062174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15d720 msgr2=0x55caaf166820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.694+0000 7f3c219beb80 1 --2- 172.21.15.32:0/1270062174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15d720 0x55caaf166820 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400e4b0 tx=0x7f3c0400ee00 comp rx=0 tx=0).stop 2023-09-30T14:43:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.694+0000 7f3c219beb80 1 -- 172.21.15.32:0/1270062174 shutdown_connections 2023-09-30T14:43:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.694+0000 7f3c219beb80 1 --2- 172.21.15.32:0/1270062174 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55caaf172020 0x55caaf174440 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.694+0000 7f3c219beb80 1 --2- 172.21.15.32:0/1270062174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15e0e0 0x55caaf16d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.694+0000 7f3bf2ff5700 0 client.15930 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.694+0000 7f3c219beb80 1 --2- 172.21.15.32:0/1270062174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15c610 0x55caaf1662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.694+0000 7f3c219beb80 1 --2- 172.21.15.32:0/1270062174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15d720 0x55caaf166820 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.694+0000 7f3c219beb80 1 -- 172.21.15.32:0/1270062174 >> 172.21.15.32:0/1270062174 conn(0x55caaf0cc200 msgr2=0x55caaf152590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.694+0000 7f3c219beb80 1 -- 172.21.15.32:0/1270062174 shutdown_connections 2023-09-30T14:43:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.695+0000 7f3c219beb80 1 -- 172.21.15.32:0/1270062174 wait complete. 2023-09-30T14:43:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.695+0000 7f3c219beb80 20 client.15930 trim_cache size 0 max 16384 2023-09-30T14:43:06.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 463313 10.40 MB/sec execute 100 sec latency 1831.324 ms 2023-09-30T14:43:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.711+0000 7f3c219beb80 1 Processor -- start 2023-09-30T14:43:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.711+0000 7f3c219beb80 1 -- start start 2023-09-30T14:43:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.711+0000 7f3c219beb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15c610 0x55caaf15db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.711+0000 7f3c219beb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15d720 0x55caaf16d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.711+0000 7f3c219beb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15e0e0 0x55caaf1644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.711+0000 7f3c219beb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caaef2b8c0 con 0x55caaf15e0e0 2023-09-30T14:43:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.711+0000 7f3c219beb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caaf165790 con 0x55caaf15c610 2023-09-30T14:43:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.711+0000 7f3c219beb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caaf15ce90 con 0x55caaf15d720 2023-09-30T14:43:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.711+0000 7f3c0f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15e0e0 0x55caaf1644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.711+0000 7f3c0effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15c610 0x55caaf15db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.711+0000 7f3c0f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15e0e0 0x55caaf1644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36720/0 (socket says 172.21.15.32:36720) 2023-09-30T14:43:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.711+0000 7f3c0f7fe700 1 -- 172.21.15.32:0/116615048 learned_addr learned my addr 172.21.15.32:0/116615048 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.711+0000 7f3c0effd700 1 --2- 172.21.15.32:0/116615048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15c610 0x55caaf15db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48192/0 (socket says 172.21.15.32:48192) 2023-09-30T14:43:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.711+0000 7f3c0e7fc700 1 --2- 172.21.15.32:0/116615048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15d720 0x55caaf16d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.711+0000 7f3c0effd700 1 -- 172.21.15.32:0/116615048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15d720 msgr2=0x55caaf16d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.712+0000 7f3c0effd700 1 --2- 172.21.15.32:0/116615048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15d720 0x55caaf16d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.712+0000 7f3c0effd700 1 -- 172.21.15.32:0/116615048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15e0e0 msgr2=0x55caaf1644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.712+0000 7f3c0effd700 1 --2- 172.21.15.32:0/116615048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15e0e0 0x55caaf1644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.712+0000 7f3c0effd700 1 -- 172.21.15.32:0/116615048 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caaeebec00 con 0x55caaf15c610 2023-09-30T14:43:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.712+0000 7f3c0e7fc700 1 --2- 172.21.15.32:0/116615048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15d720 0x55caaf16d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.712+0000 7f3c0f7fe700 1 --2- 172.21.15.32:0/116615048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15e0e0 0x55caaf1644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.712+0000 7f3c0effd700 1 --2- 172.21.15.32:0/116615048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15c610 0x55caaf15db00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3c00099e40 tx=0x7f3c000ca650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6efdcbbff4cbac5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.712+0000 7f3bf5ffb700 1 -- 172.21.15.32:0/116615048 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c00016330 con 0x55caaf15c610 2023-09-30T14:43:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.712+0000 7f3bf5ffb700 1 -- 172.21.15.32:0/116615048 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c0009a840 con 0x55caaf15c610 2023-09-30T14:43:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.712+0000 7f3bf5ffb700 1 -- 172.21.15.32:0/116615048 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c000984e0 con 0x55caaf15c610 2023-09-30T14:43:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.713+0000 7f3c219beb80 1 -- 172.21.15.32:0/116615048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15c610 msgr2=0x55caaf15db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.713+0000 7f3c219beb80 1 --2- 172.21.15.32:0/116615048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15c610 0x55caaf15db00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3c00099e40 tx=0x7f3c000ca650 comp rx=0 tx=0).stop 2023-09-30T14:43:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.713+0000 7f3c219beb80 1 -- 172.21.15.32:0/116615048 shutdown_connections 2023-09-30T14:43:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.713+0000 7f3c219beb80 1 --2- 172.21.15.32:0/116615048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15e0e0 0x55caaf1644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.713+0000 7f3c219beb80 1 --2- 172.21.15.32:0/116615048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15c610 0x55caaf15db00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.713+0000 7f3c219beb80 1 --2- 172.21.15.32:0/116615048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15d720 0x55caaf16d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.713+0000 7f3c219beb80 1 -- 172.21.15.32:0/116615048 >> 172.21.15.32:0/116615048 conn(0x55caaf0cc200 msgr2=0x55caaf165020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.713+0000 7f3c219beb80 1 -- 172.21.15.32:0/116615048 shutdown_connections 2023-09-30T14:43:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.713+0000 7f3c219beb80 1 -- 172.21.15.32:0/116615048 wait complete. 2023-09-30T14:43:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.713+0000 7f3c219beb80 1 -- start start 2023-09-30T14:43:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.713+0000 7f3c219beb80 1 Processor -- start 2023-09-30T14:43:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c219beb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15c610 0x55caaf16b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c219beb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15d720 0x55caaf16b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c219beb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15e0e0 0x55caaf16c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c219beb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caaf16cd10 con 0x55caaf15c610 2023-09-30T14:43:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c219beb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caaf16ce80 con 0x55caaf15d720 2023-09-30T14:43:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c219beb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caaf16d040 con 0x55caaf15e0e0 2023-09-30T14:43:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c0effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15c610 0x55caaf16b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c219beb80 10 client.0 _ll_register_callbacks cb 0x7f3c000091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:43:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c0effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15c610 0x55caaf16b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36722/0 (socket says 172.21.15.32:36722) 2023-09-30T14:43:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c0effd700 1 -- 172.21.15.32:0/4076166710 learned_addr learned my addr 172.21.15.32:0/4076166710 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c0e7fc700 1 --2- 172.21.15.32:0/4076166710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15d720 0x55caaf16b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c0f7fe700 1 --2- 172.21.15.32:0/4076166710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15e0e0 0x55caaf16c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c0effd700 1 -- 172.21.15.32:0/4076166710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15e0e0 msgr2=0x55caaf16c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c0effd700 1 --2- 172.21.15.32:0/4076166710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15e0e0 0x55caaf16c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c0effd700 1 -- 172.21.15.32:0/4076166710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15d720 msgr2=0x55caaf16b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c0effd700 1 --2- 172.21.15.32:0/4076166710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15d720 0x55caaf16b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c0effd700 1 -- 172.21.15.32:0/4076166710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55caaeed5d80 con 0x55caaf15c610 2023-09-30T14:43:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c0effd700 1 --2- 172.21.15.32:0/4076166710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15c610 0x55caaf16b280 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3c00096bb0 tx=0x7f3c0000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6d5eda83290496 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3bf2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:43:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3bf2ff5700 1 -- 172.21.15.32:0/4076166710 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c00016330 con 0x55caaf15c610 2023-09-30T14:43:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3bf2ff5700 1 -- 172.21.15.32:0/4076166710 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3c000a24d0 con 0x55caaf15c610 2023-09-30T14:43:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.714+0000 7f3c219beb80 1 -- 172.21.15.32:0/4076166710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c04021070 con 0x55caaf15c610 2023-09-30T14:43:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.715+0000 7f3bf2ff5700 1 -- 172.21.15.32:0/4076166710 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c0009a140 con 0x55caaf15c610 2023-09-30T14:43:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.715+0000 7f3c219beb80 10 client.15936 fetch_fsmap learned FSMap version 31 2023-09-30T14:43:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.715+0000 7f3c219beb80 1 -- 172.21.15.32:0/4076166710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f3c10109bc0 con 0x55caaf15c610 2023-09-30T14:43:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.715+0000 7f3bf2ff5700 1 -- 172.21.15.32:0/4076166710 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c04021070 con 0x55caaf15c610 2023-09-30T14:43:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.715+0000 7f3bf2ff5700 1 -- 172.21.15.32:0/4076166710 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3c000a2040 con 0x55caaf15c610 2023-09-30T14:43:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.715+0000 7f3c219beb80 10 client.15936 fetch_fsmap finished waiting for FSMap version 31 2023-09-30T14:43:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.715+0000 7f3c219beb80 10 client.15936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:43:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.715+0000 7f3c219beb80 20 client.15936 populate_metadata read hostname 'smithi032' 2023-09-30T14:43:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.715+0000 7f3c219beb80 1 --2- 172.21.15.32:0/4076166710 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55caaf1c67e0 0x55caaf1c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.715+0000 7f3c219beb80 4 client.15936 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:43:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.715+0000 7f3c219beb80 1 -- 172.21.15.32:0/4076166710 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55caaf0a3ad0 con 0x55caaf1c67e0 2023-09-30T14:43:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.715+0000 7f3c0e7fc700 1 --2- 172.21.15.32:0/4076166710 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55caaf1c67e0 0x55caaf1c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.715+0000 7f3bf2ff5700 10 client.15936 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.715+0000 7f3c0e7fc700 1 --2- 172.21.15.32:0/4076166710 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55caaf1c67e0 0x55caaf1c8c00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff1453bc22bd81c6 server_cookie=b8ccba7d2e9eeb87 in_seq=0 out_seq=0 2023-09-30T14:43:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.750+0000 7f3bf2ff5700 1 -- 172.21.15.32:0/4076166710 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55caaf0a3ad0 con 0x55caaf1c67e0 2023-09-30T14:43:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.750+0000 7f3bf2ff5700 10 client.15936 handle_command_reply: tid=0 2023-09-30T14:43:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.750+0000 7f3c219beb80 1 client.15936 shutdown 2023-09-30T14:43:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.750+0000 7f3c219beb80 1 -- 172.21.15.32:0/4076166710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15c610 msgr2=0x55caaf16b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.750+0000 7f3c219beb80 1 --2- 172.21.15.32:0/4076166710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15c610 0x55caaf16b280 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3c00096bb0 tx=0x7f3c0000ef90 comp rx=0 tx=0).stop 2023-09-30T14:43:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.750+0000 7f3c219beb80 1 -- 172.21.15.32:0/4076166710 shutdown_connections 2023-09-30T14:43:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.750+0000 7f3c219beb80 1 --2- 172.21.15.32:0/4076166710 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55caaf1c67e0 0x55caaf1c8c00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.750+0000 7f3c219beb80 1 --2- 172.21.15.32:0/4076166710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55caaf15e0e0 0x55caaf16c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.750+0000 7f3bf2ff5700 0 client.15936 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.750+0000 7f3c219beb80 1 --2- 172.21.15.32:0/4076166710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55caaf15c610 0x55caaf16b280 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.750+0000 7f3c219beb80 1 --2- 172.21.15.32:0/4076166710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55caaf15d720 0x55caaf16b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.750+0000 7f3c219beb80 1 -- 172.21.15.32:0/4076166710 >> 172.21.15.32:0/4076166710 conn(0x55caaf0cc200 msgr2=0x55caaf1644b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.751+0000 7f3c219beb80 1 -- 172.21.15.32:0/4076166710 shutdown_connections 2023-09-30T14:43:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.751+0000 7f3c219beb80 1 -- 172.21.15.32:0/4076166710 wait complete. 2023-09-30T14:43:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.751+0000 7f3c219beb80 20 client.15936 trim_cache size 0 max 16384 2023-09-30T14:43:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:43:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:43:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:43:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:43:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.751+0000 7f3c157d6700 1 -- 172.21.15.32:0/1420721315 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3bf80a0fa0 msgr2=0x7f3bf80a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.751+0000 7f3c157d6700 1 --2- 172.21.15.32:0/1420721315 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3bf80a0fa0 0x7f3bf80a3460 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009c60 tx=0x7f3c04009260 comp rx=0 tx=0).stop 2023-09-30T14:43:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.751+0000 7f3c157d6700 1 -- 172.21.15.32:0/1420721315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c10100e60 msgr2=0x7f3c10196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:06.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.751+0000 7f3c157d6700 1 --2- 172.21.15.32:0/1420721315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c10100e60 0x7f3c10196ae0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000ec10 tx=0x7f3c0000ef20 comp rx=0 tx=0).stop 2023-09-30T14:43:06.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.751+0000 7f3c157d6700 1 -- 172.21.15.32:0/1420721315 shutdown_connections 2023-09-30T14:43:06.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.751+0000 7f3c157d6700 1 --2- 172.21.15.32:0/1420721315 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f3bf80a0fa0 0x7f3bf80a3460 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.751+0000 7f3c157d6700 1 --2- 172.21.15.32:0/1420721315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c10102970 0x7f3c10197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.751+0000 7f3c157d6700 1 --2- 172.21.15.32:0/1420721315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c10100e60 0x7f3c10196ae0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.751+0000 7f3c157d6700 1 --2- 172.21.15.32:0/1420721315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c10101fb0 0x7f3c10197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:06.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.751+0000 7f3c157d6700 1 -- 172.21.15.32:0/1420721315 >> 172.21.15.32:0/1420721315 conn(0x7f3c100fc3f0 msgr2=0x7f3c10105520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:06.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.751+0000 7f3c157d6700 1 -- 172.21.15.32:0/1420721315 shutdown_connections 2023-09-30T14:43:06.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:06.751+0000 7f3c157d6700 1 -- 172.21.15.32:0/1420721315 wait complete. 2023-09-30T14:43:07.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 465708 10.43 MB/sec execute 101 sec latency 30.184 ms 2023-09-30T14:43:07.998 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:07 smithi148 ceph-mon[102355]: pgmap v432: 97 pgs: 97 active+clean; 156 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 2.8 MiB/s wr, 254 op/s 2023-09-30T14:43:07.999 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:07 smithi156 ceph-mon[103579]: pgmap v432: 97 pgs: 97 active+clean; 156 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 2.8 MiB/s wr, 254 op/s 2023-09-30T14:43:07.999 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:07 smithi032 ceph-mon[98592]: pgmap v432: 97 pgs: 97 active+clean; 156 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 2.8 MiB/s wr, 254 op/s 2023-09-30T14:43:08.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 467836 10.40 MB/sec execute 102 sec latency 17.740 ms 2023-09-30T14:43:09.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 469703 10.42 MB/sec execute 103 sec latency 14.739 ms 2023-09-30T14:43:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:09 smithi156 ceph-mon[103579]: pgmap v433: 97 pgs: 97 active+clean; 155 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 2.3 MiB/s wr, 183 op/s 2023-09-30T14:43:09.870 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:09 smithi148 ceph-mon[102355]: pgmap v433: 97 pgs: 97 active+clean; 155 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 2.3 MiB/s wr, 183 op/s 2023-09-30T14:43:09.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:09 smithi032 ceph-mon[98592]: pgmap v433: 97 pgs: 97 active+clean; 155 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 2.3 MiB/s wr, 183 op/s 2023-09-30T14:43:10.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 472613 10.49 MB/sec execute 104 sec latency 11.700 ms 2023-09-30T14:43:11.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 473942 10.45 MB/sec execute 105 sec latency 59.055 ms 2023-09-30T14:43:11.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:11 smithi156 ceph-mon[103579]: pgmap v434: 97 pgs: 97 active+clean; 152 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 2.4 MiB/s wr, 214 op/s 2023-09-30T14:43:11.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:11 smithi148 ceph-mon[102355]: pgmap v434: 97 pgs: 97 active+clean; 152 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 2.4 MiB/s wr, 214 op/s 2023-09-30T14:43:11.940 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:11 smithi032 ceph-mon[98592]: pgmap v434: 97 pgs: 97 active+clean; 152 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 2.4 MiB/s wr, 214 op/s 2023-09-30T14:43:12.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 476300 10.48 MB/sec execute 106 sec latency 47.026 ms 2023-09-30T14:43:13.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 478149 10.44 MB/sec execute 107 sec latency 16.554 ms 2023-09-30T14:43:13.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:13 smithi156 ceph-mon[103579]: pgmap v435: 97 pgs: 97 active+clean; 153 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 2.5 MiB/s wr, 187 op/s 2023-09-30T14:43:13.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:13 smithi148 ceph-mon[102355]: pgmap v435: 97 pgs: 97 active+clean; 153 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 2.5 MiB/s wr, 187 op/s 2023-09-30T14:43:13.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:13 smithi032 ceph-mon[98592]: pgmap v435: 97 pgs: 97 active+clean; 153 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 2.5 MiB/s wr, 187 op/s 2023-09-30T14:43:14.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 480271 10.47 MB/sec execute 108 sec latency 14.176 ms 2023-09-30T14:43:15.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 483216 10.53 MB/sec execute 109 sec latency 12.328 ms 2023-09-30T14:43:15.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:15 smithi156 ceph-mon[103579]: pgmap v436: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 2.4 MiB/s wr, 181 op/s 2023-09-30T14:43:15.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:15 smithi148 ceph-mon[102355]: pgmap v436: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 2.4 MiB/s wr, 181 op/s 2023-09-30T14:43:15.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:15 smithi032 ceph-mon[98592]: pgmap v436: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 2.4 MiB/s wr, 181 op/s 2023-09-30T14:43:16.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 484749 10.50 MB/sec execute 110 sec latency 11.866 ms 2023-09-30T14:43:17.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 487055 10.53 MB/sec execute 111 sec latency 31.570 ms 2023-09-30T14:43:17.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:17 smithi156 ceph-mon[103579]: pgmap v437: 97 pgs: 97 active+clean; 156 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 2.7 MiB/s wr, 238 op/s 2023-09-30T14:43:17.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:17 smithi148 ceph-mon[102355]: pgmap v437: 97 pgs: 97 active+clean; 156 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 2.7 MiB/s wr, 238 op/s 2023-09-30T14:43:17.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:17 smithi032 ceph-mon[98592]: pgmap v437: 97 pgs: 97 active+clean; 156 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 2.7 MiB/s wr, 238 op/s 2023-09-30T14:43:18.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 489332 10.51 MB/sec execute 112 sec latency 16.937 ms 2023-09-30T14:43:19.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 491252 10.52 MB/sec execute 113 sec latency 19.581 ms 2023-09-30T14:43:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:19 smithi156 ceph-mon[103579]: pgmap v438: 97 pgs: 97 active+clean; 156 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 2.7 MiB/s wr, 235 op/s 2023-09-30T14:43:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:19 smithi148 ceph-mon[102355]: pgmap v438: 97 pgs: 97 active+clean; 156 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 2.7 MiB/s wr, 235 op/s 2023-09-30T14:43:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.865+0000 7f7d0c0d4700 1 -- 172.21.15.32:0/2539060269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d04101320 msgr2=0x7f7d04101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.865+0000 7f7d0c0d4700 1 --2- 172.21.15.32:0/2539060269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d04101320 0x7f7d04101740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d000098f0 tx=0x7f7d00009c00 comp rx=0 tx=0).stop 2023-09-30T14:43:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.865+0000 7f7d0c0d4700 1 -- 172.21.15.32:0/2539060269 shutdown_connections 2023-09-30T14:43:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.865+0000 7f7d0c0d4700 1 --2- 172.21.15.32:0/2539060269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d04100930 0x7f7d041081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.865+0000 7f7d0c0d4700 1 --2- 172.21.15.32:0/2539060269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d04101320 0x7f7d04101740 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.865+0000 7f7d0c0d4700 1 --2- 172.21.15.32:0/2539060269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d040fff70 0x7f7d041003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.865+0000 7f7d0c0d4700 1 -- 172.21.15.32:0/2539060269 >> 172.21.15.32:0/2539060269 conn(0x7f7d040faa60 msgr2=0x7f7d040fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.865+0000 7f7d0c0d4700 1 -- 172.21.15.32:0/2539060269 shutdown_connections 2023-09-30T14:43:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.865+0000 7f7d0c0d4700 1 -- 172.21.15.32:0/2539060269 wait complete. 2023-09-30T14:43:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.866+0000 7f7d0c0d4700 1 Processor -- start 2023-09-30T14:43:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.866+0000 7f7d0c0d4700 1 -- start start 2023-09-30T14:43:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.866+0000 7f7d0c0d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d040fff70 0x7f7d04199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.866+0000 7f7d0c0d4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d04100930 0x7f7d0419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d0c0d4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d04101320 0x7f7d0419e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d0c0d4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0419e8d0 con 0x7f7d040fff70 2023-09-30T14:43:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d0c0d4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0419ea10 con 0x7f7d04101320 2023-09-30T14:43:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d0c0d4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0419ebd0 con 0x7f7d04100930 2023-09-30T14:43:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d09e70700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d040fff70 0x7f7d04199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d09e70700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d040fff70 0x7f7d04199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36736/0 (socket says 172.21.15.32:36736) 2023-09-30T14:43:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d09e70700 1 -- 172.21.15.32:0/207706349 learned_addr learned my addr 172.21.15.32:0/207706349 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d0966f700 1 --2- 172.21.15.32:0/207706349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d04100930 0x7f7d0419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d0a671700 1 --2- 172.21.15.32:0/207706349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d04101320 0x7f7d0419e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d09e70700 1 -- 172.21.15.32:0/207706349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d04100930 msgr2=0x7f7d0419bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d09e70700 1 --2- 172.21.15.32:0/207706349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d04100930 0x7f7d0419bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d09e70700 1 -- 172.21.15.32:0/207706349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d04101320 msgr2=0x7f7d0419e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d09e70700 1 --2- 172.21.15.32:0/207706349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d04101320 0x7f7d0419e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d09e70700 1 -- 172.21.15.32:0/207706349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf4009520 con 0x7f7d040fff70 2023-09-30T14:43:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d09e70700 1 --2- 172.21.15.32:0/207706349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d040fff70 0x7f7d04199860 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7d00011040 tx=0x7f7d0000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7cfaffd700 1 -- 172.21.15.32:0/207706349 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0001c070 con 0x7f7d040fff70 2023-09-30T14:43:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d0c0d4700 1 -- 172.21.15.32:0/207706349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc009d00 con 0x7f7d040fff70 2023-09-30T14:43:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7cfaffd700 1 -- 172.21.15.32:0/207706349 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d0000f7d0 con 0x7f7d040fff70 2023-09-30T14:43:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7cfaffd700 1 -- 172.21.15.32:0/207706349 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000049c0 con 0x7f7d040fff70 2023-09-30T14:43:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.867+0000 7f7d0c0d4700 1 -- 172.21.15.32:0/207706349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d00009520 con 0x7f7d040fff70 2023-09-30T14:43:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.870+0000 7f7cfaffd700 1 -- 172.21.15.32:0/207706349 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f7d00020070 con 0x7f7d040fff70 2023-09-30T14:43:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.870+0000 7f7cfaffd700 1 --2- 172.21.15.32:0/207706349 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7cf00a0fa0 0x7f7cf00a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.870+0000 7f7d0966f700 1 --2- 172.21.15.32:0/207706349 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7cf00a0fa0 0x7f7cf00a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.870+0000 7f7cfaffd700 1 -- 172.21.15.32:0/207706349 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7d000cd480 con 0x7f7d040fff70 2023-09-30T14:43:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.871+0000 7f7d0966f700 1 --2- 172.21.15.32:0/207706349 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7cf00a0fa0 0x7f7cf00a3460 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7d040ff330 tx=0x7f7cf4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.871+0000 7f7d182bcb80 1 Processor -- start 2023-09-30T14:43:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d182bcb80 1 -- start start 2023-09-30T14:43:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d182bcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 0x5572386ee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d182bcb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386ef310 0x5572386ef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d182bcb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386efcd0 0x5572386f6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d182bcb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557238656400 con 0x5572386ef310 2023-09-30T14:43:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d182bcb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557238647bc0 con 0x5572386efcd0 2023-09-30T14:43:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d182bcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572386454e0 con 0x5572386ee200 2023-09-30T14:43:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d0966f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386ef310 0x5572386ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d0966f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386ef310 0x5572386ef790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36746/0 (socket says 172.21.15.32:36746) 2023-09-30T14:43:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d0966f700 1 -- 172.21.15.32:0/3186275493 learned_addr learned my addr 172.21.15.32:0/3186275493 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d09e70700 1 --2- 172.21.15.32:0/3186275493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 0x5572386ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d0a671700 1 --2- 172.21.15.32:0/3186275493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386efcd0 0x5572386f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d0966f700 1 -- 172.21.15.32:0/3186275493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 msgr2=0x5572386ee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d0966f700 1 --2- 172.21.15.32:0/3186275493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 0x5572386ee620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d0966f700 1 -- 172.21.15.32:0/3186275493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386efcd0 msgr2=0x5572386f6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d0966f700 1 --2- 172.21.15.32:0/3186275493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386efcd0 0x5572386f6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d0966f700 1 -- 172.21.15.32:0/3186275493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572386a3870 con 0x5572386ef310 2023-09-30T14:43:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d09e70700 1 --2- 172.21.15.32:0/3186275493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 0x5572386ee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.872+0000 7f7d0966f700 1 --2- 172.21.15.32:0/3186275493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386ef310 0x5572386ef790 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400e600 tx=0x7f7cf4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f328638f74a30189 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.873+0000 7f7ceffff700 1 -- 172.21.15.32:0/3186275493 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4004030 con 0x5572386ef310 2023-09-30T14:43:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.873+0000 7f7ceffff700 1 -- 172.21.15.32:0/3186275493 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cf4021070 con 0x5572386ef310 2023-09-30T14:43:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.873+0000 7f7ceffff700 1 -- 172.21.15.32:0/3186275493 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf401c670 con 0x5572386ef310 2023-09-30T14:43:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.873+0000 7f7d182bcb80 1 -- 172.21.15.32:0/3186275493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386ef310 msgr2=0x5572386ef790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.873+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/3186275493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386ef310 0x5572386ef790 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400e600 tx=0x7f7cf4014000 comp rx=0 tx=0).stop 2023-09-30T14:43:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.873+0000 7f7d182bcb80 1 -- 172.21.15.32:0/3186275493 shutdown_connections 2023-09-30T14:43:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.873+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/3186275493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386efcd0 0x5572386f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.873+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/3186275493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 0x5572386ee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.873+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/3186275493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386ef310 0x5572386ef790 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.873+0000 7f7d182bcb80 1 -- 172.21.15.32:0/3186275493 >> 172.21.15.32:0/3186275493 conn(0x557238662200 msgr2=0x5572386e8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.874+0000 7f7d182bcb80 1 -- 172.21.15.32:0/3186275493 shutdown_connections 2023-09-30T14:43:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.874+0000 7f7d182bcb80 1 -- 172.21.15.32:0/3186275493 wait complete. 2023-09-30T14:43:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.874+0000 7f7d182bcb80 1 -- start start 2023-09-30T14:43:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.874+0000 7f7d182bcb80 1 Processor -- start 2023-09-30T14:43:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d182bcb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ee200 0x5572386f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d182bcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ef310 0x5572386f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d182bcb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 0x5572386ff4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d182bcb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557238640dd0 con 0x5572386efcd0 2023-09-30T14:43:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d182bcb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572385dd6b0 con 0x5572386ee200 2023-09-30T14:43:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d182bcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572385dd350 con 0x5572386ef310 2023-09-30T14:43:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d182bcb80 10 client.0 _ll_register_callbacks cb 0x55723843a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:43:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d0a671700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 0x5572386ff4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d09e70700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ee200 0x5572386f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d0a671700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 0x5572386ff4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36754/0 (socket says 172.21.15.32:36754) 2023-09-30T14:43:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d0a671700 1 -- 172.21.15.32:0/860535755 learned_addr learned my addr 172.21.15.32:0/860535755 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d09e70700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ee200 0x5572386f7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48226/0 (socket says 172.21.15.32:48226) 2023-09-30T14:43:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d0966f700 1 --2- 172.21.15.32:0/860535755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ef310 0x5572386f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d0a671700 1 -- 172.21.15.32:0/860535755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ef310 msgr2=0x5572386f84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d0a671700 1 --2- 172.21.15.32:0/860535755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ef310 0x5572386f84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d0a671700 1 -- 172.21.15.32:0/860535755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ee200 msgr2=0x5572386f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d0a671700 1 --2- 172.21.15.32:0/860535755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ee200 0x5572386f7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d0a671700 1 -- 172.21.15.32:0/860535755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d00009520 con 0x5572386efcd0 2023-09-30T14:43:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d09e70700 1 --2- 172.21.15.32:0/860535755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ee200 0x5572386f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d0a671700 1 --2- 172.21.15.32:0/860535755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 0x5572386ff4e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc007870 tx=0x7f7cfc007b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4b42ab26e0ceca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7ced7fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:43:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7ced7fa700 1 -- 172.21.15.32:0/860535755 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc00e020 con 0x5572386efcd0 2023-09-30T14:43:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7ced7fa700 1 -- 172.21.15.32:0/860535755 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7cfc004db0 con 0x5572386efcd0 2023-09-30T14:43:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.875+0000 7f7d182bcb80 1 -- 172.21.15.32:0/860535755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572386e1090 con 0x5572386efcd0 2023-09-30T14:43:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.876+0000 7f7ced7fa700 1 -- 172.21.15.32:0/860535755 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cfc004370 con 0x5572386efcd0 2023-09-30T14:43:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.876+0000 7f7d182bcb80 10 client.15948 fetch_fsmap learned FSMap version 31 2023-09-30T14:43:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.876+0000 7f7d182bcb80 1 -- 172.21.15.32:0/860535755 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f7cf4017070 con 0x5572386efcd0 2023-09-30T14:43:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.876+0000 7f7ced7fa700 1 -- 172.21.15.32:0/860535755 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5572386e1090 con 0x5572386efcd0 2023-09-30T14:43:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.876+0000 7f7ced7fa700 1 -- 172.21.15.32:0/860535755 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f7cfc027070 con 0x5572386efcd0 2023-09-30T14:43:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.876+0000 7f7d182bcb80 10 client.15948 fetch_fsmap finished waiting for FSMap version 31 2023-09-30T14:43:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.876+0000 7f7d182bcb80 10 client.15948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:43:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.876+0000 7f7d182bcb80 20 client.15948 populate_metadata read hostname 'smithi032' 2023-09-30T14:43:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.876+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/860535755 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557238703f10 0x557238706330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.876+0000 7f7d182bcb80 4 client.15948 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:43:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.876+0000 7f7d182bcb80 1 -- 172.21.15.32:0/860535755 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557238639db0 con 0x557238703f10 2023-09-30T14:43:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.876+0000 7f7d09e70700 1 --2- 172.21.15.32:0/860535755 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557238703f10 0x557238706330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.877+0000 7f7ced7fa700 10 client.15948 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.877+0000 7f7d09e70700 1 --2- 172.21.15.32:0/860535755 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557238703f10 0x557238706330 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b2cd8d7cdefa127 server_cookie=f26a5134fa4d631d in_seq=0 out_seq=0 2023-09-30T14:43:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.877+0000 7f7ced7fa700 1 -- 172.21.15.32:0/860535755 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557238639db0 con 0x557238703f10 2023-09-30T14:43:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.877+0000 7f7ced7fa700 10 client.15948 handle_command_reply: tid=0 2023-09-30T14:43:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.877+0000 7f7d182bcb80 1 client.15948 shutdown 2023-09-30T14:43:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.877+0000 7f7d182bcb80 1 -- 172.21.15.32:0/860535755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 msgr2=0x5572386ff4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.877+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/860535755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 0x5572386ff4e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc007870 tx=0x7f7cfc007b80 comp rx=0 tx=0).stop 2023-09-30T14:43:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.877+0000 7f7d182bcb80 1 -- 172.21.15.32:0/860535755 shutdown_connections 2023-09-30T14:43:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.878+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/860535755 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557238703f10 0x557238706330 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.878+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/860535755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 0x5572386ff4e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.878+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/860535755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ee200 0x5572386f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.878+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/860535755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ef310 0x5572386f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.878+0000 7f7d182bcb80 1 -- 172.21.15.32:0/860535755 >> 172.21.15.32:0/860535755 conn(0x557238662200 msgr2=0x5572386e8590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.878+0000 7f7ced7fa700 0 client.15948 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.878+0000 7f7d182bcb80 1 -- 172.21.15.32:0/860535755 shutdown_connections 2023-09-30T14:43:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.878+0000 7f7d182bcb80 1 -- 172.21.15.32:0/860535755 wait complete. 2023-09-30T14:43:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.878+0000 7f7d182bcb80 20 client.15948 trim_cache size 0 max 16384 2023-09-30T14:43:19.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:19 smithi032 ceph-mon[98592]: pgmap v438: 97 pgs: 97 active+clean; 156 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 2.7 MiB/s wr, 235 op/s 2023-09-30T14:43:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.893+0000 7f7d182bcb80 1 Processor -- start 2023-09-30T14:43:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.893+0000 7f7d182bcb80 1 -- start start 2023-09-30T14:43:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d182bcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 0x5572386f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d182bcb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ef310 0x5572386ff400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d182bcb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 0x5572386f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d182bcb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572384c18c0 con 0x5572386efcd0 2023-09-30T14:43:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d182bcb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572386e0bd0 con 0x5572386ef310 2023-09-30T14:43:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d182bcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572386f8be0 con 0x5572386ee200 2023-09-30T14:43:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d0966f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ef310 0x5572386ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d0a671700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 0x5572386f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d09e70700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 0x5572386f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d0966f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ef310 0x5572386ff400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48236/0 (socket says 172.21.15.32:48236) 2023-09-30T14:43:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d0a671700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 0x5572386f2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36762/0 (socket says 172.21.15.32:36762) 2023-09-30T14:43:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d0966f700 1 -- 172.21.15.32:0/1849129356 learned_addr learned my addr 172.21.15.32:0/1849129356 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d0966f700 1 -- 172.21.15.32:0/1849129356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 msgr2=0x5572386f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d0966f700 1 --2- 172.21.15.32:0/1849129356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 0x5572386f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d0966f700 1 -- 172.21.15.32:0/1849129356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 msgr2=0x5572386f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d0966f700 1 --2- 172.21.15.32:0/1849129356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 0x5572386f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d0966f700 1 -- 172.21.15.32:0/1849129356 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55723846bd80 con 0x5572386ef310 2023-09-30T14:43:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d09e70700 1 --2- 172.21.15.32:0/1849129356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 0x5572386f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7d0966f700 1 --2- 172.21.15.32:0/1849129356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ef310 0x5572386ff400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009c70 tx=0x7f7cf4009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a46308543201cc2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.894+0000 7f7cf8ff9700 1 -- 172.21.15.32:0/1849129356 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf402c040 con 0x5572386ef310 2023-09-30T14:43:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.895+0000 7f7cf8ff9700 1 -- 172.21.15.32:0/1849129356 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cf4005070 con 0x5572386ef310 2023-09-30T14:43:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.895+0000 7f7cf8ff9700 1 -- 172.21.15.32:0/1849129356 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4026860 con 0x5572386ef310 2023-09-30T14:43:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.895+0000 7f7d182bcb80 1 -- 172.21.15.32:0/1849129356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ef310 msgr2=0x5572386ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.895+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/1849129356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ef310 0x5572386ff400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009c70 tx=0x7f7cf4009460 comp rx=0 tx=0).stop 2023-09-30T14:43:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.895+0000 7f7d182bcb80 1 -- 172.21.15.32:0/1849129356 shutdown_connections 2023-09-30T14:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.895+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/1849129356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 0x5572386f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.895+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/1849129356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 0x5572386f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.895+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/1849129356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ef310 0x5572386ff400 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.895+0000 7f7d182bcb80 1 -- 172.21.15.32:0/1849129356 >> 172.21.15.32:0/1849129356 conn(0x557238662200 msgr2=0x5572386ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.895+0000 7f7d182bcb80 1 -- 172.21.15.32:0/1849129356 shutdown_connections 2023-09-30T14:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.895+0000 7f7d182bcb80 1 -- 172.21.15.32:0/1849129356 wait complete. 2023-09-30T14:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.896+0000 7f7d182bcb80 1 -- start start 2023-09-30T14:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.896+0000 7f7d182bcb80 1 Processor -- start 2023-09-30T14:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.896+0000 7f7d182bcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 0x5572386fcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.896+0000 7f7d182bcb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ef310 0x5572386fd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.896+0000 7f7d182bcb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 0x5572386fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.896+0000 7f7d182bcb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572386fe800 con 0x5572386efcd0 2023-09-30T14:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.896+0000 7f7d182bcb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572386fe970 con 0x5572386ef310 2023-09-30T14:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.896+0000 7f7d182bcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572386feb30 con 0x5572386ee200 2023-09-30T14:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.896+0000 7f7d182bcb80 10 client.0 _ll_register_callbacks cb 0x7f7cfc003da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.896+0000 7f7d09e70700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 0x5572386fcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.896+0000 7f7d09e70700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 0x5572386fcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51146/0 (socket says 172.21.15.32:51146) 2023-09-30T14:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.896+0000 7f7d09e70700 1 -- 172.21.15.32:0/1141252368 learned_addr learned my addr 172.21.15.32:0/1141252368 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.896+0000 7f7d0966f700 1 --2- 172.21.15.32:0/1141252368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ef310 0x5572386fd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.896+0000 7f7d0a671700 1 --2- 172.21.15.32:0/1141252368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 0x5572386fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.897+0000 7f7d09e70700 1 -- 172.21.15.32:0/1141252368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ef310 msgr2=0x5572386fd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.897+0000 7f7d09e70700 1 --2- 172.21.15.32:0/1141252368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ef310 0x5572386fd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.897+0000 7f7d09e70700 1 -- 172.21.15.32:0/1141252368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 msgr2=0x5572386fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.897+0000 7f7d09e70700 1 --2- 172.21.15.32:0/1141252368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 0x5572386fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.897+0000 7f7d09e70700 1 -- 172.21.15.32:0/1141252368 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc009d00 con 0x5572386ee200 2023-09-30T14:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.897+0000 7f7d0966f700 1 --2- 172.21.15.32:0/1141252368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ef310 0x5572386fd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.897+0000 7f7d09e70700 1 --2- 172.21.15.32:0/1141252368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 0x5572386fcd70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d0009aa90 tx=0x7f7d000cd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3195a8d2eaa77c27 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.897+0000 7f7ced7fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.897+0000 7f7ced7fa700 1 -- 172.21.15.32:0/1141252368 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0009ab60 con 0x5572386ee200 2023-09-30T14:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.897+0000 7f7d182bcb80 1 -- 172.21.15.32:0/1141252368 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7cf4021070 con 0x5572386ee200 2023-09-30T14:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.897+0000 7f7ced7fa700 1 -- 172.21.15.32:0/1141252368 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7d000b3070 con 0x5572386ee200 2023-09-30T14:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.897+0000 7f7ced7fa700 1 -- 172.21.15.32:0/1141252368 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d0009ec50 con 0x5572386ee200 2023-09-30T14:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.897+0000 7f7d182bcb80 10 client.25633 fetch_fsmap learned FSMap version 31 2023-09-30T14:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.897+0000 7f7d182bcb80 1 -- 172.21.15.32:0/1141252368 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5572386a3870 con 0x5572386ee200 2023-09-30T14:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.897+0000 7f7ced7fa700 1 -- 172.21.15.32:0/1141252368 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cf4021070 con 0x5572386ee200 2023-09-30T14:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.898+0000 7f7ced7fa700 1 -- 172.21.15.32:0/1141252368 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f7d0009ad60 con 0x5572386ee200 2023-09-30T14:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.898+0000 7f7d182bcb80 10 client.25633 fetch_fsmap finished waiting for FSMap version 31 2023-09-30T14:43:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.898+0000 7f7d182bcb80 10 client.25633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:43:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.898+0000 7f7d182bcb80 20 client.25633 populate_metadata read hostname 'smithi032' 2023-09-30T14:43:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.898+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/1141252368 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55723875c860 0x55723875ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.898+0000 7f7d182bcb80 4 client.25633 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:43:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.898+0000 7f7d182bcb80 1 -- 172.21.15.32:0/1141252368 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557238639ad0 con 0x55723875c860 2023-09-30T14:43:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.898+0000 7f7d0966f700 1 --2- 172.21.15.32:0/1141252368 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55723875c860 0x55723875ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.898+0000 7f7ced7fa700 10 client.25633 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.898+0000 7f7d0966f700 1 --2- 172.21.15.32:0/1141252368 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55723875c860 0x55723875ec80 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1fa11f5c9b73e5 server_cookie=b167349517c0353b in_seq=0 out_seq=0 2023-09-30T14:43:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.944+0000 7f7ced7fa700 1 -- 172.21.15.32:0/1141252368 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557238639ad0 con 0x55723875c860 2023-09-30T14:43:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.944+0000 7f7ced7fa700 10 client.25633 handle_command_reply: tid=0 2023-09-30T14:43:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.944+0000 7f7d182bcb80 1 client.25633 shutdown 2023-09-30T14:43:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.944+0000 7f7d182bcb80 1 -- 172.21.15.32:0/1141252368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 msgr2=0x5572386fcd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.944+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/1141252368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 0x5572386fcd70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d0009aa90 tx=0x7f7d000cd040 comp rx=0 tx=0).stop 2023-09-30T14:43:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.944+0000 7f7d182bcb80 1 -- 172.21.15.32:0/1141252368 shutdown_connections 2023-09-30T14:43:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.944+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/1141252368 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55723875c860 0x55723875ec80 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.944+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/1141252368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5572386efcd0 0x5572386fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.944+0000 7f7ced7fa700 0 client.25633 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.944+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/1141252368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5572386ee200 0x5572386fcd70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.944+0000 7f7d182bcb80 1 --2- 172.21.15.32:0/1141252368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572386ef310 0x5572386fd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.944+0000 7f7d182bcb80 1 -- 172.21.15.32:0/1141252368 >> 172.21.15.32:0/1141252368 conn(0x557238662200 msgr2=0x5572386ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.944+0000 7f7d182bcb80 1 -- 172.21.15.32:0/1141252368 shutdown_connections 2023-09-30T14:43:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.944+0000 7f7d182bcb80 1 -- 172.21.15.32:0/1141252368 wait complete. 2023-09-30T14:43:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.944+0000 7f7d182bcb80 20 client.25633 trim_cache size 0 max 16384 2023-09-30T14:43:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:43:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:43:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:43:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:43:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.945+0000 7f7d0c0d4700 1 -- 172.21.15.32:0/207706349 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7cf00a0fa0 msgr2=0x7f7cf00a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.945+0000 7f7d0c0d4700 1 --2- 172.21.15.32:0/207706349 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7cf00a0fa0 0x7f7cf00a3460 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7d040ff330 tx=0x7f7cf4009260 comp rx=0 tx=0).stop 2023-09-30T14:43:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.945+0000 7f7d0c0d4700 1 -- 172.21.15.32:0/207706349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d040fff70 msgr2=0x7f7d04199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.945+0000 7f7d0c0d4700 1 --2- 172.21.15.32:0/207706349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d040fff70 0x7f7d04199860 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7d00011040 tx=0x7f7d0000b790 comp rx=0 tx=0).stop 2023-09-30T14:43:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.945+0000 7f7d0c0d4700 1 -- 172.21.15.32:0/207706349 shutdown_connections 2023-09-30T14:43:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.945+0000 7f7d0c0d4700 1 --2- 172.21.15.32:0/207706349 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7cf00a0fa0 0x7f7cf00a3460 secure :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7d040ff330 tx=0x7f7cf4009260 comp rx=0 tx=0).stop 2023-09-30T14:43:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.945+0000 7f7d0c0d4700 1 --2- 172.21.15.32:0/207706349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d04101320 0x7f7d0419e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.945+0000 7f7d0c0d4700 1 --2- 172.21.15.32:0/207706349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d040fff70 0x7f7d04199860 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.945+0000 7f7d0c0d4700 1 --2- 172.21.15.32:0/207706349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d04100930 0x7f7d0419bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.945+0000 7f7d0c0d4700 1 -- 172.21.15.32:0/207706349 >> 172.21.15.32:0/207706349 conn(0x7f7d040faa60 msgr2=0x7f7d040fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.945+0000 7f7d0c0d4700 1 -- 172.21.15.32:0/207706349 shutdown_connections 2023-09-30T14:43:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:19.945+0000 7f7d0c0d4700 1 -- 172.21.15.32:0/207706349 wait complete. 2023-09-30T14:43:20.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 494076 10.59 MB/sec execute 114 sec latency 16.949 ms 2023-09-30T14:43:21.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 496253 10.56 MB/sec execute 115 sec latency 13.493 ms 2023-09-30T14:43:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:21 smithi156 ceph-mon[103579]: pgmap v439: 97 pgs: 97 active+clean; 158 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 2.9 MiB/s wr, 274 op/s 2023-09-30T14:43:21.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:21 smithi148 ceph-mon[102355]: pgmap v439: 97 pgs: 97 active+clean; 158 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 2.9 MiB/s wr, 274 op/s 2023-09-30T14:43:21.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:21 smithi032 ceph-mon[98592]: pgmap v439: 97 pgs: 97 active+clean; 158 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 2.9 MiB/s wr, 274 op/s 2023-09-30T14:43:22.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 497988 10.57 MB/sec execute 116 sec latency 35.150 ms 2023-09-30T14:43:23.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 500329 10.59 MB/sec execute 117 sec latency 100.550 ms 2023-09-30T14:43:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:23 smithi156 ceph-mon[103579]: pgmap v440: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 3.1 MiB/s wr, 267 op/s 2023-09-30T14:43:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:23 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:43:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:23 smithi148 ceph-mon[102355]: pgmap v440: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 3.1 MiB/s wr, 267 op/s 2023-09-30T14:43:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:23 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:43:23.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:23 smithi032 ceph-mon[98592]: pgmap v440: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 3.1 MiB/s wr, 267 op/s 2023-09-30T14:43:23.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:23 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:43:24.700 INFO:tasks.workunit.client.0.smithi032.stdout: 1 500607 10.50 MB/sec execute 118 sec latency 569.514 ms 2023-09-30T14:43:25.701 INFO:tasks.workunit.client.0.smithi032.stdout: 1 500607 10.42 MB/sec execute 119 sec latency 1569.624 ms 2023-09-30T14:43:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:25 smithi156 ceph-mon[103579]: pgmap v441: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 2.8 MiB/s wr, 286 op/s 2023-09-30T14:43:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:25 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:43:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:25 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:43:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:25 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:43:25.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:25 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:43:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:25 smithi148 ceph-mon[102355]: pgmap v441: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 2.8 MiB/s wr, 286 op/s 2023-09-30T14:43:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:25 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:43:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:25 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:43:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:25 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:43:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:25 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:43:25.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:25 smithi032 ceph-mon[98592]: pgmap v441: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 2.8 MiB/s wr, 286 op/s 2023-09-30T14:43:25.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:25 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:43:25.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:25 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:43:25.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:25 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:43:25.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:25 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:43:26.701 INFO:tasks.workunit.client.0.smithi032.stdout: 1 500607 10.33 MB/sec execute 120 sec latency 2569.737 ms 2023-09-30T14:43:27.701 INFO:tasks.workunit.client.0.smithi032.stdout: 1 500607 10.24 MB/sec execute 121 sec latency 3569.843 ms 2023-09-30T14:43:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:27 smithi156 ceph-mon[103579]: pgmap v442: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.9 MiB/s wr, 301 op/s 2023-09-30T14:43:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:27 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:43:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:27 smithi148 ceph-mon[102355]: pgmap v442: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.9 MiB/s wr, 301 op/s 2023-09-30T14:43:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:27 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:43:27.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:27 smithi032 ceph-mon[98592]: pgmap v442: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.9 MiB/s wr, 301 op/s 2023-09-30T14:43:27.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:27 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:43:28.701 INFO:tasks.workunit.client.0.smithi032.stdout: 1 500607 10.16 MB/sec execute 122 sec latency 4569.948 ms 2023-09-30T14:43:28.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:28 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:43:28.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:28 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:43:28.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:28 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:43:29.701 INFO:tasks.workunit.client.0.smithi032.stdout: 1 501550 10.12 MB/sec execute 123 sec latency 4915.149 ms 2023-09-30T14:43:29.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:29 smithi156 ceph-mon[103579]: pgmap v443: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 2.7 MiB/s wr, 257 op/s 2023-09-30T14:43:29.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:29 smithi148 ceph-mon[102355]: pgmap v443: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 2.7 MiB/s wr, 257 op/s 2023-09-30T14:43:29.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:29 smithi032 ceph-mon[98592]: pgmap v443: 97 pgs: 97 active+clean; 155 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 2.7 MiB/s wr, 257 op/s 2023-09-30T14:43:30.701 INFO:tasks.workunit.client.0.smithi032.stdout: 1 504108 10.15 MB/sec execute 124 sec latency 44.423 ms 2023-09-30T14:43:31.701 INFO:tasks.workunit.client.0.smithi032.stdout: 1 505916 10.16 MB/sec execute 125 sec latency 13.511 ms 2023-09-30T14:43:31.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:31 smithi156 ceph-mon[103579]: pgmap v444: 97 pgs: 97 active+clean; 158 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 2.6 MiB/s wr, 235 op/s 2023-09-30T14:43:31.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:31 smithi148 ceph-mon[102355]: pgmap v444: 97 pgs: 97 active+clean; 158 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 2.6 MiB/s wr, 235 op/s 2023-09-30T14:43:31.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:31 smithi032 ceph-mon[98592]: pgmap v444: 97 pgs: 97 active+clean; 158 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 2.6 MiB/s wr, 235 op/s 2023-09-30T14:43:32.701 INFO:tasks.workunit.client.0.smithi032.stdout: 1 508377 10.19 MB/sec execute 126 sec latency 9.580 ms 2023-09-30T14:43:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.058+0000 7fd325f2b700 1 -- 172.21.15.32:0/977391596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd320100f50 msgr2=0x7fd3201013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.058+0000 7fd325f2b700 1 --2- 172.21.15.32:0/977391596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd320100f50 0x7fd3201013b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd3140099c0 tx=0x7fd314009cd0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.058+0000 7fd325f2b700 1 -- 172.21.15.32:0/977391596 shutdown_connections 2023-09-30T14:43:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.058+0000 7fd325f2b700 1 --2- 172.21.15.32:0/977391596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3201018f0 0x7fd3201081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.058+0000 7fd325f2b700 1 --2- 172.21.15.32:0/977391596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3200ffd60 0x7fd320100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.058+0000 7fd325f2b700 1 --2- 172.21.15.32:0/977391596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd320100f50 0x7fd3201013b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.058+0000 7fd325f2b700 1 -- 172.21.15.32:0/977391596 >> 172.21.15.32:0/977391596 conn(0x7fd3200fb2b0 msgr2=0x7fd3200fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.058+0000 7fd325f2b700 1 -- 172.21.15.32:0/977391596 shutdown_connections 2023-09-30T14:43:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.058+0000 7fd325f2b700 1 -- 172.21.15.32:0/977391596 wait complete. 2023-09-30T14:43:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.058+0000 7fd325f2b700 1 Processor -- start 2023-09-30T14:43:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd325f2b700 1 -- start start 2023-09-30T14:43:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd325f2b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3200ffd60 0x7fd3201998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd325f2b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd320100f50 0x7fd32019be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd325f2b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3201018f0 0x7fd32019e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd325f2b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32019e920 con 0x7fd320100f50 2023-09-30T14:43:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd325f2b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32019ea60 con 0x7fd3200ffd60 2023-09-30T14:43:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd325f2b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32019ec20 con 0x7fd3201018f0 2023-09-30T14:43:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd31effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd320100f50 0x7fd32019be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd31f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3200ffd60 0x7fd3201998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd31effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd320100f50 0x7fd32019be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36780/0 (socket says 172.21.15.32:36780) 2023-09-30T14:43:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd31effd700 1 -- 172.21.15.32:0/537396037 learned_addr learned my addr 172.21.15.32:0/537396037 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd31ffff700 1 --2- 172.21.15.32:0/537396037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3201018f0 0x7fd32019e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd31effd700 1 -- 172.21.15.32:0/537396037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3201018f0 msgr2=0x7fd32019e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd31effd700 1 --2- 172.21.15.32:0/537396037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3201018f0 0x7fd32019e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd31effd700 1 -- 172.21.15.32:0/537396037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3200ffd60 msgr2=0x7fd3201998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd31effd700 1 --2- 172.21.15.32:0/537396037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3200ffd60 0x7fd3201998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd31effd700 1 -- 172.21.15.32:0/537396037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3140095f0 con 0x7fd320100f50 2023-09-30T14:43:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.059+0000 7fd31f7fe700 1 --2- 172.21.15.32:0/537396037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3200ffd60 0x7fd3201998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.060+0000 7fd31effd700 1 --2- 172.21.15.32:0/537396037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd320100f50 0x7fd32019be00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd314009990 tx=0x7fd31400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.060+0000 7fd31cff9700 1 -- 172.21.15.32:0/537396037 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31401d070 con 0x7fd320100f50 2023-09-30T14:43:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.060+0000 7fd31cff9700 1 -- 172.21.15.32:0/537396037 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd314005030 con 0x7fd320100f50 2023-09-30T14:43:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.060+0000 7fd31cff9700 1 -- 172.21.15.32:0/537396037 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314004a70 con 0x7fd320100f50 2023-09-30T14:43:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.060+0000 7fd325f2b700 1 -- 172.21.15.32:0/537396037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32019eea0 con 0x7fd320100f50 2023-09-30T14:43:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.060+0000 7fd325f2b700 1 -- 172.21.15.32:0/537396037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32019f210 con 0x7fd320100f50 2023-09-30T14:43:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.063+0000 7fd332113b80 1 Processor -- start 2023-09-30T14:43:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.063+0000 7fd332113b80 1 -- start start 2023-09-30T14:43:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.063+0000 7fd332113b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c4200 0x55e4635c4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.063+0000 7fd332113b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c5310 0x55e4635c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.063+0000 7fd332113b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635c5cd0 0x55e4635cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.063+0000 7fd332113b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46352c400 con 0x55e4635c4200 2023-09-30T14:43:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.063+0000 7fd332113b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46351dbc0 con 0x55e4635c5cd0 2023-09-30T14:43:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.063+0000 7fd332113b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46351b4e0 con 0x55e4635c5310 2023-09-30T14:43:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.064+0000 7fd31ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c4200 0x55e4635c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.064+0000 7fd31ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c4200 0x55e4635c4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36784/0 (socket says 172.21.15.32:36784) 2023-09-30T14:43:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.064+0000 7fd31ffff700 1 -- 172.21.15.32:0/2009990268 learned_addr learned my addr 172.21.15.32:0/2009990268 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.064+0000 7fd31f7fe700 1 --2- 172.21.15.32:0/2009990268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c5310 0x55e4635c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.064+0000 7fd31ffff700 1 -- 172.21.15.32:0/2009990268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c5310 msgr2=0x55e4635c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.064+0000 7fd31ffff700 1 --2- 172.21.15.32:0/2009990268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c5310 0x55e4635c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.064+0000 7fd31ffff700 1 -- 172.21.15.32:0/2009990268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635c5cd0 msgr2=0x55e4635cc560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T14:43:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.064+0000 7fd31ffff700 1 --2- 172.21.15.32:0/2009990268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635c5cd0 0x55e4635cc560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.064+0000 7fd31ffff700 1 -- 172.21.15.32:0/2009990268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e463579870 con 0x55e4635c4200 2023-09-30T14:43:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.064+0000 7fd31cff9700 1 -- 172.21.15.32:0/537396037 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fd314005410 con 0x7fd320100f50 2023-09-30T14:43:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.064+0000 7fd31ffff700 1 --2- 172.21.15.32:0/2009990268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c4200 0x55e4635c4620 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd318011680 tx=0x7fd318009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2a5decf0534561 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.064+0000 7fd305ffb700 1 -- 172.21.15.32:0/2009990268 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31800e040 con 0x55e4635c4200 2023-09-30T14:43:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.064+0000 7fd305ffb700 1 -- 172.21.15.32:0/2009990268 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd31800f040 con 0x55e4635c4200 2023-09-30T14:43:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.064+0000 7fd305ffb700 1 -- 172.21.15.32:0/2009990268 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3180047f0 con 0x55e4635c4200 2023-09-30T14:43:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.065+0000 7fd31cff9700 1 --2- 172.21.15.32:0/537396037 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd3080a1020 0x7fd3080a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.065+0000 7fd31cff9700 1 -- 172.21.15.32:0/537396037 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd3140cd440 con 0x7fd320100f50 2023-09-30T14:43:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.065+0000 7fd31f7fe700 1 --2- 172.21.15.32:0/537396037 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd3080a1020 0x7fd3080a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.065+0000 7fd332113b80 1 -- 172.21.15.32:0/2009990268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c4200 msgr2=0x55e4635c4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.065+0000 7fd332113b80 1 --2- 172.21.15.32:0/2009990268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c4200 0x55e4635c4620 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd318011680 tx=0x7fd318009920 comp rx=0 tx=0).stop 2023-09-30T14:43:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.065+0000 7fd31f7fe700 1 --2- 172.21.15.32:0/537396037 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd3080a1020 0x7fd3080a34e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd320068d20 tx=0x7fd310009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.065+0000 7fd332113b80 1 -- 172.21.15.32:0/2009990268 shutdown_connections 2023-09-30T14:43:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.065+0000 7fd332113b80 1 --2- 172.21.15.32:0/2009990268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635c5cd0 0x55e4635cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.065+0000 7fd332113b80 1 --2- 172.21.15.32:0/2009990268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c4200 0x55e4635c4620 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.065+0000 7fd332113b80 1 --2- 172.21.15.32:0/2009990268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c5310 0x55e4635c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.065+0000 7fd332113b80 1 -- 172.21.15.32:0/2009990268 >> 172.21.15.32:0/2009990268 conn(0x55e463538200 msgr2=0x55e4635beef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.065+0000 7fd332113b80 1 -- 172.21.15.32:0/2009990268 shutdown_connections 2023-09-30T14:43:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.065+0000 7fd332113b80 1 -- 172.21.15.32:0/2009990268 wait complete. 2023-09-30T14:43:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.065+0000 7fd332113b80 1 -- start start 2023-09-30T14:43:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.066+0000 7fd332113b80 1 Processor -- start 2023-09-30T14:43:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.066+0000 7fd332113b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635c4200 0x55e4635d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.066+0000 7fd332113b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5310 0x55e4635d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.066+0000 7fd332113b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c5cd0 0x55e4635d98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.066+0000 7fd332113b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e463516dd0 con 0x55e4635c5310 2023-09-30T14:43:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.066+0000 7fd332113b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4634b36b0 con 0x55e4635c4200 2023-09-30T14:43:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.066+0000 7fd332113b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4634b3350 con 0x55e4635c5cd0 2023-09-30T14:43:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.066+0000 7fd332113b80 10 client.0 _ll_register_callbacks cb 0x55e4633107a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:43:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.066+0000 7fd31effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5310 0x55e4635d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.066+0000 7fd31effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5310 0x55e4635d2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36792/0 (socket says 172.21.15.32:36792) 2023-09-30T14:43:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.066+0000 7fd31effd700 1 -- 172.21.15.32:0/2967245918 learned_addr learned my addr 172.21.15.32:0/2967245918 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.066+0000 7fd31f7fe700 1 --2- 172.21.15.32:0/2967245918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635c4200 0x55e4635d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.066+0000 7fd31ffff700 1 --2- 172.21.15.32:0/2967245918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c5cd0 0x55e4635d98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd31effd700 1 -- 172.21.15.32:0/2967245918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c5cd0 msgr2=0x55e4635d98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd31effd700 1 --2- 172.21.15.32:0/2967245918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c5cd0 0x55e4635d98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd31effd700 1 -- 172.21.15.32:0/2967245918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635c4200 msgr2=0x55e4635d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd31effd700 1 --2- 172.21.15.32:0/2967245918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635c4200 0x55e4635d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd31effd700 1 -- 172.21.15.32:0/2967245918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd318011330 con 0x55e4635c5310 2023-09-30T14:43:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd31f7fe700 1 --2- 172.21.15.32:0/2967245918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635c4200 0x55e4635d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd31ffff700 1 --2- 172.21.15.32:0/2967245918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c5cd0 0x55e4635d98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd31effd700 1 --2- 172.21.15.32:0/2967245918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5310 0x55e4635d2870 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd314096e40 tx=0x7fd314004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60585e7591e31989 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd2feffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:43:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd2feffd700 1 -- 172.21.15.32:0/2967245918 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31409a940 con 0x55e4635c5310 2023-09-30T14:43:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd2feffd700 1 -- 172.21.15.32:0/2967245918 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd31409ea90 con 0x55e4635c5310 2023-09-30T14:43:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd332113b80 1 -- 172.21.15.32:0/2967245918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4635b7090 con 0x55e4635c5310 2023-09-30T14:43:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd2feffd700 1 -- 172.21.15.32:0/2967245918 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3140a2c40 con 0x55e4635c5310 2023-09-30T14:43:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd332113b80 10 client.15966 fetch_fsmap learned FSMap version 34 2023-09-30T14:43:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd332113b80 1 -- 172.21.15.32:0/2967245918 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e463341d80 con 0x55e4635c5310 2023-09-30T14:43:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd2feffd700 1 -- 172.21.15.32:0/2967245918 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e4635b7090 con 0x55e4635c5310 2023-09-30T14:43:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd2feffd700 1 -- 172.21.15.32:0/2967245918 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd3140a2840 con 0x55e4635c5310 2023-09-30T14:43:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd332113b80 10 client.15966 fetch_fsmap finished waiting for FSMap version 34 2023-09-30T14:43:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd332113b80 10 client.15966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:43:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.067+0000 7fd332113b80 20 client.15966 populate_metadata read hostname 'smithi032' 2023-09-30T14:43:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.068+0000 7fd332113b80 1 --2- 172.21.15.32:0/2967245918 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e4635ddd90 0x55e4635e0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.068+0000 7fd332113b80 4 client.15966 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:43:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.068+0000 7fd332113b80 1 -- 172.21.15.32:0/2967245918 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e46350fdb0 con 0x55e4635ddd90 2023-09-30T14:43:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.068+0000 7fd31f7fe700 1 --2- 172.21.15.32:0/2967245918 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e4635ddd90 0x55e4635e0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.068+0000 7fd2feffd700 10 client.15966 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.068+0000 7fd31f7fe700 1 --2- 172.21.15.32:0/2967245918 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e4635ddd90 0x55e4635e0180 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ae82200908072e5 server_cookie=373d5a3fdceb9276 in_seq=0 out_seq=0 2023-09-30T14:43:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.069+0000 7fd2feffd700 1 -- 172.21.15.32:0/2967245918 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e46350fdb0 con 0x55e4635ddd90 2023-09-30T14:43:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.069+0000 7fd2feffd700 10 client.15966 handle_command_reply: tid=0 2023-09-30T14:43:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.069+0000 7fd332113b80 1 client.15966 shutdown 2023-09-30T14:43:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.069+0000 7fd332113b80 1 -- 172.21.15.32:0/2967245918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5310 msgr2=0x55e4635d2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.069+0000 7fd332113b80 1 --2- 172.21.15.32:0/2967245918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5310 0x55e4635d2870 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd314096e40 tx=0x7fd314004ef0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.069+0000 7fd332113b80 1 -- 172.21.15.32:0/2967245918 shutdown_connections 2023-09-30T14:43:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.069+0000 7fd332113b80 1 --2- 172.21.15.32:0/2967245918 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e4635ddd90 0x55e4635e0180 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.069+0000 7fd332113b80 1 --2- 172.21.15.32:0/2967245918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c5cd0 0x55e4635d98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.069+0000 7fd2feffd700 0 client.15966 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.069+0000 7fd332113b80 1 --2- 172.21.15.32:0/2967245918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635c4200 0x55e4635d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.069+0000 7fd332113b80 1 --2- 172.21.15.32:0/2967245918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5310 0x55e4635d2870 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.069+0000 7fd332113b80 1 -- 172.21.15.32:0/2967245918 >> 172.21.15.32:0/2967245918 conn(0x55e463538200 msgr2=0x55e4635be590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.069+0000 7fd332113b80 1 -- 172.21.15.32:0/2967245918 shutdown_connections 2023-09-30T14:43:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.069+0000 7fd332113b80 1 -- 172.21.15.32:0/2967245918 wait complete. 2023-09-30T14:43:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.069+0000 7fd332113b80 20 client.15966 trim_cache size 0 max 16384 2023-09-30T14:43:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.085+0000 7fd332113b80 1 Processor -- start 2023-09-30T14:43:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.085+0000 7fd332113b80 1 -- start start 2023-09-30T14:43:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.085+0000 7fd332113b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c4200 0x55e4635c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.085+0000 7fd332113b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5310 0x55e4635d9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.085+0000 7fd332113b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635c5cd0 0x55e4635cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.085+0000 7fd332113b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4633978c0 con 0x55e4635c5310 2023-09-30T14:43:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.085+0000 7fd332113b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e463589b90 con 0x55e4635c5cd0 2023-09-30T14:43:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.085+0000 7fd332113b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4635bfcd0 con 0x55e4635c4200 2023-09-30T14:43:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.086+0000 7fd31effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5310 0x55e4635d9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.086+0000 7fd31f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c4200 0x55e4635c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.086+0000 7fd31f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c4200 0x55e4635c56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51180/0 (socket says 172.21.15.32:51180) 2023-09-30T14:43:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.086+0000 7fd31ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635c5cd0 0x55e4635cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.086+0000 7fd31f7fe700 1 -- 172.21.15.32:0/791681044 learned_addr learned my addr 172.21.15.32:0/791681044 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.086+0000 7fd31effd700 1 -- 172.21.15.32:0/791681044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c4200 msgr2=0x55e4635c56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.086+0000 7fd31effd700 1 --2- 172.21.15.32:0/791681044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c4200 0x55e4635c56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.086+0000 7fd31effd700 1 -- 172.21.15.32:0/791681044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635c5cd0 msgr2=0x55e4635cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.086+0000 7fd31effd700 1 --2- 172.21.15.32:0/791681044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635c5cd0 0x55e4635cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.086+0000 7fd31effd700 1 -- 172.21.15.32:0/791681044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e46332ac00 con 0x55e4635c5310 2023-09-30T14:43:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.086+0000 7fd31ffff700 1 --2- 172.21.15.32:0/791681044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635c5cd0 0x55e4635cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.086+0000 7fd31effd700 1 --2- 172.21.15.32:0/791681044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5310 0x55e4635d9870 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd31409a640 tx=0x7fd31409aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fbe65bc3b4c171 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.086+0000 7fd3067fc700 1 -- 172.21.15.32:0/791681044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314096ae0 con 0x55e4635c5310 2023-09-30T14:43:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.086+0000 7fd3067fc700 1 -- 172.21.15.32:0/791681044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd314096c80 con 0x55e4635c5310 2023-09-30T14:43:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.086+0000 7fd3067fc700 1 -- 172.21.15.32:0/791681044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314098d70 con 0x55e4635c5310 2023-09-30T14:43:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.087+0000 7fd332113b80 1 -- 172.21.15.32:0/791681044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5310 msgr2=0x55e4635d9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.087+0000 7fd332113b80 1 --2- 172.21.15.32:0/791681044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5310 0x55e4635d9870 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd31409a640 tx=0x7fd31409aec0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.087+0000 7fd332113b80 1 -- 172.21.15.32:0/791681044 shutdown_connections 2023-09-30T14:43:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.087+0000 7fd332113b80 1 --2- 172.21.15.32:0/791681044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635c5cd0 0x55e4635cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.087+0000 7fd332113b80 1 --2- 172.21.15.32:0/791681044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c4200 0x55e4635c56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.087+0000 7fd332113b80 1 --2- 172.21.15.32:0/791681044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5310 0x55e4635d9870 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.087+0000 7fd332113b80 1 -- 172.21.15.32:0/791681044 >> 172.21.15.32:0/791681044 conn(0x55e463538200 msgr2=0x55e4635ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.087+0000 7fd332113b80 1 -- 172.21.15.32:0/791681044 shutdown_connections 2023-09-30T14:43:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.087+0000 7fd332113b80 1 -- 172.21.15.32:0/791681044 wait complete. 2023-09-30T14:43:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.087+0000 7fd332113b80 1 -- start start 2023-09-30T14:43:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd332113b80 1 Processor -- start 2023-09-30T14:43:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd332113b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c4200 0x55e4635d7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd332113b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5cd0 0x55e4635d7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd332113b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635d8570 0x55e4635d89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd332113b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4635d8f30 con 0x55e4635c5cd0 2023-09-30T14:43:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd332113b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4635d90a0 con 0x55e4635d8570 2023-09-30T14:43:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd332113b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4635d9260 con 0x55e4635c4200 2023-09-30T14:43:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd332113b80 10 client.0 _ll_register_callbacks cb 0x7fd31409a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:43:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd31effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5cd0 0x55e4635d7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd31effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5cd0 0x55e4635d7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36806/0 (socket says 172.21.15.32:36806) 2023-09-30T14:43:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd31effd700 1 -- 172.21.15.32:0/3278933471 learned_addr learned my addr 172.21.15.32:0/3278933471 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd31f7fe700 1 --2- 172.21.15.32:0/3278933471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c4200 0x55e4635d7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd31ffff700 1 --2- 172.21.15.32:0/3278933471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635d8570 0x55e4635d89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd31effd700 1 -- 172.21.15.32:0/3278933471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c4200 msgr2=0x55e4635d7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd31effd700 1 --2- 172.21.15.32:0/3278933471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c4200 0x55e4635d7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd31effd700 1 -- 172.21.15.32:0/3278933471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635d8570 msgr2=0x55e4635d89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd31effd700 1 --2- 172.21.15.32:0/3278933471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635d8570 0x55e4635d89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd31effd700 1 -- 172.21.15.32:0/3278933471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e463579870 con 0x55e4635c5cd0 2023-09-30T14:43:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd31effd700 1 --2- 172.21.15.32:0/3278933471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5cd0 0x55e4635d7560 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd314096df0 tx=0x7fd31409a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b89ceaf5a9f3d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.088+0000 7fd2feffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:43:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.089+0000 7fd2feffd700 1 -- 172.21.15.32:0/3278933471 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314098d50 con 0x55e4635c5cd0 2023-09-30T14:43:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.089+0000 7fd332113b80 1 -- 172.21.15.32:0/3278933471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd31800f040 con 0x55e4635c5cd0 2023-09-30T14:43:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.089+0000 7fd2feffd700 1 -- 172.21.15.32:0/3278933471 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd314096a90 con 0x55e4635c5cd0 2023-09-30T14:43:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.089+0000 7fd2feffd700 1 -- 172.21.15.32:0/3278933471 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd31409ea90 con 0x55e4635c5cd0 2023-09-30T14:43:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.089+0000 7fd332113b80 10 client.15972 fetch_fsmap learned FSMap version 34 2023-09-30T14:43:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.089+0000 7fd332113b80 1 -- 172.21.15.32:0/3278933471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e463341d80 con 0x55e4635c5cd0 2023-09-30T14:43:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.089+0000 7fd2feffd700 1 -- 172.21.15.32:0/3278933471 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd31407f080 con 0x55e4635c5cd0 2023-09-30T14:43:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.089+0000 7fd2feffd700 1 -- 172.21.15.32:0/3278933471 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd314096040 con 0x55e4635c5cd0 2023-09-30T14:43:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.089+0000 7fd332113b80 10 client.15972 fetch_fsmap finished waiting for FSMap version 34 2023-09-30T14:43:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.089+0000 7fd332113b80 10 client.15972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:43:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.089+0000 7fd332113b80 20 client.15972 populate_metadata read hostname 'smithi032' 2023-09-30T14:43:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.089+0000 7fd332113b80 1 --2- 172.21.15.32:0/3278933471 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e463632310 0x55e463634700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.089+0000 7fd332113b80 4 client.15972 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:43:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.089+0000 7fd332113b80 1 -- 172.21.15.32:0/3278933471 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e46350fad0 con 0x55e463632310 2023-09-30T14:43:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.089+0000 7fd31f7fe700 1 --2- 172.21.15.32:0/3278933471 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e463632310 0x55e463634700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.090+0000 7fd31f7fe700 1 --2- 172.21.15.32:0/3278933471 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e463632310 0x55e463634700 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8392ab4c164659a server_cookie=d9d975123fc7becc in_seq=0 out_seq=0 2023-09-30T14:43:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.090+0000 7fd2feffd700 10 client.15972 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.123+0000 7fd2feffd700 1 -- 172.21.15.32:0/3278933471 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e46350fad0 con 0x55e463632310 2023-09-30T14:43:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.123+0000 7fd2feffd700 10 client.15972 handle_command_reply: tid=0 2023-09-30T14:43:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.124+0000 7fd332113b80 1 client.15972 shutdown 2023-09-30T14:43:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.124+0000 7fd332113b80 1 -- 172.21.15.32:0/3278933471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5cd0 msgr2=0x55e4635d7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.124+0000 7fd332113b80 1 --2- 172.21.15.32:0/3278933471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5cd0 0x55e4635d7560 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd314096df0 tx=0x7fd31409a060 comp rx=0 tx=0).stop 2023-09-30T14:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.124+0000 7fd332113b80 1 -- 172.21.15.32:0/3278933471 shutdown_connections 2023-09-30T14:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.124+0000 7fd332113b80 1 --2- 172.21.15.32:0/3278933471 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e463632310 0x55e463634700 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.124+0000 7fd332113b80 1 --2- 172.21.15.32:0/3278933471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4635d8570 0x55e4635d89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.124+0000 7fd2feffd700 0 client.15972 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.124+0000 7fd332113b80 1 --2- 172.21.15.32:0/3278933471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e4635c4200 0x55e4635d7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.124+0000 7fd332113b80 1 --2- 172.21.15.32:0/3278933471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e4635c5cd0 0x55e4635d7560 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.124+0000 7fd332113b80 1 -- 172.21.15.32:0/3278933471 >> 172.21.15.32:0/3278933471 conn(0x55e463538200 msgr2=0x55e4635cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.124+0000 7fd332113b80 1 -- 172.21.15.32:0/3278933471 shutdown_connections 2023-09-30T14:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.124+0000 7fd332113b80 1 -- 172.21.15.32:0/3278933471 wait complete. 2023-09-30T14:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.124+0000 7fd332113b80 20 client.15972 trim_cache size 0 max 16384 2023-09-30T14:43:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:43:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:43:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:43:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:43:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.125+0000 7fd325f2b700 1 -- 172.21.15.32:0/537396037 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd3080a1020 msgr2=0x7fd3080a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.125+0000 7fd325f2b700 1 --2- 172.21.15.32:0/537396037 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd3080a1020 0x7fd3080a34e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd320068d20 tx=0x7fd310009260 comp rx=0 tx=0).stop 2023-09-30T14:43:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.125+0000 7fd325f2b700 1 -- 172.21.15.32:0/537396037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd320100f50 msgr2=0x7fd32019be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.125+0000 7fd325f2b700 1 --2- 172.21.15.32:0/537396037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd320100f50 0x7fd32019be00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd314009990 tx=0x7fd31400f690 comp rx=0 tx=0).stop 2023-09-30T14:43:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.125+0000 7fd325f2b700 1 -- 172.21.15.32:0/537396037 shutdown_connections 2023-09-30T14:43:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.125+0000 7fd325f2b700 1 --2- 172.21.15.32:0/537396037 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd3080a1020 0x7fd3080a34e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.125+0000 7fd325f2b700 1 --2- 172.21.15.32:0/537396037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3201018f0 0x7fd32019e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.125+0000 7fd325f2b700 1 --2- 172.21.15.32:0/537396037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3200ffd60 0x7fd3201998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.125+0000 7fd325f2b700 1 --2- 172.21.15.32:0/537396037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd320100f50 0x7fd32019be00 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.125+0000 7fd325f2b700 1 -- 172.21.15.32:0/537396037 >> 172.21.15.32:0/537396037 conn(0x7fd3200fb2b0 msgr2=0x7fd320104490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.125+0000 7fd325f2b700 1 -- 172.21.15.32:0/537396037 shutdown_connections 2023-09-30T14:43:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:33.125+0000 7fd325f2b700 1 -- 172.21.15.32:0/537396037 wait complete. 2023-09-30T14:43:33.701 INFO:tasks.workunit.client.0.smithi032.stdout: 1 510394 10.16 MB/sec execute 127 sec latency 29.909 ms 2023-09-30T14:43:33.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:33 smithi156 ceph-mon[103579]: pgmap v445: 97 pgs: 97 active+clean; 157 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 2.4 MiB/s wr, 205 op/s 2023-09-30T14:43:33.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:33 smithi148 ceph-mon[102355]: pgmap v445: 97 pgs: 97 active+clean; 157 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 2.4 MiB/s wr, 205 op/s 2023-09-30T14:43:33.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:33 smithi032 ceph-mon[98592]: pgmap v445: 97 pgs: 97 active+clean; 157 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 2.4 MiB/s wr, 205 op/s 2023-09-30T14:43:34.701 INFO:tasks.workunit.client.0.smithi032.stdout: 1 512139 10.18 MB/sec execute 128 sec latency 15.031 ms 2023-09-30T14:43:35.701 INFO:tasks.workunit.client.0.smithi032.stdout: 1 514603 10.20 MB/sec execute 129 sec latency 41.843 ms 2023-09-30T14:43:35.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:35 smithi156 ceph-mon[103579]: pgmap v446: 97 pgs: 97 active+clean; 157 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 2.2 MiB/s wr, 200 op/s 2023-09-30T14:43:35.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:35 smithi148 ceph-mon[102355]: pgmap v446: 97 pgs: 97 active+clean; 157 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 2.2 MiB/s wr, 200 op/s 2023-09-30T14:43:35.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:35 smithi032 ceph-mon[98592]: pgmap v446: 97 pgs: 97 active+clean; 157 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 2.2 MiB/s wr, 200 op/s 2023-09-30T14:43:36.701 INFO:tasks.workunit.client.0.smithi032.stdout: 1 516357 10.18 MB/sec execute 130 sec latency 14.407 ms 2023-09-30T14:43:37.701 INFO:tasks.workunit.client.0.smithi032.stdout: 1 518935 10.23 MB/sec execute 131 sec latency 12.224 ms 2023-09-30T14:43:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:37 smithi156 ceph-mon[103579]: pgmap v447: 97 pgs: 97 active+clean; 169 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 3.2 MiB/s wr, 184 op/s 2023-09-30T14:43:37.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:37 smithi148 ceph-mon[102355]: pgmap v447: 97 pgs: 97 active+clean; 169 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 3.2 MiB/s wr, 184 op/s 2023-09-30T14:43:37.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:37 smithi032 ceph-mon[98592]: pgmap v447: 97 pgs: 97 active+clean; 169 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 3.2 MiB/s wr, 184 op/s 2023-09-30T14:43:38.701 INFO:tasks.workunit.client.0.smithi032.stdout: 1 520733 10.20 MB/sec execute 132 sec latency 31.898 ms 2023-09-30T14:43:39.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 522660 10.23 MB/sec execute 133 sec latency 26.200 ms 2023-09-30T14:43:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:39 smithi156 ceph-mon[103579]: pgmap v448: 97 pgs: 97 active+clean; 179 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 3.7 MiB/s wr, 214 op/s 2023-09-30T14:43:39.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:39 smithi148 ceph-mon[102355]: pgmap v448: 97 pgs: 97 active+clean; 179 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 3.7 MiB/s wr, 214 op/s 2023-09-30T14:43:39.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:39 smithi032 ceph-mon[98592]: pgmap v448: 97 pgs: 97 active+clean; 179 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 3.7 MiB/s wr, 214 op/s 2023-09-30T14:43:40.701 INFO:tasks.workunit.client.0.smithi032.stdout: 1 525146 10.24 MB/sec execute 134 sec latency 14.362 ms 2023-09-30T14:43:41.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 526833 10.23 MB/sec execute 135 sec latency 14.229 ms 2023-09-30T14:43:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:41 smithi156 ceph-mon[103579]: pgmap v449: 97 pgs: 97 active+clean; 180 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 3.5 MiB/s wr, 211 op/s 2023-09-30T14:43:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:41 smithi148 ceph-mon[102355]: pgmap v449: 97 pgs: 97 active+clean; 180 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 3.5 MiB/s wr, 211 op/s 2023-09-30T14:43:41.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:41 smithi032 ceph-mon[98592]: pgmap v449: 97 pgs: 97 active+clean; 180 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 3.5 MiB/s wr, 211 op/s 2023-09-30T14:43:42.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 529546 10.28 MB/sec execute 136 sec latency 9.728 ms 2023-09-30T14:43:43.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 530999 10.24 MB/sec execute 137 sec latency 70.615 ms 2023-09-30T14:43:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:43 smithi156 ceph-mon[103579]: pgmap v450: 97 pgs: 97 active+clean; 182 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 3.6 MiB/s wr, 246 op/s 2023-09-30T14:43:43.933 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:43 smithi032 ceph-mon[98592]: pgmap v450: 97 pgs: 97 active+clean; 182 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 3.6 MiB/s wr, 246 op/s 2023-09-30T14:43:43.933 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:43 smithi148 ceph-mon[102355]: pgmap v450: 97 pgs: 97 active+clean; 182 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 3.6 MiB/s wr, 246 op/s 2023-09-30T14:43:44.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 533319 10.27 MB/sec execute 138 sec latency 16.789 ms 2023-09-30T14:43:45.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 535387 10.25 MB/sec execute 139 sec latency 74.360 ms 2023-09-30T14:43:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:45 smithi156 ceph-mon[103579]: pgmap v451: 97 pgs: 97 active+clean; 183 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 3.5 MiB/s wr, 249 op/s 2023-09-30T14:43:45.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:45 smithi148 ceph-mon[102355]: pgmap v451: 97 pgs: 97 active+clean; 183 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 3.5 MiB/s wr, 249 op/s 2023-09-30T14:43:45.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:45 smithi032 ceph-mon[98592]: pgmap v451: 97 pgs: 97 active+clean; 183 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 3.5 MiB/s wr, 249 op/s 2023-09-30T14:43:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.239+0000 7fdebc807700 1 -- 172.21.15.32:0/907456398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb80ffd60 msgr2=0x7fdeb8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.239+0000 7fdebc807700 1 --2- 172.21.15.32:0/907456398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb80ffd60 0x7fdeb8100180 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdea80099c0 tx=0x7fdea8009cd0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.239+0000 7fdebc807700 1 -- 172.21.15.32:0/907456398 shutdown_connections 2023-09-30T14:43:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.239+0000 7fdebc807700 1 --2- 172.21.15.32:0/907456398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb81018f0 0x7fdeb81081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.239+0000 7fdebc807700 1 --2- 172.21.15.32:0/907456398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb80ffd60 0x7fdeb8100180 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.239+0000 7fdebc807700 1 --2- 172.21.15.32:0/907456398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdeb8100f50 0x7fdeb81013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.239+0000 7fdebc807700 1 -- 172.21.15.32:0/907456398 >> 172.21.15.32:0/907456398 conn(0x7fdeb80fb2b0 msgr2=0x7fdeb80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.239+0000 7fdebc807700 1 -- 172.21.15.32:0/907456398 shutdown_connections 2023-09-30T14:43:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.239+0000 7fdebc807700 1 -- 172.21.15.32:0/907456398 wait complete. 2023-09-30T14:43:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.240+0000 7fdebc807700 1 Processor -- start 2023-09-30T14:43:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.240+0000 7fdebc807700 1 -- start start 2023-09-30T14:43:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.240+0000 7fdebc807700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb80ffd60 0x7fdeb819bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.240+0000 7fdebc807700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb8100f50 0x7fdeb819e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.240+0000 7fdebc807700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdeb81018f0 0x7fdeb81a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.240+0000 7fdebc807700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb8069190 con 0x7fdeb80ffd60 2023-09-30T14:43:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.240+0000 7fdebc807700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb81a0c80 con 0x7fdeb8100f50 2023-09-30T14:43:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.240+0000 7fdebc807700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb81a0df0 con 0x7fdeb81018f0 2023-09-30T14:43:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.240+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb80ffd60 0x7fdeb819bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.240+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb80ffd60 0x7fdeb819bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36818/0 (socket says 172.21.15.32:36818) 2023-09-30T14:43:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.240+0000 7fdeadd9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb8100f50 0x7fdeb819e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.240+0000 7fdeb659c700 1 -- 172.21.15.32:0/3649642130 learned_addr learned my addr 172.21.15.32:0/3649642130 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.240+0000 7fdeb6d9d700 1 --2- 172.21.15.32:0/3649642130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdeb81018f0 0x7fdeb81a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.241+0000 7fdeadd9b700 1 -- 172.21.15.32:0/3649642130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdeb81018f0 msgr2=0x7fdeb81a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.241+0000 7fdeadd9b700 1 --2- 172.21.15.32:0/3649642130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdeb81018f0 0x7fdeb81a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.241+0000 7fdeadd9b700 1 -- 172.21.15.32:0/3649642130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb80ffd60 msgr2=0x7fdeb819bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.241+0000 7fdeadd9b700 1 --2- 172.21.15.32:0/3649642130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb80ffd60 0x7fdeb819bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.241+0000 7fdeadd9b700 1 -- 172.21.15.32:0/3649642130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea80095f0 con 0x7fdeb8100f50 2023-09-30T14:43:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.241+0000 7fdeb6d9d700 1 --2- 172.21.15.32:0/3649642130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdeb81018f0 0x7fdeb81a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.241+0000 7fdeb659c700 1 --2- 172.21.15.32:0/3649642130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb80ffd60 0x7fdeb819bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.241+0000 7fdeadd9b700 1 --2- 172.21.15.32:0/3649642130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb8100f50 0x7fdeb819e1f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdea000e9f0 tx=0x7fdea000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.241+0000 7fdeaffff700 1 -- 172.21.15.32:0/3649642130 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea000c9c0 con 0x7fdeb8100f50 2023-09-30T14:43:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.241+0000 7fdeaffff700 1 -- 172.21.15.32:0/3649642130 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea0010430 con 0x7fdeb8100f50 2023-09-30T14:43:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.241+0000 7fdebc807700 1 -- 172.21.15.32:0/3649642130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb81a10d0 con 0x7fdeb8100f50 2023-09-30T14:43:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.241+0000 7fdeaffff700 1 -- 172.21.15.32:0/3649642130 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea0018680 con 0x7fdeb8100f50 2023-09-30T14:43:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.241+0000 7fdebc807700 1 -- 172.21.15.32:0/3649642130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb81a15e0 con 0x7fdeb8100f50 2023-09-30T14:43:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.243+0000 7fdeaffff700 1 -- 172.21.15.32:0/3649642130 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fdea0010860 con 0x7fdeb8100f50 2023-09-30T14:43:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.244+0000 7fdeaffff700 1 --2- 172.21.15.32:0/3649642130 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdea40a0fa0 0x7fdea40a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.244+0000 7fdeaffff700 1 -- 172.21.15.32:0/3649642130 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdea0014070 con 0x7fdeb8100f50 2023-09-30T14:43:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.244+0000 7fdeb659c700 1 --2- 172.21.15.32:0/3649642130 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdea40a0fa0 0x7fdea40a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.245+0000 7fdeb659c700 1 --2- 172.21.15.32:0/3649642130 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdea40a0fa0 0x7fdea40a3460 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdea8004800 tx=0x7fdea801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.245+0000 7fdec89efb80 1 Processor -- start 2023-09-30T14:43:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.245+0000 7fdec89efb80 1 -- start start 2023-09-30T14:43:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.245+0000 7fdec89efb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x557742756620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.245+0000 7fdec89efb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757310 0x557742757790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.245+0000 7fdec89efb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757cd0 0x55774275e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.245+0000 7fdec89efb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577426be400 con 0x557742756200 2023-09-30T14:43:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.245+0000 7fdec89efb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577426afbc0 con 0x557742757310 2023-09-30T14:43:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.245+0000 7fdec89efb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577426ad4e0 con 0x557742757cd0 2023-09-30T14:43:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.245+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x557742756620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.245+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x557742756620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36826/0 (socket says 172.21.15.32:36826) 2023-09-30T14:43:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.245+0000 7fdeb659c700 1 -- 172.21.15.32:0/2637568990 learned_addr learned my addr 172.21.15.32:0/2637568990 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.245+0000 7fdeb6d9d700 1 --2- 172.21.15.32:0/2637568990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757cd0 0x55774275e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.245+0000 7fdeadd9b700 1 --2- 172.21.15.32:0/2637568990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757310 0x557742757790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.246+0000 7fdeb659c700 1 -- 172.21.15.32:0/2637568990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757cd0 msgr2=0x55774275e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.246+0000 7fdeb659c700 1 --2- 172.21.15.32:0/2637568990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757cd0 0x55774275e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.246+0000 7fdeb659c700 1 -- 172.21.15.32:0/2637568990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757310 msgr2=0x557742757790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.246+0000 7fdeb659c700 1 --2- 172.21.15.32:0/2637568990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757310 0x557742757790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.246+0000 7fdeb659c700 1 -- 172.21.15.32:0/2637568990 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55774270b870 con 0x557742756200 2023-09-30T14:43:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.246+0000 7fdeb659c700 1 --2- 172.21.15.32:0/2637568990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x557742756620 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdea80098e0 tx=0x7fdea801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d8ee21e693a9b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.246+0000 7fdeacd99700 1 -- 172.21.15.32:0/2637568990 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea802a070 con 0x557742756200 2023-09-30T14:43:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.246+0000 7fdeacd99700 1 -- 172.21.15.32:0/2637568990 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea801e040 con 0x557742756200 2023-09-30T14:43:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.246+0000 7fdeacd99700 1 -- 172.21.15.32:0/2637568990 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8005280 con 0x557742756200 2023-09-30T14:43:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.247+0000 7fdec89efb80 1 -- 172.21.15.32:0/2637568990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 msgr2=0x557742756620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.247+0000 7fdec89efb80 1 --2- 172.21.15.32:0/2637568990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x557742756620 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdea80098e0 tx=0x7fdea801f000 comp rx=0 tx=0).stop 2023-09-30T14:43:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.247+0000 7fdec89efb80 1 -- 172.21.15.32:0/2637568990 shutdown_connections 2023-09-30T14:43:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.247+0000 7fdec89efb80 1 --2- 172.21.15.32:0/2637568990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757cd0 0x55774275e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.247+0000 7fdec89efb80 1 --2- 172.21.15.32:0/2637568990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x557742756620 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.247+0000 7fdec89efb80 1 --2- 172.21.15.32:0/2637568990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757310 0x557742757790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.247+0000 7fdec89efb80 1 -- 172.21.15.32:0/2637568990 >> 172.21.15.32:0/2637568990 conn(0x5577426ca200 msgr2=0x557742750ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.247+0000 7fdec89efb80 1 -- 172.21.15.32:0/2637568990 shutdown_connections 2023-09-30T14:43:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.247+0000 7fdec89efb80 1 -- 172.21.15.32:0/2637568990 wait complete. 2023-09-30T14:43:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.247+0000 7fdec89efb80 1 -- start start 2023-09-30T14:43:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.247+0000 7fdec89efb80 1 Processor -- start 2023-09-30T14:43:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdec89efb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742756200 0x55774275fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdec89efb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757310 0x557742760360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdec89efb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742757cd0 0x5577427673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdec89efb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577426a8dd0 con 0x557742757cd0 2023-09-30T14:43:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdec89efb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577426456b0 con 0x557742757310 2023-09-30T14:43:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdec89efb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557742645350 con 0x557742756200 2023-09-30T14:43:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdec89efb80 10 client.0 _ll_register_callbacks cb 0x5577424a27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:43:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdeb6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742757cd0 0x5577427673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742756200 0x55774275fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdeb6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742757cd0 0x5577427673a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36836/0 (socket says 172.21.15.32:36836) 2023-09-30T14:43:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdeb6d9d700 1 -- 172.21.15.32:0/453851603 learned_addr learned my addr 172.21.15.32:0/453851603 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdeadd9b700 1 --2- 172.21.15.32:0/453851603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757310 0x557742760360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdeb6d9d700 1 -- 172.21.15.32:0/453851603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742756200 msgr2=0x55774275fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdeb6d9d700 1 --2- 172.21.15.32:0/453851603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742756200 0x55774275fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdeb6d9d700 1 -- 172.21.15.32:0/453851603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757310 msgr2=0x557742760360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdeb6d9d700 1 --2- 172.21.15.32:0/453851603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757310 0x557742760360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdeb6d9d700 1 -- 172.21.15.32:0/453851603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb8108af0 con 0x557742757cd0 2023-09-30T14:43:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fdeb6d9d700 1 --2- 172.21.15.32:0/453851603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742757cd0 0x5577427673a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0003d50 tx=0x7fdeb000e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1db81393c979158 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.248+0000 7fde9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:43:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.249+0000 7fde9dffb700 1 -- 172.21.15.32:0/453851603 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0003bf0 con 0x557742757cd0 2023-09-30T14:43:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.249+0000 7fde9dffb700 1 -- 172.21.15.32:0/453851603 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdeb00092d0 con 0x557742757cd0 2023-09-30T14:43:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.249+0000 7fdec89efb80 1 -- 172.21.15.32:0/453851603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557742749090 con 0x557742757cd0 2023-09-30T14:43:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.249+0000 7fde9dffb700 1 -- 172.21.15.32:0/453851603 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeb0024030 con 0x557742757cd0 2023-09-30T14:43:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.249+0000 7fdec89efb80 10 client.15984 fetch_fsmap learned FSMap version 34 2023-09-30T14:43:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.249+0000 7fdec89efb80 1 -- 172.21.15.32:0/453851603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5577424d3d80 con 0x557742757cd0 2023-09-30T14:43:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.249+0000 7fde9dffb700 1 -- 172.21.15.32:0/453851603 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557742749090 con 0x557742757cd0 2023-09-30T14:43:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.249+0000 7fde9dffb700 1 -- 172.21.15.32:0/453851603 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fdeb001c020 con 0x557742757cd0 2023-09-30T14:43:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.249+0000 7fdec89efb80 10 client.15984 fetch_fsmap finished waiting for FSMap version 34 2023-09-30T14:43:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.249+0000 7fdec89efb80 10 client.15984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:43:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.249+0000 7fdec89efb80 20 client.15984 populate_metadata read hostname 'smithi032' 2023-09-30T14:43:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.249+0000 7fdec89efb80 1 --2- 172.21.15.32:0/453851603 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55774276a580 0x55774276c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.249+0000 7fdec89efb80 4 client.15984 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:43:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.249+0000 7fdec89efb80 1 -- 172.21.15.32:0/453851603 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577426a1db0 con 0x55774276a580 2023-09-30T14:43:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.250+0000 7fdeb659c700 1 --2- 172.21.15.32:0/453851603 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55774276a580 0x55774276c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.250+0000 7fde9dffb700 10 client.15984 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.250+0000 7fdeb659c700 1 --2- 172.21.15.32:0/453851603 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55774276a580 0x55774276c9a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f06a4a51e2e12e2 server_cookie=6c683dce2dc3fefd in_seq=0 out_seq=0 2023-09-30T14:43:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.251+0000 7fde9dffb700 1 -- 172.21.15.32:0/453851603 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5577426a1db0 con 0x55774276a580 2023-09-30T14:43:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.251+0000 7fde9dffb700 10 client.15984 handle_command_reply: tid=0 2023-09-30T14:43:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.251+0000 7fdec89efb80 1 client.15984 shutdown 2023-09-30T14:43:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.251+0000 7fdec89efb80 1 -- 172.21.15.32:0/453851603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742757cd0 msgr2=0x5577427673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.251+0000 7fdec89efb80 1 --2- 172.21.15.32:0/453851603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742757cd0 0x5577427673a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0003d50 tx=0x7fdeb000e7d0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.251+0000 7fdec89efb80 1 -- 172.21.15.32:0/453851603 shutdown_connections 2023-09-30T14:43:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.251+0000 7fdec89efb80 1 --2- 172.21.15.32:0/453851603 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55774276a580 0x55774276c9a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.251+0000 7fdec89efb80 1 --2- 172.21.15.32:0/453851603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742757cd0 0x5577427673a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.251+0000 7fde9dffb700 0 client.15984 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.251+0000 7fdec89efb80 1 --2- 172.21.15.32:0/453851603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742756200 0x55774275fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.251+0000 7fdec89efb80 1 --2- 172.21.15.32:0/453851603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757310 0x557742760360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.251+0000 7fdec89efb80 1 -- 172.21.15.32:0/453851603 >> 172.21.15.32:0/453851603 conn(0x5577426ca200 msgr2=0x557742750590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.251+0000 7fdec89efb80 1 -- 172.21.15.32:0/453851603 shutdown_connections 2023-09-30T14:43:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.251+0000 7fdec89efb80 1 -- 172.21.15.32:0/453851603 wait complete. 2023-09-30T14:43:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.251+0000 7fdec89efb80 20 client.15984 trim_cache size 0 max 16384 2023-09-30T14:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.266+0000 7fdec89efb80 1 Processor -- start 2023-09-30T14:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.266+0000 7fdec89efb80 1 -- start start 2023-09-30T14:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.266+0000 7fdec89efb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x5577427672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.266+0000 7fdec89efb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757310 0x557742775890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.266+0000 7fdec89efb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757cd0 0x55774275abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.266+0000 7fdec89efb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577425298c0 con 0x557742756200 2023-09-30T14:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.266+0000 7fdec89efb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557742751d40 con 0x557742757310 2023-09-30T14:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.266+0000 7fdec89efb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577427529a0 con 0x557742757cd0 2023-09-30T14:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.266+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x5577427672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.266+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x5577427672c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36840/0 (socket says 172.21.15.32:36840) 2023-09-30T14:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.266+0000 7fdeb659c700 1 -- 172.21.15.32:0/863049552 learned_addr learned my addr 172.21.15.32:0/863049552 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.267+0000 7fdeadd9b700 1 --2- 172.21.15.32:0/863049552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757310 0x557742775890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.267+0000 7fdeb6d9d700 1 --2- 172.21.15.32:0/863049552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757cd0 0x55774275abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.267+0000 7fdeb659c700 1 -- 172.21.15.32:0/863049552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757cd0 msgr2=0x55774275abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.267+0000 7fdeb659c700 1 --2- 172.21.15.32:0/863049552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757cd0 0x55774275abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.267+0000 7fdeb659c700 1 -- 172.21.15.32:0/863049552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757310 msgr2=0x557742775890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.267+0000 7fdeb659c700 1 --2- 172.21.15.32:0/863049552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757310 0x557742775890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.267+0000 7fdeb659c700 1 -- 172.21.15.32:0/863049552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577424bcc00 con 0x557742756200 2023-09-30T14:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.267+0000 7fdeb6d9d700 1 --2- 172.21.15.32:0/863049552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757cd0 0x55774275abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.267+0000 7fdeadd9b700 1 --2- 172.21.15.32:0/863049552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757310 0x557742775890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.267+0000 7fdeb659c700 1 --2- 172.21.15.32:0/863049552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x5577427672c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdea8031040 tx=0x7fdea8005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd27f4aa9fc4103e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.267+0000 7fdead59a700 1 -- 172.21.15.32:0/863049552 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea802a040 con 0x557742756200 2023-09-30T14:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.267+0000 7fdead59a700 1 -- 172.21.15.32:0/863049552 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea802b040 con 0x557742756200 2023-09-30T14:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.267+0000 7fdead59a700 1 -- 172.21.15.32:0/863049552 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea800fc20 con 0x557742756200 2023-09-30T14:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.268+0000 7fdec89efb80 1 -- 172.21.15.32:0/863049552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 msgr2=0x5577427672c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.268+0000 7fdec89efb80 1 --2- 172.21.15.32:0/863049552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x5577427672c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdea8031040 tx=0x7fdea8005a90 comp rx=0 tx=0).stop 2023-09-30T14:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.268+0000 7fdec89efb80 1 -- 172.21.15.32:0/863049552 shutdown_connections 2023-09-30T14:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.268+0000 7fdec89efb80 1 --2- 172.21.15.32:0/863049552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757cd0 0x55774275abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.268+0000 7fdec89efb80 1 --2- 172.21.15.32:0/863049552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x5577427672c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.268+0000 7fdec89efb80 1 --2- 172.21.15.32:0/863049552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757310 0x557742775890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.268+0000 7fdec89efb80 1 -- 172.21.15.32:0/863049552 >> 172.21.15.32:0/863049552 conn(0x5577426ca200 msgr2=0x55774275e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.268+0000 7fdec89efb80 1 -- 172.21.15.32:0/863049552 shutdown_connections 2023-09-30T14:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.268+0000 7fdec89efb80 1 -- 172.21.15.32:0/863049552 wait complete. 2023-09-30T14:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.268+0000 7fdec89efb80 1 -- start start 2023-09-30T14:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.268+0000 7fdec89efb80 1 Processor -- start 2023-09-30T14:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdec89efb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x557742775980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdec89efb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757310 0x557742773f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdec89efb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757cd0 0x5577427744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdec89efb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557742774bb0 con 0x557742756200 2023-09-30T14:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdec89efb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557742774d20 con 0x557742757cd0 2023-09-30T14:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdec89efb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577427bea00 con 0x557742757310 2023-09-30T14:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdec89efb80 10 client.0 _ll_register_callbacks cb 0x7fdeb00043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x557742775980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdeadd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757310 0x557742773f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x557742775980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36846/0 (socket says 172.21.15.32:36846) 2023-09-30T14:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdeb659c700 1 -- 172.21.15.32:0/2787494853 learned_addr learned my addr 172.21.15.32:0/2787494853 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdeb6d9d700 1 --2- 172.21.15.32:0/2787494853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757cd0 0x5577427744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdeadd9b700 1 -- 172.21.15.32:0/2787494853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757cd0 msgr2=0x5577427744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdeadd9b700 1 --2- 172.21.15.32:0/2787494853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757cd0 0x5577427744d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdeadd9b700 1 -- 172.21.15.32:0/2787494853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 msgr2=0x557742775980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdeadd9b700 1 --2- 172.21.15.32:0/2787494853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x557742775980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdeadd9b700 1 -- 172.21.15.32:0/2787494853 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55774270b870 con 0x557742757310 2023-09-30T14:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdeb659c700 1 --2- 172.21.15.32:0/2787494853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x557742775980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fdeadd9b700 1 --2- 172.21.15.32:0/2787494853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757310 0x557742773f90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdea00ca350 tx=0x7fdea009a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f84e14a63b26fc77 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fde9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.269+0000 7fde9dffb700 1 -- 172.21.15.32:0/2787494853 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea0099d10 con 0x557742757310 2023-09-30T14:43:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.270+0000 7fde9dffb700 1 -- 172.21.15.32:0/2787494853 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdea009de60 con 0x557742757310 2023-09-30T14:43:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.270+0000 7fdec89efb80 1 -- 172.21.15.32:0/2787494853 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdea801e040 con 0x557742757310 2023-09-30T14:43:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.270+0000 7fde9dffb700 1 -- 172.21.15.32:0/2787494853 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdea00a20a0 con 0x557742757310 2023-09-30T14:43:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.270+0000 7fdec89efb80 10 client.25666 fetch_fsmap learned FSMap version 34 2023-09-30T14:43:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.270+0000 7fdec89efb80 1 -- 172.21.15.32:0/2787494853 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5577424fa1f0 con 0x557742757310 2023-09-30T14:43:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.270+0000 7fde9dffb700 1 -- 172.21.15.32:0/2787494853 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea801e040 con 0x557742757310 2023-09-30T14:43:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.270+0000 7fde9dffb700 1 -- 172.21.15.32:0/2787494853 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fdea0095b00 con 0x557742757310 2023-09-30T14:43:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.270+0000 7fdec89efb80 10 client.25666 fetch_fsmap finished waiting for FSMap version 34 2023-09-30T14:43:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.270+0000 7fdec89efb80 10 client.25666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:43:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.270+0000 7fdec89efb80 20 client.25666 populate_metadata read hostname 'smithi032' 2023-09-30T14:43:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.270+0000 7fdec89efb80 1 --2- 172.21.15.32:0/2787494853 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5577427c02e0 0x5577427c2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.270+0000 7fdec89efb80 4 client.25666 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:43:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.270+0000 7fdec89efb80 1 -- 172.21.15.32:0/2787494853 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5577426a1ad0 con 0x5577427c02e0 2023-09-30T14:43:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.270+0000 7fdeb659c700 1 --2- 172.21.15.32:0/2787494853 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5577427c02e0 0x5577427c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.271+0000 7fde9dffb700 10 client.25666 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.271+0000 7fdeb659c700 1 --2- 172.21.15.32:0/2787494853 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5577427c02e0 0x5577427c2700 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f26c2f9b6fc6700 server_cookie=dfd9b97e9edf11d1 in_seq=0 out_seq=0 2023-09-30T14:43:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.312+0000 7fde9dffb700 1 -- 172.21.15.32:0/2787494853 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5577426a1ad0 con 0x5577427c02e0 2023-09-30T14:43:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.312+0000 7fde9dffb700 10 client.25666 handle_command_reply: tid=0 2023-09-30T14:43:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.312+0000 7fdec89efb80 1 client.25666 shutdown 2023-09-30T14:43:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.312+0000 7fdec89efb80 1 -- 172.21.15.32:0/2787494853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757310 msgr2=0x557742773f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.312+0000 7fdec89efb80 1 --2- 172.21.15.32:0/2787494853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757310 0x557742773f90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdea00ca350 tx=0x7fdea009a250 comp rx=0 tx=0).stop 2023-09-30T14:43:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.312+0000 7fdec89efb80 1 -- 172.21.15.32:0/2787494853 shutdown_connections 2023-09-30T14:43:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.312+0000 7fdec89efb80 1 --2- 172.21.15.32:0/2787494853 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5577427c02e0 0x5577427c2700 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.312+0000 7fdec89efb80 1 --2- 172.21.15.32:0/2787494853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557742757cd0 0x5577427744d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.312+0000 7fde9dffb700 0 client.25666 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.312+0000 7fdec89efb80 1 --2- 172.21.15.32:0/2787494853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557742756200 0x557742775980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.312+0000 7fdec89efb80 1 --2- 172.21.15.32:0/2787494853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557742757310 0x557742773f90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.312+0000 7fdec89efb80 1 -- 172.21.15.32:0/2787494853 >> 172.21.15.32:0/2787494853 conn(0x5577426ca200 msgr2=0x55774275abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.312+0000 7fdec89efb80 1 -- 172.21.15.32:0/2787494853 shutdown_connections 2023-09-30T14:43:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.312+0000 7fdec89efb80 1 -- 172.21.15.32:0/2787494853 wait complete. 2023-09-30T14:43:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.312+0000 7fdec89efb80 20 client.25666 trim_cache size 0 max 16384 2023-09-30T14:43:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:43:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:43:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:43:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:43:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.313+0000 7fdebc807700 1 -- 172.21.15.32:0/3649642130 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdea40a0fa0 msgr2=0x7fdea40a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.313+0000 7fdebc807700 1 --2- 172.21.15.32:0/3649642130 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdea40a0fa0 0x7fdea40a3460 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdea8004800 tx=0x7fdea801a040 comp rx=0 tx=0).stop 2023-09-30T14:43:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.313+0000 7fdebc807700 1 -- 172.21.15.32:0/3649642130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb8100f50 msgr2=0x7fdeb819e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.313+0000 7fdebc807700 1 --2- 172.21.15.32:0/3649642130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb8100f50 0x7fdeb819e1f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdea000e9f0 tx=0x7fdea000ed00 comp rx=0 tx=0).stop 2023-09-30T14:43:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.313+0000 7fdebc807700 1 -- 172.21.15.32:0/3649642130 shutdown_connections 2023-09-30T14:43:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.313+0000 7fdebc807700 1 --2- 172.21.15.32:0/3649642130 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdea40a0fa0 0x7fdea40a3460 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.313+0000 7fdebc807700 1 --2- 172.21.15.32:0/3649642130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdeb81018f0 0x7fdeb81a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.313+0000 7fdebc807700 1 --2- 172.21.15.32:0/3649642130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb80ffd60 0x7fdeb819bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.313+0000 7fdebc807700 1 --2- 172.21.15.32:0/3649642130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb8100f50 0x7fdeb819e1f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.313+0000 7fdebc807700 1 -- 172.21.15.32:0/3649642130 >> 172.21.15.32:0/3649642130 conn(0x7fdeb80fb2b0 msgr2=0x7fdeb80fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.313+0000 7fdebc807700 1 -- 172.21.15.32:0/3649642130 shutdown_connections 2023-09-30T14:43:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:46.313+0000 7fdebc807700 1 -- 172.21.15.32:0/3649642130 wait complete. 2023-09-30T14:43:46.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 537039 10.26 MB/sec execute 140 sec latency 26.423 ms 2023-09-30T14:43:47.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 539636 10.28 MB/sec execute 141 sec latency 14.004 ms 2023-09-30T14:43:47.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:47 smithi156 ceph-mon[103579]: pgmap v452: 97 pgs: 97 active+clean; 184 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 3.7 MiB/s wr, 248 op/s 2023-09-30T14:43:47.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:47 smithi148 ceph-mon[102355]: pgmap v452: 97 pgs: 97 active+clean; 184 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 3.7 MiB/s wr, 248 op/s 2023-09-30T14:43:47.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:47 smithi032 ceph-mon[98592]: pgmap v452: 97 pgs: 97 active+clean; 184 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 3.7 MiB/s wr, 248 op/s 2023-09-30T14:43:48.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 540913 10.26 MB/sec execute 142 sec latency 50.512 ms 2023-09-30T14:43:49.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 543640 10.30 MB/sec execute 143 sec latency 12.264 ms 2023-09-30T14:43:49.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:49 smithi156 ceph-mon[103579]: pgmap v453: 97 pgs: 97 active+clean; 186 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 2.9 MiB/s wr, 276 op/s 2023-09-30T14:43:49.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:49 smithi148 ceph-mon[102355]: pgmap v453: 97 pgs: 97 active+clean; 186 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 2.9 MiB/s wr, 276 op/s 2023-09-30T14:43:49.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:49 smithi032 ceph-mon[98592]: pgmap v453: 97 pgs: 97 active+clean; 186 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 2.9 MiB/s wr, 276 op/s 2023-09-30T14:43:50.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 544955 10.28 MB/sec execute 144 sec latency 136.046 ms 2023-09-30T14:43:51.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 547322 10.30 MB/sec execute 145 sec latency 54.915 ms 2023-09-30T14:43:51.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:51 smithi156 ceph-mon[103579]: pgmap v454: 97 pgs: 97 active+clean; 186 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 2.2 MiB/s wr, 232 op/s 2023-09-30T14:43:51.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:51 smithi148 ceph-mon[102355]: pgmap v454: 97 pgs: 97 active+clean; 186 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 2.2 MiB/s wr, 232 op/s 2023-09-30T14:43:51.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:51 smithi032 ceph-mon[98592]: pgmap v454: 97 pgs: 97 active+clean; 186 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 2.2 MiB/s wr, 232 op/s 2023-09-30T14:43:52.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 549521 10.28 MB/sec execute 146 sec latency 13.678 ms 2023-09-30T14:43:53.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 551206 10.29 MB/sec execute 147 sec latency 32.900 ms 2023-09-30T14:43:53.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:53 smithi156 ceph-mon[103579]: pgmap v455: 97 pgs: 97 active+clean; 189 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 2.7 MiB/s wr, 292 op/s 2023-09-30T14:43:53.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:53 smithi148 ceph-mon[102355]: pgmap v455: 97 pgs: 97 active+clean; 189 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 2.7 MiB/s wr, 292 op/s 2023-09-30T14:43:53.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:53 smithi032 ceph-mon[98592]: pgmap v455: 97 pgs: 97 active+clean; 189 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 2.7 MiB/s wr, 292 op/s 2023-09-30T14:43:54.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 553963 10.32 MB/sec execute 148 sec latency 15.117 ms 2023-09-30T14:43:55.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 555648 10.32 MB/sec execute 149 sec latency 13.408 ms 2023-09-30T14:43:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:55 smithi156 ceph-mon[103579]: pgmap v456: 97 pgs: 97 active+clean; 190 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.7 MiB/s wr, 282 op/s 2023-09-30T14:43:55.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:55 smithi148 ceph-mon[102355]: pgmap v456: 97 pgs: 97 active+clean; 190 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.7 MiB/s wr, 282 op/s 2023-09-30T14:43:55.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:55 smithi032 ceph-mon[98592]: pgmap v456: 97 pgs: 97 active+clean; 190 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 2.7 MiB/s wr, 282 op/s 2023-09-30T14:43:56.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 558009 10.34 MB/sec execute 150 sec latency 36.255 ms 2023-09-30T14:43:57.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 560306 10.33 MB/sec execute 151 sec latency 13.537 ms 2023-09-30T14:43:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:57 smithi156 ceph-mon[103579]: pgmap v457: 97 pgs: 97 active+clean; 191 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 3.1 MiB/s wr, 303 op/s 2023-09-30T14:43:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:57 smithi148 ceph-mon[102355]: pgmap v457: 97 pgs: 97 active+clean; 191 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 3.1 MiB/s wr, 303 op/s 2023-09-30T14:43:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:57 smithi032 ceph-mon[98592]: pgmap v457: 97 pgs: 97 active+clean; 191 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 3.1 MiB/s wr, 303 op/s 2023-09-30T14:43:58.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 562061 10.34 MB/sec execute 152 sec latency 17.810 ms 2023-09-30T14:43:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.426+0000 7ff976ca3700 1 -- 172.21.15.32:0/3672598741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff970100e80 msgr2=0x7ff9701012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.426+0000 7ff976ca3700 1 --2- 172.21.15.32:0/3672598741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff970100e80 0x7ff9701012a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff964009990 tx=0x7ff964009ca0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.426+0000 7ff976ca3700 1 -- 172.21.15.32:0/3672598741 shutdown_connections 2023-09-30T14:43:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.426+0000 7ff976ca3700 1 --2- 172.21.15.32:0/3672598741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff970102990 0x7ff9701092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.426+0000 7ff976ca3700 1 --2- 172.21.15.32:0/3672598741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff970100e80 0x7ff9701012a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.426+0000 7ff976ca3700 1 --2- 172.21.15.32:0/3672598741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff970101fd0 0x7ff970102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.426+0000 7ff976ca3700 1 -- 172.21.15.32:0/3672598741 >> 172.21.15.32:0/3672598741 conn(0x7ff9700fc3f0 msgr2=0x7ff9700fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.426+0000 7ff976ca3700 1 -- 172.21.15.32:0/3672598741 shutdown_connections 2023-09-30T14:43:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.426+0000 7ff976ca3700 1 -- 172.21.15.32:0/3672598741 wait complete. 2023-09-30T14:43:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.427+0000 7ff976ca3700 1 Processor -- start 2023-09-30T14:43:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.427+0000 7ff976ca3700 1 -- start start 2023-09-30T14:43:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.427+0000 7ff976ca3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff970100e80 0x7ff970196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.427+0000 7ff976ca3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff970101fd0 0x7ff970197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.427+0000 7ff976ca3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff970102990 0x7ff970197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.427+0000 7ff976ca3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff970198660 con 0x7ff970102990 2023-09-30T14:43:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.427+0000 7ff976ca3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9701987d0 con 0x7ff970100e80 2023-09-30T14:43:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.427+0000 7ff976ca3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff970198990 con 0x7ff970101fd0 2023-09-30T14:43:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.428+0000 7ff974a3f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff970100e80 0x7ff970196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.428+0000 7ff974a3f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff970100e80 0x7ff970196ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48336/0 (socket says 172.21.15.32:48336) 2023-09-30T14:43:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.428+0000 7ff974a3f700 1 -- 172.21.15.32:0/3740099287 learned_addr learned my addr 172.21.15.32:0/3740099287 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.428+0000 7ff96ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff970101fd0 0x7ff970197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.428+0000 7ff975240700 1 --2- 172.21.15.32:0/3740099287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff970102990 0x7ff970197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.428+0000 7ff974a3f700 1 -- 172.21.15.32:0/3740099287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff970101fd0 msgr2=0x7ff970197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.428+0000 7ff974a3f700 1 --2- 172.21.15.32:0/3740099287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff970101fd0 0x7ff970197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.428+0000 7ff974a3f700 1 -- 172.21.15.32:0/3740099287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff970102990 msgr2=0x7ff970197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.428+0000 7ff974a3f700 1 --2- 172.21.15.32:0/3740099287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff970102990 0x7ff970197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.428+0000 7ff974a3f700 1 -- 172.21.15.32:0/3740099287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9640095f0 con 0x7ff970100e80 2023-09-30T14:43:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.428+0000 7ff974a3f700 1 --2- 172.21.15.32:0/3740099287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff970100e80 0x7ff970196ce0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff96400b620 tx=0x7ff964004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.428+0000 7ff96dffb700 1 -- 172.21.15.32:0/3740099287 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96401d070 con 0x7ff970100e80 2023-09-30T14:43:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.428+0000 7ff96dffb700 1 -- 172.21.15.32:0/3740099287 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff964005030 con 0x7ff970100e80 2023-09-30T14:43:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.428+0000 7ff976ca3700 1 -- 172.21.15.32:0/3740099287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff970198c10 con 0x7ff970100e80 2023-09-30T14:43:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.428+0000 7ff96dffb700 1 -- 172.21.15.32:0/3740099287 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964004bf0 con 0x7ff970100e80 2023-09-30T14:43:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.429+0000 7ff976ca3700 1 -- 172.21.15.32:0/3740099287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97019f390 con 0x7ff970100e80 2023-09-30T14:43:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.431+0000 7ff96dffb700 1 -- 172.21.15.32:0/3740099287 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7ff964004d50 con 0x7ff970100e80 2023-09-30T14:43:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.432+0000 7ff96dffb700 1 --2- 172.21.15.32:0/3740099287 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff9600a0fa0 0x7ff9600a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.432+0000 7ff96dffb700 1 -- 172.21.15.32:0/3740099287 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff9640cd850 con 0x7ff970100e80 2023-09-30T14:43:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.432+0000 7ff96ffff700 1 --2- 172.21.15.32:0/3740099287 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff9600a0fa0 0x7ff9600a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.432+0000 7ff982e8bb80 1 Processor -- start 2023-09-30T14:43:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.432+0000 7ff982e8bb80 1 -- start start 2023-09-30T14:43:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.432+0000 7ff982e8bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e79200 0x55cc36e79620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.432+0000 7ff982e8bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e7a310 0x55cc36e7a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.432+0000 7ff982e8bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e7acd0 0x55cc36e81560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.432+0000 7ff982e8bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc36de1400 con 0x55cc36e7a310 2023-09-30T14:43:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.432+0000 7ff982e8bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc36dd2bc0 con 0x55cc36e7acd0 2023-09-30T14:43:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.432+0000 7ff982e8bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc36dd04e0 con 0x55cc36e79200 2023-09-30T14:43:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.432+0000 7ff96ffff700 1 --2- 172.21.15.32:0/3740099287 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff9600a0fa0 0x7ff9600a3460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009c60 tx=0x7ff95c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.432+0000 7ff974a3f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e79200 0x55cc36e79620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.432+0000 7ff974a3f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e79200 0x55cc36e79620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51250/0 (socket says 172.21.15.32:51250) 2023-09-30T14:43:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.432+0000 7ff974a3f700 1 -- 172.21.15.32:0/649350417 learned_addr learned my addr 172.21.15.32:0/649350417 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.432+0000 7ff975240700 1 --2- 172.21.15.32:0/649350417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e7acd0 0x55cc36e81560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.433+0000 7ff974a3f700 1 -- 172.21.15.32:0/649350417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e7acd0 msgr2=0x55cc36e81560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.433+0000 7ff974a3f700 1 --2- 172.21.15.32:0/649350417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e7acd0 0x55cc36e81560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.433+0000 7ff974a3f700 1 -- 172.21.15.32:0/649350417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e7a310 msgr2=0x55cc36e7a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.433+0000 7ff974a3f700 1 --2- 172.21.15.32:0/649350417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e7a310 0x55cc36e7a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.433+0000 7ff974a3f700 1 -- 172.21.15.32:0/649350417 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc36e2e870 con 0x55cc36e79200 2023-09-30T14:43:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.433+0000 7ff974a3f700 1 --2- 172.21.15.32:0/649350417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e79200 0x55cc36e79620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff964097250 tx=0x7ff964005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe6fec5ed252c796 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.433+0000 7ff95affd700 1 -- 172.21.15.32:0/649350417 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96409ad50 con 0x55cc36e79200 2023-09-30T14:43:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.433+0000 7ff95affd700 1 -- 172.21.15.32:0/649350417 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff96409eea0 con 0x55cc36e79200 2023-09-30T14:43:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.433+0000 7ff95affd700 1 -- 172.21.15.32:0/649350417 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96409afb0 con 0x55cc36e79200 2023-09-30T14:43:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.434+0000 7ff982e8bb80 1 -- 172.21.15.32:0/649350417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e79200 msgr2=0x55cc36e79620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.434+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/649350417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e79200 0x55cc36e79620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff964097250 tx=0x7ff964005ef0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.434+0000 7ff982e8bb80 1 -- 172.21.15.32:0/649350417 shutdown_connections 2023-09-30T14:43:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.434+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/649350417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e7acd0 0x55cc36e81560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.434+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/649350417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e79200 0x55cc36e79620 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.434+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/649350417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e7a310 0x55cc36e7a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.434+0000 7ff982e8bb80 1 -- 172.21.15.32:0/649350417 >> 172.21.15.32:0/649350417 conn(0x55cc36ded200 msgr2=0x55cc36e73ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.434+0000 7ff982e8bb80 1 -- 172.21.15.32:0/649350417 shutdown_connections 2023-09-30T14:43:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.434+0000 7ff982e8bb80 1 -- 172.21.15.32:0/649350417 wait complete. 2023-09-30T14:43:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.434+0000 7ff982e8bb80 1 -- start start 2023-09-30T14:43:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.434+0000 7ff982e8bb80 1 Processor -- start 2023-09-30T14:43:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff982e8bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e79200 0x55cc36e82ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff982e8bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e7a310 0x55cc36e83010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff982e8bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e7acd0 0x55cc36e8a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff982e8bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc36dcbdd0 con 0x55cc36e7acd0 2023-09-30T14:43:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff982e8bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc36d686b0 con 0x55cc36e79200 2023-09-30T14:43:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff982e8bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc36d68350 con 0x55cc36e7a310 2023-09-30T14:43:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff982e8bb80 10 client.0 _ll_register_callbacks cb 0x55cc36bc57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:43:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff975240700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e7acd0 0x55cc36e8a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff96ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e7a310 0x55cc36e83010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff974a3f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e79200 0x55cc36e82ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff96ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e7a310 0x55cc36e83010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51258/0 (socket says 172.21.15.32:51258) 2023-09-30T14:43:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff96ffff700 1 -- 172.21.15.32:0/3170307937 learned_addr learned my addr 172.21.15.32:0/3170307937 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff974a3f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e79200 0x55cc36e82ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48350/0 (socket says 172.21.15.32:48350) 2023-09-30T14:43:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff975240700 1 -- 172.21.15.32:0/3170307937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e7a310 msgr2=0x55cc36e83010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff975240700 1 --2- 172.21.15.32:0/3170307937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e7a310 0x55cc36e83010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff975240700 1 -- 172.21.15.32:0/3170307937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e79200 msgr2=0x55cc36e82ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff975240700 1 --2- 172.21.15.32:0/3170307937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e79200 0x55cc36e82ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff975240700 1 -- 172.21.15.32:0/3170307937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97019f390 con 0x55cc36e7acd0 2023-09-30T14:43:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff96ffff700 1 --2- 172.21.15.32:0/3170307937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e7a310 0x55cc36e83010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.435+0000 7ff974a3f700 1 --2- 172.21.15.32:0/3170307937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e79200 0x55cc36e82ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.436+0000 7ff975240700 1 --2- 172.21.15.32:0/3170307937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e7acd0 0x55cc36e8a050 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff968007ab0 tx=0x7ff968007ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7410110dc591d24 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.436+0000 7ff94ffff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:43:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.436+0000 7ff94ffff700 1 -- 172.21.15.32:0/3170307937 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96800e030 con 0x55cc36e7acd0 2023-09-30T14:43:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.436+0000 7ff94ffff700 1 -- 172.21.15.32:0/3170307937 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff968004db0 con 0x55cc36e7acd0 2023-09-30T14:43:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.436+0000 7ff982e8bb80 1 -- 172.21.15.32:0/3170307937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc36e6c090 con 0x55cc36e7acd0 2023-09-30T14:43:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.436+0000 7ff94ffff700 1 -- 172.21.15.32:0/3170307937 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff968016700 con 0x55cc36e7acd0 2023-09-30T14:43:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.436+0000 7ff982e8bb80 10 client.15999 fetch_fsmap learned FSMap version 34 2023-09-30T14:43:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.436+0000 7ff982e8bb80 1 -- 172.21.15.32:0/3170307937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55cc36bf6d80 con 0x55cc36e7acd0 2023-09-30T14:43:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.436+0000 7ff94ffff700 1 -- 172.21.15.32:0/3170307937 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cc36e6c090 con 0x55cc36e7acd0 2023-09-30T14:43:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.436+0000 7ff94ffff700 1 -- 172.21.15.32:0/3170307937 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff968015980 con 0x55cc36e7acd0 2023-09-30T14:43:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.436+0000 7ff982e8bb80 10 client.15999 fetch_fsmap finished waiting for FSMap version 34 2023-09-30T14:43:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.436+0000 7ff982e8bb80 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:43:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.436+0000 7ff982e8bb80 20 client.15999 populate_metadata read hostname 'smithi032' 2023-09-30T14:43:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.436+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/3170307937 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55cc36e8e930 0x55cc36e90d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.436+0000 7ff982e8bb80 4 client.15999 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:43:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.436+0000 7ff982e8bb80 1 -- 172.21.15.32:0/3170307937 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc36dc4db0 con 0x55cc36e8e930 2023-09-30T14:43:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.437+0000 7ff974a3f700 1 --2- 172.21.15.32:0/3170307937 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55cc36e8e930 0x55cc36e90d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.437+0000 7ff974a3f700 1 --2- 172.21.15.32:0/3170307937 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55cc36e8e930 0x55cc36e90d50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6cf5f56b5be80fb server_cookie=eb373971405ff61 in_seq=0 out_seq=0 2023-09-30T14:43:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.437+0000 7ff94ffff700 10 client.15999 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.438+0000 7ff94ffff700 1 -- 172.21.15.32:0/3170307937 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cc36dc4db0 con 0x55cc36e8e930 2023-09-30T14:43:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.438+0000 7ff94ffff700 10 client.15999 handle_command_reply: tid=0 2023-09-30T14:43:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.438+0000 7ff982e8bb80 1 client.15999 shutdown 2023-09-30T14:43:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.438+0000 7ff982e8bb80 1 -- 172.21.15.32:0/3170307937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e7acd0 msgr2=0x55cc36e8a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.438+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/3170307937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e7acd0 0x55cc36e8a050 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff968007ab0 tx=0x7ff968007ae0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.438+0000 7ff982e8bb80 1 -- 172.21.15.32:0/3170307937 shutdown_connections 2023-09-30T14:43:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.438+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/3170307937 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55cc36e8e930 0x55cc36e90d50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.438+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/3170307937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e7acd0 0x55cc36e8a050 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.438+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/3170307937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e79200 0x55cc36e82ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.438+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/3170307937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e7a310 0x55cc36e83010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.438+0000 7ff94ffff700 0 client.15999 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.438+0000 7ff982e8bb80 1 -- 172.21.15.32:0/3170307937 >> 172.21.15.32:0/3170307937 conn(0x55cc36ded200 msgr2=0x55cc36e73590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.438+0000 7ff982e8bb80 1 -- 172.21.15.32:0/3170307937 shutdown_connections 2023-09-30T14:43:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.438+0000 7ff982e8bb80 1 -- 172.21.15.32:0/3170307937 wait complete. 2023-09-30T14:43:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.438+0000 7ff982e8bb80 20 client.15999 trim_cache size 0 max 16384 2023-09-30T14:43:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.454+0000 7ff982e8bb80 1 Processor -- start 2023-09-30T14:43:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.454+0000 7ff982e8bb80 1 -- start start 2023-09-30T14:43:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.454+0000 7ff982e8bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e79200 0x55cc36e90d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.454+0000 7ff982e8bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e7a310 0x55cc36e8a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.454+0000 7ff982e8bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e7acd0 0x55cc36e7d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.454+0000 7ff982e8bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc36c4c8c0 con 0x55cc36e7a310 2023-09-30T14:43:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.454+0000 7ff982e8bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc36e80cb0 con 0x55cc36e7acd0 2023-09-30T14:43:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.454+0000 7ff982e8bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc36e7e580 con 0x55cc36e79200 2023-09-30T14:43:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.454+0000 7ff96ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e7a310 0x55cc36e8a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.454+0000 7ff96ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e7a310 0x55cc36e8a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36884/0 (socket says 172.21.15.32:36884) 2023-09-30T14:43:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.454+0000 7ff96ffff700 1 -- 172.21.15.32:0/4043062229 learned_addr learned my addr 172.21.15.32:0/4043062229 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.454+0000 7ff975240700 1 --2- 172.21.15.32:0/4043062229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e7acd0 0x55cc36e7d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.454+0000 7ff974a3f700 1 --2- 172.21.15.32:0/4043062229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e79200 0x55cc36e90d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.455+0000 7ff975240700 1 -- 172.21.15.32:0/4043062229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e79200 msgr2=0x55cc36e90d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.455+0000 7ff975240700 1 --2- 172.21.15.32:0/4043062229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e79200 0x55cc36e90d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.455+0000 7ff975240700 1 -- 172.21.15.32:0/4043062229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e7a310 msgr2=0x55cc36e8a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.455+0000 7ff975240700 1 --2- 172.21.15.32:0/4043062229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e7a310 0x55cc36e8a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.455+0000 7ff975240700 1 -- 172.21.15.32:0/4043062229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc36bdfc00 con 0x55cc36e7acd0 2023-09-30T14:43:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.455+0000 7ff975240700 1 --2- 172.21.15.32:0/4043062229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e7acd0 0x55cc36e7d9e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff968015d90 tx=0x7ff968009990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d156a715dab45fb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.455+0000 7ff95b7fe700 1 -- 172.21.15.32:0/4043062229 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96800e070 con 0x55cc36e7acd0 2023-09-30T14:43:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.455+0000 7ff95b7fe700 1 -- 172.21.15.32:0/4043062229 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff96800f040 con 0x55cc36e7acd0 2023-09-30T14:43:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.455+0000 7ff95b7fe700 1 -- 172.21.15.32:0/4043062229 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968015610 con 0x55cc36e7acd0 2023-09-30T14:43:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.456+0000 7ff982e8bb80 1 -- 172.21.15.32:0/4043062229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e7acd0 msgr2=0x55cc36e7d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.456+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/4043062229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e7acd0 0x55cc36e7d9e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff968015d90 tx=0x7ff968009990 comp rx=0 tx=0).stop 2023-09-30T14:43:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.456+0000 7ff982e8bb80 1 -- 172.21.15.32:0/4043062229 shutdown_connections 2023-09-30T14:43:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.456+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/4043062229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e7acd0 0x55cc36e7d9e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.456+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/4043062229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e79200 0x55cc36e90d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.456+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/4043062229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e7a310 0x55cc36e8a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.456+0000 7ff982e8bb80 1 -- 172.21.15.32:0/4043062229 >> 172.21.15.32:0/4043062229 conn(0x55cc36ded200 msgr2=0x55cc36e956b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.456+0000 7ff982e8bb80 1 -- 172.21.15.32:0/4043062229 shutdown_connections 2023-09-30T14:43:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.456+0000 7ff982e8bb80 1 -- 172.21.15.32:0/4043062229 wait complete. 2023-09-30T14:43:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.456+0000 7ff982e8bb80 1 -- start start 2023-09-30T14:43:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.456+0000 7ff982e8bb80 1 Processor -- start 2023-09-30T14:43:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff982e8bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e79200 0x55cc36e879c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff982e8bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e7a310 0x55cc36e87f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff982e8bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e7acd0 0x55cc36e88f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff982e8bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc36e89450 con 0x55cc36e79200 2023-09-30T14:43:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff982e8bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc36e895c0 con 0x55cc36e7a310 2023-09-30T14:43:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff982e8bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc36e89780 con 0x55cc36e7acd0 2023-09-30T14:43:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff982e8bb80 10 client.0 _ll_register_callbacks cb 0x7ff968009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:43:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff974a3f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e79200 0x55cc36e879c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff96ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e7a310 0x55cc36e87f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff974a3f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e79200 0x55cc36e879c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36888/0 (socket says 172.21.15.32:36888) 2023-09-30T14:43:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff975240700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e7acd0 0x55cc36e88f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff974a3f700 1 -- 172.21.15.32:0/1903649058 learned_addr learned my addr 172.21.15.32:0/1903649058 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:43:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff974a3f700 1 -- 172.21.15.32:0/1903649058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e7acd0 msgr2=0x55cc36e88f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff974a3f700 1 --2- 172.21.15.32:0/1903649058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e7acd0 0x55cc36e88f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff974a3f700 1 -- 172.21.15.32:0/1903649058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e7a310 msgr2=0x55cc36e87f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:43:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff974a3f700 1 --2- 172.21.15.32:0/1903649058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e7a310 0x55cc36e87f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff974a3f700 1 -- 172.21.15.32:0/1903649058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc36bf6d80 con 0x55cc36e79200 2023-09-30T14:43:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff975240700 1 --2- 172.21.15.32:0/1903649058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e7acd0 0x55cc36e88f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:43:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff974a3f700 1 --2- 172.21.15.32:0/1903649058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e79200 0x55cc36e879c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff96409ace0 tx=0x7ff96409b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d088b1383a3f00a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:43:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff94ffff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:43:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.457+0000 7ff94ffff700 1 -- 172.21.15.32:0/1903649058 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96409fbe0 con 0x55cc36e79200 2023-09-30T14:43:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.458+0000 7ff982e8bb80 1 -- 172.21.15.32:0/1903649058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff96409eea0 con 0x55cc36e79200 2023-09-30T14:43:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.458+0000 7ff94ffff700 1 -- 172.21.15.32:0/1903649058 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff96409ad50 con 0x55cc36e79200 2023-09-30T14:43:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.458+0000 7ff94ffff700 1 -- 172.21.15.32:0/1903649058 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff96409ab20 con 0x55cc36e79200 2023-09-30T14:43:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.458+0000 7ff982e8bb80 10 client.16005 fetch_fsmap learned FSMap version 34 2023-09-30T14:43:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.458+0000 7ff982e8bb80 1 -- 172.21.15.32:0/1903649058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55cc36c1d1f0 con 0x55cc36e79200 2023-09-30T14:43:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.458+0000 7ff94ffff700 1 -- 172.21.15.32:0/1903649058 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff96409eea0 con 0x55cc36e79200 2023-09-30T14:43:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.458+0000 7ff94ffff700 1 -- 172.21.15.32:0/1903649058 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff96409b0d0 con 0x55cc36e79200 2023-09-30T14:43:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.458+0000 7ff982e8bb80 10 client.16005 fetch_fsmap finished waiting for FSMap version 34 2023-09-30T14:43:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.458+0000 7ff982e8bb80 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:43:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.458+0000 7ff982e8bb80 20 client.16005 populate_metadata read hostname 'smithi032' 2023-09-30T14:43:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.458+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/1903649058 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55cc36ee2eb0 0x55cc36ee52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:43:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.458+0000 7ff982e8bb80 4 client.16005 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:43:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.458+0000 7ff982e8bb80 1 -- 172.21.15.32:0/1903649058 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cc36dc4ad0 con 0x55cc36ee2eb0 2023-09-30T14:43:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.458+0000 7ff96ffff700 1 --2- 172.21.15.32:0/1903649058 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55cc36ee2eb0 0x55cc36ee52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:43:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.459+0000 7ff94ffff700 10 client.16005 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.459+0000 7ff96ffff700 1 --2- 172.21.15.32:0/1903649058 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55cc36ee2eb0 0x55cc36ee52a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=440a57bdf5e4e350 server_cookie=842ef4823cb386b5 in_seq=0 out_seq=0 2023-09-30T14:43:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.512+0000 7ff94ffff700 1 -- 172.21.15.32:0/1903649058 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cc36dc4ad0 con 0x55cc36ee2eb0 2023-09-30T14:43:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.512+0000 7ff94ffff700 10 client.16005 handle_command_reply: tid=0 2023-09-30T14:43:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.512+0000 7ff982e8bb80 1 client.16005 shutdown 2023-09-30T14:43:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.512+0000 7ff982e8bb80 1 -- 172.21.15.32:0/1903649058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e79200 msgr2=0x55cc36e879c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.512+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/1903649058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e79200 0x55cc36e879c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff96409ace0 tx=0x7ff96409b590 comp rx=0 tx=0).stop 2023-09-30T14:43:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.512+0000 7ff982e8bb80 1 -- 172.21.15.32:0/1903649058 shutdown_connections 2023-09-30T14:43:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.512+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/1903649058 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55cc36ee2eb0 0x55cc36ee52a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.512+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/1903649058 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc36e7acd0 0x55cc36e88f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.512+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/1903649058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cc36e79200 0x55cc36e879c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.512+0000 7ff94ffff700 0 client.16005 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:43:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.512+0000 7ff982e8bb80 1 --2- 172.21.15.32:0/1903649058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cc36e7a310 0x55cc36e87f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.512+0000 7ff982e8bb80 1 -- 172.21.15.32:0/1903649058 >> 172.21.15.32:0/1903649058 conn(0x55cc36ded200 msgr2=0x55cc36e7d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.512+0000 7ff982e8bb80 1 -- 172.21.15.32:0/1903649058 shutdown_connections 2023-09-30T14:43:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.512+0000 7ff982e8bb80 1 -- 172.21.15.32:0/1903649058 wait complete. 2023-09-30T14:43:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.512+0000 7ff982e8bb80 20 client.16005 trim_cache size 0 max 16384 2023-09-30T14:43:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:43:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:43:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:43:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:43:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.513+0000 7ff976ca3700 1 -- 172.21.15.32:0/3740099287 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff9600a0fa0 msgr2=0x7ff9600a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.513+0000 7ff976ca3700 1 --2- 172.21.15.32:0/3740099287 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff9600a0fa0 0x7ff9600a3460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009c60 tx=0x7ff95c009260 comp rx=0 tx=0).stop 2023-09-30T14:43:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.513+0000 7ff976ca3700 1 -- 172.21.15.32:0/3740099287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff970100e80 msgr2=0x7ff970196ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:43:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.513+0000 7ff976ca3700 1 --2- 172.21.15.32:0/3740099287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff970100e80 0x7ff970196ce0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff96400b620 tx=0x7ff964004730 comp rx=0 tx=0).stop 2023-09-30T14:43:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.513+0000 7ff976ca3700 1 -- 172.21.15.32:0/3740099287 shutdown_connections 2023-09-30T14:43:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.513+0000 7ff976ca3700 1 --2- 172.21.15.32:0/3740099287 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff9600a0fa0 0x7ff9600a3460 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.513+0000 7ff976ca3700 1 --2- 172.21.15.32:0/3740099287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff970102990 0x7ff970197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.513+0000 7ff976ca3700 1 --2- 172.21.15.32:0/3740099287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff970100e80 0x7ff970196ce0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.513+0000 7ff976ca3700 1 --2- 172.21.15.32:0/3740099287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff970101fd0 0x7ff970197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:43:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.513+0000 7ff976ca3700 1 -- 172.21.15.32:0/3740099287 >> 172.21.15.32:0/3740099287 conn(0x7ff9700fc3f0 msgr2=0x7ff9700fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:43:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.513+0000 7ff976ca3700 1 -- 172.21.15.32:0/3740099287 shutdown_connections 2023-09-30T14:43:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:43:59.513+0000 7ff976ca3700 1 -- 172.21.15.32:0/3740099287 wait complete. 2023-09-30T14:43:59.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 564905 10.38 MB/sec execute 153 sec latency 24.358 ms 2023-09-30T14:43:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:43:59 smithi156 ceph-mon[103579]: pgmap v458: 97 pgs: 97 active+clean; 193 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 3.1 MiB/s wr, 327 op/s 2023-09-30T14:43:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:43:59 smithi148 ceph-mon[102355]: pgmap v458: 97 pgs: 97 active+clean; 193 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 3.1 MiB/s wr, 327 op/s 2023-09-30T14:43:59.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:43:59 smithi032 ceph-mon[98592]: pgmap v458: 97 pgs: 97 active+clean; 193 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 3.1 MiB/s wr, 327 op/s 2023-09-30T14:44:00.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 566857 10.36 MB/sec execute 154 sec latency 12.629 ms 2023-09-30T14:44:01.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 568902 10.38 MB/sec execute 155 sec latency 11.364 ms 2023-09-30T14:44:01.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:01 smithi156 ceph-mon[103579]: pgmap v459: 97 pgs: 97 active+clean; 195 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 2.8 MiB/s wr, 269 op/s 2023-09-30T14:44:01.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:01 smithi148 ceph-mon[102355]: pgmap v459: 97 pgs: 97 active+clean; 195 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 2.8 MiB/s wr, 269 op/s 2023-09-30T14:44:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:01 smithi032 ceph-mon[98592]: pgmap v459: 97 pgs: 97 active+clean; 195 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 2.8 MiB/s wr, 269 op/s 2023-09-30T14:44:02.702 INFO:tasks.workunit.client.0.smithi032.stdout: 1 571552 10.40 MB/sec execute 156 sec latency 12.975 ms 2023-09-30T14:44:03.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 572997 10.38 MB/sec execute 157 sec latency 37.299 ms 2023-09-30T14:44:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:03 smithi156 ceph-mon[103579]: pgmap v460: 97 pgs: 97 active+clean; 197 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 2.7 MiB/s wr, 295 op/s 2023-09-30T14:44:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:03 smithi148 ceph-mon[102355]: pgmap v460: 97 pgs: 97 active+clean; 197 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 2.7 MiB/s wr, 295 op/s 2023-09-30T14:44:03.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:03 smithi032 ceph-mon[98592]: pgmap v460: 97 pgs: 97 active+clean; 197 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 2.7 MiB/s wr, 295 op/s 2023-09-30T14:44:04.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 575691 10.42 MB/sec execute 158 sec latency 15.346 ms 2023-09-30T14:44:05.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 577890 10.40 MB/sec execute 159 sec latency 15.938 ms 2023-09-30T14:44:05.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:05 smithi156 ceph-mon[103579]: pgmap v461: 97 pgs: 97 active+clean; 198 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 2.2 MiB/s wr, 237 op/s 2023-09-30T14:44:05.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:05 smithi148 ceph-mon[102355]: pgmap v461: 97 pgs: 97 active+clean; 198 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 2.2 MiB/s wr, 237 op/s 2023-09-30T14:44:05.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:05 smithi032 ceph-mon[98592]: pgmap v461: 97 pgs: 97 active+clean; 198 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 2.2 MiB/s wr, 237 op/s 2023-09-30T14:44:06.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 579768 10.42 MB/sec execute 160 sec latency 13.666 ms 2023-09-30T14:44:07.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 582584 10.46 MB/sec execute 161 sec latency 13.082 ms 2023-09-30T14:44:07.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:07 smithi156 ceph-mon[103579]: pgmap v462: 97 pgs: 97 active+clean; 202 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.6 MiB/s wr, 294 op/s 2023-09-30T14:44:07.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:07 smithi148 ceph-mon[102355]: pgmap v462: 97 pgs: 97 active+clean; 202 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.6 MiB/s wr, 294 op/s 2023-09-30T14:44:07.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:07 smithi032 ceph-mon[98592]: pgmap v462: 97 pgs: 97 active+clean; 202 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.6 MiB/s wr, 294 op/s 2023-09-30T14:44:08.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 583952 10.43 MB/sec execute 162 sec latency 69.561 ms 2023-09-30T14:44:09.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 586383 10.45 MB/sec execute 163 sec latency 14.646 ms 2023-09-30T14:44:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:09 smithi156 ceph-mon[103579]: pgmap v463: 97 pgs: 97 active+clean; 203 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 2.2 MiB/s wr, 273 op/s 2023-09-30T14:44:09.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:09 smithi148 ceph-mon[102355]: pgmap v463: 97 pgs: 97 active+clean; 203 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 2.2 MiB/s wr, 273 op/s 2023-09-30T14:44:09.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:09 smithi032 ceph-mon[98592]: pgmap v463: 97 pgs: 97 active+clean; 203 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 2.2 MiB/s wr, 273 op/s 2023-09-30T14:44:10.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 588562 10.44 MB/sec execute 164 sec latency 14.723 ms 2023-09-30T14:44:11.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 590337 10.45 MB/sec execute 165 sec latency 17.429 ms 2023-09-30T14:44:11.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:11 smithi156 ceph-mon[103579]: pgmap v464: 97 pgs: 97 active+clean; 205 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 2.1 MiB/s wr, 258 op/s 2023-09-30T14:44:11.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:11 smithi148 ceph-mon[102355]: pgmap v464: 97 pgs: 97 active+clean; 205 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 2.1 MiB/s wr, 258 op/s 2023-09-30T14:44:11.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:11 smithi032 ceph-mon[98592]: pgmap v464: 97 pgs: 97 active+clean; 205 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 2.1 MiB/s wr, 258 op/s 2023-09-30T14:44:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.625+0000 7f4513d3c700 1 -- 172.21.15.32:0/3310107229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f450c0ffcd0 msgr2=0x7f450c1000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.625+0000 7f4513d3c700 1 --2- 172.21.15.32:0/3310107229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f450c0ffcd0 0x7f450c1000f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f45080099c0 tx=0x7f4508009cd0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.625+0000 7f4513d3c700 1 -- 172.21.15.32:0/3310107229 shutdown_connections 2023-09-30T14:44:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.625+0000 7f4513d3c700 1 --2- 172.21.15.32:0/3310107229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f450c101860 0x7f450c108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.625+0000 7f4513d3c700 1 --2- 172.21.15.32:0/3310107229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f450c0ffcd0 0x7f450c1000f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.625+0000 7f4513d3c700 1 --2- 172.21.15.32:0/3310107229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f450c100ec0 0x7f450c101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.625+0000 7f4513d3c700 1 -- 172.21.15.32:0/3310107229 >> 172.21.15.32:0/3310107229 conn(0x7f450c0fb260 msgr2=0x7f450c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.625+0000 7f4513d3c700 1 -- 172.21.15.32:0/3310107229 shutdown_connections 2023-09-30T14:44:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.625+0000 7f4513d3c700 1 -- 172.21.15.32:0/3310107229 wait complete. 2023-09-30T14:44:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.625+0000 7f4513d3c700 1 Processor -- start 2023-09-30T14:44:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.625+0000 7f4513d3c700 1 -- start start 2023-09-30T14:44:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4513d3c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f450c0ffcd0 0x7f450c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4513d3c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f450c100ec0 0x7f450c19dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4513d3c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f450c101860 0x7f450c1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4513d3c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c1a0850 con 0x7f450c0ffcd0 2023-09-30T14:44:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4513d3c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c1a09c0 con 0x7f450c100ec0 2023-09-30T14:44:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4513d3c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c1a0b80 con 0x7f450c101860 2023-09-30T14:44:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4511ad8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f450c0ffcd0 0x7f450c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4511ad8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f450c0ffcd0 0x7f450c068db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36902/0 (socket says 172.21.15.32:36902) 2023-09-30T14:44:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4511ad8700 1 -- 172.21.15.32:0/2006981960 learned_addr learned my addr 172.21.15.32:0/2006981960 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f45112d7700 1 --2- 172.21.15.32:0/2006981960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f450c100ec0 0x7f450c19dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f45122d9700 1 --2- 172.21.15.32:0/2006981960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f450c101860 0x7f450c1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4511ad8700 1 -- 172.21.15.32:0/2006981960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f450c101860 msgr2=0x7f450c1a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4511ad8700 1 --2- 172.21.15.32:0/2006981960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f450c101860 0x7f450c1a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4511ad8700 1 -- 172.21.15.32:0/2006981960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f450c100ec0 msgr2=0x7f450c19dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4511ad8700 1 --2- 172.21.15.32:0/2006981960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f450c100ec0 0x7f450c19dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4511ad8700 1 -- 172.21.15.32:0/2006981960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45080095f0 con 0x7f450c0ffcd0 2023-09-30T14:44:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4511ad8700 1 --2- 172.21.15.32:0/2006981960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f450c0ffcd0 0x7f450c068db0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f45080058e0 tx=0x7f450800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4502ffd700 1 -- 172.21.15.32:0/2006981960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450801d070 con 0x7f450c0ffcd0 2023-09-30T14:44:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4513d3c700 1 -- 172.21.15.32:0/2006981960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450c1a0e00 con 0x7f450c0ffcd0 2023-09-30T14:44:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4502ffd700 1 -- 172.21.15.32:0/2006981960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4508005050 con 0x7f450c0ffcd0 2023-09-30T14:44:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.626+0000 7f4502ffd700 1 -- 172.21.15.32:0/2006981960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508004b10 con 0x7f450c0ffcd0 2023-09-30T14:44:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.627+0000 7f4513d3c700 1 -- 172.21.15.32:0/2006981960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450c1a11d0 con 0x7f450c0ffcd0 2023-09-30T14:44:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.629+0000 7f4502ffd700 1 -- 172.21.15.32:0/2006981960 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f45080051c0 con 0x7f450c0ffcd0 2023-09-30T14:44:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.629+0000 7f4502ffd700 1 --2- 172.21.15.32:0/2006981960 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44f80a0fa0 0x7f44f80a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.630+0000 7f45112d7700 1 --2- 172.21.15.32:0/2006981960 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44f80a0fa0 0x7f44f80a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.630+0000 7f4502ffd700 1 -- 172.21.15.32:0/2006981960 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f45080cd560 con 0x7f450c0ffcd0 2023-09-30T14:44:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.630+0000 7f45112d7700 1 --2- 172.21.15.32:0/2006981960 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44f80a0fa0 0x7f44f80a3460 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009c60 tx=0x7f44fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.630+0000 7f451ff24b80 1 Processor -- start 2023-09-30T14:44:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f451ff24b80 1 -- start start 2023-09-30T14:44:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f451ff24b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c3200 0x5624a77c3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f451ff24b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c4310 0x5624a77c4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f451ff24b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4cd0 0x5624a77cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f451ff24b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a772b400 con 0x5624a77c4cd0 2023-09-30T14:44:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f451ff24b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a771cbc0 con 0x5624a77c3200 2023-09-30T14:44:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f451ff24b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a771a4e0 con 0x5624a77c4310 2023-09-30T14:44:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f4511ad8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c3200 0x5624a77c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f45122d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4cd0 0x5624a77cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f45112d7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c4310 0x5624a77c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f4511ad8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c3200 0x5624a77c3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48386/0 (socket says 172.21.15.32:48386) 2023-09-30T14:44:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f45112d7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c4310 0x5624a77c4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51294/0 (socket says 172.21.15.32:51294) 2023-09-30T14:44:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f4511ad8700 1 -- 172.21.15.32:0/3531725725 learned_addr learned my addr 172.21.15.32:0/3531725725 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f4511ad8700 1 -- 172.21.15.32:0/3531725725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c4310 msgr2=0x5624a77c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f4511ad8700 1 --2- 172.21.15.32:0/3531725725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c4310 0x5624a77c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f4511ad8700 1 -- 172.21.15.32:0/3531725725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4cd0 msgr2=0x5624a77cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f4511ad8700 1 --2- 172.21.15.32:0/3531725725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4cd0 0x5624a77cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f4511ad8700 1 -- 172.21.15.32:0/3531725725 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624a7778870 con 0x5624a77c3200 2023-09-30T14:44:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f45122d9700 1 --2- 172.21.15.32:0/3531725725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4cd0 0x5624a77cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:44:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.631+0000 7f4511ad8700 1 --2- 172.21.15.32:0/3531725725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c3200 0x5624a77c3620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f450809aa50 tx=0x7f450809b290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d55a7e72fb9d8f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.632+0000 7f44f7fff700 1 -- 172.21.15.32:0/3531725725 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45080a2dc0 con 0x5624a77c3200 2023-09-30T14:44:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.632+0000 7f44f7fff700 1 -- 172.21.15.32:0/3531725725 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4508098e60 con 0x5624a77c3200 2023-09-30T14:44:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.632+0000 7f44f7fff700 1 -- 172.21.15.32:0/3531725725 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450809af80 con 0x5624a77c3200 2023-09-30T14:44:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.632+0000 7f451ff24b80 1 -- 172.21.15.32:0/3531725725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c3200 msgr2=0x5624a77c3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.632+0000 7f451ff24b80 1 --2- 172.21.15.32:0/3531725725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c3200 0x5624a77c3620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f450809aa50 tx=0x7f450809b290 comp rx=0 tx=0).stop 2023-09-30T14:44:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.632+0000 7f451ff24b80 1 -- 172.21.15.32:0/3531725725 shutdown_connections 2023-09-30T14:44:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.632+0000 7f451ff24b80 1 --2- 172.21.15.32:0/3531725725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4cd0 0x5624a77cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.632+0000 7f451ff24b80 1 --2- 172.21.15.32:0/3531725725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c3200 0x5624a77c3620 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.632+0000 7f451ff24b80 1 --2- 172.21.15.32:0/3531725725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c4310 0x5624a77c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.632+0000 7f451ff24b80 1 -- 172.21.15.32:0/3531725725 >> 172.21.15.32:0/3531725725 conn(0x5624a7737200 msgr2=0x5624a77bdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.633+0000 7f451ff24b80 1 -- 172.21.15.32:0/3531725725 shutdown_connections 2023-09-30T14:44:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.633+0000 7f451ff24b80 1 -- 172.21.15.32:0/3531725725 wait complete. 2023-09-30T14:44:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.633+0000 7f451ff24b80 1 -- start start 2023-09-30T14:44:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.633+0000 7f451ff24b80 1 Processor -- start 2023-09-30T14:44:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.633+0000 7f451ff24b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c3200 0x5624a77cca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.633+0000 7f451ff24b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c4310 0x5624a77ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.633+0000 7f451ff24b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c4cd0 0x5624a77d4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.633+0000 7f451ff24b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a7715dd0 con 0x5624a77c3200 2023-09-30T14:44:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.633+0000 7f451ff24b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a76b26b0 con 0x5624a77c4310 2023-09-30T14:44:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.633+0000 7f451ff24b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a76b2350 con 0x5624a77c4cd0 2023-09-30T14:44:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.633+0000 7f4511ad8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c3200 0x5624a77cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.633+0000 7f451ff24b80 10 client.0 _ll_register_callbacks cb 0x5624a750f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:44:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f4511ad8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c3200 0x5624a77cca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36916/0 (socket says 172.21.15.32:36916) 2023-09-30T14:44:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f4511ad8700 1 -- 172.21.15.32:0/2215030121 learned_addr learned my addr 172.21.15.32:0/2215030121 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f45112d7700 1 --2- 172.21.15.32:0/2215030121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c4310 0x5624a77ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f45122d9700 1 --2- 172.21.15.32:0/2215030121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c4cd0 0x5624a77d4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f4511ad8700 1 -- 172.21.15.32:0/2215030121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c4cd0 msgr2=0x5624a77d4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f4511ad8700 1 --2- 172.21.15.32:0/2215030121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c4cd0 0x5624a77d4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f4511ad8700 1 -- 172.21.15.32:0/2215030121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c4310 msgr2=0x5624a77ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f4511ad8700 1 --2- 172.21.15.32:0/2215030121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c4310 0x5624a77ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f4511ad8700 1 -- 172.21.15.32:0/2215030121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4504003ac0 con 0x5624a77c3200 2023-09-30T14:44:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f4511ad8700 1 --2- 172.21.15.32:0/2215030121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c3200 0x5624a77cca90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f45080a2890 tx=0x7f450809ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c9a89bdb95fff4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f44f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:44:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f44f57fa700 1 -- 172.21.15.32:0/2215030121 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450809a1b0 con 0x5624a77c3200 2023-09-30T14:44:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f44f57fa700 1 -- 172.21.15.32:0/2215030121 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f45080a2dc0 con 0x5624a77c3200 2023-09-30T14:44:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f451ff24b80 1 -- 172.21.15.32:0/2215030121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624a77b6090 con 0x5624a77c3200 2023-09-30T14:44:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f44f57fa700 1 -- 172.21.15.32:0/2215030121 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f450809ad20 con 0x5624a77c3200 2023-09-30T14:44:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f451ff24b80 10 client.16017 fetch_fsmap learned FSMap version 34 2023-09-30T14:44:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.634+0000 7f451ff24b80 1 -- 172.21.15.32:0/2215030121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f450c1a11d0 con 0x5624a77c3200 2023-09-30T14:44:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.635+0000 7f44f57fa700 1 -- 172.21.15.32:0/2215030121 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f450807d020 con 0x5624a77c3200 2023-09-30T14:44:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.635+0000 7f44f57fa700 1 -- 172.21.15.32:0/2215030121 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f45080cd110 con 0x5624a77c3200 2023-09-30T14:44:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.635+0000 7f451ff24b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 34 2023-09-30T14:44:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.635+0000 7f451ff24b80 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:44:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.635+0000 7f451ff24b80 20 client.16017 populate_metadata read hostname 'smithi032' 2023-09-30T14:44:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.635+0000 7f451ff24b80 1 --2- 172.21.15.32:0/2215030121 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5624a77d8490 0x5624a77da8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.635+0000 7f451ff24b80 4 client.16017 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:44:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.635+0000 7f451ff24b80 1 -- 172.21.15.32:0/2215030121 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624a770edb0 con 0x5624a77d8490 2023-09-30T14:44:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.635+0000 7f45112d7700 1 --2- 172.21.15.32:0/2215030121 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5624a77d8490 0x5624a77da8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.635+0000 7f44f57fa700 10 client.16017 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:44:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.635+0000 7f45112d7700 1 --2- 172.21.15.32:0/2215030121 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5624a77d8490 0x5624a77da8b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=588bdd362aa236ae server_cookie=97665c2a24c2caa1 in_seq=0 out_seq=0 2023-09-30T14:44:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.636+0000 7f44f57fa700 1 -- 172.21.15.32:0/2215030121 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5624a770edb0 con 0x5624a77d8490 2023-09-30T14:44:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.636+0000 7f44f57fa700 10 client.16017 handle_command_reply: tid=0 2023-09-30T14:44:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.636+0000 7f451ff24b80 1 client.16017 shutdown 2023-09-30T14:44:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.636+0000 7f451ff24b80 1 -- 172.21.15.32:0/2215030121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c3200 msgr2=0x5624a77cca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.636+0000 7f451ff24b80 1 --2- 172.21.15.32:0/2215030121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c3200 0x5624a77cca90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f45080a2890 tx=0x7f450809ed50 comp rx=0 tx=0).stop 2023-09-30T14:44:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.636+0000 7f451ff24b80 1 -- 172.21.15.32:0/2215030121 shutdown_connections 2023-09-30T14:44:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.636+0000 7f451ff24b80 1 --2- 172.21.15.32:0/2215030121 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5624a77d8490 0x5624a77da8b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.636+0000 7f451ff24b80 1 --2- 172.21.15.32:0/2215030121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c4cd0 0x5624a77d4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.636+0000 7f451ff24b80 1 --2- 172.21.15.32:0/2215030121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c3200 0x5624a77cca90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.636+0000 7f451ff24b80 1 --2- 172.21.15.32:0/2215030121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c4310 0x5624a77ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.636+0000 7f44f57fa700 0 client.16017 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:44:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.636+0000 7f451ff24b80 1 -- 172.21.15.32:0/2215030121 >> 172.21.15.32:0/2215030121 conn(0x5624a7737200 msgr2=0x5624a77bd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.636+0000 7f451ff24b80 1 -- 172.21.15.32:0/2215030121 shutdown_connections 2023-09-30T14:44:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.636+0000 7f451ff24b80 1 -- 172.21.15.32:0/2215030121 wait complete. 2023-09-30T14:44:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.636+0000 7f451ff24b80 20 client.16017 trim_cache size 0 max 16384 2023-09-30T14:44:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f451ff24b80 1 Processor -- start 2023-09-30T14:44:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f451ff24b80 1 -- start start 2023-09-30T14:44:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f451ff24b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c3200 0x5624a77bd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f451ff24b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4310 0x5624a77d3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f451ff24b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c4cd0 0x5624a77c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f451ff24b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a75968c0 con 0x5624a77c4310 2023-09-30T14:44:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f451ff24b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a7788b90 con 0x5624a77c3200 2023-09-30T14:44:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f451ff24b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a77cbf90 con 0x5624a77c4cd0 2023-09-30T14:44:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f45112d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4310 0x5624a77d3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f45112d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4310 0x5624a77d3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36926/0 (socket says 172.21.15.32:36926) 2023-09-30T14:44:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f45112d7700 1 -- 172.21.15.32:0/4061260074 learned_addr learned my addr 172.21.15.32:0/4061260074 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f45122d9700 1 --2- 172.21.15.32:0/4061260074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c4cd0 0x5624a77c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f4511ad8700 1 --2- 172.21.15.32:0/4061260074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c3200 0x5624a77bd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f45112d7700 1 -- 172.21.15.32:0/4061260074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c4cd0 msgr2=0x5624a77c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f45112d7700 1 --2- 172.21.15.32:0/4061260074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c4cd0 0x5624a77c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f45112d7700 1 -- 172.21.15.32:0/4061260074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c3200 msgr2=0x5624a77bd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f45112d7700 1 --2- 172.21.15.32:0/4061260074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c3200 0x5624a77bd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f45112d7700 1 -- 172.21.15.32:0/4061260074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624a7540d80 con 0x5624a77c4310 2023-09-30T14:44:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f45112d7700 1 --2- 172.21.15.32:0/4061260074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4310 0x5624a77d3fd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x5624a77db180 tx=0x7f44fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffbe9d3a77a7c56 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f4500ff9700 1 -- 172.21.15.32:0/4061260074 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc022070 con 0x5624a77c4310 2023-09-30T14:44:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f4500ff9700 1 -- 172.21.15.32:0/4061260074 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44fc023040 con 0x5624a77c4310 2023-09-30T14:44:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.652+0000 7f4500ff9700 1 -- 172.21.15.32:0/4061260074 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc01d410 con 0x5624a77c4310 2023-09-30T14:44:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.653+0000 7f451ff24b80 1 -- 172.21.15.32:0/4061260074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4310 msgr2=0x5624a77d3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.653+0000 7f451ff24b80 1 --2- 172.21.15.32:0/4061260074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4310 0x5624a77d3fd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x5624a77db180 tx=0x7f44fc014000 comp rx=0 tx=0).stop 2023-09-30T14:44:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.653+0000 7f451ff24b80 1 -- 172.21.15.32:0/4061260074 shutdown_connections 2023-09-30T14:44:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.653+0000 7f451ff24b80 1 --2- 172.21.15.32:0/4061260074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c4cd0 0x5624a77c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.653+0000 7f451ff24b80 1 --2- 172.21.15.32:0/4061260074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c3200 0x5624a77bd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.653+0000 7f451ff24b80 1 --2- 172.21.15.32:0/4061260074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4310 0x5624a77d3fd0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.653+0000 7f451ff24b80 1 -- 172.21.15.32:0/4061260074 >> 172.21.15.32:0/4061260074 conn(0x5624a7737200 msgr2=0x5624a77bdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.653+0000 7f451ff24b80 1 -- 172.21.15.32:0/4061260074 shutdown_connections 2023-09-30T14:44:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.653+0000 7f451ff24b80 1 -- 172.21.15.32:0/4061260074 wait complete. 2023-09-30T14:44:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.653+0000 7f451ff24b80 1 -- start start 2023-09-30T14:44:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.653+0000 7f451ff24b80 1 Processor -- start 2023-09-30T14:44:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f451ff24b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c3200 0x5624a77d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f451ff24b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c4310 0x5624a77d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f451ff24b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4cd0 0x5624a77d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f451ff24b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a77d3710 con 0x5624a77c4cd0 2023-09-30T14:44:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f451ff24b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a77d3880 con 0x5624a77c4310 2023-09-30T14:44:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f451ff24b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a77d3a40 con 0x5624a77c3200 2023-09-30T14:44:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f451ff24b80 10 client.0 _ll_register_callbacks cb 0x7f4504009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:44:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f45122d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4cd0 0x5624a77d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f45122d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4cd0 0x5624a77d31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36934/0 (socket says 172.21.15.32:36934) 2023-09-30T14:44:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f45122d9700 1 -- 172.21.15.32:0/952102411 learned_addr learned my addr 172.21.15.32:0/952102411 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f45112d7700 1 --2- 172.21.15.32:0/952102411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c4310 0x5624a77d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f4511ad8700 1 --2- 172.21.15.32:0/952102411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c3200 0x5624a77d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f45122d9700 1 -- 172.21.15.32:0/952102411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c3200 msgr2=0x5624a77d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f45122d9700 1 --2- 172.21.15.32:0/952102411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c3200 0x5624a77d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f45122d9700 1 -- 172.21.15.32:0/952102411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c4310 msgr2=0x5624a77d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f45122d9700 1 --2- 172.21.15.32:0/952102411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c4310 0x5624a77d21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f45122d9700 1 -- 172.21.15.32:0/952102411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44fc017070 con 0x5624a77c4cd0 2023-09-30T14:44:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f45122d9700 1 --2- 172.21.15.32:0/952102411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4cd0 0x5624a77d31d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4504007c80 tx=0x7f4504007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35718cb60dd488ba server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f44f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:44:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f44f57fa700 1 -- 172.21.15.32:0/952102411 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450401c070 con 0x5624a77c4cd0 2023-09-30T14:44:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f451ff24b80 1 -- 172.21.15.32:0/952102411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4508098e60 con 0x5624a77c4cd0 2023-09-30T14:44:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.654+0000 7f44f57fa700 1 -- 172.21.15.32:0/952102411 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4504017850 con 0x5624a77c4cd0 2023-09-30T14:44:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.655+0000 7f44f57fa700 1 -- 172.21.15.32:0/952102411 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4504017a10 con 0x5624a77c4cd0 2023-09-30T14:44:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.655+0000 7f451ff24b80 10 client.16023 fetch_fsmap learned FSMap version 34 2023-09-30T14:44:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.655+0000 7f451ff24b80 1 -- 172.21.15.32:0/952102411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5624a7529c00 con 0x5624a77c4cd0 2023-09-30T14:44:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.655+0000 7f44f57fa700 1 -- 172.21.15.32:0/952102411 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4508098e60 con 0x5624a77c4cd0 2023-09-30T14:44:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.655+0000 7f44f57fa700 1 -- 172.21.15.32:0/952102411 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f4504021cb0 con 0x5624a77c4cd0 2023-09-30T14:44:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.655+0000 7f451ff24b80 10 client.16023 fetch_fsmap finished waiting for FSMap version 34 2023-09-30T14:44:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.655+0000 7f451ff24b80 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:44:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.655+0000 7f451ff24b80 20 client.16023 populate_metadata read hostname 'smithi032' 2023-09-30T14:44:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.655+0000 7f451ff24b80 1 --2- 172.21.15.32:0/952102411 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5624a782d200 0x5624a782f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.655+0000 7f451ff24b80 4 client.16023 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:44:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.655+0000 7f451ff24b80 1 -- 172.21.15.32:0/952102411 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5624a770ead0 con 0x5624a782d200 2023-09-30T14:44:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.655+0000 7f4511ad8700 1 --2- 172.21.15.32:0/952102411 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5624a782d200 0x5624a782f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.656+0000 7f44f57fa700 10 client.16023 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:44:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:12.656+0000 7f4511ad8700 1 --2- 172.21.15.32:0/952102411 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5624a782d200 0x5624a782f5f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d52f3f66cd7ee7 server_cookie=3b56e0909c07dd15 in_seq=0 out_seq=0 2023-09-30T14:44:12.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 592827 10.46 MB/sec execute 166 sec latency 70.028 ms 2023-09-30T14:44:13.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 594077 10.44 MB/sec execute 167 sec latency 107.237 ms 2023-09-30T14:44:13.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:13 smithi156 ceph-mon[103579]: pgmap v465: 97 pgs: 97 active+clean; 206 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 2.6 MiB/s wr, 297 op/s 2023-09-30T14:44:13.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:13 smithi148 ceph-mon[102355]: pgmap v465: 97 pgs: 97 active+clean; 206 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 2.6 MiB/s wr, 297 op/s 2023-09-30T14:44:13.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:13 smithi032 ceph-mon[98592]: pgmap v465: 97 pgs: 97 active+clean; 206 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 2.6 MiB/s wr, 297 op/s 2023-09-30T14:44:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.977+0000 7f44f57fa700 1 -- 172.21.15.32:0/952102411 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5624a770ead0 con 0x5624a782d200 2023-09-30T14:44:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.977+0000 7f44f57fa700 10 client.16023 handle_command_reply: tid=0 2023-09-30T14:44:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.977+0000 7f451ff24b80 1 client.16023 shutdown 2023-09-30T14:44:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.977+0000 7f451ff24b80 1 -- 172.21.15.32:0/952102411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4cd0 msgr2=0x5624a77d31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.977+0000 7f451ff24b80 1 --2- 172.21.15.32:0/952102411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4cd0 0x5624a77d31d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4504007c80 tx=0x7f4504007cb0 comp rx=0 tx=0).stop 2023-09-30T14:44:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.977+0000 7f451ff24b80 1 -- 172.21.15.32:0/952102411 shutdown_connections 2023-09-30T14:44:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.977+0000 7f451ff24b80 1 --2- 172.21.15.32:0/952102411 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5624a782d200 0x5624a782f5f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.977+0000 7f451ff24b80 1 --2- 172.21.15.32:0/952102411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5624a77c4cd0 0x5624a77d31d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.977+0000 7f451ff24b80 1 --2- 172.21.15.32:0/952102411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5624a77c3200 0x5624a77d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.977+0000 7f451ff24b80 1 --2- 172.21.15.32:0/952102411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624a77c4310 0x5624a77d21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.977+0000 7f451ff24b80 1 -- 172.21.15.32:0/952102411 >> 172.21.15.32:0/952102411 conn(0x5624a7737200 msgr2=0x5624a77cc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.977+0000 7f44f57fa700 0 client.16023 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:44:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.977+0000 7f451ff24b80 1 -- 172.21.15.32:0/952102411 shutdown_connections 2023-09-30T14:44:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.977+0000 7f451ff24b80 1 -- 172.21.15.32:0/952102411 wait complete. 2023-09-30T14:44:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.977+0000 7f451ff24b80 20 client.16023 trim_cache size 0 max 16384 2023-09-30T14:44:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:44:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:44:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:44:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:44:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.978+0000 7f4513d3c700 1 -- 172.21.15.32:0/2006981960 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44f80a0fa0 msgr2=0x7f44f80a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.978+0000 7f4513d3c700 1 --2- 172.21.15.32:0/2006981960 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44f80a0fa0 0x7f44f80a3460 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009c60 tx=0x7f44fc009260 comp rx=0 tx=0).stop 2023-09-30T14:44:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.978+0000 7f4513d3c700 1 -- 172.21.15.32:0/2006981960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f450c0ffcd0 msgr2=0x7f450c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.978+0000 7f4513d3c700 1 --2- 172.21.15.32:0/2006981960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f450c0ffcd0 0x7f450c068db0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f45080058e0 tx=0x7f450800f690 comp rx=0 tx=0).stop 2023-09-30T14:44:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.978+0000 7f4513d3c700 1 -- 172.21.15.32:0/2006981960 shutdown_connections 2023-09-30T14:44:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.978+0000 7f4513d3c700 1 --2- 172.21.15.32:0/2006981960 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44f80a0fa0 0x7f44f80a3460 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.978+0000 7f4513d3c700 1 --2- 172.21.15.32:0/2006981960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f450c101860 0x7f450c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.978+0000 7f4513d3c700 1 --2- 172.21.15.32:0/2006981960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f450c0ffcd0 0x7f450c068db0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.978+0000 7f4513d3c700 1 --2- 172.21.15.32:0/2006981960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f450c100ec0 0x7f450c19dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.978+0000 7f4513d3c700 1 -- 172.21.15.32:0/2006981960 >> 172.21.15.32:0/2006981960 conn(0x7f450c0fb260 msgr2=0x7f450c0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.978+0000 7f4513d3c700 1 -- 172.21.15.32:0/2006981960 shutdown_connections 2023-09-30T14:44:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:13.978+0000 7f4513d3c700 1 -- 172.21.15.32:0/2006981960 wait complete. 2023-09-30T14:44:14.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 596534 10.46 MB/sec execute 168 sec latency 50.396 ms 2023-09-30T14:44:15.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 598217 10.45 MB/sec execute 169 sec latency 13.831 ms 2023-09-30T14:44:15.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:15 smithi156 ceph-mon[103579]: pgmap v466: 97 pgs: 97 active+clean; 207 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.6 MiB/s wr, 278 op/s 2023-09-30T14:44:15.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:15 smithi148 ceph-mon[102355]: pgmap v466: 97 pgs: 97 active+clean; 207 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.6 MiB/s wr, 278 op/s 2023-09-30T14:44:15.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:15 smithi032 ceph-mon[98592]: pgmap v466: 97 pgs: 97 active+clean; 207 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.6 MiB/s wr, 278 op/s 2023-09-30T14:44:16.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 600576 10.47 MB/sec execute 170 sec latency 12.307 ms 2023-09-30T14:44:17.703 INFO:tasks.workunit.client.0.smithi032.stdout: 1 602812 10.46 MB/sec execute 171 sec latency 13.962 ms 2023-09-30T14:44:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:17 smithi156 ceph-mon[103579]: pgmap v467: 97 pgs: 97 active+clean; 209 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 3.1 MiB/s wr, 323 op/s 2023-09-30T14:44:18.598 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:17 smithi032 ceph-mon[98592]: pgmap v467: 97 pgs: 97 active+clean; 209 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 3.1 MiB/s wr, 323 op/s 2023-09-30T14:44:18.598 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:17 smithi148 ceph-mon[102355]: pgmap v467: 97 pgs: 97 active+clean; 209 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 3.1 MiB/s wr, 323 op/s 2023-09-30T14:44:18.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 604423 10.46 MB/sec execute 172 sec latency 27.660 ms 2023-09-30T14:44:19.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 606794 10.48 MB/sec execute 173 sec latency 42.105 ms 2023-09-30T14:44:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:19 smithi156 ceph-mon[103579]: pgmap v468: 97 pgs: 97 active+clean; 205 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 2.7 MiB/s wr, 266 op/s 2023-09-30T14:44:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:19 smithi148 ceph-mon[102355]: pgmap v468: 97 pgs: 97 active+clean; 205 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 2.7 MiB/s wr, 266 op/s 2023-09-30T14:44:19.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:19 smithi032 ceph-mon[98592]: pgmap v468: 97 pgs: 97 active+clean; 205 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 2.7 MiB/s wr, 266 op/s 2023-09-30T14:44:20.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 608337 10.46 MB/sec execute 174 sec latency 16.760 ms 2023-09-30T14:44:21.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 611103 10.50 MB/sec execute 175 sec latency 12.875 ms 2023-09-30T14:44:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:21 smithi156 ceph-mon[103579]: pgmap v469: 97 pgs: 97 active+clean; 207 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 2.9 MiB/s wr, 279 op/s 2023-09-30T14:44:21.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:21 smithi148 ceph-mon[102355]: pgmap v469: 97 pgs: 97 active+clean; 207 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 2.9 MiB/s wr, 279 op/s 2023-09-30T14:44:21.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:21 smithi032 ceph-mon[98592]: pgmap v469: 97 pgs: 97 active+clean; 207 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 2.9 MiB/s wr, 279 op/s 2023-09-30T14:44:22.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 612786 10.47 MB/sec execute 176 sec latency 18.251 ms 2023-09-30T14:44:23.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 614826 10.49 MB/sec execute 177 sec latency 28.971 ms 2023-09-30T14:44:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:23 smithi156 ceph-mon[103579]: pgmap v470: 97 pgs: 97 active+clean; 208 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 2.9 MiB/s wr, 273 op/s 2023-09-30T14:44:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:23 smithi148 ceph-mon[102355]: pgmap v470: 97 pgs: 97 active+clean; 208 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 2.9 MiB/s wr, 273 op/s 2023-09-30T14:44:23.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:23 smithi032 ceph-mon[98592]: pgmap v470: 97 pgs: 97 active+clean; 208 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 2.9 MiB/s wr, 273 op/s 2023-09-30T14:44:24.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 616867 10.47 MB/sec execute 178 sec latency 35.905 ms 2023-09-30T14:44:25.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 618603 10.48 MB/sec execute 179 sec latency 12.452 ms 2023-09-30T14:44:25.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:25 smithi156 ceph-mon[103579]: pgmap v471: 97 pgs: 97 active+clean; 211 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 2.8 MiB/s wr, 288 op/s 2023-09-30T14:44:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:25 smithi148 ceph-mon[102355]: pgmap v471: 97 pgs: 97 active+clean; 211 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 2.8 MiB/s wr, 288 op/s 2023-09-30T14:44:25.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:25 smithi032 ceph-mon[98592]: pgmap v471: 97 pgs: 97 active+clean; 211 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 2.8 MiB/s wr, 288 op/s 2023-09-30T14:44:26.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 620958 10.49 MB/sec execute 180 sec latency 40.630 ms 2023-09-30T14:44:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:44:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:44:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:44:26.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:44:26.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:44:26.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:44:26.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:44:26.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:44:26.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:44:26.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:44:26.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:44:26.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:44:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.090+0000 7fb3990a8700 1 -- 172.21.15.32:0/3508713075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394104280 msgr2=0x7fb394106670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.090+0000 7fb3990a8700 1 --2- 172.21.15.32:0/3508713075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394104280 0x7fb394106670 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb388009990 tx=0x7fb388009ca0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.090+0000 7fb3990a8700 1 -- 172.21.15.32:0/3508713075 shutdown_connections 2023-09-30T14:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.090+0000 7fb3990a8700 1 --2- 172.21.15.32:0/3508713075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb394106bb0 0x7fb394108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.090+0000 7fb3990a8700 1 --2- 172.21.15.32:0/3508713075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb394101950 0x7fb394103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.090+0000 7fb3990a8700 1 --2- 172.21.15.32:0/3508713075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394104280 0x7fb394106670 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.090+0000 7fb3990a8700 1 -- 172.21.15.32:0/3508713075 >> 172.21.15.32:0/3508713075 conn(0x7fb3940fb2a0 msgr2=0x7fb3940fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.090+0000 7fb3990a8700 1 -- 172.21.15.32:0/3508713075 shutdown_connections 2023-09-30T14:44:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.091+0000 7fb3990a8700 1 -- 172.21.15.32:0/3508713075 wait complete. 2023-09-30T14:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.091+0000 7fb3990a8700 1 Processor -- start 2023-09-30T14:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb3990a8700 1 -- start start 2023-09-30T14:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb3990a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394101950 0x7fb3940ff900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb3990a8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb394104280 0x7fb3940ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb3990a8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb394106bb0 0x7fb394100380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb3990a8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3941a0b80 con 0x7fb394101950 2023-09-30T14:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb3990a8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3941a0cc0 con 0x7fb394104280 2023-09-30T14:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb3990a8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3941a0e00 con 0x7fb394106bb0 2023-09-30T14:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb392d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394101950 0x7fb3940ff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb392d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394101950 0x7fb3940ff900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36944/0 (socket says 172.21.15.32:36944) 2023-09-30T14:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb392d9d700 1 -- 172.21.15.32:0/2384429801 learned_addr learned my addr 172.21.15.32:0/2384429801 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb39359e700 1 --2- 172.21.15.32:0/2384429801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb394106bb0 0x7fb394100380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb39259c700 1 --2- 172.21.15.32:0/2384429801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb394104280 0x7fb3940ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb39359e700 1 -- 172.21.15.32:0/2384429801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb394104280 msgr2=0x7fb3940ffe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb39359e700 1 --2- 172.21.15.32:0/2384429801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb394104280 0x7fb3940ffe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb39359e700 1 -- 172.21.15.32:0/2384429801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394101950 msgr2=0x7fb3940ff900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb39359e700 1 --2- 172.21.15.32:0/2384429801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394101950 0x7fb3940ff900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb39359e700 1 -- 172.21.15.32:0/2384429801 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3880095f0 con 0x7fb394106bb0 2023-09-30T14:44:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.092+0000 7fb392d9d700 1 --2- 172.21.15.32:0/2384429801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394101950 0x7fb3940ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:44:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.093+0000 7fb39359e700 1 --2- 172.21.15.32:0/2384429801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb394106bb0 0x7fb394100380 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00efc0 tx=0x7fb38c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.093+0000 7fb37bfff700 1 -- 172.21.15.32:0/2384429801 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c018070 con 0x7fb394106bb0 2023-09-30T14:44:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.093+0000 7fb3990a8700 1 -- 172.21.15.32:0/2384429801 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3941a1030 con 0x7fb394106bb0 2023-09-30T14:44:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.093+0000 7fb37bfff700 1 -- 172.21.15.32:0/2384429801 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb38c00f040 con 0x7fb394106bb0 2023-09-30T14:44:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.093+0000 7fb37bfff700 1 -- 172.21.15.32:0/2384429801 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c00cd40 con 0x7fb394106bb0 2023-09-30T14:44:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.093+0000 7fb3990a8700 1 -- 172.21.15.32:0/2384429801 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3941a1540 con 0x7fb394106bb0 2023-09-30T14:44:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.095+0000 7fb37bfff700 1 -- 172.21.15.32:0/2384429801 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fb38c003a00 con 0x7fb394106bb0 2023-09-30T14:44:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.096+0000 7fb37bfff700 1 --2- 172.21.15.32:0/2384429801 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb37c0a0fa0 0x7fb37c0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.096+0000 7fb392d9d700 1 --2- 172.21.15.32:0/2384429801 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb37c0a0fa0 0x7fb37c0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.096+0000 7fb37bfff700 1 -- 172.21.15.32:0/2384429801 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb38c0cc320 con 0x7fb394106bb0 2023-09-30T14:44:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.096+0000 7fb392d9d700 1 --2- 172.21.15.32:0/2384429801 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb37c0a0fa0 0x7fb37c0a3460 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb394101210 tx=0x7fb384009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.096+0000 7fb3a5290b80 1 Processor -- start 2023-09-30T14:44:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.096+0000 7fb3a5290b80 1 -- start start 2023-09-30T14:44:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb3a5290b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c905200 0x56217c905620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb3a5290b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c906310 0x56217c906790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb3a5290b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906cd0 0x56217c90d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb3a5290b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56217c86d400 con 0x56217c906310 2023-09-30T14:44:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb3a5290b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56217c85ebc0 con 0x56217c906cd0 2023-09-30T14:44:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb3a5290b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56217c85c4e0 con 0x56217c905200 2023-09-30T14:44:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb39259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c906310 0x56217c906790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb39359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906cd0 0x56217c90d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb392d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c905200 0x56217c905620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb39359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906cd0 0x56217c90d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48432/0 (socket says 172.21.15.32:48432) 2023-09-30T14:44:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb39259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c906310 0x56217c906790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36954/0 (socket says 172.21.15.32:36954) 2023-09-30T14:44:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb39359e700 1 -- 172.21.15.32:0/385497840 learned_addr learned my addr 172.21.15.32:0/385497840 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb39359e700 1 -- 172.21.15.32:0/385497840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c905200 msgr2=0x56217c905620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb39359e700 1 --2- 172.21.15.32:0/385497840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c905200 0x56217c905620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb39359e700 1 -- 172.21.15.32:0/385497840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c906310 msgr2=0x56217c906790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb39359e700 1 --2- 172.21.15.32:0/385497840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c906310 0x56217c906790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb39359e700 1 -- 172.21.15.32:0/385497840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56217c8ba870 con 0x56217c906cd0 2023-09-30T14:44:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb39359e700 1 --2- 172.21.15.32:0/385497840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906cd0 0x56217c90d560 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0090a0 tx=0x7fb38c0092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac4e938e043b0efb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb3797fa700 1 -- 172.21.15.32:0/385497840 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c099180 con 0x56217c906cd0 2023-09-30T14:44:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb3797fa700 1 -- 172.21.15.32:0/385497840 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb38c004ad0 con 0x56217c906cd0 2023-09-30T14:44:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.097+0000 7fb3797fa700 1 -- 172.21.15.32:0/385497840 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c01b580 con 0x56217c906cd0 2023-09-30T14:44:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.098+0000 7fb3a5290b80 1 -- 172.21.15.32:0/385497840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906cd0 msgr2=0x56217c90d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.098+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/385497840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906cd0 0x56217c90d560 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0090a0 tx=0x7fb38c0092e0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.098+0000 7fb3a5290b80 1 -- 172.21.15.32:0/385497840 shutdown_connections 2023-09-30T14:44:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.098+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/385497840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906cd0 0x56217c90d560 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.098+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/385497840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c905200 0x56217c905620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.098+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/385497840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c906310 0x56217c906790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.098+0000 7fb3a5290b80 1 -- 172.21.15.32:0/385497840 >> 172.21.15.32:0/385497840 conn(0x56217c879200 msgr2=0x56217c8ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.098+0000 7fb3a5290b80 1 -- 172.21.15.32:0/385497840 shutdown_connections 2023-09-30T14:44:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.098+0000 7fb3a5290b80 1 -- 172.21.15.32:0/385497840 wait complete. 2023-09-30T14:44:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.098+0000 7fb3a5290b80 1 -- start start 2023-09-30T14:44:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb3a5290b80 1 Processor -- start 2023-09-30T14:44:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb3a5290b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c905200 0x56217c90ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb3a5290b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906310 0x56217c90f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb3a5290b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906cd0 0x56217c916480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb3a5290b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56217c857dd0 con 0x56217c905200 2023-09-30T14:44:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb3a5290b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56217c7f46b0 con 0x56217c906cd0 2023-09-30T14:44:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb3a5290b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56217c7f4350 con 0x56217c906310 2023-09-30T14:44:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb3a5290b80 10 client.0 _ll_register_callbacks cb 0x56217c6517a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:44:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb39259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906310 0x56217c90f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb39359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906cd0 0x56217c916480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb39259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906310 0x56217c90f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51342/0 (socket says 172.21.15.32:51342) 2023-09-30T14:44:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb39259c700 1 -- 172.21.15.32:0/3957992859 learned_addr learned my addr 172.21.15.32:0/3957992859 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb392d9d700 1 --2- 172.21.15.32:0/3957992859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c905200 0x56217c90ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb39359e700 1 -- 172.21.15.32:0/3957992859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906310 msgr2=0x56217c90f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb39359e700 1 --2- 172.21.15.32:0/3957992859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906310 0x56217c90f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb39359e700 1 -- 172.21.15.32:0/3957992859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c905200 msgr2=0x56217c90ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb39359e700 1 --2- 172.21.15.32:0/3957992859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c905200 0x56217c90ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb39359e700 1 -- 172.21.15.32:0/3957992859 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3941a1540 con 0x56217c906cd0 2023-09-30T14:44:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.099+0000 7fb39259c700 1 --2- 172.21.15.32:0/3957992859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906310 0x56217c90f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:44:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.100+0000 7fb39359e700 1 --2- 172.21.15.32:0/3957992859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906cd0 0x56217c916480 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0a15f0 tx=0x7fb38c099210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a340887336930cb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.100+0000 7fb376ff5700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:44:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.100+0000 7fb376ff5700 1 -- 172.21.15.32:0/3957992859 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c0842c0 con 0x56217c906cd0 2023-09-30T14:44:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.100+0000 7fb376ff5700 1 -- 172.21.15.32:0/3957992859 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb38c09d970 con 0x56217c906cd0 2023-09-30T14:44:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.100+0000 7fb3a5290b80 1 -- 172.21.15.32:0/3957992859 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56217c8f8090 con 0x56217c906cd0 2023-09-30T14:44:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.100+0000 7fb376ff5700 1 -- 172.21.15.32:0/3957992859 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb38c065030 con 0x56217c906cd0 2023-09-30T14:44:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.100+0000 7fb3a5290b80 10 client.25721 fetch_fsmap learned FSMap version 34 2023-09-30T14:44:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.100+0000 7fb3a5290b80 1 -- 172.21.15.32:0/3957992859 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56217c682d80 con 0x56217c906cd0 2023-09-30T14:44:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.100+0000 7fb376ff5700 1 -- 172.21.15.32:0/3957992859 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb38c07c020 con 0x56217c906cd0 2023-09-30T14:44:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.100+0000 7fb376ff5700 1 -- 172.21.15.32:0/3957992859 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb38c098e30 con 0x56217c906cd0 2023-09-30T14:44:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.100+0000 7fb3a5290b80 10 client.25721 fetch_fsmap finished waiting for FSMap version 34 2023-09-30T14:44:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.100+0000 7fb3a5290b80 10 client.25721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:44:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.100+0000 7fb3a5290b80 20 client.25721 populate_metadata read hostname 'smithi032' 2023-09-30T14:44:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.100+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/3957992859 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56217c91a9a0 0x56217c91cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.100+0000 7fb3a5290b80 4 client.25721 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:44:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.100+0000 7fb3a5290b80 1 -- 172.21.15.32:0/3957992859 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56217c850db0 con 0x56217c91a9a0 2023-09-30T14:44:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.101+0000 7fb392d9d700 1 --2- 172.21.15.32:0/3957992859 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56217c91a9a0 0x56217c91cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.101+0000 7fb376ff5700 10 client.25721 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:44:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.101+0000 7fb392d9d700 1 --2- 172.21.15.32:0/3957992859 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56217c91a9a0 0x56217c91cdc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bac2cdfbabba4b9b server_cookie=e0bac27f0786ee5d in_seq=0 out_seq=0 2023-09-30T14:44:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.101+0000 7fb376ff5700 1 -- 172.21.15.32:0/3957992859 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56217c850db0 con 0x56217c91a9a0 2023-09-30T14:44:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.101+0000 7fb376ff5700 10 client.25721 handle_command_reply: tid=0 2023-09-30T14:44:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.102+0000 7fb3a5290b80 1 client.25721 shutdown 2023-09-30T14:44:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.102+0000 7fb3a5290b80 1 -- 172.21.15.32:0/3957992859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906cd0 msgr2=0x56217c916480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.102+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/3957992859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906cd0 0x56217c916480 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0a15f0 tx=0x7fb38c099210 comp rx=0 tx=0).stop 2023-09-30T14:44:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.102+0000 7fb3a5290b80 1 -- 172.21.15.32:0/3957992859 shutdown_connections 2023-09-30T14:44:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.102+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/3957992859 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56217c91a9a0 0x56217c91cdc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.102+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/3957992859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906cd0 0x56217c916480 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.102+0000 7fb376ff5700 0 client.25721 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:44:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.102+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/3957992859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c905200 0x56217c90ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.102+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/3957992859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906310 0x56217c90f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.102+0000 7fb3a5290b80 1 -- 172.21.15.32:0/3957992859 >> 172.21.15.32:0/3957992859 conn(0x56217c879200 msgr2=0x56217c8ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.102+0000 7fb3a5290b80 1 -- 172.21.15.32:0/3957992859 shutdown_connections 2023-09-30T14:44:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.102+0000 7fb3a5290b80 1 -- 172.21.15.32:0/3957992859 wait complete. 2023-09-30T14:44:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.102+0000 7fb3a5290b80 20 client.25721 trim_cache size 0 max 16384 2023-09-30T14:44:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb3a5290b80 1 Processor -- start 2023-09-30T14:44:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb3a5290b80 1 -- start start 2023-09-30T14:44:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb3a5290b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c905200 0x56217c90ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb3a5290b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906310 0x56217c90f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb3a5290b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906cd0 0x56217c916440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb3a5290b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56217c6d88c0 con 0x56217c905200 2023-09-30T14:44:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb3a5290b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56217c900bf0 con 0x56217c906310 2023-09-30T14:44:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb3a5290b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56217c9019a0 con 0x56217c906cd0 2023-09-30T14:44:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb392d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c905200 0x56217c90ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb392d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c905200 0x56217c90ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36966/0 (socket says 172.21.15.32:36966) 2023-09-30T14:44:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb392d9d700 1 -- 172.21.15.32:0/2488240295 learned_addr learned my addr 172.21.15.32:0/2488240295 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb39259c700 1 --2- 172.21.15.32:0/2488240295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906310 0x56217c90f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb39359e700 1 --2- 172.21.15.32:0/2488240295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906cd0 0x56217c916440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb392d9d700 1 -- 172.21.15.32:0/2488240295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906cd0 msgr2=0x56217c916440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb392d9d700 1 --2- 172.21.15.32:0/2488240295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906cd0 0x56217c916440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb392d9d700 1 -- 172.21.15.32:0/2488240295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906310 msgr2=0x56217c90f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb392d9d700 1 --2- 172.21.15.32:0/2488240295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906310 0x56217c90f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb392d9d700 1 -- 172.21.15.32:0/2488240295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56217c66bc00 con 0x56217c905200 2023-09-30T14:44:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.117+0000 7fb39259c700 1 --2- 172.21.15.32:0/2488240295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906310 0x56217c90f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:44:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.118+0000 7fb392d9d700 1 --2- 172.21.15.32:0/2488240295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c905200 0x56217c90ef00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb384013010 tx=0x7fb384014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8818d6340cfe5c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.118+0000 7fb379ffb700 1 -- 172.21.15.32:0/2488240295 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb384024040 con 0x56217c905200 2023-09-30T14:44:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.118+0000 7fb379ffb700 1 -- 172.21.15.32:0/2488240295 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb384029dc0 con 0x56217c905200 2023-09-30T14:44:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.118+0000 7fb379ffb700 1 -- 172.21.15.32:0/2488240295 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38401e9c0 con 0x56217c905200 2023-09-30T14:44:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.118+0000 7fb3a5290b80 1 -- 172.21.15.32:0/2488240295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c905200 msgr2=0x56217c90ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.118+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/2488240295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c905200 0x56217c90ef00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb384013010 tx=0x7fb384014000 comp rx=0 tx=0).stop 2023-09-30T14:44:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.118+0000 7fb3a5290b80 1 -- 172.21.15.32:0/2488240295 shutdown_connections 2023-09-30T14:44:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.118+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/2488240295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906cd0 0x56217c916440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.118+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/2488240295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c905200 0x56217c90ef00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.118+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/2488240295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c906310 0x56217c90f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.118+0000 7fb3a5290b80 1 -- 172.21.15.32:0/2488240295 >> 172.21.15.32:0/2488240295 conn(0x56217c879200 msgr2=0x56217c90dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.118+0000 7fb3a5290b80 1 -- 172.21.15.32:0/2488240295 shutdown_connections 2023-09-30T14:44:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.118+0000 7fb3a5290b80 1 -- 172.21.15.32:0/2488240295 wait complete. 2023-09-30T14:44:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb3a5290b80 1 -- start start 2023-09-30T14:44:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb3a5290b80 1 Processor -- start 2023-09-30T14:44:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb3a5290b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c905200 0x56217c914100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb3a5290b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906310 0x56217c914640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb3a5290b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c906cd0 0x56217c915650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb3a5290b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56217c915b90 con 0x56217c906cd0 2023-09-30T14:44:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb3a5290b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56217c915d00 con 0x56217c905200 2023-09-30T14:44:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb3a5290b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56217c915ec0 con 0x56217c906310 2023-09-30T14:44:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb3a5290b80 10 client.0 _ll_register_callbacks cb 0x7fb38c0097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:44:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb39359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c906cd0 0x56217c915650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb39359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c906cd0 0x56217c915650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36976/0 (socket says 172.21.15.32:36976) 2023-09-30T14:44:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb39359e700 1 -- 172.21.15.32:0/3200783535 learned_addr learned my addr 172.21.15.32:0/3200783535 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb39259c700 1 --2- 172.21.15.32:0/3200783535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906310 0x56217c914640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb392d9d700 1 --2- 172.21.15.32:0/3200783535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c905200 0x56217c914100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb39359e700 1 -- 172.21.15.32:0/3200783535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906310 msgr2=0x56217c914640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb39359e700 1 --2- 172.21.15.32:0/3200783535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906310 0x56217c914640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb39359e700 1 -- 172.21.15.32:0/3200783535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c905200 msgr2=0x56217c914100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb39359e700 1 --2- 172.21.15.32:0/3200783535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c905200 0x56217c914100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb39359e700 1 -- 172.21.15.32:0/3200783535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb384017070 con 0x56217c906cd0 2023-09-30T14:44:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.119+0000 7fb392d9d700 1 --2- 172.21.15.32:0/3200783535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c905200 0x56217c914100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:44:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb39359e700 1 --2- 172.21.15.32:0/3200783535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c906cd0 0x56217c915650 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb38c095d20 tx=0x7fb38c0653b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5292bc6cf4ff17b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb376ff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:44:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb376ff5700 1 -- 172.21.15.32:0/3200783535 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c065040 con 0x56217c906cd0 2023-09-30T14:44:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb3a5290b80 1 -- 172.21.15.32:0/3200783535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb38c004ad0 con 0x56217c906cd0 2023-09-30T14:44:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb376ff5700 1 -- 172.21.15.32:0/3200783535 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb38c07d080 con 0x56217c906cd0 2023-09-30T14:44:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb376ff5700 1 -- 172.21.15.32:0/3200783535 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb38c099f20 con 0x56217c906cd0 2023-09-30T14:44:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb3a5290b80 10 client.16041 fetch_fsmap learned FSMap version 34 2023-09-30T14:44:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb3a5290b80 1 -- 172.21.15.32:0/3200783535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56217c6a91f0 con 0x56217c906cd0 2023-09-30T14:44:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb376ff5700 1 -- 172.21.15.32:0/3200783535 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb38c004ad0 con 0x56217c906cd0 2023-09-30T14:44:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb376ff5700 1 -- 172.21.15.32:0/3200783535 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb38c098d10 con 0x56217c906cd0 2023-09-30T14:44:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb3a5290b80 10 client.16041 fetch_fsmap finished waiting for FSMap version 34 2023-09-30T14:44:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb3a5290b80 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:44:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb3a5290b80 20 client.16041 populate_metadata read hostname 'smithi032' 2023-09-30T14:44:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/3200783535 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56217c96f200 0x56217c9715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb3a5290b80 4 client.16041 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:44:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb3a5290b80 1 -- 172.21.15.32:0/3200783535 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56217c850ad0 con 0x56217c96f200 2023-09-30T14:44:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.120+0000 7fb392d9d700 1 --2- 172.21.15.32:0/3200783535 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56217c96f200 0x56217c9715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.121+0000 7fb392d9d700 1 --2- 172.21.15.32:0/3200783535 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56217c96f200 0x56217c9715f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edab9f2a06124611 server_cookie=9f85f4aa28ee67f6 in_seq=0 out_seq=0 2023-09-30T14:44:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.121+0000 7fb376ff5700 10 client.16041 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:44:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.166+0000 7fb376ff5700 1 -- 172.21.15.32:0/3200783535 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56217c850ad0 con 0x56217c96f200 2023-09-30T14:44:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.166+0000 7fb376ff5700 10 client.16041 handle_command_reply: tid=0 2023-09-30T14:44:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.166+0000 7fb3a5290b80 1 client.16041 shutdown 2023-09-30T14:44:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.166+0000 7fb3a5290b80 1 -- 172.21.15.32:0/3200783535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c906cd0 msgr2=0x56217c915650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.166+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/3200783535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c906cd0 0x56217c915650 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb38c095d20 tx=0x7fb38c0653b0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.166+0000 7fb3a5290b80 1 -- 172.21.15.32:0/3200783535 shutdown_connections 2023-09-30T14:44:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.166+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/3200783535 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56217c96f200 0x56217c9715f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.166+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/3200783535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56217c906cd0 0x56217c915650 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.166+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/3200783535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56217c905200 0x56217c914100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.166+0000 7fb3a5290b80 1 --2- 172.21.15.32:0/3200783535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56217c906310 0x56217c914640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.166+0000 7fb376ff5700 0 client.16041 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:44:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.166+0000 7fb3a5290b80 1 -- 172.21.15.32:0/3200783535 >> 172.21.15.32:0/3200783535 conn(0x56217c879200 msgr2=0x56217c90eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.166+0000 7fb3a5290b80 1 -- 172.21.15.32:0/3200783535 shutdown_connections 2023-09-30T14:44:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.166+0000 7fb3a5290b80 1 -- 172.21.15.32:0/3200783535 wait complete. 2023-09-30T14:44:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.166+0000 7fb3a5290b80 20 client.16041 trim_cache size 0 max 16384 2023-09-30T14:44:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:44:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:44:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:44:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:44:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.167+0000 7fb3990a8700 1 -- 172.21.15.32:0/2384429801 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb37c0a0fa0 msgr2=0x7fb37c0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.167+0000 7fb3990a8700 1 --2- 172.21.15.32:0/2384429801 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb37c0a0fa0 0x7fb37c0a3460 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb394101210 tx=0x7fb384009260 comp rx=0 tx=0).stop 2023-09-30T14:44:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.167+0000 7fb3990a8700 1 -- 172.21.15.32:0/2384429801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb394106bb0 msgr2=0x7fb394100380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.167+0000 7fb3990a8700 1 --2- 172.21.15.32:0/2384429801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb394106bb0 0x7fb394100380 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00efc0 tx=0x7fb38c00c3a0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.167+0000 7fb3990a8700 1 -- 172.21.15.32:0/2384429801 shutdown_connections 2023-09-30T14:44:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.167+0000 7fb3990a8700 1 --2- 172.21.15.32:0/2384429801 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb37c0a0fa0 0x7fb37c0a3460 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.167+0000 7fb3990a8700 1 --2- 172.21.15.32:0/2384429801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb394106bb0 0x7fb394100380 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.167+0000 7fb3990a8700 1 --2- 172.21.15.32:0/2384429801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb394101950 0x7fb3940ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.167+0000 7fb3990a8700 1 --2- 172.21.15.32:0/2384429801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb394104280 0x7fb3940ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.167+0000 7fb3990a8700 1 -- 172.21.15.32:0/2384429801 >> 172.21.15.32:0/2384429801 conn(0x7fb3940fb2a0 msgr2=0x7fb3941025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.167+0000 7fb3990a8700 1 -- 172.21.15.32:0/2384429801 shutdown_connections 2023-09-30T14:44:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:27.167+0000 7fb3990a8700 1 -- 172.21.15.32:0/2384429801 wait complete. 2023-09-30T14:44:27.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 622461 10.48 MB/sec execute 181 sec latency 23.036 ms 2023-09-30T14:44:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:27 smithi156 ceph-mon[103579]: pgmap v472: 97 pgs: 97 active+clean; 209 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 3.0 MiB/s wr, 278 op/s 2023-09-30T14:44:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:27 smithi148 ceph-mon[102355]: pgmap v472: 97 pgs: 97 active+clean; 209 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 3.0 MiB/s wr, 278 op/s 2023-09-30T14:44:27.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:27 smithi032 ceph-mon[98592]: pgmap v472: 97 pgs: 97 active+clean; 209 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 3.0 MiB/s wr, 278 op/s 2023-09-30T14:44:28.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 625189 10.51 MB/sec execute 182 sec latency 12.765 ms 2023-09-30T14:44:29.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 626200 10.47 MB/sec execute 183 sec latency 74.863 ms 2023-09-30T14:44:29.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:29 smithi156 ceph-mon[103579]: pgmap v473: 97 pgs: 97 active+clean; 212 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 2.7 MiB/s wr, 272 op/s 2023-09-30T14:44:29.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:29 smithi148 ceph-mon[102355]: pgmap v473: 97 pgs: 97 active+clean; 212 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 2.7 MiB/s wr, 272 op/s 2023-09-30T14:44:29.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:29 smithi032 ceph-mon[98592]: pgmap v473: 97 pgs: 97 active+clean; 212 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 2.7 MiB/s wr, 272 op/s 2023-09-30T14:44:30.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 628821 10.51 MB/sec execute 184 sec latency 54.370 ms 2023-09-30T14:44:31.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 630866 10.48 MB/sec execute 185 sec latency 13.441 ms 2023-09-30T14:44:31.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:31 smithi156 ceph-mon[103579]: pgmap v474: 97 pgs: 97 active+clean; 215 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 3.0 MiB/s wr, 284 op/s 2023-09-30T14:44:31.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:31 smithi148 ceph-mon[102355]: pgmap v474: 97 pgs: 97 active+clean; 215 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 3.0 MiB/s wr, 284 op/s 2023-09-30T14:44:31.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:31 smithi032 ceph-mon[98592]: pgmap v474: 97 pgs: 97 active+clean; 215 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 3.0 MiB/s wr, 284 op/s 2023-09-30T14:44:32.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 632745 10.50 MB/sec execute 186 sec latency 13.604 ms 2023-09-30T14:44:33.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 635521 10.52 MB/sec execute 187 sec latency 15.325 ms 2023-09-30T14:44:33.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:33 smithi156 ceph-mon[103579]: pgmap v475: 97 pgs: 97 active+clean; 214 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 2.9 MiB/s wr, 281 op/s 2023-09-30T14:44:33.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:33 smithi148 ceph-mon[102355]: pgmap v475: 97 pgs: 97 active+clean; 214 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 2.9 MiB/s wr, 281 op/s 2023-09-30T14:44:33.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:33 smithi032 ceph-mon[98592]: pgmap v475: 97 pgs: 97 active+clean; 214 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 2.9 MiB/s wr, 281 op/s 2023-09-30T14:44:34.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 636653 10.49 MB/sec execute 188 sec latency 63.533 ms 2023-09-30T14:44:35.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 639471 10.53 MB/sec execute 189 sec latency 39.630 ms 2023-09-30T14:44:35.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:35 smithi156 ceph-mon[103579]: pgmap v476: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 3.0 MiB/s wr, 314 op/s 2023-09-30T14:44:35.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:35 smithi148 ceph-mon[102355]: pgmap v476: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 3.0 MiB/s wr, 314 op/s 2023-09-30T14:44:35.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:35 smithi032 ceph-mon[98592]: pgmap v476: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 3.0 MiB/s wr, 314 op/s 2023-09-30T14:44:36.704 INFO:tasks.workunit.client.0.smithi032.stdout: 1 641319 10.51 MB/sec execute 190 sec latency 14.071 ms 2023-09-30T14:44:37.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 643305 10.52 MB/sec execute 191 sec latency 9.486 ms 2023-09-30T14:44:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:37 smithi156 ceph-mon[103579]: pgmap v477: 97 pgs: 97 active+clean; 216 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 2.8 MiB/s wr, 273 op/s 2023-09-30T14:44:37.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:37 smithi148 ceph-mon[102355]: pgmap v477: 97 pgs: 97 active+clean; 216 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 2.8 MiB/s wr, 273 op/s 2023-09-30T14:44:37.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:37 smithi032 ceph-mon[98592]: pgmap v477: 97 pgs: 97 active+clean; 216 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 2.8 MiB/s wr, 273 op/s 2023-09-30T14:44:38.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 645812 10.54 MB/sec execute 192 sec latency 14.406 ms 2023-09-30T14:44:39.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 647176 10.52 MB/sec execute 193 sec latency 34.384 ms 2023-09-30T14:44:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:39 smithi156 ceph-mon[103579]: pgmap v478: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.9 MiB/s wr, 316 op/s 2023-09-30T14:44:39.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:39 smithi148 ceph-mon[102355]: pgmap v478: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.9 MiB/s wr, 316 op/s 2023-09-30T14:44:39.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:39 smithi032 ceph-mon[98592]: pgmap v478: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.9 MiB/s wr, 316 op/s 2023-09-30T14:44:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.277+0000 7ff32df99700 1 -- 172.21.15.32:0/1354707236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff328102970 msgr2=0x7ff328109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.277+0000 7ff32df99700 1 --2- 172.21.15.32:0/1354707236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff328102970 0x7ff328109290 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff32000b0a0 tx=0x7ff32000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.277+0000 7ff32df99700 1 -- 172.21.15.32:0/1354707236 shutdown_connections 2023-09-30T14:44:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.277+0000 7ff32df99700 1 --2- 172.21.15.32:0/1354707236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff328102970 0x7ff328109290 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.277+0000 7ff32df99700 1 --2- 172.21.15.32:0/1354707236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff328100e60 0x7ff328101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.277+0000 7ff32df99700 1 --2- 172.21.15.32:0/1354707236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff328101fb0 0x7ff328102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.277+0000 7ff32df99700 1 -- 172.21.15.32:0/1354707236 >> 172.21.15.32:0/1354707236 conn(0x7ff3280fc3d0 msgr2=0x7ff3280fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.277+0000 7ff32df99700 1 -- 172.21.15.32:0/1354707236 shutdown_connections 2023-09-30T14:44:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.277+0000 7ff32df99700 1 -- 172.21.15.32:0/1354707236 wait complete. 2023-09-30T14:44:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.278+0000 7ff32df99700 1 Processor -- start 2023-09-30T14:44:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.278+0000 7ff32df99700 1 -- start start 2023-09-30T14:44:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff32df99700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff328100e60 0x7ff328196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff32df99700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff328101fb0 0x7ff3281972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff32df99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff328102970 0x7ff328198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff32df99700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff328198780 con 0x7ff328102970 2023-09-30T14:44:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff32df99700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3281988f0 con 0x7ff328101fb0 2023-09-30T14:44:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff32df99700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff328198ab0 con 0x7ff328100e60 2023-09-30T14:44:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff327fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff328102970 0x7ff328198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff327fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff328102970 0x7ff328198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37002/0 (socket says 172.21.15.32:37002) 2023-09-30T14:44:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff327fff700 1 -- 172.21.15.32:0/4207453923 learned_addr learned my addr 172.21.15.32:0/4207453923 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff326ffd700 1 --2- 172.21.15.32:0/4207453923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff328101fb0 0x7ff3281972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff3277fe700 1 --2- 172.21.15.32:0/4207453923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff328100e60 0x7ff328196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff327fff700 1 -- 172.21.15.32:0/4207453923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff328100e60 msgr2=0x7ff328196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff327fff700 1 --2- 172.21.15.32:0/4207453923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff328100e60 0x7ff328196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff327fff700 1 -- 172.21.15.32:0/4207453923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff328101fb0 msgr2=0x7ff3281972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff327fff700 1 --2- 172.21.15.32:0/4207453923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff328101fb0 0x7ff3281972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff327fff700 1 -- 172.21.15.32:0/4207453923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff320009d00 con 0x7ff328102970 2023-09-30T14:44:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff327fff700 1 --2- 172.21.15.32:0/4207453923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff328102970 0x7ff328198070 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff32000b780 tx=0x7ff32000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff324ff9700 1 -- 172.21.15.32:0/4207453923 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32000e040 con 0x7ff328102970 2023-09-30T14:44:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff324ff9700 1 -- 172.21.15.32:0/4207453923 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff320004c60 con 0x7ff328102970 2023-09-30T14:44:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff32df99700 1 -- 172.21.15.32:0/4207453923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff328198d30 con 0x7ff328102970 2023-09-30T14:44:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff324ff9700 1 -- 172.21.15.32:0/4207453923 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32001b6b0 con 0x7ff328102970 2023-09-30T14:44:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.279+0000 7ff32df99700 1 -- 172.21.15.32:0/4207453923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32819f4a0 con 0x7ff328102970 2023-09-30T14:44:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.282+0000 7ff324ff9700 1 -- 172.21.15.32:0/4207453923 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7ff32001d480 con 0x7ff328102970 2023-09-30T14:44:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.282+0000 7ff324ff9700 1 --2- 172.21.15.32:0/4207453923 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff3140a0fa0 0x7ff3140a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.282+0000 7ff3277fe700 1 --2- 172.21.15.32:0/4207453923 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff3140a0fa0 0x7ff3140a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.282+0000 7ff324ff9700 1 -- 172.21.15.32:0/4207453923 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff3200cde50 con 0x7ff328102970 2023-09-30T14:44:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.283+0000 7ff3277fe700 1 --2- 172.21.15.32:0/4207453923 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff3140a0fa0 0x7ff3140a3460 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff328196ae0 tx=0x7ff310009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.283+0000 7ff33a181b80 1 Processor -- start 2023-09-30T14:44:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.283+0000 7ff33a181b80 1 -- start start 2023-09-30T14:44:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.283+0000 7ff33a181b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1046610 0x5639a1046a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.283+0000 7ff33a181b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1047720 0x5639a1047ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.283+0000 7ff33a181b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a104e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.283+0000 7ff33a181b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a0faa400 con 0x5639a10480e0 2023-09-30T14:44:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.283+0000 7ff33a181b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a0f9bbc0 con 0x5639a1047720 2023-09-30T14:44:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.283+0000 7ff33a181b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a0f994e0 con 0x5639a1046610 2023-09-30T14:44:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.283+0000 7ff327fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a104e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.283+0000 7ff327fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a104e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37008/0 (socket says 172.21.15.32:37008) 2023-09-30T14:44:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.283+0000 7ff327fff700 1 -- 172.21.15.32:0/187661601 learned_addr learned my addr 172.21.15.32:0/187661601 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.283+0000 7ff326ffd700 1 --2- 172.21.15.32:0/187661601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1047720 0x5639a1047ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.284+0000 7ff3277fe700 1 --2- 172.21.15.32:0/187661601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1046610 0x5639a1046a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.284+0000 7ff327fff700 1 -- 172.21.15.32:0/187661601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1046610 msgr2=0x5639a1046a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.284+0000 7ff327fff700 1 --2- 172.21.15.32:0/187661601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1046610 0x5639a1046a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.284+0000 7ff327fff700 1 -- 172.21.15.32:0/187661601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1047720 msgr2=0x5639a1047ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.284+0000 7ff327fff700 1 --2- 172.21.15.32:0/187661601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1047720 0x5639a1047ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.284+0000 7ff327fff700 1 -- 172.21.15.32:0/187661601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639a0ff7870 con 0x5639a10480e0 2023-09-30T14:44:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.284+0000 7ff327fff700 1 --2- 172.21.15.32:0/187661601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a104e970 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff3200978b0 tx=0x7ff32000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69a547cc1db2ba1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.284+0000 7ff31dffb700 1 -- 172.21.15.32:0/187661601 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32001ce80 con 0x5639a10480e0 2023-09-30T14:44:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.284+0000 7ff31dffb700 1 -- 172.21.15.32:0/187661601 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff32009b3b0 con 0x5639a10480e0 2023-09-30T14:44:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.284+0000 7ff31dffb700 1 -- 172.21.15.32:0/187661601 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32009f170 con 0x5639a10480e0 2023-09-30T14:44:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.285+0000 7ff33a181b80 1 -- 172.21.15.32:0/187661601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 msgr2=0x5639a104e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.285+0000 7ff33a181b80 1 --2- 172.21.15.32:0/187661601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a104e970 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff3200978b0 tx=0x7ff32000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.285+0000 7ff33a181b80 1 -- 172.21.15.32:0/187661601 shutdown_connections 2023-09-30T14:44:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.285+0000 7ff33a181b80 1 --2- 172.21.15.32:0/187661601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a104e970 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.285+0000 7ff33a181b80 1 --2- 172.21.15.32:0/187661601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1046610 0x5639a1046a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.285+0000 7ff33a181b80 1 --2- 172.21.15.32:0/187661601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1047720 0x5639a1047ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.285+0000 7ff33a181b80 1 -- 172.21.15.32:0/187661601 >> 172.21.15.32:0/187661601 conn(0x5639a0fb6200 msgr2=0x5639a103cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.285+0000 7ff33a181b80 1 -- 172.21.15.32:0/187661601 shutdown_connections 2023-09-30T14:44:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.285+0000 7ff33a181b80 1 -- 172.21.15.32:0/187661601 wait complete. 2023-09-30T14:44:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.285+0000 7ff33a181b80 1 -- start start 2023-09-30T14:44:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff33a181b80 1 Processor -- start 2023-09-30T14:44:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff33a181b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1046610 0x5639a1050060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff33a181b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1047720 0x5639a10505a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff33a181b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a10575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff33a181b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a0f94dd0 con 0x5639a10480e0 2023-09-30T14:44:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff33a181b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a0f316b0 con 0x5639a1047720 2023-09-30T14:44:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff33a181b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a0f31350 con 0x5639a1046610 2023-09-30T14:44:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff33a181b80 10 client.0 _ll_register_callbacks cb 0x5639a0d8e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:44:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff327fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a10575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff3277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1046610 0x5639a1050060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff3277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1046610 0x5639a1050060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51394/0 (socket says 172.21.15.32:51394) 2023-09-30T14:44:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff326ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1047720 0x5639a10505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff3277fe700 1 -- 172.21.15.32:0/3670509458 learned_addr learned my addr 172.21.15.32:0/3670509458 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff327fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a10575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37016/0 (socket says 172.21.15.32:37016) 2023-09-30T14:44:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff3277fe700 1 -- 172.21.15.32:0/3670509458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1047720 msgr2=0x5639a10505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff3277fe700 1 --2- 172.21.15.32:0/3670509458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1047720 0x5639a10505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff3277fe700 1 -- 172.21.15.32:0/3670509458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 msgr2=0x5639a10575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff3277fe700 1 --2- 172.21.15.32:0/3670509458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a10575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff3277fe700 1 -- 172.21.15.32:0/3670509458 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32819f4a0 con 0x5639a1046610 2023-09-30T14:44:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff3277fe700 1 --2- 172.21.15.32:0/3670509458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1046610 0x5639a1050060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff310004580 tx=0x7ff3100047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa74b3439d7b70b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.286+0000 7ff306ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:44:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.287+0000 7ff306ffd700 1 -- 172.21.15.32:0/3670509458 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310022070 con 0x5639a1046610 2023-09-30T14:44:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.287+0000 7ff306ffd700 1 -- 172.21.15.32:0/3670509458 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff310004830 con 0x5639a1046610 2023-09-30T14:44:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.287+0000 7ff33a181b80 1 -- 172.21.15.32:0/3670509458 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639a1035090 con 0x5639a1046610 2023-09-30T14:44:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.287+0000 7ff306ffd700 1 -- 172.21.15.32:0/3670509458 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3100045b0 con 0x5639a1046610 2023-09-30T14:44:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.287+0000 7ff33a181b80 10 client.25729 fetch_fsmap learned FSMap version 34 2023-09-30T14:44:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.287+0000 7ff33a181b80 1 -- 172.21.15.32:0/3670509458 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5639a0dbfd80 con 0x5639a1046610 2023-09-30T14:44:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.287+0000 7ff306ffd700 1 -- 172.21.15.32:0/3670509458 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5639a1035090 con 0x5639a1046610 2023-09-30T14:44:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.287+0000 7ff306ffd700 1 -- 172.21.15.32:0/3670509458 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff310030070 con 0x5639a1046610 2023-09-30T14:44:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.287+0000 7ff33a181b80 10 client.25729 fetch_fsmap finished waiting for FSMap version 34 2023-09-30T14:44:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.287+0000 7ff33a181b80 10 client.25729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:44:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.287+0000 7ff33a181b80 20 client.25729 populate_metadata read hostname 'smithi032' 2023-09-30T14:44:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.287+0000 7ff33a181b80 1 --2- 172.21.15.32:0/3670509458 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5639a105bf60 0x5639a105e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.287+0000 7ff33a181b80 4 client.25729 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:44:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.287+0000 7ff33a181b80 1 -- 172.21.15.32:0/3670509458 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639a0f8ddb0 con 0x5639a105bf60 2023-09-30T14:44:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.287+0000 7ff326ffd700 1 --2- 172.21.15.32:0/3670509458 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5639a105bf60 0x5639a105e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.288+0000 7ff306ffd700 10 client.25729 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:44:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.288+0000 7ff326ffd700 1 --2- 172.21.15.32:0/3670509458 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5639a105bf60 0x5639a105e380 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8289c4dc3bc86c3 server_cookie=19dcafeee7d28444 in_seq=0 out_seq=0 2023-09-30T14:44:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.289+0000 7ff306ffd700 1 -- 172.21.15.32:0/3670509458 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5639a0f8ddb0 con 0x5639a105bf60 2023-09-30T14:44:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.289+0000 7ff306ffd700 10 client.25729 handle_command_reply: tid=0 2023-09-30T14:44:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.289+0000 7ff33a181b80 1 client.25729 shutdown 2023-09-30T14:44:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.289+0000 7ff33a181b80 1 -- 172.21.15.32:0/3670509458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1046610 msgr2=0x5639a1050060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.289+0000 7ff33a181b80 1 --2- 172.21.15.32:0/3670509458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1046610 0x5639a1050060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff310004580 tx=0x7ff3100047c0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.289+0000 7ff33a181b80 1 -- 172.21.15.32:0/3670509458 shutdown_connections 2023-09-30T14:44:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.289+0000 7ff33a181b80 1 --2- 172.21.15.32:0/3670509458 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5639a105bf60 0x5639a105e380 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.289+0000 7ff33a181b80 1 --2- 172.21.15.32:0/3670509458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a10575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.289+0000 7ff33a181b80 1 --2- 172.21.15.32:0/3670509458 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1046610 0x5639a1050060 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.289+0000 7ff33a181b80 1 --2- 172.21.15.32:0/3670509458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1047720 0x5639a10505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.289+0000 7ff306ffd700 0 client.25729 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:44:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.289+0000 7ff33a181b80 1 -- 172.21.15.32:0/3670509458 >> 172.21.15.32:0/3670509458 conn(0x5639a0fb6200 msgr2=0x5639a103c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.289+0000 7ff33a181b80 1 -- 172.21.15.32:0/3670509458 shutdown_connections 2023-09-30T14:44:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.289+0000 7ff33a181b80 1 -- 172.21.15.32:0/3670509458 wait complete. 2023-09-30T14:44:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.289+0000 7ff33a181b80 20 client.25729 trim_cache size 0 max 16384 2023-09-30T14:44:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.304+0000 7ff33a181b80 1 Processor -- start 2023-09-30T14:44:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.304+0000 7ff33a181b80 1 -- start start 2023-09-30T14:44:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.304+0000 7ff33a181b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1046610 0x5639a103e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.304+0000 7ff33a181b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a1047720 0x5639a1057570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.304+0000 7ff33a181b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a10480e0 0x5639a104af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.304+0000 7ff33a181b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a0e158c0 con 0x5639a1047720 2023-09-30T14:44:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.304+0000 7ff33a181b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a0f8dde0 con 0x5639a1046610 2023-09-30T14:44:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.304+0000 7ff33a181b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a104fbd0 con 0x5639a10480e0 2023-09-30T14:44:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.305+0000 7ff326ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a1047720 0x5639a1057570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.305+0000 7ff326ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a1047720 0x5639a1057570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37022/0 (socket says 172.21.15.32:37022) 2023-09-30T14:44:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.305+0000 7ff326ffd700 1 -- 172.21.15.32:0/784241308 learned_addr learned my addr 172.21.15.32:0/784241308 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.305+0000 7ff3277fe700 1 --2- 172.21.15.32:0/784241308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1046610 0x5639a103e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.305+0000 7ff327fff700 1 --2- 172.21.15.32:0/784241308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a10480e0 0x5639a104af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.305+0000 7ff326ffd700 1 -- 172.21.15.32:0/784241308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a10480e0 msgr2=0x5639a104af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.305+0000 7ff326ffd700 1 --2- 172.21.15.32:0/784241308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a10480e0 0x5639a104af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.305+0000 7ff326ffd700 1 -- 172.21.15.32:0/784241308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1046610 msgr2=0x5639a103e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.305+0000 7ff326ffd700 1 --2- 172.21.15.32:0/784241308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1046610 0x5639a103e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.305+0000 7ff326ffd700 1 -- 172.21.15.32:0/784241308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639a0da8c00 con 0x5639a1047720 2023-09-30T14:44:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.305+0000 7ff326ffd700 1 --2- 172.21.15.32:0/784241308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a1047720 0x5639a1057570 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff3180097c0 tx=0x7ff318009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c717e2fac9d9bb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.305+0000 7ff31e7fc700 1 -- 172.21.15.32:0/784241308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318017070 con 0x5639a1047720 2023-09-30T14:44:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.305+0000 7ff31e7fc700 1 -- 172.21.15.32:0/784241308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff318018040 con 0x5639a1047720 2023-09-30T14:44:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.305+0000 7ff31e7fc700 1 -- 172.21.15.32:0/784241308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31801d880 con 0x5639a1047720 2023-09-30T14:44:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.306+0000 7ff33a181b80 1 -- 172.21.15.32:0/784241308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a1047720 msgr2=0x5639a1057570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.306+0000 7ff33a181b80 1 --2- 172.21.15.32:0/784241308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a1047720 0x5639a1057570 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff3180097c0 tx=0x7ff318009260 comp rx=0 tx=0).stop 2023-09-30T14:44:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.306+0000 7ff33a181b80 1 -- 172.21.15.32:0/784241308 shutdown_connections 2023-09-30T14:44:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.306+0000 7ff33a181b80 1 --2- 172.21.15.32:0/784241308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a10480e0 0x5639a104af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.306+0000 7ff33a181b80 1 --2- 172.21.15.32:0/784241308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1046610 0x5639a103e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.306+0000 7ff33a181b80 1 --2- 172.21.15.32:0/784241308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a1047720 0x5639a1057570 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.306+0000 7ff33a181b80 1 -- 172.21.15.32:0/784241308 >> 172.21.15.32:0/784241308 conn(0x5639a0fb6200 msgr2=0x5639a103c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.306+0000 7ff33a181b80 1 -- 172.21.15.32:0/784241308 shutdown_connections 2023-09-30T14:44:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.306+0000 7ff33a181b80 1 -- 172.21.15.32:0/784241308 wait complete. 2023-09-30T14:44:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.306+0000 7ff33a181b80 1 -- start start 2023-09-30T14:44:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff33a181b80 1 Processor -- start 2023-09-30T14:44:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff33a181b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1046610 0x5639a1055340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff33a181b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1047720 0x5639a1055880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff33a181b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a1056890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff33a181b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a1056dd0 con 0x5639a10480e0 2023-09-30T14:44:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff33a181b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a1056f40 con 0x5639a1046610 2023-09-30T14:44:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff33a181b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a1057100 con 0x5639a1047720 2023-09-30T14:44:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff33a181b80 10 client.0 _ll_register_callbacks cb 0x7ff3100043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:44:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff327fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a1056890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff3277fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1046610 0x5639a1055340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff326ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1047720 0x5639a1055880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff327fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a1056890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37030/0 (socket says 172.21.15.32:37030) 2023-09-30T14:44:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff327fff700 1 -- 172.21.15.32:0/3599960761 learned_addr learned my addr 172.21.15.32:0/3599960761 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff327fff700 1 -- 172.21.15.32:0/3599960761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1047720 msgr2=0x5639a1055880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff327fff700 1 --2- 172.21.15.32:0/3599960761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1047720 0x5639a1055880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff327fff700 1 -- 172.21.15.32:0/3599960761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1046610 msgr2=0x5639a1055340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff327fff700 1 --2- 172.21.15.32:0/3599960761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1046610 0x5639a1055340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff327fff700 1 -- 172.21.15.32:0/3599960761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff318009b30 con 0x5639a10480e0 2023-09-30T14:44:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff327fff700 1 --2- 172.21.15.32:0/3599960761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a1056890 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff32009b1c0 tx=0x7ff3200977a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6119fae45d5251 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff306ffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:44:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.307+0000 7ff306ffd700 1 -- 172.21.15.32:0/3599960761 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32001ce80 con 0x5639a10480e0 2023-09-30T14:44:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.308+0000 7ff306ffd700 1 -- 172.21.15.32:0/3599960761 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff3200b3070 con 0x5639a10480e0 2023-09-30T14:44:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.308+0000 7ff33a181b80 1 -- 172.21.15.32:0/3599960761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff32009b3b0 con 0x5639a10480e0 2023-09-30T14:44:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.308+0000 7ff306ffd700 1 -- 172.21.15.32:0/3599960761 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff32009f500 con 0x5639a10480e0 2023-09-30T14:44:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.308+0000 7ff33a181b80 10 client.16059 fetch_fsmap learned FSMap version 34 2023-09-30T14:44:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.308+0000 7ff33a181b80 1 -- 172.21.15.32:0/3599960761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5639a0de61f0 con 0x5639a10480e0 2023-09-30T14:44:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.308+0000 7ff306ffd700 1 -- 172.21.15.32:0/3599960761 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff32009b3b0 con 0x5639a10480e0 2023-09-30T14:44:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.308+0000 7ff306ffd700 1 -- 172.21.15.32:0/3599960761 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 34) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ff32009a9a0 con 0x5639a10480e0 2023-09-30T14:44:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.308+0000 7ff33a181b80 10 client.16059 fetch_fsmap finished waiting for FSMap version 34 2023-09-30T14:44:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.308+0000 7ff33a181b80 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:44:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.308+0000 7ff33a181b80 20 client.16059 populate_metadata read hostname 'smithi032' 2023-09-30T14:44:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.308+0000 7ff33a181b80 1 --2- 172.21.15.32:0/3599960761 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5639a10b0900 0x5639a10b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.308+0000 7ff33a181b80 4 client.16059 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:44:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.308+0000 7ff33a181b80 1 -- 172.21.15.32:0/3599960761 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5639a0f8dad0 con 0x5639a10b0900 2023-09-30T14:44:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.308+0000 7ff3277fe700 1 --2- 172.21.15.32:0/3599960761 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5639a10b0900 0x5639a10b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.309+0000 7ff3277fe700 1 --2- 172.21.15.32:0/3599960761 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5639a10b0900 0x5639a10b2d20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=599bfae6b87c1e4a server_cookie=9a052c622b37a581 in_seq=0 out_seq=0 2023-09-30T14:44:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.309+0000 7ff306ffd700 10 client.16059 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:44:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.355+0000 7ff306ffd700 1 -- 172.21.15.32:0/3599960761 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5639a0f8dad0 con 0x5639a10b0900 2023-09-30T14:44:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.355+0000 7ff306ffd700 10 client.16059 handle_command_reply: tid=0 2023-09-30T14:44:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.355+0000 7ff33a181b80 1 client.16059 shutdown 2023-09-30T14:44:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.355+0000 7ff33a181b80 1 -- 172.21.15.32:0/3599960761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 msgr2=0x5639a1056890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.355+0000 7ff33a181b80 1 --2- 172.21.15.32:0/3599960761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a1056890 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff32009b1c0 tx=0x7ff3200977a0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.355+0000 7ff33a181b80 1 -- 172.21.15.32:0/3599960761 shutdown_connections 2023-09-30T14:44:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.355+0000 7ff33a181b80 1 --2- 172.21.15.32:0/3599960761 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5639a10b0900 0x5639a10b2d20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.355+0000 7ff33a181b80 1 --2- 172.21.15.32:0/3599960761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639a10480e0 0x5639a1056890 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.355+0000 7ff33a181b80 1 --2- 172.21.15.32:0/3599960761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639a1046610 0x5639a1055340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.355+0000 7ff33a181b80 1 --2- 172.21.15.32:0/3599960761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5639a1047720 0x5639a1055880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.355+0000 7ff306ffd700 0 client.16059 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:44:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.355+0000 7ff33a181b80 1 -- 172.21.15.32:0/3599960761 >> 172.21.15.32:0/3599960761 conn(0x5639a0fb6200 msgr2=0x5639a104af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.355+0000 7ff33a181b80 1 -- 172.21.15.32:0/3599960761 shutdown_connections 2023-09-30T14:44:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.355+0000 7ff33a181b80 1 -- 172.21.15.32:0/3599960761 wait complete. 2023-09-30T14:44:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.355+0000 7ff33a181b80 20 client.16059 trim_cache size 0 max 16384 2023-09-30T14:44:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:44:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:44:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:44:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:44:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.356+0000 7ff32df99700 1 -- 172.21.15.32:0/4207453923 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff3140a0fa0 msgr2=0x7ff3140a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.356+0000 7ff32df99700 1 --2- 172.21.15.32:0/4207453923 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff3140a0fa0 0x7ff3140a3460 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff328196ae0 tx=0x7ff310009260 comp rx=0 tx=0).stop 2023-09-30T14:44:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.356+0000 7ff32df99700 1 -- 172.21.15.32:0/4207453923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff328102970 msgr2=0x7ff328198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.356+0000 7ff32df99700 1 --2- 172.21.15.32:0/4207453923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff328102970 0x7ff328198070 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff32000b780 tx=0x7ff32000bd00 comp rx=0 tx=0).stop 2023-09-30T14:44:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.356+0000 7ff32df99700 1 -- 172.21.15.32:0/4207453923 shutdown_connections 2023-09-30T14:44:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.356+0000 7ff32df99700 1 --2- 172.21.15.32:0/4207453923 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff3140a0fa0 0x7ff3140a3460 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.356+0000 7ff32df99700 1 --2- 172.21.15.32:0/4207453923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff328102970 0x7ff328198070 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.356+0000 7ff32df99700 1 --2- 172.21.15.32:0/4207453923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff328100e60 0x7ff328196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.356+0000 7ff32df99700 1 --2- 172.21.15.32:0/4207453923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff328101fb0 0x7ff3281972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.356+0000 7ff32df99700 1 -- 172.21.15.32:0/4207453923 >> 172.21.15.32:0/4207453923 conn(0x7ff3280fc3d0 msgr2=0x7ff3280fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.356+0000 7ff32df99700 1 -- 172.21.15.32:0/4207453923 shutdown_connections 2023-09-30T14:44:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:40.356+0000 7ff32df99700 1 -- 172.21.15.32:0/4207453923 wait complete. 2023-09-30T14:44:40.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 649557 10.53 MB/sec execute 194 sec latency 34.381 ms 2023-09-30T14:44:41.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 651122 10.52 MB/sec execute 195 sec latency 16.035 ms 2023-09-30T14:44:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:41 smithi156 ceph-mon[103579]: pgmap v479: 97 pgs: 97 active+clean; 219 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2023-09-30T14:44:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:41 smithi148 ceph-mon[102355]: pgmap v479: 97 pgs: 97 active+clean; 219 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2023-09-30T14:44:41.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:41 smithi032 ceph-mon[98592]: pgmap v479: 97 pgs: 97 active+clean; 219 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2023-09-30T14:44:42.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 653682 10.55 MB/sec execute 196 sec latency 10.353 ms 2023-09-30T14:44:43.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 655531 10.52 MB/sec execute 197 sec latency 15.823 ms 2023-09-30T14:44:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:43 smithi156 ceph-mon[103579]: pgmap v480: 97 pgs: 97 active+clean; 220 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 2.9 MiB/s wr, 275 op/s 2023-09-30T14:44:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:43 smithi148 ceph-mon[102355]: pgmap v480: 97 pgs: 97 active+clean; 220 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 2.9 MiB/s wr, 275 op/s 2023-09-30T14:44:43.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:43 smithi032 ceph-mon[98592]: pgmap v480: 97 pgs: 97 active+clean; 220 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 2.9 MiB/s wr, 275 op/s 2023-09-30T14:44:44.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 657357 10.54 MB/sec execute 198 sec latency 34.075 ms 2023-09-30T14:44:45.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 659188 10.51 MB/sec execute 199 sec latency 43.380 ms 2023-09-30T14:44:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:45 smithi156 ceph-mon[103579]: pgmap v481: 97 pgs: 97 active+clean; 222 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 3.0 MiB/s wr, 308 op/s 2023-09-30T14:44:45.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:45 smithi148 ceph-mon[102355]: pgmap v481: 97 pgs: 97 active+clean; 222 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 3.0 MiB/s wr, 308 op/s 2023-09-30T14:44:45.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:45 smithi032 ceph-mon[98592]: pgmap v481: 97 pgs: 97 active+clean; 222 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 3.0 MiB/s wr, 308 op/s 2023-09-30T14:44:46.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 660946 10.53 MB/sec execute 200 sec latency 14.931 ms 2023-09-30T14:44:47.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 663127 10.52 MB/sec execute 201 sec latency 40.861 ms 2023-09-30T14:44:47.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:47 smithi156 ceph-mon[103579]: pgmap v482: 97 pgs: 97 active+clean; 222 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 2.9 MiB/s wr, 271 op/s 2023-09-30T14:44:47.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:47 smithi148 ceph-mon[102355]: pgmap v482: 97 pgs: 97 active+clean; 222 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 2.9 MiB/s wr, 271 op/s 2023-09-30T14:44:47.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:47 smithi032 ceph-mon[98592]: pgmap v482: 97 pgs: 97 active+clean; 222 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 2.9 MiB/s wr, 271 op/s 2023-09-30T14:44:48.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 664796 10.52 MB/sec execute 202 sec latency 14.837 ms 2023-09-30T14:44:49.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 667053 10.53 MB/sec execute 203 sec latency 30.356 ms 2023-09-30T14:44:49.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:49 smithi156 ceph-mon[103579]: pgmap v483: 97 pgs: 97 active+clean; 225 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 3.1 MiB/s wr, 311 op/s 2023-09-30T14:44:49.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:49 smithi148 ceph-mon[102355]: pgmap v483: 97 pgs: 97 active+clean; 225 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 3.1 MiB/s wr, 311 op/s 2023-09-30T14:44:49.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:49 smithi032 ceph-mon[98592]: pgmap v483: 97 pgs: 97 active+clean; 225 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 3.1 MiB/s wr, 311 op/s 2023-09-30T14:44:50.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 668601 10.51 MB/sec execute 204 sec latency 28.838 ms 2023-09-30T14:44:51.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 671272 10.55 MB/sec execute 205 sec latency 18.530 ms 2023-09-30T14:44:51.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:51 smithi156 ceph-mon[103579]: pgmap v484: 97 pgs: 97 active+clean; 225 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 2.9 MiB/s wr, 275 op/s 2023-09-30T14:44:51.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:51 smithi148 ceph-mon[102355]: pgmap v484: 97 pgs: 97 active+clean; 225 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 2.9 MiB/s wr, 275 op/s 2023-09-30T14:44:51.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:51 smithi032 ceph-mon[98592]: pgmap v484: 97 pgs: 97 active+clean; 225 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 2.9 MiB/s wr, 275 op/s 2023-09-30T14:44:52.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 672832 10.53 MB/sec execute 206 sec latency 14.487 ms 2023-09-30T14:44:52.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:52 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:44:52.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:52 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:44:52.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:52 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:44:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.476+0000 7f947d9eb700 1 -- 172.21.15.32:0/2421356376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9478101ff0 msgr2=0x7f9478102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.476+0000 7f947d9eb700 1 --2- 172.21.15.32:0/2421356376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9478101ff0 0x7f9478102470 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f946c009990 tx=0x7f946c009ca0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.476+0000 7f947d9eb700 1 -- 172.21.15.32:0/2421356376 shutdown_connections 2023-09-30T14:44:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.476+0000 7f947d9eb700 1 --2- 172.21.15.32:0/2421356376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94781029b0 0x7f94781092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.476+0000 7f947d9eb700 1 --2- 172.21.15.32:0/2421356376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9478100ea0 0x7f94781012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.476+0000 7f947d9eb700 1 --2- 172.21.15.32:0/2421356376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9478101ff0 0x7f9478102470 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.476+0000 7f947d9eb700 1 -- 172.21.15.32:0/2421356376 >> 172.21.15.32:0/2421356376 conn(0x7f94780fc3f0 msgr2=0x7f94780fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.476+0000 7f947d9eb700 1 -- 172.21.15.32:0/2421356376 shutdown_connections 2023-09-30T14:44:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.476+0000 7f947d9eb700 1 -- 172.21.15.32:0/2421356376 wait complete. 2023-09-30T14:44:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.477+0000 7f947d9eb700 1 Processor -- start 2023-09-30T14:44:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.477+0000 7f947d9eb700 1 -- start start 2023-09-30T14:44:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.477+0000 7f947d9eb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9478100ea0 0x7f9478073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.477+0000 7f947d9eb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9478101ff0 0x7f9478073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.477+0000 7f947d9eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94781029b0 0x7f9478073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.477+0000 7f947d9eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781a0b80 con 0x7f94781029b0 2023-09-30T14:44:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.477+0000 7f947d9eb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781a0cc0 con 0x7f9478101ff0 2023-09-30T14:44:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.477+0000 7f947d9eb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781a0e00 con 0x7f9478100ea0 2023-09-30T14:44:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.477+0000 7f94777fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94781029b0 0x7f9478073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.477+0000 7f9476ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9478100ea0 0x7f9478073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.477+0000 7f9476ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9478100ea0 0x7f9478073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51422/0 (socket says 172.21.15.32:51422) 2023-09-30T14:44:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.477+0000 7f94777fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94781029b0 0x7f9478073c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37044/0 (socket says 172.21.15.32:37044) 2023-09-30T14:44:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.477+0000 7f9476ffd700 1 -- 172.21.15.32:0/2762276262 learned_addr learned my addr 172.21.15.32:0/2762276262 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.477+0000 7f94767fc700 1 --2- 172.21.15.32:0/2762276262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9478101ff0 0x7f9478073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.478+0000 7f94777fe700 1 -- 172.21.15.32:0/2762276262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9478100ea0 msgr2=0x7f9478073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.478+0000 7f94777fe700 1 --2- 172.21.15.32:0/2762276262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9478100ea0 0x7f9478073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.478+0000 7f94777fe700 1 -- 172.21.15.32:0/2762276262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9478101ff0 msgr2=0x7f9478073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.478+0000 7f94777fe700 1 --2- 172.21.15.32:0/2762276262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9478101ff0 0x7f9478073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.478+0000 7f94777fe700 1 -- 172.21.15.32:0/2762276262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c0095f0 con 0x7f94781029b0 2023-09-30T14:44:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.478+0000 7f9476ffd700 1 --2- 172.21.15.32:0/2762276262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9478100ea0 0x7f9478073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:44:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.478+0000 7f94777fe700 1 --2- 172.21.15.32:0/2762276262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94781029b0 0x7f9478073c90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f947000efc0 tx=0x7f947000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.478+0000 7f947c9e9700 1 -- 172.21.15.32:0/2762276262 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470018070 con 0x7f94781029b0 2023-09-30T14:44:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.478+0000 7f947c9e9700 1 -- 172.21.15.32:0/2762276262 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f947000f040 con 0x7f94781029b0 2023-09-30T14:44:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.478+0000 7f947d9eb700 1 -- 172.21.15.32:0/2762276262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94781a1030 con 0x7f94781029b0 2023-09-30T14:44:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.478+0000 7f947c9e9700 1 -- 172.21.15.32:0/2762276262 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947000cd40 con 0x7f94781029b0 2023-09-30T14:44:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.478+0000 7f947d9eb700 1 -- 172.21.15.32:0/2762276262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94781a1510 con 0x7f94781029b0 2023-09-30T14:44:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.480+0000 7f947c9e9700 1 -- 172.21.15.32:0/2762276262 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f9470003a00 con 0x7f94781029b0 2023-09-30T14:44:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.481+0000 7f947c9e9700 1 --2- 172.21.15.32:0/2762276262 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f94600a1020 0x7f94600a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.481+0000 7f947c9e9700 1 -- 172.21.15.32:0/2762276262 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f94700cc290 con 0x7f94781029b0 2023-09-30T14:44:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.481+0000 7f9476ffd700 1 --2- 172.21.15.32:0/2762276262 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f94600a1020 0x7f94600a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.481+0000 7f9476ffd700 1 --2- 172.21.15.32:0/2762276262 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f94600a1020 0x7f94600a34e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9478074b20 tx=0x7f9468009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.482+0000 7f9489bd3b80 1 Processor -- start 2023-09-30T14:44:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.482+0000 7f9489bd3b80 1 -- start start 2023-09-30T14:44:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.482+0000 7f9489bd3b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976a200 0x56392976a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.482+0000 7f9489bd3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976b310 0x56392976b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.482+0000 7f9489bd3b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976bcd0 0x563929772560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.482+0000 7f9489bd3b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639296d2400 con 0x56392976bcd0 2023-09-30T14:44:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.482+0000 7f9489bd3b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639296c3bc0 con 0x56392976a200 2023-09-30T14:44:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.482+0000 7f9489bd3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639296c14e0 con 0x56392976b310 2023-09-30T14:44:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.482+0000 7f94777fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976bcd0 0x563929772560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.482+0000 7f9476ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976a200 0x56392976a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.482+0000 7f94767fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976b310 0x56392976b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.482+0000 7f94767fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976b310 0x56392976b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51432/0 (socket says 172.21.15.32:51432) 2023-09-30T14:44:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.482+0000 7f94767fc700 1 -- 172.21.15.32:0/2922231143 learned_addr learned my addr 172.21.15.32:0/2922231143 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.482+0000 7f9476ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976a200 0x56392976a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48524/0 (socket says 172.21.15.32:48524) 2023-09-30T14:44:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.483+0000 7f94767fc700 1 -- 172.21.15.32:0/2922231143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976a200 msgr2=0x56392976a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.483+0000 7f94767fc700 1 --2- 172.21.15.32:0/2922231143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976a200 0x56392976a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.483+0000 7f94767fc700 1 -- 172.21.15.32:0/2922231143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976bcd0 msgr2=0x563929772560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.483+0000 7f94767fc700 1 --2- 172.21.15.32:0/2922231143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976bcd0 0x563929772560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.483+0000 7f94767fc700 1 -- 172.21.15.32:0/2922231143 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56392971f870 con 0x56392976b310 2023-09-30T14:44:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.483+0000 7f94767fc700 1 --2- 172.21.15.32:0/2922231143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976b310 0x56392976b790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f946c009990 tx=0x7f946c005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccdcfff8cee1cb9b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.483+0000 7f945dffb700 1 -- 172.21.15.32:0/2922231143 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c01d050 con 0x56392976b310 2023-09-30T14:44:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.483+0000 7f945dffb700 1 -- 172.21.15.32:0/2922231143 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f946c01e040 con 0x56392976b310 2023-09-30T14:44:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.483+0000 7f945dffb700 1 -- 172.21.15.32:0/2922231143 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c023da0 con 0x56392976b310 2023-09-30T14:44:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.484+0000 7f9489bd3b80 1 -- 172.21.15.32:0/2922231143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976b310 msgr2=0x56392976b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.484+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/2922231143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976b310 0x56392976b790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f946c009990 tx=0x7f946c005f00 comp rx=0 tx=0).stop 2023-09-30T14:44:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.484+0000 7f9489bd3b80 1 -- 172.21.15.32:0/2922231143 shutdown_connections 2023-09-30T14:44:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.484+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/2922231143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976bcd0 0x563929772560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.484+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/2922231143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976a200 0x56392976a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.484+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/2922231143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976b310 0x56392976b790 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.484+0000 7f9489bd3b80 1 -- 172.21.15.32:0/2922231143 >> 172.21.15.32:0/2922231143 conn(0x5639296de200 msgr2=0x563929764ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.484+0000 7f9489bd3b80 1 -- 172.21.15.32:0/2922231143 shutdown_connections 2023-09-30T14:44:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.484+0000 7f9489bd3b80 1 -- 172.21.15.32:0/2922231143 wait complete. 2023-09-30T14:44:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.484+0000 7f9489bd3b80 1 -- start start 2023-09-30T14:44:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.484+0000 7f9489bd3b80 1 Processor -- start 2023-09-30T14:44:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.484+0000 7f9489bd3b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976a200 0x563929773e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f9489bd3b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976b310 0x5639297743b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f9489bd3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976bcd0 0x56392977b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f9489bd3b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639296bcdd0 con 0x56392976a200 2023-09-30T14:44:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f9489bd3b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639296596b0 con 0x56392976b310 2023-09-30T14:44:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f9489bd3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563929659350 con 0x56392976bcd0 2023-09-30T14:44:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f9489bd3b80 10 client.0 _ll_register_callbacks cb 0x5639294b67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:44:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f9476ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976a200 0x563929773e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f94767fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976b310 0x5639297743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f94767fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976b310 0x5639297743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48532/0 (socket says 172.21.15.32:48532) 2023-09-30T14:44:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f9476ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976a200 0x563929773e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37054/0 (socket says 172.21.15.32:37054) 2023-09-30T14:44:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f94767fc700 1 -- 172.21.15.32:0/507014876 learned_addr learned my addr 172.21.15.32:0/507014876 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f94777fe700 1 --2- 172.21.15.32:0/507014876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976bcd0 0x56392977b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f94767fc700 1 -- 172.21.15.32:0/507014876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976bcd0 msgr2=0x56392977b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f94767fc700 1 --2- 172.21.15.32:0/507014876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976bcd0 0x56392977b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f94767fc700 1 -- 172.21.15.32:0/507014876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976a200 msgr2=0x563929773e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f94767fc700 1 --2- 172.21.15.32:0/507014876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976a200 0x563929773e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f94767fc700 1 -- 172.21.15.32:0/507014876 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9478109c00 con 0x56392976b310 2023-09-30T14:44:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f9476ffd700 1 --2- 172.21.15.32:0/507014876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976a200 0x563929773e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:44:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f94767fc700 1 --2- 172.21.15.32:0/507014876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976b310 0x5639297743b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f946c00bb60 tx=0x7f946c00bb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3759aed0e49db5f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f9456ffd700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:44:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f9456ffd700 1 -- 172.21.15.32:0/507014876 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c005230 con 0x56392976b310 2023-09-30T14:44:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f9489bd3b80 1 -- 172.21.15.32:0/507014876 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56392975d090 con 0x56392976b310 2023-09-30T14:44:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.485+0000 7f9456ffd700 1 -- 172.21.15.32:0/507014876 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f946c023460 con 0x56392976b310 2023-09-30T14:44:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.486+0000 7f9456ffd700 1 -- 172.21.15.32:0/507014876 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f946c005390 con 0x56392976b310 2023-09-30T14:44:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.486+0000 7f9489bd3b80 10 client.25757 fetch_fsmap learned FSMap version 35 2023-09-30T14:44:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.486+0000 7f9489bd3b80 1 -- 172.21.15.32:0/507014876 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5639294e7d80 con 0x56392976b310 2023-09-30T14:44:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.486+0000 7f9456ffd700 1 -- 172.21.15.32:0/507014876 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56392975d090 con 0x56392976b310 2023-09-30T14:44:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.486+0000 7f9456ffd700 1 -- 172.21.15.32:0/507014876 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 35) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f946c00f460 con 0x56392976b310 2023-09-30T14:44:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.486+0000 7f9489bd3b80 10 client.25757 fetch_fsmap finished waiting for FSMap version 35 2023-09-30T14:44:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.486+0000 7f9489bd3b80 10 client.25757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:44:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.486+0000 7f9489bd3b80 20 client.25757 populate_metadata read hostname 'smithi032' 2023-09-30T14:44:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.486+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/507014876 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56392977f840 0x563929781c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.486+0000 7f9489bd3b80 4 client.25757 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:44:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.486+0000 7f9489bd3b80 1 -- 172.21.15.32:0/507014876 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639296b5db0 con 0x56392977f840 2023-09-30T14:44:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.486+0000 7f9476ffd700 1 --2- 172.21.15.32:0/507014876 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56392977f840 0x563929781c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.487+0000 7f9456ffd700 10 client.25757 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:44:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.487+0000 7f9476ffd700 1 --2- 172.21.15.32:0/507014876 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56392977f840 0x563929781c60 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da33cf119619f8d server_cookie=12f2082cd425b918 in_seq=0 out_seq=0 2023-09-30T14:44:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.487+0000 7f9456ffd700 1 -- 172.21.15.32:0/507014876 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5639296b5db0 con 0x56392977f840 2023-09-30T14:44:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.487+0000 7f9456ffd700 10 client.25757 handle_command_reply: tid=0 2023-09-30T14:44:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.487+0000 7f9489bd3b80 1 client.25757 shutdown 2023-09-30T14:44:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.487+0000 7f9489bd3b80 1 -- 172.21.15.32:0/507014876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976b310 msgr2=0x5639297743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.487+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/507014876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976b310 0x5639297743b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f946c00bb60 tx=0x7f946c00bb90 comp rx=0 tx=0).stop 2023-09-30T14:44:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.487+0000 7f9489bd3b80 1 -- 172.21.15.32:0/507014876 shutdown_connections 2023-09-30T14:44:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.487+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/507014876 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56392977f840 0x563929781c60 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.487+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/507014876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976bcd0 0x56392977b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.487+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/507014876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976a200 0x563929773e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.487+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/507014876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976b310 0x5639297743b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.487+0000 7f9456ffd700 0 client.25757 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:44:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.487+0000 7f9489bd3b80 1 -- 172.21.15.32:0/507014876 >> 172.21.15.32:0/507014876 conn(0x5639296de200 msgr2=0x563929764590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.487+0000 7f9489bd3b80 1 -- 172.21.15.32:0/507014876 shutdown_connections 2023-09-30T14:44:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.487+0000 7f9489bd3b80 1 -- 172.21.15.32:0/507014876 wait complete. 2023-09-30T14:44:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.488+0000 7f9489bd3b80 20 client.25757 trim_cache size 0 max 16384 2023-09-30T14:44:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f9489bd3b80 1 Processor -- start 2023-09-30T14:44:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f9489bd3b80 1 -- start start 2023-09-30T14:44:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f9489bd3b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976a200 0x56392976eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f9489bd3b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976b310 0x56392977b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f9489bd3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976bcd0 0x56392976f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f9489bd3b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392953d8c0 con 0x56392976a200 2023-09-30T14:44:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f9489bd3b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563929765c90 con 0x56392976b310 2023-09-30T14:44:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f9489bd3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392976b020 con 0x56392976bcd0 2023-09-30T14:44:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f9476ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976a200 0x56392976eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f9476ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976a200 0x56392976eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37062/0 (socket says 172.21.15.32:37062) 2023-09-30T14:44:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f9476ffd700 1 -- 172.21.15.32:0/389986467 learned_addr learned my addr 172.21.15.32:0/389986467 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f94777fe700 1 --2- 172.21.15.32:0/389986467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976bcd0 0x56392976f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f94767fc700 1 --2- 172.21.15.32:0/389986467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976b310 0x56392977b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f94767fc700 1 -- 172.21.15.32:0/389986467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976bcd0 msgr2=0x56392976f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f94767fc700 1 --2- 172.21.15.32:0/389986467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976bcd0 0x56392976f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f94767fc700 1 -- 172.21.15.32:0/389986467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976a200 msgr2=0x56392976eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f94767fc700 1 --2- 172.21.15.32:0/389986467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976a200 0x56392976eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f94767fc700 1 -- 172.21.15.32:0/389986467 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639294d0c00 con 0x56392976b310 2023-09-30T14:44:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f9476ffd700 1 --2- 172.21.15.32:0/389986467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976a200 0x56392976eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:44:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.503+0000 7f94777fe700 1 --2- 172.21.15.32:0/389986467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976bcd0 0x56392976f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:44:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.504+0000 7f94767fc700 1 --2- 172.21.15.32:0/389986467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976b310 0x56392977b3b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f946c00f840 tx=0x7f946c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36d151fbf576625f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.504+0000 7f945e7fc700 1 -- 172.21.15.32:0/389986467 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c01d040 con 0x56392976b310 2023-09-30T14:44:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.504+0000 7f945e7fc700 1 -- 172.21.15.32:0/389986467 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f946c00bc00 con 0x56392976b310 2023-09-30T14:44:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.504+0000 7f945e7fc700 1 -- 172.21.15.32:0/389986467 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c023970 con 0x56392976b310 2023-09-30T14:44:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.504+0000 7f9489bd3b80 1 -- 172.21.15.32:0/389986467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976b310 msgr2=0x56392977b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.504+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/389986467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976b310 0x56392977b3b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f946c00f840 tx=0x7f946c009310 comp rx=0 tx=0).stop 2023-09-30T14:44:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.505+0000 7f9489bd3b80 1 -- 172.21.15.32:0/389986467 shutdown_connections 2023-09-30T14:44:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.505+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/389986467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976bcd0 0x56392976f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.505+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/389986467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976a200 0x56392976eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.505+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/389986467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976b310 0x56392977b3b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.505+0000 7f9489bd3b80 1 -- 172.21.15.32:0/389986467 >> 172.21.15.32:0/389986467 conn(0x5639296de200 msgr2=0x563929786580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.505+0000 7f9489bd3b80 1 -- 172.21.15.32:0/389986467 shutdown_connections 2023-09-30T14:44:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.505+0000 7f9489bd3b80 1 -- 172.21.15.32:0/389986467 wait complete. 2023-09-30T14:44:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.505+0000 7f9489bd3b80 1 -- start start 2023-09-30T14:44:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.505+0000 7f9489bd3b80 1 Processor -- start 2023-09-30T14:44:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9489bd3b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976a200 0x563929778a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9489bd3b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976b310 0x563929778fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9489bd3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976bcd0 0x563929779fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9489bd3b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392977a500 con 0x56392976b310 2023-09-30T14:44:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9489bd3b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392977a670 con 0x56392976a200 2023-09-30T14:44:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9489bd3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392977a830 con 0x56392976bcd0 2023-09-30T14:44:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9489bd3b80 10 client.0 _ll_register_callbacks cb 0x7f947000cec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:44:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9476ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976a200 0x563929778a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9476ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976a200 0x563929778a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48544/0 (socket says 172.21.15.32:48544) 2023-09-30T14:44:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9476ffd700 1 -- 172.21.15.32:0/3254738588 learned_addr learned my addr 172.21.15.32:0/3254738588 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:44:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f94777fe700 1 --2- 172.21.15.32:0/3254738588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976bcd0 0x563929779fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f94767fc700 1 --2- 172.21.15.32:0/3254738588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976b310 0x563929778fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9476ffd700 1 -- 172.21.15.32:0/3254738588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976bcd0 msgr2=0x563929779fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9476ffd700 1 --2- 172.21.15.32:0/3254738588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976bcd0 0x563929779fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9476ffd700 1 -- 172.21.15.32:0/3254738588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976b310 msgr2=0x563929778fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:44:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9476ffd700 1 --2- 172.21.15.32:0/3254738588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976b310 0x563929778fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9476ffd700 1 -- 172.21.15.32:0/3254738588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9468017070 con 0x56392976a200 2023-09-30T14:44:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9476ffd700 1 --2- 172.21.15.32:0/3254738588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976a200 0x563929778a70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9468011c70 tx=0x7f94680047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=898d5de519ebabd5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:44:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9456ffd700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:44:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9456ffd700 1 -- 172.21.15.32:0/3254738588 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468024040 con 0x56392976a200 2023-09-30T14:44:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9456ffd700 1 -- 172.21.15.32:0/3254738588 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9468029560 con 0x56392976a200 2023-09-30T14:44:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.506+0000 7f9489bd3b80 1 -- 172.21.15.32:0/3254738588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f946c01e040 con 0x56392976a200 2023-09-30T14:44:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.507+0000 7f9456ffd700 1 -- 172.21.15.32:0/3254738588 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9468029b00 con 0x56392976a200 2023-09-30T14:44:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.507+0000 7f9489bd3b80 10 client.25763 fetch_fsmap learned FSMap version 35 2023-09-30T14:44:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.507+0000 7f9489bd3b80 1 -- 172.21.15.32:0/3254738588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f94781a1510 con 0x56392976a200 2023-09-30T14:44:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.507+0000 7f9456ffd700 1 -- 172.21.15.32:0/3254738588 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f946c01e040 con 0x56392976a200 2023-09-30T14:44:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.507+0000 7f9456ffd700 1 -- 172.21.15.32:0/3254738588 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 35) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f946801e420 con 0x56392976a200 2023-09-30T14:44:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.507+0000 7f9489bd3b80 10 client.25763 fetch_fsmap finished waiting for FSMap version 35 2023-09-30T14:44:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.507+0000 7f9489bd3b80 10 client.25763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:44:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.507+0000 7f9489bd3b80 20 client.25763 populate_metadata read hostname 'smithi032' 2023-09-30T14:44:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.507+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/3254738588 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5639297d3ff0 0x5639297d63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:44:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.507+0000 7f9489bd3b80 4 client.25763 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:44:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.507+0000 7f9489bd3b80 1 -- 172.21.15.32:0/3254738588 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5639296b5ad0 con 0x5639297d3ff0 2023-09-30T14:44:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.507+0000 7f94767fc700 1 --2- 172.21.15.32:0/3254738588 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5639297d3ff0 0x5639297d63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:44:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.508+0000 7f94767fc700 1 --2- 172.21.15.32:0/3254738588 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5639297d3ff0 0x5639297d63e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa18f7d9120f8266 server_cookie=a37eb64b99ec0938 in_seq=0 out_seq=0 2023-09-30T14:44:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.508+0000 7f9456ffd700 10 client.25763 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:44:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.549+0000 7f9456ffd700 1 -- 172.21.15.32:0/3254738588 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5639296b5ad0 con 0x5639297d3ff0 2023-09-30T14:44:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.549+0000 7f9456ffd700 10 client.25763 handle_command_reply: tid=0 2023-09-30T14:44:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.549+0000 7f9489bd3b80 1 client.25763 shutdown 2023-09-30T14:44:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f9489bd3b80 1 -- 172.21.15.32:0/3254738588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976a200 msgr2=0x563929778a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/3254738588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976a200 0x563929778a70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9468011c70 tx=0x7f94680047c0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f9489bd3b80 1 -- 172.21.15.32:0/3254738588 shutdown_connections 2023-09-30T14:44:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/3254738588 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5639297d3ff0 0x5639297d63e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/3254738588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56392976bcd0 0x563929779fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/3254738588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56392976a200 0x563929778a70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f9489bd3b80 1 --2- 172.21.15.32:0/3254738588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56392976b310 0x563929778fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f9456ffd700 0 client.25763 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:44:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f9489bd3b80 1 -- 172.21.15.32:0/3254738588 >> 172.21.15.32:0/3254738588 conn(0x5639296de200 msgr2=0x56392976eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f9489bd3b80 1 -- 172.21.15.32:0/3254738588 shutdown_connections 2023-09-30T14:44:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f9489bd3b80 1 -- 172.21.15.32:0/3254738588 wait complete. 2023-09-30T14:44:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f9489bd3b80 20 client.25763 trim_cache size 0 max 16384 2023-09-30T14:44:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:44:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:44:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:44:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:44:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f947d9eb700 1 -- 172.21.15.32:0/2762276262 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f94600a1020 msgr2=0x7f94600a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f947d9eb700 1 --2- 172.21.15.32:0/2762276262 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f94600a1020 0x7f94600a34e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9478074b20 tx=0x7f9468009260 comp rx=0 tx=0).stop 2023-09-30T14:44:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f947d9eb700 1 -- 172.21.15.32:0/2762276262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94781029b0 msgr2=0x7f9478073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:44:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f947d9eb700 1 --2- 172.21.15.32:0/2762276262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94781029b0 0x7f9478073c90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f947000efc0 tx=0x7f947000c3a0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f947d9eb700 1 -- 172.21.15.32:0/2762276262 shutdown_connections 2023-09-30T14:44:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f947d9eb700 1 --2- 172.21.15.32:0/2762276262 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f94600a1020 0x7f94600a34e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f947d9eb700 1 --2- 172.21.15.32:0/2762276262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94781029b0 0x7f9478073c90 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f947d9eb700 1 --2- 172.21.15.32:0/2762276262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9478100ea0 0x7f9478073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f947d9eb700 1 --2- 172.21.15.32:0/2762276262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9478101ff0 0x7f9478073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:44:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.550+0000 7f947d9eb700 1 -- 172.21.15.32:0/2762276262 >> 172.21.15.32:0/2762276262 conn(0x7f94780fc3f0 msgr2=0x7f9478105580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:44:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.551+0000 7f947d9eb700 1 -- 172.21.15.32:0/2762276262 shutdown_connections 2023-09-30T14:44:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:44:53.551+0000 7f947d9eb700 1 -- 172.21.15.32:0/2762276262 wait complete. 2023-09-30T14:44:53.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 675135 10.54 MB/sec execute 207 sec latency 19.029 ms 2023-09-30T14:44:53.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:53 smithi156 ceph-mon[103579]: pgmap v485: 97 pgs: 97 active+clean; 225 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 2.8 MiB/s wr, 259 op/s 2023-09-30T14:44:53.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:53 smithi148 ceph-mon[102355]: pgmap v485: 97 pgs: 97 active+clean; 225 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 2.8 MiB/s wr, 259 op/s 2023-09-30T14:44:53.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:53 smithi032 ceph-mon[98592]: pgmap v485: 97 pgs: 97 active+clean; 225 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 2.8 MiB/s wr, 259 op/s 2023-09-30T14:44:54.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 677162 10.53 MB/sec execute 208 sec latency 62.338 ms 2023-09-30T14:44:55.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 678766 10.53 MB/sec execute 209 sec latency 60.388 ms 2023-09-30T14:44:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:55 smithi156 ceph-mon[103579]: pgmap v486: 97 pgs: 97 active+clean; 226 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 2.6 MiB/s wr, 260 op/s 2023-09-30T14:44:55.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:55 smithi148 ceph-mon[102355]: pgmap v486: 97 pgs: 97 active+clean; 226 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 2.6 MiB/s wr, 260 op/s 2023-09-30T14:44:55.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:55 smithi032 ceph-mon[98592]: pgmap v486: 97 pgs: 97 active+clean; 226 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 2.6 MiB/s wr, 260 op/s 2023-09-30T14:44:56.705 INFO:tasks.workunit.client.0.smithi032.stdout: 1 680918 10.52 MB/sec execute 210 sec latency 30.140 ms 2023-09-30T14:44:57.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 682289 10.52 MB/sec execute 211 sec latency 156.196 ms 2023-09-30T14:44:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:57 smithi156 ceph-mon[103579]: pgmap v487: 97 pgs: 97 active+clean; 231 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 2.7 MiB/s wr, 281 op/s 2023-09-30T14:44:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:57 smithi148 ceph-mon[102355]: pgmap v487: 97 pgs: 97 active+clean; 231 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 2.7 MiB/s wr, 281 op/s 2023-09-30T14:44:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:57 smithi032 ceph-mon[98592]: pgmap v487: 97 pgs: 97 active+clean; 231 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 2.7 MiB/s wr, 281 op/s 2023-09-30T14:44:58.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 684632 10.53 MB/sec execute 212 sec latency 15.362 ms 2023-09-30T14:44:59.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 686458 10.52 MB/sec execute 213 sec latency 17.571 ms 2023-09-30T14:44:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:44:59 smithi156 ceph-mon[103579]: pgmap v488: 97 pgs: 97 active+clean; 227 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 2.7 MiB/s wr, 268 op/s 2023-09-30T14:44:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:44:59 smithi148 ceph-mon[102355]: pgmap v488: 97 pgs: 97 active+clean; 227 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 2.7 MiB/s wr, 268 op/s 2023-09-30T14:44:59.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:44:59 smithi032 ceph-mon[98592]: pgmap v488: 97 pgs: 97 active+clean; 227 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 2.7 MiB/s wr, 268 op/s 2023-09-30T14:45:00.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 689062 10.55 MB/sec execute 214 sec latency 30.746 ms 2023-09-30T14:45:01.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 690712 10.53 MB/sec execute 215 sec latency 24.037 ms 2023-09-30T14:45:01.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:01 smithi148 ceph-mon[102355]: pgmap v489: 97 pgs: 97 active+clean; 230 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 2.7 MiB/s wr, 262 op/s 2023-09-30T14:45:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:01 smithi032 ceph-mon[98592]: pgmap v489: 97 pgs: 97 active+clean; 230 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 2.7 MiB/s wr, 262 op/s 2023-09-30T14:45:02.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:01 smithi156 ceph-mon[103579]: pgmap v489: 97 pgs: 97 active+clean; 230 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 2.7 MiB/s wr, 262 op/s 2023-09-30T14:45:02.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 692937 10.55 MB/sec execute 216 sec latency 24.968 ms 2023-09-30T14:45:03.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 695464 10.56 MB/sec execute 217 sec latency 17.698 ms 2023-09-30T14:45:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:03 smithi148 ceph-mon[102355]: pgmap v490: 97 pgs: 97 active+clean; 231 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.0 MiB/s wr, 283 op/s 2023-09-30T14:45:03.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:03 smithi032 ceph-mon[98592]: pgmap v490: 97 pgs: 97 active+clean; 231 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.0 MiB/s wr, 283 op/s 2023-09-30T14:45:04.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:03 smithi156 ceph-mon[103579]: pgmap v490: 97 pgs: 97 active+clean; 231 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.0 MiB/s wr, 283 op/s 2023-09-30T14:45:04.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 697195 10.55 MB/sec execute 218 sec latency 14.575 ms 2023-09-30T14:45:05.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 699741 10.57 MB/sec execute 219 sec latency 69.592 ms 2023-09-30T14:45:05.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:05 smithi148 ceph-mon[102355]: pgmap v491: 97 pgs: 97 active+clean; 231 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 2.8 MiB/s wr, 281 op/s 2023-09-30T14:45:05.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:05 smithi032 ceph-mon[98592]: pgmap v491: 97 pgs: 97 active+clean; 231 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 2.8 MiB/s wr, 281 op/s 2023-09-30T14:45:06.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:05 smithi156 ceph-mon[103579]: pgmap v491: 97 pgs: 97 active+clean; 231 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 2.8 MiB/s wr, 281 op/s 2023-09-30T14:45:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.667+0000 7f1d0753e700 1 -- 172.21.15.32:0/3048629693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d00101ff0 msgr2=0x7f1d00102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.667+0000 7f1d0753e700 1 --2- 172.21.15.32:0/3048629693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d00101ff0 0x7f1d00102470 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009990 tx=0x7f1cf0009ca0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.667+0000 7f1d0753e700 1 -- 172.21.15.32:0/3048629693 shutdown_connections 2023-09-30T14:45:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.667+0000 7f1d0753e700 1 --2- 172.21.15.32:0/3048629693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d001029b0 0x7f1d001092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.667+0000 7f1d0753e700 1 --2- 172.21.15.32:0/3048629693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d00100ea0 0x7f1d001012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.667+0000 7f1d0753e700 1 --2- 172.21.15.32:0/3048629693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d00101ff0 0x7f1d00102470 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.667+0000 7f1d0753e700 1 -- 172.21.15.32:0/3048629693 >> 172.21.15.32:0/3048629693 conn(0x7f1d000fc3f0 msgr2=0x7f1d000fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.667+0000 7f1d0753e700 1 -- 172.21.15.32:0/3048629693 shutdown_connections 2023-09-30T14:45:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.667+0000 7f1d0753e700 1 -- 172.21.15.32:0/3048629693 wait complete. 2023-09-30T14:45:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.667+0000 7f1d0753e700 1 Processor -- start 2023-09-30T14:45:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.667+0000 7f1d0753e700 1 -- start start 2023-09-30T14:45:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d0753e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d00100ea0 0x7f1d00196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d0753e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d00101ff0 0x7f1d00197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d0753e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d001029b0 0x7f1d00197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d0753e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d00198690 con 0x7f1d00101ff0 2023-09-30T14:45:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d0753e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d00198800 con 0x7f1d00100ea0 2023-09-30T14:45:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d0753e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d001989c0 con 0x7f1d001029b0 2023-09-30T14:45:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d04ad9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d00101ff0 0x7f1d00197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d052da700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d00100ea0 0x7f1d00196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d04ad9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d00101ff0 0x7f1d00197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37084/0 (socket says 172.21.15.32:37084) 2023-09-30T14:45:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d04ad9700 1 -- 172.21.15.32:0/2980208358 learned_addr learned my addr 172.21.15.32:0/2980208358 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d05adb700 1 --2- 172.21.15.32:0/2980208358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d001029b0 0x7f1d00197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d04ad9700 1 -- 172.21.15.32:0/2980208358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d001029b0 msgr2=0x7f1d00197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d04ad9700 1 --2- 172.21.15.32:0/2980208358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d001029b0 0x7f1d00197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d04ad9700 1 -- 172.21.15.32:0/2980208358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d00100ea0 msgr2=0x7f1d00196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d04ad9700 1 --2- 172.21.15.32:0/2980208358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d00100ea0 0x7f1d00196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d04ad9700 1 -- 172.21.15.32:0/2980208358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf00095f0 con 0x7f1d00101ff0 2023-09-30T14:45:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d05adb700 1 --2- 172.21.15.32:0/2980208358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d001029b0 0x7f1d00197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:45:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d04ad9700 1 --2- 172.21.15.32:0/2980208358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d00101ff0 0x7f1d00197250 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0000c00 tx=0x7f1cf00047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1cf67fc700 1 -- 172.21.15.32:0/2980208358 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf001d070 con 0x7f1d00101ff0 2023-09-30T14:45:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1cf67fc700 1 -- 172.21.15.32:0/2980208358 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cf0022470 con 0x7f1d00101ff0 2023-09-30T14:45:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1d0753e700 1 -- 172.21.15.32:0/2980208358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d00198c40 con 0x7f1d00101ff0 2023-09-30T14:45:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.668+0000 7f1cf67fc700 1 -- 172.21.15.32:0/2980208358 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0017650 con 0x7f1d00101ff0 2023-09-30T14:45:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.669+0000 7f1d0753e700 1 -- 172.21.15.32:0/2980208358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0019f3c0 con 0x7f1d00101ff0 2023-09-30T14:45:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.671+0000 7f1cf67fc700 1 -- 172.21.15.32:0/2980208358 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f1cf00225e0 con 0x7f1d00101ff0 2023-09-30T14:45:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.671+0000 7f1cf67fc700 1 --2- 172.21.15.32:0/2980208358 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1cec0a0fa0 0x7f1cec0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.671+0000 7f1cf67fc700 1 -- 172.21.15.32:0/2980208358 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1cf00cdd70 con 0x7f1d00101ff0 2023-09-30T14:45:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.671+0000 7f1d052da700 1 --2- 172.21.15.32:0/2980208358 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1cec0a0fa0 0x7f1cec0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.672+0000 7f1d052da700 1 --2- 172.21.15.32:0/2980208358 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1cec0a0fa0 0x7f1cec0a3460 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009c60 tx=0x7f1cfc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.672+0000 7f1d13726b80 1 Processor -- start 2023-09-30T14:45:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.672+0000 7f1d13726b80 1 -- start start 2023-09-30T14:45:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d13726b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ec200 0x560f6b4ec620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d13726b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4ed310 0x560f6b4ed790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d13726b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4edcd0 0x560f6b4f4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d13726b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6b454400 con 0x560f6b4ec200 2023-09-30T14:45:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d13726b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6b445bc0 con 0x560f6b4edcd0 2023-09-30T14:45:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d13726b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6b4434e0 con 0x560f6b4ed310 2023-09-30T14:45:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d052da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ec200 0x560f6b4ec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d052da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ec200 0x560f6b4ec620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37090/0 (socket says 172.21.15.32:37090) 2023-09-30T14:45:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d052da700 1 -- 172.21.15.32:0/1973101543 learned_addr learned my addr 172.21.15.32:0/1973101543 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d04ad9700 1 --2- 172.21.15.32:0/1973101543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4ed310 0x560f6b4ed790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d05adb700 1 --2- 172.21.15.32:0/1973101543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4edcd0 0x560f6b4f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d052da700 1 -- 172.21.15.32:0/1973101543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4ed310 msgr2=0x560f6b4ed790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d052da700 1 --2- 172.21.15.32:0/1973101543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4ed310 0x560f6b4ed790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d052da700 1 -- 172.21.15.32:0/1973101543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4edcd0 msgr2=0x560f6b4f4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d052da700 1 --2- 172.21.15.32:0/1973101543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4edcd0 0x560f6b4f4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d052da700 1 -- 172.21.15.32:0/1973101543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f6b4a1870 con 0x560f6b4ec200 2023-09-30T14:45:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1d052da700 1 --2- 172.21.15.32:0/1973101543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ec200 0x560f6b4ec620 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00e4e0 tx=0x7f1cfc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c78ca286a5f84d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1ceb7fe700 1 -- 172.21.15.32:0/1973101543 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc00ee00 con 0x560f6b4ec200 2023-09-30T14:45:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1ceb7fe700 1 -- 172.21.15.32:0/1973101543 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cfc021070 con 0x560f6b4ec200 2023-09-30T14:45:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.673+0000 7f1ceb7fe700 1 -- 172.21.15.32:0/1973101543 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc01d7b0 con 0x560f6b4ec200 2023-09-30T14:45:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.674+0000 7f1d13726b80 1 -- 172.21.15.32:0/1973101543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ec200 msgr2=0x560f6b4ec620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.674+0000 7f1d13726b80 1 --2- 172.21.15.32:0/1973101543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ec200 0x560f6b4ec620 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00e4e0 tx=0x7f1cfc014000 comp rx=0 tx=0).stop 2023-09-30T14:45:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.674+0000 7f1d13726b80 1 -- 172.21.15.32:0/1973101543 shutdown_connections 2023-09-30T14:45:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.674+0000 7f1d13726b80 1 --2- 172.21.15.32:0/1973101543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4edcd0 0x560f6b4f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.674+0000 7f1d13726b80 1 --2- 172.21.15.32:0/1973101543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ec200 0x560f6b4ec620 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.674+0000 7f1d13726b80 1 --2- 172.21.15.32:0/1973101543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4ed310 0x560f6b4ed790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.674+0000 7f1d13726b80 1 -- 172.21.15.32:0/1973101543 >> 172.21.15.32:0/1973101543 conn(0x560f6b460200 msgr2=0x560f6b4e6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.674+0000 7f1d13726b80 1 -- 172.21.15.32:0/1973101543 shutdown_connections 2023-09-30T14:45:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.674+0000 7f1d13726b80 1 -- 172.21.15.32:0/1973101543 wait complete. 2023-09-30T14:45:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.675+0000 7f1d13726b80 1 -- start start 2023-09-30T14:45:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.675+0000 7f1d13726b80 1 Processor -- start 2023-09-30T14:45:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.675+0000 7f1d13726b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4ec200 0x560f6b4f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.675+0000 7f1d13726b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ed310 0x560f6b4f6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.675+0000 7f1d13726b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4edcd0 0x560f6b4fd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.675+0000 7f1d13726b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6b43edd0 con 0x560f6b4ed310 2023-09-30T14:45:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.675+0000 7f1d13726b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6b3db6b0 con 0x560f6b4ec200 2023-09-30T14:45:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.675+0000 7f1d13726b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6b3db350 con 0x560f6b4edcd0 2023-09-30T14:45:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.675+0000 7f1d13726b80 10 client.0 _ll_register_callbacks cb 0x560f6b2387a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:45:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1d04ad9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ed310 0x560f6b4f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1d04ad9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ed310 0x560f6b4f6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37098/0 (socket says 172.21.15.32:37098) 2023-09-30T14:45:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1d04ad9700 1 -- 172.21.15.32:0/1263579968 learned_addr learned my addr 172.21.15.32:0/1263579968 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1d052da700 1 --2- 172.21.15.32:0/1263579968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4ec200 0x560f6b4f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1d05adb700 1 --2- 172.21.15.32:0/1263579968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4edcd0 0x560f6b4fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1d04ad9700 1 -- 172.21.15.32:0/1263579968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4edcd0 msgr2=0x560f6b4fd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1d04ad9700 1 --2- 172.21.15.32:0/1263579968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4edcd0 0x560f6b4fd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1d04ad9700 1 -- 172.21.15.32:0/1263579968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4ec200 msgr2=0x560f6b4f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1d04ad9700 1 --2- 172.21.15.32:0/1263579968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4ec200 0x560f6b4f5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1d04ad9700 1 -- 172.21.15.32:0/1263579968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cfc017070 con 0x560f6b4ed310 2023-09-30T14:45:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1d04ad9700 1 --2- 172.21.15.32:0/1263579968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ed310 0x560f6b4f6400 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1cf009b210 tx=0x7f1cf0022ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4550e0a1ec552287 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1ce8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:45:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1ce8ff9700 1 -- 172.21.15.32:0/1263579968 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00a35d0 con 0x560f6b4ed310 2023-09-30T14:45:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1ce8ff9700 1 -- 172.21.15.32:0/1263579968 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1cf009a7c0 con 0x560f6b4ed310 2023-09-30T14:45:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1d13726b80 1 -- 172.21.15.32:0/1263579968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f6b4df090 con 0x560f6b4ed310 2023-09-30T14:45:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1ce8ff9700 1 -- 172.21.15.32:0/1263579968 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cf009a980 con 0x560f6b4ed310 2023-09-30T14:45:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1d13726b80 10 client.16089 fetch_fsmap learned FSMap version 35 2023-09-30T14:45:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.676+0000 7f1d13726b80 1 -- 172.21.15.32:0/1263579968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x560f6b269d80 con 0x560f6b4ed310 2023-09-30T14:45:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.677+0000 7f1ce8ff9700 1 -- 172.21.15.32:0/1263579968 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560f6b4df090 con 0x560f6b4ed310 2023-09-30T14:45:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.677+0000 7f1ce8ff9700 1 -- 172.21.15.32:0/1263579968 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 35) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1cf00a11d0 con 0x560f6b4ed310 2023-09-30T14:45:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.677+0000 7f1d13726b80 10 client.16089 fetch_fsmap finished waiting for FSMap version 35 2023-09-30T14:45:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.677+0000 7f1d13726b80 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:45:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.677+0000 7f1d13726b80 20 client.16089 populate_metadata read hostname 'smithi032' 2023-09-30T14:45:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.677+0000 7f1d13726b80 1 --2- 172.21.15.32:0/1263579968 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560f6b501900 0x560f6b503cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.677+0000 7f1d13726b80 4 client.16089 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:45:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.677+0000 7f1d13726b80 1 -- 172.21.15.32:0/1263579968 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f6b437db0 con 0x560f6b501900 2023-09-30T14:45:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.677+0000 7f1d052da700 1 --2- 172.21.15.32:0/1263579968 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560f6b501900 0x560f6b503cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.677+0000 7f1ce8ff9700 10 client.16089 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.677+0000 7f1d052da700 1 --2- 172.21.15.32:0/1263579968 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560f6b501900 0x560f6b503cf0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86515c57f5c80ec2 server_cookie=5744180d91ff3c83 in_seq=0 out_seq=0 2023-09-30T14:45:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.678+0000 7f1ce8ff9700 1 -- 172.21.15.32:0/1263579968 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560f6b437db0 con 0x560f6b501900 2023-09-30T14:45:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.678+0000 7f1ce8ff9700 10 client.16089 handle_command_reply: tid=0 2023-09-30T14:45:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.678+0000 7f1d13726b80 1 client.16089 shutdown 2023-09-30T14:45:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.678+0000 7f1d13726b80 1 -- 172.21.15.32:0/1263579968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ed310 msgr2=0x560f6b4f6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.678+0000 7f1d13726b80 1 --2- 172.21.15.32:0/1263579968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ed310 0x560f6b4f6400 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1cf009b210 tx=0x7f1cf0022ba0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.678+0000 7f1d13726b80 1 -- 172.21.15.32:0/1263579968 shutdown_connections 2023-09-30T14:45:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.678+0000 7f1d13726b80 1 --2- 172.21.15.32:0/1263579968 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560f6b501900 0x560f6b503cf0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.678+0000 7f1d13726b80 1 --2- 172.21.15.32:0/1263579968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4edcd0 0x560f6b4fd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.678+0000 7f1d13726b80 1 --2- 172.21.15.32:0/1263579968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4ec200 0x560f6b4f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.678+0000 7f1ce8ff9700 0 client.16089 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.678+0000 7f1d13726b80 1 --2- 172.21.15.32:0/1263579968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ed310 0x560f6b4f6400 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.678+0000 7f1d13726b80 1 -- 172.21.15.32:0/1263579968 >> 172.21.15.32:0/1263579968 conn(0x560f6b460200 msgr2=0x560f6b4e6590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.679+0000 7f1d13726b80 1 -- 172.21.15.32:0/1263579968 shutdown_connections 2023-09-30T14:45:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.679+0000 7f1d13726b80 1 -- 172.21.15.32:0/1263579968 wait complete. 2023-09-30T14:45:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.679+0000 7f1d13726b80 20 client.16089 trim_cache size 0 max 16384 2023-09-30T14:45:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.694+0000 7f1d13726b80 1 Processor -- start 2023-09-30T14:45:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.694+0000 7f1d13726b80 1 -- start start 2023-09-30T14:45:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.694+0000 7f1d13726b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ec200 0x560f6b4e6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.694+0000 7f1d13726b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4ed310 0x560f6b4fd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.694+0000 7f1d13726b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4edcd0 0x560f6b4fdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1d13726b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6b2bf8c0 con 0x560f6b4ec200 2023-09-30T14:45:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1d13726b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6b4f5a30 con 0x560f6b4edcd0 2023-09-30T14:45:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1d13726b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6b4ee0b0 con 0x560f6b4ed310 2023-09-30T14:45:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1d052da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ec200 0x560f6b4e6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1d052da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ec200 0x560f6b4e6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37104/0 (socket says 172.21.15.32:37104) 2023-09-30T14:45:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1d052da700 1 -- 172.21.15.32:0/2722901792 learned_addr learned my addr 172.21.15.32:0/2722901792 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1d04ad9700 1 --2- 172.21.15.32:0/2722901792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4ed310 0x560f6b4fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1d05adb700 1 --2- 172.21.15.32:0/2722901792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4edcd0 0x560f6b4fdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1d052da700 1 -- 172.21.15.32:0/2722901792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4ed310 msgr2=0x560f6b4fd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1d052da700 1 --2- 172.21.15.32:0/2722901792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4ed310 0x560f6b4fd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1d052da700 1 -- 172.21.15.32:0/2722901792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4edcd0 msgr2=0x560f6b4fdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1d052da700 1 --2- 172.21.15.32:0/2722901792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4edcd0 0x560f6b4fdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1d052da700 1 -- 172.21.15.32:0/2722901792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f6b252c00 con 0x560f6b4ec200 2023-09-30T14:45:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1d05adb700 1 --2- 172.21.15.32:0/2722901792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4edcd0 0x560f6b4fdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:45:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1d052da700 1 --2- 172.21.15.32:0/2722901792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ec200 0x560f6b4e6ec0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00e4e0 tx=0x7f1cfc005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c41010d6e766eb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1cebfff700 1 -- 172.21.15.32:0/2722901792 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc00ec00 con 0x560f6b4ec200 2023-09-30T14:45:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1cebfff700 1 -- 172.21.15.32:0/2722901792 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cfc033170 con 0x560f6b4ec200 2023-09-30T14:45:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.695+0000 7f1cebfff700 1 -- 172.21.15.32:0/2722901792 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc005450 con 0x560f6b4ec200 2023-09-30T14:45:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.696+0000 7f1d13726b80 1 -- 172.21.15.32:0/2722901792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ec200 msgr2=0x560f6b4e6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.696+0000 7f1d13726b80 1 --2- 172.21.15.32:0/2722901792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ec200 0x560f6b4e6ec0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00e4e0 tx=0x7f1cfc005c80 comp rx=0 tx=0).stop 2023-09-30T14:45:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.696+0000 7f1d13726b80 1 -- 172.21.15.32:0/2722901792 shutdown_connections 2023-09-30T14:45:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.696+0000 7f1d13726b80 1 --2- 172.21.15.32:0/2722901792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4edcd0 0x560f6b4fdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.696+0000 7f1d13726b80 1 --2- 172.21.15.32:0/2722901792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ec200 0x560f6b4e6ec0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.696+0000 7f1d13726b80 1 --2- 172.21.15.32:0/2722901792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4ed310 0x560f6b4fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.696+0000 7f1d13726b80 1 -- 172.21.15.32:0/2722901792 >> 172.21.15.32:0/2722901792 conn(0x560f6b460200 msgr2=0x560f6b4f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.696+0000 7f1d13726b80 1 -- 172.21.15.32:0/2722901792 shutdown_connections 2023-09-30T14:45:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.696+0000 7f1d13726b80 1 -- 172.21.15.32:0/2722901792 wait complete. 2023-09-30T14:45:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d13726b80 1 -- start start 2023-09-30T14:45:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d13726b80 1 Processor -- start 2023-09-30T14:45:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d13726b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4ec200 0x560f6b4fb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d13726b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ed310 0x560f6b4fb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d13726b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4edcd0 0x560f6b4fc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d13726b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6b4fcce0 con 0x560f6b4ed310 2023-09-30T14:45:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d13726b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6b4fce50 con 0x560f6b4ec200 2023-09-30T14:45:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d13726b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6b4fd010 con 0x560f6b4edcd0 2023-09-30T14:45:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d13726b80 10 client.0 _ll_register_callbacks cb 0x7f1cf009ad20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:45:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d04ad9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ed310 0x560f6b4fb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d04ad9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ed310 0x560f6b4fb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37112/0 (socket says 172.21.15.32:37112) 2023-09-30T14:45:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d04ad9700 1 -- 172.21.15.32:0/3302689855 learned_addr learned my addr 172.21.15.32:0/3302689855 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d052da700 1 --2- 172.21.15.32:0/3302689855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4ec200 0x560f6b4fb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d05adb700 1 --2- 172.21.15.32:0/3302689855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4edcd0 0x560f6b4fc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d04ad9700 1 -- 172.21.15.32:0/3302689855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4edcd0 msgr2=0x560f6b4fc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d04ad9700 1 --2- 172.21.15.32:0/3302689855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4edcd0 0x560f6b4fc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d04ad9700 1 -- 172.21.15.32:0/3302689855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4ec200 msgr2=0x560f6b4fb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d04ad9700 1 --2- 172.21.15.32:0/3302689855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4ec200 0x560f6b4fb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.697+0000 7f1d04ad9700 1 -- 172.21.15.32:0/3302689855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cfc02a040 con 0x560f6b4ed310 2023-09-30T14:45:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.698+0000 7f1d04ad9700 1 --2- 172.21.15.32:0/3302689855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ed310 0x560f6b4fb790 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000c010 tx=0x7f1cf0099880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a13f3e67f943300 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.698+0000 7f1ce8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:45:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.698+0000 7f1ce8ff9700 1 -- 172.21.15.32:0/3302689855 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00a35d0 con 0x560f6b4ed310 2023-09-30T14:45:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.698+0000 7f1ce8ff9700 1 -- 172.21.15.32:0/3302689855 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1cf0097200 con 0x560f6b4ed310 2023-09-30T14:45:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.698+0000 7f1d13726b80 1 -- 172.21.15.32:0/3302689855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1cfc021070 con 0x560f6b4ed310 2023-09-30T14:45:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.698+0000 7f1ce8ff9700 1 -- 172.21.15.32:0/3302689855 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cf00973c0 con 0x560f6b4ed310 2023-09-30T14:45:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.698+0000 7f1d13726b80 10 client.16095 fetch_fsmap learned FSMap version 35 2023-09-30T14:45:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.698+0000 7f1d13726b80 1 -- 172.21.15.32:0/3302689855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x560f6b2901f0 con 0x560f6b4ed310 2023-09-30T14:45:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.698+0000 7f1ce8ff9700 1 -- 172.21.15.32:0/3302689855 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cfc021070 con 0x560f6b4ed310 2023-09-30T14:45:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.698+0000 7f1ce8ff9700 1 -- 172.21.15.32:0/3302689855 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 35) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1cf009a2a0 con 0x560f6b4ed310 2023-09-30T14:45:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.698+0000 7f1d13726b80 10 client.16095 fetch_fsmap finished waiting for FSMap version 35 2023-09-30T14:45:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.698+0000 7f1d13726b80 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:45:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.698+0000 7f1d13726b80 20 client.16095 populate_metadata read hostname 'smithi032' 2023-09-30T14:45:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.698+0000 7f1d13726b80 1 --2- 172.21.15.32:0/3302689855 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560f6b556120 0x560f6b558510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.698+0000 7f1d13726b80 4 client.16095 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:45:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.698+0000 7f1d13726b80 1 -- 172.21.15.32:0/3302689855 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560f6b437ad0 con 0x560f6b556120 2023-09-30T14:45:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.699+0000 7f1d052da700 1 --2- 172.21.15.32:0/3302689855 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560f6b556120 0x560f6b558510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.699+0000 7f1ce8ff9700 10 client.16095 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.699+0000 7f1d052da700 1 --2- 172.21.15.32:0/3302689855 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560f6b556120 0x560f6b558510 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=571495893479f62b server_cookie=e53cde4de95da82d in_seq=0 out_seq=0 2023-09-30T14:45:06.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 701551 10.55 MB/sec execute 220 sec latency 23.016 ms 2023-09-30T14:45:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.757+0000 7f1ce8ff9700 1 -- 172.21.15.32:0/3302689855 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560f6b437ad0 con 0x560f6b556120 2023-09-30T14:45:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.757+0000 7f1ce8ff9700 10 client.16095 handle_command_reply: tid=0 2023-09-30T14:45:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.757+0000 7f1d13726b80 1 client.16095 shutdown 2023-09-30T14:45:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.757+0000 7f1d13726b80 1 -- 172.21.15.32:0/3302689855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ed310 msgr2=0x560f6b4fb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.757+0000 7f1d13726b80 1 --2- 172.21.15.32:0/3302689855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ed310 0x560f6b4fb790 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000c010 tx=0x7f1cf0099880 comp rx=0 tx=0).stop 2023-09-30T14:45:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.757+0000 7f1d13726b80 1 -- 172.21.15.32:0/3302689855 shutdown_connections 2023-09-30T14:45:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.757+0000 7f1d13726b80 1 --2- 172.21.15.32:0/3302689855 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560f6b556120 0x560f6b558510 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.757+0000 7f1d13726b80 1 --2- 172.21.15.32:0/3302689855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560f6b4edcd0 0x560f6b4fc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.757+0000 7f1d13726b80 1 --2- 172.21.15.32:0/3302689855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f6b4ec200 0x560f6b4fb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.757+0000 7f1d13726b80 1 --2- 172.21.15.32:0/3302689855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560f6b4ed310 0x560f6b4fb790 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.757+0000 7f1ce8ff9700 0 client.16095 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.757+0000 7f1d13726b80 1 -- 172.21.15.32:0/3302689855 >> 172.21.15.32:0/3302689855 conn(0x560f6b460200 msgr2=0x560f6b4fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.757+0000 7f1d13726b80 1 -- 172.21.15.32:0/3302689855 shutdown_connections 2023-09-30T14:45:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.757+0000 7f1d13726b80 1 -- 172.21.15.32:0/3302689855 wait complete. 2023-09-30T14:45:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.757+0000 7f1d13726b80 20 client.16095 trim_cache size 0 max 16384 2023-09-30T14:45:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:45:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:45:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:45:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:45:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.758+0000 7f1d0753e700 1 -- 172.21.15.32:0/2980208358 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1cec0a0fa0 msgr2=0x7f1cec0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.758+0000 7f1d0753e700 1 --2- 172.21.15.32:0/2980208358 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1cec0a0fa0 0x7f1cec0a3460 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009c60 tx=0x7f1cfc009260 comp rx=0 tx=0).stop 2023-09-30T14:45:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.758+0000 7f1d0753e700 1 -- 172.21.15.32:0/2980208358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d00101ff0 msgr2=0x7f1d00197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.758+0000 7f1d0753e700 1 --2- 172.21.15.32:0/2980208358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d00101ff0 0x7f1d00197250 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0000c00 tx=0x7f1cf00047a0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.758+0000 7f1d0753e700 1 -- 172.21.15.32:0/2980208358 shutdown_connections 2023-09-30T14:45:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.758+0000 7f1d0753e700 1 --2- 172.21.15.32:0/2980208358 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1cec0a0fa0 0x7f1cec0a3460 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.758+0000 7f1d0753e700 1 --2- 172.21.15.32:0/2980208358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d001029b0 0x7f1d00197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.758+0000 7f1d0753e700 1 --2- 172.21.15.32:0/2980208358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d00100ea0 0x7f1d00196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.758+0000 7f1d0753e700 1 --2- 172.21.15.32:0/2980208358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d00101ff0 0x7f1d00197250 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.758+0000 7f1d0753e700 1 -- 172.21.15.32:0/2980208358 >> 172.21.15.32:0/2980208358 conn(0x7f1d000fc3f0 msgr2=0x7f1d001055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.758+0000 7f1d0753e700 1 -- 172.21.15.32:0/2980208358 shutdown_connections 2023-09-30T14:45:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:06.758+0000 7f1d0753e700 1 -- 172.21.15.32:0/2980208358 wait complete. 2023-09-30T14:45:07.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 703599 10.57 MB/sec execute 221 sec latency 16.135 ms 2023-09-30T14:45:07.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:07 smithi148 ceph-mon[102355]: pgmap v492: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 3.3 MiB/s wr, 332 op/s 2023-09-30T14:45:07.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:07 smithi032 ceph-mon[98592]: pgmap v492: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 3.3 MiB/s wr, 332 op/s 2023-09-30T14:45:08.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:07 smithi156 ceph-mon[103579]: pgmap v492: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 3.3 MiB/s wr, 332 op/s 2023-09-30T14:45:08.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 706129 10.58 MB/sec execute 222 sec latency 15.509 ms 2023-09-30T14:45:09.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 707924 10.57 MB/sec execute 223 sec latency 11.292 ms 2023-09-30T14:45:09.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:09 smithi148 ceph-mon[102355]: pgmap v493: 97 pgs: 97 active+clean; 236 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 3.2 MiB/s wr, 271 op/s 2023-09-30T14:45:09.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:09 smithi032 ceph-mon[98592]: pgmap v493: 97 pgs: 97 active+clean; 236 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 3.2 MiB/s wr, 271 op/s 2023-09-30T14:45:10.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:09 smithi156 ceph-mon[103579]: pgmap v493: 97 pgs: 97 active+clean; 236 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 3.2 MiB/s wr, 271 op/s 2023-09-30T14:45:10.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 710377 10.59 MB/sec execute 224 sec latency 35.467 ms 2023-09-30T14:45:10.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:10 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:45:10.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:10 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:45:11.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:10 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:45:11.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 712191 10.57 MB/sec execute 225 sec latency 17.918 ms 2023-09-30T14:45:11.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:11 smithi148 ceph-mon[102355]: pgmap v494: 97 pgs: 97 active+clean; 236 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 3.0 MiB/s wr, 276 op/s 2023-09-30T14:45:11.913 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:11 smithi032 ceph-mon[98592]: pgmap v494: 97 pgs: 97 active+clean; 236 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 3.0 MiB/s wr, 276 op/s 2023-09-30T14:45:12.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:11 smithi156 ceph-mon[103579]: pgmap v494: 97 pgs: 97 active+clean; 236 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 3.0 MiB/s wr, 276 op/s 2023-09-30T14:45:12.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 714334 10.59 MB/sec execute 226 sec latency 9.529 ms 2023-09-30T14:45:13.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 716973 10.60 MB/sec execute 227 sec latency 18.204 ms 2023-09-30T14:45:13.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:13 smithi148 ceph-mon[102355]: pgmap v495: 97 pgs: 97 active+clean; 239 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 3.1 MiB/s wr, 299 op/s 2023-09-30T14:45:13.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:13 smithi032 ceph-mon[98592]: pgmap v495: 97 pgs: 97 active+clean; 239 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 3.1 MiB/s wr, 299 op/s 2023-09-30T14:45:14.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:13 smithi156 ceph-mon[103579]: pgmap v495: 97 pgs: 97 active+clean; 239 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 3.1 MiB/s wr, 299 op/s 2023-09-30T14:45:14.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 718801 10.60 MB/sec execute 228 sec latency 12.647 ms 2023-09-30T14:45:15.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 721142 10.61 MB/sec execute 229 sec latency 31.039 ms 2023-09-30T14:45:15.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:15 smithi148 ceph-mon[102355]: pgmap v496: 97 pgs: 97 active+clean; 239 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 2.8 MiB/s wr, 268 op/s 2023-09-30T14:45:15.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:15 smithi032 ceph-mon[98592]: pgmap v496: 97 pgs: 97 active+clean; 239 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 2.8 MiB/s wr, 268 op/s 2023-09-30T14:45:16.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:15 smithi156 ceph-mon[103579]: pgmap v496: 97 pgs: 97 active+clean; 239 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 2.8 MiB/s wr, 268 op/s 2023-09-30T14:45:16.706 INFO:tasks.workunit.client.0.smithi032.stdout: 1 723280 10.60 MB/sec execute 230 sec latency 15.190 ms 2023-09-30T14:45:17.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 725071 10.61 MB/sec execute 231 sec latency 14.696 ms 2023-09-30T14:45:17.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:17 smithi148 ceph-mon[102355]: pgmap v497: 97 pgs: 97 active+clean; 241 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 3.2 MiB/s wr, 333 op/s 2023-09-30T14:45:17.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:17 smithi032 ceph-mon[98592]: pgmap v497: 97 pgs: 97 active+clean; 241 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 3.2 MiB/s wr, 333 op/s 2023-09-30T14:45:18.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:17 smithi156 ceph-mon[103579]: pgmap v497: 97 pgs: 97 active+clean; 241 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 3.2 MiB/s wr, 333 op/s 2023-09-30T14:45:18.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 727578 10.62 MB/sec execute 232 sec latency 27.192 ms 2023-09-30T14:45:18.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:18 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:45:18.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:18 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:45:19.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:18 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:45:19.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 728669 10.60 MB/sec execute 233 sec latency 86.060 ms 2023-09-30T14:45:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:19 smithi148 ceph-mon[102355]: pgmap v498: 97 pgs: 97 active+clean; 241 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.8 MiB/s wr, 269 op/s 2023-09-30T14:45:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.873+0000 7f9d077f3700 1 -- 172.21.15.32:0/543741785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d00101940 msgr2=0x7f9d00103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.873+0000 7f9d077f3700 1 --2- 172.21.15.32:0/543741785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d00101940 0x7f9d00103d30 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009990 tx=0x7f9cf0009ca0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.873+0000 7f9d077f3700 1 -- 172.21.15.32:0/543741785 shutdown_connections 2023-09-30T14:45:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.873+0000 7f9d077f3700 1 --2- 172.21.15.32:0/543741785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d00106ba0 0x7f9d00108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.873+0000 7f9d077f3700 1 --2- 172.21.15.32:0/543741785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d00101940 0x7f9d00103d30 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.873+0000 7f9d077f3700 1 --2- 172.21.15.32:0/543741785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d00104270 0x7f9d00106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.873+0000 7f9d077f3700 1 -- 172.21.15.32:0/543741785 >> 172.21.15.32:0/543741785 conn(0x7f9d000fb2b0 msgr2=0x7f9d000fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.873+0000 7f9d077f3700 1 -- 172.21.15.32:0/543741785 shutdown_connections 2023-09-30T14:45:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.873+0000 7f9d077f3700 1 -- 172.21.15.32:0/543741785 wait complete. 2023-09-30T14:45:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.874+0000 7f9d077f3700 1 Processor -- start 2023-09-30T14:45:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.874+0000 7f9d077f3700 1 -- start start 2023-09-30T14:45:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.874+0000 7f9d077f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d00101940 0x7f9d000ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d077f3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d00104270 0x7f9d000ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d077f3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d00106ba0 0x7f9d00100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d077f3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d001a0b70 con 0x7f9d00101940 2023-09-30T14:45:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d077f3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d001a0cb0 con 0x7f9d00106ba0 2023-09-30T14:45:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d077f3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d001a0df0 con 0x7f9d00104270 2023-09-30T14:45:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d0558f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d00101940 0x7f9d000ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d0558f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d00101940 0x7f9d000ff8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37124/0 (socket says 172.21.15.32:37124) 2023-09-30T14:45:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d0558f700 1 -- 172.21.15.32:0/192788776 learned_addr learned my addr 172.21.15.32:0/192788776 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d05d90700 1 --2- 172.21.15.32:0/192788776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d00106ba0 0x7f9d00100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d04d8e700 1 --2- 172.21.15.32:0/192788776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d00104270 0x7f9d000ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d0558f700 1 -- 172.21.15.32:0/192788776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d00104270 msgr2=0x7f9d000ffe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d0558f700 1 --2- 172.21.15.32:0/192788776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d00104270 0x7f9d000ffe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d0558f700 1 -- 172.21.15.32:0/192788776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d00106ba0 msgr2=0x7f9d00100370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d0558f700 1 --2- 172.21.15.32:0/192788776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d00106ba0 0x7f9d00100370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d0558f700 1 -- 172.21.15.32:0/192788776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf00095f0 con 0x7f9d00101940 2023-09-30T14:45:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d0558f700 1 --2- 172.21.15.32:0/192788776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d00101940 0x7f9d000ff8f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00059d0 tx=0x7f9cf0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9cf67fc700 1 -- 172.21.15.32:0/192788776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf001d070 con 0x7f9d00101940 2023-09-30T14:45:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d077f3700 1 -- 172.21.15.32:0/192788776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d001a1020 con 0x7f9d00101940 2023-09-30T14:45:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9cf67fc700 1 -- 172.21.15.32:0/192788776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cf000fb90 con 0x7f9d00101940 2023-09-30T14:45:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9cf67fc700 1 -- 172.21.15.32:0/192788776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00173f0 con 0x7f9d00101940 2023-09-30T14:45:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.875+0000 7f9d077f3700 1 -- 172.21.15.32:0/192788776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d001a1440 con 0x7f9d00101940 2023-09-30T14:45:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.879+0000 7f9d139dbb80 1 Processor -- start 2023-09-30T14:45:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.879+0000 7f9d139dbb80 1 -- start start 2023-09-30T14:45:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.879+0000 7f9d139dbb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df515200 0x5641df515620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.879+0000 7f9d139dbb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 0x5641df516790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.879+0000 7f9d139dbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516cd0 0x5641df51d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.879+0000 7f9d139dbb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641df47d400 con 0x5641df516310 2023-09-30T14:45:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.879+0000 7f9d139dbb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641df46ebc0 con 0x5641df515200 2023-09-30T14:45:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.879+0000 7f9d139dbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641df46c4e0 con 0x5641df516cd0 2023-09-30T14:45:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.879+0000 7f9d05d90700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df515200 0x5641df515620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.879+0000 7f9d04d8e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516cd0 0x5641df51d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.880+0000 7f9d04d8e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516cd0 0x5641df51d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51514/0 (socket says 172.21.15.32:51514) 2023-09-30T14:45:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.879+0000 7f9d05d90700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df515200 0x5641df515620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48606/0 (socket says 172.21.15.32:48606) 2023-09-30T14:45:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.880+0000 7f9d04d8e700 1 -- 172.21.15.32:0/1273199555 learned_addr learned my addr 172.21.15.32:0/1273199555 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.880+0000 7f9d04d8e700 1 -- 172.21.15.32:0/1273199555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df515200 msgr2=0x5641df515620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.880+0000 7f9d04d8e700 1 --2- 172.21.15.32:0/1273199555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df515200 0x5641df515620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.880+0000 7f9d04d8e700 1 -- 172.21.15.32:0/1273199555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 msgr2=0x5641df516790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T14:45:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.880+0000 7f9d04d8e700 1 --2- 172.21.15.32:0/1273199555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 0x5641df516790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.880+0000 7f9d04d8e700 1 -- 172.21.15.32:0/1273199555 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641df4ca870 con 0x5641df516cd0 2023-09-30T14:45:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.880+0000 7f9cf67fc700 1 -- 172.21.15.32:0/192788776 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f9cf000fd00 con 0x7f9d00101940 2023-09-30T14:45:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.880+0000 7f9d04d8e700 1 --2- 172.21.15.32:0/1273199555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516cd0 0x5641df51d560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009d20 tx=0x7f9cfc011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=807348431f4f7bab server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.880+0000 7f9ceb7fe700 1 -- 172.21.15.32:0/1273199555 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc019630 con 0x5641df516cd0 2023-09-30T14:45:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.880+0000 7f9ceb7fe700 1 -- 172.21.15.32:0/1273199555 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cfc019c50 con 0x5641df516cd0 2023-09-30T14:45:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.880+0000 7f9ceb7fe700 1 -- 172.21.15.32:0/1273199555 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc00ec10 con 0x5641df516cd0 2023-09-30T14:45:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.881+0000 7f9cf67fc700 1 --2- 172.21.15.32:0/192788776 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9cec0a0fa0 0x7f9cec0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.881+0000 7f9d05d90700 1 --2- 172.21.15.32:0/192788776 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9cec0a0fa0 0x7f9cec0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.881+0000 7f9cf67fc700 1 -- 172.21.15.32:0/192788776 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9cf00cdb20 con 0x7f9d00101940 2023-09-30T14:45:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.881+0000 7f9d139dbb80 1 -- 172.21.15.32:0/1273199555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516cd0 msgr2=0x5641df51d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.881+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/1273199555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516cd0 0x5641df51d560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009d20 tx=0x7f9cfc011720 comp rx=0 tx=0).stop 2023-09-30T14:45:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.881+0000 7f9d139dbb80 1 -- 172.21.15.32:0/1273199555 shutdown_connections 2023-09-30T14:45:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.881+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/1273199555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516cd0 0x5641df51d560 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.881+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/1273199555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df515200 0x5641df515620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.881+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/1273199555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 0x5641df516790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.881+0000 7f9d139dbb80 1 -- 172.21.15.32:0/1273199555 >> 172.21.15.32:0/1273199555 conn(0x5641df489200 msgr2=0x5641df50fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.881+0000 7f9d139dbb80 1 -- 172.21.15.32:0/1273199555 shutdown_connections 2023-09-30T14:45:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.881+0000 7f9d139dbb80 1 -- 172.21.15.32:0/1273199555 wait complete. 2023-09-30T14:45:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.881+0000 7f9d05d90700 1 --2- 172.21.15.32:0/192788776 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9cec0a0fa0 0x7f9cec0a3460 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5641df514fa0 tx=0x7f9cf800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.881+0000 7f9d139dbb80 1 -- start start 2023-09-30T14:45:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.882+0000 7f9d139dbb80 1 Processor -- start 2023-09-30T14:45:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.882+0000 7f9d139dbb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df515200 0x5641df51eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.882+0000 7f9d139dbb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 0x5641df51f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.882+0000 7f9d139dbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516cd0 0x5641df526470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.882+0000 7f9d139dbb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641df467dd0 con 0x5641df516310 2023-09-30T14:45:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.882+0000 7f9d139dbb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641df4046b0 con 0x5641df515200 2023-09-30T14:45:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.882+0000 7f9d139dbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641df404350 con 0x5641df516cd0 2023-09-30T14:45:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.882+0000 7f9d139dbb80 10 client.0 _ll_register_callbacks cb 0x5641df2617a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:45:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.882+0000 7f9d04d8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 0x5641df51f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.882+0000 7f9d04d8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 0x5641df51f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37138/0 (socket says 172.21.15.32:37138) 2023-09-30T14:45:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.882+0000 7f9d04d8e700 1 -- 172.21.15.32:0/3277529765 learned_addr learned my addr 172.21.15.32:0/3277529765 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.882+0000 7f9d0558f700 1 --2- 172.21.15.32:0/3277529765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df515200 0x5641df51eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.883+0000 7f9d05d90700 1 --2- 172.21.15.32:0/3277529765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516cd0 0x5641df526470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.883+0000 7f9d04d8e700 1 -- 172.21.15.32:0/3277529765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516cd0 msgr2=0x5641df526470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.883+0000 7f9d04d8e700 1 --2- 172.21.15.32:0/3277529765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516cd0 0x5641df526470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.883+0000 7f9d04d8e700 1 -- 172.21.15.32:0/3277529765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df515200 msgr2=0x5641df51eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.883+0000 7f9d04d8e700 1 --2- 172.21.15.32:0/3277529765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df515200 0x5641df51eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.883+0000 7f9d04d8e700 1 -- 172.21.15.32:0/3277529765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cfc009980 con 0x5641df516310 2023-09-30T14:45:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.883+0000 7f9d04d8e700 1 --2- 172.21.15.32:0/3277529765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 0x5641df51f430 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc005c50 tx=0x7f9cfc00e460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2ffd421e7bc082 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.883+0000 7f9ce8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:45:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.883+0000 7f9ce8ff9700 1 -- 172.21.15.32:0/3277529765 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc005450 con 0x5641df516310 2023-09-30T14:45:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.883+0000 7f9ce8ff9700 1 -- 172.21.15.32:0/3277529765 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9cfc005ce0 con 0x5641df516310 2023-09-30T14:45:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.883+0000 7f9d139dbb80 1 -- 172.21.15.32:0/3277529765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641df508090 con 0x5641df516310 2023-09-30T14:45:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.883+0000 7f9ce8ff9700 1 -- 172.21.15.32:0/3277529765 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cfc0055b0 con 0x5641df516310 2023-09-30T14:45:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.883+0000 7f9d139dbb80 10 client.16104 fetch_fsmap learned FSMap version 37 2023-09-30T14:45:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.883+0000 7f9d139dbb80 1 -- 172.21.15.32:0/3277529765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5641df292d80 con 0x5641df516310 2023-09-30T14:45:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.883+0000 7f9ce8ff9700 1 -- 172.21.15.32:0/3277529765 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5641df508090 con 0x5641df516310 2023-09-30T14:45:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.884+0000 7f9ce8ff9700 1 -- 172.21.15.32:0/3277529765 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 37) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f9cfc034070 con 0x5641df516310 2023-09-30T14:45:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.884+0000 7f9d139dbb80 10 client.16104 fetch_fsmap finished waiting for FSMap version 37 2023-09-30T14:45:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.884+0000 7f9d139dbb80 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:45:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.884+0000 7f9d139dbb80 20 client.16104 populate_metadata read hostname 'smithi032' 2023-09-30T14:45:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.884+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/3277529765 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5641df52ac90 0x5641df52d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.884+0000 7f9d139dbb80 4 client.16104 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:45:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.884+0000 7f9d139dbb80 1 -- 172.21.15.32:0/3277529765 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641df460db0 con 0x5641df52ac90 2023-09-30T14:45:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.884+0000 7f9d0558f700 1 --2- 172.21.15.32:0/3277529765 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5641df52ac90 0x5641df52d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.884+0000 7f9ce8ff9700 10 client.16104 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.884+0000 7f9d0558f700 1 --2- 172.21.15.32:0/3277529765 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5641df52ac90 0x5641df52d0b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aa4834b62bef9e6 server_cookie=3083d1ad7a6719b7 in_seq=0 out_seq=0 2023-09-30T14:45:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.885+0000 7f9ce8ff9700 1 -- 172.21.15.32:0/3277529765 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5641df460db0 con 0x5641df52ac90 2023-09-30T14:45:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.885+0000 7f9ce8ff9700 10 client.16104 handle_command_reply: tid=0 2023-09-30T14:45:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.885+0000 7f9d139dbb80 1 client.16104 shutdown 2023-09-30T14:45:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.885+0000 7f9d139dbb80 1 -- 172.21.15.32:0/3277529765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 msgr2=0x5641df51f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.885+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/3277529765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 0x5641df51f430 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc005c50 tx=0x7f9cfc00e460 comp rx=0 tx=0).stop 2023-09-30T14:45:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.885+0000 7f9d139dbb80 1 -- 172.21.15.32:0/3277529765 shutdown_connections 2023-09-30T14:45:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.885+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/3277529765 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5641df52ac90 0x5641df52d0b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.885+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/3277529765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516cd0 0x5641df526470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.885+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/3277529765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df515200 0x5641df51eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.885+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/3277529765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 0x5641df51f430 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.885+0000 7f9ce8ff9700 0 client.16104 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.885+0000 7f9d139dbb80 1 -- 172.21.15.32:0/3277529765 >> 172.21.15.32:0/3277529765 conn(0x5641df489200 msgr2=0x5641df50f590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.885+0000 7f9d139dbb80 1 -- 172.21.15.32:0/3277529765 shutdown_connections 2023-09-30T14:45:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.885+0000 7f9d139dbb80 1 -- 172.21.15.32:0/3277529765 wait complete. 2023-09-30T14:45:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.885+0000 7f9d139dbb80 20 client.16104 trim_cache size 0 max 16384 2023-09-30T14:45:19.892 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:19 smithi032 ceph-mon[98592]: pgmap v498: 97 pgs: 97 active+clean; 241 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.8 MiB/s wr, 269 op/s 2023-09-30T14:45:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.900+0000 7f9d139dbb80 1 Processor -- start 2023-09-30T14:45:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.900+0000 7f9d139dbb80 1 -- start start 2023-09-30T14:45:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.900+0000 7f9d139dbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df515200 0x5641df51dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.900+0000 7f9d139dbb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 0x5641df526430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.900+0000 7f9d139dbb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df516cd0 0x5641df51d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.900+0000 7f9d139dbb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641df51eac0 con 0x5641df516310 2023-09-30T14:45:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.900+0000 7f9d139dbb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641df2e88c0 con 0x5641df516cd0 2023-09-30T14:45:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.900+0000 7f9d139dbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641df50cba0 con 0x5641df515200 2023-09-30T14:45:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.900+0000 7f9d04d8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 0x5641df526430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.901+0000 7f9d04d8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 0x5641df526430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37146/0 (socket says 172.21.15.32:37146) 2023-09-30T14:45:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.901+0000 7f9d04d8e700 1 -- 172.21.15.32:0/2285384845 learned_addr learned my addr 172.21.15.32:0/2285384845 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.901+0000 7f9d0558f700 1 --2- 172.21.15.32:0/2285384845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df515200 0x5641df51dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.901+0000 7f9d05d90700 1 --2- 172.21.15.32:0/2285384845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df516cd0 0x5641df51d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.901+0000 7f9d04d8e700 1 -- 172.21.15.32:0/2285384845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df515200 msgr2=0x5641df51dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.901+0000 7f9d04d8e700 1 --2- 172.21.15.32:0/2285384845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df515200 0x5641df51dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.901+0000 7f9d04d8e700 1 -- 172.21.15.32:0/2285384845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df516cd0 msgr2=0x5641df51d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.901+0000 7f9d04d8e700 1 --2- 172.21.15.32:0/2285384845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df516cd0 0x5641df51d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.901+0000 7f9d04d8e700 1 -- 172.21.15.32:0/2285384845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641df27bc00 con 0x5641df516310 2023-09-30T14:45:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.901+0000 7f9d0558f700 1 --2- 172.21.15.32:0/2285384845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df515200 0x5641df51dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:45:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.901+0000 7f9d04d8e700 1 --2- 172.21.15.32:0/2285384845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 0x5641df526430 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc005c20 tx=0x7f9cfc011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7ef8f5f61dae3e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.901+0000 7f9cebfff700 1 -- 172.21.15.32:0/2285384845 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc004ce0 con 0x5641df516310 2023-09-30T14:45:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.901+0000 7f9cebfff700 1 -- 172.21.15.32:0/2285384845 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cfc031070 con 0x5641df516310 2023-09-30T14:45:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.901+0000 7f9cebfff700 1 -- 172.21.15.32:0/2285384845 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0198d0 con 0x5641df516310 2023-09-30T14:45:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 -- 172.21.15.32:0/2285384845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 msgr2=0x5641df526430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/2285384845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 0x5641df526430 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc005c20 tx=0x7f9cfc011720 comp rx=0 tx=0).stop 2023-09-30T14:45:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 -- 172.21.15.32:0/2285384845 shutdown_connections 2023-09-30T14:45:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/2285384845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df516cd0 0x5641df51d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/2285384845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df515200 0x5641df51dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/2285384845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516310 0x5641df526430 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 -- 172.21.15.32:0/2285384845 >> 172.21.15.32:0/2285384845 conn(0x5641df489200 msgr2=0x5641df5319d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 -- 172.21.15.32:0/2285384845 shutdown_connections 2023-09-30T14:45:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 -- 172.21.15.32:0/2285384845 wait complete. 2023-09-30T14:45:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 -- start start 2023-09-30T14:45:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 Processor -- start 2023-09-30T14:45:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df515200 0x5641df523b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516310 0x5641df524060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516cd0 0x5641df525070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641df5255b0 con 0x5641df516cd0 2023-09-30T14:45:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641df525720 con 0x5641df515200 2023-09-30T14:45:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.902+0000 7f9d139dbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641df5258e0 con 0x5641df516310 2023-09-30T14:45:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9d139dbb80 10 client.0 _ll_register_callbacks cb 0x7f9cf8009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:45:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9d05d90700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516cd0 0x5641df525070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9d04d8e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516310 0x5641df524060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9d05d90700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516cd0 0x5641df525070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37154/0 (socket says 172.21.15.32:37154) 2023-09-30T14:45:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9d05d90700 1 -- 172.21.15.32:0/1362922904 learned_addr learned my addr 172.21.15.32:0/1362922904 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9d0558f700 1 --2- 172.21.15.32:0/1362922904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df515200 0x5641df523b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9d04d8e700 1 -- 172.21.15.32:0/1362922904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df515200 msgr2=0x5641df523b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9d04d8e700 1 --2- 172.21.15.32:0/1362922904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df515200 0x5641df523b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9d04d8e700 1 -- 172.21.15.32:0/1362922904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516cd0 msgr2=0x5641df525070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9d04d8e700 1 --2- 172.21.15.32:0/1362922904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516cd0 0x5641df525070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9d04d8e700 1 -- 172.21.15.32:0/1362922904 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641df292d80 con 0x5641df516310 2023-09-30T14:45:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9d05d90700 1 --2- 172.21.15.32:0/1362922904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516cd0 0x5641df525070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:45:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9d04d8e700 1 --2- 172.21.15.32:0/1362922904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516310 0x5641df524060 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc038040 tx=0x7f9cfc019b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e1e7f31cfe7bb00 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9ce8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:45:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9ce8ff9700 1 -- 172.21.15.32:0/1362922904 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc005450 con 0x5641df516310 2023-09-30T14:45:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9ce8ff9700 1 -- 172.21.15.32:0/1362922904 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9cfc000e40 con 0x5641df516310 2023-09-30T14:45:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.903+0000 7f9d139dbb80 1 -- 172.21.15.32:0/1362922904 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cfc019c50 con 0x5641df516310 2023-09-30T14:45:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.904+0000 7f9ce8ff9700 1 -- 172.21.15.32:0/1362922904 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cfc005d20 con 0x5641df516310 2023-09-30T14:45:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.904+0000 7f9d139dbb80 10 client.25786 fetch_fsmap learned FSMap version 37 2023-09-30T14:45:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.904+0000 7f9d139dbb80 1 -- 172.21.15.32:0/1362922904 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5641df2b91f0 con 0x5641df516310 2023-09-30T14:45:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.904+0000 7f9ce8ff9700 1 -- 172.21.15.32:0/1362922904 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cfc04f020 con 0x5641df516310 2023-09-30T14:45:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.904+0000 7f9ce8ff9700 1 -- 172.21.15.32:0/1362922904 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 37) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f9cfc034070 con 0x5641df516310 2023-09-30T14:45:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.904+0000 7f9d139dbb80 10 client.25786 fetch_fsmap finished waiting for FSMap version 37 2023-09-30T14:45:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.904+0000 7f9d139dbb80 10 client.25786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:45:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.904+0000 7f9d139dbb80 20 client.25786 populate_metadata read hostname 'smithi032' 2023-09-30T14:45:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.904+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/1362922904 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5641df57ef00 0x5641df5812f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.904+0000 7f9d139dbb80 4 client.25786 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:45:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.904+0000 7f9d139dbb80 1 -- 172.21.15.32:0/1362922904 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5641df460ad0 con 0x5641df57ef00 2023-09-30T14:45:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.904+0000 7f9d0558f700 1 --2- 172.21.15.32:0/1362922904 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5641df57ef00 0x5641df5812f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.904+0000 7f9ce8ff9700 10 client.25786 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.904+0000 7f9d0558f700 1 --2- 172.21.15.32:0/1362922904 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5641df57ef00 0x5641df5812f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a006ae89ad42c9e server_cookie=3413978494335284 in_seq=0 out_seq=0 2023-09-30T14:45:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.946+0000 7f9ce8ff9700 1 -- 172.21.15.32:0/1362922904 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5641df460ad0 con 0x5641df57ef00 2023-09-30T14:45:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.946+0000 7f9ce8ff9700 10 client.25786 handle_command_reply: tid=0 2023-09-30T14:45:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.946+0000 7f9d139dbb80 1 client.25786 shutdown 2023-09-30T14:45:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.946+0000 7f9d139dbb80 1 -- 172.21.15.32:0/1362922904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516310 msgr2=0x5641df524060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.946+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/1362922904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516310 0x5641df524060 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc038040 tx=0x7f9cfc019b50 comp rx=0 tx=0).stop 2023-09-30T14:45:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.946+0000 7f9d139dbb80 1 -- 172.21.15.32:0/1362922904 shutdown_connections 2023-09-30T14:45:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.946+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/1362922904 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5641df57ef00 0x5641df5812f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.946+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/1362922904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5641df516cd0 0x5641df525070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.946+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/1362922904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5641df515200 0x5641df523b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.946+0000 7f9d139dbb80 1 --2- 172.21.15.32:0/1362922904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641df516310 0x5641df524060 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.946+0000 7f9ce8ff9700 0 client.25786 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.946+0000 7f9d139dbb80 1 -- 172.21.15.32:0/1362922904 >> 172.21.15.32:0/1362922904 conn(0x5641df489200 msgr2=0x5641df51dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.946+0000 7f9d139dbb80 1 -- 172.21.15.32:0/1362922904 shutdown_connections 2023-09-30T14:45:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.946+0000 7f9d139dbb80 1 -- 172.21.15.32:0/1362922904 wait complete. 2023-09-30T14:45:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.946+0000 7f9d139dbb80 20 client.25786 trim_cache size 0 max 16384 2023-09-30T14:45:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:45:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:45:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:45:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:45:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.947+0000 7f9d077f3700 1 -- 172.21.15.32:0/192788776 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9cec0a0fa0 msgr2=0x7f9cec0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.947+0000 7f9d077f3700 1 --2- 172.21.15.32:0/192788776 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9cec0a0fa0 0x7f9cec0a3460 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5641df514fa0 tx=0x7f9cf800b040 comp rx=0 tx=0).stop 2023-09-30T14:45:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.947+0000 7f9d077f3700 1 -- 172.21.15.32:0/192788776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d00101940 msgr2=0x7f9d000ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.947+0000 7f9d077f3700 1 --2- 172.21.15.32:0/192788776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d00101940 0x7f9d000ff8f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00059d0 tx=0x7f9cf0004730 comp rx=0 tx=0).stop 2023-09-30T14:45:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.947+0000 7f9d077f3700 1 -- 172.21.15.32:0/192788776 shutdown_connections 2023-09-30T14:45:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.947+0000 7f9d077f3700 1 --2- 172.21.15.32:0/192788776 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9cec0a0fa0 0x7f9cec0a3460 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.947+0000 7f9d077f3700 1 --2- 172.21.15.32:0/192788776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d00106ba0 0x7f9d00100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.947+0000 7f9d077f3700 1 --2- 172.21.15.32:0/192788776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d00101940 0x7f9d000ff8f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.947+0000 7f9d077f3700 1 --2- 172.21.15.32:0/192788776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d00104270 0x7f9d000ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.947+0000 7f9d077f3700 1 -- 172.21.15.32:0/192788776 >> 172.21.15.32:0/192788776 conn(0x7f9d000fb2b0 msgr2=0x7f9d00102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.947+0000 7f9d077f3700 1 -- 172.21.15.32:0/192788776 shutdown_connections 2023-09-30T14:45:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:19.947+0000 7f9d077f3700 1 -- 172.21.15.32:0/192788776 wait complete. 2023-09-30T14:45:20.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:19 smithi156 ceph-mon[103579]: pgmap v498: 97 pgs: 97 active+clean; 241 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.8 MiB/s wr, 269 op/s 2023-09-30T14:45:20.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 730840 10.60 MB/sec execute 234 sec latency 76.170 ms 2023-09-30T14:45:21.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 732303 10.59 MB/sec execute 235 sec latency 28.101 ms 2023-09-30T14:45:21.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:21 smithi148 ceph-mon[102355]: pgmap v499: 97 pgs: 97 active+clean; 244 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 2.8 MiB/s wr, 290 op/s 2023-09-30T14:45:21.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:21 smithi032 ceph-mon[98592]: pgmap v499: 97 pgs: 97 active+clean; 244 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 2.8 MiB/s wr, 290 op/s 2023-09-30T14:45:22.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:21 smithi156 ceph-mon[103579]: pgmap v499: 97 pgs: 97 active+clean; 244 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 2.8 MiB/s wr, 290 op/s 2023-09-30T14:45:22.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 735006 10.62 MB/sec execute 236 sec latency 10.989 ms 2023-09-30T14:45:23.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 736473 10.60 MB/sec execute 237 sec latency 15.401 ms 2023-09-30T14:45:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:23 smithi148 ceph-mon[102355]: pgmap v500: 97 pgs: 97 active+clean; 245 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 3.0 MiB/s wr, 308 op/s 2023-09-30T14:45:23.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:23 smithi032 ceph-mon[98592]: pgmap v500: 97 pgs: 97 active+clean; 245 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 3.0 MiB/s wr, 308 op/s 2023-09-30T14:45:24.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:23 smithi156 ceph-mon[103579]: pgmap v500: 97 pgs: 97 active+clean; 245 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 3.0 MiB/s wr, 308 op/s 2023-09-30T14:45:24.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 738818 10.61 MB/sec execute 238 sec latency 34.305 ms 2023-09-30T14:45:25.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 739845 10.58 MB/sec execute 239 sec latency 77.210 ms 2023-09-30T14:45:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:25 smithi148 ceph-mon[102355]: pgmap v501: 97 pgs: 97 active+clean; 247 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2023-09-30T14:45:25.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:25 smithi032 ceph-mon[98592]: pgmap v501: 97 pgs: 97 active+clean; 247 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2023-09-30T14:45:26.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:25 smithi156 ceph-mon[103579]: pgmap v501: 97 pgs: 97 active+clean; 247 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2023-09-30T14:45:26.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 742236 10.60 MB/sec execute 240 sec latency 23.527 ms 2023-09-30T14:45:26.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:26 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:45:26.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:26 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:45:27.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:26 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:45:27.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 743737 10.58 MB/sec execute 241 sec latency 15.082 ms 2023-09-30T14:45:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:27 smithi148 ceph-mon[102355]: pgmap v502: 97 pgs: 97 active+clean; 249 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 3.1 MiB/s wr, 303 op/s 2023-09-30T14:45:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:27 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:45:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:27 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:45:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:27 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:45:27.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:27 smithi032 ceph-mon[98592]: pgmap v502: 97 pgs: 97 active+clean; 249 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 3.1 MiB/s wr, 303 op/s 2023-09-30T14:45:27.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:27 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:45:27.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:27 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:45:27.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:27 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:45:28.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:27 smithi156 ceph-mon[103579]: pgmap v502: 97 pgs: 97 active+clean; 249 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 3.1 MiB/s wr, 303 op/s 2023-09-30T14:45:28.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:27 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:45:28.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:27 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:45:28.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:27 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:45:28.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 746056 10.60 MB/sec execute 242 sec latency 11.616 ms 2023-09-30T14:45:28.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:28 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:45:29.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:28 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:45:29.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:28 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:45:29.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 748436 10.60 MB/sec execute 243 sec latency 37.282 ms 2023-09-30T14:45:29.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:29 smithi032 ceph-mon[98592]: pgmap v503: 97 pgs: 97 active+clean; 249 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 2.8 MiB/s wr, 275 op/s 2023-09-30T14:45:30.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:29 smithi156 ceph-mon[103579]: pgmap v503: 97 pgs: 97 active+clean; 249 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 2.8 MiB/s wr, 275 op/s 2023-09-30T14:45:30.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:29 smithi148 ceph-mon[102355]: pgmap v503: 97 pgs: 97 active+clean; 249 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 2.8 MiB/s wr, 275 op/s 2023-09-30T14:45:30.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 749886 10.59 MB/sec execute 244 sec latency 26.086 ms 2023-09-30T14:45:31.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 752505 10.61 MB/sec execute 245 sec latency 54.570 ms 2023-09-30T14:45:31.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:31 smithi032 ceph-mon[98592]: pgmap v504: 97 pgs: 97 active+clean; 250 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 2.9 MiB/s wr, 285 op/s 2023-09-30T14:45:32.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:31 smithi156 ceph-mon[103579]: pgmap v504: 97 pgs: 97 active+clean; 250 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 2.9 MiB/s wr, 285 op/s 2023-09-30T14:45:32.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:31 smithi148 ceph-mon[102355]: pgmap v504: 97 pgs: 97 active+clean; 250 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 2.9 MiB/s wr, 285 op/s 2023-09-30T14:45:32.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 754378 10.60 MB/sec execute 246 sec latency 14.941 ms 2023-09-30T14:45:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.059+0000 7f4cf5793700 1 -- 172.21.15.32:0/918998796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf00ffd70 msgr2=0x7f4cf0100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.059+0000 7f4cf5793700 1 --2- 172.21.15.32:0/918998796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf00ffd70 0x7f4cf0100190 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00099c0 tx=0x7f4ce0009cd0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.059+0000 7f4cf5793700 1 -- 172.21.15.32:0/918998796 shutdown_connections 2023-09-30T14:45:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.059+0000 7f4cf5793700 1 --2- 172.21.15.32:0/918998796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf0101900 0x7f4cf01081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.059+0000 7f4cf5793700 1 --2- 172.21.15.32:0/918998796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf00ffd70 0x7f4cf0100190 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.059+0000 7f4cf5793700 1 --2- 172.21.15.32:0/918998796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cf0100f60 0x7f4cf01013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.059+0000 7f4cf5793700 1 -- 172.21.15.32:0/918998796 >> 172.21.15.32:0/918998796 conn(0x7f4cf00fb300 msgr2=0x7f4cf00fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.059+0000 7f4cf5793700 1 -- 172.21.15.32:0/918998796 shutdown_connections 2023-09-30T14:45:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.059+0000 7f4cf5793700 1 -- 172.21.15.32:0/918998796 wait complete. 2023-09-30T14:45:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.060+0000 7f4cf5793700 1 Processor -- start 2023-09-30T14:45:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.060+0000 7f4cf5793700 1 -- start start 2023-09-30T14:45:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.060+0000 7f4cf5793700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cf00ffd70 0x7f4cf0198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.060+0000 7f4cf5793700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf0100f60 0x7f4cf0199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.060+0000 7f4cf5793700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf0101900 0x7f4cf019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.060+0000 7f4cf5793700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf019a8c0 con 0x7f4cf0101900 2023-09-30T14:45:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.060+0000 7f4cf5793700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf019aa30 con 0x7f4cf0100f60 2023-09-30T14:45:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.060+0000 7f4cf5793700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf019abf0 con 0x7f4cf00ffd70 2023-09-30T14:45:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.060+0000 7f4cef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf0101900 0x7f4cf019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.060+0000 7f4ceeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cf00ffd70 0x7f4cf0198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.060+0000 7f4cee7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf0100f60 0x7f4cf0199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.060+0000 7f4cef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf0101900 0x7f4cf019a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37168/0 (socket says 172.21.15.32:37168) 2023-09-30T14:45:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.060+0000 7f4ceeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cf00ffd70 0x7f4cf0198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51546/0 (socket says 172.21.15.32:51546) 2023-09-30T14:45:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.060+0000 7f4cef7fe700 1 -- 172.21.15.32:0/3574476223 learned_addr learned my addr 172.21.15.32:0/3574476223 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.061+0000 7f4cee7fc700 1 -- 172.21.15.32:0/3574476223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cf00ffd70 msgr2=0x7f4cf0198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.061+0000 7f4cee7fc700 1 --2- 172.21.15.32:0/3574476223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cf00ffd70 0x7f4cf0198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.061+0000 7f4cee7fc700 1 -- 172.21.15.32:0/3574476223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf0101900 msgr2=0x7f4cf019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.061+0000 7f4cee7fc700 1 --2- 172.21.15.32:0/3574476223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf0101900 0x7f4cf019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.061+0000 7f4cee7fc700 1 -- 172.21.15.32:0/3574476223 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce00095f0 con 0x7f4cf0100f60 2023-09-30T14:45:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.061+0000 7f4cef7fe700 1 --2- 172.21.15.32:0/3574476223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf0101900 0x7f4cf019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:45:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.061+0000 7f4ceeffd700 1 --2- 172.21.15.32:0/3574476223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cf00ffd70 0x7f4cf0198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:45:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.061+0000 7f4cee7fc700 1 --2- 172.21.15.32:0/3574476223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf0100f60 0x7f4cf0199410 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400e9f0 tx=0x7f4ce400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.061+0000 7f4cd7fff700 1 -- 172.21.15.32:0/3574476223 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce400c9c0 con 0x7f4cf0100f60 2023-09-30T14:45:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.061+0000 7f4cd7fff700 1 -- 172.21.15.32:0/3574476223 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ce4010430 con 0x7f4cf0100f60 2023-09-30T14:45:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.061+0000 7f4cf5793700 1 -- 172.21.15.32:0/3574476223 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf006a0f0 con 0x7f4cf0100f60 2023-09-30T14:45:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.061+0000 7f4cd7fff700 1 -- 172.21.15.32:0/3574476223 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4018680 con 0x7f4cf0100f60 2023-09-30T14:45:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.061+0000 7f4cf5793700 1 -- 172.21.15.32:0/3574476223 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf01a15e0 con 0x7f4cf0100f60 2023-09-30T14:45:33.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.063+0000 7f4cd7fff700 1 -- 172.21.15.32:0/3574476223 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f4ce4010860 con 0x7f4cf0100f60 2023-09-30T14:45:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.064+0000 7f4cd7fff700 1 --2- 172.21.15.32:0/3574476223 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4cd80a0fa0 0x7f4cd80a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.064+0000 7f4cd7fff700 1 -- 172.21.15.32:0/3574476223 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4ce4014070 con 0x7f4cf0100f60 2023-09-30T14:45:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.064+0000 7f4ceeffd700 1 --2- 172.21.15.32:0/3574476223 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4cd80a0fa0 0x7f4cd80a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.065+0000 7f4ceeffd700 1 --2- 172.21.15.32:0/3574476223 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4cd80a0fa0 0x7f4cd80a3460 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009990 tx=0x7f4ce001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.065+0000 7f4d0197bb80 1 Processor -- start 2023-09-30T14:45:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.065+0000 7f4d0197bb80 1 -- start start 2023-09-30T14:45:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.065+0000 7f4d0197bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf0200 0x55dfc2bf0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.065+0000 7f4d0197bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1310 0x55dfc2bf1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.065+0000 7f4d0197bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2bf8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.065+0000 7f4d0197bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc2b58400 con 0x55dfc2bf1310 2023-09-30T14:45:33.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.065+0000 7f4d0197bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc2b49bc0 con 0x55dfc2bf0200 2023-09-30T14:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.065+0000 7f4d0197bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc2b474e0 con 0x55dfc2bf1cd0 2023-09-30T14:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.065+0000 7f4cee7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1310 0x55dfc2bf1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.065+0000 7f4cee7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1310 0x55dfc2bf1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37174/0 (socket says 172.21.15.32:37174) 2023-09-30T14:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.065+0000 7f4cee7fc700 1 -- 172.21.15.32:0/75419781 learned_addr learned my addr 172.21.15.32:0/75419781 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.066+0000 7f4cef7fe700 1 --2- 172.21.15.32:0/75419781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2bf8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.066+0000 7f4ceeffd700 1 --2- 172.21.15.32:0/75419781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf0200 0x55dfc2bf0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.066+0000 7f4cee7fc700 1 -- 172.21.15.32:0/75419781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf1cd0 msgr2=0x55dfc2bf8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.066+0000 7f4cee7fc700 1 --2- 172.21.15.32:0/75419781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2bf8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.066+0000 7f4cee7fc700 1 -- 172.21.15.32:0/75419781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf0200 msgr2=0x55dfc2bf0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.066+0000 7f4cee7fc700 1 --2- 172.21.15.32:0/75419781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf0200 0x55dfc2bf0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.066+0000 7f4cee7fc700 1 -- 172.21.15.32:0/75419781 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfc2ba5870 con 0x55dfc2bf1310 2023-09-30T14:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.066+0000 7f4cee7fc700 1 --2- 172.21.15.32:0/75419781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1310 0x55dfc2bf1790 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40a1b70 tx=0x7f4ce40996f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2237f6da8b03aa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:33.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.066+0000 7f4cd57fa700 1 -- 172.21.15.32:0/75419781 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4099d10 con 0x55dfc2bf1310 2023-09-30T14:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.066+0000 7f4cd57fa700 1 -- 172.21.15.32:0/75419781 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ce409de60 con 0x55dfc2bf1310 2023-09-30T14:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.066+0000 7f4cd57fa700 1 -- 172.21.15.32:0/75419781 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce40a1c10 con 0x55dfc2bf1310 2023-09-30T14:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.067+0000 7f4d0197bb80 1 -- 172.21.15.32:0/75419781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1310 msgr2=0x55dfc2bf1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.067+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/75419781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1310 0x55dfc2bf1790 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40a1b70 tx=0x7f4ce40996f0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.067+0000 7f4d0197bb80 1 -- 172.21.15.32:0/75419781 shutdown_connections 2023-09-30T14:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.067+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/75419781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2bf8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.067+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/75419781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf0200 0x55dfc2bf0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.067+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/75419781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1310 0x55dfc2bf1790 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.067+0000 7f4d0197bb80 1 -- 172.21.15.32:0/75419781 >> 172.21.15.32:0/75419781 conn(0x55dfc2b64200 msgr2=0x55dfc2beaef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.067+0000 7f4d0197bb80 1 -- 172.21.15.32:0/75419781 shutdown_connections 2023-09-30T14:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.067+0000 7f4d0197bb80 1 -- 172.21.15.32:0/75419781 wait complete. 2023-09-30T14:45:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.067+0000 7f4d0197bb80 1 -- start start 2023-09-30T14:45:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.067+0000 7f4d0197bb80 1 Processor -- start 2023-09-30T14:45:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4d0197bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf0200 0x55dfc2bf9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4d0197bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1310 0x55dfc2bfa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4d0197bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2c014d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4d0197bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc2b42dd0 con 0x55dfc2bf1310 2023-09-30T14:45:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4d0197bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc2adf6b0 con 0x55dfc2bf0200 2023-09-30T14:45:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4d0197bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc2adf350 con 0x55dfc2bf1cd0 2023-09-30T14:45:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4d0197bb80 10 client.0 _ll_register_callbacks cb 0x55dfc293c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:45:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4cee7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1310 0x55dfc2bfa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4cee7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1310 0x55dfc2bfa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37180/0 (socket says 172.21.15.32:37180) 2023-09-30T14:45:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4cee7fc700 1 -- 172.21.15.32:0/882987738 learned_addr learned my addr 172.21.15.32:0/882987738 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4ceeffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf0200 0x55dfc2bf9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4cef7fe700 1 --2- 172.21.15.32:0/882987738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2c014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4cee7fc700 1 -- 172.21.15.32:0/882987738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf1cd0 msgr2=0x55dfc2c014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4cee7fc700 1 --2- 172.21.15.32:0/882987738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2c014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4cee7fc700 1 -- 172.21.15.32:0/882987738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf0200 msgr2=0x55dfc2bf9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4cee7fc700 1 --2- 172.21.15.32:0/882987738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf0200 0x55dfc2bf9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4cee7fc700 1 -- 172.21.15.32:0/882987738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf01a15e0 con 0x55dfc2bf1310 2023-09-30T14:45:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4cef7fe700 1 --2- 172.21.15.32:0/882987738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2c014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:45:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4cee7fc700 1 --2- 172.21.15.32:0/882987738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1310 0x55dfc2bfa490 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4011fd0 tx=0x7f4ce4010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11a81f2422c710c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4cd2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:45:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.068+0000 7f4cd2ff5700 1 -- 172.21.15.32:0/882987738 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce40b2470 con 0x55dfc2bf1310 2023-09-30T14:45:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.069+0000 7f4cd2ff5700 1 -- 172.21.15.32:0/882987738 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4ce4099d10 con 0x55dfc2bf1310 2023-09-30T14:45:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.069+0000 7f4d0197bb80 1 -- 172.21.15.32:0/882987738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfc2be3090 con 0x55dfc2bf1310 2023-09-30T14:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.069+0000 7f4cd2ff5700 1 -- 172.21.15.32:0/882987738 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ce40a1c10 con 0x55dfc2bf1310 2023-09-30T14:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.069+0000 7f4d0197bb80 10 client.16122 fetch_fsmap learned FSMap version 38 2023-09-30T14:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.069+0000 7f4d0197bb80 1 -- 172.21.15.32:0/882987738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55dfc296dd80 con 0x55dfc2bf1310 2023-09-30T14:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.069+0000 7f4cd2ff5700 1 -- 172.21.15.32:0/882987738 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ce407d020 con 0x55dfc2bf1310 2023-09-30T14:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.069+0000 7f4cd2ff5700 1 -- 172.21.15.32:0/882987738 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4ce4098c70 con 0x55dfc2bf1310 2023-09-30T14:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.069+0000 7f4d0197bb80 10 client.16122 fetch_fsmap finished waiting for FSMap version 38 2023-09-30T14:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.069+0000 7f4d0197bb80 10 client.16122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.069+0000 7f4d0197bb80 20 client.16122 populate_metadata read hostname 'smithi032' 2023-09-30T14:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.069+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/882987738 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dfc2c05b40 0x55dfc2c07f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.069+0000 7f4d0197bb80 4 client.16122 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.069+0000 7f4d0197bb80 1 -- 172.21.15.32:0/882987738 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfc2b3bdb0 con 0x55dfc2c05b40 2023-09-30T14:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.069+0000 7f4ceeffd700 1 --2- 172.21.15.32:0/882987738 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dfc2c05b40 0x55dfc2c07f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.070+0000 7f4cd2ff5700 10 client.16122 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.070+0000 7f4ceeffd700 1 --2- 172.21.15.32:0/882987738 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dfc2c05b40 0x55dfc2c07f60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd4a1f1fa55e3533 server_cookie=c58f82ec8b7e3900 in_seq=0 out_seq=0 2023-09-30T14:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.070+0000 7f4cd2ff5700 1 -- 172.21.15.32:0/882987738 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dfc2b3bdb0 con 0x55dfc2c05b40 2023-09-30T14:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.070+0000 7f4cd2ff5700 10 client.16122 handle_command_reply: tid=0 2023-09-30T14:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.070+0000 7f4d0197bb80 1 client.16122 shutdown 2023-09-30T14:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.070+0000 7f4d0197bb80 1 -- 172.21.15.32:0/882987738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1310 msgr2=0x55dfc2bfa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.070+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/882987738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1310 0x55dfc2bfa490 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4011fd0 tx=0x7f4ce4010b60 comp rx=0 tx=0).stop 2023-09-30T14:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.070+0000 7f4d0197bb80 1 -- 172.21.15.32:0/882987738 shutdown_connections 2023-09-30T14:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.070+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/882987738 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dfc2c05b40 0x55dfc2c07f60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.070+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/882987738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2c014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.070+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/882987738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf0200 0x55dfc2bf9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.070+0000 7f4cd2ff5700 0 client.16122 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.070+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/882987738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1310 0x55dfc2bfa490 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.070+0000 7f4d0197bb80 1 -- 172.21.15.32:0/882987738 >> 172.21.15.32:0/882987738 conn(0x55dfc2b64200 msgr2=0x55dfc2bea590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.070+0000 7f4d0197bb80 1 -- 172.21.15.32:0/882987738 shutdown_connections 2023-09-30T14:45:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.070+0000 7f4d0197bb80 1 -- 172.21.15.32:0/882987738 wait complete. 2023-09-30T14:45:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.071+0000 7f4d0197bb80 20 client.16122 trim_cache size 0 max 16384 2023-09-30T14:45:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.086+0000 7f4d0197bb80 1 Processor -- start 2023-09-30T14:45:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.086+0000 7f4d0197bb80 1 -- start start 2023-09-30T14:45:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.086+0000 7f4d0197bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf0200 0x55dfc2bf9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.086+0000 7f4d0197bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf1310 0x55dfc2c01490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.086+0000 7f4d0197bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2bf8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.086+0000 7f4d0197bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc29c38c0 con 0x55dfc2bf0200 2023-09-30T14:45:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.086+0000 7f4d0197bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc2bf1af0 con 0x55dfc2bf1cd0 2023-09-30T14:45:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.086+0000 7f4d0197bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc2bf20b0 con 0x55dfc2bf1310 2023-09-30T14:45:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.086+0000 7f4ceeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf0200 0x55dfc2bf9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.086+0000 7f4ceeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf0200 0x55dfc2bf9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37186/0 (socket says 172.21.15.32:37186) 2023-09-30T14:45:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.086+0000 7f4ceeffd700 1 -- 172.21.15.32:0/1969470551 learned_addr learned my addr 172.21.15.32:0/1969470551 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.086+0000 7f4cee7fc700 1 --2- 172.21.15.32:0/1969470551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf1310 0x55dfc2c01490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.087+0000 7f4cef7fe700 1 --2- 172.21.15.32:0/1969470551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2bf8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.087+0000 7f4ceeffd700 1 -- 172.21.15.32:0/1969470551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf1310 msgr2=0x55dfc2c01490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.087+0000 7f4ceeffd700 1 --2- 172.21.15.32:0/1969470551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf1310 0x55dfc2c01490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.087+0000 7f4ceeffd700 1 -- 172.21.15.32:0/1969470551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf1cd0 msgr2=0x55dfc2bf8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.087+0000 7f4ceeffd700 1 --2- 172.21.15.32:0/1969470551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2bf8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.087+0000 7f4ceeffd700 1 -- 172.21.15.32:0/1969470551 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfc2956c00 con 0x55dfc2bf0200 2023-09-30T14:45:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.087+0000 7f4ceeffd700 1 --2- 172.21.15.32:0/1969470551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf0200 0x55dfc2bf9600 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0100dc0 tx=0x7f4ce001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc50b8185d9ed47 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.087+0000 7f4cd5ffb700 1 -- 172.21.15.32:0/1969470551 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce002c040 con 0x55dfc2bf0200 2023-09-30T14:45:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.087+0000 7f4cd5ffb700 1 -- 172.21.15.32:0/1969470551 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ce002d040 con 0x55dfc2bf0200 2023-09-30T14:45:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.087+0000 7f4cd5ffb700 1 -- 172.21.15.32:0/1969470551 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce000f460 con 0x55dfc2bf0200 2023-09-30T14:45:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.088+0000 7f4d0197bb80 1 -- 172.21.15.32:0/1969470551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf0200 msgr2=0x55dfc2bf9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.088+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/1969470551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf0200 0x55dfc2bf9600 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0100dc0 tx=0x7f4ce001f000 comp rx=0 tx=0).stop 2023-09-30T14:45:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.088+0000 7f4d0197bb80 1 -- 172.21.15.32:0/1969470551 shutdown_connections 2023-09-30T14:45:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.088+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/1969470551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2bf8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.088+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/1969470551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf0200 0x55dfc2bf9600 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.088+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/1969470551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf1310 0x55dfc2c01490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.088+0000 7f4d0197bb80 1 -- 172.21.15.32:0/1969470551 >> 172.21.15.32:0/1969470551 conn(0x55dfc2b64200 msgr2=0x55dfc2c0c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.088+0000 7f4d0197bb80 1 -- 172.21.15.32:0/1969470551 shutdown_connections 2023-09-30T14:45:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.088+0000 7f4d0197bb80 1 -- 172.21.15.32:0/1969470551 wait complete. 2023-09-30T14:45:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.088+0000 7f4d0197bb80 1 -- start start 2023-09-30T14:45:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.088+0000 7f4d0197bb80 1 Processor -- start 2023-09-30T14:45:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4d0197bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf0200 0x55dfc2c00c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4d0197bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf1310 0x55dfc2c011a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4d0197bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2bff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4d0197bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc2bff7b0 con 0x55dfc2bf1cd0 2023-09-30T14:45:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4d0197bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc2bff920 con 0x55dfc2bf1310 2023-09-30T14:45:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4d0197bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfc2bffae0 con 0x55dfc2bf0200 2023-09-30T14:45:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4d0197bb80 10 client.0 _ll_register_callbacks cb 0x7f4ce000bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:45:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4cef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2bff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4ceeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf0200 0x55dfc2c00c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4cee7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf1310 0x55dfc2c011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4cee7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf1310 0x55dfc2c011a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48670/0 (socket says 172.21.15.32:48670) 2023-09-30T14:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4ceeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf0200 0x55dfc2c00c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51574/0 (socket says 172.21.15.32:51574) 2023-09-30T14:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4cef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2bff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37196/0 (socket says 172.21.15.32:37196) 2023-09-30T14:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4cee7fc700 1 -- 172.21.15.32:0/4145343687 learned_addr learned my addr 172.21.15.32:0/4145343687 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4cef7fe700 1 -- 172.21.15.32:0/4145343687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf0200 msgr2=0x55dfc2c00c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4cef7fe700 1 --2- 172.21.15.32:0/4145343687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf0200 0x55dfc2c00c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4cef7fe700 1 -- 172.21.15.32:0/4145343687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf1310 msgr2=0x55dfc2c011a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4cef7fe700 1 --2- 172.21.15.32:0/4145343687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf1310 0x55dfc2c011a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4cef7fe700 1 -- 172.21.15.32:0/4145343687 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf0108b00 con 0x55dfc2bf1cd0 2023-09-30T14:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.089+0000 7f4cee7fc700 1 --2- 172.21.15.32:0/4145343687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf1310 0x55dfc2c011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.090+0000 7f4cef7fe700 1 --2- 172.21.15.32:0/4145343687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2bff270 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x55dfc2be56f0 tx=0x7f4ce800e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63694c4f9e7e6bb2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.090+0000 7f4cd2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.090+0000 7f4cd2ff5700 1 -- 172.21.15.32:0/4145343687 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce801a070 con 0x55dfc2bf1cd0 2023-09-30T14:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.090+0000 7f4cd2ff5700 1 -- 172.21.15.32:0/4145343687 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4ce800ee10 con 0x55dfc2bf1cd0 2023-09-30T14:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.090+0000 7f4d0197bb80 1 -- 172.21.15.32:0/4145343687 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ce409de60 con 0x55dfc2bf1cd0 2023-09-30T14:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.090+0000 7f4cd2ff5700 1 -- 172.21.15.32:0/4145343687 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ce801e430 con 0x55dfc2bf1cd0 2023-09-30T14:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.090+0000 7f4d0197bb80 10 client.16128 fetch_fsmap learned FSMap version 38 2023-09-30T14:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.090+0000 7f4d0197bb80 1 -- 172.21.15.32:0/4145343687 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55dfc29941f0 con 0x55dfc2bf1cd0 2023-09-30T14:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.090+0000 7f4cd2ff5700 1 -- 172.21.15.32:0/4145343687 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ce409de60 con 0x55dfc2bf1cd0 2023-09-30T14:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.090+0000 7f4cd2ff5700 1 -- 172.21.15.32:0/4145343687 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4ce80155b0 con 0x55dfc2bf1cd0 2023-09-30T14:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.090+0000 7f4d0197bb80 10 client.16128 fetch_fsmap finished waiting for FSMap version 38 2023-09-30T14:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.090+0000 7f4d0197bb80 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.090+0000 7f4d0197bb80 20 client.16128 populate_metadata read hostname 'smithi032' 2023-09-30T14:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.090+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/4145343687 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dfc2c05b40 0x55dfc2c5cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.090+0000 7f4d0197bb80 4 client.16128 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.090+0000 7f4d0197bb80 1 -- 172.21.15.32:0/4145343687 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dfc2b3bad0 con 0x55dfc2c05b40 2023-09-30T14:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.091+0000 7f4ceeffd700 1 --2- 172.21.15.32:0/4145343687 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dfc2c05b40 0x55dfc2c5cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.091+0000 7f4ceeffd700 1 --2- 172.21.15.32:0/4145343687 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dfc2c05b40 0x55dfc2c5cd20 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda330af5318b0f1 server_cookie=e289e0d989ee05e1 in_seq=0 out_seq=0 2023-09-30T14:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.091+0000 7f4cd2ff5700 10 client.16128 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.135+0000 7f4cd2ff5700 1 -- 172.21.15.32:0/4145343687 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dfc2b3bad0 con 0x55dfc2c05b40 2023-09-30T14:45:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.135+0000 7f4cd2ff5700 10 client.16128 handle_command_reply: tid=0 2023-09-30T14:45:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.135+0000 7f4d0197bb80 1 client.16128 shutdown 2023-09-30T14:45:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4d0197bb80 1 -- 172.21.15.32:0/4145343687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1cd0 msgr2=0x55dfc2bff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/4145343687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2bff270 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x55dfc2be56f0 tx=0x7f4ce800e7d0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4d0197bb80 1 -- 172.21.15.32:0/4145343687 shutdown_connections 2023-09-30T14:45:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/4145343687 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dfc2c05b40 0x55dfc2c5cd20 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/4145343687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dfc2bf1cd0 0x55dfc2bff270 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4cd2ff5700 0 client.16128 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/4145343687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dfc2bf0200 0x55dfc2c00c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4d0197bb80 1 --2- 172.21.15.32:0/4145343687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfc2bf1310 0x55dfc2c011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4d0197bb80 1 -- 172.21.15.32:0/4145343687 >> 172.21.15.32:0/4145343687 conn(0x55dfc2b64200 msgr2=0x55dfc2c0c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4d0197bb80 1 -- 172.21.15.32:0/4145343687 shutdown_connections 2023-09-30T14:45:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4d0197bb80 1 -- 172.21.15.32:0/4145343687 wait complete. 2023-09-30T14:45:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4d0197bb80 20 client.16128 trim_cache size 0 max 16384 2023-09-30T14:45:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:45:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:45:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:45:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:45:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4cf5793700 1 -- 172.21.15.32:0/3574476223 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4cd80a0fa0 msgr2=0x7f4cd80a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4cf5793700 1 --2- 172.21.15.32:0/3574476223 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4cd80a0fa0 0x7f4cd80a3460 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009990 tx=0x7f4ce001a040 comp rx=0 tx=0).stop 2023-09-30T14:45:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4cf5793700 1 -- 172.21.15.32:0/3574476223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf0100f60 msgr2=0x7f4cf0199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4cf5793700 1 --2- 172.21.15.32:0/3574476223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf0100f60 0x7f4cf0199410 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400e9f0 tx=0x7f4ce400ed00 comp rx=0 tx=0).stop 2023-09-30T14:45:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4cf5793700 1 -- 172.21.15.32:0/3574476223 shutdown_connections 2023-09-30T14:45:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.136+0000 7f4cf5793700 1 --2- 172.21.15.32:0/3574476223 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4cd80a0fa0 0x7f4cd80a3460 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.137+0000 7f4cf5793700 1 --2- 172.21.15.32:0/3574476223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf0101900 0x7f4cf019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.137+0000 7f4cf5793700 1 --2- 172.21.15.32:0/3574476223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cf00ffd70 0x7f4cf0198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.137+0000 7f4cf5793700 1 --2- 172.21.15.32:0/3574476223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf0100f60 0x7f4cf0199410 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.137+0000 7f4cf5793700 1 -- 172.21.15.32:0/3574476223 >> 172.21.15.32:0/3574476223 conn(0x7f4cf00fb300 msgr2=0x7f4cf00fbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.137+0000 7f4cf5793700 1 -- 172.21.15.32:0/3574476223 shutdown_connections 2023-09-30T14:45:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:33.137+0000 7f4cf5793700 1 -- 172.21.15.32:0/3574476223 wait complete. 2023-09-30T14:45:33.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 756693 10.62 MB/sec execute 247 sec latency 21.371 ms 2023-09-30T14:45:33.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:33 smithi032 ceph-mon[98592]: pgmap v505: 97 pgs: 97 active+clean; 250 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 2.8 MiB/s wr, 264 op/s 2023-09-30T14:45:34.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:33 smithi156 ceph-mon[103579]: pgmap v505: 97 pgs: 97 active+clean; 250 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 2.8 MiB/s wr, 264 op/s 2023-09-30T14:45:34.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:33 smithi148 ceph-mon[102355]: pgmap v505: 97 pgs: 97 active+clean; 250 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 2.8 MiB/s wr, 264 op/s 2023-09-30T14:45:34.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 759242 10.62 MB/sec execute 248 sec latency 13.336 ms 2023-09-30T14:45:35.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 760740 10.61 MB/sec execute 249 sec latency 56.530 ms 2023-09-30T14:45:35.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:35 smithi032 ceph-mon[98592]: pgmap v506: 97 pgs: 97 active+clean; 252 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 2.7 MiB/s wr, 265 op/s 2023-09-30T14:45:36.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:35 smithi156 ceph-mon[103579]: pgmap v506: 97 pgs: 97 active+clean; 252 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 2.7 MiB/s wr, 265 op/s 2023-09-30T14:45:36.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:35 smithi148 ceph-mon[102355]: pgmap v506: 97 pgs: 97 active+clean; 252 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 2.7 MiB/s wr, 265 op/s 2023-09-30T14:45:36.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 763407 10.63 MB/sec execute 250 sec latency 38.123 ms 2023-09-30T14:45:37.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 765049 10.62 MB/sec execute 251 sec latency 13.847 ms 2023-09-30T14:45:37.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:37 smithi032 ceph-mon[98592]: pgmap v507: 97 pgs: 97 active+clean; 253 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 2.6 MiB/s wr, 269 op/s 2023-09-30T14:45:38.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:37 smithi156 ceph-mon[103579]: pgmap v507: 97 pgs: 97 active+clean; 253 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 2.6 MiB/s wr, 269 op/s 2023-09-30T14:45:38.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:37 smithi148 ceph-mon[102355]: pgmap v507: 97 pgs: 97 active+clean; 253 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 2.6 MiB/s wr, 269 op/s 2023-09-30T14:45:38.707 INFO:tasks.workunit.client.0.smithi032.stdout: 1 767329 10.63 MB/sec execute 252 sec latency 10.601 ms 2023-09-30T14:45:39.708 INFO:tasks.workunit.client.0.smithi032.stdout: 1 769406 10.62 MB/sec execute 253 sec latency 19.781 ms 2023-09-30T14:45:39.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:39 smithi032 ceph-mon[98592]: pgmap v508: 97 pgs: 97 active+clean; 250 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.7 MiB/s wr, 300 op/s 2023-09-30T14:45:40.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:39 smithi156 ceph-mon[103579]: pgmap v508: 97 pgs: 97 active+clean; 250 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.7 MiB/s wr, 300 op/s 2023-09-30T14:45:40.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:39 smithi148 ceph-mon[102355]: pgmap v508: 97 pgs: 97 active+clean; 250 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.7 MiB/s wr, 300 op/s 2023-09-30T14:45:40.708 INFO:tasks.workunit.client.0.smithi032.stdout: 1 771051 10.62 MB/sec execute 254 sec latency 11.594 ms 2023-09-30T14:45:41.708 INFO:tasks.workunit.client.0.smithi032.stdout: 1 772895 10.61 MB/sec execute 255 sec latency 98.384 ms 2023-09-30T14:45:41.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:41 smithi032 ceph-mon[98592]: pgmap v509: 97 pgs: 97 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2023-09-30T14:45:42.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:41 smithi156 ceph-mon[103579]: pgmap v509: 97 pgs: 97 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2023-09-30T14:45:42.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:41 smithi148 ceph-mon[102355]: pgmap v509: 97 pgs: 97 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2023-09-30T14:45:42.708 INFO:tasks.workunit.client.0.smithi032.stdout: 1 774494 10.62 MB/sec execute 256 sec latency 18.288 ms 2023-09-30T14:45:43.708 INFO:tasks.workunit.client.0.smithi032.stdout: 1 776652 10.61 MB/sec execute 257 sec latency 19.963 ms 2023-09-30T14:45:43.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:43 smithi032 ceph-mon[98592]: pgmap v510: 97 pgs: 97 active+clean; 254 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 2.8 MiB/s wr, 274 op/s 2023-09-30T14:45:44.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:43 smithi156 ceph-mon[103579]: pgmap v510: 97 pgs: 97 active+clean; 254 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 2.8 MiB/s wr, 274 op/s 2023-09-30T14:45:44.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:43 smithi148 ceph-mon[102355]: pgmap v510: 97 pgs: 97 active+clean; 254 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 2.8 MiB/s wr, 274 op/s 2023-09-30T14:45:44.708 INFO:tasks.workunit.client.0.smithi032.stdout: 1 778198 10.61 MB/sec execute 258 sec latency 12.871 ms 2023-09-30T14:45:45.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 780572 10.62 MB/sec execute 259 sec latency 18.631 ms 2023-09-30T14:45:45.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:45 smithi032 ceph-mon[98592]: pgmap v511: 97 pgs: 97 active+clean; 256 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 2.9 MiB/s wr, 299 op/s 2023-09-30T14:45:46.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:45 smithi156 ceph-mon[103579]: pgmap v511: 97 pgs: 97 active+clean; 256 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 2.9 MiB/s wr, 299 op/s 2023-09-30T14:45:46.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:45 smithi148 ceph-mon[102355]: pgmap v511: 97 pgs: 97 active+clean; 256 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 2.9 MiB/s wr, 299 op/s 2023-09-30T14:45:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.250+0000 7fdb1d51b700 1 -- 172.21.15.32:0/786998886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb180ffd90 msgr2=0x7fdb181001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.250+0000 7fdb1d51b700 1 --2- 172.21.15.32:0/786998886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb180ffd90 0x7fdb181001b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdb080099c0 tx=0x7fdb08009cd0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.250+0000 7fdb1d51b700 1 -- 172.21.15.32:0/786998886 shutdown_connections 2023-09-30T14:45:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.250+0000 7fdb1d51b700 1 --2- 172.21.15.32:0/786998886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb18101920 0x7fdb181081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.250+0000 7fdb1d51b700 1 --2- 172.21.15.32:0/786998886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb180ffd90 0x7fdb181001b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.250+0000 7fdb1d51b700 1 --2- 172.21.15.32:0/786998886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb18100f80 0x7fdb181013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.250+0000 7fdb1d51b700 1 -- 172.21.15.32:0/786998886 >> 172.21.15.32:0/786998886 conn(0x7fdb180fb300 msgr2=0x7fdb180fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.250+0000 7fdb1d51b700 1 -- 172.21.15.32:0/786998886 shutdown_connections 2023-09-30T14:45:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.250+0000 7fdb1d51b700 1 -- 172.21.15.32:0/786998886 wait complete. 2023-09-30T14:45:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.251+0000 7fdb1d51b700 1 Processor -- start 2023-09-30T14:45:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.251+0000 7fdb1d51b700 1 -- start start 2023-09-30T14:45:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.251+0000 7fdb1d51b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb180ffd90 0x7fdb18102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.251+0000 7fdb1d51b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb18100f80 0x7fdb18102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.251+0000 7fdb1d51b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb18101920 0x7fdb181030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.251+0000 7fdb1d51b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181a0bf0 con 0x7fdb18100f80 2023-09-30T14:45:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.251+0000 7fdb1d51b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181a0d30 con 0x7fdb180ffd90 2023-09-30T14:45:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.251+0000 7fdb1d51b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181a0e70 con 0x7fdb18101920 2023-09-30T14:45:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.251+0000 7fdb167fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb18100f80 0x7fdb18102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.251+0000 7fdb16ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb180ffd90 0x7fdb18102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.251+0000 7fdb177fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb18101920 0x7fdb181030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.251+0000 7fdb167fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb18100f80 0x7fdb18102bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37208/0 (socket says 172.21.15.32:37208) 2023-09-30T14:45:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.251+0000 7fdb16ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb180ffd90 0x7fdb18102670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48682/0 (socket says 172.21.15.32:48682) 2023-09-30T14:45:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.251+0000 7fdb167fc700 1 -- 172.21.15.32:0/1031297001 learned_addr learned my addr 172.21.15.32:0/1031297001 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.251+0000 7fdb167fc700 1 -- 172.21.15.32:0/1031297001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb18101920 msgr2=0x7fdb181030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.251+0000 7fdb167fc700 1 --2- 172.21.15.32:0/1031297001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb18101920 0x7fdb181030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.252+0000 7fdb167fc700 1 -- 172.21.15.32:0/1031297001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb180ffd90 msgr2=0x7fdb18102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.252+0000 7fdb167fc700 1 --2- 172.21.15.32:0/1031297001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb180ffd90 0x7fdb18102670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.252+0000 7fdb167fc700 1 -- 172.21.15.32:0/1031297001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb080095f0 con 0x7fdb18100f80 2023-09-30T14:45:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.252+0000 7fdb177fe700 1 --2- 172.21.15.32:0/1031297001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb18101920 0x7fdb181030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:45:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.252+0000 7fdb167fc700 1 --2- 172.21.15.32:0/1031297001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb18100f80 0x7fdb18102bb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00e9a0 tx=0x7fdb0c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.252+0000 7fdafffff700 1 -- 172.21.15.32:0/1031297001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c009790 con 0x7fdb18100f80 2023-09-30T14:45:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.252+0000 7fdb1d51b700 1 -- 172.21.15.32:0/1031297001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb181a1100 con 0x7fdb18100f80 2023-09-30T14:45:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.252+0000 7fdafffff700 1 -- 172.21.15.32:0/1031297001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb0c010430 con 0x7fdb18100f80 2023-09-30T14:45:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.252+0000 7fdafffff700 1 -- 172.21.15.32:0/1031297001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c018680 con 0x7fdb18100f80 2023-09-30T14:45:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.252+0000 7fdb1d51b700 1 -- 172.21.15.32:0/1031297001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb181a1660 con 0x7fdb18100f80 2023-09-30T14:45:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.254+0000 7fdafffff700 1 -- 172.21.15.32:0/1031297001 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fdb0c0105a0 con 0x7fdb18100f80 2023-09-30T14:45:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.255+0000 7fdafffff700 1 --2- 172.21.15.32:0/1031297001 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdb000a0fa0 0x7fdb000a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.255+0000 7fdb16ffd700 1 --2- 172.21.15.32:0/1031297001 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdb000a0fa0 0x7fdb000a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.255+0000 7fdafffff700 1 -- 172.21.15.32:0/1031297001 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdb0c014070 con 0x7fdb18100f80 2023-09-30T14:45:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.255+0000 7fdb16ffd700 1 --2- 172.21.15.32:0/1031297001 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdb000a0fa0 0x7fdb000a3460 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdb18103f80 tx=0x7fdb0801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb29703b80 1 Processor -- start 2023-09-30T14:45:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb29703b80 1 -- start start 2023-09-30T14:45:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb29703b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba3200 0x55b6c3ba3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb29703b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4310 0x55b6c3ba4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb29703b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3bab560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb29703b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6c3b0b400 con 0x55b6c3ba4cd0 2023-09-30T14:45:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb29703b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6c3afcbc0 con 0x55b6c3ba4310 2023-09-30T14:45:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb29703b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6c3afa4e0 con 0x55b6c3ba3200 2023-09-30T14:45:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3bab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb16ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba3200 0x55b6c3ba3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb167fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4310 0x55b6c3ba4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb16ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba3200 0x55b6c3ba3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51596/0 (socket says 172.21.15.32:51596) 2023-09-30T14:45:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb16ffd700 1 -- 172.21.15.32:0/3175115913 learned_addr learned my addr 172.21.15.32:0/3175115913 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb167fc700 1 -- 172.21.15.32:0/3175115913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba3200 msgr2=0x55b6c3ba3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb167fc700 1 --2- 172.21.15.32:0/3175115913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba3200 0x55b6c3ba3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb167fc700 1 -- 172.21.15.32:0/3175115913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba4cd0 msgr2=0x55b6c3bab560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb167fc700 1 --2- 172.21.15.32:0/3175115913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3bab560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb167fc700 1 -- 172.21.15.32:0/3175115913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6c3b58870 con 0x55b6c3ba4310 2023-09-30T14:45:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.256+0000 7fdb177fe700 1 --2- 172.21.15.32:0/3175115913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3bab560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:45:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.257+0000 7fdb167fc700 1 --2- 172.21.15.32:0/3175115913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4310 0x55b6c3ba4790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00e970 tx=0x7fdb0c0997d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=331a0e9b9b02832a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.257+0000 7fdafd7fa700 1 -- 172.21.15.32:0/3175115913 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c099d80 con 0x55b6c3ba4310 2023-09-30T14:45:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.257+0000 7fdafd7fa700 1 -- 172.21.15.32:0/3175115913 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb0c09ded0 con 0x55b6c3ba4310 2023-09-30T14:45:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.257+0000 7fdafd7fa700 1 -- 172.21.15.32:0/3175115913 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c0b2210 con 0x55b6c3ba4310 2023-09-30T14:45:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.257+0000 7fdb29703b80 1 -- 172.21.15.32:0/3175115913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4310 msgr2=0x55b6c3ba4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.257+0000 7fdb29703b80 1 --2- 172.21.15.32:0/3175115913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4310 0x55b6c3ba4790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00e970 tx=0x7fdb0c0997d0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.257+0000 7fdb29703b80 1 -- 172.21.15.32:0/3175115913 shutdown_connections 2023-09-30T14:45:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.257+0000 7fdb29703b80 1 --2- 172.21.15.32:0/3175115913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3bab560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.257+0000 7fdb29703b80 1 --2- 172.21.15.32:0/3175115913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba3200 0x55b6c3ba3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.257+0000 7fdb29703b80 1 --2- 172.21.15.32:0/3175115913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4310 0x55b6c3ba4790 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.257+0000 7fdb29703b80 1 -- 172.21.15.32:0/3175115913 >> 172.21.15.32:0/3175115913 conn(0x55b6c3b17200 msgr2=0x55b6c3b9def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.257+0000 7fdb29703b80 1 -- 172.21.15.32:0/3175115913 shutdown_connections 2023-09-30T14:45:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.258+0000 7fdb29703b80 1 -- 172.21.15.32:0/3175115913 wait complete. 2023-09-30T14:45:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.258+0000 7fdb29703b80 1 -- start start 2023-09-30T14:45:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.258+0000 7fdb29703b80 1 Processor -- start 2023-09-30T14:45:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.258+0000 7fdb29703b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba3200 0x55b6c3bacf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.258+0000 7fdb29703b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4310 0x55b6c3bad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.258+0000 7fdb29703b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3bb44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.258+0000 7fdb29703b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6c3af5dd0 con 0x55b6c3ba3200 2023-09-30T14:45:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.258+0000 7fdb29703b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6c3a926b0 con 0x55b6c3ba4310 2023-09-30T14:45:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.258+0000 7fdb29703b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6c3a92350 con 0x55b6c3ba4cd0 2023-09-30T14:45:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.258+0000 7fdb29703b80 10 client.0 _ll_register_callbacks cb 0x55b6c38ef7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:45:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.258+0000 7fdb16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba3200 0x55b6c3bacf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.258+0000 7fdb16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba3200 0x55b6c3bacf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37220/0 (socket says 172.21.15.32:37220) 2023-09-30T14:45:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.258+0000 7fdb16ffd700 1 -- 172.21.15.32:0/915946455 learned_addr learned my addr 172.21.15.32:0/915946455 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.258+0000 7fdb167fc700 1 --2- 172.21.15.32:0/915946455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4310 0x55b6c3bad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.258+0000 7fdb177fe700 1 --2- 172.21.15.32:0/915946455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3bb44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.259+0000 7fdb16ffd700 1 -- 172.21.15.32:0/915946455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba4cd0 msgr2=0x55b6c3bb44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.259+0000 7fdb16ffd700 1 --2- 172.21.15.32:0/915946455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3bb44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.259+0000 7fdb16ffd700 1 -- 172.21.15.32:0/915946455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4310 msgr2=0x55b6c3bad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.259+0000 7fdb16ffd700 1 --2- 172.21.15.32:0/915946455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4310 0x55b6c3bad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.259+0000 7fdb16ffd700 1 -- 172.21.15.32:0/915946455 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb10003a60 con 0x55b6c3ba3200 2023-09-30T14:45:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.259+0000 7fdb167fc700 1 --2- 172.21.15.32:0/915946455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4310 0x55b6c3bad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:45:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.259+0000 7fdb16ffd700 1 --2- 172.21.15.32:0/915946455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba3200 0x55b6c3bacf20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdb080098e0 tx=0x7fdb0801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33aa975f3ac65867 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.259+0000 7fdafaff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:45:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.259+0000 7fdafaff5700 1 -- 172.21.15.32:0/915946455 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08034040 con 0x55b6c3ba3200 2023-09-30T14:45:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.259+0000 7fdafaff5700 1 -- 172.21.15.32:0/915946455 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdb0802bd90 con 0x55b6c3ba3200 2023-09-30T14:45:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.259+0000 7fdb29703b80 1 -- 172.21.15.32:0/915946455 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6c3b96090 con 0x55b6c3ba3200 2023-09-30T14:45:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.259+0000 7fdafaff5700 1 -- 172.21.15.32:0/915946455 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb080046f0 con 0x55b6c3ba3200 2023-09-30T14:45:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.259+0000 7fdb29703b80 10 client.16140 fetch_fsmap learned FSMap version 38 2023-09-30T14:45:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.259+0000 7fdb29703b80 1 -- 172.21.15.32:0/915946455 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7fdb181a1660 con 0x55b6c3ba3200 2023-09-30T14:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.259+0000 7fdafaff5700 1 -- 172.21.15.32:0/915946455 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b6c3b96090 con 0x55b6c3ba3200 2023-09-30T14:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.259+0000 7fdafaff5700 1 -- 172.21.15.32:0/915946455 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdb0802d4a0 con 0x55b6c3ba3200 2023-09-30T14:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.260+0000 7fdb29703b80 10 client.16140 fetch_fsmap finished waiting for FSMap version 38 2023-09-30T14:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.260+0000 7fdb29703b80 10 client.16140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.260+0000 7fdb29703b80 20 client.16140 populate_metadata read hostname 'smithi032' 2023-09-30T14:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.260+0000 7fdb29703b80 1 --2- 172.21.15.32:0/915946455 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b6c3bb8cf0 0x55b6c3bbb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.260+0000 7fdb29703b80 4 client.16140 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.260+0000 7fdb29703b80 1 -- 172.21.15.32:0/915946455 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6c3aeedb0 con 0x55b6c3bb8cf0 2023-09-30T14:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.260+0000 7fdb167fc700 1 --2- 172.21.15.32:0/915946455 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b6c3bb8cf0 0x55b6c3bbb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.260+0000 7fdafaff5700 10 client.16140 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.260+0000 7fdb167fc700 1 --2- 172.21.15.32:0/915946455 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b6c3bb8cf0 0x55b6c3bbb110 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da81f041a79ab78 server_cookie=348aecb6c74acb18 in_seq=0 out_seq=0 2023-09-30T14:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.261+0000 7fdafaff5700 1 -- 172.21.15.32:0/915946455 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b6c3aeedb0 con 0x55b6c3bb8cf0 2023-09-30T14:45:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.261+0000 7fdafaff5700 10 client.16140 handle_command_reply: tid=0 2023-09-30T14:45:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.261+0000 7fdb29703b80 1 client.16140 shutdown 2023-09-30T14:45:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.261+0000 7fdb29703b80 1 -- 172.21.15.32:0/915946455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba3200 msgr2=0x55b6c3bacf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.261+0000 7fdb29703b80 1 --2- 172.21.15.32:0/915946455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba3200 0x55b6c3bacf20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdb080098e0 tx=0x7fdb0801f000 comp rx=0 tx=0).stop 2023-09-30T14:45:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.261+0000 7fdb29703b80 1 -- 172.21.15.32:0/915946455 shutdown_connections 2023-09-30T14:45:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.261+0000 7fdb29703b80 1 --2- 172.21.15.32:0/915946455 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b6c3bb8cf0 0x55b6c3bbb110 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.261+0000 7fdb29703b80 1 --2- 172.21.15.32:0/915946455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3bb44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.261+0000 7fdafaff5700 0 client.16140 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.261+0000 7fdb29703b80 1 --2- 172.21.15.32:0/915946455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba3200 0x55b6c3bacf20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.261+0000 7fdb29703b80 1 --2- 172.21.15.32:0/915946455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4310 0x55b6c3bad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.261+0000 7fdb29703b80 1 -- 172.21.15.32:0/915946455 >> 172.21.15.32:0/915946455 conn(0x55b6c3b17200 msgr2=0x55b6c3b9d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.261+0000 7fdb29703b80 1 -- 172.21.15.32:0/915946455 shutdown_connections 2023-09-30T14:45:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.261+0000 7fdb29703b80 1 -- 172.21.15.32:0/915946455 wait complete. 2023-09-30T14:45:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.261+0000 7fdb29703b80 20 client.16140 trim_cache size 0 max 16384 2023-09-30T14:45:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb29703b80 1 Processor -- start 2023-09-30T14:45:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb29703b80 1 -- start start 2023-09-30T14:45:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb29703b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba3200 0x55b6c3b9dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb29703b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba4310 0x55b6c3bb4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb29703b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3babc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb29703b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6c39768c0 con 0x55b6c3ba4310 2023-09-30T14:45:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb29703b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6c3b95bd0 con 0x55b6c3ba4cd0 2023-09-30T14:45:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb29703b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6c3b9e400 con 0x55b6c3ba3200 2023-09-30T14:45:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb167fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba4310 0x55b6c3bb4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb167fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba4310 0x55b6c3bb4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37230/0 (socket says 172.21.15.32:37230) 2023-09-30T14:45:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb167fc700 1 -- 172.21.15.32:0/4148970533 learned_addr learned my addr 172.21.15.32:0/4148970533 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb16ffd700 1 --2- 172.21.15.32:0/4148970533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba3200 0x55b6c3b9dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb177fe700 1 --2- 172.21.15.32:0/4148970533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3babc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb167fc700 1 -- 172.21.15.32:0/4148970533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba3200 msgr2=0x55b6c3b9dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb167fc700 1 --2- 172.21.15.32:0/4148970533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba3200 0x55b6c3b9dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb167fc700 1 -- 172.21.15.32:0/4148970533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4cd0 msgr2=0x55b6c3babc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb167fc700 1 --2- 172.21.15.32:0/4148970533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3babc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.276+0000 7fdb167fc700 1 -- 172.21.15.32:0/4148970533 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6c3920d80 con 0x55b6c3ba4310 2023-09-30T14:45:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.277+0000 7fdb167fc700 1 --2- 172.21.15.32:0/4148970533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba4310 0x55b6c3bb4460 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0967f0 tx=0x7fdb0c099c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6c90ceb17e32c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.277+0000 7fdafdffb700 1 -- 172.21.15.32:0/4148970533 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c098120 con 0x55b6c3ba4310 2023-09-30T14:45:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.277+0000 7fdafdffb700 1 -- 172.21.15.32:0/4148970533 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb0c010880 con 0x55b6c3ba4310 2023-09-30T14:45:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.277+0000 7fdafdffb700 1 -- 172.21.15.32:0/4148970533 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c0a1ee0 con 0x55b6c3ba4310 2023-09-30T14:45:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.277+0000 7fdb29703b80 1 -- 172.21.15.32:0/4148970533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba4310 msgr2=0x55b6c3bb4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.277+0000 7fdb29703b80 1 --2- 172.21.15.32:0/4148970533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba4310 0x55b6c3bb4460 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0967f0 tx=0x7fdb0c099c80 comp rx=0 tx=0).stop 2023-09-30T14:45:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.277+0000 7fdb29703b80 1 -- 172.21.15.32:0/4148970533 shutdown_connections 2023-09-30T14:45:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.277+0000 7fdb29703b80 1 --2- 172.21.15.32:0/4148970533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3babc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.277+0000 7fdb29703b80 1 --2- 172.21.15.32:0/4148970533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba3200 0x55b6c3b9dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.277+0000 7fdb29703b80 1 --2- 172.21.15.32:0/4148970533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba4310 0x55b6c3bb4460 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.277+0000 7fdb29703b80 1 -- 172.21.15.32:0/4148970533 >> 172.21.15.32:0/4148970533 conn(0x55b6c3b17200 msgr2=0x55b6c3b9d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.277+0000 7fdb29703b80 1 -- 172.21.15.32:0/4148970533 shutdown_connections 2023-09-30T14:45:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.277+0000 7fdb29703b80 1 -- 172.21.15.32:0/4148970533 wait complete. 2023-09-30T14:45:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb29703b80 1 -- start start 2023-09-30T14:45:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb29703b80 1 Processor -- start 2023-09-30T14:45:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb29703b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba3200 0x55b6c3bb1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb29703b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba4310 0x55b6c3bb2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb29703b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3bb3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb29703b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6c3bb3960 con 0x55b6c3ba3200 2023-09-30T14:45:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb29703b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6c3bb3ad0 con 0x55b6c3ba4cd0 2023-09-30T14:45:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb29703b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6c3bb3c90 con 0x55b6c3ba4310 2023-09-30T14:45:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb29703b80 10 client.0 _ll_register_callbacks cb 0x55b6c38ef7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:45:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba3200 0x55b6c3bb1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba3200 0x55b6c3bb1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37234/0 (socket says 172.21.15.32:37234) 2023-09-30T14:45:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb16ffd700 1 -- 172.21.15.32:0/211200234 learned_addr learned my addr 172.21.15.32:0/211200234 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb167fc700 1 --2- 172.21.15.32:0/211200234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba4310 0x55b6c3bb2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb177fe700 1 --2- 172.21.15.32:0/211200234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3bb3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb16ffd700 1 -- 172.21.15.32:0/211200234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba4310 msgr2=0x55b6c3bb2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb16ffd700 1 --2- 172.21.15.32:0/211200234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba4310 0x55b6c3bb2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb16ffd700 1 -- 172.21.15.32:0/211200234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4cd0 msgr2=0x55b6c3bb3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb16ffd700 1 --2- 172.21.15.32:0/211200234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3bb3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb16ffd700 1 -- 172.21.15.32:0/211200234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6c3b58870 con 0x55b6c3ba3200 2023-09-30T14:45:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.278+0000 7fdb177fe700 1 --2- 172.21.15.32:0/211200234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3bb3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:45:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.279+0000 7fdb16ffd700 1 --2- 172.21.15.32:0/211200234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba3200 0x55b6c3bb1ed0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdb0802d830 tx=0x7fdb0802dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddece889e17d4a05 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.279+0000 7fdafaff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:45:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.279+0000 7fdafaff5700 1 -- 172.21.15.32:0/211200234 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08034070 con 0x55b6c3ba3200 2023-09-30T14:45:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.279+0000 7fdafaff5700 1 -- 172.21.15.32:0/211200234 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdb08004670 con 0x55b6c3ba3200 2023-09-30T14:45:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.279+0000 7fdb29703b80 1 -- 172.21.15.32:0/211200234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb0c09ded0 con 0x55b6c3ba3200 2023-09-30T14:45:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.279+0000 7fdafaff5700 1 -- 172.21.15.32:0/211200234 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb0802bda0 con 0x55b6c3ba3200 2023-09-30T14:45:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.279+0000 7fdb29703b80 10 client.16146 fetch_fsmap learned FSMap version 38 2023-09-30T14:45:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.279+0000 7fdb29703b80 1 -- 172.21.15.32:0/211200234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55b6c3909c00 con 0x55b6c3ba3200 2023-09-30T14:45:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.279+0000 7fdafaff5700 1 -- 172.21.15.32:0/211200234 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb0c09ded0 con 0x55b6c3ba3200 2023-09-30T14:45:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.279+0000 7fdafaff5700 1 -- 172.21.15.32:0/211200234 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdb08056070 con 0x55b6c3ba3200 2023-09-30T14:45:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.279+0000 7fdb29703b80 10 client.16146 fetch_fsmap finished waiting for FSMap version 38 2023-09-30T14:45:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.279+0000 7fdb29703b80 10 client.16146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:45:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.279+0000 7fdb29703b80 20 client.16146 populate_metadata read hostname 'smithi032' 2023-09-30T14:45:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.279+0000 7fdb29703b80 1 --2- 172.21.15.32:0/211200234 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b6c3c0d200 0x55b6c3c0f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.279+0000 7fdb29703b80 4 client.16146 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:45:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.279+0000 7fdb29703b80 1 -- 172.21.15.32:0/211200234 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b6c3aeead0 con 0x55b6c3c0d200 2023-09-30T14:45:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.280+0000 7fdb167fc700 1 --2- 172.21.15.32:0/211200234 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b6c3c0d200 0x55b6c3c0f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.280+0000 7fdafaff5700 10 client.16146 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.280+0000 7fdb167fc700 1 --2- 172.21.15.32:0/211200234 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b6c3c0d200 0x55b6c3c0f5f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f0daef0da95648 server_cookie=4009df121001f97d in_seq=0 out_seq=0 2023-09-30T14:45:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.339+0000 7fdafaff5700 1 -- 172.21.15.32:0/211200234 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b6c3aeead0 con 0x55b6c3c0d200 2023-09-30T14:45:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.339+0000 7fdafaff5700 10 client.16146 handle_command_reply: tid=0 2023-09-30T14:45:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.339+0000 7fdb29703b80 1 client.16146 shutdown 2023-09-30T14:45:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.339+0000 7fdb29703b80 1 -- 172.21.15.32:0/211200234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba3200 msgr2=0x55b6c3bb1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.339+0000 7fdb29703b80 1 --2- 172.21.15.32:0/211200234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba3200 0x55b6c3bb1ed0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdb0802d830 tx=0x7fdb0802dd80 comp rx=0 tx=0).stop 2023-09-30T14:45:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.339+0000 7fdb29703b80 1 -- 172.21.15.32:0/211200234 shutdown_connections 2023-09-30T14:45:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.339+0000 7fdb29703b80 1 --2- 172.21.15.32:0/211200234 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b6c3c0d200 0x55b6c3c0f5f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.339+0000 7fdb29703b80 1 --2- 172.21.15.32:0/211200234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c3ba4cd0 0x55b6c3bb3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.339+0000 7fdb29703b80 1 --2- 172.21.15.32:0/211200234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b6c3ba3200 0x55b6c3bb1ed0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.339+0000 7fdb29703b80 1 --2- 172.21.15.32:0/211200234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b6c3ba4310 0x55b6c3bb2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.339+0000 7fdafaff5700 0 client.16146 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.339+0000 7fdb29703b80 1 -- 172.21.15.32:0/211200234 >> 172.21.15.32:0/211200234 conn(0x55b6c3b17200 msgr2=0x55b6c3b9ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.339+0000 7fdb29703b80 1 -- 172.21.15.32:0/211200234 shutdown_connections 2023-09-30T14:45:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.339+0000 7fdb29703b80 1 -- 172.21.15.32:0/211200234 wait complete. 2023-09-30T14:45:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.339+0000 7fdb29703b80 20 client.16146 trim_cache size 0 max 16384 2023-09-30T14:45:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:45:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:45:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:45:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:45:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.340+0000 7fdb1d51b700 1 -- 172.21.15.32:0/1031297001 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdb000a0fa0 msgr2=0x7fdb000a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.340+0000 7fdb1d51b700 1 --2- 172.21.15.32:0/1031297001 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdb000a0fa0 0x7fdb000a3460 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdb18103f80 tx=0x7fdb0801a040 comp rx=0 tx=0).stop 2023-09-30T14:45:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.340+0000 7fdb1d51b700 1 -- 172.21.15.32:0/1031297001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb18100f80 msgr2=0x7fdb18102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.340+0000 7fdb1d51b700 1 --2- 172.21.15.32:0/1031297001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb18100f80 0x7fdb18102bb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00e9a0 tx=0x7fdb0c00ecb0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.340+0000 7fdb1d51b700 1 -- 172.21.15.32:0/1031297001 shutdown_connections 2023-09-30T14:45:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.340+0000 7fdb1d51b700 1 --2- 172.21.15.32:0/1031297001 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fdb000a0fa0 0x7fdb000a3460 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.340+0000 7fdb1d51b700 1 --2- 172.21.15.32:0/1031297001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb18101920 0x7fdb181030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.340+0000 7fdb1d51b700 1 --2- 172.21.15.32:0/1031297001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb180ffd90 0x7fdb18102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.340+0000 7fdb1d51b700 1 --2- 172.21.15.32:0/1031297001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb18100f80 0x7fdb18102bb0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.340+0000 7fdb1d51b700 1 -- 172.21.15.32:0/1031297001 >> 172.21.15.32:0/1031297001 conn(0x7fdb180fb300 msgr2=0x7fdb180fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.340+0000 7fdb1d51b700 1 -- 172.21.15.32:0/1031297001 shutdown_connections 2023-09-30T14:45:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:46.340+0000 7fdb1d51b700 1 -- 172.21.15.32:0/1031297001 wait complete. 2023-09-30T14:45:46.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 781690 10.60 MB/sec execute 260 sec latency 42.558 ms 2023-09-30T14:45:47.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 784032 10.60 MB/sec execute 261 sec latency 43.360 ms 2023-09-30T14:45:47.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:47 smithi032 ceph-mon[98592]: pgmap v512: 97 pgs: 97 active+clean; 257 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 3.0 MiB/s wr, 290 op/s 2023-09-30T14:45:48.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:47 smithi156 ceph-mon[103579]: pgmap v512: 97 pgs: 97 active+clean; 257 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 3.0 MiB/s wr, 290 op/s 2023-09-30T14:45:48.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:47 smithi148 ceph-mon[102355]: pgmap v512: 97 pgs: 97 active+clean; 257 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 3.0 MiB/s wr, 290 op/s 2023-09-30T14:45:48.708 INFO:tasks.workunit.client.0.smithi032.stdout: 1 785605 10.59 MB/sec execute 262 sec latency 13.068 ms 2023-09-30T14:45:49.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 788467 10.62 MB/sec execute 263 sec latency 10.500 ms 2023-09-30T14:45:49.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:49 smithi032 ceph-mon[98592]: pgmap v513: 97 pgs: 97 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 3.2 MiB/s wr, 285 op/s 2023-09-30T14:45:50.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:49 smithi156 ceph-mon[103579]: pgmap v513: 97 pgs: 97 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 3.2 MiB/s wr, 285 op/s 2023-09-30T14:45:50.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:49 smithi148 ceph-mon[102355]: pgmap v513: 97 pgs: 97 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 3.2 MiB/s wr, 285 op/s 2023-09-30T14:45:50.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 790665 10.61 MB/sec execute 264 sec latency 12.715 ms 2023-09-30T14:45:51.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 792278 10.62 MB/sec execute 265 sec latency 53.756 ms 2023-09-30T14:45:52.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:51 smithi156 ceph-mon[103579]: pgmap v514: 97 pgs: 97 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 2.9 MiB/s wr, 258 op/s 2023-09-30T14:45:52.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:51 smithi148 ceph-mon[102355]: pgmap v514: 97 pgs: 97 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 2.9 MiB/s wr, 258 op/s 2023-09-30T14:45:52.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:51 smithi032 ceph-mon[98592]: pgmap v514: 97 pgs: 97 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 2.9 MiB/s wr, 258 op/s 2023-09-30T14:45:52.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 794762 10.62 MB/sec execute 266 sec latency 46.294 ms 2023-09-30T14:45:53.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 796478 10.62 MB/sec execute 267 sec latency 14.714 ms 2023-09-30T14:45:54.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:53 smithi156 ceph-mon[103579]: pgmap v515: 97 pgs: 97 active+clean; 259 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 2.9 MiB/s wr, 277 op/s 2023-09-30T14:45:54.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:53 smithi148 ceph-mon[102355]: pgmap v515: 97 pgs: 97 active+clean; 259 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 2.9 MiB/s wr, 277 op/s 2023-09-30T14:45:54.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:53 smithi032 ceph-mon[98592]: pgmap v515: 97 pgs: 97 active+clean; 259 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 2.9 MiB/s wr, 277 op/s 2023-09-30T14:45:54.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 799150 10.64 MB/sec execute 268 sec latency 11.498 ms 2023-09-30T14:45:55.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 801482 10.63 MB/sec execute 269 sec latency 13.641 ms 2023-09-30T14:45:56.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:55 smithi156 ceph-mon[103579]: pgmap v516: 97 pgs: 97 active+clean; 260 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 2.7 MiB/s wr, 268 op/s 2023-09-30T14:45:56.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:55 smithi148 ceph-mon[102355]: pgmap v516: 97 pgs: 97 active+clean; 260 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 2.7 MiB/s wr, 268 op/s 2023-09-30T14:45:56.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:55 smithi032 ceph-mon[98592]: pgmap v516: 97 pgs: 97 active+clean; 260 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 2.7 MiB/s wr, 268 op/s 2023-09-30T14:45:56.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 803202 10.63 MB/sec execute 270 sec latency 43.634 ms 2023-09-30T14:45:57.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 805951 10.66 MB/sec execute 271 sec latency 39.473 ms 2023-09-30T14:45:58.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:57 smithi156 ceph-mon[103579]: pgmap v517: 97 pgs: 97 active+clean; 264 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 3.1 MiB/s wr, 309 op/s 2023-09-30T14:45:58.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:57 smithi148 ceph-mon[102355]: pgmap v517: 97 pgs: 97 active+clean; 264 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 3.1 MiB/s wr, 309 op/s 2023-09-30T14:45:58.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:57 smithi032 ceph-mon[98592]: pgmap v517: 97 pgs: 97 active+clean; 264 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 3.1 MiB/s wr, 309 op/s 2023-09-30T14:45:58.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 807649 10.64 MB/sec execute 272 sec latency 17.340 ms 2023-09-30T14:45:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.453+0000 7f1cea9d2700 1 -- 172.21.15.32:0/3513984636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4100f50 msgr2=0x7f1ce41013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.453+0000 7f1cea9d2700 1 --2- 172.21.15.32:0/3513984636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4100f50 0x7f1ce41013b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80099c0 tx=0x7f1cd8009cd0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.453+0000 7f1cea9d2700 1 -- 172.21.15.32:0/3513984636 shutdown_connections 2023-09-30T14:45:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.453+0000 7f1cea9d2700 1 --2- 172.21.15.32:0/3513984636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ce41018f0 0x7f1ce41081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.453+0000 7f1cea9d2700 1 --2- 172.21.15.32:0/3513984636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ce40ffd60 0x7f1ce4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.453+0000 7f1cea9d2700 1 --2- 172.21.15.32:0/3513984636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4100f50 0x7f1ce41013b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.453+0000 7f1cea9d2700 1 -- 172.21.15.32:0/3513984636 >> 172.21.15.32:0/3513984636 conn(0x7f1ce40fb2b0 msgr2=0x7f1ce40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.453+0000 7f1cea9d2700 1 -- 172.21.15.32:0/3513984636 shutdown_connections 2023-09-30T14:45:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.453+0000 7f1cea9d2700 1 -- 172.21.15.32:0/3513984636 wait complete. 2023-09-30T14:45:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.454+0000 7f1cea9d2700 1 Processor -- start 2023-09-30T14:45:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.454+0000 7f1cea9d2700 1 -- start start 2023-09-30T14:45:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.454+0000 7f1cea9d2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ce40ffd60 0x7f1ce4196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.454+0000 7f1cea9d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4100f50 0x7f1ce4197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.454+0000 7f1cea9d2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ce41018f0 0x7f1ce4197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.454+0000 7f1cea9d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce4198630 con 0x7f1ce4100f50 2023-09-30T14:45:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.454+0000 7f1cea9d2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce41987a0 con 0x7f1ce40ffd60 2023-09-30T14:45:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.454+0000 7f1cea9d2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce4198960 con 0x7f1ce41018f0 2023-09-30T14:45:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1ce37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4100f50 0x7f1ce4197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1ce3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ce40ffd60 0x7f1ce4196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1ce37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4100f50 0x7f1ce4197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37250/0 (socket says 172.21.15.32:37250) 2023-09-30T14:45:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1ce37fe700 1 -- 172.21.15.32:0/951521832 learned_addr learned my addr 172.21.15.32:0/951521832 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1ce8f6f700 1 --2- 172.21.15.32:0/951521832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ce41018f0 0x7f1ce4197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1ce3fff700 1 -- 172.21.15.32:0/951521832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ce41018f0 msgr2=0x7f1ce4197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1ce3fff700 1 --2- 172.21.15.32:0/951521832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ce41018f0 0x7f1ce4197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1ce3fff700 1 -- 172.21.15.32:0/951521832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4100f50 msgr2=0x7f1ce4197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1ce3fff700 1 --2- 172.21.15.32:0/951521832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4100f50 0x7f1ce4197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1ce3fff700 1 -- 172.21.15.32:0/951521832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd80095f0 con 0x7f1ce40ffd60 2023-09-30T14:45:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1ce8f6f700 1 --2- 172.21.15.32:0/951521832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ce41018f0 0x7f1ce4197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:45:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1ce37fe700 1 --2- 172.21.15.32:0/951521832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4100f50 0x7f1ce4197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:45:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1ce3fff700 1 --2- 172.21.15.32:0/951521832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ce40ffd60 0x7f1ce4196c10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400e9c0 tx=0x7f1cd400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1ce17fa700 1 -- 172.21.15.32:0/951521832 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4004220 con 0x7f1ce40ffd60 2023-09-30T14:45:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1cea9d2700 1 -- 172.21.15.32:0/951521832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce4198c40 con 0x7f1ce40ffd60 2023-09-30T14:45:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1ce17fa700 1 -- 172.21.15.32:0/951521832 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cd4004380 con 0x7f1ce40ffd60 2023-09-30T14:45:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1ce17fa700 1 -- 172.21.15.32:0/951521832 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4003be0 con 0x7f1ce40ffd60 2023-09-30T14:45:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.455+0000 7f1cea9d2700 1 -- 172.21.15.32:0/951521832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce419f400 con 0x7f1ce40ffd60 2023-09-30T14:45:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.458+0000 7f1ce17fa700 1 -- 172.21.15.32:0/951521832 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f1cd4003d40 con 0x7f1ce40ffd60 2023-09-30T14:45:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.458+0000 7f1ce17fa700 1 --2- 172.21.15.32:0/951521832 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ccc0a0fa0 0x7f1ccc0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.459+0000 7f1ce37fe700 1 --2- 172.21.15.32:0/951521832 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ccc0a0fa0 0x7f1ccc0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.459+0000 7f1ce17fa700 1 -- 172.21.15.32:0/951521832 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1cd4014070 con 0x7f1ce40ffd60 2023-09-30T14:45:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.459+0000 7f1ce37fe700 1 --2- 172.21.15.32:0/951521832 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ccc0a0fa0 0x7f1ccc0a3460 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8004800 tx=0x7f1cd801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.459+0000 7f1cf6bbab80 1 Processor -- start 2023-09-30T14:45:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.459+0000 7f1cf6bbab80 1 -- start start 2023-09-30T14:45:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.459+0000 7f1cf6bbab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f44610 0x55adf6f44a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.459+0000 7f1cf6bbab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f45720 0x55adf6f45ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.459+0000 7f1cf6bbab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f460e0 0x55adf6f4c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.459+0000 7f1cf6bbab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf6ea8400 con 0x55adf6f45720 2023-09-30T14:45:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.459+0000 7f1cf6bbab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf6e99bc0 con 0x55adf6f44610 2023-09-30T14:45:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.459+0000 7f1cf6bbab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf6e974e0 con 0x55adf6f460e0 2023-09-30T14:45:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.460+0000 7f1ce37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f45720 0x55adf6f45ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.460+0000 7f1ce3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f44610 0x55adf6f44a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.460+0000 7f1ce37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f45720 0x55adf6f45ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37258/0 (socket says 172.21.15.32:37258) 2023-09-30T14:45:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.460+0000 7f1ce3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f44610 0x55adf6f44a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48732/0 (socket says 172.21.15.32:48732) 2023-09-30T14:45:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.460+0000 7f1ce3fff700 1 -- 172.21.15.32:0/1205866542 learned_addr learned my addr 172.21.15.32:0/1205866542 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.460+0000 7f1ce8f6f700 1 --2- 172.21.15.32:0/1205866542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f460e0 0x55adf6f4c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.460+0000 7f1ce8f6f700 1 -- 172.21.15.32:0/1205866542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f44610 msgr2=0x55adf6f44a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.460+0000 7f1ce8f6f700 1 --2- 172.21.15.32:0/1205866542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f44610 0x55adf6f44a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.460+0000 7f1ce8f6f700 1 -- 172.21.15.32:0/1205866542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f45720 msgr2=0x55adf6f45ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.460+0000 7f1ce8f6f700 1 --2- 172.21.15.32:0/1205866542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f45720 0x55adf6f45ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.460+0000 7f1ce8f6f700 1 -- 172.21.15.32:0/1205866542 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adf6ef5870 con 0x55adf6f460e0 2023-09-30T14:45:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.460+0000 7f1ce3fff700 1 --2- 172.21.15.32:0/1205866542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f44610 0x55adf6f44a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:45:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.460+0000 7f1ce8f6f700 1 --2- 172.21.15.32:0/1205866542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f460e0 0x55adf6f4c970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc003db0 tx=0x7f1cdc009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a590194098a83a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.460+0000 7f1cca7fc700 1 -- 172.21.15.32:0/1205866542 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc004bb0 con 0x55adf6f460e0 2023-09-30T14:45:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.460+0000 7f1cca7fc700 1 -- 172.21.15.32:0/1205866542 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cdc00f040 con 0x55adf6f460e0 2023-09-30T14:45:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.460+0000 7f1cca7fc700 1 -- 172.21.15.32:0/1205866542 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc007c20 con 0x55adf6f460e0 2023-09-30T14:45:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.461+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1205866542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f460e0 msgr2=0x55adf6f4c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.461+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/1205866542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f460e0 0x55adf6f4c970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc003db0 tx=0x7f1cdc009920 comp rx=0 tx=0).stop 2023-09-30T14:45:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.461+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1205866542 shutdown_connections 2023-09-30T14:45:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.461+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/1205866542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f460e0 0x55adf6f4c970 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.461+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/1205866542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f44610 0x55adf6f44a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.461+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/1205866542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f45720 0x55adf6f45ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.461+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1205866542 >> 172.21.15.32:0/1205866542 conn(0x55adf6eb4200 msgr2=0x55adf6f3aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.461+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1205866542 shutdown_connections 2023-09-30T14:45:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.461+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1205866542 wait complete. 2023-09-30T14:45:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.461+0000 7f1cf6bbab80 1 -- start start 2023-09-30T14:45:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1cf6bbab80 1 Processor -- start 2023-09-30T14:45:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1cf6bbab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f44610 0x55adf6f4e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1cf6bbab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 0x55adf6f4e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1cf6bbab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f460e0 0x55adf6f55890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1cf6bbab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf6e92dd0 con 0x55adf6f460e0 2023-09-30T14:45:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1cf6bbab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf6e2f6b0 con 0x55adf6f45720 2023-09-30T14:45:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1cf6bbab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf6e2f350 con 0x55adf6f44610 2023-09-30T14:45:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1cf6bbab80 10 client.0 _ll_register_callbacks cb 0x55adf6c8c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:45:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1ce8f6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f460e0 0x55adf6f55890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1ce3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f44610 0x55adf6f4e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1ce37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 0x55adf6f4e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1ce8f6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f460e0 0x55adf6f55890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37266/0 (socket says 172.21.15.32:37266) 2023-09-30T14:45:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1ce8f6f700 1 -- 172.21.15.32:0/1557579784 learned_addr learned my addr 172.21.15.32:0/1557579784 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1ce8f6f700 1 -- 172.21.15.32:0/1557579784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f44610 msgr2=0x55adf6f4e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1ce8f6f700 1 --2- 172.21.15.32:0/1557579784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f44610 0x55adf6f4e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1ce8f6f700 1 -- 172.21.15.32:0/1557579784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 msgr2=0x55adf6f4e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1ce8f6f700 1 --2- 172.21.15.32:0/1557579784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 0x55adf6f4e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.462+0000 7f1ce8f6f700 1 -- 172.21.15.32:0/1557579784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce419f400 con 0x55adf6f460e0 2023-09-30T14:45:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.463+0000 7f1ce8f6f700 1 --2- 172.21.15.32:0/1557579784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f460e0 0x55adf6f55890 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc007730 tx=0x7f1cdc007760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ae6cc1c8162f16 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.463+0000 7f1cc37fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:45:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.463+0000 7f1cc37fe700 1 -- 172.21.15.32:0/1557579784 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc0090d0 con 0x55adf6f460e0 2023-09-30T14:45:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.463+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1557579784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55adf6f33090 con 0x55adf6f460e0 2023-09-30T14:45:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.463+0000 7f1cc37fe700 1 -- 172.21.15.32:0/1557579784 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1cdc007c40 con 0x55adf6f460e0 2023-09-30T14:45:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.463+0000 7f1cc37fe700 1 -- 172.21.15.32:0/1557579784 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cdc007e00 con 0x55adf6f460e0 2023-09-30T14:45:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.463+0000 7f1cf6bbab80 10 client.16158 fetch_fsmap learned FSMap version 38 2023-09-30T14:45:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.463+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1557579784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f1cdc003a60 con 0x55adf6f460e0 2023-09-30T14:45:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.463+0000 7f1cc37fe700 1 -- 172.21.15.32:0/1557579784 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55adf6f33090 con 0x55adf6f460e0 2023-09-30T14:45:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.463+0000 7f1cc37fe700 1 -- 172.21.15.32:0/1557579784 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1cdc030070 con 0x55adf6f460e0 2023-09-30T14:45:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.463+0000 7f1cf6bbab80 10 client.16158 fetch_fsmap finished waiting for FSMap version 38 2023-09-30T14:45:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.463+0000 7f1cf6bbab80 10 client.16158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:45:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.463+0000 7f1cf6bbab80 20 client.16158 populate_metadata read hostname 'smithi032' 2023-09-30T14:45:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.463+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/1557579784 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55adf6f5a080 0x55adf6f5c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.463+0000 7f1cf6bbab80 4 client.16158 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:45:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.463+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1557579784 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55adf6e8bdb0 con 0x55adf6f5a080 2023-09-30T14:45:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.464+0000 7f1ce3fff700 1 --2- 172.21.15.32:0/1557579784 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55adf6f5a080 0x55adf6f5c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.464+0000 7f1cc37fe700 10 client.16158 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.464+0000 7f1ce3fff700 1 --2- 172.21.15.32:0/1557579784 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55adf6f5a080 0x55adf6f5c4a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=497f186cc83de6d9 server_cookie=11eed3d96571dd6b in_seq=0 out_seq=0 2023-09-30T14:45:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.465+0000 7f1cc37fe700 1 -- 172.21.15.32:0/1557579784 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55adf6e8bdb0 con 0x55adf6f5a080 2023-09-30T14:45:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.465+0000 7f1cc37fe700 10 client.16158 handle_command_reply: tid=0 2023-09-30T14:45:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.465+0000 7f1cf6bbab80 1 client.16158 shutdown 2023-09-30T14:45:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.465+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1557579784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f460e0 msgr2=0x55adf6f55890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.465+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/1557579784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f460e0 0x55adf6f55890 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc007730 tx=0x7f1cdc007760 comp rx=0 tx=0).stop 2023-09-30T14:45:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.465+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1557579784 shutdown_connections 2023-09-30T14:45:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.465+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/1557579784 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55adf6f5a080 0x55adf6f5c4a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.465+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/1557579784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f460e0 0x55adf6f55890 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.465+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/1557579784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f44610 0x55adf6f4e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.465+0000 7f1cc37fe700 0 client.16158 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.465+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/1557579784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 0x55adf6f4e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.465+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1557579784 >> 172.21.15.32:0/1557579784 conn(0x55adf6eb4200 msgr2=0x55adf6f3a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.465+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1557579784 shutdown_connections 2023-09-30T14:45:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.465+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1557579784 wait complete. 2023-09-30T14:45:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.465+0000 7f1cf6bbab80 20 client.16158 trim_cache size 0 max 16384 2023-09-30T14:45:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1cf6bbab80 1 Processor -- start 2023-09-30T14:45:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1cf6bbab80 1 -- start start 2023-09-30T14:45:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1cf6bbab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f44610 0x55adf6f4d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1cf6bbab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 0x55adf6f55850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1cf6bbab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f460e0 0x55adf6f3bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1cf6bbab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf6d138c0 con 0x55adf6f460e0 2023-09-30T14:45:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1cf6bbab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf6f4c2b0 con 0x55adf6f45720 2023-09-30T14:45:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1cf6bbab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf6f49b70 con 0x55adf6f44610 2023-09-30T14:45:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1ce8f6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f460e0 0x55adf6f3bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1ce37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 0x55adf6f55850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1ce8f6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f460e0 0x55adf6f3bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37274/0 (socket says 172.21.15.32:37274) 2023-09-30T14:45:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1ce37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 0x55adf6f55850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48748/0 (socket says 172.21.15.32:48748) 2023-09-30T14:45:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1ce8f6f700 1 -- 172.21.15.32:0/2576358530 learned_addr learned my addr 172.21.15.32:0/2576358530 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1ce3fff700 1 --2- 172.21.15.32:0/2576358530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f44610 0x55adf6f4d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1ce8f6f700 1 -- 172.21.15.32:0/2576358530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f44610 msgr2=0x55adf6f4d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1ce8f6f700 1 --2- 172.21.15.32:0/2576358530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f44610 0x55adf6f4d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1ce8f6f700 1 -- 172.21.15.32:0/2576358530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 msgr2=0x55adf6f55850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1ce8f6f700 1 --2- 172.21.15.32:0/2576358530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 0x55adf6f55850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1ce8f6f700 1 -- 172.21.15.32:0/2576358530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adf6cbdd80 con 0x55adf6f460e0 2023-09-30T14:45:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1ce3fff700 1 --2- 172.21.15.32:0/2576358530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f44610 0x55adf6f4d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:45:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1ce8f6f700 1 --2- 172.21.15.32:0/2576358530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f460e0 0x55adf6f3bbf0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009ec0 tx=0x7f1cdc0099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e332bcdd23536a11 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1ccaffd700 1 -- 172.21.15.32:0/2576358530 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc007880 con 0x55adf6f460e0 2023-09-30T14:45:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1ccaffd700 1 -- 172.21.15.32:0/2576358530 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cdc02d070 con 0x55adf6f460e0 2023-09-30T14:45:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.481+0000 7f1ccaffd700 1 -- 172.21.15.32:0/2576358530 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc00e750 con 0x55adf6f460e0 2023-09-30T14:45:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/2576358530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f460e0 msgr2=0x55adf6f3bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/2576358530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f460e0 0x55adf6f3bbf0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009ec0 tx=0x7f1cdc0099a0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/2576358530 shutdown_connections 2023-09-30T14:45:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/2576358530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f460e0 0x55adf6f3bbf0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/2576358530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f44610 0x55adf6f4d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/2576358530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 0x55adf6f55850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/2576358530 >> 172.21.15.32:0/2576358530 conn(0x55adf6eb4200 msgr2=0x55adf6f3a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/2576358530 shutdown_connections 2023-09-30T14:45:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/2576358530 wait complete. 2023-09-30T14:45:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 -- start start 2023-09-30T14:45:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 Processor -- start 2023-09-30T14:45:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f44610 0x55adf6f53080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 0x55adf6f535c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f460e0 0x55adf6f545d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf6f54b10 con 0x55adf6f44610 2023-09-30T14:45:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf6f54c80 con 0x55adf6f45720 2023-09-30T14:45:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf6f54e40 con 0x55adf6f460e0 2023-09-30T14:45:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.482+0000 7f1cf6bbab80 10 client.0 _ll_register_callbacks cb 0x7f1cd800bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:45:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1ce3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f44610 0x55adf6f53080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1ce3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f44610 0x55adf6f53080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37276/0 (socket says 172.21.15.32:37276) 2023-09-30T14:45:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1ce3fff700 1 -- 172.21.15.32:0/1849702677 learned_addr learned my addr 172.21.15.32:0/1849702677 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:45:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1ce8f6f700 1 --2- 172.21.15.32:0/1849702677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f460e0 0x55adf6f545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1ce37fe700 1 --2- 172.21.15.32:0/1849702677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 0x55adf6f535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1ce3fff700 1 -- 172.21.15.32:0/1849702677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f460e0 msgr2=0x55adf6f545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1ce3fff700 1 --2- 172.21.15.32:0/1849702677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f460e0 0x55adf6f545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1ce3fff700 1 -- 172.21.15.32:0/1849702677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 msgr2=0x55adf6f535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:45:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1ce3fff700 1 --2- 172.21.15.32:0/1849702677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 0x55adf6f535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1ce3fff700 1 -- 172.21.15.32:0/1849702677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc003a60 con 0x55adf6f44610 2023-09-30T14:45:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1ce37fe700 1 --2- 172.21.15.32:0/1849702677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 0x55adf6f535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:45:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1ce3fff700 1 --2- 172.21.15.32:0/1849702677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f44610 0x55adf6f53080 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4004800 tx=0x7f1cd4096260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec7ec3bb3af6bc2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:45:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1cc37fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:45:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1cc37fe700 1 -- 172.21.15.32:0/1849702677 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40a2200 con 0x55adf6f44610 2023-09-30T14:45:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1cc37fe700 1 -- 172.21.15.32:0/1849702677 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1cd40b6070 con 0x55adf6f44610 2023-09-30T14:45:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1849702677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1cdc00f040 con 0x55adf6f44610 2023-09-30T14:45:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1cc37fe700 1 -- 172.21.15.32:0/1849702677 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cd4098210 con 0x55adf6f44610 2023-09-30T14:45:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1cf6bbab80 10 client.16164 fetch_fsmap learned FSMap version 38 2023-09-30T14:45:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1849702677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55adf6ca6c00 con 0x55adf6f44610 2023-09-30T14:45:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.483+0000 7f1cc37fe700 1 -- 172.21.15.32:0/1849702677 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cd407c070 con 0x55adf6f44610 2023-09-30T14:45:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.484+0000 7f1cc37fe700 1 -- 172.21.15.32:0/1849702677 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1cd4099360 con 0x55adf6f44610 2023-09-30T14:45:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.484+0000 7f1cf6bbab80 10 client.16164 fetch_fsmap finished waiting for FSMap version 38 2023-09-30T14:45:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.484+0000 7f1cf6bbab80 10 client.16164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:45:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.484+0000 7f1cf6bbab80 20 client.16164 populate_metadata read hostname 'smithi032' 2023-09-30T14:45:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.484+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/1849702677 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55adf6fae8c0 0x55adf6fb0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:45:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.484+0000 7f1cf6bbab80 4 client.16164 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:45:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.484+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1849702677 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55adf6e8bad0 con 0x55adf6fae8c0 2023-09-30T14:45:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.484+0000 7f1ce37fe700 1 --2- 172.21.15.32:0/1849702677 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55adf6fae8c0 0x55adf6fb0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:45:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.484+0000 7f1cc37fe700 10 client.16164 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.484+0000 7f1ce37fe700 1 --2- 172.21.15.32:0/1849702677 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55adf6fae8c0 0x55adf6fb0ce0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91a9c4677a961d60 server_cookie=53004260200bab7c in_seq=0 out_seq=0 2023-09-30T14:45:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cc37fe700 1 -- 172.21.15.32:0/1849702677 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55adf6e8bad0 con 0x55adf6fae8c0 2023-09-30T14:45:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cc37fe700 10 client.16164 handle_command_reply: tid=0 2023-09-30T14:45:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cf6bbab80 1 client.16164 shutdown 2023-09-30T14:45:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1849702677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f44610 msgr2=0x55adf6f53080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/1849702677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f44610 0x55adf6f53080 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4004800 tx=0x7f1cd4096260 comp rx=0 tx=0).stop 2023-09-30T14:45:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1849702677 shutdown_connections 2023-09-30T14:45:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/1849702677 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55adf6fae8c0 0x55adf6fb0ce0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/1849702677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55adf6f460e0 0x55adf6f545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/1849702677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55adf6f44610 0x55adf6f53080 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cf6bbab80 1 --2- 172.21.15.32:0/1849702677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55adf6f45720 0x55adf6f535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cc37fe700 0 client.16164 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:45:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1849702677 >> 172.21.15.32:0/1849702677 conn(0x55adf6eb4200 msgr2=0x55adf6f3bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1849702677 shutdown_connections 2023-09-30T14:45:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cf6bbab80 1 -- 172.21.15.32:0/1849702677 wait complete. 2023-09-30T14:45:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cf6bbab80 20 client.16164 trim_cache size 0 max 16384 2023-09-30T14:45:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:45:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:45:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:45:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:45:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cea9d2700 1 -- 172.21.15.32:0/951521832 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ccc0a0fa0 msgr2=0x7f1ccc0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cea9d2700 1 --2- 172.21.15.32:0/951521832 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ccc0a0fa0 0x7f1ccc0a3460 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8004800 tx=0x7f1cd801a040 comp rx=0 tx=0).stop 2023-09-30T14:45:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cea9d2700 1 -- 172.21.15.32:0/951521832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ce40ffd60 msgr2=0x7f1ce4196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:45:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cea9d2700 1 --2- 172.21.15.32:0/951521832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ce40ffd60 0x7f1ce4196c10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400e9c0 tx=0x7f1cd400ecd0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cea9d2700 1 -- 172.21.15.32:0/951521832 shutdown_connections 2023-09-30T14:45:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cea9d2700 1 --2- 172.21.15.32:0/951521832 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ccc0a0fa0 0x7f1ccc0a3460 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cea9d2700 1 --2- 172.21.15.32:0/951521832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ce41018f0 0x7f1ce4197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cea9d2700 1 --2- 172.21.15.32:0/951521832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ce40ffd60 0x7f1ce4196c10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cea9d2700 1 --2- 172.21.15.32:0/951521832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ce4100f50 0x7f1ce4197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:45:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cea9d2700 1 -- 172.21.15.32:0/951521832 >> 172.21.15.32:0/951521832 conn(0x7f1ce40fb2b0 msgr2=0x7f1ce4068280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:45:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cea9d2700 1 -- 172.21.15.32:0/951521832 shutdown_connections 2023-09-30T14:45:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:45:59.531+0000 7f1cea9d2700 1 -- 172.21.15.32:0/951521832 wait complete. 2023-09-30T14:45:59.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 809901 10.66 MB/sec execute 273 sec latency 20.298 ms 2023-09-30T14:46:00.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:45:59 smithi156 ceph-mon[103579]: pgmap v518: 97 pgs: 97 active+clean; 262 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 2.7 MiB/s wr, 276 op/s 2023-09-30T14:46:00.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:45:59 smithi148 ceph-mon[102355]: pgmap v518: 97 pgs: 97 active+clean; 262 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 2.7 MiB/s wr, 276 op/s 2023-09-30T14:46:00.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:45:59 smithi032 ceph-mon[98592]: pgmap v518: 97 pgs: 97 active+clean; 262 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 2.7 MiB/s wr, 276 op/s 2023-09-30T14:46:00.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 812432 10.66 MB/sec execute 274 sec latency 18.363 ms 2023-09-30T14:46:01.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 813947 10.65 MB/sec execute 275 sec latency 30.425 ms 2023-09-30T14:46:02.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:01 smithi156 ceph-mon[103579]: pgmap v519: 97 pgs: 97 active+clean; 263 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 2.4 MiB/s wr, 251 op/s 2023-09-30T14:46:02.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:01 smithi148 ceph-mon[102355]: pgmap v519: 97 pgs: 97 active+clean; 263 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 2.4 MiB/s wr, 251 op/s 2023-09-30T14:46:02.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:01 smithi032 ceph-mon[98592]: pgmap v519: 97 pgs: 97 active+clean; 263 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 2.4 MiB/s wr, 251 op/s 2023-09-30T14:46:02.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 816678 10.67 MB/sec execute 276 sec latency 19.605 ms 2023-09-30T14:46:03.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 818003 10.66 MB/sec execute 277 sec latency 61.861 ms 2023-09-30T14:46:04.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:03 smithi156 ceph-mon[103579]: pgmap v520: 97 pgs: 97 active+clean; 267 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 2.8 MiB/s wr, 299 op/s 2023-09-30T14:46:04.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:03 smithi148 ceph-mon[102355]: pgmap v520: 97 pgs: 97 active+clean; 267 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 2.8 MiB/s wr, 299 op/s 2023-09-30T14:46:04.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:03 smithi032 ceph-mon[98592]: pgmap v520: 97 pgs: 97 active+clean; 267 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 2.8 MiB/s wr, 299 op/s 2023-09-30T14:46:04.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 820476 10.67 MB/sec execute 278 sec latency 11.176 ms 2023-09-30T14:46:05.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 822614 10.66 MB/sec execute 279 sec latency 48.532 ms 2023-09-30T14:46:06.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:05 smithi156 ceph-mon[103579]: pgmap v521: 97 pgs: 97 active+clean; 267 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.5 MiB/s wr, 265 op/s 2023-09-30T14:46:06.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:05 smithi148 ceph-mon[102355]: pgmap v521: 97 pgs: 97 active+clean; 267 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.5 MiB/s wr, 265 op/s 2023-09-30T14:46:06.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:05 smithi032 ceph-mon[98592]: pgmap v521: 97 pgs: 97 active+clean; 267 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.5 MiB/s wr, 265 op/s 2023-09-30T14:46:06.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 824279 10.66 MB/sec execute 280 sec latency 40.612 ms 2023-09-30T14:46:07.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 826850 10.67 MB/sec execute 281 sec latency 40.495 ms 2023-09-30T14:46:08.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:07 smithi156 ceph-mon[103579]: pgmap v522: 97 pgs: 97 active+clean; 270 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 3.0 MiB/s wr, 339 op/s 2023-09-30T14:46:08.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:07 smithi148 ceph-mon[102355]: pgmap v522: 97 pgs: 97 active+clean; 270 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 3.0 MiB/s wr, 339 op/s 2023-09-30T14:46:08.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:07 smithi032 ceph-mon[98592]: pgmap v522: 97 pgs: 97 active+clean; 270 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 3.0 MiB/s wr, 339 op/s 2023-09-30T14:46:08.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 828217 10.66 MB/sec execute 282 sec latency 45.330 ms 2023-09-30T14:46:09.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 831001 10.68 MB/sec execute 283 sec latency 18.753 ms 2023-09-30T14:46:10.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:09 smithi156 ceph-mon[103579]: pgmap v523: 97 pgs: 97 active+clean; 270 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.5 MiB/s wr, 262 op/s 2023-09-30T14:46:10.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:09 smithi148 ceph-mon[102355]: pgmap v523: 97 pgs: 97 active+clean; 270 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.5 MiB/s wr, 262 op/s 2023-09-30T14:46:10.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:09 smithi032 ceph-mon[98592]: pgmap v523: 97 pgs: 97 active+clean; 270 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.5 MiB/s wr, 262 op/s 2023-09-30T14:46:10.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 832601 10.67 MB/sec execute 284 sec latency 55.806 ms 2023-09-30T14:46:11.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 834741 10.68 MB/sec execute 285 sec latency 40.480 ms 2023-09-30T14:46:12.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:11 smithi156 ceph-mon[103579]: pgmap v524: 97 pgs: 97 active+clean; 272 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 2.8 MiB/s wr, 287 op/s 2023-09-30T14:46:12.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:11 smithi148 ceph-mon[102355]: pgmap v524: 97 pgs: 97 active+clean; 272 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 2.8 MiB/s wr, 287 op/s 2023-09-30T14:46:12.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:11 smithi032 ceph-mon[98592]: pgmap v524: 97 pgs: 97 active+clean; 272 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 2.8 MiB/s wr, 287 op/s 2023-09-30T14:46:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.654+0000 7fe0caf68700 1 -- 172.21.15.32:0/1528659326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0c4100f50 msgr2=0x7fe0c41013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.654+0000 7fe0caf68700 1 --2- 172.21.15.32:0/1528659326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0c4100f50 0x7fe0c41013b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40099c0 tx=0x7fe0b4009cd0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.654+0000 7fe0caf68700 1 -- 172.21.15.32:0/1528659326 shutdown_connections 2023-09-30T14:46:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.654+0000 7fe0caf68700 1 --2- 172.21.15.32:0/1528659326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0c41018f0 0x7fe0c41081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.654+0000 7fe0caf68700 1 --2- 172.21.15.32:0/1528659326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0c40ffd60 0x7fe0c4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.654+0000 7fe0caf68700 1 --2- 172.21.15.32:0/1528659326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0c4100f50 0x7fe0c41013b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.654+0000 7fe0caf68700 1 -- 172.21.15.32:0/1528659326 >> 172.21.15.32:0/1528659326 conn(0x7fe0c40fb2b0 msgr2=0x7fe0c40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.654+0000 7fe0caf68700 1 -- 172.21.15.32:0/1528659326 shutdown_connections 2023-09-30T14:46:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.654+0000 7fe0caf68700 1 -- 172.21.15.32:0/1528659326 wait complete. 2023-09-30T14:46:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0caf68700 1 Processor -- start 2023-09-30T14:46:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0caf68700 1 -- start start 2023-09-30T14:46:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0caf68700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0c40ffd60 0x7fe0c4198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0caf68700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0c4100f50 0x7fe0c4199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0caf68700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0c41018f0 0x7fe0c419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0caf68700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c419a8e0 con 0x7fe0c41018f0 2023-09-30T14:46:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0caf68700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c419aa50 con 0x7fe0c40ffd60 2023-09-30T14:46:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0caf68700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c419ac10 con 0x7fe0c4100f50 2023-09-30T14:46:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0c9505700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0c41018f0 0x7fe0c419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0c8d04700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0c40ffd60 0x7fe0c4198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0c8d04700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0c40ffd60 0x7fe0c4198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48766/0 (socket says 172.21.15.32:48766) 2023-09-30T14:46:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0c9505700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0c41018f0 0x7fe0c419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37292/0 (socket says 172.21.15.32:37292) 2023-09-30T14:46:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0bbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0c4100f50 0x7fe0c4199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0c8d04700 1 -- 172.21.15.32:0/2288916975 learned_addr learned my addr 172.21.15.32:0/2288916975 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0c9505700 1 -- 172.21.15.32:0/2288916975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0c4100f50 msgr2=0x7fe0c4199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0c9505700 1 --2- 172.21.15.32:0/2288916975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0c4100f50 0x7fe0c4199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0c9505700 1 -- 172.21.15.32:0/2288916975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0c40ffd60 msgr2=0x7fe0c4198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0c9505700 1 --2- 172.21.15.32:0/2288916975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0c40ffd60 0x7fe0c4198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0c9505700 1 -- 172.21.15.32:0/2288916975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b40095f0 con 0x7fe0c41018f0 2023-09-30T14:46:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.655+0000 7fe0c8d04700 1 --2- 172.21.15.32:0/2288916975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0c40ffd60 0x7fe0c4198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:46:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.656+0000 7fe0c9505700 1 --2- 172.21.15.32:0/2288916975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0c41018f0 0x7fe0c419a1d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00efc0 tx=0x7fe0bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.656+0000 7fe0b9ffb700 1 -- 172.21.15.32:0/2288916975 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc018070 con 0x7fe0c41018f0 2023-09-30T14:46:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.656+0000 7fe0b9ffb700 1 -- 172.21.15.32:0/2288916975 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0bc00f040 con 0x7fe0c41018f0 2023-09-30T14:46:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.656+0000 7fe0caf68700 1 -- 172.21.15.32:0/2288916975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c419ae90 con 0x7fe0c41018f0 2023-09-30T14:46:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.656+0000 7fe0b9ffb700 1 -- 172.21.15.32:0/2288916975 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc00cd40 con 0x7fe0c41018f0 2023-09-30T14:46:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.656+0000 7fe0caf68700 1 -- 172.21.15.32:0/2288916975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c41a1600 con 0x7fe0c41018f0 2023-09-30T14:46:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.658+0000 7fe0b9ffb700 1 -- 172.21.15.32:0/2288916975 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fe0bc003a00 con 0x7fe0c41018f0 2023-09-30T14:46:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.659+0000 7fe0b9ffb700 1 --2- 172.21.15.32:0/2288916975 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0ac0a0fa0 0x7fe0ac0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.659+0000 7fe0b9ffb700 1 -- 172.21.15.32:0/2288916975 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe0bc0cc2c0 con 0x7fe0c41018f0 2023-09-30T14:46:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.659+0000 7fe0c8d04700 1 --2- 172.21.15.32:0/2288916975 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0ac0a0fa0 0x7fe0ac0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.659+0000 7fe0c8d04700 1 --2- 172.21.15.32:0/2288916975 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0ac0a0fa0 0x7fe0ac0a3460 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4198c40 tx=0x7fe0c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0d7150b80 1 Processor -- start 2023-09-30T14:46:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0d7150b80 1 -- start start 2023-09-30T14:46:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0d7150b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808d200 0x55882808d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0d7150b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808e310 0x55882808e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0d7150b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808ecd0 0x558828095560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0d7150b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558827ff5400 con 0x55882808e310 2023-09-30T14:46:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0d7150b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558827fe6bc0 con 0x55882808d200 2023-09-30T14:46:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0d7150b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558827fe44e0 con 0x55882808ecd0 2023-09-30T14:46:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0bbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808e310 0x55882808e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0bbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808e310 0x55882808e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37300/0 (socket says 172.21.15.32:37300) 2023-09-30T14:46:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0bbfff700 1 -- 172.21.15.32:0/1289705339 learned_addr learned my addr 172.21.15.32:0/1289705339 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0c8d04700 1 --2- 172.21.15.32:0/1289705339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808d200 0x55882808d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0bbfff700 1 -- 172.21.15.32:0/1289705339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808ecd0 msgr2=0x558828095560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0bbfff700 1 --2- 172.21.15.32:0/1289705339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808ecd0 0x558828095560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0bbfff700 1 -- 172.21.15.32:0/1289705339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808d200 msgr2=0x55882808d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0bbfff700 1 --2- 172.21.15.32:0/1289705339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808d200 0x55882808d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0bbfff700 1 -- 172.21.15.32:0/1289705339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558828042870 con 0x55882808e310 2023-09-30T14:46:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.660+0000 7fe0c8d04700 1 --2- 172.21.15.32:0/1289705339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808d200 0x55882808d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:46:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.661+0000 7fe0bbfff700 1 --2- 172.21.15.32:0/1289705339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808e310 0x55882808e790 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40099c0 tx=0x7fe0b4005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af80ed73b514a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.661+0000 7fe0aaffd700 1 -- 172.21.15.32:0/1289705339 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400bad0 con 0x55882808e310 2023-09-30T14:46:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.661+0000 7fe0aaffd700 1 -- 172.21.15.32:0/1289705339 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0b401e040 con 0x55882808e310 2023-09-30T14:46:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.661+0000 7fe0aaffd700 1 -- 172.21.15.32:0/1289705339 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4003b30 con 0x55882808e310 2023-09-30T14:46:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.662+0000 7fe0d7150b80 1 -- 172.21.15.32:0/1289705339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808e310 msgr2=0x55882808e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.662+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/1289705339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808e310 0x55882808e790 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40099c0 tx=0x7fe0b4005ee0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.662+0000 7fe0d7150b80 1 -- 172.21.15.32:0/1289705339 shutdown_connections 2023-09-30T14:46:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.662+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/1289705339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808ecd0 0x558828095560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.662+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/1289705339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808d200 0x55882808d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.662+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/1289705339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808e310 0x55882808e790 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.662+0000 7fe0d7150b80 1 -- 172.21.15.32:0/1289705339 >> 172.21.15.32:0/1289705339 conn(0x558828001200 msgr2=0x558828087ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.662+0000 7fe0d7150b80 1 -- 172.21.15.32:0/1289705339 shutdown_connections 2023-09-30T14:46:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.662+0000 7fe0d7150b80 1 -- 172.21.15.32:0/1289705339 wait complete. 2023-09-30T14:46:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.662+0000 7fe0d7150b80 1 -- start start 2023-09-30T14:46:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.662+0000 7fe0d7150b80 1 Processor -- start 2023-09-30T14:46:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0d7150b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808d200 0x558828096ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0d7150b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808e310 0x5588280973e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0d7150b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808ecd0 0x55882809e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0d7150b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558827fdfdd0 con 0x55882808ecd0 2023-09-30T14:46:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0d7150b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558827f7c6b0 con 0x55882808e310 2023-09-30T14:46:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0d7150b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558827f7c350 con 0x55882808d200 2023-09-30T14:46:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0d7150b80 10 client.0 _ll_register_callbacks cb 0x558827dd97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:46:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0c9505700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808ecd0 0x55882809e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0c8d04700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808d200 0x558828096ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0bbfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808e310 0x5588280973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0c8d04700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808d200 0x558828096ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51686/0 (socket says 172.21.15.32:51686) 2023-09-30T14:46:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0c8d04700 1 -- 172.21.15.32:0/1408097157 learned_addr learned my addr 172.21.15.32:0/1408097157 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0bbfff700 1 -- 172.21.15.32:0/1408097157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808d200 msgr2=0x558828096ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0bbfff700 1 --2- 172.21.15.32:0/1408097157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808d200 0x558828096ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0bbfff700 1 -- 172.21.15.32:0/1408097157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808ecd0 msgr2=0x55882809e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0bbfff700 1 --2- 172.21.15.32:0/1408097157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808ecd0 0x55882809e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0bbfff700 1 -- 172.21.15.32:0/1408097157 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c0017070 con 0x55882808e310 2023-09-30T14:46:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.663+0000 7fe0c9505700 1 --2- 172.21.15.32:0/1408097157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808ecd0 0x55882809e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:46:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.664+0000 7fe0bbfff700 1 --2- 172.21.15.32:0/1408097157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808e310 0x5588280973e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4005850 tx=0x7fe0b4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ba1814ae1e2bc8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.664+0000 7fe0a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:46:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.665+0000 7fe0a3fff700 1 -- 172.21.15.32:0/1408097157 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40054e0 con 0x55882808e310 2023-09-30T14:46:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.665+0000 7fe0a3fff700 1 -- 172.21.15.32:0/1408097157 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe0b40038b0 con 0x55882808e310 2023-09-30T14:46:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.665+0000 7fe0d7150b80 1 -- 172.21.15.32:0/1408097157 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558828080090 con 0x55882808e310 2023-09-30T14:46:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.665+0000 7fe0a3fff700 1 -- 172.21.15.32:0/1408097157 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0b4003a70 con 0x55882808e310 2023-09-30T14:46:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.665+0000 7fe0d7150b80 10 client.25865 fetch_fsmap learned FSMap version 38 2023-09-30T14:46:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.665+0000 7fe0d7150b80 1 -- 172.21.15.32:0/1408097157 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7fe0c4108af0 con 0x55882808e310 2023-09-30T14:46:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.666+0000 7fe0a3fff700 1 -- 172.21.15.32:0/1408097157 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558828080090 con 0x55882808e310 2023-09-30T14:46:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.666+0000 7fe0a3fff700 1 -- 172.21.15.32:0/1408097157 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe0b402dc50 con 0x55882808e310 2023-09-30T14:46:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.666+0000 7fe0d7150b80 10 client.25865 fetch_fsmap finished waiting for FSMap version 38 2023-09-30T14:46:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.666+0000 7fe0d7150b80 10 client.25865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:46:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.666+0000 7fe0d7150b80 20 client.25865 populate_metadata read hostname 'smithi032' 2023-09-30T14:46:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.666+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/1408097157 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5588280a28b0 0x5588280a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.666+0000 7fe0d7150b80 4 client.25865 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:46:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.666+0000 7fe0d7150b80 1 -- 172.21.15.32:0/1408097157 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558827fd8db0 con 0x5588280a28b0 2023-09-30T14:46:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.666+0000 7fe0c8d04700 1 --2- 172.21.15.32:0/1408097157 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5588280a28b0 0x5588280a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.666+0000 7fe0a3fff700 10 client.25865 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:46:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.666+0000 7fe0c8d04700 1 --2- 172.21.15.32:0/1408097157 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5588280a28b0 0x5588280a4cd0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79867763eff873ce server_cookie=ba97694a6aa85b9f in_seq=0 out_seq=0 2023-09-30T14:46:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.667+0000 7fe0a3fff700 1 -- 172.21.15.32:0/1408097157 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558827fd8db0 con 0x5588280a28b0 2023-09-30T14:46:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.667+0000 7fe0a3fff700 10 client.25865 handle_command_reply: tid=0 2023-09-30T14:46:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.667+0000 7fe0d7150b80 1 client.25865 shutdown 2023-09-30T14:46:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.667+0000 7fe0d7150b80 1 -- 172.21.15.32:0/1408097157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808e310 msgr2=0x5588280973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.667+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/1408097157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808e310 0x5588280973e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4005850 tx=0x7fe0b4000bc0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.667+0000 7fe0d7150b80 1 -- 172.21.15.32:0/1408097157 shutdown_connections 2023-09-30T14:46:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.667+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/1408097157 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5588280a28b0 0x5588280a4cd0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.667+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/1408097157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808ecd0 0x55882809e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.667+0000 7fe0a3fff700 0 client.25865 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:46:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.667+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/1408097157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808d200 0x558828096ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.667+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/1408097157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808e310 0x5588280973e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.667+0000 7fe0d7150b80 1 -- 172.21.15.32:0/1408097157 >> 172.21.15.32:0/1408097157 conn(0x558828001200 msgr2=0x558828087590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.667+0000 7fe0d7150b80 1 -- 172.21.15.32:0/1408097157 shutdown_connections 2023-09-30T14:46:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.667+0000 7fe0d7150b80 1 -- 172.21.15.32:0/1408097157 wait complete. 2023-09-30T14:46:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.668+0000 7fe0d7150b80 20 client.25865 trim_cache size 0 max 16384 2023-09-30T14:46:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.684+0000 7fe0d7150b80 1 Processor -- start 2023-09-30T14:46:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.684+0000 7fe0d7150b80 1 -- start start 2023-09-30T14:46:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0d7150b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808d200 0x55882808e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0d7150b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808e310 0x55882809e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0d7150b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808ecd0 0x558828087ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0d7150b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558827e608c0 con 0x55882808d200 2023-09-30T14:46:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0d7150b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588280917f0 con 0x55882808ecd0 2023-09-30T14:46:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0d7150b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558828088430 con 0x55882808e310 2023-09-30T14:46:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0c8d04700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808d200 0x55882808e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0c8d04700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808d200 0x55882808e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37312/0 (socket says 172.21.15.32:37312) 2023-09-30T14:46:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0c8d04700 1 -- 172.21.15.32:0/946701139 learned_addr learned my addr 172.21.15.32:0/946701139 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0bbfff700 1 --2- 172.21.15.32:0/946701139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808e310 0x55882809e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0c9505700 1 --2- 172.21.15.32:0/946701139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808ecd0 0x558828087ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0c8d04700 1 -- 172.21.15.32:0/946701139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808e310 msgr2=0x55882809e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0c8d04700 1 --2- 172.21.15.32:0/946701139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808e310 0x55882809e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0c8d04700 1 -- 172.21.15.32:0/946701139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808ecd0 msgr2=0x558828087ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0c8d04700 1 --2- 172.21.15.32:0/946701139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808ecd0 0x558828087ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0c8d04700 1 -- 172.21.15.32:0/946701139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558827e0ad80 con 0x55882808d200 2023-09-30T14:46:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.685+0000 7fe0c8d04700 1 --2- 172.21.15.32:0/946701139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808d200 0x55882808e6f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x5588280a55b0 tx=0x7fe0c0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbd7d3083d96719 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.686+0000 7fe0ab7fe700 1 -- 172.21.15.32:0/946701139 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c002e070 con 0x55882808d200 2023-09-30T14:46:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.686+0000 7fe0ab7fe700 1 -- 172.21.15.32:0/946701139 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0c0029460 con 0x55882808d200 2023-09-30T14:46:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.686+0000 7fe0ab7fe700 1 -- 172.21.15.32:0/946701139 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0024040 con 0x55882808d200 2023-09-30T14:46:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.687+0000 7fe0d7150b80 1 -- 172.21.15.32:0/946701139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808d200 msgr2=0x55882808e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.687+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/946701139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808d200 0x55882808e6f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x5588280a55b0 tx=0x7fe0c0014000 comp rx=0 tx=0).stop 2023-09-30T14:46:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.687+0000 7fe0d7150b80 1 -- 172.21.15.32:0/946701139 shutdown_connections 2023-09-30T14:46:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.687+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/946701139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808ecd0 0x558828087ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.687+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/946701139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808d200 0x55882808e6f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.687+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/946701139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808e310 0x55882809e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.687+0000 7fe0d7150b80 1 -- 172.21.15.32:0/946701139 >> 172.21.15.32:0/946701139 conn(0x558828001200 msgr2=0x5588280a9630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.687+0000 7fe0d7150b80 1 -- 172.21.15.32:0/946701139 shutdown_connections 2023-09-30T14:46:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.687+0000 7fe0d7150b80 1 -- 172.21.15.32:0/946701139 wait complete. 2023-09-30T14:46:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.687+0000 7fe0d7150b80 1 -- start start 2023-09-30T14:46:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.687+0000 7fe0d7150b80 1 Processor -- start 2023-09-30T14:46:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0d7150b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808d200 0x55882809bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0d7150b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808e310 0x55882809c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0d7150b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808ecd0 0x55882809cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0d7150b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882809d520 con 0x55882808d200 2023-09-30T14:46:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0d7150b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882809d690 con 0x55882808e310 2023-09-30T14:46:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0d7150b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882809d850 con 0x55882808ecd0 2023-09-30T14:46:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0d7150b80 10 client.0 _ll_register_callbacks cb 0x7fe0c0004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:46:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0c8d04700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808d200 0x55882809bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0bbfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808e310 0x55882809c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0c8d04700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808d200 0x55882809bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37318/0 (socket says 172.21.15.32:37318) 2023-09-30T14:46:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0c8d04700 1 -- 172.21.15.32:0/3388783799 learned_addr learned my addr 172.21.15.32:0/3388783799 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0c9505700 1 --2- 172.21.15.32:0/3388783799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808ecd0 0x55882809cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0c8d04700 1 -- 172.21.15.32:0/3388783799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808ecd0 msgr2=0x55882809cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0c8d04700 1 --2- 172.21.15.32:0/3388783799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808ecd0 0x55882809cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0c8d04700 1 -- 172.21.15.32:0/3388783799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808e310 msgr2=0x55882809c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0c8d04700 1 --2- 172.21.15.32:0/3388783799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808e310 0x55882809c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0c8d04700 1 -- 172.21.15.32:0/3388783799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c00040e0 con 0x55882808d200 2023-09-30T14:46:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0bbfff700 1 --2- 172.21.15.32:0/3388783799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808e310 0x55882809c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:46:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0c8d04700 1 --2- 172.21.15.32:0/3388783799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808d200 0x55882809bc20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0005d90 tx=0x7fe0c00053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f66f42242b1a1ee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:46:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0a3fff700 1 -- 172.21.15.32:0/3388783799 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0024070 con 0x55882808d200 2023-09-30T14:46:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0a3fff700 1 -- 172.21.15.32:0/3388783799 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe0c0003880 con 0x55882808d200 2023-09-30T14:46:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.688+0000 7fe0d7150b80 1 -- 172.21.15.32:0/3388783799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0b401e040 con 0x55882808d200 2023-09-30T14:46:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.689+0000 7fe0a3fff700 1 -- 172.21.15.32:0/3388783799 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0c002e050 con 0x55882808d200 2023-09-30T14:46:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.689+0000 7fe0d7150b80 10 client.16182 fetch_fsmap learned FSMap version 38 2023-09-30T14:46:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.689+0000 7fe0d7150b80 1 -- 172.21.15.32:0/3388783799 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x558827df3c00 con 0x55882808d200 2023-09-30T14:46:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.689+0000 7fe0a3fff700 1 -- 172.21.15.32:0/3388783799 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0b401e040 con 0x55882808d200 2023-09-30T14:46:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.689+0000 7fe0a3fff700 1 -- 172.21.15.32:0/3388783799 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe0c0033430 con 0x55882808d200 2023-09-30T14:46:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.689+0000 7fe0d7150b80 10 client.16182 fetch_fsmap finished waiting for FSMap version 38 2023-09-30T14:46:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.689+0000 7fe0d7150b80 10 client.16182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:46:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.689+0000 7fe0d7150b80 20 client.16182 populate_metadata read hostname 'smithi032' 2023-09-30T14:46:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.689+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/3388783799 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5588280f6fd0 0x5588280f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.689+0000 7fe0d7150b80 4 client.16182 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:46:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.689+0000 7fe0d7150b80 1 -- 172.21.15.32:0/3388783799 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558827fd8ad0 con 0x5588280f6fd0 2023-09-30T14:46:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.689+0000 7fe0bbfff700 1 --2- 172.21.15.32:0/3388783799 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5588280f6fd0 0x5588280f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.690+0000 7fe0a3fff700 10 client.16182 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:46:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.690+0000 7fe0bbfff700 1 --2- 172.21.15.32:0/3388783799 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5588280f6fd0 0x5588280f93c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77fa6f9f5584a5eb server_cookie=229a00c6ceea73ab in_seq=0 out_seq=0 2023-09-30T14:46:12.709 INFO:tasks.workunit.client.0.smithi032.stdout: 1 837234 10.68 MB/sec execute 286 sec latency 29.286 ms 2023-09-30T14:46:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.736+0000 7fe0a3fff700 1 -- 172.21.15.32:0/3388783799 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558827fd8ad0 con 0x5588280f6fd0 2023-09-30T14:46:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.736+0000 7fe0a3fff700 10 client.16182 handle_command_reply: tid=0 2023-09-30T14:46:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.736+0000 7fe0d7150b80 1 client.16182 shutdown 2023-09-30T14:46:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.736+0000 7fe0d7150b80 1 -- 172.21.15.32:0/3388783799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808d200 msgr2=0x55882809bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.736+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/3388783799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808d200 0x55882809bc20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0005d90 tx=0x7fe0c00053f0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.736+0000 7fe0d7150b80 1 -- 172.21.15.32:0/3388783799 shutdown_connections 2023-09-30T14:46:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.736+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/3388783799 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5588280f6fd0 0x5588280f93c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.736+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/3388783799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55882808ecd0 0x55882809cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.736+0000 7fe0a3fff700 0 client.16182 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:46:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.736+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/3388783799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55882808d200 0x55882809bc20 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.736+0000 7fe0d7150b80 1 --2- 172.21.15.32:0/3388783799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55882808e310 0x55882809c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.736+0000 7fe0d7150b80 1 -- 172.21.15.32:0/3388783799 >> 172.21.15.32:0/3388783799 conn(0x558828001200 msgr2=0x558828095ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.736+0000 7fe0d7150b80 1 -- 172.21.15.32:0/3388783799 shutdown_connections 2023-09-30T14:46:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.736+0000 7fe0d7150b80 1 -- 172.21.15.32:0/3388783799 wait complete. 2023-09-30T14:46:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.736+0000 7fe0d7150b80 20 client.16182 trim_cache size 0 max 16384 2023-09-30T14:46:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:46:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:46:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:46:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:46:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.737+0000 7fe0caf68700 1 -- 172.21.15.32:0/2288916975 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0ac0a0fa0 msgr2=0x7fe0ac0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.737+0000 7fe0caf68700 1 --2- 172.21.15.32:0/2288916975 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0ac0a0fa0 0x7fe0ac0a3460 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4198c40 tx=0x7fe0c0009260 comp rx=0 tx=0).stop 2023-09-30T14:46:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.737+0000 7fe0caf68700 1 -- 172.21.15.32:0/2288916975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0c41018f0 msgr2=0x7fe0c419a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.737+0000 7fe0caf68700 1 --2- 172.21.15.32:0/2288916975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0c41018f0 0x7fe0c419a1d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00efc0 tx=0x7fe0bc00c3a0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.737+0000 7fe0caf68700 1 -- 172.21.15.32:0/2288916975 shutdown_connections 2023-09-30T14:46:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.737+0000 7fe0caf68700 1 --2- 172.21.15.32:0/2288916975 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe0ac0a0fa0 0x7fe0ac0a3460 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.737+0000 7fe0caf68700 1 --2- 172.21.15.32:0/2288916975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0c41018f0 0x7fe0c419a1d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.737+0000 7fe0caf68700 1 --2- 172.21.15.32:0/2288916975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0c40ffd60 0x7fe0c4198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.737+0000 7fe0caf68700 1 --2- 172.21.15.32:0/2288916975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0c4100f50 0x7fe0c4199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.737+0000 7fe0caf68700 1 -- 172.21.15.32:0/2288916975 >> 172.21.15.32:0/2288916975 conn(0x7fe0c40fb2b0 msgr2=0x7fe0c4068300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.737+0000 7fe0caf68700 1 -- 172.21.15.32:0/2288916975 shutdown_connections 2023-09-30T14:46:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:12.737+0000 7fe0caf68700 1 -- 172.21.15.32:0/2288916975 wait complete. 2023-09-30T14:46:13.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 838885 10.67 MB/sec execute 287 sec latency 19.184 ms 2023-09-30T14:46:14.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:13 smithi156 ceph-mon[103579]: pgmap v525: 97 pgs: 97 active+clean; 274 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 3.3 MiB/s wr, 341 op/s 2023-09-30T14:46:14.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:13 smithi148 ceph-mon[102355]: pgmap v525: 97 pgs: 97 active+clean; 274 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 3.3 MiB/s wr, 341 op/s 2023-09-30T14:46:14.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:13 smithi032 ceph-mon[98592]: pgmap v525: 97 pgs: 97 active+clean; 274 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 3.3 MiB/s wr, 341 op/s 2023-09-30T14:46:14.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 841677 10.70 MB/sec execute 288 sec latency 10.964 ms 2023-09-30T14:46:15.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 843807 10.68 MB/sec execute 289 sec latency 18.604 ms 2023-09-30T14:46:16.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:15 smithi156 ceph-mon[103579]: pgmap v526: 97 pgs: 97 active+clean; 275 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 2.9 MiB/s wr, 265 op/s 2023-09-30T14:46:16.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:15 smithi148 ceph-mon[102355]: pgmap v526: 97 pgs: 97 active+clean; 275 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 2.9 MiB/s wr, 265 op/s 2023-09-30T14:46:16.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:15 smithi032 ceph-mon[98592]: pgmap v526: 97 pgs: 97 active+clean; 275 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 2.9 MiB/s wr, 265 op/s 2023-09-30T14:46:16.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 845489 10.69 MB/sec execute 290 sec latency 58.926 ms 2023-09-30T14:46:17.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:16 smithi156 ceph-mon[103579]: pgmap v527: 97 pgs: 97 active+clean; 272 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 3.2 MiB/s wr, 309 op/s 2023-09-30T14:46:17.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:16 smithi148 ceph-mon[102355]: pgmap v527: 97 pgs: 97 active+clean; 272 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 3.2 MiB/s wr, 309 op/s 2023-09-30T14:46:17.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:16 smithi032 ceph-mon[98592]: pgmap v527: 97 pgs: 97 active+clean; 272 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 3.2 MiB/s wr, 309 op/s 2023-09-30T14:46:17.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 847939 10.70 MB/sec execute 291 sec latency 26.556 ms 2023-09-30T14:46:18.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 849532 10.69 MB/sec execute 292 sec latency 17.700 ms 2023-09-30T14:46:19.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:19 smithi148 ceph-mon[102355]: pgmap v528: 97 pgs: 97 active+clean; 273 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2023-09-30T14:46:19.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:19 smithi032 ceph-mon[98592]: pgmap v528: 97 pgs: 97 active+clean; 273 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2023-09-30T14:46:19.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:19 smithi156 ceph-mon[103579]: pgmap v528: 97 pgs: 97 active+clean; 273 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2023-09-30T14:46:19.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 852196 10.71 MB/sec execute 293 sec latency 56.866 ms 2023-09-30T14:46:20.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 853634 10.69 MB/sec execute 294 sec latency 54.481 ms 2023-09-30T14:46:21.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:21 smithi148 ceph-mon[102355]: pgmap v529: 97 pgs: 97 active+clean; 274 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.9 MiB/s wr, 304 op/s 2023-09-30T14:46:21.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:21 smithi032 ceph-mon[98592]: pgmap v529: 97 pgs: 97 active+clean; 274 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.9 MiB/s wr, 304 op/s 2023-09-30T14:46:21.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:21 smithi156 ceph-mon[103579]: pgmap v529: 97 pgs: 97 active+clean; 274 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.9 MiB/s wr, 304 op/s 2023-09-30T14:46:21.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 855975 10.70 MB/sec execute 295 sec latency 23.100 ms 2023-09-30T14:46:22.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 858083 10.69 MB/sec execute 296 sec latency 30.480 ms 2023-09-30T14:46:23.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:23 smithi148 ceph-mon[102355]: pgmap v530: 97 pgs: 97 active+clean; 276 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 3.1 MiB/s wr, 313 op/s 2023-09-30T14:46:23.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:23 smithi032 ceph-mon[98592]: pgmap v530: 97 pgs: 97 active+clean; 276 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 3.1 MiB/s wr, 313 op/s 2023-09-30T14:46:23.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:23 smithi156 ceph-mon[103579]: pgmap v530: 97 pgs: 97 active+clean; 276 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 3.1 MiB/s wr, 313 op/s 2023-09-30T14:46:23.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 859796 10.70 MB/sec execute 297 sec latency 47.976 ms 2023-09-30T14:46:24.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 862397 10.70 MB/sec execute 298 sec latency 18.050 ms 2023-09-30T14:46:25.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:25 smithi148 ceph-mon[102355]: pgmap v531: 97 pgs: 97 active+clean; 276 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 2.4 MiB/s wr, 236 op/s 2023-09-30T14:46:25.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:25 smithi032 ceph-mon[98592]: pgmap v531: 97 pgs: 97 active+clean; 276 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 2.4 MiB/s wr, 236 op/s 2023-09-30T14:46:25.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:25 smithi156 ceph-mon[103579]: pgmap v531: 97 pgs: 97 active+clean; 276 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 2.4 MiB/s wr, 236 op/s 2023-09-30T14:46:25.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 863899 10.69 MB/sec execute 299 sec latency 33.559 ms 2023-09-30T14:46:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.853+0000 7f91cbb3c700 1 -- 172.21.15.32:0/463563150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c4101920 msgr2=0x7f91c41081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.853+0000 7f91cbb3c700 1 --2- 172.21.15.32:0/463563150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c4101920 0x7f91c41081d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00b0d0 tx=0x7f91bc00b3e0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.853+0000 7f91cbb3c700 1 -- 172.21.15.32:0/463563150 shutdown_connections 2023-09-30T14:46:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.853+0000 7f91cbb3c700 1 --2- 172.21.15.32:0/463563150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c4101920 0x7f91c41081d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.853+0000 7f91cbb3c700 1 --2- 172.21.15.32:0/463563150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c40ffd90 0x7f91c41001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.853+0000 7f91cbb3c700 1 --2- 172.21.15.32:0/463563150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91c4100f80 0x7f91c41013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.853+0000 7f91cbb3c700 1 -- 172.21.15.32:0/463563150 >> 172.21.15.32:0/463563150 conn(0x7f91c40fb320 msgr2=0x7f91c40fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.853+0000 7f91cbb3c700 1 -- 172.21.15.32:0/463563150 shutdown_connections 2023-09-30T14:46:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.853+0000 7f91cbb3c700 1 -- 172.21.15.32:0/463563150 wait complete. 2023-09-30T14:46:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.854+0000 7f91cbb3c700 1 Processor -- start 2023-09-30T14:46:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.854+0000 7f91cbb3c700 1 -- start start 2023-09-30T14:46:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.854+0000 7f91cbb3c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c40ffd90 0x7f91c4199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.854+0000 7f91cbb3c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c4100f80 0x7f91c419bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.854+0000 7f91cbb3c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91c4101920 0x7f91c419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.854+0000 7f91cbb3c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c419ea00 con 0x7f91c40ffd90 2023-09-30T14:46:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.854+0000 7f91cbb3c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c419eb40 con 0x7f91c4100f80 2023-09-30T14:46:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.854+0000 7f91cbb3c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c419ed00 con 0x7f91c4101920 2023-09-30T14:46:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.854+0000 7f91c98d8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c40ffd90 0x7f91c4199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.854+0000 7f91c98d8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c40ffd90 0x7f91c4199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37342/0 (socket says 172.21.15.32:37342) 2023-09-30T14:46:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.854+0000 7f91c98d8700 1 -- 172.21.15.32:0/1101759231 learned_addr learned my addr 172.21.15.32:0/1101759231 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.854+0000 7f91c90d7700 1 --2- 172.21.15.32:0/1101759231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c4100f80 0x7f91c419bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.854+0000 7f91ca0d9700 1 --2- 172.21.15.32:0/1101759231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91c4101920 0x7f91c419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.855+0000 7f91c98d8700 1 -- 172.21.15.32:0/1101759231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91c4101920 msgr2=0x7f91c419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.855+0000 7f91c98d8700 1 --2- 172.21.15.32:0/1101759231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91c4101920 0x7f91c419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.855+0000 7f91c98d8700 1 -- 172.21.15.32:0/1101759231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c4100f80 msgr2=0x7f91c419bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.855+0000 7f91c98d8700 1 --2- 172.21.15.32:0/1101759231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c4100f80 0x7f91c419bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.855+0000 7f91c98d8700 1 -- 172.21.15.32:0/1101759231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc009d00 con 0x7f91c40ffd90 2023-09-30T14:46:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.855+0000 7f91c98d8700 1 --2- 172.21.15.32:0/1101759231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c40ffd90 0x7f91c4199990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f91c000e970 tx=0x7f91c000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.855+0000 7f91baffd700 1 -- 172.21.15.32:0/1101759231 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0009790 con 0x7f91c40ffd90 2023-09-30T14:46:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.855+0000 7f91baffd700 1 -- 172.21.15.32:0/1101759231 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91c000c7e0 con 0x7f91c40ffd90 2023-09-30T14:46:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.855+0000 7f91cbb3c700 1 -- 172.21.15.32:0/1101759231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c419efe0 con 0x7f91c40ffd90 2023-09-30T14:46:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.855+0000 7f91baffd700 1 -- 172.21.15.32:0/1101759231 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0003c70 con 0x7f91c40ffd90 2023-09-30T14:46:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.855+0000 7f91cbb3c700 1 -- 172.21.15.32:0/1101759231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c419f400 con 0x7f91c40ffd90 2023-09-30T14:46:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.857+0000 7f91baffd700 1 -- 172.21.15.32:0/1101759231 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f91c000cc50 con 0x7f91c40ffd90 2023-09-30T14:46:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.858+0000 7f91baffd700 1 --2- 172.21.15.32:0/1101759231 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91b00a0fa0 0x7f91b00a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.858+0000 7f91c90d7700 1 --2- 172.21.15.32:0/1101759231 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91b00a0fa0 0x7f91b00a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.858+0000 7f91baffd700 1 -- 172.21.15.32:0/1101759231 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f91c0014070 con 0x7f91c40ffd90 2023-09-30T14:46:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.859+0000 7f91c90d7700 1 --2- 172.21.15.32:0/1101759231 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91b00a0fa0 0x7f91b00a3460 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f91c4069640 tx=0x7f91b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.859+0000 7f91d7d24b80 1 Processor -- start 2023-09-30T14:46:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.859+0000 7f91d7d24b80 1 -- start start 2023-09-30T14:46:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.859+0000 7f91d7d24b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a2719200 0x5635a2719620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.859+0000 7f91d7d24b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a271a310 0x5635a271a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.859+0000 7f91d7d24b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a271acd0 0x5635a2721560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91d7d24b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a2681400 con 0x5635a2719200 2023-09-30T14:46:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91d7d24b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a2672bc0 con 0x5635a271acd0 2023-09-30T14:46:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91d7d24b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a26704e0 con 0x5635a271a310 2023-09-30T14:46:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91c98d8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a2719200 0x5635a2719620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91c98d8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a2719200 0x5635a2719620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37350/0 (socket says 172.21.15.32:37350) 2023-09-30T14:46:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91c98d8700 1 -- 172.21.15.32:0/545011068 learned_addr learned my addr 172.21.15.32:0/545011068 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91c90d7700 1 --2- 172.21.15.32:0/545011068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a271a310 0x5635a271a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91ca0d9700 1 --2- 172.21.15.32:0/545011068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a271acd0 0x5635a2721560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91c98d8700 1 -- 172.21.15.32:0/545011068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a271a310 msgr2=0x5635a271a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91c98d8700 1 --2- 172.21.15.32:0/545011068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a271a310 0x5635a271a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91c98d8700 1 -- 172.21.15.32:0/545011068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a271acd0 msgr2=0x5635a2721560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91c98d8700 1 --2- 172.21.15.32:0/545011068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a271acd0 0x5635a2721560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91c98d8700 1 -- 172.21.15.32:0/545011068 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635a26ce870 con 0x5635a2719200 2023-09-30T14:46:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91c98d8700 1 --2- 172.21.15.32:0/545011068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a2719200 0x5635a2719620 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f91c00a1bc0 tx=0x7f91c000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53720fb47d0343de server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91affff700 1 -- 172.21.15.32:0/545011068 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0099d60 con 0x5635a2719200 2023-09-30T14:46:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91affff700 1 -- 172.21.15.32:0/545011068 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91c009deb0 con 0x5635a2719200 2023-09-30T14:46:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.860+0000 7f91affff700 1 -- 172.21.15.32:0/545011068 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c00b2480 con 0x5635a2719200 2023-09-30T14:46:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.861+0000 7f91d7d24b80 1 -- 172.21.15.32:0/545011068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a2719200 msgr2=0x5635a2719620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.861+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/545011068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a2719200 0x5635a2719620 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f91c00a1bc0 tx=0x7f91c000cf50 comp rx=0 tx=0).stop 2023-09-30T14:46:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.861+0000 7f91d7d24b80 1 -- 172.21.15.32:0/545011068 shutdown_connections 2023-09-30T14:46:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.861+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/545011068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a271acd0 0x5635a2721560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.861+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/545011068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a2719200 0x5635a2719620 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.861+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/545011068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a271a310 0x5635a271a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.861+0000 7f91d7d24b80 1 -- 172.21.15.32:0/545011068 >> 172.21.15.32:0/545011068 conn(0x5635a268d200 msgr2=0x5635a2713ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.861+0000 7f91d7d24b80 1 -- 172.21.15.32:0/545011068 shutdown_connections 2023-09-30T14:46:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.861+0000 7f91d7d24b80 1 -- 172.21.15.32:0/545011068 wait complete. 2023-09-30T14:46:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.862+0000 7f91d7d24b80 1 -- start start 2023-09-30T14:46:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.862+0000 7f91d7d24b80 1 Processor -- start 2023-09-30T14:46:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.862+0000 7f91d7d24b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a2719200 0x5635a2722ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.862+0000 7f91d7d24b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271a310 0x5635a2723400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.862+0000 7f91d7d24b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a271acd0 0x5635a272a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.862+0000 7f91d7d24b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a266bdd0 con 0x5635a271a310 2023-09-30T14:46:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.862+0000 7f91d7d24b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a26086b0 con 0x5635a2719200 2023-09-30T14:46:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.862+0000 7f91d7d24b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a2608350 con 0x5635a271acd0 2023-09-30T14:46:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.862+0000 7f91d7d24b80 10 client.0 _ll_register_callbacks cb 0x5635a24657a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:46:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.862+0000 7f91c90d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271a310 0x5635a2723400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.862+0000 7f91c98d8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a2719200 0x5635a2722ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.862+0000 7f91c90d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271a310 0x5635a2723400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37358/0 (socket says 172.21.15.32:37358) 2023-09-30T14:46:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.862+0000 7f91c90d7700 1 -- 172.21.15.32:0/524929851 learned_addr learned my addr 172.21.15.32:0/524929851 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.862+0000 7f91ca0d9700 1 --2- 172.21.15.32:0/524929851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a271acd0 0x5635a272a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.863+0000 7f91c90d7700 1 -- 172.21.15.32:0/524929851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a271acd0 msgr2=0x5635a272a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.863+0000 7f91c90d7700 1 --2- 172.21.15.32:0/524929851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a271acd0 0x5635a272a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.863+0000 7f91c90d7700 1 -- 172.21.15.32:0/524929851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a2719200 msgr2=0x5635a2722ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.863+0000 7f91c90d7700 1 --2- 172.21.15.32:0/524929851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a2719200 0x5635a2722ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.863+0000 7f91c90d7700 1 -- 172.21.15.32:0/524929851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c419f400 con 0x5635a271a310 2023-09-30T14:46:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.863+0000 7f91c90d7700 1 --2- 172.21.15.32:0/524929851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271a310 0x5635a2723400 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f91b40116f0 tx=0x7f91b40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e84cf7e88ce7b4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.863+0000 7f91ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:46:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.863+0000 7f91ad7fa700 1 -- 172.21.15.32:0/524929851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b4028470 con 0x5635a271a310 2023-09-30T14:46:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.863+0000 7f91ad7fa700 1 -- 172.21.15.32:0/524929851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f91b4017070 con 0x5635a271a310 2023-09-30T14:46:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.863+0000 7f91d7d24b80 1 -- 172.21.15.32:0/524929851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635a270c090 con 0x5635a271a310 2023-09-30T14:46:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.863+0000 7f91ad7fa700 1 -- 172.21.15.32:0/524929851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91b402e030 con 0x5635a271a310 2023-09-30T14:46:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.863+0000 7f91d7d24b80 10 client.16194 fetch_fsmap learned FSMap version 38 2023-09-30T14:46:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.863+0000 7f91d7d24b80 1 -- 172.21.15.32:0/524929851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5635a2496d80 con 0x5635a271a310 2023-09-30T14:46:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.863+0000 7f91ad7fa700 1 -- 172.21.15.32:0/524929851 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5635a270c090 con 0x5635a271a310 2023-09-30T14:46:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.864+0000 7f91ad7fa700 1 -- 172.21.15.32:0/524929851 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f91b401e550 con 0x5635a271a310 2023-09-30T14:46:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.864+0000 7f91d7d24b80 10 client.16194 fetch_fsmap finished waiting for FSMap version 38 2023-09-30T14:46:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.864+0000 7f91d7d24b80 10 client.16194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:46:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.864+0000 7f91d7d24b80 20 client.16194 populate_metadata read hostname 'smithi032' 2023-09-30T14:46:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.864+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/524929851 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5635a272edf0 0x5635a2731210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.864+0000 7f91d7d24b80 4 client.16194 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:46:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.864+0000 7f91d7d24b80 1 -- 172.21.15.32:0/524929851 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635a2664db0 con 0x5635a272edf0 2023-09-30T14:46:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.864+0000 7f91c98d8700 1 --2- 172.21.15.32:0/524929851 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5635a272edf0 0x5635a2731210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.864+0000 7f91ad7fa700 10 client.16194 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:46:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.864+0000 7f91c98d8700 1 --2- 172.21.15.32:0/524929851 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5635a272edf0 0x5635a2731210 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=239f74b4e4d0c183 server_cookie=85d25a93d329fdf2 in_seq=0 out_seq=0 2023-09-30T14:46:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.865+0000 7f91ad7fa700 1 -- 172.21.15.32:0/524929851 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5635a2664db0 con 0x5635a272edf0 2023-09-30T14:46:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.865+0000 7f91ad7fa700 10 client.16194 handle_command_reply: tid=0 2023-09-30T14:46:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.865+0000 7f91d7d24b80 1 client.16194 shutdown 2023-09-30T14:46:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.865+0000 7f91d7d24b80 1 -- 172.21.15.32:0/524929851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271a310 msgr2=0x5635a2723400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.865+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/524929851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271a310 0x5635a2723400 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f91b40116f0 tx=0x7f91b40047c0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.865+0000 7f91d7d24b80 1 -- 172.21.15.32:0/524929851 shutdown_connections 2023-09-30T14:46:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.865+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/524929851 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5635a272edf0 0x5635a2731210 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.865+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/524929851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a271acd0 0x5635a272a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.865+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/524929851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a2719200 0x5635a2722ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.865+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/524929851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271a310 0x5635a2723400 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.865+0000 7f91ad7fa700 0 client.16194 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:46:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.865+0000 7f91d7d24b80 1 -- 172.21.15.32:0/524929851 >> 172.21.15.32:0/524929851 conn(0x5635a268d200 msgr2=0x5635a2713590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.865+0000 7f91d7d24b80 1 -- 172.21.15.32:0/524929851 shutdown_connections 2023-09-30T14:46:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.865+0000 7f91d7d24b80 1 -- 172.21.15.32:0/524929851 wait complete. 2023-09-30T14:46:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.865+0000 7f91d7d24b80 20 client.16194 trim_cache size 0 max 16384 2023-09-30T14:46:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.882+0000 7f91d7d24b80 1 Processor -- start 2023-09-30T14:46:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.882+0000 7f91d7d24b80 1 -- start start 2023-09-30T14:46:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.882+0000 7f91d7d24b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a2719200 0x5635a2713ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.882+0000 7f91d7d24b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271a310 0x5635a272a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.882+0000 7f91d7d24b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a271acd0 0x5635a272ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.882+0000 7f91d7d24b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a24ec8c0 con 0x5635a271a310 2023-09-30T14:46:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.882+0000 7f91d7d24b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a2722a30 con 0x5635a271acd0 2023-09-30T14:46:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.882+0000 7f91d7d24b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a2713cc0 con 0x5635a2719200 2023-09-30T14:46:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.883+0000 7f91c98d8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a2719200 0x5635a2713ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.883+0000 7f91c90d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271a310 0x5635a272a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.883+0000 7f91c98d8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a2719200 0x5635a2713ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51746/0 (socket says 172.21.15.32:51746) 2023-09-30T14:46:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.883+0000 7f91c98d8700 1 -- 172.21.15.32:0/1848051315 learned_addr learned my addr 172.21.15.32:0/1848051315 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.883+0000 7f91ca0d9700 1 --2- 172.21.15.32:0/1848051315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a271acd0 0x5635a272ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.883+0000 7f91c98d8700 1 -- 172.21.15.32:0/1848051315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a271acd0 msgr2=0x5635a272ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.883+0000 7f91c98d8700 1 --2- 172.21.15.32:0/1848051315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a271acd0 0x5635a272ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.883+0000 7f91c98d8700 1 -- 172.21.15.32:0/1848051315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271a310 msgr2=0x5635a272a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.883+0000 7f91c98d8700 1 --2- 172.21.15.32:0/1848051315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271a310 0x5635a272a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.883+0000 7f91c98d8700 1 -- 172.21.15.32:0/1848051315 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635a247fc00 con 0x5635a2719200 2023-09-30T14:46:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.883+0000 7f91c98d8700 1 --2- 172.21.15.32:0/1848051315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a2719200 0x5635a2713ec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f91c00967d0 tx=0x7f91c0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc0386a8a6f13c7e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.883+0000 7f91b8d82700 1 -- 172.21.15.32:0/1848051315 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0099d60 con 0x5635a2719200 2023-09-30T14:46:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.883+0000 7f91b8d82700 1 -- 172.21.15.32:0/1848051315 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91c0098100 con 0x5635a2719200 2023-09-30T14:46:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.883+0000 7f91b8d82700 1 -- 172.21.15.32:0/1848051315 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0099fc0 con 0x5635a2719200 2023-09-30T14:46:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.884+0000 7f91d7d24b80 1 -- 172.21.15.32:0/1848051315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a2719200 msgr2=0x5635a2713ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.884+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/1848051315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a2719200 0x5635a2713ec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f91c00967d0 tx=0x7f91c0004750 comp rx=0 tx=0).stop 2023-09-30T14:46:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.884+0000 7f91d7d24b80 1 -- 172.21.15.32:0/1848051315 shutdown_connections 2023-09-30T14:46:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.884+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/1848051315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a271acd0 0x5635a272ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.884+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/1848051315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a2719200 0x5635a2713ec0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.884+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/1848051315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271a310 0x5635a272a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.884+0000 7f91d7d24b80 1 -- 172.21.15.32:0/1848051315 >> 172.21.15.32:0/1848051315 conn(0x5635a268d200 msgr2=0x5635a2713590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.884+0000 7f91d7d24b80 1 -- 172.21.15.32:0/1848051315 shutdown_connections 2023-09-30T14:46:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.884+0000 7f91d7d24b80 1 -- 172.21.15.32:0/1848051315 wait complete. 2023-09-30T14:46:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.884+0000 7f91d7d24b80 1 -- start start 2023-09-30T14:46:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.885+0000 7f91d7d24b80 1 Processor -- start 2023-09-30T14:46:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.885+0000 7f91d7d24b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a2719200 0x5635a2727ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.885+0000 7f91d7d24b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a271a310 0x5635a2728000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.885+0000 7f91d7d24b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271acd0 0x5635a2729010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.885+0000 7f91d7d24b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a2729550 con 0x5635a271acd0 2023-09-30T14:46:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.885+0000 7f91d7d24b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a27296c0 con 0x5635a271a310 2023-09-30T14:46:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.885+0000 7f91d7d24b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a2729880 con 0x5635a2719200 2023-09-30T14:46:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.885+0000 7f91d7d24b80 10 client.0 _ll_register_callbacks cb 0x5635a24657a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:46:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.885+0000 7f91ca0d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271acd0 0x5635a2729010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.885+0000 7f91ca0d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271acd0 0x5635a2729010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37374/0 (socket says 172.21.15.32:37374) 2023-09-30T14:46:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.885+0000 7f91ca0d9700 1 -- 172.21.15.32:0/3800826827 learned_addr learned my addr 172.21.15.32:0/3800826827 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.885+0000 7f91c90d7700 1 --2- 172.21.15.32:0/3800826827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a271a310 0x5635a2728000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.885+0000 7f91c98d8700 1 --2- 172.21.15.32:0/3800826827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a2719200 0x5635a2727ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.886+0000 7f91ca0d9700 1 -- 172.21.15.32:0/3800826827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a2719200 msgr2=0x5635a2727ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.886+0000 7f91ca0d9700 1 --2- 172.21.15.32:0/3800826827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a2719200 0x5635a2727ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.886+0000 7f91ca0d9700 1 -- 172.21.15.32:0/3800826827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a271a310 msgr2=0x5635a2728000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.886+0000 7f91ca0d9700 1 --2- 172.21.15.32:0/3800826827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a271a310 0x5635a2728000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.886+0000 7f91ca0d9700 1 -- 172.21.15.32:0/3800826827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c00b6040 con 0x5635a271acd0 2023-09-30T14:46:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.886+0000 7f91c98d8700 1 --2- 172.21.15.32:0/3800826827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a2719200 0x5635a2727ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:46:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.886+0000 7f91ca0d9700 1 --2- 172.21.15.32:0/3800826827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271acd0 0x5635a2729010 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00b0d0 tx=0x7f91bc007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e412820d660464a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.886+0000 7f91ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:46:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.886+0000 7f91ad7fa700 1 -- 172.21.15.32:0/3800826827 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc022070 con 0x5635a271acd0 2023-09-30T14:46:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.886+0000 7f91ad7fa700 1 -- 172.21.15.32:0/3800826827 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f91bc00bcb0 con 0x5635a271acd0 2023-09-30T14:46:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.886+0000 7f91d7d24b80 1 -- 172.21.15.32:0/3800826827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91c009deb0 con 0x5635a271acd0 2023-09-30T14:46:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.886+0000 7f91ad7fa700 1 -- 172.21.15.32:0/3800826827 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91bc00e040 con 0x5635a271acd0 2023-09-30T14:46:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.886+0000 7f91d7d24b80 10 client.16200 fetch_fsmap learned FSMap version 38 2023-09-30T14:46:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.886+0000 7f91d7d24b80 1 -- 172.21.15.32:0/3800826827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5635a24bd1f0 con 0x5635a271acd0 2023-09-30T14:46:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.886+0000 7f91ad7fa700 1 -- 172.21.15.32:0/3800826827 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91c009deb0 con 0x5635a271acd0 2023-09-30T14:46:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.887+0000 7f91ad7fa700 1 -- 172.21.15.32:0/3800826827 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f91bc00b100 con 0x5635a271acd0 2023-09-30T14:46:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.887+0000 7f91d7d24b80 10 client.16200 fetch_fsmap finished waiting for FSMap version 38 2023-09-30T14:46:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.887+0000 7f91d7d24b80 10 client.16200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:46:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.887+0000 7f91d7d24b80 20 client.16200 populate_metadata read hostname 'smithi032' 2023-09-30T14:46:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.887+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/3800826827 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5635a27835c0 0x5635a27859e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.887+0000 7f91d7d24b80 4 client.16200 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:46:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.887+0000 7f91d7d24b80 1 -- 172.21.15.32:0/3800826827 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5635a2664ad0 con 0x5635a27835c0 2023-09-30T14:46:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.887+0000 7f91c98d8700 1 --2- 172.21.15.32:0/3800826827 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5635a27835c0 0x5635a27859e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.887+0000 7f91c98d8700 1 --2- 172.21.15.32:0/3800826827 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5635a27835c0 0x5635a27859e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a42e422724f01a server_cookie=8b3b095f74deee77 in_seq=0 out_seq=0 2023-09-30T14:46:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.887+0000 7f91ad7fa700 10 client.16200 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:46:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.929+0000 7f91ad7fa700 1 -- 172.21.15.32:0/3800826827 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5635a2664ad0 con 0x5635a27835c0 2023-09-30T14:46:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.929+0000 7f91ad7fa700 10 client.16200 handle_command_reply: tid=0 2023-09-30T14:46:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.929+0000 7f91d7d24b80 1 client.16200 shutdown 2023-09-30T14:46:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.929+0000 7f91d7d24b80 1 -- 172.21.15.32:0/3800826827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271acd0 msgr2=0x5635a2729010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.929+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/3800826827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271acd0 0x5635a2729010 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00b0d0 tx=0x7f91bc007780 comp rx=0 tx=0).stop 2023-09-30T14:46:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.929+0000 7f91d7d24b80 1 -- 172.21.15.32:0/3800826827 shutdown_connections 2023-09-30T14:46:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.929+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/3800826827 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5635a27835c0 0x5635a27859e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.929+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/3800826827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5635a271acd0 0x5635a2729010 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.929+0000 7f91ad7fa700 0 client.16200 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:46:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.929+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/3800826827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5635a2719200 0x5635a2727ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.929+0000 7f91d7d24b80 1 --2- 172.21.15.32:0/3800826827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635a271a310 0x5635a2728000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.929+0000 7f91d7d24b80 1 -- 172.21.15.32:0/3800826827 >> 172.21.15.32:0/3800826827 conn(0x5635a268d200 msgr2=0x5635a272ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.929+0000 7f91d7d24b80 1 -- 172.21.15.32:0/3800826827 shutdown_connections 2023-09-30T14:46:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.929+0000 7f91d7d24b80 1 -- 172.21.15.32:0/3800826827 wait complete. 2023-09-30T14:46:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.930+0000 7f91d7d24b80 20 client.16200 trim_cache size 0 max 16384 2023-09-30T14:46:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:46:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:46:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:46:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:46:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.930+0000 7f91cbb3c700 1 -- 172.21.15.32:0/1101759231 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91b00a0fa0 msgr2=0x7f91b00a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.930+0000 7f91cbb3c700 1 --2- 172.21.15.32:0/1101759231 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91b00a0fa0 0x7f91b00a3460 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f91c4069640 tx=0x7f91b400b410 comp rx=0 tx=0).stop 2023-09-30T14:46:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.930+0000 7f91cbb3c700 1 -- 172.21.15.32:0/1101759231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c40ffd90 msgr2=0x7f91c4199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.930+0000 7f91cbb3c700 1 --2- 172.21.15.32:0/1101759231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c40ffd90 0x7f91c4199990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f91c000e970 tx=0x7f91c000ec80 comp rx=0 tx=0).stop 2023-09-30T14:46:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.930+0000 7f91cbb3c700 1 -- 172.21.15.32:0/1101759231 shutdown_connections 2023-09-30T14:46:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.930+0000 7f91cbb3c700 1 --2- 172.21.15.32:0/1101759231 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91b00a0fa0 0x7f91b00a3460 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.930+0000 7f91cbb3c700 1 --2- 172.21.15.32:0/1101759231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91c4101920 0x7f91c419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.930+0000 7f91cbb3c700 1 --2- 172.21.15.32:0/1101759231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c40ffd90 0x7f91c4199990 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.930+0000 7f91cbb3c700 1 --2- 172.21.15.32:0/1101759231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c4100f80 0x7f91c419bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.930+0000 7f91cbb3c700 1 -- 172.21.15.32:0/1101759231 >> 172.21.15.32:0/1101759231 conn(0x7f91c40fb320 msgr2=0x7f91c41043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.930+0000 7f91cbb3c700 1 -- 172.21.15.32:0/1101759231 shutdown_connections 2023-09-30T14:46:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:25.930+0000 7f91cbb3c700 1 -- 172.21.15.32:0/1101759231 wait complete. 2023-09-30T14:46:26.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 866506 10.72 MB/sec execute 300 sec latency 15.059 ms 2023-09-30T14:46:27.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:27 smithi032 ceph-mon[98592]: pgmap v532: 97 pgs: 97 active+clean; 279 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 3.0 MiB/s wr, 313 op/s 2023-09-30T14:46:27.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:27 smithi156 ceph-mon[103579]: pgmap v532: 97 pgs: 97 active+clean; 279 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 3.0 MiB/s wr, 313 op/s 2023-09-30T14:46:27.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:27 smithi148 ceph-mon[102355]: pgmap v532: 97 pgs: 97 active+clean; 279 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 3.0 MiB/s wr, 313 op/s 2023-09-30T14:46:27.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 867781 10.70 MB/sec execute 301 sec latency 63.504 ms 2023-09-30T14:46:28.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:46:28.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:46:28.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:46:28.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:28 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:46:28.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:46:28.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:46:28.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:46:28.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:28 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:46:28.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:46:28.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:46:28.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:46:28.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:46:28.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:28 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:46:28.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 870122 10.71 MB/sec execute 302 sec latency 14.390 ms 2023-09-30T14:46:29.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:29 smithi032 ceph-mon[98592]: pgmap v533: 97 pgs: 97 active+clean; 281 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 2.9 MiB/s wr, 274 op/s 2023-09-30T14:46:29.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:29 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi148 to 196.2M 2023-09-30T14:46:29.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:29 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi032 to 197.2M 2023-09-30T14:46:29.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:29 smithi032 ceph-mon[98592]: Unable to set osd_memory_target on smithi148 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-30T14:46:29.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:29 smithi032 ceph-mon[98592]: Unable to set osd_memory_target on smithi032 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-30T14:46:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:29 smithi156 ceph-mon[103579]: pgmap v533: 97 pgs: 97 active+clean; 281 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 2.9 MiB/s wr, 274 op/s 2023-09-30T14:46:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:29 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi148 to 196.2M 2023-09-30T14:46:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:29 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi032 to 197.2M 2023-09-30T14:46:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:29 smithi156 ceph-mon[103579]: Unable to set osd_memory_target on smithi148 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-30T14:46:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:29 smithi156 ceph-mon[103579]: Unable to set osd_memory_target on smithi032 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-30T14:46:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:29 smithi148 ceph-mon[102355]: pgmap v533: 97 pgs: 97 active+clean; 281 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 2.9 MiB/s wr, 274 op/s 2023-09-30T14:46:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:29 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi148 to 196.2M 2023-09-30T14:46:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:29 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi032 to 197.2M 2023-09-30T14:46:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:29 smithi148 ceph-mon[102355]: Unable to set osd_memory_target on smithi148 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-30T14:46:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:29 smithi148 ceph-mon[102355]: Unable to set osd_memory_target on smithi032 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-30T14:46:29.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 872291 10.70 MB/sec execute 303 sec latency 13.978 ms 2023-09-30T14:46:30.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 873936 10.70 MB/sec execute 304 sec latency 43.172 ms 2023-09-30T14:46:31.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:31 smithi032 ceph-mon[98592]: pgmap v534: 97 pgs: 97 active+clean; 282 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 2.9 MiB/s wr, 264 op/s 2023-09-30T14:46:31.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:31 smithi156 ceph-mon[103579]: pgmap v534: 97 pgs: 97 active+clean; 282 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 2.9 MiB/s wr, 264 op/s 2023-09-30T14:46:31.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:31 smithi148 ceph-mon[102355]: pgmap v534: 97 pgs: 97 active+clean; 282 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 2.9 MiB/s wr, 264 op/s 2023-09-30T14:46:31.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 876397 10.71 MB/sec execute 305 sec latency 14.615 ms 2023-09-30T14:46:32.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 877830 10.70 MB/sec execute 306 sec latency 41.401 ms 2023-09-30T14:46:33.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:33 smithi032 ceph-mon[98592]: pgmap v535: 97 pgs: 97 active+clean; 283 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 3.0 MiB/s wr, 274 op/s 2023-09-30T14:46:33.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:33 smithi156 ceph-mon[103579]: pgmap v535: 97 pgs: 97 active+clean; 283 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 3.0 MiB/s wr, 274 op/s 2023-09-30T14:46:33.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:33 smithi148 ceph-mon[102355]: pgmap v535: 97 pgs: 97 active+clean; 283 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 3.0 MiB/s wr, 274 op/s 2023-09-30T14:46:33.710 INFO:tasks.workunit.client.0.smithi032.stdout: 1 880534 10.72 MB/sec execute 307 sec latency 48.834 ms 2023-09-30T14:46:34.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 882241 10.70 MB/sec execute 308 sec latency 13.207 ms 2023-09-30T14:46:35.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:35 smithi032 ceph-mon[98592]: pgmap v536: 97 pgs: 97 active+clean; 285 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2023-09-30T14:46:35.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:35 smithi156 ceph-mon[103579]: pgmap v536: 97 pgs: 97 active+clean; 285 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2023-09-30T14:46:35.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:35 smithi148 ceph-mon[102355]: pgmap v536: 97 pgs: 97 active+clean; 285 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2023-09-30T14:46:35.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 884440 10.71 MB/sec execute 309 sec latency 10.382 ms 2023-09-30T14:46:36.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 886911 10.72 MB/sec execute 310 sec latency 18.090 ms 2023-09-30T14:46:37.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:37 smithi032 ceph-mon[98592]: pgmap v537: 97 pgs: 97 active+clean; 285 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 3.1 MiB/s wr, 317 op/s 2023-09-30T14:46:37.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:37 smithi156 ceph-mon[103579]: pgmap v537: 97 pgs: 97 active+clean; 285 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 3.1 MiB/s wr, 317 op/s 2023-09-30T14:46:37.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:37 smithi148 ceph-mon[102355]: pgmap v537: 97 pgs: 97 active+clean; 285 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 3.1 MiB/s wr, 317 op/s 2023-09-30T14:46:37.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 888288 10.71 MB/sec execute 311 sec latency 56.971 ms 2023-09-30T14:46:38.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 890775 10.71 MB/sec execute 312 sec latency 20.808 ms 2023-09-30T14:46:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.052+0000 7fb41dce1700 1 -- 172.21.15.32:0/2690459971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb418100f80 msgr2=0x7fb4181013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.052+0000 7fb41dce1700 1 --2- 172.21.15.32:0/2690459971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb418100f80 0x7fb4181013e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb4000099c0 tx=0x7fb400009cd0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.052+0000 7fb41dce1700 1 -- 172.21.15.32:0/2690459971 shutdown_connections 2023-09-30T14:46:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.052+0000 7fb41dce1700 1 --2- 172.21.15.32:0/2690459971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb418101920 0x7fb4181081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.052+0000 7fb41dce1700 1 --2- 172.21.15.32:0/2690459971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4180ffd90 0x7fb4181001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.052+0000 7fb41dce1700 1 --2- 172.21.15.32:0/2690459971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb418100f80 0x7fb4181013e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.052+0000 7fb41dce1700 1 -- 172.21.15.32:0/2690459971 >> 172.21.15.32:0/2690459971 conn(0x7fb4180fb300 msgr2=0x7fb4180fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.052+0000 7fb41dce1700 1 -- 172.21.15.32:0/2690459971 shutdown_connections 2023-09-30T14:46:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.052+0000 7fb41dce1700 1 -- 172.21.15.32:0/2690459971 wait complete. 2023-09-30T14:46:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.053+0000 7fb41dce1700 1 Processor -- start 2023-09-30T14:46:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.053+0000 7fb41dce1700 1 -- start start 2023-09-30T14:46:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.053+0000 7fb41dce1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180ffd90 0x7fb418195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.053+0000 7fb41dce1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb418100f80 0x7fb418196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.053+0000 7fb41dce1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb418101920 0x7fb418196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.053+0000 7fb41dce1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb418197620 con 0x7fb418101920 2023-09-30T14:46:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.053+0000 7fb41dce1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb418197790 con 0x7fb418100f80 2023-09-30T14:46:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.053+0000 7fb41dce1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb418197950 con 0x7fb4180ffd90 2023-09-30T14:46:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.053+0000 7fb40effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb418100f80 0x7fb418196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.053+0000 7fb40effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb418100f80 0x7fb418196140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48860/0 (socket says 172.21.15.32:48860) 2023-09-30T14:46:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.053+0000 7fb40effd700 1 -- 172.21.15.32:0/1469397508 learned_addr learned my addr 172.21.15.32:0/1469397508 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.054+0000 7fb40effd700 1 -- 172.21.15.32:0/1469397508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180ffd90 msgr2=0x7fb418195c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.054+0000 7fb4177fe700 1 --2- 172.21.15.32:0/1469397508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180ffd90 0x7fb418195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.054+0000 7fb40effd700 1 --2- 172.21.15.32:0/1469397508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180ffd90 0x7fb418195c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.054+0000 7fb40effd700 1 -- 172.21.15.32:0/1469397508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb418101920 msgr2=0x7fb418196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.054+0000 7fb40effd700 1 --2- 172.21.15.32:0/1469397508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb418101920 0x7fb418196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.054+0000 7fb40effd700 1 -- 172.21.15.32:0/1469397508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4000095f0 con 0x7fb418100f80 2023-09-30T14:46:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.054+0000 7fb4177fe700 1 --2- 172.21.15.32:0/1469397508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180ffd90 0x7fb418195c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:46:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.054+0000 7fb40effd700 1 --2- 172.21.15.32:0/1469397508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb418100f80 0x7fb418196140 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb400009990 tx=0x7fb40000f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.054+0000 7fb4157fa700 1 -- 172.21.15.32:0/1469397508 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40001d070 con 0x7fb418100f80 2023-09-30T14:46:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.054+0000 7fb4157fa700 1 -- 172.21.15.32:0/1469397508 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4000054f0 con 0x7fb418100f80 2023-09-30T14:46:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.054+0000 7fb41dce1700 1 -- 172.21.15.32:0/1469397508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb418197bd0 con 0x7fb418100f80 2023-09-30T14:46:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.054+0000 7fb4157fa700 1 -- 172.21.15.32:0/1469397508 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400004c70 con 0x7fb418100f80 2023-09-30T14:46:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.054+0000 7fb41dce1700 1 -- 172.21.15.32:0/1469397508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41819e3f0 con 0x7fb418100f80 2023-09-30T14:46:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.057+0000 7fb4157fa700 1 -- 172.21.15.32:0/1469397508 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fb40000fc30 con 0x7fb418100f80 2023-09-30T14:46:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.057+0000 7fb4157fa700 1 --2- 172.21.15.32:0/1469397508 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb4040a0fa0 0x7fb4040a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.057+0000 7fb4177fe700 1 --2- 172.21.15.32:0/1469397508 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb4040a0fa0 0x7fb4040a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.057+0000 7fb4157fa700 1 -- 172.21.15.32:0/1469397508 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb4000cd650 con 0x7fb418100f80 2023-09-30T14:46:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.058+0000 7fb4177fe700 1 --2- 172.21.15.32:0/1469397508 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb4040a0fa0 0x7fb4040a3460 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb418100de0 tx=0x7fb408009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.058+0000 7fb429ec9b80 1 Processor -- start 2023-09-30T14:46:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.058+0000 7fb429ec9b80 1 -- start start 2023-09-30T14:46:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb429ec9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af0200 0x556751af0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb429ec9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1310 0x556751af1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb429ec9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af1cd0 0x556751af8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb429ec9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556751a58400 con 0x556751af1310 2023-09-30T14:46:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb429ec9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556751a49bc0 con 0x556751af1cd0 2023-09-30T14:46:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb429ec9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556751a474e0 con 0x556751af0200 2023-09-30T14:46:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb40effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1310 0x556751af1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb40effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1310 0x556751af1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37394/0 (socket says 172.21.15.32:37394) 2023-09-30T14:46:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb40effd700 1 -- 172.21.15.32:0/3321678188 learned_addr learned my addr 172.21.15.32:0/3321678188 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb417fff700 1 --2- 172.21.15.32:0/3321678188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af1cd0 0x556751af8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb4177fe700 1 --2- 172.21.15.32:0/3321678188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af0200 0x556751af0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb40effd700 1 -- 172.21.15.32:0/3321678188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af0200 msgr2=0x556751af0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb40effd700 1 --2- 172.21.15.32:0/3321678188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af0200 0x556751af0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb40effd700 1 -- 172.21.15.32:0/3321678188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af1cd0 msgr2=0x556751af8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb40effd700 1 --2- 172.21.15.32:0/3321678188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af1cd0 0x556751af8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb40effd700 1 -- 172.21.15.32:0/3321678188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556751aa5870 con 0x556751af1310 2023-09-30T14:46:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb417fff700 1 --2- 172.21.15.32:0/3321678188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af1cd0 0x556751af8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:46:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb40effd700 1 --2- 172.21.15.32:0/3321678188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1310 0x556751af1790 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb4000970b0 tx=0x7fb40009a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2788c678eb19f5d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb40dffb700 1 -- 172.21.15.32:0/3321678188 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40009abb0 con 0x556751af1310 2023-09-30T14:46:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb40dffb700 1 -- 172.21.15.32:0/3321678188 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb40009ed00 con 0x556751af1310 2023-09-30T14:46:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.059+0000 7fb40dffb700 1 -- 172.21.15.32:0/3321678188 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4000ba320 con 0x556751af1310 2023-09-30T14:46:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.060+0000 7fb429ec9b80 1 -- 172.21.15.32:0/3321678188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1310 msgr2=0x556751af1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.060+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/3321678188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1310 0x556751af1790 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb4000970b0 tx=0x7fb40009a820 comp rx=0 tx=0).stop 2023-09-30T14:46:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.060+0000 7fb429ec9b80 1 -- 172.21.15.32:0/3321678188 shutdown_connections 2023-09-30T14:46:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.060+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/3321678188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af1cd0 0x556751af8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.060+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/3321678188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af0200 0x556751af0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.060+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/3321678188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1310 0x556751af1790 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.060+0000 7fb429ec9b80 1 -- 172.21.15.32:0/3321678188 >> 172.21.15.32:0/3321678188 conn(0x556751a64200 msgr2=0x556751aeaef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.060+0000 7fb429ec9b80 1 -- 172.21.15.32:0/3321678188 shutdown_connections 2023-09-30T14:46:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.060+0000 7fb429ec9b80 1 -- 172.21.15.32:0/3321678188 wait complete. 2023-09-30T14:46:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.060+0000 7fb429ec9b80 1 -- start start 2023-09-30T14:46:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.061+0000 7fb429ec9b80 1 Processor -- start 2023-09-30T14:46:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.061+0000 7fb429ec9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af0200 0x556751af9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.061+0000 7fb429ec9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af1310 0x556751afa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.061+0000 7fb429ec9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1cd0 0x556751b010e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.061+0000 7fb429ec9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556751a42dd0 con 0x556751af1cd0 2023-09-30T14:46:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.061+0000 7fb429ec9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567519df6b0 con 0x556751af0200 2023-09-30T14:46:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.061+0000 7fb429ec9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567519df350 con 0x556751af1310 2023-09-30T14:46:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.061+0000 7fb429ec9b80 10 client.0 _ll_register_callbacks cb 0x55675183c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:46:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.061+0000 7fb417fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1cd0 0x556751b010e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.061+0000 7fb4177fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af0200 0x556751af9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.061+0000 7fb417fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1cd0 0x556751b010e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37402/0 (socket says 172.21.15.32:37402) 2023-09-30T14:46:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.061+0000 7fb4177fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af0200 0x556751af9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48874/0 (socket says 172.21.15.32:48874) 2023-09-30T14:46:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.061+0000 7fb417fff700 1 -- 172.21.15.32:0/440530909 learned_addr learned my addr 172.21.15.32:0/440530909 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.061+0000 7fb40effd700 1 --2- 172.21.15.32:0/440530909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af1310 0x556751afa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb417fff700 1 -- 172.21.15.32:0/440530909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af1310 msgr2=0x556751afa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb417fff700 1 --2- 172.21.15.32:0/440530909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af1310 0x556751afa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb417fff700 1 -- 172.21.15.32:0/440530909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af0200 msgr2=0x556751af9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb417fff700 1 --2- 172.21.15.32:0/440530909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af0200 0x556751af9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb417fff700 1 -- 172.21.15.32:0/440530909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41819e3f0 con 0x556751af1cd0 2023-09-30T14:46:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb4177fe700 1 --2- 172.21.15.32:0/440530909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af0200 0x556751af9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:46:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb417fff700 1 --2- 172.21.15.32:0/440530909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1cd0 0x556751b010e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb410008bc0 tx=0x7fb410008ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5c889416a647db server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb3f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:46:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb3f6ffd700 1 -- 172.21.15.32:0/440530909 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41000ea30 con 0x556751af1cd0 2023-09-30T14:46:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb3f6ffd700 1 -- 172.21.15.32:0/440530909 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb41000eb90 con 0x556751af1cd0 2023-09-30T14:46:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb429ec9b80 1 -- 172.21.15.32:0/440530909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556751ae3090 con 0x556751af1cd0 2023-09-30T14:46:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb3f6ffd700 1 -- 172.21.15.32:0/440530909 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb410007b10 con 0x556751af1cd0 2023-09-30T14:46:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb429ec9b80 10 client.16209 fetch_fsmap learned FSMap version 38 2023-09-30T14:46:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb429ec9b80 1 -- 172.21.15.32:0/440530909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55675186dd80 con 0x556751af1cd0 2023-09-30T14:46:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb3f6ffd700 1 -- 172.21.15.32:0/440530909 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556751ae3090 con 0x556751af1cd0 2023-09-30T14:46:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb3f6ffd700 1 -- 172.21.15.32:0/440530909 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb410013070 con 0x556751af1cd0 2023-09-30T14:46:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb429ec9b80 10 client.16209 fetch_fsmap finished waiting for FSMap version 38 2023-09-30T14:46:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb429ec9b80 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:46:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb429ec9b80 20 client.16209 populate_metadata read hostname 'smithi032' 2023-09-30T14:46:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/440530909 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556751b059c0 0x556751b07de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.062+0000 7fb429ec9b80 4 client.16209 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:46:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.063+0000 7fb429ec9b80 1 -- 172.21.15.32:0/440530909 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556751a3bdb0 con 0x556751b059c0 2023-09-30T14:46:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.063+0000 7fb4177fe700 1 --2- 172.21.15.32:0/440530909 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556751b059c0 0x556751b07de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.063+0000 7fb3f6ffd700 10 client.16209 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:46:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.063+0000 7fb4177fe700 1 --2- 172.21.15.32:0/440530909 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556751b059c0 0x556751b07de0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43a169d925d3efb2 server_cookie=6e528a5da0ab90ef in_seq=0 out_seq=0 2023-09-30T14:46:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.064+0000 7fb3f6ffd700 1 -- 172.21.15.32:0/440530909 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556751a3bdb0 con 0x556751b059c0 2023-09-30T14:46:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.064+0000 7fb3f6ffd700 10 client.16209 handle_command_reply: tid=0 2023-09-30T14:46:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.064+0000 7fb429ec9b80 1 client.16209 shutdown 2023-09-30T14:46:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.064+0000 7fb429ec9b80 1 -- 172.21.15.32:0/440530909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1cd0 msgr2=0x556751b010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.064+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/440530909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1cd0 0x556751b010e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb410008bc0 tx=0x7fb410008ed0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.064+0000 7fb429ec9b80 1 -- 172.21.15.32:0/440530909 shutdown_connections 2023-09-30T14:46:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.064+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/440530909 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556751b059c0 0x556751b07de0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.064+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/440530909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1cd0 0x556751b010e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.064+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/440530909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af0200 0x556751af9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.064+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/440530909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af1310 0x556751afa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.064+0000 7fb429ec9b80 1 -- 172.21.15.32:0/440530909 >> 172.21.15.32:0/440530909 conn(0x556751a64200 msgr2=0x556751aea590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.064+0000 7fb3f6ffd700 0 client.16209 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:46:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.065+0000 7fb429ec9b80 1 -- 172.21.15.32:0/440530909 shutdown_connections 2023-09-30T14:46:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.065+0000 7fb429ec9b80 1 -- 172.21.15.32:0/440530909 wait complete. 2023-09-30T14:46:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.065+0000 7fb429ec9b80 20 client.16209 trim_cache size 0 max 16384 2023-09-30T14:46:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.080+0000 7fb429ec9b80 1 Processor -- start 2023-09-30T14:46:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.080+0000 7fb429ec9b80 1 -- start start 2023-09-30T14:46:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.080+0000 7fb429ec9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af0200 0x556751aebe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.080+0000 7fb429ec9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af1310 0x556751af9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.080+0000 7fb429ec9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1cd0 0x556751afa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.080+0000 7fb429ec9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567518c38c0 con 0x556751af1cd0 2023-09-30T14:46:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.080+0000 7fb429ec9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556751af9010 con 0x556751af1310 2023-09-30T14:46:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.080+0000 7fb429ec9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556751af9700 con 0x556751af0200 2023-09-30T14:46:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.080+0000 7fb417fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1cd0 0x556751afa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.081+0000 7fb417fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1cd0 0x556751afa060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37410/0 (socket says 172.21.15.32:37410) 2023-09-30T14:46:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.081+0000 7fb417fff700 1 -- 172.21.15.32:0/1355852382 learned_addr learned my addr 172.21.15.32:0/1355852382 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.081+0000 7fb40effd700 1 --2- 172.21.15.32:0/1355852382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af1310 0x556751af9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.081+0000 7fb4177fe700 1 --2- 172.21.15.32:0/1355852382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af0200 0x556751aebe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.081+0000 7fb417fff700 1 -- 172.21.15.32:0/1355852382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af0200 msgr2=0x556751aebe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.081+0000 7fb417fff700 1 --2- 172.21.15.32:0/1355852382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af0200 0x556751aebe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.081+0000 7fb417fff700 1 -- 172.21.15.32:0/1355852382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af1310 msgr2=0x556751af9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.081+0000 7fb417fff700 1 --2- 172.21.15.32:0/1355852382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af1310 0x556751af9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.081+0000 7fb417fff700 1 -- 172.21.15.32:0/1355852382 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556751856c00 con 0x556751af1cd0 2023-09-30T14:46:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.081+0000 7fb417fff700 1 --2- 172.21.15.32:0/1355852382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1cd0 0x556751afa060 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb410000f80 tx=0x7fb4100075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a5f60122ddf5f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.081+0000 7fb40e7fc700 1 -- 172.21.15.32:0/1355852382 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41000e3b0 con 0x556751af1cd0 2023-09-30T14:46:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.081+0000 7fb40e7fc700 1 -- 172.21.15.32:0/1355852382 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb410010040 con 0x556751af1cd0 2023-09-30T14:46:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.081+0000 7fb40e7fc700 1 -- 172.21.15.32:0/1355852382 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41000e830 con 0x556751af1cd0 2023-09-30T14:46:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.082+0000 7fb429ec9b80 1 -- 172.21.15.32:0/1355852382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1cd0 msgr2=0x556751afa060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.082+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/1355852382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1cd0 0x556751afa060 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb410000f80 tx=0x7fb4100075f0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.082+0000 7fb429ec9b80 1 -- 172.21.15.32:0/1355852382 shutdown_connections 2023-09-30T14:46:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.082+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/1355852382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af1cd0 0x556751afa060 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.082+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/1355852382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af0200 0x556751aebe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.082+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/1355852382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af1310 0x556751af9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.082+0000 7fb429ec9b80 1 -- 172.21.15.32:0/1355852382 >> 172.21.15.32:0/1355852382 conn(0x556751a64200 msgr2=0x556751af4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.082+0000 7fb429ec9b80 1 -- 172.21.15.32:0/1355852382 shutdown_connections 2023-09-30T14:46:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.082+0000 7fb429ec9b80 1 -- 172.21.15.32:0/1355852382 wait complete. 2023-09-30T14:46:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.082+0000 7fb429ec9b80 1 -- start start 2023-09-30T14:46:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.082+0000 7fb429ec9b80 1 Processor -- start 2023-09-30T14:46:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb429ec9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af0200 0x556751b557c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb429ec9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af1310 0x556751b57d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb429ec9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af1cd0 0x556751b5a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb429ec9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556751b5a8d0 con 0x556751af0200 2023-09-30T14:46:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb429ec9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556751b5aa40 con 0x556751af1cd0 2023-09-30T14:46:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb429ec9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556751b5ac00 con 0x556751af1310 2023-09-30T14:46:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb429ec9b80 10 client.0 _ll_register_callbacks cb 0x7fb410008480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:46:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb4177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af0200 0x556751b557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb4177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af0200 0x556751b557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37412/0 (socket says 172.21.15.32:37412) 2023-09-30T14:46:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb4177fe700 1 -- 172.21.15.32:0/894172116 learned_addr learned my addr 172.21.15.32:0/894172116 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb40effd700 1 --2- 172.21.15.32:0/894172116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af1310 0x556751b57d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb4177fe700 1 -- 172.21.15.32:0/894172116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af1310 msgr2=0x556751b57d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb4177fe700 1 --2- 172.21.15.32:0/894172116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af1310 0x556751b57d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb4177fe700 1 -- 172.21.15.32:0/894172116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af1cd0 msgr2=0x556751b5a260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-30T14:46:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb4177fe700 1 --2- 172.21.15.32:0/894172116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af1cd0 0x556751b5a260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb4177fe700 1 -- 172.21.15.32:0/894172116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55675186dd80 con 0x556751af0200 2023-09-30T14:46:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb4177fe700 1 --2- 172.21.15.32:0/894172116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af0200 0x556751b557c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x556751b086f0 tx=0x7fb4080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0792279ffac2e11 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb3f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:46:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb3f6ffd700 1 -- 172.21.15.32:0/894172116 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408012450 con 0x556751af0200 2023-09-30T14:46:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb3f6ffd700 1 -- 172.21.15.32:0/894172116 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb4080125b0 con 0x556751af0200 2023-09-30T14:46:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.083+0000 7fb429ec9b80 1 -- 172.21.15.32:0/894172116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb40009ed00 con 0x556751af0200 2023-09-30T14:46:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.084+0000 7fb3f6ffd700 1 -- 172.21.15.32:0/894172116 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb408012770 con 0x556751af0200 2023-09-30T14:46:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.085+0000 7fb429ec9b80 10 client.16215 fetch_fsmap learned FSMap version 38 2023-09-30T14:46:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.085+0000 7fb429ec9b80 1 -- 172.21.15.32:0/894172116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5567518941f0 con 0x556751af0200 2023-09-30T14:46:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.085+0000 7fb3f6ffd700 1 -- 172.21.15.32:0/894172116 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb40009ed00 con 0x556751af0200 2023-09-30T14:46:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.086+0000 7fb3f6ffd700 1 -- 172.21.15.32:0/894172116 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb40801fa20 con 0x556751af0200 2023-09-30T14:46:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.086+0000 7fb429ec9b80 10 client.16215 fetch_fsmap finished waiting for FSMap version 38 2023-09-30T14:46:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.086+0000 7fb429ec9b80 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:46:39.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.086+0000 7fb429ec9b80 20 client.16215 populate_metadata read hostname 'smithi032' 2023-09-30T14:46:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.086+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/894172116 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556751b5c710 0x556751b5eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.086+0000 7fb429ec9b80 4 client.16215 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:46:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.086+0000 7fb429ec9b80 1 -- 172.21.15.32:0/894172116 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556751a3bad0 con 0x556751b5c710 2023-09-30T14:46:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.086+0000 7fb40effd700 1 --2- 172.21.15.32:0/894172116 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556751b5c710 0x556751b5eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.086+0000 7fb40effd700 1 --2- 172.21.15.32:0/894172116 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556751b5c710 0x556751b5eb30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da8f63f50d9cb47f server_cookie=36ec0222a55142c1 in_seq=0 out_seq=0 2023-09-30T14:46:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.086+0000 7fb3f6ffd700 10 client.16215 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:46:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.140+0000 7fb3f6ffd700 1 -- 172.21.15.32:0/894172116 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556751a3bad0 con 0x556751b5c710 2023-09-30T14:46:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.140+0000 7fb3f6ffd700 10 client.16215 handle_command_reply: tid=0 2023-09-30T14:46:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.140+0000 7fb429ec9b80 1 client.16215 shutdown 2023-09-30T14:46:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.140+0000 7fb429ec9b80 1 -- 172.21.15.32:0/894172116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af0200 msgr2=0x556751b557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.140+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/894172116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af0200 0x556751b557c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x556751b086f0 tx=0x7fb4080047c0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.140+0000 7fb429ec9b80 1 -- 172.21.15.32:0/894172116 shutdown_connections 2023-09-30T14:46:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.140+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/894172116 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556751b5c710 0x556751b5eb30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.140+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/894172116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556751af1cd0 0x556751b5a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.140+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/894172116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556751af0200 0x556751b557c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.140+0000 7fb3f6ffd700 0 client.16215 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:46:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.140+0000 7fb429ec9b80 1 --2- 172.21.15.32:0/894172116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556751af1310 0x556751b57d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.140+0000 7fb429ec9b80 1 -- 172.21.15.32:0/894172116 >> 172.21.15.32:0/894172116 conn(0x556751a64200 msgr2=0x556751af4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.140+0000 7fb429ec9b80 1 -- 172.21.15.32:0/894172116 shutdown_connections 2023-09-30T14:46:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.140+0000 7fb429ec9b80 1 -- 172.21.15.32:0/894172116 wait complete. 2023-09-30T14:46:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.141+0000 7fb429ec9b80 20 client.16215 trim_cache size 0 max 16384 2023-09-30T14:46:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:46:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:46:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:46:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:46:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.141+0000 7fb41dce1700 1 -- 172.21.15.32:0/1469397508 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb4040a0fa0 msgr2=0x7fb4040a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.141+0000 7fb41dce1700 1 --2- 172.21.15.32:0/1469397508 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb4040a0fa0 0x7fb4040a3460 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb418100de0 tx=0x7fb408009260 comp rx=0 tx=0).stop 2023-09-30T14:46:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.141+0000 7fb41dce1700 1 -- 172.21.15.32:0/1469397508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb418100f80 msgr2=0x7fb418196140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.141+0000 7fb41dce1700 1 --2- 172.21.15.32:0/1469397508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb418100f80 0x7fb418196140 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb400009990 tx=0x7fb40000f740 comp rx=0 tx=0).stop 2023-09-30T14:46:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.141+0000 7fb41dce1700 1 -- 172.21.15.32:0/1469397508 shutdown_connections 2023-09-30T14:46:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.141+0000 7fb41dce1700 1 --2- 172.21.15.32:0/1469397508 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb4040a0fa0 0x7fb4040a3460 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.141+0000 7fb41dce1700 1 --2- 172.21.15.32:0/1469397508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb418101920 0x7fb418196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.141+0000 7fb41dce1700 1 --2- 172.21.15.32:0/1469397508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4180ffd90 0x7fb418195c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.141+0000 7fb41dce1700 1 --2- 172.21.15.32:0/1469397508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb418100f80 0x7fb418196140 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.141+0000 7fb41dce1700 1 -- 172.21.15.32:0/1469397508 >> 172.21.15.32:0/1469397508 conn(0x7fb4180fb300 msgr2=0x7fb4181044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.141+0000 7fb41dce1700 1 -- 172.21.15.32:0/1469397508 shutdown_connections 2023-09-30T14:46:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:39.141+0000 7fb41dce1700 1 -- 172.21.15.32:0/1469397508 wait complete. 2023-09-30T14:46:39.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:39 smithi032 ceph-mon[98592]: pgmap v538: 97 pgs: 97 active+clean; 288 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 2.9 MiB/s wr, 280 op/s 2023-09-30T14:46:39.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:39 smithi156 ceph-mon[103579]: pgmap v538: 97 pgs: 97 active+clean; 288 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 2.9 MiB/s wr, 280 op/s 2023-09-30T14:46:39.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:39 smithi148 ceph-mon[102355]: pgmap v538: 97 pgs: 97 active+clean; 288 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 2.9 MiB/s wr, 280 op/s 2023-09-30T14:46:39.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 892288 10.71 MB/sec execute 313 sec latency 16.658 ms 2023-09-30T14:46:40.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 894792 10.72 MB/sec execute 314 sec latency 52.466 ms 2023-09-30T14:46:41.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:41 smithi032 ceph-mon[98592]: pgmap v539: 97 pgs: 97 active+clean; 289 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 2.9 MiB/s wr, 282 op/s 2023-09-30T14:46:41.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:41 smithi156 ceph-mon[103579]: pgmap v539: 97 pgs: 97 active+clean; 289 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 2.9 MiB/s wr, 282 op/s 2023-09-30T14:46:41.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:41 smithi148 ceph-mon[102355]: pgmap v539: 97 pgs: 97 active+clean; 289 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 2.9 MiB/s wr, 282 op/s 2023-09-30T14:46:41.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 896067 10.71 MB/sec execute 315 sec latency 83.685 ms 2023-09-30T14:46:42.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 898165 10.71 MB/sec execute 316 sec latency 86.633 ms 2023-09-30T14:46:43.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:43 smithi032 ceph-mon[98592]: pgmap v540: 97 pgs: 97 active+clean; 291 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 3.1 MiB/s wr, 309 op/s 2023-09-30T14:46:43.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:43 smithi156 ceph-mon[103579]: pgmap v540: 97 pgs: 97 active+clean; 291 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 3.1 MiB/s wr, 309 op/s 2023-09-30T14:46:43.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:43 smithi148 ceph-mon[102355]: pgmap v540: 97 pgs: 97 active+clean; 291 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 3.1 MiB/s wr, 309 op/s 2023-09-30T14:46:43.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 899461 10.69 MB/sec execute 317 sec latency 16.329 ms 2023-09-30T14:46:44.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 901915 10.71 MB/sec execute 318 sec latency 69.773 ms 2023-09-30T14:46:45.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:45 smithi032 ceph-mon[98592]: pgmap v541: 97 pgs: 97 active+clean; 292 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 3.0 MiB/s wr, 280 op/s 2023-09-30T14:46:45.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:45 smithi156 ceph-mon[103579]: pgmap v541: 97 pgs: 97 active+clean; 292 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 3.0 MiB/s wr, 280 op/s 2023-09-30T14:46:45.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:45 smithi148 ceph-mon[102355]: pgmap v541: 97 pgs: 97 active+clean; 292 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 3.0 MiB/s wr, 280 op/s 2023-09-30T14:46:45.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 903416 10.69 MB/sec execute 319 sec latency 15.204 ms 2023-09-30T14:46:46.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 905657 10.70 MB/sec execute 320 sec latency 12.396 ms 2023-09-30T14:46:47.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:47 smithi156 ceph-mon[103579]: pgmap v542: 97 pgs: 97 active+clean; 295 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 3.2 MiB/s wr, 311 op/s 2023-09-30T14:46:47.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:47 smithi148 ceph-mon[102355]: pgmap v542: 97 pgs: 97 active+clean; 295 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 3.2 MiB/s wr, 311 op/s 2023-09-30T14:46:47.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:47 smithi032 ceph-mon[98592]: pgmap v542: 97 pgs: 97 active+clean; 295 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 3.2 MiB/s wr, 311 op/s 2023-09-30T14:46:47.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 907431 10.69 MB/sec execute 321 sec latency 31.993 ms 2023-09-30T14:46:48.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 909313 10.70 MB/sec execute 322 sec latency 32.004 ms 2023-09-30T14:46:49.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:49 smithi156 ceph-mon[103579]: pgmap v543: 97 pgs: 97 active+clean; 293 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 3.0 MiB/s wr, 278 op/s 2023-09-30T14:46:49.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:49 smithi148 ceph-mon[102355]: pgmap v543: 97 pgs: 97 active+clean; 293 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 3.0 MiB/s wr, 278 op/s 2023-09-30T14:46:49.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:49 smithi032 ceph-mon[98592]: pgmap v543: 97 pgs: 97 active+clean; 293 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 3.0 MiB/s wr, 278 op/s 2023-09-30T14:46:49.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 911836 10.70 MB/sec execute 323 sec latency 14.387 ms 2023-09-30T14:46:50.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 913561 10.70 MB/sec execute 324 sec latency 18.458 ms 2023-09-30T14:46:51.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:51 smithi156 ceph-mon[103579]: pgmap v544: 97 pgs: 97 active+clean; 294 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 2.6 MiB/s wr, 245 op/s 2023-09-30T14:46:51.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:51 smithi148 ceph-mon[102355]: pgmap v544: 97 pgs: 97 active+clean; 294 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 2.6 MiB/s wr, 245 op/s 2023-09-30T14:46:51.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:51 smithi032 ceph-mon[98592]: pgmap v544: 97 pgs: 97 active+clean; 294 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 2.6 MiB/s wr, 245 op/s 2023-09-30T14:46:51.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 916206 10.71 MB/sec execute 325 sec latency 12.909 ms 2023-09-30T14:46:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.260+0000 7f4cbbc32700 1 -- 172.21.15.32:0/3565032415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb41018f0 msgr2=0x7f4cb41081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.260+0000 7f4cbbc32700 1 --2- 172.21.15.32:0/3565032415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb41018f0 0x7f4cb41081a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00b0d0 tx=0x7f4cac00b3e0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.260+0000 7f4cbbc32700 1 -- 172.21.15.32:0/3565032415 shutdown_connections 2023-09-30T14:46:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.260+0000 7f4cbbc32700 1 --2- 172.21.15.32:0/3565032415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb41018f0 0x7f4cb41081a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.260+0000 7f4cbbc32700 1 --2- 172.21.15.32:0/3565032415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb40ffd60 0x7f4cb4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.260+0000 7f4cbbc32700 1 --2- 172.21.15.32:0/3565032415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb4100f50 0x7f4cb41013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.260+0000 7f4cbbc32700 1 -- 172.21.15.32:0/3565032415 >> 172.21.15.32:0/3565032415 conn(0x7f4cb40fb2b0 msgr2=0x7f4cb40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.260+0000 7f4cbbc32700 1 -- 172.21.15.32:0/3565032415 shutdown_connections 2023-09-30T14:46:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.260+0000 7f4cbbc32700 1 -- 172.21.15.32:0/3565032415 wait complete. 2023-09-30T14:46:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.261+0000 7f4cbbc32700 1 Processor -- start 2023-09-30T14:46:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.261+0000 7f4cbbc32700 1 -- start start 2023-09-30T14:46:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.261+0000 7f4cbbc32700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb40ffd60 0x7f4cb4108cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.261+0000 7f4cbbc32700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb4100f50 0x7f4cb41091f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.261+0000 7f4cbbc32700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb41018f0 0x7f4cb4109730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.261+0000 7f4cbbc32700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb410a850 con 0x7f4cb40ffd60 2023-09-30T14:46:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.261+0000 7f4cbbc32700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb410a9c0 con 0x7f4cb41018f0 2023-09-30T14:46:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.261+0000 7f4cbbc32700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb410ab80 con 0x7f4cb4100f50 2023-09-30T14:46:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.261+0000 7f4cb99ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb40ffd60 0x7f4cb4108cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.261+0000 7f4cb99ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb40ffd60 0x7f4cb4108cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37426/0 (socket says 172.21.15.32:37426) 2023-09-30T14:46:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.261+0000 7f4cb99ce700 1 -- 172.21.15.32:0/1083509746 learned_addr learned my addr 172.21.15.32:0/1083509746 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.261+0000 7f4cb91cd700 1 --2- 172.21.15.32:0/1083509746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb4100f50 0x7f4cb41091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.261+0000 7f4cba1cf700 1 --2- 172.21.15.32:0/1083509746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb41018f0 0x7f4cb4109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.262+0000 7f4cb99ce700 1 -- 172.21.15.32:0/1083509746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb4100f50 msgr2=0x7f4cb41091f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.262+0000 7f4cb99ce700 1 --2- 172.21.15.32:0/1083509746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb4100f50 0x7f4cb41091f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.262+0000 7f4cb99ce700 1 -- 172.21.15.32:0/1083509746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb41018f0 msgr2=0x7f4cb4109730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.262+0000 7f4cb99ce700 1 --2- 172.21.15.32:0/1083509746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb41018f0 0x7f4cb4109730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.262+0000 7f4cb99ce700 1 -- 172.21.15.32:0/1083509746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cac009d00 con 0x7f4cb40ffd60 2023-09-30T14:46:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.262+0000 7f4cb99ce700 1 --2- 172.21.15.32:0/1083509746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb40ffd60 0x7f4cb4108cb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000d9b0 tx=0x7f4cb000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.262+0000 7f4caaffd700 1 -- 172.21.15.32:0/1083509746 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb000b950 con 0x7f4cb40ffd60 2023-09-30T14:46:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.262+0000 7f4cbbc32700 1 -- 172.21.15.32:0/1083509746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb41a5600 con 0x7f4cb40ffd60 2023-09-30T14:46:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.262+0000 7f4caaffd700 1 -- 172.21.15.32:0/1083509746 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cb0007620 con 0x7f4cb40ffd60 2023-09-30T14:46:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.262+0000 7f4cbbc32700 1 -- 172.21.15.32:0/1083509746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb41a5b30 con 0x7f4cb40ffd60 2023-09-30T14:46:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.262+0000 7f4caaffd700 1 -- 172.21.15.32:0/1083509746 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb000f3c0 con 0x7f4cb40ffd60 2023-09-30T14:46:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.264+0000 7f4caaffd700 1 -- 172.21.15.32:0/1083509746 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f4cb0010460 con 0x7f4cb40ffd60 2023-09-30T14:46:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.265+0000 7f4caaffd700 1 --2- 172.21.15.32:0/1083509746 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4ca00a0fa0 0x7f4ca00a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.265+0000 7f4cb91cd700 1 --2- 172.21.15.32:0/1083509746 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4ca00a0fa0 0x7f4ca00a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.265+0000 7f4caaffd700 1 -- 172.21.15.32:0/1083509746 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4cb00cc5a0 con 0x7f4cb40ffd60 2023-09-30T14:46:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.265+0000 7f4cb91cd700 1 --2- 172.21.15.32:0/1083509746 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4ca00a0fa0 0x7f4ca00a3460 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4109fb0 tx=0x7f4ca4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.265+0000 7f4cc7e1ab80 1 Processor -- start 2023-09-30T14:46:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cc7e1ab80 1 -- start start 2023-09-30T14:46:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cc7e1ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba1200 0x560ddeba1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cc7e1ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba2310 0x560ddeba2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cc7e1ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2cd0 0x560ddeba9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cc7e1ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ddeb09400 con 0x560ddeba2310 2023-09-30T14:46:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cc7e1ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ddeafabc0 con 0x560ddeba2cd0 2023-09-30T14:46:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cc7e1ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ddeaf84e0 con 0x560ddeba1200 2023-09-30T14:46:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cb91cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba2310 0x560ddeba2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cb99ce700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba1200 0x560ddeba1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cb91cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba2310 0x560ddeba2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37436/0 (socket says 172.21.15.32:37436) 2023-09-30T14:46:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cb91cd700 1 -- 172.21.15.32:0/3733316103 learned_addr learned my addr 172.21.15.32:0/3733316103 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cba1cf700 1 --2- 172.21.15.32:0/3733316103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2cd0 0x560ddeba9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cb99ce700 1 -- 172.21.15.32:0/3733316103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2cd0 msgr2=0x560ddeba9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cb99ce700 1 --2- 172.21.15.32:0/3733316103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2cd0 0x560ddeba9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cb99ce700 1 -- 172.21.15.32:0/3733316103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba2310 msgr2=0x560ddeba2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cb99ce700 1 --2- 172.21.15.32:0/3733316103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba2310 0x560ddeba2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cb99ce700 1 -- 172.21.15.32:0/3733316103 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ddeb56870 con 0x560ddeba1200 2023-09-30T14:46:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cb91cd700 1 --2- 172.21.15.32:0/3733316103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba2310 0x560ddeba2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:46:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.266+0000 7f4cb99ce700 1 --2- 172.21.15.32:0/3733316103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba1200 0x560ddeba1620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000b540 tx=0x7f4cb0095dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=668084ff6536a018 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.267+0000 7f4c9ffff700 1 -- 172.21.15.32:0/3733316103 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0099a10 con 0x560ddeba1200 2023-09-30T14:46:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.267+0000 7f4c9ffff700 1 -- 172.21.15.32:0/3733316103 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cb009db60 con 0x560ddeba1200 2023-09-30T14:46:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.267+0000 7f4c9ffff700 1 -- 172.21.15.32:0/3733316103 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0099c70 con 0x560ddeba1200 2023-09-30T14:46:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.267+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/3733316103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba1200 msgr2=0x560ddeba1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.267+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/3733316103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba1200 0x560ddeba1620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000b540 tx=0x7f4cb0095dd0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.267+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/3733316103 shutdown_connections 2023-09-30T14:46:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.267+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/3733316103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2cd0 0x560ddeba9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.267+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/3733316103 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba1200 0x560ddeba1620 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.267+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/3733316103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba2310 0x560ddeba2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.267+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/3733316103 >> 172.21.15.32:0/3733316103 conn(0x560ddeb15200 msgr2=0x560ddeb9bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.268+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/3733316103 shutdown_connections 2023-09-30T14:46:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.268+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/3733316103 wait complete. 2023-09-30T14:46:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.268+0000 7f4cc7e1ab80 1 -- start start 2023-09-30T14:46:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.268+0000 7f4cc7e1ab80 1 Processor -- start 2023-09-30T14:46:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.268+0000 7f4cc7e1ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddebaae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.268+0000 7f4cc7e1ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2310 0x560ddebab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.268+0000 7f4cc7e1ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba2cd0 0x560ddebb23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.268+0000 7f4cc7e1ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ddeaf3dd0 con 0x560ddeba1200 2023-09-30T14:46:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.268+0000 7f4cc7e1ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ddea906b0 con 0x560ddeba2310 2023-09-30T14:46:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.268+0000 7f4cc7e1ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ddea90350 con 0x560ddeba2cd0 2023-09-30T14:46:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.268+0000 7f4cc7e1ab80 10 client.0 _ll_register_callbacks cb 0x560dde8ed7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:46:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4cb99ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddebaae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4cb91cd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2310 0x560ddebab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4cb99ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddebaae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37440/0 (socket says 172.21.15.32:37440) 2023-09-30T14:46:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4cb99ce700 1 -- 172.21.15.32:0/3808638926 learned_addr learned my addr 172.21.15.32:0/3808638926 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4cba1cf700 1 --2- 172.21.15.32:0/3808638926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba2cd0 0x560ddebb23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4cb99ce700 1 -- 172.21.15.32:0/3808638926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba2cd0 msgr2=0x560ddebb23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4cb99ce700 1 --2- 172.21.15.32:0/3808638926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba2cd0 0x560ddebb23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4cb99ce700 1 -- 172.21.15.32:0/3808638926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2310 msgr2=0x560ddebab3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4cb99ce700 1 --2- 172.21.15.32:0/3808638926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2310 0x560ddebab3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4cb99ce700 1 -- 172.21.15.32:0/3808638926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca4017070 con 0x560ddeba1200 2023-09-30T14:46:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4cb91cd700 1 --2- 172.21.15.32:0/3808638926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2310 0x560ddebab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:46:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4cb99ce700 1 --2- 172.21.15.32:0/3808638926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddebaae60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0010980 tx=0x7f4cb000dfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91e484181be391a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4c9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:46:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4c9d7fa700 1 -- 172.21.15.32:0/3808638926 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb00b2170 con 0x560ddeba1200 2023-09-30T14:46:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4c9d7fa700 1 -- 172.21.15.32:0/3808638926 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4cb0099a10 con 0x560ddeba1200 2023-09-30T14:46:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/3808638926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ddeb94090 con 0x560ddeba1200 2023-09-30T14:46:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4c9d7fa700 1 -- 172.21.15.32:0/3808638926 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cb0098f00 con 0x560ddeba1200 2023-09-30T14:46:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4cc7e1ab80 10 client.16227 fetch_fsmap learned FSMap version 38 2023-09-30T14:46:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.269+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/3808638926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f4cb41a5b30 con 0x560ddeba1200 2023-09-30T14:46:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.270+0000 7f4c9d7fa700 1 -- 172.21.15.32:0/3808638926 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560ddeb94090 con 0x560ddeba1200 2023-09-30T14:46:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.270+0000 7f4c9d7fa700 1 -- 172.21.15.32:0/3808638926 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4cb00a1910 con 0x560ddeba1200 2023-09-30T14:46:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.270+0000 7f4cc7e1ab80 10 client.16227 fetch_fsmap finished waiting for FSMap version 38 2023-09-30T14:46:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.270+0000 7f4cc7e1ab80 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:46:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.270+0000 7f4cc7e1ab80 20 client.16227 populate_metadata read hostname 'smithi032' 2023-09-30T14:46:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.270+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/3808638926 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560ddebb68d0 0x560ddebb8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.270+0000 7f4cc7e1ab80 4 client.16227 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:46:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.270+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/3808638926 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ddeaecdb0 con 0x560ddebb68d0 2023-09-30T14:46:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.270+0000 7f4cb91cd700 1 --2- 172.21.15.32:0/3808638926 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560ddebb68d0 0x560ddebb8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.270+0000 7f4c9d7fa700 10 client.16227 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:46:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.270+0000 7f4cb91cd700 1 --2- 172.21.15.32:0/3808638926 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560ddebb68d0 0x560ddebb8cf0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd34c1ebf0b51fd server_cookie=6f87887e1807a911 in_seq=0 out_seq=0 2023-09-30T14:46:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.271+0000 7f4c9d7fa700 1 -- 172.21.15.32:0/3808638926 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560ddeaecdb0 con 0x560ddebb68d0 2023-09-30T14:46:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.271+0000 7f4c9d7fa700 10 client.16227 handle_command_reply: tid=0 2023-09-30T14:46:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.271+0000 7f4cc7e1ab80 1 client.16227 shutdown 2023-09-30T14:46:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.271+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/3808638926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 msgr2=0x560ddebaae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.271+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/3808638926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddebaae60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0010980 tx=0x7f4cb000dfb0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.271+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/3808638926 shutdown_connections 2023-09-30T14:46:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.271+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/3808638926 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560ddebb68d0 0x560ddebb8cf0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.271+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/3808638926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba2cd0 0x560ddebb23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.271+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/3808638926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddebaae60 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.271+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/3808638926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2310 0x560ddebab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.271+0000 7f4c9d7fa700 0 client.16227 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:46:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.271+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/3808638926 >> 172.21.15.32:0/3808638926 conn(0x560ddeb15200 msgr2=0x560ddeb9b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.271+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/3808638926 shutdown_connections 2023-09-30T14:46:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.271+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/3808638926 wait complete. 2023-09-30T14:46:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.271+0000 7f4cc7e1ab80 20 client.16227 trim_cache size 0 max 16384 2023-09-30T14:46:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.286+0000 7f4cc7e1ab80 1 Processor -- start 2023-09-30T14:46:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.286+0000 7f4cc7e1ab80 1 -- start start 2023-09-30T14:46:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.286+0000 7f4cc7e1ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddeba26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.286+0000 7f4cc7e1ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba2310 0x560ddebb23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.286+0000 7f4cc7e1ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2cd0 0x560ddeba9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.286+0000 7f4cc7e1ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dde9748c0 con 0x560ddeba1200 2023-09-30T14:46:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.286+0000 7f4cc7e1ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ddeb9d630 con 0x560ddeba2cd0 2023-09-30T14:46:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.286+0000 7f4cc7e1ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ddeba20e0 con 0x560ddeba2310 2023-09-30T14:46:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.286+0000 7f4cb99ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddeba26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.287+0000 7f4cb99ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddeba26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37448/0 (socket says 172.21.15.32:37448) 2023-09-30T14:46:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.287+0000 7f4cb99ce700 1 -- 172.21.15.32:0/1535644426 learned_addr learned my addr 172.21.15.32:0/1535644426 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.287+0000 7f4cb91cd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba2310 0x560ddebb23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.287+0000 7f4cba1cf700 1 --2- 172.21.15.32:0/1535644426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2cd0 0x560ddeba9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.287+0000 7f4cb99ce700 1 -- 172.21.15.32:0/1535644426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba2310 msgr2=0x560ddebb23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.287+0000 7f4cb99ce700 1 --2- 172.21.15.32:0/1535644426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba2310 0x560ddebb23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.287+0000 7f4cb99ce700 1 -- 172.21.15.32:0/1535644426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2cd0 msgr2=0x560ddeba9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.287+0000 7f4cb99ce700 1 --2- 172.21.15.32:0/1535644426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2cd0 0x560ddeba9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.287+0000 7f4cb99ce700 1 -- 172.21.15.32:0/1535644426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dde91ed80 con 0x560ddeba1200 2023-09-30T14:46:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.287+0000 7f4cb99ce700 1 --2- 172.21.15.32:0/1535644426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddeba26f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0099710 tx=0x7f4cb00a1ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956ae47b63d3a302 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.287+0000 7f4ca8d82700 1 -- 172.21.15.32:0/1535644426 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb00b2170 con 0x560ddeba1200 2023-09-30T14:46:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.287+0000 7f4ca8d82700 1 -- 172.21.15.32:0/1535644426 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cb00cc240 con 0x560ddeba1200 2023-09-30T14:46:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.287+0000 7f4ca8d82700 1 -- 172.21.15.32:0/1535644426 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0099c70 con 0x560ddeba1200 2023-09-30T14:46:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.288+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/1535644426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 msgr2=0x560ddeba26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.288+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/1535644426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddeba26f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0099710 tx=0x7f4cb00a1ee0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.288+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/1535644426 shutdown_connections 2023-09-30T14:46:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.288+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/1535644426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2cd0 0x560ddeba9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.288+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/1535644426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddeba26f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.288+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/1535644426 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba2310 0x560ddebb23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.288+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/1535644426 >> 172.21.15.32:0/1535644426 conn(0x560ddeb15200 msgr2=0x560ddeb9bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.288+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/1535644426 shutdown_connections 2023-09-30T14:46:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.288+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/1535644426 wait complete. 2023-09-30T14:46:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.288+0000 7f4cc7e1ab80 1 -- start start 2023-09-30T14:46:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.288+0000 7f4cc7e1ab80 1 Processor -- start 2023-09-30T14:46:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cc7e1ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddebaf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cc7e1ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2310 0x560ddebafec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cc7e1ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba2cd0 0x560ddebb0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cc7e1ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ddebb1410 con 0x560ddeba1200 2023-09-30T14:46:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cc7e1ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ddebb1580 con 0x560ddeba2310 2023-09-30T14:46:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cc7e1ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ddebb1740 con 0x560ddeba2cd0 2023-09-30T14:46:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cb99ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddebaf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cc7e1ab80 10 client.0 _ll_register_callbacks cb 0x7f4ca40119e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:46:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cb99ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddebaf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37454/0 (socket says 172.21.15.32:37454) 2023-09-30T14:46:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cb99ce700 1 -- 172.21.15.32:0/1612985553 learned_addr learned my addr 172.21.15.32:0/1612985553 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:46:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cb91cd700 1 --2- 172.21.15.32:0/1612985553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2310 0x560ddebafec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cba1cf700 1 --2- 172.21.15.32:0/1612985553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba2cd0 0x560ddebb0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cb99ce700 1 -- 172.21.15.32:0/1612985553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba2cd0 msgr2=0x560ddebb0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cb99ce700 1 --2- 172.21.15.32:0/1612985553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba2cd0 0x560ddebb0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cb99ce700 1 -- 172.21.15.32:0/1612985553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2310 msgr2=0x560ddebafec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:46:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cb99ce700 1 --2- 172.21.15.32:0/1612985553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2310 0x560ddebafec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cb99ce700 1 -- 172.21.15.32:0/1612985553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb41a5b30 con 0x560ddeba1200 2023-09-30T14:46:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cb99ce700 1 --2- 172.21.15.32:0/1612985553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddebaf980 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0099390 tx=0x7f4cb0099e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c80b9721c34b31 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:46:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4c9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:46:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4c9d7fa700 1 -- 172.21.15.32:0/1612985553 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb00b2170 con 0x560ddeba1200 2023-09-30T14:46:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4c9d7fa700 1 -- 172.21.15.32:0/1612985553 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4cb0099a10 con 0x560ddeba1200 2023-09-30T14:46:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.289+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/1612985553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4cb009db60 con 0x560ddeba1200 2023-09-30T14:46:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.290+0000 7f4c9d7fa700 1 -- 172.21.15.32:0/1612985553 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cb0095d50 con 0x560ddeba1200 2023-09-30T14:46:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.290+0000 7f4cc7e1ab80 10 client.16233 fetch_fsmap learned FSMap version 38 2023-09-30T14:46:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.290+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/1612985553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x560dde907c00 con 0x560ddeba1200 2023-09-30T14:46:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.290+0000 7f4c9d7fa700 1 -- 172.21.15.32:0/1612985553 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cb009db60 con 0x560ddeba1200 2023-09-30T14:46:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.290+0000 7f4c9d7fa700 1 -- 172.21.15.32:0/1612985553 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4cb0065340 con 0x560ddeba1200 2023-09-30T14:46:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.290+0000 7f4cc7e1ab80 10 client.16233 fetch_fsmap finished waiting for FSMap version 38 2023-09-30T14:46:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.290+0000 7f4cc7e1ab80 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:46:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.290+0000 7f4cc7e1ab80 20 client.16233 populate_metadata read hostname 'smithi032' 2023-09-30T14:46:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.290+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/1612985553 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560ddec0a770 0x560ddec0cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:46:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.290+0000 7f4cc7e1ab80 4 client.16233 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:46:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.290+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/1612985553 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560ddeaecad0 con 0x560ddec0a770 2023-09-30T14:46:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.290+0000 7f4cb91cd700 1 --2- 172.21.15.32:0/1612985553 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560ddec0a770 0x560ddec0cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:46:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.291+0000 7f4c9d7fa700 10 client.16233 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:46:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.291+0000 7f4cb91cd700 1 --2- 172.21.15.32:0/1612985553 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560ddec0a770 0x560ddec0cb60 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3450726749ccc161 server_cookie=b1ba3ba8ac5f3bb3 in_seq=0 out_seq=0 2023-09-30T14:46:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.459+0000 7f4c9d7fa700 1 -- 172.21.15.32:0/1612985553 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560ddeaecad0 con 0x560ddec0a770 2023-09-30T14:46:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.459+0000 7f4c9d7fa700 10 client.16233 handle_command_reply: tid=0 2023-09-30T14:46:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.459+0000 7f4cc7e1ab80 1 client.16233 shutdown 2023-09-30T14:46:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.459+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/1612985553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 msgr2=0x560ddebaf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/1612985553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddebaf980 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0099390 tx=0x7f4cb0099e80 comp rx=0 tx=0).stop 2023-09-30T14:46:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/1612985553 shutdown_connections 2023-09-30T14:46:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/1612985553 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x560ddec0a770 0x560ddec0cb60 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/1612985553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ddeba2cd0 0x560ddebb0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/1612985553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560ddeba1200 0x560ddebaf980 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cc7e1ab80 1 --2- 172.21.15.32:0/1612985553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ddeba2310 0x560ddebafec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/1612985553 >> 172.21.15.32:0/1612985553 conn(0x560ddeb15200 msgr2=0x560ddeba9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4c9d7fa700 0 client.16233 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:46:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/1612985553 shutdown_connections 2023-09-30T14:46:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cc7e1ab80 1 -- 172.21.15.32:0/1612985553 wait complete. 2023-09-30T14:46:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cc7e1ab80 20 client.16233 trim_cache size 0 max 16384 2023-09-30T14:46:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:46:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:46:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:46:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:46:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cbbc32700 1 -- 172.21.15.32:0/1083509746 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4ca00a0fa0 msgr2=0x7f4ca00a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cbbc32700 1 --2- 172.21.15.32:0/1083509746 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4ca00a0fa0 0x7f4ca00a3460 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4109fb0 tx=0x7f4ca4009260 comp rx=0 tx=0).stop 2023-09-30T14:46:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cbbc32700 1 -- 172.21.15.32:0/1083509746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb40ffd60 msgr2=0x7f4cb4108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:46:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cbbc32700 1 --2- 172.21.15.32:0/1083509746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb40ffd60 0x7f4cb4108cb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000d9b0 tx=0x7f4cb000dcc0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cbbc32700 1 -- 172.21.15.32:0/1083509746 shutdown_connections 2023-09-30T14:46:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cbbc32700 1 --2- 172.21.15.32:0/1083509746 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4ca00a0fa0 0x7f4ca00a3460 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cbbc32700 1 --2- 172.21.15.32:0/1083509746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cb41018f0 0x7f4cb4109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cbbc32700 1 --2- 172.21.15.32:0/1083509746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb40ffd60 0x7f4cb4108cb0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.460+0000 7f4cbbc32700 1 --2- 172.21.15.32:0/1083509746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb4100f50 0x7f4cb41091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:46:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.461+0000 7f4cbbc32700 1 -- 172.21.15.32:0/1083509746 >> 172.21.15.32:0/1083509746 conn(0x7f4cb40fb2b0 msgr2=0x7f4cb40fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:46:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.461+0000 7f4cbbc32700 1 -- 172.21.15.32:0/1083509746 shutdown_connections 2023-09-30T14:46:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:46:52.461+0000 7f4cbbc32700 1 -- 172.21.15.32:0/1083509746 wait complete. 2023-09-30T14:46:52.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 916993 10.72 MB/sec execute 326 sec latency 133.512 ms 2023-09-30T14:46:53.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:53 smithi156 ceph-mon[103579]: pgmap v545: 97 pgs: 97 active+clean; 297 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 3.1 MiB/s wr, 314 op/s 2023-09-30T14:46:53.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:53 smithi148 ceph-mon[102355]: pgmap v545: 97 pgs: 97 active+clean; 297 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 3.1 MiB/s wr, 314 op/s 2023-09-30T14:46:53.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:53 smithi032 ceph-mon[98592]: pgmap v545: 97 pgs: 97 active+clean; 297 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 3.1 MiB/s wr, 314 op/s 2023-09-30T14:46:53.711 INFO:tasks.workunit.client.0.smithi032.stdout: 1 919259 10.74 MB/sec execute 327 sec latency 31.101 ms 2023-09-30T14:46:54.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 920958 10.74 MB/sec execute 328 sec latency 13.108 ms 2023-09-30T14:46:55.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:55 smithi156 ceph-mon[103579]: pgmap v546: 97 pgs: 97 active+clean; 295 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 2.6 MiB/s wr, 249 op/s 2023-09-30T14:46:55.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:55 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:46:55.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:55 smithi148 ceph-mon[102355]: pgmap v546: 97 pgs: 97 active+clean; 295 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 2.6 MiB/s wr, 249 op/s 2023-09-30T14:46:55.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:55 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:46:55.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:55 smithi032 ceph-mon[98592]: pgmap v546: 97 pgs: 97 active+clean; 295 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 2.6 MiB/s wr, 249 op/s 2023-09-30T14:46:55.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:55 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:46:55.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 923779 10.76 MB/sec execute 329 sec latency 11.681 ms 2023-09-30T14:46:56.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 924385 10.73 MB/sec execute 330 sec latency 436.544 ms 2023-09-30T14:46:57.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:57 smithi156 ceph-mon[103579]: pgmap v547: 97 pgs: 97 active+clean; 284 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.7 MiB/s wr, 290 op/s 2023-09-30T14:46:57.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:57 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:46:57.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:57 smithi148 ceph-mon[102355]: pgmap v547: 97 pgs: 97 active+clean; 284 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.7 MiB/s wr, 290 op/s 2023-09-30T14:46:57.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:57 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:46:57.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:57 smithi032 ceph-mon[98592]: pgmap v547: 97 pgs: 97 active+clean; 284 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.7 MiB/s wr, 290 op/s 2023-09-30T14:46:57.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:57 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:46:57.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 924385 10.70 MB/sec execute 331 sec latency 1436.624 ms 2023-09-30T14:46:58.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 924385 10.67 MB/sec execute 332 sec latency 2436.749 ms 2023-09-30T14:46:59.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:46:59 smithi156 ceph-mon[103579]: pgmap v548: 97 pgs: 97 active+clean; 281 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 2.7 MiB/s wr, 269 op/s 2023-09-30T14:46:59.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:46:59 smithi148 ceph-mon[102355]: pgmap v548: 97 pgs: 97 active+clean; 281 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 2.7 MiB/s wr, 269 op/s 2023-09-30T14:46:59.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:46:59 smithi032 ceph-mon[98592]: pgmap v548: 97 pgs: 97 active+clean; 281 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 2.7 MiB/s wr, 269 op/s 2023-09-30T14:46:59.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 924385 10.64 MB/sec execute 333 sec latency 3436.835 ms 2023-09-30T14:47:00.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 924385 10.60 MB/sec execute 334 sec latency 4436.916 ms 2023-09-30T14:47:01.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:01 smithi156 ceph-mon[103579]: pgmap v549: 97 pgs: 97 active+clean; 285 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 2.7 MiB/s wr, 260 op/s 2023-09-30T14:47:01.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:01 smithi148 ceph-mon[102355]: pgmap v549: 97 pgs: 97 active+clean; 285 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 2.7 MiB/s wr, 260 op/s 2023-09-30T14:47:01.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:01 smithi032 ceph-mon[98592]: pgmap v549: 97 pgs: 97 active+clean; 285 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 2.7 MiB/s wr, 260 op/s 2023-09-30T14:47:01.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 926667 10.61 MB/sec execute 335 sec latency 4576.522 ms 2023-09-30T14:47:02.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 928249 10.60 MB/sec execute 336 sec latency 36.647 ms 2023-09-30T14:47:03.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:03 smithi156 ceph-mon[103579]: pgmap v550: 97 pgs: 97 active+clean; 285 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 2.8 MiB/s wr, 260 op/s 2023-09-30T14:47:03.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:03 smithi148 ceph-mon[102355]: pgmap v550: 97 pgs: 97 active+clean; 285 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 2.8 MiB/s wr, 260 op/s 2023-09-30T14:47:03.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:03 smithi032 ceph-mon[98592]: pgmap v550: 97 pgs: 97 active+clean; 285 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 2.8 MiB/s wr, 260 op/s 2023-09-30T14:47:03.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 930974 10.62 MB/sec execute 337 sec latency 65.164 ms 2023-09-30T14:47:04.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 932442 10.61 MB/sec execute 338 sec latency 33.183 ms 2023-09-30T14:47:05.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:05 smithi156 ceph-mon[103579]: pgmap v551: 97 pgs: 97 active+clean; 283 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 2.1 MiB/s wr, 176 op/s 2023-09-30T14:47:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.578+0000 7ffb88de8700 1 -- 172.21.15.32:0/3353080346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb840ffd60 msgr2=0x7ffb84100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.578+0000 7ffb88de8700 1 --2- 172.21.15.32:0/3353080346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb840ffd60 0x7ffb84100180 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ffb740099c0 tx=0x7ffb74009cd0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.578+0000 7ffb88de8700 1 -- 172.21.15.32:0/3353080346 shutdown_connections 2023-09-30T14:47:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.578+0000 7ffb88de8700 1 --2- 172.21.15.32:0/3353080346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb841018f0 0x7ffb841081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.578+0000 7ffb88de8700 1 --2- 172.21.15.32:0/3353080346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb840ffd60 0x7ffb84100180 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.578+0000 7ffb88de8700 1 --2- 172.21.15.32:0/3353080346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb84100f50 0x7ffb841013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.578+0000 7ffb88de8700 1 -- 172.21.15.32:0/3353080346 >> 172.21.15.32:0/3353080346 conn(0x7ffb840fb2b0 msgr2=0x7ffb840fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.578+0000 7ffb88de8700 1 -- 172.21.15.32:0/3353080346 shutdown_connections 2023-09-30T14:47:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.578+0000 7ffb88de8700 1 -- 172.21.15.32:0/3353080346 wait complete. 2023-09-30T14:47:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.579+0000 7ffb88de8700 1 Processor -- start 2023-09-30T14:47:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.579+0000 7ffb88de8700 1 -- start start 2023-09-30T14:47:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.579+0000 7ffb88de8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb840ffd60 0x7ffb84196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.579+0000 7ffb88de8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb84100f50 0x7ffb84197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.579+0000 7ffb88de8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb841018f0 0x7ffb84197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.579+0000 7ffb88de8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb84198650 con 0x7ffb841018f0 2023-09-30T14:47:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.579+0000 7ffb88de8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb841987c0 con 0x7ffb840ffd60 2023-09-30T14:47:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.579+0000 7ffb88de8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb84198980 con 0x7ffb84100f50 2023-09-30T14:47:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.579+0000 7ffb82d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb841018f0 0x7ffb84197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.579+0000 7ffb8259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb840ffd60 0x7ffb84196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.579+0000 7ffb8259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb840ffd60 0x7ffb84196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48944/0 (socket says 172.21.15.32:48944) 2023-09-30T14:47:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.579+0000 7ffb82d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb841018f0 0x7ffb84197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37472/0 (socket says 172.21.15.32:37472) 2023-09-30T14:47:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.579+0000 7ffb81d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb84100f50 0x7ffb84197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.579+0000 7ffb8259c700 1 -- 172.21.15.32:0/1923986959 learned_addr learned my addr 172.21.15.32:0/1923986959 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.579+0000 7ffb8259c700 1 -- 172.21.15.32:0/1923986959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb84100f50 msgr2=0x7ffb84197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.580+0000 7ffb8259c700 1 --2- 172.21.15.32:0/1923986959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb84100f50 0x7ffb84197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.580+0000 7ffb8259c700 1 -- 172.21.15.32:0/1923986959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb841018f0 msgr2=0x7ffb84197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.580+0000 7ffb8259c700 1 --2- 172.21.15.32:0/1923986959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb841018f0 0x7ffb84197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.580+0000 7ffb8259c700 1 -- 172.21.15.32:0/1923986959 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb740095f0 con 0x7ffb840ffd60 2023-09-30T14:47:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.580+0000 7ffb81d9b700 1 --2- 172.21.15.32:0/1923986959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb84100f50 0x7ffb84197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:47:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.580+0000 7ffb82d9d700 1 --2- 172.21.15.32:0/1923986959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb841018f0 0x7ffb84197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:47:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.580+0000 7ffb8259c700 1 --2- 172.21.15.32:0/1923986959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb840ffd60 0x7ffb84196c30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009fd0 tx=0x7ffb74009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.580+0000 7ffb737fe700 1 -- 172.21.15.32:0/1923986959 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7401d070 con 0x7ffb840ffd60 2023-09-30T14:47:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.580+0000 7ffb737fe700 1 -- 172.21.15.32:0/1923986959 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb74005030 con 0x7ffb840ffd60 2023-09-30T14:47:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.580+0000 7ffb737fe700 1 -- 172.21.15.32:0/1923986959 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb74004a90 con 0x7ffb840ffd60 2023-09-30T14:47:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.580+0000 7ffb88de8700 1 -- 172.21.15.32:0/1923986959 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb84198c00 con 0x7ffb840ffd60 2023-09-30T14:47:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.580+0000 7ffb88de8700 1 -- 172.21.15.32:0/1923986959 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8419f420 con 0x7ffb840ffd60 2023-09-30T14:47:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.582+0000 7ffb737fe700 1 -- 172.21.15.32:0/1923986959 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7ffb740051a0 con 0x7ffb840ffd60 2023-09-30T14:47:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.583+0000 7ffb737fe700 1 --2- 172.21.15.32:0/1923986959 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffb6c0a1020 0x7ffb6c0a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.583+0000 7ffb81d9b700 1 --2- 172.21.15.32:0/1923986959 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffb6c0a1020 0x7ffb6c0a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.583+0000 7ffb737fe700 1 -- 172.21.15.32:0/1923986959 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffb740cd420 con 0x7ffb840ffd60 2023-09-30T14:47:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.583+0000 7ffb81d9b700 1 --2- 172.21.15.32:0/1923986959 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffb6c0a1020 0x7ffb6c0a34e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ffb78011490 tx=0x7ffb78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb94fd0b80 1 Processor -- start 2023-09-30T14:47:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb94fd0b80 1 -- start start 2023-09-30T14:47:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb94fd0b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcb610 0x55685bbcba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb94fd0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 0x55685bbccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb94fd0b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcd0e0 0x55685bbd3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb94fd0b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685bb2f400 con 0x55685bbcd0e0 2023-09-30T14:47:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb94fd0b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685bb20bc0 con 0x55685bbcb610 2023-09-30T14:47:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb94fd0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685bb1e4e0 con 0x55685bbcc720 2023-09-30T14:47:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb81d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 0x55685bbccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb8259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcb610 0x55685bbcba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb81d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 0x55685bbccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51860/0 (socket says 172.21.15.32:51860) 2023-09-30T14:47:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb81d9b700 1 -- 172.21.15.32:0/3599457462 learned_addr learned my addr 172.21.15.32:0/3599457462 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb82d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcd0e0 0x55685bbd3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb81d9b700 1 -- 172.21.15.32:0/3599457462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcb610 msgr2=0x55685bbcba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb81d9b700 1 --2- 172.21.15.32:0/3599457462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcb610 0x55685bbcba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb81d9b700 1 -- 172.21.15.32:0/3599457462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcd0e0 msgr2=0x55685bbd3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.584+0000 7ffb81d9b700 1 --2- 172.21.15.32:0/3599457462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcd0e0 0x55685bbd3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.585+0000 7ffb81d9b700 1 -- 172.21.15.32:0/3599457462 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55685bb7c870 con 0x55685bbcc720 2023-09-30T14:47:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.585+0000 7ffb81d9b700 1 --2- 172.21.15.32:0/3599457462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 0x55685bbccba0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffb780043c0 tx=0x7ffb78014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abbe01f65ff0721c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.585+0000 7ffb70ff9700 1 -- 172.21.15.32:0/3599457462 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78003d10 con 0x55685bbcc720 2023-09-30T14:47:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.585+0000 7ffb70ff9700 1 -- 172.21.15.32:0/3599457462 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb78003e70 con 0x55685bbcc720 2023-09-30T14:47:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.585+0000 7ffb70ff9700 1 -- 172.21.15.32:0/3599457462 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7800e7a0 con 0x55685bbcc720 2023-09-30T14:47:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.586+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3599457462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 msgr2=0x55685bbccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.586+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/3599457462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 0x55685bbccba0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffb780043c0 tx=0x7ffb78014000 comp rx=0 tx=0).stop 2023-09-30T14:47:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.586+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3599457462 shutdown_connections 2023-09-30T14:47:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.586+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/3599457462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcd0e0 0x55685bbd3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.586+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/3599457462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcb610 0x55685bbcba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.586+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/3599457462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 0x55685bbccba0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.586+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3599457462 >> 172.21.15.32:0/3599457462 conn(0x55685bb3b200 msgr2=0x55685bbc1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.586+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3599457462 shutdown_connections 2023-09-30T14:47:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.586+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3599457462 wait complete. 2023-09-30T14:47:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.586+0000 7ffb94fd0b80 1 -- start start 2023-09-30T14:47:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.586+0000 7ffb94fd0b80 1 Processor -- start 2023-09-30T14:47:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb94fd0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcb610 0x55685bbd52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb94fd0b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcc720 0x55685bbd57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb94fd0b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcd0e0 0x55685bbdc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb94fd0b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685bb19dd0 con 0x55685bbcc720 2023-09-30T14:47:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb94fd0b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685bab66b0 con 0x55685bbcd0e0 2023-09-30T14:47:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb94fd0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685bab6350 con 0x55685bbcb610 2023-09-30T14:47:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb94fd0b80 10 client.0 _ll_register_callbacks cb 0x55685b9137a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:47:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb81d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcc720 0x55685bbd57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb8259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcb610 0x55685bbd52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb8259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcb610 0x55685bbd52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51864/0 (socket says 172.21.15.32:51864) 2023-09-30T14:47:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb82d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcd0e0 0x55685bbdc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb8259c700 1 -- 172.21.15.32:0/3797169073 learned_addr learned my addr 172.21.15.32:0/3797169073 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb81d9b700 1 -- 172.21.15.32:0/3797169073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcb610 msgr2=0x55685bbd52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb81d9b700 1 --2- 172.21.15.32:0/3797169073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcb610 0x55685bbd52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb81d9b700 1 -- 172.21.15.32:0/3797169073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcd0e0 msgr2=0x55685bbdc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb81d9b700 1 --2- 172.21.15.32:0/3797169073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcd0e0 0x55685bbdc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb81d9b700 1 -- 172.21.15.32:0/3797169073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb78017070 con 0x55685bbcc720 2023-09-30T14:47:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb82d9d700 1 --2- 172.21.15.32:0/3797169073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcd0e0 0x55685bbdc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:47:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb81d9b700 1 --2- 172.21.15.32:0/3797169073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcc720 0x55685bbd57e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ffb78011be0 tx=0x7ffb78005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519a9f94b65c78ba server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:47:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb69ffb700 1 -- 172.21.15.32:0/3797169073 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7802c070 con 0x55685bbcc720 2023-09-30T14:47:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3797169073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55685bbba090 con 0x55685bbcc720 2023-09-30T14:47:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.587+0000 7ffb69ffb700 1 -- 172.21.15.32:0/3797169073 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffb780055f0 con 0x55685bbcc720 2023-09-30T14:47:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.588+0000 7ffb69ffb700 1 -- 172.21.15.32:0/3797169073 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb78004710 con 0x55685bbcc720 2023-09-30T14:47:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.588+0000 7ffb94fd0b80 10 client.16245 fetch_fsmap learned FSMap version 40 2023-09-30T14:47:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.588+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3797169073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55685b944d80 con 0x55685bbcc720 2023-09-30T14:47:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.588+0000 7ffb69ffb700 1 -- 172.21.15.32:0/3797169073 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55685bbba090 con 0x55685bbcc720 2023-09-30T14:47:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.588+0000 7ffb69ffb700 1 -- 172.21.15.32:0/3797169073 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffb7803a070 con 0x55685bbcc720 2023-09-30T14:47:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.588+0000 7ffb94fd0b80 10 client.16245 fetch_fsmap finished waiting for FSMap version 40 2023-09-30T14:47:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.588+0000 7ffb94fd0b80 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:47:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.588+0000 7ffb94fd0b80 20 client.16245 populate_metadata read hostname 'smithi032' 2023-09-30T14:47:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.588+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/3797169073 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55685bbe1030 0x55685bbe3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.588+0000 7ffb94fd0b80 4 client.16245 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:47:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.588+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3797169073 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55685bb12db0 con 0x55685bbe1030 2023-09-30T14:47:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.588+0000 7ffb8259c700 1 --2- 172.21.15.32:0/3797169073 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55685bbe1030 0x55685bbe3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.589+0000 7ffb8259c700 1 --2- 172.21.15.32:0/3797169073 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55685bbe1030 0x55685bbe3450 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53cb4d2641405d2e server_cookie=258eb03adf6d8a3b in_seq=0 out_seq=0 2023-09-30T14:47:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.589+0000 7ffb69ffb700 10 client.16245 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.589+0000 7ffb69ffb700 1 -- 172.21.15.32:0/3797169073 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55685bb12db0 con 0x55685bbe1030 2023-09-30T14:47:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.589+0000 7ffb69ffb700 10 client.16245 handle_command_reply: tid=0 2023-09-30T14:47:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.589+0000 7ffb94fd0b80 1 client.16245 shutdown 2023-09-30T14:47:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.589+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3797169073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcc720 msgr2=0x55685bbd57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.589+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/3797169073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcc720 0x55685bbd57e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ffb78011be0 tx=0x7ffb78005270 comp rx=0 tx=0).stop 2023-09-30T14:47:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.590+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3797169073 shutdown_connections 2023-09-30T14:47:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.590+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/3797169073 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55685bbe1030 0x55685bbe3450 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.590+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/3797169073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcd0e0 0x55685bbdc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.590+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/3797169073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcb610 0x55685bbd52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.590+0000 7ffb69ffb700 0 client.16245 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.590+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/3797169073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcc720 0x55685bbd57e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.590+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3797169073 >> 172.21.15.32:0/3797169073 conn(0x55685bb3b200 msgr2=0x55685bbc1590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.590+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3797169073 shutdown_connections 2023-09-30T14:47:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.590+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3797169073 wait complete. 2023-09-30T14:47:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.590+0000 7ffb94fd0b80 20 client.16245 trim_cache size 0 max 16384 2023-09-30T14:47:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb94fd0b80 1 Processor -- start 2023-09-30T14:47:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb94fd0b80 1 -- start start 2023-09-30T14:47:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb94fd0b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcb610 0x55685bbd34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb94fd0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 0x55685bbd39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb94fd0b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcd0e0 0x55685bbdc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb94fd0b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685b99a8c0 con 0x55685bbcb610 2023-09-30T14:47:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb94fd0b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685bbd4780 con 0x55685bbcd0e0 2023-09-30T14:47:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb94fd0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685bbc3290 con 0x55685bbcc720 2023-09-30T14:47:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb8259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcb610 0x55685bbd34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb8259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcb610 0x55685bbd34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37490/0 (socket says 172.21.15.32:37490) 2023-09-30T14:47:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb8259c700 1 -- 172.21.15.32:0/1692943108 learned_addr learned my addr 172.21.15.32:0/1692943108 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb82d9d700 1 --2- 172.21.15.32:0/1692943108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcd0e0 0x55685bbdc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb81d9b700 1 --2- 172.21.15.32:0/1692943108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 0x55685bbd39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb8259c700 1 -- 172.21.15.32:0/1692943108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 msgr2=0x55685bbd39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb8259c700 1 --2- 172.21.15.32:0/1692943108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 0x55685bbd39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb8259c700 1 -- 172.21.15.32:0/1692943108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcd0e0 msgr2=0x55685bbdc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb8259c700 1 --2- 172.21.15.32:0/1692943108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcd0e0 0x55685bbdc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.605+0000 7ffb8259c700 1 -- 172.21.15.32:0/1692943108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55685b92dc00 con 0x55685bbcb610 2023-09-30T14:47:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.606+0000 7ffb8259c700 1 --2- 172.21.15.32:0/1692943108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcb610 0x55685bbd34b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400bf40 tx=0x7ffb7401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d4ba42cacf8f6b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.606+0000 7ffb717fa700 1 -- 172.21.15.32:0/1692943108 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb740a2c80 con 0x55685bbcb610 2023-09-30T14:47:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.606+0000 7ffb717fa700 1 -- 172.21.15.32:0/1692943108 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb740cb040 con 0x55685bbcb610 2023-09-30T14:47:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.606+0000 7ffb717fa700 1 -- 172.21.15.32:0/1692943108 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb74098db0 con 0x55685bbcb610 2023-09-30T14:47:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.606+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/1692943108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcb610 msgr2=0x55685bbd34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.606+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/1692943108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcb610 0x55685bbd34b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400bf40 tx=0x7ffb7401a040 comp rx=0 tx=0).stop 2023-09-30T14:47:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/1692943108 shutdown_connections 2023-09-30T14:47:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/1692943108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcd0e0 0x55685bbdc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/1692943108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcb610 0x55685bbd34b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/1692943108 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 0x55685bbd39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/1692943108 >> 172.21.15.32:0/1692943108 conn(0x55685bb3b200 msgr2=0x55685bbe7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/1692943108 shutdown_connections 2023-09-30T14:47:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/1692943108 wait complete. 2023-09-30T14:47:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb94fd0b80 1 -- start start 2023-09-30T14:47:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb94fd0b80 1 Processor -- start 2023-09-30T14:47:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb94fd0b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcb610 0x55685bbda330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb94fd0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 0x55685bbda870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb94fd0b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcd0e0 0x55685bbdb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb94fd0b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685bbdbdc0 con 0x55685bbcd0e0 2023-09-30T14:47:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb94fd0b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685bbdbf30 con 0x55685bbcb610 2023-09-30T14:47:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb94fd0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685bbdc0f0 con 0x55685bbcc720 2023-09-30T14:47:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb94fd0b80 10 client.0 _ll_register_callbacks cb 0x7ffb7c009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:47:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb8259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcb610 0x55685bbda330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb82d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcd0e0 0x55685bbdb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb81d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 0x55685bbda870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb8259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcb610 0x55685bbda330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:48972/0 (socket says 172.21.15.32:48972) 2023-09-30T14:47:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.607+0000 7ffb8259c700 1 -- 172.21.15.32:0/3913994282 learned_addr learned my addr 172.21.15.32:0/3913994282 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.608+0000 7ffb81d9b700 1 -- 172.21.15.32:0/3913994282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcb610 msgr2=0x55685bbda330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.608+0000 7ffb81d9b700 1 --2- 172.21.15.32:0/3913994282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcb610 0x55685bbda330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.608+0000 7ffb81d9b700 1 -- 172.21.15.32:0/3913994282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcd0e0 msgr2=0x55685bbdb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.608+0000 7ffb81d9b700 1 --2- 172.21.15.32:0/3913994282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcd0e0 0x55685bbdb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.608+0000 7ffb81d9b700 1 -- 172.21.15.32:0/3913994282 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8419f420 con 0x55685bbcc720 2023-09-30T14:47:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.608+0000 7ffb81d9b700 1 --2- 172.21.15.32:0/3913994282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 0x55685bbda870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffb78006010 tx=0x7ffb7801e830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e43bd9d499ec580 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.608+0000 7ffb69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:47:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.608+0000 7ffb69ffb700 1 -- 172.21.15.32:0/3913994282 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7802c040 con 0x55685bbcc720 2023-09-30T14:47:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.608+0000 7ffb69ffb700 1 -- 172.21.15.32:0/3913994282 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffb7800e3b0 con 0x55685bbcc720 2023-09-30T14:47:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.608+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3913994282 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb78003e70 con 0x55685bbcc720 2023-09-30T14:47:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.608+0000 7ffb69ffb700 1 -- 172.21.15.32:0/3913994282 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb7800e700 con 0x55685bbcc720 2023-09-30T14:47:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.608+0000 7ffb94fd0b80 10 client.25921 fetch_fsmap learned FSMap version 40 2023-09-30T14:47:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.608+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3913994282 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55685b96b1f0 con 0x55685bbcc720 2023-09-30T14:47:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.608+0000 7ffb69ffb700 1 -- 172.21.15.32:0/3913994282 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb78003e70 con 0x55685bbcc720 2023-09-30T14:47:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.608+0000 7ffb69ffb700 1 -- 172.21.15.32:0/3913994282 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffb78003680 con 0x55685bbcc720 2023-09-30T14:47:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.609+0000 7ffb94fd0b80 10 client.25921 fetch_fsmap finished waiting for FSMap version 40 2023-09-30T14:47:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.609+0000 7ffb94fd0b80 10 client.25921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:47:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.609+0000 7ffb94fd0b80 20 client.25921 populate_metadata read hostname 'smithi032' 2023-09-30T14:47:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.609+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/3913994282 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55685bc35640 0x55685bc37a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.609+0000 7ffb94fd0b80 4 client.25921 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:47:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.609+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3913994282 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55685bb12ad0 con 0x55685bc35640 2023-09-30T14:47:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.609+0000 7ffb8259c700 1 --2- 172.21.15.32:0/3913994282 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55685bc35640 0x55685bc37a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.609+0000 7ffb8259c700 1 --2- 172.21.15.32:0/3913994282 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55685bc35640 0x55685bc37a60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d677611e95381acb server_cookie=2e1384f1ceacf684 in_seq=0 out_seq=0 2023-09-30T14:47:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.609+0000 7ffb69ffb700 10 client.25921 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:05.616 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:05 smithi148 ceph-mon[102355]: pgmap v551: 97 pgs: 97 active+clean; 283 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 2.1 MiB/s wr, 176 op/s 2023-09-30T14:47:05.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:05 smithi032 ceph-mon[98592]: pgmap v551: 97 pgs: 97 active+clean; 283 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 2.1 MiB/s wr, 176 op/s 2023-09-30T14:47:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.650+0000 7ffb69ffb700 1 -- 172.21.15.32:0/3913994282 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55685bb12ad0 con 0x55685bc35640 2023-09-30T14:47:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.650+0000 7ffb69ffb700 10 client.25921 handle_command_reply: tid=0 2023-09-30T14:47:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.650+0000 7ffb94fd0b80 1 client.25921 shutdown 2023-09-30T14:47:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.650+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3913994282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 msgr2=0x55685bbda870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.650+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/3913994282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 0x55685bbda870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffb78006010 tx=0x7ffb7801e830 comp rx=0 tx=0).stop 2023-09-30T14:47:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.650+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3913994282 shutdown_connections 2023-09-30T14:47:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.650+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/3913994282 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55685bc35640 0x55685bc37a60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.650+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/3913994282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55685bbcd0e0 0x55685bbdb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.650+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/3913994282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55685bbcb610 0x55685bbda330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.650+0000 7ffb69ffb700 0 client.25921 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.650+0000 7ffb94fd0b80 1 --2- 172.21.15.32:0/3913994282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55685bbcc720 0x55685bbda870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.650+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3913994282 >> 172.21.15.32:0/3913994282 conn(0x55685bb3b200 msgr2=0x55685bbd34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.650+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3913994282 shutdown_connections 2023-09-30T14:47:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.650+0000 7ffb94fd0b80 1 -- 172.21.15.32:0/3913994282 wait complete. 2023-09-30T14:47:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.650+0000 7ffb94fd0b80 20 client.25921 trim_cache size 0 max 16384 2023-09-30T14:47:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:47:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:47:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:47:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:47:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.651+0000 7ffb88de8700 1 -- 172.21.15.32:0/1923986959 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffb6c0a1020 msgr2=0x7ffb6c0a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.651+0000 7ffb88de8700 1 --2- 172.21.15.32:0/1923986959 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffb6c0a1020 0x7ffb6c0a34e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ffb78011490 tx=0x7ffb78009260 comp rx=0 tx=0).stop 2023-09-30T14:47:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.651+0000 7ffb88de8700 1 -- 172.21.15.32:0/1923986959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb840ffd60 msgr2=0x7ffb84196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.651+0000 7ffb88de8700 1 --2- 172.21.15.32:0/1923986959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb840ffd60 0x7ffb84196c30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009fd0 tx=0x7ffb74009580 comp rx=0 tx=0).stop 2023-09-30T14:47:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.651+0000 7ffb88de8700 1 -- 172.21.15.32:0/1923986959 shutdown_connections 2023-09-30T14:47:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.651+0000 7ffb88de8700 1 --2- 172.21.15.32:0/1923986959 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffb6c0a1020 0x7ffb6c0a34e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.651+0000 7ffb88de8700 1 --2- 172.21.15.32:0/1923986959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb841018f0 0x7ffb84197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.651+0000 7ffb88de8700 1 --2- 172.21.15.32:0/1923986959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb840ffd60 0x7ffb84196c30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.651+0000 7ffb88de8700 1 --2- 172.21.15.32:0/1923986959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb84100f50 0x7ffb84197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.651+0000 7ffb88de8700 1 -- 172.21.15.32:0/1923986959 >> 172.21.15.32:0/1923986959 conn(0x7ffb840fb2b0 msgr2=0x7ffb840fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.651+0000 7ffb88de8700 1 -- 172.21.15.32:0/1923986959 shutdown_connections 2023-09-30T14:47:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:05.651+0000 7ffb88de8700 1 -- 172.21.15.32:0/1923986959 wait complete. 2023-09-30T14:47:05.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 934826 10.62 MB/sec execute 339 sec latency 14.637 ms 2023-09-30T14:47:06.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 937348 10.62 MB/sec execute 340 sec latency 13.708 ms 2023-09-30T14:47:07.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:07 smithi156 ceph-mon[103579]: pgmap v552: 97 pgs: 97 active+clean; 282 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 2.8 MiB/s wr, 234 op/s 2023-09-30T14:47:07.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:07 smithi148 ceph-mon[102355]: pgmap v552: 97 pgs: 97 active+clean; 282 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 2.8 MiB/s wr, 234 op/s 2023-09-30T14:47:07.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:07 smithi032 ceph-mon[98592]: pgmap v552: 97 pgs: 97 active+clean; 282 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 2.8 MiB/s wr, 234 op/s 2023-09-30T14:47:07.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 938992 10.61 MB/sec execute 341 sec latency 12.943 ms 2023-09-30T14:47:08.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 941572 10.63 MB/sec execute 342 sec latency 37.638 ms 2023-09-30T14:47:09.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:09 smithi156 ceph-mon[103579]: pgmap v553: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 2.5 MiB/s wr, 198 op/s 2023-09-30T14:47:09.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:09 smithi148 ceph-mon[102355]: pgmap v553: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 2.5 MiB/s wr, 198 op/s 2023-09-30T14:47:09.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:09 smithi032 ceph-mon[98592]: pgmap v553: 97 pgs: 97 active+clean; 281 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 2.5 MiB/s wr, 198 op/s 2023-09-30T14:47:09.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 943015 10.62 MB/sec execute 343 sec latency 31.657 ms 2023-09-30T14:47:10.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 945486 10.63 MB/sec execute 344 sec latency 9.847 ms 2023-09-30T14:47:11.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:11 smithi156 ceph-mon[103579]: pgmap v554: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 2.5 MiB/s wr, 214 op/s 2023-09-30T14:47:11.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:11 smithi148 ceph-mon[102355]: pgmap v554: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 2.5 MiB/s wr, 214 op/s 2023-09-30T14:47:11.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:11 smithi032 ceph-mon[98592]: pgmap v554: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 2.5 MiB/s wr, 214 op/s 2023-09-30T14:47:11.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 947900 10.63 MB/sec execute 345 sec latency 11.155 ms 2023-09-30T14:47:12.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 949576 10.63 MB/sec execute 346 sec latency 12.285 ms 2023-09-30T14:47:13.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:13 smithi156 ceph-mon[103579]: pgmap v555: 97 pgs: 97 active+clean; 284 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 2.7 MiB/s wr, 242 op/s 2023-09-30T14:47:13.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:13 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:47:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:13 smithi148 ceph-mon[102355]: pgmap v555: 97 pgs: 97 active+clean; 284 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 2.7 MiB/s wr, 242 op/s 2023-09-30T14:47:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:13 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:47:13.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:13 smithi032 ceph-mon[98592]: pgmap v555: 97 pgs: 97 active+clean; 284 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 2.7 MiB/s wr, 242 op/s 2023-09-30T14:47:13.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:13 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:47:13.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 952014 10.64 MB/sec execute 347 sec latency 64.726 ms 2023-09-30T14:47:14.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 953384 10.62 MB/sec execute 348 sec latency 42.631 ms 2023-09-30T14:47:15.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:15 smithi156 ceph-mon[103579]: pgmap v556: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 2.6 MiB/s wr, 232 op/s 2023-09-30T14:47:15.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:15 smithi148 ceph-mon[102355]: pgmap v556: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 2.6 MiB/s wr, 232 op/s 2023-09-30T14:47:15.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:15 smithi032 ceph-mon[98592]: pgmap v556: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 2.6 MiB/s wr, 232 op/s 2023-09-30T14:47:15.712 INFO:tasks.workunit.client.0.smithi032.stdout: 1 955956 10.64 MB/sec execute 349 sec latency 39.867 ms 2023-09-30T14:47:16.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 957836 10.63 MB/sec execute 350 sec latency 14.350 ms 2023-09-30T14:47:17.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:17 smithi156 ceph-mon[103579]: pgmap v557: 97 pgs: 97 active+clean; 286 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 3.1 MiB/s wr, 308 op/s 2023-09-30T14:47:17.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:17 smithi148 ceph-mon[102355]: pgmap v557: 97 pgs: 97 active+clean; 286 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 3.1 MiB/s wr, 308 op/s 2023-09-30T14:47:17.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:17 smithi032 ceph-mon[98592]: pgmap v557: 97 pgs: 97 active+clean; 286 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 3.1 MiB/s wr, 308 op/s 2023-09-30T14:47:17.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 959752 10.64 MB/sec execute 351 sec latency 12.228 ms 2023-09-30T14:47:18.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 961844 10.63 MB/sec execute 352 sec latency 43.670 ms 2023-09-30T14:47:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.780+0000 7f5a21d59700 1 -- 172.21.15.32:0/377791222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0ffd70 msgr2=0x7f5a1c100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.780+0000 7f5a21d59700 1 --2- 172.21.15.32:0/377791222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0ffd70 0x7f5a1c100190 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f5a100099c0 tx=0x7f5a10009cd0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.780+0000 7f5a21d59700 1 -- 172.21.15.32:0/377791222 shutdown_connections 2023-09-30T14:47:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.780+0000 7f5a21d59700 1 --2- 172.21.15.32:0/377791222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c101900 0x7f5a1c1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.780+0000 7f5a21d59700 1 --2- 172.21.15.32:0/377791222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0ffd70 0x7f5a1c100190 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.780+0000 7f5a21d59700 1 --2- 172.21.15.32:0/377791222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a1c100f60 0x7f5a1c1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.780+0000 7f5a21d59700 1 -- 172.21.15.32:0/377791222 >> 172.21.15.32:0/377791222 conn(0x7f5a1c0fb2a0 msgr2=0x7f5a1c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.780+0000 7f5a21d59700 1 -- 172.21.15.32:0/377791222 shutdown_connections 2023-09-30T14:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.780+0000 7f5a21d59700 1 -- 172.21.15.32:0/377791222 wait complete. 2023-09-30T14:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.780+0000 7f5a21d59700 1 Processor -- start 2023-09-30T14:47:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a21d59700 1 -- start start 2023-09-30T14:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a21d59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0ffd70 0x7f5a1c198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a21d59700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a1c100f60 0x7f5a1c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a21d59700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c101900 0x7f5a1c19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a21d59700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c19a900 con 0x7f5a1c0ffd70 2023-09-30T14:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a21d59700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c19aa70 con 0x7f5a1c101900 2023-09-30T14:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a21d59700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c19ac30 con 0x7f5a1c100f60 2023-09-30T14:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a1b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0ffd70 0x7f5a1c198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a1b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0ffd70 0x7f5a1c198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37510/0 (socket says 172.21.15.32:37510) 2023-09-30T14:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a1b7fe700 1 -- 172.21.15.32:0/3099443424 learned_addr learned my addr 172.21.15.32:0/3099443424 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a1affd700 1 --2- 172.21.15.32:0/3099443424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a1c100f60 0x7f5a1c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a1bfff700 1 --2- 172.21.15.32:0/3099443424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c101900 0x7f5a1c19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a1b7fe700 1 -- 172.21.15.32:0/3099443424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a1c100f60 msgr2=0x7f5a1c199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a1b7fe700 1 --2- 172.21.15.32:0/3099443424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a1c100f60 0x7f5a1c199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a1b7fe700 1 -- 172.21.15.32:0/3099443424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c101900 msgr2=0x7f5a1c19a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a1b7fe700 1 --2- 172.21.15.32:0/3099443424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c101900 0x7f5a1c19a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a1b7fe700 1 -- 172.21.15.32:0/3099443424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a100095f0 con 0x7f5a1c0ffd70 2023-09-30T14:47:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a1affd700 1 --2- 172.21.15.32:0/3099443424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a1c100f60 0x7f5a1c199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:47:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.781+0000 7f5a1b7fe700 1 --2- 172.21.15.32:0/3099443424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0ffd70 0x7f5a1c198ee0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009990 tx=0x7f5a1000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.782+0000 7f5a18ff9700 1 -- 172.21.15.32:0/3099443424 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1001d070 con 0x7f5a1c0ffd70 2023-09-30T14:47:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.782+0000 7f5a18ff9700 1 -- 172.21.15.32:0/3099443424 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a10005030 con 0x7f5a1c0ffd70 2023-09-30T14:47:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.782+0000 7f5a21d59700 1 -- 172.21.15.32:0/3099443424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1c19aeb0 con 0x7f5a1c0ffd70 2023-09-30T14:47:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.782+0000 7f5a18ff9700 1 -- 172.21.15.32:0/3099443424 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10004b30 con 0x7f5a1c0ffd70 2023-09-30T14:47:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.782+0000 7f5a21d59700 1 -- 172.21.15.32:0/3099443424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1c1a16d0 con 0x7f5a1c0ffd70 2023-09-30T14:47:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.784+0000 7f5a18ff9700 1 -- 172.21.15.32:0/3099443424 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f5a10004cb0 con 0x7f5a1c0ffd70 2023-09-30T14:47:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.785+0000 7f5a18ff9700 1 --2- 172.21.15.32:0/3099443424 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5a040a0fa0 0x7f5a040a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.785+0000 7f5a18ff9700 1 -- 172.21.15.32:0/3099443424 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5a100cd580 con 0x7f5a1c0ffd70 2023-09-30T14:47:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.785+0000 7f5a1affd700 1 --2- 172.21.15.32:0/3099443424 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5a040a0fa0 0x7f5a040a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.786+0000 7f5a1affd700 1 --2- 172.21.15.32:0/3099443424 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5a040a0fa0 0x7f5a040a3460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c011490 tx=0x7f5a0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.786+0000 7f5a2df41b80 1 Processor -- start 2023-09-30T14:47:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.786+0000 7f5a2df41b80 1 -- start start 2023-09-30T14:47:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.786+0000 7f5a2df41b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4fe200 0x558b5f4fe620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.786+0000 7f5a2df41b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ff310 0x558b5f4ff790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.786+0000 7f5a2df41b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4ffcd0 0x558b5f506560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.786+0000 7f5a2df41b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5f466400 con 0x558b5f4ff310 2023-09-30T14:47:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.786+0000 7f5a2df41b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5f457bc0 con 0x558b5f4ffcd0 2023-09-30T14:47:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.786+0000 7f5a2df41b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5f4554e0 con 0x558b5f4fe200 2023-09-30T14:47:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.786+0000 7f5a1affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ff310 0x558b5f4ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.786+0000 7f5a1affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ff310 0x558b5f4ff790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37520/0 (socket says 172.21.15.32:37520) 2023-09-30T14:47:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.786+0000 7f5a1affd700 1 -- 172.21.15.32:0/3449104237 learned_addr learned my addr 172.21.15.32:0/3449104237 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.786+0000 7f5a1bfff700 1 --2- 172.21.15.32:0/3449104237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4ffcd0 0x558b5f506560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.786+0000 7f5a1b7fe700 1 --2- 172.21.15.32:0/3449104237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4fe200 0x558b5f4fe620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.787+0000 7f5a1affd700 1 -- 172.21.15.32:0/3449104237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4fe200 msgr2=0x558b5f4fe620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.787+0000 7f5a1affd700 1 --2- 172.21.15.32:0/3449104237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4fe200 0x558b5f4fe620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.787+0000 7f5a1affd700 1 -- 172.21.15.32:0/3449104237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4ffcd0 msgr2=0x558b5f506560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.787+0000 7f5a1affd700 1 --2- 172.21.15.32:0/3449104237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4ffcd0 0x558b5f506560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.787+0000 7f5a1affd700 1 -- 172.21.15.32:0/3449104237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b5f4b3870 con 0x558b5f4ff310 2023-09-30T14:47:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.787+0000 7f5a1affd700 1 --2- 172.21.15.32:0/3449104237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ff310 0x558b5f4ff790 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0043c0 tx=0x7f5a0c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3682e376beff20 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.787+0000 7f5a01ffb700 1 -- 172.21.15.32:0/3449104237 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c003d10 con 0x558b5f4ff310 2023-09-30T14:47:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.787+0000 7f5a01ffb700 1 -- 172.21.15.32:0/3449104237 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a0c003e70 con 0x558b5f4ff310 2023-09-30T14:47:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.787+0000 7f5a01ffb700 1 -- 172.21.15.32:0/3449104237 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c00e7a0 con 0x558b5f4ff310 2023-09-30T14:47:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.788+0000 7f5a2df41b80 1 -- 172.21.15.32:0/3449104237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ff310 msgr2=0x558b5f4ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.788+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/3449104237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ff310 0x558b5f4ff790 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0043c0 tx=0x7f5a0c014000 comp rx=0 tx=0).stop 2023-09-30T14:47:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.788+0000 7f5a2df41b80 1 -- 172.21.15.32:0/3449104237 shutdown_connections 2023-09-30T14:47:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.788+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/3449104237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4ffcd0 0x558b5f506560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.788+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/3449104237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4fe200 0x558b5f4fe620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.788+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/3449104237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ff310 0x558b5f4ff790 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.788+0000 7f5a2df41b80 1 -- 172.21.15.32:0/3449104237 >> 172.21.15.32:0/3449104237 conn(0x558b5f472200 msgr2=0x558b5f4f8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.788+0000 7f5a2df41b80 1 -- 172.21.15.32:0/3449104237 shutdown_connections 2023-09-30T14:47:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.788+0000 7f5a2df41b80 1 -- 172.21.15.32:0/3449104237 wait complete. 2023-09-30T14:47:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.788+0000 7f5a2df41b80 1 -- start start 2023-09-30T14:47:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a2df41b80 1 Processor -- start 2023-09-30T14:47:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a2df41b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4fe200 0x558b5f507fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a2df41b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4ff310 0x558b5f5084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a2df41b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ffcd0 0x558b5f50f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a2df41b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5f450dd0 con 0x558b5f4ffcd0 2023-09-30T14:47:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a2df41b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5f3ed6b0 con 0x558b5f4ff310 2023-09-30T14:47:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a2df41b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5f3ed350 con 0x558b5f4fe200 2023-09-30T14:47:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a2df41b80 10 client.0 _ll_register_callbacks cb 0x558b5f24a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:47:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a1b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4fe200 0x558b5f507fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a1affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4ff310 0x558b5f5084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a1bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ffcd0 0x558b5f50f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a1b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4fe200 0x558b5f507fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51906/0 (socket says 172.21.15.32:51906) 2023-09-30T14:47:18.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a1affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4ff310 0x558b5f5084f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49002/0 (socket says 172.21.15.32:49002) 2023-09-30T14:47:18.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a1b7fe700 1 -- 172.21.15.32:0/2706297461 learned_addr learned my addr 172.21.15.32:0/2706297461 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:18.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a1bfff700 1 -- 172.21.15.32:0/2706297461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4fe200 msgr2=0x558b5f507fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:18.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a1bfff700 1 --2- 172.21.15.32:0/2706297461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4fe200 0x558b5f507fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a1bfff700 1 -- 172.21.15.32:0/2706297461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4ff310 msgr2=0x558b5f5084f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a1bfff700 1 --2- 172.21.15.32:0/2706297461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4ff310 0x558b5f5084f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.789+0000 7f5a1bfff700 1 -- 172.21.15.32:0/2706297461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0c017070 con 0x558b5f4ffcd0 2023-09-30T14:47:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.790+0000 7f5a1bfff700 1 --2- 172.21.15.32:0/2706297461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ffcd0 0x558b5f50f530 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400ae80 tx=0x7f5a14012510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9849d41dcd71dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.790+0000 7f59faffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:47:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.790+0000 7f59faffd700 1 -- 172.21.15.32:0/2706297461 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1400f040 con 0x558b5f4ffcd0 2023-09-30T14:47:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.790+0000 7f59faffd700 1 -- 172.21.15.32:0/2706297461 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5a14012bf0 con 0x558b5f4ffcd0 2023-09-30T14:47:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.790+0000 7f5a2df41b80 1 -- 172.21.15.32:0/2706297461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b5f4f1090 con 0x558b5f4ffcd0 2023-09-30T14:47:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.790+0000 7f59faffd700 1 -- 172.21.15.32:0/2706297461 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a14012db0 con 0x558b5f4ffcd0 2023-09-30T14:47:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.790+0000 7f5a2df41b80 10 client.16263 fetch_fsmap learned FSMap version 41 2023-09-30T14:47:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.790+0000 7f5a2df41b80 1 -- 172.21.15.32:0/2706297461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x558b5f27bd80 con 0x558b5f4ffcd0 2023-09-30T14:47:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.790+0000 7f59faffd700 1 -- 172.21.15.32:0/2706297461 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558b5f4f1090 con 0x558b5f4ffcd0 2023-09-30T14:47:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.790+0000 7f59faffd700 1 -- 172.21.15.32:0/2706297461 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5a140083f0 con 0x558b5f4ffcd0 2023-09-30T14:47:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.791+0000 7f5a2df41b80 10 client.16263 fetch_fsmap finished waiting for FSMap version 41 2023-09-30T14:47:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.791+0000 7f5a2df41b80 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:47:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.791+0000 7f5a2df41b80 20 client.16263 populate_metadata read hostname 'smithi032' 2023-09-30T14:47:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.791+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/2706297461 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558b5f513ec0 0x558b5f5162e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.791+0000 7f5a2df41b80 4 client.16263 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:47:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.791+0000 7f5a2df41b80 1 -- 172.21.15.32:0/2706297461 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b5f449db0 con 0x558b5f513ec0 2023-09-30T14:47:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.791+0000 7f5a1b7fe700 1 --2- 172.21.15.32:0/2706297461 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558b5f513ec0 0x558b5f5162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.791+0000 7f5a1b7fe700 1 --2- 172.21.15.32:0/2706297461 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558b5f513ec0 0x558b5f5162e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5973d5e422294cf server_cookie=b864b7cc572d90f2 in_seq=0 out_seq=0 2023-09-30T14:47:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.791+0000 7f59faffd700 10 client.16263 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.792+0000 7f59faffd700 1 -- 172.21.15.32:0/2706297461 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558b5f449db0 con 0x558b5f513ec0 2023-09-30T14:47:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.792+0000 7f59faffd700 10 client.16263 handle_command_reply: tid=0 2023-09-30T14:47:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.792+0000 7f5a2df41b80 1 client.16263 shutdown 2023-09-30T14:47:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.792+0000 7f5a2df41b80 1 -- 172.21.15.32:0/2706297461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ffcd0 msgr2=0x558b5f50f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.792+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/2706297461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ffcd0 0x558b5f50f530 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400ae80 tx=0x7f5a14012510 comp rx=0 tx=0).stop 2023-09-30T14:47:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.792+0000 7f5a2df41b80 1 -- 172.21.15.32:0/2706297461 shutdown_connections 2023-09-30T14:47:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.792+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/2706297461 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558b5f513ec0 0x558b5f5162e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.792+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/2706297461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ffcd0 0x558b5f50f530 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.792+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/2706297461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4fe200 0x558b5f507fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.792+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/2706297461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4ff310 0x558b5f5084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.792+0000 7f59faffd700 0 client.16263 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.792+0000 7f5a2df41b80 1 -- 172.21.15.32:0/2706297461 >> 172.21.15.32:0/2706297461 conn(0x558b5f472200 msgr2=0x558b5f4f8590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.792+0000 7f5a2df41b80 1 -- 172.21.15.32:0/2706297461 shutdown_connections 2023-09-30T14:47:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.792+0000 7f5a2df41b80 1 -- 172.21.15.32:0/2706297461 wait complete. 2023-09-30T14:47:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.792+0000 7f5a2df41b80 20 client.16263 trim_cache size 0 max 16384 2023-09-30T14:47:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.807+0000 7f5a2df41b80 1 Processor -- start 2023-09-30T14:47:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.807+0000 7f5a2df41b80 1 -- start start 2023-09-30T14:47:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a2df41b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4fe200 0x558b5f4ff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a2df41b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4ff310 0x558b5f50f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a2df41b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ffcd0 0x558b5f5061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a2df41b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5f2d18c0 con 0x558b5f4ffcd0 2023-09-30T14:47:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a2df41b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5f4f5c10 con 0x558b5f4fe200 2023-09-30T14:47:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a2df41b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5f500130 con 0x558b5f4ff310 2023-09-30T14:47:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a1bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ffcd0 0x558b5f5061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a1b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4fe200 0x558b5f4ff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a1b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4fe200 0x558b5f4ff6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49008/0 (socket says 172.21.15.32:49008) 2023-09-30T14:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a1b7fe700 1 -- 172.21.15.32:0/898619150 learned_addr learned my addr 172.21.15.32:0/898619150 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a1bfff700 1 --2- 172.21.15.32:0/898619150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ffcd0 0x558b5f5061c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37536/0 (socket says 172.21.15.32:37536) 2023-09-30T14:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a1affd700 1 --2- 172.21.15.32:0/898619150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4ff310 0x558b5f50f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a1b7fe700 1 -- 172.21.15.32:0/898619150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4ff310 msgr2=0x558b5f50f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a1b7fe700 1 --2- 172.21.15.32:0/898619150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4ff310 0x558b5f50f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a1b7fe700 1 -- 172.21.15.32:0/898619150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ffcd0 msgr2=0x558b5f5061c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a1b7fe700 1 --2- 172.21.15.32:0/898619150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ffcd0 0x558b5f5061c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a1b7fe700 1 -- 172.21.15.32:0/898619150 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b5f264c00 con 0x558b5f4fe200 2023-09-30T14:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a1bfff700 1 --2- 172.21.15.32:0/898619150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ffcd0 0x558b5f5061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.808+0000 7f5a1b7fe700 1 --2- 172.21.15.32:0/898619150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4fe200 0x558b5f4ff6f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5a10005cd0 tx=0x7f5a1001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad0911a828aad13e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.809+0000 7f5a027fc700 1 -- 172.21.15.32:0/898619150 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a100b2070 con 0x558b5f4fe200 2023-09-30T14:47:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.809+0000 7f5a027fc700 1 -- 172.21.15.32:0/898619150 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a100a2d80 con 0x558b5f4fe200 2023-09-30T14:47:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.809+0000 7f5a027fc700 1 -- 172.21.15.32:0/898619150 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10098e20 con 0x558b5f4fe200 2023-09-30T14:47:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.809+0000 7f5a2df41b80 1 -- 172.21.15.32:0/898619150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4fe200 msgr2=0x558b5f4ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.809+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/898619150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4fe200 0x558b5f4ff6f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5a10005cd0 tx=0x7f5a1001a040 comp rx=0 tx=0).stop 2023-09-30T14:47:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.809+0000 7f5a2df41b80 1 -- 172.21.15.32:0/898619150 shutdown_connections 2023-09-30T14:47:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.809+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/898619150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ffcd0 0x558b5f5061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.809+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/898619150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4fe200 0x558b5f4ff6f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.809+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/898619150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4ff310 0x558b5f50f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.809+0000 7f5a2df41b80 1 -- 172.21.15.32:0/898619150 >> 172.21.15.32:0/898619150 conn(0x558b5f472200 msgr2=0x558b5f506c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.809+0000 7f5a2df41b80 1 -- 172.21.15.32:0/898619150 shutdown_connections 2023-09-30T14:47:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.809+0000 7f5a2df41b80 1 -- 172.21.15.32:0/898619150 wait complete. 2023-09-30T14:47:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.810+0000 7f5a2df41b80 1 -- start start 2023-09-30T14:47:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.810+0000 7f5a2df41b80 1 Processor -- start 2023-09-30T14:47:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.810+0000 7f5a2df41b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4fe200 0x558b5f50d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.810+0000 7f5a2df41b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ff310 0x558b5f50d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.810+0000 7f5a2df41b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4ffcd0 0x558b5f50e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.810+0000 7f5a2df41b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5f50eb10 con 0x558b5f4ff310 2023-09-30T14:47:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.810+0000 7f5a2df41b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5f50ec80 con 0x558b5f4fe200 2023-09-30T14:47:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.810+0000 7f5a2df41b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5f50ee40 con 0x558b5f4ffcd0 2023-09-30T14:47:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.810+0000 7f5a2df41b80 10 client.0 _ll_register_callbacks cb 0x7f5a1400a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:47:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.810+0000 7f5a1affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ff310 0x558b5f50d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.810+0000 7f5a1affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ff310 0x558b5f50d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37540/0 (socket says 172.21.15.32:37540) 2023-09-30T14:47:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.810+0000 7f5a1affd700 1 -- 172.21.15.32:0/3935789841 learned_addr learned my addr 172.21.15.32:0/3935789841 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.810+0000 7f5a1bfff700 1 --2- 172.21.15.32:0/3935789841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4ffcd0 0x558b5f50e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.810+0000 7f5a1b7fe700 1 --2- 172.21.15.32:0/3935789841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4fe200 0x558b5f50d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f5a1affd700 1 -- 172.21.15.32:0/3935789841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4ffcd0 msgr2=0x558b5f50e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f5a1affd700 1 --2- 172.21.15.32:0/3935789841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4ffcd0 0x558b5f50e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f5a1affd700 1 -- 172.21.15.32:0/3935789841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4fe200 msgr2=0x558b5f50d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f5a1affd700 1 --2- 172.21.15.32:0/3935789841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4fe200 0x558b5f50d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f5a1affd700 1 -- 172.21.15.32:0/3935789841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b5f27bd80 con 0x558b5f4ff310 2023-09-30T14:47:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f5a1b7fe700 1 --2- 172.21.15.32:0/3935789841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4fe200 0x558b5f50d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:47:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f5a1bfff700 1 --2- 172.21.15.32:0/3935789841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4ffcd0 0x558b5f50e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:47:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f5a1affd700 1 --2- 172.21.15.32:0/3935789841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ff310 0x558b5f50d5c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0043c0 tx=0x7f5a0c026630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae1f2cf24911e2e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f59faffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:47:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f59faffd700 1 -- 172.21.15.32:0/3935789841 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c02c040 con 0x558b5f4ff310 2023-09-30T14:47:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f59faffd700 1 -- 172.21.15.32:0/3935789841 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5a0c026a50 con 0x558b5f4ff310 2023-09-30T14:47:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f5a2df41b80 1 -- 172.21.15.32:0/3935789841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a0c003e70 con 0x558b5f4ff310 2023-09-30T14:47:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f59faffd700 1 -- 172.21.15.32:0/3935789841 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a0c026c10 con 0x558b5f4ff310 2023-09-30T14:47:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f5a2df41b80 10 client.16269 fetch_fsmap learned FSMap version 41 2023-09-30T14:47:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f5a2df41b80 1 -- 172.21.15.32:0/3935789841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f5a1c1a16d0 con 0x558b5f4ff310 2023-09-30T14:47:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f59faffd700 1 -- 172.21.15.32:0/3935789841 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a0c003e70 con 0x558b5f4ff310 2023-09-30T14:47:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.811+0000 7f59faffd700 1 -- 172.21.15.32:0/3935789841 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5a0c03b070 con 0x558b5f4ff310 2023-09-30T14:47:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.812+0000 7f5a2df41b80 10 client.16269 fetch_fsmap finished waiting for FSMap version 41 2023-09-30T14:47:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.812+0000 7f5a2df41b80 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:47:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.812+0000 7f5a2df41b80 20 client.16269 populate_metadata read hostname 'smithi032' 2023-09-30T14:47:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.812+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/3935789841 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558b5f568a70 0x558b5f56ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.812+0000 7f5a2df41b80 4 client.16269 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:47:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.812+0000 7f5a2df41b80 1 -- 172.21.15.32:0/3935789841 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558b5f449ad0 con 0x558b5f568a70 2023-09-30T14:47:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.812+0000 7f5a1b7fe700 1 --2- 172.21.15.32:0/3935789841 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558b5f568a70 0x558b5f56ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.812+0000 7f5a1b7fe700 1 --2- 172.21.15.32:0/3935789841 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558b5f568a70 0x558b5f56ae90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6942a4ab8789104a server_cookie=da3f88919943a090 in_seq=0 out_seq=0 2023-09-30T14:47:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.812+0000 7f59faffd700 10 client.16269 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.860+0000 7f59faffd700 1 -- 172.21.15.32:0/3935789841 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558b5f449ad0 con 0x558b5f568a70 2023-09-30T14:47:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.860+0000 7f59faffd700 10 client.16269 handle_command_reply: tid=0 2023-09-30T14:47:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.860+0000 7f5a2df41b80 1 client.16269 shutdown 2023-09-30T14:47:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a2df41b80 1 -- 172.21.15.32:0/3935789841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ff310 msgr2=0x558b5f50d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/3935789841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ff310 0x558b5f50d5c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0043c0 tx=0x7f5a0c026630 comp rx=0 tx=0).stop 2023-09-30T14:47:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a2df41b80 1 -- 172.21.15.32:0/3935789841 shutdown_connections 2023-09-30T14:47:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/3935789841 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558b5f568a70 0x558b5f56ae90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/3935789841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558b5f4ffcd0 0x558b5f50e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/3935789841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b5f4fe200 0x558b5f50d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a2df41b80 1 --2- 172.21.15.32:0/3935789841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558b5f4ff310 0x558b5f50d5c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f59faffd700 0 client.16269 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a2df41b80 1 -- 172.21.15.32:0/3935789841 >> 172.21.15.32:0/3935789841 conn(0x558b5f472200 msgr2=0x558b5f5061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a2df41b80 1 -- 172.21.15.32:0/3935789841 shutdown_connections 2023-09-30T14:47:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a2df41b80 1 -- 172.21.15.32:0/3935789841 wait complete. 2023-09-30T14:47:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a2df41b80 20 client.16269 trim_cache size 0 max 16384 2023-09-30T14:47:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:47:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:47:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:47:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:47:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a21d59700 1 -- 172.21.15.32:0/3099443424 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5a040a0fa0 msgr2=0x7f5a040a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a21d59700 1 --2- 172.21.15.32:0/3099443424 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5a040a0fa0 0x7f5a040a3460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c011490 tx=0x7f5a0c009260 comp rx=0 tx=0).stop 2023-09-30T14:47:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a21d59700 1 -- 172.21.15.32:0/3099443424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0ffd70 msgr2=0x7f5a1c198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a21d59700 1 --2- 172.21.15.32:0/3099443424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0ffd70 0x7f5a1c198ee0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009990 tx=0x7f5a1000f690 comp rx=0 tx=0).stop 2023-09-30T14:47:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a21d59700 1 -- 172.21.15.32:0/3099443424 shutdown_connections 2023-09-30T14:47:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a21d59700 1 --2- 172.21.15.32:0/3099443424 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f5a040a0fa0 0x7f5a040a3460 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a21d59700 1 --2- 172.21.15.32:0/3099443424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1c101900 0x7f5a1c19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a21d59700 1 --2- 172.21.15.32:0/3099443424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a1c0ffd70 0x7f5a1c198ee0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a21d59700 1 --2- 172.21.15.32:0/3099443424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a1c100f60 0x7f5a1c199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.861+0000 7f5a21d59700 1 -- 172.21.15.32:0/3099443424 >> 172.21.15.32:0/3099443424 conn(0x7f5a1c0fb2a0 msgr2=0x7f5a1c068ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.862+0000 7f5a21d59700 1 -- 172.21.15.32:0/3099443424 shutdown_connections 2023-09-30T14:47:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:18.862+0000 7f5a21d59700 1 -- 172.21.15.32:0/3099443424 wait complete. 2023-09-30T14:47:19.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:19 smithi156 ceph-mon[103579]: pgmap v558: 97 pgs: 97 active+clean; 286 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 2.7 MiB/s wr, 249 op/s 2023-09-30T14:47:19.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:19 smithi148 ceph-mon[102355]: pgmap v558: 97 pgs: 97 active+clean; 286 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 2.7 MiB/s wr, 249 op/s 2023-09-30T14:47:19.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:19 smithi032 ceph-mon[98592]: pgmap v558: 97 pgs: 97 active+clean; 286 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 2.7 MiB/s wr, 249 op/s 2023-09-30T14:47:19.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 963393 10.63 MB/sec execute 353 sec latency 51.215 ms 2023-09-30T14:47:20.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 966020 10.64 MB/sec execute 354 sec latency 13.705 ms 2023-09-30T14:47:21.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:21 smithi156 ceph-mon[103579]: pgmap v559: 97 pgs: 97 active+clean; 288 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 3.0 MiB/s wr, 301 op/s 2023-09-30T14:47:21.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:21 smithi148 ceph-mon[102355]: pgmap v559: 97 pgs: 97 active+clean; 288 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 3.0 MiB/s wr, 301 op/s 2023-09-30T14:47:21.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:21 smithi032 ceph-mon[98592]: pgmap v559: 97 pgs: 97 active+clean; 288 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 3.0 MiB/s wr, 301 op/s 2023-09-30T14:47:21.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 967657 10.63 MB/sec execute 355 sec latency 14.611 ms 2023-09-30T14:47:22.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 970190 10.65 MB/sec execute 356 sec latency 11.529 ms 2023-09-30T14:47:23.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:23 smithi156 ceph-mon[103579]: pgmap v560: 97 pgs: 97 active+clean; 288 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 2.8 MiB/s wr, 277 op/s 2023-09-30T14:47:23.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:23 smithi148 ceph-mon[102355]: pgmap v560: 97 pgs: 97 active+clean; 288 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 2.8 MiB/s wr, 277 op/s 2023-09-30T14:47:23.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:23 smithi032 ceph-mon[98592]: pgmap v560: 97 pgs: 97 active+clean; 288 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 2.8 MiB/s wr, 277 op/s 2023-09-30T14:47:23.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 972015 10.63 MB/sec execute 357 sec latency 67.285 ms 2023-09-30T14:47:24.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 973903 10.64 MB/sec execute 358 sec latency 30.311 ms 2023-09-30T14:47:25.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:25 smithi156 ceph-mon[103579]: pgmap v561: 97 pgs: 97 active+clean; 292 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 2.9 MiB/s wr, 261 op/s 2023-09-30T14:47:25.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:25 smithi148 ceph-mon[102355]: pgmap v561: 97 pgs: 97 active+clean; 292 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 2.9 MiB/s wr, 261 op/s 2023-09-30T14:47:25.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:25 smithi032 ceph-mon[98592]: pgmap v561: 97 pgs: 97 active+clean; 292 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 2.9 MiB/s wr, 261 op/s 2023-09-30T14:47:25.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 976344 10.64 MB/sec execute 359 sec latency 17.094 ms 2023-09-30T14:47:26.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 977751 10.64 MB/sec execute 360 sec latency 19.416 ms 2023-09-30T14:47:27.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:27 smithi156 ceph-mon[103579]: pgmap v562: 97 pgs: 97 active+clean; 302 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.8 MiB/s wr, 339 op/s 2023-09-30T14:47:27.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:27 smithi148 ceph-mon[102355]: pgmap v562: 97 pgs: 97 active+clean; 302 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.8 MiB/s wr, 339 op/s 2023-09-30T14:47:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:27 smithi032 ceph-mon[98592]: pgmap v562: 97 pgs: 97 active+clean; 302 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.8 MiB/s wr, 339 op/s 2023-09-30T14:47:27.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 980549 10.65 MB/sec execute 361 sec latency 11.986 ms 2023-09-30T14:47:28.714 INFO:tasks.workunit.client.0.smithi032.stdout: 1 981741 10.64 MB/sec execute 362 sec latency 19.471 ms 2023-09-30T14:47:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:29 smithi156 ceph-mon[103579]: pgmap v563: 97 pgs: 97 active+clean; 311 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 4.3 MiB/s wr, 267 op/s 2023-09-30T14:47:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:29 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:47:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:29 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:47:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:29 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:47:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:29 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:47:29.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:29 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:47:29.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:29 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:47:29.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:29 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:47:29.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:29 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:47:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:29 smithi148 ceph-mon[102355]: pgmap v563: 97 pgs: 97 active+clean; 311 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 4.3 MiB/s wr, 267 op/s 2023-09-30T14:47:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:29 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:47:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:29 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:47:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:29 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:47:29.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:29 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:47:29.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:29 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:47:29.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:29 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:47:29.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:29 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:47:29.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:29 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:47:29.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:29 smithi032 ceph-mon[98592]: pgmap v563: 97 pgs: 97 active+clean; 311 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 4.3 MiB/s wr, 267 op/s 2023-09-30T14:47:29.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:29 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:47:29.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:29 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:47:29.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:29 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:47:29.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:29 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:47:29.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:29 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:47:29.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:29 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:47:29.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:29 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:47:29.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:29 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:47:29.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 984264 10.65 MB/sec execute 363 sec latency 37.239 ms 2023-09-30T14:47:30.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:30 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-09-30T14:47:30.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:30 smithi156 ceph-mon[103579]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-30T14:47:30.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:30 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-09-30T14:47:30.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:30 smithi148 ceph-mon[102355]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-30T14:47:30.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:30 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-09-30T14:47:30.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:30 smithi032 ceph-mon[98592]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-30T14:47:30.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 985982 10.64 MB/sec execute 364 sec latency 15.189 ms 2023-09-30T14:47:31.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:31 smithi156 ceph-mon[103579]: pgmap v564: 97 pgs: 97 active+clean; 313 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 4.1 MiB/s wr, 282 op/s 2023-09-30T14:47:31.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:31 smithi148 ceph-mon[102355]: pgmap v564: 97 pgs: 97 active+clean; 313 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 4.1 MiB/s wr, 282 op/s 2023-09-30T14:47:31.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:31 smithi032 ceph-mon[98592]: pgmap v564: 97 pgs: 97 active+clean; 313 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 4.1 MiB/s wr, 282 op/s 2023-09-30T14:47:31.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 988052 10.65 MB/sec execute 365 sec latency 22.932 ms 2023-09-30T14:47:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.982+0000 7f70209a3700 1 -- 172.21.15.32:0/3794186675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f701c100e60 msgr2=0x7f701c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.982+0000 7f70209a3700 1 --2- 172.21.15.32:0/3794186675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f701c100e60 0x7f701c101280 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f700c009990 tx=0x7f700c009ca0 comp rx=0 tx=0).stop 2023-09-30T14:47:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.983+0000 7f70209a3700 1 -- 172.21.15.32:0/3794186675 shutdown_connections 2023-09-30T14:47:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.983+0000 7f70209a3700 1 --2- 172.21.15.32:0/3794186675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c102970 0x7f701c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.983+0000 7f70209a3700 1 --2- 172.21.15.32:0/3794186675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f701c100e60 0x7f701c101280 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.983+0000 7f70209a3700 1 --2- 172.21.15.32:0/3794186675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f701c101fb0 0x7f701c102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.983+0000 7f70209a3700 1 -- 172.21.15.32:0/3794186675 >> 172.21.15.32:0/3794186675 conn(0x7f701c0fc3d0 msgr2=0x7f701c0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.983+0000 7f70209a3700 1 -- 172.21.15.32:0/3794186675 shutdown_connections 2023-09-30T14:47:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.983+0000 7f70209a3700 1 -- 172.21.15.32:0/3794186675 wait complete. 2023-09-30T14:47:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.983+0000 7f70209a3700 1 Processor -- start 2023-09-30T14:47:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f70209a3700 1 -- start start 2023-09-30T14:47:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f70209a3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f701c100e60 0x7f701c073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f70209a3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c101fb0 0x7f701c073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f70209a3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f701c102970 0x7f701c073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f70209a3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c1a0c20 con 0x7f701c102970 2023-09-30T14:47:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f70209a3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c1a0d60 con 0x7f701c101fb0 2023-09-30T14:47:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f70209a3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c1a0ea0 con 0x7f701c100e60 2023-09-30T14:47:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f701a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f701c100e60 0x7f701c073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f701a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f701c100e60 0x7f701c073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51934/0 (socket says 172.21.15.32:51934) 2023-09-30T14:47:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f701a59c700 1 -- 172.21.15.32:0/4218115205 learned_addr learned my addr 172.21.15.32:0/4218115205 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f701ad9d700 1 --2- 172.21.15.32:0/4218115205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f701c102970 0x7f701c073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f7019d9b700 1 --2- 172.21.15.32:0/4218115205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c101fb0 0x7f701c073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f701ad9d700 1 -- 172.21.15.32:0/4218115205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f701c100e60 msgr2=0x7f701c073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f701ad9d700 1 --2- 172.21.15.32:0/4218115205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f701c100e60 0x7f701c073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f701ad9d700 1 -- 172.21.15.32:0/4218115205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c101fb0 msgr2=0x7f701c073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f701ad9d700 1 --2- 172.21.15.32:0/4218115205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c101fb0 0x7f701c073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.984+0000 7f701ad9d700 1 -- 172.21.15.32:0/4218115205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7010009520 con 0x7f701c102970 2023-09-30T14:47:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.985+0000 7f701a59c700 1 --2- 172.21.15.32:0/4218115205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f701c100e60 0x7f701c073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:47:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.985+0000 7f701ad9d700 1 --2- 172.21.15.32:0/4218115205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f701c102970 0x7f701c073c90 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f701400efc0 tx=0x7f701400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.985+0000 7f700b7fe700 1 -- 172.21.15.32:0/4218115205 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7014018070 con 0x7f701c102970 2023-09-30T14:47:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.985+0000 7f700b7fe700 1 -- 172.21.15.32:0/4218115205 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f701400f040 con 0x7f701c102970 2023-09-30T14:47:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.985+0000 7f70209a3700 1 -- 172.21.15.32:0/4218115205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c0095f0 con 0x7f701c102970 2023-09-30T14:47:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.985+0000 7f700b7fe700 1 -- 172.21.15.32:0/4218115205 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701400cd40 con 0x7f701c102970 2023-09-30T14:47:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.985+0000 7f70209a3700 1 -- 172.21.15.32:0/4218115205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c1a1400 con 0x7f701c102970 2023-09-30T14:47:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.988+0000 7f700b7fe700 1 -- 172.21.15.32:0/4218115205 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f7014003a00 con 0x7f701c102970 2023-09-30T14:47:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.989+0000 7f700b7fe700 1 --2- 172.21.15.32:0/4218115205 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f70040a0fa0 0x7f70040a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.989+0000 7f701a59c700 1 --2- 172.21.15.32:0/4218115205 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f70040a0fa0 0x7f70040a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.989+0000 7f700b7fe700 1 -- 172.21.15.32:0/4218115205 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f70140cc370 con 0x7f701c102970 2023-09-30T14:47:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.989+0000 7f702cb8bb80 1 Processor -- start 2023-09-30T14:47:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.989+0000 7f702cb8bb80 1 -- start start 2023-09-30T14:47:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f702cb8bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab30200 0x55ba8ab30620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f702cb8bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab31310 0x55ba8ab31790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f702cb8bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab38560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f702cb8bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8aa98400 con 0x55ba8ab31cd0 2023-09-30T14:47:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f702cb8bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8aa89bc0 con 0x55ba8ab30200 2023-09-30T14:47:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f702cb8bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8aa874e0 con 0x55ba8ab31310 2023-09-30T14:47:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f701ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab38560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f701a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab30200 0x55ba8ab30620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f701ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab38560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37562/0 (socket says 172.21.15.32:37562) 2023-09-30T14:47:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f701ad9d700 1 -- 172.21.15.32:0/1331407840 learned_addr learned my addr 172.21.15.32:0/1331407840 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f7019d9b700 1 --2- 172.21.15.32:0/1331407840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab31310 0x55ba8ab31790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f701a59c700 1 --2- 172.21.15.32:0/4218115205 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f70040a0fa0 0x7f70040a3460 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f700c00b2e0 tx=0x7f700c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f701ad9d700 1 -- 172.21.15.32:0/1331407840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab31310 msgr2=0x55ba8ab31790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f701ad9d700 1 --2- 172.21.15.32:0/1331407840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab31310 0x55ba8ab31790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f701ad9d700 1 -- 172.21.15.32:0/1331407840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab30200 msgr2=0x55ba8ab30620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f701ad9d700 1 --2- 172.21.15.32:0/1331407840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab30200 0x55ba8ab30620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f701ad9d700 1 -- 172.21.15.32:0/1331407840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba8aae5870 con 0x55ba8ab31cd0 2023-09-30T14:47:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.990+0000 7f701ad9d700 1 --2- 172.21.15.32:0/1331407840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab38560 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f70140a1640 tx=0x7f7014095cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d327d49e6b6cda5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.991+0000 7f7008ff9700 1 -- 172.21.15.32:0/1331407840 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7014082ea0 con 0x55ba8ab31cd0 2023-09-30T14:47:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.991+0000 7f7008ff9700 1 -- 172.21.15.32:0/1331407840 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70140991d0 con 0x55ba8ab31cd0 2023-09-30T14:47:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.991+0000 7f7008ff9700 1 -- 172.21.15.32:0/1331407840 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701401b320 con 0x55ba8ab31cd0 2023-09-30T14:47:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.991+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1331407840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 msgr2=0x55ba8ab38560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.991+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/1331407840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab38560 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f70140a1640 tx=0x7f7014095cb0 comp rx=0 tx=0).stop 2023-09-30T14:47:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.991+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1331407840 shutdown_connections 2023-09-30T14:47:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.991+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/1331407840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab38560 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.991+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/1331407840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab30200 0x55ba8ab30620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.991+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/1331407840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab31310 0x55ba8ab31790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.991+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1331407840 >> 172.21.15.32:0/1331407840 conn(0x55ba8aaa4200 msgr2=0x55ba8ab2aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.991+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1331407840 shutdown_connections 2023-09-30T14:47:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.992+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1331407840 wait complete. 2023-09-30T14:47:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.992+0000 7f702cb8bb80 1 -- start start 2023-09-30T14:47:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.992+0000 7f702cb8bb80 1 Processor -- start 2023-09-30T14:47:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.992+0000 7f702cb8bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab30200 0x55ba8ab39f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.992+0000 7f702cb8bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab31310 0x55ba8ab3a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.992+0000 7f702cb8bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab41480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.992+0000 7f702cb8bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8aa82dd0 con 0x55ba8ab30200 2023-09-30T14:47:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.992+0000 7f702cb8bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8aa1f6b0 con 0x55ba8ab31cd0 2023-09-30T14:47:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.992+0000 7f702cb8bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8aa1f350 con 0x55ba8ab31310 2023-09-30T14:47:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.992+0000 7f702cb8bb80 10 client.0 _ll_register_callbacks cb 0x55ba8a87c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:47:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.992+0000 7f701a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab30200 0x55ba8ab39f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.992+0000 7f701a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab30200 0x55ba8ab39f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37566/0 (socket says 172.21.15.32:37566) 2023-09-30T14:47:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.992+0000 7f701a59c700 1 -- 172.21.15.32:0/1298228399 learned_addr learned my addr 172.21.15.32:0/1298228399 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.993+0000 7f7019d9b700 1 --2- 172.21.15.32:0/1298228399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab31310 0x55ba8ab3a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.993+0000 7f701ad9d700 1 --2- 172.21.15.32:0/1298228399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab41480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.993+0000 7f701a59c700 1 -- 172.21.15.32:0/1298228399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab31310 msgr2=0x55ba8ab3a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.993+0000 7f701a59c700 1 --2- 172.21.15.32:0/1298228399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab31310 0x55ba8ab3a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.993+0000 7f701a59c700 1 -- 172.21.15.32:0/1298228399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab31cd0 msgr2=0x55ba8ab41480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.993+0000 7f701a59c700 1 --2- 172.21.15.32:0/1298228399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab41480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.993+0000 7f701a59c700 1 -- 172.21.15.32:0/1298228399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c1a1400 con 0x55ba8ab30200 2023-09-30T14:47:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.993+0000 7f701ad9d700 1 --2- 172.21.15.32:0/1298228399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab41480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:47:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.993+0000 7f7019d9b700 1 --2- 172.21.15.32:0/1298228399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab31310 0x55ba8ab3a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:47:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.993+0000 7f701a59c700 1 --2- 172.21.15.32:0/1298228399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab30200 0x55ba8ab39f00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f700c025c80 tx=0x7f700c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8b5e419d85725f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.993+0000 7f7001ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:47:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.993+0000 7f7001ffb700 1 -- 172.21.15.32:0/1298228399 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c00fe80 con 0x55ba8ab30200 2023-09-30T14:47:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.993+0000 7f7001ffb700 1 -- 172.21.15.32:0/1298228399 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f700c005050 con 0x55ba8ab30200 2023-09-30T14:47:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.993+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1298228399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba8ab23090 con 0x55ba8ab30200 2023-09-30T14:47:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.994+0000 7f7001ffb700 1 -- 172.21.15.32:0/1298228399 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f700c005210 con 0x55ba8ab30200 2023-09-30T14:47:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.994+0000 7f702cb8bb80 10 client.16281 fetch_fsmap learned FSMap version 41 2023-09-30T14:47:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.994+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1298228399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ba8a8add80 con 0x55ba8ab30200 2023-09-30T14:47:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.994+0000 7f7001ffb700 1 -- 172.21.15.32:0/1298228399 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba8ab23090 con 0x55ba8ab30200 2023-09-30T14:47:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.994+0000 7f7001ffb700 1 -- 172.21.15.32:0/1298228399 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f700c030400 con 0x55ba8ab30200 2023-09-30T14:47:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.994+0000 7f702cb8bb80 10 client.16281 fetch_fsmap finished waiting for FSMap version 41 2023-09-30T14:47:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.994+0000 7f702cb8bb80 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:47:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.994+0000 7f702cb8bb80 20 client.16281 populate_metadata read hostname 'smithi032' 2023-09-30T14:47:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.994+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/1298228399 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ba8ab45d50 0x55ba8ab48170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.994+0000 7f702cb8bb80 4 client.16281 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:47:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.994+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1298228399 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba8aa7bdb0 con 0x55ba8ab45d50 2023-09-30T14:47:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.994+0000 7f7019d9b700 1 --2- 172.21.15.32:0/1298228399 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ba8ab45d50 0x55ba8ab48170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.994+0000 7f7019d9b700 1 --2- 172.21.15.32:0/1298228399 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ba8ab45d50 0x55ba8ab48170 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a2448899f9cf8f4 server_cookie=587a0ffb8977b28d in_seq=0 out_seq=0 2023-09-30T14:47:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.994+0000 7f7001ffb700 10 client.16281 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.995+0000 7f7001ffb700 1 -- 172.21.15.32:0/1298228399 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ba8aa7bdb0 con 0x55ba8ab45d50 2023-09-30T14:47:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.995+0000 7f7001ffb700 10 client.16281 handle_command_reply: tid=0 2023-09-30T14:47:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.995+0000 7f702cb8bb80 1 client.16281 shutdown 2023-09-30T14:47:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.995+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1298228399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab30200 msgr2=0x55ba8ab39f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.995+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/1298228399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab30200 0x55ba8ab39f00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f700c025c80 tx=0x7f700c01a050 comp rx=0 tx=0).stop 2023-09-30T14:47:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.995+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1298228399 shutdown_connections 2023-09-30T14:47:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.995+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/1298228399 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ba8ab45d50 0x55ba8ab48170 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.995+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/1298228399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab41480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.995+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/1298228399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab30200 0x55ba8ab39f00 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.995+0000 7f7001ffb700 0 client.16281 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.995+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/1298228399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab31310 0x55ba8ab3a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.995+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1298228399 >> 172.21.15.32:0/1298228399 conn(0x55ba8aaa4200 msgr2=0x55ba8ab2a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.995+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1298228399 shutdown_connections 2023-09-30T14:47:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.995+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1298228399 wait complete. 2023-09-30T14:47:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:31.995+0000 7f702cb8bb80 20 client.16281 trim_cache size 0 max 16384 2023-09-30T14:47:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.011+0000 7f702cb8bb80 1 Processor -- start 2023-09-30T14:47:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.011+0000 7f702cb8bb80 1 -- start start 2023-09-30T14:47:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f702cb8bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab30200 0x55ba8ab316f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f702cb8bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab31310 0x55ba8ab41440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f702cb8bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab38c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f702cb8bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8a9038c0 con 0x55ba8ab31cd0 2023-09-30T14:47:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f702cb8bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8ab2c9e0 con 0x55ba8ab30200 2023-09-30T14:47:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f702cb8bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8ab310e0 con 0x55ba8ab31310 2023-09-30T14:47:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f701ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab38c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f701a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab30200 0x55ba8ab316f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f701ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab38c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37578/0 (socket says 172.21.15.32:37578) 2023-09-30T14:47:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f7019d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab31310 0x55ba8ab41440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f701a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab30200 0x55ba8ab316f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49050/0 (socket says 172.21.15.32:49050) 2023-09-30T14:47:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f701ad9d700 1 -- 172.21.15.32:0/1951395324 learned_addr learned my addr 172.21.15.32:0/1951395324 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f701a59c700 1 -- 172.21.15.32:0/1951395324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab31310 msgr2=0x55ba8ab41440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f701a59c700 1 --2- 172.21.15.32:0/1951395324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab31310 0x55ba8ab41440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f701a59c700 1 -- 172.21.15.32:0/1951395324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 msgr2=0x55ba8ab38c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f701a59c700 1 --2- 172.21.15.32:0/1951395324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab38c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f701a59c700 1 -- 172.21.15.32:0/1951395324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba8a896c00 con 0x55ba8ab30200 2023-09-30T14:47:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f701a59c700 1 --2- 172.21.15.32:0/1951395324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab30200 0x55ba8ab316f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f700c000c00 tx=0x7f700c004500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=300608d17842759e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.012+0000 7f70097fa700 1 -- 172.21.15.32:0/1951395324 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c0309a0 con 0x55ba8ab30200 2023-09-30T14:47:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.013+0000 7f70097fa700 1 -- 172.21.15.32:0/1951395324 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f700c01e040 con 0x55ba8ab30200 2023-09-30T14:47:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.013+0000 7f70097fa700 1 -- 172.21.15.32:0/1951395324 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c03cb90 con 0x55ba8ab30200 2023-09-30T14:47:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.013+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1951395324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab30200 msgr2=0x55ba8ab316f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.013+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/1951395324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab30200 0x55ba8ab316f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f700c000c00 tx=0x7f700c004500 comp rx=0 tx=0).stop 2023-09-30T14:47:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.013+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1951395324 shutdown_connections 2023-09-30T14:47:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.013+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/1951395324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab38c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.013+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/1951395324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab30200 0x55ba8ab316f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.013+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/1951395324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab31310 0x55ba8ab41440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.013+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1951395324 >> 172.21.15.32:0/1951395324 conn(0x55ba8aaa4200 msgr2=0x55ba8ab2a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.013+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1951395324 shutdown_connections 2023-09-30T14:47:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.013+0000 7f702cb8bb80 1 -- 172.21.15.32:0/1951395324 wait complete. 2023-09-30T14:47:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f702cb8bb80 1 -- start start 2023-09-30T14:47:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f702cb8bb80 1 Processor -- start 2023-09-30T14:47:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f702cb8bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab30200 0x55ba8ab3ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f702cb8bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab31310 0x55ba8ab3f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f702cb8bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab40260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f702cb8bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8ab407a0 con 0x55ba8ab31cd0 2023-09-30T14:47:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f702cb8bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8ab40910 con 0x55ba8ab31310 2023-09-30T14:47:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f702cb8bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8ab40ad0 con 0x55ba8ab30200 2023-09-30T14:47:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f702cb8bb80 10 client.0 _ll_register_callbacks cb 0x7f700c025830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:47:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f701ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab40260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f701a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab30200 0x55ba8ab3ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f701ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab40260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37584/0 (socket says 172.21.15.32:37584) 2023-09-30T14:47:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f701ad9d700 1 -- 172.21.15.32:0/2023846033 learned_addr learned my addr 172.21.15.32:0/2023846033 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f701a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab30200 0x55ba8ab3ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:51962/0 (socket says 172.21.15.32:51962) 2023-09-30T14:47:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f7019d9b700 1 --2- 172.21.15.32:0/2023846033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab31310 0x55ba8ab3f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f701a59c700 1 -- 172.21.15.32:0/2023846033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab31310 msgr2=0x55ba8ab3f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f701a59c700 1 --2- 172.21.15.32:0/2023846033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab31310 0x55ba8ab3f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f701a59c700 1 -- 172.21.15.32:0/2023846033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 msgr2=0x55ba8ab40260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f701a59c700 1 --2- 172.21.15.32:0/2023846033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab40260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f701a59c700 1 -- 172.21.15.32:0/2023846033 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba8a8add80 con 0x55ba8ab30200 2023-09-30T14:47:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f7019d9b700 1 --2- 172.21.15.32:0/2023846033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab31310 0x55ba8ab3f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:47:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.014+0000 7f701ad9d700 1 --2- 172.21.15.32:0/2023846033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab40260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:47:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.015+0000 7f701a59c700 1 --2- 172.21.15.32:0/2023846033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab30200 0x55ba8ab3ed10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f700c0307b0 tx=0x7f700c03c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edad09b8c001fec3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.015+0000 7f7001ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:47:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.015+0000 7f7001ffb700 1 -- 172.21.15.32:0/2023846033 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c030b50 con 0x55ba8ab30200 2023-09-30T14:47:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.015+0000 7f702cb8bb80 1 -- 172.21.15.32:0/2023846033 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70140991d0 con 0x55ba8ab30200 2023-09-30T14:47:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.015+0000 7f7001ffb700 1 -- 172.21.15.32:0/2023846033 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f700c00fad0 con 0x55ba8ab30200 2023-09-30T14:47:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.015+0000 7f7001ffb700 1 -- 172.21.15.32:0/2023846033 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f700c00fc90 con 0x55ba8ab30200 2023-09-30T14:47:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.015+0000 7f702cb8bb80 10 client.25957 fetch_fsmap learned FSMap version 41 2023-09-30T14:47:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.015+0000 7f702cb8bb80 1 -- 172.21.15.32:0/2023846033 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ba8a8d41f0 con 0x55ba8ab30200 2023-09-30T14:47:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.015+0000 7f7001ffb700 1 -- 172.21.15.32:0/2023846033 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70140991d0 con 0x55ba8ab30200 2023-09-30T14:47:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.015+0000 7f7001ffb700 1 -- 172.21.15.32:0/2023846033 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f700c04d770 con 0x55ba8ab30200 2023-09-30T14:47:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.015+0000 7f702cb8bb80 10 client.25957 fetch_fsmap finished waiting for FSMap version 41 2023-09-30T14:47:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.015+0000 7f702cb8bb80 10 client.25957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:47:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.016+0000 7f702cb8bb80 20 client.25957 populate_metadata read hostname 'smithi032' 2023-09-30T14:47:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.016+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/2023846033 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ba8ab9a240 0x55ba8ab9c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.016+0000 7f702cb8bb80 4 client.25957 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:47:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.016+0000 7f702cb8bb80 1 -- 172.21.15.32:0/2023846033 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ba8aa7bad0 con 0x55ba8ab9a240 2023-09-30T14:47:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.016+0000 7f7019d9b700 1 --2- 172.21.15.32:0/2023846033 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ba8ab9a240 0x55ba8ab9c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.016+0000 7f7019d9b700 1 --2- 172.21.15.32:0/2023846033 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ba8ab9a240 0x55ba8ab9c630 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b380094c67e30b5 server_cookie=84130b486c496dd4 in_seq=0 out_seq=0 2023-09-30T14:47:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.016+0000 7f7001ffb700 10 client.25957 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.059+0000 7f7001ffb700 1 -- 172.21.15.32:0/2023846033 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ba8aa7bad0 con 0x55ba8ab9a240 2023-09-30T14:47:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.059+0000 7f7001ffb700 10 client.25957 handle_command_reply: tid=0 2023-09-30T14:47:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.059+0000 7f702cb8bb80 1 client.25957 shutdown 2023-09-30T14:47:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.059+0000 7f702cb8bb80 1 -- 172.21.15.32:0/2023846033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab30200 msgr2=0x55ba8ab3ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.059+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/2023846033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab30200 0x55ba8ab3ed10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f700c0307b0 tx=0x7f700c03c470 comp rx=0 tx=0).stop 2023-09-30T14:47:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.059+0000 7f702cb8bb80 1 -- 172.21.15.32:0/2023846033 shutdown_connections 2023-09-30T14:47:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.059+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/2023846033 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ba8ab9a240 0x55ba8ab9c630 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.059+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/2023846033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ba8ab31cd0 0x55ba8ab40260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.059+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/2023846033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba8ab30200 0x55ba8ab3ed10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.059+0000 7f7001ffb700 0 client.25957 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.059+0000 7f702cb8bb80 1 --2- 172.21.15.32:0/2023846033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba8ab31310 0x55ba8ab3f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.059+0000 7f702cb8bb80 1 -- 172.21.15.32:0/2023846033 >> 172.21.15.32:0/2023846033 conn(0x55ba8aaa4200 msgr2=0x55ba8ab39cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.060+0000 7f702cb8bb80 1 -- 172.21.15.32:0/2023846033 shutdown_connections 2023-09-30T14:47:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.060+0000 7f702cb8bb80 1 -- 172.21.15.32:0/2023846033 wait complete. 2023-09-30T14:47:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.060+0000 7f702cb8bb80 20 client.25957 trim_cache size 0 max 16384 2023-09-30T14:47:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:47:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:47:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:47:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:47:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.060+0000 7f70209a3700 1 -- 172.21.15.32:0/4218115205 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f70040a0fa0 msgr2=0x7f70040a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.060+0000 7f70209a3700 1 --2- 172.21.15.32:0/4218115205 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f70040a0fa0 0x7f70040a3460 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f700c00b2e0 tx=0x7f700c005fb0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.060+0000 7f70209a3700 1 -- 172.21.15.32:0/4218115205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f701c102970 msgr2=0x7f701c073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.060+0000 7f70209a3700 1 --2- 172.21.15.32:0/4218115205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f701c102970 0x7f701c073c90 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f701400efc0 tx=0x7f701400c3a0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.061+0000 7f70209a3700 1 -- 172.21.15.32:0/4218115205 shutdown_connections 2023-09-30T14:47:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.061+0000 7f70209a3700 1 --2- 172.21.15.32:0/4218115205 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f70040a0fa0 0x7f70040a3460 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.061+0000 7f70209a3700 1 --2- 172.21.15.32:0/4218115205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f701c102970 0x7f701c073c90 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.061+0000 7f70209a3700 1 --2- 172.21.15.32:0/4218115205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f701c100e60 0x7f701c073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.061+0000 7f70209a3700 1 --2- 172.21.15.32:0/4218115205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c101fb0 0x7f701c073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.061+0000 7f70209a3700 1 -- 172.21.15.32:0/4218115205 >> 172.21.15.32:0/4218115205 conn(0x7f701c0fc3d0 msgr2=0x7f701c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.061+0000 7f70209a3700 1 -- 172.21.15.32:0/4218115205 shutdown_connections 2023-09-30T14:47:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:32.061+0000 7f70209a3700 1 -- 172.21.15.32:0/4218115205 wait complete. 2023-09-30T14:47:32.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 990443 10.65 MB/sec execute 366 sec latency 18.249 ms 2023-09-30T14:47:33.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:33 smithi156 ceph-mon[103579]: pgmap v565: 97 pgs: 97 active+clean; 315 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 4.1 MiB/s wr, 264 op/s 2023-09-30T14:47:33.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:33 smithi148 ceph-mon[102355]: pgmap v565: 97 pgs: 97 active+clean; 315 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 4.1 MiB/s wr, 264 op/s 2023-09-30T14:47:33.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:33 smithi032 ceph-mon[98592]: pgmap v565: 97 pgs: 97 active+clean; 315 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 4.1 MiB/s wr, 264 op/s 2023-09-30T14:47:33.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 991953 10.64 MB/sec execute 367 sec latency 22.265 ms 2023-09-30T14:47:34.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 994442 10.65 MB/sec execute 368 sec latency 16.475 ms 2023-09-30T14:47:35.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:35 smithi156 ceph-mon[103579]: pgmap v566: 97 pgs: 97 active+clean; 317 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 4.1 MiB/s wr, 248 op/s 2023-09-30T14:47:35.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:35 smithi148 ceph-mon[102355]: pgmap v566: 97 pgs: 97 active+clean; 317 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 4.1 MiB/s wr, 248 op/s 2023-09-30T14:47:35.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:35 smithi032 ceph-mon[98592]: pgmap v566: 97 pgs: 97 active+clean; 317 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 4.1 MiB/s wr, 248 op/s 2023-09-30T14:47:35.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 996209 10.65 MB/sec execute 369 sec latency 14.811 ms 2023-09-30T14:47:36.714 INFO:tasks.workunit.client.0.smithi032.stdout: 1 998574 10.66 MB/sec execute 370 sec latency 33.476 ms 2023-09-30T14:47:37.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:37 smithi156 ceph-mon[103579]: pgmap v567: 97 pgs: 97 active+clean; 318 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 3.9 MiB/s wr, 286 op/s 2023-09-30T14:47:37.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:37 smithi148 ceph-mon[102355]: pgmap v567: 97 pgs: 97 active+clean; 318 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 3.9 MiB/s wr, 286 op/s 2023-09-30T14:47:37.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:37 smithi032 ceph-mon[98592]: pgmap v567: 97 pgs: 97 active+clean; 318 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 3.9 MiB/s wr, 286 op/s 2023-09-30T14:47:37.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1000525 10.65 MB/sec execute 371 sec latency 18.785 ms 2023-09-30T14:47:38.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1002284 10.65 MB/sec execute 372 sec latency 48.634 ms 2023-09-30T14:47:39.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:39 smithi156 ceph-mon[103579]: pgmap v568: 97 pgs: 97 active+clean; 320 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 3.0 MiB/s wr, 253 op/s 2023-09-30T14:47:39.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:39 smithi148 ceph-mon[102355]: pgmap v568: 97 pgs: 97 active+clean; 320 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 3.0 MiB/s wr, 253 op/s 2023-09-30T14:47:39.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:39 smithi032 ceph-mon[98592]: pgmap v568: 97 pgs: 97 active+clean; 320 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 3.0 MiB/s wr, 253 op/s 2023-09-30T14:47:39.713 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1004423 10.65 MB/sec execute 373 sec latency 36.923 ms 2023-09-30T14:47:40.714 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1006204 10.65 MB/sec execute 374 sec latency 17.888 ms 2023-09-30T14:47:41.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:41 smithi156 ceph-mon[103579]: pgmap v569: 97 pgs: 97 active+clean; 322 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 2.3 MiB/s wr, 250 op/s 2023-09-30T14:47:41.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:41 smithi148 ceph-mon[102355]: pgmap v569: 97 pgs: 97 active+clean; 322 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 2.3 MiB/s wr, 250 op/s 2023-09-30T14:47:41.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:41 smithi032 ceph-mon[98592]: pgmap v569: 97 pgs: 97 active+clean; 322 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 2.3 MiB/s wr, 250 op/s 2023-09-30T14:47:41.714 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1009089 10.67 MB/sec execute 375 sec latency 9.625 ms 2023-09-30T14:47:42.715 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1010737 10.66 MB/sec execute 376 sec latency 45.705 ms 2023-09-30T14:47:43.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:43 smithi156 ceph-mon[103579]: pgmap v570: 97 pgs: 97 active+clean; 324 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 2.7 MiB/s wr, 272 op/s 2023-09-30T14:47:43.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:43 smithi148 ceph-mon[102355]: pgmap v570: 97 pgs: 97 active+clean; 324 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 2.7 MiB/s wr, 272 op/s 2023-09-30T14:47:43.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:43 smithi032 ceph-mon[98592]: pgmap v570: 97 pgs: 97 active+clean; 324 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 2.7 MiB/s wr, 272 op/s 2023-09-30T14:47:43.715 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1012890 10.66 MB/sec execute 377 sec latency 33.616 ms 2023-09-30T14:47:44.715 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1015265 10.66 MB/sec execute 378 sec latency 29.533 ms 2023-09-30T14:47:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.176+0000 7f19048e9700 1 -- 172.21.15.32:0/2575433070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19000ffd70 msgr2=0x7f1900100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.176+0000 7f19048e9700 1 --2- 172.21.15.32:0/2575433070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19000ffd70 0x7f1900100190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f18f00099c0 tx=0x7f18f0009cd0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.176+0000 7f19048e9700 1 -- 172.21.15.32:0/2575433070 shutdown_connections 2023-09-30T14:47:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.176+0000 7f19048e9700 1 --2- 172.21.15.32:0/2575433070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1900101900 0x7f19001081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.176+0000 7f19048e9700 1 --2- 172.21.15.32:0/2575433070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19000ffd70 0x7f1900100190 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.176+0000 7f19048e9700 1 --2- 172.21.15.32:0/2575433070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1900100f60 0x7f19001013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.176+0000 7f19048e9700 1 -- 172.21.15.32:0/2575433070 >> 172.21.15.32:0/2575433070 conn(0x7f19000fb2e0 msgr2=0x7f19000fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.176+0000 7f19048e9700 1 -- 172.21.15.32:0/2575433070 shutdown_connections 2023-09-30T14:47:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.176+0000 7f19048e9700 1 -- 172.21.15.32:0/2575433070 wait complete. 2023-09-30T14:47:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.177+0000 7f19048e9700 1 Processor -- start 2023-09-30T14:47:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.177+0000 7f19048e9700 1 -- start start 2023-09-30T14:47:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.177+0000 7f19048e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19000ffd70 0x7f1900196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.177+0000 7f19048e9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1900100f60 0x7f1900197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.177+0000 7f19048e9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1900101900 0x7f1900197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.177+0000 7f19048e9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19001986c0 con 0x7f19000ffd70 2023-09-30T14:47:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.177+0000 7f19048e9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1900198830 con 0x7f1900100f60 2023-09-30T14:47:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.177+0000 7f19048e9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19001989f0 con 0x7f1900101900 2023-09-30T14:47:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f18fe59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19000ffd70 0x7f1900196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f18fdd9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1900100f60 0x7f1900197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f18fdd9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1900100f60 0x7f1900197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49074/0 (socket says 172.21.15.32:49074) 2023-09-30T14:47:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f18fed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1900101900 0x7f1900197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f18fe59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19000ffd70 0x7f1900196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37596/0 (socket says 172.21.15.32:37596) 2023-09-30T14:47:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f18fdd9b700 1 -- 172.21.15.32:0/772785546 learned_addr learned my addr 172.21.15.32:0/772785546 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f18fed9d700 1 -- 172.21.15.32:0/772785546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1900100f60 msgr2=0x7f1900197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f18fed9d700 1 --2- 172.21.15.32:0/772785546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1900100f60 0x7f1900197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f18fed9d700 1 -- 172.21.15.32:0/772785546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19000ffd70 msgr2=0x7f1900196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f18fed9d700 1 --2- 172.21.15.32:0/772785546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19000ffd70 0x7f1900196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f18fed9d700 1 -- 172.21.15.32:0/772785546 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f00095f0 con 0x7f1900101900 2023-09-30T14:47:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f18fed9d700 1 --2- 172.21.15.32:0/772785546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1900101900 0x7f1900197fc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f18f800efc0 tx=0x7f18f800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f18f77fe700 1 -- 172.21.15.32:0/772785546 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8018070 con 0x7f1900101900 2023-09-30T14:47:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f19048e9700 1 -- 172.21.15.32:0/772785546 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1900198c70 con 0x7f1900101900 2023-09-30T14:47:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f18f77fe700 1 -- 172.21.15.32:0/772785546 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18f800f040 con 0x7f1900101900 2023-09-30T14:47:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f18f77fe700 1 -- 172.21.15.32:0/772785546 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f800cd40 con 0x7f1900101900 2023-09-30T14:47:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.178+0000 7f19048e9700 1 -- 172.21.15.32:0/772785546 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190019f440 con 0x7f1900101900 2023-09-30T14:47:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.180+0000 7f18f77fe700 1 -- 172.21.15.32:0/772785546 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f18f8003a00 con 0x7f1900101900 2023-09-30T14:47:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.181+0000 7f18f77fe700 1 --2- 172.21.15.32:0/772785546 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f18ec0a1020 0x7f18ec0a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.181+0000 7f18f77fe700 1 -- 172.21.15.32:0/772785546 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f18f80cc230 con 0x7f1900101900 2023-09-30T14:47:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.181+0000 7f18fe59c700 1 --2- 172.21.15.32:0/772785546 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f18ec0a1020 0x7f18ec0a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.182+0000 7f18fe59c700 1 --2- 172.21.15.32:0/772785546 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f18ec0a1020 0x7f18ec0a34e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009990 tx=0x7f18f0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.182+0000 7f1910ad1b80 1 Processor -- start 2023-09-30T14:47:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.182+0000 7f1910ad1b80 1 -- start start 2023-09-30T14:47:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.182+0000 7f1910ad1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9e200 0x55b2cba9e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.182+0000 7f1910ad1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9f310 0x55b2cba9f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.182+0000 7f1910ad1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaa6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.182+0000 7f1910ad1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2cba06400 con 0x55b2cba9f310 2023-09-30T14:47:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.182+0000 7f1910ad1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2cb9f7bc0 con 0x55b2cba9e200 2023-09-30T14:47:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.182+0000 7f1910ad1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2cb9f54e0 con 0x55b2cba9fcd0 2023-09-30T14:47:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.182+0000 7f18fdd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9f310 0x55b2cba9f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.182+0000 7f18fe59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9e200 0x55b2cba9e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.182+0000 7f18fdd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9f310 0x55b2cba9f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37606/0 (socket says 172.21.15.32:37606) 2023-09-30T14:47:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.182+0000 7f18fe59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9e200 0x55b2cba9e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49080/0 (socket says 172.21.15.32:49080) 2023-09-30T14:47:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.182+0000 7f18fdd9b700 1 -- 172.21.15.32:0/1388120912 learned_addr learned my addr 172.21.15.32:0/1388120912 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.182+0000 7f18fed9d700 1 --2- 172.21.15.32:0/1388120912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaa6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.183+0000 7f18fdd9b700 1 -- 172.21.15.32:0/1388120912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9fcd0 msgr2=0x55b2cbaa6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.183+0000 7f18fdd9b700 1 --2- 172.21.15.32:0/1388120912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaa6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.183+0000 7f18fdd9b700 1 -- 172.21.15.32:0/1388120912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9e200 msgr2=0x55b2cba9e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.183+0000 7f18fdd9b700 1 --2- 172.21.15.32:0/1388120912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9e200 0x55b2cba9e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.183+0000 7f18fdd9b700 1 -- 172.21.15.32:0/1388120912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2cba53870 con 0x55b2cba9f310 2023-09-30T14:47:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.183+0000 7f18fe59c700 1 --2- 172.21.15.32:0/1388120912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9e200 0x55b2cba9e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:47:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.183+0000 7f18fdd9b700 1 --2- 172.21.15.32:0/1388120912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9f310 0x55b2cba9f790 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f18e8009e50 tx=0x7f18e8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be65bdbb19829b31 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.183+0000 7f18f4ff9700 1 -- 172.21.15.32:0/1388120912 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e8004a30 con 0x55b2cba9f310 2023-09-30T14:47:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.183+0000 7f18f4ff9700 1 -- 172.21.15.32:0/1388120912 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18e80054e0 con 0x55b2cba9f310 2023-09-30T14:47:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.183+0000 7f18f4ff9700 1 -- 172.21.15.32:0/1388120912 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e80198f0 con 0x55b2cba9f310 2023-09-30T14:47:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.184+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1388120912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9f310 msgr2=0x55b2cba9f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.184+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/1388120912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9f310 0x55b2cba9f790 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f18e8009e50 tx=0x7f18e8011720 comp rx=0 tx=0).stop 2023-09-30T14:47:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.184+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1388120912 shutdown_connections 2023-09-30T14:47:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.184+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/1388120912 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaa6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.184+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/1388120912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9e200 0x55b2cba9e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.184+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/1388120912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9f310 0x55b2cba9f790 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.184+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1388120912 >> 172.21.15.32:0/1388120912 conn(0x55b2cba12200 msgr2=0x55b2cba98ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.184+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1388120912 shutdown_connections 2023-09-30T14:47:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.184+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1388120912 wait complete. 2023-09-30T14:47:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.184+0000 7f1910ad1b80 1 -- start start 2023-09-30T14:47:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.184+0000 7f1910ad1b80 1 Processor -- start 2023-09-30T14:47:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f1910ad1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9e200 0x55b2cbaa7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f1910ad1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9f310 0x55b2cbaa83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f1910ad1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaaf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f1910ad1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2cb9f0dd0 con 0x55b2cba9fcd0 2023-09-30T14:47:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f1910ad1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2cb98d6b0 con 0x55b2cba9e200 2023-09-30T14:47:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f1910ad1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2cb98d350 con 0x55b2cba9f310 2023-09-30T14:47:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f1910ad1b80 10 client.0 _ll_register_callbacks cb 0x55b2cb7ea7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:47:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f18fe59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9e200 0x55b2cbaa7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f18fdd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9f310 0x55b2cbaa83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f18fed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaaf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f18fe59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9e200 0x55b2cbaa7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49086/0 (socket says 172.21.15.32:49086) 2023-09-30T14:47:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f18fe59c700 1 -- 172.21.15.32:0/1434918746 learned_addr learned my addr 172.21.15.32:0/1434918746 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f18fe59c700 1 -- 172.21.15.32:0/1434918746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9f310 msgr2=0x55b2cbaa83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f18fe59c700 1 --2- 172.21.15.32:0/1434918746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9f310 0x55b2cbaa83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f18fe59c700 1 -- 172.21.15.32:0/1434918746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9fcd0 msgr2=0x55b2cbaaf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f18fe59c700 1 --2- 172.21.15.32:0/1434918746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaaf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f18fe59c700 1 -- 172.21.15.32:0/1434918746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1900108b00 con 0x55b2cba9e200 2023-09-30T14:47:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f18fdd9b700 1 --2- 172.21.15.32:0/1434918746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9f310 0x55b2cbaa83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:47:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f18fe59c700 1 --2- 172.21.15.32:0/1434918746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9e200 0x55b2cbaa7e70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1900100dc0 tx=0x7f18f0009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6912c30af93cb634 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.185+0000 7f18e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:47:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.186+0000 7f18e5ffb700 1 -- 172.21.15.32:0/1434918746 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0029050 con 0x55b2cba9e200 2023-09-30T14:47:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.186+0000 7f18e5ffb700 1 -- 172.21.15.32:0/1434918746 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f18f000f460 con 0x55b2cba9e200 2023-09-30T14:47:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.186+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1434918746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2cba91090 con 0x55b2cba9e200 2023-09-30T14:47:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.186+0000 7f18e5ffb700 1 -- 172.21.15.32:0/1434918746 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18f0005720 con 0x55b2cba9e200 2023-09-30T14:47:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.186+0000 7f1910ad1b80 10 client.25985 fetch_fsmap learned FSMap version 41 2023-09-30T14:47:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.186+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1434918746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f18e8009a80 con 0x55b2cba9e200 2023-09-30T14:47:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.186+0000 7f18e5ffb700 1 -- 172.21.15.32:0/1434918746 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b2cba91090 con 0x55b2cba9e200 2023-09-30T14:47:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.186+0000 7f18e5ffb700 1 -- 172.21.15.32:0/1434918746 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f18f0022770 con 0x55b2cba9e200 2023-09-30T14:47:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.186+0000 7f1910ad1b80 10 client.25985 fetch_fsmap finished waiting for FSMap version 41 2023-09-30T14:47:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.186+0000 7f1910ad1b80 10 client.25985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:47:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.186+0000 7f1910ad1b80 20 client.25985 populate_metadata read hostname 'smithi032' 2023-09-30T14:47:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.186+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/1434918746 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b2cbab3d90 0x55b2cbab61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.186+0000 7f1910ad1b80 4 client.25985 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:47:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.186+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1434918746 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2cb9e9db0 con 0x55b2cbab3d90 2023-09-30T14:47:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.186+0000 7f18fdd9b700 1 --2- 172.21.15.32:0/1434918746 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b2cbab3d90 0x55b2cbab61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.187+0000 7f18e5ffb700 10 client.25985 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.187+0000 7f18fdd9b700 1 --2- 172.21.15.32:0/1434918746 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b2cbab3d90 0x55b2cbab61b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f2c8bdeae74a1a0 server_cookie=867f9b082d3a3dd1 in_seq=0 out_seq=0 2023-09-30T14:47:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.187+0000 7f18e5ffb700 1 -- 172.21.15.32:0/1434918746 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b2cb9e9db0 con 0x55b2cbab3d90 2023-09-30T14:47:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.187+0000 7f18e5ffb700 10 client.25985 handle_command_reply: tid=0 2023-09-30T14:47:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.187+0000 7f1910ad1b80 1 client.25985 shutdown 2023-09-30T14:47:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.188+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1434918746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9e200 msgr2=0x55b2cbaa7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.188+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/1434918746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9e200 0x55b2cbaa7e70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1900100dc0 tx=0x7f18f0009cd0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.188+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1434918746 shutdown_connections 2023-09-30T14:47:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.188+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/1434918746 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b2cbab3d90 0x55b2cbab61b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.188+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/1434918746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaaf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.188+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/1434918746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9e200 0x55b2cbaa7e70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.188+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/1434918746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9f310 0x55b2cbaa83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.188+0000 7f18e5ffb700 0 client.25985 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.188+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1434918746 >> 172.21.15.32:0/1434918746 conn(0x55b2cba12200 msgr2=0x55b2cba98590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.188+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1434918746 shutdown_connections 2023-09-30T14:47:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.188+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1434918746 wait complete. 2023-09-30T14:47:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.188+0000 7f1910ad1b80 20 client.25985 trim_cache size 0 max 16384 2023-09-30T14:47:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f1910ad1b80 1 Processor -- start 2023-09-30T14:47:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f1910ad1b80 1 -- start start 2023-09-30T14:47:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f1910ad1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9e200 0x55b2cba9f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f1910ad1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9f310 0x55b2cbaaf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f1910ad1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaa2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f1910ad1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2cb8718c0 con 0x55b2cba9fcd0 2023-09-30T14:47:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f1910ad1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2cbaa7a10 con 0x55b2cba9f310 2023-09-30T14:47:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f1910ad1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2cba9aa10 con 0x55b2cba9e200 2023-09-30T14:47:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f18fed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaa2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f18fed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaa2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37622/0 (socket says 172.21.15.32:37622) 2023-09-30T14:47:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f18fed9d700 1 -- 172.21.15.32:0/2473513905 learned_addr learned my addr 172.21.15.32:0/2473513905 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f18fdd9b700 1 --2- 172.21.15.32:0/2473513905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9f310 0x55b2cbaaf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f18fe59c700 1 --2- 172.21.15.32:0/2473513905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9e200 0x55b2cba9f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f18fed9d700 1 -- 172.21.15.32:0/2473513905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9e200 msgr2=0x55b2cba9f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f18fed9d700 1 --2- 172.21.15.32:0/2473513905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9e200 0x55b2cba9f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f18fed9d700 1 -- 172.21.15.32:0/2473513905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9f310 msgr2=0x55b2cbaaf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f18fed9d700 1 --2- 172.21.15.32:0/2473513905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9f310 0x55b2cbaaf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.203+0000 7f18fed9d700 1 -- 172.21.15.32:0/2473513905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2cb81bd80 con 0x55b2cba9fcd0 2023-09-30T14:47:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.204+0000 7f18fed9d700 1 --2- 172.21.15.32:0/2473513905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaa2b90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f18f8099490 tx=0x7f18f8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a445fd5d4d59fd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.204+0000 7f18f57fa700 1 -- 172.21.15.32:0/2473513905 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f801b320 con 0x55b2cba9fcd0 2023-09-30T14:47:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.204+0000 7f18f57fa700 1 -- 172.21.15.32:0/2473513905 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18f8004ad0 con 0x55b2cba9fcd0 2023-09-30T14:47:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.204+0000 7f18f57fa700 1 -- 172.21.15.32:0/2473513905 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f80840b0 con 0x55b2cba9fcd0 2023-09-30T14:47:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.204+0000 7f1910ad1b80 1 -- 172.21.15.32:0/2473513905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9fcd0 msgr2=0x55b2cbaa2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.204+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/2473513905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaa2b90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f18f8099490 tx=0x7f18f8000f40 comp rx=0 tx=0).stop 2023-09-30T14:47:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.204+0000 7f1910ad1b80 1 -- 172.21.15.32:0/2473513905 shutdown_connections 2023-09-30T14:47:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.204+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/2473513905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaa2b90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.204+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/2473513905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9e200 0x55b2cba9f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.204+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/2473513905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9f310 0x55b2cbaaf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.204+0000 7f1910ad1b80 1 -- 172.21.15.32:0/2473513905 >> 172.21.15.32:0/2473513905 conn(0x55b2cba12200 msgr2=0x55b2cbabab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.205+0000 7f1910ad1b80 1 -- 172.21.15.32:0/2473513905 shutdown_connections 2023-09-30T14:47:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.205+0000 7f1910ad1b80 1 -- 172.21.15.32:0/2473513905 wait complete. 2023-09-30T14:47:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.205+0000 7f1910ad1b80 1 -- start start 2023-09-30T14:47:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.205+0000 7f1910ad1b80 1 Processor -- start 2023-09-30T14:47:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.205+0000 7f1910ad1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9e200 0x55b2cbaaccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.205+0000 7f1910ad1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9f310 0x55b2cbaad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.205+0000 7f1910ad1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.205+0000 7f1910ad1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2cbaae760 con 0x55b2cba9e200 2023-09-30T14:47:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.205+0000 7f1910ad1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2cbaae8d0 con 0x55b2cba9f310 2023-09-30T14:47:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.205+0000 7f1910ad1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2cbaaea90 con 0x55b2cba9fcd0 2023-09-30T14:47:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.205+0000 7f1910ad1b80 10 client.0 _ll_register_callbacks cb 0x7f18f0005070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:47:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.205+0000 7f18fe59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9e200 0x55b2cbaaccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.205+0000 7f18fe59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9e200 0x55b2cbaaccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37624/0 (socket says 172.21.15.32:37624) 2023-09-30T14:47:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.205+0000 7f18fe59c700 1 -- 172.21.15.32:0/1878197456 learned_addr learned my addr 172.21.15.32:0/1878197456 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.205+0000 7f18fdd9b700 1 --2- 172.21.15.32:0/1878197456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9f310 0x55b2cbaad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.205+0000 7f18fed9d700 1 --2- 172.21.15.32:0/1878197456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.206+0000 7f18fe59c700 1 -- 172.21.15.32:0/1878197456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9fcd0 msgr2=0x55b2cbaae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.206+0000 7f18fe59c700 1 --2- 172.21.15.32:0/1878197456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.206+0000 7f18fe59c700 1 -- 172.21.15.32:0/1878197456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9f310 msgr2=0x55b2cbaad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.206+0000 7f18fe59c700 1 --2- 172.21.15.32:0/1878197456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9f310 0x55b2cbaad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.206+0000 7f18fe59c700 1 -- 172.21.15.32:0/1878197456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190019f440 con 0x55b2cba9e200 2023-09-30T14:47:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.206+0000 7f18fe59c700 1 --2- 172.21.15.32:0/1878197456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9e200 0x55b2cbaaccd0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f18f0005040 tx=0x7f18f0022930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e88f2bca7ceed6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.206+0000 7f18e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:47:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.206+0000 7f18e5ffb700 1 -- 172.21.15.32:0/1878197456 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0029040 con 0x55b2cba9e200 2023-09-30T14:47:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.206+0000 7f18e5ffb700 1 -- 172.21.15.32:0/1878197456 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f18f0005720 con 0x55b2cba9e200 2023-09-30T14:47:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.206+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1878197456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18e80054e0 con 0x55b2cba9e200 2023-09-30T14:47:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.207+0000 7f18e5ffb700 1 -- 172.21.15.32:0/1878197456 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18f000ba30 con 0x55b2cba9e200 2023-09-30T14:47:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.207+0000 7f1910ad1b80 10 client.16305 fetch_fsmap learned FSMap version 41 2023-09-30T14:47:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.207+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1878197456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55b2cb804c00 con 0x55b2cba9e200 2023-09-30T14:47:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.207+0000 7f18e5ffb700 1 -- 172.21.15.32:0/1878197456 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18f004a030 con 0x55b2cba9e200 2023-09-30T14:47:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.207+0000 7f18e5ffb700 1 -- 172.21.15.32:0/1878197456 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f18f003aac0 con 0x55b2cba9e200 2023-09-30T14:47:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.207+0000 7f1910ad1b80 10 client.16305 fetch_fsmap finished waiting for FSMap version 41 2023-09-30T14:47:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.207+0000 7f1910ad1b80 10 client.16305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:47:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.207+0000 7f1910ad1b80 20 client.16305 populate_metadata read hostname 'smithi032' 2023-09-30T14:47:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.207+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/1878197456 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b2cbb08260 0x55b2cbb0a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.207+0000 7f1910ad1b80 4 client.16305 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:47:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.207+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1878197456 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b2cb9e9ad0 con 0x55b2cbb08260 2023-09-30T14:47:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.207+0000 7f18fdd9b700 1 --2- 172.21.15.32:0/1878197456 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b2cbb08260 0x55b2cbb0a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.208+0000 7f18e5ffb700 10 client.16305 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:45.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.208+0000 7f18fdd9b700 1 --2- 172.21.15.32:0/1878197456 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b2cbb08260 0x55b2cbb0a650 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d38ad0552ee307e server_cookie=c92f5b11634da66a in_seq=0 out_seq=0 2023-09-30T14:47:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.254+0000 7f18e5ffb700 1 -- 172.21.15.32:0/1878197456 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b2cb9e9ad0 con 0x55b2cbb08260 2023-09-30T14:47:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.254+0000 7f18e5ffb700 10 client.16305 handle_command_reply: tid=0 2023-09-30T14:47:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.254+0000 7f1910ad1b80 1 client.16305 shutdown 2023-09-30T14:47:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.254+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1878197456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9e200 msgr2=0x55b2cbaaccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.254+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/1878197456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9e200 0x55b2cbaaccd0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f18f0005040 tx=0x7f18f0022930 comp rx=0 tx=0).stop 2023-09-30T14:47:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.254+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1878197456 shutdown_connections 2023-09-30T14:47:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.254+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/1878197456 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b2cbb08260 0x55b2cbb0a650 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.254+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/1878197456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b2cba9fcd0 0x55b2cbaae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.254+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/1878197456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b2cba9e200 0x55b2cbaaccd0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.254+0000 7f18e5ffb700 0 client.16305 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.254+0000 7f1910ad1b80 1 --2- 172.21.15.32:0/1878197456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2cba9f310 0x55b2cbaad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.254+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1878197456 >> 172.21.15.32:0/1878197456 conn(0x55b2cba12200 msgr2=0x55b2cbaa2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.254+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1878197456 shutdown_connections 2023-09-30T14:47:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.254+0000 7f1910ad1b80 1 -- 172.21.15.32:0/1878197456 wait complete. 2023-09-30T14:47:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.254+0000 7f1910ad1b80 20 client.16305 trim_cache size 0 max 16384 2023-09-30T14:47:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:47:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:47:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:47:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:47:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.255+0000 7f19048e9700 1 -- 172.21.15.32:0/772785546 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f18ec0a1020 msgr2=0x7f18ec0a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.255+0000 7f19048e9700 1 --2- 172.21.15.32:0/772785546 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f18ec0a1020 0x7f18ec0a34e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009990 tx=0x7f18f0005fb0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.255+0000 7f19048e9700 1 -- 172.21.15.32:0/772785546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1900101900 msgr2=0x7f1900197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.255+0000 7f19048e9700 1 --2- 172.21.15.32:0/772785546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1900101900 0x7f1900197fc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f18f800efc0 tx=0x7f18f800c3a0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.255+0000 7f19048e9700 1 -- 172.21.15.32:0/772785546 shutdown_connections 2023-09-30T14:47:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.255+0000 7f19048e9700 1 --2- 172.21.15.32:0/772785546 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f18ec0a1020 0x7f18ec0a34e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.255+0000 7f19048e9700 1 --2- 172.21.15.32:0/772785546 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1900101900 0x7f1900197fc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.255+0000 7f19048e9700 1 --2- 172.21.15.32:0/772785546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19000ffd70 0x7f1900196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.255+0000 7f19048e9700 1 --2- 172.21.15.32:0/772785546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1900100f60 0x7f1900197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.255+0000 7f19048e9700 1 -- 172.21.15.32:0/772785546 >> 172.21.15.32:0/772785546 conn(0x7f19000fb2e0 msgr2=0x7f19000fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.255+0000 7f19048e9700 1 -- 172.21.15.32:0/772785546 shutdown_connections 2023-09-30T14:47:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:45.255+0000 7f19048e9700 1 -- 172.21.15.32:0/772785546 wait complete. 2023-09-30T14:47:45.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:45 smithi156 ceph-mon[103579]: pgmap v571: 97 pgs: 97 active+clean; 322 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 2.4 MiB/s wr, 244 op/s 2023-09-30T14:47:45.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:45 smithi148 ceph-mon[102355]: pgmap v571: 97 pgs: 97 active+clean; 322 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 2.4 MiB/s wr, 244 op/s 2023-09-30T14:47:45.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:45 smithi032 ceph-mon[98592]: pgmap v571: 97 pgs: 97 active+clean; 322 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 2.4 MiB/s wr, 244 op/s 2023-09-30T14:47:45.714 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1016782 10.66 MB/sec execute 379 sec latency 38.516 ms 2023-09-30T14:47:46.714 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1019590 10.68 MB/sec execute 380 sec latency 11.853 ms 2023-09-30T14:47:47.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:47 smithi156 ceph-mon[103579]: pgmap v572: 97 pgs: 97 active+clean; 324 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 2.8 MiB/s wr, 307 op/s 2023-09-30T14:47:47.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:47 smithi148 ceph-mon[102355]: pgmap v572: 97 pgs: 97 active+clean; 324 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 2.8 MiB/s wr, 307 op/s 2023-09-30T14:47:47.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:47 smithi032 ceph-mon[98592]: pgmap v572: 97 pgs: 97 active+clean; 324 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 2.8 MiB/s wr, 307 op/s 2023-09-30T14:47:47.714 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1021128 10.67 MB/sec execute 381 sec latency 33.630 ms 2023-09-30T14:47:48.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1023461 10.67 MB/sec execute 382 sec latency 75.038 ms 2023-09-30T14:47:49.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:49 smithi156 ceph-mon[103579]: pgmap v573: 97 pgs: 97 active+clean; 324 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 2.7 MiB/s wr, 274 op/s 2023-09-30T14:47:49.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:49 smithi148 ceph-mon[102355]: pgmap v573: 97 pgs: 97 active+clean; 324 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 2.7 MiB/s wr, 274 op/s 2023-09-30T14:47:49.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:49 smithi032 ceph-mon[98592]: pgmap v573: 97 pgs: 97 active+clean; 324 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 2.7 MiB/s wr, 274 op/s 2023-09-30T14:47:49.715 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1025516 10.66 MB/sec execute 383 sec latency 28.036 ms 2023-09-30T14:47:50.715 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1027240 10.67 MB/sec execute 384 sec latency 76.615 ms 2023-09-30T14:47:51.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:51 smithi156 ceph-mon[103579]: pgmap v574: 97 pgs: 97 active+clean; 323 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 2.7 MiB/s wr, 239 op/s 2023-09-30T14:47:51.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:51 smithi148 ceph-mon[102355]: pgmap v574: 97 pgs: 97 active+clean; 323 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 2.7 MiB/s wr, 239 op/s 2023-09-30T14:47:51.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:51 smithi032 ceph-mon[98592]: pgmap v574: 97 pgs: 97 active+clean; 323 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 2.7 MiB/s wr, 239 op/s 2023-09-30T14:47:51.715 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1029860 10.67 MB/sec execute 385 sec latency 14.653 ms 2023-09-30T14:47:52.715 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1031616 10.68 MB/sec execute 386 sec latency 18.345 ms 2023-09-30T14:47:53.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:53 smithi156 ceph-mon[103579]: pgmap v575: 97 pgs: 97 active+clean; 326 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 3.1 MiB/s wr, 323 op/s 2023-09-30T14:47:53.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:53 smithi148 ceph-mon[102355]: pgmap v575: 97 pgs: 97 active+clean; 326 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 3.1 MiB/s wr, 323 op/s 2023-09-30T14:47:53.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:53 smithi032 ceph-mon[98592]: pgmap v575: 97 pgs: 97 active+clean; 326 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 3.1 MiB/s wr, 323 op/s 2023-09-30T14:47:53.715 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1034020 10.68 MB/sec execute 387 sec latency 31.532 ms 2023-09-30T14:47:54.715 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1035639 10.67 MB/sec execute 388 sec latency 50.720 ms 2023-09-30T14:47:55.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:55 smithi156 ceph-mon[103579]: pgmap v576: 97 pgs: 97 active+clean; 326 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 2.6 MiB/s wr, 280 op/s 2023-09-30T14:47:55.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:55 smithi148 ceph-mon[102355]: pgmap v576: 97 pgs: 97 active+clean; 326 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 2.6 MiB/s wr, 280 op/s 2023-09-30T14:47:55.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:55 smithi032 ceph-mon[98592]: pgmap v576: 97 pgs: 97 active+clean; 326 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 2.6 MiB/s wr, 280 op/s 2023-09-30T14:47:55.715 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1037788 10.68 MB/sec execute 389 sec latency 14.991 ms 2023-09-30T14:47:56.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1040293 10.68 MB/sec execute 390 sec latency 18.488 ms 2023-09-30T14:47:57.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:57 smithi156 ceph-mon[103579]: pgmap v577: 97 pgs: 97 active+clean; 328 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 3.1 MiB/s wr, 342 op/s 2023-09-30T14:47:57.721 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:57 smithi032 ceph-mon[98592]: pgmap v577: 97 pgs: 97 active+clean; 328 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 3.1 MiB/s wr, 342 op/s 2023-09-30T14:47:57.722 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:57 smithi148 ceph-mon[102355]: pgmap v577: 97 pgs: 97 active+clean; 328 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 3.1 MiB/s wr, 342 op/s 2023-09-30T14:47:57.722 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1041850 10.68 MB/sec execute 391 sec latency 15.840 ms 2023-09-30T14:47:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.369+0000 7f84552a3700 1 -- 172.21.15.32:0/2353775925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84500ffda0 msgr2=0x7f84501001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.369+0000 7f84552a3700 1 --2- 172.21.15.32:0/2353775925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84500ffda0 0x7f84501001c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f84400099c0 tx=0x7f8440009cd0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.369+0000 7f84552a3700 1 -- 172.21.15.32:0/2353775925 shutdown_connections 2023-09-30T14:47:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.369+0000 7f84552a3700 1 --2- 172.21.15.32:0/2353775925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8450101930 0x7f84501081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.369+0000 7f84552a3700 1 --2- 172.21.15.32:0/2353775925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84500ffda0 0x7f84501001c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.369+0000 7f84552a3700 1 --2- 172.21.15.32:0/2353775925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8450100f90 0x7f84501013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.369+0000 7f84552a3700 1 -- 172.21.15.32:0/2353775925 >> 172.21.15.32:0/2353775925 conn(0x7f84500fb2f0 msgr2=0x7f84500fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.369+0000 7f84552a3700 1 -- 172.21.15.32:0/2353775925 shutdown_connections 2023-09-30T14:47:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.369+0000 7f84552a3700 1 -- 172.21.15.32:0/2353775925 wait complete. 2023-09-30T14:47:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.370+0000 7f84552a3700 1 Processor -- start 2023-09-30T14:47:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.370+0000 7f84552a3700 1 -- start start 2023-09-30T14:47:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.370+0000 7f84552a3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84500ffda0 0x7f845019bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.370+0000 7f84552a3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8450100f90 0x7f845019e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.370+0000 7f84552a3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8450101930 0x7f84501a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.370+0000 7f84552a3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8450069c90 con 0x7f84500ffda0 2023-09-30T14:47:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.370+0000 7f84552a3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84501a0c60 con 0x7f8450100f90 2023-09-30T14:47:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.370+0000 7f84552a3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84501a0df0 con 0x7f8450101930 2023-09-30T14:47:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.371+0000 7f844effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84500ffda0 0x7f845019bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.371+0000 7f844effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84500ffda0 0x7f845019bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37638/0 (socket says 172.21.15.32:37638) 2023-09-30T14:47:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.371+0000 7f844effd700 1 -- 172.21.15.32:0/1970791709 learned_addr learned my addr 172.21.15.32:0/1970791709 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.371+0000 7f844e7fc700 1 --2- 172.21.15.32:0/1970791709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8450100f90 0x7f845019e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.371+0000 7f844f7fe700 1 --2- 172.21.15.32:0/1970791709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8450101930 0x7f84501a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.371+0000 7f844effd700 1 -- 172.21.15.32:0/1970791709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8450101930 msgr2=0x7f84501a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.371+0000 7f844effd700 1 --2- 172.21.15.32:0/1970791709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8450101930 0x7f84501a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.371+0000 7f844effd700 1 -- 172.21.15.32:0/1970791709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8450100f90 msgr2=0x7f845019e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.371+0000 7f844effd700 1 --2- 172.21.15.32:0/1970791709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8450100f90 0x7f845019e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.371+0000 7f844effd700 1 -- 172.21.15.32:0/1970791709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84400095f0 con 0x7f84500ffda0 2023-09-30T14:47:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.371+0000 7f844effd700 1 --2- 172.21.15.32:0/1970791709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84500ffda0 0x7f845019bc80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8440009fd0 tx=0x7f8440009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.371+0000 7f8437fff700 1 -- 172.21.15.32:0/1970791709 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844001d070 con 0x7f84500ffda0 2023-09-30T14:47:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.371+0000 7f84552a3700 1 -- 172.21.15.32:0/1970791709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84501a1070 con 0x7f84500ffda0 2023-09-30T14:47:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.371+0000 7f8437fff700 1 -- 172.21.15.32:0/1970791709 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8440005030 con 0x7f84500ffda0 2023-09-30T14:47:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.371+0000 7f8437fff700 1 -- 172.21.15.32:0/1970791709 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8440004a90 con 0x7f84500ffda0 2023-09-30T14:47:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.371+0000 7f84552a3700 1 -- 172.21.15.32:0/1970791709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84501a14f0 con 0x7f84500ffda0 2023-09-30T14:47:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.374+0000 7f8437fff700 1 -- 172.21.15.32:0/1970791709 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f84400051a0 con 0x7f84500ffda0 2023-09-30T14:47:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.375+0000 7f8437fff700 1 --2- 172.21.15.32:0/1970791709 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f84380a0fa0 0x7f84380a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.375+0000 7f8437fff700 1 -- 172.21.15.32:0/1970791709 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f84400cd490 con 0x7f84500ffda0 2023-09-30T14:47:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.375+0000 7f844e7fc700 1 --2- 172.21.15.32:0/1970791709 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f84380a0fa0 0x7f84380a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.375+0000 7f846148bb80 1 Processor -- start 2023-09-30T14:47:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.375+0000 7f846148bb80 1 -- start start 2023-09-30T14:47:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.375+0000 7f844e7fc700 1 --2- 172.21.15.32:0/1970791709 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f84380a0fa0 0x7f84380a3460 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f84500691f0 tx=0x7f8444009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.375+0000 7f846148bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296aa200 0x55f9296aa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.375+0000 7f846148bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.375+0000 7f846148bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296abcd0 0x55f9296b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.375+0000 7f846148bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f929612400 con 0x55f9296ab310 2023-09-30T14:47:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.375+0000 7f846148bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f929603bc0 con 0x55f9296aa200 2023-09-30T14:47:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.375+0000 7f846148bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9296014e0 con 0x55f9296abcd0 2023-09-30T14:47:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.376+0000 7f844e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.376+0000 7f844e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296ab790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37648/0 (socket says 172.21.15.32:37648) 2023-09-30T14:47:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.376+0000 7f844e7fc700 1 -- 172.21.15.32:0/3683904725 learned_addr learned my addr 172.21.15.32:0/3683904725 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.376+0000 7f844f7fe700 1 --2- 172.21.15.32:0/3683904725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296abcd0 0x55f9296b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.376+0000 7f844effd700 1 --2- 172.21.15.32:0/3683904725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296aa200 0x55f9296aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.376+0000 7f844e7fc700 1 -- 172.21.15.32:0/3683904725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296abcd0 msgr2=0x55f9296b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.376+0000 7f844e7fc700 1 --2- 172.21.15.32:0/3683904725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296abcd0 0x55f9296b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.376+0000 7f844e7fc700 1 -- 172.21.15.32:0/3683904725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296aa200 msgr2=0x55f9296aa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.376+0000 7f844e7fc700 1 --2- 172.21.15.32:0/3683904725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296aa200 0x55f9296aa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.376+0000 7f844e7fc700 1 -- 172.21.15.32:0/3683904725 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f92965f870 con 0x55f9296ab310 2023-09-30T14:47:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.376+0000 7f844effd700 1 --2- 172.21.15.32:0/3683904725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296aa200 0x55f9296aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:47:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.376+0000 7f844e7fc700 1 --2- 172.21.15.32:0/3683904725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296ab790 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8444011d50 tx=0x7f8444014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66c6935352f9247 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.376+0000 7f84357fa700 1 -- 172.21.15.32:0/3683904725 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844400eb20 con 0x55f9296ab310 2023-09-30T14:47:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.376+0000 7f84357fa700 1 -- 172.21.15.32:0/3683904725 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8444021070 con 0x55f9296ab310 2023-09-30T14:47:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.376+0000 7f84357fa700 1 -- 172.21.15.32:0/3683904725 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444003b80 con 0x55f9296ab310 2023-09-30T14:47:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.377+0000 7f846148bb80 1 -- 172.21.15.32:0/3683904725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 msgr2=0x55f9296ab790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.377+0000 7f846148bb80 1 --2- 172.21.15.32:0/3683904725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296ab790 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8444011d50 tx=0x7f8444014000 comp rx=0 tx=0).stop 2023-09-30T14:47:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.377+0000 7f846148bb80 1 -- 172.21.15.32:0/3683904725 shutdown_connections 2023-09-30T14:47:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.377+0000 7f846148bb80 1 --2- 172.21.15.32:0/3683904725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296abcd0 0x55f9296b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.377+0000 7f846148bb80 1 --2- 172.21.15.32:0/3683904725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296aa200 0x55f9296aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.377+0000 7f846148bb80 1 --2- 172.21.15.32:0/3683904725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296ab790 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.377+0000 7f846148bb80 1 -- 172.21.15.32:0/3683904725 >> 172.21.15.32:0/3683904725 conn(0x55f92961e200 msgr2=0x55f9296a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.377+0000 7f846148bb80 1 -- 172.21.15.32:0/3683904725 shutdown_connections 2023-09-30T14:47:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.377+0000 7f846148bb80 1 -- 172.21.15.32:0/3683904725 wait complete. 2023-09-30T14:47:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.377+0000 7f846148bb80 1 -- start start 2023-09-30T14:47:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f846148bb80 1 Processor -- start 2023-09-30T14:47:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f846148bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296aa200 0x55f9296b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f846148bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f846148bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296abcd0 0x55f9296bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f846148bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9295fcdd0 con 0x55f9296ab310 2023-09-30T14:47:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f846148bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9295996b0 con 0x55f9296aa200 2023-09-30T14:47:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f846148bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f929599350 con 0x55f9296abcd0 2023-09-30T14:47:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f846148bb80 10 client.0 _ll_register_callbacks cb 0x55f9293f67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:47:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f844e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f844e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37654/0 (socket says 172.21.15.32:37654) 2023-09-30T14:47:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f844e7fc700 1 -- 172.21.15.32:0/1244993175 learned_addr learned my addr 172.21.15.32:0/1244993175 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f844f7fe700 1 --2- 172.21.15.32:0/1244993175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296abcd0 0x55f9296bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f844effd700 1 --2- 172.21.15.32:0/1244993175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296aa200 0x55f9296b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f844e7fc700 1 -- 172.21.15.32:0/1244993175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296abcd0 msgr2=0x55f9296bb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f844e7fc700 1 --2- 172.21.15.32:0/1244993175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296abcd0 0x55f9296bb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f844e7fc700 1 -- 172.21.15.32:0/1244993175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296aa200 msgr2=0x55f9296b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f844e7fc700 1 --2- 172.21.15.32:0/1244993175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296aa200 0x55f9296b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f844e7fc700 1 -- 172.21.15.32:0/1244993175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8444017070 con 0x55f9296ab310 2023-09-30T14:47:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f844e7fc700 1 --2- 172.21.15.32:0/1244993175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296b4470 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f844400ebb0 tx=0x7f844400ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff97a7cbfcb7bd47 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.378+0000 7f8432ff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:47:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.379+0000 7f8432ff5700 1 -- 172.21.15.32:0/1244993175 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444003d50 con 0x55f9296ab310 2023-09-30T14:47:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.379+0000 7f8432ff5700 1 -- 172.21.15.32:0/1244993175 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f84440050d0 con 0x55f9296ab310 2023-09-30T14:47:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.379+0000 7f846148bb80 1 -- 172.21.15.32:0/1244993175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f92969d090 con 0x55f9296ab310 2023-09-30T14:47:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.379+0000 7f8432ff5700 1 -- 172.21.15.32:0/1244993175 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8444005290 con 0x55f9296ab310 2023-09-30T14:47:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.379+0000 7f846148bb80 10 client.16317 fetch_fsmap learned FSMap version 41 2023-09-30T14:47:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.379+0000 7f846148bb80 1 -- 172.21.15.32:0/1244993175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55f929427d80 con 0x55f9296ab310 2023-09-30T14:47:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.379+0000 7f8432ff5700 1 -- 172.21.15.32:0/1244993175 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f92969d090 con 0x55f9296ab310 2023-09-30T14:47:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.379+0000 7f8432ff5700 1 -- 172.21.15.32:0/1244993175 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f8444031b00 con 0x55f9296ab310 2023-09-30T14:47:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.379+0000 7f846148bb80 10 client.16317 fetch_fsmap finished waiting for FSMap version 41 2023-09-30T14:47:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.379+0000 7f846148bb80 10 client.16317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:47:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.379+0000 7f846148bb80 20 client.16317 populate_metadata read hostname 'smithi032' 2023-09-30T14:47:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.379+0000 7f846148bb80 1 --2- 172.21.15.32:0/1244993175 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f9296bfbc0 0x55f9296c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.379+0000 7f846148bb80 4 client.16317 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:47:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.379+0000 7f846148bb80 1 -- 172.21.15.32:0/1244993175 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9295f5db0 con 0x55f9296bfbc0 2023-09-30T14:47:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.380+0000 7f844effd700 1 --2- 172.21.15.32:0/1244993175 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f9296bfbc0 0x55f9296c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.380+0000 7f844effd700 1 --2- 172.21.15.32:0/1244993175 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f9296bfbc0 0x55f9296c1fe0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf65e0cb957e828d server_cookie=3f7109c3aeace308 in_seq=0 out_seq=0 2023-09-30T14:47:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.380+0000 7f8432ff5700 10 client.16317 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.381+0000 7f8432ff5700 1 -- 172.21.15.32:0/1244993175 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f9295f5db0 con 0x55f9296bfbc0 2023-09-30T14:47:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.381+0000 7f8432ff5700 10 client.16317 handle_command_reply: tid=0 2023-09-30T14:47:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.381+0000 7f846148bb80 1 client.16317 shutdown 2023-09-30T14:47:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.381+0000 7f846148bb80 1 -- 172.21.15.32:0/1244993175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 msgr2=0x55f9296b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.381+0000 7f846148bb80 1 --2- 172.21.15.32:0/1244993175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296b4470 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f844400ebb0 tx=0x7f844400ebe0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.381+0000 7f846148bb80 1 -- 172.21.15.32:0/1244993175 shutdown_connections 2023-09-30T14:47:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.381+0000 7f846148bb80 1 --2- 172.21.15.32:0/1244993175 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f9296bfbc0 0x55f9296c1fe0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.381+0000 7f846148bb80 1 --2- 172.21.15.32:0/1244993175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296abcd0 0x55f9296bb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.381+0000 7f8432ff5700 0 client.16317 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.381+0000 7f846148bb80 1 --2- 172.21.15.32:0/1244993175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296aa200 0x55f9296b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.381+0000 7f846148bb80 1 --2- 172.21.15.32:0/1244993175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296b4470 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.381+0000 7f846148bb80 1 -- 172.21.15.32:0/1244993175 >> 172.21.15.32:0/1244993175 conn(0x55f92961e200 msgr2=0x55f9296a4590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.381+0000 7f846148bb80 1 -- 172.21.15.32:0/1244993175 shutdown_connections 2023-09-30T14:47:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.381+0000 7f846148bb80 1 -- 172.21.15.32:0/1244993175 wait complete. 2023-09-30T14:47:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.382+0000 7f846148bb80 20 client.16317 trim_cache size 0 max 16384 2023-09-30T14:47:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f846148bb80 1 Processor -- start 2023-09-30T14:47:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f846148bb80 1 -- start start 2023-09-30T14:47:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f846148bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296aa200 0x55f9296a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f846148bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296bb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f846148bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296abcd0 0x55f9296b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f846148bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92947d8c0 con 0x55f9296ab310 2023-09-30T14:47:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f846148bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9296ae790 con 0x55f9296aa200 2023-09-30T14:47:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f846148bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92969cbd0 con 0x55f9296abcd0 2023-09-30T14:47:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f844e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296bb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f844e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296bb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37662/0 (socket says 172.21.15.32:37662) 2023-09-30T14:47:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f844e7fc700 1 -- 172.21.15.32:0/1273158134 learned_addr learned my addr 172.21.15.32:0/1273158134 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f844effd700 1 --2- 172.21.15.32:0/1273158134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296aa200 0x55f9296a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f844f7fe700 1 --2- 172.21.15.32:0/1273158134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296abcd0 0x55f9296b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f844e7fc700 1 -- 172.21.15.32:0/1273158134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296abcd0 msgr2=0x55f9296b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f844e7fc700 1 --2- 172.21.15.32:0/1273158134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296abcd0 0x55f9296b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f844e7fc700 1 -- 172.21.15.32:0/1273158134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296aa200 msgr2=0x55f9296a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f844e7fc700 1 --2- 172.21.15.32:0/1273158134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296aa200 0x55f9296a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.399+0000 7f844e7fc700 1 -- 172.21.15.32:0/1273158134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f929410c00 con 0x55f9296ab310 2023-09-30T14:47:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.400+0000 7f844e7fc700 1 --2- 172.21.15.32:0/1273158134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296bb470 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8444031e60 tx=0x7f8444009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a499d6444d2492b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.400+0000 7f8435ffb700 1 -- 172.21.15.32:0/1273158134 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444005c40 con 0x55f9296ab310 2023-09-30T14:47:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.400+0000 7f8435ffb700 1 -- 172.21.15.32:0/1273158134 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f844400ec50 con 0x55f9296ab310 2023-09-30T14:47:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.400+0000 7f8435ffb700 1 -- 172.21.15.32:0/1273158134 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84440316e0 con 0x55f9296ab310 2023-09-30T14:47:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.401+0000 7f846148bb80 1 -- 172.21.15.32:0/1273158134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 msgr2=0x55f9296bb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.401+0000 7f846148bb80 1 --2- 172.21.15.32:0/1273158134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296bb470 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8444031e60 tx=0x7f8444009ac0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.401+0000 7f846148bb80 1 -- 172.21.15.32:0/1273158134 shutdown_connections 2023-09-30T14:47:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.401+0000 7f846148bb80 1 --2- 172.21.15.32:0/1273158134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296abcd0 0x55f9296b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.401+0000 7f846148bb80 1 --2- 172.21.15.32:0/1273158134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296aa200 0x55f9296a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.401+0000 7f846148bb80 1 --2- 172.21.15.32:0/1273158134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296bb470 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.401+0000 7f846148bb80 1 -- 172.21.15.32:0/1273158134 >> 172.21.15.32:0/1273158134 conn(0x55f92961e200 msgr2=0x55f9296b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.401+0000 7f846148bb80 1 -- 172.21.15.32:0/1273158134 shutdown_connections 2023-09-30T14:47:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.401+0000 7f846148bb80 1 -- 172.21.15.32:0/1273158134 wait complete. 2023-09-30T14:47:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.401+0000 7f846148bb80 1 -- start start 2023-09-30T14:47:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.401+0000 7f846148bb80 1 Processor -- start 2023-09-30T14:47:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.401+0000 7f846148bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296aa200 0x55f9296b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.401+0000 7f846148bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f846148bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296abcd0 0x55f9296ba390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f846148bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9296ba8d0 con 0x55f9296ab310 2023-09-30T14:47:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f846148bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9296baa40 con 0x55f9296abcd0 2023-09-30T14:47:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f846148bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9296bac00 con 0x55f9296aa200 2023-09-30T14:47:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f846148bb80 10 client.0 _ll_register_callbacks cb 0x7f8448009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:47:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f844e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f844e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296b9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37668/0 (socket says 172.21.15.32:37668) 2023-09-30T14:47:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f844e7fc700 1 -- 172.21.15.32:0/576106892 learned_addr learned my addr 172.21.15.32:0/576106892 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:47:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f844f7fe700 1 --2- 172.21.15.32:0/576106892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296abcd0 0x55f9296ba390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f844effd700 1 --2- 172.21.15.32:0/576106892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296aa200 0x55f9296b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f844e7fc700 1 -- 172.21.15.32:0/576106892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296aa200 msgr2=0x55f9296b8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f844e7fc700 1 --2- 172.21.15.32:0/576106892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296aa200 0x55f9296b8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f844e7fc700 1 -- 172.21.15.32:0/576106892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296abcd0 msgr2=0x55f9296ba390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:47:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f844e7fc700 1 --2- 172.21.15.32:0/576106892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296abcd0 0x55f9296ba390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f844e7fc700 1 -- 172.21.15.32:0/576106892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f929427d80 con 0x55f9296ab310 2023-09-30T14:47:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f844e7fc700 1 --2- 172.21.15.32:0/576106892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296b9380 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8444011b30 tx=0x7f84440038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee00ae7b25850cc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:47:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f8432ff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:47:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f8432ff5700 1 -- 172.21.15.32:0/576106892 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444031890 con 0x55f9296ab310 2023-09-30T14:47:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f846148bb80 1 -- 172.21.15.32:0/576106892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8444021070 con 0x55f9296ab310 2023-09-30T14:47:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.402+0000 7f8432ff5700 1 -- 172.21.15.32:0/576106892 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8444005640 con 0x55f9296ab310 2023-09-30T14:47:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.403+0000 7f8432ff5700 1 -- 172.21.15.32:0/576106892 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8444049030 con 0x55f9296ab310 2023-09-30T14:47:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.403+0000 7f846148bb80 10 client.16323 fetch_fsmap learned FSMap version 41 2023-09-30T14:47:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.403+0000 7f846148bb80 1 -- 172.21.15.32:0/576106892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55f92944e1f0 con 0x55f9296ab310 2023-09-30T14:47:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.403+0000 7f8432ff5700 1 -- 172.21.15.32:0/576106892 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8444021070 con 0x55f9296ab310 2023-09-30T14:47:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.403+0000 7f8432ff5700 1 -- 172.21.15.32:0/576106892 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f8444039070 con 0x55f9296ab310 2023-09-30T14:47:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.403+0000 7f846148bb80 10 client.16323 fetch_fsmap finished waiting for FSMap version 41 2023-09-30T14:47:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.403+0000 7f846148bb80 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:47:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.403+0000 7f846148bb80 20 client.16323 populate_metadata read hostname 'smithi032' 2023-09-30T14:47:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.403+0000 7f846148bb80 1 --2- 172.21.15.32:0/576106892 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f929714130 0x55f929716520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:47:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.403+0000 7f846148bb80 4 client.16323 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:47:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.403+0000 7f846148bb80 1 -- 172.21.15.32:0/576106892 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f9295f5ad0 con 0x55f929714130 2023-09-30T14:47:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.403+0000 7f844effd700 1 --2- 172.21.15.32:0/576106892 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f929714130 0x55f929716520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:47:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.403+0000 7f844effd700 1 --2- 172.21.15.32:0/576106892 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f929714130 0x55f929716520 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebede496eec11cca server_cookie=452474962433c2e1 in_seq=0 out_seq=0 2023-09-30T14:47:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.403+0000 7f8432ff5700 10 client.16323 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.454+0000 7f8432ff5700 1 -- 172.21.15.32:0/576106892 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f9295f5ad0 con 0x55f929714130 2023-09-30T14:47:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.454+0000 7f8432ff5700 10 client.16323 handle_command_reply: tid=0 2023-09-30T14:47:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.454+0000 7f846148bb80 1 client.16323 shutdown 2023-09-30T14:47:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.454+0000 7f846148bb80 1 -- 172.21.15.32:0/576106892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 msgr2=0x55f9296b9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.454+0000 7f846148bb80 1 --2- 172.21.15.32:0/576106892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296b9380 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8444011b30 tx=0x7f84440038b0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.454+0000 7f846148bb80 1 -- 172.21.15.32:0/576106892 shutdown_connections 2023-09-30T14:47:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.454+0000 7f846148bb80 1 --2- 172.21.15.32:0/576106892 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f929714130 0x55f929716520 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.454+0000 7f846148bb80 1 --2- 172.21.15.32:0/576106892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9296abcd0 0x55f9296ba390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.454+0000 7f846148bb80 1 --2- 172.21.15.32:0/576106892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9296aa200 0x55f9296b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.454+0000 7f8432ff5700 0 client.16323 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:47:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.454+0000 7f846148bb80 1 --2- 172.21.15.32:0/576106892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9296ab310 0x55f9296b9380 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.454+0000 7f846148bb80 1 -- 172.21.15.32:0/576106892 >> 172.21.15.32:0/576106892 conn(0x55f92961e200 msgr2=0x55f9296b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.454+0000 7f846148bb80 1 -- 172.21.15.32:0/576106892 shutdown_connections 2023-09-30T14:47:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.454+0000 7f846148bb80 1 -- 172.21.15.32:0/576106892 wait complete. 2023-09-30T14:47:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.455+0000 7f846148bb80 20 client.16323 trim_cache size 0 max 16384 2023-09-30T14:47:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:47:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:47:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:47:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:47:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.455+0000 7f84552a3700 1 -- 172.21.15.32:0/1970791709 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f84380a0fa0 msgr2=0x7f84380a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.455+0000 7f84552a3700 1 --2- 172.21.15.32:0/1970791709 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f84380a0fa0 0x7f84380a3460 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f84500691f0 tx=0x7f8444009260 comp rx=0 tx=0).stop 2023-09-30T14:47:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.455+0000 7f84552a3700 1 -- 172.21.15.32:0/1970791709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84500ffda0 msgr2=0x7f845019bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:47:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.455+0000 7f84552a3700 1 --2- 172.21.15.32:0/1970791709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84500ffda0 0x7f845019bc80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8440009fd0 tx=0x7f8440009580 comp rx=0 tx=0).stop 2023-09-30T14:47:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.455+0000 7f84552a3700 1 -- 172.21.15.32:0/1970791709 shutdown_connections 2023-09-30T14:47:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.455+0000 7f84552a3700 1 --2- 172.21.15.32:0/1970791709 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f84380a0fa0 0x7f84380a3460 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.455+0000 7f84552a3700 1 --2- 172.21.15.32:0/1970791709 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8450101930 0x7f84501a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.455+0000 7f84552a3700 1 --2- 172.21.15.32:0/1970791709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84500ffda0 0x7f845019bc80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.455+0000 7f84552a3700 1 --2- 172.21.15.32:0/1970791709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8450100f90 0x7f845019e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:47:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.455+0000 7f84552a3700 1 -- 172.21.15.32:0/1970791709 >> 172.21.15.32:0/1970791709 conn(0x7f84500fb2f0 msgr2=0x7f8450104470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:47:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.455+0000 7f84552a3700 1 -- 172.21.15.32:0/1970791709 shutdown_connections 2023-09-30T14:47:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:47:58.455+0000 7f84552a3700 1 -- 172.21.15.32:0/1970791709 wait complete. 2023-09-30T14:47:58.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1044222 10.68 MB/sec execute 392 sec latency 64.888 ms 2023-09-30T14:47:59.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:47:59 smithi156 ceph-mon[103579]: pgmap v578: 97 pgs: 97 active+clean; 328 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 2.6 MiB/s wr, 257 op/s 2023-09-30T14:47:59.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:47:59 smithi148 ceph-mon[102355]: pgmap v578: 97 pgs: 97 active+clean; 328 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 2.6 MiB/s wr, 257 op/s 2023-09-30T14:47:59.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:47:59 smithi032 ceph-mon[98592]: pgmap v578: 97 pgs: 97 active+clean; 328 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 2.6 MiB/s wr, 257 op/s 2023-09-30T14:47:59.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1045523 10.67 MB/sec execute 393 sec latency 58.663 ms 2023-09-30T14:48:00.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1048190 10.69 MB/sec execute 394 sec latency 18.695 ms 2023-09-30T14:48:01.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:01 smithi156 ceph-mon[103579]: pgmap v579: 97 pgs: 97 active+clean; 330 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.7 MiB/s wr, 256 op/s 2023-09-30T14:48:01.699 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:01 smithi032 ceph-mon[98592]: pgmap v579: 97 pgs: 97 active+clean; 330 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.7 MiB/s wr, 256 op/s 2023-09-30T14:48:01.699 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:01 smithi148 ceph-mon[102355]: pgmap v579: 97 pgs: 97 active+clean; 330 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.7 MiB/s wr, 256 op/s 2023-09-30T14:48:01.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1049921 10.68 MB/sec execute 395 sec latency 50.715 ms 2023-09-30T14:48:02.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1052060 10.68 MB/sec execute 396 sec latency 10.644 ms 2023-09-30T14:48:03.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:03 smithi156 ceph-mon[103579]: pgmap v580: 97 pgs: 97 active+clean; 333 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 3.0 MiB/s wr, 328 op/s 2023-09-30T14:48:03.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:03 smithi148 ceph-mon[102355]: pgmap v580: 97 pgs: 97 active+clean; 333 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 3.0 MiB/s wr, 328 op/s 2023-09-30T14:48:03.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:03 smithi032 ceph-mon[98592]: pgmap v580: 97 pgs: 97 active+clean; 333 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 3.0 MiB/s wr, 328 op/s 2023-09-30T14:48:03.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1054674 10.69 MB/sec execute 397 sec latency 14.633 ms 2023-09-30T14:48:04.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1055929 10.68 MB/sec execute 398 sec latency 110.304 ms 2023-09-30T14:48:05.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:05 smithi156 ceph-mon[103579]: pgmap v581: 97 pgs: 97 active+clean; 333 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 2.5 MiB/s wr, 248 op/s 2023-09-30T14:48:05.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:05 smithi148 ceph-mon[102355]: pgmap v581: 97 pgs: 97 active+clean; 333 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 2.5 MiB/s wr, 248 op/s 2023-09-30T14:48:05.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:05 smithi032 ceph-mon[98592]: pgmap v581: 97 pgs: 97 active+clean; 333 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 2.5 MiB/s wr, 248 op/s 2023-09-30T14:48:05.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1058418 10.69 MB/sec execute 399 sec latency 31.110 ms 2023-09-30T14:48:06.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1060047 10.68 MB/sec execute 400 sec latency 14.919 ms 2023-09-30T14:48:07.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:07 smithi156 ceph-mon[103579]: pgmap v582: 97 pgs: 97 active+clean; 337 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 3.1 MiB/s wr, 328 op/s 2023-09-30T14:48:07.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:07 smithi148 ceph-mon[102355]: pgmap v582: 97 pgs: 97 active+clean; 337 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 3.1 MiB/s wr, 328 op/s 2023-09-30T14:48:07.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:07 smithi032 ceph-mon[98592]: pgmap v582: 97 pgs: 97 active+clean; 337 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 3.1 MiB/s wr, 328 op/s 2023-09-30T14:48:07.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1062406 10.69 MB/sec execute 401 sec latency 9.739 ms 2023-09-30T14:48:08.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1064550 10.68 MB/sec execute 402 sec latency 14.718 ms 2023-09-30T14:48:09.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:09 smithi156 ceph-mon[103579]: pgmap v583: 97 pgs: 97 active+clean; 337 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 2.7 MiB/s wr, 250 op/s 2023-09-30T14:48:09.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:09 smithi148 ceph-mon[102355]: pgmap v583: 97 pgs: 97 active+clean; 337 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 2.7 MiB/s wr, 250 op/s 2023-09-30T14:48:09.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:09 smithi032 ceph-mon[98592]: pgmap v583: 97 pgs: 97 active+clean; 337 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 2.7 MiB/s wr, 250 op/s 2023-09-30T14:48:09.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1066250 10.69 MB/sec execute 403 sec latency 55.157 ms 2023-09-30T14:48:10.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1068472 10.69 MB/sec execute 404 sec latency 41.315 ms 2023-09-30T14:48:11.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:11 smithi156 ceph-mon[103579]: pgmap v584: 97 pgs: 97 active+clean; 336 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 2.8 MiB/s wr, 287 op/s 2023-09-30T14:48:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.568+0000 7f109b043700 1 -- 172.21.15.32:0/4267534431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1094101940 msgr2=0x7f1094103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.568+0000 7f109b043700 1 --2- 172.21.15.32:0/4267534431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1094101940 0x7f1094103d30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1088009990 tx=0x7f1088009ca0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f109b043700 1 -- 172.21.15.32:0/4267534431 shutdown_connections 2023-09-30T14:48:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f109b043700 1 --2- 172.21.15.32:0/4267534431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1094106ba0 0x7f1094108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f109b043700 1 --2- 172.21.15.32:0/4267534431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1094101940 0x7f1094103d30 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f109b043700 1 --2- 172.21.15.32:0/4267534431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1094104270 0x7f1094106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f109b043700 1 -- 172.21.15.32:0/4267534431 >> 172.21.15.32:0/4267534431 conn(0x7f10940fb2b0 msgr2=0x7f10940fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f109b043700 1 -- 172.21.15.32:0/4267534431 shutdown_connections 2023-09-30T14:48:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f109b043700 1 -- 172.21.15.32:0/4267534431 wait complete. 2023-09-30T14:48:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f109b043700 1 Processor -- start 2023-09-30T14:48:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f109b043700 1 -- start start 2023-09-30T14:48:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f109b043700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1094101940 0x7f1094196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f109b043700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1094104270 0x7f10941970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f109b043700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1094106ba0 0x7f1094197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f109b043700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10941985a0 con 0x7f1094101940 2023-09-30T14:48:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f109b043700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1094198710 con 0x7f1094104270 2023-09-30T14:48:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f109b043700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10941988d0 con 0x7f1094106ba0 2023-09-30T14:48:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f1098ddf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1094101940 0x7f1094196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.569+0000 7f1098ddf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1094101940 0x7f1094196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37680/0 (socket says 172.21.15.32:37680) 2023-09-30T14:48:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.570+0000 7f1098ddf700 1 -- 172.21.15.32:0/2963684128 learned_addr learned my addr 172.21.15.32:0/2963684128 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.570+0000 7f1093fff700 1 --2- 172.21.15.32:0/2963684128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1094104270 0x7f10941970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.570+0000 7f10995e0700 1 --2- 172.21.15.32:0/2963684128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1094106ba0 0x7f1094197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.570+0000 7f1098ddf700 1 -- 172.21.15.32:0/2963684128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1094106ba0 msgr2=0x7f1094197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.570+0000 7f1098ddf700 1 --2- 172.21.15.32:0/2963684128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1094106ba0 0x7f1094197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.570+0000 7f1098ddf700 1 -- 172.21.15.32:0/2963684128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1094104270 msgr2=0x7f10941970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.570+0000 7f1098ddf700 1 --2- 172.21.15.32:0/2963684128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1094104270 0x7f10941970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.570+0000 7f1098ddf700 1 -- 172.21.15.32:0/2963684128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10880095f0 con 0x7f1094101940 2023-09-30T14:48:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.570+0000 7f1098ddf700 1 --2- 172.21.15.32:0/2963684128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1094101940 0x7f1094196b80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f10880059d0 tx=0x7f1088004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.570+0000 7f1091ffb700 1 -- 172.21.15.32:0/2963684128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108801d070 con 0x7f1094101940 2023-09-30T14:48:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.570+0000 7f109b043700 1 -- 172.21.15.32:0/2963684128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1094198b50 con 0x7f1094101940 2023-09-30T14:48:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.570+0000 7f1091ffb700 1 -- 172.21.15.32:0/2963684128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f108800fb90 con 0x7f1094101940 2023-09-30T14:48:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.570+0000 7f1091ffb700 1 -- 172.21.15.32:0/2963684128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10880173f0 con 0x7f1094101940 2023-09-30T14:48:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.570+0000 7f109b043700 1 -- 172.21.15.32:0/2963684128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109419f370 con 0x7f1094101940 2023-09-30T14:48:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.572+0000 7f1091ffb700 1 -- 172.21.15.32:0/2963684128 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f108800fd00 con 0x7f1094101940 2023-09-30T14:48:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.573+0000 7f1091ffb700 1 --2- 172.21.15.32:0/2963684128 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f10840a0fa0 0x7f10840a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.573+0000 7f1091ffb700 1 -- 172.21.15.32:0/2963684128 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f10880cdb10 con 0x7f1094101940 2023-09-30T14:48:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.573+0000 7f1093fff700 1 --2- 172.21.15.32:0/2963684128 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f10840a0fa0 0x7f10840a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.574+0000 7f1093fff700 1 --2- 172.21.15.32:0/2963684128 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f10840a0fa0 0x7f10840a3460 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1080009c60 tx=0x7f1080009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.574+0000 7f10a722bb80 1 Processor -- start 2023-09-30T14:48:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.574+0000 7f10a722bb80 1 -- start start 2023-09-30T14:48:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.574+0000 7f10a722bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177d610 0x55dcf177da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.574+0000 7f10a722bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177e720 0x55dcf177eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.574+0000 7f10a722bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177f0e0 0x55dcf1785970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.574+0000 7f10a722bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf16e1400 con 0x55dcf177e720 2023-09-30T14:48:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.574+0000 7f10a722bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf16d2bc0 con 0x55dcf177f0e0 2023-09-30T14:48:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.574+0000 7f10a722bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf16d04e0 con 0x55dcf177d610 2023-09-30T14:48:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.574+0000 7f1098ddf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177d610 0x55dcf177da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.574+0000 7f1093fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177e720 0x55dcf177eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.574+0000 7f10995e0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177f0e0 0x55dcf1785970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.574+0000 7f1093fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177e720 0x55dcf177eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37690/0 (socket says 172.21.15.32:37690) 2023-09-30T14:48:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.574+0000 7f1098ddf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177d610 0x55dcf177da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52070/0 (socket says 172.21.15.32:52070) 2023-09-30T14:48:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.574+0000 7f1093fff700 1 -- 172.21.15.32:0/805909471 learned_addr learned my addr 172.21.15.32:0/805909471 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.575+0000 7f10995e0700 1 -- 172.21.15.32:0/805909471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177d610 msgr2=0x55dcf177da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.575+0000 7f10995e0700 1 --2- 172.21.15.32:0/805909471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177d610 0x55dcf177da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.575+0000 7f10995e0700 1 -- 172.21.15.32:0/805909471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177e720 msgr2=0x55dcf177eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.575+0000 7f10995e0700 1 --2- 172.21.15.32:0/805909471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177e720 0x55dcf177eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.575+0000 7f10995e0700 1 -- 172.21.15.32:0/805909471 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcf172e870 con 0x55dcf177f0e0 2023-09-30T14:48:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.575+0000 7f1093fff700 1 --2- 172.21.15.32:0/805909471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177e720 0x55dcf177eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:48:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.575+0000 7f1098ddf700 1 --2- 172.21.15.32:0/805909471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177d610 0x55dcf177da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:48:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.575+0000 7f10995e0700 1 --2- 172.21.15.32:0/805909471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177f0e0 0x55dcf1785970 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f108c011580 tx=0x7f108c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0b3fc05ce01003e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.575+0000 7f107effd700 1 -- 172.21.15.32:0/805909471 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c00e040 con 0x55dcf177f0e0 2023-09-30T14:48:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.575+0000 7f107effd700 1 -- 172.21.15.32:0/805909471 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f108c00f040 con 0x55dcf177f0e0 2023-09-30T14:48:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.575+0000 7f107effd700 1 -- 172.21.15.32:0/805909471 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c0043f0 con 0x55dcf177f0e0 2023-09-30T14:48:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.576+0000 7f10a722bb80 1 -- 172.21.15.32:0/805909471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177f0e0 msgr2=0x55dcf1785970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.576+0000 7f10a722bb80 1 --2- 172.21.15.32:0/805909471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177f0e0 0x55dcf1785970 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f108c011580 tx=0x7f108c009920 comp rx=0 tx=0).stop 2023-09-30T14:48:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.576+0000 7f10a722bb80 1 -- 172.21.15.32:0/805909471 shutdown_connections 2023-09-30T14:48:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.576+0000 7f10a722bb80 1 --2- 172.21.15.32:0/805909471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177f0e0 0x55dcf1785970 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.576+0000 7f10a722bb80 1 --2- 172.21.15.32:0/805909471 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177d610 0x55dcf177da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.576+0000 7f10a722bb80 1 --2- 172.21.15.32:0/805909471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177e720 0x55dcf177eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.576+0000 7f10a722bb80 1 -- 172.21.15.32:0/805909471 >> 172.21.15.32:0/805909471 conn(0x55dcf16ed200 msgr2=0x55dcf1773ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.576+0000 7f10a722bb80 1 -- 172.21.15.32:0/805909471 shutdown_connections 2023-09-30T14:48:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.576+0000 7f10a722bb80 1 -- 172.21.15.32:0/805909471 wait complete. 2023-09-30T14:48:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.576+0000 7f10a722bb80 1 -- start start 2023-09-30T14:48:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.576+0000 7f10a722bb80 1 Processor -- start 2023-09-30T14:48:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f10a722bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177d610 0x55dcf1787130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f10a722bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177e720 0x55dcf1787670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f10a722bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177f0e0 0x55dcf178e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f10a722bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf16cbdd0 con 0x55dcf177f0e0 2023-09-30T14:48:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f10a722bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf16686b0 con 0x55dcf177e720 2023-09-30T14:48:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f10a722bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf1668350 con 0x55dcf177d610 2023-09-30T14:48:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f10a722bb80 10 client.0 _ll_register_callbacks cb 0x55dcf14c57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:48:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f10995e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177f0e0 0x55dcf178e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f10995e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177f0e0 0x55dcf178e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37698/0 (socket says 172.21.15.32:37698) 2023-09-30T14:48:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f10995e0700 1 -- 172.21.15.32:0/2637470321 learned_addr learned my addr 172.21.15.32:0/2637470321 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f1098ddf700 1 --2- 172.21.15.32:0/2637470321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177d610 0x55dcf1787130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f1093fff700 1 --2- 172.21.15.32:0/2637470321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177e720 0x55dcf1787670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f10995e0700 1 -- 172.21.15.32:0/2637470321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177d610 msgr2=0x55dcf1787130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f10995e0700 1 --2- 172.21.15.32:0/2637470321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177d610 0x55dcf1787130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f10995e0700 1 -- 172.21.15.32:0/2637470321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177e720 msgr2=0x55dcf1787670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f10995e0700 1 --2- 172.21.15.32:0/2637470321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177e720 0x55dcf1787670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f10995e0700 1 -- 172.21.15.32:0/2637470321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1080017070 con 0x55dcf177f0e0 2023-09-30T14:48:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f10995e0700 1 --2- 172.21.15.32:0/2637470321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177f0e0 0x55dcf178e6b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f108c006050 tx=0x7f108c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a6af4a6de41fac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.577+0000 7f1073fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:48:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.578+0000 7f1073fff700 1 -- 172.21.15.32:0/2637470321 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c00e040 con 0x55dcf177f0e0 2023-09-30T14:48:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.578+0000 7f1073fff700 1 -- 172.21.15.32:0/2637470321 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f108c0041b0 con 0x55dcf177f0e0 2023-09-30T14:48:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.578+0000 7f10a722bb80 1 -- 172.21.15.32:0/2637470321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dcf176c090 con 0x55dcf177f0e0 2023-09-30T14:48:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.578+0000 7f1073fff700 1 -- 172.21.15.32:0/2637470321 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f108c004370 con 0x55dcf177f0e0 2023-09-30T14:48:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.578+0000 7f10a722bb80 10 client.16335 fetch_fsmap learned FSMap version 41 2023-09-30T14:48:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.578+0000 7f10a722bb80 1 -- 172.21.15.32:0/2637470321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f108c011230 con 0x55dcf177f0e0 2023-09-30T14:48:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.578+0000 7f1073fff700 1 -- 172.21.15.32:0/2637470321 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dcf176c090 con 0x55dcf177f0e0 2023-09-30T14:48:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.578+0000 7f1073fff700 1 -- 172.21.15.32:0/2637470321 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f108c01b020 con 0x55dcf177f0e0 2023-09-30T14:48:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.578+0000 7f10a722bb80 10 client.16335 fetch_fsmap finished waiting for FSMap version 41 2023-09-30T14:48:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.578+0000 7f10a722bb80 10 client.16335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:48:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.578+0000 7f10a722bb80 20 client.16335 populate_metadata read hostname 'smithi032' 2023-09-30T14:48:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.578+0000 7f10a722bb80 1 --2- 172.21.15.32:0/2637470321 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dcf1792e80 0x55dcf17952a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.578+0000 7f10a722bb80 4 client.16335 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:48:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.578+0000 7f10a722bb80 1 -- 172.21.15.32:0/2637470321 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dcf16c4db0 con 0x55dcf1792e80 2023-09-30T14:48:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.578+0000 7f1098ddf700 1 --2- 172.21.15.32:0/2637470321 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dcf1792e80 0x55dcf17952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.579+0000 7f1073fff700 10 client.16335 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:48:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.579+0000 7f1098ddf700 1 --2- 172.21.15.32:0/2637470321 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dcf1792e80 0x55dcf17952a0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5236047a67e955d6 server_cookie=a72383c1fcb314b6 in_seq=0 out_seq=0 2023-09-30T14:48:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.579+0000 7f1073fff700 1 -- 172.21.15.32:0/2637470321 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dcf16c4db0 con 0x55dcf1792e80 2023-09-30T14:48:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.579+0000 7f1073fff700 10 client.16335 handle_command_reply: tid=0 2023-09-30T14:48:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.579+0000 7f10a722bb80 1 client.16335 shutdown 2023-09-30T14:48:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.579+0000 7f10a722bb80 1 -- 172.21.15.32:0/2637470321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177f0e0 msgr2=0x55dcf178e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.579+0000 7f10a722bb80 1 --2- 172.21.15.32:0/2637470321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177f0e0 0x55dcf178e6b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f108c006050 tx=0x7f108c011f00 comp rx=0 tx=0).stop 2023-09-30T14:48:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.580+0000 7f10a722bb80 1 -- 172.21.15.32:0/2637470321 shutdown_connections 2023-09-30T14:48:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.580+0000 7f10a722bb80 1 --2- 172.21.15.32:0/2637470321 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dcf1792e80 0x55dcf17952a0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.580+0000 7f10a722bb80 1 --2- 172.21.15.32:0/2637470321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177f0e0 0x55dcf178e6b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.580+0000 7f10a722bb80 1 --2- 172.21.15.32:0/2637470321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177d610 0x55dcf1787130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.580+0000 7f10a722bb80 1 --2- 172.21.15.32:0/2637470321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177e720 0x55dcf1787670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.580+0000 7f1073fff700 0 client.16335 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:48:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.580+0000 7f10a722bb80 1 -- 172.21.15.32:0/2637470321 >> 172.21.15.32:0/2637470321 conn(0x55dcf16ed200 msgr2=0x55dcf1773590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.580+0000 7f10a722bb80 1 -- 172.21.15.32:0/2637470321 shutdown_connections 2023-09-30T14:48:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.580+0000 7f10a722bb80 1 -- 172.21.15.32:0/2637470321 wait complete. 2023-09-30T14:48:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.580+0000 7f10a722bb80 20 client.16335 trim_cache size 0 max 16384 2023-09-30T14:48:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.594+0000 7f10a722bb80 1 Processor -- start 2023-09-30T14:48:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.594+0000 7f10a722bb80 1 -- start start 2023-09-30T14:48:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f10a722bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177d610 0x55dcf1785f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f10a722bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177e720 0x55dcf178e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f10a722bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177f0e0 0x55dcf1781f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f10a722bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf154c8c0 con 0x55dcf177d610 2023-09-30T14:48:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f10a722bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf17865e0 con 0x55dcf177f0e0 2023-09-30T14:48:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f10a722bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf1786cd0 con 0x55dcf177e720 2023-09-30T14:48:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f1098ddf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177d610 0x55dcf1785f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f1098ddf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177d610 0x55dcf1785f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37702/0 (socket says 172.21.15.32:37702) 2023-09-30T14:48:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f1098ddf700 1 -- 172.21.15.32:0/2936435150 learned_addr learned my addr 172.21.15.32:0/2936435150 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f10995e0700 1 --2- 172.21.15.32:0/2936435150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177f0e0 0x55dcf1781f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f1093fff700 1 --2- 172.21.15.32:0/2936435150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177e720 0x55dcf178e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f1098ddf700 1 -- 172.21.15.32:0/2936435150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177e720 msgr2=0x55dcf178e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f1098ddf700 1 --2- 172.21.15.32:0/2936435150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177e720 0x55dcf178e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f1098ddf700 1 -- 172.21.15.32:0/2936435150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177f0e0 msgr2=0x55dcf1781f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f1098ddf700 1 --2- 172.21.15.32:0/2936435150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177f0e0 0x55dcf1781f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f1098ddf700 1 -- 172.21.15.32:0/2936435150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcf14f6d80 con 0x55dcf177d610 2023-09-30T14:48:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f10995e0700 1 --2- 172.21.15.32:0/2936435150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177f0e0 0x55dcf1781f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:48:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f1098ddf700 1 --2- 172.21.15.32:0/2936435150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177d610 0x55dcf1785f40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1088005910 tx=0x7f10880a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45eeba17e03d1f9f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.595+0000 7f107f7fe700 1 -- 172.21.15.32:0/2936435150 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10880a3370 con 0x55dcf177d610 2023-09-30T14:48:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.596+0000 7f107f7fe700 1 -- 172.21.15.32:0/2936435150 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10880991b0 con 0x55dcf177d610 2023-09-30T14:48:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.596+0000 7f107f7fe700 1 -- 172.21.15.32:0/2936435150 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108809a7c0 con 0x55dcf177d610 2023-09-30T14:48:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.596+0000 7f10a722bb80 1 -- 172.21.15.32:0/2936435150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177d610 msgr2=0x55dcf1785f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.596+0000 7f10a722bb80 1 --2- 172.21.15.32:0/2936435150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177d610 0x55dcf1785f40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1088005910 tx=0x7f10880a1040 comp rx=0 tx=0).stop 2023-09-30T14:48:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.596+0000 7f10a722bb80 1 -- 172.21.15.32:0/2936435150 shutdown_connections 2023-09-30T14:48:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.596+0000 7f10a722bb80 1 --2- 172.21.15.32:0/2936435150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177f0e0 0x55dcf1781f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.596+0000 7f10a722bb80 1 --2- 172.21.15.32:0/2936435150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177d610 0x55dcf1785f40 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.596+0000 7f10a722bb80 1 --2- 172.21.15.32:0/2936435150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177e720 0x55dcf178e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.596+0000 7f10a722bb80 1 -- 172.21.15.32:0/2936435150 >> 172.21.15.32:0/2936435150 conn(0x55dcf16ed200 msgr2=0x55dcf1785460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.596+0000 7f10a722bb80 1 -- 172.21.15.32:0/2936435150 shutdown_connections 2023-09-30T14:48:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.596+0000 7f10a722bb80 1 -- 172.21.15.32:0/2936435150 wait complete. 2023-09-30T14:48:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10a722bb80 1 -- start start 2023-09-30T14:48:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10a722bb80 1 Processor -- start 2023-09-30T14:48:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10a722bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177d610 0x55dcf178c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10a722bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177e720 0x55dcf178c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10a722bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177f0e0 0x55dcf178d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10a722bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf178de40 con 0x55dcf177f0e0 2023-09-30T14:48:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10a722bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf178dfb0 con 0x55dcf177d610 2023-09-30T14:48:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10a722bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf178e170 con 0x55dcf177e720 2023-09-30T14:48:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10a722bb80 10 client.0 _ll_register_callbacks cb 0x7f1080009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:48:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10995e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177f0e0 0x55dcf178d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10995e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177f0e0 0x55dcf178d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37712/0 (socket says 172.21.15.32:37712) 2023-09-30T14:48:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10995e0700 1 -- 172.21.15.32:0/529886040 learned_addr learned my addr 172.21.15.32:0/529886040 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f1093fff700 1 --2- 172.21.15.32:0/529886040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177e720 0x55dcf178c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f1098ddf700 1 --2- 172.21.15.32:0/529886040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177d610 0x55dcf178c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10995e0700 1 -- 172.21.15.32:0/529886040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177e720 msgr2=0x55dcf178c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10995e0700 1 --2- 172.21.15.32:0/529886040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177e720 0x55dcf178c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10995e0700 1 -- 172.21.15.32:0/529886040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177d610 msgr2=0x55dcf178c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10995e0700 1 --2- 172.21.15.32:0/529886040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177d610 0x55dcf178c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.597+0000 7f10995e0700 1 -- 172.21.15.32:0/529886040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108c011230 con 0x55dcf177f0e0 2023-09-30T14:48:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.598+0000 7f10995e0700 1 --2- 172.21.15.32:0/529886040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177f0e0 0x55dcf178d900 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f108c007c20 tx=0x7f108c007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa31cfa8afbe311 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.598+0000 7f1073fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:48:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.598+0000 7f1073fff700 1 -- 172.21.15.32:0/529886040 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c00e040 con 0x55dcf177f0e0 2023-09-30T14:48:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.598+0000 7f1073fff700 1 -- 172.21.15.32:0/529886040 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f108c007700 con 0x55dcf177f0e0 2023-09-30T14:48:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.598+0000 7f10a722bb80 1 -- 172.21.15.32:0/529886040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f108c00f040 con 0x55dcf177f0e0 2023-09-30T14:48:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.598+0000 7f1073fff700 1 -- 172.21.15.32:0/529886040 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f108c0078e0 con 0x55dcf177f0e0 2023-09-30T14:48:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.598+0000 7f10a722bb80 10 client.16341 fetch_fsmap learned FSMap version 41 2023-09-30T14:48:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.598+0000 7f10a722bb80 1 -- 172.21.15.32:0/529886040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f1094198b50 con 0x55dcf177f0e0 2023-09-30T14:48:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.598+0000 7f1073fff700 1 -- 172.21.15.32:0/529886040 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f108c00f040 con 0x55dcf177f0e0 2023-09-30T14:48:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.598+0000 7f1073fff700 1 -- 172.21.15.32:0/529886040 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f108c01b070 con 0x55dcf177f0e0 2023-09-30T14:48:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.598+0000 7f10a722bb80 10 client.16341 fetch_fsmap finished waiting for FSMap version 41 2023-09-30T14:48:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.598+0000 7f10a722bb80 10 client.16341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:48:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.598+0000 7f10a722bb80 20 client.16341 populate_metadata read hostname 'smithi032' 2023-09-30T14:48:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.598+0000 7f10a722bb80 1 --2- 172.21.15.32:0/529886040 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dcf17e7700 0x55dcf17e9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.598+0000 7f10a722bb80 4 client.16341 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:48:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.598+0000 7f10a722bb80 1 -- 172.21.15.32:0/529886040 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dcf16c4ad0 con 0x55dcf17e7700 2023-09-30T14:48:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.599+0000 7f1098ddf700 1 --2- 172.21.15.32:0/529886040 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dcf17e7700 0x55dcf17e9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.599+0000 7f1073fff700 10 client.16341 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:48:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.599+0000 7f1098ddf700 1 --2- 172.21.15.32:0/529886040 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dcf17e7700 0x55dcf17e9b20 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0b0925e8df400f5 server_cookie=3a783423696dfc09 in_seq=0 out_seq=0 2023-09-30T14:48:11.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:11 smithi148 ceph-mon[102355]: pgmap v584: 97 pgs: 97 active+clean; 336 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 2.8 MiB/s wr, 287 op/s 2023-09-30T14:48:11.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:11 smithi032 ceph-mon[98592]: pgmap v584: 97 pgs: 97 active+clean; 336 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 2.8 MiB/s wr, 287 op/s 2023-09-30T14:48:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.651+0000 7f1073fff700 1 -- 172.21.15.32:0/529886040 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dcf16c4ad0 con 0x55dcf17e7700 2023-09-30T14:48:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.651+0000 7f1073fff700 10 client.16341 handle_command_reply: tid=0 2023-09-30T14:48:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.651+0000 7f10a722bb80 1 client.16341 shutdown 2023-09-30T14:48:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f10a722bb80 1 -- 172.21.15.32:0/529886040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177f0e0 msgr2=0x55dcf178d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f10a722bb80 1 --2- 172.21.15.32:0/529886040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177f0e0 0x55dcf178d900 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f108c007c20 tx=0x7f108c007e90 comp rx=0 tx=0).stop 2023-09-30T14:48:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f10a722bb80 1 -- 172.21.15.32:0/529886040 shutdown_connections 2023-09-30T14:48:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f10a722bb80 1 --2- 172.21.15.32:0/529886040 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55dcf17e7700 0x55dcf17e9b20 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f10a722bb80 1 --2- 172.21.15.32:0/529886040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55dcf177f0e0 0x55dcf178d900 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f10a722bb80 1 --2- 172.21.15.32:0/529886040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dcf177d610 0x55dcf178c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f10a722bb80 1 --2- 172.21.15.32:0/529886040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dcf177e720 0x55dcf178c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f1073fff700 0 client.16341 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:48:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f10a722bb80 1 -- 172.21.15.32:0/529886040 >> 172.21.15.32:0/529886040 conn(0x55dcf16ed200 msgr2=0x55dcf1781f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f10a722bb80 1 -- 172.21.15.32:0/529886040 shutdown_connections 2023-09-30T14:48:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f10a722bb80 1 -- 172.21.15.32:0/529886040 wait complete. 2023-09-30T14:48:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f10a722bb80 20 client.16341 trim_cache size 0 max 16384 2023-09-30T14:48:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:48:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:48:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:48:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:48:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f109b043700 1 -- 172.21.15.32:0/2963684128 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f10840a0fa0 msgr2=0x7f10840a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f109b043700 1 --2- 172.21.15.32:0/2963684128 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f10840a0fa0 0x7f10840a3460 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1080009c60 tx=0x7f1080009260 comp rx=0 tx=0).stop 2023-09-30T14:48:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f109b043700 1 -- 172.21.15.32:0/2963684128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1094101940 msgr2=0x7f1094196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f109b043700 1 --2- 172.21.15.32:0/2963684128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1094101940 0x7f1094196b80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f10880059d0 tx=0x7f1088004730 comp rx=0 tx=0).stop 2023-09-30T14:48:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f109b043700 1 -- 172.21.15.32:0/2963684128 shutdown_connections 2023-09-30T14:48:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.652+0000 7f109b043700 1 --2- 172.21.15.32:0/2963684128 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f10840a0fa0 0x7f10840a3460 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.653+0000 7f109b043700 1 --2- 172.21.15.32:0/2963684128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1094106ba0 0x7f1094197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.653+0000 7f109b043700 1 --2- 172.21.15.32:0/2963684128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1094101940 0x7f1094196b80 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.653+0000 7f109b043700 1 --2- 172.21.15.32:0/2963684128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1094104270 0x7f10941970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.653+0000 7f109b043700 1 -- 172.21.15.32:0/2963684128 >> 172.21.15.32:0/2963684128 conn(0x7f10940fb2b0 msgr2=0x7f1094102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.653+0000 7f109b043700 1 -- 172.21.15.32:0/2963684128 shutdown_connections 2023-09-30T14:48:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:11.653+0000 7f109b043700 1 -- 172.21.15.32:0/2963684128 wait complete. 2023-09-30T14:48:11.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1070029 10.68 MB/sec execute 405 sec latency 30.492 ms 2023-09-30T14:48:12.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1072792 10.70 MB/sec execute 406 sec latency 11.677 ms 2023-09-30T14:48:13.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:13 smithi156 ceph-mon[103579]: pgmap v585: 97 pgs: 97 active+clean; 338 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 3.1 MiB/s wr, 319 op/s 2023-09-30T14:48:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:13 smithi148 ceph-mon[102355]: pgmap v585: 97 pgs: 97 active+clean; 338 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 3.1 MiB/s wr, 319 op/s 2023-09-30T14:48:13.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:13 smithi032 ceph-mon[98592]: pgmap v585: 97 pgs: 97 active+clean; 338 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 3.1 MiB/s wr, 319 op/s 2023-09-30T14:48:13.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1074340 10.69 MB/sec execute 407 sec latency 14.853 ms 2023-09-30T14:48:14.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1076609 10.70 MB/sec execute 408 sec latency 40.925 ms 2023-09-30T14:48:15.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:15 smithi156 ceph-mon[103579]: pgmap v586: 97 pgs: 97 active+clean; 338 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.7 MiB/s wr, 241 op/s 2023-09-30T14:48:15.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:15 smithi148 ceph-mon[102355]: pgmap v586: 97 pgs: 97 active+clean; 338 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.7 MiB/s wr, 241 op/s 2023-09-30T14:48:15.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:15 smithi032 ceph-mon[98592]: pgmap v586: 97 pgs: 97 active+clean; 338 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.7 MiB/s wr, 241 op/s 2023-09-30T14:48:15.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1078170 10.68 MB/sec execute 409 sec latency 45.211 ms 2023-09-30T14:48:16.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1080317 10.69 MB/sec execute 410 sec latency 10.842 ms 2023-09-30T14:48:17.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:17 smithi156 ceph-mon[103579]: pgmap v587: 97 pgs: 97 active+clean; 343 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 3.2 MiB/s wr, 325 op/s 2023-09-30T14:48:17.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:17 smithi148 ceph-mon[102355]: pgmap v587: 97 pgs: 97 active+clean; 343 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 3.2 MiB/s wr, 325 op/s 2023-09-30T14:48:17.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:17 smithi032 ceph-mon[98592]: pgmap v587: 97 pgs: 97 active+clean; 343 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 3.2 MiB/s wr, 325 op/s 2023-09-30T14:48:17.716 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1082770 10.69 MB/sec execute 411 sec latency 16.615 ms 2023-09-30T14:48:18.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1084292 10.69 MB/sec execute 412 sec latency 45.136 ms 2023-09-30T14:48:19.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:19 smithi156 ceph-mon[103579]: pgmap v588: 97 pgs: 97 active+clean; 343 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.8 MiB/s wr, 256 op/s 2023-09-30T14:48:19.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:19 smithi148 ceph-mon[102355]: pgmap v588: 97 pgs: 97 active+clean; 343 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.8 MiB/s wr, 256 op/s 2023-09-30T14:48:19.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:19 smithi032 ceph-mon[98592]: pgmap v588: 97 pgs: 97 active+clean; 343 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.8 MiB/s wr, 256 op/s 2023-09-30T14:48:19.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1086777 10.69 MB/sec execute 413 sec latency 49.393 ms 2023-09-30T14:48:20.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1088033 10.68 MB/sec execute 414 sec latency 43.832 ms 2023-09-30T14:48:21.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:21 smithi156 ceph-mon[103579]: pgmap v589: 97 pgs: 97 active+clean; 341 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.7 MiB/s wr, 255 op/s 2023-09-30T14:48:21.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:21 smithi148 ceph-mon[102355]: pgmap v589: 97 pgs: 97 active+clean; 341 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.7 MiB/s wr, 255 op/s 2023-09-30T14:48:21.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:21 smithi032 ceph-mon[98592]: pgmap v589: 97 pgs: 97 active+clean; 341 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.7 MiB/s wr, 255 op/s 2023-09-30T14:48:21.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1090730 10.70 MB/sec execute 415 sec latency 13.089 ms 2023-09-30T14:48:22.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1092489 10.69 MB/sec execute 416 sec latency 14.075 ms 2023-09-30T14:48:23.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:23 smithi156 ceph-mon[103579]: pgmap v590: 97 pgs: 97 active+clean; 344 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 3.0 MiB/s wr, 289 op/s 2023-09-30T14:48:23.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:23 smithi148 ceph-mon[102355]: pgmap v590: 97 pgs: 97 active+clean; 344 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 3.0 MiB/s wr, 289 op/s 2023-09-30T14:48:23.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:23 smithi032 ceph-mon[98592]: pgmap v590: 97 pgs: 97 active+clean; 344 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 3.0 MiB/s wr, 289 op/s 2023-09-30T14:48:23.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1094523 10.70 MB/sec execute 417 sec latency 17.414 ms 2023-09-30T14:48:24.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1096539 10.69 MB/sec execute 418 sec latency 80.447 ms 2023-09-30T14:48:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.764+0000 7fc33b63b700 1 -- 172.21.15.32:0/2048004319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc334100f50 msgr2=0x7fc3341013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.764+0000 7fc33b63b700 1 --2- 172.21.15.32:0/2048004319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc334100f50 0x7fc3341013b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc3280099c0 tx=0x7fc328009cd0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.764+0000 7fc33b63b700 1 -- 172.21.15.32:0/2048004319 shutdown_connections 2023-09-30T14:48:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.764+0000 7fc33b63b700 1 --2- 172.21.15.32:0/2048004319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3341018f0 0x7fc3341081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.764+0000 7fc33b63b700 1 --2- 172.21.15.32:0/2048004319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3340ffd60 0x7fc334100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.764+0000 7fc33b63b700 1 --2- 172.21.15.32:0/2048004319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc334100f50 0x7fc3341013b0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.764+0000 7fc33b63b700 1 -- 172.21.15.32:0/2048004319 >> 172.21.15.32:0/2048004319 conn(0x7fc3340fb2b0 msgr2=0x7fc3340fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.764+0000 7fc33b63b700 1 -- 172.21.15.32:0/2048004319 shutdown_connections 2023-09-30T14:48:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.764+0000 7fc33b63b700 1 -- 172.21.15.32:0/2048004319 wait complete. 2023-09-30T14:48:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc33b63b700 1 Processor -- start 2023-09-30T14:48:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc33b63b700 1 -- start start 2023-09-30T14:48:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc33b63b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3340ffd60 0x7fc334196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc33b63b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc334100f50 0x7fc334197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc33b63b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3341018f0 0x7fc334197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc33b63b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc334198690 con 0x7fc334100f50 2023-09-30T14:48:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc33b63b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc334198800 con 0x7fc3340ffd60 2023-09-30T14:48:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc33b63b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3341989c0 con 0x7fc3341018f0 2023-09-30T14:48:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc338bd6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc334100f50 0x7fc334197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc3393d7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3340ffd60 0x7fc334196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc338bd6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc334100f50 0x7fc334197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37732/0 (socket says 172.21.15.32:37732) 2023-09-30T14:48:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc338bd6700 1 -- 172.21.15.32:0/1581278327 learned_addr learned my addr 172.21.15.32:0/1581278327 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc3393d7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3340ffd60 0x7fc334196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49206/0 (socket says 172.21.15.32:49206) 2023-09-30T14:48:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc339bd8700 1 --2- 172.21.15.32:0/1581278327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3341018f0 0x7fc334197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc338bd6700 1 -- 172.21.15.32:0/1581278327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3341018f0 msgr2=0x7fc334197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc338bd6700 1 --2- 172.21.15.32:0/1581278327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3341018f0 0x7fc334197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc338bd6700 1 -- 172.21.15.32:0/1581278327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3340ffd60 msgr2=0x7fc334196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc338bd6700 1 --2- 172.21.15.32:0/1581278327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3340ffd60 0x7fc334196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.765+0000 7fc338bd6700 1 -- 172.21.15.32:0/1581278327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3280095f0 con 0x7fc334100f50 2023-09-30T14:48:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.766+0000 7fc338bd6700 1 --2- 172.21.15.32:0/1581278327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc334100f50 0x7fc334197250 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc32800bf40 tx=0x7fc32800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.766+0000 7fc3267fc700 1 -- 172.21.15.32:0/1581278327 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32801d070 con 0x7fc334100f50 2023-09-30T14:48:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.766+0000 7fc33b63b700 1 -- 172.21.15.32:0/1581278327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc334198c40 con 0x7fc334100f50 2023-09-30T14:48:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.766+0000 7fc3267fc700 1 -- 172.21.15.32:0/1581278327 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc328005050 con 0x7fc334100f50 2023-09-30T14:48:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.766+0000 7fc3267fc700 1 -- 172.21.15.32:0/1581278327 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328004b30 con 0x7fc334100f50 2023-09-30T14:48:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.766+0000 7fc33b63b700 1 -- 172.21.15.32:0/1581278327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc334102350 con 0x7fc334100f50 2023-09-30T14:48:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.770+0000 7fc3267fc700 1 -- 172.21.15.32:0/1581278327 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fc328004c90 con 0x7fc334100f50 2023-09-30T14:48:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.770+0000 7fc347823b80 1 Processor -- start 2023-09-30T14:48:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.770+0000 7fc347823b80 1 -- start start 2023-09-30T14:48:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.770+0000 7fc347823b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4aa10 0x564282d4ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.770+0000 7fc347823b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4bb20 0x564282d4bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.770+0000 7fc347823b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4c4e0 0x564282d52d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.770+0000 7fc347823b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564282caa400 con 0x564282d4aa10 2023-09-30T14:48:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.770+0000 7fc347823b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564282c9bbc0 con 0x564282d4c4e0 2023-09-30T14:48:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc347823b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564282c994e0 con 0x564282d4bb20 2023-09-30T14:48:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc3267fc700 1 --2- 172.21.15.32:0/1581278327 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc3200a0fa0 0x7fc3200a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc339bd8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4aa10 0x564282d4ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc339bd8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4aa10 0x564282d4ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37736/0 (socket says 172.21.15.32:37736) 2023-09-30T14:48:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc339bd8700 1 -- 172.21.15.32:0/2345337170 learned_addr learned my addr 172.21.15.32:0/2345337170 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc3393d7700 1 --2- 172.21.15.32:0/2345337170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4bb20 0x564282d4bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc338bd6700 1 --2- 172.21.15.32:0/2345337170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4c4e0 0x564282d52d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc339bd8700 1 --2- 172.21.15.32:0/1581278327 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc3200a0fa0 0x7fc3200a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc3393d7700 1 -- 172.21.15.32:0/2345337170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4c4e0 msgr2=0x564282d52d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc3393d7700 1 --2- 172.21.15.32:0/2345337170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4c4e0 0x564282d52d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc3393d7700 1 -- 172.21.15.32:0/2345337170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4aa10 msgr2=0x564282d4ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc3393d7700 1 --2- 172.21.15.32:0/2345337170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4aa10 0x564282d4ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc3393d7700 1 -- 172.21.15.32:0/2345337170 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564282cf7870 con 0x564282d4bb20 2023-09-30T14:48:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc3267fc700 1 -- 172.21.15.32:0/1581278327 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc3280cd500 con 0x7fc334100f50 2023-09-30T14:48:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc339bd8700 1 --2- 172.21.15.32:0/2345337170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4aa10 0x564282d4ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:48:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc3393d7700 1 --2- 172.21.15.32:0/2345337170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4bb20 0x564282d4bfa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc330007c30 tx=0x7fc330007f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c28e6cfd1ef45030 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.771+0000 7fc339bd8700 1 --2- 172.21.15.32:0/1581278327 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc3200a0fa0 0x7fc3200a3460 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x564282d4a7b0 tx=0x7fc32c004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.772+0000 7fc31f7fe700 1 -- 172.21.15.32:0/2345337170 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330004310 con 0x564282d4bb20 2023-09-30T14:48:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.772+0000 7fc31f7fe700 1 -- 172.21.15.32:0/2345337170 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc330010e40 con 0x564282d4bb20 2023-09-30T14:48:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.772+0000 7fc31f7fe700 1 -- 172.21.15.32:0/2345337170 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330005020 con 0x564282d4bb20 2023-09-30T14:48:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.772+0000 7fc347823b80 1 -- 172.21.15.32:0/2345337170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4bb20 msgr2=0x564282d4bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.772+0000 7fc347823b80 1 --2- 172.21.15.32:0/2345337170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4bb20 0x564282d4bfa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc330007c30 tx=0x7fc330007f40 comp rx=0 tx=0).stop 2023-09-30T14:48:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.772+0000 7fc347823b80 1 -- 172.21.15.32:0/2345337170 shutdown_connections 2023-09-30T14:48:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.772+0000 7fc347823b80 1 --2- 172.21.15.32:0/2345337170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4c4e0 0x564282d52d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.772+0000 7fc347823b80 1 --2- 172.21.15.32:0/2345337170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4aa10 0x564282d4ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.772+0000 7fc347823b80 1 --2- 172.21.15.32:0/2345337170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4bb20 0x564282d4bfa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.773+0000 7fc347823b80 1 -- 172.21.15.32:0/2345337170 >> 172.21.15.32:0/2345337170 conn(0x564282cb6200 msgr2=0x564282d3cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.773+0000 7fc347823b80 1 -- 172.21.15.32:0/2345337170 shutdown_connections 2023-09-30T14:48:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.773+0000 7fc347823b80 1 -- 172.21.15.32:0/2345337170 wait complete. 2023-09-30T14:48:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.773+0000 7fc347823b80 1 -- start start 2023-09-30T14:48:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.773+0000 7fc347823b80 1 Processor -- start 2023-09-30T14:48:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc347823b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4aa10 0x564282d432f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc347823b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4bb20 0x564282d43830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc347823b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4c4e0 0x564282d48860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc347823b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564282c94dd0 con 0x564282d4aa10 2023-09-30T14:48:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc347823b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564282c316b0 con 0x564282d4c4e0 2023-09-30T14:48:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc347823b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564282c31350 con 0x564282d4bb20 2023-09-30T14:48:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc347823b80 10 client.0 _ll_register_callbacks cb 0x564282a8e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:48:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc3393d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4aa10 0x564282d432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc3393d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4aa10 0x564282d432f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37744/0 (socket says 172.21.15.32:37744) 2023-09-30T14:48:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc3393d7700 1 -- 172.21.15.32:0/1561266691 learned_addr learned my addr 172.21.15.32:0/1561266691 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc338bd6700 1 --2- 172.21.15.32:0/1561266691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4bb20 0x564282d43830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc339bd8700 1 --2- 172.21.15.32:0/1561266691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4c4e0 0x564282d48860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc338bd6700 1 -- 172.21.15.32:0/1561266691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4c4e0 msgr2=0x564282d48860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc338bd6700 1 --2- 172.21.15.32:0/1561266691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4c4e0 0x564282d48860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc338bd6700 1 -- 172.21.15.32:0/1561266691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4aa10 msgr2=0x564282d432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc338bd6700 1 --2- 172.21.15.32:0/1561266691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4aa10 0x564282d432f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc338bd6700 1 -- 172.21.15.32:0/1561266691 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32c0106d0 con 0x564282d4bb20 2023-09-30T14:48:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc338bd6700 1 --2- 172.21.15.32:0/1561266691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4bb20 0x564282d43830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc3280cb060 tx=0x7fc32809a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecd3d507db427160 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.774+0000 7fc31cff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:48:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.775+0000 7fc31cff9700 1 -- 172.21.15.32:0/1561266691 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3280056c0 con 0x564282d4bb20 2023-09-30T14:48:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.775+0000 7fc31cff9700 1 -- 172.21.15.32:0/1561266691 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc3280051c0 con 0x564282d4bb20 2023-09-30T14:48:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.775+0000 7fc347823b80 1 -- 172.21.15.32:0/1561266691 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564282d35090 con 0x564282d4bb20 2023-09-30T14:48:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.775+0000 7fc31cff9700 1 -- 172.21.15.32:0/1561266691 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc32809aa60 con 0x564282d4bb20 2023-09-30T14:48:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.775+0000 7fc347823b80 10 client.26023 fetch_fsmap learned FSMap version 41 2023-09-30T14:48:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.775+0000 7fc347823b80 1 -- 172.21.15.32:0/1561266691 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fc330007890 con 0x564282d4bb20 2023-09-30T14:48:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.775+0000 7fc31cff9700 1 -- 172.21.15.32:0/1561266691 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564282d35090 con 0x564282d4bb20 2023-09-30T14:48:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.775+0000 7fc31cff9700 1 -- 172.21.15.32:0/1561266691 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc32809acc0 con 0x564282d4bb20 2023-09-30T14:48:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.775+0000 7fc347823b80 10 client.26023 fetch_fsmap finished waiting for FSMap version 41 2023-09-30T14:48:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.775+0000 7fc347823b80 10 client.26023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:48:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.775+0000 7fc347823b80 20 client.26023 populate_metadata read hostname 'smithi032' 2023-09-30T14:48:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.775+0000 7fc347823b80 1 --2- 172.21.15.32:0/1561266691 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564282d68940 0x564282d6ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.775+0000 7fc347823b80 4 client.26023 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:48:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.775+0000 7fc347823b80 1 -- 172.21.15.32:0/1561266691 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564282c8ddb0 con 0x564282d68940 2023-09-30T14:48:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.775+0000 7fc3393d7700 1 --2- 172.21.15.32:0/1561266691 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564282d68940 0x564282d6ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.776+0000 7fc3393d7700 1 --2- 172.21.15.32:0/1561266691 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564282d68940 0x564282d6ad60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8be7098daaea883 server_cookie=f53246d41acac2ed in_seq=0 out_seq=0 2023-09-30T14:48:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.776+0000 7fc31cff9700 10 client.26023 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:48:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.776+0000 7fc31cff9700 1 -- 172.21.15.32:0/1561266691 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564282c8ddb0 con 0x564282d68940 2023-09-30T14:48:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.776+0000 7fc31cff9700 10 client.26023 handle_command_reply: tid=0 2023-09-30T14:48:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.776+0000 7fc347823b80 1 client.26023 shutdown 2023-09-30T14:48:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.777+0000 7fc347823b80 1 -- 172.21.15.32:0/1561266691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4bb20 msgr2=0x564282d43830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.777+0000 7fc347823b80 1 --2- 172.21.15.32:0/1561266691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4bb20 0x564282d43830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc3280cb060 tx=0x7fc32809a630 comp rx=0 tx=0).stop 2023-09-30T14:48:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.777+0000 7fc347823b80 1 -- 172.21.15.32:0/1561266691 shutdown_connections 2023-09-30T14:48:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.777+0000 7fc347823b80 1 --2- 172.21.15.32:0/1561266691 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564282d68940 0x564282d6ad60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.777+0000 7fc347823b80 1 --2- 172.21.15.32:0/1561266691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4c4e0 0x564282d48860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.777+0000 7fc347823b80 1 --2- 172.21.15.32:0/1561266691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4aa10 0x564282d432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.777+0000 7fc347823b80 1 --2- 172.21.15.32:0/1561266691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4bb20 0x564282d43830 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.777+0000 7fc31cff9700 0 client.26023 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:48:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.777+0000 7fc347823b80 1 -- 172.21.15.32:0/1561266691 >> 172.21.15.32:0/1561266691 conn(0x564282cb6200 msgr2=0x564282d3c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.777+0000 7fc347823b80 1 -- 172.21.15.32:0/1561266691 shutdown_connections 2023-09-30T14:48:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.777+0000 7fc347823b80 1 -- 172.21.15.32:0/1561266691 wait complete. 2023-09-30T14:48:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.777+0000 7fc347823b80 20 client.26023 trim_cache size 0 max 16384 2023-09-30T14:48:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.797+0000 7fc347823b80 1 Processor -- start 2023-09-30T14:48:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc347823b80 1 -- start start 2023-09-30T14:48:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc347823b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4aa10 0x564282d4bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc347823b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4bb20 0x564282d78540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc347823b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4c4e0 0x564282d52950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc347823b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564282b158c0 con 0x564282d4c4e0 2023-09-30T14:48:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc347823b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564282d4b290 con 0x564282d4bb20 2023-09-30T14:48:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc347823b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564282d42e60 con 0x564282d4aa10 2023-09-30T14:48:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc338bd6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4bb20 0x564282d78540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc339bd8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4c4e0 0x564282d52950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc338bd6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4bb20 0x564282d78540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49228/0 (socket says 172.21.15.32:49228) 2023-09-30T14:48:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc338bd6700 1 -- 172.21.15.32:0/1613508739 learned_addr learned my addr 172.21.15.32:0/1613508739 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc3393d7700 1 --2- 172.21.15.32:0/1613508739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4aa10 0x564282d4bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc338bd6700 1 -- 172.21.15.32:0/1613508739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4aa10 msgr2=0x564282d4bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc338bd6700 1 --2- 172.21.15.32:0/1613508739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4aa10 0x564282d4bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc338bd6700 1 -- 172.21.15.32:0/1613508739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4c4e0 msgr2=0x564282d52950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc338bd6700 1 --2- 172.21.15.32:0/1613508739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4c4e0 0x564282d52950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.798+0000 7fc338bd6700 1 -- 172.21.15.32:0/1613508739 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564282abfd80 con 0x564282d4bb20 2023-09-30T14:48:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.799+0000 7fc338bd6700 1 --2- 172.21.15.32:0/1613508739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4bb20 0x564282d78540 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc3280cb380 tx=0x7fc3280cb060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1517f6df5c421265 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.799+0000 7fc31ffff700 1 -- 172.21.15.32:0/1613508739 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32809a3c0 con 0x564282d4bb20 2023-09-30T14:48:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.799+0000 7fc31ffff700 1 -- 172.21.15.32:0/1613508739 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3280051c0 con 0x564282d4bb20 2023-09-30T14:48:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.799+0000 7fc31ffff700 1 -- 172.21.15.32:0/1613508739 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3280056e0 con 0x564282d4bb20 2023-09-30T14:48:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.800+0000 7fc347823b80 1 -- 172.21.15.32:0/1613508739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4bb20 msgr2=0x564282d78540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.800+0000 7fc347823b80 1 --2- 172.21.15.32:0/1613508739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4bb20 0x564282d78540 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc3280cb380 tx=0x7fc3280cb060 comp rx=0 tx=0).stop 2023-09-30T14:48:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.800+0000 7fc347823b80 1 -- 172.21.15.32:0/1613508739 shutdown_connections 2023-09-30T14:48:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.800+0000 7fc347823b80 1 --2- 172.21.15.32:0/1613508739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4c4e0 0x564282d52950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.800+0000 7fc347823b80 1 --2- 172.21.15.32:0/1613508739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4aa10 0x564282d4bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.800+0000 7fc347823b80 1 --2- 172.21.15.32:0/1613508739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4bb20 0x564282d78540 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.800+0000 7fc347823b80 1 -- 172.21.15.32:0/1613508739 >> 172.21.15.32:0/1613508739 conn(0x564282cb6200 msgr2=0x564282d42050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.800+0000 7fc347823b80 1 -- 172.21.15.32:0/1613508739 shutdown_connections 2023-09-30T14:48:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.800+0000 7fc347823b80 1 -- 172.21.15.32:0/1613508739 wait complete. 2023-09-30T14:48:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.800+0000 7fc347823b80 1 -- start start 2023-09-30T14:48:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.800+0000 7fc347823b80 1 Processor -- start 2023-09-30T14:48:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.801+0000 7fc347823b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4aa10 0x564282d76530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.801+0000 7fc347823b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4bb20 0x564282dc9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.801+0000 7fc347823b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4c4e0 0x564282dcc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.801+0000 7fc347823b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564282db8c50 con 0x564282d4c4e0 2023-09-30T14:48:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.801+0000 7fc347823b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564282dcc920 con 0x564282d4bb20 2023-09-30T14:48:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.801+0000 7fc347823b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564282dcca60 con 0x564282d4aa10 2023-09-30T14:48:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.801+0000 7fc347823b80 10 client.0 _ll_register_callbacks cb 0x7fc3280053c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:48:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.801+0000 7fc3393d7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4aa10 0x564282d76530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.801+0000 7fc3393d7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4aa10 0x564282d76530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52138/0 (socket says 172.21.15.32:52138) 2023-09-30T14:48:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.801+0000 7fc3393d7700 1 -- 172.21.15.32:0/38917271 learned_addr learned my addr 172.21.15.32:0/38917271 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.801+0000 7fc338bd6700 1 --2- 172.21.15.32:0/38917271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4bb20 0x564282dc9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.801+0000 7fc339bd8700 1 --2- 172.21.15.32:0/38917271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4c4e0 0x564282dcc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.802+0000 7fc3393d7700 1 -- 172.21.15.32:0/38917271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4bb20 msgr2=0x564282dc9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.802+0000 7fc3393d7700 1 --2- 172.21.15.32:0/38917271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4bb20 0x564282dc9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.802+0000 7fc3393d7700 1 -- 172.21.15.32:0/38917271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4c4e0 msgr2=0x564282dcc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.802+0000 7fc3393d7700 1 --2- 172.21.15.32:0/38917271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4c4e0 0x564282dcc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.802+0000 7fc3393d7700 1 -- 172.21.15.32:0/38917271 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc330007890 con 0x564282d4aa10 2023-09-30T14:48:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.802+0000 7fc3393d7700 1 --2- 172.21.15.32:0/38917271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4aa10 0x564282d76530 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc330007b80 tx=0x7fc3300058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=869422b934990a10 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.802+0000 7fc31cff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:48:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.802+0000 7fc31cff9700 1 -- 172.21.15.32:0/38917271 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33001d040 con 0x564282d4aa10 2023-09-30T14:48:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.802+0000 7fc31cff9700 1 -- 172.21.15.32:0/38917271 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc330028cf0 con 0x564282d4aa10 2023-09-30T14:48:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.802+0000 7fc347823b80 1 -- 172.21.15.32:0/38917271 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc330010e40 con 0x564282d4aa10 2023-09-30T14:48:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.802+0000 7fc31cff9700 1 -- 172.21.15.32:0/38917271 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3300280c0 con 0x564282d4aa10 2023-09-30T14:48:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.802+0000 7fc347823b80 10 client.26029 fetch_fsmap learned FSMap version 41 2023-09-30T14:48:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.802+0000 7fc347823b80 1 -- 172.21.15.32:0/38917271 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564282aa8c00 con 0x564282d4aa10 2023-09-30T14:48:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.803+0000 7fc31cff9700 1 -- 172.21.15.32:0/38917271 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc330010e40 con 0x564282d4aa10 2023-09-30T14:48:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.803+0000 7fc31cff9700 1 -- 172.21.15.32:0/38917271 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc33003a070 con 0x564282d4aa10 2023-09-30T14:48:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.803+0000 7fc347823b80 10 client.26029 fetch_fsmap finished waiting for FSMap version 41 2023-09-30T14:48:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.803+0000 7fc347823b80 10 client.26029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:48:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.803+0000 7fc347823b80 20 client.26029 populate_metadata read hostname 'smithi032' 2023-09-30T14:48:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.803+0000 7fc347823b80 1 --2- 172.21.15.32:0/38917271 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564282dce3a0 0x564282dd07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.803+0000 7fc347823b80 4 client.26029 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:48:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.803+0000 7fc347823b80 1 -- 172.21.15.32:0/38917271 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564282c8dad0 con 0x564282dce3a0 2023-09-30T14:48:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.803+0000 7fc338bd6700 1 --2- 172.21.15.32:0/38917271 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564282dce3a0 0x564282dd07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.804+0000 7fc338bd6700 1 --2- 172.21.15.32:0/38917271 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564282dce3a0 0x564282dd07c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d1fe2911c5356a server_cookie=cdf83e7811bcda73 in_seq=0 out_seq=0 2023-09-30T14:48:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.804+0000 7fc31cff9700 10 client.26029 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:48:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.864+0000 7fc31cff9700 1 -- 172.21.15.32:0/38917271 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564282c8dad0 con 0x564282dce3a0 2023-09-30T14:48:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.864+0000 7fc31cff9700 10 client.26029 handle_command_reply: tid=0 2023-09-30T14:48:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.864+0000 7fc347823b80 1 client.26029 shutdown 2023-09-30T14:48:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc347823b80 1 -- 172.21.15.32:0/38917271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4aa10 msgr2=0x564282d76530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc347823b80 1 --2- 172.21.15.32:0/38917271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4aa10 0x564282d76530 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc330007b80 tx=0x7fc3300058e0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc347823b80 1 -- 172.21.15.32:0/38917271 shutdown_connections 2023-09-30T14:48:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc347823b80 1 --2- 172.21.15.32:0/38917271 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564282dce3a0 0x564282dd07c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc347823b80 1 --2- 172.21.15.32:0/38917271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564282d4c4e0 0x564282dcc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc347823b80 1 --2- 172.21.15.32:0/38917271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564282d4aa10 0x564282d76530 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc347823b80 1 --2- 172.21.15.32:0/38917271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282d4bb20 0x564282dc9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc31cff9700 0 client.26029 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:48:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc347823b80 1 -- 172.21.15.32:0/38917271 >> 172.21.15.32:0/38917271 conn(0x564282cb6200 msgr2=0x564282d52950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc347823b80 1 -- 172.21.15.32:0/38917271 shutdown_connections 2023-09-30T14:48:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc347823b80 1 -- 172.21.15.32:0/38917271 wait complete. 2023-09-30T14:48:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc347823b80 20 client.26029 trim_cache size 0 max 16384 2023-09-30T14:48:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:48:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:48:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:48:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:48:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc33b63b700 1 -- 172.21.15.32:0/1581278327 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc3200a0fa0 msgr2=0x7fc3200a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc33b63b700 1 --2- 172.21.15.32:0/1581278327 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc3200a0fa0 0x7fc3200a3460 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x564282d4a7b0 tx=0x7fc32c004b40 comp rx=0 tx=0).stop 2023-09-30T14:48:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc33b63b700 1 -- 172.21.15.32:0/1581278327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc334100f50 msgr2=0x7fc334197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc33b63b700 1 --2- 172.21.15.32:0/1581278327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc334100f50 0x7fc334197250 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc32800bf40 tx=0x7fc32800f690 comp rx=0 tx=0).stop 2023-09-30T14:48:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc33b63b700 1 -- 172.21.15.32:0/1581278327 shutdown_connections 2023-09-30T14:48:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc33b63b700 1 --2- 172.21.15.32:0/1581278327 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc3200a0fa0 0x7fc3200a3460 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc33b63b700 1 --2- 172.21.15.32:0/1581278327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3341018f0 0x7fc334197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc33b63b700 1 --2- 172.21.15.32:0/1581278327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3340ffd60 0x7fc334196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc33b63b700 1 --2- 172.21.15.32:0/1581278327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc334100f50 0x7fc334197250 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.865+0000 7fc33b63b700 1 -- 172.21.15.32:0/1581278327 >> 172.21.15.32:0/1581278327 conn(0x7fc3340fb2b0 msgr2=0x7fc3340682c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.866+0000 7fc33b63b700 1 -- 172.21.15.32:0/1581278327 shutdown_connections 2023-09-30T14:48:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:24.866+0000 7fc33b63b700 1 -- 172.21.15.32:0/1581278327 wait complete. 2023-09-30T14:48:25.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:25 smithi156 ceph-mon[103579]: pgmap v591: 97 pgs: 97 active+clean; 345 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 2.6 MiB/s wr, 257 op/s 2023-09-30T14:48:25.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:25 smithi148 ceph-mon[102355]: pgmap v591: 97 pgs: 97 active+clean; 345 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 2.6 MiB/s wr, 257 op/s 2023-09-30T14:48:25.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:25 smithi032 ceph-mon[98592]: pgmap v591: 97 pgs: 97 active+clean; 345 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 2.6 MiB/s wr, 257 op/s 2023-09-30T14:48:25.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1098141 10.69 MB/sec execute 419 sec latency 36.102 ms 2023-09-30T14:48:26.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1100577 10.69 MB/sec execute 420 sec latency 16.563 ms 2023-09-30T14:48:27.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:27 smithi156 ceph-mon[103579]: pgmap v592: 97 pgs: 97 active+clean; 344 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 3.1 MiB/s wr, 315 op/s 2023-09-30T14:48:27.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:27 smithi148 ceph-mon[102355]: pgmap v592: 97 pgs: 97 active+clean; 344 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 3.1 MiB/s wr, 315 op/s 2023-09-30T14:48:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:27 smithi032 ceph-mon[98592]: pgmap v592: 97 pgs: 97 active+clean; 344 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 3.1 MiB/s wr, 315 op/s 2023-09-30T14:48:27.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1102122 10.69 MB/sec execute 421 sec latency 14.913 ms 2023-09-30T14:48:28.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1104848 10.70 MB/sec execute 422 sec latency 13.491 ms 2023-09-30T14:48:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:29 smithi156 ceph-mon[103579]: pgmap v593: 97 pgs: 97 active+clean; 345 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 2.8 MiB/s wr, 259 op/s 2023-09-30T14:48:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:29 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:48:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:29 smithi148 ceph-mon[102355]: pgmap v593: 97 pgs: 97 active+clean; 345 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 2.8 MiB/s wr, 259 op/s 2023-09-30T14:48:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:29 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:48:29.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:29 smithi032 ceph-mon[98592]: pgmap v593: 97 pgs: 97 active+clean; 345 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 2.8 MiB/s wr, 259 op/s 2023-09-30T14:48:29.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:29 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:48:29.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1106178 10.69 MB/sec execute 423 sec latency 28.138 ms 2023-09-30T14:48:30.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:30 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:48:30.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:30 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:48:30.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:30 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:48:30.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:30 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:48:30.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:30 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:48:30.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:30 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:48:30.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:30 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:48:30.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:30 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:48:30.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:30 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:48:30.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1108452 10.70 MB/sec execute 424 sec latency 53.848 ms 2023-09-30T14:48:31.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:31 smithi156 ceph-mon[103579]: pgmap v594: 97 pgs: 97 active+clean; 345 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.7 MiB/s wr, 265 op/s 2023-09-30T14:48:31.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:31 smithi148 ceph-mon[102355]: pgmap v594: 97 pgs: 97 active+clean; 345 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.7 MiB/s wr, 265 op/s 2023-09-30T14:48:31.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:31 smithi032 ceph-mon[98592]: pgmap v594: 97 pgs: 97 active+clean; 345 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 2.7 MiB/s wr, 265 op/s 2023-09-30T14:48:31.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1110078 10.69 MB/sec execute 425 sec latency 14.496 ms 2023-09-30T14:48:32.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1112070 10.69 MB/sec execute 426 sec latency 91.730 ms 2023-09-30T14:48:33.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:33 smithi156 ceph-mon[103579]: pgmap v595: 97 pgs: 97 active+clean; 348 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 3.2 MiB/s wr, 309 op/s 2023-09-30T14:48:33.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:33 smithi148 ceph-mon[102355]: pgmap v595: 97 pgs: 97 active+clean; 348 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 3.2 MiB/s wr, 309 op/s 2023-09-30T14:48:33.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:33 smithi032 ceph-mon[98592]: pgmap v595: 97 pgs: 97 active+clean; 348 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 3.2 MiB/s wr, 309 op/s 2023-09-30T14:48:33.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1114219 10.69 MB/sec execute 427 sec latency 14.561 ms 2023-09-30T14:48:34.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1115935 10.69 MB/sec execute 428 sec latency 10.867 ms 2023-09-30T14:48:35.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:35 smithi156 ceph-mon[103579]: pgmap v596: 97 pgs: 97 active+clean; 348 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 2.8 MiB/s wr, 252 op/s 2023-09-30T14:48:35.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:35 smithi148 ceph-mon[102355]: pgmap v596: 97 pgs: 97 active+clean; 348 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 2.8 MiB/s wr, 252 op/s 2023-09-30T14:48:35.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:35 smithi032 ceph-mon[98592]: pgmap v596: 97 pgs: 97 active+clean; 348 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 2.8 MiB/s wr, 252 op/s 2023-09-30T14:48:35.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1118086 10.69 MB/sec execute 429 sec latency 54.290 ms 2023-09-30T14:48:36.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1119625 10.68 MB/sec execute 430 sec latency 18.573 ms 2023-09-30T14:48:37.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:37 smithi156 ceph-mon[103579]: pgmap v597: 97 pgs: 97 active+clean; 351 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 3.0 MiB/s wr, 294 op/s 2023-09-30T14:48:37.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:37 smithi148 ceph-mon[102355]: pgmap v597: 97 pgs: 97 active+clean; 351 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 3.0 MiB/s wr, 294 op/s 2023-09-30T14:48:37.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:37 smithi032 ceph-mon[98592]: pgmap v597: 97 pgs: 97 active+clean; 351 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 3.0 MiB/s wr, 294 op/s 2023-09-30T14:48:37.717 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1122082 10.69 MB/sec execute 431 sec latency 23.356 ms 2023-09-30T14:48:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.983+0000 7f85571a9700 1 -- 172.21.15.32:0/3742857352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8550104240 msgr2=0x7f8550106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.983+0000 7f85571a9700 1 --2- 172.21.15.32:0/3742857352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8550104240 0x7f8550106630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f853c0098f0 tx=0x7f853c009c00 comp rx=0 tx=0).stop 2023-09-30T14:48:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.983+0000 7f85571a9700 1 -- 172.21.15.32:0/3742857352 shutdown_connections 2023-09-30T14:48:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.983+0000 7f85571a9700 1 --2- 172.21.15.32:0/3742857352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8550106b70 0x7f8550108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.983+0000 7f85571a9700 1 --2- 172.21.15.32:0/3742857352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8550101910 0x7f8550103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.983+0000 7f85571a9700 1 --2- 172.21.15.32:0/3742857352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8550104240 0x7f8550106630 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.983+0000 7f85571a9700 1 -- 172.21.15.32:0/3742857352 >> 172.21.15.32:0/3742857352 conn(0x7f85500fb320 msgr2=0x7f85500fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.983+0000 7f85571a9700 1 -- 172.21.15.32:0/3742857352 shutdown_connections 2023-09-30T14:48:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.983+0000 7f85571a9700 1 -- 172.21.15.32:0/3742857352 wait complete. 2023-09-30T14:48:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.984+0000 7f85571a9700 1 Processor -- start 2023-09-30T14:48:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.984+0000 7f85571a9700 1 -- start start 2023-09-30T14:48:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f85571a9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8550101910 0x7f8550196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f85571a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8550104240 0x7f8550197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f85571a9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8550106b70 0x7f8550197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f85571a9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85501986c0 con 0x7f8550104240 2023-09-30T14:48:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f85571a9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8550198830 con 0x7f8550101910 2023-09-30T14:48:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f85571a9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85501989f0 con 0x7f8550106b70 2023-09-30T14:48:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f854ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8550104240 0x7f8550197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f8554f45700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8550101910 0x7f8550196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f8554f45700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8550101910 0x7f8550196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49244/0 (socket says 172.21.15.32:49244) 2023-09-30T14:48:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f854ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8550104240 0x7f8550197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37770/0 (socket says 172.21.15.32:37770) 2023-09-30T14:48:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f8554f45700 1 -- 172.21.15.32:0/785093401 learned_addr learned my addr 172.21.15.32:0/785093401 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f8555746700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8550106b70 0x7f8550197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f8554f45700 1 -- 172.21.15.32:0/785093401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8550106b70 msgr2=0x7f8550197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f8554f45700 1 --2- 172.21.15.32:0/785093401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8550106b70 0x7f8550197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f8554f45700 1 -- 172.21.15.32:0/785093401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8550104240 msgr2=0x7f8550197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f8554f45700 1 --2- 172.21.15.32:0/785093401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8550104240 0x7f8550197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f8554f45700 1 -- 172.21.15.32:0/785093401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c009520 con 0x7f8550101910 2023-09-30T14:48:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f854ffff700 1 --2- 172.21.15.32:0/785093401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8550104240 0x7f8550197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:48:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f8554f45700 1 --2- 172.21.15.32:0/785093401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8550101910 0x7f8550196d40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f854400e9a0 tx=0x7f854400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.985+0000 7f854dffb700 1 -- 172.21.15.32:0/785093401 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544009790 con 0x7f8550101910 2023-09-30T14:48:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.986+0000 7f85571a9700 1 -- 172.21.15.32:0/785093401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8550198cd0 con 0x7f8550101910 2023-09-30T14:48:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.986+0000 7f854dffb700 1 -- 172.21.15.32:0/785093401 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8544010430 con 0x7f8550101910 2023-09-30T14:48:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.986+0000 7f854dffb700 1 -- 172.21.15.32:0/785093401 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544018680 con 0x7f8550101910 2023-09-30T14:48:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.986+0000 7f85571a9700 1 -- 172.21.15.32:0/785093401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85500ff940 con 0x7f8550101910 2023-09-30T14:48:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.989+0000 7f854dffb700 1 -- 172.21.15.32:0/785093401 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f85440105a0 con 0x7f8550101910 2023-09-30T14:48:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.989+0000 7f854dffb700 1 --2- 172.21.15.32:0/785093401 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f85400a0fa0 0x7f85400a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.989+0000 7f854dffb700 1 -- 172.21.15.32:0/785093401 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8544014070 con 0x7f8550101910 2023-09-30T14:48:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.989+0000 7f854ffff700 1 --2- 172.21.15.32:0/785093401 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f85400a0fa0 0x7f85400a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.990+0000 7f8563391b80 1 Processor -- start 2023-09-30T14:48:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.990+0000 7f8563391b80 1 -- start start 2023-09-30T14:48:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.990+0000 7f854ffff700 1 --2- 172.21.15.32:0/785093401 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f85400a0fa0 0x7f85400a3460 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f853c005e90 tx=0x7f853c01d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.990+0000 7f8563391b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443254200 0x562443254620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.990+0000 7f8563391b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255310 0x562443255790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.990+0000 7f8563391b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443255cd0 0x56244325c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.990+0000 7f8563391b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624431bc400 con 0x562443255310 2023-09-30T14:48:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.990+0000 7f8563391b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624431adbc0 con 0x562443254200 2023-09-30T14:48:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.990+0000 7f8563391b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624431ab4e0 con 0x562443255cd0 2023-09-30T14:48:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.990+0000 7f854ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255310 0x562443255790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.990+0000 7f854ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255310 0x562443255790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37778/0 (socket says 172.21.15.32:37778) 2023-09-30T14:48:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.990+0000 7f854ffff700 1 -- 172.21.15.32:0/4236816663 learned_addr learned my addr 172.21.15.32:0/4236816663 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.990+0000 7f8554f45700 1 --2- 172.21.15.32:0/4236816663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443254200 0x562443254620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.990+0000 7f8555746700 1 --2- 172.21.15.32:0/4236816663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443255cd0 0x56244325c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.991+0000 7f8555746700 1 -- 172.21.15.32:0/4236816663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443254200 msgr2=0x562443254620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.991+0000 7f8555746700 1 --2- 172.21.15.32:0/4236816663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443254200 0x562443254620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.991+0000 7f8555746700 1 -- 172.21.15.32:0/4236816663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255310 msgr2=0x562443255790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.991+0000 7f8555746700 1 --2- 172.21.15.32:0/4236816663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255310 0x562443255790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.991+0000 7f8555746700 1 -- 172.21.15.32:0/4236816663 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562443209870 con 0x562443255cd0 2023-09-30T14:48:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.991+0000 7f854ffff700 1 --2- 172.21.15.32:0/4236816663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255310 0x562443255790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:48:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.991+0000 7f8554f45700 1 --2- 172.21.15.32:0/4236816663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443254200 0x562443254620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:48:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.991+0000 7f8555746700 1 --2- 172.21.15.32:0/4236816663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443255cd0 0x56244325c560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8548011580 tx=0x7f85480099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b195886cda316a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.991+0000 7f853affd700 1 -- 172.21.15.32:0/4236816663 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854800e040 con 0x562443255cd0 2023-09-30T14:48:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.991+0000 7f853affd700 1 -- 172.21.15.32:0/4236816663 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f854800f040 con 0x562443255cd0 2023-09-30T14:48:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.991+0000 7f853affd700 1 -- 172.21.15.32:0/4236816663 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548004290 con 0x562443255cd0 2023-09-30T14:48:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.992+0000 7f8563391b80 1 -- 172.21.15.32:0/4236816663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443255cd0 msgr2=0x56244325c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.992+0000 7f8563391b80 1 --2- 172.21.15.32:0/4236816663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443255cd0 0x56244325c560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8548011580 tx=0x7f85480099e0 comp rx=0 tx=0).stop 2023-09-30T14:48:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.992+0000 7f8563391b80 1 -- 172.21.15.32:0/4236816663 shutdown_connections 2023-09-30T14:48:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.992+0000 7f8563391b80 1 --2- 172.21.15.32:0/4236816663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443255cd0 0x56244325c560 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.992+0000 7f8563391b80 1 --2- 172.21.15.32:0/4236816663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443254200 0x562443254620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.992+0000 7f8563391b80 1 --2- 172.21.15.32:0/4236816663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255310 0x562443255790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.992+0000 7f8563391b80 1 -- 172.21.15.32:0/4236816663 >> 172.21.15.32:0/4236816663 conn(0x5624431c8200 msgr2=0x56244324eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.992+0000 7f8563391b80 1 -- 172.21.15.32:0/4236816663 shutdown_connections 2023-09-30T14:48:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.992+0000 7f8563391b80 1 -- 172.21.15.32:0/4236816663 wait complete. 2023-09-30T14:48:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.992+0000 7f8563391b80 1 -- start start 2023-09-30T14:48:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.992+0000 7f8563391b80 1 Processor -- start 2023-09-30T14:48:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8563391b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443254200 0x56244325dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8563391b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443255310 0x56244325e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8563391b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443255cd0 0x562443265360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8563391b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624431a6dd0 con 0x562443254200 2023-09-30T14:48:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8563391b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624431436b0 con 0x562443255cd0 2023-09-30T14:48:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8563391b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562443143350 con 0x562443255310 2023-09-30T14:48:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8563391b80 10 client.0 _ll_register_callbacks cb 0x562442fa07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:48:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8554f45700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443254200 0x56244325dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8554f45700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443254200 0x56244325dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37782/0 (socket says 172.21.15.32:37782) 2023-09-30T14:48:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8554f45700 1 -- 172.21.15.32:0/3101137669 learned_addr learned my addr 172.21.15.32:0/3101137669 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f854ffff700 1 --2- 172.21.15.32:0/3101137669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443255310 0x56244325e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8555746700 1 --2- 172.21.15.32:0/3101137669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443255cd0 0x562443265360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8554f45700 1 -- 172.21.15.32:0/3101137669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443255310 msgr2=0x56244325e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8554f45700 1 --2- 172.21.15.32:0/3101137669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443255310 0x56244325e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8554f45700 1 -- 172.21.15.32:0/3101137669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443255cd0 msgr2=0x562443265360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8554f45700 1 --2- 172.21.15.32:0/3101137669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443255cd0 0x562443265360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8554f45700 1 -- 172.21.15.32:0/3101137669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8550109bd0 con 0x562443254200 2023-09-30T14:48:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f854ffff700 1 --2- 172.21.15.32:0/3101137669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443255310 0x56244325e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:48:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8555746700 1 --2- 172.21.15.32:0/3101137669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443255cd0 0x562443265360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:48:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f8554f45700 1 --2- 172.21.15.32:0/3101137669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443254200 0x56244325dde0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8544099d10 tx=0x7f854409a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0aa355e0ac0f94 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.993+0000 7f852ffff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:48:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.994+0000 7f852ffff700 1 -- 172.21.15.32:0/3101137669 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854409d070 con 0x562443254200 2023-09-30T14:48:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.994+0000 7f852ffff700 1 -- 172.21.15.32:0/3101137669 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f854409ded0 con 0x562443254200 2023-09-30T14:48:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.994+0000 7f8563391b80 1 -- 172.21.15.32:0/3101137669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562443247090 con 0x562443254200 2023-09-30T14:48:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.994+0000 7f852ffff700 1 -- 172.21.15.32:0/3101137669 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85440a2110 con 0x562443254200 2023-09-30T14:48:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.994+0000 7f8563391b80 10 client.16371 fetch_fsmap learned FSMap version 41 2023-09-30T14:48:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.994+0000 7f8563391b80 1 -- 172.21.15.32:0/3101137669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f8548011230 con 0x562443254200 2023-09-30T14:48:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.994+0000 7f852ffff700 1 -- 172.21.15.32:0/3101137669 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562443247090 con 0x562443254200 2023-09-30T14:48:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.994+0000 7f852ffff700 1 -- 172.21.15.32:0/3101137669 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f8544098ce0 con 0x562443254200 2023-09-30T14:48:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.994+0000 7f8563391b80 10 client.16371 fetch_fsmap finished waiting for FSMap version 41 2023-09-30T14:48:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.994+0000 7f8563391b80 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:48:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.994+0000 7f8563391b80 20 client.16371 populate_metadata read hostname 'smithi032' 2023-09-30T14:48:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.994+0000 7f8563391b80 1 --2- 172.21.15.32:0/3101137669 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562443269c40 0x56244326c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.994+0000 7f8563391b80 4 client.16371 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:48:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.994+0000 7f8563391b80 1 -- 172.21.15.32:0/3101137669 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56244319fdb0 con 0x562443269c40 2023-09-30T14:48:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.994+0000 7f854ffff700 1 --2- 172.21.15.32:0/3101137669 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562443269c40 0x56244326c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.995+0000 7f854ffff700 1 --2- 172.21.15.32:0/3101137669 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562443269c40 0x56244326c060 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd3f60b1c2d02ad4 server_cookie=865b1df776b135f in_seq=0 out_seq=0 2023-09-30T14:48:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.995+0000 7f852ffff700 10 client.16371 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:48:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.995+0000 7f852ffff700 1 -- 172.21.15.32:0/3101137669 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56244319fdb0 con 0x562443269c40 2023-09-30T14:48:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.995+0000 7f852ffff700 10 client.16371 handle_command_reply: tid=0 2023-09-30T14:48:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.996+0000 7f8563391b80 1 client.16371 shutdown 2023-09-30T14:48:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.996+0000 7f8563391b80 1 -- 172.21.15.32:0/3101137669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443254200 msgr2=0x56244325dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.996+0000 7f8563391b80 1 --2- 172.21.15.32:0/3101137669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443254200 0x56244325dde0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8544099d10 tx=0x7f854409a2c0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.996+0000 7f8563391b80 1 -- 172.21.15.32:0/3101137669 shutdown_connections 2023-09-30T14:48:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.996+0000 7f8563391b80 1 --2- 172.21.15.32:0/3101137669 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562443269c40 0x56244326c060 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.996+0000 7f8563391b80 1 --2- 172.21.15.32:0/3101137669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443255cd0 0x562443265360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.996+0000 7f8563391b80 1 --2- 172.21.15.32:0/3101137669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443254200 0x56244325dde0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.996+0000 7f8563391b80 1 --2- 172.21.15.32:0/3101137669 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443255310 0x56244325e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.996+0000 7f8563391b80 1 -- 172.21.15.32:0/3101137669 >> 172.21.15.32:0/3101137669 conn(0x5624431c8200 msgr2=0x56244324e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.996+0000 7f852ffff700 0 client.16371 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:48:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.996+0000 7f8563391b80 1 -- 172.21.15.32:0/3101137669 shutdown_connections 2023-09-30T14:48:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.996+0000 7f8563391b80 1 -- 172.21.15.32:0/3101137669 wait complete. 2023-09-30T14:48:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:37.996+0000 7f8563391b80 20 client.16371 trim_cache size 0 max 16384 2023-09-30T14:48:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.012+0000 7f8563391b80 1 Processor -- start 2023-09-30T14:48:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f8563391b80 1 -- start start 2023-09-30T14:48:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f8563391b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443254200 0x56244325cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f8563391b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255310 0x562443265320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f8563391b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443255cd0 0x56244325c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f8563391b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624430278c0 con 0x562443255310 2023-09-30T14:48:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f8563391b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562443250290 con 0x562443255cd0 2023-09-30T14:48:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f8563391b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244325d950 con 0x562443254200 2023-09-30T14:48:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f854ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255310 0x562443265320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f8554f45700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443254200 0x56244325cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f854ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255310 0x562443265320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37790/0 (socket says 172.21.15.32:37790) 2023-09-30T14:48:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f854ffff700 1 -- 172.21.15.32:0/3196820269 learned_addr learned my addr 172.21.15.32:0/3196820269 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f8554f45700 1 -- 172.21.15.32:0/3196820269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443255cd0 msgr2=0x56244325c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f8554f45700 1 --2- 172.21.15.32:0/3196820269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443255cd0 0x56244325c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f8554f45700 1 -- 172.21.15.32:0/3196820269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255310 msgr2=0x562443265320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f8554f45700 1 --2- 172.21.15.32:0/3196820269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255310 0x562443265320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f8554f45700 1 -- 172.21.15.32:0/3196820269 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562442fd1d80 con 0x562443254200 2023-09-30T14:48:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f854ffff700 1 --2- 172.21.15.32:0/3196820269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255310 0x562443265320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:48:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.013+0000 7f8554f45700 1 --2- 172.21.15.32:0/3196820269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443254200 0x56244325cc10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f85440967f0 tx=0x7f8544098330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b89b84e371f79ed0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.014+0000 7f853b7fe700 1 -- 172.21.15.32:0/3196820269 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854409d070 con 0x562443254200 2023-09-30T14:48:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.014+0000 7f853b7fe700 1 -- 172.21.15.32:0/3196820269 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85440a2110 con 0x562443254200 2023-09-30T14:48:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.014+0000 7f853b7fe700 1 -- 172.21.15.32:0/3196820269 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85440a1ee0 con 0x562443254200 2023-09-30T14:48:38.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.014+0000 7f8563391b80 1 -- 172.21.15.32:0/3196820269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443254200 msgr2=0x56244325cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:38.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.014+0000 7f8563391b80 1 --2- 172.21.15.32:0/3196820269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443254200 0x56244325cc10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f85440967f0 tx=0x7f8544098330 comp rx=0 tx=0).stop 2023-09-30T14:48:38.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.014+0000 7f8563391b80 1 -- 172.21.15.32:0/3196820269 shutdown_connections 2023-09-30T14:48:38.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.014+0000 7f8563391b80 1 --2- 172.21.15.32:0/3196820269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443255cd0 0x56244325c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.014+0000 7f8563391b80 1 --2- 172.21.15.32:0/3196820269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443254200 0x56244325cc10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.014+0000 7f8563391b80 1 --2- 172.21.15.32:0/3196820269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255310 0x562443265320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.014+0000 7f8563391b80 1 -- 172.21.15.32:0/3196820269 >> 172.21.15.32:0/3196820269 conn(0x5624431c8200 msgr2=0x56244326bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.015+0000 7f8563391b80 1 -- 172.21.15.32:0/3196820269 shutdown_connections 2023-09-30T14:48:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.015+0000 7f8563391b80 1 -- 172.21.15.32:0/3196820269 wait complete. 2023-09-30T14:48:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.015+0000 7f8563391b80 1 -- start start 2023-09-30T14:48:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.015+0000 7f8563391b80 1 Processor -- start 2023-09-30T14:48:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.015+0000 7f8563391b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443254200 0x562443262c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.015+0000 7f8563391b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443255310 0x562443263150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.015+0000 7f8563391b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255cd0 0x562443264160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.015+0000 7f8563391b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624432646a0 con 0x562443255cd0 2023-09-30T14:48:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.015+0000 7f8563391b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562443264810 con 0x562443255310 2023-09-30T14:48:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.015+0000 7f8563391b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624432649d0 con 0x562443254200 2023-09-30T14:48:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f8563391b80 10 client.0 _ll_register_callbacks cb 0x562442fa07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:48:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f8555746700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255cd0 0x562443264160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f8554f45700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443254200 0x562443262c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f854ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443255310 0x562443263150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f8555746700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255cd0 0x562443264160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37798/0 (socket says 172.21.15.32:37798) 2023-09-30T14:48:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f8555746700 1 -- 172.21.15.32:0/146287740 learned_addr learned my addr 172.21.15.32:0/146287740 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f8555746700 1 -- 172.21.15.32:0/146287740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443254200 msgr2=0x562443262c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f8555746700 1 --2- 172.21.15.32:0/146287740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443254200 0x562443262c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f8555746700 1 -- 172.21.15.32:0/146287740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443255310 msgr2=0x562443263150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f8555746700 1 --2- 172.21.15.32:0/146287740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443255310 0x562443263150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f8555746700 1 -- 172.21.15.32:0/146287740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8548011230 con 0x562443255cd0 2023-09-30T14:48:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f8555746700 1 --2- 172.21.15.32:0/146287740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255cd0 0x562443264160 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8548011550 tx=0x7f8548003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab6cc98838d00e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f852ffff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:48:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f852ffff700 1 -- 172.21.15.32:0/146287740 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854800e040 con 0x562443255cd0 2023-09-30T14:48:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f8563391b80 1 -- 172.21.15.32:0/146287740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f854800f040 con 0x562443255cd0 2023-09-30T14:48:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f852ffff700 1 -- 172.21.15.32:0/146287740 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8548009450 con 0x562443255cd0 2023-09-30T14:48:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f852ffff700 1 -- 172.21.15.32:0/146287740 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f854802d030 con 0x562443255cd0 2023-09-30T14:48:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f8563391b80 10 client.16377 fetch_fsmap learned FSMap version 41 2023-09-30T14:48:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.016+0000 7f8563391b80 1 -- 172.21.15.32:0/146287740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562442fbac00 con 0x562443255cd0 2023-09-30T14:48:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.017+0000 7f852ffff700 1 -- 172.21.15.32:0/146287740 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f854800f040 con 0x562443255cd0 2023-09-30T14:48:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.017+0000 7f852ffff700 1 -- 172.21.15.32:0/146287740 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f854801b020 con 0x562443255cd0 2023-09-30T14:48:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.017+0000 7f8563391b80 10 client.16377 fetch_fsmap finished waiting for FSMap version 41 2023-09-30T14:48:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.017+0000 7f8563391b80 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:48:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.017+0000 7f8563391b80 20 client.16377 populate_metadata read hostname 'smithi032' 2023-09-30T14:48:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.017+0000 7f8563391b80 1 --2- 172.21.15.32:0/146287740 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5624432be5b0 0x5624432c09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.017+0000 7f8563391b80 4 client.16377 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:48:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.017+0000 7f8563391b80 1 -- 172.21.15.32:0/146287740 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56244319fad0 con 0x5624432be5b0 2023-09-30T14:48:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.017+0000 7f8554f45700 1 --2- 172.21.15.32:0/146287740 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5624432be5b0 0x5624432c09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.017+0000 7f852ffff700 10 client.16377 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:48:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.017+0000 7f8554f45700 1 --2- 172.21.15.32:0/146287740 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5624432be5b0 0x5624432c09d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4762531ff2f6b712 server_cookie=9f2e98fe5bc4e3e4 in_seq=0 out_seq=0 2023-09-30T14:48:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.067+0000 7f852ffff700 1 -- 172.21.15.32:0/146287740 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56244319fad0 con 0x5624432be5b0 2023-09-30T14:48:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.067+0000 7f852ffff700 10 client.16377 handle_command_reply: tid=0 2023-09-30T14:48:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.067+0000 7f8563391b80 1 client.16377 shutdown 2023-09-30T14:48:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.067+0000 7f8563391b80 1 -- 172.21.15.32:0/146287740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255cd0 msgr2=0x562443264160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.067+0000 7f8563391b80 1 --2- 172.21.15.32:0/146287740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255cd0 0x562443264160 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8548011550 tx=0x7f8548003c30 comp rx=0 tx=0).stop 2023-09-30T14:48:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.067+0000 7f8563391b80 1 -- 172.21.15.32:0/146287740 shutdown_connections 2023-09-30T14:48:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.067+0000 7f8563391b80 1 --2- 172.21.15.32:0/146287740 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5624432be5b0 0x5624432c09d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.067+0000 7f8563391b80 1 --2- 172.21.15.32:0/146287740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562443255cd0 0x562443264160 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.067+0000 7f8563391b80 1 --2- 172.21.15.32:0/146287740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562443254200 0x562443262c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.067+0000 7f8563391b80 1 --2- 172.21.15.32:0/146287740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562443255310 0x562443263150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.067+0000 7f852ffff700 0 client.16377 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:48:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.067+0000 7f8563391b80 1 -- 172.21.15.32:0/146287740 >> 172.21.15.32:0/146287740 conn(0x5624431c8200 msgr2=0x56244325c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.067+0000 7f8563391b80 1 -- 172.21.15.32:0/146287740 shutdown_connections 2023-09-30T14:48:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.067+0000 7f8563391b80 1 -- 172.21.15.32:0/146287740 wait complete. 2023-09-30T14:48:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.067+0000 7f8563391b80 20 client.16377 trim_cache size 0 max 16384 2023-09-30T14:48:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:48:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:48:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:48:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:48:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.068+0000 7f85571a9700 1 -- 172.21.15.32:0/785093401 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f85400a0fa0 msgr2=0x7f85400a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.068+0000 7f85571a9700 1 --2- 172.21.15.32:0/785093401 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f85400a0fa0 0x7f85400a3460 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f853c005e90 tx=0x7f853c01d040 comp rx=0 tx=0).stop 2023-09-30T14:48:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.068+0000 7f85571a9700 1 -- 172.21.15.32:0/785093401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8550101910 msgr2=0x7f8550196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.068+0000 7f85571a9700 1 --2- 172.21.15.32:0/785093401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8550101910 0x7f8550196d40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f854400e9a0 tx=0x7f854400ecb0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.068+0000 7f85571a9700 1 -- 172.21.15.32:0/785093401 shutdown_connections 2023-09-30T14:48:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.068+0000 7f85571a9700 1 --2- 172.21.15.32:0/785093401 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f85400a0fa0 0x7f85400a3460 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.068+0000 7f85571a9700 1 --2- 172.21.15.32:0/785093401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8550106b70 0x7f8550197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.068+0000 7f85571a9700 1 --2- 172.21.15.32:0/785093401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8550101910 0x7f8550196d40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.068+0000 7f85571a9700 1 --2- 172.21.15.32:0/785093401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8550104240 0x7f8550197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.068+0000 7f85571a9700 1 -- 172.21.15.32:0/785093401 >> 172.21.15.32:0/785093401 conn(0x7f85500fb320 msgr2=0x7f8550102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.068+0000 7f85571a9700 1 -- 172.21.15.32:0/785093401 shutdown_connections 2023-09-30T14:48:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:38.068+0000 7f85571a9700 1 -- 172.21.15.32:0/785093401 wait complete. 2023-09-30T14:48:38.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1123397 10.68 MB/sec execute 432 sec latency 20.478 ms 2023-09-30T14:48:39.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:39 smithi156 ceph-mon[103579]: pgmap v598: 97 pgs: 97 active+clean; 352 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 2.8 MiB/s wr, 253 op/s 2023-09-30T14:48:39.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:39 smithi148 ceph-mon[102355]: pgmap v598: 97 pgs: 97 active+clean; 352 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 2.8 MiB/s wr, 253 op/s 2023-09-30T14:48:39.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:39 smithi032 ceph-mon[98592]: pgmap v598: 97 pgs: 97 active+clean; 352 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 2.8 MiB/s wr, 253 op/s 2023-09-30T14:48:39.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1126041 10.69 MB/sec execute 433 sec latency 13.437 ms 2023-09-30T14:48:40.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1127351 10.68 MB/sec execute 434 sec latency 82.464 ms 2023-09-30T14:48:41.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:41 smithi156 ceph-mon[103579]: pgmap v599: 97 pgs: 97 active+clean; 354 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 2.8 MiB/s wr, 278 op/s 2023-09-30T14:48:41.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:41 smithi148 ceph-mon[102355]: pgmap v599: 97 pgs: 97 active+clean; 354 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 2.8 MiB/s wr, 278 op/s 2023-09-30T14:48:41.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:41 smithi032 ceph-mon[98592]: pgmap v599: 97 pgs: 97 active+clean; 354 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 2.8 MiB/s wr, 278 op/s 2023-09-30T14:48:41.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1129803 10.69 MB/sec execute 435 sec latency 16.043 ms 2023-09-30T14:48:42.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1132153 10.69 MB/sec execute 436 sec latency 13.542 ms 2023-09-30T14:48:43.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:43 smithi156 ceph-mon[103579]: pgmap v600: 97 pgs: 97 active+clean; 353 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.0 MiB/s wr, 286 op/s 2023-09-30T14:48:43.632 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:43 smithi148 ceph-mon[102355]: pgmap v600: 97 pgs: 97 active+clean; 353 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.0 MiB/s wr, 286 op/s 2023-09-30T14:48:43.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:43 smithi032 ceph-mon[98592]: pgmap v600: 97 pgs: 97 active+clean; 353 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.0 MiB/s wr, 286 op/s 2023-09-30T14:48:43.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1133832 10.69 MB/sec execute 437 sec latency 32.301 ms 2023-09-30T14:48:44.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1136636 10.70 MB/sec execute 438 sec latency 15.029 ms 2023-09-30T14:48:45.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:45 smithi156 ceph-mon[103579]: pgmap v601: 97 pgs: 97 active+clean; 354 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.8 MiB/s wr, 282 op/s 2023-09-30T14:48:45.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:45 smithi148 ceph-mon[102355]: pgmap v601: 97 pgs: 97 active+clean; 354 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.8 MiB/s wr, 282 op/s 2023-09-30T14:48:45.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:45 smithi032 ceph-mon[98592]: pgmap v601: 97 pgs: 97 active+clean; 354 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.8 MiB/s wr, 282 op/s 2023-09-30T14:48:45.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1138056 10.69 MB/sec execute 439 sec latency 62.126 ms 2023-09-30T14:48:46.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1140510 10.70 MB/sec execute 440 sec latency 14.304 ms 2023-09-30T14:48:47.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:47 smithi156 ceph-mon[103579]: pgmap v602: 97 pgs: 97 active+clean; 356 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 2.9 MiB/s wr, 289 op/s 2023-09-30T14:48:47.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:47 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:48:47.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:47 smithi148 ceph-mon[102355]: pgmap v602: 97 pgs: 97 active+clean; 356 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 2.9 MiB/s wr, 289 op/s 2023-09-30T14:48:47.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:47 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:48:47.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:47 smithi032 ceph-mon[98592]: pgmap v602: 97 pgs: 97 active+clean; 356 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 2.9 MiB/s wr, 289 op/s 2023-09-30T14:48:47.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:47 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:48:47.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1142808 10.70 MB/sec execute 441 sec latency 17.403 ms 2023-09-30T14:48:48.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1144533 10.70 MB/sec execute 442 sec latency 58.961 ms 2023-09-30T14:48:49.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:49 smithi156 ceph-mon[103579]: pgmap v603: 97 pgs: 97 active+clean; 359 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 2.8 MiB/s wr, 290 op/s 2023-09-30T14:48:49.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:49 smithi148 ceph-mon[102355]: pgmap v603: 97 pgs: 97 active+clean; 359 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 2.8 MiB/s wr, 290 op/s 2023-09-30T14:48:49.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:49 smithi032 ceph-mon[98592]: pgmap v603: 97 pgs: 97 active+clean; 359 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 2.8 MiB/s wr, 290 op/s 2023-09-30T14:48:49.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1146953 10.70 MB/sec execute 443 sec latency 63.383 ms 2023-09-30T14:48:50.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1148271 10.69 MB/sec execute 444 sec latency 24.256 ms 2023-09-30T14:48:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.196+0000 7f443cb7b700 1 -- 172.21.15.32:0/3160159157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44381018f0 msgr2=0x7f44381081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.196+0000 7f443cb7b700 1 --2- 172.21.15.32:0/3160159157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44381018f0 0x7f44381081a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f443000b0d0 tx=0x7f443000b3e0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.196+0000 7f443cb7b700 1 -- 172.21.15.32:0/3160159157 shutdown_connections 2023-09-30T14:48:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.196+0000 7f443cb7b700 1 --2- 172.21.15.32:0/3160159157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44381018f0 0x7f44381081a0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.196+0000 7f443cb7b700 1 --2- 172.21.15.32:0/3160159157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44380ffd60 0x7f4438100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.196+0000 7f443cb7b700 1 --2- 172.21.15.32:0/3160159157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4438100f50 0x7f44381013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.196+0000 7f443cb7b700 1 -- 172.21.15.32:0/3160159157 >> 172.21.15.32:0/3160159157 conn(0x7f44380fb2b0 msgr2=0x7f44380fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.196+0000 7f443cb7b700 1 -- 172.21.15.32:0/3160159157 shutdown_connections 2023-09-30T14:48:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.196+0000 7f443cb7b700 1 -- 172.21.15.32:0/3160159157 wait complete. 2023-09-30T14:48:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.197+0000 7f443cb7b700 1 Processor -- start 2023-09-30T14:48:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.197+0000 7f443cb7b700 1 -- start start 2023-09-30T14:48:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.197+0000 7f443cb7b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44380ffd60 0x7f4438199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.197+0000 7f443cb7b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4438100f50 0x7f443819bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.197+0000 7f443cb7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44381018f0 0x7f443819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.197+0000 7f443cb7b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443819e9c0 con 0x7f44381018f0 2023-09-30T14:48:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.197+0000 7f443cb7b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443819eb00 con 0x7f4438100f50 2023-09-30T14:48:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.197+0000 7f443cb7b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443819ecc0 con 0x7f44380ffd60 2023-09-30T14:48:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.197+0000 7f4436d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44381018f0 0x7f443819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.198+0000 7f4436d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44381018f0 0x7f443819e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37812/0 (socket says 172.21.15.32:37812) 2023-09-30T14:48:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.198+0000 7f443659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44380ffd60 0x7f4438199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.198+0000 7f4435d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4438100f50 0x7f443819bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.198+0000 7f4436d9d700 1 -- 172.21.15.32:0/1028510798 learned_addr learned my addr 172.21.15.32:0/1028510798 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.198+0000 7f4436d9d700 1 -- 172.21.15.32:0/1028510798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44380ffd60 msgr2=0x7f4438199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.198+0000 7f4436d9d700 1 --2- 172.21.15.32:0/1028510798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44380ffd60 0x7f4438199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.198+0000 7f4436d9d700 1 -- 172.21.15.32:0/1028510798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4438100f50 msgr2=0x7f443819bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.198+0000 7f4436d9d700 1 --2- 172.21.15.32:0/1028510798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4438100f50 0x7f443819bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.198+0000 7f4436d9d700 1 -- 172.21.15.32:0/1028510798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4430009d00 con 0x7f44381018f0 2023-09-30T14:48:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.198+0000 7f4436d9d700 1 --2- 172.21.15.32:0/1028510798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44381018f0 0x7f443819e3f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4430008ec0 tx=0x7f4430008ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.198+0000 7f44277fe700 1 -- 172.21.15.32:0/1028510798 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443000e040 con 0x7f44381018f0 2023-09-30T14:48:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.198+0000 7f44277fe700 1 -- 172.21.15.32:0/1028510798 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4430005070 con 0x7f44381018f0 2023-09-30T14:48:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.198+0000 7f443cb7b700 1 -- 172.21.15.32:0/1028510798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443819ef40 con 0x7f44381018f0 2023-09-30T14:48:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.198+0000 7f44277fe700 1 -- 172.21.15.32:0/1028510798 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443001b6c0 con 0x7f44381018f0 2023-09-30T14:48:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.198+0000 7f443cb7b700 1 -- 172.21.15.32:0/1028510798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443819f2b0 con 0x7f44381018f0 2023-09-30T14:48:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.202+0000 7f44277fe700 1 -- 172.21.15.32:0/1028510798 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f44300051e0 con 0x7f44381018f0 2023-09-30T14:48:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.202+0000 7f44277fe700 1 --2- 172.21.15.32:0/1028510798 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44200a0fa0 0x7f44200a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f4448d63b80 1 Processor -- start 2023-09-30T14:48:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f443659c700 1 --2- 172.21.15.32:0/1028510798 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44200a0fa0 0x7f44200a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f44277fe700 1 -- 172.21.15.32:0/1028510798 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f44300cd940 con 0x7f44381018f0 2023-09-30T14:48:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f4448d63b80 1 -- start start 2023-09-30T14:48:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f4448d63b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff2610 0x55978eff2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f4448d63b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff3720 0x55978eff3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f4448d63b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff40e0 0x55978effa970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f4448d63b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978ef56400 con 0x55978eff3720 2023-09-30T14:48:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f4448d63b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978ef47bc0 con 0x55978eff40e0 2023-09-30T14:48:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f4448d63b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978ef454e0 con 0x55978eff2610 2023-09-30T14:48:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f4435d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff3720 0x55978eff3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f443659c700 1 --2- 172.21.15.32:0/1028510798 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44200a0fa0 0x7f44200a3460 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4438068d20 tx=0x7f4428009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f4435d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff3720 0x55978eff3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37820/0 (socket says 172.21.15.32:37820) 2023-09-30T14:48:51.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f4435d9b700 1 -- 172.21.15.32:0/1674906172 learned_addr learned my addr 172.21.15.32:0/1674906172 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f443659c700 1 --2- 172.21.15.32:0/1674906172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff2610 0x55978eff2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f4436d9d700 1 --2- 172.21.15.32:0/1674906172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff40e0 0x55978effa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f4435d9b700 1 -- 172.21.15.32:0/1674906172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff2610 msgr2=0x55978eff2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.203+0000 7f4435d9b700 1 --2- 172.21.15.32:0/1674906172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff2610 0x55978eff2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.204+0000 7f4435d9b700 1 -- 172.21.15.32:0/1674906172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff40e0 msgr2=0x55978effa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.204+0000 7f4435d9b700 1 --2- 172.21.15.32:0/1674906172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff40e0 0x55978effa970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.204+0000 7f4435d9b700 1 -- 172.21.15.32:0/1674906172 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55978efa3870 con 0x55978eff3720 2023-09-30T14:48:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.204+0000 7f4435d9b700 1 --2- 172.21.15.32:0/1674906172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff3720 0x55978eff3ba0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f442c009d20 tx=0x7f442c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a0a6c243c984af server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.204+0000 7f4424ff9700 1 -- 172.21.15.32:0/1674906172 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c004a30 con 0x55978eff3720 2023-09-30T14:48:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.204+0000 7f4424ff9700 1 -- 172.21.15.32:0/1674906172 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f442c0054e0 con 0x55978eff3720 2023-09-30T14:48:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.204+0000 7f4424ff9700 1 -- 172.21.15.32:0/1674906172 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c019760 con 0x55978eff3720 2023-09-30T14:48:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.205+0000 7f4448d63b80 1 -- 172.21.15.32:0/1674906172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff3720 msgr2=0x55978eff3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.205+0000 7f4448d63b80 1 --2- 172.21.15.32:0/1674906172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff3720 0x55978eff3ba0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f442c009d20 tx=0x7f442c011720 comp rx=0 tx=0).stop 2023-09-30T14:48:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.205+0000 7f4448d63b80 1 -- 172.21.15.32:0/1674906172 shutdown_connections 2023-09-30T14:48:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.205+0000 7f4448d63b80 1 --2- 172.21.15.32:0/1674906172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff40e0 0x55978effa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.205+0000 7f4448d63b80 1 --2- 172.21.15.32:0/1674906172 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff2610 0x55978eff2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.205+0000 7f4448d63b80 1 --2- 172.21.15.32:0/1674906172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff3720 0x55978eff3ba0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.205+0000 7f4448d63b80 1 -- 172.21.15.32:0/1674906172 >> 172.21.15.32:0/1674906172 conn(0x55978ef62200 msgr2=0x55978efe8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.205+0000 7f4448d63b80 1 -- 172.21.15.32:0/1674906172 shutdown_connections 2023-09-30T14:48:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.205+0000 7f4448d63b80 1 -- 172.21.15.32:0/1674906172 wait complete. 2023-09-30T14:48:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.205+0000 7f4448d63b80 1 -- start start 2023-09-30T14:48:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.205+0000 7f4448d63b80 1 Processor -- start 2023-09-30T14:48:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.205+0000 7f4448d63b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff2610 0x55978effc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.205+0000 7f4448d63b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff3720 0x55978effc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.205+0000 7f4448d63b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff40e0 0x55978f003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.205+0000 7f4448d63b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978ef40dd0 con 0x55978eff40e0 2023-09-30T14:48:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.205+0000 7f4448d63b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978eedd6b0 con 0x55978eff3720 2023-09-30T14:48:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f4448d63b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978eedd350 con 0x55978eff2610 2023-09-30T14:48:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f4448d63b80 10 client.0 _ll_register_callbacks cb 0x55978ed3a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:48:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f4435d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff3720 0x55978effc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f443659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff2610 0x55978effc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f4435d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff3720 0x55978effc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49302/0 (socket says 172.21.15.32:49302) 2023-09-30T14:48:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f443659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff2610 0x55978effc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52206/0 (socket says 172.21.15.32:52206) 2023-09-30T14:48:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f4435d9b700 1 -- 172.21.15.32:0/2543601005 learned_addr learned my addr 172.21.15.32:0/2543601005 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f4436d9d700 1 --2- 172.21.15.32:0/2543601005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff40e0 0x55978f003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f4435d9b700 1 -- 172.21.15.32:0/2543601005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff2610 msgr2=0x55978effc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f4435d9b700 1 --2- 172.21.15.32:0/2543601005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff2610 0x55978effc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f4435d9b700 1 -- 172.21.15.32:0/2543601005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff40e0 msgr2=0x55978f003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f4435d9b700 1 --2- 172.21.15.32:0/2543601005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff40e0 0x55978f003800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f4435d9b700 1 -- 172.21.15.32:0/2543601005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442c009980 con 0x55978eff3720 2023-09-30T14:48:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f4435d9b700 1 --2- 172.21.15.32:0/2543601005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff3720 0x55978effc7c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f442c0095f0 tx=0x7f442c009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a98504ad91daddba server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f441dffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:48:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f441dffb700 1 -- 172.21.15.32:0/2543601005 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c004960 con 0x55978eff3720 2023-09-30T14:48:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f4448d63b80 1 -- 172.21.15.32:0/2543601005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55978efe1090 con 0x55978eff3720 2023-09-30T14:48:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.206+0000 7f441dffb700 1 -- 172.21.15.32:0/2543601005 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f442c004ac0 con 0x55978eff3720 2023-09-30T14:48:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.207+0000 7f441dffb700 1 -- 172.21.15.32:0/2543601005 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f442c0194a0 con 0x55978eff3720 2023-09-30T14:48:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.207+0000 7f4448d63b80 10 client.26072 fetch_fsmap learned FSMap version 42 2023-09-30T14:48:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.207+0000 7f4448d63b80 1 -- 172.21.15.32:0/2543601005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55978ed6bd80 con 0x55978eff3720 2023-09-30T14:48:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.207+0000 7f441dffb700 1 -- 172.21.15.32:0/2543601005 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55978efe1090 con 0x55978eff3720 2023-09-30T14:48:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.207+0000 7f441dffb700 1 -- 172.21.15.32:0/2543601005 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 42) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f442c0245f0 con 0x55978eff3720 2023-09-30T14:48:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.207+0000 7f4448d63b80 10 client.26072 fetch_fsmap finished waiting for FSMap version 42 2023-09-30T14:48:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.207+0000 7f4448d63b80 10 client.26072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:48:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.207+0000 7f4448d63b80 20 client.26072 populate_metadata read hostname 'smithi032' 2023-09-30T14:48:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.207+0000 7f4448d63b80 1 --2- 172.21.15.32:0/2543601005 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55978f0080d0 0x55978f00a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.207+0000 7f4448d63b80 4 client.26072 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:48:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.207+0000 7f4448d63b80 1 -- 172.21.15.32:0/2543601005 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55978ef39db0 con 0x55978f0080d0 2023-09-30T14:48:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.207+0000 7f443659c700 1 --2- 172.21.15.32:0/2543601005 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55978f0080d0 0x55978f00a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.208+0000 7f441dffb700 10 client.26072 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:48:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.208+0000 7f443659c700 1 --2- 172.21.15.32:0/2543601005 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55978f0080d0 0x55978f00a4f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e362f56f5e5a1fa server_cookie=1602690718714940 in_seq=0 out_seq=0 2023-09-30T14:48:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.208+0000 7f441dffb700 1 -- 172.21.15.32:0/2543601005 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55978ef39db0 con 0x55978f0080d0 2023-09-30T14:48:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.208+0000 7f441dffb700 10 client.26072 handle_command_reply: tid=0 2023-09-30T14:48:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.208+0000 7f4448d63b80 1 client.26072 shutdown 2023-09-30T14:48:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.208+0000 7f4448d63b80 1 -- 172.21.15.32:0/2543601005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff3720 msgr2=0x55978effc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.208+0000 7f4448d63b80 1 --2- 172.21.15.32:0/2543601005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff3720 0x55978effc7c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f442c0095f0 tx=0x7f442c009770 comp rx=0 tx=0).stop 2023-09-30T14:48:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.208+0000 7f4448d63b80 1 -- 172.21.15.32:0/2543601005 shutdown_connections 2023-09-30T14:48:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.208+0000 7f4448d63b80 1 --2- 172.21.15.32:0/2543601005 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55978f0080d0 0x55978f00a4f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.208+0000 7f4448d63b80 1 --2- 172.21.15.32:0/2543601005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff40e0 0x55978f003800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.208+0000 7f441dffb700 0 client.26072 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:48:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.208+0000 7f4448d63b80 1 --2- 172.21.15.32:0/2543601005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff2610 0x55978effc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.208+0000 7f4448d63b80 1 --2- 172.21.15.32:0/2543601005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff3720 0x55978effc7c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.208+0000 7f4448d63b80 1 -- 172.21.15.32:0/2543601005 >> 172.21.15.32:0/2543601005 conn(0x55978ef62200 msgr2=0x55978efe8590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.208+0000 7f4448d63b80 1 -- 172.21.15.32:0/2543601005 shutdown_connections 2023-09-30T14:48:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.208+0000 7f4448d63b80 1 -- 172.21.15.32:0/2543601005 wait complete. 2023-09-30T14:48:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.209+0000 7f4448d63b80 20 client.26072 trim_cache size 0 max 16384 2023-09-30T14:48:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.224+0000 7f4448d63b80 1 Processor -- start 2023-09-30T14:48:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.224+0000 7f4448d63b80 1 -- start start 2023-09-30T14:48:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f4448d63b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff2610 0x55978effc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f4448d63b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff3720 0x55978effc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f4448d63b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff40e0 0x55978f0037c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f4448d63b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978effbe50 con 0x55978eff3720 2023-09-30T14:48:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f4448d63b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978edc18c0 con 0x55978eff40e0 2023-09-30T14:48:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f4448d63b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978efea270 con 0x55978eff2610 2023-09-30T14:48:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f4435d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff3720 0x55978effc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f4435d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff3720 0x55978effc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37834/0 (socket says 172.21.15.32:37834) 2023-09-30T14:48:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f4435d9b700 1 -- 172.21.15.32:0/3022421495 learned_addr learned my addr 172.21.15.32:0/3022421495 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f4436d9d700 1 --2- 172.21.15.32:0/3022421495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff40e0 0x55978f0037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f443659c700 1 --2- 172.21.15.32:0/3022421495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff2610 0x55978effc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f4435d9b700 1 -- 172.21.15.32:0/3022421495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff2610 msgr2=0x55978effc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f4435d9b700 1 --2- 172.21.15.32:0/3022421495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff2610 0x55978effc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f4435d9b700 1 -- 172.21.15.32:0/3022421495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff40e0 msgr2=0x55978f0037c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f4435d9b700 1 --2- 172.21.15.32:0/3022421495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff40e0 0x55978f0037c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f4435d9b700 1 -- 172.21.15.32:0/3022421495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55978ed54c00 con 0x55978eff3720 2023-09-30T14:48:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f4435d9b700 1 --2- 172.21.15.32:0/3022421495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff3720 0x55978effc7c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f442c009c70 tx=0x7f442c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6131feabadc70c04 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f44257fa700 1 -- 172.21.15.32:0/3022421495 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c005c20 con 0x55978eff3720 2023-09-30T14:48:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f44257fa700 1 -- 172.21.15.32:0/3022421495 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f442c004960 con 0x55978eff3720 2023-09-30T14:48:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.225+0000 7f44257fa700 1 -- 172.21.15.32:0/3022421495 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c003680 con 0x55978eff3720 2023-09-30T14:48:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.226+0000 7f4448d63b80 1 -- 172.21.15.32:0/3022421495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff3720 msgr2=0x55978effc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.226+0000 7f4448d63b80 1 --2- 172.21.15.32:0/3022421495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff3720 0x55978effc7c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f442c009c70 tx=0x7f442c011720 comp rx=0 tx=0).stop 2023-09-30T14:48:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.226+0000 7f4448d63b80 1 -- 172.21.15.32:0/3022421495 shutdown_connections 2023-09-30T14:48:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.226+0000 7f4448d63b80 1 --2- 172.21.15.32:0/3022421495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff40e0 0x55978f0037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.226+0000 7f4448d63b80 1 --2- 172.21.15.32:0/3022421495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff2610 0x55978effc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.226+0000 7f4448d63b80 1 --2- 172.21.15.32:0/3022421495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff3720 0x55978effc7c0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.226+0000 7f4448d63b80 1 -- 172.21.15.32:0/3022421495 >> 172.21.15.32:0/3022421495 conn(0x55978ef62200 msgr2=0x55978f00ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.226+0000 7f4448d63b80 1 -- 172.21.15.32:0/3022421495 shutdown_connections 2023-09-30T14:48:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.226+0000 7f4448d63b80 1 -- 172.21.15.32:0/3022421495 wait complete. 2023-09-30T14:48:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.226+0000 7f4448d63b80 1 -- start start 2023-09-30T14:48:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f4448d63b80 1 Processor -- start 2023-09-30T14:48:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f4448d63b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff2610 0x55978f001120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f4448d63b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff3720 0x55978f001660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f4448d63b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff40e0 0x55978f002670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f4448d63b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978f002bb0 con 0x55978eff2610 2023-09-30T14:48:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f4448d63b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978f002d20 con 0x55978eff40e0 2023-09-30T14:48:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f4448d63b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978f002ee0 con 0x55978eff3720 2023-09-30T14:48:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f4448d63b80 10 client.0 _ll_register_callbacks cb 0x7f44300074e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:48:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f443659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff2610 0x55978f001120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f443659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff2610 0x55978f001120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37838/0 (socket says 172.21.15.32:37838) 2023-09-30T14:48:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f443659c700 1 -- 172.21.15.32:0/4209485644 learned_addr learned my addr 172.21.15.32:0/4209485644 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:48:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f4435d9b700 1 --2- 172.21.15.32:0/4209485644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff3720 0x55978f001660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f4436d9d700 1 --2- 172.21.15.32:0/4209485644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff40e0 0x55978f002670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f443659c700 1 -- 172.21.15.32:0/4209485644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff3720 msgr2=0x55978f001660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f443659c700 1 --2- 172.21.15.32:0/4209485644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff3720 0x55978f001660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f443659c700 1 -- 172.21.15.32:0/4209485644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff40e0 msgr2=0x55978f002670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:48:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f443659c700 1 --2- 172.21.15.32:0/4209485644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff40e0 0x55978f002670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.227+0000 7f443659c700 1 -- 172.21.15.32:0/4209485644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55978ed6bd80 con 0x55978eff2610 2023-09-30T14:48:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.228+0000 7f443659c700 1 --2- 172.21.15.32:0/4209485644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff2610 0x55978f001120 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4428005950 tx=0x7f44280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60407f46d3c788da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:48:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.228+0000 7f441dffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:48:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.228+0000 7f441dffb700 1 -- 172.21.15.32:0/4209485644 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428022040 con 0x55978eff2610 2023-09-30T14:48:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.228+0000 7f4448d63b80 1 -- 172.21.15.32:0/4209485644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f442c0054e0 con 0x55978eff2610 2023-09-30T14:48:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.228+0000 7f441dffb700 1 -- 172.21.15.32:0/4209485644 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f442802b2b0 con 0x55978eff2610 2023-09-30T14:48:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.228+0000 7f441dffb700 1 -- 172.21.15.32:0/4209485644 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f442802b470 con 0x55978eff2610 2023-09-30T14:48:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.228+0000 7f4448d63b80 10 client.16395 fetch_fsmap learned FSMap version 42 2023-09-30T14:48:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.228+0000 7f4448d63b80 1 -- 172.21.15.32:0/4209485644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55978ed921f0 con 0x55978eff2610 2023-09-30T14:48:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.228+0000 7f441dffb700 1 -- 172.21.15.32:0/4209485644 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f442c0054e0 con 0x55978eff2610 2023-09-30T14:48:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.228+0000 7f441dffb700 1 -- 172.21.15.32:0/4209485644 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 42) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f442803d070 con 0x55978eff2610 2023-09-30T14:48:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.229+0000 7f4448d63b80 10 client.16395 fetch_fsmap finished waiting for FSMap version 42 2023-09-30T14:48:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.229+0000 7f4448d63b80 10 client.16395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:48:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.229+0000 7f4448d63b80 20 client.16395 populate_metadata read hostname 'smithi032' 2023-09-30T14:48:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.229+0000 7f4448d63b80 1 --2- 172.21.15.32:0/4209485644 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55978efeea60 0x55978eff0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:48:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.229+0000 7f4448d63b80 4 client.16395 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:48:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.229+0000 7f4448d63b80 1 -- 172.21.15.32:0/4209485644 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55978ef39ad0 con 0x55978efeea60 2023-09-30T14:48:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.229+0000 7f4435d9b700 1 --2- 172.21.15.32:0/4209485644 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55978efeea60 0x55978eff0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:48:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.229+0000 7f441dffb700 10 client.16395 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:48:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.229+0000 7f4435d9b700 1 --2- 172.21.15.32:0/4209485644 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55978efeea60 0x55978eff0e80 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=174b481e41aacddd server_cookie=3b99f54a8306c778 in_seq=0 out_seq=0 2023-09-30T14:48:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.283+0000 7f441dffb700 1 -- 172.21.15.32:0/4209485644 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55978ef39ad0 con 0x55978efeea60 2023-09-30T14:48:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.283+0000 7f441dffb700 10 client.16395 handle_command_reply: tid=0 2023-09-30T14:48:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.283+0000 7f4448d63b80 1 client.16395 shutdown 2023-09-30T14:48:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.283+0000 7f4448d63b80 1 -- 172.21.15.32:0/4209485644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff2610 msgr2=0x55978f001120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.283+0000 7f4448d63b80 1 --2- 172.21.15.32:0/4209485644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff2610 0x55978f001120 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4428005950 tx=0x7f44280047c0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.283+0000 7f4448d63b80 1 -- 172.21.15.32:0/4209485644 shutdown_connections 2023-09-30T14:48:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.283+0000 7f4448d63b80 1 --2- 172.21.15.32:0/4209485644 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55978efeea60 0x55978eff0e80 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.283+0000 7f4448d63b80 1 --2- 172.21.15.32:0/4209485644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55978eff40e0 0x55978f002670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.283+0000 7f4448d63b80 1 --2- 172.21.15.32:0/4209485644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55978eff2610 0x55978f001120 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.283+0000 7f4448d63b80 1 --2- 172.21.15.32:0/4209485644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55978eff3720 0x55978f001660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.283+0000 7f4448d63b80 1 -- 172.21.15.32:0/4209485644 >> 172.21.15.32:0/4209485644 conn(0x55978ef62200 msgr2=0x55978effc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.283+0000 7f441dffb700 0 client.16395 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:48:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.283+0000 7f4448d63b80 1 -- 172.21.15.32:0/4209485644 shutdown_connections 2023-09-30T14:48:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.283+0000 7f4448d63b80 1 -- 172.21.15.32:0/4209485644 wait complete. 2023-09-30T14:48:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.283+0000 7f4448d63b80 20 client.16395 trim_cache size 0 max 16384 2023-09-30T14:48:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:48:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:48:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:48:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:48:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.284+0000 7f443cb7b700 1 -- 172.21.15.32:0/1028510798 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44200a0fa0 msgr2=0x7f44200a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.284+0000 7f443cb7b700 1 --2- 172.21.15.32:0/1028510798 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44200a0fa0 0x7f44200a3460 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4438068d20 tx=0x7f4428009260 comp rx=0 tx=0).stop 2023-09-30T14:48:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.284+0000 7f443cb7b700 1 -- 172.21.15.32:0/1028510798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44381018f0 msgr2=0x7f443819e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:48:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.284+0000 7f443cb7b700 1 --2- 172.21.15.32:0/1028510798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44381018f0 0x7f443819e3f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4430008ec0 tx=0x7f4430008ef0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.284+0000 7f443cb7b700 1 -- 172.21.15.32:0/1028510798 shutdown_connections 2023-09-30T14:48:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.284+0000 7f443cb7b700 1 --2- 172.21.15.32:0/1028510798 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f44200a0fa0 0x7f44200a3460 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.284+0000 7f443cb7b700 1 --2- 172.21.15.32:0/1028510798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44381018f0 0x7f443819e3f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.284+0000 7f443cb7b700 1 --2- 172.21.15.32:0/1028510798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44380ffd60 0x7f4438199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.284+0000 7f443cb7b700 1 --2- 172.21.15.32:0/1028510798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4438100f50 0x7f443819bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:48:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.284+0000 7f443cb7b700 1 -- 172.21.15.32:0/1028510798 >> 172.21.15.32:0/1028510798 conn(0x7f44380fb2b0 msgr2=0x7f4438104400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:48:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.284+0000 7f443cb7b700 1 -- 172.21.15.32:0/1028510798 shutdown_connections 2023-09-30T14:48:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:48:51.284+0000 7f443cb7b700 1 -- 172.21.15.32:0/1028510798 wait complete. 2023-09-30T14:48:51.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:51 smithi156 ceph-mon[103579]: pgmap v604: 97 pgs: 97 active+clean; 360 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 2.9 MiB/s wr, 301 op/s 2023-09-30T14:48:51.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:51 smithi148 ceph-mon[102355]: pgmap v604: 97 pgs: 97 active+clean; 360 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 2.9 MiB/s wr, 301 op/s 2023-09-30T14:48:51.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:51 smithi032 ceph-mon[98592]: pgmap v604: 97 pgs: 97 active+clean; 360 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 2.9 MiB/s wr, 301 op/s 2023-09-30T14:48:51.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1150925 10.71 MB/sec execute 445 sec latency 22.442 ms 2023-09-30T14:48:52.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1152494 10.70 MB/sec execute 446 sec latency 15.268 ms 2023-09-30T14:48:53.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:53 smithi156 ceph-mon[103579]: pgmap v605: 97 pgs: 97 active+clean; 362 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 3.0 MiB/s wr, 258 op/s 2023-09-30T14:48:53.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:53 smithi148 ceph-mon[102355]: pgmap v605: 97 pgs: 97 active+clean; 362 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 3.0 MiB/s wr, 258 op/s 2023-09-30T14:48:53.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:53 smithi032 ceph-mon[98592]: pgmap v605: 97 pgs: 97 active+clean; 362 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 3.0 MiB/s wr, 258 op/s 2023-09-30T14:48:53.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1154773 10.70 MB/sec execute 447 sec latency 12.044 ms 2023-09-30T14:48:54.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1156852 10.70 MB/sec execute 448 sec latency 39.921 ms 2023-09-30T14:48:55.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:55 smithi156 ceph-mon[103579]: pgmap v606: 97 pgs: 97 active+clean; 363 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 2.8 MiB/s wr, 260 op/s 2023-09-30T14:48:55.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:55 smithi148 ceph-mon[102355]: pgmap v606: 97 pgs: 97 active+clean; 363 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 2.8 MiB/s wr, 260 op/s 2023-09-30T14:48:55.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:55 smithi032 ceph-mon[98592]: pgmap v606: 97 pgs: 97 active+clean; 363 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 2.8 MiB/s wr, 260 op/s 2023-09-30T14:48:55.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1158515 10.70 MB/sec execute 449 sec latency 33.138 ms 2023-09-30T14:48:56.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1160906 10.70 MB/sec execute 450 sec latency 31.269 ms 2023-09-30T14:48:57.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:57 smithi156 ceph-mon[103579]: pgmap v607: 97 pgs: 97 active+clean; 364 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 2.9 MiB/s wr, 307 op/s 2023-09-30T14:48:57.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:57 smithi148 ceph-mon[102355]: pgmap v607: 97 pgs: 97 active+clean; 364 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 2.9 MiB/s wr, 307 op/s 2023-09-30T14:48:57.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:57 smithi032 ceph-mon[98592]: pgmap v607: 97 pgs: 97 active+clean; 364 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 2.9 MiB/s wr, 307 op/s 2023-09-30T14:48:57.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1162533 10.70 MB/sec execute 451 sec latency 14.329 ms 2023-09-30T14:48:58.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1165228 10.71 MB/sec execute 452 sec latency 14.776 ms 2023-09-30T14:48:59.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:48:59 smithi156 ceph-mon[103579]: pgmap v608: 97 pgs: 97 active+clean; 364 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 3.0 MiB/s wr, 286 op/s 2023-09-30T14:48:59.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:48:59 smithi148 ceph-mon[102355]: pgmap v608: 97 pgs: 97 active+clean; 364 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 3.0 MiB/s wr, 286 op/s 2023-09-30T14:48:59.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:48:59 smithi032 ceph-mon[98592]: pgmap v608: 97 pgs: 97 active+clean; 364 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 3.0 MiB/s wr, 286 op/s 2023-09-30T14:48:59.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1166703 10.70 MB/sec execute 453 sec latency 34.709 ms 2023-09-30T14:49:00.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:00 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:49:00.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:00 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:49:00.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:00 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:49:00.718 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1168885 10.71 MB/sec execute 454 sec latency 64.829 ms 2023-09-30T14:49:01.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:01 smithi148 ceph-mon[102355]: pgmap v609: 97 pgs: 97 active+clean; 368 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 3.0 MiB/s wr, 299 op/s 2023-09-30T14:49:01.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:01 smithi032 ceph-mon[98592]: pgmap v609: 97 pgs: 97 active+clean; 368 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 3.0 MiB/s wr, 299 op/s 2023-09-30T14:49:01.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1170957 10.70 MB/sec execute 455 sec latency 24.907 ms 2023-09-30T14:49:01.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:01 smithi156 ceph-mon[103579]: pgmap v609: 97 pgs: 97 active+clean; 368 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 3.0 MiB/s wr, 299 op/s 2023-09-30T14:49:02.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1172719 10.70 MB/sec execute 456 sec latency 11.222 ms 2023-09-30T14:49:03.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:03 smithi148 ceph-mon[102355]: pgmap v610: 97 pgs: 97 active+clean; 368 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 2.9 MiB/s wr, 285 op/s 2023-09-30T14:49:03.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:03 smithi032 ceph-mon[98592]: pgmap v610: 97 pgs: 97 active+clean; 368 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 2.9 MiB/s wr, 285 op/s 2023-09-30T14:49:03.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1175122 10.71 MB/sec execute 457 sec latency 18.453 ms 2023-09-30T14:49:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:03 smithi156 ceph-mon[103579]: pgmap v610: 97 pgs: 97 active+clean; 368 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 2.9 MiB/s wr, 285 op/s 2023-09-30T14:49:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.400+0000 7fd98e91c700 1 -- 172.21.15.32:0/4089913246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd988101950 msgr2=0x7fd9881081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.400+0000 7fd98e91c700 1 --2- 172.21.15.32:0/4089913246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd988101950 0x7fd9881081e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd98000b0a0 tx=0x7fd98000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.400+0000 7fd98e91c700 1 -- 172.21.15.32:0/4089913246 shutdown_connections 2023-09-30T14:49:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.400+0000 7fd98e91c700 1 --2- 172.21.15.32:0/4089913246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd988101950 0x7fd9881081e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.400+0000 7fd98e91c700 1 --2- 172.21.15.32:0/4089913246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd988100280 0x7fd9881006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.400+0000 7fd98e91c700 1 --2- 172.21.15.32:0/4089913246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd988100f90 0x7fd988101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.400+0000 7fd98e91c700 1 -- 172.21.15.32:0/4089913246 >> 172.21.15.32:0/4089913246 conn(0x7fd9880faa30 msgr2=0x7fd9880fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.400+0000 7fd98e91c700 1 -- 172.21.15.32:0/4089913246 shutdown_connections 2023-09-30T14:49:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.400+0000 7fd98e91c700 1 -- 172.21.15.32:0/4089913246 wait complete. 2023-09-30T14:49:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.400+0000 7fd98e91c700 1 Processor -- start 2023-09-30T14:49:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd98e91c700 1 -- start start 2023-09-30T14:49:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd98e91c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd988100280 0x7fd988195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd98e91c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd988100f90 0x7fd988196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd98e91c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd988101950 0x7fd988196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd98e91c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd988197670 con 0x7fd988101950 2023-09-30T14:49:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd98e91c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9881977e0 con 0x7fd988100280 2023-09-30T14:49:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd98e91c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9881979a0 con 0x7fd988100f90 2023-09-30T14:49:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd98ceb9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd988101950 0x7fd988196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd987fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd988100280 0x7fd988195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd98ceb9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd988101950 0x7fd988196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37856/0 (socket says 172.21.15.32:37856) 2023-09-30T14:49:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd9877fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd988100f90 0x7fd988196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd98ceb9700 1 -- 172.21.15.32:0/3660014232 learned_addr learned my addr 172.21.15.32:0/3660014232 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd98ceb9700 1 -- 172.21.15.32:0/3660014232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd988100f90 msgr2=0x7fd988196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd98ceb9700 1 --2- 172.21.15.32:0/3660014232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd988100f90 0x7fd988196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd98ceb9700 1 -- 172.21.15.32:0/3660014232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd988100280 msgr2=0x7fd988195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd98ceb9700 1 --2- 172.21.15.32:0/3660014232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd988100280 0x7fd988195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd98ceb9700 1 -- 172.21.15.32:0/3660014232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd978009520 con 0x7fd988101950 2023-09-30T14:49:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.401+0000 7fd9877fe700 1 --2- 172.21.15.32:0/3660014232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd988100f90 0x7fd988196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:49:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.402+0000 7fd98ceb9700 1 --2- 172.21.15.32:0/3660014232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd988101950 0x7fd988196f70 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd98000b780 tx=0x7fd98000be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.402+0000 7fd9857fa700 1 -- 172.21.15.32:0/3660014232 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98000e040 con 0x7fd988101950 2023-09-30T14:49:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.402+0000 7fd9857fa700 1 -- 172.21.15.32:0/3660014232 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd98001d940 con 0x7fd988101950 2023-09-30T14:49:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.402+0000 7fd9857fa700 1 -- 172.21.15.32:0/3660014232 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98001b990 con 0x7fd988101950 2023-09-30T14:49:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.402+0000 7fd98e91c700 1 -- 172.21.15.32:0/3660014232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd980009d00 con 0x7fd988101950 2023-09-30T14:49:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.402+0000 7fd98e91c700 1 -- 172.21.15.32:0/3660014232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98819e1c0 con 0x7fd988101950 2023-09-30T14:49:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.404+0000 7fd9857fa700 1 -- 172.21.15.32:0/3660014232 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fd98001d480 con 0x7fd988101950 2023-09-30T14:49:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.405+0000 7fd9857fa700 1 --2- 172.21.15.32:0/3660014232 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd9700a0fa0 0x7fd9700a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.405+0000 7fd987fff700 1 --2- 172.21.15.32:0/3660014232 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd9700a0fa0 0x7fd9700a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.405+0000 7fd9857fa700 1 -- 172.21.15.32:0/3660014232 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd9800ce520 con 0x7fd988101950 2023-09-30T14:49:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.405+0000 7fd987fff700 1 --2- 172.21.15.32:0/3660014232 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd9700a0fa0 0x7fd9700a3460 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd988067db0 tx=0x7fd978009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.406+0000 7fd99ab04b80 1 Processor -- start 2023-09-30T14:49:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.406+0000 7fd99ab04b80 1 -- start start 2023-09-30T14:49:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.406+0000 7fd99ab04b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db38200 0x55f18db38620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.406+0000 7fd99ab04b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db39310 0x55f18db39790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.406+0000 7fd99ab04b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db39cd0 0x55f18db40560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.406+0000 7fd99ab04b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18daa0400 con 0x55f18db38200 2023-09-30T14:49:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.406+0000 7fd99ab04b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18da91bc0 con 0x55f18db39310 2023-09-30T14:49:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.406+0000 7fd99ab04b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18da8f4e0 con 0x55f18db39cd0 2023-09-30T14:49:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.406+0000 7fd987fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db38200 0x55f18db38620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.406+0000 7fd987fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db38200 0x55f18db38620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37860/0 (socket says 172.21.15.32:37860) 2023-09-30T14:49:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.406+0000 7fd987fff700 1 -- 172.21.15.32:0/3569782466 learned_addr learned my addr 172.21.15.32:0/3569782466 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.406+0000 7fd9877fe700 1 --2- 172.21.15.32:0/3569782466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db39310 0x55f18db39790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.406+0000 7fd98ceb9700 1 --2- 172.21.15.32:0/3569782466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db39cd0 0x55f18db40560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.407+0000 7fd987fff700 1 -- 172.21.15.32:0/3569782466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db39cd0 msgr2=0x55f18db40560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.407+0000 7fd987fff700 1 --2- 172.21.15.32:0/3569782466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db39cd0 0x55f18db40560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.407+0000 7fd987fff700 1 -- 172.21.15.32:0/3569782466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db39310 msgr2=0x55f18db39790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.407+0000 7fd987fff700 1 --2- 172.21.15.32:0/3569782466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db39310 0x55f18db39790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.407+0000 7fd987fff700 1 -- 172.21.15.32:0/3569782466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f18daed870 con 0x55f18db38200 2023-09-30T14:49:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.407+0000 7fd987fff700 1 --2- 172.21.15.32:0/3569782466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db38200 0x55f18db38620 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd9780042c0 tx=0x7fd978014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a384665f04aae961 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.407+0000 7fd96e7fc700 1 -- 172.21.15.32:0/3569782466 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978003d10 con 0x55f18db38200 2023-09-30T14:49:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.407+0000 7fd96e7fc700 1 -- 172.21.15.32:0/3569782466 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd978003e70 con 0x55f18db38200 2023-09-30T14:49:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.407+0000 7fd96e7fc700 1 -- 172.21.15.32:0/3569782466 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978026700 con 0x55f18db38200 2023-09-30T14:49:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.408+0000 7fd99ab04b80 1 -- 172.21.15.32:0/3569782466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db38200 msgr2=0x55f18db38620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.408+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/3569782466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db38200 0x55f18db38620 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd9780042c0 tx=0x7fd978014000 comp rx=0 tx=0).stop 2023-09-30T14:49:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.408+0000 7fd99ab04b80 1 -- 172.21.15.32:0/3569782466 shutdown_connections 2023-09-30T14:49:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.408+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/3569782466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db39cd0 0x55f18db40560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.408+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/3569782466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db38200 0x55f18db38620 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.408+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/3569782466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db39310 0x55f18db39790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.408+0000 7fd99ab04b80 1 -- 172.21.15.32:0/3569782466 >> 172.21.15.32:0/3569782466 conn(0x55f18daac200 msgr2=0x55f18db32ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.408+0000 7fd99ab04b80 1 -- 172.21.15.32:0/3569782466 shutdown_connections 2023-09-30T14:49:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.408+0000 7fd99ab04b80 1 -- 172.21.15.32:0/3569782466 wait complete. 2023-09-30T14:49:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.408+0000 7fd99ab04b80 1 -- start start 2023-09-30T14:49:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.408+0000 7fd99ab04b80 1 Processor -- start 2023-09-30T14:49:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.408+0000 7fd99ab04b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db38200 0x55f18db41f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd99ab04b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db39310 0x55f18db424d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd99ab04b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db39cd0 0x55f18db49510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd99ab04b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18da8add0 con 0x55f18db38200 2023-09-30T14:49:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd99ab04b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18da276b0 con 0x55f18db39310 2023-09-30T14:49:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd99ab04b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18da27350 con 0x55f18db39cd0 2023-09-30T14:49:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd99ab04b80 10 client.0 _ll_register_callbacks cb 0x55f18d8847a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:49:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd987fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db38200 0x55f18db41f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd987fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db38200 0x55f18db41f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37866/0 (socket says 172.21.15.32:37866) 2023-09-30T14:49:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd987fff700 1 -- 172.21.15.32:0/949514081 learned_addr learned my addr 172.21.15.32:0/949514081 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd98ceb9700 1 --2- 172.21.15.32:0/949514081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db39cd0 0x55f18db49510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd9877fe700 1 --2- 172.21.15.32:0/949514081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db39310 0x55f18db424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd987fff700 1 -- 172.21.15.32:0/949514081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db39cd0 msgr2=0x55f18db49510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd987fff700 1 --2- 172.21.15.32:0/949514081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db39cd0 0x55f18db49510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd987fff700 1 -- 172.21.15.32:0/949514081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db39310 msgr2=0x55f18db424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd987fff700 1 --2- 172.21.15.32:0/949514081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db39310 0x55f18db424d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd987fff700 1 -- 172.21.15.32:0/949514081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd978017070 con 0x55f18db38200 2023-09-30T14:49:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd987fff700 1 --2- 172.21.15.32:0/949514081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db38200 0x55f18db41f90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd978005270 tx=0x7fd97801ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc0036891432a47 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd9677fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:49:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd9677fe700 1 -- 172.21.15.32:0/949514081 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97802c070 con 0x55f18db38200 2023-09-30T14:49:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd9677fe700 1 -- 172.21.15.32:0/949514081 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd9780055e0 con 0x55f18db38200 2023-09-30T14:49:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.409+0000 7fd99ab04b80 1 -- 172.21.15.32:0/949514081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f18db2b090 con 0x55f18db38200 2023-09-30T14:49:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.410+0000 7fd9677fe700 1 -- 172.21.15.32:0/949514081 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd978037030 con 0x55f18db38200 2023-09-30T14:49:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.410+0000 7fd99ab04b80 10 client.16407 fetch_fsmap learned FSMap version 43 2023-09-30T14:49:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.410+0000 7fd99ab04b80 1 -- 172.21.15.32:0/949514081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f18d8b5d80 con 0x55f18db38200 2023-09-30T14:49:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.410+0000 7fd9677fe700 1 -- 172.21.15.32:0/949514081 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f18db2b090 con 0x55f18db38200 2023-09-30T14:49:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.410+0000 7fd9677fe700 1 -- 172.21.15.32:0/949514081 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd978024020 con 0x55f18db38200 2023-09-30T14:49:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.410+0000 7fd99ab04b80 10 client.16407 fetch_fsmap finished waiting for FSMap version 43 2023-09-30T14:49:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.410+0000 7fd99ab04b80 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:49:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.410+0000 7fd99ab04b80 20 client.16407 populate_metadata read hostname 'smithi032' 2023-09-30T14:49:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.410+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/949514081 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f18db4dc50 0x55f18db50070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.410+0000 7fd99ab04b80 4 client.16407 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:49:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.410+0000 7fd99ab04b80 1 -- 172.21.15.32:0/949514081 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f18da83db0 con 0x55f18db4dc50 2023-09-30T14:49:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.410+0000 7fd9877fe700 1 --2- 172.21.15.32:0/949514081 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f18db4dc50 0x55f18db50070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.411+0000 7fd9677fe700 10 client.16407 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.411+0000 7fd9877fe700 1 --2- 172.21.15.32:0/949514081 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f18db4dc50 0x55f18db50070 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b28d00e1c7a3a5 server_cookie=c51259095ae1b8e in_seq=0 out_seq=0 2023-09-30T14:49:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.411+0000 7fd9677fe700 1 -- 172.21.15.32:0/949514081 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f18da83db0 con 0x55f18db4dc50 2023-09-30T14:49:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.411+0000 7fd9677fe700 10 client.16407 handle_command_reply: tid=0 2023-09-30T14:49:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.411+0000 7fd99ab04b80 1 client.16407 shutdown 2023-09-30T14:49:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.411+0000 7fd99ab04b80 1 -- 172.21.15.32:0/949514081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db38200 msgr2=0x55f18db41f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.411+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/949514081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db38200 0x55f18db41f90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd978005270 tx=0x7fd97801ee00 comp rx=0 tx=0).stop 2023-09-30T14:49:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.411+0000 7fd99ab04b80 1 -- 172.21.15.32:0/949514081 shutdown_connections 2023-09-30T14:49:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.411+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/949514081 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f18db4dc50 0x55f18db50070 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.412+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/949514081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db39cd0 0x55f18db49510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.412+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/949514081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db38200 0x55f18db41f90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.412+0000 7fd9677fe700 0 client.16407 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.412+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/949514081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db39310 0x55f18db424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.412+0000 7fd99ab04b80 1 -- 172.21.15.32:0/949514081 >> 172.21.15.32:0/949514081 conn(0x55f18daac200 msgr2=0x55f18db32590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.412+0000 7fd99ab04b80 1 -- 172.21.15.32:0/949514081 shutdown_connections 2023-09-30T14:49:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.412+0000 7fd99ab04b80 1 -- 172.21.15.32:0/949514081 wait complete. 2023-09-30T14:49:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.412+0000 7fd99ab04b80 20 client.16407 trim_cache size 0 max 16384 2023-09-30T14:49:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd99ab04b80 1 Processor -- start 2023-09-30T14:49:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd99ab04b80 1 -- start start 2023-09-30T14:49:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd99ab04b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db38200 0x55f18db40150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd99ab04b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db39310 0x55f18db494d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd99ab04b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db39cd0 0x55f18db3cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd99ab04b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18d90b8c0 con 0x55f18db39cd0 2023-09-30T14:49:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd99ab04b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18db3ff00 con 0x55f18db38200 2023-09-30T14:49:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd99ab04b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18db3d7f0 con 0x55f18db39310 2023-09-30T14:49:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd98ceb9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db39cd0 0x55f18db3cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd98ceb9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db39cd0 0x55f18db3cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37876/0 (socket says 172.21.15.32:37876) 2023-09-30T14:49:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd98ceb9700 1 -- 172.21.15.32:0/2892919365 learned_addr learned my addr 172.21.15.32:0/2892919365 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd987fff700 1 --2- 172.21.15.32:0/2892919365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db38200 0x55f18db40150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd9877fe700 1 --2- 172.21.15.32:0/2892919365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db39310 0x55f18db494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd98ceb9700 1 -- 172.21.15.32:0/2892919365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db39310 msgr2=0x55f18db494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd98ceb9700 1 --2- 172.21.15.32:0/2892919365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db39310 0x55f18db494d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd98ceb9700 1 -- 172.21.15.32:0/2892919365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db38200 msgr2=0x55f18db40150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd98ceb9700 1 --2- 172.21.15.32:0/2892919365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db38200 0x55f18db40150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.427+0000 7fd98ceb9700 1 -- 172.21.15.32:0/2892919365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f18d89ec00 con 0x55f18db39cd0 2023-09-30T14:49:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.428+0000 7fd98ceb9700 1 --2- 172.21.15.32:0/2892919365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db39cd0 0x55f18db3cc60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd980009fd0 tx=0x7fd980009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4323bcb9b1e8c596 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.428+0000 7fd96effd700 1 -- 172.21.15.32:0/2892919365 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98009c120 con 0x55f18db39cd0 2023-09-30T14:49:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.428+0000 7fd96effd700 1 -- 172.21.15.32:0/2892919365 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd98009ba20 con 0x55f18db39cd0 2023-09-30T14:49:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.428+0000 7fd96effd700 1 -- 172.21.15.32:0/2892919365 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd980099ea0 con 0x55f18db39cd0 2023-09-30T14:49:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.428+0000 7fd99ab04b80 1 -- 172.21.15.32:0/2892919365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db39cd0 msgr2=0x55f18db3cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.428+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/2892919365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db39cd0 0x55f18db3cc60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd980009fd0 tx=0x7fd980009040 comp rx=0 tx=0).stop 2023-09-30T14:49:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.429+0000 7fd99ab04b80 1 -- 172.21.15.32:0/2892919365 shutdown_connections 2023-09-30T14:49:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.429+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/2892919365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db39cd0 0x55f18db3cc60 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.429+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/2892919365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db38200 0x55f18db40150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.429+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/2892919365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db39310 0x55f18db494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.429+0000 7fd99ab04b80 1 -- 172.21.15.32:0/2892919365 >> 172.21.15.32:0/2892919365 conn(0x55f18daac200 msgr2=0x55f18db40c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.429+0000 7fd99ab04b80 1 -- 172.21.15.32:0/2892919365 shutdown_connections 2023-09-30T14:49:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.429+0000 7fd99ab04b80 1 -- 172.21.15.32:0/2892919365 wait complete. 2023-09-30T14:49:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.429+0000 7fd99ab04b80 1 -- start start 2023-09-30T14:49:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.429+0000 7fd99ab04b80 1 Processor -- start 2023-09-30T14:49:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd99ab04b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db38200 0x55f18db48ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd99ab04b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db39310 0x55f18db49420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd99ab04b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db39cd0 0x55f18db474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd99ab04b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18db47c00 con 0x55f18db39310 2023-09-30T14:49:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd99ab04b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18db47d70 con 0x55f18db39cd0 2023-09-30T14:49:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd99ab04b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18db47f30 con 0x55f18db38200 2023-09-30T14:49:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd99ab04b80 10 client.0 _ll_register_callbacks cb 0x7fd97c009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:49:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd9877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db39310 0x55f18db49420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd9877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db39310 0x55f18db49420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37880/0 (socket says 172.21.15.32:37880) 2023-09-30T14:49:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd9877fe700 1 -- 172.21.15.32:0/2928878400 learned_addr learned my addr 172.21.15.32:0/2928878400 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd98ceb9700 1 --2- 172.21.15.32:0/2928878400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db39cd0 0x55f18db474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd987fff700 1 --2- 172.21.15.32:0/2928878400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db38200 0x55f18db48ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd9877fe700 1 -- 172.21.15.32:0/2928878400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db38200 msgr2=0x55f18db48ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd9877fe700 1 --2- 172.21.15.32:0/2928878400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db38200 0x55f18db48ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd9877fe700 1 -- 172.21.15.32:0/2928878400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db39cd0 msgr2=0x55f18db474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd9877fe700 1 --2- 172.21.15.32:0/2928878400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db39cd0 0x55f18db474f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd9877fe700 1 -- 172.21.15.32:0/2928878400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98819e1c0 con 0x55f18db39310 2023-09-30T14:49:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd987fff700 1 --2- 172.21.15.32:0/2928878400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db38200 0x55f18db48ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:49:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd9877fe700 1 --2- 172.21.15.32:0/2928878400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db39310 0x55f18db49420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd97c005d90 tx=0x7fd97c004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ab16ee3832717d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd9677fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:49:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd9677fe700 1 -- 172.21.15.32:0/2928878400 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c017040 con 0x55f18db39310 2023-09-30T14:49:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd9677fe700 1 -- 172.21.15.32:0/2928878400 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd97c013b80 con 0x55f18db39310 2023-09-30T14:49:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd99ab04b80 1 -- 172.21.15.32:0/2928878400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd978003e70 con 0x55f18db39310 2023-09-30T14:49:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.430+0000 7fd9677fe700 1 -- 172.21.15.32:0/2928878400 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd97c02c030 con 0x55f18db39310 2023-09-30T14:49:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.431+0000 7fd99ab04b80 10 client.16413 fetch_fsmap learned FSMap version 43 2023-09-30T14:49:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.431+0000 7fd99ab04b80 1 -- 172.21.15.32:0/2928878400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f18d8dc1f0 con 0x55f18db39310 2023-09-30T14:49:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.431+0000 7fd9677fe700 1 -- 172.21.15.32:0/2928878400 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd978003e70 con 0x55f18db39310 2023-09-30T14:49:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.431+0000 7fd9677fe700 1 -- 172.21.15.32:0/2928878400 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd97c024020 con 0x55f18db39310 2023-09-30T14:49:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.431+0000 7fd99ab04b80 10 client.16413 fetch_fsmap finished waiting for FSMap version 43 2023-09-30T14:49:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.431+0000 7fd99ab04b80 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:49:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.431+0000 7fd99ab04b80 20 client.16413 populate_metadata read hostname 'smithi032' 2023-09-30T14:49:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.431+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/2928878400 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f18dba4fa0 0x55f18dba73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.431+0000 7fd99ab04b80 4 client.16413 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:49:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.431+0000 7fd99ab04b80 1 -- 172.21.15.32:0/2928878400 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f18da83ad0 con 0x55f18dba4fa0 2023-09-30T14:49:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.431+0000 7fd987fff700 1 --2- 172.21.15.32:0/2928878400 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f18dba4fa0 0x55f18dba73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.431+0000 7fd987fff700 1 --2- 172.21.15.32:0/2928878400 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f18dba4fa0 0x55f18dba73c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5d82cfa4f8449b1 server_cookie=3e609f4e7973fb79 in_seq=0 out_seq=0 2023-09-30T14:49:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.431+0000 7fd9677fe700 10 client.16413 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.476+0000 7fd9677fe700 1 -- 172.21.15.32:0/2928878400 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f18da83ad0 con 0x55f18dba4fa0 2023-09-30T14:49:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.476+0000 7fd9677fe700 10 client.16413 handle_command_reply: tid=0 2023-09-30T14:49:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.477+0000 7fd99ab04b80 1 client.16413 shutdown 2023-09-30T14:49:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.477+0000 7fd99ab04b80 1 -- 172.21.15.32:0/2928878400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db39310 msgr2=0x55f18db49420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.477+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/2928878400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db39310 0x55f18db49420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd97c005d90 tx=0x7fd97c004320 comp rx=0 tx=0).stop 2023-09-30T14:49:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.477+0000 7fd99ab04b80 1 -- 172.21.15.32:0/2928878400 shutdown_connections 2023-09-30T14:49:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.477+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/2928878400 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f18dba4fa0 0x55f18dba73c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.477+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/2928878400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f18db39cd0 0x55f18db474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.477+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/2928878400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f18db38200 0x55f18db48ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.477+0000 7fd99ab04b80 1 --2- 172.21.15.32:0/2928878400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f18db39310 0x55f18db49420 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.477+0000 7fd99ab04b80 1 -- 172.21.15.32:0/2928878400 >> 172.21.15.32:0/2928878400 conn(0x55f18daac200 msgr2=0x55f18db40150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.477+0000 7fd9677fe700 0 client.16413 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.477+0000 7fd99ab04b80 1 -- 172.21.15.32:0/2928878400 shutdown_connections 2023-09-30T14:49:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.477+0000 7fd99ab04b80 1 -- 172.21.15.32:0/2928878400 wait complete. 2023-09-30T14:49:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.477+0000 7fd99ab04b80 20 client.16413 trim_cache size 0 max 16384 2023-09-30T14:49:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:49:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:49:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:49:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:49:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.478+0000 7fd98e91c700 1 -- 172.21.15.32:0/3660014232 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd9700a0fa0 msgr2=0x7fd9700a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.478+0000 7fd98e91c700 1 --2- 172.21.15.32:0/3660014232 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd9700a0fa0 0x7fd9700a3460 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd988067db0 tx=0x7fd978009260 comp rx=0 tx=0).stop 2023-09-30T14:49:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.478+0000 7fd98e91c700 1 -- 172.21.15.32:0/3660014232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd988101950 msgr2=0x7fd988196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.478+0000 7fd98e91c700 1 --2- 172.21.15.32:0/3660014232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd988101950 0x7fd988196f70 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd98000b780 tx=0x7fd98000be40 comp rx=0 tx=0).stop 2023-09-30T14:49:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.478+0000 7fd98e91c700 1 -- 172.21.15.32:0/3660014232 shutdown_connections 2023-09-30T14:49:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.478+0000 7fd98e91c700 1 --2- 172.21.15.32:0/3660014232 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd9700a0fa0 0x7fd9700a3460 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.478+0000 7fd98e91c700 1 --2- 172.21.15.32:0/3660014232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd988101950 0x7fd988196f70 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.478+0000 7fd98e91c700 1 --2- 172.21.15.32:0/3660014232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd988100280 0x7fd988195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.478+0000 7fd98e91c700 1 --2- 172.21.15.32:0/3660014232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd988100f90 0x7fd988196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.478+0000 7fd98e91c700 1 -- 172.21.15.32:0/3660014232 >> 172.21.15.32:0/3660014232 conn(0x7fd9880faa30 msgr2=0x7fd988104450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.478+0000 7fd98e91c700 1 -- 172.21.15.32:0/3660014232 shutdown_connections 2023-09-30T14:49:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:04.478+0000 7fd98e91c700 1 -- 172.21.15.32:0/3660014232 wait complete. 2023-09-30T14:49:04.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1176581 10.70 MB/sec execute 458 sec latency 36.052 ms 2023-09-30T14:49:05.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:05 smithi148 ceph-mon[102355]: pgmap v611: 97 pgs: 97 active+clean; 368 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 2.7 MiB/s wr, 272 op/s 2023-09-30T14:49:05.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:05 smithi032 ceph-mon[98592]: pgmap v611: 97 pgs: 97 active+clean; 368 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 2.7 MiB/s wr, 272 op/s 2023-09-30T14:49:05.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1179033 10.71 MB/sec execute 459 sec latency 31.337 ms 2023-09-30T14:49:05.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:05 smithi156 ceph-mon[103579]: pgmap v611: 97 pgs: 97 active+clean; 368 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 2.7 MiB/s wr, 272 op/s 2023-09-30T14:49:06.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1180438 10.70 MB/sec execute 460 sec latency 20.932 ms 2023-09-30T14:49:07.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:07 smithi148 ceph-mon[102355]: pgmap v612: 97 pgs: 97 active+clean; 370 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 3.0 MiB/s wr, 331 op/s 2023-09-30T14:49:07.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:07 smithi032 ceph-mon[98592]: pgmap v612: 97 pgs: 97 active+clean; 370 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 3.0 MiB/s wr, 331 op/s 2023-09-30T14:49:07.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1182956 10.71 MB/sec execute 461 sec latency 14.837 ms 2023-09-30T14:49:07.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:07 smithi156 ceph-mon[103579]: pgmap v612: 97 pgs: 97 active+clean; 370 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 3.0 MiB/s wr, 331 op/s 2023-09-30T14:49:08.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1184441 10.70 MB/sec execute 462 sec latency 15.249 ms 2023-09-30T14:49:09.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:09 smithi148 ceph-mon[102355]: pgmap v613: 97 pgs: 97 active+clean; 371 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 2.7 MiB/s wr, 261 op/s 2023-09-30T14:49:09.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:09 smithi032 ceph-mon[98592]: pgmap v613: 97 pgs: 97 active+clean; 371 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 2.7 MiB/s wr, 261 op/s 2023-09-30T14:49:09.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1186666 10.71 MB/sec execute 463 sec latency 12.003 ms 2023-09-30T14:49:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:09 smithi156 ceph-mon[103579]: pgmap v613: 97 pgs: 97 active+clean; 371 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 2.7 MiB/s wr, 261 op/s 2023-09-30T14:49:10.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1188639 10.70 MB/sec execute 464 sec latency 95.496 ms 2023-09-30T14:49:11.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:11 smithi148 ceph-mon[102355]: pgmap v614: 97 pgs: 97 active+clean; 372 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 2.5 MiB/s wr, 281 op/s 2023-09-30T14:49:11.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:11 smithi032 ceph-mon[98592]: pgmap v614: 97 pgs: 97 active+clean; 372 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 2.5 MiB/s wr, 281 op/s 2023-09-30T14:49:11.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1190300 10.70 MB/sec execute 465 sec latency 99.976 ms 2023-09-30T14:49:11.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:11 smithi156 ceph-mon[103579]: pgmap v614: 97 pgs: 97 active+clean; 372 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 2.5 MiB/s wr, 281 op/s 2023-09-30T14:49:12.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1192829 10.71 MB/sec execute 466 sec latency 19.520 ms 2023-09-30T14:49:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:13 smithi148 ceph-mon[102355]: pgmap v615: 97 pgs: 97 active+clean; 374 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 2.5 MiB/s wr, 259 op/s 2023-09-30T14:49:13.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:13 smithi032 ceph-mon[98592]: pgmap v615: 97 pgs: 97 active+clean; 374 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 2.5 MiB/s wr, 259 op/s 2023-09-30T14:49:13.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1194683 10.70 MB/sec execute 467 sec latency 14.023 ms 2023-09-30T14:49:13.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:13 smithi156 ceph-mon[103579]: pgmap v615: 97 pgs: 97 active+clean; 374 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 2.5 MiB/s wr, 259 op/s 2023-09-30T14:49:14.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1197237 10.71 MB/sec execute 468 sec latency 12.063 ms 2023-09-30T14:49:15.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:15 smithi148 ceph-mon[102355]: pgmap v616: 97 pgs: 97 active+clean; 377 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.6 MiB/s wr, 275 op/s 2023-09-30T14:49:15.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:15 smithi032 ceph-mon[98592]: pgmap v616: 97 pgs: 97 active+clean; 377 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.6 MiB/s wr, 275 op/s 2023-09-30T14:49:15.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1199487 10.71 MB/sec execute 469 sec latency 15.112 ms 2023-09-30T14:49:15.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:15 smithi156 ceph-mon[103579]: pgmap v616: 97 pgs: 97 active+clean; 377 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.6 MiB/s wr, 275 op/s 2023-09-30T14:49:16.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1201010 10.71 MB/sec execute 470 sec latency 88.271 ms 2023-09-30T14:49:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.589+0000 7ffac9570700 1 -- 172.21.15.32:0/69657803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac41018f0 msgr2=0x7ffac41081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.589+0000 7ffac9570700 1 --2- 172.21.15.32:0/69657803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac41018f0 0x7ffac41081a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00b0d0 tx=0x7ffabc00b3e0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.590+0000 7ffac9570700 1 -- 172.21.15.32:0/69657803 shutdown_connections 2023-09-30T14:49:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.590+0000 7ffac9570700 1 --2- 172.21.15.32:0/69657803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac41018f0 0x7ffac41081a0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.590+0000 7ffac9570700 1 --2- 172.21.15.32:0/69657803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffac40ffd60 0x7ffac4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.590+0000 7ffac9570700 1 --2- 172.21.15.32:0/69657803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffac4100f50 0x7ffac41013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.590+0000 7ffac9570700 1 -- 172.21.15.32:0/69657803 >> 172.21.15.32:0/69657803 conn(0x7ffac40fb2b0 msgr2=0x7ffac40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.590+0000 7ffac9570700 1 -- 172.21.15.32:0/69657803 shutdown_connections 2023-09-30T14:49:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.590+0000 7ffac9570700 1 -- 172.21.15.32:0/69657803 wait complete. 2023-09-30T14:49:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.590+0000 7ffac9570700 1 Processor -- start 2023-09-30T14:49:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.590+0000 7ffac9570700 1 -- start start 2023-09-30T14:49:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffac9570700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffac40ffd60 0x7ffac4199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffac9570700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac4100f50 0x7ffac419be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffac9570700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffac41018f0 0x7ffac419e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffac9570700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac419e9d0 con 0x7ffac4100f50 2023-09-30T14:49:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffac9570700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac419eb40 con 0x7ffac40ffd60 2023-09-30T14:49:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffac9570700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac419ed00 con 0x7ffac41018f0 2023-09-30T14:49:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffaba7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac4100f50 0x7ffac419be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffac2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffac40ffd60 0x7ffac4199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffaba7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac4100f50 0x7ffac419be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37894/0 (socket says 172.21.15.32:37894) 2023-09-30T14:49:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffac2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffac40ffd60 0x7ffac4199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49368/0 (socket says 172.21.15.32:49368) 2023-09-30T14:49:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffaba7fc700 1 -- 172.21.15.32:0/360782651 learned_addr learned my addr 172.21.15.32:0/360782651 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffac37fe700 1 --2- 172.21.15.32:0/360782651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffac41018f0 0x7ffac419e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffaba7fc700 1 -- 172.21.15.32:0/360782651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffac41018f0 msgr2=0x7ffac419e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffaba7fc700 1 --2- 172.21.15.32:0/360782651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffac41018f0 0x7ffac419e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffaba7fc700 1 -- 172.21.15.32:0/360782651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffac40ffd60 msgr2=0x7ffac4199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffaba7fc700 1 --2- 172.21.15.32:0/360782651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffac40ffd60 0x7ffac4199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffaba7fc700 1 -- 172.21.15.32:0/360782651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac009520 con 0x7ffac4100f50 2023-09-30T14:49:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffaba7fc700 1 --2- 172.21.15.32:0/360782651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac4100f50 0x7ffac419be80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00eb30 tx=0x7ffaac00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.591+0000 7ffac0ff9700 1 -- 172.21.15.32:0/360782651 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac009880 con 0x7ffac4100f50 2023-09-30T14:49:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.592+0000 7ffac9570700 1 -- 172.21.15.32:0/360782651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffabc009d00 con 0x7ffac4100f50 2023-09-30T14:49:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.592+0000 7ffac0ff9700 1 -- 172.21.15.32:0/360782651 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaac0044e0 con 0x7ffac4100f50 2023-09-30T14:49:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.592+0000 7ffac0ff9700 1 -- 172.21.15.32:0/360782651 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac003d80 con 0x7ffac4100f50 2023-09-30T14:49:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.592+0000 7ffac9570700 1 -- 172.21.15.32:0/360782651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac419f2d0 con 0x7ffac4100f50 2023-09-30T14:49:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.594+0000 7ffac0ff9700 1 -- 172.21.15.32:0/360782651 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7ffaac004000 con 0x7ffac4100f50 2023-09-30T14:49:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.594+0000 7ffac0ff9700 1 --2- 172.21.15.32:0/360782651 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffab00a0fa0 0x7ffab00a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.594+0000 7ffac0ff9700 1 -- 172.21.15.32:0/360782651 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffaac014070 con 0x7ffac4100f50 2023-09-30T14:49:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.594+0000 7ffac2ffd700 1 --2- 172.21.15.32:0/360782651 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffab00a0fa0 0x7ffab00a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.595+0000 7ffac2ffd700 1 --2- 172.21.15.32:0/360782651 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffab00a0fa0 0x7ffab00a3460 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffac4068d60 tx=0x7ffab400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.595+0000 7ffad5758b80 1 Processor -- start 2023-09-30T14:49:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.595+0000 7ffad5758b80 1 -- start start 2023-09-30T14:49:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffad5758b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1d610 0x55b0b8e1da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffad5758b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1e720 0x55b0b8e1eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffad5758b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e25970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffad5758b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b8d81400 con 0x55b0b8e1d610 2023-09-30T14:49:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffad5758b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b8d72bc0 con 0x55b0b8e1f0e0 2023-09-30T14:49:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffad5758b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b8d704e0 con 0x55b0b8e1e720 2023-09-30T14:49:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffac2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1d610 0x55b0b8e1da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffac2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1d610 0x55b0b8e1da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37900/0 (socket says 172.21.15.32:37900) 2023-09-30T14:49:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffac2ffd700 1 -- 172.21.15.32:0/1680579081 learned_addr learned my addr 172.21.15.32:0/1680579081 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffaba7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1e720 0x55b0b8e1eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffac37fe700 1 --2- 172.21.15.32:0/1680579081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e25970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffac2ffd700 1 -- 172.21.15.32:0/1680579081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1e720 msgr2=0x55b0b8e1eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffac2ffd700 1 --2- 172.21.15.32:0/1680579081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1e720 0x55b0b8e1eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffac2ffd700 1 -- 172.21.15.32:0/1680579081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1f0e0 msgr2=0x55b0b8e25970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffac2ffd700 1 --2- 172.21.15.32:0/1680579081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e25970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffac2ffd700 1 -- 172.21.15.32:0/1680579081 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0b8dce870 con 0x55b0b8e1d610 2023-09-30T14:49:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffaba7fc700 1 --2- 172.21.15.32:0/1680579081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1e720 0x55b0b8e1eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:49:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.596+0000 7ffac2ffd700 1 --2- 172.21.15.32:0/1680579081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1d610 0x55b0b8e1da30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ffab4010f70 tx=0x7ffab4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f474d9327505fe1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.597+0000 7ffab97fa700 1 -- 172.21.15.32:0/1680579081 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4011db0 con 0x55b0b8e1d610 2023-09-30T14:49:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.597+0000 7ffab97fa700 1 -- 172.21.15.32:0/1680579081 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffab4015070 con 0x55b0b8e1d610 2023-09-30T14:49:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.597+0000 7ffab97fa700 1 -- 172.21.15.32:0/1680579081 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4003d10 con 0x55b0b8e1d610 2023-09-30T14:49:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.597+0000 7ffad5758b80 1 -- 172.21.15.32:0/1680579081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1d610 msgr2=0x55b0b8e1da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.597+0000 7ffad5758b80 1 --2- 172.21.15.32:0/1680579081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1d610 0x55b0b8e1da30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ffab4010f70 tx=0x7ffab4013000 comp rx=0 tx=0).stop 2023-09-30T14:49:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.597+0000 7ffad5758b80 1 -- 172.21.15.32:0/1680579081 shutdown_connections 2023-09-30T14:49:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.597+0000 7ffad5758b80 1 --2- 172.21.15.32:0/1680579081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e25970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.597+0000 7ffad5758b80 1 --2- 172.21.15.32:0/1680579081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1d610 0x55b0b8e1da30 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.597+0000 7ffad5758b80 1 --2- 172.21.15.32:0/1680579081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1e720 0x55b0b8e1eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.597+0000 7ffad5758b80 1 -- 172.21.15.32:0/1680579081 >> 172.21.15.32:0/1680579081 conn(0x55b0b8d8d200 msgr2=0x55b0b8e13ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.598+0000 7ffad5758b80 1 -- 172.21.15.32:0/1680579081 shutdown_connections 2023-09-30T14:49:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.598+0000 7ffad5758b80 1 -- 172.21.15.32:0/1680579081 wait complete. 2023-09-30T14:49:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.598+0000 7ffad5758b80 1 -- start start 2023-09-30T14:49:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.598+0000 7ffad5758b80 1 Processor -- start 2023-09-30T14:49:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.598+0000 7ffad5758b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1d610 0x55b0b8e272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.598+0000 7ffad5758b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1e720 0x55b0b8e27820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.598+0000 7ffad5758b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e2e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.598+0000 7ffad5758b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b8d6bdd0 con 0x55b0b8e1d610 2023-09-30T14:49:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.598+0000 7ffad5758b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b8d086b0 con 0x55b0b8e1e720 2023-09-30T14:49:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.598+0000 7ffad5758b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b8d08350 con 0x55b0b8e1f0e0 2023-09-30T14:49:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.598+0000 7ffad5758b80 10 client.0 _ll_register_callbacks cb 0x55b0b8b657a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:49:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.598+0000 7ffac2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1d610 0x55b0b8e272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.598+0000 7ffac2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1d610 0x55b0b8e272e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37906/0 (socket says 172.21.15.32:37906) 2023-09-30T14:49:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.598+0000 7ffac2ffd700 1 -- 172.21.15.32:0/1468880080 learned_addr learned my addr 172.21.15.32:0/1468880080 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:17.609 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.598+0000 7ffac37fe700 1 --2- 172.21.15.32:0/1468880080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e2e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.598+0000 7ffaba7fc700 1 --2- 172.21.15.32:0/1468880080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1e720 0x55b0b8e27820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.599+0000 7ffac2ffd700 1 -- 172.21.15.32:0/1468880080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1f0e0 msgr2=0x55b0b8e2e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.599+0000 7ffac2ffd700 1 --2- 172.21.15.32:0/1468880080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e2e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.599+0000 7ffac2ffd700 1 -- 172.21.15.32:0/1468880080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1e720 msgr2=0x55b0b8e27820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.599+0000 7ffac2ffd700 1 --2- 172.21.15.32:0/1468880080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1e720 0x55b0b8e27820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.599+0000 7ffac2ffd700 1 -- 172.21.15.32:0/1468880080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac419f2d0 con 0x55b0b8e1d610 2023-09-30T14:49:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.599+0000 7ffaba7fc700 1 --2- 172.21.15.32:0/1468880080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1e720 0x55b0b8e27820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:49:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.599+0000 7ffac2ffd700 1 --2- 172.21.15.32:0/1468880080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1d610 0x55b0b8e272e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ffab400bfd0 tx=0x7ffab4011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba47a4ec8223641 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.599+0000 7ffaaa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:49:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.599+0000 7ffaaa7fc700 1 -- 172.21.15.32:0/1468880080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4003ea0 con 0x55b0b8e1d610 2023-09-30T14:49:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.599+0000 7ffad5758b80 1 -- 172.21.15.32:0/1468880080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0b8e0c090 con 0x55b0b8e1d610 2023-09-30T14:49:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.599+0000 7ffaaa7fc700 1 -- 172.21.15.32:0/1468880080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffab4017070 con 0x55b0b8e1d610 2023-09-30T14:49:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.599+0000 7ffaaa7fc700 1 -- 172.21.15.32:0/1468880080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffab4004000 con 0x55b0b8e1d610 2023-09-30T14:49:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.599+0000 7ffad5758b80 10 client.16425 fetch_fsmap learned FSMap version 44 2023-09-30T14:49:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.599+0000 7ffad5758b80 1 -- 172.21.15.32:0/1468880080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7ffab4010c20 con 0x55b0b8e1d610 2023-09-30T14:49:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.599+0000 7ffaaa7fc700 1 -- 172.21.15.32:0/1468880080 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b0b8e0c090 con 0x55b0b8e1d610 2023-09-30T14:49:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.600+0000 7ffaaa7fc700 1 -- 172.21.15.32:0/1468880080 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffab401ea90 con 0x55b0b8e1d610 2023-09-30T14:49:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.600+0000 7ffad5758b80 10 client.16425 fetch_fsmap finished waiting for FSMap version 44 2023-09-30T14:49:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.600+0000 7ffad5758b80 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:49:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.600+0000 7ffad5758b80 20 client.16425 populate_metadata read hostname 'smithi032' 2023-09-30T14:49:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.600+0000 7ffad5758b80 1 --2- 172.21.15.32:0/1468880080 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b0b8e33080 0x55b0b8e354a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.600+0000 7ffad5758b80 4 client.16425 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:49:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.600+0000 7ffad5758b80 1 -- 172.21.15.32:0/1468880080 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0b8d64db0 con 0x55b0b8e33080 2023-09-30T14:49:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.600+0000 7ffaba7fc700 1 --2- 172.21.15.32:0/1468880080 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b0b8e33080 0x55b0b8e354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.600+0000 7ffaaa7fc700 10 client.16425 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.600+0000 7ffaba7fc700 1 --2- 172.21.15.32:0/1468880080 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b0b8e33080 0x55b0b8e354a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59910e851e5de590 server_cookie=2be6ee3d5cab069c in_seq=0 out_seq=0 2023-09-30T14:49:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.601+0000 7ffaaa7fc700 1 -- 172.21.15.32:0/1468880080 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b0b8d64db0 con 0x55b0b8e33080 2023-09-30T14:49:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.601+0000 7ffaaa7fc700 10 client.16425 handle_command_reply: tid=0 2023-09-30T14:49:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.601+0000 7ffad5758b80 1 client.16425 shutdown 2023-09-30T14:49:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.601+0000 7ffad5758b80 1 -- 172.21.15.32:0/1468880080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1d610 msgr2=0x55b0b8e272e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.601+0000 7ffad5758b80 1 --2- 172.21.15.32:0/1468880080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1d610 0x55b0b8e272e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ffab400bfd0 tx=0x7ffab4011de0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.601+0000 7ffad5758b80 1 -- 172.21.15.32:0/1468880080 shutdown_connections 2023-09-30T14:49:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.601+0000 7ffad5758b80 1 --2- 172.21.15.32:0/1468880080 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b0b8e33080 0x55b0b8e354a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.601+0000 7ffad5758b80 1 --2- 172.21.15.32:0/1468880080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e2e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.601+0000 7ffad5758b80 1 --2- 172.21.15.32:0/1468880080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1d610 0x55b0b8e272e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.601+0000 7ffaaa7fc700 0 client.16425 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.601+0000 7ffad5758b80 1 --2- 172.21.15.32:0/1468880080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1e720 0x55b0b8e27820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.601+0000 7ffad5758b80 1 -- 172.21.15.32:0/1468880080 >> 172.21.15.32:0/1468880080 conn(0x55b0b8d8d200 msgr2=0x55b0b8e13590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.601+0000 7ffad5758b80 1 -- 172.21.15.32:0/1468880080 shutdown_connections 2023-09-30T14:49:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.601+0000 7ffad5758b80 1 -- 172.21.15.32:0/1468880080 wait complete. 2023-09-30T14:49:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.601+0000 7ffad5758b80 20 client.16425 trim_cache size 0 max 16384 2023-09-30T14:49:17.615 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:17 smithi148 ceph-mon[102355]: pgmap v617: 97 pgs: 97 active+clean; 376 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 2.7 MiB/s wr, 290 op/s 2023-09-30T14:49:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.617+0000 7ffad5758b80 1 Processor -- start 2023-09-30T14:49:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.617+0000 7ffad5758b80 1 -- start start 2023-09-30T14:49:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffad5758b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1d610 0x55b0b8e13ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffad5758b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1e720 0x55b0b8e2e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffad5758b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e13590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffad5758b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b8bec8c0 con 0x55b0b8e1e720 2023-09-30T14:49:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffad5758b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b8e10ba0 con 0x55b0b8e1d610 2023-09-30T14:49:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffad5758b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b8e0bbd0 con 0x55b0b8e1f0e0 2023-09-30T14:49:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffaba7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1e720 0x55b0b8e2e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffac2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1d610 0x55b0b8e13ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffaba7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1e720 0x55b0b8e2e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37916/0 (socket says 172.21.15.32:37916) 2023-09-30T14:49:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffaba7fc700 1 -- 172.21.15.32:0/1295051921 learned_addr learned my addr 172.21.15.32:0/1295051921 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffac37fe700 1 --2- 172.21.15.32:0/1295051921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e13590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffac2ffd700 1 -- 172.21.15.32:0/1295051921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1f0e0 msgr2=0x55b0b8e13590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffac2ffd700 1 --2- 172.21.15.32:0/1295051921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e13590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffac2ffd700 1 -- 172.21.15.32:0/1295051921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1e720 msgr2=0x55b0b8e2e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffac2ffd700 1 --2- 172.21.15.32:0/1295051921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1e720 0x55b0b8e2e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffac2ffd700 1 -- 172.21.15.32:0/1295051921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0b8b96d80 con 0x55b0b8e1d610 2023-09-30T14:49:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffaba7fc700 1 --2- 172.21.15.32:0/1295051921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1e720 0x55b0b8e2e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:49:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffac2ffd700 1 --2- 172.21.15.32:0/1295051921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1d610 0x55b0b8e13ec0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffab401eef0 tx=0x7ffab4005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=821292271a2a853d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.618+0000 7ffab9ffb700 1 -- 172.21.15.32:0/1295051921 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4026440 con 0x55b0b8e1d610 2023-09-30T14:49:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.619+0000 7ffab9ffb700 1 -- 172.21.15.32:0/1295051921 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffab400b580 con 0x55b0b8e1d610 2023-09-30T14:49:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.619+0000 7ffab9ffb700 1 -- 172.21.15.32:0/1295051921 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab401e820 con 0x55b0b8e1d610 2023-09-30T14:49:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.619+0000 7ffad5758b80 1 -- 172.21.15.32:0/1295051921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1d610 msgr2=0x55b0b8e13ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.619+0000 7ffad5758b80 1 --2- 172.21.15.32:0/1295051921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1d610 0x55b0b8e13ec0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffab401eef0 tx=0x7ffab4005a00 comp rx=0 tx=0).stop 2023-09-30T14:49:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.619+0000 7ffad5758b80 1 -- 172.21.15.32:0/1295051921 shutdown_connections 2023-09-30T14:49:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.619+0000 7ffad5758b80 1 --2- 172.21.15.32:0/1295051921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e13590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.619+0000 7ffad5758b80 1 --2- 172.21.15.32:0/1295051921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1d610 0x55b0b8e13ec0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.619+0000 7ffad5758b80 1 --2- 172.21.15.32:0/1295051921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1e720 0x55b0b8e2e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.619+0000 7ffad5758b80 1 -- 172.21.15.32:0/1295051921 >> 172.21.15.32:0/1295051921 conn(0x55b0b8d8d200 msgr2=0x55b0b8e39e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.619+0000 7ffad5758b80 1 -- 172.21.15.32:0/1295051921 shutdown_connections 2023-09-30T14:49:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.619+0000 7ffad5758b80 1 -- 172.21.15.32:0/1295051921 wait complete. 2023-09-30T14:49:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffad5758b80 1 -- start start 2023-09-30T14:49:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffad5758b80 1 Processor -- start 2023-09-30T14:49:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffad5758b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1d610 0x55b0b8e2c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffad5758b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1e720 0x55b0b8e2c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffad5758b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e2d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffad5758b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b8e2deb0 con 0x55b0b8e1f0e0 2023-09-30T14:49:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffad5758b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b8e2e020 con 0x55b0b8e1d610 2023-09-30T14:49:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffad5758b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b8e2e1e0 con 0x55b0b8e1e720 2023-09-30T14:49:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffad5758b80 10 client.0 _ll_register_callbacks cb 0x7ffaac005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:49:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffac37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e2d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffac37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e2d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37922/0 (socket says 172.21.15.32:37922) 2023-09-30T14:49:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffac37fe700 1 -- 172.21.15.32:0/3634448324 learned_addr learned my addr 172.21.15.32:0/3634448324 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffaba7fc700 1 --2- 172.21.15.32:0/3634448324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1e720 0x55b0b8e2c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffac2ffd700 1 --2- 172.21.15.32:0/3634448324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1d610 0x55b0b8e2c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffac37fe700 1 -- 172.21.15.32:0/3634448324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1e720 msgr2=0x55b0b8e2c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffac37fe700 1 --2- 172.21.15.32:0/3634448324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1e720 0x55b0b8e2c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffac37fe700 1 -- 172.21.15.32:0/3634448324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1d610 msgr2=0x55b0b8e2c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.620+0000 7ffac37fe700 1 --2- 172.21.15.32:0/3634448324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1d610 0x55b0b8e2c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffac37fe700 1 -- 172.21.15.32:0/3634448324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffabc009d00 con 0x55b0b8e1f0e0 2023-09-30T14:49:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffac37fe700 1 --2- 172.21.15.32:0/3634448324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e2d970 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00b0d0 tx=0x7ffabc007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7cc977cb2d64dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffaaa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:49:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffaaa7fc700 1 -- 172.21.15.32:0/3634448324 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc00e040 con 0x55b0b8e1f0e0 2023-09-30T14:49:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffaaa7fc700 1 -- 172.21.15.32:0/3634448324 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffabc007800 con 0x55b0b8e1f0e0 2023-09-30T14:49:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffad5758b80 1 -- 172.21.15.32:0/3634448324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffab4015070 con 0x55b0b8e1f0e0 2023-09-30T14:49:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffaaa7fc700 1 -- 172.21.15.32:0/3634448324 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffabc003a00 con 0x55b0b8e1f0e0 2023-09-30T14:49:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffad5758b80 10 client.16431 fetch_fsmap learned FSMap version 44 2023-09-30T14:49:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffad5758b80 1 -- 172.21.15.32:0/3634448324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7ffab4010c20 con 0x55b0b8e1f0e0 2023-09-30T14:49:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffaaa7fc700 1 -- 172.21.15.32:0/3634448324 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffab4015070 con 0x55b0b8e1f0e0 2023-09-30T14:49:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffaaa7fc700 1 -- 172.21.15.32:0/3634448324 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffabc00b100 con 0x55b0b8e1f0e0 2023-09-30T14:49:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffad5758b80 10 client.16431 fetch_fsmap finished waiting for FSMap version 44 2023-09-30T14:49:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffad5758b80 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:49:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffad5758b80 20 client.16431 populate_metadata read hostname 'smithi032' 2023-09-30T14:49:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffad5758b80 1 --2- 172.21.15.32:0/3634448324 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b0b8e87da0 0x55b0b8e8a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffad5758b80 4 client.16431 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:49:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.621+0000 7ffad5758b80 1 -- 172.21.15.32:0/3634448324 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b0b8d64ad0 con 0x55b0b8e87da0 2023-09-30T14:49:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.622+0000 7ffac2ffd700 1 --2- 172.21.15.32:0/3634448324 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b0b8e87da0 0x55b0b8e8a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.622+0000 7ffaaa7fc700 10 client.16431 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.622+0000 7ffac2ffd700 1 --2- 172.21.15.32:0/3634448324 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b0b8e87da0 0x55b0b8e8a1c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e74695a5f9231bd server_cookie=c348eae44349eab5 in_seq=0 out_seq=0 2023-09-30T14:49:17.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:17 smithi032 ceph-mon[98592]: pgmap v617: 97 pgs: 97 active+clean; 376 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 2.7 MiB/s wr, 290 op/s 2023-09-30T14:49:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.675+0000 7ffaaa7fc700 1 -- 172.21.15.32:0/3634448324 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b0b8d64ad0 con 0x55b0b8e87da0 2023-09-30T14:49:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.675+0000 7ffaaa7fc700 10 client.16431 handle_command_reply: tid=0 2023-09-30T14:49:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.675+0000 7ffad5758b80 1 client.16431 shutdown 2023-09-30T14:49:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.675+0000 7ffad5758b80 1 -- 172.21.15.32:0/3634448324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1f0e0 msgr2=0x55b0b8e2d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.675+0000 7ffad5758b80 1 --2- 172.21.15.32:0/3634448324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e2d970 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00b0d0 tx=0x7ffabc007710 comp rx=0 tx=0).stop 2023-09-30T14:49:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.675+0000 7ffad5758b80 1 -- 172.21.15.32:0/3634448324 shutdown_connections 2023-09-30T14:49:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.675+0000 7ffad5758b80 1 --2- 172.21.15.32:0/3634448324 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b0b8e87da0 0x55b0b8e8a1c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.675+0000 7ffad5758b80 1 --2- 172.21.15.32:0/3634448324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b0b8e1f0e0 0x55b0b8e2d970 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.675+0000 7ffaaa7fc700 0 client.16431 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.675+0000 7ffad5758b80 1 --2- 172.21.15.32:0/3634448324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b0b8e1d610 0x55b0b8e2c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.675+0000 7ffad5758b80 1 --2- 172.21.15.32:0/3634448324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b0b8e1e720 0x55b0b8e2c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.675+0000 7ffad5758b80 1 -- 172.21.15.32:0/3634448324 >> 172.21.15.32:0/3634448324 conn(0x55b0b8d8d200 msgr2=0x55b0b8e254f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.675+0000 7ffad5758b80 1 -- 172.21.15.32:0/3634448324 shutdown_connections 2023-09-30T14:49:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.675+0000 7ffad5758b80 1 -- 172.21.15.32:0/3634448324 wait complete. 2023-09-30T14:49:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.675+0000 7ffad5758b80 20 client.16431 trim_cache size 0 max 16384 2023-09-30T14:49:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:49:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:49:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:49:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:49:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.676+0000 7ffac9570700 1 -- 172.21.15.32:0/360782651 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffab00a0fa0 msgr2=0x7ffab00a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.676+0000 7ffac9570700 1 --2- 172.21.15.32:0/360782651 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffab00a0fa0 0x7ffab00a3460 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffac4068d60 tx=0x7ffab400b410 comp rx=0 tx=0).stop 2023-09-30T14:49:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.676+0000 7ffac9570700 1 -- 172.21.15.32:0/360782651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac4100f50 msgr2=0x7ffac419be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.676+0000 7ffac9570700 1 --2- 172.21.15.32:0/360782651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac4100f50 0x7ffac419be80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00eb30 tx=0x7ffaac00ee40 comp rx=0 tx=0).stop 2023-09-30T14:49:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.676+0000 7ffac9570700 1 -- 172.21.15.32:0/360782651 shutdown_connections 2023-09-30T14:49:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.676+0000 7ffac9570700 1 --2- 172.21.15.32:0/360782651 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffab00a0fa0 0x7ffab00a3460 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.676+0000 7ffac9570700 1 --2- 172.21.15.32:0/360782651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffac41018f0 0x7ffac419e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.676+0000 7ffac9570700 1 --2- 172.21.15.32:0/360782651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffac40ffd60 0x7ffac4199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.676+0000 7ffac9570700 1 --2- 172.21.15.32:0/360782651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffac4100f50 0x7ffac419be80 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.676+0000 7ffac9570700 1 -- 172.21.15.32:0/360782651 >> 172.21.15.32:0/360782651 conn(0x7ffac40fb2b0 msgr2=0x7ffac40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.676+0000 7ffac9570700 1 -- 172.21.15.32:0/360782651 shutdown_connections 2023-09-30T14:49:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:17.676+0000 7ffac9570700 1 -- 172.21.15.32:0/360782651 wait complete. 2023-09-30T14:49:17.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1203459 10.71 MB/sec execute 471 sec latency 25.104 ms 2023-09-30T14:49:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:17 smithi156 ceph-mon[103579]: pgmap v617: 97 pgs: 97 active+clean; 376 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 2.7 MiB/s wr, 290 op/s 2023-09-30T14:49:18.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:18 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:49:18.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:18 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:49:18.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1205000 10.71 MB/sec execute 472 sec latency 48.200 ms 2023-09-30T14:49:18.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:18 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:49:19.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:19 smithi148 ceph-mon[102355]: pgmap v618: 97 pgs: 97 active+clean; 379 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 2.7 MiB/s wr, 283 op/s 2023-09-30T14:49:19.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:19 smithi032 ceph-mon[98592]: pgmap v618: 97 pgs: 97 active+clean; 379 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 2.7 MiB/s wr, 283 op/s 2023-09-30T14:49:19.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1207704 10.72 MB/sec execute 473 sec latency 50.901 ms 2023-09-30T14:49:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:19 smithi156 ceph-mon[103579]: pgmap v618: 97 pgs: 97 active+clean; 379 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 2.7 MiB/s wr, 283 op/s 2023-09-30T14:49:20.719 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1209255 10.71 MB/sec execute 474 sec latency 14.051 ms 2023-09-30T14:49:21.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:21 smithi148 ceph-mon[102355]: pgmap v619: 97 pgs: 97 active+clean; 375 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 2.8 MiB/s wr, 273 op/s 2023-09-30T14:49:21.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:21 smithi032 ceph-mon[98592]: pgmap v619: 97 pgs: 97 active+clean; 375 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 2.8 MiB/s wr, 273 op/s 2023-09-30T14:49:21.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1211379 10.72 MB/sec execute 475 sec latency 36.778 ms 2023-09-30T14:49:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:21 smithi156 ceph-mon[103579]: pgmap v619: 97 pgs: 97 active+clean; 375 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 2.8 MiB/s wr, 273 op/s 2023-09-30T14:49:22.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1213005 10.71 MB/sec execute 476 sec latency 19.776 ms 2023-09-30T14:49:23.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:23 smithi148 ceph-mon[102355]: pgmap v620: 97 pgs: 97 active+clean; 377 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 3.0 MiB/s wr, 272 op/s 2023-09-30T14:49:23.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:23 smithi032 ceph-mon[98592]: pgmap v620: 97 pgs: 97 active+clean; 377 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 3.0 MiB/s wr, 272 op/s 2023-09-30T14:49:23.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1215096 10.71 MB/sec execute 477 sec latency 11.615 ms 2023-09-30T14:49:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:23 smithi156 ceph-mon[103579]: pgmap v620: 97 pgs: 97 active+clean; 377 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 3.0 MiB/s wr, 272 op/s 2023-09-30T14:49:24.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1217162 10.71 MB/sec execute 478 sec latency 48.772 ms 2023-09-30T14:49:25.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:25 smithi148 ceph-mon[102355]: pgmap v621: 97 pgs: 97 active+clean; 379 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 2.9 MiB/s wr, 271 op/s 2023-09-30T14:49:25.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:25 smithi032 ceph-mon[98592]: pgmap v621: 97 pgs: 97 active+clean; 379 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 2.9 MiB/s wr, 271 op/s 2023-09-30T14:49:25.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1218932 10.71 MB/sec execute 479 sec latency 18.291 ms 2023-09-30T14:49:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:25 smithi156 ceph-mon[103579]: pgmap v621: 97 pgs: 97 active+clean; 379 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 2.9 MiB/s wr, 271 op/s 2023-09-30T14:49:26.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1221088 10.71 MB/sec execute 480 sec latency 36.548 ms 2023-09-30T14:49:27.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:27 smithi148 ceph-mon[102355]: pgmap v622: 97 pgs: 97 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 2.9 MiB/s wr, 279 op/s 2023-09-30T14:49:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:27 smithi032 ceph-mon[98592]: pgmap v622: 97 pgs: 97 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 2.9 MiB/s wr, 279 op/s 2023-09-30T14:49:27.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1222498 10.70 MB/sec execute 481 sec latency 23.234 ms 2023-09-30T14:49:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:27 smithi156 ceph-mon[103579]: pgmap v622: 97 pgs: 97 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 2.9 MiB/s wr, 279 op/s 2023-09-30T14:49:28.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1224940 10.71 MB/sec execute 482 sec latency 15.231 ms 2023-09-30T14:49:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:29 smithi148 ceph-mon[102355]: pgmap v623: 97 pgs: 97 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 2.9 MiB/s wr, 293 op/s 2023-09-30T14:49:29.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:29 smithi032 ceph-mon[98592]: pgmap v623: 97 pgs: 97 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 2.9 MiB/s wr, 293 op/s 2023-09-30T14:49:29.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1226480 10.70 MB/sec execute 483 sec latency 12.422 ms 2023-09-30T14:49:29.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:29 smithi156 ceph-mon[103579]: pgmap v623: 97 pgs: 97 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 2.9 MiB/s wr, 293 op/s 2023-09-30T14:49:30.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:30 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:49:30.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:30 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:49:30.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1228907 10.71 MB/sec execute 484 sec latency 18.601 ms 2023-09-30T14:49:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.795+0000 7f491e61f700 1 -- 172.21.15.32:0/2009288278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49181009c0 msgr2=0x7f4918108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.795+0000 7f491e61f700 1 --2- 172.21.15.32:0/2009288278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49181009c0 0x7f4918108260 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f491000f530 tx=0x7f491000f840 comp rx=0 tx=0).stop 2023-09-30T14:49:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.795+0000 7f491e61f700 1 -- 172.21.15.32:0/2009288278 shutdown_connections 2023-09-30T14:49:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.795+0000 7f491e61f700 1 --2- 172.21.15.32:0/2009288278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49181009c0 0x7f4918108260 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.795+0000 7f491e61f700 1 --2- 172.21.15.32:0/2009288278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49181013b0 0x7f49181017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.795+0000 7f491e61f700 1 --2- 172.21.15.32:0/2009288278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4918100000 0x7f4918100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.795+0000 7f491e61f700 1 -- 172.21.15.32:0/2009288278 >> 172.21.15.32:0/2009288278 conn(0x7f49180faa90 msgr2=0x7f49180fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.795+0000 7f491e61f700 1 -- 172.21.15.32:0/2009288278 shutdown_connections 2023-09-30T14:49:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.795+0000 7f491e61f700 1 -- 172.21.15.32:0/2009288278 wait complete. 2023-09-30T14:49:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.795+0000 7f491e61f700 1 Processor -- start 2023-09-30T14:49:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.795+0000 7f491e61f700 1 -- start start 2023-09-30T14:49:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f491e61f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4918100000 0x7f4918194ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f491e61f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49181009c0 0x7f49181950e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f491e61f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49181013b0 0x7f4918195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f491e61f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49181965c0 con 0x7f49181013b0 2023-09-30T14:49:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f491e61f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4918196730 con 0x7f4918100000 2023-09-30T14:49:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f491e61f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49181968f0 con 0x7f49181009c0 2023-09-30T14:49:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f491cbbc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49181013b0 0x7f4918195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f491cbbc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49181013b0 0x7f4918195f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37938/0 (socket says 172.21.15.32:37938) 2023-09-30T14:49:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f4917fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4918100000 0x7f4918194ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f491cbbc700 1 -- 172.21.15.32:0/364391153 learned_addr learned my addr 172.21.15.32:0/364391153 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f49177fe700 1 --2- 172.21.15.32:0/364391153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49181009c0 0x7f49181950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f49177fe700 1 -- 172.21.15.32:0/364391153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4918100000 msgr2=0x7f4918194ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f49177fe700 1 --2- 172.21.15.32:0/364391153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4918100000 0x7f4918194ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f49177fe700 1 -- 172.21.15.32:0/364391153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49181013b0 msgr2=0x7f4918195f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f49177fe700 1 --2- 172.21.15.32:0/364391153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49181013b0 0x7f4918195f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f49177fe700 1 -- 172.21.15.32:0/364391153 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4910009d00 con 0x7f49181009c0 2023-09-30T14:49:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f4917fff700 1 --2- 172.21.15.32:0/364391153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4918100000 0x7f4918194ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:49:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.796+0000 7f491cbbc700 1 --2- 172.21.15.32:0/364391153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49181013b0 0x7f4918195f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:49:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.797+0000 7f49177fe700 1 --2- 172.21.15.32:0/364391153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49181009c0 0x7f49181950e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f490c007950 tx=0x7f490c007c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.797+0000 7f49157fa700 1 -- 172.21.15.32:0/364391153 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c010040 con 0x7f49181009c0 2023-09-30T14:49:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.797+0000 7f491e61f700 1 -- 172.21.15.32:0/364391153 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4918196bd0 con 0x7f49181009c0 2023-09-30T14:49:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.797+0000 7f49157fa700 1 -- 172.21.15.32:0/364391153 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f490c0044e0 con 0x7f49181009c0 2023-09-30T14:49:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.797+0000 7f49157fa700 1 -- 172.21.15.32:0/364391153 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c005280 con 0x7f49181009c0 2023-09-30T14:49:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.797+0000 7f491e61f700 1 -- 172.21.15.32:0/364391153 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491819d390 con 0x7f49181009c0 2023-09-30T14:49:30.811 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:30 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:49:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.799+0000 7f49157fa700 1 -- 172.21.15.32:0/364391153 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f490c0048c0 con 0x7f49181009c0 2023-09-30T14:49:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.800+0000 7f49157fa700 1 --2- 172.21.15.32:0/364391153 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f49000a0fa0 0x7f49000a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.800+0000 7f49157fa700 1 -- 172.21.15.32:0/364391153 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f490c0cc430 con 0x7f49181009c0 2023-09-30T14:49:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.800+0000 7f4917fff700 1 --2- 172.21.15.32:0/364391153 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f49000a0fa0 0x7f49000a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.800+0000 7f492a807b80 1 Processor -- start 2023-09-30T14:49:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.800+0000 7f492a807b80 1 -- start start 2023-09-30T14:49:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f4917fff700 1 --2- 172.21.15.32:0/364391153 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f49000a0fa0 0x7f49000a3460 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4918068f30 tx=0x7f4908009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f492a807b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139202200 0x55b139202620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f492a807b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203310 0x55b139203790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f492a807b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203cd0 0x55b13920a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f492a807b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13916a400 con 0x55b139202200 2023-09-30T14:49:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f492a807b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13915bbc0 con 0x55b139203cd0 2023-09-30T14:49:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f492a807b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1391594e0 con 0x55b139203310 2023-09-30T14:49:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f49177fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203310 0x55b139203790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f49177fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203310 0x55b139203790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52326/0 (socket says 172.21.15.32:52326) 2023-09-30T14:49:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f491cbbc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203cd0 0x55b13920a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f49177fe700 1 -- 172.21.15.32:0/122151754 learned_addr learned my addr 172.21.15.32:0/122151754 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f4917fff700 1 --2- 172.21.15.32:0/122151754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139202200 0x55b139202620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f49177fe700 1 -- 172.21.15.32:0/122151754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203cd0 msgr2=0x55b13920a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f49177fe700 1 --2- 172.21.15.32:0/122151754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203cd0 0x55b13920a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f49177fe700 1 -- 172.21.15.32:0/122151754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139202200 msgr2=0x55b139202620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f49177fe700 1 --2- 172.21.15.32:0/122151754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139202200 0x55b139202620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f49177fe700 1 -- 172.21.15.32:0/122151754 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1391b7870 con 0x55b139203310 2023-09-30T14:49:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.801+0000 7f49177fe700 1 --2- 172.21.15.32:0/122151754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203310 0x55b139203790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f490c099830 tx=0x7f490c099280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ab70d90a4dbde71 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.802+0000 7f48fe7fc700 1 -- 172.21.15.32:0/122151754 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0b90d0 con 0x55b139203310 2023-09-30T14:49:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.802+0000 7f48fe7fc700 1 -- 172.21.15.32:0/122151754 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f490c0998a0 con 0x55b139203310 2023-09-30T14:49:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.802+0000 7f48fe7fc700 1 -- 172.21.15.32:0/122151754 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c098d90 con 0x55b139203310 2023-09-30T14:49:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.802+0000 7f492a807b80 1 -- 172.21.15.32:0/122151754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203310 msgr2=0x55b139203790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.802+0000 7f492a807b80 1 --2- 172.21.15.32:0/122151754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203310 0x55b139203790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f490c099830 tx=0x7f490c099280 comp rx=0 tx=0).stop 2023-09-30T14:49:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.802+0000 7f492a807b80 1 -- 172.21.15.32:0/122151754 shutdown_connections 2023-09-30T14:49:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.802+0000 7f492a807b80 1 --2- 172.21.15.32:0/122151754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203cd0 0x55b13920a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.802+0000 7f492a807b80 1 --2- 172.21.15.32:0/122151754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139202200 0x55b139202620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.802+0000 7f492a807b80 1 --2- 172.21.15.32:0/122151754 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203310 0x55b139203790 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.802+0000 7f492a807b80 1 -- 172.21.15.32:0/122151754 >> 172.21.15.32:0/122151754 conn(0x55b139176200 msgr2=0x55b1391fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.803+0000 7f492a807b80 1 -- 172.21.15.32:0/122151754 shutdown_connections 2023-09-30T14:49:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.803+0000 7f492a807b80 1 -- 172.21.15.32:0/122151754 wait complete. 2023-09-30T14:49:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.803+0000 7f492a807b80 1 -- start start 2023-09-30T14:49:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.803+0000 7f492a807b80 1 Processor -- start 2023-09-30T14:49:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.803+0000 7f492a807b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139202200 0x55b13920bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.803+0000 7f492a807b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203310 0x55b13920c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f492a807b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203cd0 0x55b139213480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f492a807b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b139154dd0 con 0x55b139202200 2023-09-30T14:49:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f492a807b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1390f16b0 con 0x55b139203cd0 2023-09-30T14:49:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f492a807b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1390f1350 con 0x55b139203310 2023-09-30T14:49:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f492a807b80 10 client.0 _ll_register_callbacks cb 0x55b138f4e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:49:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f49177fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203310 0x55b13920c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f491cbbc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203cd0 0x55b139213480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f49177fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203310 0x55b13920c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52330/0 (socket says 172.21.15.32:52330) 2023-09-30T14:49:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f49177fe700 1 -- 172.21.15.32:0/1320679369 learned_addr learned my addr 172.21.15.32:0/1320679369 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f491cbbc700 1 -- 172.21.15.32:0/1320679369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203310 msgr2=0x55b13920c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f491cbbc700 1 --2- 172.21.15.32:0/1320679369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203310 0x55b13920c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f491cbbc700 1 -- 172.21.15.32:0/1320679369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139202200 msgr2=0x55b13920bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f491cbbc700 1 --2- 172.21.15.32:0/1320679369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139202200 0x55b13920bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f491cbbc700 1 -- 172.21.15.32:0/1320679369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491819d390 con 0x55b139203cd0 2023-09-30T14:49:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f49177fe700 1 --2- 172.21.15.32:0/1320679369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203310 0x55b13920c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:49:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f491cbbc700 1 --2- 172.21.15.32:0/1320679369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203cd0 0x55b139213480 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4910012040 tx=0x7f4910017e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f70908a308316a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.804+0000 7f48f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:49:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.805+0000 7f48f77fe700 1 -- 172.21.15.32:0/1320679369 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910017860 con 0x55b139203cd0 2023-09-30T14:49:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.805+0000 7f48f77fe700 1 -- 172.21.15.32:0/1320679369 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4910004a50 con 0x55b139203cd0 2023-09-30T14:49:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.805+0000 7f492a807b80 1 -- 172.21.15.32:0/1320679369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1391f5090 con 0x55b139203cd0 2023-09-30T14:49:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.805+0000 7f48f77fe700 1 -- 172.21.15.32:0/1320679369 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49100179c0 con 0x55b139203cd0 2023-09-30T14:49:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.805+0000 7f492a807b80 10 client.26126 fetch_fsmap learned FSMap version 44 2023-09-30T14:49:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.805+0000 7f492a807b80 1 -- 172.21.15.32:0/1320679369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55b138f7fd80 con 0x55b139203cd0 2023-09-30T14:49:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.805+0000 7f48f77fe700 1 -- 172.21.15.32:0/1320679369 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b1391f5090 con 0x55b139203cd0 2023-09-30T14:49:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.805+0000 7f48f77fe700 1 -- 172.21.15.32:0/1320679369 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4910005040 con 0x55b139203cd0 2023-09-30T14:49:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.805+0000 7f492a807b80 10 client.26126 fetch_fsmap finished waiting for FSMap version 44 2023-09-30T14:49:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.805+0000 7f492a807b80 10 client.26126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:49:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.805+0000 7f492a807b80 20 client.26126 populate_metadata read hostname 'smithi032' 2023-09-30T14:49:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.805+0000 7f492a807b80 1 --2- 172.21.15.32:0/1320679369 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b139217e00 0x55b13921a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.805+0000 7f492a807b80 4 client.26126 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:49:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.805+0000 7f492a807b80 1 -- 172.21.15.32:0/1320679369 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b13914ddb0 con 0x55b139217e00 2023-09-30T14:49:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.805+0000 7f4917fff700 1 --2- 172.21.15.32:0/1320679369 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b139217e00 0x55b13921a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.806+0000 7f4917fff700 1 --2- 172.21.15.32:0/1320679369 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b139217e00 0x55b13921a220 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b34304f11d0ce94 server_cookie=a6c195a0c3891a22 in_seq=0 out_seq=0 2023-09-30T14:49:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.808+0000 7f48f77fe700 10 client.26126 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.808+0000 7f48f77fe700 1 -- 172.21.15.32:0/1320679369 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b13914ddb0 con 0x55b139217e00 2023-09-30T14:49:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.808+0000 7f48f77fe700 10 client.26126 handle_command_reply: tid=0 2023-09-30T14:49:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.808+0000 7f492a807b80 1 client.26126 shutdown 2023-09-30T14:49:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.809+0000 7f492a807b80 1 -- 172.21.15.32:0/1320679369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203cd0 msgr2=0x55b139213480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.809+0000 7f492a807b80 1 --2- 172.21.15.32:0/1320679369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203cd0 0x55b139213480 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4910012040 tx=0x7f4910017e00 comp rx=0 tx=0).stop 2023-09-30T14:49:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.809+0000 7f492a807b80 1 -- 172.21.15.32:0/1320679369 shutdown_connections 2023-09-30T14:49:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.809+0000 7f492a807b80 1 --2- 172.21.15.32:0/1320679369 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b139217e00 0x55b13921a220 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.809+0000 7f492a807b80 1 --2- 172.21.15.32:0/1320679369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203cd0 0x55b139213480 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.809+0000 7f48f77fe700 0 client.26126 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.809+0000 7f492a807b80 1 --2- 172.21.15.32:0/1320679369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139202200 0x55b13920bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.809+0000 7f492a807b80 1 --2- 172.21.15.32:0/1320679369 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203310 0x55b13920c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.809+0000 7f492a807b80 1 -- 172.21.15.32:0/1320679369 >> 172.21.15.32:0/1320679369 conn(0x55b139176200 msgr2=0x55b1391fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.809+0000 7f492a807b80 1 -- 172.21.15.32:0/1320679369 shutdown_connections 2023-09-30T14:49:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.809+0000 7f492a807b80 1 -- 172.21.15.32:0/1320679369 wait complete. 2023-09-30T14:49:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.809+0000 7f492a807b80 20 client.26126 trim_cache size 0 max 16384 2023-09-30T14:49:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f492a807b80 1 Processor -- start 2023-09-30T14:49:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f492a807b80 1 -- start start 2023-09-30T14:49:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f492a807b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139202200 0x55b13920b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f492a807b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203310 0x55b139213440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f492a807b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203cd0 0x55b13920aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f492a807b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b138fd58c0 con 0x55b139202200 2023-09-30T14:49:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f492a807b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1391fdbf0 con 0x55b139203310 2023-09-30T14:49:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f492a807b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1392030c0 con 0x55b139203cd0 2023-09-30T14:49:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f49177fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203310 0x55b139213440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f4917fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139202200 0x55b13920b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f49177fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203310 0x55b139213440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49432/0 (socket says 172.21.15.32:49432) 2023-09-30T14:49:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f4917fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139202200 0x55b13920b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37956/0 (socket says 172.21.15.32:37956) 2023-09-30T14:49:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f49177fe700 1 -- 172.21.15.32:0/3903952696 learned_addr learned my addr 172.21.15.32:0/3903952696 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f491cbbc700 1 --2- 172.21.15.32:0/3903952696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203cd0 0x55b13920aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f49177fe700 1 -- 172.21.15.32:0/3903952696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203cd0 msgr2=0x55b13920aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f49177fe700 1 --2- 172.21.15.32:0/3903952696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203cd0 0x55b13920aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f49177fe700 1 -- 172.21.15.32:0/3903952696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139202200 msgr2=0x55b13920b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.825+0000 7f49177fe700 1 --2- 172.21.15.32:0/3903952696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139202200 0x55b13920b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.826+0000 7f49177fe700 1 -- 172.21.15.32:0/3903952696 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b138f68c00 con 0x55b139203310 2023-09-30T14:49:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.826+0000 7f4917fff700 1 --2- 172.21.15.32:0/3903952696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139202200 0x55b13920b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:49:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.826+0000 7f49177fe700 1 --2- 172.21.15.32:0/3903952696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203310 0x55b139213440 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f490c099830 tx=0x7f490c007f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9fcfabb37723f43 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.826+0000 7f48feffd700 1 -- 172.21.15.32:0/3903952696 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0b90d0 con 0x55b139203310 2023-09-30T14:49:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.826+0000 7f48feffd700 1 -- 172.21.15.32:0/3903952696 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f490c09d9f0 con 0x55b139203310 2023-09-30T14:49:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.826+0000 7f48feffd700 1 -- 172.21.15.32:0/3903952696 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c098ff0 con 0x55b139203310 2023-09-30T14:49:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.827+0000 7f492a807b80 1 -- 172.21.15.32:0/3903952696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203310 msgr2=0x55b139213440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.827+0000 7f492a807b80 1 --2- 172.21.15.32:0/3903952696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203310 0x55b139213440 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f490c099830 tx=0x7f490c007f60 comp rx=0 tx=0).stop 2023-09-30T14:49:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.827+0000 7f492a807b80 1 -- 172.21.15.32:0/3903952696 shutdown_connections 2023-09-30T14:49:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.827+0000 7f492a807b80 1 --2- 172.21.15.32:0/3903952696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139203cd0 0x55b13920aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.827+0000 7f492a807b80 1 --2- 172.21.15.32:0/3903952696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139202200 0x55b13920b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.827+0000 7f492a807b80 1 --2- 172.21.15.32:0/3903952696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203310 0x55b139213440 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.827+0000 7f492a807b80 1 -- 172.21.15.32:0/3903952696 >> 172.21.15.32:0/3903952696 conn(0x55b139176200 msgr2=0x55b1391fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.827+0000 7f492a807b80 1 -- 172.21.15.32:0/3903952696 shutdown_connections 2023-09-30T14:49:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.827+0000 7f492a807b80 1 -- 172.21.15.32:0/3903952696 wait complete. 2023-09-30T14:49:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.827+0000 7f492a807b80 1 -- start start 2023-09-30T14:49:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.828+0000 7f492a807b80 1 Processor -- start 2023-09-30T14:49:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.828+0000 7f492a807b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139202200 0x55b139210d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.828+0000 7f492a807b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139203310 0x55b1392112b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.828+0000 7f492a807b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203cd0 0x55b1392122c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.828+0000 7f492a807b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b139212800 con 0x55b139203310 2023-09-30T14:49:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.828+0000 7f492a807b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b139212970 con 0x55b139203cd0 2023-09-30T14:49:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.828+0000 7f492a807b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b139212b30 con 0x55b139202200 2023-09-30T14:49:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.828+0000 7f492a807b80 10 client.0 _ll_register_callbacks cb 0x7f491001a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:49:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.828+0000 7f4917fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139202200 0x55b139210d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.828+0000 7f49177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139203310 0x55b1392112b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.828+0000 7f4917fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139202200 0x55b139210d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52344/0 (socket says 172.21.15.32:52344) 2023-09-30T14:49:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.828+0000 7f49177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139203310 0x55b1392112b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37964/0 (socket says 172.21.15.32:37964) 2023-09-30T14:49:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.828+0000 7f4917fff700 1 -- 172.21.15.32:0/3496070614 learned_addr learned my addr 172.21.15.32:0/3496070614 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.828+0000 7f491cbbc700 1 --2- 172.21.15.32:0/3496070614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203cd0 0x55b1392122c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.828+0000 7f4917fff700 1 -- 172.21.15.32:0/3496070614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203cd0 msgr2=0x55b1392122c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.829+0000 7f4917fff700 1 --2- 172.21.15.32:0/3496070614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203cd0 0x55b1392122c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.829+0000 7f4917fff700 1 -- 172.21.15.32:0/3496070614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139203310 msgr2=0x55b1392112b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.829+0000 7f4917fff700 1 --2- 172.21.15.32:0/3496070614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139203310 0x55b1392112b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.829+0000 7f4917fff700 1 -- 172.21.15.32:0/3496070614 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4908017070 con 0x55b139202200 2023-09-30T14:49:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.829+0000 7f4917fff700 1 --2- 172.21.15.32:0/3496070614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139202200 0x55b139210d70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55b13921ab20 tx=0x7f4908004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f24539ea74a987ee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.829+0000 7f48f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:49:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.829+0000 7f48f77fe700 1 -- 172.21.15.32:0/3496070614 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49080275b0 con 0x55b139202200 2023-09-30T14:49:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.829+0000 7f492a807b80 1 -- 172.21.15.32:0/3496070614 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f490c0998a0 con 0x55b139202200 2023-09-30T14:49:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.829+0000 7f48f77fe700 1 -- 172.21.15.32:0/3496070614 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4908024040 con 0x55b139202200 2023-09-30T14:49:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.829+0000 7f48f77fe700 1 -- 172.21.15.32:0/3496070614 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4908030470 con 0x55b139202200 2023-09-30T14:49:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.829+0000 7f492a807b80 10 client.26119 fetch_fsmap learned FSMap version 44 2023-09-30T14:49:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.829+0000 7f492a807b80 1 -- 172.21.15.32:0/3496070614 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55b1391b7870 con 0x55b139202200 2023-09-30T14:49:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.829+0000 7f48f77fe700 1 -- 172.21.15.32:0/3496070614 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f490c0998a0 con 0x55b139202200 2023-09-30T14:49:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.830+0000 7f48f77fe700 1 -- 172.21.15.32:0/3496070614 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4908039070 con 0x55b139202200 2023-09-30T14:49:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.830+0000 7f492a807b80 10 client.26119 fetch_fsmap finished waiting for FSMap version 44 2023-09-30T14:49:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.830+0000 7f492a807b80 10 client.26119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:49:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.830+0000 7f492a807b80 20 client.26119 populate_metadata read hostname 'smithi032' 2023-09-30T14:49:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.830+0000 7f492a807b80 1 --2- 172.21.15.32:0/3496070614 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b13926c6d0 0x55b13926eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.830+0000 7f492a807b80 4 client.26119 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:49:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.830+0000 7f492a807b80 1 -- 172.21.15.32:0/3496070614 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b13914dad0 con 0x55b13926c6d0 2023-09-30T14:49:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.830+0000 7f49177fe700 1 --2- 172.21.15.32:0/3496070614 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b13926c6d0 0x55b13926eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.830+0000 7f48f77fe700 10 client.26119 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.830+0000 7f49177fe700 1 --2- 172.21.15.32:0/3496070614 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b13926c6d0 0x55b13926eb90 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a94ff5b05bd91162 server_cookie=513a589cd6374e4d in_seq=0 out_seq=0 2023-09-30T14:49:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.876+0000 7f48f77fe700 1 -- 172.21.15.32:0/3496070614 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b13914dad0 con 0x55b13926c6d0 2023-09-30T14:49:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.876+0000 7f48f77fe700 10 client.26119 handle_command_reply: tid=0 2023-09-30T14:49:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.876+0000 7f492a807b80 1 client.26119 shutdown 2023-09-30T14:49:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.876+0000 7f492a807b80 1 -- 172.21.15.32:0/3496070614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139202200 msgr2=0x55b139210d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.876+0000 7f492a807b80 1 --2- 172.21.15.32:0/3496070614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139202200 0x55b139210d70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55b13921ab20 tx=0x7f4908004750 comp rx=0 tx=0).stop 2023-09-30T14:49:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.876+0000 7f492a807b80 1 -- 172.21.15.32:0/3496070614 shutdown_connections 2023-09-30T14:49:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.876+0000 7f492a807b80 1 --2- 172.21.15.32:0/3496070614 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b13926c6d0 0x55b13926eb90 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.876+0000 7f492a807b80 1 --2- 172.21.15.32:0/3496070614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b139203cd0 0x55b1392122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.876+0000 7f492a807b80 1 --2- 172.21.15.32:0/3496070614 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b139202200 0x55b139210d70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.876+0000 7f492a807b80 1 --2- 172.21.15.32:0/3496070614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b139203310 0x55b1392112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.876+0000 7f48f77fe700 0 client.26119 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.876+0000 7f492a807b80 1 -- 172.21.15.32:0/3496070614 >> 172.21.15.32:0/3496070614 conn(0x55b139176200 msgr2=0x55b13920a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.876+0000 7f492a807b80 1 -- 172.21.15.32:0/3496070614 shutdown_connections 2023-09-30T14:49:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.876+0000 7f492a807b80 1 -- 172.21.15.32:0/3496070614 wait complete. 2023-09-30T14:49:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.876+0000 7f492a807b80 20 client.26119 trim_cache size 0 max 16384 2023-09-30T14:49:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:49:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:49:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:49:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:49:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.877+0000 7f491e61f700 1 -- 172.21.15.32:0/364391153 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f49000a0fa0 msgr2=0x7f49000a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.877+0000 7f491e61f700 1 --2- 172.21.15.32:0/364391153 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f49000a0fa0 0x7f49000a3460 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4918068f30 tx=0x7f4908009260 comp rx=0 tx=0).stop 2023-09-30T14:49:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.877+0000 7f491e61f700 1 -- 172.21.15.32:0/364391153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49181009c0 msgr2=0x7f49181950e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.877+0000 7f491e61f700 1 --2- 172.21.15.32:0/364391153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49181009c0 0x7f49181950e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f490c007950 tx=0x7f490c007c60 comp rx=0 tx=0).stop 2023-09-30T14:49:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.877+0000 7f491e61f700 1 -- 172.21.15.32:0/364391153 shutdown_connections 2023-09-30T14:49:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.877+0000 7f491e61f700 1 --2- 172.21.15.32:0/364391153 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f49000a0fa0 0x7f49000a3460 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.877+0000 7f491e61f700 1 --2- 172.21.15.32:0/364391153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49181013b0 0x7f4918195f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.877+0000 7f491e61f700 1 --2- 172.21.15.32:0/364391153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4918100000 0x7f4918194ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.877+0000 7f491e61f700 1 --2- 172.21.15.32:0/364391153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49181009c0 0x7f49181950e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.877+0000 7f491e61f700 1 -- 172.21.15.32:0/364391153 >> 172.21.15.32:0/364391153 conn(0x7f49180faa90 msgr2=0x7f49180fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.877+0000 7f491e61f700 1 -- 172.21.15.32:0/364391153 shutdown_connections 2023-09-30T14:49:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:30.877+0000 7f491e61f700 1 -- 172.21.15.32:0/364391153 wait complete. 2023-09-30T14:49:31.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:31 smithi148 ceph-mon[102355]: pgmap v624: 97 pgs: 97 active+clean; 383 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 2.7 MiB/s wr, 256 op/s 2023-09-30T14:49:31.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:31 smithi032 ceph-mon[98592]: pgmap v624: 97 pgs: 97 active+clean; 383 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 2.7 MiB/s wr, 256 op/s 2023-09-30T14:49:31.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1229881 10.70 MB/sec execute 485 sec latency 70.137 ms 2023-09-30T14:49:31.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:31 smithi156 ceph-mon[103579]: pgmap v624: 97 pgs: 97 active+clean; 383 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 2.7 MiB/s wr, 256 op/s 2023-09-30T14:49:32.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1232182 10.70 MB/sec execute 486 sec latency 76.645 ms 2023-09-30T14:49:33.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:33 smithi148 ceph-mon[102355]: pgmap v625: 97 pgs: 97 active+clean; 382 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 2.8 MiB/s wr, 259 op/s 2023-09-30T14:49:33.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:33 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:33 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:33 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:33 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:33 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:33 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:33 smithi032 ceph-mon[98592]: pgmap v625: 97 pgs: 97 active+clean; 382 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 2.8 MiB/s wr, 259 op/s 2023-09-30T14:49:33.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:33 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:33 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:33 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:33 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:33 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:33 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1233372 10.69 MB/sec execute 487 sec latency 91.688 ms 2023-09-30T14:49:33.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:33 smithi156 ceph-mon[103579]: pgmap v625: 97 pgs: 97 active+clean; 382 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 2.8 MiB/s wr, 259 op/s 2023-09-30T14:49:33.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:33 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:33 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:33 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:33 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:33 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:33.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:33 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:34.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:34 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:49:34.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:34 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:49:34.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:34 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:34.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:34 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:49:34.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:34 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:49:34.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:34 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:34.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1236068 10.70 MB/sec execute 488 sec latency 35.270 ms 2023-09-30T14:49:34.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:34 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:49:34.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:34 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:49:34.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:34 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:49:35.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1237760 10.70 MB/sec execute 489 sec latency 14.741 ms 2023-09-30T14:49:35.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:35 smithi156 ceph-mon[103579]: pgmap v626: 97 pgs: 97 active+clean; 384 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.6 MiB/s wr, 276 op/s 2023-09-30T14:49:35.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:35 smithi148 ceph-mon[102355]: pgmap v626: 97 pgs: 97 active+clean; 384 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.6 MiB/s wr, 276 op/s 2023-09-30T14:49:35.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:35 smithi032 ceph-mon[98592]: pgmap v626: 97 pgs: 97 active+clean; 384 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.6 MiB/s wr, 276 op/s 2023-09-30T14:49:36.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1239871 10.70 MB/sec execute 490 sec latency 32.491 ms 2023-09-30T14:49:37.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1241912 10.69 MB/sec execute 491 sec latency 22.085 ms 2023-09-30T14:49:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:37 smithi156 ceph-mon[103579]: pgmap v627: 97 pgs: 97 active+clean; 384 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 2.6 MiB/s wr, 267 op/s 2023-09-30T14:49:37.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:37 smithi148 ceph-mon[102355]: pgmap v627: 97 pgs: 97 active+clean; 384 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 2.6 MiB/s wr, 267 op/s 2023-09-30T14:49:37.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:37 smithi032 ceph-mon[98592]: pgmap v627: 97 pgs: 97 active+clean; 384 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 2.6 MiB/s wr, 267 op/s 2023-09-30T14:49:38.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1243627 10.70 MB/sec execute 492 sec latency 29.910 ms 2023-09-30T14:49:39.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1246047 10.70 MB/sec execute 493 sec latency 14.078 ms 2023-09-30T14:49:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:39 smithi156 ceph-mon[103579]: pgmap v628: 97 pgs: 97 active+clean; 387 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 2.8 MiB/s wr, 295 op/s 2023-09-30T14:49:39.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:39 smithi148 ceph-mon[102355]: pgmap v628: 97 pgs: 97 active+clean; 387 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 2.8 MiB/s wr, 295 op/s 2023-09-30T14:49:39.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:39 smithi032 ceph-mon[98592]: pgmap v628: 97 pgs: 97 active+clean; 387 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 2.8 MiB/s wr, 295 op/s 2023-09-30T14:49:40.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1247709 10.70 MB/sec execute 494 sec latency 20.953 ms 2023-09-30T14:49:41.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1250081 10.71 MB/sec execute 495 sec latency 39.369 ms 2023-09-30T14:49:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:41 smithi156 ceph-mon[103579]: pgmap v629: 97 pgs: 97 active+clean; 387 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 2.8 MiB/s wr, 268 op/s 2023-09-30T14:49:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:41 smithi148 ceph-mon[102355]: pgmap v629: 97 pgs: 97 active+clean; 387 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 2.8 MiB/s wr, 268 op/s 2023-09-30T14:49:41.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:41 smithi032 ceph-mon[98592]: pgmap v629: 97 pgs: 97 active+clean; 387 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 2.8 MiB/s wr, 268 op/s 2023-09-30T14:49:42.720 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1251295 10.69 MB/sec execute 496 sec latency 27.291 ms 2023-09-30T14:49:43.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1253733 10.70 MB/sec execute 497 sec latency 13.974 ms 2023-09-30T14:49:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:43 smithi156 ceph-mon[103579]: pgmap v630: 97 pgs: 97 active+clean; 387 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 2.8 MiB/s wr, 237 op/s 2023-09-30T14:49:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:43 smithi148 ceph-mon[102355]: pgmap v630: 97 pgs: 97 active+clean; 387 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 2.8 MiB/s wr, 237 op/s 2023-09-30T14:49:43.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:43 smithi032 ceph-mon[98592]: pgmap v630: 97 pgs: 97 active+clean; 387 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 2.8 MiB/s wr, 237 op/s 2023-09-30T14:49:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.997+0000 7f726c31b700 1 -- 172.21.15.32:0/89663455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7264100f50 msgr2=0x7f72641013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.997+0000 7f726c31b700 1 --2- 172.21.15.32:0/89663455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7264100f50 0x7f72641013b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f72540099c0 tx=0x7f7254009cd0 comp rx=0 tx=0).stop 2023-09-30T14:49:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.997+0000 7f726c31b700 1 -- 172.21.15.32:0/89663455 shutdown_connections 2023-09-30T14:49:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.997+0000 7f726c31b700 1 --2- 172.21.15.32:0/89663455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72641018f0 0x7f72641081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.997+0000 7f726c31b700 1 --2- 172.21.15.32:0/89663455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72640ffd60 0x7f7264100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.997+0000 7f726c31b700 1 --2- 172.21.15.32:0/89663455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7264100f50 0x7f72641013b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.997+0000 7f726c31b700 1 -- 172.21.15.32:0/89663455 >> 172.21.15.32:0/89663455 conn(0x7f72640fb2b0 msgr2=0x7f72640fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.997+0000 7f726c31b700 1 -- 172.21.15.32:0/89663455 shutdown_connections 2023-09-30T14:49:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.997+0000 7f726c31b700 1 -- 172.21.15.32:0/89663455 wait complete. 2023-09-30T14:49:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.998+0000 7f726c31b700 1 Processor -- start 2023-09-30T14:49:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.998+0000 7f726c31b700 1 -- start start 2023-09-30T14:49:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.998+0000 7f726c31b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72640ffd60 0x7f7264102610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.998+0000 7f726c31b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7264100f50 0x7f7264102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.998+0000 7f726c31b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72641018f0 0x7f7264103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.998+0000 7f726c31b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72641a0b90 con 0x7f72641018f0 2023-09-30T14:49:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.998+0000 7f726c31b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72641a0cd0 con 0x7f72640ffd60 2023-09-30T14:49:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.998+0000 7f726c31b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72641a0e10 con 0x7f7264100f50 2023-09-30T14:49:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.998+0000 7f726a0b7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72640ffd60 0x7f7264102610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.998+0000 7f726a8b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72641018f0 0x7f7264103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.998+0000 7f726a0b7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72640ffd60 0x7f7264102610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49450/0 (socket says 172.21.15.32:49450) 2023-09-30T14:49:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.998+0000 7f726a0b7700 1 -- 172.21.15.32:0/56625403 learned_addr learned my addr 172.21.15.32:0/56625403 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.998+0000 7f72698b6700 1 --2- 172.21.15.32:0/56625403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7264100f50 0x7f7264102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.999+0000 7f726a0b7700 1 -- 172.21.15.32:0/56625403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7264100f50 msgr2=0x7f7264102b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.999+0000 7f726a0b7700 1 --2- 172.21.15.32:0/56625403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7264100f50 0x7f7264102b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.999+0000 7f726a0b7700 1 -- 172.21.15.32:0/56625403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72641018f0 msgr2=0x7f7264103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.999+0000 7f726a0b7700 1 --2- 172.21.15.32:0/56625403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72641018f0 0x7f7264103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.999+0000 7f726a0b7700 1 -- 172.21.15.32:0/56625403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72540095f0 con 0x7f72640ffd60 2023-09-30T14:49:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.999+0000 7f726a0b7700 1 --2- 172.21.15.32:0/56625403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72640ffd60 0x7f7264102610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f726000e970 tx=0x7f726000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.999+0000 7f725b7fe700 1 -- 172.21.15.32:0/56625403 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260009790 con 0x7f72640ffd60 2023-09-30T14:49:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.999+0000 7f725b7fe700 1 -- 172.21.15.32:0/56625403 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f726000c7e0 con 0x7f72640ffd60 2023-09-30T14:49:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.999+0000 7f725b7fe700 1 -- 172.21.15.32:0/56625403 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260003c70 con 0x7f72640ffd60 2023-09-30T14:49:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.999+0000 7f726c31b700 1 -- 172.21.15.32:0/56625403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72641a10a0 con 0x7f72640ffd60 2023-09-30T14:49:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:43.999+0000 7f726c31b700 1 -- 172.21.15.32:0/56625403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72641a1560 con 0x7f72640ffd60 2023-09-30T14:49:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.001+0000 7f725b7fe700 1 -- 172.21.15.32:0/56625403 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f726000cc50 con 0x7f72640ffd60 2023-09-30T14:49:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.002+0000 7f725b7fe700 1 --2- 172.21.15.32:0/56625403 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f72500a1020 0x7f72500a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.002+0000 7f72698b6700 1 --2- 172.21.15.32:0/56625403 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f72500a1020 0x7f72500a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.002+0000 7f725b7fe700 1 -- 172.21.15.32:0/56625403 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7260014070 con 0x7f72640ffd60 2023-09-30T14:49:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.003+0000 7f7278503b80 1 Processor -- start 2023-09-30T14:49:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.003+0000 7f72698b6700 1 --2- 172.21.15.32:0/56625403 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f72500a1020 0x7f72500a34e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7254009990 tx=0x7f7254005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.003+0000 7f7278503b80 1 -- start start 2023-09-30T14:49:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.003+0000 7f7278503b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f4f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.003+0000 7f7278503b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4faf0 0x55e919f502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.003+0000 7f7278503b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f508d0 0x55e919f50d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.003+0000 7f7278503b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e919eba400 con 0x55e919f4f190 2023-09-30T14:49:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.003+0000 7f7278503b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e919eabbc0 con 0x55e919f508d0 2023-09-30T14:49:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.003+0000 7f7278503b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e919ea94e0 con 0x55e919f4faf0 2023-09-30T14:49:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.004+0000 7f72698b6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4faf0 0x55e919f502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.004+0000 7f726a8b8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f508d0 0x55e919f50d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.004+0000 7f72698b6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4faf0 0x55e919f502f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52366/0 (socket says 172.21.15.32:52366) 2023-09-30T14:49:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.004+0000 7f72698b6700 1 -- 172.21.15.32:0/3466660253 learned_addr learned my addr 172.21.15.32:0/3466660253 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.004+0000 7f726a0b7700 1 --2- 172.21.15.32:0/3466660253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f4f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.004+0000 7f726a0b7700 1 -- 172.21.15.32:0/3466660253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4faf0 msgr2=0x55e919f502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.004+0000 7f726a0b7700 1 --2- 172.21.15.32:0/3466660253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4faf0 0x55e919f502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.004+0000 7f726a0b7700 1 -- 172.21.15.32:0/3466660253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f508d0 msgr2=0x55e919f50d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.004+0000 7f726a0b7700 1 --2- 172.21.15.32:0/3466660253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f508d0 0x55e919f50d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.004+0000 7f726a0b7700 1 -- 172.21.15.32:0/3466660253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e919f07870 con 0x55e919f4f190 2023-09-30T14:49:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.004+0000 7f726a8b8700 1 --2- 172.21.15.32:0/3466660253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f508d0 0x55e919f50d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:49:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.005+0000 7f726a0b7700 1 --2- 172.21.15.32:0/3466660253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f4f5b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7260005950 tx=0x7f7260099650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a25ff5040ce145 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.006+0000 7f7258d82700 1 -- 172.21.15.32:0/3466660253 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260099cf0 con 0x55e919f4f190 2023-09-30T14:49:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.006+0000 7f7258d82700 1 -- 172.21.15.32:0/3466660253 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f726009de40 con 0x55e919f4f190 2023-09-30T14:49:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.006+0000 7f7258d82700 1 -- 172.21.15.32:0/3466660253 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260099f50 con 0x55e919f4f190 2023-09-30T14:49:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.007+0000 7f7278503b80 1 -- 172.21.15.32:0/3466660253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 msgr2=0x55e919f4f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.007+0000 7f7278503b80 1 --2- 172.21.15.32:0/3466660253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f4f5b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7260005950 tx=0x7f7260099650 comp rx=0 tx=0).stop 2023-09-30T14:49:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.007+0000 7f7278503b80 1 -- 172.21.15.32:0/3466660253 shutdown_connections 2023-09-30T14:49:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.007+0000 7f7278503b80 1 --2- 172.21.15.32:0/3466660253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f508d0 0x55e919f50d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.007+0000 7f7278503b80 1 --2- 172.21.15.32:0/3466660253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f4f5b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.007+0000 7f7278503b80 1 --2- 172.21.15.32:0/3466660253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4faf0 0x55e919f502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.007+0000 7f7278503b80 1 -- 172.21.15.32:0/3466660253 >> 172.21.15.32:0/3466660253 conn(0x55e919ec6200 msgr2=0x55e919f4ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.007+0000 7f7278503b80 1 -- 172.21.15.32:0/3466660253 shutdown_connections 2023-09-30T14:49:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.007+0000 7f7278503b80 1 -- 172.21.15.32:0/3466660253 wait complete. 2023-09-30T14:49:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.007+0000 7f7278503b80 1 -- start start 2023-09-30T14:49:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.007+0000 7f7278503b80 1 Processor -- start 2023-09-30T14:49:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f7278503b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f6cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f7278503b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4faf0 0x55e919f6d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f7278503b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f508d0 0x55e919f74500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f7278503b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e919ea4dd0 con 0x55e919f4f190 2023-09-30T14:49:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f7278503b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e919e416b0 con 0x55e919f508d0 2023-09-30T14:49:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f7278503b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e919e41350 con 0x55e919f4faf0 2023-09-30T14:49:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f7278503b80 10 client.0 _ll_register_callbacks cb 0x55e919c9e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:49:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f726a0b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f6cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f726a0b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f6cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37988/0 (socket says 172.21.15.32:37988) 2023-09-30T14:49:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f726a0b7700 1 -- 172.21.15.32:0/1783068332 learned_addr learned my addr 172.21.15.32:0/1783068332 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f72698b6700 1 --2- 172.21.15.32:0/1783068332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4faf0 0x55e919f6d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f726a8b8700 1 --2- 172.21.15.32:0/1783068332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f508d0 0x55e919f74500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f726a0b7700 1 -- 172.21.15.32:0/1783068332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4faf0 msgr2=0x55e919f6d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f726a0b7700 1 --2- 172.21.15.32:0/1783068332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4faf0 0x55e919f6d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f726a0b7700 1 -- 172.21.15.32:0/1783068332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f508d0 msgr2=0x55e919f74500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f726a0b7700 1 --2- 172.21.15.32:0/1783068332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f508d0 0x55e919f74500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f726a0b7700 1 -- 172.21.15.32:0/1783068332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c007e10 con 0x55e919f4f190 2023-09-30T14:49:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f726a0b7700 1 --2- 172.21.15.32:0/1783068332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f6cf80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f72600961f0 tx=0x7f72600ca2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc02824c21a622b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.008+0000 7f724dffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:49:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.009+0000 7f724dffb700 1 -- 172.21.15.32:0/1783068332 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72600993e0 con 0x55e919f4f190 2023-09-30T14:49:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.009+0000 7f724dffb700 1 -- 172.21.15.32:0/1783068332 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7260099cf0 con 0x55e919f4f190 2023-09-30T14:49:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.009+0000 7f7278503b80 1 -- 172.21.15.32:0/1783068332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e919f45090 con 0x55e919f4f190 2023-09-30T14:49:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.009+0000 7f724dffb700 1 -- 172.21.15.32:0/1783068332 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7260099f50 con 0x55e919f4f190 2023-09-30T14:49:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.009+0000 7f7278503b80 10 client.16458 fetch_fsmap learned FSMap version 44 2023-09-30T14:49:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.009+0000 7f7278503b80 1 -- 172.21.15.32:0/1783068332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f72641a1560 con 0x55e919f4f190 2023-09-30T14:49:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.009+0000 7f724dffb700 1 -- 172.21.15.32:0/1783068332 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e919f45090 con 0x55e919f4f190 2023-09-30T14:49:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.009+0000 7f724dffb700 1 -- 172.21.15.32:0/1783068332 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7260095ae0 con 0x55e919f4f190 2023-09-30T14:49:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.009+0000 7f7278503b80 10 client.16458 fetch_fsmap finished waiting for FSMap version 44 2023-09-30T14:49:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.009+0000 7f7278503b80 10 client.16458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:49:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.009+0000 7f7278503b80 20 client.16458 populate_metadata read hostname 'smithi032' 2023-09-30T14:49:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.009+0000 7f7278503b80 1 --2- 172.21.15.32:0/1783068332 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e919f78970 0x55e919f7ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.009+0000 7f7278503b80 4 client.16458 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:49:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.009+0000 7f7278503b80 1 -- 172.21.15.32:0/1783068332 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e919e9ddb0 con 0x55e919f78970 2023-09-30T14:49:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.010+0000 7f72698b6700 1 --2- 172.21.15.32:0/1783068332 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e919f78970 0x55e919f7ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.010+0000 7f724dffb700 10 client.16458 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.010+0000 7f72698b6700 1 --2- 172.21.15.32:0/1783068332 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e919f78970 0x55e919f7ad90 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99b28b8acb07a56 server_cookie=92b03598125ea7f2 in_seq=0 out_seq=0 2023-09-30T14:49:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.011+0000 7f724dffb700 1 -- 172.21.15.32:0/1783068332 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e919e9ddb0 con 0x55e919f78970 2023-09-30T14:49:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.011+0000 7f724dffb700 10 client.16458 handle_command_reply: tid=0 2023-09-30T14:49:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.011+0000 7f7278503b80 1 client.16458 shutdown 2023-09-30T14:49:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.011+0000 7f7278503b80 1 -- 172.21.15.32:0/1783068332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 msgr2=0x55e919f6cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.011+0000 7f7278503b80 1 --2- 172.21.15.32:0/1783068332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f6cf80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f72600961f0 tx=0x7f72600ca2a0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.011+0000 7f7278503b80 1 -- 172.21.15.32:0/1783068332 shutdown_connections 2023-09-30T14:49:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.011+0000 7f7278503b80 1 --2- 172.21.15.32:0/1783068332 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e919f78970 0x55e919f7ad90 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.011+0000 7f7278503b80 1 --2- 172.21.15.32:0/1783068332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f508d0 0x55e919f74500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.011+0000 7f724dffb700 0 client.16458 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.011+0000 7f7278503b80 1 --2- 172.21.15.32:0/1783068332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f6cf80 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.011+0000 7f7278503b80 1 --2- 172.21.15.32:0/1783068332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4faf0 0x55e919f6d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.011+0000 7f7278503b80 1 -- 172.21.15.32:0/1783068332 >> 172.21.15.32:0/1783068332 conn(0x55e919ec6200 msgr2=0x55e919f4c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.011+0000 7f7278503b80 1 -- 172.21.15.32:0/1783068332 shutdown_connections 2023-09-30T14:49:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.011+0000 7f7278503b80 1 -- 172.21.15.32:0/1783068332 wait complete. 2023-09-30T14:49:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.011+0000 7f7278503b80 20 client.16458 trim_cache size 0 max 16384 2023-09-30T14:49:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.028+0000 7f7278503b80 1 Processor -- start 2023-09-30T14:49:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.028+0000 7f7278503b80 1 -- start start 2023-09-30T14:49:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.028+0000 7f7278503b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f50cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.028+0000 7f7278503b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f4faf0 0x55e919f744c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.028+0000 7f7278503b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f508d0 0x55e919f59dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.028+0000 7f7278503b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e919d258c0 con 0x55e919f4f190 2023-09-30T14:49:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.028+0000 7f7278503b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e919f49ba0 con 0x55e919f4faf0 2023-09-30T14:49:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.028+0000 7f7278503b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e919f4fed0 con 0x55e919f508d0 2023-09-30T14:49:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.028+0000 7f726a0b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f50cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.028+0000 7f726a0b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f50cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37996/0 (socket says 172.21.15.32:37996) 2023-09-30T14:49:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.029+0000 7f726a0b7700 1 -- 172.21.15.32:0/2592835731 learned_addr learned my addr 172.21.15.32:0/2592835731 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.029+0000 7f72698b6700 1 --2- 172.21.15.32:0/2592835731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f4faf0 0x55e919f744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.029+0000 7f726a8b8700 1 --2- 172.21.15.32:0/2592835731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f508d0 0x55e919f59dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.029+0000 7f726a0b7700 1 -- 172.21.15.32:0/2592835731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f508d0 msgr2=0x55e919f59dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.029+0000 7f726a0b7700 1 --2- 172.21.15.32:0/2592835731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f508d0 0x55e919f59dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.029+0000 7f726a0b7700 1 -- 172.21.15.32:0/2592835731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f4faf0 msgr2=0x55e919f744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.029+0000 7f726a0b7700 1 --2- 172.21.15.32:0/2592835731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f4faf0 0x55e919f744c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.029+0000 7f726a0b7700 1 -- 172.21.15.32:0/2592835731 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e919ccfd80 con 0x55e919f4f190 2023-09-30T14:49:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.029+0000 7f726a0b7700 1 --2- 172.21.15.32:0/2592835731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f50cb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7260000c00 tx=0x7f72600982a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19331aec6f8d295d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.029+0000 7f7259583700 1 -- 172.21.15.32:0/2592835731 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260099cf0 con 0x55e919f4f190 2023-09-30T14:49:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.029+0000 7f7259583700 1 -- 172.21.15.32:0/2592835731 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7260096400 con 0x55e919f4f190 2023-09-30T14:49:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.029+0000 7f7259583700 1 -- 172.21.15.32:0/2592835731 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72600a1bf0 con 0x55e919f4f190 2023-09-30T14:49:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.030+0000 7f7278503b80 1 -- 172.21.15.32:0/2592835731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 msgr2=0x55e919f50cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.030+0000 7f7278503b80 1 --2- 172.21.15.32:0/2592835731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f50cb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7260000c00 tx=0x7f72600982a0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.030+0000 7f7278503b80 1 -- 172.21.15.32:0/2592835731 shutdown_connections 2023-09-30T14:49:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.030+0000 7f7278503b80 1 --2- 172.21.15.32:0/2592835731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f508d0 0x55e919f59dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.030+0000 7f7278503b80 1 --2- 172.21.15.32:0/2592835731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4f190 0x55e919f50cb0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.030+0000 7f7278503b80 1 --2- 172.21.15.32:0/2592835731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f4faf0 0x55e919f744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.030+0000 7f7278503b80 1 -- 172.21.15.32:0/2592835731 >> 172.21.15.32:0/2592835731 conn(0x55e919ec6200 msgr2=0x55e919f6bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.032+0000 7f7278503b80 1 -- 172.21.15.32:0/2592835731 shutdown_connections 2023-09-30T14:49:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.032+0000 7f7278503b80 1 -- 172.21.15.32:0/2592835731 wait complete. 2023-09-30T14:49:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.032+0000 7f7278503b80 1 -- start start 2023-09-30T14:49:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.032+0000 7f7278503b80 1 Processor -- start 2023-09-30T14:49:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f7278503b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4f190 0x55e919f71e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f7278503b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4faf0 0x55e919f72350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f7278503b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f508d0 0x55e919f73360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f7278503b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e919f738a0 con 0x55e919f4faf0 2023-09-30T14:49:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f7278503b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e919f73a10 con 0x55e919f508d0 2023-09-30T14:49:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f7278503b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e919f73bd0 con 0x55e919f4f190 2023-09-30T14:49:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f7278503b80 10 client.0 _ll_register_callbacks cb 0x7f725c007c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:49:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f72698b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4faf0 0x55e919f72350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f72698b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4faf0 0x55e919f72350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38004/0 (socket says 172.21.15.32:38004) 2023-09-30T14:49:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f72698b6700 1 -- 172.21.15.32:0/1305654430 learned_addr learned my addr 172.21.15.32:0/1305654430 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f726a0b7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4f190 0x55e919f71e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f726a8b8700 1 --2- 172.21.15.32:0/1305654430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f508d0 0x55e919f73360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f72698b6700 1 -- 172.21.15.32:0/1305654430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4f190 msgr2=0x55e919f71e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f72698b6700 1 --2- 172.21.15.32:0/1305654430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4f190 0x55e919f71e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f72698b6700 1 -- 172.21.15.32:0/1305654430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f508d0 msgr2=0x55e919f73360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f72698b6700 1 --2- 172.21.15.32:0/1305654430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f508d0 0x55e919f73360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f72698b6700 1 -- 172.21.15.32:0/1305654430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72641a1560 con 0x55e919f4faf0 2023-09-30T14:49:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f726a0b7700 1 --2- 172.21.15.32:0/1305654430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4f190 0x55e919f71e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:49:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f72698b6700 1 --2- 172.21.15.32:0/1305654430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4faf0 0x55e919f72350 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x55e919f500b0 tx=0x7f7254009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5663583e4ee0dcc7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.033+0000 7f724dffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:49:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.034+0000 7f724dffb700 1 -- 172.21.15.32:0/1305654430 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254003b30 con 0x55e919f4faf0 2023-09-30T14:49:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.034+0000 7f724dffb700 1 -- 172.21.15.32:0/1305654430 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7254003c90 con 0x55e919f4faf0 2023-09-30T14:49:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.034+0000 7f7278503b80 1 -- 172.21.15.32:0/1305654430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f726009de40 con 0x55e919f4faf0 2023-09-30T14:49:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.034+0000 7f724dffb700 1 -- 172.21.15.32:0/1305654430 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7254003e90 con 0x55e919f4faf0 2023-09-30T14:49:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.034+0000 7f7278503b80 10 client.16464 fetch_fsmap learned FSMap version 44 2023-09-30T14:49:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.034+0000 7f7278503b80 1 -- 172.21.15.32:0/1305654430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55e919cb8c00 con 0x55e919f4faf0 2023-09-30T14:49:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.034+0000 7f724dffb700 1 -- 172.21.15.32:0/1305654430 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f726009de40 con 0x55e919f4faf0 2023-09-30T14:49:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.034+0000 7f724dffb700 1 -- 172.21.15.32:0/1305654430 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7254043070 con 0x55e919f4faf0 2023-09-30T14:49:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.034+0000 7f7278503b80 10 client.16464 fetch_fsmap finished waiting for FSMap version 44 2023-09-30T14:49:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.034+0000 7f7278503b80 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:49:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.034+0000 7f7278503b80 20 client.16464 populate_metadata read hostname 'smithi032' 2023-09-30T14:49:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.034+0000 7f7278503b80 1 --2- 172.21.15.32:0/1305654430 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e919fcd2a0 0x55e919fcf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.034+0000 7f7278503b80 4 client.16464 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:49:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.034+0000 7f7278503b80 1 -- 172.21.15.32:0/1305654430 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e919e9dad0 con 0x55e919fcd2a0 2023-09-30T14:49:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.034+0000 7f726a0b7700 1 --2- 172.21.15.32:0/1305654430 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e919fcd2a0 0x55e919fcf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.035+0000 7f726a0b7700 1 --2- 172.21.15.32:0/1305654430 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e919fcd2a0 0x55e919fcf6c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1c3799ca9bfa464 server_cookie=8c611102bfc58dc2 in_seq=0 out_seq=0 2023-09-30T14:49:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.035+0000 7f724dffb700 10 client.16464 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.088+0000 7f724dffb700 1 -- 172.21.15.32:0/1305654430 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e919e9dad0 con 0x55e919fcd2a0 2023-09-30T14:49:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.088+0000 7f724dffb700 10 client.16464 handle_command_reply: tid=0 2023-09-30T14:49:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.088+0000 7f7278503b80 1 client.16464 shutdown 2023-09-30T14:49:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.088+0000 7f7278503b80 1 -- 172.21.15.32:0/1305654430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4faf0 msgr2=0x55e919f72350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.088+0000 7f7278503b80 1 --2- 172.21.15.32:0/1305654430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4faf0 0x55e919f72350 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x55e919f500b0 tx=0x7f7254009310 comp rx=0 tx=0).stop 2023-09-30T14:49:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.088+0000 7f7278503b80 1 -- 172.21.15.32:0/1305654430 shutdown_connections 2023-09-30T14:49:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.088+0000 7f7278503b80 1 --2- 172.21.15.32:0/1305654430 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e919fcd2a0 0x55e919fcf6c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.088+0000 7f7278503b80 1 --2- 172.21.15.32:0/1305654430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e919f508d0 0x55e919f73360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.088+0000 7f724dffb700 0 client.16464 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.088+0000 7f7278503b80 1 --2- 172.21.15.32:0/1305654430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e919f4f190 0x55e919f71e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.088+0000 7f7278503b80 1 --2- 172.21.15.32:0/1305654430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e919f4faf0 0x55e919f72350 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.088+0000 7f7278503b80 1 -- 172.21.15.32:0/1305654430 >> 172.21.15.32:0/1305654430 conn(0x55e919ec6200 msgr2=0x55e919f538e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.088+0000 7f7278503b80 1 -- 172.21.15.32:0/1305654430 shutdown_connections 2023-09-30T14:49:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.088+0000 7f7278503b80 1 -- 172.21.15.32:0/1305654430 wait complete. 2023-09-30T14:49:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.089+0000 7f7278503b80 20 client.16464 trim_cache size 0 max 16384 2023-09-30T14:49:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:49:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:49:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:49:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:49:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.089+0000 7f726c31b700 1 -- 172.21.15.32:0/56625403 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f72500a1020 msgr2=0x7f72500a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.089+0000 7f726c31b700 1 --2- 172.21.15.32:0/56625403 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f72500a1020 0x7f72500a34e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7254009990 tx=0x7f7254005fb0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.089+0000 7f726c31b700 1 -- 172.21.15.32:0/56625403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72640ffd60 msgr2=0x7f7264102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.089+0000 7f726c31b700 1 --2- 172.21.15.32:0/56625403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72640ffd60 0x7f7264102610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f726000e970 tx=0x7f726000ec80 comp rx=0 tx=0).stop 2023-09-30T14:49:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.089+0000 7f726c31b700 1 -- 172.21.15.32:0/56625403 shutdown_connections 2023-09-30T14:49:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.089+0000 7f726c31b700 1 --2- 172.21.15.32:0/56625403 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f72500a1020 0x7f72500a34e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.089+0000 7f726c31b700 1 --2- 172.21.15.32:0/56625403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72641018f0 0x7f7264103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.089+0000 7f726c31b700 1 --2- 172.21.15.32:0/56625403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72640ffd60 0x7f7264102610 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.089+0000 7f726c31b700 1 --2- 172.21.15.32:0/56625403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7264100f50 0x7f7264102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.089+0000 7f726c31b700 1 -- 172.21.15.32:0/56625403 >> 172.21.15.32:0/56625403 conn(0x7f72640fb2b0 msgr2=0x7f72640fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.089+0000 7f726c31b700 1 -- 172.21.15.32:0/56625403 shutdown_connections 2023-09-30T14:49:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:44.089+0000 7f726c31b700 1 -- 172.21.15.32:0/56625403 wait complete. 2023-09-30T14:49:44.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1255090 10.69 MB/sec execute 498 sec latency 24.316 ms 2023-09-30T14:49:45.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1257423 10.70 MB/sec execute 499 sec latency 12.231 ms 2023-09-30T14:49:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:45 smithi156 ceph-mon[103579]: pgmap v631: 97 pgs: 97 active+clean; 389 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 2.8 MiB/s wr, 274 op/s 2023-09-30T14:49:45.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:45 smithi148 ceph-mon[102355]: pgmap v631: 97 pgs: 97 active+clean; 389 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 2.8 MiB/s wr, 274 op/s 2023-09-30T14:49:45.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:45 smithi032 ceph-mon[98592]: pgmap v631: 97 pgs: 97 active+clean; 389 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 2.8 MiB/s wr, 274 op/s 2023-09-30T14:49:46.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1258712 10.69 MB/sec execute 500 sec latency 28.304 ms 2023-09-30T14:49:47.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1260903 10.69 MB/sec execute 501 sec latency 26.544 ms 2023-09-30T14:49:47.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:47 smithi156 ceph-mon[103579]: pgmap v632: 97 pgs: 97 active+clean; 391 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 3.0 MiB/s wr, 284 op/s 2023-09-30T14:49:47.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:47 smithi148 ceph-mon[102355]: pgmap v632: 97 pgs: 97 active+clean; 391 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 3.0 MiB/s wr, 284 op/s 2023-09-30T14:49:47.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:47 smithi032 ceph-mon[98592]: pgmap v632: 97 pgs: 97 active+clean; 391 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 3.0 MiB/s wr, 284 op/s 2023-09-30T14:49:48.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1262349 10.68 MB/sec execute 502 sec latency 15.836 ms 2023-09-30T14:49:49.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1264549 10.69 MB/sec execute 503 sec latency 19.723 ms 2023-09-30T14:49:49.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:49 smithi156 ceph-mon[103579]: pgmap v633: 97 pgs: 97 active+clean; 385 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 3.1 MiB/s wr, 289 op/s 2023-09-30T14:49:49.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:49 smithi148 ceph-mon[102355]: pgmap v633: 97 pgs: 97 active+clean; 385 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 3.1 MiB/s wr, 289 op/s 2023-09-30T14:49:49.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:49 smithi032 ceph-mon[98592]: pgmap v633: 97 pgs: 97 active+clean; 385 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 3.1 MiB/s wr, 289 op/s 2023-09-30T14:49:50.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1265993 10.68 MB/sec execute 504 sec latency 21.106 ms 2023-09-30T14:49:51.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1268091 10.69 MB/sec execute 505 sec latency 64.275 ms 2023-09-30T14:49:51.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:51 smithi156 ceph-mon[103579]: pgmap v634: 97 pgs: 97 active+clean; 383 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 2.9 MiB/s wr, 257 op/s 2023-09-30T14:49:51.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:51 smithi148 ceph-mon[102355]: pgmap v634: 97 pgs: 97 active+clean; 383 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 2.9 MiB/s wr, 257 op/s 2023-09-30T14:49:51.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:51 smithi032 ceph-mon[98592]: pgmap v634: 97 pgs: 97 active+clean; 383 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 2.9 MiB/s wr, 257 op/s 2023-09-30T14:49:52.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1269825 10.68 MB/sec execute 506 sec latency 21.178 ms 2023-09-30T14:49:53.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1271823 10.68 MB/sec execute 507 sec latency 14.775 ms 2023-09-30T14:49:53.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:53 smithi156 ceph-mon[103579]: pgmap v635: 97 pgs: 97 active+clean; 370 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 2.7 MiB/s wr, 263 op/s 2023-09-30T14:49:53.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:53 smithi148 ceph-mon[102355]: pgmap v635: 97 pgs: 97 active+clean; 370 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 2.7 MiB/s wr, 263 op/s 2023-09-30T14:49:53.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:53 smithi032 ceph-mon[98592]: pgmap v635: 97 pgs: 97 active+clean; 370 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 2.7 MiB/s wr, 263 op/s 2023-09-30T14:49:54.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1273964 10.68 MB/sec execute 508 sec latency 59.824 ms 2023-09-30T14:49:55.721 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1275414 10.68 MB/sec execute 509 sec latency 85.029 ms 2023-09-30T14:49:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:55 smithi156 ceph-mon[103579]: pgmap v636: 97 pgs: 97 active+clean; 371 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.6 MiB/s wr, 277 op/s 2023-09-30T14:49:55.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:55 smithi148 ceph-mon[102355]: pgmap v636: 97 pgs: 97 active+clean; 371 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.6 MiB/s wr, 277 op/s 2023-09-30T14:49:55.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:55 smithi032 ceph-mon[98592]: pgmap v636: 97 pgs: 97 active+clean; 371 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.6 MiB/s wr, 277 op/s 2023-09-30T14:49:56.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1277705 10.68 MB/sec execute 510 sec latency 30.387 ms 2023-09-30T14:49:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.206+0000 7fb91d262700 1 -- 172.21.15.32:0/1274456043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918106bb0 msgr2=0x7fb918108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.206+0000 7fb91d262700 1 --2- 172.21.15.32:0/1274456043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918106bb0 0x7fb918108fa0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb91000b0a0 tx=0x7fb91000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.206+0000 7fb91d262700 1 -- 172.21.15.32:0/1274456043 shutdown_connections 2023-09-30T14:49:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.206+0000 7fb91d262700 1 --2- 172.21.15.32:0/1274456043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918106bb0 0x7fb918108fa0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.206+0000 7fb91d262700 1 --2- 172.21.15.32:0/1274456043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb918101950 0x7fb918103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.206+0000 7fb91d262700 1 --2- 172.21.15.32:0/1274456043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb918104280 0x7fb918106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.206+0000 7fb91d262700 1 -- 172.21.15.32:0/1274456043 >> 172.21.15.32:0/1274456043 conn(0x7fb9180fb2a0 msgr2=0x7fb9180fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.206+0000 7fb91d262700 1 -- 172.21.15.32:0/1274456043 shutdown_connections 2023-09-30T14:49:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.206+0000 7fb91d262700 1 -- 172.21.15.32:0/1274456043 wait complete. 2023-09-30T14:49:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.207+0000 7fb91d262700 1 Processor -- start 2023-09-30T14:49:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.207+0000 7fb91d262700 1 -- start start 2023-09-30T14:49:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.207+0000 7fb91d262700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb918101950 0x7fb9180ff900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.207+0000 7fb91d262700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb918104280 0x7fb9180ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.207+0000 7fb91d262700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918106bb0 0x7fb918100380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.207+0000 7fb91d262700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb918101800 con 0x7fb918106bb0 2023-09-30T14:49:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.207+0000 7fb91d262700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9181a0ce0 con 0x7fb918104280 2023-09-30T14:49:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.207+0000 7fb91d262700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9181a0ea0 con 0x7fb918101950 2023-09-30T14:49:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.207+0000 7fb916ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb918101950 0x7fb9180ff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.207+0000 7fb916ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb918101950 0x7fb9180ff900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52400/0 (socket says 172.21.15.32:52400) 2023-09-30T14:49:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.207+0000 7fb916ffd700 1 -- 172.21.15.32:0/1942790603 learned_addr learned my addr 172.21.15.32:0/1942790603 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.207+0000 7fb9177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918106bb0 0x7fb918100380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.207+0000 7fb9167fc700 1 --2- 172.21.15.32:0/1942790603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb918104280 0x7fb9180ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.208+0000 7fb916ffd700 1 -- 172.21.15.32:0/1942790603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb918104280 msgr2=0x7fb9180ffe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.208+0000 7fb916ffd700 1 --2- 172.21.15.32:0/1942790603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb918104280 0x7fb9180ffe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.208+0000 7fb916ffd700 1 -- 172.21.15.32:0/1942790603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918106bb0 msgr2=0x7fb918100380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.208+0000 7fb916ffd700 1 --2- 172.21.15.32:0/1942790603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918106bb0 0x7fb918100380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.208+0000 7fb916ffd700 1 -- 172.21.15.32:0/1942790603 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c009520 con 0x7fb918101950 2023-09-30T14:49:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.208+0000 7fb9177fe700 1 --2- 172.21.15.32:0/1942790603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918106bb0 0x7fb918100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:49:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.208+0000 7fb916ffd700 1 --2- 172.21.15.32:0/1942790603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb918101950 0x7fb9180ff900 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb90800e9a0 tx=0x7fb90800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.208+0000 7fb8fffff700 1 -- 172.21.15.32:0/1942790603 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb908009790 con 0x7fb918101950 2023-09-30T14:49:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.208+0000 7fb91d262700 1 -- 172.21.15.32:0/1942790603 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb910009d00 con 0x7fb918101950 2023-09-30T14:49:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.208+0000 7fb8fffff700 1 -- 172.21.15.32:0/1942790603 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb908010430 con 0x7fb918101950 2023-09-30T14:49:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.208+0000 7fb8fffff700 1 -- 172.21.15.32:0/1942790603 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb908018680 con 0x7fb918101950 2023-09-30T14:49:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.208+0000 7fb91d262700 1 -- 172.21.15.32:0/1942790603 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9181a1550 con 0x7fb918101950 2023-09-30T14:49:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.210+0000 7fb8fffff700 1 -- 172.21.15.32:0/1942790603 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fb9080105a0 con 0x7fb918101950 2023-09-30T14:49:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.211+0000 7fb8fffff700 1 --2- 172.21.15.32:0/1942790603 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9000a0fa0 0x7fb9000a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.211+0000 7fb9167fc700 1 --2- 172.21.15.32:0/1942790603 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9000a0fa0 0x7fb9000a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.211+0000 7fb8fffff700 1 -- 172.21.15.32:0/1942790603 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb908014070 con 0x7fb918101950 2023-09-30T14:49:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.211+0000 7fb9167fc700 1 --2- 172.21.15.32:0/1942790603 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9000a0fa0 0x7fb9000a3460 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb918100bb0 tx=0x7fb90c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.211+0000 7fb92944ab80 1 Processor -- start 2023-09-30T14:49:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb92944ab80 1 -- start start 2023-09-30T14:49:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb92944ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944668200 0x555944668620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb92944ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944669310 0x555944669790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb92944ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 0x555944670560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb92944ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559445d0400 con 0x555944669cd0 2023-09-30T14:49:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb92944ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559445c1bc0 con 0x555944668200 2023-09-30T14:49:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb92944ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559445bf4e0 con 0x555944669310 2023-09-30T14:49:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb916ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944668200 0x555944668620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb916ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944668200 0x555944668620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49502/0 (socket says 172.21.15.32:49502) 2023-09-30T14:49:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb9177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 0x555944670560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb916ffd700 1 -- 172.21.15.32:0/4236903674 learned_addr learned my addr 172.21.15.32:0/4236903674 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb9167fc700 1 --2- 172.21.15.32:0/4236903674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944669310 0x555944669790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb916ffd700 1 -- 172.21.15.32:0/4236903674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944669310 msgr2=0x555944669790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb916ffd700 1 --2- 172.21.15.32:0/4236903674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944669310 0x555944669790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb916ffd700 1 -- 172.21.15.32:0/4236903674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 msgr2=0x555944670560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb916ffd700 1 --2- 172.21.15.32:0/4236903674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 0x555944670560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb916ffd700 1 -- 172.21.15.32:0/4236903674 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55594461d870 con 0x555944668200 2023-09-30T14:49:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.212+0000 7fb916ffd700 1 --2- 172.21.15.32:0/4236903674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944668200 0x555944668620 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb908099d10 tx=0x7fb9080997e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d98adb47f05caa72 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.213+0000 7fb8fd7fa700 1 -- 172.21.15.32:0/4236903674 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90809ded0 con 0x555944668200 2023-09-30T14:49:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.213+0000 7fb8fd7fa700 1 -- 172.21.15.32:0/4236903674 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9080a2110 con 0x555944668200 2023-09-30T14:49:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.213+0000 7fb8fd7fa700 1 -- 172.21.15.32:0/4236903674 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9080b2210 con 0x555944668200 2023-09-30T14:49:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.213+0000 7fb92944ab80 1 -- 172.21.15.32:0/4236903674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944668200 msgr2=0x555944668620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.213+0000 7fb92944ab80 1 --2- 172.21.15.32:0/4236903674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944668200 0x555944668620 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb908099d10 tx=0x7fb9080997e0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.213+0000 7fb92944ab80 1 -- 172.21.15.32:0/4236903674 shutdown_connections 2023-09-30T14:49:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.213+0000 7fb92944ab80 1 --2- 172.21.15.32:0/4236903674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 0x555944670560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.213+0000 7fb92944ab80 1 --2- 172.21.15.32:0/4236903674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944668200 0x555944668620 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.213+0000 7fb92944ab80 1 --2- 172.21.15.32:0/4236903674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944669310 0x555944669790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.213+0000 7fb92944ab80 1 -- 172.21.15.32:0/4236903674 >> 172.21.15.32:0/4236903674 conn(0x5559445dc200 msgr2=0x555944662ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.213+0000 7fb92944ab80 1 -- 172.21.15.32:0/4236903674 shutdown_connections 2023-09-30T14:49:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.213+0000 7fb92944ab80 1 -- 172.21.15.32:0/4236903674 wait complete. 2023-09-30T14:49:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb92944ab80 1 -- start start 2023-09-30T14:49:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb92944ab80 1 Processor -- start 2023-09-30T14:49:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb92944ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944668200 0x555944671f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb92944ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944669310 0x555944672490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb92944ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 0x5559446794d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb92944ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559445badd0 con 0x555944669cd0 2023-09-30T14:49:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb92944ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559445576b0 con 0x555944669310 2023-09-30T14:49:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb92944ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555944557350 con 0x555944668200 2023-09-30T14:49:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb92944ab80 10 client.0 _ll_register_callbacks cb 0x5559443b47a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:49:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb916ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944668200 0x555944671f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb9177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 0x5559446794d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb916ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944668200 0x555944671f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52414/0 (socket says 172.21.15.32:52414) 2023-09-30T14:49:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb916ffd700 1 -- 172.21.15.32:0/2937779520 learned_addr learned my addr 172.21.15.32:0/2937779520 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb9167fc700 1 --2- 172.21.15.32:0/2937779520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944669310 0x555944672490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb916ffd700 1 -- 172.21.15.32:0/2937779520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944669310 msgr2=0x555944672490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb916ffd700 1 --2- 172.21.15.32:0/2937779520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944669310 0x555944672490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb916ffd700 1 -- 172.21.15.32:0/2937779520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 msgr2=0x5559446794d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb916ffd700 1 --2- 172.21.15.32:0/2937779520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 0x5559446794d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.214+0000 7fb916ffd700 1 -- 172.21.15.32:0/2937779520 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9181a1550 con 0x555944668200 2023-09-30T14:49:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb9167fc700 1 --2- 172.21.15.32:0/2937779520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944669310 0x555944672490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:49:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb9177fe700 1 --2- 172.21.15.32:0/2937779520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 0x5559446794d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:49:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb916ffd700 1 --2- 172.21.15.32:0/2937779520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944668200 0x555944671f50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb908009230 tx=0x7fb908099eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2aaec729c69b50e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb8faff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:49:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb8faff5700 1 -- 172.21.15.32:0/2937779520 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9080b2410 con 0x555944668200 2023-09-30T14:49:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb8faff5700 1 -- 172.21.15.32:0/2937779520 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9080a1c80 con 0x555944668200 2023-09-30T14:49:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb92944ab80 1 -- 172.21.15.32:0/2937779520 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55594465b090 con 0x555944668200 2023-09-30T14:49:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb8faff5700 1 -- 172.21.15.32:0/2937779520 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9080a1e40 con 0x555944668200 2023-09-30T14:49:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb92944ab80 10 client.26146 fetch_fsmap learned FSMap version 44 2023-09-30T14:49:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb92944ab80 1 -- 172.21.15.32:0/2937779520 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5559443e5d80 con 0x555944668200 2023-09-30T14:49:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb8faff5700 1 -- 172.21.15.32:0/2937779520 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55594465b090 con 0x555944668200 2023-09-30T14:49:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb8faff5700 1 -- 172.21.15.32:0/2937779520 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb90809ec10 con 0x555944668200 2023-09-30T14:49:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb92944ab80 10 client.26146 fetch_fsmap finished waiting for FSMap version 44 2023-09-30T14:49:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb92944ab80 10 client.26146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:49:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb92944ab80 20 client.26146 populate_metadata read hostname 'smithi032' 2023-09-30T14:49:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb92944ab80 1 --2- 172.21.15.32:0/2937779520 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55594467d9c0 0x55594467fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb92944ab80 4 client.26146 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:49:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.215+0000 7fb92944ab80 1 -- 172.21.15.32:0/2937779520 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559445b3db0 con 0x55594467d9c0 2023-09-30T14:49:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.216+0000 7fb9167fc700 1 --2- 172.21.15.32:0/2937779520 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55594467d9c0 0x55594467fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.216+0000 7fb8faff5700 10 client.26146 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.216+0000 7fb9167fc700 1 --2- 172.21.15.32:0/2937779520 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55594467d9c0 0x55594467fde0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e67b19019d70895 server_cookie=4b13b32fc8646dd6 in_seq=0 out_seq=0 2023-09-30T14:49:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.217+0000 7fb8faff5700 1 -- 172.21.15.32:0/2937779520 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5559445b3db0 con 0x55594467d9c0 2023-09-30T14:49:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.217+0000 7fb8faff5700 10 client.26146 handle_command_reply: tid=0 2023-09-30T14:49:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.217+0000 7fb92944ab80 1 client.26146 shutdown 2023-09-30T14:49:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.217+0000 7fb92944ab80 1 -- 172.21.15.32:0/2937779520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944668200 msgr2=0x555944671f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.217+0000 7fb92944ab80 1 --2- 172.21.15.32:0/2937779520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944668200 0x555944671f50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb908009230 tx=0x7fb908099eb0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.217+0000 7fb92944ab80 1 -- 172.21.15.32:0/2937779520 shutdown_connections 2023-09-30T14:49:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.217+0000 7fb92944ab80 1 --2- 172.21.15.32:0/2937779520 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55594467d9c0 0x55594467fde0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.217+0000 7fb92944ab80 1 --2- 172.21.15.32:0/2937779520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 0x5559446794d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.217+0000 7fb92944ab80 1 --2- 172.21.15.32:0/2937779520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944668200 0x555944671f50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.217+0000 7fb8faff5700 0 client.26146 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.217+0000 7fb92944ab80 1 --2- 172.21.15.32:0/2937779520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944669310 0x555944672490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.217+0000 7fb92944ab80 1 -- 172.21.15.32:0/2937779520 >> 172.21.15.32:0/2937779520 conn(0x5559445dc200 msgr2=0x555944662590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.217+0000 7fb92944ab80 1 -- 172.21.15.32:0/2937779520 shutdown_connections 2023-09-30T14:49:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.217+0000 7fb92944ab80 1 -- 172.21.15.32:0/2937779520 wait complete. 2023-09-30T14:49:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.217+0000 7fb92944ab80 20 client.26146 trim_cache size 0 max 16384 2023-09-30T14:49:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.233+0000 7fb92944ab80 1 Processor -- start 2023-09-30T14:49:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.233+0000 7fb92944ab80 1 -- start start 2023-09-30T14:49:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.233+0000 7fb92944ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944668200 0x555944670160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.233+0000 7fb92944ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944669310 0x555944679490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.233+0000 7fb92944ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 0x55594466ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.233+0000 7fb92944ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594443b8c0 con 0x555944669cd0 2023-09-30T14:49:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.233+0000 7fb92944ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559446730d0 con 0x555944669310 2023-09-30T14:49:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.233+0000 7fb92944ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555944664290 con 0x555944668200 2023-09-30T14:49:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.234+0000 7fb9167fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944669310 0x555944679490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.234+0000 7fb916ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944668200 0x555944670160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.234+0000 7fb916ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944668200 0x555944670160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52422/0 (socket says 172.21.15.32:52422) 2023-09-30T14:49:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.234+0000 7fb9167fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944669310 0x555944679490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49518/0 (socket says 172.21.15.32:49518) 2023-09-30T14:49:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.234+0000 7fb916ffd700 1 -- 172.21.15.32:0/3704728788 learned_addr learned my addr 172.21.15.32:0/3704728788 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.234+0000 7fb9177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 0x55594466ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.234+0000 7fb916ffd700 1 -- 172.21.15.32:0/3704728788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944669310 msgr2=0x555944679490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.234+0000 7fb916ffd700 1 --2- 172.21.15.32:0/3704728788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944669310 0x555944679490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.234+0000 7fb916ffd700 1 -- 172.21.15.32:0/3704728788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 msgr2=0x55594466ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.234+0000 7fb916ffd700 1 --2- 172.21.15.32:0/3704728788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 0x55594466ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.234+0000 7fb916ffd700 1 -- 172.21.15.32:0/3704728788 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559443cec00 con 0x555944668200 2023-09-30T14:49:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.234+0000 7fb9177fe700 1 --2- 172.21.15.32:0/3704728788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 0x55594466ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:49:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.234+0000 7fb916ffd700 1 --2- 172.21.15.32:0/3704728788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944668200 0x555944670160 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb908099f40 tx=0x7fb908098330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acb9ab34f049d77 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.234+0000 7fb8fdffb700 1 -- 172.21.15.32:0/3704728788 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb908010880 con 0x555944668200 2023-09-30T14:49:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.234+0000 7fb8fdffb700 1 -- 172.21.15.32:0/3704728788 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb90809ec10 con 0x555944668200 2023-09-30T14:49:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.234+0000 7fb8fdffb700 1 -- 172.21.15.32:0/3704728788 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9080a1ee0 con 0x555944668200 2023-09-30T14:49:57.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.235+0000 7fb92944ab80 1 -- 172.21.15.32:0/3704728788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944668200 msgr2=0x555944670160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.235+0000 7fb92944ab80 1 --2- 172.21.15.32:0/3704728788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944668200 0x555944670160 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb908099f40 tx=0x7fb908098330 comp rx=0 tx=0).stop 2023-09-30T14:49:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.235+0000 7fb92944ab80 1 -- 172.21.15.32:0/3704728788 shutdown_connections 2023-09-30T14:49:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.235+0000 7fb92944ab80 1 --2- 172.21.15.32:0/3704728788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669cd0 0x55594466ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.235+0000 7fb92944ab80 1 --2- 172.21.15.32:0/3704728788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944668200 0x555944670160 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.235+0000 7fb92944ab80 1 --2- 172.21.15.32:0/3704728788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944669310 0x555944679490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.235+0000 7fb92944ab80 1 -- 172.21.15.32:0/3704728788 >> 172.21.15.32:0/3704728788 conn(0x5559445dc200 msgr2=0x555944670c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.235+0000 7fb92944ab80 1 -- 172.21.15.32:0/3704728788 shutdown_connections 2023-09-30T14:49:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.235+0000 7fb92944ab80 1 -- 172.21.15.32:0/3704728788 wait complete. 2023-09-30T14:49:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.236+0000 7fb92944ab80 1 -- start start 2023-09-30T14:49:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.236+0000 7fb92944ab80 1 Processor -- start 2023-09-30T14:49:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.236+0000 7fb92944ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944668200 0x555944676950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.236+0000 7fb92944ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669310 0x555944676e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.236+0000 7fb92944ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944669cd0 0x555944677ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.236+0000 7fb92944ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559446783e0 con 0x555944669310 2023-09-30T14:49:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.236+0000 7fb92944ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555944678550 con 0x555944668200 2023-09-30T14:49:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.236+0000 7fb92944ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555944678710 con 0x555944669cd0 2023-09-30T14:49:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.236+0000 7fb92944ab80 10 client.0 _ll_register_callbacks cb 0x7fb908005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:49:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.236+0000 7fb9167fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669310 0x555944676e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.236+0000 7fb916ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944668200 0x555944676950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.236+0000 7fb9177fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944669cd0 0x555944677ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.236+0000 7fb9167fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669310 0x555944676e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38048/0 (socket says 172.21.15.32:38048) 2023-09-30T14:49:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.236+0000 7fb9167fc700 1 -- 172.21.15.32:0/933492002 learned_addr learned my addr 172.21.15.32:0/933492002 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:49:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.237+0000 7fb9167fc700 1 -- 172.21.15.32:0/933492002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944669cd0 msgr2=0x555944677ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.237+0000 7fb9167fc700 1 --2- 172.21.15.32:0/933492002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944669cd0 0x555944677ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.237+0000 7fb9167fc700 1 -- 172.21.15.32:0/933492002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944668200 msgr2=0x555944676950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:49:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.237+0000 7fb9167fc700 1 --2- 172.21.15.32:0/933492002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944668200 0x555944676950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.237+0000 7fb9167fc700 1 -- 172.21.15.32:0/933492002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb918109b90 con 0x555944669310 2023-09-30T14:49:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.237+0000 7fb916ffd700 1 --2- 172.21.15.32:0/933492002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944668200 0x555944676950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:49:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.237+0000 7fb9167fc700 1 --2- 172.21.15.32:0/933492002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669310 0x555944676e90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x555944680680 tx=0x7fb90c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73deac0a70a6e89 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:49:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.237+0000 7fb8faff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:49:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.237+0000 7fb8faff5700 1 -- 172.21.15.32:0/933492002 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c02a220 con 0x555944669310 2023-09-30T14:49:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.237+0000 7fb8faff5700 1 -- 172.21.15.32:0/933492002 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb90c030070 con 0x555944669310 2023-09-30T14:49:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.237+0000 7fb92944ab80 1 -- 172.21.15.32:0/933492002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9080a2110 con 0x555944669310 2023-09-30T14:49:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.237+0000 7fb8faff5700 1 -- 172.21.15.32:0/933492002 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb90c03a030 con 0x555944669310 2023-09-30T14:49:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.237+0000 7fb92944ab80 10 client.16482 fetch_fsmap learned FSMap version 44 2023-09-30T14:49:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.237+0000 7fb92944ab80 1 -- 172.21.15.32:0/933492002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5559443e5d80 con 0x555944669310 2023-09-30T14:49:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.237+0000 7fb8faff5700 1 -- 172.21.15.32:0/933492002 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9080a2110 con 0x555944669310 2023-09-30T14:49:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.238+0000 7fb8faff5700 1 -- 172.21.15.32:0/933492002 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb90c01d910 con 0x555944669310 2023-09-30T14:49:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.238+0000 7fb92944ab80 10 client.16482 fetch_fsmap finished waiting for FSMap version 44 2023-09-30T14:49:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.238+0000 7fb92944ab80 10 client.16482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:49:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.238+0000 7fb92944ab80 20 client.16482 populate_metadata read hostname 'smithi032' 2023-09-30T14:49:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.238+0000 7fb92944ab80 1 --2- 172.21.15.32:0/933492002 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5559446d1f40 0x5559446d4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:49:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.238+0000 7fb92944ab80 4 client.16482 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:49:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.238+0000 7fb92944ab80 1 -- 172.21.15.32:0/933492002 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5559445b3ad0 con 0x5559446d1f40 2023-09-30T14:49:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.238+0000 7fb916ffd700 1 --2- 172.21.15.32:0/933492002 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5559446d1f40 0x5559446d4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:49:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.238+0000 7fb8faff5700 10 client.16482 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.238+0000 7fb916ffd700 1 --2- 172.21.15.32:0/933492002 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5559446d1f40 0x5559446d4330 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=555fd50f37bbe327 server_cookie=413528e48d4c0909 in_seq=0 out_seq=0 2023-09-30T14:49:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.312+0000 7fb8faff5700 1 -- 172.21.15.32:0/933492002 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5559445b3ad0 con 0x5559446d1f40 2023-09-30T14:49:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.312+0000 7fb8faff5700 10 client.16482 handle_command_reply: tid=0 2023-09-30T14:49:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.312+0000 7fb92944ab80 1 client.16482 shutdown 2023-09-30T14:49:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.312+0000 7fb92944ab80 1 -- 172.21.15.32:0/933492002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669310 msgr2=0x555944676e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.312+0000 7fb92944ab80 1 --2- 172.21.15.32:0/933492002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669310 0x555944676e90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x555944680680 tx=0x7fb90c0047c0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.312+0000 7fb92944ab80 1 -- 172.21.15.32:0/933492002 shutdown_connections 2023-09-30T14:49:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.312+0000 7fb92944ab80 1 --2- 172.21.15.32:0/933492002 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5559446d1f40 0x5559446d4330 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.312+0000 7fb92944ab80 1 --2- 172.21.15.32:0/933492002 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555944669cd0 0x555944677ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb8faff5700 0 client.16482 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:49:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb92944ab80 1 --2- 172.21.15.32:0/933492002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555944668200 0x555944676950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb92944ab80 1 --2- 172.21.15.32:0/933492002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555944669310 0x555944676e90 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb92944ab80 1 -- 172.21.15.32:0/933492002 >> 172.21.15.32:0/933492002 conn(0x5559445dc200 msgr2=0x555944670160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb92944ab80 1 -- 172.21.15.32:0/933492002 shutdown_connections 2023-09-30T14:49:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb92944ab80 1 -- 172.21.15.32:0/933492002 wait complete. 2023-09-30T14:49:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb92944ab80 20 client.16482 trim_cache size 0 max 16384 2023-09-30T14:49:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:49:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:49:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:49:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:49:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb91d262700 1 -- 172.21.15.32:0/1942790603 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9000a0fa0 msgr2=0x7fb9000a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb91d262700 1 --2- 172.21.15.32:0/1942790603 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9000a0fa0 0x7fb9000a3460 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb918100bb0 tx=0x7fb90c009260 comp rx=0 tx=0).stop 2023-09-30T14:49:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb91d262700 1 -- 172.21.15.32:0/1942790603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb918101950 msgr2=0x7fb9180ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:49:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb91d262700 1 --2- 172.21.15.32:0/1942790603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb918101950 0x7fb9180ff900 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb90800e9a0 tx=0x7fb90800ecb0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb91d262700 1 -- 172.21.15.32:0/1942790603 shutdown_connections 2023-09-30T14:49:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb91d262700 1 --2- 172.21.15.32:0/1942790603 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9000a0fa0 0x7fb9000a3460 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb91d262700 1 --2- 172.21.15.32:0/1942790603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb918106bb0 0x7fb918100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb91d262700 1 --2- 172.21.15.32:0/1942790603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb918101950 0x7fb9180ff900 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb91d262700 1 --2- 172.21.15.32:0/1942790603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb918104280 0x7fb9180ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:49:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.313+0000 7fb91d262700 1 -- 172.21.15.32:0/1942790603 >> 172.21.15.32:0/1942790603 conn(0x7fb9180fb2a0 msgr2=0x7fb9181025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:49:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.314+0000 7fb91d262700 1 -- 172.21.15.32:0/1942790603 shutdown_connections 2023-09-30T14:49:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:49:57.314+0000 7fb91d262700 1 -- 172.21.15.32:0/1942790603 wait complete. 2023-09-30T14:49:57.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1278951 10.67 MB/sec execute 511 sec latency 64.545 ms 2023-09-30T14:49:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:57 smithi156 ceph-mon[103579]: pgmap v637: 97 pgs: 97 active+clean; 372 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 2.8 MiB/s wr, 262 op/s 2023-09-30T14:49:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:57 smithi148 ceph-mon[102355]: pgmap v637: 97 pgs: 97 active+clean; 372 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 2.8 MiB/s wr, 262 op/s 2023-09-30T14:49:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:57 smithi032 ceph-mon[98592]: pgmap v637: 97 pgs: 97 active+clean; 372 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 2.8 MiB/s wr, 262 op/s 2023-09-30T14:49:58.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1281358 10.67 MB/sec execute 512 sec latency 18.548 ms 2023-09-30T14:49:59.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1282828 10.67 MB/sec execute 513 sec latency 24.856 ms 2023-09-30T14:49:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:49:59 smithi156 ceph-mon[103579]: pgmap v638: 97 pgs: 97 active+clean; 370 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.7 MiB/s wr, 270 op/s 2023-09-30T14:49:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:49:59 smithi148 ceph-mon[102355]: pgmap v638: 97 pgs: 97 active+clean; 370 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.7 MiB/s wr, 270 op/s 2023-09-30T14:49:59.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:49:59 smithi032 ceph-mon[98592]: pgmap v638: 97 pgs: 97 active+clean; 370 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.7 MiB/s wr, 270 op/s 2023-09-30T14:50:00.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:00 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-a[98568]: 2023-09-30T14:49:59.998+0000 7f49afbf4700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2023-09-30T14:50:00.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1285194 10.67 MB/sec execute 514 sec latency 46.725 ms 2023-09-30T14:50:00.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:00 smithi156 ceph-mon[103579]: overall HEALTH_ERR 1 MDSs report damaged metadata 2023-09-30T14:50:00.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:00 smithi148 ceph-mon[102355]: overall HEALTH_ERR 1 MDSs report damaged metadata 2023-09-30T14:50:00.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:00 smithi032 ceph-mon[98592]: overall HEALTH_ERR 1 MDSs report damaged metadata 2023-09-30T14:50:01.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1286551 10.67 MB/sec execute 515 sec latency 16.765 ms 2023-09-30T14:50:01.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:01 smithi156 ceph-mon[103579]: pgmap v639: 97 pgs: 97 active+clean; 370 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 2.5 MiB/s wr, 248 op/s 2023-09-30T14:50:01.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:01 smithi148 ceph-mon[102355]: pgmap v639: 97 pgs: 97 active+clean; 370 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 2.5 MiB/s wr, 248 op/s 2023-09-30T14:50:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:01 smithi032 ceph-mon[98592]: pgmap v639: 97 pgs: 97 active+clean; 370 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 2.5 MiB/s wr, 248 op/s 2023-09-30T14:50:02.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1289114 10.68 MB/sec execute 516 sec latency 26.891 ms 2023-09-30T14:50:03.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1290529 10.67 MB/sec execute 517 sec latency 16.532 ms 2023-09-30T14:50:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:03 smithi156 ceph-mon[103579]: pgmap v640: 97 pgs: 97 active+clean; 372 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.6 MiB/s wr, 270 op/s 2023-09-30T14:50:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:03 smithi148 ceph-mon[102355]: pgmap v640: 97 pgs: 97 active+clean; 372 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.6 MiB/s wr, 270 op/s 2023-09-30T14:50:03.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:03 smithi032 ceph-mon[98592]: pgmap v640: 97 pgs: 97 active+clean; 372 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.6 MiB/s wr, 270 op/s 2023-09-30T14:50:04.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1292903 10.68 MB/sec execute 518 sec latency 11.618 ms 2023-09-30T14:50:05.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1294522 10.67 MB/sec execute 519 sec latency 30.075 ms 2023-09-30T14:50:05.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:05 smithi156 ceph-mon[103579]: pgmap v641: 97 pgs: 97 active+clean; 372 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 2.6 MiB/s wr, 270 op/s 2023-09-30T14:50:05.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:05 smithi148 ceph-mon[102355]: pgmap v641: 97 pgs: 97 active+clean; 372 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 2.6 MiB/s wr, 270 op/s 2023-09-30T14:50:05.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:05 smithi032 ceph-mon[98592]: pgmap v641: 97 pgs: 97 active+clean; 372 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 2.6 MiB/s wr, 270 op/s 2023-09-30T14:50:06.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1296634 10.67 MB/sec execute 520 sec latency 26.712 ms 2023-09-30T14:50:07.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1298629 10.67 MB/sec execute 521 sec latency 53.107 ms 2023-09-30T14:50:07.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:07 smithi156 ceph-mon[103579]: pgmap v642: 97 pgs: 97 active+clean; 375 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 3.0 MiB/s wr, 305 op/s 2023-09-30T14:50:07.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:07 smithi148 ceph-mon[102355]: pgmap v642: 97 pgs: 97 active+clean; 375 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 3.0 MiB/s wr, 305 op/s 2023-09-30T14:50:07.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:07 smithi032 ceph-mon[98592]: pgmap v642: 97 pgs: 97 active+clean; 375 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 3.0 MiB/s wr, 305 op/s 2023-09-30T14:50:08.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1300258 10.67 MB/sec execute 522 sec latency 20.244 ms 2023-09-30T14:50:09.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1302418 10.66 MB/sec execute 523 sec latency 45.157 ms 2023-09-30T14:50:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:09 smithi156 ceph-mon[103579]: pgmap v643: 97 pgs: 97 active+clean; 375 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.6 MiB/s wr, 273 op/s 2023-09-30T14:50:09.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:09 smithi148 ceph-mon[102355]: pgmap v643: 97 pgs: 97 active+clean; 375 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.6 MiB/s wr, 273 op/s 2023-09-30T14:50:09.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:09 smithi032 ceph-mon[98592]: pgmap v643: 97 pgs: 97 active+clean; 375 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.6 MiB/s wr, 273 op/s 2023-09-30T14:50:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.429+0000 7f1ae596b700 1 -- 172.21.15.32:0/111172312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae0100ea0 msgr2=0x7f1ae01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.429+0000 7f1ae596b700 1 --2- 172.21.15.32:0/111172312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae0100ea0 0x7f1ae01012c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009990 tx=0x7f1ad0009ca0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.429+0000 7f1ae596b700 1 -- 172.21.15.32:0/111172312 shutdown_connections 2023-09-30T14:50:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.429+0000 7f1ae596b700 1 --2- 172.21.15.32:0/111172312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae01029b0 0x7f1ae01092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.429+0000 7f1ae596b700 1 --2- 172.21.15.32:0/111172312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae0100ea0 0x7f1ae01012c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.429+0000 7f1ae596b700 1 --2- 172.21.15.32:0/111172312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ae0101ff0 0x7f1ae0102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.429+0000 7f1ae596b700 1 -- 172.21.15.32:0/111172312 >> 172.21.15.32:0/111172312 conn(0x7f1ae00fc3f0 msgr2=0x7f1ae00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.429+0000 7f1ae596b700 1 -- 172.21.15.32:0/111172312 shutdown_connections 2023-09-30T14:50:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.429+0000 7f1ae596b700 1 -- 172.21.15.32:0/111172312 wait complete. 2023-09-30T14:50:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.429+0000 7f1ae596b700 1 Processor -- start 2023-09-30T14:50:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1ae596b700 1 -- start start 2023-09-30T14:50:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1ae596b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae0100ea0 0x7f1ae0196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1ae596b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ae0101ff0 0x7f1ae0197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1ae596b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae01029b0 0x7f1ae0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1ae596b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae01986b0 con 0x7f1ae01029b0 2023-09-30T14:50:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1ae596b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae0198820 con 0x7f1ae0101ff0 2023-09-30T14:50:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1ae596b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae01989e0 con 0x7f1ae0100ea0 2023-09-30T14:50:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1adf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae01029b0 0x7f1ae0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1adeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae0100ea0 0x7f1ae0196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1adeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae0100ea0 0x7f1ae0196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52442/0 (socket says 172.21.15.32:52442) 2023-09-30T14:50:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1ade7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ae0101ff0 0x7f1ae0197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1adeffd700 1 -- 172.21.15.32:0/106197903 learned_addr learned my addr 172.21.15.32:0/106197903 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1adf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae01029b0 0x7f1ae0197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38064/0 (socket says 172.21.15.32:38064) 2023-09-30T14:50:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1adeffd700 1 -- 172.21.15.32:0/106197903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ae0101ff0 msgr2=0x7f1ae0197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1adeffd700 1 --2- 172.21.15.32:0/106197903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ae0101ff0 0x7f1ae0197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1adeffd700 1 -- 172.21.15.32:0/106197903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae01029b0 msgr2=0x7f1ae0197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1adeffd700 1 --2- 172.21.15.32:0/106197903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae01029b0 0x7f1ae0197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1adeffd700 1 -- 172.21.15.32:0/106197903 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad8009c40 con 0x7f1ae0100ea0 2023-09-30T14:50:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1ade7fc700 1 --2- 172.21.15.32:0/106197903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ae0101ff0 0x7f1ae0197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:50:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.430+0000 7f1adf7fe700 1 --2- 172.21.15.32:0/106197903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae01029b0 0x7f1ae0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:50:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.431+0000 7f1adeffd700 1 --2- 172.21.15.32:0/106197903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae0100ea0 0x7f1ae0196d30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000b620 tx=0x7f1ad0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.431+0000 7f1ae4969700 1 -- 172.21.15.32:0/106197903 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad001d070 con 0x7f1ae0100ea0 2023-09-30T14:50:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.431+0000 7f1ae4969700 1 -- 172.21.15.32:0/106197903 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ad0022470 con 0x7f1ae0100ea0 2023-09-30T14:50:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.431+0000 7f1ae596b700 1 -- 172.21.15.32:0/106197903 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad00095f0 con 0x7f1ae0100ea0 2023-09-30T14:50:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.431+0000 7f1ae4969700 1 -- 172.21.15.32:0/106197903 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0017650 con 0x7f1ae0100ea0 2023-09-30T14:50:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.431+0000 7f1ae596b700 1 -- 172.21.15.32:0/106197903 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae019f200 con 0x7f1ae0100ea0 2023-09-30T14:50:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.433+0000 7f1ae4969700 1 -- 172.21.15.32:0/106197903 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f1ad00225e0 con 0x7f1ae0100ea0 2023-09-30T14:50:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.434+0000 7f1ae4969700 1 --2- 172.21.15.32:0/106197903 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ac80a0fa0 0x7f1ac80a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.434+0000 7f1ade7fc700 1 --2- 172.21.15.32:0/106197903 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ac80a0fa0 0x7f1ac80a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.434+0000 7f1ae4969700 1 -- 172.21.15.32:0/106197903 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ad00cdb60 con 0x7f1ae0100ea0 2023-09-30T14:50:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.434+0000 7f1af1b53b80 1 Processor -- start 2023-09-30T14:50:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.434+0000 7f1af1b53b80 1 -- start start 2023-09-30T14:50:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.434+0000 7f1ade7fc700 1 --2- 172.21.15.32:0/106197903 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ac80a0fa0 0x7f1ac80a3460 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4011490 tx=0x7f1ad4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.434+0000 7f1af1b53b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e4200 0x55a71e4e4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.434+0000 7f1af1b53b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5310 0x55a71e4e5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1af1b53b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e5cd0 0x55a71e4ec560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1af1b53b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71e44c400 con 0x55a71e4e5cd0 2023-09-30T14:50:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1af1b53b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71e43dbc0 con 0x55a71e4e5310 2023-09-30T14:50:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1af1b53b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71e43b4e0 con 0x55a71e4e4200 2023-09-30T14:50:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1adeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e4200 0x55a71e4e4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1adf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e5cd0 0x55a71e4ec560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1adeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e4200 0x55a71e4e4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52450/0 (socket says 172.21.15.32:52450) 2023-09-30T14:50:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1adeffd700 1 -- 172.21.15.32:0/319744933 learned_addr learned my addr 172.21.15.32:0/319744933 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1ade7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5310 0x55a71e4e5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1adeffd700 1 -- 172.21.15.32:0/319744933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5310 msgr2=0x55a71e4e5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1adeffd700 1 --2- 172.21.15.32:0/319744933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5310 0x55a71e4e5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1adeffd700 1 -- 172.21.15.32:0/319744933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e5cd0 msgr2=0x55a71e4ec560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1adeffd700 1 --2- 172.21.15.32:0/319744933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e5cd0 0x55a71e4ec560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1adeffd700 1 -- 172.21.15.32:0/319744933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a71e499870 con 0x55a71e4e4200 2023-09-30T14:50:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1adf7fe700 1 --2- 172.21.15.32:0/319744933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e5cd0 0x55a71e4ec560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:50:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1adeffd700 1 --2- 172.21.15.32:0/319744933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e4200 0x55a71e4e4620 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00975c0 tx=0x7f1ad009af10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f13d3a69909aa895 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1ac5ffb700 1 -- 172.21.15.32:0/319744933 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad009b7c0 con 0x55a71e4e4200 2023-09-30T14:50:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1ac5ffb700 1 -- 172.21.15.32:0/319744933 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ad009b0c0 con 0x55a71e4e4200 2023-09-30T14:50:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.435+0000 7f1ac5ffb700 1 -- 172.21.15.32:0/319744933 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad009b320 con 0x55a71e4e4200 2023-09-30T14:50:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.436+0000 7f1af1b53b80 1 -- 172.21.15.32:0/319744933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e4200 msgr2=0x55a71e4e4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.436+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/319744933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e4200 0x55a71e4e4620 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00975c0 tx=0x7f1ad009af10 comp rx=0 tx=0).stop 2023-09-30T14:50:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.436+0000 7f1af1b53b80 1 -- 172.21.15.32:0/319744933 shutdown_connections 2023-09-30T14:50:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.436+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/319744933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e5cd0 0x55a71e4ec560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.436+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/319744933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e4200 0x55a71e4e4620 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.436+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/319744933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5310 0x55a71e4e5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.436+0000 7f1af1b53b80 1 -- 172.21.15.32:0/319744933 >> 172.21.15.32:0/319744933 conn(0x55a71e458200 msgr2=0x55a71e4deef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.436+0000 7f1af1b53b80 1 -- 172.21.15.32:0/319744933 shutdown_connections 2023-09-30T14:50:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.436+0000 7f1af1b53b80 1 -- 172.21.15.32:0/319744933 wait complete. 2023-09-30T14:50:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.437+0000 7f1af1b53b80 1 -- start start 2023-09-30T14:50:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.437+0000 7f1af1b53b80 1 Processor -- start 2023-09-30T14:50:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.437+0000 7f1af1b53b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e4200 0x55a71e4edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.437+0000 7f1af1b53b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e5310 0x55a71e4ee110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.437+0000 7f1af1b53b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5cd0 0x55a71e4f5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.437+0000 7f1af1b53b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71e436dd0 con 0x55a71e4e4200 2023-09-30T14:50:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.437+0000 7f1af1b53b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71e3d36b0 con 0x55a71e4e5cd0 2023-09-30T14:50:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.437+0000 7f1af1b53b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71e3d3350 con 0x55a71e4e5310 2023-09-30T14:50:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.437+0000 7f1af1b53b80 10 client.0 _ll_register_callbacks cb 0x55a71e2307a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:50:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.437+0000 7f1adeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e4200 0x55a71e4edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.437+0000 7f1adeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e4200 0x55a71e4edbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38074/0 (socket says 172.21.15.32:38074) 2023-09-30T14:50:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.437+0000 7f1adeffd700 1 -- 172.21.15.32:0/1258559378 learned_addr learned my addr 172.21.15.32:0/1258559378 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.437+0000 7f1ade7fc700 1 --2- 172.21.15.32:0/1258559378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e5310 0x55a71e4ee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.437+0000 7f1adf7fe700 1 --2- 172.21.15.32:0/1258559378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5cd0 0x55a71e4f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.438+0000 7f1adeffd700 1 -- 172.21.15.32:0/1258559378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e5310 msgr2=0x55a71e4ee110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.438+0000 7f1adeffd700 1 --2- 172.21.15.32:0/1258559378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e5310 0x55a71e4ee110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.438+0000 7f1adeffd700 1 -- 172.21.15.32:0/1258559378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5cd0 msgr2=0x55a71e4f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.438+0000 7f1adeffd700 1 --2- 172.21.15.32:0/1258559378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5cd0 0x55a71e4f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.438+0000 7f1adeffd700 1 -- 172.21.15.32:0/1258559378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad80090f0 con 0x55a71e4e4200 2023-09-30T14:50:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.438+0000 7f1adf7fe700 1 --2- 172.21.15.32:0/1258559378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5cd0 0x55a71e4f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:50:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.438+0000 7f1adeffd700 1 --2- 172.21.15.32:0/1258559378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e4200 0x55a71e4edbd0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00cb6c0 tx=0x7f1ad0099460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3047cadcb1c7d3d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.438+0000 7f1abeffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:50:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.438+0000 7f1abeffd700 1 -- 172.21.15.32:0/1258559378 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad00a33c0 con 0x55a71e4e4200 2023-09-30T14:50:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.438+0000 7f1abeffd700 1 -- 172.21.15.32:0/1258559378 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ad009b7c0 con 0x55a71e4e4200 2023-09-30T14:50:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.438+0000 7f1af1b53b80 1 -- 172.21.15.32:0/1258559378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a71e4d7090 con 0x55a71e4e4200 2023-09-30T14:50:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.438+0000 7f1abeffd700 1 -- 172.21.15.32:0/1258559378 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ad009b320 con 0x55a71e4e4200 2023-09-30T14:50:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.438+0000 7f1af1b53b80 10 client.16494 fetch_fsmap learned FSMap version 44 2023-09-30T14:50:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.438+0000 7f1af1b53b80 1 -- 172.21.15.32:0/1258559378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f1ae019f200 con 0x55a71e4e4200 2023-09-30T14:50:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.438+0000 7f1abeffd700 1 -- 172.21.15.32:0/1258559378 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a71e4d7090 con 0x55a71e4e4200 2023-09-30T14:50:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.439+0000 7f1abeffd700 1 -- 172.21.15.32:0/1258559378 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1ad00cbb30 con 0x55a71e4e4200 2023-09-30T14:50:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.439+0000 7f1af1b53b80 10 client.16494 fetch_fsmap finished waiting for FSMap version 44 2023-09-30T14:50:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.439+0000 7f1af1b53b80 10 client.16494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:50:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.439+0000 7f1af1b53b80 20 client.16494 populate_metadata read hostname 'smithi032' 2023-09-30T14:50:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.439+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/1258559378 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a71e4f9590 0x55a71e4fb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.439+0000 7f1af1b53b80 4 client.16494 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:50:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.439+0000 7f1af1b53b80 1 -- 172.21.15.32:0/1258559378 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a71e42fdb0 con 0x55a71e4f9590 2023-09-30T14:50:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.439+0000 7f1ade7fc700 1 --2- 172.21.15.32:0/1258559378 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a71e4f9590 0x55a71e4fb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.439+0000 7f1ade7fc700 1 --2- 172.21.15.32:0/1258559378 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a71e4f9590 0x55a71e4fb9b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a872f41204227d2e server_cookie=5b957a1b76b2c2fd in_seq=0 out_seq=0 2023-09-30T14:50:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.439+0000 7f1abeffd700 10 client.16494 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:50:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.440+0000 7f1abeffd700 1 -- 172.21.15.32:0/1258559378 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a71e42fdb0 con 0x55a71e4f9590 2023-09-30T14:50:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.440+0000 7f1abeffd700 10 client.16494 handle_command_reply: tid=0 2023-09-30T14:50:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.440+0000 7f1af1b53b80 1 client.16494 shutdown 2023-09-30T14:50:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.440+0000 7f1af1b53b80 1 -- 172.21.15.32:0/1258559378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e4200 msgr2=0x55a71e4edbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.440+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/1258559378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e4200 0x55a71e4edbd0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00cb6c0 tx=0x7f1ad0099460 comp rx=0 tx=0).stop 2023-09-30T14:50:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.440+0000 7f1af1b53b80 1 -- 172.21.15.32:0/1258559378 shutdown_connections 2023-09-30T14:50:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.440+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/1258559378 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a71e4f9590 0x55a71e4fb9b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.440+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/1258559378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5cd0 0x55a71e4f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.440+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/1258559378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e4200 0x55a71e4edbd0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.440+0000 7f1abeffd700 0 client.16494 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:50:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.440+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/1258559378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e5310 0x55a71e4ee110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.440+0000 7f1af1b53b80 1 -- 172.21.15.32:0/1258559378 >> 172.21.15.32:0/1258559378 conn(0x55a71e458200 msgr2=0x55a71e4de590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.440+0000 7f1af1b53b80 1 -- 172.21.15.32:0/1258559378 shutdown_connections 2023-09-30T14:50:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.440+0000 7f1af1b53b80 1 -- 172.21.15.32:0/1258559378 wait complete. 2023-09-30T14:50:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.440+0000 7f1af1b53b80 20 client.16494 trim_cache size 0 max 16384 2023-09-30T14:50:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.456+0000 7f1af1b53b80 1 Processor -- start 2023-09-30T14:50:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.456+0000 7f1af1b53b80 1 -- start start 2023-09-30T14:50:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1af1b53b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e4200 0x55a71e4fb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1af1b53b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5310 0x55a71e4f5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1af1b53b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e5cd0 0x55a71e4e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1af1b53b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71e2b78c0 con 0x55a71e4e5cd0 2023-09-30T14:50:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1af1b53b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71e4dfd70 con 0x55a71e4e5310 2023-09-30T14:50:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1af1b53b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71e4ed740 con 0x55a71e4e4200 2023-09-30T14:50:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1ade7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5310 0x55a71e4f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1adf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e5cd0 0x55a71e4e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1adf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e5cd0 0x55a71e4e8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38086/0 (socket says 172.21.15.32:38086) 2023-09-30T14:50:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1ade7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5310 0x55a71e4f5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49560/0 (socket says 172.21.15.32:49560) 2023-09-30T14:50:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1adf7fe700 1 -- 172.21.15.32:0/2251539211 learned_addr learned my addr 172.21.15.32:0/2251539211 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1adeffd700 1 --2- 172.21.15.32:0/2251539211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e4200 0x55a71e4fb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1adf7fe700 1 -- 172.21.15.32:0/2251539211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e4200 msgr2=0x55a71e4fb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1adf7fe700 1 --2- 172.21.15.32:0/2251539211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e4200 0x55a71e4fb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1adf7fe700 1 -- 172.21.15.32:0/2251539211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5310 msgr2=0x55a71e4f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1adf7fe700 1 --2- 172.21.15.32:0/2251539211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5310 0x55a71e4f5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1adf7fe700 1 -- 172.21.15.32:0/2251539211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a71e261d80 con 0x55a71e4e5cd0 2023-09-30T14:50:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1ade7fc700 1 --2- 172.21.15.32:0/2251539211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5310 0x55a71e4f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:50:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1adf7fe700 1 --2- 172.21.15.32:0/2251539211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e5cd0 0x55a71e4e8ae0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8003d20 tx=0x7f1ad800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a707d47b4bc1da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.457+0000 7f1ac67fc700 1 -- 172.21.15.32:0/2251539211 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8007700 con 0x55a71e4e5cd0 2023-09-30T14:50:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.458+0000 7f1ac67fc700 1 -- 172.21.15.32:0/2251539211 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ad800f040 con 0x55a71e4e5cd0 2023-09-30T14:50:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.458+0000 7f1ac67fc700 1 -- 172.21.15.32:0/2251539211 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad800ea50 con 0x55a71e4e5cd0 2023-09-30T14:50:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.458+0000 7f1af1b53b80 1 -- 172.21.15.32:0/2251539211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e5cd0 msgr2=0x55a71e4e8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.458+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/2251539211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e5cd0 0x55a71e4e8ae0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8003d20 tx=0x7f1ad800b040 comp rx=0 tx=0).stop 2023-09-30T14:50:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.458+0000 7f1af1b53b80 1 -- 172.21.15.32:0/2251539211 shutdown_connections 2023-09-30T14:50:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.458+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/2251539211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e5cd0 0x55a71e4e8ae0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.458+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/2251539211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e4200 0x55a71e4fb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.458+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/2251539211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5310 0x55a71e4f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.458+0000 7f1af1b53b80 1 -- 172.21.15.32:0/2251539211 >> 172.21.15.32:0/2251539211 conn(0x55a71e458200 msgr2=0x55a71e500310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.458+0000 7f1af1b53b80 1 -- 172.21.15.32:0/2251539211 shutdown_connections 2023-09-30T14:50:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.458+0000 7f1af1b53b80 1 -- 172.21.15.32:0/2251539211 wait complete. 2023-09-30T14:50:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.459+0000 7f1af1b53b80 1 -- start start 2023-09-30T14:50:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.459+0000 7f1af1b53b80 1 Processor -- start 2023-09-30T14:50:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.459+0000 7f1af1b53b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e4200 0x55a71e4f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.459+0000 7f1af1b53b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e5310 0x55a71e4f3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.459+0000 7f1af1b53b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5cd0 0x55a71e4f4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.459+0000 7f1af1b53b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71e4f48b0 con 0x55a71e4e4200 2023-09-30T14:50:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.459+0000 7f1af1b53b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71e4f4a20 con 0x55a71e4e5cd0 2023-09-30T14:50:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.459+0000 7f1af1b53b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71e4f4be0 con 0x55a71e4e5310 2023-09-30T14:50:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.459+0000 7f1af1b53b80 10 client.0 _ll_register_callbacks cb 0x7f1ae01978f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:50:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.459+0000 7f1adeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e4200 0x55a71e4f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.459+0000 7f1adeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e4200 0x55a71e4f2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38088/0 (socket says 172.21.15.32:38088) 2023-09-30T14:50:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.459+0000 7f1adf7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5cd0 0x55a71e4f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.459+0000 7f1adeffd700 1 -- 172.21.15.32:0/2001944507 learned_addr learned my addr 172.21.15.32:0/2001944507 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.459+0000 7f1ade7fc700 1 --2- 172.21.15.32:0/2001944507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e5310 0x55a71e4f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1adf7fe700 1 -- 172.21.15.32:0/2001944507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e5310 msgr2=0x55a71e4f3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1adf7fe700 1 --2- 172.21.15.32:0/2001944507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e5310 0x55a71e4f3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1adf7fe700 1 -- 172.21.15.32:0/2001944507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e4200 msgr2=0x55a71e4f2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1adf7fe700 1 --2- 172.21.15.32:0/2001944507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e4200 0x55a71e4f2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1adf7fe700 1 -- 172.21.15.32:0/2001944507 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad4017070 con 0x55a71e4e5cd0 2023-09-30T14:50:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1adf7fe700 1 --2- 172.21.15.32:0/2001944507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5cd0 0x55a71e4f4370 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55a71e4e4d20 tx=0x7f1ad800ea00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33f80c9a74276cc4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1abeffd700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:50:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1abeffd700 1 -- 172.21.15.32:0/2001944507 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8024070 con 0x55a71e4e5cd0 2023-09-30T14:50:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1af1b53b80 1 -- 172.21.15.32:0/2001944507 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ad009b0c0 con 0x55a71e4e5cd0 2023-09-30T14:50:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1abeffd700 1 -- 172.21.15.32:0/2001944507 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ad8003e70 con 0x55a71e4e5cd0 2023-09-30T14:50:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1abeffd700 1 -- 172.21.15.32:0/2001944507 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ad802e030 con 0x55a71e4e5cd0 2023-09-30T14:50:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1af1b53b80 10 client.26186 fetch_fsmap learned FSMap version 44 2023-09-30T14:50:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1af1b53b80 1 -- 172.21.15.32:0/2001944507 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f1ad8003a00 con 0x55a71e4e5cd0 2023-09-30T14:50:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1abeffd700 1 -- 172.21.15.32:0/2001944507 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ad009b0c0 con 0x55a71e4e5cd0 2023-09-30T14:50:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1abeffd700 1 -- 172.21.15.32:0/2001944507 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1ad801f470 con 0x55a71e4e5cd0 2023-09-30T14:50:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1af1b53b80 10 client.26186 fetch_fsmap finished waiting for FSMap version 44 2023-09-30T14:50:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1af1b53b80 10 client.26186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:50:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.460+0000 7f1af1b53b80 20 client.26186 populate_metadata read hostname 'smithi032' 2023-09-30T14:50:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.461+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/2001944507 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a71e54e160 0x55a71e550550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.461+0000 7f1af1b53b80 4 client.26186 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:50:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.461+0000 7f1af1b53b80 1 -- 172.21.15.32:0/2001944507 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a71e42fad0 con 0x55a71e54e160 2023-09-30T14:50:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.461+0000 7f1adeffd700 1 --2- 172.21.15.32:0/2001944507 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a71e54e160 0x55a71e550550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.461+0000 7f1abeffd700 10 client.26186 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:50:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.461+0000 7f1adeffd700 1 --2- 172.21.15.32:0/2001944507 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a71e54e160 0x55a71e550550 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5198eefcd881e66e server_cookie=700a5bf5b00ef0a2 in_seq=0 out_seq=0 2023-09-30T14:50:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.513+0000 7f1abeffd700 1 -- 172.21.15.32:0/2001944507 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a71e42fad0 con 0x55a71e54e160 2023-09-30T14:50:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.513+0000 7f1abeffd700 10 client.26186 handle_command_reply: tid=0 2023-09-30T14:50:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.513+0000 7f1af1b53b80 1 client.26186 shutdown 2023-09-30T14:50:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1af1b53b80 1 -- 172.21.15.32:0/2001944507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5cd0 msgr2=0x55a71e4f4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/2001944507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5cd0 0x55a71e4f4370 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55a71e4e4d20 tx=0x7f1ad800ea00 comp rx=0 tx=0).stop 2023-09-30T14:50:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1af1b53b80 1 -- 172.21.15.32:0/2001944507 shutdown_connections 2023-09-30T14:50:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/2001944507 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a71e54e160 0x55a71e550550 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/2001944507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a71e4e5cd0 0x55a71e4f4370 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1abeffd700 0 client.26186 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:50:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/2001944507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a71e4e4200 0x55a71e4f2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1af1b53b80 1 --2- 172.21.15.32:0/2001944507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a71e4e5310 0x55a71e4f3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1af1b53b80 1 -- 172.21.15.32:0/2001944507 >> 172.21.15.32:0/2001944507 conn(0x55a71e458200 msgr2=0x55a71e4e8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1af1b53b80 1 -- 172.21.15.32:0/2001944507 shutdown_connections 2023-09-30T14:50:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1af1b53b80 1 -- 172.21.15.32:0/2001944507 wait complete. 2023-09-30T14:50:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1af1b53b80 20 client.26186 trim_cache size 0 max 16384 2023-09-30T14:50:10.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:50:10.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:50:10.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:50:10.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:50:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1ae596b700 1 -- 172.21.15.32:0/106197903 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ac80a0fa0 msgr2=0x7f1ac80a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1ae596b700 1 --2- 172.21.15.32:0/106197903 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ac80a0fa0 0x7f1ac80a3460 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4011490 tx=0x7f1ad4009260 comp rx=0 tx=0).stop 2023-09-30T14:50:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1ae596b700 1 -- 172.21.15.32:0/106197903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae0100ea0 msgr2=0x7f1ae0196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1ae596b700 1 --2- 172.21.15.32:0/106197903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae0100ea0 0x7f1ae0196d30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000b620 tx=0x7f1ad0005740 comp rx=0 tx=0).stop 2023-09-30T14:50:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1ae596b700 1 -- 172.21.15.32:0/106197903 shutdown_connections 2023-09-30T14:50:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1ae596b700 1 --2- 172.21.15.32:0/106197903 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f1ac80a0fa0 0x7f1ac80a3460 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1ae596b700 1 --2- 172.21.15.32:0/106197903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae01029b0 0x7f1ae0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1ae596b700 1 --2- 172.21.15.32:0/106197903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae0100ea0 0x7f1ae0196d30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1ae596b700 1 --2- 172.21.15.32:0/106197903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ae0101ff0 0x7f1ae0197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:10.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.514+0000 7f1ae596b700 1 -- 172.21.15.32:0/106197903 >> 172.21.15.32:0/106197903 conn(0x7f1ae00fc3f0 msgr2=0x7f1ae00fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:10.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.515+0000 7f1ae596b700 1 -- 172.21.15.32:0/106197903 shutdown_connections 2023-09-30T14:50:10.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:10.515+0000 7f1ae596b700 1 -- 172.21.15.32:0/106197903 wait complete. 2023-09-30T14:50:10.723 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1303931 10.66 MB/sec execute 524 sec latency 26.173 ms 2023-09-30T14:50:11.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1306278 10.67 MB/sec execute 525 sec latency 24.627 ms 2023-09-30T14:50:11.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:11 smithi156 ceph-mon[103579]: pgmap v644: 97 pgs: 97 active+clean; 377 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 2.7 MiB/s wr, 260 op/s 2023-09-30T14:50:11.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:11 smithi148 ceph-mon[102355]: pgmap v644: 97 pgs: 97 active+clean; 377 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 2.7 MiB/s wr, 260 op/s 2023-09-30T14:50:11.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:11 smithi032 ceph-mon[98592]: pgmap v644: 97 pgs: 97 active+clean; 377 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 2.7 MiB/s wr, 260 op/s 2023-09-30T14:50:12.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1307527 10.66 MB/sec execute 526 sec latency 66.378 ms 2023-09-30T14:50:13.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1309911 10.66 MB/sec execute 527 sec latency 20.358 ms 2023-09-30T14:50:13.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:13 smithi148 ceph-mon[102355]: pgmap v645: 97 pgs: 97 active+clean; 378 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 2.9 MiB/s wr, 276 op/s 2023-09-30T14:50:13.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:13 smithi032 ceph-mon[98592]: pgmap v645: 97 pgs: 97 active+clean; 378 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 2.9 MiB/s wr, 276 op/s 2023-09-30T14:50:14.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:13 smithi156 ceph-mon[103579]: pgmap v645: 97 pgs: 97 active+clean; 378 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 2.9 MiB/s wr, 276 op/s 2023-09-30T14:50:14.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1311367 10.66 MB/sec execute 528 sec latency 16.024 ms 2023-09-30T14:50:15.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1314006 10.67 MB/sec execute 529 sec latency 13.864 ms 2023-09-30T14:50:15.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:15 smithi148 ceph-mon[102355]: pgmap v646: 97 pgs: 97 active+clean; 379 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 2.6 MiB/s wr, 221 op/s 2023-09-30T14:50:15.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:15 smithi032 ceph-mon[98592]: pgmap v646: 97 pgs: 97 active+clean; 379 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 2.6 MiB/s wr, 221 op/s 2023-09-30T14:50:16.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:15 smithi156 ceph-mon[103579]: pgmap v646: 97 pgs: 97 active+clean; 379 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 2.6 MiB/s wr, 221 op/s 2023-09-30T14:50:16.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1315407 10.66 MB/sec execute 530 sec latency 55.851 ms 2023-09-30T14:50:17.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1317510 10.66 MB/sec execute 531 sec latency 85.692 ms 2023-09-30T14:50:17.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:17 smithi148 ceph-mon[102355]: pgmap v647: 97 pgs: 97 active+clean; 381 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 3.0 MiB/s wr, 300 op/s 2023-09-30T14:50:17.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:17 smithi032 ceph-mon[98592]: pgmap v647: 97 pgs: 97 active+clean; 381 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 3.0 MiB/s wr, 300 op/s 2023-09-30T14:50:18.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:17 smithi156 ceph-mon[103579]: pgmap v647: 97 pgs: 97 active+clean; 381 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 3.0 MiB/s wr, 300 op/s 2023-09-30T14:50:18.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1318956 10.66 MB/sec execute 532 sec latency 16.531 ms 2023-09-30T14:50:19.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1321333 10.66 MB/sec execute 533 sec latency 11.897 ms 2023-09-30T14:50:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:19 smithi148 ceph-mon[102355]: pgmap v648: 97 pgs: 97 active+clean; 382 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 2.6 MiB/s wr, 248 op/s 2023-09-30T14:50:19.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:19 smithi032 ceph-mon[98592]: pgmap v648: 97 pgs: 97 active+clean; 382 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 2.6 MiB/s wr, 248 op/s 2023-09-30T14:50:20.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:19 smithi156 ceph-mon[103579]: pgmap v648: 97 pgs: 97 active+clean; 382 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 2.6 MiB/s wr, 248 op/s 2023-09-30T14:50:20.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1323229 10.65 MB/sec execute 534 sec latency 20.400 ms 2023-09-30T14:50:21.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1325079 10.66 MB/sec execute 535 sec latency 13.264 ms 2023-09-30T14:50:21.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:21 smithi148 ceph-mon[102355]: pgmap v649: 97 pgs: 97 active+clean; 383 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 2.7 MiB/s wr, 254 op/s 2023-09-30T14:50:21.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:21 smithi032 ceph-mon[98592]: pgmap v649: 97 pgs: 97 active+clean; 383 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 2.7 MiB/s wr, 254 op/s 2023-09-30T14:50:22.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:21 smithi156 ceph-mon[103579]: pgmap v649: 97 pgs: 97 active+clean; 383 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 2.7 MiB/s wr, 254 op/s 2023-09-30T14:50:22.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1327014 10.65 MB/sec execute 536 sec latency 94.440 ms 2023-09-30T14:50:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.631+0000 7fb9fbc41700 1 -- 172.21.15.32:0/1859778297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4100f40 msgr2=0x7fb9f41013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.631+0000 7fb9fbc41700 1 --2- 172.21.15.32:0/1859778297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4100f40 0x7fb9f41013a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40099c0 tx=0x7fb9e4009cd0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.631+0000 7fb9fbc41700 1 -- 172.21.15.32:0/1859778297 shutdown_connections 2023-09-30T14:50:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.631+0000 7fb9fbc41700 1 --2- 172.21.15.32:0/1859778297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9f41018e0 0x7fb9f4108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.631+0000 7fb9fbc41700 1 --2- 172.21.15.32:0/1859778297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9f40ffd50 0x7fb9f4100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.631+0000 7fb9fbc41700 1 --2- 172.21.15.32:0/1859778297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4100f40 0x7fb9f41013a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.631+0000 7fb9fbc41700 1 -- 172.21.15.32:0/1859778297 >> 172.21.15.32:0/1859778297 conn(0x7fb9f40fb2a0 msgr2=0x7fb9f40fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.631+0000 7fb9fbc41700 1 -- 172.21.15.32:0/1859778297 shutdown_connections 2023-09-30T14:50:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.631+0000 7fb9fbc41700 1 -- 172.21.15.32:0/1859778297 wait complete. 2023-09-30T14:50:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9fbc41700 1 Processor -- start 2023-09-30T14:50:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9fbc41700 1 -- start start 2023-09-30T14:50:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9fbc41700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9f40ffd50 0x7fb9f4198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9fbc41700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4100f40 0x7fb9f4199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9fbc41700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9f41018e0 0x7fb9f419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9fbc41700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f419a8d0 con 0x7fb9f4100f40 2023-09-30T14:50:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9fbc41700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f419aa40 con 0x7fb9f40ffd50 2023-09-30T14:50:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9fbc41700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f419ac00 con 0x7fb9f41018e0 2023-09-30T14:50:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9f91dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4100f40 0x7fb9f4199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9fa1de700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9f41018e0 0x7fb9f419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9f91dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4100f40 0x7fb9f4199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38104/0 (socket says 172.21.15.32:38104) 2023-09-30T14:50:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9f91dc700 1 -- 172.21.15.32:0/1583856564 learned_addr learned my addr 172.21.15.32:0/1583856564 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9f99dd700 1 --2- 172.21.15.32:0/1583856564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9f40ffd50 0x7fb9f4198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9f91dc700 1 -- 172.21.15.32:0/1583856564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9f41018e0 msgr2=0x7fb9f419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9f91dc700 1 --2- 172.21.15.32:0/1583856564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9f41018e0 0x7fb9f419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9f91dc700 1 -- 172.21.15.32:0/1583856564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9f40ffd50 msgr2=0x7fb9f4198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9f91dc700 1 --2- 172.21.15.32:0/1583856564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9f40ffd50 0x7fb9f4198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9f91dc700 1 -- 172.21.15.32:0/1583856564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e40095f0 con 0x7fb9f4100f40 2023-09-30T14:50:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9f99dd700 1 --2- 172.21.15.32:0/1583856564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9f40ffd50 0x7fb9f4198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:50:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.632+0000 7fb9fa1de700 1 --2- 172.21.15.32:0/1583856564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9f41018e0 0x7fb9f419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:50:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.633+0000 7fb9f91dc700 1 --2- 172.21.15.32:0/1583856564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4100f40 0x7fb9f4199420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009fd0 tx=0x7fb9e4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.633+0000 7fb9eaffd700 1 -- 172.21.15.32:0/1583856564 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e401d070 con 0x7fb9f4100f40 2023-09-30T14:50:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.633+0000 7fb9eaffd700 1 -- 172.21.15.32:0/1583856564 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9e4005030 con 0x7fb9f4100f40 2023-09-30T14:50:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.633+0000 7fb9fbc41700 1 -- 172.21.15.32:0/1583856564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f419ae80 con 0x7fb9f4100f40 2023-09-30T14:50:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.633+0000 7fb9eaffd700 1 -- 172.21.15.32:0/1583856564 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4004a90 con 0x7fb9f4100f40 2023-09-30T14:50:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.633+0000 7fb9fbc41700 1 -- 172.21.15.32:0/1583856564 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f41a15f0 con 0x7fb9f4100f40 2023-09-30T14:50:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.635+0000 7fb9eaffd700 1 -- 172.21.15.32:0/1583856564 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fb9e40051a0 con 0x7fb9f4100f40 2023-09-30T14:50:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.636+0000 7fb9eaffd700 1 --2- 172.21.15.32:0/1583856564 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9e00a0fa0 0x7fb9e00a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.636+0000 7fb9eaffd700 1 -- 172.21.15.32:0/1583856564 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9e40cd4d0 con 0x7fb9f4100f40 2023-09-30T14:50:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.636+0000 7fb9f99dd700 1 --2- 172.21.15.32:0/1583856564 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9e00a0fa0 0x7fb9e00a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.637+0000 7fb9f99dd700 1 --2- 172.21.15.32:0/1583856564 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9e00a0fa0 0x7fb9e00a3460 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4198c30 tx=0x7fb9f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.637+0000 7fba07e29b80 1 Processor -- start 2023-09-30T14:50:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.637+0000 7fba07e29b80 1 -- start start 2023-09-30T14:50:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.637+0000 7fba07e29b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9db2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.637+0000 7fba07e29b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 0x55d5c9db3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.637+0000 7fba07e29b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3cd0 0x55d5c9dba560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.637+0000 7fba07e29b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c9d1a400 con 0x55d5c9db2200 2023-09-30T14:50:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.637+0000 7fba07e29b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c9d0bbc0 con 0x55d5c9db3cd0 2023-09-30T14:50:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.637+0000 7fba07e29b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c9d094e0 con 0x55d5c9db3310 2023-09-30T14:50:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.638+0000 7fb9f99dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9db2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.638+0000 7fb9f99dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9db2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38110/0 (socket says 172.21.15.32:38110) 2023-09-30T14:50:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.638+0000 7fb9f99dd700 1 -- 172.21.15.32:0/58947652 learned_addr learned my addr 172.21.15.32:0/58947652 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.638+0000 7fb9f91dc700 1 --2- 172.21.15.32:0/58947652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 0x55d5c9db3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.638+0000 7fb9fa1de700 1 --2- 172.21.15.32:0/58947652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3cd0 0x55d5c9dba560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.638+0000 7fb9f91dc700 1 -- 172.21.15.32:0/58947652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3cd0 msgr2=0x55d5c9dba560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.638+0000 7fb9f91dc700 1 --2- 172.21.15.32:0/58947652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3cd0 0x55d5c9dba560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.638+0000 7fb9f91dc700 1 -- 172.21.15.32:0/58947652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 msgr2=0x55d5c9db2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.638+0000 7fb9f91dc700 1 --2- 172.21.15.32:0/58947652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9db2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.638+0000 7fb9f91dc700 1 -- 172.21.15.32:0/58947652 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5c9d67870 con 0x55d5c9db3310 2023-09-30T14:50:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.638+0000 7fb9f99dd700 1 --2- 172.21.15.32:0/58947652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9db2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:50:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.638+0000 7fb9f91dc700 1 --2- 172.21.15.32:0/58947652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 0x55d5c9db3790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb9e400bf40 tx=0x7fb9e409a4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f965a95d9d11e121 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.638+0000 7fb9dffff700 1 -- 172.21.15.32:0/58947652 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e409eb80 con 0x55d5c9db3310 2023-09-30T14:50:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.638+0000 7fb9dffff700 1 -- 172.21.15.32:0/58947652 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9e40a2d30 con 0x55d5c9db3310 2023-09-30T14:50:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.638+0000 7fb9dffff700 1 -- 172.21.15.32:0/58947652 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4004c10 con 0x55d5c9db3310 2023-09-30T14:50:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.639+0000 7fba07e29b80 1 -- 172.21.15.32:0/58947652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 msgr2=0x55d5c9db3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.639+0000 7fba07e29b80 1 --2- 172.21.15.32:0/58947652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 0x55d5c9db3790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb9e400bf40 tx=0x7fb9e409a4b0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.639+0000 7fba07e29b80 1 -- 172.21.15.32:0/58947652 shutdown_connections 2023-09-30T14:50:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.639+0000 7fba07e29b80 1 --2- 172.21.15.32:0/58947652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3cd0 0x55d5c9dba560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.639+0000 7fba07e29b80 1 --2- 172.21.15.32:0/58947652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9db2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.639+0000 7fba07e29b80 1 --2- 172.21.15.32:0/58947652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 0x55d5c9db3790 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.639+0000 7fba07e29b80 1 -- 172.21.15.32:0/58947652 >> 172.21.15.32:0/58947652 conn(0x55d5c9d26200 msgr2=0x55d5c9dacef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.639+0000 7fba07e29b80 1 -- 172.21.15.32:0/58947652 shutdown_connections 2023-09-30T14:50:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.639+0000 7fba07e29b80 1 -- 172.21.15.32:0/58947652 wait complete. 2023-09-30T14:50:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.639+0000 7fba07e29b80 1 -- start start 2023-09-30T14:50:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fba07e29b80 1 Processor -- start 2023-09-30T14:50:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fba07e29b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dbbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fba07e29b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 0x55d5c9dbc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fba07e29b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3cd0 0x55d5c9dc30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fba07e29b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c9d04dd0 con 0x55d5c9db2200 2023-09-30T14:50:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fba07e29b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c9ca16b0 con 0x55d5c9db3cd0 2023-09-30T14:50:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fba07e29b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c9ca1350 con 0x55d5c9db3310 2023-09-30T14:50:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fba07e29b80 10 client.0 _ll_register_callbacks cb 0x55d5c9afe7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:50:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fb9f99dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dbbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fb9f99dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dbbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38116/0 (socket says 172.21.15.32:38116) 2023-09-30T14:50:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fb9f99dd700 1 -- 172.21.15.32:0/1630437254 learned_addr learned my addr 172.21.15.32:0/1630437254 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fb9fa1de700 1 --2- 172.21.15.32:0/1630437254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3cd0 0x55d5c9dc30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fb9f91dc700 1 --2- 172.21.15.32:0/1630437254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 0x55d5c9dbc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fb9f99dd700 1 -- 172.21.15.32:0/1630437254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 msgr2=0x55d5c9dbc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fb9f99dd700 1 --2- 172.21.15.32:0/1630437254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 0x55d5c9dbc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fb9f99dd700 1 -- 172.21.15.32:0/1630437254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3cd0 msgr2=0x55d5c9dc30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fb9f99dd700 1 --2- 172.21.15.32:0/1630437254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3cd0 0x55d5c9dc30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fb9f99dd700 1 -- 172.21.15.32:0/1630437254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f0017070 con 0x55d5c9db2200 2023-09-30T14:50:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.640+0000 7fb9f91dc700 1 --2- 172.21.15.32:0/1630437254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 0x55d5c9dbc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:50:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.641+0000 7fb9f99dd700 1 --2- 172.21.15.32:0/1630437254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dbbb40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00037c0 tx=0x7fb9f00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4473cf890cae656a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.641+0000 7fb9dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:50:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.641+0000 7fb9dd7fa700 1 -- 172.21.15.32:0/1630437254 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f001f740 con 0x55d5c9db2200 2023-09-30T14:50:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.641+0000 7fb9dd7fa700 1 -- 172.21.15.32:0/1630437254 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9f001f8a0 con 0x55d5c9db2200 2023-09-30T14:50:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.641+0000 7fba07e29b80 1 -- 172.21.15.32:0/1630437254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5c9da5090 con 0x55d5c9db2200 2023-09-30T14:50:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.641+0000 7fb9dd7fa700 1 -- 172.21.15.32:0/1630437254 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9f001fa60 con 0x55d5c9db2200 2023-09-30T14:50:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.641+0000 7fba07e29b80 10 client.16512 fetch_fsmap learned FSMap version 44 2023-09-30T14:50:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.641+0000 7fba07e29b80 1 -- 172.21.15.32:0/1630437254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7fb9f41a15f0 con 0x55d5c9db2200 2023-09-30T14:50:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.641+0000 7fb9dd7fa700 1 -- 172.21.15.32:0/1630437254 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d5c9da5090 con 0x55d5c9db2200 2023-09-30T14:50:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.641+0000 7fb9dd7fa700 1 -- 172.21.15.32:0/1630437254 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb9f0031070 con 0x55d5c9db2200 2023-09-30T14:50:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.641+0000 7fba07e29b80 10 client.16512 fetch_fsmap finished waiting for FSMap version 44 2023-09-30T14:50:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.641+0000 7fba07e29b80 10 client.16512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:50:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.641+0000 7fba07e29b80 20 client.16512 populate_metadata read hostname 'smithi032' 2023-09-30T14:50:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.641+0000 7fba07e29b80 1 --2- 172.21.15.32:0/1630437254 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d5c9dc79f0 0x55d5c9dc9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.641+0000 7fba07e29b80 4 client.16512 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:50:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.641+0000 7fba07e29b80 1 -- 172.21.15.32:0/1630437254 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5c9cfddb0 con 0x55d5c9dc79f0 2023-09-30T14:50:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.642+0000 7fb9f91dc700 1 --2- 172.21.15.32:0/1630437254 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d5c9dc79f0 0x55d5c9dc9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.642+0000 7fb9dd7fa700 10 client.16512 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:50:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.642+0000 7fb9f91dc700 1 --2- 172.21.15.32:0/1630437254 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d5c9dc79f0 0x55d5c9dc9e10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=178e0e82807d5309 server_cookie=183204d896f421b in_seq=0 out_seq=0 2023-09-30T14:50:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.643+0000 7fb9dd7fa700 1 -- 172.21.15.32:0/1630437254 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d5c9cfddb0 con 0x55d5c9dc79f0 2023-09-30T14:50:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.643+0000 7fb9dd7fa700 10 client.16512 handle_command_reply: tid=0 2023-09-30T14:50:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.643+0000 7fba07e29b80 1 client.16512 shutdown 2023-09-30T14:50:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.643+0000 7fba07e29b80 1 -- 172.21.15.32:0/1630437254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 msgr2=0x55d5c9dbbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.643+0000 7fba07e29b80 1 --2- 172.21.15.32:0/1630437254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dbbb40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00037c0 tx=0x7fb9f00047c0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.643+0000 7fba07e29b80 1 -- 172.21.15.32:0/1630437254 shutdown_connections 2023-09-30T14:50:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.643+0000 7fba07e29b80 1 --2- 172.21.15.32:0/1630437254 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d5c9dc79f0 0x55d5c9dc9e10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.643+0000 7fba07e29b80 1 --2- 172.21.15.32:0/1630437254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3cd0 0x55d5c9dc30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.643+0000 7fb9dd7fa700 0 client.16512 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:50:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.643+0000 7fba07e29b80 1 --2- 172.21.15.32:0/1630437254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dbbb40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.643+0000 7fba07e29b80 1 --2- 172.21.15.32:0/1630437254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 0x55d5c9dbc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.643+0000 7fba07e29b80 1 -- 172.21.15.32:0/1630437254 >> 172.21.15.32:0/1630437254 conn(0x55d5c9d26200 msgr2=0x55d5c9dac590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.643+0000 7fba07e29b80 1 -- 172.21.15.32:0/1630437254 shutdown_connections 2023-09-30T14:50:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.643+0000 7fba07e29b80 1 -- 172.21.15.32:0/1630437254 wait complete. 2023-09-30T14:50:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.643+0000 7fba07e29b80 20 client.16512 trim_cache size 0 max 16384 2023-09-30T14:50:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fba07e29b80 1 Processor -- start 2023-09-30T14:50:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fba07e29b80 1 -- start start 2023-09-30T14:50:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fba07e29b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dae630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fba07e29b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 0x55d5c9dc3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fba07e29b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3cd0 0x55d5c9dac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fba07e29b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c9b858c0 con 0x55d5c9db2200 2023-09-30T14:50:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fba07e29b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c9db6540 con 0x55d5c9db3cd0 2023-09-30T14:50:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fba07e29b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c9dacad0 con 0x55d5c9db3310 2023-09-30T14:50:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fb9f99dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fb9f99dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dae630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38124/0 (socket says 172.21.15.32:38124) 2023-09-30T14:50:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fb9f99dd700 1 -- 172.21.15.32:0/2300818300 learned_addr learned my addr 172.21.15.32:0/2300818300 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fb9fa1de700 1 --2- 172.21.15.32:0/2300818300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3cd0 0x55d5c9dac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fb9f91dc700 1 --2- 172.21.15.32:0/2300818300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 0x55d5c9dc3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fb9f99dd700 1 -- 172.21.15.32:0/2300818300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 msgr2=0x55d5c9dc3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fb9f99dd700 1 --2- 172.21.15.32:0/2300818300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 0x55d5c9dc3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fb9f99dd700 1 -- 172.21.15.32:0/2300818300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3cd0 msgr2=0x55d5c9dac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fb9f99dd700 1 --2- 172.21.15.32:0/2300818300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3cd0 0x55d5c9dac590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.659+0000 7fb9f99dd700 1 -- 172.21.15.32:0/2300818300 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5c9b2fd80 con 0x55d5c9db2200 2023-09-30T14:50:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.660+0000 7fb9f99dd700 1 --2- 172.21.15.32:0/2300818300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dae630 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0003e60 tx=0x7fb9f0003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79758268bdb5322f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.660+0000 7fb9e8d82700 1 -- 172.21.15.32:0/2300818300 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f001f240 con 0x55d5c9db2200 2023-09-30T14:50:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.660+0000 7fb9e8d82700 1 -- 172.21.15.32:0/2300818300 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9f002e070 con 0x55d5c9db2200 2023-09-30T14:50:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.660+0000 7fb9e8d82700 1 -- 172.21.15.32:0/2300818300 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0005690 con 0x55d5c9db2200 2023-09-30T14:50:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.660+0000 7fba07e29b80 1 -- 172.21.15.32:0/2300818300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 msgr2=0x55d5c9dae630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.660+0000 7fba07e29b80 1 --2- 172.21.15.32:0/2300818300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dae630 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0003e60 tx=0x7fb9f0003e90 comp rx=0 tx=0).stop 2023-09-30T14:50:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.660+0000 7fba07e29b80 1 -- 172.21.15.32:0/2300818300 shutdown_connections 2023-09-30T14:50:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.660+0000 7fba07e29b80 1 --2- 172.21.15.32:0/2300818300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3cd0 0x55d5c9dac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.660+0000 7fba07e29b80 1 --2- 172.21.15.32:0/2300818300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dae630 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.660+0000 7fba07e29b80 1 --2- 172.21.15.32:0/2300818300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3310 0x55d5c9dc3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.660+0000 7fba07e29b80 1 -- 172.21.15.32:0/2300818300 >> 172.21.15.32:0/2300818300 conn(0x55d5c9d26200 msgr2=0x55d5c9dce790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.660+0000 7fba07e29b80 1 -- 172.21.15.32:0/2300818300 shutdown_connections 2023-09-30T14:50:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.661+0000 7fba07e29b80 1 -- 172.21.15.32:0/2300818300 wait complete. 2023-09-30T14:50:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.661+0000 7fba07e29b80 1 -- start start 2023-09-30T14:50:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.661+0000 7fba07e29b80 1 Processor -- start 2023-09-30T14:50:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.661+0000 7fba07e29b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dc0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.661+0000 7fba07e29b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3310 0x55d5c9dc1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.661+0000 7fba07e29b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3cd0 0x55d5c9dc2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.661+0000 7fba07e29b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c9dc26a0 con 0x55d5c9db2200 2023-09-30T14:50:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.661+0000 7fba07e29b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c9dc2810 con 0x55d5c9db3310 2023-09-30T14:50:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.661+0000 7fba07e29b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c9dc29d0 con 0x55d5c9db3cd0 2023-09-30T14:50:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.661+0000 7fba07e29b80 10 client.0 _ll_register_callbacks cb 0x7fb9f0004660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:50:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.661+0000 7fb9f99dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dc0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.661+0000 7fb9f99dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dc0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38130/0 (socket says 172.21.15.32:38130) 2023-09-30T14:50:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.661+0000 7fb9f99dd700 1 -- 172.21.15.32:0/655528206 learned_addr learned my addr 172.21.15.32:0/655528206 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.661+0000 7fb9f91dc700 1 --2- 172.21.15.32:0/655528206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3310 0x55d5c9dc1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.662+0000 7fb9fa1de700 1 --2- 172.21.15.32:0/655528206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3cd0 0x55d5c9dc2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.662+0000 7fb9f99dd700 1 -- 172.21.15.32:0/655528206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3cd0 msgr2=0x55d5c9dc2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.662+0000 7fb9f99dd700 1 --2- 172.21.15.32:0/655528206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3cd0 0x55d5c9dc2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.662+0000 7fb9f99dd700 1 -- 172.21.15.32:0/655528206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3310 msgr2=0x55d5c9dc1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.662+0000 7fb9f99dd700 1 --2- 172.21.15.32:0/655528206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3310 0x55d5c9dc1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.662+0000 7fb9f99dd700 1 -- 172.21.15.32:0/655528206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f41a15f0 con 0x55d5c9db2200 2023-09-30T14:50:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.662+0000 7fb9f91dc700 1 --2- 172.21.15.32:0/655528206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3310 0x55d5c9dc1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:50:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.662+0000 7fb9f99dd700 1 --2- 172.21.15.32:0/655528206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dc0c10 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0005210 tx=0x7fb9f0029f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffaa49880b15577 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.662+0000 7fb9dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:50:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.662+0000 7fb9dd7fa700 1 -- 172.21.15.32:0/655528206 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0004b90 con 0x55d5c9db2200 2023-09-30T14:50:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.662+0000 7fb9dd7fa700 1 -- 172.21.15.32:0/655528206 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9f0004cf0 con 0x55d5c9db2200 2023-09-30T14:50:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.662+0000 7fba07e29b80 1 -- 172.21.15.32:0/655528206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9e40a2d30 con 0x55d5c9db2200 2023-09-30T14:50:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.662+0000 7fb9dd7fa700 1 -- 172.21.15.32:0/655528206 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9f0028440 con 0x55d5c9db2200 2023-09-30T14:50:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.662+0000 7fba07e29b80 10 client.16518 fetch_fsmap learned FSMap version 44 2023-09-30T14:50:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.662+0000 7fba07e29b80 1 -- 172.21.15.32:0/655528206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55d5c9b18c00 con 0x55d5c9db2200 2023-09-30T14:50:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.662+0000 7fb9dd7fa700 1 -- 172.21.15.32:0/655528206 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9e40a2d30 con 0x55d5c9db2200 2023-09-30T14:50:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.663+0000 7fb9dd7fa700 1 -- 172.21.15.32:0/655528206 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb9f0031070 con 0x55d5c9db2200 2023-09-30T14:50:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.663+0000 7fba07e29b80 10 client.16518 fetch_fsmap finished waiting for FSMap version 44 2023-09-30T14:50:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.663+0000 7fba07e29b80 10 client.16518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:50:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.663+0000 7fba07e29b80 20 client.16518 populate_metadata read hostname 'smithi032' 2023-09-30T14:50:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.663+0000 7fba07e29b80 1 --2- 172.21.15.32:0/655528206 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d5c9e1c250 0x55d5c9e1e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.663+0000 7fba07e29b80 4 client.16518 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:50:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.663+0000 7fba07e29b80 1 -- 172.21.15.32:0/655528206 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d5c9cfdad0 con 0x55d5c9e1c250 2023-09-30T14:50:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.663+0000 7fb9f91dc700 1 --2- 172.21.15.32:0/655528206 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d5c9e1c250 0x55d5c9e1e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.663+0000 7fb9dd7fa700 10 client.16518 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:50:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.663+0000 7fb9f91dc700 1 --2- 172.21.15.32:0/655528206 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d5c9e1c250 0x55d5c9e1e670 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7310b644d0525c6 server_cookie=3ac14dd9e6b7ce95 in_seq=0 out_seq=0 2023-09-30T14:50:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.714+0000 7fb9dd7fa700 1 -- 172.21.15.32:0/655528206 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d5c9cfdad0 con 0x55d5c9e1c250 2023-09-30T14:50:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.714+0000 7fb9dd7fa700 10 client.16518 handle_command_reply: tid=0 2023-09-30T14:50:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.714+0000 7fba07e29b80 1 client.16518 shutdown 2023-09-30T14:50:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.714+0000 7fba07e29b80 1 -- 172.21.15.32:0/655528206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 msgr2=0x55d5c9dc0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.714+0000 7fba07e29b80 1 --2- 172.21.15.32:0/655528206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dc0c10 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0005210 tx=0x7fb9f0029f60 comp rx=0 tx=0).stop 2023-09-30T14:50:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.714+0000 7fba07e29b80 1 -- 172.21.15.32:0/655528206 shutdown_connections 2023-09-30T14:50:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.714+0000 7fba07e29b80 1 --2- 172.21.15.32:0/655528206 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d5c9e1c250 0x55d5c9e1e670 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.714+0000 7fba07e29b80 1 --2- 172.21.15.32:0/655528206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5c9db3cd0 0x55d5c9dc2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.714+0000 7fba07e29b80 1 --2- 172.21.15.32:0/655528206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d5c9db2200 0x55d5c9dc0c10 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.714+0000 7fb9dd7fa700 0 client.16518 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:50:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.714+0000 7fba07e29b80 1 --2- 172.21.15.32:0/655528206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d5c9db3310 0x55d5c9dc1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.714+0000 7fba07e29b80 1 -- 172.21.15.32:0/655528206 >> 172.21.15.32:0/655528206 conn(0x55d5c9d26200 msgr2=0x55d5c9dace70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.714+0000 7fba07e29b80 1 -- 172.21.15.32:0/655528206 shutdown_connections 2023-09-30T14:50:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.715+0000 7fba07e29b80 1 -- 172.21.15.32:0/655528206 wait complete. 2023-09-30T14:50:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.715+0000 7fba07e29b80 20 client.16518 trim_cache size 0 max 16384 2023-09-30T14:50:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:50:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:50:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:50:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:50:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.715+0000 7fb9fbc41700 1 -- 172.21.15.32:0/1583856564 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9e00a0fa0 msgr2=0x7fb9e00a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.715+0000 7fb9fbc41700 1 --2- 172.21.15.32:0/1583856564 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9e00a0fa0 0x7fb9e00a3460 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4198c30 tx=0x7fb9f0009260 comp rx=0 tx=0).stop 2023-09-30T14:50:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.715+0000 7fb9fbc41700 1 -- 172.21.15.32:0/1583856564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4100f40 msgr2=0x7fb9f4199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.715+0000 7fb9fbc41700 1 --2- 172.21.15.32:0/1583856564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4100f40 0x7fb9f4199420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009fd0 tx=0x7fb9e4009580 comp rx=0 tx=0).stop 2023-09-30T14:50:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.715+0000 7fb9fbc41700 1 -- 172.21.15.32:0/1583856564 shutdown_connections 2023-09-30T14:50:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.715+0000 7fb9fbc41700 1 --2- 172.21.15.32:0/1583856564 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb9e00a0fa0 0x7fb9e00a3460 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.715+0000 7fb9fbc41700 1 --2- 172.21.15.32:0/1583856564 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9f41018e0 0x7fb9f419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.715+0000 7fb9fbc41700 1 --2- 172.21.15.32:0/1583856564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9f40ffd50 0x7fb9f4198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.715+0000 7fb9fbc41700 1 --2- 172.21.15.32:0/1583856564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4100f40 0x7fb9f4199420 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.715+0000 7fb9fbc41700 1 -- 172.21.15.32:0/1583856564 >> 172.21.15.32:0/1583856564 conn(0x7fb9f40fb2a0 msgr2=0x7fb9f4069210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.715+0000 7fb9fbc41700 1 -- 172.21.15.32:0/1583856564 shutdown_connections 2023-09-30T14:50:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:23.715+0000 7fb9fbc41700 1 -- 172.21.15.32:0/1583856564 wait complete. 2023-09-30T14:50:23.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1328605 10.66 MB/sec execute 537 sec latency 23.846 ms 2023-09-30T14:50:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:23 smithi148 ceph-mon[102355]: pgmap v650: 97 pgs: 97 active+clean; 385 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.8 MiB/s wr, 275 op/s 2023-09-30T14:50:23.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:23 smithi032 ceph-mon[98592]: pgmap v650: 97 pgs: 97 active+clean; 385 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.8 MiB/s wr, 275 op/s 2023-09-30T14:50:24.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:23 smithi156 ceph-mon[103579]: pgmap v650: 97 pgs: 97 active+clean; 385 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.8 MiB/s wr, 275 op/s 2023-09-30T14:50:24.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1330954 10.66 MB/sec execute 538 sec latency 20.265 ms 2023-09-30T14:50:25.724 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1332450 10.65 MB/sec execute 539 sec latency 16.595 ms 2023-09-30T14:50:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:25 smithi148 ceph-mon[102355]: pgmap v651: 97 pgs: 97 active+clean; 385 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 2.6 MiB/s wr, 239 op/s 2023-09-30T14:50:25.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:25 smithi032 ceph-mon[98592]: pgmap v651: 97 pgs: 97 active+clean; 385 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 2.6 MiB/s wr, 239 op/s 2023-09-30T14:50:26.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:25 smithi156 ceph-mon[103579]: pgmap v651: 97 pgs: 97 active+clean; 385 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 2.6 MiB/s wr, 239 op/s 2023-09-30T14:50:26.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1335085 10.66 MB/sec execute 540 sec latency 11.783 ms 2023-09-30T14:50:27.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1336250 10.65 MB/sec execute 541 sec latency 46.782 ms 2023-09-30T14:50:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:27 smithi148 ceph-mon[102355]: pgmap v652: 97 pgs: 97 active+clean; 388 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 3.0 MiB/s wr, 315 op/s 2023-09-30T14:50:27.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:27 smithi032 ceph-mon[98592]: pgmap v652: 97 pgs: 97 active+clean; 388 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 3.0 MiB/s wr, 315 op/s 2023-09-30T14:50:28.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:27 smithi156 ceph-mon[103579]: pgmap v652: 97 pgs: 97 active+clean; 388 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 3.0 MiB/s wr, 315 op/s 2023-09-30T14:50:28.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1338955 10.66 MB/sec execute 542 sec latency 31.255 ms 2023-09-30T14:50:29.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1340800 10.65 MB/sec execute 543 sec latency 11.511 ms 2023-09-30T14:50:29.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:29 smithi148 ceph-mon[102355]: pgmap v653: 97 pgs: 97 active+clean; 389 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.6 MiB/s wr, 244 op/s 2023-09-30T14:50:29.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:29 smithi032 ceph-mon[98592]: pgmap v653: 97 pgs: 97 active+clean; 389 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.6 MiB/s wr, 244 op/s 2023-09-30T14:50:30.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:29 smithi156 ceph-mon[103579]: pgmap v653: 97 pgs: 97 active+clean; 389 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.6 MiB/s wr, 244 op/s 2023-09-30T14:50:30.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1342897 10.66 MB/sec execute 544 sec latency 9.918 ms 2023-09-30T14:50:31.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1345360 10.66 MB/sec execute 545 sec latency 13.286 ms 2023-09-30T14:50:31.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:31 smithi148 ceph-mon[102355]: pgmap v654: 97 pgs: 97 active+clean; 389 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 2.4 MiB/s wr, 248 op/s 2023-09-30T14:50:31.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:31 smithi032 ceph-mon[98592]: pgmap v654: 97 pgs: 97 active+clean; 389 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 2.4 MiB/s wr, 248 op/s 2023-09-30T14:50:32.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:31 smithi156 ceph-mon[103579]: pgmap v654: 97 pgs: 97 active+clean; 389 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 2.4 MiB/s wr, 248 op/s 2023-09-30T14:50:32.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1346889 10.66 MB/sec execute 546 sec latency 60.435 ms 2023-09-30T14:50:33.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1349119 10.66 MB/sec execute 547 sec latency 74.968 ms 2023-09-30T14:50:33.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:33 smithi148 ceph-mon[102355]: pgmap v655: 97 pgs: 97 active+clean; 391 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 2.7 MiB/s wr, 294 op/s 2023-09-30T14:50:33.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:33 smithi032 ceph-mon[98592]: pgmap v655: 97 pgs: 97 active+clean; 391 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 2.7 MiB/s wr, 294 op/s 2023-09-30T14:50:34.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:33 smithi156 ceph-mon[103579]: pgmap v655: 97 pgs: 97 active+clean; 391 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 2.7 MiB/s wr, 294 op/s 2023-09-30T14:50:34.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1350522 10.65 MB/sec execute 548 sec latency 18.788 ms 2023-09-30T14:50:34.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:34 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:50:34.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:34 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:50:34.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:34 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:50:34.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:34 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:50:34.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:34 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:50:34.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:34 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:50:34.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:34 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:50:34.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:34 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:50:35.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:34 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:50:35.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:34 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:50:35.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:34 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:50:35.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:34 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:50:35.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1353119 10.66 MB/sec execute 549 sec latency 34.296 ms 2023-09-30T14:50:35.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:35 smithi148 ceph-mon[102355]: pgmap v656: 97 pgs: 97 active+clean; 392 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 2.5 MiB/s wr, 249 op/s 2023-09-30T14:50:35.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:35 smithi032 ceph-mon[98592]: pgmap v656: 97 pgs: 97 active+clean; 392 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 2.5 MiB/s wr, 249 op/s 2023-09-30T14:50:36.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:35 smithi156 ceph-mon[103579]: pgmap v656: 97 pgs: 97 active+clean; 392 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 2.5 MiB/s wr, 249 op/s 2023-09-30T14:50:36.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1354705 10.66 MB/sec execute 550 sec latency 14.967 ms 2023-09-30T14:50:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.831+0000 7f92155c4700 1 -- 172.21.15.32:0/1492963934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9210101310 msgr2=0x7f9210101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.831+0000 7f92155c4700 1 --2- 172.21.15.32:0/1492963934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9210101310 0x7f9210101730 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9200009990 tx=0x7f9200009ca0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.831+0000 7f92155c4700 1 -- 172.21.15.32:0/1492963934 shutdown_connections 2023-09-30T14:50:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.831+0000 7f92155c4700 1 --2- 172.21.15.32:0/1492963934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9210100920 0x7f92101081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.831+0000 7f92155c4700 1 --2- 172.21.15.32:0/1492963934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9210101310 0x7f9210101730 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.831+0000 7f92155c4700 1 --2- 172.21.15.32:0/1492963934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92100fff60 0x7f92101003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.831+0000 7f92155c4700 1 -- 172.21.15.32:0/1492963934 >> 172.21.15.32:0/1492963934 conn(0x7f92100faa30 msgr2=0x7f92100fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.831+0000 7f92155c4700 1 -- 172.21.15.32:0/1492963934 shutdown_connections 2023-09-30T14:50:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.831+0000 7f92155c4700 1 -- 172.21.15.32:0/1492963934 wait complete. 2023-09-30T14:50:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.832+0000 7f92155c4700 1 Processor -- start 2023-09-30T14:50:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.832+0000 7f92155c4700 1 -- start start 2023-09-30T14:50:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.832+0000 7f92155c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92100fff60 0x7f9210196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.832+0000 7f92155c4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9210100920 0x7f9210197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.832+0000 7f92155c4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9210101310 0x7f9210197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.832+0000 7f92155c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9210198670 con 0x7f92100fff60 2023-09-30T14:50:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.832+0000 7f92155c4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92101987e0 con 0x7f9210101310 2023-09-30T14:50:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.832+0000 7f92155c4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92101989a0 con 0x7f9210100920 2023-09-30T14:50:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.833+0000 7f920effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92100fff60 0x7f9210196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.833+0000 7f920effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92100fff60 0x7f9210196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38142/0 (socket says 172.21.15.32:38142) 2023-09-30T14:50:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.833+0000 7f920effd700 1 -- 172.21.15.32:0/3859371318 learned_addr learned my addr 172.21.15.32:0/3859371318 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.833+0000 7f920f7fe700 1 --2- 172.21.15.32:0/3859371318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9210101310 0x7f9210197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.833+0000 7f92067fc700 1 --2- 172.21.15.32:0/3859371318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9210100920 0x7f9210197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.833+0000 7f920effd700 1 -- 172.21.15.32:0/3859371318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9210100920 msgr2=0x7f9210197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.833+0000 7f920effd700 1 --2- 172.21.15.32:0/3859371318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9210100920 0x7f9210197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.833+0000 7f920effd700 1 -- 172.21.15.32:0/3859371318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9210101310 msgr2=0x7f9210197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.833+0000 7f920effd700 1 --2- 172.21.15.32:0/3859371318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9210101310 0x7f9210197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.833+0000 7f920effd700 1 -- 172.21.15.32:0/3859371318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92000095f0 con 0x7f92100fff60 2023-09-30T14:50:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.833+0000 7f920effd700 1 --2- 172.21.15.32:0/3859371318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92100fff60 0x7f9210196cf0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9200004760 tx=0x7f9200004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.833+0000 7f920cff9700 1 -- 172.21.15.32:0/3859371318 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920001d070 con 0x7f92100fff60 2023-09-30T14:50:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.833+0000 7f92155c4700 1 -- 172.21.15.32:0/3859371318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9210198c20 con 0x7f92100fff60 2023-09-30T14:50:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.833+0000 7f920cff9700 1 -- 172.21.15.32:0/3859371318 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9200004830 con 0x7f92100fff60 2023-09-30T14:50:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.833+0000 7f920cff9700 1 -- 172.21.15.32:0/3859371318 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200005350 con 0x7f92100fff60 2023-09-30T14:50:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.833+0000 7f92155c4700 1 -- 172.21.15.32:0/3859371318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921019f3a0 con 0x7f92100fff60 2023-09-30T14:50:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.836+0000 7f920cff9700 1 -- 172.21.15.32:0/3859371318 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f9200005590 con 0x7f92100fff60 2023-09-30T14:50:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.836+0000 7f920cff9700 1 --2- 172.21.15.32:0/3859371318 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91fc0a0fa0 0x7f91fc0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.836+0000 7f920cff9700 1 -- 172.21.15.32:0/3859371318 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f92000cd900 con 0x7f92100fff60 2023-09-30T14:50:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.836+0000 7f92067fc700 1 --2- 172.21.15.32:0/3859371318 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91fc0a0fa0 0x7f91fc0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.837+0000 7f92067fc700 1 --2- 172.21.15.32:0/3859371318 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91fc0a0fa0 0x7f91fc0a3460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009c60 tx=0x7f91f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:36.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.837+0000 7f92217acb80 1 Processor -- start 2023-09-30T14:50:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.837+0000 7f92217acb80 1 -- start start 2023-09-30T14:50:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.837+0000 7f92217acb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc675200 0x5616dc675620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.837+0000 7f92217acb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc676310 0x5616dc676790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.837+0000 7f92217acb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 0x5616dc67d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.837+0000 7f92217acb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616dc5dd400 con 0x5616dc676cd0 2023-09-30T14:50:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.837+0000 7f92217acb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616dc5cebc0 con 0x5616dc675200 2023-09-30T14:50:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.837+0000 7f92217acb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616dc5cc4e0 con 0x5616dc676310 2023-09-30T14:50:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.837+0000 7f920f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 0x5616dc67d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.837+0000 7f920effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc675200 0x5616dc675620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.838+0000 7f920effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc675200 0x5616dc675620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49626/0 (socket says 172.21.15.32:49626) 2023-09-30T14:50:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.838+0000 7f92067fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc676310 0x5616dc676790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.838+0000 7f920effd700 1 -- 172.21.15.32:0/1883302265 learned_addr learned my addr 172.21.15.32:0/1883302265 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.838+0000 7f920effd700 1 -- 172.21.15.32:0/1883302265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc676310 msgr2=0x5616dc676790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.838+0000 7f920effd700 1 --2- 172.21.15.32:0/1883302265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc676310 0x5616dc676790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.838+0000 7f920effd700 1 -- 172.21.15.32:0/1883302265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 msgr2=0x5616dc67d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.838+0000 7f920effd700 1 --2- 172.21.15.32:0/1883302265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 0x5616dc67d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.838+0000 7f920effd700 1 -- 172.21.15.32:0/1883302265 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616dc62a870 con 0x5616dc675200 2023-09-30T14:50:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.838+0000 7f920effd700 1 --2- 172.21.15.32:0/1883302265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc675200 0x5616dc675620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9200097360 tx=0x7f920009b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14c8a878b03a4a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.838+0000 7f92057fa700 1 -- 172.21.15.32:0/1883302265 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92000ba030 con 0x5616dc675200 2023-09-30T14:50:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.838+0000 7f92057fa700 1 -- 172.21.15.32:0/1883302265 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f920009efb0 con 0x5616dc675200 2023-09-30T14:50:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.838+0000 7f92057fa700 1 -- 172.21.15.32:0/1883302265 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92000a2cd0 con 0x5616dc675200 2023-09-30T14:50:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.839+0000 7f92217acb80 1 -- 172.21.15.32:0/1883302265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc675200 msgr2=0x5616dc675620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.839+0000 7f92217acb80 1 --2- 172.21.15.32:0/1883302265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc675200 0x5616dc675620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9200097360 tx=0x7f920009b560 comp rx=0 tx=0).stop 2023-09-30T14:50:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.839+0000 7f92217acb80 1 -- 172.21.15.32:0/1883302265 shutdown_connections 2023-09-30T14:50:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.839+0000 7f92217acb80 1 --2- 172.21.15.32:0/1883302265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 0x5616dc67d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.839+0000 7f92217acb80 1 --2- 172.21.15.32:0/1883302265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc675200 0x5616dc675620 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.839+0000 7f92217acb80 1 --2- 172.21.15.32:0/1883302265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc676310 0x5616dc676790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.839+0000 7f92217acb80 1 -- 172.21.15.32:0/1883302265 >> 172.21.15.32:0/1883302265 conn(0x5616dc5e9200 msgr2=0x5616dc66fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.839+0000 7f92217acb80 1 -- 172.21.15.32:0/1883302265 shutdown_connections 2023-09-30T14:50:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.839+0000 7f92217acb80 1 -- 172.21.15.32:0/1883302265 wait complete. 2023-09-30T14:50:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.839+0000 7f92217acb80 1 -- start start 2023-09-30T14:50:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.839+0000 7f92217acb80 1 Processor -- start 2023-09-30T14:50:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f92217acb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc675200 0x5616dc67eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f92217acb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc676310 0x5616dc67f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f92217acb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc676cd0 0x5616dc686100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f92217acb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616dc5c7dd0 con 0x5616dc675200 2023-09-30T14:50:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f92217acb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616dc5646b0 con 0x5616dc676310 2023-09-30T14:50:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f92217acb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616dc564350 con 0x5616dc676cd0 2023-09-30T14:50:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f92217acb80 10 client.0 _ll_register_callbacks cb 0x5616dc3c17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:50:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f920effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc675200 0x5616dc67eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f920effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc675200 0x5616dc67eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38156/0 (socket says 172.21.15.32:38156) 2023-09-30T14:50:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f920effd700 1 -- 172.21.15.32:0/2127146441 learned_addr learned my addr 172.21.15.32:0/2127146441 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f92067fc700 1 --2- 172.21.15.32:0/2127146441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc676310 0x5616dc67f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f920f7fe700 1 --2- 172.21.15.32:0/2127146441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc676cd0 0x5616dc686100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f920effd700 1 -- 172.21.15.32:0/2127146441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc676cd0 msgr2=0x5616dc686100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f920effd700 1 --2- 172.21.15.32:0/2127146441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc676cd0 0x5616dc686100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f920effd700 1 -- 172.21.15.32:0/2127146441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc676310 msgr2=0x5616dc67f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f920effd700 1 --2- 172.21.15.32:0/2127146441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc676310 0x5616dc67f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f920effd700 1 -- 172.21.15.32:0/2127146441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921019f3a0 con 0x5616dc675200 2023-09-30T14:50:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f920effd700 1 --2- 172.21.15.32:0/2127146441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc675200 0x5616dc67eb80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9200004730 tx=0x7f9200009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113cda5c1f5485ed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f91f67fc700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:50:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f91f67fc700 1 -- 172.21.15.32:0/2127146441 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92000ba070 con 0x5616dc675200 2023-09-30T14:50:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f91f67fc700 1 -- 172.21.15.32:0/2127146441 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f920009d070 con 0x5616dc675200 2023-09-30T14:50:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.840+0000 7f92217acb80 1 -- 172.21.15.32:0/2127146441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616dc668090 con 0x5616dc675200 2023-09-30T14:50:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.841+0000 7f91f67fc700 1 -- 172.21.15.32:0/2127146441 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f920009a5d0 con 0x5616dc675200 2023-09-30T14:50:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.841+0000 7f92217acb80 10 client.16530 fetch_fsmap learned FSMap version 44 2023-09-30T14:50:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.841+0000 7f92217acb80 1 -- 172.21.15.32:0/2127146441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5616dc3f2d80 con 0x5616dc675200 2023-09-30T14:50:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.841+0000 7f91f67fc700 1 -- 172.21.15.32:0/2127146441 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9200064080 con 0x5616dc675200 2023-09-30T14:50:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.841+0000 7f91f67fc700 1 -- 172.21.15.32:0/2127146441 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9200099dc0 con 0x5616dc675200 2023-09-30T14:50:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.841+0000 7f92217acb80 10 client.16530 fetch_fsmap finished waiting for FSMap version 44 2023-09-30T14:50:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.841+0000 7f92217acb80 10 client.16530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:50:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.841+0000 7f92217acb80 20 client.16530 populate_metadata read hostname 'smithi032' 2023-09-30T14:50:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.841+0000 7f92217acb80 1 --2- 172.21.15.32:0/2127146441 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5616dc68a4a0 0x5616dc68c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.841+0000 7f92217acb80 4 client.16530 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:50:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.841+0000 7f92217acb80 1 -- 172.21.15.32:0/2127146441 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616dc5c0db0 con 0x5616dc68a4a0 2023-09-30T14:50:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.841+0000 7f92067fc700 1 --2- 172.21.15.32:0/2127146441 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5616dc68a4a0 0x5616dc68c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.842+0000 7f91f67fc700 10 client.16530 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:50:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.842+0000 7f92067fc700 1 --2- 172.21.15.32:0/2127146441 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5616dc68a4a0 0x5616dc68c890 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b91d5e9f89ee129 server_cookie=e21c6cb567dbc563 in_seq=0 out_seq=0 2023-09-30T14:50:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.843+0000 7f91f67fc700 1 -- 172.21.15.32:0/2127146441 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5616dc5c0db0 con 0x5616dc68a4a0 2023-09-30T14:50:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.843+0000 7f91f67fc700 10 client.16530 handle_command_reply: tid=0 2023-09-30T14:50:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.843+0000 7f92217acb80 1 client.16530 shutdown 2023-09-30T14:50:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.843+0000 7f92217acb80 1 -- 172.21.15.32:0/2127146441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc675200 msgr2=0x5616dc67eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.843+0000 7f92217acb80 1 --2- 172.21.15.32:0/2127146441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc675200 0x5616dc67eb80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9200004730 tx=0x7f9200009310 comp rx=0 tx=0).stop 2023-09-30T14:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.843+0000 7f92217acb80 1 -- 172.21.15.32:0/2127146441 shutdown_connections 2023-09-30T14:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.843+0000 7f92217acb80 1 --2- 172.21.15.32:0/2127146441 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5616dc68a4a0 0x5616dc68c890 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.843+0000 7f92217acb80 1 --2- 172.21.15.32:0/2127146441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc676cd0 0x5616dc686100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.843+0000 7f92217acb80 1 --2- 172.21.15.32:0/2127146441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc675200 0x5616dc67eb80 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.843+0000 7f92217acb80 1 --2- 172.21.15.32:0/2127146441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc676310 0x5616dc67f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.843+0000 7f91f67fc700 0 client.16530 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.843+0000 7f92217acb80 1 -- 172.21.15.32:0/2127146441 >> 172.21.15.32:0/2127146441 conn(0x5616dc5e9200 msgr2=0x5616dc66f590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.843+0000 7f92217acb80 1 -- 172.21.15.32:0/2127146441 shutdown_connections 2023-09-30T14:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.843+0000 7f92217acb80 1 -- 172.21.15.32:0/2127146441 wait complete. 2023-09-30T14:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.843+0000 7f92217acb80 20 client.16530 trim_cache size 0 max 16384 2023-09-30T14:50:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.859+0000 7f92217acb80 1 Processor -- start 2023-09-30T14:50:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.859+0000 7f92217acb80 1 -- start start 2023-09-30T14:50:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f92217acb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc675200 0x5616dc66f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f92217acb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc676310 0x5616dc6860c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f92217acb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 0x5616dc66fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f92217acb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616dc4488c0 con 0x5616dc676cd0 2023-09-30T14:50:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f92217acb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616dc675d00 con 0x5616dc675200 2023-09-30T14:50:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f92217acb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616dc66cba0 con 0x5616dc676310 2023-09-30T14:50:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f920f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 0x5616dc66fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f920f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 0x5616dc66fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38168/0 (socket says 172.21.15.32:38168) 2023-09-30T14:50:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f920f7fe700 1 -- 172.21.15.32:0/2999202718 learned_addr learned my addr 172.21.15.32:0/2999202718 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f920effd700 1 --2- 172.21.15.32:0/2999202718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc675200 0x5616dc66f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f92067fc700 1 --2- 172.21.15.32:0/2999202718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc676310 0x5616dc6860c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f920f7fe700 1 -- 172.21.15.32:0/2999202718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc676310 msgr2=0x5616dc6860c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f920f7fe700 1 --2- 172.21.15.32:0/2999202718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc676310 0x5616dc6860c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f920f7fe700 1 -- 172.21.15.32:0/2999202718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc675200 msgr2=0x5616dc66f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f920f7fe700 1 --2- 172.21.15.32:0/2999202718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc675200 0x5616dc66f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f920f7fe700 1 -- 172.21.15.32:0/2999202718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616dc3dbc00 con 0x5616dc676cd0 2023-09-30T14:50:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f920f7fe700 1 --2- 172.21.15.32:0/2999202718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 0x5616dc66fec0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9208015790 tx=0x7f9208009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8569fb21ce2040 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f9205ffb700 1 -- 172.21.15.32:0/2999202718 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920800e040 con 0x5616dc676cd0 2023-09-30T14:50:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f9205ffb700 1 -- 172.21.15.32:0/2999202718 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f920800f040 con 0x5616dc676cd0 2023-09-30T14:50:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.860+0000 7f9205ffb700 1 -- 172.21.15.32:0/2999202718 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208007500 con 0x5616dc676cd0 2023-09-30T14:50:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.861+0000 7f92217acb80 1 -- 172.21.15.32:0/2999202718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 msgr2=0x5616dc66fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.861+0000 7f92217acb80 1 --2- 172.21.15.32:0/2999202718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 0x5616dc66fec0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9208015790 tx=0x7f9208009920 comp rx=0 tx=0).stop 2023-09-30T14:50:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.861+0000 7f92217acb80 1 -- 172.21.15.32:0/2999202718 shutdown_connections 2023-09-30T14:50:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.861+0000 7f92217acb80 1 --2- 172.21.15.32:0/2999202718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 0x5616dc66fec0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.861+0000 7f92217acb80 1 --2- 172.21.15.32:0/2999202718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc675200 0x5616dc66f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.861+0000 7f92217acb80 1 --2- 172.21.15.32:0/2999202718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc676310 0x5616dc6860c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.861+0000 7f92217acb80 1 -- 172.21.15.32:0/2999202718 >> 172.21.15.32:0/2999202718 conn(0x5616dc5e9200 msgr2=0x5616dc6911f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.861+0000 7f92217acb80 1 -- 172.21.15.32:0/2999202718 shutdown_connections 2023-09-30T14:50:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.861+0000 7f92217acb80 1 -- 172.21.15.32:0/2999202718 wait complete. 2023-09-30T14:50:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.861+0000 7f92217acb80 1 -- start start 2023-09-30T14:50:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f92217acb80 1 Processor -- start 2023-09-30T14:50:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f92217acb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc675200 0x5616dc683fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f92217acb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc676310 0x5616dc6844e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f92217acb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 0x5616dc6854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f92217acb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616dc685a30 con 0x5616dc676cd0 2023-09-30T14:50:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f92217acb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616dc685ba0 con 0x5616dc676310 2023-09-30T14:50:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f92217acb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616dc685d60 con 0x5616dc675200 2023-09-30T14:50:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f92217acb80 10 client.0 _ll_register_callbacks cb 0x7f92101978b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:50:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f920f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 0x5616dc6854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f92067fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc676310 0x5616dc6844e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f920effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc675200 0x5616dc683fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f92067fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc676310 0x5616dc6844e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49646/0 (socket says 172.21.15.32:49646) 2023-09-30T14:50:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f920f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 0x5616dc6854f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38174/0 (socket says 172.21.15.32:38174) 2023-09-30T14:50:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f92067fc700 1 -- 172.21.15.32:0/2253113836 learned_addr learned my addr 172.21.15.32:0/2253113836 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f92067fc700 1 -- 172.21.15.32:0/2253113836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc675200 msgr2=0x5616dc683fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f92067fc700 1 --2- 172.21.15.32:0/2253113836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc675200 0x5616dc683fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f92067fc700 1 -- 172.21.15.32:0/2253113836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 msgr2=0x5616dc6854f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f92067fc700 1 --2- 172.21.15.32:0/2253113836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 0x5616dc6854f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f92067fc700 1 -- 172.21.15.32:0/2253113836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9208015440 con 0x5616dc676310 2023-09-30T14:50:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.862+0000 7f920f7fe700 1 --2- 172.21.15.32:0/2253113836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 0x5616dc6854f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:50:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.863+0000 7f92067fc700 1 --2- 172.21.15.32:0/2253113836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc676310 0x5616dc6844e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f91f8013010 tx=0x7f91f80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29e480b989ecd064 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.863+0000 7f91f67fc700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:50:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.863+0000 7f91f67fc700 1 -- 172.21.15.32:0/2253113836 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8022040 con 0x5616dc676310 2023-09-30T14:50:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.863+0000 7f92217acb80 1 -- 172.21.15.32:0/2253113836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f920009efb0 con 0x5616dc676310 2023-09-30T14:50:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.863+0000 7f91f67fc700 1 -- 172.21.15.32:0/2253113836 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f91f8003a60 con 0x5616dc676310 2023-09-30T14:50:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.863+0000 7f91f67fc700 1 -- 172.21.15.32:0/2253113836 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91f803a030 con 0x5616dc676310 2023-09-30T14:50:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.863+0000 7f92217acb80 10 client.26222 fetch_fsmap learned FSMap version 44 2023-09-30T14:50:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.863+0000 7f92217acb80 1 -- 172.21.15.32:0/2253113836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5616dc4191f0 con 0x5616dc676310 2023-09-30T14:50:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.863+0000 7f91f67fc700 1 -- 172.21.15.32:0/2253113836 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f920009efb0 con 0x5616dc676310 2023-09-30T14:50:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.863+0000 7f91f67fc700 1 -- 172.21.15.32:0/2253113836 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f91f80346d0 con 0x5616dc676310 2023-09-30T14:50:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.863+0000 7f92217acb80 10 client.26222 fetch_fsmap finished waiting for FSMap version 44 2023-09-30T14:50:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.863+0000 7f92217acb80 10 client.26222 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:50:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.863+0000 7f92217acb80 20 client.26222 populate_metadata read hostname 'smithi032' 2023-09-30T14:50:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.863+0000 7f92217acb80 1 --2- 172.21.15.32:0/2253113836 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5616dc6df460 0x5616dc6e1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.863+0000 7f92217acb80 4 client.26222 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:50:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.863+0000 7f92217acb80 1 -- 172.21.15.32:0/2253113836 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5616dc5c0ad0 con 0x5616dc6df460 2023-09-30T14:50:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.864+0000 7f920effd700 1 --2- 172.21.15.32:0/2253113836 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5616dc6df460 0x5616dc6e1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.864+0000 7f91f67fc700 10 client.26222 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:50:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:36.864+0000 7f920effd700 1 --2- 172.21.15.32:0/2253113836 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5616dc6df460 0x5616dc6e1880 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3874448dd0e8191 server_cookie=9b06e0cc55080b2d in_seq=0 out_seq=0 2023-09-30T14:50:37.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1356915 10.66 MB/sec execute 551 sec latency 54.358 ms 2023-09-30T14:50:37.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:37 smithi148 ceph-mon[102355]: pgmap v657: 97 pgs: 97 active+clean; 394 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 2.5 MiB/s wr, 292 op/s 2023-09-30T14:50:37.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:37 smithi032 ceph-mon[98592]: pgmap v657: 97 pgs: 97 active+clean; 394 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 2.5 MiB/s wr, 292 op/s 2023-09-30T14:50:38.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:37 smithi156 ceph-mon[103579]: pgmap v657: 97 pgs: 97 active+clean; 394 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 2.5 MiB/s wr, 292 op/s 2023-09-30T14:50:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.079+0000 7f91f67fc700 1 -- 172.21.15.32:0/2253113836 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5616dc5c0ad0 con 0x5616dc6df460 2023-09-30T14:50:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.079+0000 7f91f67fc700 10 client.26222 handle_command_reply: tid=0 2023-09-30T14:50:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.079+0000 7f92217acb80 1 client.26222 shutdown 2023-09-30T14:50:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.080+0000 7f92217acb80 1 -- 172.21.15.32:0/2253113836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc676310 msgr2=0x5616dc6844e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.080+0000 7f92217acb80 1 --2- 172.21.15.32:0/2253113836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc676310 0x5616dc6844e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f91f8013010 tx=0x7f91f80047c0 comp rx=0 tx=0).stop 2023-09-30T14:50:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.080+0000 7f92217acb80 1 -- 172.21.15.32:0/2253113836 shutdown_connections 2023-09-30T14:50:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.080+0000 7f92217acb80 1 --2- 172.21.15.32:0/2253113836 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5616dc6df460 0x5616dc6e1880 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.080+0000 7f92217acb80 1 --2- 172.21.15.32:0/2253113836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5616dc676cd0 0x5616dc6854f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.080+0000 7f91f67fc700 0 client.26222 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:50:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.080+0000 7f92217acb80 1 --2- 172.21.15.32:0/2253113836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616dc675200 0x5616dc683fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.080+0000 7f92217acb80 1 --2- 172.21.15.32:0/2253113836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616dc676310 0x5616dc6844e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.080+0000 7f92217acb80 1 -- 172.21.15.32:0/2253113836 >> 172.21.15.32:0/2253113836 conn(0x5616dc5e9200 msgr2=0x5616dc66f590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.080+0000 7f92217acb80 1 -- 172.21.15.32:0/2253113836 shutdown_connections 2023-09-30T14:50:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.080+0000 7f92217acb80 1 -- 172.21.15.32:0/2253113836 wait complete. 2023-09-30T14:50:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.080+0000 7f92217acb80 20 client.26222 trim_cache size 0 max 16384 2023-09-30T14:50:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:50:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:50:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:50:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:50:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.081+0000 7f92155c4700 1 -- 172.21.15.32:0/3859371318 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91fc0a0fa0 msgr2=0x7f91fc0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.081+0000 7f92155c4700 1 --2- 172.21.15.32:0/3859371318 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91fc0a0fa0 0x7f91fc0a3460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009c60 tx=0x7f91f8009260 comp rx=0 tx=0).stop 2023-09-30T14:50:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.081+0000 7f92155c4700 1 -- 172.21.15.32:0/3859371318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92100fff60 msgr2=0x7f9210196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.081+0000 7f92155c4700 1 --2- 172.21.15.32:0/3859371318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92100fff60 0x7f9210196cf0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9200004760 tx=0x7f9200004790 comp rx=0 tx=0).stop 2023-09-30T14:50:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.081+0000 7f92155c4700 1 -- 172.21.15.32:0/3859371318 shutdown_connections 2023-09-30T14:50:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.081+0000 7f92155c4700 1 --2- 172.21.15.32:0/3859371318 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f91fc0a0fa0 0x7f91fc0a3460 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.081+0000 7f92155c4700 1 --2- 172.21.15.32:0/3859371318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9210101310 0x7f9210197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.081+0000 7f92155c4700 1 --2- 172.21.15.32:0/3859371318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92100fff60 0x7f9210196cf0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.081+0000 7f92155c4700 1 --2- 172.21.15.32:0/3859371318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9210100920 0x7f9210197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.081+0000 7f92155c4700 1 -- 172.21.15.32:0/3859371318 >> 172.21.15.32:0/3859371318 conn(0x7f92100faa30 msgr2=0x7f92100feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.081+0000 7f92155c4700 1 -- 172.21.15.32:0/3859371318 shutdown_connections 2023-09-30T14:50:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:38.081+0000 7f92155c4700 1 -- 172.21.15.32:0/3859371318 wait complete. 2023-09-30T14:50:38.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1358522 10.65 MB/sec execute 552 sec latency 50.491 ms 2023-09-30T14:50:39.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1360467 10.66 MB/sec execute 553 sec latency 61.902 ms 2023-09-30T14:50:39.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:39 smithi148 ceph-mon[102355]: pgmap v658: 97 pgs: 97 active+clean; 396 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.6 MiB/s wr, 264 op/s 2023-09-30T14:50:39.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:39 smithi032 ceph-mon[98592]: pgmap v658: 97 pgs: 97 active+clean; 396 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.6 MiB/s wr, 264 op/s 2023-09-30T14:50:40.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:39 smithi156 ceph-mon[103579]: pgmap v658: 97 pgs: 97 active+clean; 396 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.6 MiB/s wr, 264 op/s 2023-09-30T14:50:40.725 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1362591 10.65 MB/sec execute 554 sec latency 16.169 ms 2023-09-30T14:50:41.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1364254 10.65 MB/sec execute 555 sec latency 10.754 ms 2023-09-30T14:50:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:41 smithi148 ceph-mon[102355]: pgmap v659: 97 pgs: 97 active+clean; 392 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 2.6 MiB/s wr, 260 op/s 2023-09-30T14:50:41.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:41 smithi032 ceph-mon[98592]: pgmap v659: 97 pgs: 97 active+clean; 392 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 2.6 MiB/s wr, 260 op/s 2023-09-30T14:50:42.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:41 smithi156 ceph-mon[103579]: pgmap v659: 97 pgs: 97 active+clean; 392 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 2.6 MiB/s wr, 260 op/s 2023-09-30T14:50:42.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1366808 10.66 MB/sec execute 556 sec latency 15.408 ms 2023-09-30T14:50:43.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1368086 10.65 MB/sec execute 557 sec latency 26.865 ms 2023-09-30T14:50:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:43 smithi148 ceph-mon[102355]: pgmap v660: 97 pgs: 97 active+clean; 392 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 2.8 MiB/s wr, 272 op/s 2023-09-30T14:50:43.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:43 smithi032 ceph-mon[98592]: pgmap v660: 97 pgs: 97 active+clean; 392 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 2.8 MiB/s wr, 272 op/s 2023-09-30T14:50:44.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:43 smithi156 ceph-mon[103579]: pgmap v660: 97 pgs: 97 active+clean; 392 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 2.8 MiB/s wr, 272 op/s 2023-09-30T14:50:44.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1370723 10.66 MB/sec execute 558 sec latency 23.920 ms 2023-09-30T14:50:45.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1372218 10.66 MB/sec execute 559 sec latency 15.819 ms 2023-09-30T14:50:45.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:45 smithi148 ceph-mon[102355]: pgmap v661: 97 pgs: 97 active+clean; 395 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.7 MiB/s wr, 287 op/s 2023-09-30T14:50:45.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:45 smithi032 ceph-mon[98592]: pgmap v661: 97 pgs: 97 active+clean; 395 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.7 MiB/s wr, 287 op/s 2023-09-30T14:50:46.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:45 smithi156 ceph-mon[103579]: pgmap v661: 97 pgs: 97 active+clean; 395 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 2.7 MiB/s wr, 287 op/s 2023-09-30T14:50:46.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1374570 10.66 MB/sec execute 560 sec latency 12.607 ms 2023-09-30T14:50:47.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1375574 10.68 MB/sec execute 561 sec latency 55.929 ms 2023-09-30T14:50:47.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:47 smithi148 ceph-mon[102355]: pgmap v662: 97 pgs: 97 active+clean; 396 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 2.8 MiB/s wr, 290 op/s 2023-09-30T14:50:47.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:47 smithi032 ceph-mon[98592]: pgmap v662: 97 pgs: 97 active+clean; 396 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 2.8 MiB/s wr, 290 op/s 2023-09-30T14:50:48.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:47 smithi156 ceph-mon[103579]: pgmap v662: 97 pgs: 97 active+clean; 396 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 2.8 MiB/s wr, 290 op/s 2023-09-30T14:50:48.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1377793 10.68 MB/sec execute 562 sec latency 87.916 ms 2023-09-30T14:50:49.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1379258 10.68 MB/sec execute 563 sec latency 27.297 ms 2023-09-30T14:50:49.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:49 smithi148 ceph-mon[102355]: pgmap v663: 97 pgs: 97 active+clean; 398 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 3.0 MiB/s wr, 286 op/s 2023-09-30T14:50:49.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:49 smithi032 ceph-mon[98592]: pgmap v663: 97 pgs: 97 active+clean; 398 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 3.0 MiB/s wr, 286 op/s 2023-09-30T14:50:50.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:49 smithi156 ceph-mon[103579]: pgmap v663: 97 pgs: 97 active+clean; 398 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 3.0 MiB/s wr, 286 op/s 2023-09-30T14:50:50.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1381587 10.68 MB/sec execute 564 sec latency 36.596 ms 2023-09-30T14:50:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.196+0000 7ff79eaa1700 1 -- 172.21.15.32:0/3242686357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff798100f50 msgr2=0x7ff7981013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:51.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.196+0000 7ff79eaa1700 1 --2- 172.21.15.32:0/3242686357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff798100f50 0x7ff7981013b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff7840099c0 tx=0x7ff784009cd0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.196+0000 7ff79eaa1700 1 -- 172.21.15.32:0/3242686357 shutdown_connections 2023-09-30T14:50:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.196+0000 7ff79eaa1700 1 --2- 172.21.15.32:0/3242686357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7981018f0 0x7ff7981081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.196+0000 7ff79eaa1700 1 --2- 172.21.15.32:0/3242686357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7980ffd60 0x7ff798100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.196+0000 7ff79eaa1700 1 --2- 172.21.15.32:0/3242686357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff798100f50 0x7ff7981013b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.196+0000 7ff79eaa1700 1 -- 172.21.15.32:0/3242686357 >> 172.21.15.32:0/3242686357 conn(0x7ff7980fb2b0 msgr2=0x7ff7980fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.196+0000 7ff79eaa1700 1 -- 172.21.15.32:0/3242686357 shutdown_connections 2023-09-30T14:50:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.196+0000 7ff79eaa1700 1 -- 172.21.15.32:0/3242686357 wait complete. 2023-09-30T14:50:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.197+0000 7ff79eaa1700 1 Processor -- start 2023-09-30T14:50:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.197+0000 7ff79eaa1700 1 -- start start 2023-09-30T14:50:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.197+0000 7ff79eaa1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7980ffd60 0x7ff798102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.197+0000 7ff79eaa1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff798100f50 0x7ff798102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.197+0000 7ff79eaa1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7981018f0 0x7ff798102db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.197+0000 7ff79eaa1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7981a0bd0 con 0x7ff798100f50 2023-09-30T14:50:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.197+0000 7ff79eaa1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7981a0d10 con 0x7ff7981018f0 2023-09-30T14:50:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.197+0000 7ff79eaa1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7981a0e50 con 0x7ff7980ffd60 2023-09-30T14:50:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.197+0000 7ff797fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff798100f50 0x7ff798102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.197+0000 7ff797fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff798100f50 0x7ff798102870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38186/0 (socket says 172.21.15.32:38186) 2023-09-30T14:50:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.197+0000 7ff797fff700 1 -- 172.21.15.32:0/529923805 learned_addr learned my addr 172.21.15.32:0/529923805 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.197+0000 7ff79d03e700 1 --2- 172.21.15.32:0/529923805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7981018f0 0x7ff798102db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.198+0000 7ff79c83d700 1 --2- 172.21.15.32:0/529923805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7980ffd60 0x7ff798102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.198+0000 7ff797fff700 1 -- 172.21.15.32:0/529923805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7980ffd60 msgr2=0x7ff798102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.198+0000 7ff797fff700 1 --2- 172.21.15.32:0/529923805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7980ffd60 0x7ff798102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.198+0000 7ff797fff700 1 -- 172.21.15.32:0/529923805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7981018f0 msgr2=0x7ff798102db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.198+0000 7ff797fff700 1 --2- 172.21.15.32:0/529923805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7981018f0 0x7ff798102db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.198+0000 7ff797fff700 1 -- 172.21.15.32:0/529923805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7840095f0 con 0x7ff798100f50 2023-09-30T14:50:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.198+0000 7ff79c83d700 1 --2- 172.21.15.32:0/529923805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7980ffd60 0x7ff798102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:50:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.198+0000 7ff797fff700 1 --2- 172.21.15.32:0/529923805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff798100f50 0x7ff798102870 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff784009990 tx=0x7ff78400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.198+0000 7ff795ffb700 1 -- 172.21.15.32:0/529923805 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78401d070 con 0x7ff798100f50 2023-09-30T14:50:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.198+0000 7ff79eaa1700 1 -- 172.21.15.32:0/529923805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7981a1080 con 0x7ff798100f50 2023-09-30T14:50:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.198+0000 7ff795ffb700 1 -- 172.21.15.32:0/529923805 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7840054f0 con 0x7ff798100f50 2023-09-30T14:50:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.198+0000 7ff795ffb700 1 -- 172.21.15.32:0/529923805 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784004b00 con 0x7ff798100f50 2023-09-30T14:50:51.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.198+0000 7ff79eaa1700 1 -- 172.21.15.32:0/529923805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7981a1500 con 0x7ff798100f50 2023-09-30T14:50:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.200+0000 7ff795ffb700 1 -- 172.21.15.32:0/529923805 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7ff78400fb80 con 0x7ff798100f50 2023-09-30T14:50:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.201+0000 7ff795ffb700 1 --2- 172.21.15.32:0/529923805 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff7880a1020 0x7ff7880a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.201+0000 7ff795ffb700 1 -- 172.21.15.32:0/529923805 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff7840cd460 con 0x7ff798100f50 2023-09-30T14:50:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.201+0000 7ff79c83d700 1 --2- 172.21.15.32:0/529923805 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff7880a1020 0x7ff7880a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.201+0000 7ff79c83d700 1 --2- 172.21.15.32:0/529923805 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff7880a1020 0x7ff7880a34e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff798103c40 tx=0x7ff78c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff7aac89b80 1 Processor -- start 2023-09-30T14:50:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff7aac89b80 1 -- start start 2023-09-30T14:50:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff7aac89b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae24200 0x56254ae24620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff7aac89b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25310 0x56254ae25790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff7aac89b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae25cd0 0x56254ae2c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff7aac89b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254ad8c400 con 0x56254ae25310 2023-09-30T14:50:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff7aac89b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254ad7dbc0 con 0x56254ae25cd0 2023-09-30T14:50:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff7aac89b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254ad7b4e0 con 0x56254ae24200 2023-09-30T14:50:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff797fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25310 0x56254ae25790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff797fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25310 0x56254ae25790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38194/0 (socket says 172.21.15.32:38194) 2023-09-30T14:50:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff797fff700 1 -- 172.21.15.32:0/148329462 learned_addr learned my addr 172.21.15.32:0/148329462 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff79d03e700 1 --2- 172.21.15.32:0/148329462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae25cd0 0x56254ae2c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff79c83d700 1 --2- 172.21.15.32:0/148329462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae24200 0x56254ae24620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff797fff700 1 -- 172.21.15.32:0/148329462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae24200 msgr2=0x56254ae24620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff797fff700 1 --2- 172.21.15.32:0/148329462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae24200 0x56254ae24620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff797fff700 1 -- 172.21.15.32:0/148329462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae25cd0 msgr2=0x56254ae2c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff797fff700 1 --2- 172.21.15.32:0/148329462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae25cd0 0x56254ae2c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.202+0000 7ff797fff700 1 -- 172.21.15.32:0/148329462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56254add9870 con 0x56254ae25310 2023-09-30T14:50:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.203+0000 7ff797fff700 1 --2- 172.21.15.32:0/148329462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25310 0x56254ae25790 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff784096ec0 tx=0x7ff78409a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112f602d35e92749 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.203+0000 7ff782ffd700 1 -- 172.21.15.32:0/148329462 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78409b0c0 con 0x56254ae25310 2023-09-30T14:50:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.203+0000 7ff782ffd700 1 -- 172.21.15.32:0/148329462 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff78409a9c0 con 0x56254ae25310 2023-09-30T14:50:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.203+0000 7ff782ffd700 1 -- 172.21.15.32:0/148329462 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784099eb0 con 0x56254ae25310 2023-09-30T14:50:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.203+0000 7ff7aac89b80 1 -- 172.21.15.32:0/148329462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25310 msgr2=0x56254ae25790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.203+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/148329462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25310 0x56254ae25790 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff784096ec0 tx=0x7ff78409a370 comp rx=0 tx=0).stop 2023-09-30T14:50:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.203+0000 7ff7aac89b80 1 -- 172.21.15.32:0/148329462 shutdown_connections 2023-09-30T14:50:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.203+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/148329462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae25cd0 0x56254ae2c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.203+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/148329462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae24200 0x56254ae24620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.203+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/148329462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25310 0x56254ae25790 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.203+0000 7ff7aac89b80 1 -- 172.21.15.32:0/148329462 >> 172.21.15.32:0/148329462 conn(0x56254ad98200 msgr2=0x56254ae1eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.204+0000 7ff7aac89b80 1 -- 172.21.15.32:0/148329462 shutdown_connections 2023-09-30T14:50:51.211 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.204+0000 7ff7aac89b80 1 -- 172.21.15.32:0/148329462 wait complete. 2023-09-30T14:50:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.204+0000 7ff7aac89b80 1 -- start start 2023-09-30T14:50:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.204+0000 7ff7aac89b80 1 Processor -- start 2023-09-30T14:50:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.204+0000 7ff7aac89b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae24200 0x56254ae2da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.204+0000 7ff7aac89b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25310 0x56254ae2dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.204+0000 7ff7aac89b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae25cd0 0x56254ae35010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.204+0000 7ff7aac89b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254ad76dd0 con 0x56254ae25310 2023-09-30T14:50:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.204+0000 7ff7aac89b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254ad136b0 con 0x56254ae25cd0 2023-09-30T14:50:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.204+0000 7ff7aac89b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254ad13350 con 0x56254ae24200 2023-09-30T14:50:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.204+0000 7ff7aac89b80 10 client.0 _ll_register_callbacks cb 0x56254ab707a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:50:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.204+0000 7ff797fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25310 0x56254ae2dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff79d03e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae25cd0 0x56254ae35010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff79c83d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae24200 0x56254ae2da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff797fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25310 0x56254ae2dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38200/0 (socket says 172.21.15.32:38200) 2023-09-30T14:50:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff797fff700 1 -- 172.21.15.32:0/268603052 learned_addr learned my addr 172.21.15.32:0/268603052 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff797fff700 1 -- 172.21.15.32:0/268603052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae24200 msgr2=0x56254ae2da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff797fff700 1 --2- 172.21.15.32:0/268603052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae24200 0x56254ae2da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff797fff700 1 -- 172.21.15.32:0/268603052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae25cd0 msgr2=0x56254ae35010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff797fff700 1 --2- 172.21.15.32:0/268603052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae25cd0 0x56254ae35010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff797fff700 1 -- 172.21.15.32:0/268603052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7981a1500 con 0x56254ae25310 2023-09-30T14:50:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff79d03e700 1 --2- 172.21.15.32:0/268603052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae25cd0 0x56254ae35010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:50:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff797fff700 1 --2- 172.21.15.32:0/268603052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25310 0x56254ae2dfd0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff78409a950 tx=0x7ff78409b130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83be2b29e511092c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff777fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:50:51.213 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff777fff700 1 -- 172.21.15.32:0/268603052 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78409a0b0 con 0x56254ae25310 2023-09-30T14:50:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff777fff700 1 -- 172.21.15.32:0/268603052 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff7840a2cc0 con 0x56254ae25310 2023-09-30T14:50:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff7aac89b80 1 -- 172.21.15.32:0/268603052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56254ae17090 con 0x56254ae25310 2023-09-30T14:50:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff777fff700 1 -- 172.21.15.32:0/268603052 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff78407e020 con 0x56254ae25310 2023-09-30T14:50:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff7aac89b80 10 client.16548 fetch_fsmap learned FSMap version 44 2023-09-30T14:50:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.205+0000 7ff7aac89b80 1 -- 172.21.15.32:0/268603052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x56254aba1d80 con 0x56254ae25310 2023-09-30T14:50:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.206+0000 7ff777fff700 1 -- 172.21.15.32:0/268603052 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56254ae17090 con 0x56254ae25310 2023-09-30T14:50:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.206+0000 7ff777fff700 1 -- 172.21.15.32:0/268603052 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff7840999b0 con 0x56254ae25310 2023-09-30T14:50:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.206+0000 7ff7aac89b80 10 client.16548 fetch_fsmap finished waiting for FSMap version 44 2023-09-30T14:50:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.206+0000 7ff7aac89b80 10 client.16548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:50:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.206+0000 7ff7aac89b80 20 client.16548 populate_metadata read hostname 'smithi032' 2023-09-30T14:50:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.206+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/268603052 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56254ae39540 0x56254ae3b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.206+0000 7ff7aac89b80 4 client.16548 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:50:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.206+0000 7ff7aac89b80 1 -- 172.21.15.32:0/268603052 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56254ad6fdb0 con 0x56254ae39540 2023-09-30T14:50:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.206+0000 7ff79c83d700 1 --2- 172.21.15.32:0/268603052 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56254ae39540 0x56254ae3b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.206+0000 7ff777fff700 10 client.16548 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:50:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.206+0000 7ff79c83d700 1 --2- 172.21.15.32:0/268603052 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56254ae39540 0x56254ae3b930 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f652812f6c10900 server_cookie=31b3ee18f0da5482 in_seq=0 out_seq=0 2023-09-30T14:50:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.207+0000 7ff777fff700 1 -- 172.21.15.32:0/268603052 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56254ad6fdb0 con 0x56254ae39540 2023-09-30T14:50:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.207+0000 7ff777fff700 10 client.16548 handle_command_reply: tid=0 2023-09-30T14:50:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.207+0000 7ff7aac89b80 1 client.16548 shutdown 2023-09-30T14:50:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.207+0000 7ff7aac89b80 1 -- 172.21.15.32:0/268603052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25310 msgr2=0x56254ae2dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.207+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/268603052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25310 0x56254ae2dfd0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff78409a950 tx=0x7ff78409b130 comp rx=0 tx=0).stop 2023-09-30T14:50:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.207+0000 7ff7aac89b80 1 -- 172.21.15.32:0/268603052 shutdown_connections 2023-09-30T14:50:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.207+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/268603052 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56254ae39540 0x56254ae3b930 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.207+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/268603052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae25cd0 0x56254ae35010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.207+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/268603052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae24200 0x56254ae2da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.207+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/268603052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25310 0x56254ae2dfd0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.207+0000 7ff777fff700 0 client.16548 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:50:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.207+0000 7ff7aac89b80 1 -- 172.21.15.32:0/268603052 >> 172.21.15.32:0/268603052 conn(0x56254ad98200 msgr2=0x56254ae1e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.208+0000 7ff7aac89b80 1 -- 172.21.15.32:0/268603052 shutdown_connections 2023-09-30T14:50:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.208+0000 7ff7aac89b80 1 -- 172.21.15.32:0/268603052 wait complete. 2023-09-30T14:50:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.208+0000 7ff7aac89b80 20 client.16548 trim_cache size 0 max 16384 2023-09-30T14:50:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.223+0000 7ff7aac89b80 1 Processor -- start 2023-09-30T14:50:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.223+0000 7ff7aac89b80 1 -- start start 2023-09-30T14:50:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.223+0000 7ff7aac89b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae24200 0x56254ae2da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.223+0000 7ff7aac89b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae25310 0x56254ae2dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.223+0000 7ff7aac89b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25cd0 0x56254ae34fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.223+0000 7ff7aac89b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254abf78c0 con 0x56254ae25cd0 2023-09-30T14:50:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.224+0000 7ff7aac89b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254ae2cf40 con 0x56254ae25310 2023-09-30T14:50:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.224+0000 7ff7aac89b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254ae1bba0 con 0x56254ae24200 2023-09-30T14:50:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.224+0000 7ff79d03e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25cd0 0x56254ae34fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.224+0000 7ff797fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae25310 0x56254ae2dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.224+0000 7ff79d03e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25cd0 0x56254ae34fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38210/0 (socket says 172.21.15.32:38210) 2023-09-30T14:50:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.224+0000 7ff79d03e700 1 -- 172.21.15.32:0/2871981758 learned_addr learned my addr 172.21.15.32:0/2871981758 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.224+0000 7ff79c83d700 1 --2- 172.21.15.32:0/2871981758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae24200 0x56254ae2da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.224+0000 7ff79d03e700 1 -- 172.21.15.32:0/2871981758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae24200 msgr2=0x56254ae2da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.224+0000 7ff79d03e700 1 --2- 172.21.15.32:0/2871981758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae24200 0x56254ae2da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.224+0000 7ff79d03e700 1 -- 172.21.15.32:0/2871981758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae25310 msgr2=0x56254ae2dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.224+0000 7ff79d03e700 1 --2- 172.21.15.32:0/2871981758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae25310 0x56254ae2dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.224+0000 7ff79d03e700 1 -- 172.21.15.32:0/2871981758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56254ab8ac00 con 0x56254ae25cd0 2023-09-30T14:50:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.224+0000 7ff79d03e700 1 --2- 172.21.15.32:0/2871981758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25cd0 0x56254ae34fd0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff790012ac0 tx=0x7ff790009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f60edeecc191a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.224+0000 7ff7837fe700 1 -- 172.21.15.32:0/2871981758 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79000e040 con 0x56254ae25cd0 2023-09-30T14:50:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.224+0000 7ff7837fe700 1 -- 172.21.15.32:0/2871981758 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff79000f040 con 0x56254ae25cd0 2023-09-30T14:50:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.224+0000 7ff7837fe700 1 -- 172.21.15.32:0/2871981758 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790007b50 con 0x56254ae25cd0 2023-09-30T14:50:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.225+0000 7ff7aac89b80 1 -- 172.21.15.32:0/2871981758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25cd0 msgr2=0x56254ae34fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.225+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/2871981758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25cd0 0x56254ae34fd0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff790012ac0 tx=0x7ff790009920 comp rx=0 tx=0).stop 2023-09-30T14:50:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.225+0000 7ff7aac89b80 1 -- 172.21.15.32:0/2871981758 shutdown_connections 2023-09-30T14:50:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.225+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/2871981758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25cd0 0x56254ae34fd0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.225+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/2871981758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae24200 0x56254ae2da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.225+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/2871981758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae25310 0x56254ae2dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.225+0000 7ff7aac89b80 1 -- 172.21.15.32:0/2871981758 >> 172.21.15.32:0/2871981758 conn(0x56254ad98200 msgr2=0x56254ae1fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.225+0000 7ff7aac89b80 1 -- 172.21.15.32:0/2871981758 shutdown_connections 2023-09-30T14:50:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.225+0000 7ff7aac89b80 1 -- 172.21.15.32:0/2871981758 wait complete. 2023-09-30T14:50:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff7aac89b80 1 -- start start 2023-09-30T14:50:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff7aac89b80 1 Processor -- start 2023-09-30T14:50:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff7aac89b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae24200 0x56254ae31150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff7aac89b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae25310 0x56254ae31690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff7aac89b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25cd0 0x56254ae346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff7aac89b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254ae34bf0 con 0x56254ae25cd0 2023-09-30T14:50:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff7aac89b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254ae34d60 con 0x56254ae24200 2023-09-30T14:50:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff7aac89b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254ae34f20 con 0x56254ae25310 2023-09-30T14:50:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff7aac89b80 10 client.0 _ll_register_callbacks cb 0x7ff790009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:50:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff79d03e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25cd0 0x56254ae346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff79d03e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25cd0 0x56254ae346b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38216/0 (socket says 172.21.15.32:38216) 2023-09-30T14:50:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff79c83d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae24200 0x56254ae31150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff79d03e700 1 -- 172.21.15.32:0/2797335160 learned_addr learned my addr 172.21.15.32:0/2797335160 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:50:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff797fff700 1 --2- 172.21.15.32:0/2797335160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae25310 0x56254ae31690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff79d03e700 1 -- 172.21.15.32:0/2797335160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae25310 msgr2=0x56254ae31690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff79d03e700 1 --2- 172.21.15.32:0/2797335160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae25310 0x56254ae31690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff79d03e700 1 -- 172.21.15.32:0/2797335160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae24200 msgr2=0x56254ae31150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:50:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff79d03e700 1 --2- 172.21.15.32:0/2797335160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae24200 0x56254ae31150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff79d03e700 1 -- 172.21.15.32:0/2797335160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff790012770 con 0x56254ae25cd0 2023-09-30T14:50:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.226+0000 7ff79c83d700 1 --2- 172.21.15.32:0/2797335160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae24200 0x56254ae31150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:50:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.227+0000 7ff79d03e700 1 --2- 172.21.15.32:0/2797335160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25cd0 0x56254ae346b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff790009510 tx=0x7ff790007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfea4290f9dc46d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:50:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.227+0000 7ff777fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:50:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.227+0000 7ff777fff700 1 -- 172.21.15.32:0/2797335160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79000e070 con 0x56254ae25cd0 2023-09-30T14:50:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.227+0000 7ff777fff700 1 -- 172.21.15.32:0/2797335160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff790007a60 con 0x56254ae25cd0 2023-09-30T14:50:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.227+0000 7ff7aac89b80 1 -- 172.21.15.32:0/2797335160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff78409a9c0 con 0x56254ae25cd0 2023-09-30T14:50:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.227+0000 7ff777fff700 1 -- 172.21.15.32:0/2797335160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff790007c20 con 0x56254ae25cd0 2023-09-30T14:50:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.227+0000 7ff7aac89b80 10 client.16554 fetch_fsmap learned FSMap version 44 2023-09-30T14:50:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.227+0000 7ff7aac89b80 1 -- 172.21.15.32:0/2797335160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x56254abc81f0 con 0x56254ae25cd0 2023-09-30T14:50:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.227+0000 7ff777fff700 1 -- 172.21.15.32:0/2797335160 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff78409a9c0 con 0x56254ae25cd0 2023-09-30T14:50:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.227+0000 7ff777fff700 1 -- 172.21.15.32:0/2797335160 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 44) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff7900040f0 con 0x56254ae25cd0 2023-09-30T14:50:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.227+0000 7ff7aac89b80 10 client.16554 fetch_fsmap finished waiting for FSMap version 44 2023-09-30T14:50:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.227+0000 7ff7aac89b80 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:50:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.227+0000 7ff7aac89b80 20 client.16554 populate_metadata read hostname 'smithi032' 2023-09-30T14:50:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.227+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/2797335160 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56254ae8e5f0 0x56254ae90a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:50:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.227+0000 7ff7aac89b80 4 client.16554 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:50:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.227+0000 7ff7aac89b80 1 -- 172.21.15.32:0/2797335160 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56254ad6fad0 con 0x56254ae8e5f0 2023-09-30T14:50:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.228+0000 7ff79c83d700 1 --2- 172.21.15.32:0/2797335160 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56254ae8e5f0 0x56254ae90a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:50:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.228+0000 7ff777fff700 10 client.16554 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:50:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.228+0000 7ff79c83d700 1 --2- 172.21.15.32:0/2797335160 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56254ae8e5f0 0x56254ae90a90 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ededf4448acb6f7e server_cookie=f20863cd4182cc4e in_seq=0 out_seq=0 2023-09-30T14:50:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.286+0000 7ff777fff700 1 -- 172.21.15.32:0/2797335160 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56254ad6fad0 con 0x56254ae8e5f0 2023-09-30T14:50:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.286+0000 7ff777fff700 10 client.16554 handle_command_reply: tid=0 2023-09-30T14:50:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.286+0000 7ff7aac89b80 1 client.16554 shutdown 2023-09-30T14:50:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.286+0000 7ff7aac89b80 1 -- 172.21.15.32:0/2797335160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25cd0 msgr2=0x56254ae346b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.286+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/2797335160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25cd0 0x56254ae346b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff790009510 tx=0x7ff790007500 comp rx=0 tx=0).stop 2023-09-30T14:50:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.286+0000 7ff7aac89b80 1 -- 172.21.15.32:0/2797335160 shutdown_connections 2023-09-30T14:50:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.286+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/2797335160 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56254ae8e5f0 0x56254ae90a90 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.286+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/2797335160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56254ae25cd0 0x56254ae346b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.286+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/2797335160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56254ae24200 0x56254ae31150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.286+0000 7ff7aac89b80 1 --2- 172.21.15.32:0/2797335160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56254ae25310 0x56254ae31690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.286+0000 7ff7aac89b80 1 -- 172.21.15.32:0/2797335160 >> 172.21.15.32:0/2797335160 conn(0x56254ad98200 msgr2=0x56254ae28960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.286+0000 7ff777fff700 0 client.16554 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:50:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.286+0000 7ff7aac89b80 1 -- 172.21.15.32:0/2797335160 shutdown_connections 2023-09-30T14:50:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.286+0000 7ff7aac89b80 1 -- 172.21.15.32:0/2797335160 wait complete. 2023-09-30T14:50:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.286+0000 7ff7aac89b80 20 client.16554 trim_cache size 0 max 16384 2023-09-30T14:50:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:50:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:50:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:50:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:50:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.287+0000 7ff79eaa1700 1 -- 172.21.15.32:0/529923805 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff7880a1020 msgr2=0x7ff7880a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.287+0000 7ff79eaa1700 1 --2- 172.21.15.32:0/529923805 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff7880a1020 0x7ff7880a34e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff798103c40 tx=0x7ff78c009260 comp rx=0 tx=0).stop 2023-09-30T14:50:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.287+0000 7ff79eaa1700 1 -- 172.21.15.32:0/529923805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff798100f50 msgr2=0x7ff798102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:50:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.287+0000 7ff79eaa1700 1 --2- 172.21.15.32:0/529923805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff798100f50 0x7ff798102870 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff784009990 tx=0x7ff78400f690 comp rx=0 tx=0).stop 2023-09-30T14:50:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.287+0000 7ff79eaa1700 1 -- 172.21.15.32:0/529923805 shutdown_connections 2023-09-30T14:50:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.287+0000 7ff79eaa1700 1 --2- 172.21.15.32:0/529923805 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff7880a1020 0x7ff7880a34e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.287+0000 7ff79eaa1700 1 --2- 172.21.15.32:0/529923805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7981018f0 0x7ff798102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.287+0000 7ff79eaa1700 1 --2- 172.21.15.32:0/529923805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff7980ffd60 0x7ff798102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.287+0000 7ff79eaa1700 1 --2- 172.21.15.32:0/529923805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff798100f50 0x7ff798102870 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:50:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.287+0000 7ff79eaa1700 1 -- 172.21.15.32:0/529923805 >> 172.21.15.32:0/529923805 conn(0x7ff7980fb2b0 msgr2=0x7ff7980682c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:50:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.287+0000 7ff79eaa1700 1 -- 172.21.15.32:0/529923805 shutdown_connections 2023-09-30T14:50:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:50:51.287+0000 7ff79eaa1700 1 -- 172.21.15.32:0/529923805 wait complete. 2023-09-30T14:50:51.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1382914 10.67 MB/sec execute 565 sec latency 21.810 ms 2023-09-30T14:50:51.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:51 smithi148 ceph-mon[102355]: pgmap v664: 97 pgs: 97 active+clean; 392 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 2.4 MiB/s wr, 234 op/s 2023-09-30T14:50:51.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:51 smithi032 ceph-mon[98592]: pgmap v664: 97 pgs: 97 active+clean; 392 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 2.4 MiB/s wr, 234 op/s 2023-09-30T14:50:52.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:51 smithi156 ceph-mon[103579]: pgmap v664: 97 pgs: 97 active+clean; 392 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 2.4 MiB/s wr, 234 op/s 2023-09-30T14:50:52.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1385428 10.68 MB/sec execute 566 sec latency 12.533 ms 2023-09-30T14:50:52.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:52 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:50:52.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:52 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:50:53.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:52 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:50:53.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1386583 10.67 MB/sec execute 567 sec latency 55.835 ms 2023-09-30T14:50:53.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:53 smithi032 ceph-mon[98592]: pgmap v665: 97 pgs: 97 active+clean; 388 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 2.9 MiB/s wr, 276 op/s 2023-09-30T14:50:54.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:53 smithi156 ceph-mon[103579]: pgmap v665: 97 pgs: 97 active+clean; 388 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 2.9 MiB/s wr, 276 op/s 2023-09-30T14:50:54.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:53 smithi148 ceph-mon[102355]: pgmap v665: 97 pgs: 97 active+clean; 388 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 2.9 MiB/s wr, 276 op/s 2023-09-30T14:50:54.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1389040 10.68 MB/sec execute 568 sec latency 18.258 ms 2023-09-30T14:50:55.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1390363 10.67 MB/sec execute 569 sec latency 30.661 ms 2023-09-30T14:50:55.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:55 smithi032 ceph-mon[98592]: pgmap v666: 97 pgs: 97 active+clean; 385 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2023-09-30T14:50:56.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:55 smithi156 ceph-mon[103579]: pgmap v666: 97 pgs: 97 active+clean; 385 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2023-09-30T14:50:56.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:55 smithi148 ceph-mon[102355]: pgmap v666: 97 pgs: 97 active+clean; 385 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2023-09-30T14:50:56.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1392904 10.68 MB/sec execute 570 sec latency 16.807 ms 2023-09-30T14:50:57.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1394444 10.67 MB/sec execute 571 sec latency 15.548 ms 2023-09-30T14:50:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:57 smithi032 ceph-mon[98592]: pgmap v667: 97 pgs: 97 active+clean; 387 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2023-09-30T14:50:58.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:57 smithi156 ceph-mon[103579]: pgmap v667: 97 pgs: 97 active+clean; 387 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2023-09-30T14:50:58.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:57 smithi148 ceph-mon[102355]: pgmap v667: 97 pgs: 97 active+clean; 387 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2023-09-30T14:50:58.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1396641 10.68 MB/sec execute 572 sec latency 48.438 ms 2023-09-30T14:50:59.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1398556 10.67 MB/sec execute 573 sec latency 19.355 ms 2023-09-30T14:50:59.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:50:59 smithi032 ceph-mon[98592]: pgmap v668: 97 pgs: 97 active+clean; 388 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 2.8 MiB/s wr, 271 op/s 2023-09-30T14:51:00.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:50:59 smithi156 ceph-mon[103579]: pgmap v668: 97 pgs: 97 active+clean; 388 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 2.8 MiB/s wr, 271 op/s 2023-09-30T14:51:00.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:50:59 smithi148 ceph-mon[102355]: pgmap v668: 97 pgs: 97 active+clean; 388 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 2.8 MiB/s wr, 271 op/s 2023-09-30T14:51:00.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1400357 10.67 MB/sec execute 574 sec latency 24.572 ms 2023-09-30T14:51:01.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1402374 10.67 MB/sec execute 575 sec latency 28.272 ms 2023-09-30T14:51:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:01 smithi032 ceph-mon[98592]: pgmap v669: 97 pgs: 97 active+clean; 389 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 2.7 MiB/s wr, 274 op/s 2023-09-30T14:51:02.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:01 smithi156 ceph-mon[103579]: pgmap v669: 97 pgs: 97 active+clean; 389 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 2.7 MiB/s wr, 274 op/s 2023-09-30T14:51:02.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:01 smithi148 ceph-mon[102355]: pgmap v669: 97 pgs: 97 active+clean; 389 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 2.7 MiB/s wr, 274 op/s 2023-09-30T14:51:02.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1404006 10.67 MB/sec execute 576 sec latency 12.079 ms 2023-09-30T14:51:03.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1406208 10.67 MB/sec execute 577 sec latency 71.355 ms 2023-09-30T14:51:03.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:03 smithi032 ceph-mon[98592]: pgmap v670: 97 pgs: 97 active+clean; 390 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 3.0 MiB/s wr, 299 op/s 2023-09-30T14:51:04.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:03 smithi156 ceph-mon[103579]: pgmap v670: 97 pgs: 97 active+clean; 390 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 3.0 MiB/s wr, 299 op/s 2023-09-30T14:51:04.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:03 smithi148 ceph-mon[102355]: pgmap v670: 97 pgs: 97 active+clean; 390 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 3.0 MiB/s wr, 299 op/s 2023-09-30T14:51:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.408+0000 7f516d03b700 1 -- 172.21.15.32:0/1859942320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51680ffd60 msgr2=0x7f5168100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.408+0000 7f516d03b700 1 --2- 172.21.15.32:0/1859942320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51680ffd60 0x7f5168100180 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f51580099c0 tx=0x7f5158009cd0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.408+0000 7f516d03b700 1 -- 172.21.15.32:0/1859942320 shutdown_connections 2023-09-30T14:51:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.408+0000 7f516d03b700 1 --2- 172.21.15.32:0/1859942320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51681018f0 0x7f51681081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.408+0000 7f516d03b700 1 --2- 172.21.15.32:0/1859942320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51680ffd60 0x7f5168100180 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.408+0000 7f516d03b700 1 --2- 172.21.15.32:0/1859942320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5168100f50 0x7f51681013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.408+0000 7f516d03b700 1 -- 172.21.15.32:0/1859942320 >> 172.21.15.32:0/1859942320 conn(0x7f51680fb2b0 msgr2=0x7f51680fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.409+0000 7f516d03b700 1 -- 172.21.15.32:0/1859942320 shutdown_connections 2023-09-30T14:51:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.409+0000 7f516d03b700 1 -- 172.21.15.32:0/1859942320 wait complete. 2023-09-30T14:51:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.409+0000 7f516d03b700 1 Processor -- start 2023-09-30T14:51:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.410+0000 7f516d03b700 1 -- start start 2023-09-30T14:51:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.410+0000 7f516d03b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51680ffd60 0x7f5168199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.410+0000 7f516d03b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5168100f50 0x7f516819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.410+0000 7f516d03b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51681018f0 0x7f516819e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.410+0000 7f516d03b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516819e8d0 con 0x7f51681018f0 2023-09-30T14:51:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.410+0000 7f516d03b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516819ea40 con 0x7f5168100f50 2023-09-30T14:51:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.410+0000 7f516d03b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516819ec00 con 0x7f51680ffd60 2023-09-30T14:51:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.410+0000 7f5166d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51680ffd60 0x7f5168199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.410+0000 7f516759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51681018f0 0x7f516819e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.410+0000 7f5166d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51680ffd60 0x7f5168199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52608/0 (socket says 172.21.15.32:52608) 2023-09-30T14:51:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.410+0000 7f5166d9d700 1 -- 172.21.15.32:0/1505303633 learned_addr learned my addr 172.21.15.32:0/1505303633 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.410+0000 7f516659c700 1 --2- 172.21.15.32:0/1505303633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5168100f50 0x7f516819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.411+0000 7f516759e700 1 -- 172.21.15.32:0/1505303633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51680ffd60 msgr2=0x7f5168199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.411+0000 7f516759e700 1 --2- 172.21.15.32:0/1505303633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51680ffd60 0x7f5168199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.411+0000 7f516759e700 1 -- 172.21.15.32:0/1505303633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5168100f50 msgr2=0x7f516819bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.411+0000 7f516759e700 1 --2- 172.21.15.32:0/1505303633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5168100f50 0x7f516819bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.411+0000 7f516759e700 1 -- 172.21.15.32:0/1505303633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51580095f0 con 0x7f51681018f0 2023-09-30T14:51:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.411+0000 7f516659c700 1 --2- 172.21.15.32:0/1505303633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5168100f50 0x7f516819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.411+0000 7f5166d9d700 1 --2- 172.21.15.32:0/1505303633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51680ffd60 0x7f5168199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.411+0000 7f516759e700 1 --2- 172.21.15.32:0/1505303633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51681018f0 0x7f516819e2d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f516000efc0 tx=0x7f516000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.411+0000 7f514ffff700 1 -- 172.21.15.32:0/1505303633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160018070 con 0x7f51681018f0 2023-09-30T14:51:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.411+0000 7f516d03b700 1 -- 172.21.15.32:0/1505303633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516819ee80 con 0x7f51681018f0 2023-09-30T14:51:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.411+0000 7f514ffff700 1 -- 172.21.15.32:0/1505303633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f516000f040 con 0x7f51681018f0 2023-09-30T14:51:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.411+0000 7f514ffff700 1 -- 172.21.15.32:0/1505303633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516000cd40 con 0x7f51681018f0 2023-09-30T14:51:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.411+0000 7f516d03b700 1 -- 172.21.15.32:0/1505303633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516819f300 con 0x7f51681018f0 2023-09-30T14:51:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.413+0000 7f514ffff700 1 -- 172.21.15.32:0/1505303633 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f5160003a00 con 0x7f51681018f0 2023-09-30T14:51:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.414+0000 7f514ffff700 1 --2- 172.21.15.32:0/1505303633 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51500a0fa0 0x7f51500a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.414+0000 7f514ffff700 1 -- 172.21.15.32:0/1505303633 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f51600cc310 con 0x7f51681018f0 2023-09-30T14:51:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.414+0000 7f5166d9d700 1 --2- 172.21.15.32:0/1505303633 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51500a0fa0 0x7f51500a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.414+0000 7f5166d9d700 1 --2- 172.21.15.32:0/1505303633 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51500a0fa0 0x7f51500a3460 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5158004800 tx=0x7f515801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.415+0000 7f5179223b80 1 Processor -- start 2023-09-30T14:51:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.415+0000 7f5179223b80 1 -- start start 2023-09-30T14:51:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.415+0000 7f5179223b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 0x55a940fd6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.415+0000 7f5179223b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7310 0x55a940fd7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.415+0000 7f5179223b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7cd0 0x55a940fde560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.415+0000 7f5179223b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940f3e400 con 0x55a940fd7cd0 2023-09-30T14:51:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.415+0000 7f5179223b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940f2fbc0 con 0x55a940fd6200 2023-09-30T14:51:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.415+0000 7f5179223b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940f2d4e0 con 0x55a940fd7310 2023-09-30T14:51:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.415+0000 7f516759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7cd0 0x55a940fde560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.415+0000 7f5166d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 0x55a940fd6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.416+0000 7f5166d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 0x55a940fd6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49710/0 (socket says 172.21.15.32:49710) 2023-09-30T14:51:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.415+0000 7f516759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7cd0 0x55a940fde560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38238/0 (socket says 172.21.15.32:38238) 2023-09-30T14:51:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.416+0000 7f5166d9d700 1 -- 172.21.15.32:0/883881634 learned_addr learned my addr 172.21.15.32:0/883881634 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.416+0000 7f516659c700 1 --2- 172.21.15.32:0/883881634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7310 0x55a940fd7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.416+0000 7f5166d9d700 1 -- 172.21.15.32:0/883881634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7310 msgr2=0x55a940fd7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.416+0000 7f5166d9d700 1 --2- 172.21.15.32:0/883881634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7310 0x55a940fd7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.416+0000 7f5166d9d700 1 -- 172.21.15.32:0/883881634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7cd0 msgr2=0x55a940fde560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.416+0000 7f5166d9d700 1 --2- 172.21.15.32:0/883881634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7cd0 0x55a940fde560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.416+0000 7f5166d9d700 1 -- 172.21.15.32:0/883881634 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a940f8b870 con 0x55a940fd6200 2023-09-30T14:51:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.416+0000 7f516759e700 1 --2- 172.21.15.32:0/883881634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7cd0 0x55a940fde560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.416+0000 7f516659c700 1 --2- 172.21.15.32:0/883881634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7310 0x55a940fd7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.416+0000 7f5166d9d700 1 --2- 172.21.15.32:0/883881634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 0x55a940fd6620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f51580098e0 tx=0x7f515801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be33834757145975 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.416+0000 7f514d7fa700 1 -- 172.21.15.32:0/883881634 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515802a070 con 0x55a940fd6200 2023-09-30T14:51:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.416+0000 7f514d7fa700 1 -- 172.21.15.32:0/883881634 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f515801e040 con 0x55a940fd6200 2023-09-30T14:51:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.416+0000 7f514d7fa700 1 -- 172.21.15.32:0/883881634 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158005280 con 0x55a940fd6200 2023-09-30T14:51:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.417+0000 7f5179223b80 1 -- 172.21.15.32:0/883881634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 msgr2=0x55a940fd6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.417+0000 7f5179223b80 1 --2- 172.21.15.32:0/883881634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 0x55a940fd6620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f51580098e0 tx=0x7f515801f000 comp rx=0 tx=0).stop 2023-09-30T14:51:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.417+0000 7f5179223b80 1 -- 172.21.15.32:0/883881634 shutdown_connections 2023-09-30T14:51:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.417+0000 7f5179223b80 1 --2- 172.21.15.32:0/883881634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7cd0 0x55a940fde560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.417+0000 7f5179223b80 1 --2- 172.21.15.32:0/883881634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 0x55a940fd6620 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.417+0000 7f5179223b80 1 --2- 172.21.15.32:0/883881634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7310 0x55a940fd7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.417+0000 7f5179223b80 1 -- 172.21.15.32:0/883881634 >> 172.21.15.32:0/883881634 conn(0x55a940f4a200 msgr2=0x55a940fd0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.417+0000 7f5179223b80 1 -- 172.21.15.32:0/883881634 shutdown_connections 2023-09-30T14:51:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.417+0000 7f5179223b80 1 -- 172.21.15.32:0/883881634 wait complete. 2023-09-30T14:51:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.417+0000 7f5179223b80 1 -- start start 2023-09-30T14:51:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f5179223b80 1 Processor -- start 2023-09-30T14:51:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f5179223b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 0x55a940fdfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f5179223b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7310 0x55a940fe02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f5179223b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7cd0 0x55a940fe7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f5179223b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940f28dd0 con 0x55a940fd7310 2023-09-30T14:51:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f5179223b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940ec56b0 con 0x55a940fd6200 2023-09-30T14:51:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f5179223b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940ec5350 con 0x55a940fd7cd0 2023-09-30T14:51:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f5179223b80 10 client.0 _ll_register_callbacks cb 0x55a940d227a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:51:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f516659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7310 0x55a940fe02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f5166d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 0x55a940fdfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f516659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7310 0x55a940fe02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38242/0 (socket says 172.21.15.32:38242) 2023-09-30T14:51:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f516659c700 1 -- 172.21.15.32:0/134895970 learned_addr learned my addr 172.21.15.32:0/134895970 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f516759e700 1 --2- 172.21.15.32:0/134895970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7cd0 0x55a940fe7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f5166d9d700 1 -- 172.21.15.32:0/134895970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7cd0 msgr2=0x55a940fe7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f5166d9d700 1 --2- 172.21.15.32:0/134895970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7cd0 0x55a940fe7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f5166d9d700 1 -- 172.21.15.32:0/134895970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7310 msgr2=0x55a940fe02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f5166d9d700 1 --2- 172.21.15.32:0/134895970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7310 0x55a940fe02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f5166d9d700 1 -- 172.21.15.32:0/134895970 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516819f300 con 0x55a940fd6200 2023-09-30T14:51:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f516659c700 1 --2- 172.21.15.32:0/134895970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7310 0x55a940fe02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f5166d9d700 1 --2- 172.21.15.32:0/134895970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 0x55a940fdfd80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5158000c00 tx=0x7f5158004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7625f9c195fe483 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.418+0000 7f514aff5700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:51:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.419+0000 7f514aff5700 1 -- 172.21.15.32:0/134895970 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515803c070 con 0x55a940fd6200 2023-09-30T14:51:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.419+0000 7f5179223b80 1 -- 172.21.15.32:0/134895970 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a940fc9090 con 0x55a940fd6200 2023-09-30T14:51:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.419+0000 7f514aff5700 1 -- 172.21.15.32:0/134895970 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5158005040 con 0x55a940fd6200 2023-09-30T14:51:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.419+0000 7f514aff5700 1 -- 172.21.15.32:0/134895970 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f515802a050 con 0x55a940fd6200 2023-09-30T14:51:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.419+0000 7f5179223b80 10 client.26252 fetch_fsmap learned FSMap version 45 2023-09-30T14:51:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.419+0000 7f5179223b80 1 -- 172.21.15.32:0/134895970 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f5168108af0 con 0x55a940fd6200 2023-09-30T14:51:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.419+0000 7f514aff5700 1 -- 172.21.15.32:0/134895970 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5158047080 con 0x55a940fd6200 2023-09-30T14:51:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.419+0000 7f514aff5700 1 -- 172.21.15.32:0/134895970 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 45) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f515803f030 con 0x55a940fd6200 2023-09-30T14:51:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.419+0000 7f5179223b80 10 client.26252 fetch_fsmap finished waiting for FSMap version 45 2023-09-30T14:51:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.419+0000 7f5179223b80 10 client.26252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:51:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.419+0000 7f5179223b80 20 client.26252 populate_metadata read hostname 'smithi032' 2023-09-30T14:51:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.419+0000 7f5179223b80 1 --2- 172.21.15.32:0/134895970 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a940feb6a0 0x55a940fedac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.419+0000 7f5179223b80 4 client.26252 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:51:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.419+0000 7f5179223b80 1 -- 172.21.15.32:0/134895970 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a940f21db0 con 0x55a940feb6a0 2023-09-30T14:51:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.419+0000 7f516659c700 1 --2- 172.21.15.32:0/134895970 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a940feb6a0 0x55a940fedac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.420+0000 7f514aff5700 10 client.26252 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.420+0000 7f516659c700 1 --2- 172.21.15.32:0/134895970 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a940feb6a0 0x55a940fedac0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31f5ed95c9ab25e8 server_cookie=3ee4fb8d5aeae337 in_seq=0 out_seq=0 2023-09-30T14:51:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.420+0000 7f514aff5700 1 -- 172.21.15.32:0/134895970 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a940f21db0 con 0x55a940feb6a0 2023-09-30T14:51:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.420+0000 7f514aff5700 10 client.26252 handle_command_reply: tid=0 2023-09-30T14:51:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.420+0000 7f5179223b80 1 client.26252 shutdown 2023-09-30T14:51:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.421+0000 7f5179223b80 1 -- 172.21.15.32:0/134895970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 msgr2=0x55a940fdfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.421+0000 7f5179223b80 1 --2- 172.21.15.32:0/134895970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 0x55a940fdfd80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5158000c00 tx=0x7f5158004a60 comp rx=0 tx=0).stop 2023-09-30T14:51:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.421+0000 7f5179223b80 1 -- 172.21.15.32:0/134895970 shutdown_connections 2023-09-30T14:51:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.421+0000 7f5179223b80 1 --2- 172.21.15.32:0/134895970 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a940feb6a0 0x55a940fedac0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.421+0000 7f5179223b80 1 --2- 172.21.15.32:0/134895970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7cd0 0x55a940fe7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.421+0000 7f5179223b80 1 --2- 172.21.15.32:0/134895970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 0x55a940fdfd80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.421+0000 7f5179223b80 1 --2- 172.21.15.32:0/134895970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7310 0x55a940fe02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.421+0000 7f514aff5700 0 client.26252 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.421+0000 7f5179223b80 1 -- 172.21.15.32:0/134895970 >> 172.21.15.32:0/134895970 conn(0x55a940f4a200 msgr2=0x55a940fd0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.421+0000 7f5179223b80 1 -- 172.21.15.32:0/134895970 shutdown_connections 2023-09-30T14:51:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.421+0000 7f5179223b80 1 -- 172.21.15.32:0/134895970 wait complete. 2023-09-30T14:51:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.421+0000 7f5179223b80 20 client.26252 trim_cache size 0 max 16384 2023-09-30T14:51:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.436+0000 7f5179223b80 1 Processor -- start 2023-09-30T14:51:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.436+0000 7f5179223b80 1 -- start start 2023-09-30T14:51:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.436+0000 7f5179223b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd6200 0x55a940fdf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.436+0000 7f5179223b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7310 0x55a940fe72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.436+0000 7f5179223b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd7cd0 0x55a940fdab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.436+0000 7f5179223b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940da98c0 con 0x55a940fd6200 2023-09-30T14:51:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.436+0000 7f5179223b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940f21db0 con 0x55a940fd7cd0 2023-09-30T14:51:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.436+0000 7f5179223b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940fc8bd0 con 0x55a940fd7310 2023-09-30T14:51:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.436+0000 7f5166d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd6200 0x55a940fdf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.436+0000 7f5166d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd6200 0x55a940fdf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38248/0 (socket says 172.21.15.32:38248) 2023-09-30T14:51:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.436+0000 7f5166d9d700 1 -- 172.21.15.32:0/636056804 learned_addr learned my addr 172.21.15.32:0/636056804 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.436+0000 7f516759e700 1 --2- 172.21.15.32:0/636056804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd7cd0 0x55a940fdab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.436+0000 7f516659c700 1 --2- 172.21.15.32:0/636056804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7310 0x55a940fe72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.437+0000 7f516759e700 1 -- 172.21.15.32:0/636056804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7310 msgr2=0x55a940fe72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.437+0000 7f516759e700 1 --2- 172.21.15.32:0/636056804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7310 0x55a940fe72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.437+0000 7f516759e700 1 -- 172.21.15.32:0/636056804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd6200 msgr2=0x55a940fdf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.437+0000 7f516759e700 1 --2- 172.21.15.32:0/636056804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd6200 0x55a940fdf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.437+0000 7f516759e700 1 -- 172.21.15.32:0/636056804 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a940d53d80 con 0x55a940fd7cd0 2023-09-30T14:51:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.437+0000 7f5166d9d700 1 --2- 172.21.15.32:0/636056804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd6200 0x55a940fdf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.437+0000 7f516759e700 1 --2- 172.21.15.32:0/636056804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd7cd0 0x55a940fdab80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f516000c6f0 tx=0x7f5160004dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=360868f7ec25fa69 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.437+0000 7f514dffb700 1 -- 172.21.15.32:0/636056804 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160082ea0 con 0x55a940fd7cd0 2023-09-30T14:51:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.437+0000 7f514dffb700 1 -- 172.21.15.32:0/636056804 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5160099f70 con 0x55a940fd7cd0 2023-09-30T14:51:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.437+0000 7f514dffb700 1 -- 172.21.15.32:0/636056804 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160097d30 con 0x55a940fd7cd0 2023-09-30T14:51:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 -- 172.21.15.32:0/636056804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd7cd0 msgr2=0x55a940fdab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 --2- 172.21.15.32:0/636056804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd7cd0 0x55a940fdab80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f516000c6f0 tx=0x7f5160004dc0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 -- 172.21.15.32:0/636056804 shutdown_connections 2023-09-30T14:51:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 --2- 172.21.15.32:0/636056804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd7cd0 0x55a940fdab80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 --2- 172.21.15.32:0/636056804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd6200 0x55a940fdf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 --2- 172.21.15.32:0/636056804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7310 0x55a940fe72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 -- 172.21.15.32:0/636056804 >> 172.21.15.32:0/636056804 conn(0x55a940f4a200 msgr2=0x55a940fde070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 -- 172.21.15.32:0/636056804 shutdown_connections 2023-09-30T14:51:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 -- 172.21.15.32:0/636056804 wait complete. 2023-09-30T14:51:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 -- start start 2023-09-30T14:51:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 Processor -- start 2023-09-30T14:51:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 0x55a940fe6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7310 0x55a940fe71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7cd0 0x55a940fe5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940fe5920 con 0x55a940fd7310 2023-09-30T14:51:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940fe5a90 con 0x55a940fd6200 2023-09-30T14:51:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940fe5c50 con 0x55a940fd7cd0 2023-09-30T14:51:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.438+0000 7f5179223b80 10 client.0 _ll_register_callbacks cb 0x7f516000cec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:51:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f516659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7310 0x55a940fe71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f516659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7310 0x55a940fe71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38256/0 (socket says 172.21.15.32:38256) 2023-09-30T14:51:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f5166d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 0x55a940fe6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f516659c700 1 -- 172.21.15.32:0/3789685605 learned_addr learned my addr 172.21.15.32:0/3789685605 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f516759e700 1 --2- 172.21.15.32:0/3789685605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7cd0 0x55a940fe5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f516659c700 1 -- 172.21.15.32:0/3789685605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7cd0 msgr2=0x55a940fe5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f516659c700 1 --2- 172.21.15.32:0/3789685605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7cd0 0x55a940fe5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f516659c700 1 -- 172.21.15.32:0/3789685605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 msgr2=0x55a940fe6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f516659c700 1 --2- 172.21.15.32:0/3789685605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 0x55a940fe6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f516659c700 1 -- 172.21.15.32:0/3789685605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5168108af0 con 0x55a940fd7310 2023-09-30T14:51:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f516659c700 1 --2- 172.21.15.32:0/3789685605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7310 0x55a940fe71b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f515c000c00 tx=0x7f515c021ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c410e757536ba49 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f514aff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:51:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f514aff5700 1 -- 172.21.15.32:0/3789685605 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c017040 con 0x55a940fd7310 2023-09-30T14:51:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f514aff5700 1 -- 172.21.15.32:0/3789685605 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f515c0210b0 con 0x55a940fd7310 2023-09-30T14:51:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f5179223b80 1 -- 172.21.15.32:0/3789685605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f515801e040 con 0x55a940fd7310 2023-09-30T14:51:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f514aff5700 1 -- 172.21.15.32:0/3789685605 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f515c0122d0 con 0x55a940fd7310 2023-09-30T14:51:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f5179223b80 10 client.16572 fetch_fsmap learned FSMap version 45 2023-09-30T14:51:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.439+0000 7f5179223b80 1 -- 172.21.15.32:0/3789685605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f516819ee80 con 0x55a940fd7310 2023-09-30T14:51:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.440+0000 7f514aff5700 1 -- 172.21.15.32:0/3789685605 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f515801e040 con 0x55a940fd7310 2023-09-30T14:51:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.440+0000 7f514aff5700 1 -- 172.21.15.32:0/3789685605 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 45) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f515c01f8f0 con 0x55a940fd7310 2023-09-30T14:51:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.440+0000 7f5179223b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 45 2023-09-30T14:51:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.440+0000 7f5179223b80 10 client.16572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:51:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.440+0000 7f5179223b80 20 client.16572 populate_metadata read hostname 'smithi032' 2023-09-30T14:51:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.440+0000 7f5179223b80 1 --2- 172.21.15.32:0/3789685605 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a941040970 0x55a941042d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.440+0000 7f5179223b80 4 client.16572 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:51:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.440+0000 7f5179223b80 1 -- 172.21.15.32:0/3789685605 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a940f21ad0 con 0x55a941040970 2023-09-30T14:51:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.440+0000 7f5166d9d700 1 --2- 172.21.15.32:0/3789685605 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a941040970 0x55a941042d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.440+0000 7f514aff5700 10 client.16572 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.440+0000 7f5166d9d700 1 --2- 172.21.15.32:0/3789685605 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a941040970 0x55a941042d90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4efcb1eb2418068c server_cookie=5b8076c0afe87b3f in_seq=0 out_seq=0 2023-09-30T14:51:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.500+0000 7f514aff5700 1 -- 172.21.15.32:0/3789685605 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a940f21ad0 con 0x55a941040970 2023-09-30T14:51:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.500+0000 7f514aff5700 10 client.16572 handle_command_reply: tid=0 2023-09-30T14:51:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.500+0000 7f5179223b80 1 client.16572 shutdown 2023-09-30T14:51:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.500+0000 7f5179223b80 1 -- 172.21.15.32:0/3789685605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7310 msgr2=0x55a940fe71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.500+0000 7f5179223b80 1 --2- 172.21.15.32:0/3789685605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7310 0x55a940fe71b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f515c000c00 tx=0x7f515c021ce0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.500+0000 7f5179223b80 1 -- 172.21.15.32:0/3789685605 shutdown_connections 2023-09-30T14:51:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.500+0000 7f5179223b80 1 --2- 172.21.15.32:0/3789685605 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a941040970 0x55a941042d90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.500+0000 7f5179223b80 1 --2- 172.21.15.32:0/3789685605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a940fd7cd0 0x55a940fe5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.500+0000 7f5179223b80 1 --2- 172.21.15.32:0/3789685605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a940fd6200 0x55a940fe6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.500+0000 7f514aff5700 0 client.16572 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.500+0000 7f5179223b80 1 --2- 172.21.15.32:0/3789685605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a940fd7310 0x55a940fe71b0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.500+0000 7f5179223b80 1 -- 172.21.15.32:0/3789685605 >> 172.21.15.32:0/3789685605 conn(0x55a940f4a200 msgr2=0x55a940fdab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.501+0000 7f5179223b80 1 -- 172.21.15.32:0/3789685605 shutdown_connections 2023-09-30T14:51:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.501+0000 7f5179223b80 1 -- 172.21.15.32:0/3789685605 wait complete. 2023-09-30T14:51:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.501+0000 7f5179223b80 20 client.16572 trim_cache size 0 max 16384 2023-09-30T14:51:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:51:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:51:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:51:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:51:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.501+0000 7f516d03b700 1 -- 172.21.15.32:0/1505303633 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51500a0fa0 msgr2=0x7f51500a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.501+0000 7f516d03b700 1 --2- 172.21.15.32:0/1505303633 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51500a0fa0 0x7f51500a3460 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5158004800 tx=0x7f515801a040 comp rx=0 tx=0).stop 2023-09-30T14:51:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.501+0000 7f516d03b700 1 -- 172.21.15.32:0/1505303633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51681018f0 msgr2=0x7f516819e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.501+0000 7f516d03b700 1 --2- 172.21.15.32:0/1505303633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51681018f0 0x7f516819e2d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f516000efc0 tx=0x7f516000c3a0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.501+0000 7f516d03b700 1 -- 172.21.15.32:0/1505303633 shutdown_connections 2023-09-30T14:51:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.501+0000 7f516d03b700 1 --2- 172.21.15.32:0/1505303633 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f51500a0fa0 0x7f51500a3460 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.501+0000 7f516d03b700 1 --2- 172.21.15.32:0/1505303633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51681018f0 0x7f516819e2d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.501+0000 7f516d03b700 1 --2- 172.21.15.32:0/1505303633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51680ffd60 0x7f5168199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.501+0000 7f516d03b700 1 --2- 172.21.15.32:0/1505303633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5168100f50 0x7f516819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.501+0000 7f516d03b700 1 -- 172.21.15.32:0/1505303633 >> 172.21.15.32:0/1505303633 conn(0x7f51680fb2b0 msgr2=0x7f51680fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.501+0000 7f516d03b700 1 -- 172.21.15.32:0/1505303633 shutdown_connections 2023-09-30T14:51:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:04.501+0000 7f516d03b700 1 -- 172.21.15.32:0/1505303633 wait complete. 2023-09-30T14:51:04.726 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1407555 10.66 MB/sec execute 578 sec latency 21.993 ms 2023-09-30T14:51:05.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1409906 10.67 MB/sec execute 579 sec latency 16.142 ms 2023-09-30T14:51:05.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:05 smithi032 ceph-mon[98592]: pgmap v671: 97 pgs: 97 active+clean; 376 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.6 MiB/s wr, 250 op/s 2023-09-30T14:51:06.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:05 smithi156 ceph-mon[103579]: pgmap v671: 97 pgs: 97 active+clean; 376 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.6 MiB/s wr, 250 op/s 2023-09-30T14:51:06.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:05 smithi148 ceph-mon[102355]: pgmap v671: 97 pgs: 97 active+clean; 376 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.6 MiB/s wr, 250 op/s 2023-09-30T14:51:06.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1411206 10.66 MB/sec execute 580 sec latency 40.982 ms 2023-09-30T14:51:07.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1413514 10.66 MB/sec execute 581 sec latency 19.956 ms 2023-09-30T14:51:07.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:07 smithi032 ceph-mon[98592]: pgmap v672: 97 pgs: 97 active+clean; 371 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 2.9 MiB/s wr, 295 op/s 2023-09-30T14:51:08.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:07 smithi156 ceph-mon[103579]: pgmap v672: 97 pgs: 97 active+clean; 371 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 2.9 MiB/s wr, 295 op/s 2023-09-30T14:51:08.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:07 smithi148 ceph-mon[102355]: pgmap v672: 97 pgs: 97 active+clean; 371 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 2.9 MiB/s wr, 295 op/s 2023-09-30T14:51:08.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1414754 10.66 MB/sec execute 582 sec latency 61.378 ms 2023-09-30T14:51:09.729 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1416685 10.65 MB/sec execute 583 sec latency 41.435 ms 2023-09-30T14:51:09.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:09 smithi032 ceph-mon[98592]: pgmap v673: 97 pgs: 97 active+clean; 370 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 2.8 MiB/s wr, 272 op/s 2023-09-30T14:51:10.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:09 smithi156 ceph-mon[103579]: pgmap v673: 97 pgs: 97 active+clean; 370 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 2.8 MiB/s wr, 272 op/s 2023-09-30T14:51:10.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:09 smithi148 ceph-mon[102355]: pgmap v673: 97 pgs: 97 active+clean; 370 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 2.8 MiB/s wr, 272 op/s 2023-09-30T14:51:10.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1418223 10.65 MB/sec execute 584 sec latency 11.011 ms 2023-09-30T14:51:11.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1420407 10.65 MB/sec execute 585 sec latency 20.100 ms 2023-09-30T14:51:11.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:11 smithi032 ceph-mon[98592]: pgmap v674: 97 pgs: 97 active+clean; 368 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 2.7 MiB/s wr, 268 op/s 2023-09-30T14:51:12.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:11 smithi156 ceph-mon[103579]: pgmap v674: 97 pgs: 97 active+clean; 368 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 2.7 MiB/s wr, 268 op/s 2023-09-30T14:51:12.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:11 smithi148 ceph-mon[102355]: pgmap v674: 97 pgs: 97 active+clean; 368 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 2.7 MiB/s wr, 268 op/s 2023-09-30T14:51:12.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1421841 10.65 MB/sec execute 586 sec latency 12.587 ms 2023-09-30T14:51:13.728 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1423952 10.65 MB/sec execute 587 sec latency 34.643 ms 2023-09-30T14:51:13.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:13 smithi032 ceph-mon[98592]: pgmap v675: 97 pgs: 97 active+clean; 369 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 2.7 MiB/s wr, 254 op/s 2023-09-30T14:51:14.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:13 smithi156 ceph-mon[103579]: pgmap v675: 97 pgs: 97 active+clean; 369 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 2.7 MiB/s wr, 254 op/s 2023-09-30T14:51:14.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:13 smithi148 ceph-mon[102355]: pgmap v675: 97 pgs: 97 active+clean; 369 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 2.7 MiB/s wr, 254 op/s 2023-09-30T14:51:14.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1425225 10.65 MB/sec execute 588 sec latency 46.976 ms 2023-09-30T14:51:15.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1427481 10.65 MB/sec execute 589 sec latency 16.027 ms 2023-09-30T14:51:15.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:15 smithi032 ceph-mon[98592]: pgmap v676: 97 pgs: 97 active+clean; 371 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.7 MiB/s wr, 265 op/s 2023-09-30T14:51:16.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:15 smithi156 ceph-mon[103579]: pgmap v676: 97 pgs: 97 active+clean; 371 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.7 MiB/s wr, 265 op/s 2023-09-30T14:51:16.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:15 smithi148 ceph-mon[102355]: pgmap v676: 97 pgs: 97 active+clean; 371 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 2.7 MiB/s wr, 265 op/s 2023-09-30T14:51:16.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1428821 10.64 MB/sec execute 590 sec latency 15.108 ms 2023-09-30T14:51:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.615+0000 7f310db00700 1 -- 172.21.15.32:0/1119974660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31081008a0 msgr2=0x7f3108108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.615+0000 7f310db00700 1 --2- 172.21.15.32:0/1119974660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31081008a0 0x7f3108108140 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f310000b0a0 tx=0x7f310000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.615+0000 7f310db00700 1 -- 172.21.15.32:0/1119974660 shutdown_connections 2023-09-30T14:51:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.615+0000 7f310db00700 1 --2- 172.21.15.32:0/1119974660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31081008a0 0x7f3108108140 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.615+0000 7f310db00700 1 --2- 172.21.15.32:0/1119974660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3108101290 0x7f31081016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.615+0000 7f310db00700 1 --2- 172.21.15.32:0/1119974660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31080ffee0 0x7f3108100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.615+0000 7f310db00700 1 -- 172.21.15.32:0/1119974660 >> 172.21.15.32:0/1119974660 conn(0x7f31080fa9b0 msgr2=0x7f31080fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.615+0000 7f310db00700 1 -- 172.21.15.32:0/1119974660 shutdown_connections 2023-09-30T14:51:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.615+0000 7f310db00700 1 -- 172.21.15.32:0/1119974660 wait complete. 2023-09-30T14:51:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.616+0000 7f310db00700 1 Processor -- start 2023-09-30T14:51:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.616+0000 7f310db00700 1 -- start start 2023-09-30T14:51:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.616+0000 7f310db00700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31080ffee0 0x7f3108196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.616+0000 7f310db00700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31081008a0 0x7f3108197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.616+0000 7f310db00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3108101290 0x7f310819e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.616+0000 7f310db00700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310819e740 con 0x7f31080ffee0 2023-09-30T14:51:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.616+0000 7f310db00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310819e880 con 0x7f3108101290 2023-09-30T14:51:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.616+0000 7f310db00700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310819ea10 con 0x7f31081008a0 2023-09-30T14:51:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.616+0000 7f31077fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31080ffee0 0x7f3108196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.616+0000 7f31077fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31080ffee0 0x7f3108196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38268/0 (socket says 172.21.15.32:38268) 2023-09-30T14:51:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.616+0000 7f31077fe700 1 -- 172.21.15.32:0/614053324 learned_addr learned my addr 172.21.15.32:0/614053324 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.616+0000 7f3107fff700 1 --2- 172.21.15.32:0/614053324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3108101290 0x7f310819e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.616+0000 7f3106ffd700 1 --2- 172.21.15.32:0/614053324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31081008a0 0x7f3108197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.617+0000 7f31077fe700 1 -- 172.21.15.32:0/614053324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31081008a0 msgr2=0x7f3108197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.617+0000 7f31077fe700 1 --2- 172.21.15.32:0/614053324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31081008a0 0x7f3108197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.617+0000 7f31077fe700 1 -- 172.21.15.32:0/614053324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3108101290 msgr2=0x7f310819e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.617+0000 7f31077fe700 1 --2- 172.21.15.32:0/614053324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3108101290 0x7f310819e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.617+0000 7f31077fe700 1 -- 172.21.15.32:0/614053324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3100009d00 con 0x7f31080ffee0 2023-09-30T14:51:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.617+0000 7f31077fe700 1 --2- 172.21.15.32:0/614053324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31080ffee0 0x7f3108196bc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f30f800e9a0 tx=0x7f30f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.617+0000 7f3104ff9700 1 -- 172.21.15.32:0/614053324 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f8009790 con 0x7f31080ffee0 2023-09-30T14:51:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.617+0000 7f3104ff9700 1 -- 172.21.15.32:0/614053324 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30f8010430 con 0x7f31080ffee0 2023-09-30T14:51:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.617+0000 7f310db00700 1 -- 172.21.15.32:0/614053324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310819ecf0 con 0x7f31080ffee0 2023-09-30T14:51:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.617+0000 7f3104ff9700 1 -- 172.21.15.32:0/614053324 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f8018680 con 0x7f31080ffee0 2023-09-30T14:51:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.617+0000 7f310db00700 1 -- 172.21.15.32:0/614053324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310819f1b0 con 0x7f31080ffee0 2023-09-30T14:51:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.620+0000 7f3104ff9700 1 -- 172.21.15.32:0/614053324 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f30f80105a0 con 0x7f31080ffee0 2023-09-30T14:51:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.620+0000 7f3104ff9700 1 --2- 172.21.15.32:0/614053324 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f30f00a0fa0 0x7f30f00a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.621+0000 7f3106ffd700 1 --2- 172.21.15.32:0/614053324 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f30f00a0fa0 0x7f30f00a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.621+0000 7f3104ff9700 1 -- 172.21.15.32:0/614053324 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f30f8014070 con 0x7f31080ffee0 2023-09-30T14:51:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.621+0000 7f3106ffd700 1 --2- 172.21.15.32:0/614053324 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f30f00a0fa0 0x7f30f00a3460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009c60 tx=0x7f30fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.621+0000 7f3119ce8b80 1 Processor -- start 2023-09-30T14:51:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.621+0000 7f3119ce8b80 1 -- start start 2023-09-30T14:51:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.621+0000 7f3119ce8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df96610 0x55b35df96a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.621+0000 7f3119ce8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df97720 0x55b35df97ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.621+0000 7f3119ce8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df980e0 0x55b35df9e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.621+0000 7f3119ce8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35defa400 con 0x55b35df97720 2023-09-30T14:51:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f3119ce8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35deebbc0 con 0x55b35df96610 2023-09-30T14:51:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f3119ce8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35dee94e0 con 0x55b35df980e0 2023-09-30T14:51:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f3106ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df97720 0x55b35df97ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f3106ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df97720 0x55b35df97ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38278/0 (socket says 172.21.15.32:38278) 2023-09-30T14:51:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f3106ffd700 1 -- 172.21.15.32:0/2046429332 learned_addr learned my addr 172.21.15.32:0/2046429332 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f31077fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df96610 0x55b35df96a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f3107fff700 1 --2- 172.21.15.32:0/2046429332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df980e0 0x55b35df9e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f3106ffd700 1 -- 172.21.15.32:0/2046429332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df980e0 msgr2=0x55b35df9e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f3106ffd700 1 --2- 172.21.15.32:0/2046429332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df980e0 0x55b35df9e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f3106ffd700 1 -- 172.21.15.32:0/2046429332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df96610 msgr2=0x55b35df96a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f3106ffd700 1 --2- 172.21.15.32:0/2046429332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df96610 0x55b35df96a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f3106ffd700 1 -- 172.21.15.32:0/2046429332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b35df47870 con 0x55b35df97720 2023-09-30T14:51:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f31077fe700 1 --2- 172.21.15.32:0/2046429332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df96610 0x55b35df96a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f3106ffd700 1 --2- 172.21.15.32:0/2046429332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df97720 0x55b35df97ba0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00e4e0 tx=0x7f30fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976f9284de3cd053 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f30edffb700 1 -- 172.21.15.32:0/2046429332 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc00ee00 con 0x55b35df97720 2023-09-30T14:51:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f30edffb700 1 -- 172.21.15.32:0/2046429332 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30fc021070 con 0x55b35df97720 2023-09-30T14:51:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.622+0000 7f30edffb700 1 -- 172.21.15.32:0/2046429332 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc01d810 con 0x55b35df97720 2023-09-30T14:51:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.623+0000 7f3119ce8b80 1 -- 172.21.15.32:0/2046429332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df97720 msgr2=0x55b35df97ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.623+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/2046429332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df97720 0x55b35df97ba0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00e4e0 tx=0x7f30fc014000 comp rx=0 tx=0).stop 2023-09-30T14:51:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.623+0000 7f3119ce8b80 1 -- 172.21.15.32:0/2046429332 shutdown_connections 2023-09-30T14:51:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.623+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/2046429332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df980e0 0x55b35df9e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.623+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/2046429332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df96610 0x55b35df96a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.623+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/2046429332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df97720 0x55b35df97ba0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.623+0000 7f3119ce8b80 1 -- 172.21.15.32:0/2046429332 >> 172.21.15.32:0/2046429332 conn(0x55b35df06200 msgr2=0x55b35df8cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.623+0000 7f3119ce8b80 1 -- 172.21.15.32:0/2046429332 shutdown_connections 2023-09-30T14:51:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.623+0000 7f3119ce8b80 1 -- 172.21.15.32:0/2046429332 wait complete. 2023-09-30T14:51:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3119ce8b80 1 -- start start 2023-09-30T14:51:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3119ce8b80 1 Processor -- start 2023-09-30T14:51:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3119ce8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df96610 0x55b35dfa0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3119ce8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df97720 0x55b35dfa0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3119ce8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 0x55b35dfa78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3119ce8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35dee4dd0 con 0x55b35df980e0 2023-09-30T14:51:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3119ce8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35de816b0 con 0x55b35df96610 2023-09-30T14:51:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3119ce8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35de81350 con 0x55b35df97720 2023-09-30T14:51:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3119ce8b80 10 client.0 _ll_register_callbacks cb 0x55b35dcde7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:51:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3107fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 0x55b35dfa78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3106ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df97720 0x55b35dfa0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f31077fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df96610 0x55b35dfa0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3107fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 0x55b35dfa78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38286/0 (socket says 172.21.15.32:38286) 2023-09-30T14:51:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3106ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df97720 0x55b35dfa0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52666/0 (socket says 172.21.15.32:52666) 2023-09-30T14:51:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3107fff700 1 -- 172.21.15.32:0/1510403409 learned_addr learned my addr 172.21.15.32:0/1510403409 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3107fff700 1 -- 172.21.15.32:0/1510403409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df97720 msgr2=0x55b35dfa0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3107fff700 1 --2- 172.21.15.32:0/1510403409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df97720 0x55b35dfa0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3107fff700 1 -- 172.21.15.32:0/1510403409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df96610 msgr2=0x55b35dfa0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.624+0000 7f3107fff700 1 --2- 172.21.15.32:0/1510403409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df96610 0x55b35dfa0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f3107fff700 1 -- 172.21.15.32:0/1510403409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310819f1b0 con 0x55b35df980e0 2023-09-30T14:51:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f3106ffd700 1 --2- 172.21.15.32:0/1510403409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df97720 0x55b35dfa0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f3107fff700 1 --2- 172.21.15.32:0/1510403409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 0x55b35dfa78d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3100007710 tx=0x7f3100007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ca41eff6383bc6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f30e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:51:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f30e6ffd700 1 -- 172.21.15.32:0/1510403409 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310000e040 con 0x55b35df980e0 2023-09-30T14:51:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f3119ce8b80 1 -- 172.21.15.32:0/1510403409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b35df85090 con 0x55b35df980e0 2023-09-30T14:51:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f30e6ffd700 1 -- 172.21.15.32:0/1510403409 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f310000bc50 con 0x55b35df980e0 2023-09-30T14:51:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f30e6ffd700 1 -- 172.21.15.32:0/1510403409 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f310002c030 con 0x55b35df980e0 2023-09-30T14:51:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f3119ce8b80 10 client.16584 fetch_fsmap learned FSMap version 45 2023-09-30T14:51:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f3119ce8b80 1 -- 172.21.15.32:0/1510403409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f30fc017070 con 0x55b35df980e0 2023-09-30T14:51:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f30e6ffd700 1 -- 172.21.15.32:0/1510403409 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b35df85090 con 0x55b35df980e0 2023-09-30T14:51:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f30e6ffd700 1 -- 172.21.15.32:0/1510403409 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 45) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f3100019070 con 0x55b35df980e0 2023-09-30T14:51:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f3119ce8b80 10 client.16584 fetch_fsmap finished waiting for FSMap version 45 2023-09-30T14:51:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f3119ce8b80 10 client.16584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:51:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f3119ce8b80 20 client.16584 populate_metadata read hostname 'smithi032' 2023-09-30T14:51:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/1510403409 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b35dfac1f0 0x55b35dfae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f3119ce8b80 4 client.16584 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:51:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.625+0000 7f3119ce8b80 1 -- 172.21.15.32:0/1510403409 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b35dedddb0 con 0x55b35dfac1f0 2023-09-30T14:51:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.626+0000 7f31077fe700 1 --2- 172.21.15.32:0/1510403409 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b35dfac1f0 0x55b35dfae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.626+0000 7f31077fe700 1 --2- 172.21.15.32:0/1510403409 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b35dfac1f0 0x55b35dfae610 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=541f0d59b4d046e7 server_cookie=2a38559608152df7 in_seq=0 out_seq=0 2023-09-30T14:51:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.626+0000 7f30e6ffd700 10 client.16584 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.626+0000 7f30e6ffd700 1 -- 172.21.15.32:0/1510403409 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b35dedddb0 con 0x55b35dfac1f0 2023-09-30T14:51:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.626+0000 7f30e6ffd700 10 client.16584 handle_command_reply: tid=0 2023-09-30T14:51:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.627+0000 7f3119ce8b80 1 client.16584 shutdown 2023-09-30T14:51:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.627+0000 7f3119ce8b80 1 -- 172.21.15.32:0/1510403409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 msgr2=0x55b35dfa78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.627+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/1510403409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 0x55b35dfa78d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3100007710 tx=0x7f3100007740 comp rx=0 tx=0).stop 2023-09-30T14:51:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.627+0000 7f3119ce8b80 1 -- 172.21.15.32:0/1510403409 shutdown_connections 2023-09-30T14:51:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.627+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/1510403409 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b35dfac1f0 0x55b35dfae610 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.627+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/1510403409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 0x55b35dfa78d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.627+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/1510403409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df96610 0x55b35dfa0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.627+0000 7f30e6ffd700 0 client.16584 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.627+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/1510403409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df97720 0x55b35dfa0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.627+0000 7f3119ce8b80 1 -- 172.21.15.32:0/1510403409 >> 172.21.15.32:0/1510403409 conn(0x55b35df06200 msgr2=0x55b35df8c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.627+0000 7f3119ce8b80 1 -- 172.21.15.32:0/1510403409 shutdown_connections 2023-09-30T14:51:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.627+0000 7f3119ce8b80 1 -- 172.21.15.32:0/1510403409 wait complete. 2023-09-30T14:51:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.627+0000 7f3119ce8b80 20 client.16584 trim_cache size 0 max 16384 2023-09-30T14:51:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.642+0000 7f3119ce8b80 1 Processor -- start 2023-09-30T14:51:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.642+0000 7f3119ce8b80 1 -- start start 2023-09-30T14:51:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3119ce8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df96610 0x55b35df9e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3119ce8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df97720 0x55b35dfa77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3119ce8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 0x55b35df9b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3119ce8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35dd658c0 con 0x55b35df980e0 2023-09-30T14:51:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3119ce8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35dfa9590 con 0x55b35df96610 2023-09-30T14:51:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3119ce8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35df97540 con 0x55b35df97720 2023-09-30T14:51:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3107fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 0x55b35df9b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3106ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df97720 0x55b35dfa77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3106ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df97720 0x55b35dfa77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52672/0 (socket says 172.21.15.32:52672) 2023-09-30T14:51:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3106ffd700 1 -- 172.21.15.32:0/1506403206 learned_addr learned my addr 172.21.15.32:0/1506403206 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3107fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 0x55b35df9b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38292/0 (socket says 172.21.15.32:38292) 2023-09-30T14:51:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f31077fe700 1 --2- 172.21.15.32:0/1506403206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df96610 0x55b35df9e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3106ffd700 1 -- 172.21.15.32:0/1506403206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df96610 msgr2=0x55b35df9e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3106ffd700 1 --2- 172.21.15.32:0/1506403206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df96610 0x55b35df9e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3106ffd700 1 -- 172.21.15.32:0/1506403206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 msgr2=0x55b35df9b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3106ffd700 1 --2- 172.21.15.32:0/1506403206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 0x55b35df9b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3106ffd700 1 -- 172.21.15.32:0/1506403206 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b35dd0fd80 con 0x55b35df97720 2023-09-30T14:51:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.643+0000 7f3106ffd700 1 --2- 172.21.15.32:0/1506403206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df97720 0x55b35dfa77f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f30fc005950 tx=0x7f30fc0092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=378a9566f883a10b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.644+0000 7f30ee7fc700 1 -- 172.21.15.32:0/1506403206 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc00ed00 con 0x55b35df97720 2023-09-30T14:51:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.644+0000 7f30ee7fc700 1 -- 172.21.15.32:0/1506403206 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30fc00ee60 con 0x55b35df97720 2023-09-30T14:51:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.644+0000 7f30ee7fc700 1 -- 172.21.15.32:0/1506403206 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc003dd0 con 0x55b35df97720 2023-09-30T14:51:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.644+0000 7f3119ce8b80 1 -- 172.21.15.32:0/1506403206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df97720 msgr2=0x55b35dfa77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.644+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/1506403206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df97720 0x55b35dfa77f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f30fc005950 tx=0x7f30fc0092d0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.644+0000 7f3119ce8b80 1 -- 172.21.15.32:0/1506403206 shutdown_connections 2023-09-30T14:51:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.644+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/1506403206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 0x55b35df9b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.644+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/1506403206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df96610 0x55b35df9e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.644+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/1506403206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df97720 0x55b35dfa77f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.644+0000 7f3119ce8b80 1 -- 172.21.15.32:0/1506403206 >> 172.21.15.32:0/1506403206 conn(0x55b35df06200 msgr2=0x55b35df9f020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.644+0000 7f3119ce8b80 1 -- 172.21.15.32:0/1506403206 shutdown_connections 2023-09-30T14:51:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.644+0000 7f3119ce8b80 1 -- 172.21.15.32:0/1506403206 wait complete. 2023-09-30T14:51:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f3119ce8b80 1 -- start start 2023-09-30T14:51:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f3119ce8b80 1 Processor -- start 2023-09-30T14:51:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f3119ce8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df96610 0x55b35dfa5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f3119ce8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df97720 0x55b35dfa5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f3119ce8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 0x55b35dfa6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f3119ce8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35dfa72c0 con 0x55b35df980e0 2023-09-30T14:51:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f3119ce8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35dfa7430 con 0x55b35df97720 2023-09-30T14:51:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f3119ce8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b35dfa75f0 con 0x55b35df96610 2023-09-30T14:51:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f3119ce8b80 10 client.0 _ll_register_callbacks cb 0x7f3100007bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:51:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f3106ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df97720 0x55b35dfa5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f31077fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df96610 0x55b35dfa5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f3107fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 0x55b35dfa6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f31077fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df96610 0x55b35dfa5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52676/0 (socket says 172.21.15.32:52676) 2023-09-30T14:51:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f31077fe700 1 -- 172.21.15.32:0/3202287205 learned_addr learned my addr 172.21.15.32:0/3202287205 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f3107fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 0x55b35dfa6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38298/0 (socket says 172.21.15.32:38298) 2023-09-30T14:51:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f3106ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df97720 0x55b35dfa5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49772/0 (socket says 172.21.15.32:49772) 2023-09-30T14:51:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.645+0000 7f31077fe700 1 -- 172.21.15.32:0/3202287205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df97720 msgr2=0x55b35dfa5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f31077fe700 1 --2- 172.21.15.32:0/3202287205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df97720 0x55b35dfa5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f31077fe700 1 -- 172.21.15.32:0/3202287205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 msgr2=0x55b35dfa6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f31077fe700 1 --2- 172.21.15.32:0/3202287205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 0x55b35dfa6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f31077fe700 1 -- 172.21.15.32:0/3202287205 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc02a040 con 0x55b35df96610 2023-09-30T14:51:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f3106ffd700 1 --2- 172.21.15.32:0/3202287205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df97720 0x55b35dfa5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f31077fe700 1 --2- 172.21.15.32:0/3202287205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df96610 0x55b35dfa5830 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f30f800e970 tx=0x7f30f80cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=870d120cba05f32f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f30e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:51:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f30e6ffd700 1 -- 172.21.15.32:0/3202287205 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f8010880 con 0x55b35df96610 2023-09-30T14:51:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f3119ce8b80 1 -- 172.21.15.32:0/3202287205 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30fc021070 con 0x55b35df96610 2023-09-30T14:51:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f30e6ffd700 1 -- 172.21.15.32:0/3202287205 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f30f80b2240 con 0x55b35df96610 2023-09-30T14:51:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f30e6ffd700 1 -- 172.21.15.32:0/3202287205 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30f80109e0 con 0x55b35df96610 2023-09-30T14:51:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f3119ce8b80 10 client.26260 fetch_fsmap learned FSMap version 45 2023-09-30T14:51:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f3119ce8b80 1 -- 172.21.15.32:0/3202287205 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55b35dcf8c00 con 0x55b35df96610 2023-09-30T14:51:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f30e6ffd700 1 -- 172.21.15.32:0/3202287205 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30f807c080 con 0x55b35df96610 2023-09-30T14:51:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f30e6ffd700 1 -- 172.21.15.32:0/3202287205 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 45) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f30f80a1c80 con 0x55b35df96610 2023-09-30T14:51:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f3119ce8b80 10 client.26260 fetch_fsmap finished waiting for FSMap version 45 2023-09-30T14:51:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f3119ce8b80 10 client.26260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:51:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f3119ce8b80 20 client.26260 populate_metadata read hostname 'smithi032' 2023-09-30T14:51:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/3202287205 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b35e0012c0 0x55b35e0036e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f3119ce8b80 4 client.26260 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:51:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.646+0000 7f3119ce8b80 1 -- 172.21.15.32:0/3202287205 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b35deddad0 con 0x55b35e0012c0 2023-09-30T14:51:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.647+0000 7f3106ffd700 1 --2- 172.21.15.32:0/3202287205 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b35e0012c0 0x55b35e0036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.647+0000 7f30e6ffd700 10 client.26260 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.647+0000 7f3106ffd700 1 --2- 172.21.15.32:0/3202287205 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b35e0012c0 0x55b35e0036e0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94189357e28302e4 server_cookie=1219d6bd745b5f7f in_seq=0 out_seq=0 2023-09-30T14:51:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.703+0000 7f30e6ffd700 1 -- 172.21.15.32:0/3202287205 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b35deddad0 con 0x55b35e0012c0 2023-09-30T14:51:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.703+0000 7f30e6ffd700 10 client.26260 handle_command_reply: tid=0 2023-09-30T14:51:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.703+0000 7f3119ce8b80 1 client.26260 shutdown 2023-09-30T14:51:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.703+0000 7f3119ce8b80 1 -- 172.21.15.32:0/3202287205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df96610 msgr2=0x55b35dfa5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.703+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/3202287205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df96610 0x55b35dfa5830 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f30f800e970 tx=0x7f30f80cc040 comp rx=0 tx=0).stop 2023-09-30T14:51:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.703+0000 7f3119ce8b80 1 -- 172.21.15.32:0/3202287205 shutdown_connections 2023-09-30T14:51:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.703+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/3202287205 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b35e0012c0 0x55b35e0036e0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.703+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/3202287205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b35df980e0 0x55b35dfa6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.703+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/3202287205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b35df96610 0x55b35dfa5830 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.703+0000 7f30e6ffd700 0 client.26260 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.703+0000 7f3119ce8b80 1 --2- 172.21.15.32:0/3202287205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b35df97720 0x55b35dfa5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.703+0000 7f3119ce8b80 1 -- 172.21.15.32:0/3202287205 >> 172.21.15.32:0/3202287205 conn(0x55b35df06200 msgr2=0x55b35df9e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.703+0000 7f3119ce8b80 1 -- 172.21.15.32:0/3202287205 shutdown_connections 2023-09-30T14:51:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.703+0000 7f3119ce8b80 1 -- 172.21.15.32:0/3202287205 wait complete. 2023-09-30T14:51:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.703+0000 7f3119ce8b80 20 client.26260 trim_cache size 0 max 16384 2023-09-30T14:51:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:51:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:51:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:51:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:51:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.704+0000 7f310db00700 1 -- 172.21.15.32:0/614053324 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f30f00a0fa0 msgr2=0x7f30f00a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.704+0000 7f310db00700 1 --2- 172.21.15.32:0/614053324 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f30f00a0fa0 0x7f30f00a3460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009c60 tx=0x7f30fc009260 comp rx=0 tx=0).stop 2023-09-30T14:51:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.704+0000 7f310db00700 1 -- 172.21.15.32:0/614053324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31080ffee0 msgr2=0x7f3108196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.704+0000 7f310db00700 1 --2- 172.21.15.32:0/614053324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31080ffee0 0x7f3108196bc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f30f800e9a0 tx=0x7f30f800ecb0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.704+0000 7f310db00700 1 -- 172.21.15.32:0/614053324 shutdown_connections 2023-09-30T14:51:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.704+0000 7f310db00700 1 --2- 172.21.15.32:0/614053324 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f30f00a0fa0 0x7f30f00a3460 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.704+0000 7f310db00700 1 --2- 172.21.15.32:0/614053324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3108101290 0x7f310819e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.704+0000 7f310db00700 1 --2- 172.21.15.32:0/614053324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31080ffee0 0x7f3108196bc0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.704+0000 7f310db00700 1 --2- 172.21.15.32:0/614053324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31081008a0 0x7f3108197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.704+0000 7f310db00700 1 -- 172.21.15.32:0/614053324 >> 172.21.15.32:0/614053324 conn(0x7f31080fa9b0 msgr2=0x7f31080fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.704+0000 7f310db00700 1 -- 172.21.15.32:0/614053324 shutdown_connections 2023-09-30T14:51:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:17.704+0000 7f310db00700 1 -- 172.21.15.32:0/614053324 wait complete. 2023-09-30T14:51:17.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1431193 10.64 MB/sec execute 591 sec latency 23.208 ms 2023-09-30T14:51:17.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:17 smithi032 ceph-mon[98592]: pgmap v677: 97 pgs: 97 active+clean; 372 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 2.9 MiB/s wr, 281 op/s 2023-09-30T14:51:18.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:17 smithi156 ceph-mon[103579]: pgmap v677: 97 pgs: 97 active+clean; 372 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 2.9 MiB/s wr, 281 op/s 2023-09-30T14:51:18.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:17 smithi148 ceph-mon[102355]: pgmap v677: 97 pgs: 97 active+clean; 372 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 2.9 MiB/s wr, 281 op/s 2023-09-30T14:51:18.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1432602 10.64 MB/sec execute 592 sec latency 56.488 ms 2023-09-30T14:51:19.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1435320 10.65 MB/sec execute 593 sec latency 19.886 ms 2023-09-30T14:51:20.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:19 smithi156 ceph-mon[103579]: pgmap v678: 97 pgs: 97 active+clean; 374 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 2.7 MiB/s wr, 257 op/s 2023-09-30T14:51:20.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:19 smithi148 ceph-mon[102355]: pgmap v678: 97 pgs: 97 active+clean; 374 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 2.7 MiB/s wr, 257 op/s 2023-09-30T14:51:20.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:19 smithi032 ceph-mon[98592]: pgmap v678: 97 pgs: 97 active+clean; 374 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 2.7 MiB/s wr, 257 op/s 2023-09-30T14:51:20.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1436970 10.64 MB/sec execute 594 sec latency 15.599 ms 2023-09-30T14:51:21.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1439288 10.65 MB/sec execute 595 sec latency 11.541 ms 2023-09-30T14:51:22.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:21 smithi156 ceph-mon[103579]: pgmap v679: 97 pgs: 97 active+clean; 378 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 2.8 MiB/s wr, 218 op/s 2023-09-30T14:51:22.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:21 smithi148 ceph-mon[102355]: pgmap v679: 97 pgs: 97 active+clean; 378 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 2.8 MiB/s wr, 218 op/s 2023-09-30T14:51:22.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:21 smithi032 ceph-mon[98592]: pgmap v679: 97 pgs: 97 active+clean; 378 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 2.8 MiB/s wr, 218 op/s 2023-09-30T14:51:22.728 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1441680 10.65 MB/sec execute 596 sec latency 14.169 ms 2023-09-30T14:51:23.727 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1443237 10.65 MB/sec execute 597 sec latency 26.748 ms 2023-09-30T14:51:24.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:23 smithi156 ceph-mon[103579]: pgmap v680: 97 pgs: 97 active+clean; 387 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 3.7 MiB/s wr, 261 op/s 2023-09-30T14:51:24.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:23 smithi148 ceph-mon[102355]: pgmap v680: 97 pgs: 97 active+clean; 387 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 3.7 MiB/s wr, 261 op/s 2023-09-30T14:51:24.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:23 smithi032 ceph-mon[98592]: pgmap v680: 97 pgs: 97 active+clean; 387 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 3.7 MiB/s wr, 261 op/s 2023-09-30T14:51:24.728 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1445646 10.65 MB/sec execute 598 sec latency 21.044 ms 2023-09-30T14:51:25.728 INFO:tasks.workunit.client.0.smithi032.stdout: 1 1447380 10.65 MB/sec execute 599 sec latency 14.977 ms 2023-09-30T14:51:26.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:25 smithi156 ceph-mon[103579]: pgmap v681: 97 pgs: 97 active+clean; 396 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 4.2 MiB/s wr, 284 op/s 2023-09-30T14:51:26.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:25 smithi148 ceph-mon[102355]: pgmap v681: 97 pgs: 97 active+clean; 396 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 4.2 MiB/s wr, 284 op/s 2023-09-30T14:51:26.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:25 smithi032 ceph-mon[98592]: pgmap v681: 97 pgs: 97 active+clean; 396 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 4.2 MiB/s wr, 284 op/s 2023-09-30T14:51:26.728 INFO:tasks.workunit.client.0.smithi032.stdout: 1 cleanup 600 sec 2023-09-30T14:51:26.893 INFO:tasks.workunit.client.0.smithi032.stdout: 0 cleanup 600 sec 2023-09-30T14:51:26.893 INFO:tasks.workunit.client.0.smithi032.stdout: 2023-09-30T14:51:26.893 INFO:tasks.workunit.client.0.smithi032.stdout: Operation Count AvgLat MaxLat 2023-09-30T14:51:26.893 INFO:tasks.workunit.client.0.smithi032.stdout: ---------------------------------------- 2023-09-30T14:51:26.893 INFO:tasks.workunit.client.0.smithi032.stdout: NTCreateX 203307 1.246 2331.625 2023-09-30T14:51:26.893 INFO:tasks.workunit.client.0.smithi032.stdout: Close 149369 0.002 0.086 2023-09-30T14:51:26.893 INFO:tasks.workunit.client.0.smithi032.stdout: Rename 8609 5.838 4915.133 2023-09-30T14:51:26.893 INFO:tasks.workunit.client.0.smithi032.stdout: Unlink 41044 1.652 4576.491 2023-09-30T14:51:26.894 INFO:tasks.workunit.client.0.smithi032.stdout: Deltree 6 87.512 133.503 2023-09-30T14:51:26.894 INFO:tasks.workunit.client.0.smithi032.stdout: Mkdir 3 0.007 0.009 2023-09-30T14:51:26.894 INFO:tasks.workunit.client.0.smithi032.stdout: Qpathinfo 184262 0.883 95.104 2023-09-30T14:51:26.895 INFO:tasks.workunit.client.0.smithi032.stdout: Qfileinfo 32323 0.002 0.036 2023-09-30T14:51:26.895 INFO:tasks.workunit.client.0.smithi032.stdout: Qfsinfo 33790 0.187 16.178 2023-09-30T14:51:26.895 INFO:tasks.workunit.client.0.smithi032.stdout: Sfileinfo 16562 0.017 1.558 2023-09-30T14:51:26.895 INFO:tasks.workunit.client.0.smithi032.stdout: Find 71253 0.147 96.360 2023-09-30T14:51:26.895 INFO:tasks.workunit.client.0.smithi032.stdout: WriteX 101532 0.012 1.847 2023-09-30T14:51:26.895 INFO:tasks.workunit.client.0.smithi032.stdout: ReadX 318705 0.004 24.684 2023-09-30T14:51:26.895 INFO:tasks.workunit.client.0.smithi032.stdout: LockX 662 0.599 9.579 2023-09-30T14:51:26.895 INFO:tasks.workunit.client.0.smithi032.stdout: UnlockX 662 0.580 7.620 2023-09-30T14:51:26.895 INFO:tasks.workunit.client.0.smithi032.stdout: Flush 14251 2.946 59.525 2023-09-30T14:51:26.895 INFO:tasks.workunit.client.0.smithi032.stdout: 2023-09-30T14:51:26.895 INFO:tasks.workunit.client.0.smithi032.stdout:Throughput 10.6497 MB/sec 1 clients 1 procs max_latency=4915.149 ms 2023-09-30T14:51:26.896 INFO:tasks.workunit.client.0.smithi032.stderr:+ dbench 10 2023-09-30T14:51:26.896 INFO:tasks.workunit.client.0.smithi032.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-09-30T14:51:26.896 INFO:tasks.workunit.client.0.smithi032.stdout: 2023-09-30T14:51:26.896 INFO:tasks.workunit.client.0.smithi032.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-09-30T14:51:26.896 INFO:tasks.workunit.client.0.smithi032.stdout:8 of 10 processes prepared for launch 0 sec 2023-09-30T14:51:26.994 INFO:tasks.workunit.client.0.smithi032.stdout:10 of 10 processes prepared for launch 0 sec 2023-09-30T14:51:26.995 INFO:tasks.workunit.client.0.smithi032.stdout:releasing clients 2023-09-30T14:51:27.994 INFO:tasks.workunit.client.0.smithi032.stdout: 10 356 127.85 MB/sec warmup 1 sec latency 151.302 ms 2023-09-30T14:51:28.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:27 smithi156 ceph-mon[103579]: pgmap v682: 97 pgs: 97 active+clean; 396 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 4.1 MiB/s wr, 259 op/s 2023-09-30T14:51:28.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:27 smithi148 ceph-mon[102355]: pgmap v682: 97 pgs: 97 active+clean; 396 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 4.1 MiB/s wr, 259 op/s 2023-09-30T14:51:28.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:27 smithi032 ceph-mon[98592]: pgmap v682: 97 pgs: 97 active+clean; 396 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 4.1 MiB/s wr, 259 op/s 2023-09-30T14:51:28.994 INFO:tasks.workunit.client.0.smithi032.stdout: 10 718 104.58 MB/sec warmup 2 sec latency 167.600 ms 2023-09-30T14:51:29.995 INFO:tasks.workunit.client.0.smithi032.stdout: 10 1190 76.77 MB/sec warmup 3 sec latency 57.019 ms 2023-09-30T14:51:30.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:29 smithi156 ceph-mon[103579]: pgmap v683: 97 pgs: 97 active+clean; 399 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 4.2 MiB/s wr, 290 op/s 2023-09-30T14:51:30.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:29 smithi148 ceph-mon[102355]: pgmap v683: 97 pgs: 97 active+clean; 399 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 4.2 MiB/s wr, 290 op/s 2023-09-30T14:51:30.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:29 smithi032 ceph-mon[98592]: pgmap v683: 97 pgs: 97 active+clean; 399 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 4.2 MiB/s wr, 290 op/s 2023-09-30T14:51:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.825+0000 7fa76946b700 1 -- 172.21.15.32:0/2510401449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7640ff4e0 msgr2=0x7fa7640ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.825+0000 7fa76946b700 1 --2- 172.21.15.32:0/2510401449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7640ff4e0 0x7fa7640ff900 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa7540099c0 tx=0x7fa754009cd0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.825+0000 7fa76946b700 1 -- 172.21.15.32:0/2510401449 shutdown_connections 2023-09-30T14:51:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.825+0000 7fa76946b700 1 --2- 172.21.15.32:0/2510401449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa764101070 0x7fa764107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.825+0000 7fa76946b700 1 --2- 172.21.15.32:0/2510401449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7640ff4e0 0x7fa7640ff900 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.825+0000 7fa76946b700 1 --2- 172.21.15.32:0/2510401449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7641006d0 0x7fa764100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.825+0000 7fa76946b700 1 -- 172.21.15.32:0/2510401449 >> 172.21.15.32:0/2510401449 conn(0x7fa7640faa30 msgr2=0x7fa7640fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.825+0000 7fa76946b700 1 -- 172.21.15.32:0/2510401449 shutdown_connections 2023-09-30T14:51:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.825+0000 7fa76946b700 1 -- 172.21.15.32:0/2510401449 wait complete. 2023-09-30T14:51:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.826+0000 7fa76946b700 1 Processor -- start 2023-09-30T14:51:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.826+0000 7fa76946b700 1 -- start start 2023-09-30T14:51:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.826+0000 7fa76946b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7640ff4e0 0x7fa7641949f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.826+0000 7fa76946b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7641006d0 0x7fa764194f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.826+0000 7fa76946b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa764101070 0x7fa764195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.826+0000 7fa76946b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa764196410 con 0x7fa7641006d0 2023-09-30T14:51:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.826+0000 7fa76946b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa764196580 con 0x7fa7640ff4e0 2023-09-30T14:51:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.826+0000 7fa76946b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa764196740 con 0x7fa764101070 2023-09-30T14:51:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa7627fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7641006d0 0x7fa764194f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa7627fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7641006d0 0x7fa764194f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38310/0 (socket says 172.21.15.32:38310) 2023-09-30T14:51:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa7627fc700 1 -- 172.21.15.32:0/1532663700 learned_addr learned my addr 172.21.15.32:0/1532663700 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa762ffd700 1 --2- 172.21.15.32:0/1532663700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7640ff4e0 0x7fa7641949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa7637fe700 1 --2- 172.21.15.32:0/1532663700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa764101070 0x7fa764195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa7627fc700 1 -- 172.21.15.32:0/1532663700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa764101070 msgr2=0x7fa764195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa7627fc700 1 --2- 172.21.15.32:0/1532663700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa764101070 0x7fa764195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa7627fc700 1 -- 172.21.15.32:0/1532663700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7640ff4e0 msgr2=0x7fa7641949f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa7627fc700 1 --2- 172.21.15.32:0/1532663700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7640ff4e0 0x7fa7641949f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa7627fc700 1 -- 172.21.15.32:0/1532663700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75800a630 con 0x7fa7641006d0 2023-09-30T14:51:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa762ffd700 1 --2- 172.21.15.32:0/1532663700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7640ff4e0 0x7fa7641949f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa7627fc700 1 --2- 172.21.15.32:0/1532663700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7641006d0 0x7fa764194f30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa75800fec0 tx=0x7fa75800d6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa74bfff700 1 -- 172.21.15.32:0/1532663700 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758004220 con 0x7fa7641006d0 2023-09-30T14:51:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa76946b700 1 -- 172.21.15.32:0/1532663700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7540095f0 con 0x7fa7641006d0 2023-09-30T14:51:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa74bfff700 1 -- 172.21.15.32:0/1532663700 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa758004380 con 0x7fa7641006d0 2023-09-30T14:51:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa74bfff700 1 -- 172.21.15.32:0/1532663700 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7580178a0 con 0x7fa7641006d0 2023-09-30T14:51:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.827+0000 7fa76946b700 1 -- 172.21.15.32:0/1532663700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76419d000 con 0x7fa7641006d0 2023-09-30T14:51:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.830+0000 7fa74bfff700 1 -- 172.21.15.32:0/1532663700 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fa758017a40 con 0x7fa7641006d0 2023-09-30T14:51:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.831+0000 7fa74bfff700 1 --2- 172.21.15.32:0/1532663700 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa74c0a0fa0 0x7fa74c0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.831+0000 7fa74bfff700 1 -- 172.21.15.32:0/1532663700 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa7580cdb80 con 0x7fa7641006d0 2023-09-30T14:51:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.831+0000 7fa762ffd700 1 --2- 172.21.15.32:0/1532663700 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa74c0a0fa0 0x7fa74c0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa775653b80 1 Processor -- start 2023-09-30T14:51:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa775653b80 1 -- start start 2023-09-30T14:51:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa775653b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c3200 0x562c5e6c3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa775653b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c4310 0x562c5e6c4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa775653b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4cd0 0x562c5e6cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa775653b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5e62b400 con 0x562c5e6c4310 2023-09-30T14:51:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa775653b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5e61cbc0 con 0x562c5e6c4cd0 2023-09-30T14:51:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa775653b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5e61a4e0 con 0x562c5e6c3200 2023-09-30T14:51:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa7627fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c4310 0x562c5e6c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa7627fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c4310 0x562c5e6c4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38316/0 (socket says 172.21.15.32:38316) 2023-09-30T14:51:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa7627fc700 1 -- 172.21.15.32:0/3037924617 learned_addr learned my addr 172.21.15.32:0/3037924617 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa7637fe700 1 --2- 172.21.15.32:0/3037924617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4cd0 0x562c5e6cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa7627fc700 1 -- 172.21.15.32:0/3037924617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c3200 msgr2=0x562c5e6c3620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T14:51:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa7627fc700 1 --2- 172.21.15.32:0/3037924617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c3200 0x562c5e6c3620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa7627fc700 1 -- 172.21.15.32:0/3037924617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4cd0 msgr2=0x562c5e6cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa7627fc700 1 --2- 172.21.15.32:0/3037924617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4cd0 0x562c5e6cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.832+0000 7fa7627fc700 1 -- 172.21.15.32:0/3037924617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c5e678870 con 0x562c5e6c4310 2023-09-30T14:51:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.833+0000 7fa7627fc700 1 --2- 172.21.15.32:0/3037924617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c4310 0x562c5e6c4790 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa7580a2eb0 tx=0x7fa75809aac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d277e1f22f4f32a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.833+0000 7fa762ffd700 1 --2- 172.21.15.32:0/1532663700 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa74c0a0fa0 0x7fa74c0a3460 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa754009990 tx=0x7fa75401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.833+0000 7fa7497fa700 1 -- 172.21.15.32:0/3037924617 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75809b050 con 0x562c5e6c4310 2023-09-30T14:51:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.833+0000 7fa7497fa700 1 -- 172.21.15.32:0/3037924617 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa75809f1a0 con 0x562c5e6c4310 2023-09-30T14:51:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.833+0000 7fa7497fa700 1 -- 172.21.15.32:0/3037924617 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7580a2f50 con 0x562c5e6c4310 2023-09-30T14:51:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.834+0000 7fa775653b80 1 -- 172.21.15.32:0/3037924617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c4310 msgr2=0x562c5e6c4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.834+0000 7fa775653b80 1 --2- 172.21.15.32:0/3037924617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c4310 0x562c5e6c4790 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa7580a2eb0 tx=0x7fa75809aac0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.834+0000 7fa775653b80 1 -- 172.21.15.32:0/3037924617 shutdown_connections 2023-09-30T14:51:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.834+0000 7fa775653b80 1 --2- 172.21.15.32:0/3037924617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4cd0 0x562c5e6cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.834+0000 7fa775653b80 1 --2- 172.21.15.32:0/3037924617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c3200 0x562c5e6c3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.834+0000 7fa775653b80 1 --2- 172.21.15.32:0/3037924617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c4310 0x562c5e6c4790 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.834+0000 7fa775653b80 1 -- 172.21.15.32:0/3037924617 >> 172.21.15.32:0/3037924617 conn(0x562c5e637200 msgr2=0x562c5e6bdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.834+0000 7fa775653b80 1 -- 172.21.15.32:0/3037924617 shutdown_connections 2023-09-30T14:51:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.834+0000 7fa775653b80 1 -- 172.21.15.32:0/3037924617 wait complete. 2023-09-30T14:51:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.834+0000 7fa775653b80 1 -- start start 2023-09-30T14:51:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.834+0000 7fa775653b80 1 Processor -- start 2023-09-30T14:51:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa775653b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c3200 0x562c5e6e0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa775653b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4310 0x562c5e6e05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa775653b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c4cd0 0x562c5e6e75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa775653b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5e615dd0 con 0x562c5e6c3200 2023-09-30T14:51:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa775653b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5e5b26b0 con 0x562c5e6c4310 2023-09-30T14:51:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa775653b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5e5b2350 con 0x562c5e6c4cd0 2023-09-30T14:51:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa775653b80 10 client.0 _ll_register_callbacks cb 0x562c5e40f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:51:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa762ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c3200 0x562c5e6e0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa762ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c3200 0x562c5e6e0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38320/0 (socket says 172.21.15.32:38320) 2023-09-30T14:51:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa762ffd700 1 -- 172.21.15.32:0/1119477373 learned_addr learned my addr 172.21.15.32:0/1119477373 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa7627fc700 1 --2- 172.21.15.32:0/1119477373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4310 0x562c5e6e05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa7637fe700 1 --2- 172.21.15.32:0/1119477373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c4cd0 0x562c5e6e75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa762ffd700 1 -- 172.21.15.32:0/1119477373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c4cd0 msgr2=0x562c5e6e75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa762ffd700 1 --2- 172.21.15.32:0/1119477373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c4cd0 0x562c5e6e75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa762ffd700 1 -- 172.21.15.32:0/1119477373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4310 msgr2=0x562c5e6e05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa762ffd700 1 --2- 172.21.15.32:0/1119477373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4310 0x562c5e6e05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa762ffd700 1 -- 172.21.15.32:0/1119477373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76419d000 con 0x562c5e6c3200 2023-09-30T14:51:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa7637fe700 1 --2- 172.21.15.32:0/1119477373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c4cd0 0x562c5e6e75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa7627fc700 1 --2- 172.21.15.32:0/1119477373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4310 0x562c5e6e05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa762ffd700 1 --2- 172.21.15.32:0/1119477373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c3200 0x562c5e6e0070 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa7540099c0 tx=0x7fa754009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4268e088dd6d20f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.835+0000 7fa746ff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:51:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.836+0000 7fa746ff5700 1 -- 172.21.15.32:0/1119477373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75402a070 con 0x562c5e6c3200 2023-09-30T14:51:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.836+0000 7fa746ff5700 1 -- 172.21.15.32:0/1119477373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa754005310 con 0x562c5e6c3200 2023-09-30T14:51:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.836+0000 7fa775653b80 1 -- 172.21.15.32:0/1119477373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c5e6b6090 con 0x562c5e6c3200 2023-09-30T14:51:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.836+0000 7fa746ff5700 1 -- 172.21.15.32:0/1119477373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7540054d0 con 0x562c5e6c3200 2023-09-30T14:51:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.836+0000 7fa775653b80 10 client.16602 fetch_fsmap learned FSMap version 45 2023-09-30T14:51:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.836+0000 7fa775653b80 1 -- 172.21.15.32:0/1119477373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x562c5e440d80 con 0x562c5e6c3200 2023-09-30T14:51:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.836+0000 7fa746ff5700 1 -- 172.21.15.32:0/1119477373 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562c5e6b6090 con 0x562c5e6c3200 2023-09-30T14:51:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.836+0000 7fa746ff5700 1 -- 172.21.15.32:0/1119477373 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 45) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fa754024420 con 0x562c5e6c3200 2023-09-30T14:51:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.836+0000 7fa775653b80 10 client.16602 fetch_fsmap finished waiting for FSMap version 45 2023-09-30T14:51:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.836+0000 7fa775653b80 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:51:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.836+0000 7fa775653b80 20 client.16602 populate_metadata read hostname 'smithi032' 2023-09-30T14:51:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.836+0000 7fa775653b80 1 --2- 172.21.15.32:0/1119477373 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562c5e6ebec0 0x562c5e6ee2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.836+0000 7fa775653b80 4 client.16602 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:51:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.836+0000 7fa775653b80 1 -- 172.21.15.32:0/1119477373 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c5e60edb0 con 0x562c5e6ebec0 2023-09-30T14:51:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.836+0000 7fa7627fc700 1 --2- 172.21.15.32:0/1119477373 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562c5e6ebec0 0x562c5e6ee2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.837+0000 7fa746ff5700 10 client.16602 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.837+0000 7fa7627fc700 1 --2- 172.21.15.32:0/1119477373 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562c5e6ebec0 0x562c5e6ee2e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea99826ede2243dd server_cookie=3ae097e9f2abf7e in_seq=0 out_seq=0 2023-09-30T14:51:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.837+0000 7fa746ff5700 1 -- 172.21.15.32:0/1119477373 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562c5e60edb0 con 0x562c5e6ebec0 2023-09-30T14:51:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.837+0000 7fa746ff5700 10 client.16602 handle_command_reply: tid=0 2023-09-30T14:51:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.838+0000 7fa775653b80 1 client.16602 shutdown 2023-09-30T14:51:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.838+0000 7fa775653b80 1 -- 172.21.15.32:0/1119477373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c3200 msgr2=0x562c5e6e0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.838+0000 7fa775653b80 1 --2- 172.21.15.32:0/1119477373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c3200 0x562c5e6e0070 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa7540099c0 tx=0x7fa754009310 comp rx=0 tx=0).stop 2023-09-30T14:51:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.838+0000 7fa775653b80 1 -- 172.21.15.32:0/1119477373 shutdown_connections 2023-09-30T14:51:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.838+0000 7fa775653b80 1 --2- 172.21.15.32:0/1119477373 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562c5e6ebec0 0x562c5e6ee2e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.838+0000 7fa775653b80 1 --2- 172.21.15.32:0/1119477373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c4cd0 0x562c5e6e75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.838+0000 7fa775653b80 1 --2- 172.21.15.32:0/1119477373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c3200 0x562c5e6e0070 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.838+0000 7fa775653b80 1 --2- 172.21.15.32:0/1119477373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4310 0x562c5e6e05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.838+0000 7fa746ff5700 0 client.16602 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.838+0000 7fa775653b80 1 -- 172.21.15.32:0/1119477373 >> 172.21.15.32:0/1119477373 conn(0x562c5e637200 msgr2=0x562c5e6bd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.838+0000 7fa775653b80 1 -- 172.21.15.32:0/1119477373 shutdown_connections 2023-09-30T14:51:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.838+0000 7fa775653b80 1 -- 172.21.15.32:0/1119477373 wait complete. 2023-09-30T14:51:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.838+0000 7fa775653b80 20 client.16602 trim_cache size 0 max 16384 2023-09-30T14:51:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa775653b80 1 Processor -- start 2023-09-30T14:51:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa775653b80 1 -- start start 2023-09-30T14:51:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa775653b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c3200 0x562c5e6bd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa775653b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c4310 0x562c5e6cb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa775653b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4cd0 0x562c5e6c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa775653b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5e4968c0 con 0x562c5e6c4310 2023-09-30T14:51:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa775653b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5e6c8800 con 0x562c5e6c4cd0 2023-09-30T14:51:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa775653b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5e6b5bd0 con 0x562c5e6c3200 2023-09-30T14:51:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa7627fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c4310 0x562c5e6cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa762ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c3200 0x562c5e6bd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa7627fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c4310 0x562c5e6cb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38330/0 (socket says 172.21.15.32:38330) 2023-09-30T14:51:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa7627fc700 1 -- 172.21.15.32:0/3922329794 learned_addr learned my addr 172.21.15.32:0/3922329794 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa7637fe700 1 --2- 172.21.15.32:0/3922329794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4cd0 0x562c5e6c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa762ffd700 1 -- 172.21.15.32:0/3922329794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4cd0 msgr2=0x562c5e6c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa762ffd700 1 --2- 172.21.15.32:0/3922329794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4cd0 0x562c5e6c7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa762ffd700 1 -- 172.21.15.32:0/3922329794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c4310 msgr2=0x562c5e6cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa762ffd700 1 --2- 172.21.15.32:0/3922329794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c4310 0x562c5e6cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa762ffd700 1 -- 172.21.15.32:0/3922329794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c5e429c00 con 0x562c5e6c3200 2023-09-30T14:51:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.860+0000 7fa7627fc700 1 --2- 172.21.15.32:0/3922329794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c4310 0x562c5e6cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.861+0000 7fa762ffd700 1 --2- 172.21.15.32:0/3922329794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c3200 0x562c5e6bd590 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa754024780 tx=0x7fa754009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdb450a49b6a674 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.861+0000 7fa749ffb700 1 -- 172.21.15.32:0/3922329794 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75402a040 con 0x562c5e6c3200 2023-09-30T14:51:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.861+0000 7fa749ffb700 1 -- 172.21.15.32:0/3922329794 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa75401e040 con 0x562c5e6c3200 2023-09-30T14:51:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.861+0000 7fa749ffb700 1 -- 172.21.15.32:0/3922329794 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75403a950 con 0x562c5e6c3200 2023-09-30T14:51:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.862+0000 7fa775653b80 1 -- 172.21.15.32:0/3922329794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c3200 msgr2=0x562c5e6bd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.862+0000 7fa775653b80 1 --2- 172.21.15.32:0/3922329794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c3200 0x562c5e6bd590 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa754024780 tx=0x7fa754009510 comp rx=0 tx=0).stop 2023-09-30T14:51:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.862+0000 7fa775653b80 1 -- 172.21.15.32:0/3922329794 shutdown_connections 2023-09-30T14:51:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.862+0000 7fa775653b80 1 --2- 172.21.15.32:0/3922329794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4cd0 0x562c5e6c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.862+0000 7fa775653b80 1 --2- 172.21.15.32:0/3922329794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c3200 0x562c5e6bd590 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.862+0000 7fa775653b80 1 --2- 172.21.15.32:0/3922329794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c4310 0x562c5e6cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.862+0000 7fa775653b80 1 -- 172.21.15.32:0/3922329794 >> 172.21.15.32:0/3922329794 conn(0x562c5e637200 msgr2=0x562c5e6dcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.862+0000 7fa775653b80 1 -- 172.21.15.32:0/3922329794 shutdown_connections 2023-09-30T14:51:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.862+0000 7fa775653b80 1 -- 172.21.15.32:0/3922329794 wait complete. 2023-09-30T14:51:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.862+0000 7fa775653b80 1 -- start start 2023-09-30T14:51:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa775653b80 1 Processor -- start 2023-09-30T14:51:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa775653b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c3200 0x562c5e73bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa775653b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c4310 0x562c5e73e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa775653b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4cd0 0x562c5e7408c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa775653b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5e741000 con 0x562c5e6c3200 2023-09-30T14:51:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa775653b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5e741170 con 0x562c5e6c4cd0 2023-09-30T14:51:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa775653b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c5e741330 con 0x562c5e6c4310 2023-09-30T14:51:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa775653b80 10 client.0 _ll_register_callbacks cb 0x7fa754004840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:51:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa762ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c3200 0x562c5e73bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa762ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c3200 0x562c5e73bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38334/0 (socket says 172.21.15.32:38334) 2023-09-30T14:51:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa762ffd700 1 -- 172.21.15.32:0/369525671 learned_addr learned my addr 172.21.15.32:0/369525671 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa7627fc700 1 --2- 172.21.15.32:0/369525671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c4310 0x562c5e73e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa7637fe700 1 --2- 172.21.15.32:0/369525671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4cd0 0x562c5e7408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa7627fc700 1 -- 172.21.15.32:0/369525671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4cd0 msgr2=0x562c5e7408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa7627fc700 1 --2- 172.21.15.32:0/369525671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4cd0 0x562c5e7408c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa7627fc700 1 -- 172.21.15.32:0/369525671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c3200 msgr2=0x562c5e73bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa7627fc700 1 --2- 172.21.15.32:0/369525671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c3200 0x562c5e73bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.863+0000 7fa7627fc700 1 -- 172.21.15.32:0/369525671 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c5e678870 con 0x562c5e6c4310 2023-09-30T14:51:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.864+0000 7fa762ffd700 1 --2- 172.21.15.32:0/369525671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c3200 0x562c5e73bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.864+0000 7fa7627fc700 1 --2- 172.21.15.32:0/369525671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c4310 0x562c5e73e210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa75809ad50 tx=0x7fa75809f0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac12e1ac66f33f0f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.864+0000 7fa746ff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:51:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.864+0000 7fa746ff5700 1 -- 172.21.15.32:0/369525671 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75809b0d0 con 0x562c5e6c4310 2023-09-30T14:51:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.864+0000 7fa746ff5700 1 -- 172.21.15.32:0/369525671 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa75809a540 con 0x562c5e6c4310 2023-09-30T14:51:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.864+0000 7fa775653b80 1 -- 172.21.15.32:0/369525671 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa75809f1a0 con 0x562c5e6c4310 2023-09-30T14:51:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.864+0000 7fa746ff5700 1 -- 172.21.15.32:0/369525671 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa758097390 con 0x562c5e6c4310 2023-09-30T14:51:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.864+0000 7fa775653b80 10 client.26275 fetch_fsmap learned FSMap version 45 2023-09-30T14:51:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.864+0000 7fa775653b80 1 -- 172.21.15.32:0/369525671 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x562c5e440d80 con 0x562c5e6c4310 2023-09-30T14:51:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.864+0000 7fa746ff5700 1 -- 172.21.15.32:0/369525671 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa75809f1a0 con 0x562c5e6c4310 2023-09-30T14:51:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.864+0000 7fa746ff5700 1 -- 172.21.15.32:0/369525671 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 45) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fa7580cbb50 con 0x562c5e6c4310 2023-09-30T14:51:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.864+0000 7fa775653b80 10 client.26275 fetch_fsmap finished waiting for FSMap version 45 2023-09-30T14:51:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.864+0000 7fa775653b80 10 client.26275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:51:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.864+0000 7fa775653b80 20 client.26275 populate_metadata read hostname 'smithi032' 2023-09-30T14:51:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.864+0000 7fa775653b80 1 --2- 172.21.15.32:0/369525671 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562c5e742b90 0x562c5e744fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.864+0000 7fa775653b80 4 client.26275 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:51:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.865+0000 7fa775653b80 1 -- 172.21.15.32:0/369525671 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562c5e60ead0 con 0x562c5e742b90 2023-09-30T14:51:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.865+0000 7fa762ffd700 1 --2- 172.21.15.32:0/369525671 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562c5e742b90 0x562c5e744fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.865+0000 7fa746ff5700 10 client.26275 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.865+0000 7fa762ffd700 1 --2- 172.21.15.32:0/369525671 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562c5e742b90 0x562c5e744fb0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b09695ce9a32bb84 server_cookie=39782f8e1d1468f1 in_seq=0 out_seq=0 2023-09-30T14:51:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.919+0000 7fa746ff5700 1 -- 172.21.15.32:0/369525671 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562c5e60ead0 con 0x562c5e742b90 2023-09-30T14:51:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.919+0000 7fa746ff5700 10 client.26275 handle_command_reply: tid=0 2023-09-30T14:51:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.919+0000 7fa775653b80 1 client.26275 shutdown 2023-09-30T14:51:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.919+0000 7fa775653b80 1 -- 172.21.15.32:0/369525671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c4310 msgr2=0x562c5e73e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.919+0000 7fa775653b80 1 --2- 172.21.15.32:0/369525671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c4310 0x562c5e73e210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa75809ad50 tx=0x7fa75809f0d0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.919+0000 7fa775653b80 1 -- 172.21.15.32:0/369525671 shutdown_connections 2023-09-30T14:51:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.919+0000 7fa775653b80 1 --2- 172.21.15.32:0/369525671 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562c5e742b90 0x562c5e744fb0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.919+0000 7fa775653b80 1 --2- 172.21.15.32:0/369525671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c5e6c4cd0 0x562c5e7408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.919+0000 7fa746ff5700 0 client.26275 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.919+0000 7fa775653b80 1 --2- 172.21.15.32:0/369525671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562c5e6c3200 0x562c5e73bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.919+0000 7fa775653b80 1 --2- 172.21.15.32:0/369525671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c5e6c4310 0x562c5e73e210 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.919+0000 7fa775653b80 1 -- 172.21.15.32:0/369525671 >> 172.21.15.32:0/369525671 conn(0x562c5e637200 msgr2=0x562c5e6bd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.919+0000 7fa775653b80 1 -- 172.21.15.32:0/369525671 shutdown_connections 2023-09-30T14:51:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.920+0000 7fa775653b80 1 -- 172.21.15.32:0/369525671 wait complete. 2023-09-30T14:51:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.920+0000 7fa775653b80 20 client.26275 trim_cache size 0 max 16384 2023-09-30T14:51:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:51:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:51:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:51:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:51:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.920+0000 7fa76946b700 1 -- 172.21.15.32:0/1532663700 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa74c0a0fa0 msgr2=0x7fa74c0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.920+0000 7fa76946b700 1 --2- 172.21.15.32:0/1532663700 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa74c0a0fa0 0x7fa74c0a3460 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa754009990 tx=0x7fa75401a040 comp rx=0 tx=0).stop 2023-09-30T14:51:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.920+0000 7fa76946b700 1 -- 172.21.15.32:0/1532663700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7641006d0 msgr2=0x7fa764194f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.920+0000 7fa76946b700 1 --2- 172.21.15.32:0/1532663700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7641006d0 0x7fa764194f30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa75800fec0 tx=0x7fa75800d6c0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.920+0000 7fa76946b700 1 -- 172.21.15.32:0/1532663700 shutdown_connections 2023-09-30T14:51:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.920+0000 7fa76946b700 1 --2- 172.21.15.32:0/1532663700 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa74c0a0fa0 0x7fa74c0a3460 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.920+0000 7fa76946b700 1 --2- 172.21.15.32:0/1532663700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa764101070 0x7fa764195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.920+0000 7fa76946b700 1 --2- 172.21.15.32:0/1532663700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7640ff4e0 0x7fa7641949f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.920+0000 7fa76946b700 1 --2- 172.21.15.32:0/1532663700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7641006d0 0x7fa764194f30 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.920+0000 7fa76946b700 1 -- 172.21.15.32:0/1532663700 >> 172.21.15.32:0/1532663700 conn(0x7fa7640faa30 msgr2=0x7fa764103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.920+0000 7fa76946b700 1 -- 172.21.15.32:0/1532663700 shutdown_connections 2023-09-30T14:51:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:30.920+0000 7fa76946b700 1 -- 172.21.15.32:0/1532663700 wait complete. 2023-09-30T14:51:30.995 INFO:tasks.workunit.client.0.smithi032.stdout: 10 1934 66.14 MB/sec warmup 4 sec latency 80.336 ms 2023-09-30T14:51:31.995 INFO:tasks.workunit.client.0.smithi032.stdout: 10 2614 59.53 MB/sec warmup 5 sec latency 45.500 ms 2023-09-30T14:51:32.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:31 smithi156 ceph-mon[103579]: pgmap v684: 97 pgs: 97 active+clean; 395 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 3.9 MiB/s wr, 243 op/s 2023-09-30T14:51:32.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:31 smithi148 ceph-mon[102355]: pgmap v684: 97 pgs: 97 active+clean; 395 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 3.9 MiB/s wr, 243 op/s 2023-09-30T14:51:32.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:31 smithi032 ceph-mon[98592]: pgmap v684: 97 pgs: 97 active+clean; 395 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 3.9 MiB/s wr, 243 op/s 2023-09-30T14:51:32.995 INFO:tasks.workunit.client.0.smithi032.stdout: 10 3311 58.94 MB/sec warmup 6 sec latency 43.303 ms 2023-09-30T14:51:33.998 INFO:tasks.workunit.client.0.smithi032.stdout: 10 3752 56.16 MB/sec warmup 7 sec latency 61.584 ms 2023-09-30T14:51:34.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:33 smithi156 ceph-mon[103579]: pgmap v685: 97 pgs: 97 active+clean; 395 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 4.5 MiB/s wr, 336 op/s 2023-09-30T14:51:34.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:33 smithi148 ceph-mon[102355]: pgmap v685: 97 pgs: 97 active+clean; 395 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 4.5 MiB/s wr, 336 op/s 2023-09-30T14:51:34.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:33 smithi032 ceph-mon[98592]: pgmap v685: 97 pgs: 97 active+clean; 395 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 4.5 MiB/s wr, 336 op/s 2023-09-30T14:51:34.998 INFO:tasks.workunit.client.0.smithi032.stdout: 10 3937 50.02 MB/sec warmup 8 sec latency 116.711 ms 2023-09-30T14:51:35.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:34 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:51:35.115 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:34 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:51:35.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:34 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:51:35.998 INFO:tasks.workunit.client.0.smithi032.stdout: 10 4133 44.54 MB/sec warmup 9 sec latency 157.403 ms 2023-09-30T14:51:36.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:35 smithi156 ceph-mon[103579]: pgmap v686: 97 pgs: 97 active+clean; 389 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.8 MiB/s wr, 299 op/s 2023-09-30T14:51:36.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:35 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:51:36.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:35 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:51:36.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:35 smithi148 ceph-mon[102355]: pgmap v686: 97 pgs: 97 active+clean; 389 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.8 MiB/s wr, 299 op/s 2023-09-30T14:51:36.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:35 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:51:36.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:35 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:51:36.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:35 smithi032 ceph-mon[98592]: pgmap v686: 97 pgs: 97 active+clean; 389 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.8 MiB/s wr, 299 op/s 2023-09-30T14:51:36.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:35 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:51:36.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:35 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:51:36.998 INFO:tasks.workunit.client.0.smithi032.stdout: 10 4437 40.93 MB/sec warmup 10 sec latency 107.093 ms 2023-09-30T14:51:37.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:36 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:51:37.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:36 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:51:37.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:36 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:51:37.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:36 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:51:37.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:36 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:51:37.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:36 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:51:37.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:36 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:51:37.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:36 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:51:37.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:36 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:51:37.998 INFO:tasks.workunit.client.0.smithi032.stdout: 10 4938 40.33 MB/sec warmup 11 sec latency 64.728 ms 2023-09-30T14:51:38.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:37 smithi156 ceph-mon[103579]: pgmap v687: 97 pgs: 97 active+clean; 395 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.8 MiB/s wr, 289 op/s 2023-09-30T14:51:38.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:37 smithi032 ceph-mon[98592]: pgmap v687: 97 pgs: 97 active+clean; 395 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.8 MiB/s wr, 289 op/s 2023-09-30T14:51:38.148 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:37 smithi148 ceph-mon[102355]: pgmap v687: 97 pgs: 97 active+clean; 395 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.8 MiB/s wr, 289 op/s 2023-09-30T14:51:38.998 INFO:tasks.workunit.client.0.smithi032.stdout: 10 5800 38.70 MB/sec warmup 12 sec latency 53.549 ms 2023-09-30T14:51:39.998 INFO:tasks.workunit.client.0.smithi032.stdout: 10 6214 38.63 MB/sec warmup 13 sec latency 85.098 ms 2023-09-30T14:51:40.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:39 smithi156 ceph-mon[103579]: pgmap v688: 97 pgs: 97 active+clean; 401 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 325 op/s 2023-09-30T14:51:40.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:39 smithi148 ceph-mon[102355]: pgmap v688: 97 pgs: 97 active+clean; 401 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 325 op/s 2023-09-30T14:51:40.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:39 smithi032 ceph-mon[98592]: pgmap v688: 97 pgs: 97 active+clean; 401 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 325 op/s 2023-09-30T14:51:40.998 INFO:tasks.workunit.client.0.smithi032.stdout: 10 6866 39.00 MB/sec warmup 14 sec latency 41.680 ms 2023-09-30T14:51:41.999 INFO:tasks.workunit.client.0.smithi032.stdout: 10 7220 38.59 MB/sec warmup 15 sec latency 105.561 ms 2023-09-30T14:51:42.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:41 smithi156 ceph-mon[103579]: pgmap v689: 97 pgs: 97 active+clean; 402 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 3.8 MiB/s wr, 294 op/s 2023-09-30T14:51:42.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:41 smithi032 ceph-mon[98592]: pgmap v689: 97 pgs: 97 active+clean; 402 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 3.8 MiB/s wr, 294 op/s 2023-09-30T14:51:42.254 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:41 smithi148 ceph-mon[102355]: pgmap v689: 97 pgs: 97 active+clean; 402 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 3.8 MiB/s wr, 294 op/s 2023-09-30T14:51:42.998 INFO:tasks.workunit.client.0.smithi032.stdout: 10 7445 36.69 MB/sec warmup 16 sec latency 39.693 ms 2023-09-30T14:51:43.998 INFO:tasks.workunit.client.0.smithi032.stdout: 10 7664 34.84 MB/sec warmup 17 sec latency 109.574 ms 2023-09-30T14:51:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.042+0000 7f22e370d700 1 -- 172.21.15.32:0/1184268747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22dc1042b0 msgr2=0x7f22dc1066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.042+0000 7f22e370d700 1 --2- 172.21.15.32:0/1184268747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22dc1042b0 0x7f22dc1066a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009990 tx=0x7f22cc009ca0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.042+0000 7f22e370d700 1 -- 172.21.15.32:0/1184268747 shutdown_connections 2023-09-30T14:51:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.042+0000 7f22e370d700 1 --2- 172.21.15.32:0/1184268747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22dc106be0 0x7f22dc108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.042+0000 7f22e370d700 1 --2- 172.21.15.32:0/1184268747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22dc101980 0x7f22dc103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.042+0000 7f22e370d700 1 --2- 172.21.15.32:0/1184268747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22dc1042b0 0x7f22dc1066a0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.042+0000 7f22e370d700 1 -- 172.21.15.32:0/1184268747 >> 172.21.15.32:0/1184268747 conn(0x7f22dc0fb2b0 msgr2=0x7f22dc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.042+0000 7f22e370d700 1 -- 172.21.15.32:0/1184268747 shutdown_connections 2023-09-30T14:51:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.042+0000 7f22e370d700 1 -- 172.21.15.32:0/1184268747 wait complete. 2023-09-30T14:51:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e370d700 1 Processor -- start 2023-09-30T14:51:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e370d700 1 -- start start 2023-09-30T14:51:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e370d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22dc101980 0x7f22dc196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e370d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22dc1042b0 0x7f22dc197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e370d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22dc106be0 0x7f22dc197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e370d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc198600 con 0x7f22dc101980 2023-09-30T14:51:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e370d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc198770 con 0x7f22dc1042b0 2023-09-30T14:51:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e370d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc198930 con 0x7f22dc106be0 2023-09-30T14:51:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e14a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22dc101980 0x7f22dc196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e14a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22dc101980 0x7f22dc196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38348/0 (socket says 172.21.15.32:38348) 2023-09-30T14:51:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e14a9700 1 -- 172.21.15.32:0/1787396935 learned_addr learned my addr 172.21.15.32:0/1787396935 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e0ca8700 1 --2- 172.21.15.32:0/1787396935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22dc1042b0 0x7f22dc197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e1caa700 1 --2- 172.21.15.32:0/1787396935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22dc106be0 0x7f22dc197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e14a9700 1 -- 172.21.15.32:0/1787396935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22dc106be0 msgr2=0x7f22dc197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e14a9700 1 --2- 172.21.15.32:0/1787396935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22dc106be0 0x7f22dc197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e14a9700 1 -- 172.21.15.32:0/1787396935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22dc1042b0 msgr2=0x7f22dc197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e14a9700 1 --2- 172.21.15.32:0/1787396935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22dc1042b0 0x7f22dc197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e14a9700 1 -- 172.21.15.32:0/1787396935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc0095f0 con 0x7f22dc101980 2023-09-30T14:51:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.043+0000 7f22e14a9700 1 --2- 172.21.15.32:0/1787396935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22dc101980 0x7f22dc196be0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f22d800e9a0 tx=0x7f22d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.044+0000 7f22d27fc700 1 -- 172.21.15.32:0/1787396935 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8009790 con 0x7f22dc101980 2023-09-30T14:51:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.044+0000 7f22d27fc700 1 -- 172.21.15.32:0/1787396935 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22d8010430 con 0x7f22dc101980 2023-09-30T14:51:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.044+0000 7f22e370d700 1 -- 172.21.15.32:0/1787396935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22dc198c10 con 0x7f22dc101980 2023-09-30T14:51:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.044+0000 7f22d27fc700 1 -- 172.21.15.32:0/1787396935 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8018680 con 0x7f22dc101980 2023-09-30T14:51:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.044+0000 7f22e370d700 1 -- 172.21.15.32:0/1787396935 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22dc19f3d0 con 0x7f22dc101980 2023-09-30T14:51:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.046+0000 7f22d27fc700 1 -- 172.21.15.32:0/1787396935 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f22d80105a0 con 0x7f22dc101980 2023-09-30T14:51:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.046+0000 7f22d27fc700 1 --2- 172.21.15.32:0/1787396935 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f22c80a0fa0 0x7f22c80a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.046+0000 7f22e0ca8700 1 --2- 172.21.15.32:0/1787396935 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f22c80a0fa0 0x7f22c80a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.046+0000 7f22d27fc700 1 -- 172.21.15.32:0/1787396935 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f22d8014070 con 0x7f22dc101980 2023-09-30T14:51:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.047+0000 7f22e0ca8700 1 --2- 172.21.15.32:0/1787396935 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f22c80a0fa0 0x7f22c80a3460 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00b2e0 tx=0x7f22cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.047+0000 7f22ef8f5b80 1 Processor -- start 2023-09-30T14:51:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.048+0000 7f22ef8f5b80 1 -- start start 2023-09-30T14:51:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.048+0000 7f22ef8f5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d60610 0x556230d60a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.048+0000 7f22ef8f5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d61720 0x556230d61ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.048+0000 7f22ef8f5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d620e0 0x556230d68970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.048+0000 7f22ef8f5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556230cc4400 con 0x556230d60610 2023-09-30T14:51:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.048+0000 7f22ef8f5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556230cb5bc0 con 0x556230d620e0 2023-09-30T14:51:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.048+0000 7f22ef8f5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556230cb34e0 con 0x556230d61720 2023-09-30T14:51:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.048+0000 7f22e14a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d60610 0x556230d60a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.048+0000 7f22e14a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d60610 0x556230d60a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38356/0 (socket says 172.21.15.32:38356) 2023-09-30T14:51:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.048+0000 7f22e14a9700 1 -- 172.21.15.32:0/1974946796 learned_addr learned my addr 172.21.15.32:0/1974946796 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.048+0000 7f22e0ca8700 1 --2- 172.21.15.32:0/1974946796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d61720 0x556230d61ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.048+0000 7f22e14a9700 1 -- 172.21.15.32:0/1974946796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d61720 msgr2=0x556230d61ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.048+0000 7f22e14a9700 1 --2- 172.21.15.32:0/1974946796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d61720 0x556230d61ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.048+0000 7f22e14a9700 1 -- 172.21.15.32:0/1974946796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d620e0 msgr2=0x556230d68970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.048+0000 7f22e1caa700 1 --2- 172.21.15.32:0/1974946796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d620e0 0x556230d68970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.048+0000 7f22e14a9700 1 --2- 172.21.15.32:0/1974946796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d620e0 0x556230d68970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.049+0000 7f22e14a9700 1 -- 172.21.15.32:0/1974946796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556230d11870 con 0x556230d60610 2023-09-30T14:51:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.049+0000 7f22e14a9700 1 --2- 172.21.15.32:0/1974946796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d60610 0x556230d60a30 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f22d8096280 tx=0x7f22d80997b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4803e6dd50452359 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.049+0000 7f22c77fe700 1 -- 172.21.15.32:0/1974946796 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d809ded0 con 0x556230d60610 2023-09-30T14:51:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.049+0000 7f22c77fe700 1 -- 172.21.15.32:0/1974946796 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22d80a2110 con 0x556230d60610 2023-09-30T14:51:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.049+0000 7f22c77fe700 1 -- 172.21.15.32:0/1974946796 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d80b2210 con 0x556230d60610 2023-09-30T14:51:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.050+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/1974946796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d60610 msgr2=0x556230d60a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.050+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/1974946796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d60610 0x556230d60a30 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f22d8096280 tx=0x7f22d80997b0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.050+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/1974946796 shutdown_connections 2023-09-30T14:51:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.050+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/1974946796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d620e0 0x556230d68970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.050+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/1974946796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d60610 0x556230d60a30 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.050+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/1974946796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d61720 0x556230d61ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.050+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/1974946796 >> 172.21.15.32:0/1974946796 conn(0x556230cd0200 msgr2=0x556230d56ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.050+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/1974946796 shutdown_connections 2023-09-30T14:51:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.050+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/1974946796 wait complete. 2023-09-30T14:51:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.050+0000 7f22ef8f5b80 1 -- start start 2023-09-30T14:51:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.050+0000 7f22ef8f5b80 1 Processor -- start 2023-09-30T14:51:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22ef8f5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 0x556230d6a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22ef8f5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d61720 0x556230d6a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22ef8f5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 0x556230d718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22ef8f5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556230caedd0 con 0x556230d620e0 2023-09-30T14:51:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22ef8f5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556230c4b6b0 con 0x556230d61720 2023-09-30T14:51:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22ef8f5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556230c4b350 con 0x556230d60610 2023-09-30T14:51:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22ef8f5b80 10 client.0 _ll_register_callbacks cb 0x556230aa87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:51:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22e14a9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 0x556230d6a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22e14a9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 0x556230d6a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52744/0 (socket says 172.21.15.32:52744) 2023-09-30T14:51:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22e1caa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 0x556230d718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22e14a9700 1 -- 172.21.15.32:0/3812961862 learned_addr learned my addr 172.21.15.32:0/3812961862 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22e0ca8700 1 --2- 172.21.15.32:0/3812961862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d61720 0x556230d6a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22e14a9700 1 -- 172.21.15.32:0/3812961862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d61720 msgr2=0x556230d6a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22e14a9700 1 --2- 172.21.15.32:0/3812961862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d61720 0x556230d6a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22e14a9700 1 -- 172.21.15.32:0/3812961862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 msgr2=0x556230d718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22e14a9700 1 --2- 172.21.15.32:0/3812961862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 0x556230d718a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22e14a9700 1 -- 172.21.15.32:0/3812961862 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22dc19f3d0 con 0x556230d60610 2023-09-30T14:51:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.051+0000 7f22e1caa700 1 --2- 172.21.15.32:0/3812961862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 0x556230d718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.052+0000 7f22e14a9700 1 --2- 172.21.15.32:0/3812961862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 0x556230d6a320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f22d809a3e0 tx=0x7f22d800c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2a4f9f78f9db312 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.052+0000 7f22c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:51:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.052+0000 7f22c4ff9700 1 -- 172.21.15.32:0/3812961862 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d809ded0 con 0x556230d60610 2023-09-30T14:51:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.052+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/3812961862 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556230d4f090 con 0x556230d60610 2023-09-30T14:51:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.052+0000 7f22c4ff9700 1 -- 172.21.15.32:0/3812961862 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f22d80a1c80 con 0x556230d60610 2023-09-30T14:51:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.052+0000 7f22c4ff9700 1 -- 172.21.15.32:0/3812961862 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d8064020 con 0x556230d60610 2023-09-30T14:51:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.052+0000 7f22ef8f5b80 10 client.26287 fetch_fsmap learned FSMap version 45 2023-09-30T14:51:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.052+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/3812961862 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x556230ad9d80 con 0x556230d60610 2023-09-30T14:51:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.052+0000 7f22c4ff9700 1 -- 172.21.15.32:0/3812961862 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22d807c070 con 0x556230d60610 2023-09-30T14:51:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.052+0000 7f22c4ff9700 1 -- 172.21.15.32:0/3812961862 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 45) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f22d809ec10 con 0x556230d60610 2023-09-30T14:51:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.052+0000 7f22ef8f5b80 10 client.26287 fetch_fsmap finished waiting for FSMap version 45 2023-09-30T14:51:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.052+0000 7f22ef8f5b80 10 client.26287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:51:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.053+0000 7f22ef8f5b80 20 client.26287 populate_metadata read hostname 'smithi032' 2023-09-30T14:51:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.053+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/3812961862 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556230d75c40 0x556230d78060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.053+0000 7f22ef8f5b80 4 client.26287 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:51:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.053+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/3812961862 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556230ca7db0 con 0x556230d75c40 2023-09-30T14:51:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.053+0000 7f22e0ca8700 1 --2- 172.21.15.32:0/3812961862 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556230d75c40 0x556230d78060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.053+0000 7f22c4ff9700 10 client.26287 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.053+0000 7f22e0ca8700 1 --2- 172.21.15.32:0/3812961862 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556230d75c40 0x556230d78060 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95fd0b13d7d2d52d server_cookie=71d5d5bad4303611 in_seq=0 out_seq=0 2023-09-30T14:51:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.054+0000 7f22c4ff9700 1 -- 172.21.15.32:0/3812961862 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556230ca7db0 con 0x556230d75c40 2023-09-30T14:51:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.054+0000 7f22c4ff9700 10 client.26287 handle_command_reply: tid=0 2023-09-30T14:51:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.054+0000 7f22ef8f5b80 1 client.26287 shutdown 2023-09-30T14:51:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.054+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/3812961862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 msgr2=0x556230d6a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.054+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/3812961862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 0x556230d6a320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f22d809a3e0 tx=0x7f22d800c470 comp rx=0 tx=0).stop 2023-09-30T14:51:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.054+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/3812961862 shutdown_connections 2023-09-30T14:51:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.054+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/3812961862 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556230d75c40 0x556230d78060 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.054+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/3812961862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 0x556230d718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.054+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/3812961862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 0x556230d6a320 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.054+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/3812961862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d61720 0x556230d6a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.054+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/3812961862 >> 172.21.15.32:0/3812961862 conn(0x556230cd0200 msgr2=0x556230d56590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.054+0000 7f22c4ff9700 0 client.26287 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.055+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/3812961862 shutdown_connections 2023-09-30T14:51:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.055+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/3812961862 wait complete. 2023-09-30T14:51:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.055+0000 7f22ef8f5b80 20 client.26287 trim_cache size 0 max 16384 2023-09-30T14:51:44.060 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:43 smithi156 ceph-mon[103579]: pgmap v690: 97 pgs: 97 active+clean; 410 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 362 op/s 2023-09-30T14:51:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.070+0000 7f22ef8f5b80 1 Processor -- start 2023-09-30T14:51:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.070+0000 7f22ef8f5b80 1 -- start start 2023-09-30T14:51:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.070+0000 7f22ef8f5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 0x556230d77c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.070+0000 7f22ef8f5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d61720 0x556230d71860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22ef8f5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 0x556230d685a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22ef8f5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556230b2f8c0 con 0x556230d620e0 2023-09-30T14:51:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22ef8f5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556230d53ba0 con 0x556230d61720 2023-09-30T14:51:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22ef8f5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556230d58630 con 0x556230d60610 2023-09-30T14:51:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22e1caa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 0x556230d685a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22e1caa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 0x556230d685a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38374/0 (socket says 172.21.15.32:38374) 2023-09-30T14:51:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22e1caa700 1 -- 172.21.15.32:0/3003440989 learned_addr learned my addr 172.21.15.32:0/3003440989 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22e0ca8700 1 --2- 172.21.15.32:0/3003440989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d61720 0x556230d71860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22e14a9700 1 --2- 172.21.15.32:0/3003440989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 0x556230d77c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22e1caa700 1 -- 172.21.15.32:0/3003440989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 msgr2=0x556230d77c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22e1caa700 1 --2- 172.21.15.32:0/3003440989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 0x556230d77c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22e1caa700 1 -- 172.21.15.32:0/3003440989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d61720 msgr2=0x556230d71860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22e1caa700 1 --2- 172.21.15.32:0/3003440989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d61720 0x556230d71860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22e1caa700 1 -- 172.21.15.32:0/3003440989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556230ac2c00 con 0x556230d620e0 2023-09-30T14:51:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22e0ca8700 1 --2- 172.21.15.32:0/3003440989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d61720 0x556230d71860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22e1caa700 1 --2- 172.21.15.32:0/3003440989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 0x556230d685a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f22d4007850 tx=0x7f22d4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd8ad0fd531fb6a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22c7fff700 1 -- 172.21.15.32:0/3003440989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d400e040 con 0x556230d620e0 2023-09-30T14:51:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22c7fff700 1 -- 172.21.15.32:0/3003440989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22d400f040 con 0x556230d620e0 2023-09-30T14:51:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.071+0000 7f22c7fff700 1 -- 172.21.15.32:0/3003440989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40086b0 con 0x556230d620e0 2023-09-30T14:51:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.072+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/3003440989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 msgr2=0x556230d685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.072+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/3003440989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 0x556230d685a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f22d4007850 tx=0x7f22d4009920 comp rx=0 tx=0).stop 2023-09-30T14:51:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.072+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/3003440989 shutdown_connections 2023-09-30T14:51:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.072+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/3003440989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 0x556230d685a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.072+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/3003440989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 0x556230d77c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.072+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/3003440989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d61720 0x556230d71860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.072+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/3003440989 >> 172.21.15.32:0/3003440989 conn(0x556230cd0200 msgr2=0x556230d68fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.072+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/3003440989 shutdown_connections 2023-09-30T14:51:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.072+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/3003440989 wait complete. 2023-09-30T14:51:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.072+0000 7f22ef8f5b80 1 -- start start 2023-09-30T14:51:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.073+0000 7f22ef8f5b80 1 Processor -- start 2023-09-30T14:51:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.073+0000 7f22ef8f5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 0x556230d6f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.073+0000 7f22ef8f5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d61720 0x556230d6f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.073+0000 7f22ef8f5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 0x556230d707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.073+0000 7f22ef8f5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x556230d70ce0 con 0x556230d620e0 2023-09-30T14:51:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.073+0000 7f22ef8f5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556230d70e50 con 0x556230d61720 2023-09-30T14:51:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.073+0000 7f22ef8f5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556230d71010 con 0x556230d60610 2023-09-30T14:51:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.073+0000 7f22ef8f5b80 10 client.0 _ll_register_callbacks cb 0x7f22cc00b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:51:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.073+0000 7f22e14a9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 0x556230d6f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.073+0000 7f22e1caa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 0x556230d707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.073+0000 7f22e0ca8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d61720 0x556230d6f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.073+0000 7f22e14a9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 0x556230d6f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52758/0 (socket says 172.21.15.32:52758) 2023-09-30T14:51:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.073+0000 7f22e14a9700 1 -- 172.21.15.32:0/522697319 learned_addr learned my addr 172.21.15.32:0/522697319 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.074+0000 7f22e14a9700 1 -- 172.21.15.32:0/522697319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d61720 msgr2=0x556230d6f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.074+0000 7f22e14a9700 1 --2- 172.21.15.32:0/522697319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d61720 0x556230d6f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.074+0000 7f22e14a9700 1 -- 172.21.15.32:0/522697319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 msgr2=0x556230d707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.074+0000 7f22e14a9700 1 --2- 172.21.15.32:0/522697319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 0x556230d707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.074+0000 7f22e14a9700 1 -- 172.21.15.32:0/522697319 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22dc109ba0 con 0x556230d60610 2023-09-30T14:51:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.074+0000 7f22e14a9700 1 --2- 172.21.15.32:0/522697319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 0x556230d6f250 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f22d809f070 tx=0x7f22d80996e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e83f413380b9da0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.074+0000 7f22c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:51:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.074+0000 7f22c4ff9700 1 -- 172.21.15.32:0/522697319 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d809ded0 con 0x556230d60610 2023-09-30T14:51:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.074+0000 7f22c4ff9700 1 -- 172.21.15.32:0/522697319 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f22d809ec10 con 0x556230d60610 2023-09-30T14:51:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.074+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/522697319 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22d80a2110 con 0x556230d60610 2023-09-30T14:51:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.074+0000 7f22c4ff9700 1 -- 172.21.15.32:0/522697319 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d80b93f0 con 0x556230d60610 2023-09-30T14:51:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.074+0000 7f22ef8f5b80 10 client.26293 fetch_fsmap learned FSMap version 45 2023-09-30T14:51:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.074+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/522697319 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f22d40092d0 con 0x556230d60610 2023-09-30T14:51:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.075+0000 7f22c4ff9700 1 -- 172.21.15.32:0/522697319 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22d80a2110 con 0x556230d60610 2023-09-30T14:51:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.075+0000 7f22c4ff9700 1 -- 172.21.15.32:0/522697319 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 45) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f22d8099270 con 0x556230d60610 2023-09-30T14:51:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.075+0000 7f22ef8f5b80 10 client.26293 fetch_fsmap finished waiting for FSMap version 45 2023-09-30T14:51:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.075+0000 7f22ef8f5b80 10 client.26293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:51:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.075+0000 7f22ef8f5b80 20 client.26293 populate_metadata read hostname 'smithi032' 2023-09-30T14:51:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.075+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/522697319 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556230dca210 0x556230dcc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.075+0000 7f22ef8f5b80 4 client.26293 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:51:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.075+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/522697319 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556230ca7ad0 con 0x556230dca210 2023-09-30T14:51:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.075+0000 7f22e0ca8700 1 --2- 172.21.15.32:0/522697319 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556230dca210 0x556230dcc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.075+0000 7f22e0ca8700 1 --2- 172.21.15.32:0/522697319 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556230dca210 0x556230dcc630 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5055d49da49d6707 server_cookie=da57b9598137d0d3 in_seq=0 out_seq=0 2023-09-30T14:51:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.075+0000 7f22c4ff9700 10 client.26293 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.094+0000 7f22c4ff9700 1 -- 172.21.15.32:0/522697319 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556230ca7ad0 con 0x556230dca210 2023-09-30T14:51:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.094+0000 7f22c4ff9700 10 client.26293 handle_command_reply: tid=0 2023-09-30T14:51:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.094+0000 7f22ef8f5b80 1 client.26293 shutdown 2023-09-30T14:51:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.094+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/522697319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 msgr2=0x556230d6f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.094+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/522697319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 0x556230d6f250 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f22d809f070 tx=0x7f22d80996e0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.094+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/522697319 shutdown_connections 2023-09-30T14:51:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.094+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/522697319 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556230dca210 0x556230dcc630 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.094+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/522697319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x556230d620e0 0x556230d707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.094+0000 7f22c4ff9700 0 client.26293 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.094+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/522697319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556230d60610 0x556230d6f250 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.094+0000 7f22ef8f5b80 1 --2- 172.21.15.32:0/522697319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556230d61720 0x556230d6f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.094+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/522697319 >> 172.21.15.32:0/522697319 conn(0x556230cd0200 msgr2=0x556230d77c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.094+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/522697319 shutdown_connections 2023-09-30T14:51:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.094+0000 7f22ef8f5b80 1 -- 172.21.15.32:0/522697319 wait complete. 2023-09-30T14:51:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.094+0000 7f22ef8f5b80 20 client.26293 trim_cache size 0 max 16384 2023-09-30T14:51:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:51:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:51:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:51:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:51:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.095+0000 7f22e370d700 1 -- 172.21.15.32:0/1787396935 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f22c80a0fa0 msgr2=0x7f22c80a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.095+0000 7f22e370d700 1 --2- 172.21.15.32:0/1787396935 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f22c80a0fa0 0x7f22c80a3460 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00b2e0 tx=0x7f22cc005fb0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.095+0000 7f22e370d700 1 -- 172.21.15.32:0/1787396935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22dc101980 msgr2=0x7f22dc196be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.095+0000 7f22e370d700 1 --2- 172.21.15.32:0/1787396935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22dc101980 0x7f22dc196be0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f22d800e9a0 tx=0x7f22d800ecb0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.095+0000 7f22e370d700 1 -- 172.21.15.32:0/1787396935 shutdown_connections 2023-09-30T14:51:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.095+0000 7f22e370d700 1 --2- 172.21.15.32:0/1787396935 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f22c80a0fa0 0x7f22c80a3460 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.095+0000 7f22e370d700 1 --2- 172.21.15.32:0/1787396935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22dc106be0 0x7f22dc197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.095+0000 7f22e370d700 1 --2- 172.21.15.32:0/1787396935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22dc101980 0x7f22dc196be0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.095+0000 7f22e370d700 1 --2- 172.21.15.32:0/1787396935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22dc1042b0 0x7f22dc197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.095+0000 7f22e370d700 1 -- 172.21.15.32:0/1787396935 >> 172.21.15.32:0/1787396935 conn(0x7f22dc0fb2b0 msgr2=0x7f22dc1025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.095+0000 7f22e370d700 1 -- 172.21.15.32:0/1787396935 shutdown_connections 2023-09-30T14:51:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:44.095+0000 7f22e370d700 1 -- 172.21.15.32:0/1787396935 wait complete. 2023-09-30T14:51:44.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:43 smithi148 ceph-mon[102355]: pgmap v690: 97 pgs: 97 active+clean; 410 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 362 op/s 2023-09-30T14:51:44.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:43 smithi032 ceph-mon[98592]: pgmap v690: 97 pgs: 97 active+clean; 410 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 362 op/s 2023-09-30T14:51:44.999 INFO:tasks.workunit.client.0.smithi032.stdout: 10 7840 33.08 MB/sec warmup 18 sec latency 319.873 ms 2023-09-30T14:51:45.998 INFO:tasks.workunit.client.0.smithi032.stdout: 10 8217 32.42 MB/sec warmup 19 sec latency 302.759 ms 2023-09-30T14:51:46.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:45 smithi156 ceph-mon[103579]: pgmap v691: 97 pgs: 97 active+clean; 404 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 3.8 MiB/s wr, 274 op/s 2023-09-30T14:51:46.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:45 smithi148 ceph-mon[102355]: pgmap v691: 97 pgs: 97 active+clean; 404 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 3.8 MiB/s wr, 274 op/s 2023-09-30T14:51:46.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:45 smithi032 ceph-mon[98592]: pgmap v691: 97 pgs: 97 active+clean; 404 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 3.8 MiB/s wr, 274 op/s 2023-09-30T14:51:46.999 INFO:tasks.workunit.client.0.smithi032.stdout: 10 8852 32.15 MB/sec warmup 20 sec latency 53.967 ms 2023-09-30T14:51:47.999 INFO:tasks.workunit.client.0.smithi032.stdout: 10 9541 31.98 MB/sec warmup 21 sec latency 46.171 ms 2023-09-30T14:51:48.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:47 smithi156 ceph-mon[103579]: pgmap v692: 97 pgs: 97 active+clean; 391 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 310 op/s 2023-09-30T14:51:48.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:47 smithi148 ceph-mon[102355]: pgmap v692: 97 pgs: 97 active+clean; 391 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 310 op/s 2023-09-30T14:51:48.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:47 smithi032 ceph-mon[98592]: pgmap v692: 97 pgs: 97 active+clean; 391 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 310 op/s 2023-09-30T14:51:48.999 INFO:tasks.workunit.client.0.smithi032.stdout: 10 10133 32.87 MB/sec warmup 22 sec latency 63.577 ms 2023-09-30T14:51:49.999 INFO:tasks.workunit.client.0.smithi032.stdout: 10 10653 33.07 MB/sec warmup 23 sec latency 76.969 ms 2023-09-30T14:51:50.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:49 smithi156 ceph-mon[103579]: pgmap v693: 97 pgs: 97 active+clean; 395 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 307 op/s 2023-09-30T14:51:50.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:49 smithi148 ceph-mon[102355]: pgmap v693: 97 pgs: 97 active+clean; 395 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 307 op/s 2023-09-30T14:51:50.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:49 smithi032 ceph-mon[98592]: pgmap v693: 97 pgs: 97 active+clean; 395 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 307 op/s 2023-09-30T14:51:50.999 INFO:tasks.workunit.client.0.smithi032.stdout: 10 10877 32.23 MB/sec warmup 24 sec latency 49.531 ms 2023-09-30T14:51:51.999 INFO:tasks.workunit.client.0.smithi032.stdout: 10 11083 31.13 MB/sec warmup 25 sec latency 113.729 ms 2023-09-30T14:51:52.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:51 smithi156 ceph-mon[103579]: pgmap v694: 97 pgs: 97 active+clean; 402 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 325 op/s 2023-09-30T14:51:52.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:51 smithi148 ceph-mon[102355]: pgmap v694: 97 pgs: 97 active+clean; 402 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 325 op/s 2023-09-30T14:51:52.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:51 smithi032 ceph-mon[98592]: pgmap v694: 97 pgs: 97 active+clean; 402 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 325 op/s 2023-09-30T14:51:52.999 INFO:tasks.workunit.client.0.smithi032.stdout: 10 11390 30.23 MB/sec warmup 26 sec latency 110.740 ms 2023-09-30T14:51:53.999 INFO:tasks.workunit.client.0.smithi032.stdout: 10 11852 29.92 MB/sec warmup 27 sec latency 62.596 ms 2023-09-30T14:51:54.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:53 smithi156 ceph-mon[103579]: pgmap v695: 97 pgs: 97 active+clean; 401 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 326 op/s 2023-09-30T14:51:54.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:53 smithi148 ceph-mon[102355]: pgmap v695: 97 pgs: 97 active+clean; 401 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 326 op/s 2023-09-30T14:51:54.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:53 smithi032 ceph-mon[98592]: pgmap v695: 97 pgs: 97 active+clean; 401 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 326 op/s 2023-09-30T14:51:54.999 INFO:tasks.workunit.client.0.smithi032.stdout: 10 12512 29.83 MB/sec warmup 28 sec latency 106.882 ms 2023-09-30T14:51:55.999 INFO:tasks.workunit.client.0.smithi032.stdout: 10 13124 29.77 MB/sec warmup 29 sec latency 53.135 ms 2023-09-30T14:51:56.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:55 smithi156 ceph-mon[103579]: pgmap v696: 97 pgs: 97 active+clean; 402 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 302 op/s 2023-09-30T14:51:56.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:55 smithi148 ceph-mon[102355]: pgmap v696: 97 pgs: 97 active+clean; 402 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 302 op/s 2023-09-30T14:51:56.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:55 smithi032 ceph-mon[98592]: pgmap v696: 97 pgs: 97 active+clean; 402 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 302 op/s 2023-09-30T14:51:56.999 INFO:tasks.workunit.client.0.smithi032.stdout: 10 13734 30.48 MB/sec warmup 30 sec latency 98.681 ms 2023-09-30T14:51:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.213+0000 7fa372483700 1 -- 172.21.15.32:0/885407942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa36c1018f0 msgr2=0x7fa36c1081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.213+0000 7fa372483700 1 --2- 172.21.15.32:0/885407942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa36c1018f0 0x7fa36c1081a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa36400b0d0 tx=0x7fa36400b3e0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.213+0000 7fa372483700 1 -- 172.21.15.32:0/885407942 shutdown_connections 2023-09-30T14:51:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.213+0000 7fa372483700 1 --2- 172.21.15.32:0/885407942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa36c1018f0 0x7fa36c1081a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.213+0000 7fa372483700 1 --2- 172.21.15.32:0/885407942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa36c0ffd60 0x7fa36c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.213+0000 7fa372483700 1 --2- 172.21.15.32:0/885407942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa36c100f50 0x7fa36c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.213+0000 7fa372483700 1 -- 172.21.15.32:0/885407942 >> 172.21.15.32:0/885407942 conn(0x7fa36c0fb2b0 msgr2=0x7fa36c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.213+0000 7fa372483700 1 -- 172.21.15.32:0/885407942 shutdown_connections 2023-09-30T14:51:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.213+0000 7fa372483700 1 -- 172.21.15.32:0/885407942 wait complete. 2023-09-30T14:51:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.213+0000 7fa372483700 1 Processor -- start 2023-09-30T14:51:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.213+0000 7fa372483700 1 -- start start 2023-09-30T14:51:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa372483700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa36c0ffd60 0x7fa36c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa372483700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa36c100f50 0x7fa36c19bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa372483700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa36c1018f0 0x7fa36c19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa372483700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c19e9e0 con 0x7fa36c0ffd60 2023-09-30T14:51:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa372483700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c19eb20 con 0x7fa36c1018f0 2023-09-30T14:51:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa372483700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c19ece0 con 0x7fa36c100f50 2023-09-30T14:51:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa36bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa36c0ffd60 0x7fa36c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa36bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa36c0ffd60 0x7fa36c199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38394/0 (socket says 172.21.15.32:38394) 2023-09-30T14:51:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa36bfff700 1 -- 172.21.15.32:0/1669890534 learned_addr learned my addr 172.21.15.32:0/1669890534 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa370a20700 1 --2- 172.21.15.32:0/1669890534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa36c1018f0 0x7fa36c19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa36b7fe700 1 --2- 172.21.15.32:0/1669890534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa36c100f50 0x7fa36c19bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa36bfff700 1 -- 172.21.15.32:0/1669890534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa36c100f50 msgr2=0x7fa36c19bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa36bfff700 1 --2- 172.21.15.32:0/1669890534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa36c100f50 0x7fa36c19bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa36bfff700 1 -- 172.21.15.32:0/1669890534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa36c1018f0 msgr2=0x7fa36c19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa36bfff700 1 --2- 172.21.15.32:0/1669890534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa36c1018f0 0x7fa36c19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa36bfff700 1 -- 172.21.15.32:0/1669890534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa364009d00 con 0x7fa36c0ffd60 2023-09-30T14:51:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa36b7fe700 1 --2- 172.21.15.32:0/1669890534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa36c100f50 0x7fa36c19bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:51:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa36bfff700 1 --2- 172.21.15.32:0/1669890534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa36c0ffd60 0x7fa36c199970 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00e970 tx=0x7fa35c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa3697fa700 1 -- 172.21.15.32:0/1669890534 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c009790 con 0x7fa36c0ffd60 2023-09-30T14:51:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa372483700 1 -- 172.21.15.32:0/1669890534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36c19efc0 con 0x7fa36c0ffd60 2023-09-30T14:51:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.214+0000 7fa3697fa700 1 -- 172.21.15.32:0/1669890534 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa35c00eec0 con 0x7fa36c0ffd60 2023-09-30T14:51:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.215+0000 7fa3697fa700 1 -- 172.21.15.32:0/1669890534 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c003cf0 con 0x7fa36c0ffd60 2023-09-30T14:51:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.215+0000 7fa372483700 1 -- 172.21.15.32:0/1669890534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36c19f3f0 con 0x7fa36c0ffd60 2023-09-30T14:51:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.217+0000 7fa3697fa700 1 -- 172.21.15.32:0/1669890534 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fa35c00ccd0 con 0x7fa36c0ffd60 2023-09-30T14:51:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.217+0000 7fa3697fa700 1 --2- 172.21.15.32:0/1669890534 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa3540a0fa0 0x7fa3540a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.217+0000 7fa36b7fe700 1 --2- 172.21.15.32:0/1669890534 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa3540a0fa0 0x7fa3540a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.217+0000 7fa3697fa700 1 -- 172.21.15.32:0/1669890534 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa35c014070 con 0x7fa36c0ffd60 2023-09-30T14:51:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.218+0000 7fa36b7fe700 1 --2- 172.21.15.32:0/1669890534 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa3540a0fa0 0x7fa3540a3460 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0686f0 tx=0x7fa360009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.218+0000 7fa37e66bb80 1 Processor -- start 2023-09-30T14:51:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.218+0000 7fa37e66bb80 1 -- start start 2023-09-30T14:51:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.218+0000 7fa37e66bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3d200 0x557255c3d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.218+0000 7fa37e66bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3e310 0x557255c3e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.218+0000 7fa37e66bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3ecd0 0x557255c45560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.218+0000 7fa37e66bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557255ba5400 con 0x557255c3d200 2023-09-30T14:51:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.218+0000 7fa37e66bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557255b96bc0 con 0x557255c3ecd0 2023-09-30T14:51:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.219+0000 7fa37e66bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557255b944e0 con 0x557255c3e310 2023-09-30T14:51:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.219+0000 7fa36bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3d200 0x557255c3d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.219+0000 7fa36bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3d200 0x557255c3d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38402/0 (socket says 172.21.15.32:38402) 2023-09-30T14:51:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.219+0000 7fa36bfff700 1 -- 172.21.15.32:0/2669593975 learned_addr learned my addr 172.21.15.32:0/2669593975 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.219+0000 7fa370a20700 1 --2- 172.21.15.32:0/2669593975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3ecd0 0x557255c45560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.219+0000 7fa36b7fe700 1 --2- 172.21.15.32:0/2669593975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3e310 0x557255c3e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.219+0000 7fa36bfff700 1 -- 172.21.15.32:0/2669593975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3e310 msgr2=0x557255c3e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.219+0000 7fa36bfff700 1 --2- 172.21.15.32:0/2669593975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3e310 0x557255c3e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.219+0000 7fa36bfff700 1 -- 172.21.15.32:0/2669593975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3ecd0 msgr2=0x557255c45560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.219+0000 7fa36bfff700 1 --2- 172.21.15.32:0/2669593975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3ecd0 0x557255c45560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.219+0000 7fa36bfff700 1 -- 172.21.15.32:0/2669593975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557255bf2870 con 0x557255c3d200 2023-09-30T14:51:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.219+0000 7fa36bfff700 1 --2- 172.21.15.32:0/2669593975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3d200 0x557255c3d620 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0a1c50 tx=0x7fa35c099a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4c7e8392ddf317 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.219+0000 7fa3527fc700 1 -- 172.21.15.32:0/2669593975 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c099df0 con 0x557255c3d200 2023-09-30T14:51:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.219+0000 7fa3527fc700 1 -- 172.21.15.32:0/2669593975 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa35c09df40 con 0x557255c3d200 2023-09-30T14:51:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.219+0000 7fa3527fc700 1 -- 172.21.15.32:0/2669593975 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c09a2b0 con 0x557255c3d200 2023-09-30T14:51:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.220+0000 7fa37e66bb80 1 -- 172.21.15.32:0/2669593975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3d200 msgr2=0x557255c3d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.220+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/2669593975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3d200 0x557255c3d620 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0a1c50 tx=0x7fa35c099a00 comp rx=0 tx=0).stop 2023-09-30T14:51:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.220+0000 7fa37e66bb80 1 -- 172.21.15.32:0/2669593975 shutdown_connections 2023-09-30T14:51:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.220+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/2669593975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3ecd0 0x557255c45560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.220+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/2669593975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3d200 0x557255c3d620 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.220+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/2669593975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3e310 0x557255c3e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.220+0000 7fa37e66bb80 1 -- 172.21.15.32:0/2669593975 >> 172.21.15.32:0/2669593975 conn(0x557255bb1200 msgr2=0x557255c37ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.220+0000 7fa37e66bb80 1 -- 172.21.15.32:0/2669593975 shutdown_connections 2023-09-30T14:51:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.220+0000 7fa37e66bb80 1 -- 172.21.15.32:0/2669593975 wait complete. 2023-09-30T14:51:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.220+0000 7fa37e66bb80 1 -- start start 2023-09-30T14:51:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.221+0000 7fa37e66bb80 1 Processor -- start 2023-09-30T14:51:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.221+0000 7fa37e66bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3d200 0x557255c46ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.221+0000 7fa37e66bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3e310 0x557255c47420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.221+0000 7fa37e66bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3ecd0 0x557255c4e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.221+0000 7fa37e66bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557255b8fdd0 con 0x557255c3e310 2023-09-30T14:51:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.221+0000 7fa37e66bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557255b2c6b0 con 0x557255c3d200 2023-09-30T14:51:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.221+0000 7fa37e66bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557255b2c350 con 0x557255c3ecd0 2023-09-30T14:51:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.221+0000 7fa37e66bb80 10 client.0 _ll_register_callbacks cb 0x5572559897a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:51:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.221+0000 7fa36bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3d200 0x557255c46ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.221+0000 7fa370a20700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3ecd0 0x557255c4e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.221+0000 7fa36bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3d200 0x557255c46ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49884/0 (socket says 172.21.15.32:49884) 2023-09-30T14:51:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.221+0000 7fa36bfff700 1 -- 172.21.15.32:0/15863618 learned_addr learned my addr 172.21.15.32:0/15863618 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.221+0000 7fa36b7fe700 1 --2- 172.21.15.32:0/15863618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3e310 0x557255c47420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.222+0000 7fa36bfff700 1 -- 172.21.15.32:0/15863618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3ecd0 msgr2=0x557255c4e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.222+0000 7fa36bfff700 1 --2- 172.21.15.32:0/15863618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3ecd0 0x557255c4e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.222+0000 7fa36bfff700 1 -- 172.21.15.32:0/15863618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3e310 msgr2=0x557255c47420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.222+0000 7fa36bfff700 1 --2- 172.21.15.32:0/15863618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3e310 0x557255c47420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.222+0000 7fa36bfff700 1 -- 172.21.15.32:0/15863618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36c19f3f0 con 0x557255c3d200 2023-09-30T14:51:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.222+0000 7fa36bfff700 1 --2- 172.21.15.32:0/15863618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3d200 0x557255c46ee0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa35c099d80 tx=0x7fa35c095b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77fb30e938a56350 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.222+0000 7fa34b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:51:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.222+0000 7fa34b7fe700 1 -- 172.21.15.32:0/15863618 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c09a250 con 0x557255c3d200 2023-09-30T14:51:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.222+0000 7fa37e66bb80 1 -- 172.21.15.32:0/15863618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557255c30090 con 0x557255c3d200 2023-09-30T14:51:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.222+0000 7fa34b7fe700 1 -- 172.21.15.32:0/15863618 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa35c099df0 con 0x557255c3d200 2023-09-30T14:51:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.222+0000 7fa34b7fe700 1 -- 172.21.15.32:0/15863618 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa35c0a1cf0 con 0x557255c3d200 2023-09-30T14:51:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.222+0000 7fa37e66bb80 10 client.26321 fetch_fsmap learned FSMap version 46 2023-09-30T14:51:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.222+0000 7fa37e66bb80 1 -- 172.21.15.32:0/15863618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5572559bad80 con 0x557255c3d200 2023-09-30T14:51:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.222+0000 7fa34b7fe700 1 -- 172.21.15.32:0/15863618 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557255c30090 con 0x557255c3d200 2023-09-30T14:51:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.223+0000 7fa34b7fe700 1 -- 172.21.15.32:0/15863618 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 46) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa35c081070 con 0x557255c3d200 2023-09-30T14:51:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.223+0000 7fa37e66bb80 10 client.26321 fetch_fsmap finished waiting for FSMap version 46 2023-09-30T14:51:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.223+0000 7fa37e66bb80 10 client.26321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:51:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.223+0000 7fa37e66bb80 20 client.26321 populate_metadata read hostname 'smithi032' 2023-09-30T14:51:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.223+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/15863618 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557255c52b10 0x557255c54f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.223+0000 7fa37e66bb80 4 client.26321 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:51:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.223+0000 7fa37e66bb80 1 -- 172.21.15.32:0/15863618 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557255b88db0 con 0x557255c52b10 2023-09-30T14:51:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.223+0000 7fa36b7fe700 1 --2- 172.21.15.32:0/15863618 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557255c52b10 0x557255c54f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.223+0000 7fa34b7fe700 10 client.26321 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.223+0000 7fa36b7fe700 1 --2- 172.21.15.32:0/15863618 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557255c52b10 0x557255c54f30 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3345e0d0143e2404 server_cookie=9c703a60a652ef0d in_seq=0 out_seq=0 2023-09-30T14:51:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.224+0000 7fa34b7fe700 1 -- 172.21.15.32:0/15863618 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557255b88db0 con 0x557255c52b10 2023-09-30T14:51:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.224+0000 7fa34b7fe700 10 client.26321 handle_command_reply: tid=0 2023-09-30T14:51:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.224+0000 7fa37e66bb80 1 client.26321 shutdown 2023-09-30T14:51:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.224+0000 7fa37e66bb80 1 -- 172.21.15.32:0/15863618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3d200 msgr2=0x557255c46ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.224+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/15863618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3d200 0x557255c46ee0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa35c099d80 tx=0x7fa35c095b00 comp rx=0 tx=0).stop 2023-09-30T14:51:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.224+0000 7fa37e66bb80 1 -- 172.21.15.32:0/15863618 shutdown_connections 2023-09-30T14:51:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.224+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/15863618 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557255c52b10 0x557255c54f30 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.224+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/15863618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3ecd0 0x557255c4e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.224+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/15863618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3d200 0x557255c46ee0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.224+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/15863618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3e310 0x557255c47420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.224+0000 7fa34b7fe700 0 client.26321 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.224+0000 7fa37e66bb80 1 -- 172.21.15.32:0/15863618 >> 172.21.15.32:0/15863618 conn(0x557255bb1200 msgr2=0x557255c37590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.224+0000 7fa37e66bb80 1 -- 172.21.15.32:0/15863618 shutdown_connections 2023-09-30T14:51:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.224+0000 7fa37e66bb80 1 -- 172.21.15.32:0/15863618 wait complete. 2023-09-30T14:51:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.224+0000 7fa37e66bb80 20 client.26321 trim_cache size 0 max 16384 2023-09-30T14:51:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.240+0000 7fa37e66bb80 1 Processor -- start 2023-09-30T14:51:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.240+0000 7fa37e66bb80 1 -- start start 2023-09-30T14:51:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa37e66bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3d200 0x557255c45c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa37e66bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3e310 0x557255c4e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa37e66bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3ecd0 0x557255c37590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa37e66bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557255a108c0 con 0x557255c3ecd0 2023-09-30T14:51:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa37e66bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557255c39290 con 0x557255c3e310 2023-09-30T14:51:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa37e66bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557255c3da80 con 0x557255c3d200 2023-09-30T14:51:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa370a20700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3ecd0 0x557255c37590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa370a20700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3ecd0 0x557255c37590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38420/0 (socket says 172.21.15.32:38420) 2023-09-30T14:51:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa36bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3d200 0x557255c45c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa370a20700 1 -- 172.21.15.32:0/3509022011 learned_addr learned my addr 172.21.15.32:0/3509022011 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa370a20700 1 -- 172.21.15.32:0/3509022011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3d200 msgr2=0x557255c45c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa370a20700 1 --2- 172.21.15.32:0/3509022011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3d200 0x557255c45c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa370a20700 1 -- 172.21.15.32:0/3509022011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3e310 msgr2=0x557255c4e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa370a20700 1 --2- 172.21.15.32:0/3509022011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3e310 0x557255c4e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa370a20700 1 -- 172.21.15.32:0/3509022011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572559a3c00 con 0x557255c3ecd0 2023-09-30T14:51:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa36bfff700 1 --2- 172.21.15.32:0/3509022011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3d200 0x557255c45c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:51:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa370a20700 1 --2- 172.21.15.32:0/3509022011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3ecd0 0x557255c37590 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa3640094b0 tx=0x7fa3640094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edf23759ba6036e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa352ffd700 1 -- 172.21.15.32:0/3509022011 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36400e040 con 0x557255c3ecd0 2023-09-30T14:51:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa352ffd700 1 -- 172.21.15.32:0/3509022011 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa364008e90 con 0x557255c3ecd0 2023-09-30T14:51:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.241+0000 7fa352ffd700 1 -- 172.21.15.32:0/3509022011 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364005070 con 0x557255c3ecd0 2023-09-30T14:51:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.242+0000 7fa37e66bb80 1 -- 172.21.15.32:0/3509022011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3ecd0 msgr2=0x557255c37590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.242+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/3509022011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3ecd0 0x557255c37590 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa3640094b0 tx=0x7fa3640094e0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.242+0000 7fa37e66bb80 1 -- 172.21.15.32:0/3509022011 shutdown_connections 2023-09-30T14:51:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.242+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/3509022011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3ecd0 0x557255c37590 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.242+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/3509022011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3d200 0x557255c45c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.242+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/3509022011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3e310 0x557255c4e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.242+0000 7fa37e66bb80 1 -- 172.21.15.32:0/3509022011 >> 172.21.15.32:0/3509022011 conn(0x557255bb1200 msgr2=0x557255c37ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.242+0000 7fa37e66bb80 1 -- 172.21.15.32:0/3509022011 shutdown_connections 2023-09-30T14:51:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.242+0000 7fa37e66bb80 1 -- 172.21.15.32:0/3509022011 wait complete. 2023-09-30T14:51:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.243+0000 7fa37e66bb80 1 -- start start 2023-09-30T14:51:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.243+0000 7fa37e66bb80 1 Processor -- start 2023-09-30T14:51:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.243+0000 7fa37e66bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3d200 0x557255c4bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.243+0000 7fa37e66bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3e310 0x557255c4c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.243+0000 7fa37e66bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3ecd0 0x557255c4d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.243+0000 7fa37e66bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557255c4da80 con 0x557255c3d200 2023-09-30T14:51:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.243+0000 7fa37e66bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557255c4dbf0 con 0x557255c3e310 2023-09-30T14:51:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.243+0000 7fa37e66bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557255c4ddb0 con 0x557255c3ecd0 2023-09-30T14:51:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.243+0000 7fa37e66bb80 10 client.0 _ll_register_callbacks cb 0x7fa35c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:51:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.243+0000 7fa36bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3d200 0x557255c4bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.243+0000 7fa36bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3d200 0x557255c4bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38422/0 (socket says 172.21.15.32:38422) 2023-09-30T14:51:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.243+0000 7fa36bfff700 1 -- 172.21.15.32:0/3650524180 learned_addr learned my addr 172.21.15.32:0/3650524180 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:51:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.243+0000 7fa36b7fe700 1 --2- 172.21.15.32:0/3650524180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3e310 0x557255c4c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.243+0000 7fa370a20700 1 --2- 172.21.15.32:0/3650524180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3ecd0 0x557255c4d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.244+0000 7fa36bfff700 1 -- 172.21.15.32:0/3650524180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3ecd0 msgr2=0x557255c4d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.244+0000 7fa36bfff700 1 --2- 172.21.15.32:0/3650524180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3ecd0 0x557255c4d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.244+0000 7fa36bfff700 1 -- 172.21.15.32:0/3650524180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3e310 msgr2=0x557255c4c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:51:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.244+0000 7fa36bfff700 1 --2- 172.21.15.32:0/3650524180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3e310 0x557255c4c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.244+0000 7fa36bfff700 1 -- 172.21.15.32:0/3650524180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572559bad80 con 0x557255c3d200 2023-09-30T14:51:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.244+0000 7fa370a20700 1 --2- 172.21.15.32:0/3650524180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3ecd0 0x557255c4d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:51:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.244+0000 7fa36bfff700 1 --2- 172.21.15.32:0/3650524180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3d200 0x557255c4bff0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0962f0 tx=0x7fa35c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc4ba96a65b4ff7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:51:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.244+0000 7fa34b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:51:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.244+0000 7fa34b7fe700 1 -- 172.21.15.32:0/3650524180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c01fec0 con 0x557255c3d200 2023-09-30T14:51:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.244+0000 7fa34b7fe700 1 -- 172.21.15.32:0/3650524180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa35c098d50 con 0x557255c3d200 2023-09-30T14:51:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.244+0000 7fa37e66bb80 1 -- 172.21.15.32:0/3650524180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa35c09df40 con 0x557255c3d200 2023-09-30T14:51:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.244+0000 7fa34b7fe700 1 -- 172.21.15.32:0/3650524180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa35c099df0 con 0x557255c3d200 2023-09-30T14:51:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.244+0000 7fa37e66bb80 10 client.16644 fetch_fsmap learned FSMap version 46 2023-09-30T14:51:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.244+0000 7fa37e66bb80 1 -- 172.21.15.32:0/3650524180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7fa36c108af0 con 0x557255c3d200 2023-09-30T14:51:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.244+0000 7fa34b7fe700 1 -- 172.21.15.32:0/3650524180 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa35c09df40 con 0x557255c3d200 2023-09-30T14:51:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.245+0000 7fa34b7fe700 1 -- 172.21.15.32:0/3650524180 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 46) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa35c081070 con 0x557255c3d200 2023-09-30T14:51:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.245+0000 7fa37e66bb80 10 client.16644 fetch_fsmap finished waiting for FSMap version 46 2023-09-30T14:51:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.245+0000 7fa37e66bb80 10 client.16644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:51:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.245+0000 7fa37e66bb80 20 client.16644 populate_metadata read hostname 'smithi032' 2023-09-30T14:51:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.245+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/3650524180 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557255ca7250 0x557255ca9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:51:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.245+0000 7fa37e66bb80 4 client.16644 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:51:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.245+0000 7fa37e66bb80 1 -- 172.21.15.32:0/3650524180 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557255b88ad0 con 0x557255ca7250 2023-09-30T14:51:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.245+0000 7fa36b7fe700 1 --2- 172.21.15.32:0/3650524180 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557255ca7250 0x557255ca9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:51:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.245+0000 7fa34b7fe700 10 client.16644 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.245+0000 7fa36b7fe700 1 --2- 172.21.15.32:0/3650524180 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557255ca7250 0x557255ca9640 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc29a559b3a9a837 server_cookie=a6692caaec452251 in_seq=0 out_seq=0 2023-09-30T14:51:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.409+0000 7fa34b7fe700 1 -- 172.21.15.32:0/3650524180 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557255b88ad0 con 0x557255ca7250 2023-09-30T14:51:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.409+0000 7fa34b7fe700 10 client.16644 handle_command_reply: tid=0 2023-09-30T14:51:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.409+0000 7fa37e66bb80 1 client.16644 shutdown 2023-09-30T14:51:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.409+0000 7fa37e66bb80 1 -- 172.21.15.32:0/3650524180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3d200 msgr2=0x557255c4bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.409+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/3650524180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3d200 0x557255c4bff0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0962f0 tx=0x7fa35c000bc0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.409+0000 7fa37e66bb80 1 -- 172.21.15.32:0/3650524180 shutdown_connections 2023-09-30T14:51:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.409+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/3650524180 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557255ca7250 0x557255ca9640 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.409+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/3650524180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557255c3ecd0 0x557255c4d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.409+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/3650524180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557255c3d200 0x557255c4bff0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.409+0000 7fa37e66bb80 1 --2- 172.21.15.32:0/3650524180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255c3e310 0x557255c4c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.409+0000 7fa34b7fe700 0 client.16644 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:51:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.409+0000 7fa37e66bb80 1 -- 172.21.15.32:0/3650524180 >> 172.21.15.32:0/3650524180 conn(0x557255bb1200 msgr2=0x557255c41bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.409+0000 7fa37e66bb80 1 -- 172.21.15.32:0/3650524180 shutdown_connections 2023-09-30T14:51:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.409+0000 7fa37e66bb80 1 -- 172.21.15.32:0/3650524180 wait complete. 2023-09-30T14:51:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.409+0000 7fa37e66bb80 20 client.16644 trim_cache size 0 max 16384 2023-09-30T14:51:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:51:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:51:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:51:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:51:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.410+0000 7fa372483700 1 -- 172.21.15.32:0/1669890534 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa3540a0fa0 msgr2=0x7fa3540a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.410+0000 7fa372483700 1 --2- 172.21.15.32:0/1669890534 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa3540a0fa0 0x7fa3540a3460 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0686f0 tx=0x7fa360009260 comp rx=0 tx=0).stop 2023-09-30T14:51:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.410+0000 7fa372483700 1 -- 172.21.15.32:0/1669890534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa36c0ffd60 msgr2=0x7fa36c199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:51:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.410+0000 7fa372483700 1 --2- 172.21.15.32:0/1669890534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa36c0ffd60 0x7fa36c199970 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00e970 tx=0x7fa35c00ec80 comp rx=0 tx=0).stop 2023-09-30T14:51:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.410+0000 7fa372483700 1 -- 172.21.15.32:0/1669890534 shutdown_connections 2023-09-30T14:51:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.410+0000 7fa372483700 1 --2- 172.21.15.32:0/1669890534 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa3540a0fa0 0x7fa3540a3460 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.410+0000 7fa372483700 1 --2- 172.21.15.32:0/1669890534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa36c1018f0 0x7fa36c19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.410+0000 7fa372483700 1 --2- 172.21.15.32:0/1669890534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa36c0ffd60 0x7fa36c199970 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.410+0000 7fa372483700 1 --2- 172.21.15.32:0/1669890534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa36c100f50 0x7fa36c19bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:51:57.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.410+0000 7fa372483700 1 -- 172.21.15.32:0/1669890534 >> 172.21.15.32:0/1669890534 conn(0x7fa36c0fb2b0 msgr2=0x7fa36c104400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:51:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.410+0000 7fa372483700 1 -- 172.21.15.32:0/1669890534 shutdown_connections 2023-09-30T14:51:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:51:57.410+0000 7fa372483700 1 -- 172.21.15.32:0/1669890534 wait complete. 2023-09-30T14:51:57.999 INFO:tasks.workunit.client.0.smithi032.stdout: 10 14263 30.88 MB/sec warmup 31 sec latency 108.807 ms 2023-09-30T14:51:58.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:57 smithi156 ceph-mon[103579]: pgmap v697: 97 pgs: 97 active+clean; 405 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 339 op/s 2023-09-30T14:51:58.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:57 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:51:58.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:57 smithi148 ceph-mon[102355]: pgmap v697: 97 pgs: 97 active+clean; 405 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 339 op/s 2023-09-30T14:51:58.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:57 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:51:58.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:57 smithi032 ceph-mon[98592]: pgmap v697: 97 pgs: 97 active+clean; 405 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 339 op/s 2023-09-30T14:51:58.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:57 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:51:58.999 INFO:tasks.workunit.client.0.smithi032.stdout: 10 14646 30.29 MB/sec warmup 32 sec latency 76.829 ms 2023-09-30T14:52:00.000 INFO:tasks.workunit.client.0.smithi032.stdout: 10 15019 29.67 MB/sec warmup 33 sec latency 345.544 ms 2023-09-30T14:52:00.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:51:59 smithi156 ceph-mon[103579]: pgmap v698: 97 pgs: 97 active+clean; 406 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 320 op/s 2023-09-30T14:52:00.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:51:59 smithi148 ceph-mon[102355]: pgmap v698: 97 pgs: 97 active+clean; 406 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 320 op/s 2023-09-30T14:52:00.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:51:59 smithi032 ceph-mon[98592]: pgmap v698: 97 pgs: 97 active+clean; 406 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 320 op/s 2023-09-30T14:52:01.000 INFO:tasks.workunit.client.0.smithi032.stdout: 10 15499 29.67 MB/sec warmup 34 sec latency 442.975 ms 2023-09-30T14:52:01.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:00 smithi156 ceph-mon[103579]: pgmap v699: 97 pgs: 97 active+clean; 407 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 331 op/s 2023-09-30T14:52:01.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:00 smithi148 ceph-mon[102355]: pgmap v699: 97 pgs: 97 active+clean; 407 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 331 op/s 2023-09-30T14:52:01.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:00 smithi032 ceph-mon[98592]: pgmap v699: 97 pgs: 97 active+clean; 407 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 331 op/s 2023-09-30T14:52:02.000 INFO:tasks.workunit.client.0.smithi032.stdout: 10 16003 29.56 MB/sec warmup 35 sec latency 114.848 ms 2023-09-30T14:52:03.000 INFO:tasks.workunit.client.0.smithi032.stdout: 10 16741 29.73 MB/sec warmup 36 sec latency 65.737 ms 2023-09-30T14:52:03.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:03 smithi156 ceph-mon[103579]: pgmap v700: 97 pgs: 97 active+clean; 530 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 14 MiB/s wr, 413 op/s 2023-09-30T14:52:03.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:03 smithi148 ceph-mon[102355]: pgmap v700: 97 pgs: 97 active+clean; 530 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 14 MiB/s wr, 413 op/s 2023-09-30T14:52:03.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:03 smithi032 ceph-mon[98592]: pgmap v700: 97 pgs: 97 active+clean; 530 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 14 MiB/s wr, 413 op/s 2023-09-30T14:52:04.000 INFO:tasks.workunit.client.0.smithi032.stdout: 10 17482 29.88 MB/sec warmup 37 sec latency 42.845 ms 2023-09-30T14:52:05.000 INFO:tasks.workunit.client.0.smithi032.stdout: 10 18038 30.16 MB/sec warmup 38 sec latency 200.627 ms 2023-09-30T14:52:05.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:05 smithi156 ceph-mon[103579]: pgmap v701: 97 pgs: 97 active+clean; 584 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 18 MiB/s wr, 435 op/s 2023-09-30T14:52:05.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:05 smithi148 ceph-mon[102355]: pgmap v701: 97 pgs: 97 active+clean; 584 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 18 MiB/s wr, 435 op/s 2023-09-30T14:52:05.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:05 smithi032 ceph-mon[98592]: pgmap v701: 97 pgs: 97 active+clean; 584 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 18 MiB/s wr, 435 op/s 2023-09-30T14:52:06.000 INFO:tasks.workunit.client.0.smithi032.stdout: 10 18522 30.33 MB/sec warmup 39 sec latency 372.456 ms 2023-09-30T14:52:07.000 INFO:tasks.workunit.client.0.smithi032.stdout: 10 18998 30.02 MB/sec warmup 40 sec latency 99.079 ms 2023-09-30T14:52:07.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:07 smithi156 ceph-mon[103579]: pgmap v702: 97 pgs: 97 active+clean; 624 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 21 MiB/s wr, 483 op/s 2023-09-30T14:52:07.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:07 smithi148 ceph-mon[102355]: pgmap v702: 97 pgs: 97 active+clean; 624 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 21 MiB/s wr, 483 op/s 2023-09-30T14:52:07.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:07 smithi032 ceph-mon[98592]: pgmap v702: 97 pgs: 97 active+clean; 624 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 21 MiB/s wr, 483 op/s 2023-09-30T14:52:08.000 INFO:tasks.workunit.client.0.smithi032.stdout: 10 19725 30.23 MB/sec warmup 41 sec latency 91.192 ms 2023-09-30T14:52:08.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:08 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:52:08.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:08 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:52:08.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:08 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:52:09.000 INFO:tasks.workunit.client.0.smithi032.stdout: 10 20647 30.52 MB/sec warmup 42 sec latency 41.101 ms 2023-09-30T14:52:09.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:09 smithi156 ceph-mon[103579]: pgmap v703: 97 pgs: 97 active+clean; 628 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 22 MiB/s wr, 528 op/s 2023-09-30T14:52:09.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:09 smithi148 ceph-mon[102355]: pgmap v703: 97 pgs: 97 active+clean; 628 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 22 MiB/s wr, 528 op/s 2023-09-30T14:52:09.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:09 smithi032 ceph-mon[98592]: pgmap v703: 97 pgs: 97 active+clean; 628 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 22 MiB/s wr, 528 op/s 2023-09-30T14:52:10.000 INFO:tasks.workunit.client.0.smithi032.stdout: 10 21603 30.90 MB/sec warmup 43 sec latency 137.150 ms 2023-09-30T14:52:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.539+0000 7f9b92b13700 1 -- 172.21.15.32:0/1665583696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8c0ffd60 msgr2=0x7f9b8c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.539+0000 7f9b92b13700 1 --2- 172.21.15.32:0/1665583696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8c0ffd60 0x7f9b8c100180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9b800099c0 tx=0x7f9b80009cd0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.539+0000 7f9b92b13700 1 -- 172.21.15.32:0/1665583696 shutdown_connections 2023-09-30T14:52:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.539+0000 7f9b92b13700 1 --2- 172.21.15.32:0/1665583696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b8c1018f0 0x7f9b8c1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.539+0000 7f9b92b13700 1 --2- 172.21.15.32:0/1665583696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8c0ffd60 0x7f9b8c100180 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.539+0000 7f9b92b13700 1 --2- 172.21.15.32:0/1665583696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b8c100f50 0x7f9b8c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.539+0000 7f9b92b13700 1 -- 172.21.15.32:0/1665583696 >> 172.21.15.32:0/1665583696 conn(0x7f9b8c0fb2b0 msgr2=0x7f9b8c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.539+0000 7f9b92b13700 1 -- 172.21.15.32:0/1665583696 shutdown_connections 2023-09-30T14:52:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.539+0000 7f9b92b13700 1 -- 172.21.15.32:0/1665583696 wait complete. 2023-09-30T14:52:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.540+0000 7f9b92b13700 1 Processor -- start 2023-09-30T14:52:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.540+0000 7f9b92b13700 1 -- start start 2023-09-30T14:52:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.540+0000 7f9b92b13700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b8c0ffd60 0x7f9b8c1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.540+0000 7f9b92b13700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8c100f50 0x7f9b8c19be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.540+0000 7f9b92b13700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b8c1018f0 0x7f9b8c19e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.540+0000 7f9b92b13700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c19e920 con 0x7f9b8c1018f0 2023-09-30T14:52:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.540+0000 7f9b92b13700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c19ea60 con 0x7f9b8c100f50 2023-09-30T14:52:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.540+0000 7f9b92b13700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c19ec20 con 0x7f9b8c0ffd60 2023-09-30T14:52:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.540+0000 7f9b908af700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b8c0ffd60 0x7f9b8c1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.540+0000 7f9b8bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8c100f50 0x7f9b8c19be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.540+0000 7f9b8bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8c100f50 0x7f9b8c19be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49918/0 (socket says 172.21.15.32:49918) 2023-09-30T14:52:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.540+0000 7f9b908af700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b8c0ffd60 0x7f9b8c1998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52822/0 (socket says 172.21.15.32:52822) 2023-09-30T14:52:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.541+0000 7f9b8bfff700 1 -- 172.21.15.32:0/1461147378 learned_addr learned my addr 172.21.15.32:0/1461147378 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.541+0000 7f9b910b0700 1 --2- 172.21.15.32:0/1461147378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b8c1018f0 0x7f9b8c19e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.541+0000 7f9b908af700 1 -- 172.21.15.32:0/1461147378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8c100f50 msgr2=0x7f9b8c19be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.541+0000 7f9b908af700 1 --2- 172.21.15.32:0/1461147378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8c100f50 0x7f9b8c19be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.541+0000 7f9b908af700 1 -- 172.21.15.32:0/1461147378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b8c1018f0 msgr2=0x7f9b8c19e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.541+0000 7f9b908af700 1 --2- 172.21.15.32:0/1461147378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b8c1018f0 0x7f9b8c19e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.541+0000 7f9b908af700 1 -- 172.21.15.32:0/1461147378 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b800095f0 con 0x7f9b8c0ffd60 2023-09-30T14:52:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.541+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/1461147378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8c100f50 0x7f9b8c19be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:52:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.541+0000 7f9b908af700 1 --2- 172.21.15.32:0/1461147378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b8c0ffd60 0x7f9b8c1998b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9b800058e0 tx=0x7f9b8000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.541+0000 7f9b89ffb700 1 -- 172.21.15.32:0/1461147378 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8001d070 con 0x7f9b8c0ffd60 2023-09-30T14:52:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.542+0000 7f9b92b13700 1 -- 172.21.15.32:0/1461147378 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8c195690 con 0x7f9b8c0ffd60 2023-09-30T14:52:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.542+0000 7f9b89ffb700 1 -- 172.21.15.32:0/1461147378 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b80005020 con 0x7f9b8c0ffd60 2023-09-30T14:52:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.542+0000 7f9b89ffb700 1 -- 172.21.15.32:0/1461147378 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80004a90 con 0x7f9b8c0ffd60 2023-09-30T14:52:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.542+0000 7f9b92b13700 1 -- 172.21.15.32:0/1461147378 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8c195b00 con 0x7f9b8c0ffd60 2023-09-30T14:52:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.545+0000 7f9b89ffb700 1 -- 172.21.15.32:0/1461147378 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f9b80005400 con 0x7f9b8c0ffd60 2023-09-30T14:52:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.545+0000 7f9b89ffb700 1 --2- 172.21.15.32:0/1461147378 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9b7c0a1020 0x7f9b7c0a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.546+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/1461147378 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9b7c0a1020 0x7f9b7c0a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.546+0000 7f9b89ffb700 1 -- 172.21.15.32:0/1461147378 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9b800cd260 con 0x7f9b8c0ffd60 2023-09-30T14:52:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.546+0000 7f9b9ecfbb80 1 Processor -- start 2023-09-30T14:52:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.546+0000 7f9b9ecfbb80 1 -- start start 2023-09-30T14:52:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.546+0000 7f9b9ecfbb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb1200 0x562741eb1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.546+0000 7f9b9ecfbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 0x562741eb2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.546+0000 7f9b9ecfbb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb2cd0 0x562741eb9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.546+0000 7f9b9ecfbb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741e19400 con 0x562741eb1200 2023-09-30T14:52:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.546+0000 7f9b9ecfbb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741e0abc0 con 0x562741eb2cd0 2023-09-30T14:52:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.546+0000 7f9b9ecfbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741e084e0 con 0x562741eb2310 2023-09-30T14:52:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.546+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/1461147378 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9b7c0a1020 0x7f9b7c0a34e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c068710 tx=0x7f9b78012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.547+0000 7f9b908af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb1200 0x562741eb1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.547+0000 7f9b908af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb1200 0x562741eb1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38448/0 (socket says 172.21.15.32:38448) 2023-09-30T14:52:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.547+0000 7f9b908af700 1 -- 172.21.15.32:0/2435235899 learned_addr learned my addr 172.21.15.32:0/2435235899 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.547+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/2435235899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 0x562741eb2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.547+0000 7f9b910b0700 1 --2- 172.21.15.32:0/2435235899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb2cd0 0x562741eb9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.547+0000 7f9b8bfff700 1 -- 172.21.15.32:0/2435235899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb2cd0 msgr2=0x562741eb9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.547+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/2435235899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb2cd0 0x562741eb9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.547+0000 7f9b8bfff700 1 -- 172.21.15.32:0/2435235899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb1200 msgr2=0x562741eb1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.547+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/2435235899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb1200 0x562741eb1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.547+0000 7f9b8bfff700 1 -- 172.21.15.32:0/2435235899 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562741e66870 con 0x562741eb2310 2023-09-30T14:52:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.547+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/2435235899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 0x562741eb2790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b78009f10 tx=0x7f9b78018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d876f285707fd99e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.547+0000 7f9b76ffd700 1 -- 172.21.15.32:0/2435235899 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78003b10 con 0x562741eb2310 2023-09-30T14:52:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.547+0000 7f9b76ffd700 1 -- 172.21.15.32:0/2435235899 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b78016040 con 0x562741eb2310 2023-09-30T14:52:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.547+0000 7f9b76ffd700 1 -- 172.21.15.32:0/2435235899 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7801c3f0 con 0x562741eb2310 2023-09-30T14:52:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.548+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/2435235899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 msgr2=0x562741eb2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.548+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/2435235899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 0x562741eb2790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b78009f10 tx=0x7f9b78018000 comp rx=0 tx=0).stop 2023-09-30T14:52:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.548+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/2435235899 shutdown_connections 2023-09-30T14:52:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.548+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/2435235899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb2cd0 0x562741eb9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.548+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/2435235899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb1200 0x562741eb1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.548+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/2435235899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 0x562741eb2790 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.548+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/2435235899 >> 172.21.15.32:0/2435235899 conn(0x562741e25200 msgr2=0x562741eabef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.548+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/2435235899 shutdown_connections 2023-09-30T14:52:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.548+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/2435235899 wait complete. 2023-09-30T14:52:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.549+0000 7f9b9ecfbb80 1 -- start start 2023-09-30T14:52:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.549+0000 7f9b9ecfbb80 1 Processor -- start 2023-09-30T14:52:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.549+0000 7f9b9ecfbb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb1200 0x562741ebaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.549+0000 7f9b9ecfbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 0x562741ebb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b9ecfbb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb2cd0 0x562741ec24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b9ecfbb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741e03dd0 con 0x562741eb2cd0 2023-09-30T14:52:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b9ecfbb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741da06b0 con 0x562741eb1200 2023-09-30T14:52:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b9ecfbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741da0350 con 0x562741eb2310 2023-09-30T14:52:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b9ecfbb80 10 client.0 _ll_register_callbacks cb 0x562741bfd7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:52:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b908af700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb1200 0x562741ebaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b908af700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb1200 0x562741ebaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49930/0 (socket says 172.21.15.32:49930) 2023-09-30T14:52:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b908af700 1 -- 172.21.15.32:0/3807806023 learned_addr learned my addr 172.21.15.32:0/3807806023 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/3807806023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 0x562741ebb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b910b0700 1 --2- 172.21.15.32:0/3807806023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb2cd0 0x562741ec24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b8bfff700 1 -- 172.21.15.32:0/3807806023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb1200 msgr2=0x562741ebaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/3807806023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb1200 0x562741ebaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b8bfff700 1 -- 172.21.15.32:0/3807806023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb2cd0 msgr2=0x562741ec24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/3807806023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb2cd0 0x562741ec24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b8bfff700 1 -- 172.21.15.32:0/3807806023 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b78004080 con 0x562741eb2310 2023-09-30T14:52:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b910b0700 1 --2- 172.21.15.32:0/3807806023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb2cd0 0x562741ec24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:52:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/3807806023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 0x562741ebb470 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b78012010 tx=0x7f9b780105a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8463449090ca169c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.550+0000 7f9b6bfff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:52:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.551+0000 7f9b6bfff700 1 -- 172.21.15.32:0/3807806023 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78003d20 con 0x562741eb2310 2023-09-30T14:52:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.551+0000 7f9b6bfff700 1 -- 172.21.15.32:0/3807806023 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9b7800b400 con 0x562741eb2310 2023-09-30T14:52:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.551+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/3807806023 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562741ea4090 con 0x562741eb2310 2023-09-30T14:52:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.551+0000 7f9b6bfff700 1 -- 172.21.15.32:0/3807806023 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b7800b5c0 con 0x562741eb2310 2023-09-30T14:52:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.551+0000 7f9b9ecfbb80 10 client.26323 fetch_fsmap learned FSMap version 47 2023-09-30T14:52:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.551+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/3807806023 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x562741c2ed80 con 0x562741eb2310 2023-09-30T14:52:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.551+0000 7f9b6bfff700 1 -- 172.21.15.32:0/3807806023 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562741ea4090 con 0x562741eb2310 2023-09-30T14:52:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.551+0000 7f9b6bfff700 1 -- 172.21.15.32:0/3807806023 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 47) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9b78004680 con 0x562741eb2310 2023-09-30T14:52:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.551+0000 7f9b9ecfbb80 10 client.26323 fetch_fsmap finished waiting for FSMap version 47 2023-09-30T14:52:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.551+0000 7f9b9ecfbb80 10 client.26323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:52:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.551+0000 7f9b9ecfbb80 20 client.26323 populate_metadata read hostname 'smithi032' 2023-09-30T14:52:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.551+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/3807806023 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562741ec6cc0 0x562741ec90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.551+0000 7f9b9ecfbb80 4 client.26323 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:52:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.551+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/3807806023 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562741dfcdb0 con 0x562741ec6cc0 2023-09-30T14:52:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.552+0000 7f9b908af700 1 --2- 172.21.15.32:0/3807806023 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562741ec6cc0 0x562741ec90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.552+0000 7f9b6bfff700 10 client.26323 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:52:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.552+0000 7f9b908af700 1 --2- 172.21.15.32:0/3807806023 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562741ec6cc0 0x562741ec90e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=998bc3216742f53a server_cookie=ecb397c852993e64 in_seq=0 out_seq=0 2023-09-30T14:52:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.554+0000 7f9b6bfff700 1 -- 172.21.15.32:0/3807806023 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562741dfcdb0 con 0x562741ec6cc0 2023-09-30T14:52:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.554+0000 7f9b6bfff700 10 client.26323 handle_command_reply: tid=0 2023-09-30T14:52:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.554+0000 7f9b9ecfbb80 1 client.26323 shutdown 2023-09-30T14:52:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.554+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/3807806023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 msgr2=0x562741ebb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.554+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/3807806023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 0x562741ebb470 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b78012010 tx=0x7f9b780105a0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.554+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/3807806023 shutdown_connections 2023-09-30T14:52:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.554+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/3807806023 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562741ec6cc0 0x562741ec90e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.554+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/3807806023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb2cd0 0x562741ec24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.554+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/3807806023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb1200 0x562741ebaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.554+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/3807806023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 0x562741ebb470 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.554+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/3807806023 >> 172.21.15.32:0/3807806023 conn(0x562741e25200 msgr2=0x562741eab590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.554+0000 7f9b6bfff700 0 client.26323 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:52:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.555+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/3807806023 shutdown_connections 2023-09-30T14:52:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.555+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/3807806023 wait complete. 2023-09-30T14:52:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.555+0000 7f9b9ecfbb80 20 client.26323 trim_cache size 0 max 16384 2023-09-30T14:52:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.574+0000 7f9b9ecfbb80 1 Processor -- start 2023-09-30T14:52:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.574+0000 7f9b9ecfbb80 1 -- start start 2023-09-30T14:52:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.574+0000 7f9b9ecfbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb1200 0x562741ec2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.574+0000 7f9b9ecfbb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb2310 0x562741ec2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.574+0000 7f9b9ecfbb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb2cd0 0x562741eb9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.574+0000 7f9b9ecfbb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741ebab00 con 0x562741eb2cd0 2023-09-30T14:52:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.574+0000 7f9b9ecfbb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741c848c0 con 0x562741eb2310 2023-09-30T14:52:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.574+0000 7f9b9ecfbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741eacd10 con 0x562741eb1200 2023-09-30T14:52:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.574+0000 7f9b8bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb2310 0x562741ec2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.574+0000 7f9b8bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb2310 0x562741ec2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:49938/0 (socket says 172.21.15.32:49938) 2023-09-30T14:52:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.574+0000 7f9b8bfff700 1 -- 172.21.15.32:0/3335996798 learned_addr learned my addr 172.21.15.32:0/3335996798 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.574+0000 7f9b910b0700 1 --2- 172.21.15.32:0/3335996798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb2cd0 0x562741eb9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.575+0000 7f9b8bfff700 1 -- 172.21.15.32:0/3335996798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb1200 msgr2=0x562741ec2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.575+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/3335996798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb1200 0x562741ec2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.575+0000 7f9b8bfff700 1 -- 172.21.15.32:0/3335996798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb2cd0 msgr2=0x562741eb9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.575+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/3335996798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb2cd0 0x562741eb9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.575+0000 7f9b8bfff700 1 -- 172.21.15.32:0/3335996798 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562741c17c00 con 0x562741eb2310 2023-09-30T14:52:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.575+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/3335996798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb2310 0x562741ec2470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9b7803c040 tx=0x7f9b780093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de686988b833113a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.575+0000 7f9b777fe700 1 -- 172.21.15.32:0/3335996798 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78003d20 con 0x562741eb2310 2023-09-30T14:52:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.575+0000 7f9b777fe700 1 -- 172.21.15.32:0/3335996798 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b7800b8c0 con 0x562741eb2310 2023-09-30T14:52:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.575+0000 7f9b777fe700 1 -- 172.21.15.32:0/3335996798 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78005040 con 0x562741eb2310 2023-09-30T14:52:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.576+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/3335996798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb2310 msgr2=0x562741ec2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.576+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/3335996798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb2310 0x562741ec2470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9b7803c040 tx=0x7f9b780093d0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.576+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/3335996798 shutdown_connections 2023-09-30T14:52:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.576+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/3335996798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb2cd0 0x562741eb9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.576+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/3335996798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb1200 0x562741ec2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.576+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/3335996798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb2310 0x562741ec2470 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.576+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/3335996798 >> 172.21.15.32:0/3335996798 conn(0x562741e25200 msgr2=0x562741eb9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.576+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/3335996798 shutdown_connections 2023-09-30T14:52:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.576+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/3335996798 wait complete. 2023-09-30T14:52:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.576+0000 7f9b9ecfbb80 1 -- start start 2023-09-30T14:52:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.577+0000 7f9b9ecfbb80 1 Processor -- start 2023-09-30T14:52:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.577+0000 7f9b9ecfbb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb1200 0x562741ebf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.577+0000 7f9b9ecfbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 0x562741ebfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.577+0000 7f9b9ecfbb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb2cd0 0x562741ec0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.577+0000 7f9b9ecfbb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741ec1130 con 0x562741eb1200 2023-09-30T14:52:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.577+0000 7f9b9ecfbb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741ec12a0 con 0x562741eb2cd0 2023-09-30T14:52:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.577+0000 7f9b9ecfbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562741ec1460 con 0x562741eb2310 2023-09-30T14:52:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.577+0000 7f9b9ecfbb80 10 client.0 _ll_register_callbacks cb 0x7f9b800056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:52:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.577+0000 7f9b908af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb1200 0x562741ebf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.577+0000 7f9b908af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb1200 0x562741ebf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38468/0 (socket says 172.21.15.32:38468) 2023-09-30T14:52:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.577+0000 7f9b8bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 0x562741ebfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.577+0000 7f9b908af700 1 -- 172.21.15.32:0/177662692 learned_addr learned my addr 172.21.15.32:0/177662692 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.578+0000 7f9b908af700 1 -- 172.21.15.32:0/177662692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 msgr2=0x562741ebfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.578+0000 7f9b908af700 1 --2- 172.21.15.32:0/177662692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 0x562741ebfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.578+0000 7f9b908af700 1 -- 172.21.15.32:0/177662692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb2cd0 msgr2=0x562741ec0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.578+0000 7f9b908af700 1 --2- 172.21.15.32:0/177662692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb2cd0 0x562741ec0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.578+0000 7f9b908af700 1 -- 172.21.15.32:0/177662692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562741c2ed80 con 0x562741eb1200 2023-09-30T14:52:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.578+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/177662692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 0x562741ebfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:52:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.578+0000 7f9b908af700 1 --2- 172.21.15.32:0/177662692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb1200 0x562741ebf6a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9b8009a750 tx=0x7f9b80009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28dd92da3232ae3b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.578+0000 7f9b6bfff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:52:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.578+0000 7f9b6bfff700 1 -- 172.21.15.32:0/177662692 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8009aec0 con 0x562741eb1200 2023-09-30T14:52:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.578+0000 7f9b6bfff700 1 -- 172.21.15.32:0/177662692 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9b8009a7c0 con 0x562741eb1200 2023-09-30T14:52:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.578+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/177662692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b78016040 con 0x562741eb1200 2023-09-30T14:52:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.578+0000 7f9b6bfff700 1 -- 172.21.15.32:0/177662692 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b8009e910 con 0x562741eb1200 2023-09-30T14:52:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.578+0000 7f9b9ecfbb80 10 client.16662 fetch_fsmap learned FSMap version 47 2023-09-30T14:52:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.578+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/177662692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x562741c551f0 con 0x562741eb1200 2023-09-30T14:52:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.579+0000 7f9b6bfff700 1 -- 172.21.15.32:0/177662692 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b78016040 con 0x562741eb1200 2023-09-30T14:52:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.579+0000 7f9b6bfff700 1 -- 172.21.15.32:0/177662692 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 47) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9b8009aa20 con 0x562741eb1200 2023-09-30T14:52:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.579+0000 7f9b9ecfbb80 10 client.16662 fetch_fsmap finished waiting for FSMap version 47 2023-09-30T14:52:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.579+0000 7f9b9ecfbb80 10 client.16662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:52:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.579+0000 7f9b9ecfbb80 20 client.16662 populate_metadata read hostname 'smithi032' 2023-09-30T14:52:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.579+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/177662692 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562741f1ac40 0x562741f1d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.579+0000 7f9b9ecfbb80 4 client.16662 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:52:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.579+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/177662692 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562741dfcad0 con 0x562741f1ac40 2023-09-30T14:52:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.579+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/177662692 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562741f1ac40 0x562741f1d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.580+0000 7f9b6bfff700 10 client.16662 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:52:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.580+0000 7f9b8bfff700 1 --2- 172.21.15.32:0/177662692 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562741f1ac40 0x562741f1d030 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e50d7f981e36af5 server_cookie=c5566fd9463bbaa2 in_seq=0 out_seq=0 2023-09-30T14:52:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.827+0000 7f9b6bfff700 1 -- 172.21.15.32:0/177662692 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562741dfcad0 con 0x562741f1ac40 2023-09-30T14:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.827+0000 7f9b6bfff700 10 client.16662 handle_command_reply: tid=0 2023-09-30T14:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.827+0000 7f9b9ecfbb80 1 client.16662 shutdown 2023-09-30T14:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.827+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/177662692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb1200 msgr2=0x562741ebf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.827+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/177662692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb1200 0x562741ebf6a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9b8009a750 tx=0x7f9b80009cd0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.827+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/177662692 shutdown_connections 2023-09-30T14:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.827+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/177662692 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562741f1ac40 0x562741f1d030 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.827+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/177662692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562741eb2cd0 0x562741ec0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.827+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/177662692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562741eb1200 0x562741ebf6a0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.827+0000 7f9b9ecfbb80 1 --2- 172.21.15.32:0/177662692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562741eb2310 0x562741ebfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.827+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/177662692 >> 172.21.15.32:0/177662692 conn(0x562741e25200 msgr2=0x562741eb5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.827+0000 7f9b6bfff700 0 client.16662 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.828+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/177662692 shutdown_connections 2023-09-30T14:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.828+0000 7f9b9ecfbb80 1 -- 172.21.15.32:0/177662692 wait complete. 2023-09-30T14:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.828+0000 7f9b9ecfbb80 20 client.16662 trim_cache size 0 max 16384 2023-09-30T14:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:52:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.830+0000 7f9b92b13700 1 -- 172.21.15.32:0/1461147378 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9b7c0a1020 msgr2=0x7f9b7c0a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.830+0000 7f9b92b13700 1 --2- 172.21.15.32:0/1461147378 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9b7c0a1020 0x7f9b7c0a34e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c068710 tx=0x7f9b78012040 comp rx=0 tx=0).stop 2023-09-30T14:52:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.830+0000 7f9b92b13700 1 -- 172.21.15.32:0/1461147378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b8c0ffd60 msgr2=0x7f9b8c1998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.830+0000 7f9b92b13700 1 --2- 172.21.15.32:0/1461147378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b8c0ffd60 0x7f9b8c1998b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9b800058e0 tx=0x7f9b8000f690 comp rx=0 tx=0).stop 2023-09-30T14:52:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.830+0000 7f9b92b13700 1 -- 172.21.15.32:0/1461147378 shutdown_connections 2023-09-30T14:52:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.830+0000 7f9b92b13700 1 --2- 172.21.15.32:0/1461147378 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9b7c0a1020 0x7f9b7c0a34e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.830+0000 7f9b92b13700 1 --2- 172.21.15.32:0/1461147378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b8c1018f0 0x7f9b8c19e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.830+0000 7f9b92b13700 1 --2- 172.21.15.32:0/1461147378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9b8c0ffd60 0x7f9b8c1998b0 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9b800058e0 tx=0x7f9b8000f690 comp rx=0 tx=0).stop 2023-09-30T14:52:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.830+0000 7f9b92b13700 1 --2- 172.21.15.32:0/1461147378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8c100f50 0x7f9b8c19be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.830+0000 7f9b92b13700 1 -- 172.21.15.32:0/1461147378 >> 172.21.15.32:0/1461147378 conn(0x7f9b8c0fb2b0 msgr2=0x7f9b8c1044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.832+0000 7f9b92b13700 1 -- 172.21.15.32:0/1461147378 shutdown_connections 2023-09-30T14:52:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:10.834+0000 7f9b92b13700 1 -- 172.21.15.32:0/1461147378 wait complete. 2023-09-30T14:52:11.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 22266 31.14 MB/sec warmup 44 sec latency 273.264 ms 2023-09-30T14:52:11.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:11 smithi156 ceph-mon[103579]: pgmap v704: 97 pgs: 97 active+clean; 617 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 22 MiB/s wr, 526 op/s 2023-09-30T14:52:11.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:11 smithi148 ceph-mon[102355]: pgmap v704: 97 pgs: 97 active+clean; 617 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 22 MiB/s wr, 526 op/s 2023-09-30T14:52:11.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:11 smithi032 ceph-mon[98592]: pgmap v704: 97 pgs: 97 active+clean; 617 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 22 MiB/s wr, 526 op/s 2023-09-30T14:52:12.000 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23235 31.91 MB/sec warmup 45 sec latency 55.373 ms 2023-09-30T14:52:13.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 23902 31.85 MB/sec warmup 46 sec latency 61.658 ms 2023-09-30T14:52:13.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:13 smithi156 ceph-mon[103579]: pgmap v705: 97 pgs: 97 active+clean; 611 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 23 MiB/s wr, 646 op/s 2023-09-30T14:52:13.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:13 smithi148 ceph-mon[102355]: pgmap v705: 97 pgs: 97 active+clean; 611 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 23 MiB/s wr, 646 op/s 2023-09-30T14:52:13.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:13 smithi032 ceph-mon[98592]: pgmap v705: 97 pgs: 97 active+clean; 611 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 23 MiB/s wr, 646 op/s 2023-09-30T14:52:14.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 24671 31.95 MB/sec warmup 47 sec latency 61.687 ms 2023-09-30T14:52:15.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 25525 32.11 MB/sec warmup 48 sec latency 55.459 ms 2023-09-30T14:52:15.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:15 smithi156 ceph-mon[103579]: pgmap v706: 97 pgs: 97 active+clean; 614 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 14 MiB/s wr, 615 op/s 2023-09-30T14:52:15.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:15 smithi148 ceph-mon[102355]: pgmap v706: 97 pgs: 97 active+clean; 614 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 14 MiB/s wr, 615 op/s 2023-09-30T14:52:15.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:15 smithi032 ceph-mon[98592]: pgmap v706: 97 pgs: 97 active+clean; 614 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 14 MiB/s wr, 615 op/s 2023-09-30T14:52:16.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 26483 32.42 MB/sec warmup 49 sec latency 133.630 ms 2023-09-30T14:52:17.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 27438 33.22 MB/sec warmup 50 sec latency 77.638 ms 2023-09-30T14:52:17.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:17 smithi156 ceph-mon[103579]: pgmap v707: 97 pgs: 97 active+clean; 617 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 725 op/s 2023-09-30T14:52:17.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:17 smithi148 ceph-mon[102355]: pgmap v707: 97 pgs: 97 active+clean; 617 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 725 op/s 2023-09-30T14:52:17.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:17 smithi032 ceph-mon[98592]: pgmap v707: 97 pgs: 97 active+clean; 617 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 725 op/s 2023-09-30T14:52:18.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 27986 33.10 MB/sec warmup 51 sec latency 43.871 ms 2023-09-30T14:52:19.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 28635 32.98 MB/sec warmup 52 sec latency 94.652 ms 2023-09-30T14:52:19.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:19 smithi156 ceph-mon[103579]: pgmap v708: 97 pgs: 97 active+clean; 631 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.3 MiB/s wr, 807 op/s 2023-09-30T14:52:19.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:19 smithi148 ceph-mon[102355]: pgmap v708: 97 pgs: 97 active+clean; 631 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.3 MiB/s wr, 807 op/s 2023-09-30T14:52:19.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:19 smithi032 ceph-mon[98592]: pgmap v708: 97 pgs: 97 active+clean; 631 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.3 MiB/s wr, 807 op/s 2023-09-30T14:52:20.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 29464 33.16 MB/sec warmup 53 sec latency 98.154 ms 2023-09-30T14:52:21.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 30020 33.09 MB/sec warmup 54 sec latency 282.838 ms 2023-09-30T14:52:21.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:21 smithi156 ceph-mon[103579]: pgmap v709: 97 pgs: 97 active+clean; 632 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.5 MiB/s wr, 727 op/s 2023-09-30T14:52:21.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:21 smithi148 ceph-mon[102355]: pgmap v709: 97 pgs: 97 active+clean; 632 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.5 MiB/s wr, 727 op/s 2023-09-30T14:52:21.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:21 smithi032 ceph-mon[98592]: pgmap v709: 97 pgs: 97 active+clean; 632 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.5 MiB/s wr, 727 op/s 2023-09-30T14:52:22.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 30864 33.37 MB/sec warmup 55 sec latency 55.099 ms 2023-09-30T14:52:23.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 31658 33.67 MB/sec warmup 56 sec latency 57.094 ms 2023-09-30T14:52:23.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:23 smithi156 ceph-mon[103579]: pgmap v710: 97 pgs: 97 active+clean; 641 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.9 MiB/s wr, 794 op/s 2023-09-30T14:52:23.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:23 smithi148 ceph-mon[102355]: pgmap v710: 97 pgs: 97 active+clean; 641 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.9 MiB/s wr, 794 op/s 2023-09-30T14:52:23.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:23 smithi032 ceph-mon[98592]: pgmap v710: 97 pgs: 97 active+clean; 641 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.9 MiB/s wr, 794 op/s 2023-09-30T14:52:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.969+0000 7f2320227700 1 -- 172.21.15.32:0/1644500178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2318106b50 msgr2=0x7f2318108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.969+0000 7f2320227700 1 --2- 172.21.15.32:0/1644500178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2318106b50 0x7f2318108fc0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f231000b0a0 tx=0x7f231000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.969+0000 7f2320227700 1 -- 172.21.15.32:0/1644500178 shutdown_connections 2023-09-30T14:52:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.969+0000 7f2320227700 1 --2- 172.21.15.32:0/1644500178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2318106b50 0x7f2318108fc0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.969+0000 7f2320227700 1 --2- 172.21.15.32:0/1644500178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23181018f0 0x7f2318103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.969+0000 7f2320227700 1 --2- 172.21.15.32:0/1644500178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2318104220 0x7f2318106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.969+0000 7f2320227700 1 -- 172.21.15.32:0/1644500178 >> 172.21.15.32:0/1644500178 conn(0x7f23180fb2a0 msgr2=0x7f23180fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.969+0000 7f2320227700 1 -- 172.21.15.32:0/1644500178 shutdown_connections 2023-09-30T14:52:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.969+0000 7f2320227700 1 -- 172.21.15.32:0/1644500178 wait complete. 2023-09-30T14:52:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.970+0000 7f2320227700 1 Processor -- start 2023-09-30T14:52:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.970+0000 7f2320227700 1 -- start start 2023-09-30T14:52:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.970+0000 7f2320227700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23181018f0 0x7f2318196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.970+0000 7f2320227700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2318104220 0x7f23181972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.970+0000 7f2320227700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2318106b50 0x7f2318198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.970+0000 7f2320227700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23181987a0 con 0x7f23181018f0 2023-09-30T14:52:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.970+0000 7f2320227700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2318198910 con 0x7f2318106b50 2023-09-30T14:52:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.970+0000 7f2320227700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2318198ad0 con 0x7f2318104220 2023-09-30T14:52:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.970+0000 7f231dfc3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23181018f0 0x7f2318196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.970+0000 7f231dfc3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23181018f0 0x7f2318196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38482/0 (socket says 172.21.15.32:38482) 2023-09-30T14:52:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.970+0000 7f231dfc3700 1 -- 172.21.15.32:0/2773316802 learned_addr learned my addr 172.21.15.32:0/2773316802 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.970+0000 7f231d7c2700 1 --2- 172.21.15.32:0/2773316802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2318104220 0x7f23181972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.971+0000 7f231dfc3700 1 -- 172.21.15.32:0/2773316802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2318104220 msgr2=0x7f23181972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.971+0000 7f231dfc3700 1 --2- 172.21.15.32:0/2773316802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2318104220 0x7f23181972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.971+0000 7f231dfc3700 1 -- 172.21.15.32:0/2773316802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2318106b50 msgr2=0x7f2318198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.971+0000 7f231dfc3700 1 --2- 172.21.15.32:0/2773316802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2318106b50 0x7f2318198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.971+0000 7f231dfc3700 1 -- 172.21.15.32:0/2773316802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2310009d00 con 0x7f23181018f0 2023-09-30T14:52:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.971+0000 7f231d7c2700 1 --2- 172.21.15.32:0/2773316802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2318104220 0x7f23181972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:52:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.971+0000 7f231dfc3700 1 --2- 172.21.15.32:0/2773316802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23181018f0 0x7f2318196db0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f231400e9a0 tx=0x7f231400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.971+0000 7f230effd700 1 -- 172.21.15.32:0/2773316802 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314009790 con 0x7f23181018f0 2023-09-30T14:52:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.971+0000 7f2320227700 1 -- 172.21.15.32:0/2773316802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2318198db0 con 0x7f23181018f0 2023-09-30T14:52:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.971+0000 7f230effd700 1 -- 172.21.15.32:0/2773316802 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f231400cbd0 con 0x7f23181018f0 2023-09-30T14:52:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.971+0000 7f230effd700 1 -- 172.21.15.32:0/2773316802 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314018680 con 0x7f23181018f0 2023-09-30T14:52:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.971+0000 7f2320227700 1 -- 172.21.15.32:0/2773316802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231819f570 con 0x7f23181018f0 2023-09-30T14:52:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.975+0000 7f230effd700 1 -- 172.21.15.32:0/2773316802 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f231400cd40 con 0x7f23181018f0 2023-09-30T14:52:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.975+0000 7f230effd700 1 --2- 172.21.15.32:0/2773316802 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f23040a0fa0 0x7f23040a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.976+0000 7f231d7c2700 1 --2- 172.21.15.32:0/2773316802 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f23040a0fa0 0x7f23040a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.976+0000 7f232c40fb80 1 Processor -- start 2023-09-30T14:52:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.976+0000 7f230effd700 1 -- 172.21.15.32:0/2773316802 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2314014070 con 0x7f23181018f0 2023-09-30T14:52:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.976+0000 7f232c40fb80 1 -- start start 2023-09-30T14:52:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.976+0000 7f232c40fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34b200 0x55cd5c34b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.976+0000 7f232c40fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34c310 0x55cd5c34c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.976+0000 7f232c40fb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c353560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.976+0000 7f232c40fb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5c2b3400 con 0x55cd5c34ccd0 2023-09-30T14:52:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.976+0000 7f232c40fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5c2a4bc0 con 0x55cd5c34b200 2023-09-30T14:52:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.976+0000 7f232c40fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5c2a24e0 con 0x55cd5c34c310 2023-09-30T14:52:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.976+0000 7f231d7c2700 1 --2- 172.21.15.32:0/2773316802 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f23040a0fa0 0x7f23040a3460 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2308011490 tx=0x7f2308009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.976+0000 7f231e7c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c353560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.976+0000 7f231e7c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c353560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38494/0 (socket says 172.21.15.32:38494) 2023-09-30T14:52:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.976+0000 7f231e7c4700 1 -- 172.21.15.32:0/672840301 learned_addr learned my addr 172.21.15.32:0/672840301 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.976+0000 7f231d7c2700 1 --2- 172.21.15.32:0/672840301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34c310 0x55cd5c34c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.977+0000 7f231e7c4700 1 -- 172.21.15.32:0/672840301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34c310 msgr2=0x55cd5c34c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.977+0000 7f231e7c4700 1 --2- 172.21.15.32:0/672840301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34c310 0x55cd5c34c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.977+0000 7f231e7c4700 1 -- 172.21.15.32:0/672840301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34b200 msgr2=0x55cd5c34b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.977+0000 7f231e7c4700 1 --2- 172.21.15.32:0/672840301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34b200 0x55cd5c34b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.977+0000 7f231e7c4700 1 -- 172.21.15.32:0/672840301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd5c300870 con 0x55cd5c34ccd0 2023-09-30T14:52:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.977+0000 7f231e7c4700 1 --2- 172.21.15.32:0/672840301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c353560 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2310009fd0 tx=0x7f23100092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e330ac5f96ff6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.977+0000 7f2303fff700 1 -- 172.21.15.32:0/672840301 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231000e040 con 0x55cd5c34ccd0 2023-09-30T14:52:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.977+0000 7f2303fff700 1 -- 172.21.15.32:0/672840301 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2310009490 con 0x55cd5c34ccd0 2023-09-30T14:52:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.977+0000 7f2303fff700 1 -- 172.21.15.32:0/672840301 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231001b460 con 0x55cd5c34ccd0 2023-09-30T14:52:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.978+0000 7f232c40fb80 1 -- 172.21.15.32:0/672840301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 msgr2=0x55cd5c353560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.978+0000 7f232c40fb80 1 --2- 172.21.15.32:0/672840301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c353560 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2310009fd0 tx=0x7f23100092e0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.978+0000 7f232c40fb80 1 -- 172.21.15.32:0/672840301 shutdown_connections 2023-09-30T14:52:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.978+0000 7f232c40fb80 1 --2- 172.21.15.32:0/672840301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c353560 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.978+0000 7f232c40fb80 1 --2- 172.21.15.32:0/672840301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34b200 0x55cd5c34b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.978+0000 7f232c40fb80 1 --2- 172.21.15.32:0/672840301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34c310 0x55cd5c34c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.978+0000 7f232c40fb80 1 -- 172.21.15.32:0/672840301 >> 172.21.15.32:0/672840301 conn(0x55cd5c2bf200 msgr2=0x55cd5c345ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.978+0000 7f232c40fb80 1 -- 172.21.15.32:0/672840301 shutdown_connections 2023-09-30T14:52:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.978+0000 7f232c40fb80 1 -- 172.21.15.32:0/672840301 wait complete. 2023-09-30T14:52:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.978+0000 7f232c40fb80 1 -- start start 2023-09-30T14:52:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.978+0000 7f232c40fb80 1 Processor -- start 2023-09-30T14:52:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f232c40fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34b200 0x55cd5c354eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f232c40fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34c310 0x55cd5c3553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f232c40fb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c35c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f232c40fb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5c29ddd0 con 0x55cd5c34ccd0 2023-09-30T14:52:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f232c40fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5c23a6b0 con 0x55cd5c34c310 2023-09-30T14:52:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f232c40fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5c23a350 con 0x55cd5c34b200 2023-09-30T14:52:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f232c40fb80 10 client.0 _ll_register_callbacks cb 0x55cd5c0977a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:52:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f231e7c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c35c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f231dfc3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34b200 0x55cd5c354eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f231e7c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c35c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38500/0 (socket says 172.21.15.32:38500) 2023-09-30T14:52:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f231e7c4700 1 -- 172.21.15.32:0/1909706113 learned_addr learned my addr 172.21.15.32:0/1909706113 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f231d7c2700 1 --2- 172.21.15.32:0/1909706113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34c310 0x55cd5c3553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f231e7c4700 1 -- 172.21.15.32:0/1909706113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34b200 msgr2=0x55cd5c354eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f231e7c4700 1 --2- 172.21.15.32:0/1909706113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34b200 0x55cd5c354eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f231e7c4700 1 -- 172.21.15.32:0/1909706113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34c310 msgr2=0x55cd5c3553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f231e7c4700 1 --2- 172.21.15.32:0/1909706113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34c310 0x55cd5c3553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f231e7c4700 1 -- 172.21.15.32:0/1909706113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2318109bb0 con 0x55cd5c34ccd0 2023-09-30T14:52:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f231dfc3700 1 --2- 172.21.15.32:0/1909706113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34b200 0x55cd5c354eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:52:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f231e7c4700 1 --2- 172.21.15.32:0/1909706113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c35c430 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f231000b0a0 tx=0x7f2310007790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdb6207f897e46c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.979+0000 7f23017fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:52:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.980+0000 7f23017fa700 1 -- 172.21.15.32:0/1909706113 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310029070 con 0x55cd5c34ccd0 2023-09-30T14:52:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.980+0000 7f23017fa700 1 -- 172.21.15.32:0/1909706113 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f23100052d0 con 0x55cd5c34ccd0 2023-09-30T14:52:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.980+0000 7f232c40fb80 1 -- 172.21.15.32:0/1909706113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd5c33e090 con 0x55cd5c34ccd0 2023-09-30T14:52:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.980+0000 7f23017fa700 1 -- 172.21.15.32:0/1909706113 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f231000e040 con 0x55cd5c34ccd0 2023-09-30T14:52:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.980+0000 7f232c40fb80 10 client.16674 fetch_fsmap learned FSMap version 47 2023-09-30T14:52:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.980+0000 7f232c40fb80 1 -- 172.21.15.32:0/1909706113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55cd5c0c8d80 con 0x55cd5c34ccd0 2023-09-30T14:52:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.980+0000 7f23017fa700 1 -- 172.21.15.32:0/1909706113 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2310034020 con 0x55cd5c34ccd0 2023-09-30T14:52:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.980+0000 7f23017fa700 1 -- 172.21.15.32:0/1909706113 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 47) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f231002c020 con 0x55cd5c34ccd0 2023-09-30T14:52:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.980+0000 7f232c40fb80 10 client.16674 fetch_fsmap finished waiting for FSMap version 47 2023-09-30T14:52:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.980+0000 7f232c40fb80 10 client.16674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:52:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.980+0000 7f232c40fb80 20 client.16674 populate_metadata read hostname 'smithi032' 2023-09-30T14:52:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.980+0000 7f232c40fb80 1 --2- 172.21.15.32:0/1909706113 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55cd5c360910 0x55cd5c362d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.980+0000 7f232c40fb80 4 client.16674 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:52:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.980+0000 7f232c40fb80 1 -- 172.21.15.32:0/1909706113 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd5c296db0 con 0x55cd5c360910 2023-09-30T14:52:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.981+0000 7f231dfc3700 1 --2- 172.21.15.32:0/1909706113 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55cd5c360910 0x55cd5c362d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.981+0000 7f231dfc3700 1 --2- 172.21.15.32:0/1909706113 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55cd5c360910 0x55cd5c362d30 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea208716d28b524b server_cookie=663bfb213eda2ba6 in_seq=0 out_seq=0 2023-09-30T14:52:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.981+0000 7f23017fa700 10 client.16674 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:52:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.983+0000 7f23017fa700 1 -- 172.21.15.32:0/1909706113 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cd5c296db0 con 0x55cd5c360910 2023-09-30T14:52:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.983+0000 7f23017fa700 10 client.16674 handle_command_reply: tid=0 2023-09-30T14:52:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.983+0000 7f232c40fb80 1 client.16674 shutdown 2023-09-30T14:52:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.983+0000 7f232c40fb80 1 -- 172.21.15.32:0/1909706113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 msgr2=0x55cd5c35c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.983+0000 7f232c40fb80 1 --2- 172.21.15.32:0/1909706113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c35c430 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f231000b0a0 tx=0x7f2310007790 comp rx=0 tx=0).stop 2023-09-30T14:52:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.983+0000 7f232c40fb80 1 -- 172.21.15.32:0/1909706113 shutdown_connections 2023-09-30T14:52:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.983+0000 7f232c40fb80 1 --2- 172.21.15.32:0/1909706113 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55cd5c360910 0x55cd5c362d30 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.983+0000 7f232c40fb80 1 --2- 172.21.15.32:0/1909706113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c35c430 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.983+0000 7f232c40fb80 1 --2- 172.21.15.32:0/1909706113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34b200 0x55cd5c354eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.983+0000 7f232c40fb80 1 --2- 172.21.15.32:0/1909706113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34c310 0x55cd5c3553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.983+0000 7f23017fa700 0 client.16674 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:52:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.983+0000 7f232c40fb80 1 -- 172.21.15.32:0/1909706113 >> 172.21.15.32:0/1909706113 conn(0x55cd5c2bf200 msgr2=0x55cd5c345590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.983+0000 7f232c40fb80 1 -- 172.21.15.32:0/1909706113 shutdown_connections 2023-09-30T14:52:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.983+0000 7f232c40fb80 1 -- 172.21.15.32:0/1909706113 wait complete. 2023-09-30T14:52:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:23.983+0000 7f232c40fb80 20 client.16674 trim_cache size 0 max 16384 2023-09-30T14:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.000+0000 7f232c40fb80 1 Processor -- start 2023-09-30T14:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f232c40fb80 1 -- start start 2023-09-30T14:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f232c40fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34b200 0x55cd5c34c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f232c40fb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34c310 0x55cd5c35c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f232c40fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34ccd0 0x55cd5c353130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f232c40fb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5c11e8c0 con 0x55cd5c34c310 2023-09-30T14:52:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f232c40fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5c34d100 con 0x55cd5c34ccd0 2023-09-30T14:52:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f232c40fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5c347630 con 0x55cd5c34b200 2023-09-30T14:52:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f231d7c2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34c310 0x55cd5c35c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f231d7c2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34c310 0x55cd5c35c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38506/0 (socket says 172.21.15.32:38506) 2023-09-30T14:52:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f231d7c2700 1 -- 172.21.15.32:0/779045232 learned_addr learned my addr 172.21.15.32:0/779045232 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f231dfc3700 1 --2- 172.21.15.32:0/779045232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34b200 0x55cd5c34c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f231e7c4700 1 --2- 172.21.15.32:0/779045232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34ccd0 0x55cd5c353130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:24.002 INFO:tasks.workunit.client.0.smithi032.stdout: 10 32211 33.57 MB/sec warmup 57 sec latency 155.186 ms 2023-09-30T14:52:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f231d7c2700 1 -- 172.21.15.32:0/779045232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34b200 msgr2=0x55cd5c34c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f231d7c2700 1 --2- 172.21.15.32:0/779045232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34b200 0x55cd5c34c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f231d7c2700 1 -- 172.21.15.32:0/779045232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34ccd0 msgr2=0x55cd5c353130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f231d7c2700 1 --2- 172.21.15.32:0/779045232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34ccd0 0x55cd5c353130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f231d7c2700 1 -- 172.21.15.32:0/779045232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd5c0b1c00 con 0x55cd5c34c310 2023-09-30T14:52:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f231dfc3700 1 --2- 172.21.15.32:0/779045232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34b200 0x55cd5c34c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:52:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.001+0000 7f231d7c2700 1 --2- 172.21.15.32:0/779045232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34c310 0x55cd5c35c3f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f23080045c0 tx=0x7f2308014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6948e8febe52e07 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.002+0000 7f230cff9700 1 -- 172.21.15.32:0/779045232 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308024070 con 0x55cd5c34c310 2023-09-30T14:52:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.002+0000 7f230cff9700 1 -- 172.21.15.32:0/779045232 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2308003d70 con 0x55cd5c34c310 2023-09-30T14:52:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.002+0000 7f230cff9700 1 -- 172.21.15.32:0/779045232 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308028620 con 0x55cd5c34c310 2023-09-30T14:52:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.003+0000 7f232c40fb80 1 -- 172.21.15.32:0/779045232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34c310 msgr2=0x55cd5c35c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.003+0000 7f232c40fb80 1 --2- 172.21.15.32:0/779045232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34c310 0x55cd5c35c3f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f23080045c0 tx=0x7f2308014000 comp rx=0 tx=0).stop 2023-09-30T14:52:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.003+0000 7f232c40fb80 1 -- 172.21.15.32:0/779045232 shutdown_connections 2023-09-30T14:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.003+0000 7f232c40fb80 1 --2- 172.21.15.32:0/779045232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34ccd0 0x55cd5c353130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.003+0000 7f232c40fb80 1 --2- 172.21.15.32:0/779045232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34b200 0x55cd5c34c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.003+0000 7f232c40fb80 1 --2- 172.21.15.32:0/779045232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34c310 0x55cd5c35c3f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.003+0000 7f232c40fb80 1 -- 172.21.15.32:0/779045232 >> 172.21.15.32:0/779045232 conn(0x55cd5c2bf200 msgr2=0x55cd5c353ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.003+0000 7f232c40fb80 1 -- 172.21.15.32:0/779045232 shutdown_connections 2023-09-30T14:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.003+0000 7f232c40fb80 1 -- 172.21.15.32:0/779045232 wait complete. 2023-09-30T14:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.003+0000 7f232c40fb80 1 -- start start 2023-09-30T14:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.003+0000 7f232c40fb80 1 Processor -- start 2023-09-30T14:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.003+0000 7f232c40fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34b200 0x55cd5c359f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f232c40fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34c310 0x55cd5c35a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f232c40fb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c35b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f232c40fb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5c35ba10 con 0x55cd5c34ccd0 2023-09-30T14:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f232c40fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5c35bb80 con 0x55cd5c34b200 2023-09-30T14:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f232c40fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5c35bd40 con 0x55cd5c34c310 2023-09-30T14:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f232c40fb80 10 client.0 _ll_register_callbacks cb 0x7f2308004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f231e7c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c35b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f231d7c2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34c310 0x55cd5c35a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f231dfc3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34b200 0x55cd5c359f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f231e7c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c35b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38514/0 (socket says 172.21.15.32:38514) 2023-09-30T14:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f231e7c4700 1 -- 172.21.15.32:0/2043094264 learned_addr learned my addr 172.21.15.32:0/2043094264 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f231d7c2700 1 -- 172.21.15.32:0/2043094264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34b200 msgr2=0x55cd5c359f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f231d7c2700 1 --2- 172.21.15.32:0/2043094264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34b200 0x55cd5c359f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f231d7c2700 1 -- 172.21.15.32:0/2043094264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 msgr2=0x55cd5c35b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f231d7c2700 1 --2- 172.21.15.32:0/2043094264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c35b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f231d7c2700 1 -- 172.21.15.32:0/2043094264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231819f570 con 0x55cd5c34c310 2023-09-30T14:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f231e7c4700 1 --2- 172.21.15.32:0/2043094264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c35b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f231dfc3700 1 --2- 172.21.15.32:0/2043094264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34b200 0x55cd5c359f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f231d7c2700 1 --2- 172.21.15.32:0/2043094264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34c310 0x55cd5c35a4c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2308000c00 tx=0x7f230801e620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97229dede6d4d75e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f23017fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f23017fa700 1 -- 172.21.15.32:0/2043094264 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308024070 con 0x55cd5c34c310 2023-09-30T14:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f232c40fb80 1 -- 172.21.15.32:0/2043094264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2310009490 con 0x55cd5c34c310 2023-09-30T14:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.004+0000 7f23017fa700 1 -- 172.21.15.32:0/2043094264 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2308029420 con 0x55cd5c34c310 2023-09-30T14:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.005+0000 7f23017fa700 1 -- 172.21.15.32:0/2043094264 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f230801ea20 con 0x55cd5c34c310 2023-09-30T14:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.005+0000 7f232c40fb80 10 client.26344 fetch_fsmap learned FSMap version 47 2023-09-30T14:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.005+0000 7f232c40fb80 1 -- 172.21.15.32:0/2043094264 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f2308017070 con 0x55cd5c34c310 2023-09-30T14:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.005+0000 7f23017fa700 1 -- 172.21.15.32:0/2043094264 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2308040020 con 0x55cd5c34c310 2023-09-30T14:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.005+0000 7f23017fa700 1 -- 172.21.15.32:0/2043094264 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 47) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f230803c630 con 0x55cd5c34c310 2023-09-30T14:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.005+0000 7f232c40fb80 10 client.26344 fetch_fsmap finished waiting for FSMap version 47 2023-09-30T14:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.005+0000 7f232c40fb80 10 client.26344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.005+0000 7f232c40fb80 20 client.26344 populate_metadata read hostname 'smithi032' 2023-09-30T14:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.005+0000 7f232c40fb80 1 --2- 172.21.15.32:0/2043094264 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55cd5c3b5500 0x55cd5c3b7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.005+0000 7f232c40fb80 4 client.26344 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.005+0000 7f232c40fb80 1 -- 172.21.15.32:0/2043094264 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cd5c296ad0 con 0x55cd5c3b5500 2023-09-30T14:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.005+0000 7f231dfc3700 1 --2- 172.21.15.32:0/2043094264 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55cd5c3b5500 0x55cd5c3b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.006+0000 7f23017fa700 10 client.26344 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:52:24.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.006+0000 7f231dfc3700 1 --2- 172.21.15.32:0/2043094264 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55cd5c3b5500 0x55cd5c3b7920 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45f02b4c9d95f7f3 server_cookie=1933eb40da649f4b in_seq=0 out_seq=0 2023-09-30T14:52:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.245+0000 7f23017fa700 1 -- 172.21.15.32:0/2043094264 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cd5c296ad0 con 0x55cd5c3b5500 2023-09-30T14:52:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.245+0000 7f23017fa700 10 client.26344 handle_command_reply: tid=0 2023-09-30T14:52:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.246+0000 7f232c40fb80 1 client.26344 shutdown 2023-09-30T14:52:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.246+0000 7f232c40fb80 1 -- 172.21.15.32:0/2043094264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34c310 msgr2=0x55cd5c35a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.246+0000 7f232c40fb80 1 --2- 172.21.15.32:0/2043094264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34c310 0x55cd5c35a4c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2308000c00 tx=0x7f230801e620 comp rx=0 tx=0).stop 2023-09-30T14:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.246+0000 7f232c40fb80 1 -- 172.21.15.32:0/2043094264 shutdown_connections 2023-09-30T14:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.246+0000 7f232c40fb80 1 --2- 172.21.15.32:0/2043094264 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55cd5c3b5500 0x55cd5c3b7920 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.246+0000 7f232c40fb80 1 --2- 172.21.15.32:0/2043094264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55cd5c34ccd0 0x55cd5c35b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.246+0000 7f23017fa700 0 client.26344 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.246+0000 7f232c40fb80 1 --2- 172.21.15.32:0/2043094264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5c34b200 0x55cd5c359f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.246+0000 7f232c40fb80 1 --2- 172.21.15.32:0/2043094264 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd5c34c310 0x55cd5c35a4c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.246+0000 7f232c40fb80 1 -- 172.21.15.32:0/2043094264 >> 172.21.15.32:0/2043094264 conn(0x55cd5c2bf200 msgr2=0x55cd5c353130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.246+0000 7f232c40fb80 1 -- 172.21.15.32:0/2043094264 shutdown_connections 2023-09-30T14:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.246+0000 7f232c40fb80 1 -- 172.21.15.32:0/2043094264 wait complete. 2023-09-30T14:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.246+0000 7f232c40fb80 20 client.26344 trim_cache size 0 max 16384 2023-09-30T14:52:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:52:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:52:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:52:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:52:24.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.247+0000 7f2320227700 1 -- 172.21.15.32:0/2773316802 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f23040a0fa0 msgr2=0x7f23040a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.247+0000 7f2320227700 1 --2- 172.21.15.32:0/2773316802 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f23040a0fa0 0x7f23040a3460 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2308011490 tx=0x7f2308009260 comp rx=0 tx=0).stop 2023-09-30T14:52:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.247+0000 7f2320227700 1 -- 172.21.15.32:0/2773316802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23181018f0 msgr2=0x7f2318196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.247+0000 7f2320227700 1 --2- 172.21.15.32:0/2773316802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23181018f0 0x7f2318196db0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f231400e9a0 tx=0x7f231400ecb0 comp rx=0 tx=0).stop 2023-09-30T14:52:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.247+0000 7f2320227700 1 -- 172.21.15.32:0/2773316802 shutdown_connections 2023-09-30T14:52:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.247+0000 7f2320227700 1 --2- 172.21.15.32:0/2773316802 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f23040a0fa0 0x7f23040a3460 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.247+0000 7f2320227700 1 --2- 172.21.15.32:0/2773316802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2318106b50 0x7f2318198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.247+0000 7f2320227700 1 --2- 172.21.15.32:0/2773316802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23181018f0 0x7f2318196db0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.247+0000 7f2320227700 1 --2- 172.21.15.32:0/2773316802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2318104220 0x7f23181972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.247+0000 7f2320227700 1 -- 172.21.15.32:0/2773316802 >> 172.21.15.32:0/2773316802 conn(0x7f23180fb2a0 msgr2=0x7f23181025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.247+0000 7f2320227700 1 -- 172.21.15.32:0/2773316802 shutdown_connections 2023-09-30T14:52:24.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:24.247+0000 7f2320227700 1 -- 172.21.15.32:0/2773316802 wait complete. 2023-09-30T14:52:25.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 32851 33.44 MB/sec warmup 58 sec latency 109.057 ms 2023-09-30T14:52:25.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:25 smithi156 ceph-mon[103579]: pgmap v711: 97 pgs: 97 active+clean; 646 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.8 MiB/s wr, 728 op/s 2023-09-30T14:52:25.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:25 smithi148 ceph-mon[102355]: pgmap v711: 97 pgs: 97 active+clean; 646 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.8 MiB/s wr, 728 op/s 2023-09-30T14:52:25.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:25 smithi032 ceph-mon[98592]: pgmap v711: 97 pgs: 97 active+clean; 646 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.8 MiB/s wr, 728 op/s 2023-09-30T14:52:26.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 33490 33.48 MB/sec warmup 59 sec latency 227.071 ms 2023-09-30T14:52:27.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 34389 33.73 MB/sec warmup 60 sec latency 78.919 ms 2023-09-30T14:52:27.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:27 smithi156 ceph-mon[103579]: pgmap v712: 97 pgs: 97 active+clean; 642 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.2 MiB/s wr, 781 op/s 2023-09-30T14:52:27.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:27 smithi148 ceph-mon[102355]: pgmap v712: 97 pgs: 97 active+clean; 642 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.2 MiB/s wr, 781 op/s 2023-09-30T14:52:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:27 smithi032 ceph-mon[98592]: pgmap v712: 97 pgs: 97 active+clean; 642 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.2 MiB/s wr, 781 op/s 2023-09-30T14:52:28.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 35265 33.96 MB/sec warmup 61 sec latency 58.514 ms 2023-09-30T14:52:29.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 36080 34.26 MB/sec warmup 62 sec latency 138.000 ms 2023-09-30T14:52:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:29 smithi156 ceph-mon[103579]: pgmap v713: 97 pgs: 97 active+clean; 622 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.8 MiB/s wr, 780 op/s 2023-09-30T14:52:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:29 smithi148 ceph-mon[102355]: pgmap v713: 97 pgs: 97 active+clean; 622 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.8 MiB/s wr, 780 op/s 2023-09-30T14:52:29.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:29 smithi032 ceph-mon[98592]: pgmap v713: 97 pgs: 97 active+clean; 622 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.8 MiB/s wr, 780 op/s 2023-09-30T14:52:30.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 36682 34.24 MB/sec warmup 63 sec latency 106.046 ms 2023-09-30T14:52:31.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 37165 34.03 MB/sec warmup 64 sec latency 163.920 ms 2023-09-30T14:52:31.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:31 smithi156 ceph-mon[103579]: pgmap v714: 97 pgs: 97 active+clean; 621 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.7 MiB/s wr, 659 op/s 2023-09-30T14:52:31.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:31 smithi148 ceph-mon[102355]: pgmap v714: 97 pgs: 97 active+clean; 621 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.7 MiB/s wr, 659 op/s 2023-09-30T14:52:31.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:31 smithi032 ceph-mon[98592]: pgmap v714: 97 pgs: 97 active+clean; 621 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.7 MiB/s wr, 659 op/s 2023-09-30T14:52:32.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 37917 34.10 MB/sec warmup 65 sec latency 104.884 ms 2023-09-30T14:52:33.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 38881 34.22 MB/sec warmup 66 sec latency 37.176 ms 2023-09-30T14:52:33.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:33 smithi156 ceph-mon[103579]: pgmap v715: 97 pgs: 97 active+clean; 623 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.8 MiB/s wr, 757 op/s 2023-09-30T14:52:33.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:33 smithi148 ceph-mon[102355]: pgmap v715: 97 pgs: 97 active+clean; 623 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.8 MiB/s wr, 757 op/s 2023-09-30T14:52:33.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:33 smithi032 ceph-mon[98592]: pgmap v715: 97 pgs: 97 active+clean; 623 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.8 MiB/s wr, 757 op/s 2023-09-30T14:52:34.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 39675 34.49 MB/sec warmup 67 sec latency 122.302 ms 2023-09-30T14:52:35.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 40496 34.72 MB/sec warmup 68 sec latency 171.220 ms 2023-09-30T14:52:35.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:35 smithi156 ceph-mon[103579]: pgmap v716: 97 pgs: 97 active+clean; 630 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.7 MiB/s wr, 770 op/s 2023-09-30T14:52:35.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:35 smithi148 ceph-mon[102355]: pgmap v716: 97 pgs: 97 active+clean; 630 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.7 MiB/s wr, 770 op/s 2023-09-30T14:52:35.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:35 smithi032 ceph-mon[98592]: pgmap v716: 97 pgs: 97 active+clean; 630 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.7 MiB/s wr, 770 op/s 2023-09-30T14:52:36.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 40968 34.61 MB/sec warmup 69 sec latency 239.724 ms 2023-09-30T14:52:37.002 INFO:tasks.workunit.client.0.smithi032.stdout: 10 41498 34.54 MB/sec warmup 70 sec latency 79.650 ms 2023-09-30T14:52:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.371+0000 7f69475bc700 1 -- 172.21.15.32:0/2464612896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69401021b0 msgr2=0x7f6940102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.371+0000 7f69475bc700 1 --2- 172.21.15.32:0/2464612896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69401021b0 0x7f6940102610 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6930009990 tx=0x7f6930009ca0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.372+0000 7f69475bc700 1 -- 172.21.15.32:0/2464612896 shutdown_connections 2023-09-30T14:52:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.372+0000 7f69475bc700 1 --2- 172.21.15.32:0/2464612896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6940102b50 0x7f6940109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.372+0000 7f69475bc700 1 --2- 172.21.15.32:0/2464612896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69400ff140 0x7f69400ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.372+0000 7f69475bc700 1 --2- 172.21.15.32:0/2464612896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69401021b0 0x7f6940102610 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.372+0000 7f69475bc700 1 -- 172.21.15.32:0/2464612896 >> 172.21.15.32:0/2464612896 conn(0x7f69400faa30 msgr2=0x7f69400fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.372+0000 7f69475bc700 1 -- 172.21.15.32:0/2464612896 shutdown_connections 2023-09-30T14:52:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.372+0000 7f69475bc700 1 -- 172.21.15.32:0/2464612896 wait complete. 2023-09-30T14:52:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.372+0000 7f69475bc700 1 Processor -- start 2023-09-30T14:52:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.372+0000 7f69475bc700 1 -- start start 2023-09-30T14:52:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f69475bc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69400ff140 0x7f6940196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f69475bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69401021b0 0x7f69401972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f69475bc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6940102b50 0x7f6940197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f69475bc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69401986f0 con 0x7f69401021b0 2023-09-30T14:52:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f69475bc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6940198860 con 0x7f69400ff140 2023-09-30T14:52:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f69475bc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6940198a20 con 0x7f6940102b50 2023-09-30T14:52:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f6944b57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69401021b0 0x7f69401972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f6944b57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69401021b0 0x7f69401972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38526/0 (socket says 172.21.15.32:38526) 2023-09-30T14:52:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f6944b57700 1 -- 172.21.15.32:0/1792036427 learned_addr learned my addr 172.21.15.32:0/1792036427 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f6945358700 1 --2- 172.21.15.32:0/1792036427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69400ff140 0x7f6940196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f6945b59700 1 --2- 172.21.15.32:0/1792036427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6940102b50 0x7f6940197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f6944b57700 1 -- 172.21.15.32:0/1792036427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6940102b50 msgr2=0x7f6940197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f6944b57700 1 --2- 172.21.15.32:0/1792036427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6940102b50 0x7f6940197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f6944b57700 1 -- 172.21.15.32:0/1792036427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69400ff140 msgr2=0x7f6940196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f6944b57700 1 --2- 172.21.15.32:0/1792036427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69400ff140 0x7f6940196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f6944b57700 1 -- 172.21.15.32:0/1792036427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69300095f0 con 0x7f69401021b0 2023-09-30T14:52:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f6944b57700 1 --2- 172.21.15.32:0/1792036427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69401021b0 0x7f69401972b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6930004730 tx=0x7f6930004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f69367fc700 1 -- 172.21.15.32:0/1792036427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693001d070 con 0x7f69401021b0 2023-09-30T14:52:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f69367fc700 1 -- 172.21.15.32:0/1792036427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6930004c90 con 0x7f69401021b0 2023-09-30T14:52:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f69367fc700 1 -- 172.21.15.32:0/1792036427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69300054b0 con 0x7f69401021b0 2023-09-30T14:52:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.373+0000 7f69475bc700 1 -- 172.21.15.32:0/1792036427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6940198ca0 con 0x7f69401021b0 2023-09-30T14:52:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.374+0000 7f69475bc700 1 -- 172.21.15.32:0/1792036427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694019f420 con 0x7f69401021b0 2023-09-30T14:52:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.378+0000 7f69537a4b80 1 Processor -- start 2023-09-30T14:52:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.378+0000 7f69367fc700 1 -- 172.21.15.32:0/1792036427 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f693000fbc0 con 0x7f69401021b0 2023-09-30T14:52:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.378+0000 7f69537a4b80 1 -- start start 2023-09-30T14:52:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.378+0000 7f69537a4b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232482200 0x562232482620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.378+0000 7f69537a4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483310 0x562232483790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.378+0000 7f69537a4b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232483cd0 0x56223248a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.378+0000 7f69537a4b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622323ea400 con 0x562232482200 2023-09-30T14:52:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.378+0000 7f69537a4b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622323dbbc0 con 0x562232483cd0 2023-09-30T14:52:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.378+0000 7f69537a4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622323d94e0 con 0x562232483310 2023-09-30T14:52:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.378+0000 7f6945b59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232482200 0x562232482620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.378+0000 7f6945358700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483310 0x562232483790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.378+0000 7f6945358700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483310 0x562232483790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52914/0 (socket says 172.21.15.32:52914) 2023-09-30T14:52:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.378+0000 7f6945b59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232482200 0x562232482620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38530/0 (socket says 172.21.15.32:38530) 2023-09-30T14:52:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.378+0000 7f6945358700 1 -- 172.21.15.32:0/3204749887 learned_addr learned my addr 172.21.15.32:0/3204749887 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.378+0000 7f6944b57700 1 --2- 172.21.15.32:0/3204749887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232483cd0 0x56223248a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.379+0000 7f69367fc700 1 --2- 172.21.15.32:0/1792036427 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f692c0a1020 0x7f692c0a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.379+0000 7f6945358700 1 -- 172.21.15.32:0/3204749887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232483cd0 msgr2=0x56223248a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.379+0000 7f6945358700 1 --2- 172.21.15.32:0/3204749887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232483cd0 0x56223248a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.379+0000 7f6945358700 1 -- 172.21.15.32:0/3204749887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232482200 msgr2=0x562232482620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.379+0000 7f6945358700 1 --2- 172.21.15.32:0/3204749887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232482200 0x562232482620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.379+0000 7f6945358700 1 -- 172.21.15.32:0/3204749887 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562232437870 con 0x562232483310 2023-09-30T14:52:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.379+0000 7f6945b59700 1 --2- 172.21.15.32:0/1792036427 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f692c0a1020 0x7f692c0a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.379+0000 7f69367fc700 1 -- 172.21.15.32:0/1792036427 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f69300cd8f0 con 0x7f69401021b0 2023-09-30T14:52:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.379+0000 7f6945358700 1 --2- 172.21.15.32:0/3204749887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483310 0x562232483790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f693c009d30 tx=0x7f693c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f44c8cabca207a72 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.379+0000 7f692b7fe700 1 -- 172.21.15.32:0/3204749887 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c004220 con 0x562232483310 2023-09-30T14:52:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.379+0000 7f692b7fe700 1 -- 172.21.15.32:0/3204749887 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f693c004380 con 0x562232483310 2023-09-30T14:52:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.379+0000 7f692b7fe700 1 -- 172.21.15.32:0/3204749887 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c0198d0 con 0x562232483310 2023-09-30T14:52:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.380+0000 7f69537a4b80 1 -- 172.21.15.32:0/3204749887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483310 msgr2=0x562232483790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.380+0000 7f6945b59700 1 --2- 172.21.15.32:0/1792036427 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f692c0a1020 0x7f692c0a34e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x562232481fa0 tx=0x7f6938004be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.380+0000 7f69537a4b80 1 --2- 172.21.15.32:0/3204749887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483310 0x562232483790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f693c009d30 tx=0x7f693c011720 comp rx=0 tx=0).stop 2023-09-30T14:52:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.380+0000 7f69537a4b80 1 -- 172.21.15.32:0/3204749887 shutdown_connections 2023-09-30T14:52:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.380+0000 7f69537a4b80 1 --2- 172.21.15.32:0/3204749887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232483cd0 0x56223248a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.380+0000 7f69537a4b80 1 --2- 172.21.15.32:0/3204749887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232482200 0x562232482620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.380+0000 7f69537a4b80 1 --2- 172.21.15.32:0/3204749887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483310 0x562232483790 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.380+0000 7f69537a4b80 1 -- 172.21.15.32:0/3204749887 >> 172.21.15.32:0/3204749887 conn(0x5622323f6200 msgr2=0x56223247cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.380+0000 7f69537a4b80 1 -- 172.21.15.32:0/3204749887 shutdown_connections 2023-09-30T14:52:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.380+0000 7f69537a4b80 1 -- 172.21.15.32:0/3204749887 wait complete. 2023-09-30T14:52:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.381+0000 7f69537a4b80 1 -- start start 2023-09-30T14:52:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.381+0000 7f69537a4b80 1 Processor -- start 2023-09-30T14:52:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.381+0000 7f69537a4b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232482200 0x562232494670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.381+0000 7f69537a4b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232483310 0x562232494bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.381+0000 7f69537a4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483cd0 0x56223249bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.381+0000 7f69537a4b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622323d4dd0 con 0x562232483310 2023-09-30T14:52:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.381+0000 7f69537a4b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622323716b0 con 0x562232482200 2023-09-30T14:52:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.381+0000 7f69537a4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562232371350 con 0x562232483cd0 2023-09-30T14:52:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.381+0000 7f69537a4b80 10 client.0 _ll_register_callbacks cb 0x5622321ce7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:52:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.381+0000 7f6944b57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232483310 0x562232494bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.381+0000 7f6944b57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232483310 0x562232494bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38540/0 (socket says 172.21.15.32:38540) 2023-09-30T14:52:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.381+0000 7f6944b57700 1 -- 172.21.15.32:0/2941356139 learned_addr learned my addr 172.21.15.32:0/2941356139 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.381+0000 7f6945b59700 1 --2- 172.21.15.32:0/2941356139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483cd0 0x56223249bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.382+0000 7f6945b59700 1 -- 172.21.15.32:0/2941356139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232482200 msgr2=0x562232494670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.382+0000 7f6945b59700 1 --2- 172.21.15.32:0/2941356139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232482200 0x562232494670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.382+0000 7f6945b59700 1 -- 172.21.15.32:0/2941356139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232483310 msgr2=0x562232494bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.382+0000 7f6945b59700 1 --2- 172.21.15.32:0/2941356139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232483310 0x562232494bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.382+0000 7f6945b59700 1 -- 172.21.15.32:0/2941356139 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693c009960 con 0x562232483cd0 2023-09-30T14:52:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.382+0000 7f6945b59700 1 --2- 172.21.15.32:0/2941356139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483cd0 0x56223249bbf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f693800a900 tx=0x7f693800d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b49898e54a5c6947 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.382+0000 7f6928ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:52:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.382+0000 7f6928ff9700 1 -- 172.21.15.32:0/2941356139 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6938010030 con 0x562232483cd0 2023-09-30T14:52:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.382+0000 7f6928ff9700 1 -- 172.21.15.32:0/2941356139 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f693801e040 con 0x562232483cd0 2023-09-30T14:52:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.382+0000 7f69537a4b80 1 -- 172.21.15.32:0/2941356139 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562232475090 con 0x562232483cd0 2023-09-30T14:52:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.382+0000 7f6928ff9700 1 -- 172.21.15.32:0/2941356139 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6938015890 con 0x562232483cd0 2023-09-30T14:52:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.382+0000 7f69537a4b80 10 client.26356 fetch_fsmap learned FSMap version 47 2023-09-30T14:52:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.382+0000 7f69537a4b80 1 -- 172.21.15.32:0/2941356139 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5622321ffd80 con 0x562232483cd0 2023-09-30T14:52:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.383+0000 7f6928ff9700 1 -- 172.21.15.32:0/2941356139 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562232475090 con 0x562232483cd0 2023-09-30T14:52:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.383+0000 7f6928ff9700 1 -- 172.21.15.32:0/2941356139 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 47) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6938022bc0 con 0x562232483cd0 2023-09-30T14:52:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.383+0000 7f69537a4b80 10 client.26356 fetch_fsmap finished waiting for FSMap version 47 2023-09-30T14:52:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.383+0000 7f69537a4b80 10 client.26356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:52:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.383+0000 7f69537a4b80 20 client.26356 populate_metadata read hostname 'smithi032' 2023-09-30T14:52:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.383+0000 7f69537a4b80 1 --2- 172.21.15.32:0/2941356139 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5622324a05d0 0x5622324a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.383+0000 7f69537a4b80 4 client.26356 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:52:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.383+0000 7f69537a4b80 1 -- 172.21.15.32:0/2941356139 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622323cddb0 con 0x5622324a05d0 2023-09-30T14:52:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.383+0000 7f6945358700 1 --2- 172.21.15.32:0/2941356139 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5622324a05d0 0x5622324a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.383+0000 7f6928ff9700 10 client.26356 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:52:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.383+0000 7f6945358700 1 --2- 172.21.15.32:0/2941356139 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5622324a05d0 0x5622324a29f0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dee4fd4b1bd24f73 server_cookie=df00738db14916d in_seq=0 out_seq=0 2023-09-30T14:52:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.387+0000 7f6928ff9700 1 -- 172.21.15.32:0/2941356139 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5622323cddb0 con 0x5622324a05d0 2023-09-30T14:52:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.387+0000 7f6928ff9700 10 client.26356 handle_command_reply: tid=0 2023-09-30T14:52:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.387+0000 7f69537a4b80 1 client.26356 shutdown 2023-09-30T14:52:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.388+0000 7f69537a4b80 1 -- 172.21.15.32:0/2941356139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483cd0 msgr2=0x56223249bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.388+0000 7f69537a4b80 1 --2- 172.21.15.32:0/2941356139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483cd0 0x56223249bbf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f693800a900 tx=0x7f693800d040 comp rx=0 tx=0).stop 2023-09-30T14:52:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.388+0000 7f69537a4b80 1 -- 172.21.15.32:0/2941356139 shutdown_connections 2023-09-30T14:52:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.388+0000 7f69537a4b80 1 --2- 172.21.15.32:0/2941356139 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5622324a05d0 0x5622324a29f0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.388+0000 7f69537a4b80 1 --2- 172.21.15.32:0/2941356139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483cd0 0x56223249bbf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.388+0000 7f6928ff9700 0 client.26356 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:52:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.388+0000 7f69537a4b80 1 --2- 172.21.15.32:0/2941356139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232482200 0x562232494670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.388+0000 7f69537a4b80 1 --2- 172.21.15.32:0/2941356139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232483310 0x562232494bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.388+0000 7f69537a4b80 1 -- 172.21.15.32:0/2941356139 >> 172.21.15.32:0/2941356139 conn(0x5622323f6200 msgr2=0x56223247c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.388+0000 7f69537a4b80 1 -- 172.21.15.32:0/2941356139 shutdown_connections 2023-09-30T14:52:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.388+0000 7f69537a4b80 1 -- 172.21.15.32:0/2941356139 wait complete. 2023-09-30T14:52:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.388+0000 7f69537a4b80 20 client.26356 trim_cache size 0 max 16384 2023-09-30T14:52:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.403+0000 7f69537a4b80 1 Processor -- start 2023-09-30T14:52:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.403+0000 7f69537a4b80 1 -- start start 2023-09-30T14:52:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.404+0000 7f69537a4b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232482200 0x562232493d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.404+0000 7f69537a4b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232483310 0x56223249bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.404+0000 7f69537a4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483cd0 0x56223247cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.404+0000 7f69537a4b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622322558c0 con 0x562232482200 2023-09-30T14:52:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.404+0000 7f69537a4b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562232493b20 con 0x562232483310 2023-09-30T14:52:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.404+0000 7f69537a4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562232482a80 con 0x562232483cd0 2023-09-30T14:52:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.404+0000 7f6945358700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232482200 0x562232493d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.404+0000 7f6945358700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232482200 0x562232493d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38546/0 (socket says 172.21.15.32:38546) 2023-09-30T14:52:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.404+0000 7f6945358700 1 -- 172.21.15.32:0/3631359959 learned_addr learned my addr 172.21.15.32:0/3631359959 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.404+0000 7f6945358700 1 -- 172.21.15.32:0/3631359959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483cd0 msgr2=0x56223247cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.404+0000 7f6945358700 1 --2- 172.21.15.32:0/3631359959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483cd0 0x56223247cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.404+0000 7f6945358700 1 -- 172.21.15.32:0/3631359959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232483310 msgr2=0x56223249bbb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T14:52:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.404+0000 7f6945358700 1 --2- 172.21.15.32:0/3631359959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232483310 0x56223249bbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.404+0000 7f6945358700 1 -- 172.21.15.32:0/3631359959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622321e8c00 con 0x562232482200 2023-09-30T14:52:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.404+0000 7f6945358700 1 --2- 172.21.15.32:0/3631359959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232482200 0x562232493d20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f693c01b040 tx=0x7f693c009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552b633ad2533e04 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.405+0000 7f692bfff700 1 -- 172.21.15.32:0/3631359959 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c029b00 con 0x562232482200 2023-09-30T14:52:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.405+0000 7f692bfff700 1 -- 172.21.15.32:0/3631359959 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f693c029070 con 0x562232482200 2023-09-30T14:52:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.405+0000 7f692bfff700 1 -- 172.21.15.32:0/3631359959 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c003b40 con 0x562232482200 2023-09-30T14:52:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.406+0000 7f69537a4b80 1 -- 172.21.15.32:0/3631359959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232482200 msgr2=0x562232493d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.406+0000 7f69537a4b80 1 --2- 172.21.15.32:0/3631359959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232482200 0x562232493d20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f693c01b040 tx=0x7f693c009730 comp rx=0 tx=0).stop 2023-09-30T14:52:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.406+0000 7f69537a4b80 1 -- 172.21.15.32:0/3631359959 shutdown_connections 2023-09-30T14:52:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.406+0000 7f69537a4b80 1 --2- 172.21.15.32:0/3631359959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232483cd0 0x56223247cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.406+0000 7f69537a4b80 1 --2- 172.21.15.32:0/3631359959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232482200 0x562232493d20 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.406+0000 7f69537a4b80 1 --2- 172.21.15.32:0/3631359959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232483310 0x56223249bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.406+0000 7f69537a4b80 1 -- 172.21.15.32:0/3631359959 >> 172.21.15.32:0/3631359959 conn(0x5622323f6200 msgr2=0x5622324a7350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.406+0000 7f69537a4b80 1 -- 172.21.15.32:0/3631359959 shutdown_connections 2023-09-30T14:52:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.406+0000 7f69537a4b80 1 -- 172.21.15.32:0/3631359959 wait complete. 2023-09-30T14:52:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.406+0000 7f69537a4b80 1 -- start start 2023-09-30T14:52:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.406+0000 7f69537a4b80 1 Processor -- start 2023-09-30T14:52:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f69537a4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232482200 0x56223249b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f69537a4b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232483310 0x56223249b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f69537a4b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232483cd0 0x5622324999b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f69537a4b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562232499ef0 con 0x562232483cd0 2023-09-30T14:52:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f69537a4b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223249a060 con 0x562232483310 2023-09-30T14:52:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f69537a4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223249a220 con 0x562232482200 2023-09-30T14:52:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f69537a4b80 10 client.0 _ll_register_callbacks cb 0x7f693800aeb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:52:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f6945b59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232483cd0 0x5622324999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f6945b59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232483cd0 0x5622324999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38552/0 (socket says 172.21.15.32:38552) 2023-09-30T14:52:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f6945b59700 1 -- 172.21.15.32:0/1630098552 learned_addr learned my addr 172.21.15.32:0/1630098552 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f6945358700 1 --2- 172.21.15.32:0/1630098552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232482200 0x56223249b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f6944b57700 1 --2- 172.21.15.32:0/1630098552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232483310 0x56223249b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f6945b59700 1 -- 172.21.15.32:0/1630098552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232482200 msgr2=0x56223249b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f6945b59700 1 --2- 172.21.15.32:0/1630098552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232482200 0x56223249b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f6945b59700 1 -- 172.21.15.32:0/1630098552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232483310 msgr2=0x56223249b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f6945b59700 1 --2- 172.21.15.32:0/1630098552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232483310 0x56223249b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f6945b59700 1 -- 172.21.15.32:0/1630098552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562232437870 con 0x562232483cd0 2023-09-30T14:52:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f6945358700 1 --2- 172.21.15.32:0/1630098552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232482200 0x56223249b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:52:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.407+0000 7f6944b57700 1 --2- 172.21.15.32:0/1630098552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232483310 0x56223249b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:52:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.408+0000 7f6945b59700 1 --2- 172.21.15.32:0/1630098552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232483cd0 0x5622324999b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f693802d7c0 tx=0x7f693802d7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1411b052da5a162 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.408+0000 7f6928ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:52:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.408+0000 7f6928ff9700 1 -- 172.21.15.32:0/1630098552 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693802de10 con 0x562232483cd0 2023-09-30T14:52:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.408+0000 7f6928ff9700 1 -- 172.21.15.32:0/1630098552 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6938011070 con 0x562232483cd0 2023-09-30T14:52:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.408+0000 7f69537a4b80 1 -- 172.21.15.32:0/1630098552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f693c004380 con 0x562232483cd0 2023-09-30T14:52:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.408+0000 7f6928ff9700 1 -- 172.21.15.32:0/1630098552 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6938015690 con 0x562232483cd0 2023-09-30T14:52:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.408+0000 7f69537a4b80 10 client.16698 fetch_fsmap learned FSMap version 47 2023-09-30T14:52:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.408+0000 7f69537a4b80 1 -- 172.21.15.32:0/1630098552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5622322261f0 con 0x562232483cd0 2023-09-30T14:52:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.408+0000 7f6928ff9700 1 -- 172.21.15.32:0/1630098552 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f693c004380 con 0x562232483cd0 2023-09-30T14:52:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.408+0000 7f6928ff9700 1 -- 172.21.15.32:0/1630098552 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 47) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f69380228a0 con 0x562232483cd0 2023-09-30T14:52:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.408+0000 7f69537a4b80 10 client.16698 fetch_fsmap finished waiting for FSMap version 47 2023-09-30T14:52:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.408+0000 7f69537a4b80 10 client.16698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:52:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.408+0000 7f69537a4b80 20 client.16698 populate_metadata read hostname 'smithi032' 2023-09-30T14:52:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.409+0000 7f69537a4b80 1 --2- 172.21.15.32:0/1630098552 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5622324f5640 0x5622324f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.409+0000 7f69537a4b80 4 client.16698 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:52:37.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.409+0000 7f69537a4b80 1 -- 172.21.15.32:0/1630098552 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5622323cdad0 con 0x5622324f5640 2023-09-30T14:52:37.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.409+0000 7f6945358700 1 --2- 172.21.15.32:0/1630098552 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5622324f5640 0x5622324f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:37.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.409+0000 7f6928ff9700 10 client.16698 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:52:37.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.409+0000 7f6945358700 1 --2- 172.21.15.32:0/1630098552 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5622324f5640 0x5622324f7a60 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e7b86f465c67ddf server_cookie=99f347207dba9a5a in_seq=0 out_seq=0 2023-09-30T14:52:37.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:37 smithi156 ceph-mon[103579]: pgmap v717: 97 pgs: 97 active+clean; 636 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.5 MiB/s wr, 806 op/s 2023-09-30T14:52:37.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:37 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:52:37.613 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:37 smithi148 ceph-mon[102355]: pgmap v717: 97 pgs: 97 active+clean; 636 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.5 MiB/s wr, 806 op/s 2023-09-30T14:52:37.613 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:37 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:52:37.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:37 smithi032 ceph-mon[98592]: pgmap v717: 97 pgs: 97 active+clean; 636 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.5 MiB/s wr, 806 op/s 2023-09-30T14:52:37.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:37 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:52:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.706+0000 7f6928ff9700 1 -- 172.21.15.32:0/1630098552 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5622323cdad0 con 0x5622324f5640 2023-09-30T14:52:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.706+0000 7f6928ff9700 10 client.16698 handle_command_reply: tid=0 2023-09-30T14:52:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.706+0000 7f69537a4b80 1 client.16698 shutdown 2023-09-30T14:52:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.706+0000 7f69537a4b80 1 -- 172.21.15.32:0/1630098552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232483cd0 msgr2=0x5622324999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.706+0000 7f69537a4b80 1 --2- 172.21.15.32:0/1630098552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232483cd0 0x5622324999b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f693802d7c0 tx=0x7f693802d7f0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.706+0000 7f69537a4b80 1 -- 172.21.15.32:0/1630098552 shutdown_connections 2023-09-30T14:52:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.706+0000 7f69537a4b80 1 --2- 172.21.15.32:0/1630098552 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5622324f5640 0x5622324f7a60 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.706+0000 7f69537a4b80 1 --2- 172.21.15.32:0/1630098552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562232483cd0 0x5622324999b0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.706+0000 7f6928ff9700 0 client.16698 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:52:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.706+0000 7f69537a4b80 1 --2- 172.21.15.32:0/1630098552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562232482200 0x56223249b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.706+0000 7f69537a4b80 1 --2- 172.21.15.32:0/1630098552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562232483310 0x56223249b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.706+0000 7f69537a4b80 1 -- 172.21.15.32:0/1630098552 >> 172.21.15.32:0/1630098552 conn(0x5622323f6200 msgr2=0x56223249c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.706+0000 7f69537a4b80 1 -- 172.21.15.32:0/1630098552 shutdown_connections 2023-09-30T14:52:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.707+0000 7f69537a4b80 1 -- 172.21.15.32:0/1630098552 wait complete. 2023-09-30T14:52:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.707+0000 7f69537a4b80 20 client.16698 trim_cache size 0 max 16384 2023-09-30T14:52:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:52:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:52:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:52:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:52:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.707+0000 7f69475bc700 1 -- 172.21.15.32:0/1792036427 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f692c0a1020 msgr2=0x7f692c0a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.707+0000 7f69475bc700 1 --2- 172.21.15.32:0/1792036427 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f692c0a1020 0x7f692c0a34e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x562232481fa0 tx=0x7f6938004be0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.707+0000 7f69475bc700 1 -- 172.21.15.32:0/1792036427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69401021b0 msgr2=0x7f69401972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.707+0000 7f69475bc700 1 --2- 172.21.15.32:0/1792036427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69401021b0 0x7f69401972b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6930004730 tx=0x7f6930004760 comp rx=0 tx=0).stop 2023-09-30T14:52:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.707+0000 7f69475bc700 1 -- 172.21.15.32:0/1792036427 shutdown_connections 2023-09-30T14:52:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.707+0000 7f69475bc700 1 --2- 172.21.15.32:0/1792036427 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f692c0a1020 0x7f692c0a34e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.707+0000 7f69475bc700 1 --2- 172.21.15.32:0/1792036427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6940102b50 0x7f6940197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.707+0000 7f69475bc700 1 --2- 172.21.15.32:0/1792036427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69400ff140 0x7f6940196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.707+0000 7f69475bc700 1 --2- 172.21.15.32:0/1792036427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69401021b0 0x7f69401972b0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.707+0000 7f69475bc700 1 -- 172.21.15.32:0/1792036427 >> 172.21.15.32:0/1792036427 conn(0x7f69400faa30 msgr2=0x7f69401056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.708+0000 7f69475bc700 1 -- 172.21.15.32:0/1792036427 shutdown_connections 2023-09-30T14:52:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:37.708+0000 7f69475bc700 1 -- 172.21.15.32:0/1792036427 wait complete. 2023-09-30T14:52:38.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 41935 34.22 MB/sec warmup 71 sec latency 174.228 ms 2023-09-30T14:52:38.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:38 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:52:38.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:38 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:52:38.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:38 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:52:38.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:38 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:52:38.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:38 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:52:38.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:38 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:52:38.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:38 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:52:38.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:38 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:52:38.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:38 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:52:38.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:38 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:52:38.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:38 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:52:38.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:38 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:52:39.001 INFO:tasks.workunit.client.0.smithi032.stdout: 10 42680 34.27 MB/sec warmup 72 sec latency 73.356 ms 2023-09-30T14:52:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:39 smithi156 ceph-mon[103579]: pgmap v718: 97 pgs: 97 active+clean; 639 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 8.9 MiB/s wr, 719 op/s 2023-09-30T14:52:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:39 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:52:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:39 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:52:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:39 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:52:39.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:39 smithi148 ceph-mon[102355]: pgmap v718: 97 pgs: 97 active+clean; 639 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 8.9 MiB/s wr, 719 op/s 2023-09-30T14:52:39.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:39 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:52:39.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:39 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:52:39.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:39 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:52:39.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:39 smithi032 ceph-mon[98592]: pgmap v718: 97 pgs: 97 active+clean; 639 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 8.9 MiB/s wr, 719 op/s 2023-09-30T14:52:39.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:39 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:52:39.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:39 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:52:39.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:39 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:52:40.002 INFO:tasks.workunit.client.0.smithi032.stdout: 10 43583 34.44 MB/sec warmup 73 sec latency 80.106 ms 2023-09-30T14:52:41.002 INFO:tasks.workunit.client.0.smithi032.stdout: 10 44317 34.72 MB/sec warmup 74 sec latency 204.723 ms 2023-09-30T14:52:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:41 smithi156 ceph-mon[103579]: pgmap v719: 97 pgs: 97 active+clean; 634 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.9 MiB/s wr, 635 op/s 2023-09-30T14:52:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:41 smithi148 ceph-mon[102355]: pgmap v719: 97 pgs: 97 active+clean; 634 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.9 MiB/s wr, 635 op/s 2023-09-30T14:52:41.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:41 smithi032 ceph-mon[98592]: pgmap v719: 97 pgs: 97 active+clean; 634 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.9 MiB/s wr, 635 op/s 2023-09-30T14:52:42.002 INFO:tasks.workunit.client.0.smithi032.stdout: 10 44981 34.74 MB/sec warmup 75 sec latency 127.246 ms 2023-09-30T14:52:43.002 INFO:tasks.workunit.client.0.smithi032.stdout: 10 45564 34.61 MB/sec warmup 76 sec latency 77.165 ms 2023-09-30T14:52:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:43 smithi156 ceph-mon[103579]: pgmap v720: 97 pgs: 97 active+clean; 642 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 10 MiB/s wr, 823 op/s 2023-09-30T14:52:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:43 smithi148 ceph-mon[102355]: pgmap v720: 97 pgs: 97 active+clean; 642 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 10 MiB/s wr, 823 op/s 2023-09-30T14:52:43.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:43 smithi032 ceph-mon[98592]: pgmap v720: 97 pgs: 97 active+clean; 642 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 10 MiB/s wr, 823 op/s 2023-09-30T14:52:44.002 INFO:tasks.workunit.client.0.smithi032.stdout: 10 46248 34.56 MB/sec warmup 77 sec latency 273.878 ms 2023-09-30T14:52:45.002 INFO:tasks.workunit.client.0.smithi032.stdout: 10 47329 34.88 MB/sec warmup 78 sec latency 50.635 ms 2023-09-30T14:52:45.808 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:45 smithi156 ceph-mon[103579]: pgmap v721: 97 pgs: 97 active+clean; 641 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 9.3 MiB/s wr, 785 op/s 2023-09-30T14:52:45.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:45 smithi148 ceph-mon[102355]: pgmap v721: 97 pgs: 97 active+clean; 641 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 9.3 MiB/s wr, 785 op/s 2023-09-30T14:52:45.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:45 smithi032 ceph-mon[98592]: pgmap v721: 97 pgs: 97 active+clean; 641 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 9.3 MiB/s wr, 785 op/s 2023-09-30T14:52:46.002 INFO:tasks.workunit.client.0.smithi032.stdout: 10 47826 34.84 MB/sec warmup 79 sec latency 208.903 ms 2023-09-30T14:52:47.002 INFO:tasks.workunit.client.0.smithi032.stdout: 10 48550 34.93 MB/sec warmup 80 sec latency 261.742 ms 2023-09-30T14:52:47.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:47 smithi156 ceph-mon[103579]: pgmap v722: 97 pgs: 97 active+clean; 647 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.8 MiB/s wr, 833 op/s 2023-09-30T14:52:47.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:47 smithi148 ceph-mon[102355]: pgmap v722: 97 pgs: 97 active+clean; 647 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.8 MiB/s wr, 833 op/s 2023-09-30T14:52:47.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:47 smithi032 ceph-mon[98592]: pgmap v722: 97 pgs: 97 active+clean; 647 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.8 MiB/s wr, 833 op/s 2023-09-30T14:52:48.002 INFO:tasks.workunit.client.0.smithi032.stdout: 10 49259 35.00 MB/sec warmup 81 sec latency 77.488 ms 2023-09-30T14:52:49.002 INFO:tasks.workunit.client.0.smithi032.stdout: 10 49929 34.94 MB/sec warmup 82 sec latency 59.684 ms 2023-09-30T14:52:49.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:49 smithi156 ceph-mon[103579]: pgmap v723: 97 pgs: 97 active+clean; 654 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.2 MiB/s wr, 804 op/s 2023-09-30T14:52:49.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:49 smithi148 ceph-mon[102355]: pgmap v723: 97 pgs: 97 active+clean; 654 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.2 MiB/s wr, 804 op/s 2023-09-30T14:52:49.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:49 smithi032 ceph-mon[98592]: pgmap v723: 97 pgs: 97 active+clean; 654 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.2 MiB/s wr, 804 op/s 2023-09-30T14:52:50.003 INFO:tasks.workunit.client.0.smithi032.stdout: 10 50737 34.96 MB/sec warmup 83 sec latency 42.799 ms 2023-09-30T14:52:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.887+0000 7f45b4148700 1 -- 172.21.15.32:0/698816335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45ac072cf0 msgr2=0x7f45ac073110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.887+0000 7f45b4148700 1 --2- 172.21.15.32:0/698816335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45ac072cf0 0x7f45ac073110 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f459c005fd0 tx=0x7f459c0077c0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.889+0000 7f45b4148700 1 -- 172.21.15.32:0/698816335 shutdown_connections 2023-09-30T14:52:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.889+0000 7f45b4148700 1 --2- 172.21.15.32:0/698816335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45ac0785a0 0x7f45ac07aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.889+0000 7f45b4148700 1 --2- 172.21.15.32:0/698816335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45ac072cf0 0x7f45ac073110 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.889+0000 7f45b4148700 1 --2- 172.21.15.32:0/698816335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac075bd0 0x7f45ac078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.889+0000 7f45b4148700 1 -- 172.21.15.32:0/698816335 >> 172.21.15.32:0/698816335 conn(0x7f45ac06dc80 msgr2=0x7f45ac0700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.905+0000 7f45b4148700 1 -- 172.21.15.32:0/698816335 shutdown_connections 2023-09-30T14:52:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.907+0000 7f45b4148700 1 -- 172.21.15.32:0/698816335 wait complete. 2023-09-30T14:52:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.908+0000 7f45b4148700 1 Processor -- start 2023-09-30T14:52:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.908+0000 7f45b4148700 1 -- start start 2023-09-30T14:52:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.908+0000 7f45b4148700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac072cf0 0x7f45ac1aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.908+0000 7f45b4148700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45ac0785a0 0x7f45ac1af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.908+0000 7f45b4148700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45ac1b3c00 0x7f45ac1b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.908+0000 7f45b4148700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac1b6600 con 0x7f45ac072cf0 2023-09-30T14:52:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.908+0000 7f45b4148700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac1b6770 con 0x7f45ac0785a0 2023-09-30T14:52:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.908+0000 7f45b4148700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac1b6930 con 0x7f45ac1b3c00 2023-09-30T14:52:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.909+0000 7f45b1ee4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac072cf0 0x7f45ac1aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.909+0000 7f45b1ee4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac072cf0 0x7f45ac1aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38564/0 (socket says 172.21.15.32:38564) 2023-09-30T14:52:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.909+0000 7f45b1ee4700 1 -- 172.21.15.32:0/2592489413 learned_addr learned my addr 172.21.15.32:0/2592489413 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.912+0000 7f45b1ee4700 1 -- 172.21.15.32:0/2592489413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45ac1b3c00 msgr2=0x7f45ac1b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.912+0000 7f45b1ee4700 1 --2- 172.21.15.32:0/2592489413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45ac1b3c00 0x7f45ac1b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.912+0000 7f45b1ee4700 1 -- 172.21.15.32:0/2592489413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45ac0785a0 msgr2=0x7f45ac1af280 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-30T14:52:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.912+0000 7f45b1ee4700 1 --2- 172.21.15.32:0/2592489413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45ac0785a0 0x7f45ac1af280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.912+0000 7f45b1ee4700 1 -- 172.21.15.32:0/2592489413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c0071c0 con 0x7f45ac072cf0 2023-09-30T14:52:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.912+0000 7f45b1ee4700 1 --2- 172.21.15.32:0/2592489413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac072cf0 0x7f45ac1aed20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f459c0039a0 tx=0x7f459c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.914+0000 7f45a2ffd700 1 -- 172.21.15.32:0/2592489413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c00f070 con 0x7f45ac072cf0 2023-09-30T14:52:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.914+0000 7f45a2ffd700 1 -- 172.21.15.32:0/2592489413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f459c00ca80 con 0x7f45ac072cf0 2023-09-30T14:52:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.914+0000 7f45a2ffd700 1 -- 172.21.15.32:0/2592489413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c00a6c0 con 0x7f45ac072cf0 2023-09-30T14:52:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.914+0000 7f45b4148700 1 -- 172.21.15.32:0/2592489413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45ac1b6bb0 con 0x7f45ac072cf0 2023-09-30T14:52:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.914+0000 7f45b4148700 1 -- 172.21.15.32:0/2592489413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ac1b70f0 con 0x7f45ac072cf0 2023-09-30T14:52:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.919+0000 7f45a2ffd700 1 -- 172.21.15.32:0/2592489413 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f459c00a820 con 0x7f45ac072cf0 2023-09-30T14:52:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.921+0000 7f45a2ffd700 1 --2- 172.21.15.32:0/2592489413 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f45980a1120 0x7f45980a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.921+0000 7f45a2ffd700 1 -- 172.21.15.32:0/2592489413 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f459c0cd180 con 0x7f45ac072cf0 2023-09-30T14:52:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.922+0000 7f45c0330b80 1 Processor -- start 2023-09-30T14:52:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.922+0000 7f45b16e3700 1 --2- 172.21.15.32:0/2592489413 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f45980a1120 0x7f45980a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.923+0000 7f45b16e3700 1 --2- 172.21.15.32:0/2592489413 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f45980a1120 0x7f45980a35e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f45a8005e50 tx=0x7f45a8005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.930+0000 7f45c0330b80 1 -- start start 2023-09-30T14:52:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.930+0000 7f45c0330b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb65200 0x557beeb65620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.930+0000 7f45c0330b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66310 0x557beeb66790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.930+0000 7f45c0330b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 0x557beeb6d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.930+0000 7f45c0330b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beeacd400 con 0x557beeb66310 2023-09-30T14:52:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.930+0000 7f45c0330b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beeabebc0 con 0x557beeb66cd0 2023-09-30T14:52:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.930+0000 7f45c0330b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beeabc4e0 con 0x557beeb65200 2023-09-30T14:52:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.930+0000 7f45b26e5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 0x557beeb6d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.931+0000 7f45b26e5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 0x557beeb6d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:50050/0 (socket says 172.21.15.32:50050) 2023-09-30T14:52:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.931+0000 7f45b26e5700 1 -- 172.21.15.32:0/279501255 learned_addr learned my addr 172.21.15.32:0/279501255 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.931+0000 7f45b16e3700 1 --2- 172.21.15.32:0/279501255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66310 0x557beeb66790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.931+0000 7f45b16e3700 1 -- 172.21.15.32:0/279501255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb65200 msgr2=0x557beeb65620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-30T14:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.931+0000 7f45b16e3700 1 --2- 172.21.15.32:0/279501255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb65200 0x557beeb65620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.931+0000 7f45b16e3700 1 -- 172.21.15.32:0/279501255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 msgr2=0x557beeb6d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.931+0000 7f45b16e3700 1 --2- 172.21.15.32:0/279501255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 0x557beeb6d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.931+0000 7f45b16e3700 1 -- 172.21.15.32:0/279501255 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557beeb1a870 con 0x557beeb66310 2023-09-30T14:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.931+0000 7f45b16e3700 1 --2- 172.21.15.32:0/279501255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66310 0x557beeb66790 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f45ac072390 tx=0x7f45a8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496f4899ac52dacb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.931+0000 7f4597fff700 1 -- 172.21.15.32:0/279501255 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a80108c0 con 0x557beeb66310 2023-09-30T14:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.932+0000 7f4597fff700 1 -- 172.21.15.32:0/279501255 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45a801d440 con 0x557beeb66310 2023-09-30T14:52:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.932+0000 7f4597fff700 1 -- 172.21.15.32:0/279501255 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a800f620 con 0x557beeb66310 2023-09-30T14:52:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.933+0000 7f45c0330b80 1 -- 172.21.15.32:0/279501255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66310 msgr2=0x557beeb66790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.933+0000 7f45c0330b80 1 --2- 172.21.15.32:0/279501255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66310 0x557beeb66790 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f45ac072390 tx=0x7f45a8010000 comp rx=0 tx=0).stop 2023-09-30T14:52:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.933+0000 7f45c0330b80 1 -- 172.21.15.32:0/279501255 shutdown_connections 2023-09-30T14:52:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.933+0000 7f45c0330b80 1 --2- 172.21.15.32:0/279501255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 0x557beeb6d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.933+0000 7f45c0330b80 1 --2- 172.21.15.32:0/279501255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb65200 0x557beeb65620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.933+0000 7f45c0330b80 1 --2- 172.21.15.32:0/279501255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66310 0x557beeb66790 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.933+0000 7f45c0330b80 1 -- 172.21.15.32:0/279501255 >> 172.21.15.32:0/279501255 conn(0x557beead9200 msgr2=0x557beeb5fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.933+0000 7f45c0330b80 1 -- 172.21.15.32:0/279501255 shutdown_connections 2023-09-30T14:52:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.933+0000 7f45c0330b80 1 -- 172.21.15.32:0/279501255 wait complete. 2023-09-30T14:52:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.934+0000 7f45c0330b80 1 -- start start 2023-09-30T14:52:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.934+0000 7f45c0330b80 1 Processor -- start 2023-09-30T14:52:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.936+0000 7f45c0330b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb65200 0x557beeb6ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.936+0000 7f45c0330b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb66310 0x557beeb6f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.936+0000 7f45c0330b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 0x557beeb76410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.936+0000 7f45c0330b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beeab7dd0 con 0x557beeb65200 2023-09-30T14:52:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.936+0000 7f45c0330b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beea546b0 con 0x557beeb66cd0 2023-09-30T14:52:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.936+0000 7f45c0330b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beea54350 con 0x557beeb66310 2023-09-30T14:52:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.936+0000 7f45c0330b80 10 client.0 _ll_register_callbacks cb 0x557bee8b17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:52:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.936+0000 7f45b1ee4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb65200 0x557beeb6ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.936+0000 7f45b1ee4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb65200 0x557beeb6ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38580/0 (socket says 172.21.15.32:38580) 2023-09-30T14:52:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.936+0000 7f45b16e3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb66310 0x557beeb6f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.936+0000 7f45b1ee4700 1 -- 172.21.15.32:0/1227018860 learned_addr learned my addr 172.21.15.32:0/1227018860 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.936+0000 7f45b26e5700 1 --2- 172.21.15.32:0/1227018860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 0x557beeb76410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.936+0000 7f45b1ee4700 1 -- 172.21.15.32:0/1227018860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb66310 msgr2=0x557beeb6f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.936+0000 7f45b1ee4700 1 --2- 172.21.15.32:0/1227018860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb66310 0x557beeb6f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.936+0000 7f45b1ee4700 1 -- 172.21.15.32:0/1227018860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 msgr2=0x557beeb76410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.936+0000 7f45b1ee4700 1 --2- 172.21.15.32:0/1227018860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 0x557beeb76410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.937+0000 7f45b1ee4700 1 -- 172.21.15.32:0/1227018860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a800d7a0 con 0x557beeb65200 2023-09-30T14:52:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.937+0000 7f45b16e3700 1 --2- 172.21.15.32:0/1227018860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb66310 0x557beeb6f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:52:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.937+0000 7f45b26e5700 1 --2- 172.21.15.32:0/1227018860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 0x557beeb76410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:52:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.937+0000 7f45b1ee4700 1 --2- 172.21.15.32:0/1227018860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb65200 0x557beeb6ee90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f459c005e80 tx=0x7f459c09a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30a2303f0dcf697 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.937+0000 7f45957fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:52:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.937+0000 7f45957fa700 1 -- 172.21.15.32:0/1227018860 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c09ade0 con 0x557beeb65200 2023-09-30T14:52:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.937+0000 7f45957fa700 1 -- 172.21.15.32:0/1227018860 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f459c09a6e0 con 0x557beeb65200 2023-09-30T14:52:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.937+0000 7f45c0330b80 1 -- 172.21.15.32:0/1227018860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557beeb58090 con 0x557beeb65200 2023-09-30T14:52:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.937+0000 7f45957fa700 1 -- 172.21.15.32:0/1227018860 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f459c09e830 con 0x557beeb65200 2023-09-30T14:52:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.937+0000 7f45c0330b80 10 client.16710 fetch_fsmap learned FSMap version 47 2023-09-30T14:52:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.937+0000 7f45c0330b80 1 -- 172.21.15.32:0/1227018860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x557bee8e2d80 con 0x557beeb65200 2023-09-30T14:52:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.938+0000 7f45957fa700 1 -- 172.21.15.32:0/1227018860 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f459c0a6020 con 0x557beeb65200 2023-09-30T14:52:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.938+0000 7f45957fa700 1 -- 172.21.15.32:0/1227018860 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 47) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f459c099bd0 con 0x557beeb65200 2023-09-30T14:52:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.938+0000 7f45c0330b80 10 client.16710 fetch_fsmap finished waiting for FSMap version 47 2023-09-30T14:52:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.938+0000 7f45c0330b80 10 client.16710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:52:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.938+0000 7f45c0330b80 20 client.16710 populate_metadata read hostname 'smithi032' 2023-09-30T14:52:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.938+0000 7f45c0330b80 1 --2- 172.21.15.32:0/1227018860 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557beeb7aac0 0x557beeb7cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.938+0000 7f45c0330b80 4 client.16710 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:52:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.938+0000 7f45c0330b80 1 -- 172.21.15.32:0/1227018860 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557beeab0db0 con 0x557beeb7aac0 2023-09-30T14:52:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.943+0000 7f45b16e3700 1 --2- 172.21.15.32:0/1227018860 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557beeb7aac0 0x557beeb7cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.947+0000 7f45957fa700 10 client.16710 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:52:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.947+0000 7f45b16e3700 1 --2- 172.21.15.32:0/1227018860 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557beeb7aac0 0x557beeb7cee0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e87e6b1265592b77 server_cookie=4212dc9140533f49 in_seq=0 out_seq=0 2023-09-30T14:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.952+0000 7f45957fa700 1 -- 172.21.15.32:0/1227018860 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557beeab0db0 con 0x557beeb7aac0 2023-09-30T14:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.952+0000 7f45957fa700 10 client.16710 handle_command_reply: tid=0 2023-09-30T14:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.952+0000 7f45c0330b80 1 client.16710 shutdown 2023-09-30T14:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.953+0000 7f45c0330b80 1 -- 172.21.15.32:0/1227018860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb65200 msgr2=0x557beeb6ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.953+0000 7f45c0330b80 1 --2- 172.21.15.32:0/1227018860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb65200 0x557beeb6ee90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f459c005e80 tx=0x7f459c09a4b0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.953+0000 7f45c0330b80 1 -- 172.21.15.32:0/1227018860 shutdown_connections 2023-09-30T14:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.953+0000 7f45c0330b80 1 --2- 172.21.15.32:0/1227018860 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557beeb7aac0 0x557beeb7cee0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.953+0000 7f45c0330b80 1 --2- 172.21.15.32:0/1227018860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 0x557beeb76410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.953+0000 7f45957fa700 0 client.16710 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.953+0000 7f45c0330b80 1 --2- 172.21.15.32:0/1227018860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb65200 0x557beeb6ee90 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.953+0000 7f45c0330b80 1 --2- 172.21.15.32:0/1227018860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb66310 0x557beeb6f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.953+0000 7f45c0330b80 1 -- 172.21.15.32:0/1227018860 >> 172.21.15.32:0/1227018860 conn(0x557beead9200 msgr2=0x557beeb5f590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.953+0000 7f45c0330b80 1 -- 172.21.15.32:0/1227018860 shutdown_connections 2023-09-30T14:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.953+0000 7f45c0330b80 1 -- 172.21.15.32:0/1227018860 wait complete. 2023-09-30T14:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.953+0000 7f45c0330b80 20 client.16710 trim_cache size 0 max 16384 2023-09-30T14:52:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.972+0000 7f45c0330b80 1 Processor -- start 2023-09-30T14:52:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.972+0000 7f45c0330b80 1 -- start start 2023-09-30T14:52:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.972+0000 7f45c0330b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb65200 0x557beeb6dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.972+0000 7f45c0330b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66310 0x557beeb763a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.972+0000 7f45c0330b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 0x557beeb5fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.972+0000 7f45c0330b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bee9388c0 con 0x557beeb66310 2023-09-30T14:52:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.972+0000 7f45c0330b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beeb6ea00 con 0x557beeb66cd0 2023-09-30T14:52:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.972+0000 7f45c0330b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beeb6e150 con 0x557beeb65200 2023-09-30T14:52:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.973+0000 7f45b16e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66310 0x557beeb763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.973+0000 7f45b1ee4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb65200 0x557beeb6dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.973+0000 7f45b26e5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 0x557beeb5fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.973+0000 7f45b1ee4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb65200 0x557beeb6dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:52968/0 (socket says 172.21.15.32:52968) 2023-09-30T14:52:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.973+0000 7f45b1ee4700 1 -- 172.21.15.32:0/3545424729 learned_addr learned my addr 172.21.15.32:0/3545424729 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.973+0000 7f45b16e3700 1 -- 172.21.15.32:0/3545424729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb65200 msgr2=0x557beeb6dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.973+0000 7f45b16e3700 1 --2- 172.21.15.32:0/3545424729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb65200 0x557beeb6dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.973+0000 7f45b16e3700 1 -- 172.21.15.32:0/3545424729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 msgr2=0x557beeb5fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.973+0000 7f45b16e3700 1 --2- 172.21.15.32:0/3545424729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 0x557beeb5fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.973+0000 7f45b16e3700 1 -- 172.21.15.32:0/3545424729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bee8cbc00 con 0x557beeb66310 2023-09-30T14:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.973+0000 7f45b1ee4700 1 --2- 172.21.15.32:0/3545424729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb65200 0x557beeb6dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.973+0000 7f45b26e5700 1 --2- 172.21.15.32:0/3545424729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 0x557beeb5fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.973+0000 7f45b16e3700 1 --2- 172.21.15.32:0/3545424729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66310 0x557beeb763a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f45a800bfd0 tx=0x7f45a8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6e357c7b3a8d61 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.973+0000 7f45a0ff9700 1 -- 172.21.15.32:0/3545424729 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a802d040 con 0x557beeb66310 2023-09-30T14:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.973+0000 7f45a0ff9700 1 -- 172.21.15.32:0/3545424729 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45a801dd60 con 0x557beeb66310 2023-09-30T14:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.973+0000 7f45a0ff9700 1 -- 172.21.15.32:0/3545424729 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a8032e30 con 0x557beeb66310 2023-09-30T14:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.974+0000 7f45c0330b80 1 -- 172.21.15.32:0/3545424729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66310 msgr2=0x557beeb763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.974+0000 7f45c0330b80 1 --2- 172.21.15.32:0/3545424729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66310 0x557beeb763a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f45a800bfd0 tx=0x7f45a8010740 comp rx=0 tx=0).stop 2023-09-30T14:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.974+0000 7f45c0330b80 1 -- 172.21.15.32:0/3545424729 shutdown_connections 2023-09-30T14:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.974+0000 7f45c0330b80 1 --2- 172.21.15.32:0/3545424729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb66cd0 0x557beeb5fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.974+0000 7f45c0330b80 1 --2- 172.21.15.32:0/3545424729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb65200 0x557beeb6dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.974+0000 7f45c0330b80 1 --2- 172.21.15.32:0/3545424729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66310 0x557beeb763a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.974+0000 7f45c0330b80 1 -- 172.21.15.32:0/3545424729 >> 172.21.15.32:0/3545424729 conn(0x557beead9200 msgr2=0x557beeb81840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.974+0000 7f45c0330b80 1 -- 172.21.15.32:0/3545424729 shutdown_connections 2023-09-30T14:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.974+0000 7f45c0330b80 1 -- 172.21.15.32:0/3545424729 wait complete. 2023-09-30T14:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.974+0000 7f45c0330b80 1 -- start start 2023-09-30T14:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.975+0000 7f45c0330b80 1 Processor -- start 2023-09-30T14:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.975+0000 7f45c0330b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb65200 0x557beeb73de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.975+0000 7f45c0330b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb66310 0x557beeb74320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.975+0000 7f45c0330b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66cd0 0x557beeb75330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.975+0000 7f45c0330b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beeb75870 con 0x557beeb66cd0 2023-09-30T14:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.975+0000 7f45c0330b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beeb759e0 con 0x557beeb65200 2023-09-30T14:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.975+0000 7f45c0330b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beeb75ba0 con 0x557beeb66310 2023-09-30T14:52:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.975+0000 7f45c0330b80 10 client.0 _ll_register_callbacks cb 0x7f459c096380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.975+0000 7f45b26e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66cd0 0x557beeb75330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.975+0000 7f45b26e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66cd0 0x557beeb75330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38596/0 (socket says 172.21.15.32:38596) 2023-09-30T14:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.975+0000 7f45b26e5700 1 -- 172.21.15.32:0/1303564448 learned_addr learned my addr 172.21.15.32:0/1303564448 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.975+0000 7f45b1ee4700 1 --2- 172.21.15.32:0/1303564448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb65200 0x557beeb73de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.975+0000 7f45b16e3700 1 --2- 172.21.15.32:0/1303564448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb66310 0x557beeb74320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45b26e5700 1 -- 172.21.15.32:0/1303564448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb66310 msgr2=0x557beeb74320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45b26e5700 1 --2- 172.21.15.32:0/1303564448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb66310 0x557beeb74320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45b26e5700 1 -- 172.21.15.32:0/1303564448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb65200 msgr2=0x557beeb73de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45b26e5700 1 --2- 172.21.15.32:0/1303564448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb65200 0x557beeb73de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45b26e5700 1 -- 172.21.15.32:0/1303564448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bee8e2d80 con 0x557beeb66cd0 2023-09-30T14:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45b26e5700 1 --2- 172.21.15.32:0/1303564448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66cd0 0x557beeb75330 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f45a4007dd0 tx=0x7f45a4012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4e781c6c28aec5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45957fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45957fa700 1 -- 172.21.15.32:0/1303564448 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4012e10 con 0x557beeb66cd0 2023-09-30T14:52:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45957fa700 1 -- 172.21.15.32:0/1303564448 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f45a4018470 con 0x557beeb66cd0 2023-09-30T14:52:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45c0330b80 1 -- 172.21.15.32:0/1303564448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45a801d440 con 0x557beeb66cd0 2023-09-30T14:52:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45957fa700 1 -- 172.21.15.32:0/1303564448 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45a4007ec0 con 0x557beeb66cd0 2023-09-30T14:52:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45c0330b80 10 client.16716 fetch_fsmap learned FSMap version 47 2023-09-30T14:52:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45c0330b80 1 -- 172.21.15.32:0/1303564448 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f45a4009bb0 con 0x557beeb66cd0 2023-09-30T14:52:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45957fa700 1 -- 172.21.15.32:0/1303564448 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45a801d440 con 0x557beeb66cd0 2023-09-30T14:52:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45957fa700 1 -- 172.21.15.32:0/1303564448 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 47) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f45a4014040 con 0x557beeb66cd0 2023-09-30T14:52:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45c0330b80 10 client.16716 fetch_fsmap finished waiting for FSMap version 47 2023-09-30T14:52:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45c0330b80 10 client.16716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:52:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.976+0000 7f45c0330b80 20 client.16716 populate_metadata read hostname 'smithi032' 2023-09-30T14:52:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.977+0000 7f45c0330b80 1 --2- 172.21.15.32:0/1303564448 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557beebcf630 0x557beebd1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:52:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.977+0000 7f45c0330b80 4 client.16716 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:52:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.977+0000 7f45c0330b80 1 -- 172.21.15.32:0/1303564448 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557beeab0ad0 con 0x557beebcf630 2023-09-30T14:52:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.977+0000 7f45b1ee4700 1 --2- 172.21.15.32:0/1303564448 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557beebcf630 0x557beebd1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:52:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.977+0000 7f45957fa700 10 client.16716 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:52:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:50.977+0000 7f45b1ee4700 1 --2- 172.21.15.32:0/1303564448 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557beebcf630 0x557beebd1a50 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d032f87cc3e7312f server_cookie=6d31b236fe916126 in_seq=0 out_seq=0 2023-09-30T14:52:51.003 INFO:tasks.workunit.client.0.smithi032.stdout: 10 51518 35.11 MB/sec warmup 84 sec latency 220.943 ms 2023-09-30T14:52:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.422+0000 7f45957fa700 1 -- 172.21.15.32:0/1303564448 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557beeab0ad0 con 0x557beebcf630 2023-09-30T14:52:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.422+0000 7f45957fa700 10 client.16716 handle_command_reply: tid=0 2023-09-30T14:52:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.422+0000 7f45c0330b80 1 client.16716 shutdown 2023-09-30T14:52:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.422+0000 7f45c0330b80 1 -- 172.21.15.32:0/1303564448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66cd0 msgr2=0x557beeb75330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.422+0000 7f45c0330b80 1 --2- 172.21.15.32:0/1303564448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66cd0 0x557beeb75330 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f45a4007dd0 tx=0x7f45a4012520 comp rx=0 tx=0).stop 2023-09-30T14:52:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.423+0000 7f45c0330b80 1 -- 172.21.15.32:0/1303564448 shutdown_connections 2023-09-30T14:52:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.423+0000 7f45c0330b80 1 --2- 172.21.15.32:0/1303564448 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557beebcf630 0x557beebd1a50 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.423+0000 7f45c0330b80 1 --2- 172.21.15.32:0/1303564448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557beeb66cd0 0x557beeb75330 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.423+0000 7f45c0330b80 1 --2- 172.21.15.32:0/1303564448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557beeb65200 0x557beeb73de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.423+0000 7f45c0330b80 1 --2- 172.21.15.32:0/1303564448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beeb66310 0x557beeb74320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.423+0000 7f45957fa700 0 client.16716 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:52:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.423+0000 7f45c0330b80 1 -- 172.21.15.32:0/1303564448 >> 172.21.15.32:0/1303564448 conn(0x557beead9200 msgr2=0x557beeb76be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.423+0000 7f45c0330b80 1 -- 172.21.15.32:0/1303564448 shutdown_connections 2023-09-30T14:52:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.423+0000 7f45c0330b80 1 -- 172.21.15.32:0/1303564448 wait complete. 2023-09-30T14:52:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.423+0000 7f45c0330b80 20 client.16716 trim_cache size 0 max 16384 2023-09-30T14:52:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:52:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:52:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:52:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:52:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.423+0000 7f45b4148700 1 -- 172.21.15.32:0/2592489413 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f45980a1120 msgr2=0x7f45980a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.423+0000 7f45b4148700 1 --2- 172.21.15.32:0/2592489413 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f45980a1120 0x7f45980a35e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f45a8005e50 tx=0x7f45a8005de0 comp rx=0 tx=0).stop 2023-09-30T14:52:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.423+0000 7f45b4148700 1 -- 172.21.15.32:0/2592489413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac072cf0 msgr2=0x7f45ac1aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:52:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.423+0000 7f45b4148700 1 --2- 172.21.15.32:0/2592489413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac072cf0 0x7f45ac1aed20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f459c0039a0 tx=0x7f459c0042e0 comp rx=0 tx=0).stop 2023-09-30T14:52:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.423+0000 7f45b4148700 1 -- 172.21.15.32:0/2592489413 shutdown_connections 2023-09-30T14:52:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.423+0000 7f45b4148700 1 --2- 172.21.15.32:0/2592489413 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f45980a1120 0x7f45980a35e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.424+0000 7f45b4148700 1 --2- 172.21.15.32:0/2592489413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45ac1b3c00 0x7f45ac1b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.424+0000 7f45b4148700 1 --2- 172.21.15.32:0/2592489413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45ac072cf0 0x7f45ac1aed20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.424+0000 7f45b4148700 1 --2- 172.21.15.32:0/2592489413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45ac0785a0 0x7f45ac1af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:52:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.424+0000 7f45b4148700 1 -- 172.21.15.32:0/2592489413 >> 172.21.15.32:0/2592489413 conn(0x7f45ac06dc80 msgr2=0x7f45ac079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:52:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.424+0000 7f45b4148700 1 -- 172.21.15.32:0/2592489413 shutdown_connections 2023-09-30T14:52:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:52:51.424+0000 7f45b4148700 1 -- 172.21.15.32:0/2592489413 wait complete. 2023-09-30T14:52:51.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:51 smithi156 ceph-mon[103579]: pgmap v724: 97 pgs: 97 active+clean; 657 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.8 MiB/s wr, 795 op/s 2023-09-30T14:52:51.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:51 smithi148 ceph-mon[102355]: pgmap v724: 97 pgs: 97 active+clean; 657 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.8 MiB/s wr, 795 op/s 2023-09-30T14:52:51.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:51 smithi032 ceph-mon[98592]: pgmap v724: 97 pgs: 97 active+clean; 657 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.8 MiB/s wr, 795 op/s 2023-09-30T14:52:52.002 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52105 34.98 MB/sec warmup 85 sec latency 341.000 ms 2023-09-30T14:52:53.003 INFO:tasks.workunit.client.0.smithi032.stdout: 10 52989 35.30 MB/sec warmup 86 sec latency 61.617 ms 2023-09-30T14:52:53.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:53 smithi156 ceph-mon[103579]: pgmap v725: 97 pgs: 97 active+clean; 663 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 866 op/s 2023-09-30T14:52:53.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:53 smithi148 ceph-mon[102355]: pgmap v725: 97 pgs: 97 active+clean; 663 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 866 op/s 2023-09-30T14:52:53.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:53 smithi032 ceph-mon[98592]: pgmap v725: 97 pgs: 97 active+clean; 663 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 866 op/s 2023-09-30T14:52:54.003 INFO:tasks.workunit.client.0.smithi032.stdout: 10 53652 35.29 MB/sec warmup 87 sec latency 73.884 ms 2023-09-30T14:52:55.003 INFO:tasks.workunit.client.0.smithi032.stdout: 10 54282 35.21 MB/sec warmup 88 sec latency 50.728 ms 2023-09-30T14:52:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:55 smithi156 ceph-mon[103579]: pgmap v726: 97 pgs: 97 active+clean; 664 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.6 MiB/s wr, 749 op/s 2023-09-30T14:52:55.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:55 smithi148 ceph-mon[102355]: pgmap v726: 97 pgs: 97 active+clean; 664 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.6 MiB/s wr, 749 op/s 2023-09-30T14:52:55.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:55 smithi032 ceph-mon[98592]: pgmap v726: 97 pgs: 97 active+clean; 664 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.6 MiB/s wr, 749 op/s 2023-09-30T14:52:56.004 INFO:tasks.workunit.client.0.smithi032.stdout: 10 54956 35.18 MB/sec warmup 89 sec latency 105.216 ms 2023-09-30T14:52:57.004 INFO:tasks.workunit.client.0.smithi032.stdout: 10 55544 35.24 MB/sec warmup 90 sec latency 252.827 ms 2023-09-30T14:52:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:57 smithi156 ceph-mon[103579]: pgmap v727: 97 pgs: 97 active+clean; 665 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 894 op/s 2023-09-30T14:52:57.862 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:57 smithi148 ceph-mon[102355]: pgmap v727: 97 pgs: 97 active+clean; 665 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 894 op/s 2023-09-30T14:52:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:57 smithi032 ceph-mon[98592]: pgmap v727: 97 pgs: 97 active+clean; 665 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 894 op/s 2023-09-30T14:52:58.004 INFO:tasks.workunit.client.0.smithi032.stdout: 10 56327 35.15 MB/sec warmup 91 sec latency 93.044 ms 2023-09-30T14:52:59.004 INFO:tasks.workunit.client.0.smithi032.stdout: 10 57381 35.48 MB/sec warmup 92 sec latency 46.651 ms 2023-09-30T14:52:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:52:59 smithi156 ceph-mon[103579]: pgmap v728: 97 pgs: 97 active+clean; 654 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 802 op/s 2023-09-30T14:52:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:52:59 smithi148 ceph-mon[102355]: pgmap v728: 97 pgs: 97 active+clean; 654 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 802 op/s 2023-09-30T14:52:59.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:52:59 smithi032 ceph-mon[98592]: pgmap v728: 97 pgs: 97 active+clean; 654 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 802 op/s 2023-09-30T14:53:00.004 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58125 35.66 MB/sec warmup 93 sec latency 64.969 ms 2023-09-30T14:53:01.004 INFO:tasks.workunit.client.0.smithi032.stdout: 10 58699 35.52 MB/sec warmup 94 sec latency 62.919 ms 2023-09-30T14:53:01.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:01 smithi156 ceph-mon[103579]: pgmap v729: 97 pgs: 97 active+clean; 650 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 10 MiB/s wr, 775 op/s 2023-09-30T14:53:01.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:01 smithi148 ceph-mon[102355]: pgmap v729: 97 pgs: 97 active+clean; 650 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 10 MiB/s wr, 775 op/s 2023-09-30T14:53:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:01 smithi032 ceph-mon[98592]: pgmap v729: 97 pgs: 97 active+clean; 650 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 10 MiB/s wr, 775 op/s 2023-09-30T14:53:02.004 INFO:tasks.workunit.client.0.smithi032.stdout: 10 59305 35.45 MB/sec warmup 95 sec latency 238.032 ms 2023-09-30T14:53:03.004 INFO:tasks.workunit.client.0.smithi032.stdout: 10 60198 35.59 MB/sec warmup 96 sec latency 68.872 ms 2023-09-30T14:53:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:03 smithi156 ceph-mon[103579]: pgmap v730: 97 pgs: 97 active+clean; 652 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 858 op/s 2023-09-30T14:53:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:03 smithi148 ceph-mon[102355]: pgmap v730: 97 pgs: 97 active+clean; 652 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 858 op/s 2023-09-30T14:53:03.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:03 smithi032 ceph-mon[98592]: pgmap v730: 97 pgs: 97 active+clean; 652 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 858 op/s 2023-09-30T14:53:04.004 INFO:tasks.workunit.client.0.smithi032.stdout: 10 61186 35.78 MB/sec warmup 97 sec latency 49.813 ms 2023-09-30T14:53:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.554+0000 7f6ef64bb700 1 -- 172.21.15.32:0/1554271080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef0100f50 msgr2=0x7f6ef01013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.554+0000 7f6ef64bb700 1 --2- 172.21.15.32:0/1554271080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef0100f50 0x7f6ef01013b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80099c0 tx=0x7f6ed8009cd0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.554+0000 7f6ef64bb700 1 -- 172.21.15.32:0/1554271080 shutdown_connections 2023-09-30T14:53:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.554+0000 7f6ef64bb700 1 --2- 172.21.15.32:0/1554271080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ef01018f0 0x7f6ef01081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.554+0000 7f6ef64bb700 1 --2- 172.21.15.32:0/1554271080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef00ffd60 0x7f6ef0100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.554+0000 7f6ef64bb700 1 --2- 172.21.15.32:0/1554271080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef0100f50 0x7f6ef01013b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.554+0000 7f6ef64bb700 1 -- 172.21.15.32:0/1554271080 >> 172.21.15.32:0/1554271080 conn(0x7f6ef00fb2b0 msgr2=0x7f6ef00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.554+0000 7f6ef64bb700 1 -- 172.21.15.32:0/1554271080 shutdown_connections 2023-09-30T14:53:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.554+0000 7f6ef64bb700 1 -- 172.21.15.32:0/1554271080 wait complete. 2023-09-30T14:53:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.555+0000 7f6ef64bb700 1 Processor -- start 2023-09-30T14:53:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.555+0000 7f6ef64bb700 1 -- start start 2023-09-30T14:53:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.555+0000 7f6ef64bb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ef00ffd60 0x7f6ef019d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.555+0000 7f6ef64bb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0100f50 0x7f6ef019d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.555+0000 7f6ef64bb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef01018f0 0x7f6ef019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.555+0000 7f6ef64bb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef019ec50 con 0x7f6ef0100f50 2023-09-30T14:53:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.555+0000 7f6ef64bb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef019edc0 con 0x7f6ef01018f0 2023-09-30T14:53:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.555+0000 7f6ef64bb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef019ef80 con 0x7f6ef00ffd60 2023-09-30T14:53:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.555+0000 7f6ee77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0100f50 0x7f6ef019d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.555+0000 7f6eeffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ef00ffd60 0x7f6ef019d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.555+0000 7f6ee77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0100f50 0x7f6ef019d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38608/0 (socket says 172.21.15.32:38608) 2023-09-30T14:53:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.555+0000 7f6ee77fe700 1 -- 172.21.15.32:0/3967996659 learned_addr learned my addr 172.21.15.32:0/3967996659 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.555+0000 7f6ef4a58700 1 --2- 172.21.15.32:0/3967996659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef01018f0 0x7f6ef019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.556+0000 7f6ee77fe700 1 -- 172.21.15.32:0/3967996659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ef00ffd60 msgr2=0x7f6ef019d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.556+0000 7f6ee77fe700 1 --2- 172.21.15.32:0/3967996659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ef00ffd60 0x7f6ef019d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.556+0000 7f6ee77fe700 1 -- 172.21.15.32:0/3967996659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef01018f0 msgr2=0x7f6ef019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.556+0000 7f6ee77fe700 1 --2- 172.21.15.32:0/3967996659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef01018f0 0x7f6ef019e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.556+0000 7f6ee77fe700 1 -- 172.21.15.32:0/3967996659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed80095f0 con 0x7f6ef0100f50 2023-09-30T14:53:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.556+0000 7f6ee77fe700 1 --2- 172.21.15.32:0/3967996659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0100f50 0x7f6ef019d770 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009fd0 tx=0x7f6ed8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.556+0000 7f6eedffb700 1 -- 172.21.15.32:0/3967996659 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed801d070 con 0x7f6ef0100f50 2023-09-30T14:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.556+0000 7f6eedffb700 1 -- 172.21.15.32:0/3967996659 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ed8005030 con 0x7f6ef0100f50 2023-09-30T14:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.556+0000 7f6ef64bb700 1 -- 172.21.15.32:0/3967996659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef019f200 con 0x7f6ef0100f50 2023-09-30T14:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.556+0000 7f6eedffb700 1 -- 172.21.15.32:0/3967996659 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8004a90 con 0x7f6ef0100f50 2023-09-30T14:53:04.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.557+0000 7f6ef64bb700 1 -- 172.21.15.32:0/3967996659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef01a5a20 con 0x7f6ef0100f50 2023-09-30T14:53:04.561 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.560+0000 7f6eedffb700 1 -- 172.21.15.32:0/3967996659 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f6ed80051a0 con 0x7f6ef0100f50 2023-09-30T14:53:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.561+0000 7f6f026a3b80 1 Processor -- start 2023-09-30T14:53:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.561+0000 7f6eedffb700 1 --2- 172.21.15.32:0/3967996659 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6edc0a10f0 0x7f6edc0a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.561+0000 7f6f026a3b80 1 -- start start 2023-09-30T14:53:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.561+0000 7f6eedffb700 1 -- 172.21.15.32:0/3967996659 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6ed80cd3f0 con 0x7f6ef0100f50 2023-09-30T14:53:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.561+0000 7f6ef4a58700 1 --2- 172.21.15.32:0/3967996659 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6edc0a10f0 0x7f6edc0a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.561+0000 7f6f026a3b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b13140f200 0x55b13140f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.561+0000 7f6f026a3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410310 0x55b131410790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.561+0000 7f6f026a3b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b131410cd0 0x55b131417560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.561+0000 7f6f026a3b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131377400 con 0x55b13140f200 2023-09-30T14:53:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.561+0000 7f6f026a3b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131368bc0 con 0x55b131410cd0 2023-09-30T14:53:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.561+0000 7f6f026a3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1313664e0 con 0x55b131410310 2023-09-30T14:53:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.561+0000 7f6eeffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b13140f200 0x55b13140f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.561+0000 7f6eeffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b13140f200 0x55b13140f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38616/0 (socket says 172.21.15.32:38616) 2023-09-30T14:53:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.561+0000 7f6eeffff700 1 -- 172.21.15.32:0/472581301 learned_addr learned my addr 172.21.15.32:0/472581301 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.561+0000 7f6ef4a58700 1 --2- 172.21.15.32:0/472581301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b131410cd0 0x55b131417560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.562+0000 7f6ee77fe700 1 --2- 172.21.15.32:0/472581301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410310 0x55b131410790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.562+0000 7f6eeffff700 1 -- 172.21.15.32:0/472581301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410310 msgr2=0x55b131410790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.562+0000 7f6eeffff700 1 --2- 172.21.15.32:0/472581301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410310 0x55b131410790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.562+0000 7f6eeffff700 1 -- 172.21.15.32:0/472581301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b131410cd0 msgr2=0x55b131417560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.562+0000 7f6eeffff700 1 --2- 172.21.15.32:0/472581301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b131410cd0 0x55b131417560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.562+0000 7f6eeffff700 1 -- 172.21.15.32:0/472581301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1313c4870 con 0x55b13140f200 2023-09-30T14:53:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.562+0000 7f6ee77fe700 1 --2- 172.21.15.32:0/472581301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410310 0x55b131410790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:53:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.562+0000 7f6ef4a58700 1 --2- 172.21.15.32:0/3967996659 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6edc0a10f0 0x7f6edc0a35b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800f870 tx=0x7f6ee800fc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.562+0000 7f6eeffff700 1 --2- 172.21.15.32:0/472581301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b13140f200 0x55b13140f620 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009d30 tx=0x7f6ee0011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d7a2d20aa73e27 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.562+0000 7f6ee67fc700 1 -- 172.21.15.32:0/472581301 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0011da0 con 0x55b13140f200 2023-09-30T14:53:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.562+0000 7f6ee67fc700 1 -- 172.21.15.32:0/472581301 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee00044e0 con 0x55b13140f200 2023-09-30T14:53:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.562+0000 7f6ee67fc700 1 -- 172.21.15.32:0/472581301 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0019430 con 0x55b13140f200 2023-09-30T14:53:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.563+0000 7f6f026a3b80 1 -- 172.21.15.32:0/472581301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b13140f200 msgr2=0x55b13140f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.563+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/472581301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b13140f200 0x55b13140f620 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009d30 tx=0x7f6ee0011720 comp rx=0 tx=0).stop 2023-09-30T14:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.563+0000 7f6f026a3b80 1 -- 172.21.15.32:0/472581301 shutdown_connections 2023-09-30T14:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.563+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/472581301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b131410cd0 0x55b131417560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.563+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/472581301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b13140f200 0x55b13140f620 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.563+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/472581301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410310 0x55b131410790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.563+0000 7f6f026a3b80 1 -- 172.21.15.32:0/472581301 >> 172.21.15.32:0/472581301 conn(0x55b131383200 msgr2=0x55b131409ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.563+0000 7f6f026a3b80 1 -- 172.21.15.32:0/472581301 shutdown_connections 2023-09-30T14:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.563+0000 7f6f026a3b80 1 -- 172.21.15.32:0/472581301 wait complete. 2023-09-30T14:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.563+0000 7f6f026a3b80 1 -- start start 2023-09-30T14:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6f026a3b80 1 Processor -- start 2023-09-30T14:53:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6f026a3b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 0x55b131418e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6f026a3b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b1314193c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6f026a3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410cd0 0x55b131420400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6f026a3b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131361dd0 con 0x55b131410310 2023-09-30T14:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6f026a3b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1312fe6b0 con 0x55b13140f200 2023-09-30T14:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6f026a3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1312fe350 con 0x55b131410cd0 2023-09-30T14:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6f026a3b80 10 client.0 _ll_register_callbacks cb 0x55b13115b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6ee77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b1314193c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6ee77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b1314193c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38622/0 (socket says 172.21.15.32:38622) 2023-09-30T14:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6ee77fe700 1 -- 172.21.15.32:0/869741789 learned_addr learned my addr 172.21.15.32:0/869741789 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6eeffff700 1 --2- 172.21.15.32:0/869741789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 0x55b131418e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6ef4a58700 1 --2- 172.21.15.32:0/869741789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410cd0 0x55b131420400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6ee77fe700 1 -- 172.21.15.32:0/869741789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410cd0 msgr2=0x55b131420400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6ee77fe700 1 --2- 172.21.15.32:0/869741789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410cd0 0x55b131420400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6ee77fe700 1 -- 172.21.15.32:0/869741789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 msgr2=0x55b131418e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6ee77fe700 1 --2- 172.21.15.32:0/869741789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 0x55b131418e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.564+0000 7f6ee77fe700 1 -- 172.21.15.32:0/869741789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee0009960 con 0x55b131410310 2023-09-30T14:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.565+0000 7f6ee77fe700 1 --2- 172.21.15.32:0/869741789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b1314193c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80973c0 tx=0x7f6ed8005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35921c2d40a827e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.565+0000 7f6ecf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.565+0000 7f6ecf7fe700 1 -- 172.21.15.32:0/869741789 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80a2c50 con 0x55b131410310 2023-09-30T14:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.565+0000 7f6ecf7fe700 1 -- 172.21.15.32:0/869741789 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6ed809abb0 con 0x55b131410310 2023-09-30T14:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.565+0000 7f6f026a3b80 1 -- 172.21.15.32:0/869741789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b131402090 con 0x55b131410310 2023-09-30T14:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.565+0000 7f6ecf7fe700 1 -- 172.21.15.32:0/869741789 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ed80a2ea0 con 0x55b131410310 2023-09-30T14:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.565+0000 7f6f026a3b80 10 client.16728 fetch_fsmap learned FSMap version 47 2023-09-30T14:53:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.565+0000 7f6f026a3b80 1 -- 172.21.15.32:0/869741789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55b13118cd80 con 0x55b131410310 2023-09-30T14:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.565+0000 7f6ecf7fe700 1 -- 172.21.15.32:0/869741789 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ed80a6070 con 0x55b131410310 2023-09-30T14:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.565+0000 7f6ecf7fe700 1 -- 172.21.15.32:0/869741789 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 47) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6ed80a27c0 con 0x55b131410310 2023-09-30T14:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.565+0000 7f6f026a3b80 10 client.16728 fetch_fsmap finished waiting for FSMap version 47 2023-09-30T14:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.565+0000 7f6f026a3b80 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.565+0000 7f6f026a3b80 20 client.16728 populate_metadata read hostname 'smithi032' 2023-09-30T14:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.565+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/869741789 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b131424860 0x55b131426c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.565+0000 7f6f026a3b80 4 client.16728 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.565+0000 7f6f026a3b80 1 -- 172.21.15.32:0/869741789 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b13135adb0 con 0x55b131424860 2023-09-30T14:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.566+0000 7f6eeffff700 1 --2- 172.21.15.32:0/869741789 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b131424860 0x55b131426c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.566+0000 7f6ecf7fe700 10 client.16728 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.566+0000 7f6eeffff700 1 --2- 172.21.15.32:0/869741789 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b131424860 0x55b131426c80 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=932c77824c05bc6 server_cookie=fc7b2ae282902217 in_seq=0 out_seq=0 2023-09-30T14:53:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.569+0000 7f6ecf7fe700 1 -- 172.21.15.32:0/869741789 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b13135adb0 con 0x55b131424860 2023-09-30T14:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.569+0000 7f6ecf7fe700 10 client.16728 handle_command_reply: tid=0 2023-09-30T14:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.570+0000 7f6f026a3b80 1 client.16728 shutdown 2023-09-30T14:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.570+0000 7f6f026a3b80 1 -- 172.21.15.32:0/869741789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 msgr2=0x55b1314193c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.570+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/869741789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b1314193c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80973c0 tx=0x7f6ed8005760 comp rx=0 tx=0).stop 2023-09-30T14:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.570+0000 7f6f026a3b80 1 -- 172.21.15.32:0/869741789 shutdown_connections 2023-09-30T14:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.570+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/869741789 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b131424860 0x55b131426c80 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.570+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/869741789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410cd0 0x55b131420400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.570+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/869741789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 0x55b131418e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.570+0000 7f6ecf7fe700 0 client.16728 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.570+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/869741789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b1314193c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.570+0000 7f6f026a3b80 1 -- 172.21.15.32:0/869741789 >> 172.21.15.32:0/869741789 conn(0x55b131383200 msgr2=0x55b131409590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.570+0000 7f6f026a3b80 1 -- 172.21.15.32:0/869741789 shutdown_connections 2023-09-30T14:53:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.570+0000 7f6f026a3b80 1 -- 172.21.15.32:0/869741789 wait complete. 2023-09-30T14:53:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.570+0000 7f6f026a3b80 20 client.16728 trim_cache size 0 max 16384 2023-09-30T14:53:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.587+0000 7f6f026a3b80 1 Processor -- start 2023-09-30T14:53:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.587+0000 7f6f026a3b80 1 -- start start 2023-09-30T14:53:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.588+0000 7f6f026a3b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 0x55b1314106f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.588+0000 7f6f026a3b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b1314203c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.588+0000 7f6f026a3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410cd0 0x55b1314170d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.588+0000 7f6f026a3b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1311e28c0 con 0x55b131410310 2023-09-30T14:53:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.588+0000 7f6f026a3b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13140ac40 con 0x55b13140f200 2023-09-30T14:53:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.588+0000 7f6f026a3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b131410100 con 0x55b131410cd0 2023-09-30T14:53:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.588+0000 7f6ee77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b1314203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.588+0000 7f6ee77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b1314203c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38632/0 (socket says 172.21.15.32:38632) 2023-09-30T14:53:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.588+0000 7f6ee77fe700 1 -- 172.21.15.32:0/2127822074 learned_addr learned my addr 172.21.15.32:0/2127822074 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.588+0000 7f6ef4a58700 1 --2- 172.21.15.32:0/2127822074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410cd0 0x55b1314170d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.588+0000 7f6eeffff700 1 --2- 172.21.15.32:0/2127822074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 0x55b1314106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.588+0000 7f6ee77fe700 1 -- 172.21.15.32:0/2127822074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410cd0 msgr2=0x55b1314170d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.588+0000 7f6ee77fe700 1 --2- 172.21.15.32:0/2127822074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410cd0 0x55b1314170d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.588+0000 7f6ee77fe700 1 -- 172.21.15.32:0/2127822074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 msgr2=0x55b1314106f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.588+0000 7f6ee77fe700 1 --2- 172.21.15.32:0/2127822074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 0x55b1314106f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.588+0000 7f6ee77fe700 1 -- 172.21.15.32:0/2127822074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b131175c00 con 0x55b131410310 2023-09-30T14:53:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.589+0000 7f6ee77fe700 1 --2- 172.21.15.32:0/2127822074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b1314203c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80cb270 tx=0x7f6ed8096dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f771e786d0b679 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.589+0000 7f6ee6ffd700 1 -- 172.21.15.32:0/2127822074 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80a2ea0 con 0x55b131410310 2023-09-30T14:53:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.589+0000 7f6ee6ffd700 1 -- 172.21.15.32:0/2127822074 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ed80a6070 con 0x55b131410310 2023-09-30T14:53:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.589+0000 7f6ee6ffd700 1 -- 172.21.15.32:0/2127822074 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed809ae10 con 0x55b131410310 2023-09-30T14:53:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.590+0000 7f6f026a3b80 1 -- 172.21.15.32:0/2127822074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 msgr2=0x55b1314203c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.590+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/2127822074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b1314203c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80cb270 tx=0x7f6ed8096dc0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.590+0000 7f6f026a3b80 1 -- 172.21.15.32:0/2127822074 shutdown_connections 2023-09-30T14:53:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.590+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/2127822074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410cd0 0x55b1314170d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.590+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/2127822074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 0x55b1314106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.590+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/2127822074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b1314203c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.590+0000 7f6f026a3b80 1 -- 172.21.15.32:0/2127822074 >> 172.21.15.32:0/2127822074 conn(0x55b131383200 msgr2=0x55b13142b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.590+0000 7f6f026a3b80 1 -- 172.21.15.32:0/2127822074 shutdown_connections 2023-09-30T14:53:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.590+0000 7f6f026a3b80 1 -- 172.21.15.32:0/2127822074 wait complete. 2023-09-30T14:53:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.591+0000 7f6f026a3b80 1 -- start start 2023-09-30T14:53:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.591+0000 7f6f026a3b80 1 Processor -- start 2023-09-30T14:53:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.591+0000 7f6f026a3b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 0x55b13141de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.591+0000 7f6f026a3b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b13141e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.591+0000 7f6f026a3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410cd0 0x55b13141f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.591+0000 7f6f026a3b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13141f8d0 con 0x55b131410310 2023-09-30T14:53:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.591+0000 7f6f026a3b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13141fa40 con 0x55b13140f200 2023-09-30T14:53:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.591+0000 7f6f026a3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13141fc00 con 0x55b131410cd0 2023-09-30T14:53:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.591+0000 7f6f026a3b80 10 client.0 _ll_register_callbacks cb 0x7f6ed809a2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:53:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.592+0000 7f6eeffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 0x55b13141de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.592+0000 7f6eeffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 0x55b13141de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:50110/0 (socket says 172.21.15.32:50110) 2023-09-30T14:53:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.592+0000 7f6eeffff700 1 -- 172.21.15.32:0/568743653 learned_addr learned my addr 172.21.15.32:0/568743653 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.592+0000 7f6ee77fe700 1 --2- 172.21.15.32:0/568743653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b13141e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.592+0000 7f6eeffff700 1 -- 172.21.15.32:0/568743653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410cd0 msgr2=0x55b13141f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.592+0000 7f6eeffff700 1 --2- 172.21.15.32:0/568743653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410cd0 0x55b13141f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.592+0000 7f6eeffff700 1 -- 172.21.15.32:0/568743653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 msgr2=0x55b13141e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.592+0000 7f6eeffff700 1 --2- 172.21.15.32:0/568743653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b13141e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.592+0000 7f6eeffff700 1 -- 172.21.15.32:0/568743653 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b13118cd80 con 0x55b13140f200 2023-09-30T14:53:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.592+0000 7f6ee77fe700 1 --2- 172.21.15.32:0/568743653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b13141e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:53:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.592+0000 7f6eeffff700 1 --2- 172.21.15.32:0/568743653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 0x55b13141de40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00097c0 tx=0x7f6ee0005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=538a6bd7fb3410ba server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.592+0000 7f6ecf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:53:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.593+0000 7f6ecf7fe700 1 -- 172.21.15.32:0/568743653 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0029050 con 0x55b13140f200 2023-09-30T14:53:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.593+0000 7f6f026a3b80 1 -- 172.21.15.32:0/568743653 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ee00044e0 con 0x55b13140f200 2023-09-30T14:53:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.593+0000 7f6ecf7fe700 1 -- 172.21.15.32:0/568743653 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6ee0005040 con 0x55b13140f200 2023-09-30T14:53:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.594+0000 7f6ecf7fe700 1 -- 172.21.15.32:0/568743653 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ee0005200 con 0x55b13140f200 2023-09-30T14:53:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.594+0000 7f6f026a3b80 10 client.26387 fetch_fsmap learned FSMap version 47 2023-09-30T14:53:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.594+0000 7f6f026a3b80 1 -- 172.21.15.32:0/568743653 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55b1311b31f0 con 0x55b13140f200 2023-09-30T14:53:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.594+0000 7f6ecf7fe700 1 -- 172.21.15.32:0/568743653 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee00044e0 con 0x55b13140f200 2023-09-30T14:53:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.595+0000 7f6ecf7fe700 1 -- 172.21.15.32:0/568743653 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 47) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6ee003c070 con 0x55b13140f200 2023-09-30T14:53:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.595+0000 7f6f026a3b80 10 client.26387 fetch_fsmap finished waiting for FSMap version 47 2023-09-30T14:53:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.595+0000 7f6f026a3b80 10 client.26387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:53:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.595+0000 7f6f026a3b80 20 client.26387 populate_metadata read hostname 'smithi032' 2023-09-30T14:53:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.595+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/568743653 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b131479400 0x55b13147b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.595+0000 7f6f026a3b80 4 client.26387 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:53:04.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.595+0000 7f6f026a3b80 1 -- 172.21.15.32:0/568743653 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b13135aad0 con 0x55b131479400 2023-09-30T14:53:04.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.595+0000 7f6ee77fe700 1 --2- 172.21.15.32:0/568743653 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b131479400 0x55b13147b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:04.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.595+0000 7f6ecf7fe700 10 client.26387 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:04.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.595+0000 7f6ee77fe700 1 --2- 172.21.15.32:0/568743653 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b131479400 0x55b13147b840 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf15bc315b9ef54 server_cookie=6fd9ebdcd8b1ffdc in_seq=0 out_seq=0 2023-09-30T14:53:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.912+0000 7f6ecf7fe700 1 -- 172.21.15.32:0/568743653 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b13135aad0 con 0x55b131479400 2023-09-30T14:53:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.912+0000 7f6ecf7fe700 10 client.26387 handle_command_reply: tid=0 2023-09-30T14:53:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.912+0000 7f6f026a3b80 1 client.26387 shutdown 2023-09-30T14:53:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.912+0000 7f6f026a3b80 1 -- 172.21.15.32:0/568743653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 msgr2=0x55b13141de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.912+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/568743653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 0x55b13141de40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00097c0 tx=0x7f6ee0005d90 comp rx=0 tx=0).stop 2023-09-30T14:53:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.913+0000 7f6f026a3b80 1 -- 172.21.15.32:0/568743653 shutdown_connections 2023-09-30T14:53:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.913+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/568743653 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b131479400 0x55b13147b840 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.913+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/568743653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b131410cd0 0x55b13141f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.913+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/568743653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b13140f200 0x55b13141de40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.913+0000 7f6f026a3b80 1 --2- 172.21.15.32:0/568743653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b131410310 0x55b13141e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.913+0000 7f6f026a3b80 1 -- 172.21.15.32:0/568743653 >> 172.21.15.32:0/568743653 conn(0x55b131383200 msgr2=0x55b1314170d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.913+0000 7f6ecf7fe700 0 client.26387 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.913+0000 7f6f026a3b80 1 -- 172.21.15.32:0/568743653 shutdown_connections 2023-09-30T14:53:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.913+0000 7f6f026a3b80 1 -- 172.21.15.32:0/568743653 wait complete. 2023-09-30T14:53:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.913+0000 7f6f026a3b80 20 client.26387 trim_cache size 0 max 16384 2023-09-30T14:53:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:53:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:53:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:53:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:53:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.913+0000 7f6ef64bb700 1 -- 172.21.15.32:0/3967996659 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6edc0a10f0 msgr2=0x7f6edc0a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.913+0000 7f6ef64bb700 1 --2- 172.21.15.32:0/3967996659 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6edc0a10f0 0x7f6edc0a35b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800f870 tx=0x7f6ee800fc30 comp rx=0 tx=0).stop 2023-09-30T14:53:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.913+0000 7f6ef64bb700 1 -- 172.21.15.32:0/3967996659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0100f50 msgr2=0x7f6ef019d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.913+0000 7f6ef64bb700 1 --2- 172.21.15.32:0/3967996659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0100f50 0x7f6ef019d770 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009fd0 tx=0x7f6ed8009580 comp rx=0 tx=0).stop 2023-09-30T14:53:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.916+0000 7f6ef64bb700 1 -- 172.21.15.32:0/3967996659 shutdown_connections 2023-09-30T14:53:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.916+0000 7f6ef64bb700 1 --2- 172.21.15.32:0/3967996659 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6edc0a10f0 0x7f6edc0a35b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.916+0000 7f6ef64bb700 1 --2- 172.21.15.32:0/3967996659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef01018f0 0x7f6ef019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.916+0000 7f6ef64bb700 1 --2- 172.21.15.32:0/3967996659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ef00ffd60 0x7f6ef019d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.916+0000 7f6ef64bb700 1 --2- 172.21.15.32:0/3967996659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef0100f50 0x7f6ef019d770 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.916+0000 7f6ef64bb700 1 -- 172.21.15.32:0/3967996659 >> 172.21.15.32:0/3967996659 conn(0x7f6ef00fb2b0 msgr2=0x7f6ef00fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.916+0000 7f6ef64bb700 1 -- 172.21.15.32:0/3967996659 shutdown_connections 2023-09-30T14:53:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:04.916+0000 7f6ef64bb700 1 -- 172.21.15.32:0/3967996659 wait complete. 2023-09-30T14:53:05.004 INFO:tasks.workunit.client.0.smithi032.stdout: 10 61973 35.96 MB/sec warmup 98 sec latency 186.560 ms 2023-09-30T14:53:05.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:05 smithi156 ceph-mon[103579]: pgmap v731: 97 pgs: 97 active+clean; 645 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 802 op/s 2023-09-30T14:53:05.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:05 smithi148 ceph-mon[102355]: pgmap v731: 97 pgs: 97 active+clean; 645 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 802 op/s 2023-09-30T14:53:05.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:05 smithi032 ceph-mon[98592]: pgmap v731: 97 pgs: 97 active+clean; 645 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 802 op/s 2023-09-30T14:53:06.004 INFO:tasks.workunit.client.0.smithi032.stdout: 10 62666 35.87 MB/sec warmup 99 sec latency 48.124 ms 2023-09-30T14:53:07.004 INFO:tasks.workunit.client.0.smithi032.stdout: 10 63257 35.79 MB/sec warmup 100 sec latency 234.213 ms 2023-09-30T14:53:07.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:07 smithi156 ceph-mon[103579]: pgmap v732: 97 pgs: 97 active+clean; 642 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 11 MiB/s wr, 829 op/s 2023-09-30T14:53:07.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:07 smithi148 ceph-mon[102355]: pgmap v732: 97 pgs: 97 active+clean; 642 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 11 MiB/s wr, 829 op/s 2023-09-30T14:53:07.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:07 smithi032 ceph-mon[98592]: pgmap v732: 97 pgs: 97 active+clean; 642 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 11 MiB/s wr, 829 op/s 2023-09-30T14:53:08.004 INFO:tasks.workunit.client.0.smithi032.stdout: 10 63862 35.82 MB/sec warmup 101 sec latency 103.379 ms 2023-09-30T14:53:09.004 INFO:tasks.workunit.client.0.smithi032.stdout: 10 64736 35.83 MB/sec warmup 102 sec latency 54.458 ms 2023-09-30T14:53:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:09 smithi156 ceph-mon[103579]: pgmap v733: 97 pgs: 97 active+clean; 648 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 803 op/s 2023-09-30T14:53:09.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:09 smithi148 ceph-mon[102355]: pgmap v733: 97 pgs: 97 active+clean; 648 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 803 op/s 2023-09-30T14:53:09.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:09 smithi032 ceph-mon[98592]: pgmap v733: 97 pgs: 97 active+clean; 648 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 803 op/s 2023-09-30T14:53:10.004 INFO:tasks.workunit.client.0.smithi032.stdout: 10 65766 36.08 MB/sec warmup 103 sec latency 38.512 ms 2023-09-30T14:53:11.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 66549 36.25 MB/sec warmup 104 sec latency 41.653 ms 2023-09-30T14:53:11.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:11 smithi156 ceph-mon[103579]: pgmap v734: 97 pgs: 97 active+clean; 653 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 10 MiB/s wr, 855 op/s 2023-09-30T14:53:11.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:11 smithi148 ceph-mon[102355]: pgmap v734: 97 pgs: 97 active+clean; 653 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 10 MiB/s wr, 855 op/s 2023-09-30T14:53:11.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:11 smithi032 ceph-mon[98592]: pgmap v734: 97 pgs: 97 active+clean; 653 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 10 MiB/s wr, 855 op/s 2023-09-30T14:53:12.004 INFO:tasks.workunit.client.0.smithi032.stdout: 10 66890 36.03 MB/sec warmup 105 sec latency 313.207 ms 2023-09-30T14:53:13.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 67514 35.92 MB/sec warmup 106 sec latency 77.307 ms 2023-09-30T14:53:13.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:13 smithi156 ceph-mon[103579]: pgmap v735: 97 pgs: 97 active+clean; 655 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 888 op/s 2023-09-30T14:53:13.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:13 smithi148 ceph-mon[102355]: pgmap v735: 97 pgs: 97 active+clean; 655 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 888 op/s 2023-09-30T14:53:13.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:13 smithi032 ceph-mon[98592]: pgmap v735: 97 pgs: 97 active+clean; 655 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 888 op/s 2023-09-30T14:53:14.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 68371 36.07 MB/sec warmup 107 sec latency 81.785 ms 2023-09-30T14:53:15.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 69346 36.16 MB/sec warmup 108 sec latency 36.323 ms 2023-09-30T14:53:15.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:15 smithi156 ceph-mon[103579]: pgmap v736: 97 pgs: 97 active+clean; 659 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 818 op/s 2023-09-30T14:53:15.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:15 smithi148 ceph-mon[102355]: pgmap v736: 97 pgs: 97 active+clean; 659 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 818 op/s 2023-09-30T14:53:15.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:15 smithi032 ceph-mon[98592]: pgmap v736: 97 pgs: 97 active+clean; 659 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 818 op/s 2023-09-30T14:53:16.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 70294 36.40 MB/sec warmup 109 sec latency 71.949 ms 2023-09-30T14:53:17.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 70756 36.31 MB/sec warmup 110 sec latency 232.408 ms 2023-09-30T14:53:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:17 smithi156 ceph-mon[103579]: pgmap v737: 97 pgs: 97 active+clean; 667 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 895 op/s 2023-09-30T14:53:17.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:17 smithi148 ceph-mon[102355]: pgmap v737: 97 pgs: 97 active+clean; 667 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 895 op/s 2023-09-30T14:53:17.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:17 smithi032 ceph-mon[98592]: pgmap v737: 97 pgs: 97 active+clean; 667 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 895 op/s 2023-09-30T14:53:18.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 71342 36.26 MB/sec warmup 111 sec latency 56.780 ms 2023-09-30T14:53:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.036+0000 7f120364b700 1 -- 172.21.15.32:0/418519032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11fc0ffd60 msgr2=0x7f11fc100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.036+0000 7f120364b700 1 --2- 172.21.15.32:0/418519032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11fc0ffd60 0x7f11fc100180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f11f800d960 tx=0x7f11f800dc70 comp rx=0 tx=0).stop 2023-09-30T14:53:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.036+0000 7f120364b700 1 -- 172.21.15.32:0/418519032 shutdown_connections 2023-09-30T14:53:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.036+0000 7f120364b700 1 --2- 172.21.15.32:0/418519032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11fc1018f0 0x7f11fc1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.036+0000 7f120364b700 1 --2- 172.21.15.32:0/418519032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11fc0ffd60 0x7f11fc100180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.036+0000 7f120364b700 1 --2- 172.21.15.32:0/418519032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11fc100f50 0x7f11fc1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.036+0000 7f120364b700 1 -- 172.21.15.32:0/418519032 >> 172.21.15.32:0/418519032 conn(0x7f11fc0fb2b0 msgr2=0x7f11fc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.036+0000 7f120364b700 1 -- 172.21.15.32:0/418519032 shutdown_connections 2023-09-30T14:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.036+0000 7f120364b700 1 -- 172.21.15.32:0/418519032 wait complete. 2023-09-30T14:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.036+0000 7f120364b700 1 Processor -- start 2023-09-30T14:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.036+0000 7f120364b700 1 -- start start 2023-09-30T14:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f120364b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11fc0ffd60 0x7f11fc194a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f120364b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11fc100f50 0x7f11fc194fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f120364b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11fc1018f0 0x7f11fc195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f120364b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc196490 con 0x7f11fc1018f0 2023-09-30T14:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f120364b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc196600 con 0x7f11fc100f50 2023-09-30T14:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f120364b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc1967c0 con 0x7f11fc0ffd60 2023-09-30T14:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f12013e7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11fc0ffd60 0x7f11fc194a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f1201be8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11fc1018f0 0x7f11fc195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f1200be6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11fc100f50 0x7f11fc194fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f12013e7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11fc0ffd60 0x7f11fc194a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53030/0 (socket says 172.21.15.32:53030) 2023-09-30T14:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f12013e7700 1 -- 172.21.15.32:0/1042563568 learned_addr learned my addr 172.21.15.32:0/1042563568 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f12013e7700 1 -- 172.21.15.32:0/1042563568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11fc100f50 msgr2=0x7f11fc194fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f12013e7700 1 --2- 172.21.15.32:0/1042563568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11fc100f50 0x7f11fc194fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f12013e7700 1 -- 172.21.15.32:0/1042563568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11fc1018f0 msgr2=0x7f11fc195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f12013e7700 1 --2- 172.21.15.32:0/1042563568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11fc1018f0 0x7f11fc195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f12013e7700 1 -- 172.21.15.32:0/1042563568 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec00f750 con 0x7f11fc0ffd60 2023-09-30T14:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f1201be8700 1 --2- 172.21.15.32:0/1042563568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11fc1018f0 0x7f11fc195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f12013e7700 1 --2- 172.21.15.32:0/1042563568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11fc0ffd60 0x7f11fc194a70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009230 tx=0x7f11f800d280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.037+0000 7f11f27fc700 1 -- 172.21.15.32:0/1042563568 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f801b070 con 0x7f11fc0ffd60 2023-09-30T14:53:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.038+0000 7f11f27fc700 1 -- 172.21.15.32:0/1042563568 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11f80096c0 con 0x7f11fc0ffd60 2023-09-30T14:53:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.038+0000 7f120364b700 1 -- 172.21.15.32:0/1042563568 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f800d590 con 0x7f11fc0ffd60 2023-09-30T14:53:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.038+0000 7f11f27fc700 1 -- 172.21.15.32:0/1042563568 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f8003970 con 0x7f11fc0ffd60 2023-09-30T14:53:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.038+0000 7f120364b700 1 -- 172.21.15.32:0/1042563568 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc19d080 con 0x7f11fc0ffd60 2023-09-30T14:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.040+0000 7f11f27fc700 1 -- 172.21.15.32:0/1042563568 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f11f8009af0 con 0x7f11fc0ffd60 2023-09-30T14:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.041+0000 7f11f27fc700 1 --2- 172.21.15.32:0/1042563568 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f11e80a1020 0x7f11e80a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.041+0000 7f11f27fc700 1 -- 172.21.15.32:0/1042563568 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f11f80ccc50 con 0x7f11fc0ffd60 2023-09-30T14:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.042+0000 7f1200be6700 1 --2- 172.21.15.32:0/1042563568 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f11e80a1020 0x7f11e80a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.042+0000 7f120f833b80 1 Processor -- start 2023-09-30T14:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.042+0000 7f120f833b80 1 -- start start 2023-09-30T14:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.042+0000 7f120f833b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd4a10 0x563e3cbd4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.042+0000 7f120f833b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd5b20 0x563e3cbd5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.042+0000 7f120f833b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbdcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.042+0000 7f120f833b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e3cb34400 con 0x563e3cbd64e0 2023-09-30T14:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.042+0000 7f120f833b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e3cb25bc0 con 0x563e3cbd4a10 2023-09-30T14:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.042+0000 7f120f833b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e3cb234e0 con 0x563e3cbd5b20 2023-09-30T14:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.042+0000 7f1200be6700 1 --2- 172.21.15.32:0/1042563568 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f11e80a1020 0x7f11e80a34e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f11ec005fd0 tx=0x7f11ec011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.043+0000 7f1201be8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbdcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.043+0000 7f1201be8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbdcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38660/0 (socket says 172.21.15.32:38660) 2023-09-30T14:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.043+0000 7f1201be8700 1 -- 172.21.15.32:0/4096002356 learned_addr learned my addr 172.21.15.32:0/4096002356 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.043+0000 7f12013e7700 1 --2- 172.21.15.32:0/4096002356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd4a10 0x563e3cbd4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.043+0000 7f1200be6700 1 --2- 172.21.15.32:0/4096002356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd5b20 0x563e3cbd5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.043+0000 7f1201be8700 1 -- 172.21.15.32:0/4096002356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd5b20 msgr2=0x563e3cbd5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.043+0000 7f1201be8700 1 --2- 172.21.15.32:0/4096002356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd5b20 0x563e3cbd5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.043+0000 7f1201be8700 1 -- 172.21.15.32:0/4096002356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd4a10 msgr2=0x563e3cbd4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.043+0000 7f1201be8700 1 --2- 172.21.15.32:0/4096002356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd4a10 0x563e3cbd4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.043+0000 7f1201be8700 1 -- 172.21.15.32:0/4096002356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e3cb81870 con 0x563e3cbd64e0 2023-09-30T14:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.043+0000 7f1201be8700 1 --2- 172.21.15.32:0/4096002356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbdcd70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f11f4003eb0 tx=0x7f11f4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9364ca8a2ec98c9f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.043+0000 7f11e77fe700 1 -- 172.21.15.32:0/4096002356 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f40092d0 con 0x563e3cbd64e0 2023-09-30T14:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.043+0000 7f11e77fe700 1 -- 172.21.15.32:0/4096002356 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11f400f040 con 0x563e3cbd64e0 2023-09-30T14:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.043+0000 7f11e77fe700 1 -- 172.21.15.32:0/4096002356 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f400e730 con 0x563e3cbd64e0 2023-09-30T14:53:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.044+0000 7f120f833b80 1 -- 172.21.15.32:0/4096002356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 msgr2=0x563e3cbdcd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.044+0000 7f120f833b80 1 --2- 172.21.15.32:0/4096002356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbdcd70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f11f4003eb0 tx=0x7f11f4009920 comp rx=0 tx=0).stop 2023-09-30T14:53:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.044+0000 7f120f833b80 1 -- 172.21.15.32:0/4096002356 shutdown_connections 2023-09-30T14:53:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.044+0000 7f120f833b80 1 --2- 172.21.15.32:0/4096002356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbdcd70 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.044+0000 7f120f833b80 1 --2- 172.21.15.32:0/4096002356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd4a10 0x563e3cbd4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.044+0000 7f120f833b80 1 --2- 172.21.15.32:0/4096002356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd5b20 0x563e3cbd5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.044+0000 7f120f833b80 1 -- 172.21.15.32:0/4096002356 >> 172.21.15.32:0/4096002356 conn(0x563e3cb40200 msgr2=0x563e3cbc6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.044+0000 7f120f833b80 1 -- 172.21.15.32:0/4096002356 shutdown_connections 2023-09-30T14:53:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.044+0000 7f120f833b80 1 -- 172.21.15.32:0/4096002356 wait complete. 2023-09-30T14:53:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.045+0000 7f120f833b80 1 -- start start 2023-09-30T14:53:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.045+0000 7f120f833b80 1 Processor -- start 2023-09-30T14:53:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.045+0000 7f120f833b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd4a10 0x563e3cbde600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.045+0000 7f120f833b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd5b20 0x563e3cbdeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.045+0000 7f120f833b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbe5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.045+0000 7f120f833b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e3cb1edd0 con 0x563e3cbd64e0 2023-09-30T14:53:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.045+0000 7f120f833b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e3cabb6b0 con 0x563e3cbd5b20 2023-09-30T14:53:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.045+0000 7f120f833b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e3cabb350 con 0x563e3cbd4a10 2023-09-30T14:53:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.045+0000 7f120f833b80 10 client.0 _ll_register_callbacks cb 0x563e3c9187a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:53:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.046+0000 7f12013e7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd4a10 0x563e3cbde600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.046+0000 7f12013e7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd4a10 0x563e3cbde600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53044/0 (socket says 172.21.15.32:53044) 2023-09-30T14:53:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.046+0000 7f12013e7700 1 -- 172.21.15.32:0/1525823921 learned_addr learned my addr 172.21.15.32:0/1525823921 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.046+0000 7f1201be8700 1 --2- 172.21.15.32:0/1525823921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbe5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.046+0000 7f1200be6700 1 --2- 172.21.15.32:0/1525823921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd5b20 0x563e3cbdeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.046+0000 7f12013e7700 1 -- 172.21.15.32:0/1525823921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd5b20 msgr2=0x563e3cbdeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.046+0000 7f12013e7700 1 --2- 172.21.15.32:0/1525823921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd5b20 0x563e3cbdeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.046+0000 7f12013e7700 1 -- 172.21.15.32:0/1525823921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 msgr2=0x563e3cbe5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.046+0000 7f12013e7700 1 --2- 172.21.15.32:0/1525823921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbe5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.046+0000 7f12013e7700 1 -- 172.21.15.32:0/1525823921 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f4003b90 con 0x563e3cbd4a10 2023-09-30T14:53:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.046+0000 7f1201be8700 1 --2- 172.21.15.32:0/1525823921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbe5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:53:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.046+0000 7f12013e7700 1 --2- 172.21.15.32:0/1525823921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd4a10 0x563e3cbde600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f11f80966b0 tx=0x7f11f809a930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41ce313008c711f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.046+0000 7f11e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:53:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.046+0000 7f11e4ff9700 1 -- 172.21.15.32:0/1525823921 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f809e300 con 0x563e3cbd4a10 2023-09-30T14:53:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.046+0000 7f11e4ff9700 1 -- 172.21.15.32:0/1525823921 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f11f80a24b0 con 0x563e3cbd4a10 2023-09-30T14:53:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.046+0000 7f120f833b80 1 -- 172.21.15.32:0/1525823921 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e3cbbf090 con 0x563e3cbd4a10 2023-09-30T14:53:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.047+0000 7f11e4ff9700 1 -- 172.21.15.32:0/1525823921 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11f80b8170 con 0x563e3cbd4a10 2023-09-30T14:53:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.047+0000 7f120f833b80 10 client.26392 fetch_fsmap learned FSMap version 47 2023-09-30T14:53:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.047+0000 7f120f833b80 1 -- 172.21.15.32:0/1525823921 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x563e3c949d80 con 0x563e3cbd4a10 2023-09-30T14:53:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.047+0000 7f11e4ff9700 1 -- 172.21.15.32:0/1525823921 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11f807b070 con 0x563e3cbd4a10 2023-09-30T14:53:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.047+0000 7f11e4ff9700 1 -- 172.21.15.32:0/1525823921 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 47) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f11f80a20b0 con 0x563e3cbd4a10 2023-09-30T14:53:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.047+0000 7f120f833b80 10 client.26392 fetch_fsmap finished waiting for FSMap version 47 2023-09-30T14:53:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.047+0000 7f120f833b80 10 client.26392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:53:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.047+0000 7f120f833b80 20 client.26392 populate_metadata read hostname 'smithi032' 2023-09-30T14:53:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.047+0000 7f120f833b80 1 --2- 172.21.15.32:0/1525823921 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563e3cbea040 0x563e3cbec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.047+0000 7f120f833b80 4 client.26392 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:53:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.047+0000 7f120f833b80 1 -- 172.21.15.32:0/1525823921 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e3cb17db0 con 0x563e3cbea040 2023-09-30T14:53:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.047+0000 7f1200be6700 1 --2- 172.21.15.32:0/1525823921 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563e3cbea040 0x563e3cbec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.048+0000 7f1200be6700 1 --2- 172.21.15.32:0/1525823921 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563e3cbea040 0x563e3cbec430 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=761df56f7942c222 server_cookie=bb7b8befea5510e0 in_seq=0 out_seq=0 2023-09-30T14:53:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.048+0000 7f11e4ff9700 10 client.26392 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.050+0000 7f11e4ff9700 1 -- 172.21.15.32:0/1525823921 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563e3cb17db0 con 0x563e3cbea040 2023-09-30T14:53:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.050+0000 7f11e4ff9700 10 client.26392 handle_command_reply: tid=0 2023-09-30T14:53:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.050+0000 7f120f833b80 1 client.26392 shutdown 2023-09-30T14:53:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.050+0000 7f120f833b80 1 -- 172.21.15.32:0/1525823921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd4a10 msgr2=0x563e3cbde600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.050+0000 7f120f833b80 1 --2- 172.21.15.32:0/1525823921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd4a10 0x563e3cbde600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f11f80966b0 tx=0x7f11f809a930 comp rx=0 tx=0).stop 2023-09-30T14:53:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.050+0000 7f120f833b80 1 -- 172.21.15.32:0/1525823921 shutdown_connections 2023-09-30T14:53:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.050+0000 7f120f833b80 1 --2- 172.21.15.32:0/1525823921 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563e3cbea040 0x563e3cbec430 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.050+0000 7f120f833b80 1 --2- 172.21.15.32:0/1525823921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbe5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.050+0000 7f120f833b80 1 --2- 172.21.15.32:0/1525823921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd4a10 0x563e3cbde600 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.050+0000 7f120f833b80 1 --2- 172.21.15.32:0/1525823921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd5b20 0x563e3cbdeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.050+0000 7f11e4ff9700 0 client.26392 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.050+0000 7f120f833b80 1 -- 172.21.15.32:0/1525823921 >> 172.21.15.32:0/1525823921 conn(0x563e3cb40200 msgr2=0x563e3cbc6590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.050+0000 7f120f833b80 1 -- 172.21.15.32:0/1525823921 shutdown_connections 2023-09-30T14:53:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.050+0000 7f120f833b80 1 -- 172.21.15.32:0/1525823921 wait complete. 2023-09-30T14:53:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.050+0000 7f120f833b80 20 client.26392 trim_cache size 0 max 16384 2023-09-30T14:53:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f120f833b80 1 Processor -- start 2023-09-30T14:53:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f120f833b80 1 -- start start 2023-09-30T14:53:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f120f833b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd4a10 0x563e3cbd5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f120f833b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd5b20 0x563e3cbe5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f120f833b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbd9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f120f833b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e3c99f8c0 con 0x563e3cbd64e0 2023-09-30T14:53:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f120f833b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e3cb17de0 con 0x563e3cbd4a10 2023-09-30T14:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f120f833b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e3cbd5290 con 0x563e3cbd5b20 2023-09-30T14:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f1201be8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbd9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f1201be8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbd9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38674/0 (socket says 172.21.15.32:38674) 2023-09-30T14:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f1201be8700 1 -- 172.21.15.32:0/2375147183 learned_addr learned my addr 172.21.15.32:0/2375147183 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f1200be6700 1 --2- 172.21.15.32:0/2375147183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd5b20 0x563e3cbe5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f12013e7700 1 --2- 172.21.15.32:0/2375147183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd4a10 0x563e3cbd5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f1201be8700 1 -- 172.21.15.32:0/2375147183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd5b20 msgr2=0x563e3cbe5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f1201be8700 1 --2- 172.21.15.32:0/2375147183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd5b20 0x563e3cbe5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f1201be8700 1 -- 172.21.15.32:0/2375147183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd4a10 msgr2=0x563e3cbd5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f1201be8700 1 --2- 172.21.15.32:0/2375147183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd4a10 0x563e3cbd5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.066+0000 7f1201be8700 1 -- 172.21.15.32:0/2375147183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e3c932c00 con 0x563e3cbd64e0 2023-09-30T14:53:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.067+0000 7f1201be8700 1 --2- 172.21.15.32:0/2375147183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbd9390 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f11f4003e80 tx=0x7f11f4004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6013d5edec67468 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.067+0000 7f11e7fff700 1 -- 172.21.15.32:0/2375147183 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4007700 con 0x563e3cbd64e0 2023-09-30T14:53:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.067+0000 7f11e7fff700 1 -- 172.21.15.32:0/2375147183 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11f4007d20 con 0x563e3cbd64e0 2023-09-30T14:53:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.067+0000 7f11e7fff700 1 -- 172.21.15.32:0/2375147183 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f400ecf0 con 0x563e3cbd64e0 2023-09-30T14:53:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.068+0000 7f120f833b80 1 -- 172.21.15.32:0/2375147183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 msgr2=0x563e3cbd9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.068+0000 7f120f833b80 1 --2- 172.21.15.32:0/2375147183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbd9390 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f11f4003e80 tx=0x7f11f4004940 comp rx=0 tx=0).stop 2023-09-30T14:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.068+0000 7f120f833b80 1 -- 172.21.15.32:0/2375147183 shutdown_connections 2023-09-30T14:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.068+0000 7f120f833b80 1 --2- 172.21.15.32:0/2375147183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd64e0 0x563e3cbd9390 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.068+0000 7f120f833b80 1 --2- 172.21.15.32:0/2375147183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd4a10 0x563e3cbd5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.068+0000 7f120f833b80 1 --2- 172.21.15.32:0/2375147183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd5b20 0x563e3cbe5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.068+0000 7f120f833b80 1 -- 172.21.15.32:0/2375147183 >> 172.21.15.32:0/2375147183 conn(0x563e3cb40200 msgr2=0x563e3cbdc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.068+0000 7f120f833b80 1 -- 172.21.15.32:0/2375147183 shutdown_connections 2023-09-30T14:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.068+0000 7f120f833b80 1 -- 172.21.15.32:0/2375147183 wait complete. 2023-09-30T14:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.068+0000 7f120f833b80 1 -- start start 2023-09-30T14:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.068+0000 7f120f833b80 1 Processor -- start 2023-09-30T14:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f120f833b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd4a10 0x563e3cbe3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f120f833b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd5b20 0x563e3cbe4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f120f833b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd64e0 0x563e3cbe4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f120f833b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e3cbe54b0 con 0x563e3cbd5b20 2023-09-30T14:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f120f833b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e3cbe5620 con 0x563e3cbd64e0 2023-09-30T14:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f120f833b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e3cbe57e0 con 0x563e3cbd4a10 2023-09-30T14:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f120f833b80 10 client.0 _ll_register_callbacks cb 0x7f11f8099b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f1200be6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd5b20 0x563e3cbe4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f1200be6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd5b20 0x563e3cbe4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38678/0 (socket says 172.21.15.32:38678) 2023-09-30T14:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f12013e7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd4a10 0x563e3cbe3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f1200be6700 1 -- 172.21.15.32:0/4161197575 learned_addr learned my addr 172.21.15.32:0/4161197575 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f1201be8700 1 --2- 172.21.15.32:0/4161197575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd64e0 0x563e3cbe4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f1200be6700 1 -- 172.21.15.32:0/4161197575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd4a10 msgr2=0x563e3cbe3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f1200be6700 1 --2- 172.21.15.32:0/4161197575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd4a10 0x563e3cbe3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f1200be6700 1 -- 172.21.15.32:0/4161197575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd64e0 msgr2=0x563e3cbe4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f1200be6700 1 --2- 172.21.15.32:0/4161197575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd64e0 0x563e3cbe4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f1200be6700 1 -- 172.21.15.32:0/4161197575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e3cb81870 con 0x563e3cbd5b20 2023-09-30T14:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f1201be8700 1 --2- 172.21.15.32:0/4161197575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd64e0 0x563e3cbe4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f12013e7700 1 --2- 172.21.15.32:0/4161197575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd4a10 0x563e3cbe3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f1200be6700 1 --2- 172.21.15.32:0/4161197575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd5b20 0x563e3cbe4050 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x563e3cbecd10 tx=0x7f11ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c159b435cb28cec8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f11e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.069+0000 7f11e4ff9700 1 -- 172.21.15.32:0/4161197575 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec020040 con 0x563e3cbd5b20 2023-09-30T14:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.070+0000 7f11e4ff9700 1 -- 172.21.15.32:0/4161197575 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f11ec02ac70 con 0x563e3cbd5b20 2023-09-30T14:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.070+0000 7f120f833b80 1 -- 172.21.15.32:0/4161197575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11f400f040 con 0x563e3cbd5b20 2023-09-30T14:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.070+0000 7f11e4ff9700 1 -- 172.21.15.32:0/4161197575 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11ec014040 con 0x563e3cbd5b20 2023-09-30T14:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.070+0000 7f120f833b80 10 client.16749 fetch_fsmap learned FSMap version 47 2023-09-30T14:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.070+0000 7f120f833b80 1 -- 172.21.15.32:0/4161197575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x563e3c9701f0 con 0x563e3cbd5b20 2023-09-30T14:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.070+0000 7f11e4ff9700 1 -- 172.21.15.32:0/4161197575 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11f400f040 con 0x563e3cbd5b20 2023-09-30T14:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.070+0000 7f11e4ff9700 1 -- 172.21.15.32:0/4161197575 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 47) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f11ec033430 con 0x563e3cbd5b20 2023-09-30T14:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.070+0000 7f120f833b80 10 client.16749 fetch_fsmap finished waiting for FSMap version 47 2023-09-30T14:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.070+0000 7f120f833b80 10 client.16749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.070+0000 7f120f833b80 20 client.16749 populate_metadata read hostname 'smithi032' 2023-09-30T14:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.070+0000 7f120f833b80 1 --2- 172.21.15.32:0/4161197575 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563e3cc3f060 0x563e3cc41480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.070+0000 7f120f833b80 4 client.16749 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.070+0000 7f120f833b80 1 -- 172.21.15.32:0/4161197575 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563e3cb17ad0 con 0x563e3cc3f060 2023-09-30T14:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.070+0000 7f12013e7700 1 --2- 172.21.15.32:0/4161197575 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563e3cc3f060 0x563e3cc41480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.071+0000 7f12013e7700 1 --2- 172.21.15.32:0/4161197575 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563e3cc3f060 0x563e3cc41480 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f289cdf8a04bcd6 server_cookie=46d74297a14e158e in_seq=0 out_seq=0 2023-09-30T14:53:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.071+0000 7f11e4ff9700 10 client.16749 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.391+0000 7f11e4ff9700 1 -- 172.21.15.32:0/4161197575 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563e3cb17ad0 con 0x563e3cc3f060 2023-09-30T14:53:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.391+0000 7f11e4ff9700 10 client.16749 handle_command_reply: tid=0 2023-09-30T14:53:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.391+0000 7f120f833b80 1 client.16749 shutdown 2023-09-30T14:53:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.392+0000 7f120f833b80 1 -- 172.21.15.32:0/4161197575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd5b20 msgr2=0x563e3cbe4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.392+0000 7f120f833b80 1 --2- 172.21.15.32:0/4161197575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd5b20 0x563e3cbe4050 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x563e3cbecd10 tx=0x7f11ec004750 comp rx=0 tx=0).stop 2023-09-30T14:53:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.392+0000 7f120f833b80 1 -- 172.21.15.32:0/4161197575 shutdown_connections 2023-09-30T14:53:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.392+0000 7f120f833b80 1 --2- 172.21.15.32:0/4161197575 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563e3cc3f060 0x563e3cc41480 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.392+0000 7f120f833b80 1 --2- 172.21.15.32:0/4161197575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563e3cbd64e0 0x563e3cbe4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.392+0000 7f11e4ff9700 0 client.16749 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.392+0000 7f120f833b80 1 --2- 172.21.15.32:0/4161197575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563e3cbd4a10 0x563e3cbe3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.392+0000 7f120f833b80 1 --2- 172.21.15.32:0/4161197575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563e3cbd5b20 0x563e3cbe4050 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.392+0000 7f120f833b80 1 -- 172.21.15.32:0/4161197575 >> 172.21.15.32:0/4161197575 conn(0x563e3cb40200 msgr2=0x563e3cbd9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.392+0000 7f120f833b80 1 -- 172.21.15.32:0/4161197575 shutdown_connections 2023-09-30T14:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.392+0000 7f120f833b80 1 -- 172.21.15.32:0/4161197575 wait complete. 2023-09-30T14:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.392+0000 7f120f833b80 20 client.16749 trim_cache size 0 max 16384 2023-09-30T14:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:53:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:53:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.393+0000 7f120364b700 1 -- 172.21.15.32:0/1042563568 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f11e80a1020 msgr2=0x7f11e80a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.393+0000 7f120364b700 1 --2- 172.21.15.32:0/1042563568 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f11e80a1020 0x7f11e80a34e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f11ec005fd0 tx=0x7f11ec011040 comp rx=0 tx=0).stop 2023-09-30T14:53:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.393+0000 7f120364b700 1 -- 172.21.15.32:0/1042563568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11fc0ffd60 msgr2=0x7f11fc194a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.393+0000 7f120364b700 1 --2- 172.21.15.32:0/1042563568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11fc0ffd60 0x7f11fc194a70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009230 tx=0x7f11f800d280 comp rx=0 tx=0).stop 2023-09-30T14:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.393+0000 7f120364b700 1 -- 172.21.15.32:0/1042563568 shutdown_connections 2023-09-30T14:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.393+0000 7f120364b700 1 --2- 172.21.15.32:0/1042563568 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f11e80a1020 0x7f11e80a34e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.393+0000 7f120364b700 1 --2- 172.21.15.32:0/1042563568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11fc1018f0 0x7f11fc195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.393+0000 7f120364b700 1 --2- 172.21.15.32:0/1042563568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11fc0ffd60 0x7f11fc194a70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.393+0000 7f120364b700 1 --2- 172.21.15.32:0/1042563568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11fc100f50 0x7f11fc194fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.393+0000 7f120364b700 1 -- 172.21.15.32:0/1042563568 >> 172.21.15.32:0/1042563568 conn(0x7f11fc0fb2b0 msgr2=0x7f11fc104480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.394+0000 7f120364b700 1 -- 172.21.15.32:0/1042563568 shutdown_connections 2023-09-30T14:53:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:18.394+0000 7f120364b700 1 -- 172.21.15.32:0/1042563568 wait complete. 2023-09-30T14:53:19.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 71861 36.15 MB/sec warmup 112 sec latency 154.757 ms 2023-09-30T14:53:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:19 smithi156 ceph-mon[103579]: pgmap v738: 97 pgs: 97 active+clean; 674 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 842 op/s 2023-09-30T14:53:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:19 smithi148 ceph-mon[102355]: pgmap v738: 97 pgs: 97 active+clean; 674 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 842 op/s 2023-09-30T14:53:19.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:19 smithi032 ceph-mon[98592]: pgmap v738: 97 pgs: 97 active+clean; 674 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 842 op/s 2023-09-30T14:53:20.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 72806 36.24 MB/sec warmup 113 sec latency 53.258 ms 2023-09-30T14:53:21.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 73817 36.39 MB/sec warmup 114 sec latency 49.684 ms 2023-09-30T14:53:21.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:21 smithi148 ceph-mon[102355]: pgmap v739: 97 pgs: 97 active+clean; 677 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 761 op/s 2023-09-30T14:53:21.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:21 smithi032 ceph-mon[98592]: pgmap v739: 97 pgs: 97 active+clean; 677 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 761 op/s 2023-09-30T14:53:22.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 74609 36.52 MB/sec warmup 115 sec latency 174.670 ms 2023-09-30T14:53:22.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:21 smithi156 ceph-mon[103579]: pgmap v739: 97 pgs: 97 active+clean; 677 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 761 op/s 2023-09-30T14:53:23.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 75148 36.47 MB/sec warmup 116 sec latency 47.479 ms 2023-09-30T14:53:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:23 smithi148 ceph-mon[102355]: pgmap v740: 97 pgs: 97 active+clean; 675 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 833 op/s 2023-09-30T14:53:23.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:23 smithi032 ceph-mon[98592]: pgmap v740: 97 pgs: 97 active+clean; 675 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 833 op/s 2023-09-30T14:53:24.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 75900 36.44 MB/sec warmup 117 sec latency 59.923 ms 2023-09-30T14:53:24.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:23 smithi156 ceph-mon[103579]: pgmap v740: 97 pgs: 97 active+clean; 675 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 833 op/s 2023-09-30T14:53:25.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 76856 36.59 MB/sec warmup 118 sec latency 42.177 ms 2023-09-30T14:53:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:25 smithi148 ceph-mon[102355]: pgmap v741: 97 pgs: 97 active+clean; 671 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 835 op/s 2023-09-30T14:53:25.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:25 smithi032 ceph-mon[98592]: pgmap v741: 97 pgs: 97 active+clean; 671 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 835 op/s 2023-09-30T14:53:26.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 77706 36.64 MB/sec warmup 119 sec latency 52.213 ms 2023-09-30T14:53:26.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:25 smithi156 ceph-mon[103579]: pgmap v741: 97 pgs: 97 active+clean; 671 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 835 op/s 2023-09-30T14:53:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:27 smithi148 ceph-mon[102355]: pgmap v742: 97 pgs: 97 active+clean; 661 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.6 MiB/s wr, 826 op/s 2023-09-30T14:53:27.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:27 smithi032 ceph-mon[98592]: pgmap v742: 97 pgs: 97 active+clean; 661 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.6 MiB/s wr, 826 op/s 2023-09-30T14:53:28.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 79098 42.40 MB/sec execute 1 sec latency 218.931 ms 2023-09-30T14:53:28.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:27 smithi156 ceph-mon[103579]: pgmap v742: 97 pgs: 97 active+clean; 661 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.6 MiB/s wr, 826 op/s 2023-09-30T14:53:29.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 79853 40.83 MB/sec execute 2 sec latency 48.039 ms 2023-09-30T14:53:29.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:29 smithi148 ceph-mon[102355]: pgmap v743: 97 pgs: 97 active+clean; 668 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.5 MiB/s wr, 868 op/s 2023-09-30T14:53:29.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:29 smithi032 ceph-mon[98592]: pgmap v743: 97 pgs: 97 active+clean; 668 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.5 MiB/s wr, 868 op/s 2023-09-30T14:53:30.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 80429 35.78 MB/sec execute 3 sec latency 122.712 ms 2023-09-30T14:53:30.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:29 smithi156 ceph-mon[103579]: pgmap v743: 97 pgs: 97 active+clean; 668 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.5 MiB/s wr, 868 op/s 2023-09-30T14:53:31.006 INFO:tasks.workunit.client.0.smithi032.stdout: 10 81339 37.46 MB/sec execute 4 sec latency 68.914 ms 2023-09-30T14:53:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.542+0000 7f895cee1700 1 -- 172.21.15.32:0/1102184295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89580ffd60 msgr2=0x7f8958100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.542+0000 7f895cee1700 1 --2- 172.21.15.32:0/1102184295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89580ffd60 0x7f8958100180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f895000b0a0 tx=0x7f895000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.542+0000 7f895cee1700 1 -- 172.21.15.32:0/1102184295 shutdown_connections 2023-09-30T14:53:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.542+0000 7f895cee1700 1 --2- 172.21.15.32:0/1102184295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89581018f0 0x7f89581081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.542+0000 7f895cee1700 1 --2- 172.21.15.32:0/1102184295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89580ffd60 0x7f8958100180 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.542+0000 7f895cee1700 1 --2- 172.21.15.32:0/1102184295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8958100f50 0x7f89581013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.542+0000 7f895cee1700 1 -- 172.21.15.32:0/1102184295 >> 172.21.15.32:0/1102184295 conn(0x7f89580fb2b0 msgr2=0x7f89580fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.543+0000 7f895cee1700 1 -- 172.21.15.32:0/1102184295 shutdown_connections 2023-09-30T14:53:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.543+0000 7f895cee1700 1 -- 172.21.15.32:0/1102184295 wait complete. 2023-09-30T14:53:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.544+0000 7f895cee1700 1 Processor -- start 2023-09-30T14:53:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.544+0000 7f895cee1700 1 -- start start 2023-09-30T14:53:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.544+0000 7f895cee1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89580ffd60 0x7f895819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.544+0000 7f895cee1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89581018f0 0x7f895819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.544+0000 7f895cee1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f895819e800 0x7f895819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.544+0000 7f895cee1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895819f250 con 0x7f89581018f0 2023-09-30T14:53:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.544+0000 7f895cee1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895819f3c0 con 0x7f89580ffd60 2023-09-30T14:53:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.544+0000 7f895cee1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895819f580 con 0x7f895819e800 2023-09-30T14:53:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.544+0000 7f895659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89580ffd60 0x7f895819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.544+0000 7f895659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89580ffd60 0x7f895819d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:50164/0 (socket says 172.21.15.32:50164) 2023-09-30T14:53:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.544+0000 7f895659c700 1 -- 172.21.15.32:0/1934227537 learned_addr learned my addr 172.21.15.32:0/1934227537 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.544+0000 7f8956d9d700 1 --2- 172.21.15.32:0/1934227537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f895819e800 0x7f895819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.544+0000 7f8955d9b700 1 --2- 172.21.15.32:0/1934227537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89581018f0 0x7f895819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.545+0000 7f895659c700 1 -- 172.21.15.32:0/1934227537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f895819e800 msgr2=0x7f895819ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.545+0000 7f895659c700 1 --2- 172.21.15.32:0/1934227537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f895819e800 0x7f895819ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.545+0000 7f895659c700 1 -- 172.21.15.32:0/1934227537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89581018f0 msgr2=0x7f895819d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.545+0000 7f895659c700 1 --2- 172.21.15.32:0/1934227537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89581018f0 0x7f895819d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.545+0000 7f895659c700 1 -- 172.21.15.32:0/1934227537 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8950009d00 con 0x7f89580ffd60 2023-09-30T14:53:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.545+0000 7f895659c700 1 --2- 172.21.15.32:0/1934227537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89580ffd60 0x7f895819d440 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8950006130 tx=0x7f89500096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.547+0000 7f89477fe700 1 -- 172.21.15.32:0/1934227537 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895000e040 con 0x7f89580ffd60 2023-09-30T14:53:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.547+0000 7f89477fe700 1 -- 172.21.15.32:0/1934227537 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8950004e60 con 0x7f89580ffd60 2023-09-30T14:53:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.547+0000 7f89477fe700 1 -- 172.21.15.32:0/1934227537 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895001da90 con 0x7f89580ffd60 2023-09-30T14:53:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.547+0000 7f895cee1700 1 -- 172.21.15.32:0/1934227537 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89581a5a50 con 0x7f89580ffd60 2023-09-30T14:53:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.548+0000 7f895cee1700 1 -- 172.21.15.32:0/1934227537 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89581a5f00 con 0x7f89580ffd60 2023-09-30T14:53:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.550+0000 7f89477fe700 1 -- 172.21.15.32:0/1934227537 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f8950005780 con 0x7f89580ffd60 2023-09-30T14:53:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.551+0000 7f89477fe700 1 --2- 172.21.15.32:0/1934227537 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89400a1170 0x7f89400a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.552+0000 7f8955d9b700 1 --2- 172.21.15.32:0/1934227537 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89400a1170 0x7f89400a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.552+0000 7f89477fe700 1 -- 172.21.15.32:0/1934227537 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f89500cdb60 con 0x7f89580ffd60 2023-09-30T14:53:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.552+0000 7f8955d9b700 1 --2- 172.21.15.32:0/1934227537 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89400a1170 0x7f89400a3630 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8958100a90 tx=0x7f8948009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.552+0000 7f89690c9b80 1 Processor -- start 2023-09-30T14:53:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f89690c9b80 1 -- start start 2023-09-30T14:53:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f89690c9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa85200 0x5562ffa85620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f89690c9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa86310 0x5562ffa86790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f89690c9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86cd0 0x5562ffa8d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f89690c9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ff9ed400 con 0x5562ffa86310 2023-09-30T14:53:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f89690c9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ff9debc0 con 0x5562ffa85200 2023-09-30T14:53:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f89690c9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ff9dc4e0 con 0x5562ffa86cd0 2023-09-30T14:53:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f8955d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa86310 0x5562ffa86790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f895659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa85200 0x5562ffa85620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f8955d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa86310 0x5562ffa86790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38698/0 (socket says 172.21.15.32:38698) 2023-09-30T14:53:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f8955d9b700 1 -- 172.21.15.32:0/3163570723 learned_addr learned my addr 172.21.15.32:0/3163570723 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f8956d9d700 1 --2- 172.21.15.32:0/3163570723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86cd0 0x5562ffa8d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f895659c700 1 -- 172.21.15.32:0/3163570723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86cd0 msgr2=0x5562ffa8d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f895659c700 1 --2- 172.21.15.32:0/3163570723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86cd0 0x5562ffa8d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f895659c700 1 -- 172.21.15.32:0/3163570723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa86310 msgr2=0x5562ffa86790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f895659c700 1 --2- 172.21.15.32:0/3163570723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa86310 0x5562ffa86790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f895659c700 1 -- 172.21.15.32:0/3163570723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562ffa3a870 con 0x5562ffa85200 2023-09-30T14:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.553+0000 7f895659c700 1 --2- 172.21.15.32:0/3163570723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa85200 0x5562ffa85620 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8950000f80 tx=0x7f895001d480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e173182910bbffb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.554+0000 7f8944ff9700 1 -- 172.21.15.32:0/3163570723 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895009f1b0 con 0x5562ffa85200 2023-09-30T14:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.554+0000 7f8944ff9700 1 -- 172.21.15.32:0/3163570723 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89500a3360 con 0x5562ffa85200 2023-09-30T14:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.554+0000 7f8944ff9700 1 -- 172.21.15.32:0/3163570723 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895009a550 con 0x5562ffa85200 2023-09-30T14:53:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.554+0000 7f89690c9b80 1 -- 172.21.15.32:0/3163570723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa85200 msgr2=0x5562ffa85620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.554+0000 7f89690c9b80 1 --2- 172.21.15.32:0/3163570723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa85200 0x5562ffa85620 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8950000f80 tx=0x7f895001d480 comp rx=0 tx=0).stop 2023-09-30T14:53:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.554+0000 7f89690c9b80 1 -- 172.21.15.32:0/3163570723 shutdown_connections 2023-09-30T14:53:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.554+0000 7f89690c9b80 1 --2- 172.21.15.32:0/3163570723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86cd0 0x5562ffa8d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.554+0000 7f89690c9b80 1 --2- 172.21.15.32:0/3163570723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa85200 0x5562ffa85620 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.554+0000 7f89690c9b80 1 --2- 172.21.15.32:0/3163570723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa86310 0x5562ffa86790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.554+0000 7f89690c9b80 1 -- 172.21.15.32:0/3163570723 >> 172.21.15.32:0/3163570723 conn(0x5562ff9f9200 msgr2=0x5562ffa7fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.555+0000 7f89690c9b80 1 -- 172.21.15.32:0/3163570723 shutdown_connections 2023-09-30T14:53:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.555+0000 7f89690c9b80 1 -- 172.21.15.32:0/3163570723 wait complete. 2023-09-30T14:53:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.555+0000 7f89690c9b80 1 -- start start 2023-09-30T14:53:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.555+0000 7f89690c9b80 1 Processor -- start 2023-09-30T14:53:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.555+0000 7f89690c9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa85200 0x5562ffa8ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.555+0000 7f89690c9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86310 0x5562ffa8f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.555+0000 7f89690c9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa86cd0 0x5562ffa961d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.555+0000 7f89690c9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ff9d7dd0 con 0x5562ffa86cd0 2023-09-30T14:53:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.555+0000 7f89690c9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ff9746b0 con 0x5562ffa86310 2023-09-30T14:53:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.555+0000 7f89690c9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ff974350 con 0x5562ffa85200 2023-09-30T14:53:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.555+0000 7f89690c9b80 10 client.0 _ll_register_callbacks cb 0x5562ff7d17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:53:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.556+0000 7f8955d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86310 0x5562ffa8f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.556+0000 7f895659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa85200 0x5562ffa8ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.556+0000 7f8956d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa86cd0 0x5562ffa961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.556+0000 7f8955d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86310 0x5562ffa8f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:50180/0 (socket says 172.21.15.32:50180) 2023-09-30T14:53:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.556+0000 7f8955d9b700 1 -- 172.21.15.32:0/846852962 learned_addr learned my addr 172.21.15.32:0/846852962 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.556+0000 7f8955d9b700 1 -- 172.21.15.32:0/846852962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa85200 msgr2=0x5562ffa8ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.556+0000 7f8955d9b700 1 --2- 172.21.15.32:0/846852962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa85200 0x5562ffa8ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.556+0000 7f8955d9b700 1 -- 172.21.15.32:0/846852962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa86cd0 msgr2=0x5562ffa961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.556+0000 7f8955d9b700 1 --2- 172.21.15.32:0/846852962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa86cd0 0x5562ffa961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.556+0000 7f8955d9b700 1 -- 172.21.15.32:0/846852962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89581a5f00 con 0x5562ffa86310 2023-09-30T14:53:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.556+0000 7f895659c700 1 --2- 172.21.15.32:0/846852962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa85200 0x5562ffa8ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:53:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.556+0000 7f8956d9d700 1 --2- 172.21.15.32:0/846852962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa86cd0 0x5562ffa961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:53:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.556+0000 7f8955d9b700 1 --2- 172.21.15.32:0/846852962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86310 0x5562ffa8f190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f894800f000 tx=0x7f89480047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc94dc5bcfce28ef server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.556+0000 7f893dffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:53:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.556+0000 7f893dffb700 1 -- 172.21.15.32:0/846852962 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948015b20 con 0x5562ffa86310 2023-09-30T14:53:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.557+0000 7f893dffb700 1 -- 172.21.15.32:0/846852962 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8948015c80 con 0x5562ffa86310 2023-09-30T14:53:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.557+0000 7f89690c9b80 1 -- 172.21.15.32:0/846852962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562ffa78090 con 0x5562ffa86310 2023-09-30T14:53:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.557+0000 7f893dffb700 1 -- 172.21.15.32:0/846852962 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8948015e40 con 0x5562ffa86310 2023-09-30T14:53:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.557+0000 7f89690c9b80 10 client.26417 fetch_fsmap learned FSMap version 47 2023-09-30T14:53:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.557+0000 7f89690c9b80 1 -- 172.21.15.32:0/846852962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5562ff802d80 con 0x5562ffa86310 2023-09-30T14:53:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.557+0000 7f893dffb700 1 -- 172.21.15.32:0/846852962 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5562ffa78090 con 0x5562ffa86310 2023-09-30T14:53:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.557+0000 7f893dffb700 1 -- 172.21.15.32:0/846852962 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 47) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8948013450 con 0x5562ffa86310 2023-09-30T14:53:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.557+0000 7f89690c9b80 10 client.26417 fetch_fsmap finished waiting for FSMap version 47 2023-09-30T14:53:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.557+0000 7f89690c9b80 10 client.26417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:53:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.557+0000 7f89690c9b80 20 client.26417 populate_metadata read hostname 'smithi032' 2023-09-30T14:53:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.557+0000 7f89690c9b80 1 --2- 172.21.15.32:0/846852962 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5562ffa9ab00 0x5562ffa9cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.557+0000 7f89690c9b80 4 client.26417 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:53:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.557+0000 7f89690c9b80 1 -- 172.21.15.32:0/846852962 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562ff9d0db0 con 0x5562ffa9ab00 2023-09-30T14:53:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.558+0000 7f895659c700 1 --2- 172.21.15.32:0/846852962 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5562ffa9ab00 0x5562ffa9cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.558+0000 7f893dffb700 10 client.26417 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.558+0000 7f895659c700 1 --2- 172.21.15.32:0/846852962 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5562ffa9ab00 0x5562ffa9cf20 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c020ea186173a57 server_cookie=e782f7b2f09fb0c in_seq=0 out_seq=0 2023-09-30T14:53:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.561+0000 7f893dffb700 1 -- 172.21.15.32:0/846852962 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5562ff9d0db0 con 0x5562ffa9ab00 2023-09-30T14:53:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.561+0000 7f893dffb700 10 client.26417 handle_command_reply: tid=0 2023-09-30T14:53:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.561+0000 7f89690c9b80 1 client.26417 shutdown 2023-09-30T14:53:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.561+0000 7f89690c9b80 1 -- 172.21.15.32:0/846852962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86310 msgr2=0x5562ffa8f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.561+0000 7f89690c9b80 1 --2- 172.21.15.32:0/846852962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86310 0x5562ffa8f190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f894800f000 tx=0x7f89480047b0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.561+0000 7f89690c9b80 1 -- 172.21.15.32:0/846852962 shutdown_connections 2023-09-30T14:53:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.561+0000 7f89690c9b80 1 --2- 172.21.15.32:0/846852962 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5562ffa9ab00 0x5562ffa9cf20 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.561+0000 7f89690c9b80 1 --2- 172.21.15.32:0/846852962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa86cd0 0x5562ffa961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.561+0000 7f89690c9b80 1 --2- 172.21.15.32:0/846852962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa85200 0x5562ffa8ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.561+0000 7f89690c9b80 1 --2- 172.21.15.32:0/846852962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86310 0x5562ffa8f190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.561+0000 7f89690c9b80 1 -- 172.21.15.32:0/846852962 >> 172.21.15.32:0/846852962 conn(0x5562ff9f9200 msgr2=0x5562ffa7f590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.561+0000 7f893dffb700 0 client.26417 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.562+0000 7f89690c9b80 1 -- 172.21.15.32:0/846852962 shutdown_connections 2023-09-30T14:53:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.562+0000 7f89690c9b80 1 -- 172.21.15.32:0/846852962 wait complete. 2023-09-30T14:53:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.562+0000 7f89690c9b80 20 client.26417 trim_cache size 0 max 16384 2023-09-30T14:53:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.584+0000 7f89690c9b80 1 Processor -- start 2023-09-30T14:53:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.586+0000 7f89690c9b80 1 -- start start 2023-09-30T14:53:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.586+0000 7f89690c9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa85200 0x5562ffa89b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.586+0000 7f89690c9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86310 0x5562ffa8a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.586+0000 7f89690c9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86cd0 0x5562ffa960f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.586+0000 7f89690c9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ff8588c0 con 0x5562ffa85200 2023-09-30T14:53:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.586+0000 7f89690c9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ffa8fe40 con 0x5562ffa86310 2023-09-30T14:53:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.586+0000 7f89690c9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ffa80bf0 con 0x5562ffa86cd0 2023-09-30T14:53:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.587+0000 7f8956d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86cd0 0x5562ffa960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.587+0000 7f8956d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86cd0 0x5562ffa960f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53092/0 (socket says 172.21.15.32:53092) 2023-09-30T14:53:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.587+0000 7f8956d9d700 1 -- 172.21.15.32:0/1391086239 learned_addr learned my addr 172.21.15.32:0/1391086239 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.587+0000 7f895659c700 1 --2- 172.21.15.32:0/1391086239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa85200 0x5562ffa89b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.587+0000 7f8956d9d700 1 -- 172.21.15.32:0/1391086239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86310 msgr2=0x5562ffa8a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.587+0000 7f8956d9d700 1 --2- 172.21.15.32:0/1391086239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86310 0x5562ffa8a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.587+0000 7f8956d9d700 1 -- 172.21.15.32:0/1391086239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa85200 msgr2=0x5562ffa89b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.587+0000 7f8956d9d700 1 --2- 172.21.15.32:0/1391086239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa85200 0x5562ffa89b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.587+0000 7f8956d9d700 1 -- 172.21.15.32:0/1391086239 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562ff7ebc00 con 0x5562ffa86cd0 2023-09-30T14:53:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.587+0000 7f8956d9d700 1 --2- 172.21.15.32:0/1391086239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86cd0 0x5562ffa960f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f894c00b830 tx=0x7f894c00bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b3c7e983446ba4d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.588+0000 7f89457fa700 1 -- 172.21.15.32:0/1391086239 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c01f070 con 0x5562ffa86cd0 2023-09-30T14:53:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.588+0000 7f89457fa700 1 -- 172.21.15.32:0/1391086239 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f894c017040 con 0x5562ffa86cd0 2023-09-30T14:53:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.588+0000 7f89457fa700 1 -- 172.21.15.32:0/1391086239 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c016070 con 0x5562ffa86cd0 2023-09-30T14:53:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.589+0000 7f89690c9b80 1 -- 172.21.15.32:0/1391086239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86cd0 msgr2=0x5562ffa960f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.589+0000 7f89690c9b80 1 --2- 172.21.15.32:0/1391086239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86cd0 0x5562ffa960f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f894c00b830 tx=0x7f894c00bb40 comp rx=0 tx=0).stop 2023-09-30T14:53:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.589+0000 7f89690c9b80 1 -- 172.21.15.32:0/1391086239 shutdown_connections 2023-09-30T14:53:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.589+0000 7f89690c9b80 1 --2- 172.21.15.32:0/1391086239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86cd0 0x5562ffa960f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.589+0000 7f89690c9b80 1 --2- 172.21.15.32:0/1391086239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa85200 0x5562ffa89b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.589+0000 7f89690c9b80 1 --2- 172.21.15.32:0/1391086239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86310 0x5562ffa8a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.589+0000 7f89690c9b80 1 -- 172.21.15.32:0/1391086239 >> 172.21.15.32:0/1391086239 conn(0x5562ff9f9200 msgr2=0x5562ffa8ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.593+0000 7f89690c9b80 1 -- 172.21.15.32:0/1391086239 shutdown_connections 2023-09-30T14:53:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.594+0000 7f89690c9b80 1 -- 172.21.15.32:0/1391086239 wait complete. 2023-09-30T14:53:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.594+0000 7f89690c9b80 1 -- start start 2023-09-30T14:53:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.594+0000 7f89690c9b80 1 Processor -- start 2023-09-30T14:53:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.595+0000 7f89690c9b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa85200 0x5562ffa91b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.595+0000 7f89690c9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86310 0x5562ffa92090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.596+0000 7f89690c9b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86cd0 0x5562ffa950b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.596+0000 7f89690c9b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ffa955f0 con 0x5562ffa85200 2023-09-30T14:53:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.596+0000 7f89690c9b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ffa95760 con 0x5562ffa86cd0 2023-09-30T14:53:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.596+0000 7f89690c9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562ffa95920 con 0x5562ffa86310 2023-09-30T14:53:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.596+0000 7f89690c9b80 10 client.0 _ll_register_callbacks cb 0x7f894800ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:53:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.596+0000 7f8956d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86cd0 0x5562ffa950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.596+0000 7f8956d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86cd0 0x5562ffa950b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:50196/0 (socket says 172.21.15.32:50196) 2023-09-30T14:53:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.596+0000 7f8956d9d700 1 -- 172.21.15.32:0/2812046846 learned_addr learned my addr 172.21.15.32:0/2812046846 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.596+0000 7f8955d9b700 1 --2- 172.21.15.32:0/2812046846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86310 0x5562ffa92090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.596+0000 7f8956d9d700 1 -- 172.21.15.32:0/2812046846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86310 msgr2=0x5562ffa92090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.596+0000 7f8956d9d700 1 --2- 172.21.15.32:0/2812046846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86310 0x5562ffa92090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.596+0000 7f8956d9d700 1 -- 172.21.15.32:0/2812046846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa85200 msgr2=0x5562ffa91b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.596+0000 7f8956d9d700 1 --2- 172.21.15.32:0/2812046846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa85200 0x5562ffa91b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.596+0000 7f8956d9d700 1 -- 172.21.15.32:0/2812046846 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562ff8291f0 con 0x5562ffa86cd0 2023-09-30T14:53:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.598+0000 7f8956d9d700 1 --2- 172.21.15.32:0/2812046846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86cd0 0x5562ffa950b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f894c004410 tx=0x7f894c005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=233504ffcee2d7c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.598+0000 7f893dffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:53:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.598+0000 7f893dffb700 1 -- 172.21.15.32:0/2812046846 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c01f030 con 0x5562ffa86cd0 2023-09-30T14:53:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.598+0000 7f893dffb700 1 -- 172.21.15.32:0/2812046846 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f894c0051b0 con 0x5562ffa86cd0 2023-09-30T14:53:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.598+0000 7f89690c9b80 1 -- 172.21.15.32:0/2812046846 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89500a3360 con 0x5562ffa86cd0 2023-09-30T14:53:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.603+0000 7f893dffb700 1 -- 172.21.15.32:0/2812046846 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f894c016040 con 0x5562ffa86cd0 2023-09-30T14:53:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.603+0000 7f89690c9b80 10 client.26420 fetch_fsmap learned FSMap version 47 2023-09-30T14:53:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.603+0000 7f89690c9b80 1 -- 172.21.15.32:0/2812046846 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5562ff801250 con 0x5562ffa86cd0 2023-09-30T14:53:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.605+0000 7f893dffb700 1 -- 172.21.15.32:0/2812046846 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f894c034020 con 0x5562ffa86cd0 2023-09-30T14:53:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.605+0000 7f893dffb700 1 -- 172.21.15.32:0/2812046846 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 47) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f894c02c060 con 0x5562ffa86cd0 2023-09-30T14:53:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.605+0000 7f89690c9b80 10 client.26420 fetch_fsmap finished waiting for FSMap version 47 2023-09-30T14:53:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.605+0000 7f89690c9b80 10 client.26420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:53:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.605+0000 7f89690c9b80 20 client.26420 populate_metadata read hostname 'smithi032' 2023-09-30T14:53:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.605+0000 7f89690c9b80 1 --2- 172.21.15.32:0/2812046846 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5562ffb004b0 0x5562ffb028d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.605+0000 7f89690c9b80 4 client.26420 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:53:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.605+0000 7f89690c9b80 1 -- 172.21.15.32:0/2812046846 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5562ff9d0ad0 con 0x5562ffb004b0 2023-09-30T14:53:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.605+0000 7f895659c700 1 --2- 172.21.15.32:0/2812046846 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5562ffb004b0 0x5562ffb028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.605+0000 7f893dffb700 10 client.26420 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:31.606+0000 7f895659c700 1 --2- 172.21.15.32:0/2812046846 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5562ffb004b0 0x5562ffb028d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2c27c4b91d64b0f server_cookie=ffea0a96bd17f8e in_seq=0 out_seq=0 2023-09-30T14:53:31.865 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:31 smithi148 ceph-mon[102355]: pgmap v744: 97 pgs: 97 active+clean; 668 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.0 MiB/s wr, 800 op/s 2023-09-30T14:53:31.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:31 smithi032 ceph-mon[98592]: pgmap v744: 97 pgs: 97 active+clean; 668 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.0 MiB/s wr, 800 op/s 2023-09-30T14:53:32.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 82032 37.72 MB/sec execute 5 sec latency 314.000 ms 2023-09-30T14:53:32.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:31 smithi156 ceph-mon[103579]: pgmap v744: 97 pgs: 97 active+clean; 668 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.0 MiB/s wr, 800 op/s 2023-09-30T14:53:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.100+0000 7f893dffb700 1 -- 172.21.15.32:0/2812046846 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5562ff9d0ad0 con 0x5562ffb004b0 2023-09-30T14:53:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.100+0000 7f893dffb700 10 client.26420 handle_command_reply: tid=0 2023-09-30T14:53:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.100+0000 7f89690c9b80 1 client.26420 shutdown 2023-09-30T14:53:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.100+0000 7f89690c9b80 1 -- 172.21.15.32:0/2812046846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86cd0 msgr2=0x5562ffa950b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.100+0000 7f89690c9b80 1 --2- 172.21.15.32:0/2812046846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86cd0 0x5562ffa950b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f894c004410 tx=0x7f894c005db0 comp rx=0 tx=0).stop 2023-09-30T14:53:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.100+0000 7f89690c9b80 1 -- 172.21.15.32:0/2812046846 shutdown_connections 2023-09-30T14:53:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f89690c9b80 1 --2- 172.21.15.32:0/2812046846 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5562ffb004b0 0x5562ffb028d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f89690c9b80 1 --2- 172.21.15.32:0/2812046846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562ffa86cd0 0x5562ffa950b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f89690c9b80 1 --2- 172.21.15.32:0/2812046846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5562ffa85200 0x5562ffa91b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f89690c9b80 1 --2- 172.21.15.32:0/2812046846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5562ffa86310 0x5562ffa92090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f893dffb700 0 client.26420 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f89690c9b80 1 -- 172.21.15.32:0/2812046846 >> 172.21.15.32:0/2812046846 conn(0x5562ff9f9200 msgr2=0x5562ffa8ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f89690c9b80 1 -- 172.21.15.32:0/2812046846 shutdown_connections 2023-09-30T14:53:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f89690c9b80 1 -- 172.21.15.32:0/2812046846 wait complete. 2023-09-30T14:53:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f89690c9b80 20 client.26420 trim_cache size 0 max 16384 2023-09-30T14:53:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:53:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:53:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:53:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:53:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f895cee1700 1 -- 172.21.15.32:0/1934227537 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89400a1170 msgr2=0x7f89400a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f895cee1700 1 --2- 172.21.15.32:0/1934227537 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89400a1170 0x7f89400a3630 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8958100a90 tx=0x7f8948009110 comp rx=0 tx=0).stop 2023-09-30T14:53:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f895cee1700 1 -- 172.21.15.32:0/1934227537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89580ffd60 msgr2=0x7f895819d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f895cee1700 1 --2- 172.21.15.32:0/1934227537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89580ffd60 0x7f895819d440 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8950006130 tx=0x7f89500096c0 comp rx=0 tx=0).stop 2023-09-30T14:53:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f895cee1700 1 -- 172.21.15.32:0/1934227537 shutdown_connections 2023-09-30T14:53:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f895cee1700 1 --2- 172.21.15.32:0/1934227537 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f89400a1170 0x7f89400a3630 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f895cee1700 1 --2- 172.21.15.32:0/1934227537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f895819e800 0x7f895819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f895cee1700 1 --2- 172.21.15.32:0/1934227537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89580ffd60 0x7f895819d440 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f895cee1700 1 --2- 172.21.15.32:0/1934227537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89581018f0 0x7f895819d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f895cee1700 1 -- 172.21.15.32:0/1934227537 >> 172.21.15.32:0/1934227537 conn(0x7f89580fb2b0 msgr2=0x7f8958106970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f895cee1700 1 -- 172.21.15.32:0/1934227537 shutdown_connections 2023-09-30T14:53:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:32.101+0000 7f895cee1700 1 -- 172.21.15.32:0/1934227537 wait complete. 2023-09-30T14:53:33.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 82783 40.03 MB/sec execute 6 sec latency 457.599 ms 2023-09-30T14:53:33.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:33 smithi148 ceph-mon[102355]: pgmap v745: 97 pgs: 97 active+clean; 680 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 872 op/s 2023-09-30T14:53:33.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:33 smithi032 ceph-mon[98592]: pgmap v745: 97 pgs: 97 active+clean; 680 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 872 op/s 2023-09-30T14:53:34.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 83429 40.51 MB/sec execute 7 sec latency 112.394 ms 2023-09-30T14:53:34.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:33 smithi156 ceph-mon[103579]: pgmap v745: 97 pgs: 97 active+clean; 680 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 872 op/s 2023-09-30T14:53:35.006 INFO:tasks.workunit.client.0.smithi032.stdout: 10 83915 37.84 MB/sec execute 8 sec latency 123.744 ms 2023-09-30T14:53:35.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:35 smithi148 ceph-mon[102355]: pgmap v746: 97 pgs: 97 active+clean; 677 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.9 MiB/s wr, 781 op/s 2023-09-30T14:53:35.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:35 smithi032 ceph-mon[98592]: pgmap v746: 97 pgs: 97 active+clean; 677 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.9 MiB/s wr, 781 op/s 2023-09-30T14:53:36.005 INFO:tasks.workunit.client.0.smithi032.stdout: 10 84540 36.13 MB/sec execute 9 sec latency 133.286 ms 2023-09-30T14:53:36.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:35 smithi156 ceph-mon[103579]: pgmap v746: 97 pgs: 97 active+clean; 677 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.9 MiB/s wr, 781 op/s 2023-09-30T14:53:37.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 85456 37.60 MB/sec execute 10 sec latency 140.104 ms 2023-09-30T14:53:37.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:37 smithi032 ceph-mon[98592]: pgmap v747: 97 pgs: 97 active+clean; 668 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 10 MiB/s wr, 842 op/s 2023-09-30T14:53:38.006 INFO:tasks.workunit.client.0.smithi032.stdout: 10 86253 37.79 MB/sec execute 11 sec latency 246.589 ms 2023-09-30T14:53:38.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:37 smithi156 ceph-mon[103579]: pgmap v747: 97 pgs: 97 active+clean; 668 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 10 MiB/s wr, 842 op/s 2023-09-30T14:53:38.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:37 smithi148 ceph-mon[102355]: pgmap v747: 97 pgs: 97 active+clean; 668 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 10 MiB/s wr, 842 op/s 2023-09-30T14:53:39.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 86967 39.09 MB/sec execute 12 sec latency 58.086 ms 2023-09-30T14:53:39.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:39 smithi032 ceph-mon[98592]: pgmap v748: 97 pgs: 97 active+clean; 662 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 923 op/s 2023-09-30T14:53:39.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:39 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:53:40.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 87593 38.55 MB/sec execute 13 sec latency 57.674 ms 2023-09-30T14:53:40.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:39 smithi156 ceph-mon[103579]: pgmap v748: 97 pgs: 97 active+clean; 662 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 923 op/s 2023-09-30T14:53:40.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:39 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:53:40.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:39 smithi148 ceph-mon[102355]: pgmap v748: 97 pgs: 97 active+clean; 662 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 923 op/s 2023-09-30T14:53:40.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:39 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:53:40.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:53:40.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:53:40.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:53:41.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 88462 38.31 MB/sec execute 14 sec latency 54.027 ms 2023-09-30T14:53:41.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:53:41.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:53:41.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:53:41.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:53:41.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:53:41.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:53:41.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:41 smithi032 ceph-mon[98592]: pgmap v749: 97 pgs: 97 active+clean; 665 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 804 op/s 2023-09-30T14:53:42.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 89305 39.53 MB/sec execute 15 sec latency 73.754 ms 2023-09-30T14:53:42.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:41 smithi156 ceph-mon[103579]: pgmap v749: 97 pgs: 97 active+clean; 665 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 804 op/s 2023-09-30T14:53:42.116 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:41 smithi148 ceph-mon[102355]: pgmap v749: 97 pgs: 97 active+clean; 665 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 804 op/s 2023-09-30T14:53:43.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 89823 38.71 MB/sec execute 16 sec latency 230.995 ms 2023-09-30T14:53:43.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:43 smithi032 ceph-mon[98592]: pgmap v750: 97 pgs: 97 active+clean; 665 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 833 op/s 2023-09-30T14:53:44.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 90633 38.60 MB/sec execute 17 sec latency 63.888 ms 2023-09-30T14:53:44.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:43 smithi156 ceph-mon[103579]: pgmap v750: 97 pgs: 97 active+clean; 665 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 833 op/s 2023-09-30T14:53:44.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:43 smithi148 ceph-mon[102355]: pgmap v750: 97 pgs: 97 active+clean; 665 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 833 op/s 2023-09-30T14:53:45.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 91635 39.82 MB/sec execute 18 sec latency 48.124 ms 2023-09-30T14:53:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.239+0000 7f79db564700 1 -- 172.21.15.32:0/770593444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d4107e20 msgr2=0x7f79d410a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.239+0000 7f79db564700 1 --2- 172.21.15.32:0/770593444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d4107e20 0x7f79d410a250 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009990 tx=0x7f79c4009ca0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.239+0000 7f79db564700 1 -- 172.21.15.32:0/770593444 shutdown_connections 2023-09-30T14:53:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.239+0000 7f79db564700 1 --2- 172.21.15.32:0/770593444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d410d160 0x7f79d410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.239+0000 7f79db564700 1 --2- 172.21.15.32:0/770593444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d4107e20 0x7f79d410a250 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.239+0000 7f79db564700 1 --2- 172.21.15.32:0/770593444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d410a790 0x7f79d410cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.239+0000 7f79db564700 1 -- 172.21.15.32:0/770593444 >> 172.21.15.32:0/770593444 conn(0x7f79d406dc80 msgr2=0x7f79d40700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.239+0000 7f79db564700 1 -- 172.21.15.32:0/770593444 shutdown_connections 2023-09-30T14:53:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.239+0000 7f79db564700 1 -- 172.21.15.32:0/770593444 wait complete. 2023-09-30T14:53:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.240+0000 7f79db564700 1 Processor -- start 2023-09-30T14:53:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.241+0000 7f79db564700 1 -- start start 2023-09-30T14:53:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.241+0000 7f79db564700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d4107e20 0x7f79d41a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.241+0000 7f79db564700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d410a790 0x7f79d41a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.241+0000 7f79db564700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d410d160 0x7f79d41a6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.241+0000 7f79db564700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d41a7500 con 0x7f79d410a790 2023-09-30T14:53:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.241+0000 7f79db564700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d41a7670 con 0x7f79d410d160 2023-09-30T14:53:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.241+0000 7f79db564700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d41a7830 con 0x7f79d4107e20 2023-09-30T14:53:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.241+0000 7f79d9300700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d4107e20 0x7f79d41a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.241+0000 7f79d9300700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d4107e20 0x7f79d41a5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53110/0 (socket says 172.21.15.32:53110) 2023-09-30T14:53:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.241+0000 7f79d9300700 1 -- 172.21.15.32:0/2198571538 learned_addr learned my addr 172.21.15.32:0/2198571538 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.241+0000 7f79d9b01700 1 --2- 172.21.15.32:0/2198571538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d410d160 0x7f79d41a6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.242+0000 7f79d9b01700 1 -- 172.21.15.32:0/2198571538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d4107e20 msgr2=0x7f79d41a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.242+0000 7f79d9b01700 1 --2- 172.21.15.32:0/2198571538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d4107e20 0x7f79d41a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.242+0000 7f79d9b01700 1 -- 172.21.15.32:0/2198571538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d410a790 msgr2=0x7f79d41a6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.242+0000 7f79d9b01700 1 --2- 172.21.15.32:0/2198571538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d410a790 0x7f79d41a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.242+0000 7f79d9b01700 1 -- 172.21.15.32:0/2198571538 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c40095f0 con 0x7f79d410d160 2023-09-30T14:53:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.243+0000 7f79d9b01700 1 --2- 172.21.15.32:0/2198571538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d410d160 0x7f79d41a6ea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f79cc007f10 tx=0x7f79cc00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.244+0000 7f79ca7fc700 1 -- 172.21.15.32:0/2198571538 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc00ddd0 con 0x7f79d410d160 2023-09-30T14:53:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.244+0000 7f79db564700 1 -- 172.21.15.32:0/2198571538 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d41a7b10 con 0x7f79d410d160 2023-09-30T14:53:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.244+0000 7f79ca7fc700 1 -- 172.21.15.32:0/2198571538 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79cc00f040 con 0x7f79d410d160 2023-09-30T14:53:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.244+0000 7f79ca7fc700 1 -- 172.21.15.32:0/2198571538 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc005a90 con 0x7f79d410d160 2023-09-30T14:53:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.244+0000 7f79db564700 1 -- 172.21.15.32:0/2198571538 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d410f7b0 con 0x7f79d410d160 2023-09-30T14:53:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.248+0000 7f79ca7fc700 1 -- 172.21.15.32:0/2198571538 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f79cc004ad0 con 0x7f79d410d160 2023-09-30T14:53:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.249+0000 7f79e774cb80 1 Processor -- start 2023-09-30T14:53:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.250+0000 7f79e774cb80 1 -- start start 2023-09-30T14:53:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.250+0000 7f79ca7fc700 1 --2- 172.21.15.32:0/2198571538 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f79c00a1070 0x7f79c00a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.250+0000 7f79ca7fc700 1 -- 172.21.15.32:0/2198571538 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f79cc0cc030 con 0x7f79d410d160 2023-09-30T14:53:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.251+0000 7f79e774cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4084200 0x55f4b4084620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.251+0000 7f79e774cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4085310 0x55f4b4085790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.251+0000 7f79e774cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b4085cd0 0x55f4b408c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.251+0000 7f79e774cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b3fec400 con 0x55f4b4084200 2023-09-30T14:53:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.251+0000 7f79e774cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b3fddbc0 con 0x55f4b4085310 2023-09-30T14:53:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.251+0000 7f79e774cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b3fdb4e0 con 0x55f4b4085cd0 2023-09-30T14:53:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.251+0000 7f79d9300700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4084200 0x55f4b4084620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.251+0000 7f79d9300700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4084200 0x55f4b4084620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38734/0 (socket says 172.21.15.32:38734) 2023-09-30T14:53:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.251+0000 7f79d9300700 1 -- 172.21.15.32:0/636304345 learned_addr learned my addr 172.21.15.32:0/636304345 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.252+0000 7f79d9300700 1 -- 172.21.15.32:0/636304345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b4085cd0 msgr2=0x55f4b408c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.252+0000 7f79d9300700 1 --2- 172.21.15.32:0/636304345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b4085cd0 0x55f4b408c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.252+0000 7f79d9300700 1 -- 172.21.15.32:0/636304345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4085310 msgr2=0x55f4b4085790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-30T14:53:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.252+0000 7f79d9300700 1 --2- 172.21.15.32:0/636304345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4085310 0x55f4b4085790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.252+0000 7f79d9300700 1 -- 172.21.15.32:0/636304345 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4b4039870 con 0x55f4b4084200 2023-09-30T14:53:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.252+0000 7f79d9300700 1 --2- 172.21.15.32:0/636304345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4084200 0x55f4b4084620 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f79c4005c90 tx=0x7f79c400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6110e70bb557c25 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.253+0000 7f79bf7fe700 1 -- 172.21.15.32:0/636304345 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c401d050 con 0x55f4b4084200 2023-09-30T14:53:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.253+0000 7f79bf7fe700 1 -- 172.21.15.32:0/636304345 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79c401e040 con 0x55f4b4084200 2023-09-30T14:53:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.253+0000 7f79bf7fe700 1 -- 172.21.15.32:0/636304345 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c4023460 con 0x55f4b4084200 2023-09-30T14:53:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.254+0000 7f79e774cb80 1 -- 172.21.15.32:0/636304345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4084200 msgr2=0x55f4b4084620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.254+0000 7f79e774cb80 1 --2- 172.21.15.32:0/636304345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4084200 0x55f4b4084620 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f79c4005c90 tx=0x7f79c400be00 comp rx=0 tx=0).stop 2023-09-30T14:53:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.254+0000 7f79e774cb80 1 -- 172.21.15.32:0/636304345 shutdown_connections 2023-09-30T14:53:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.254+0000 7f79e774cb80 1 --2- 172.21.15.32:0/636304345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b4085cd0 0x55f4b408c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.254+0000 7f79e774cb80 1 --2- 172.21.15.32:0/636304345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4084200 0x55f4b4084620 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.254+0000 7f79e774cb80 1 --2- 172.21.15.32:0/636304345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4085310 0x55f4b4085790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.254+0000 7f79e774cb80 1 -- 172.21.15.32:0/636304345 >> 172.21.15.32:0/636304345 conn(0x55f4b3ff8200 msgr2=0x55f4b407eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.255+0000 7f79d8aff700 1 --2- 172.21.15.32:0/2198571538 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f79c00a1070 0x7f79c00a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.255+0000 7f79d8aff700 1 --2- 172.21.15.32:0/2198571538 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f79c00a1070 0x7f79c00a3530 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55f4b4080ad0 tx=0x7f79d0006da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.256+0000 7f79e774cb80 1 -- 172.21.15.32:0/636304345 shutdown_connections 2023-09-30T14:53:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.256+0000 7f79e774cb80 1 -- 172.21.15.32:0/636304345 wait complete. 2023-09-30T14:53:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.256+0000 7f79e774cb80 1 -- start start 2023-09-30T14:53:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.257+0000 7f79e774cb80 1 Processor -- start 2023-09-30T14:53:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.257+0000 7f79e774cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b4084200 0x55f4b408e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.257+0000 7f79e774cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4085cd0 0x55f4b408e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.257+0000 7f79e774cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b40935c0 0x55f4b40959b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.257+0000 7f79e774cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b3fd6dd0 con 0x55f4b4085cd0 2023-09-30T14:53:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.257+0000 7f79e774cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b3f736b0 con 0x55f4b40935c0 2023-09-30T14:53:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.257+0000 7f79e774cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b3f73350 con 0x55f4b4084200 2023-09-30T14:53:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.257+0000 7f79e774cb80 10 client.0 _ll_register_callbacks cb 0x55f4b3dd07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:53:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.257+0000 7f79d9300700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b4084200 0x55f4b408e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.257+0000 7f79d9300700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b4084200 0x55f4b408e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53124/0 (socket says 172.21.15.32:53124) 2023-09-30T14:53:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.257+0000 7f79d9300700 1 -- 172.21.15.32:0/3999085418 learned_addr learned my addr 172.21.15.32:0/3999085418 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.258+0000 7f79d8aff700 1 --2- 172.21.15.32:0/3999085418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4085cd0 0x55f4b408e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.258+0000 7f79d8aff700 1 -- 172.21.15.32:0/3999085418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b4084200 msgr2=0x55f4b408e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.258+0000 7f79d8aff700 1 --2- 172.21.15.32:0/3999085418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b4084200 0x55f4b408e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.258+0000 7f79d8aff700 1 -- 172.21.15.32:0/3999085418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b40935c0 msgr2=0x55f4b40959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.258+0000 7f79d8aff700 1 --2- 172.21.15.32:0/3999085418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b40935c0 0x55f4b40959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.258+0000 7f79d8aff700 1 -- 172.21.15.32:0/3999085418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d410e200 con 0x55f4b4085cd0 2023-09-30T14:53:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.258+0000 7f79d8aff700 1 --2- 172.21.15.32:0/3999085418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4085cd0 0x55f4b408e590 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x55f4b4080b00 tx=0x7f79d0004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3922e3805e81428d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.258+0000 7f79bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:53:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.258+0000 7f79bcff9700 1 -- 172.21.15.32:0/3999085418 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0004020 con 0x55f4b4085cd0 2023-09-30T14:53:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.258+0000 7f79e774cb80 1 -- 172.21.15.32:0/3999085418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4b4077090 con 0x55f4b4085cd0 2023-09-30T14:53:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.259+0000 7f79bcff9700 1 -- 172.21.15.32:0/3999085418 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f79d0004180 con 0x55f4b4085cd0 2023-09-30T14:53:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.260+0000 7f79bcff9700 1 -- 172.21.15.32:0/3999085418 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79d0004340 con 0x55f4b4085cd0 2023-09-30T14:53:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.260+0000 7f79e774cb80 10 client.16770 fetch_fsmap learned FSMap version 47 2023-09-30T14:53:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.260+0000 7f79e774cb80 1 -- 172.21.15.32:0/3999085418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55f4b3e01d80 con 0x55f4b4085cd0 2023-09-30T14:53:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.260+0000 7f79bcff9700 1 -- 172.21.15.32:0/3999085418 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79d002b020 con 0x55f4b4085cd0 2023-09-30T14:53:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.260+0000 7f79bcff9700 1 -- 172.21.15.32:0/3999085418 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 47) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f79d0003a70 con 0x55f4b4085cd0 2023-09-30T14:53:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.262+0000 7f79e774cb80 10 client.16770 fetch_fsmap finished waiting for FSMap version 47 2023-09-30T14:53:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.262+0000 7f79e774cb80 10 client.16770 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:53:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.262+0000 7f79e774cb80 20 client.16770 populate_metadata read hostname 'smithi032' 2023-09-30T14:53:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.262+0000 7f79e774cb80 1 --2- 172.21.15.32:0/3999085418 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f4b409a340 0x55f4b409c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.262+0000 7f79e774cb80 4 client.16770 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:53:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.262+0000 7f79e774cb80 1 -- 172.21.15.32:0/3999085418 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4b3fcfdb0 con 0x55f4b409a340 2023-09-30T14:53:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.266+0000 7f79d9300700 1 --2- 172.21.15.32:0/3999085418 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f4b409a340 0x55f4b409c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.268+0000 7f79bcff9700 10 client.16770 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.268+0000 7f79d9300700 1 --2- 172.21.15.32:0/3999085418 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f4b409a340 0x55f4b409c760 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4c7d55e03f58dcc server_cookie=c8645436441b57a6 in_seq=0 out_seq=0 2023-09-30T14:53:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.274+0000 7f79bcff9700 1 -- 172.21.15.32:0/3999085418 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f4b3fcfdb0 con 0x55f4b409a340 2023-09-30T14:53:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.274+0000 7f79bcff9700 10 client.16770 handle_command_reply: tid=0 2023-09-30T14:53:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.274+0000 7f79e774cb80 1 client.16770 shutdown 2023-09-30T14:53:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.274+0000 7f79e774cb80 1 -- 172.21.15.32:0/3999085418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4085cd0 msgr2=0x55f4b408e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.274+0000 7f79e774cb80 1 --2- 172.21.15.32:0/3999085418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4085cd0 0x55f4b408e590 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x55f4b4080b00 tx=0x7f79d0004800 comp rx=0 tx=0).stop 2023-09-30T14:53:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.274+0000 7f79e774cb80 1 -- 172.21.15.32:0/3999085418 shutdown_connections 2023-09-30T14:53:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.274+0000 7f79e774cb80 1 --2- 172.21.15.32:0/3999085418 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f4b409a340 0x55f4b409c760 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.274+0000 7f79e774cb80 1 --2- 172.21.15.32:0/3999085418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b40935c0 0x55f4b40959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.274+0000 7f79e774cb80 1 --2- 172.21.15.32:0/3999085418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b4084200 0x55f4b408e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.274+0000 7f79bcff9700 0 client.16770 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.274+0000 7f79e774cb80 1 --2- 172.21.15.32:0/3999085418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4085cd0 0x55f4b408e590 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.274+0000 7f79e774cb80 1 -- 172.21.15.32:0/3999085418 >> 172.21.15.32:0/3999085418 conn(0x55f4b3ff8200 msgr2=0x55f4b407e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.274+0000 7f79e774cb80 1 -- 172.21.15.32:0/3999085418 shutdown_connections 2023-09-30T14:53:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.274+0000 7f79e774cb80 1 -- 172.21.15.32:0/3999085418 wait complete. 2023-09-30T14:53:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.275+0000 7f79e774cb80 20 client.16770 trim_cache size 0 max 16384 2023-09-30T14:53:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.295+0000 7f79e774cb80 1 Processor -- start 2023-09-30T14:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.295+0000 7f79e774cb80 1 -- start start 2023-09-30T14:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.295+0000 7f79e774cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4084200 0x55f4b40845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.295+0000 7f79e774cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4085cd0 0x55f4b40a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.295+0000 7f79e774cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b40935c0 0x55f4b407e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.295+0000 7f79e774cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b3e578c0 con 0x55f4b4084200 2023-09-30T14:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.295+0000 7f79e774cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b4084f70 con 0x55f4b4085cd0 2023-09-30T14:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.295+0000 7f79e774cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b407fc10 con 0x55f4b40935c0 2023-09-30T14:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.295+0000 7f79d8aff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4085cd0 0x55f4b40a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.295+0000 7f79d9b01700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b40935c0 0x55f4b407e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.295+0000 7f79d8aff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4085cd0 0x55f4b40a9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:50228/0 (socket says 172.21.15.32:50228) 2023-09-30T14:53:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.295+0000 7f79d9300700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4084200 0x55f4b40845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.295+0000 7f79d8aff700 1 -- 172.21.15.32:0/3047857611 learned_addr learned my addr 172.21.15.32:0/3047857611 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.296+0000 7f79d9b01700 1 -- 172.21.15.32:0/3047857611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4085cd0 msgr2=0x55f4b40a9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.296+0000 7f79d9b01700 1 --2- 172.21.15.32:0/3047857611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4085cd0 0x55f4b40a9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.296+0000 7f79d9b01700 1 -- 172.21.15.32:0/3047857611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4084200 msgr2=0x55f4b40845e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.296+0000 7f79d9b01700 1 --2- 172.21.15.32:0/3047857611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4084200 0x55f4b40845e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.296+0000 7f79d9b01700 1 -- 172.21.15.32:0/3047857611 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4b3deac00 con 0x55f4b40935c0 2023-09-30T14:53:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.296+0000 7f79d8aff700 1 --2- 172.21.15.32:0/3047857611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4085cd0 0x55f4b40a9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:53:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.296+0000 7f79d9300700 1 --2- 172.21.15.32:0/3047857611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4084200 0x55f4b40845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:53:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.296+0000 7f79d9b01700 1 --2- 172.21.15.32:0/3047857611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b40935c0 0x55f4b407e590 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55f4b408f100 tx=0x7f79cc004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d601438a81f6343 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.296+0000 7f79bffff700 1 -- 172.21.15.32:0/3047857611 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc009e20 con 0x55f4b40935c0 2023-09-30T14:53:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.296+0000 7f79bffff700 1 -- 172.21.15.32:0/3047857611 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79cc099ba0 con 0x55f4b40935c0 2023-09-30T14:53:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.296+0000 7f79bffff700 1 -- 172.21.15.32:0/3047857611 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc099700 con 0x55f4b40935c0 2023-09-30T14:53:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.297+0000 7f79e774cb80 1 -- 172.21.15.32:0/3047857611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b40935c0 msgr2=0x55f4b407e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.297+0000 7f79e774cb80 1 --2- 172.21.15.32:0/3047857611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b40935c0 0x55f4b407e590 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55f4b408f100 tx=0x7f79cc004ef0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.297+0000 7f79e774cb80 1 -- 172.21.15.32:0/3047857611 shutdown_connections 2023-09-30T14:53:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.297+0000 7f79e774cb80 1 --2- 172.21.15.32:0/3047857611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b40935c0 0x55f4b407e590 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.297+0000 7f79e774cb80 1 --2- 172.21.15.32:0/3047857611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4084200 0x55f4b40845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.297+0000 7f79e774cb80 1 --2- 172.21.15.32:0/3047857611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4085cd0 0x55f4b40a9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.297+0000 7f79e774cb80 1 -- 172.21.15.32:0/3047857611 >> 172.21.15.32:0/3047857611 conn(0x55f4b3ff8200 msgr2=0x55f4b408c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.297+0000 7f79e774cb80 1 -- 172.21.15.32:0/3047857611 shutdown_connections 2023-09-30T14:53:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.297+0000 7f79e774cb80 1 -- 172.21.15.32:0/3047857611 wait complete. 2023-09-30T14:53:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.298+0000 7f79e774cb80 1 -- start start 2023-09-30T14:53:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.298+0000 7f79e774cb80 1 Processor -- start 2023-09-30T14:53:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.298+0000 7f79e774cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4084200 0x55f4b40e8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.298+0000 7f79e774cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4085cd0 0x55f4b40eadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.298+0000 7f79e774cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b40935c0 0x55f4b40ed310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.298+0000 7f79e774cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b40ed930 con 0x55f4b4085cd0 2023-09-30T14:53:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.298+0000 7f79e774cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b40edaa0 con 0x55f4b4084200 2023-09-30T14:53:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.298+0000 7f79e774cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4b40edc60 con 0x55f4b40935c0 2023-09-30T14:53:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.298+0000 7f79e774cb80 10 client.0 _ll_register_callbacks cb 0x7f79d0012580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:53:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.299+0000 7f79d8aff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4085cd0 0x55f4b40eadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.299+0000 7f79d9300700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4084200 0x55f4b40e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.299+0000 7f79d9300700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4084200 0x55f4b40e8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:50232/0 (socket says 172.21.15.32:50232) 2023-09-30T14:53:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.299+0000 7f79d9300700 1 -- 172.21.15.32:0/955118484 learned_addr learned my addr 172.21.15.32:0/955118484 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.299+0000 7f79d9b01700 1 --2- 172.21.15.32:0/955118484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b40935c0 0x55f4b40ed310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.299+0000 7f79d9b01700 1 -- 172.21.15.32:0/955118484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4084200 msgr2=0x55f4b40e8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.299+0000 7f79d9b01700 1 --2- 172.21.15.32:0/955118484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4084200 0x55f4b40e8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.299+0000 7f79d9b01700 1 -- 172.21.15.32:0/955118484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4085cd0 msgr2=0x55f4b40eadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.299+0000 7f79d9b01700 1 --2- 172.21.15.32:0/955118484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4085cd0 0x55f4b40eadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.299+0000 7f79d9b01700 1 -- 172.21.15.32:0/955118484 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4b4039870 con 0x55f4b40935c0 2023-09-30T14:53:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.299+0000 7f79d9b01700 1 --2- 172.21.15.32:0/955118484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b40935c0 0x55f4b40ed310 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55f4b407fd50 tx=0x7f79cc004d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e105b0c6f530dc9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.299+0000 7f79bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:53:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.300+0000 7f79bcff9700 1 -- 172.21.15.32:0/955118484 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc0994a0 con 0x55f4b40935c0 2023-09-30T14:53:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.300+0000 7f79e774cb80 1 -- 172.21.15.32:0/955118484 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79c401e040 con 0x55f4b40935c0 2023-09-30T14:53:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.300+0000 7f79bcff9700 1 -- 172.21.15.32:0/955118484 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f79cc0b1050 con 0x55f4b40935c0 2023-09-30T14:53:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.300+0000 7f79bcff9700 1 -- 172.21.15.32:0/955118484 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79cc009e20 con 0x55f4b40935c0 2023-09-30T14:53:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.300+0000 7f79e774cb80 10 client.26431 fetch_fsmap learned FSMap version 47 2023-09-30T14:53:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.300+0000 7f79e774cb80 1 -- 172.21.15.32:0/955118484 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55f4b3e01d80 con 0x55f4b40935c0 2023-09-30T14:53:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.300+0000 7f79bcff9700 1 -- 172.21.15.32:0/955118484 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79c401e040 con 0x55f4b40935c0 2023-09-30T14:53:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.300+0000 7f79bcff9700 1 -- 172.21.15.32:0/955118484 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 47) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f79cc09d120 con 0x55f4b40935c0 2023-09-30T14:53:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.300+0000 7f79e774cb80 10 client.26431 fetch_fsmap finished waiting for FSMap version 47 2023-09-30T14:53:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.300+0000 7f79e774cb80 10 client.26431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:53:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.300+0000 7f79e774cb80 20 client.26431 populate_metadata read hostname 'smithi032' 2023-09-30T14:53:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.300+0000 7f79e774cb80 1 --2- 172.21.15.32:0/955118484 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f4b40ef490 0x55f4b40f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.300+0000 7f79e774cb80 4 client.26431 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:53:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.300+0000 7f79e774cb80 1 -- 172.21.15.32:0/955118484 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f4b3fcfad0 con 0x55f4b40ef490 2023-09-30T14:53:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.301+0000 7f79d9300700 1 --2- 172.21.15.32:0/955118484 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f4b40ef490 0x55f4b40f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.301+0000 7f79bcff9700 10 client.26431 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.301+0000 7f79d9300700 1 --2- 172.21.15.32:0/955118484 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f4b40ef490 0x55f4b40f18b0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f34d901c852e0971 server_cookie=733ad737eb259ca1 in_seq=0 out_seq=0 2023-09-30T14:53:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.563+0000 7f79bcff9700 1 -- 172.21.15.32:0/955118484 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f4b3fcfad0 con 0x55f4b40ef490 2023-09-30T14:53:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.563+0000 7f79bcff9700 10 client.26431 handle_command_reply: tid=0 2023-09-30T14:53:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.563+0000 7f79e774cb80 1 client.26431 shutdown 2023-09-30T14:53:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.563+0000 7f79e774cb80 1 -- 172.21.15.32:0/955118484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b40935c0 msgr2=0x55f4b40ed310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.563+0000 7f79e774cb80 1 --2- 172.21.15.32:0/955118484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b40935c0 0x55f4b40ed310 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55f4b407fd50 tx=0x7f79cc004d80 comp rx=0 tx=0).stop 2023-09-30T14:53:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.563+0000 7f79e774cb80 1 -- 172.21.15.32:0/955118484 shutdown_connections 2023-09-30T14:53:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.563+0000 7f79e774cb80 1 --2- 172.21.15.32:0/955118484 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f4b40ef490 0x55f4b40f18b0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.563+0000 7f79e774cb80 1 --2- 172.21.15.32:0/955118484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4b40935c0 0x55f4b40ed310 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.563+0000 7f79e774cb80 1 --2- 172.21.15.32:0/955118484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4b4084200 0x55f4b40e8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.563+0000 7f79e774cb80 1 --2- 172.21.15.32:0/955118484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f4b4085cd0 0x55f4b40eadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.563+0000 7f79e774cb80 1 -- 172.21.15.32:0/955118484 >> 172.21.15.32:0/955118484 conn(0x55f4b3ff8200 msgr2=0x55f4b407e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.564+0000 7f79bcff9700 0 client.26431 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.568+0000 7f79e774cb80 1 -- 172.21.15.32:0/955118484 shutdown_connections 2023-09-30T14:53:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.568+0000 7f79e774cb80 1 -- 172.21.15.32:0/955118484 wait complete. 2023-09-30T14:53:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.568+0000 7f79e774cb80 20 client.26431 trim_cache size 0 max 16384 2023-09-30T14:53:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:53:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:53:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:53:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:53:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.569+0000 7f79db564700 1 -- 172.21.15.32:0/2198571538 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f79c00a1070 msgr2=0x7f79c00a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.569+0000 7f79db564700 1 --2- 172.21.15.32:0/2198571538 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f79c00a1070 0x7f79c00a3530 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55f4b4080ad0 tx=0x7f79d0006da0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.569+0000 7f79db564700 1 -- 172.21.15.32:0/2198571538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d410d160 msgr2=0x7f79d41a6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.569+0000 7f79db564700 1 --2- 172.21.15.32:0/2198571538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d410d160 0x7f79d41a6ea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f79cc007f10 tx=0x7f79cc00d3b0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.569+0000 7f79db564700 1 -- 172.21.15.32:0/2198571538 shutdown_connections 2023-09-30T14:53:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.569+0000 7f79db564700 1 --2- 172.21.15.32:0/2198571538 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f79c00a1070 0x7f79c00a3530 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.569+0000 7f79db564700 1 --2- 172.21.15.32:0/2198571538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d410d160 0x7f79d41a6ea0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.569+0000 7f79db564700 1 --2- 172.21.15.32:0/2198571538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79d4107e20 0x7f79d41a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.569+0000 7f79db564700 1 --2- 172.21.15.32:0/2198571538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79d410a790 0x7f79d41a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.569+0000 7f79db564700 1 -- 172.21.15.32:0/2198571538 >> 172.21.15.32:0/2198571538 conn(0x7f79d406dc80 msgr2=0x7f79d410c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.569+0000 7f79db564700 1 -- 172.21.15.32:0/2198571538 shutdown_connections 2023-09-30T14:53:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:45.569+0000 7f79db564700 1 -- 172.21.15.32:0/2198571538 wait complete. 2023-09-30T14:53:45.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:45 smithi032 ceph-mon[98592]: pgmap v751: 97 pgs: 97 active+clean; 677 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 819 op/s 2023-09-30T14:53:46.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92247 39.52 MB/sec execute 19 sec latency 177.903 ms 2023-09-30T14:53:46.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:45 smithi156 ceph-mon[103579]: pgmap v751: 97 pgs: 97 active+clean; 677 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 819 op/s 2023-09-30T14:53:46.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:45 smithi148 ceph-mon[102355]: pgmap v751: 97 pgs: 97 active+clean; 677 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 819 op/s 2023-09-30T14:53:47.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 92904 39.34 MB/sec execute 20 sec latency 44.626 ms 2023-09-30T14:53:48.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 93269 38.25 MB/sec execute 21 sec latency 363.606 ms 2023-09-30T14:53:48.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:47 smithi156 ceph-mon[103579]: pgmap v752: 97 pgs: 97 active+clean; 682 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 855 op/s 2023-09-30T14:53:48.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:47 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:53:48.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:47 smithi148 ceph-mon[102355]: pgmap v752: 97 pgs: 97 active+clean; 682 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 855 op/s 2023-09-30T14:53:48.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:47 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:53:48.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:47 smithi032 ceph-mon[98592]: pgmap v752: 97 pgs: 97 active+clean; 682 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 855 op/s 2023-09-30T14:53:48.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:47 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:53:49.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 93963 37.88 MB/sec execute 22 sec latency 93.713 ms 2023-09-30T14:53:50.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 95152 39.01 MB/sec execute 23 sec latency 39.609 ms 2023-09-30T14:53:50.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:49 smithi156 ceph-mon[103579]: pgmap v753: 97 pgs: 97 active+clean; 677 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 11 MiB/s wr, 863 op/s 2023-09-30T14:53:50.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:49 smithi148 ceph-mon[102355]: pgmap v753: 97 pgs: 97 active+clean; 677 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 11 MiB/s wr, 863 op/s 2023-09-30T14:53:50.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:49 smithi032 ceph-mon[98592]: pgmap v753: 97 pgs: 97 active+clean; 677 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 11 MiB/s wr, 863 op/s 2023-09-30T14:53:51.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 95986 40.10 MB/sec execute 24 sec latency 52.118 ms 2023-09-30T14:53:52.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 96441 39.27 MB/sec execute 25 sec latency 61.382 ms 2023-09-30T14:53:52.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:51 smithi156 ceph-mon[103579]: pgmap v754: 97 pgs: 97 active+clean; 681 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 11 MiB/s wr, 771 op/s 2023-09-30T14:53:52.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:51 smithi148 ceph-mon[102355]: pgmap v754: 97 pgs: 97 active+clean; 681 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 11 MiB/s wr, 771 op/s 2023-09-30T14:53:52.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:51 smithi032 ceph-mon[98592]: pgmap v754: 97 pgs: 97 active+clean; 681 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 11 MiB/s wr, 771 op/s 2023-09-30T14:53:53.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 96861 38.16 MB/sec execute 26 sec latency 208.005 ms 2023-09-30T14:53:54.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 97616 38.46 MB/sec execute 27 sec latency 57.601 ms 2023-09-30T14:53:54.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:53 smithi156 ceph-mon[103579]: pgmap v755: 97 pgs: 97 active+clean; 688 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 843 op/s 2023-09-30T14:53:54.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:53 smithi148 ceph-mon[102355]: pgmap v755: 97 pgs: 97 active+clean; 688 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 843 op/s 2023-09-30T14:53:54.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:53 smithi032 ceph-mon[98592]: pgmap v755: 97 pgs: 97 active+clean; 688 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 843 op/s 2023-09-30T14:53:55.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 98620 38.35 MB/sec execute 28 sec latency 98.412 ms 2023-09-30T14:53:56.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 99610 39.51 MB/sec execute 29 sec latency 70.121 ms 2023-09-30T14:53:56.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:55 smithi156 ceph-mon[103579]: pgmap v756: 97 pgs: 97 active+clean; 684 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 12 MiB/s wr, 894 op/s 2023-09-30T14:53:56.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:55 smithi148 ceph-mon[102355]: pgmap v756: 97 pgs: 97 active+clean; 684 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 12 MiB/s wr, 894 op/s 2023-09-30T14:53:56.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:55 smithi032 ceph-mon[98592]: pgmap v756: 97 pgs: 97 active+clean; 684 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 12 MiB/s wr, 894 op/s 2023-09-30T14:53:57.010 INFO:tasks.workunit.client.0.smithi032.stdout: 10 100345 39.88 MB/sec execute 30 sec latency 63.645 ms 2023-09-30T14:53:58.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 100644 38.92 MB/sec execute 31 sec latency 207.953 ms 2023-09-30T14:53:58.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:57 smithi156 ceph-mon[103579]: pgmap v757: 97 pgs: 97 active+clean; 674 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 858 op/s 2023-09-30T14:53:58.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:57 smithi148 ceph-mon[102355]: pgmap v757: 97 pgs: 97 active+clean; 674 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 858 op/s 2023-09-30T14:53:58.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:57 smithi032 ceph-mon[98592]: pgmap v757: 97 pgs: 97 active+clean; 674 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 858 op/s 2023-09-30T14:53:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.712+0000 7f72272f9700 1 -- 172.21.15.32:0/2480773352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72200feeb0 msgr2=0x7f72200ff310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.712+0000 7f72272f9700 1 --2- 172.21.15.32:0/2480773352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72200feeb0 0x7f72200ff310 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7210009990 tx=0x7f7210009ca0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.712+0000 7f72272f9700 1 -- 172.21.15.32:0/2480773352 shutdown_connections 2023-09-30T14:53:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.712+0000 7f72272f9700 1 --2- 172.21.15.32:0/2480773352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72200ff850 0x7f7220109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.712+0000 7f72272f9700 1 --2- 172.21.15.32:0/2480773352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7220102440 0x7f7220102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.712+0000 7f72272f9700 1 --2- 172.21.15.32:0/2480773352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72200feeb0 0x7f72200ff310 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.712+0000 7f72272f9700 1 -- 172.21.15.32:0/2480773352 >> 172.21.15.32:0/2480773352 conn(0x7f72200faa30 msgr2=0x7f72200fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.712+0000 7f72272f9700 1 -- 172.21.15.32:0/2480773352 shutdown_connections 2023-09-30T14:53:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.712+0000 7f72272f9700 1 -- 172.21.15.32:0/2480773352 wait complete. 2023-09-30T14:53:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.713+0000 7f72272f9700 1 Processor -- start 2023-09-30T14:53:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.713+0000 7f72272f9700 1 -- start start 2023-09-30T14:53:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.713+0000 7f72272f9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72200feeb0 0x7f7220196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.713+0000 7f72272f9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72200ff850 0x7f7220197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.713+0000 7f72272f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7220102440 0x7f7220197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.713+0000 7f72272f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7220198660 con 0x7f7220102440 2023-09-30T14:53:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.713+0000 7f72272f9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72201987d0 con 0x7f72200ff850 2023-09-30T14:53:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.713+0000 7f72272f9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7220198990 con 0x7f72200feeb0 2023-09-30T14:53:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.713+0000 7f7225896700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7220102440 0x7f7220197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.713+0000 7f7225095700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72200feeb0 0x7f7220196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.713+0000 7f7225896700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7220102440 0x7f7220197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38774/0 (socket says 172.21.15.32:38774) 2023-09-30T14:53:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.713+0000 7f7225896700 1 -- 172.21.15.32:0/3525256173 learned_addr learned my addr 172.21.15.32:0/3525256173 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.714+0000 7f7225896700 1 -- 172.21.15.32:0/3525256173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72200feeb0 msgr2=0x7f7220196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.714+0000 7f7225896700 1 --2- 172.21.15.32:0/3525256173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72200feeb0 0x7f7220196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.714+0000 7f7225896700 1 -- 172.21.15.32:0/3525256173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72200ff850 msgr2=0x7f7220197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.714+0000 7f7225896700 1 --2- 172.21.15.32:0/3525256173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72200ff850 0x7f7220197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.714+0000 7f7225896700 1 -- 172.21.15.32:0/3525256173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72100095f0 con 0x7f7220102440 2023-09-30T14:53:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.714+0000 7f7225896700 1 --2- 172.21.15.32:0/3525256173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7220102440 0x7f7220197f60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f721800ada0 tx=0x7f721800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.714+0000 7f72167fc700 1 -- 172.21.15.32:0/3525256173 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721800f070 con 0x7f7220102440 2023-09-30T14:53:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.714+0000 7f72167fc700 1 -- 172.21.15.32:0/3525256173 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f721800ce40 con 0x7f7220102440 2023-09-30T14:53:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.714+0000 7f72272f9700 1 -- 172.21.15.32:0/3525256173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7220198c70 con 0x7f7220102440 2023-09-30T14:53:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.714+0000 7f72167fc700 1 -- 172.21.15.32:0/3525256173 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218005460 con 0x7f7220102440 2023-09-30T14:53:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.714+0000 7f72272f9700 1 -- 172.21.15.32:0/3525256173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72201001a0 con 0x7f7220102440 2023-09-30T14:53:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.717+0000 7f72167fc700 1 -- 172.21.15.32:0/3525256173 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f721800c980 con 0x7f7220102440 2023-09-30T14:53:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.717+0000 7f72167fc700 1 --2- 172.21.15.32:0/3525256173 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f720c0a0fa0 0x7f720c0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.718+0000 7f72167fc700 1 -- 172.21.15.32:0/3525256173 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f72180cbb90 con 0x7f7220102440 2023-09-30T14:53:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.718+0000 7f7225095700 1 --2- 172.21.15.32:0/3525256173 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f720c0a0fa0 0x7f720c0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.718+0000 7f7225095700 1 --2- 172.21.15.32:0/3525256173 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f720c0a0fa0 0x7f720c0a3460 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7220067f60 tx=0x7f721c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.719+0000 7f72334e1b80 1 Processor -- start 2023-09-30T14:53:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.719+0000 7f72334e1b80 1 -- start start 2023-09-30T14:53:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.719+0000 7f72334e1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c7200 0x55acf50c7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.719+0000 7f72334e1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8310 0x55acf50c8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.719+0000 7f72334e1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c8cd0 0x55acf50cf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.719+0000 7f72334e1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf502f400 con 0x55acf50c8310 2023-09-30T14:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.719+0000 7f72334e1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf5020bc0 con 0x55acf50c8cd0 2023-09-30T14:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.719+0000 7f72334e1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf501e4e0 con 0x55acf50c7200 2023-09-30T14:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.719+0000 7f7224894700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8310 0x55acf50c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.719+0000 7f7224894700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8310 0x55acf50c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38780/0 (socket says 172.21.15.32:38780) 2023-09-30T14:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.719+0000 7f7224894700 1 -- 172.21.15.32:0/3029114686 learned_addr learned my addr 172.21.15.32:0/3029114686 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.719+0000 7f7225095700 1 --2- 172.21.15.32:0/3029114686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c7200 0x55acf50c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.720+0000 7f7225896700 1 --2- 172.21.15.32:0/3029114686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c8cd0 0x55acf50cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.720+0000 7f7224894700 1 -- 172.21.15.32:0/3029114686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c7200 msgr2=0x55acf50c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.720+0000 7f7224894700 1 --2- 172.21.15.32:0/3029114686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c7200 0x55acf50c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.720+0000 7f7224894700 1 -- 172.21.15.32:0/3029114686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c8cd0 msgr2=0x55acf50cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.720+0000 7f7224894700 1 --2- 172.21.15.32:0/3029114686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c8cd0 0x55acf50cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.720+0000 7f7224894700 1 -- 172.21.15.32:0/3029114686 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acf507c870 con 0x55acf50c8310 2023-09-30T14:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.720+0000 7f7225896700 1 --2- 172.21.15.32:0/3029114686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c8cd0 0x55acf50cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.720+0000 7f7224894700 1 --2- 172.21.15.32:0/3029114686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8310 0x55acf50c8790 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7210006010 tx=0x7f7210005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b2a342712fa48f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.720+0000 7f720b7fe700 1 -- 172.21.15.32:0/3029114686 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721001d070 con 0x55acf50c8310 2023-09-30T14:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.720+0000 7f720b7fe700 1 -- 172.21.15.32:0/3029114686 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7210004cd0 con 0x55acf50c8310 2023-09-30T14:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.720+0000 7f720b7fe700 1 -- 172.21.15.32:0/3029114686 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210005290 con 0x55acf50c8310 2023-09-30T14:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.721+0000 7f72334e1b80 1 -- 172.21.15.32:0/3029114686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8310 msgr2=0x55acf50c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.721+0000 7f72334e1b80 1 --2- 172.21.15.32:0/3029114686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8310 0x55acf50c8790 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7210006010 tx=0x7f7210005da0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.721+0000 7f72334e1b80 1 -- 172.21.15.32:0/3029114686 shutdown_connections 2023-09-30T14:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.721+0000 7f72334e1b80 1 --2- 172.21.15.32:0/3029114686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c8cd0 0x55acf50cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.721+0000 7f72334e1b80 1 --2- 172.21.15.32:0/3029114686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c7200 0x55acf50c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.721+0000 7f72334e1b80 1 --2- 172.21.15.32:0/3029114686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8310 0x55acf50c8790 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.721+0000 7f72334e1b80 1 -- 172.21.15.32:0/3029114686 >> 172.21.15.32:0/3029114686 conn(0x55acf503b200 msgr2=0x55acf50c1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.721+0000 7f72334e1b80 1 -- 172.21.15.32:0/3029114686 shutdown_connections 2023-09-30T14:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.721+0000 7f72334e1b80 1 -- 172.21.15.32:0/3029114686 wait complete. 2023-09-30T14:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.722+0000 7f72334e1b80 1 -- start start 2023-09-30T14:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.722+0000 7f72334e1b80 1 Processor -- start 2023-09-30T14:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.722+0000 7f72334e1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c7200 0x55acf50d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.722+0000 7f72334e1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c8310 0x55acf50d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.722+0000 7f72334e1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8cd0 0x55acf50d8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.722+0000 7f72334e1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf5019dd0 con 0x55acf50c8cd0 2023-09-30T14:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.722+0000 7f72334e1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf4fb66b0 con 0x55acf50c7200 2023-09-30T14:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.722+0000 7f72334e1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf4fb6350 con 0x55acf50c8310 2023-09-30T14:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f72334e1b80 10 client.0 _ll_register_callbacks cb 0x55acf4e137a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:53:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f7225896700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8cd0 0x55acf50d8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f7225896700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8cd0 0x55acf50d8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38788/0 (socket says 172.21.15.32:38788) 2023-09-30T14:53:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f7224894700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c8310 0x55acf50d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f7225896700 1 -- 172.21.15.32:0/29894996 learned_addr learned my addr 172.21.15.32:0/29894996 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f7225095700 1 --2- 172.21.15.32:0/29894996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c7200 0x55acf50d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f7225896700 1 -- 172.21.15.32:0/29894996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c8310 msgr2=0x55acf50d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f7225896700 1 --2- 172.21.15.32:0/29894996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c8310 0x55acf50d13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f7225896700 1 -- 172.21.15.32:0/29894996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c7200 msgr2=0x55acf50d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f7225896700 1 --2- 172.21.15.32:0/29894996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c7200 0x55acf50d0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f7225896700 1 -- 172.21.15.32:0/29894996 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7220109be0 con 0x55acf50c8cd0 2023-09-30T14:53:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f7224894700 1 --2- 172.21.15.32:0/29894996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c8310 0x55acf50d13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:53:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f7225896700 1 --2- 172.21.15.32:0/29894996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8cd0 0x55acf50d8400 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f721800c730 tx=0x7f7218099170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f8b0e5c669e23c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f7208ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:53:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f7208ff9700 1 -- 172.21.15.32:0/29894996 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218097490 con 0x55acf50c8cd0 2023-09-30T14:53:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f7208ff9700 1 -- 172.21.15.32:0/29894996 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7218004ad0 con 0x55acf50c8cd0 2023-09-30T14:53:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f72334e1b80 1 -- 172.21.15.32:0/29894996 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acf50ba090 con 0x55acf50c8cd0 2023-09-30T14:53:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f7208ff9700 1 -- 172.21.15.32:0/29894996 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7218099350 con 0x55acf50c8cd0 2023-09-30T14:53:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.723+0000 7f72334e1b80 10 client.16785 fetch_fsmap learned FSMap version 48 2023-09-30T14:53:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.724+0000 7f72334e1b80 1 -- 172.21.15.32:0/29894996 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55acf4e44d80 con 0x55acf50c8cd0 2023-09-30T14:53:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.724+0000 7f7208ff9700 1 -- 172.21.15.32:0/29894996 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55acf50ba090 con 0x55acf50c8cd0 2023-09-30T14:53:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.724+0000 7f7208ff9700 1 -- 172.21.15.32:0/29894996 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 48) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f72180985e0 con 0x55acf50c8cd0 2023-09-30T14:53:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.724+0000 7f72334e1b80 10 client.16785 fetch_fsmap finished waiting for FSMap version 48 2023-09-30T14:53:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.724+0000 7f72334e1b80 10 client.16785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:53:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.724+0000 7f72334e1b80 20 client.16785 populate_metadata read hostname 'smithi032' 2023-09-30T14:53:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.724+0000 7f72334e1b80 1 --2- 172.21.15.32:0/29894996 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55acf50dcb70 0x55acf50def60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.724+0000 7f72334e1b80 4 client.16785 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:53:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.724+0000 7f72334e1b80 1 -- 172.21.15.32:0/29894996 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acf5012db0 con 0x55acf50dcb70 2023-09-30T14:53:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.725+0000 7f7225095700 1 --2- 172.21.15.32:0/29894996 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55acf50dcb70 0x55acf50def60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.726+0000 7f7208ff9700 10 client.16785 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.726+0000 7f7225095700 1 --2- 172.21.15.32:0/29894996 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55acf50dcb70 0x55acf50def60 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b00b61f1366b646e server_cookie=d75eb9b63a6d7fa9 in_seq=0 out_seq=0 2023-09-30T14:53:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.735+0000 7f7208ff9700 1 -- 172.21.15.32:0/29894996 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55acf5012db0 con 0x55acf50dcb70 2023-09-30T14:53:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.735+0000 7f7208ff9700 10 client.16785 handle_command_reply: tid=0 2023-09-30T14:53:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.735+0000 7f72334e1b80 1 client.16785 shutdown 2023-09-30T14:53:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.735+0000 7f72334e1b80 1 -- 172.21.15.32:0/29894996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8cd0 msgr2=0x55acf50d8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.735+0000 7f72334e1b80 1 --2- 172.21.15.32:0/29894996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8cd0 0x55acf50d8400 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f721800c730 tx=0x7f7218099170 comp rx=0 tx=0).stop 2023-09-30T14:53:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.735+0000 7f72334e1b80 1 -- 172.21.15.32:0/29894996 shutdown_connections 2023-09-30T14:53:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.735+0000 7f72334e1b80 1 --2- 172.21.15.32:0/29894996 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55acf50dcb70 0x55acf50def60 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.735+0000 7f72334e1b80 1 --2- 172.21.15.32:0/29894996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8cd0 0x55acf50d8400 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.735+0000 7f72334e1b80 1 --2- 172.21.15.32:0/29894996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c7200 0x55acf50d0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.735+0000 7f7208ff9700 0 client.16785 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.735+0000 7f72334e1b80 1 --2- 172.21.15.32:0/29894996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c8310 0x55acf50d13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.735+0000 7f72334e1b80 1 -- 172.21.15.32:0/29894996 >> 172.21.15.32:0/29894996 conn(0x55acf503b200 msgr2=0x55acf50c1590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.736+0000 7f72334e1b80 1 -- 172.21.15.32:0/29894996 shutdown_connections 2023-09-30T14:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.736+0000 7f72334e1b80 1 -- 172.21.15.32:0/29894996 wait complete. 2023-09-30T14:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.736+0000 7f72334e1b80 20 client.16785 trim_cache size 0 max 16384 2023-09-30T14:53:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.754+0000 7f72334e1b80 1 Processor -- start 2023-09-30T14:53:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.754+0000 7f72334e1b80 1 -- start start 2023-09-30T14:53:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.754+0000 7f72334e1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c7200 0x55acf50cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.754+0000 7f72334e1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8310 0x55acf50d83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.754+0000 7f72334e1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c8cd0 0x55acf50d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.754+0000 7f72334e1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf4e9a8c0 con 0x55acf50c8310 2023-09-30T14:53:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.754+0000 7f72334e1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf50da060 con 0x55acf50c8cd0 2023-09-30T14:53:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.754+0000 7f72334e1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf50d0330 con 0x55acf50c7200 2023-09-30T14:53:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.754+0000 7f7224894700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8310 0x55acf50d83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.754+0000 7f7224894700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8310 0x55acf50d83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38794/0 (socket says 172.21.15.32:38794) 2023-09-30T14:53:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.754+0000 7f7224894700 1 -- 172.21.15.32:0/1366793277 learned_addr learned my addr 172.21.15.32:0/1366793277 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.754+0000 7f7225095700 1 --2- 172.21.15.32:0/1366793277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c7200 0x55acf50cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.755+0000 7f7224894700 1 -- 172.21.15.32:0/1366793277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c7200 msgr2=0x55acf50cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.755+0000 7f7224894700 1 --2- 172.21.15.32:0/1366793277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c7200 0x55acf50cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.755+0000 7f7224894700 1 -- 172.21.15.32:0/1366793277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c8cd0 msgr2=0x55acf50d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.755+0000 7f7224894700 1 --2- 172.21.15.32:0/1366793277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c8cd0 0x55acf50d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.755+0000 7f7224894700 1 -- 172.21.15.32:0/1366793277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acf4e2dc00 con 0x55acf50c8310 2023-09-30T14:53:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.755+0000 7f7224894700 1 --2- 172.21.15.32:0/1366793277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8310 0x55acf50d83c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f72100098e0 tx=0x7f721000bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353e6b885512a8df server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.755+0000 7f720bfff700 1 -- 172.21.15.32:0/1366793277 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721001d040 con 0x55acf50c8310 2023-09-30T14:53:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.755+0000 7f720bfff700 1 -- 172.21.15.32:0/1366793277 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7210004e40 con 0x55acf50c8310 2023-09-30T14:53:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.755+0000 7f720bfff700 1 -- 172.21.15.32:0/1366793277 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721000f6c0 con 0x55acf50c8310 2023-09-30T14:53:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.756+0000 7f72334e1b80 1 -- 172.21.15.32:0/1366793277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8310 msgr2=0x55acf50d83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.756+0000 7f72334e1b80 1 --2- 172.21.15.32:0/1366793277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8310 0x55acf50d83c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f72100098e0 tx=0x7f721000bb60 comp rx=0 tx=0).stop 2023-09-30T14:53:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.756+0000 7f72334e1b80 1 -- 172.21.15.32:0/1366793277 shutdown_connections 2023-09-30T14:53:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.756+0000 7f72334e1b80 1 --2- 172.21.15.32:0/1366793277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c8cd0 0x55acf50d0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.756+0000 7f72334e1b80 1 --2- 172.21.15.32:0/1366793277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c7200 0x55acf50cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.756+0000 7f72334e1b80 1 --2- 172.21.15.32:0/1366793277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c8310 0x55acf50d83c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.756+0000 7f72334e1b80 1 -- 172.21.15.32:0/1366793277 >> 172.21.15.32:0/1366793277 conn(0x55acf503b200 msgr2=0x55acf50cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.757+0000 7f72334e1b80 1 -- 172.21.15.32:0/1366793277 shutdown_connections 2023-09-30T14:53:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.757+0000 7f72334e1b80 1 -- 172.21.15.32:0/1366793277 wait complete. 2023-09-30T14:53:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.757+0000 7f72334e1b80 1 -- start start 2023-09-30T14:53:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.757+0000 7f72334e1b80 1 Processor -- start 2023-09-30T14:53:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f72334e1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c7200 0x55acf50d5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f72334e1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c8310 0x55acf50d6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f72334e1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c8cd0 0x55acf50d7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f72334e1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf50d7970 con 0x55acf50c7200 2023-09-30T14:53:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f72334e1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf50d7ae0 con 0x55acf50c8cd0 2023-09-30T14:53:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f72334e1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf50d7ca0 con 0x55acf50c8310 2023-09-30T14:53:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f72334e1b80 10 client.0 _ll_register_callbacks cb 0x7f7218098a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:53:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f7225095700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c7200 0x55acf50d5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f7225095700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c7200 0x55acf50d5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38798/0 (socket says 172.21.15.32:38798) 2023-09-30T14:53:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f7225095700 1 -- 172.21.15.32:0/1901676404 learned_addr learned my addr 172.21.15.32:0/1901676404 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:53:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f7224894700 1 --2- 172.21.15.32:0/1901676404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c8310 0x55acf50d6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f7225896700 1 --2- 172.21.15.32:0/1901676404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c8cd0 0x55acf50d7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f7225095700 1 -- 172.21.15.32:0/1901676404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c8310 msgr2=0x55acf50d6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f7225095700 1 --2- 172.21.15.32:0/1901676404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c8310 0x55acf50d6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f7225095700 1 -- 172.21.15.32:0/1901676404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c8cd0 msgr2=0x55acf50d7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:53:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f7225095700 1 --2- 172.21.15.32:0/1901676404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c8cd0 0x55acf50d7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f7225095700 1 -- 172.21.15.32:0/1901676404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acf507c870 con 0x55acf50c7200 2023-09-30T14:53:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f7225095700 1 --2- 172.21.15.32:0/1901676404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c7200 0x55acf50d5ee0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f721c0094f0 tx=0x7f721c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7d10c4bbbf1a9a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:53:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f7208ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:53:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.758+0000 7f7208ff9700 1 -- 172.21.15.32:0/1901676404 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c01ec10 con 0x55acf50c7200 2023-09-30T14:53:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.759+0000 7f7208ff9700 1 -- 172.21.15.32:0/1901676404 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f721c01ed70 con 0x55acf50c7200 2023-09-30T14:53:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.759+0000 7f72334e1b80 1 -- 172.21.15.32:0/1901676404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7210004cd0 con 0x55acf50c7200 2023-09-30T14:53:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.759+0000 7f7208ff9700 1 -- 172.21.15.32:0/1901676404 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f721c004020 con 0x55acf50c7200 2023-09-30T14:53:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.759+0000 7f72334e1b80 10 client.16791 fetch_fsmap learned FSMap version 48 2023-09-30T14:53:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.759+0000 7f72334e1b80 1 -- 172.21.15.32:0/1901676404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55acf4e6b1f0 con 0x55acf50c7200 2023-09-30T14:53:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.759+0000 7f7208ff9700 1 -- 172.21.15.32:0/1901676404 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7210004cd0 con 0x55acf50c7200 2023-09-30T14:53:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.759+0000 7f7208ff9700 1 -- 172.21.15.32:0/1901676404 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 48) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f721c0323f0 con 0x55acf50c7200 2023-09-30T14:53:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.759+0000 7f72334e1b80 10 client.16791 fetch_fsmap finished waiting for FSMap version 48 2023-09-30T14:53:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.759+0000 7f72334e1b80 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:53:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.759+0000 7f72334e1b80 20 client.16791 populate_metadata read hostname 'smithi032' 2023-09-30T14:53:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.759+0000 7f72334e1b80 1 --2- 172.21.15.32:0/1901676404 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55acf5131700 0x55acf5133ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:53:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.759+0000 7f72334e1b80 4 client.16791 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:53:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.759+0000 7f72334e1b80 1 -- 172.21.15.32:0/1901676404 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55acf5012ad0 con 0x55acf5131700 2023-09-30T14:53:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.760+0000 7f7224894700 1 --2- 172.21.15.32:0/1901676404 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55acf5131700 0x55acf5133ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:53:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.760+0000 7f7208ff9700 10 client.16791 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:58.760+0000 7f7224894700 1 --2- 172.21.15.32:0/1901676404 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55acf5131700 0x55acf5133ba0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b389bdbd6210c5b server_cookie=6bda017c3bc898b7 in_seq=0 out_seq=0 2023-09-30T14:53:59.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 101149 38.31 MB/sec execute 32 sec latency 200.960 ms 2023-09-30T14:53:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.035+0000 7f7208ff9700 1 -- 172.21.15.32:0/1901676404 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55acf5012ad0 con 0x55acf5131700 2023-09-30T14:53:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.035+0000 7f7208ff9700 10 client.16791 handle_command_reply: tid=0 2023-09-30T14:53:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.035+0000 7f72334e1b80 1 client.16791 shutdown 2023-09-30T14:53:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.035+0000 7f72334e1b80 1 -- 172.21.15.32:0/1901676404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c7200 msgr2=0x55acf50d5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.035+0000 7f72334e1b80 1 --2- 172.21.15.32:0/1901676404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c7200 0x55acf50d5ee0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f721c0094f0 tx=0x7f721c0047c0 comp rx=0 tx=0).stop 2023-09-30T14:53:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.035+0000 7f72334e1b80 1 -- 172.21.15.32:0/1901676404 shutdown_connections 2023-09-30T14:53:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.035+0000 7f72334e1b80 1 --2- 172.21.15.32:0/1901676404 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55acf5131700 0x55acf5133ba0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.035+0000 7f72334e1b80 1 --2- 172.21.15.32:0/1901676404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acf50c8cd0 0x55acf50d7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.035+0000 7f7208ff9700 0 client.16791 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:53:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.035+0000 7f72334e1b80 1 --2- 172.21.15.32:0/1901676404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55acf50c7200 0x55acf50d5ee0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.035+0000 7f72334e1b80 1 --2- 172.21.15.32:0/1901676404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf50c8310 0x55acf50d6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.035+0000 7f72334e1b80 1 -- 172.21.15.32:0/1901676404 >> 172.21.15.32:0/1901676404 conn(0x55acf503b200 msgr2=0x55acf50cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.035+0000 7f72334e1b80 1 -- 172.21.15.32:0/1901676404 shutdown_connections 2023-09-30T14:53:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.035+0000 7f72334e1b80 1 -- 172.21.15.32:0/1901676404 wait complete. 2023-09-30T14:53:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.036+0000 7f72334e1b80 20 client.16791 trim_cache size 0 max 16384 2023-09-30T14:53:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:53:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:53:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:53:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:53:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.036+0000 7f72272f9700 1 -- 172.21.15.32:0/3525256173 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f720c0a0fa0 msgr2=0x7f720c0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.036+0000 7f72272f9700 1 --2- 172.21.15.32:0/3525256173 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f720c0a0fa0 0x7f720c0a3460 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7220067f60 tx=0x7f721c009260 comp rx=0 tx=0).stop 2023-09-30T14:53:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.036+0000 7f72272f9700 1 -- 172.21.15.32:0/3525256173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7220102440 msgr2=0x7f7220197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:53:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.036+0000 7f72272f9700 1 --2- 172.21.15.32:0/3525256173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7220102440 0x7f7220197f60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f721800ada0 tx=0x7f721800c330 comp rx=0 tx=0).stop 2023-09-30T14:53:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.036+0000 7f72272f9700 1 -- 172.21.15.32:0/3525256173 shutdown_connections 2023-09-30T14:53:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.036+0000 7f72272f9700 1 --2- 172.21.15.32:0/3525256173 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f720c0a0fa0 0x7f720c0a3460 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.036+0000 7f72272f9700 1 --2- 172.21.15.32:0/3525256173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7220102440 0x7f7220197f60 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.036+0000 7f72272f9700 1 --2- 172.21.15.32:0/3525256173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72200feeb0 0x7f7220196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.036+0000 7f72272f9700 1 --2- 172.21.15.32:0/3525256173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72200ff850 0x7f7220197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:53:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.036+0000 7f72272f9700 1 -- 172.21.15.32:0/3525256173 >> 172.21.15.32:0/3525256173 conn(0x7f72200faa30 msgr2=0x7f72200fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:53:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.036+0000 7f72272f9700 1 -- 172.21.15.32:0/3525256173 shutdown_connections 2023-09-30T14:53:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:53:59.036+0000 7f72272f9700 1 -- 172.21.15.32:0/3525256173 wait complete. 2023-09-30T14:54:00.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 102045 38.46 MB/sec execute 33 sec latency 215.392 ms 2023-09-30T14:54:00.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:53:59 smithi156 ceph-mon[103579]: pgmap v758: 97 pgs: 97 active+clean; 680 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 831 op/s 2023-09-30T14:54:00.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:53:59 smithi148 ceph-mon[102355]: pgmap v758: 97 pgs: 97 active+clean; 680 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 831 op/s 2023-09-30T14:54:00.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:53:59 smithi032 ceph-mon[98592]: pgmap v758: 97 pgs: 97 active+clean; 680 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 831 op/s 2023-09-30T14:54:01.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 103071 39.03 MB/sec execute 34 sec latency 59.629 ms 2023-09-30T14:54:02.007 INFO:tasks.workunit.client.0.smithi032.stdout: 10 103865 39.21 MB/sec execute 35 sec latency 52.267 ms 2023-09-30T14:54:02.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:01 smithi156 ceph-mon[103579]: pgmap v759: 97 pgs: 97 active+clean; 681 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 771 op/s 2023-09-30T14:54:02.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:01 smithi148 ceph-mon[102355]: pgmap v759: 97 pgs: 97 active+clean; 681 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 771 op/s 2023-09-30T14:54:02.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:01 smithi032 ceph-mon[98592]: pgmap v759: 97 pgs: 97 active+clean; 681 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 771 op/s 2023-09-30T14:54:03.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 104470 39.21 MB/sec execute 36 sec latency 252.097 ms 2023-09-30T14:54:04.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 105205 39.06 MB/sec execute 37 sec latency 57.079 ms 2023-09-30T14:54:04.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:03 smithi156 ceph-mon[103579]: pgmap v760: 97 pgs: 97 active+clean; 673 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 797 op/s 2023-09-30T14:54:04.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:03 smithi148 ceph-mon[102355]: pgmap v760: 97 pgs: 97 active+clean; 673 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 797 op/s 2023-09-30T14:54:04.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:03 smithi032 ceph-mon[98592]: pgmap v760: 97 pgs: 97 active+clean; 673 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 797 op/s 2023-09-30T14:54:05.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 106090 39.48 MB/sec execute 38 sec latency 52.501 ms 2023-09-30T14:54:06.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 106686 39.25 MB/sec execute 39 sec latency 69.547 ms 2023-09-30T14:54:06.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:05 smithi156 ceph-mon[103579]: pgmap v761: 97 pgs: 97 active+clean; 680 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 10 MiB/s wr, 838 op/s 2023-09-30T14:54:06.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:05 smithi148 ceph-mon[102355]: pgmap v761: 97 pgs: 97 active+clean; 680 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 10 MiB/s wr, 838 op/s 2023-09-30T14:54:06.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:05 smithi032 ceph-mon[98592]: pgmap v761: 97 pgs: 97 active+clean; 680 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 10 MiB/s wr, 838 op/s 2023-09-30T14:54:07.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 107621 39.05 MB/sec execute 40 sec latency 39.292 ms 2023-09-30T14:54:08.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 108375 39.43 MB/sec execute 41 sec latency 248.024 ms 2023-09-30T14:54:08.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:07 smithi156 ceph-mon[103579]: pgmap v762: 97 pgs: 97 active+clean; 682 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.9 MiB/s wr, 817 op/s 2023-09-30T14:54:08.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:07 smithi148 ceph-mon[102355]: pgmap v762: 97 pgs: 97 active+clean; 682 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.9 MiB/s wr, 817 op/s 2023-09-30T14:54:08.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:07 smithi032 ceph-mon[98592]: pgmap v762: 97 pgs: 97 active+clean; 682 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.9 MiB/s wr, 817 op/s 2023-09-30T14:54:09.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 108926 39.39 MB/sec execute 42 sec latency 314.502 ms 2023-09-30T14:54:10.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 109499 39.21 MB/sec execute 43 sec latency 107.368 ms 2023-09-30T14:54:10.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:09 smithi156 ceph-mon[103579]: pgmap v763: 97 pgs: 97 active+clean; 689 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.9 MiB/s wr, 827 op/s 2023-09-30T14:54:10.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:09 smithi148 ceph-mon[102355]: pgmap v763: 97 pgs: 97 active+clean; 689 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.9 MiB/s wr, 827 op/s 2023-09-30T14:54:10.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:09 smithi032 ceph-mon[98592]: pgmap v763: 97 pgs: 97 active+clean; 689 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.9 MiB/s wr, 827 op/s 2023-09-30T14:54:11.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 110190 38.89 MB/sec execute 44 sec latency 50.420 ms 2023-09-30T14:54:12.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111000 39.08 MB/sec execute 45 sec latency 34.571 ms 2023-09-30T14:54:12.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:11 smithi156 ceph-mon[103579]: pgmap v764: 97 pgs: 97 active+clean; 692 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.1 MiB/s wr, 787 op/s 2023-09-30T14:54:12.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:11 smithi148 ceph-mon[102355]: pgmap v764: 97 pgs: 97 active+clean; 692 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.1 MiB/s wr, 787 op/s 2023-09-30T14:54:12.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:11 smithi032 ceph-mon[98592]: pgmap v764: 97 pgs: 97 active+clean; 692 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.1 MiB/s wr, 787 op/s 2023-09-30T14:54:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.165+0000 7f9d9d54e700 1 -- 172.21.15.32:0/1452509778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d981018e0 msgr2=0x7f9d98108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.165+0000 7f9d9d54e700 1 --2- 172.21.15.32:0/1452509778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d981018e0 0x7f9d98108190 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000b0d0 tx=0x7f9d9000b3e0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.165+0000 7f9d9d54e700 1 -- 172.21.15.32:0/1452509778 shutdown_connections 2023-09-30T14:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.165+0000 7f9d9d54e700 1 --2- 172.21.15.32:0/1452509778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d981018e0 0x7f9d98108190 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.165+0000 7f9d9d54e700 1 --2- 172.21.15.32:0/1452509778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d980ffd50 0x7f9d98100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.165+0000 7f9d9d54e700 1 --2- 172.21.15.32:0/1452509778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d98100f40 0x7f9d981013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.165+0000 7f9d9d54e700 1 -- 172.21.15.32:0/1452509778 >> 172.21.15.32:0/1452509778 conn(0x7f9d980fb2a0 msgr2=0x7f9d980fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.165+0000 7f9d9d54e700 1 -- 172.21.15.32:0/1452509778 shutdown_connections 2023-09-30T14:54:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.165+0000 7f9d9d54e700 1 -- 172.21.15.32:0/1452509778 wait complete. 2023-09-30T14:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.166+0000 7f9d9d54e700 1 Processor -- start 2023-09-30T14:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.166+0000 7f9d9d54e700 1 -- start start 2023-09-30T14:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.166+0000 7f9d9d54e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d980ffd50 0x7f9d98196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.166+0000 7f9d9d54e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d98100f40 0x7f9d981972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.166+0000 7f9d9d54e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d981018e0 0x7f9d98198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.166+0000 7f9d9d54e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d98198790 con 0x7f9d98100f40 2023-09-30T14:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.166+0000 7f9d9d54e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d98198900 con 0x7f9d980ffd50 2023-09-30T14:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.166+0000 7f9d9d54e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d98198ac0 con 0x7f9d981018e0 2023-09-30T14:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.167+0000 7f9d977fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d981018e0 0x7f9d98198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.167+0000 7f9d977fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d981018e0 0x7f9d98198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53196/0 (socket says 172.21.15.32:53196) 2023-09-30T14:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.167+0000 7f9d977fe700 1 -- 172.21.15.32:0/2761482603 learned_addr learned my addr 172.21.15.32:0/2761482603 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.167+0000 7f9d967fc700 1 --2- 172.21.15.32:0/2761482603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d98100f40 0x7f9d981972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.167+0000 7f9d96ffd700 1 --2- 172.21.15.32:0/2761482603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d980ffd50 0x7f9d98196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.167+0000 7f9d977fe700 1 -- 172.21.15.32:0/2761482603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d980ffd50 msgr2=0x7f9d98196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.167+0000 7f9d977fe700 1 --2- 172.21.15.32:0/2761482603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d980ffd50 0x7f9d98196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.167+0000 7f9d977fe700 1 -- 172.21.15.32:0/2761482603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d98100f40 msgr2=0x7f9d981972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.167+0000 7f9d977fe700 1 --2- 172.21.15.32:0/2761482603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d98100f40 0x7f9d981972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.167+0000 7f9d977fe700 1 -- 172.21.15.32:0/2761482603 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d88009520 con 0x7f9d981018e0 2023-09-30T14:54:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.167+0000 7f9d977fe700 1 --2- 172.21.15.32:0/2761482603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d981018e0 0x7f9d98198080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9d90008eb0 tx=0x7f9d90008ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.168+0000 7f9d7ffff700 1 -- 172.21.15.32:0/2761482603 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9000e040 con 0x7f9d981018e0 2023-09-30T14:54:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.168+0000 7f9d7ffff700 1 -- 172.21.15.32:0/2761482603 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d9000bdb0 con 0x7f9d981018e0 2023-09-30T14:54:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.168+0000 7f9d7ffff700 1 -- 172.21.15.32:0/2761482603 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9001d470 con 0x7f9d981018e0 2023-09-30T14:54:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.168+0000 7f9d9d54e700 1 -- 172.21.15.32:0/2761482603 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d90009d00 con 0x7f9d981018e0 2023-09-30T14:54:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.168+0000 7f9d9d54e700 1 -- 172.21.15.32:0/2761482603 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9819f2d0 con 0x7f9d981018e0 2023-09-30T14:54:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.170+0000 7f9d7ffff700 1 -- 172.21.15.32:0/2761482603 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f9d90005020 con 0x7f9d981018e0 2023-09-30T14:54:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.171+0000 7f9d7ffff700 1 --2- 172.21.15.32:0/2761482603 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9d800a1020 0x7f9d800a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.171+0000 7f9d96ffd700 1 --2- 172.21.15.32:0/2761482603 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9d800a1020 0x7f9d800a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.171+0000 7f9d7ffff700 1 -- 172.21.15.32:0/2761482603 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9d900cd590 con 0x7f9d981018e0 2023-09-30T14:54:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.172+0000 7f9d96ffd700 1 --2- 172.21.15.32:0/2761482603 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9d800a1020 0x7f9d800a34e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9d98196af0 tx=0x7f9d88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.172+0000 7f9da9736b80 1 Processor -- start 2023-09-30T14:54:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.172+0000 7f9da9736b80 1 -- start start 2023-09-30T14:54:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.172+0000 7f9da9736b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb369200 0x55f9bb369620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.172+0000 7f9da9736b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36a310 0x55f9bb36a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.172+0000 7f9da9736b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb36acd0 0x55f9bb371560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.172+0000 7f9da9736b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bb2d1400 con 0x55f9bb369200 2023-09-30T14:54:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.172+0000 7f9da9736b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bb2c2bc0 con 0x55f9bb36acd0 2023-09-30T14:54:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.172+0000 7f9da9736b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bb2c04e0 con 0x55f9bb36a310 2023-09-30T14:54:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.173+0000 7f9d96ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb369200 0x55f9bb369620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.173+0000 7f9d967fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36a310 0x55f9bb36a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.173+0000 7f9d96ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb369200 0x55f9bb369620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38820/0 (socket says 172.21.15.32:38820) 2023-09-30T14:54:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.173+0000 7f9d967fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36a310 0x55f9bb36a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53204/0 (socket says 172.21.15.32:53204) 2023-09-30T14:54:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.173+0000 7f9d96ffd700 1 -- 172.21.15.32:0/3272015562 learned_addr learned my addr 172.21.15.32:0/3272015562 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.173+0000 7f9d977fe700 1 --2- 172.21.15.32:0/3272015562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb36acd0 0x55f9bb371560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.173+0000 7f9d96ffd700 1 -- 172.21.15.32:0/3272015562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36a310 msgr2=0x55f9bb36a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.173+0000 7f9d96ffd700 1 --2- 172.21.15.32:0/3272015562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36a310 0x55f9bb36a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.173+0000 7f9d96ffd700 1 -- 172.21.15.32:0/3272015562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb36acd0 msgr2=0x55f9bb371560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.173+0000 7f9d96ffd700 1 --2- 172.21.15.32:0/3272015562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb36acd0 0x55f9bb371560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.173+0000 7f9d96ffd700 1 -- 172.21.15.32:0/3272015562 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9bb31e870 con 0x55f9bb369200 2023-09-30T14:54:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.173+0000 7f9d967fc700 1 --2- 172.21.15.32:0/3272015562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36a310 0x55f9bb36a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:54:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.173+0000 7f9d96ffd700 1 --2- 172.21.15.32:0/3272015562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb369200 0x55f9bb369620 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800e600 tx=0x7f9d88014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f959abe6f84025 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.173+0000 7f9d7d7fa700 1 -- 172.21.15.32:0/3272015562 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88026430 con 0x55f9bb369200 2023-09-30T14:54:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.173+0000 7f9d7d7fa700 1 -- 172.21.15.32:0/3272015562 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d88021070 con 0x55f9bb369200 2023-09-30T14:54:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.173+0000 7f9d7d7fa700 1 -- 172.21.15.32:0/3272015562 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8801d810 con 0x55f9bb369200 2023-09-30T14:54:12.178 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.174+0000 7f9da9736b80 1 -- 172.21.15.32:0/3272015562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb369200 msgr2=0x55f9bb369620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.174+0000 7f9da9736b80 1 --2- 172.21.15.32:0/3272015562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb369200 0x55f9bb369620 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800e600 tx=0x7f9d88014000 comp rx=0 tx=0).stop 2023-09-30T14:54:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.175+0000 7f9da9736b80 1 -- 172.21.15.32:0/3272015562 shutdown_connections 2023-09-30T14:54:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.175+0000 7f9da9736b80 1 --2- 172.21.15.32:0/3272015562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb36acd0 0x55f9bb371560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.175+0000 7f9da9736b80 1 --2- 172.21.15.32:0/3272015562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb369200 0x55f9bb369620 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.175+0000 7f9da9736b80 1 --2- 172.21.15.32:0/3272015562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36a310 0x55f9bb36a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.175+0000 7f9da9736b80 1 -- 172.21.15.32:0/3272015562 >> 172.21.15.32:0/3272015562 conn(0x55f9bb2dd200 msgr2=0x55f9bb363ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.175+0000 7f9da9736b80 1 -- 172.21.15.32:0/3272015562 shutdown_connections 2023-09-30T14:54:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.175+0000 7f9da9736b80 1 -- 172.21.15.32:0/3272015562 wait complete. 2023-09-30T14:54:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.175+0000 7f9da9736b80 1 -- start start 2023-09-30T14:54:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.175+0000 7f9da9736b80 1 Processor -- start 2023-09-30T14:54:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.175+0000 7f9da9736b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb369200 0x55f9bb372ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.175+0000 7f9da9736b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb36a310 0x55f9bb3733e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.175+0000 7f9da9736b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 0x55f9bb37a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.175+0000 7f9da9736b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bb2bbdd0 con 0x55f9bb369200 2023-09-30T14:54:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.175+0000 7f9da9736b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bb2586b0 con 0x55f9bb36a310 2023-09-30T14:54:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.175+0000 7f9da9736b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bb258350 con 0x55f9bb36acd0 2023-09-30T14:54:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.175+0000 7f9da9736b80 10 client.0 _ll_register_callbacks cb 0x55f9bb0b57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:54:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.176+0000 7f9d96ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb369200 0x55f9bb372ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.176+0000 7f9d977fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 0x55f9bb37a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.176+0000 7f9d96ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb369200 0x55f9bb372ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38826/0 (socket says 172.21.15.32:38826) 2023-09-30T14:54:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.176+0000 7f9d96ffd700 1 -- 172.21.15.32:0/877663520 learned_addr learned my addr 172.21.15.32:0/877663520 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.176+0000 7f9d977fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 0x55f9bb37a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53212/0 (socket says 172.21.15.32:53212) 2023-09-30T14:54:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.176+0000 7f9d96ffd700 1 -- 172.21.15.32:0/877663520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 msgr2=0x55f9bb37a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.176+0000 7f9d96ffd700 1 --2- 172.21.15.32:0/877663520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 0x55f9bb37a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.176+0000 7f9d96ffd700 1 -- 172.21.15.32:0/877663520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb36a310 msgr2=0x55f9bb3733e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.176+0000 7f9d96ffd700 1 --2- 172.21.15.32:0/877663520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb36a310 0x55f9bb3733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.176+0000 7f9d96ffd700 1 -- 172.21.15.32:0/877663520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8c0098e0 con 0x55f9bb369200 2023-09-30T14:54:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.176+0000 7f9d977fe700 1 --2- 172.21.15.32:0/877663520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 0x55f9bb37a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:54:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.176+0000 7f9d96ffd700 1 --2- 172.21.15.32:0/877663520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb369200 0x55f9bb372ea0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9d88005170 tx=0x7f9d88005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4302a24137cb751f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.176+0000 7f9d7aff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:54:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.176+0000 7f9d7aff5700 1 -- 172.21.15.32:0/877663520 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88003680 con 0x55f9bb369200 2023-09-30T14:54:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.176+0000 7f9da9736b80 1 -- 172.21.15.32:0/877663520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9bb35c090 con 0x55f9bb369200 2023-09-30T14:54:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.176+0000 7f9d7aff5700 1 -- 172.21.15.32:0/877663520 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9d880037e0 con 0x55f9bb369200 2023-09-30T14:54:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.177+0000 7f9d7aff5700 1 -- 172.21.15.32:0/877663520 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d880039a0 con 0x55f9bb369200 2023-09-30T14:54:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.177+0000 7f9da9736b80 10 client.16803 fetch_fsmap learned FSMap version 48 2023-09-30T14:54:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.177+0000 7f9da9736b80 1 -- 172.21.15.32:0/877663520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x7f9d88017070 con 0x55f9bb369200 2023-09-30T14:54:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.177+0000 7f9d7aff5700 1 -- 172.21.15.32:0/877663520 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f9bb35c090 con 0x55f9bb369200 2023-09-30T14:54:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.177+0000 7f9d7aff5700 1 -- 172.21.15.32:0/877663520 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 48) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9d88038070 con 0x55f9bb369200 2023-09-30T14:54:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.177+0000 7f9da9736b80 10 client.16803 fetch_fsmap finished waiting for FSMap version 48 2023-09-30T14:54:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.177+0000 7f9da9736b80 10 client.16803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:54:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.177+0000 7f9da9736b80 20 client.16803 populate_metadata read hostname 'smithi032' 2023-09-30T14:54:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.177+0000 7f9da9736b80 1 --2- 172.21.15.32:0/877663520 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f9bb37ec90 0x55f9bb3810b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.178+0000 7f9da9736b80 4 client.16803 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:54:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.178+0000 7f9da9736b80 1 -- 172.21.15.32:0/877663520 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9bb2b4db0 con 0x55f9bb37ec90 2023-09-30T14:54:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.178+0000 7f9d967fc700 1 --2- 172.21.15.32:0/877663520 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f9bb37ec90 0x55f9bb3810b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.179+0000 7f9d967fc700 1 --2- 172.21.15.32:0/877663520 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f9bb37ec90 0x55f9bb3810b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e410570849691e4 server_cookie=b7b806a6469be945 in_seq=0 out_seq=0 2023-09-30T14:54:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.179+0000 7f9d7aff5700 10 client.16803 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:54:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.180+0000 7f9d7aff5700 1 -- 172.21.15.32:0/877663520 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f9bb2b4db0 con 0x55f9bb37ec90 2023-09-30T14:54:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.180+0000 7f9d7aff5700 10 client.16803 handle_command_reply: tid=0 2023-09-30T14:54:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.180+0000 7f9da9736b80 1 client.16803 shutdown 2023-09-30T14:54:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.180+0000 7f9da9736b80 1 -- 172.21.15.32:0/877663520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb369200 msgr2=0x55f9bb372ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.180+0000 7f9da9736b80 1 --2- 172.21.15.32:0/877663520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb369200 0x55f9bb372ea0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9d88005170 tx=0x7f9d88005250 comp rx=0 tx=0).stop 2023-09-30T14:54:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.180+0000 7f9da9736b80 1 -- 172.21.15.32:0/877663520 shutdown_connections 2023-09-30T14:54:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.180+0000 7f9da9736b80 1 --2- 172.21.15.32:0/877663520 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f9bb37ec90 0x55f9bb3810b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.180+0000 7f9da9736b80 1 --2- 172.21.15.32:0/877663520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 0x55f9bb37a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.180+0000 7f9da9736b80 1 --2- 172.21.15.32:0/877663520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb369200 0x55f9bb372ea0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.180+0000 7f9d7aff5700 0 client.16803 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:54:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.180+0000 7f9da9736b80 1 --2- 172.21.15.32:0/877663520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb36a310 0x55f9bb3733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.180+0000 7f9da9736b80 1 -- 172.21.15.32:0/877663520 >> 172.21.15.32:0/877663520 conn(0x55f9bb2dd200 msgr2=0x55f9bb363590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.180+0000 7f9da9736b80 1 -- 172.21.15.32:0/877663520 shutdown_connections 2023-09-30T14:54:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.180+0000 7f9da9736b80 1 -- 172.21.15.32:0/877663520 wait complete. 2023-09-30T14:54:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.180+0000 7f9da9736b80 20 client.16803 trim_cache size 0 max 16384 2023-09-30T14:54:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.197+0000 7f9da9736b80 1 Processor -- start 2023-09-30T14:54:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.197+0000 7f9da9736b80 1 -- start start 2023-09-30T14:54:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.197+0000 7f9da9736b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb369200 0x55f9bb36a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.197+0000 7f9da9736b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb36a310 0x55f9bb37a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.197+0000 7f9da9736b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 0x55f9bb3710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.197+0000 7f9da9736b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bb13c8c0 con 0x55f9bb36a310 2023-09-30T14:54:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.197+0000 7f9da9736b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bb372a10 con 0x55f9bb369200 2023-09-30T14:54:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.197+0000 7f9da9736b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bb360ba0 con 0x55f9bb36acd0 2023-09-30T14:54:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.197+0000 7f9d967fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb36a310 0x55f9bb37a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.197+0000 7f9d967fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb36a310 0x55f9bb37a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38836/0 (socket says 172.21.15.32:38836) 2023-09-30T14:54:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.197+0000 7f9d967fc700 1 -- 172.21.15.32:0/638267249 learned_addr learned my addr 172.21.15.32:0/638267249 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.197+0000 7f9d96ffd700 1 --2- 172.21.15.32:0/638267249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb369200 0x55f9bb36a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.197+0000 7f9d977fe700 1 --2- 172.21.15.32:0/638267249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 0x55f9bb3710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.198+0000 7f9d967fc700 1 -- 172.21.15.32:0/638267249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 msgr2=0x55f9bb3710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.198+0000 7f9d967fc700 1 --2- 172.21.15.32:0/638267249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 0x55f9bb3710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.198+0000 7f9d967fc700 1 -- 172.21.15.32:0/638267249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb369200 msgr2=0x55f9bb36a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.198+0000 7f9d967fc700 1 --2- 172.21.15.32:0/638267249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb369200 0x55f9bb36a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.198+0000 7f9d967fc700 1 -- 172.21.15.32:0/638267249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9bb0e6d80 con 0x55f9bb36a310 2023-09-30T14:54:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.198+0000 7f9d967fc700 1 --2- 172.21.15.32:0/638267249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb36a310 0x55f9bb37a3e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c004790 tx=0x7f9d8c0109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34cfbd5a91f2c521 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.198+0000 7f9d7dffb700 1 -- 172.21.15.32:0/638267249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c017040 con 0x55f9bb36a310 2023-09-30T14:54:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.198+0000 7f9d7dffb700 1 -- 172.21.15.32:0/638267249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d8c01c5a0 con 0x55f9bb36a310 2023-09-30T14:54:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.198+0000 7f9d7dffb700 1 -- 172.21.15.32:0/638267249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0038e0 con 0x55f9bb36a310 2023-09-30T14:54:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.199+0000 7f9da9736b80 1 -- 172.21.15.32:0/638267249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb36a310 msgr2=0x55f9bb37a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.199+0000 7f9da9736b80 1 --2- 172.21.15.32:0/638267249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb36a310 0x55f9bb37a3e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c004790 tx=0x7f9d8c0109e0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.199+0000 7f9da9736b80 1 -- 172.21.15.32:0/638267249 shutdown_connections 2023-09-30T14:54:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.199+0000 7f9da9736b80 1 --2- 172.21.15.32:0/638267249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 0x55f9bb3710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.199+0000 7f9da9736b80 1 --2- 172.21.15.32:0/638267249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb369200 0x55f9bb36a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.199+0000 7f9da9736b80 1 --2- 172.21.15.32:0/638267249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb36a310 0x55f9bb37a3e0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.199+0000 7f9da9736b80 1 -- 172.21.15.32:0/638267249 >> 172.21.15.32:0/638267249 conn(0x55f9bb2dd200 msgr2=0x55f9bb371c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.199+0000 7f9da9736b80 1 -- 172.21.15.32:0/638267249 shutdown_connections 2023-09-30T14:54:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.199+0000 7f9da9736b80 1 -- 172.21.15.32:0/638267249 wait complete. 2023-09-30T14:54:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.199+0000 7f9da9736b80 1 -- start start 2023-09-30T14:54:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.199+0000 7f9da9736b80 1 Processor -- start 2023-09-30T14:54:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.199+0000 7f9da9736b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb369200 0x55f9bb377e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.199+0000 7f9da9736b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb36a310 0x55f9bb3783a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.199+0000 7f9da9736b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 0x55f9bb3793b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9da9736b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bb3798f0 con 0x55f9bb36a310 2023-09-30T14:54:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9da9736b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bb379a60 con 0x55f9bb369200 2023-09-30T14:54:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9da9736b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9bb379c20 con 0x55f9bb36acd0 2023-09-30T14:54:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9da9736b80 10 client.0 _ll_register_callbacks cb 0x55f9bb0b57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:54:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9d967fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb36a310 0x55f9bb3783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9d967fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb36a310 0x55f9bb3783a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38842/0 (socket says 172.21.15.32:38842) 2023-09-30T14:54:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9d977fe700 1 --2- 172.21.15.32:0/3803471175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 0x55f9bb3793b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9d967fc700 1 -- 172.21.15.32:0/3803471175 learned_addr learned my addr 172.21.15.32:0/3803471175 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9d96ffd700 1 --2- 172.21.15.32:0/3803471175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb369200 0x55f9bb377e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9d967fc700 1 -- 172.21.15.32:0/3803471175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 msgr2=0x55f9bb3793b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9d967fc700 1 --2- 172.21.15.32:0/3803471175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 0x55f9bb3793b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9d967fc700 1 -- 172.21.15.32:0/3803471175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb369200 msgr2=0x55f9bb377e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9d967fc700 1 --2- 172.21.15.32:0/3803471175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb369200 0x55f9bb377e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9d967fc700 1 -- 172.21.15.32:0/3803471175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8c010430 con 0x55f9bb36a310 2023-09-30T14:54:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9d977fe700 1 --2- 172.21.15.32:0/3803471175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 0x55f9bb3793b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:54:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9d96ffd700 1 --2- 172.21.15.32:0/3803471175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb369200 0x55f9bb377e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:54:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9d967fc700 1 --2- 172.21.15.32:0/3803471175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb36a310 0x55f9bb3783a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009de0 tx=0x7f9d8c01df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b779ab454bd2dbfb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9d7aff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:54:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9d7aff5700 1 -- 172.21.15.32:0/3803471175 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c017040 con 0x55f9bb36a310 2023-09-30T14:54:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9d7aff5700 1 -- 172.21.15.32:0/3803471175 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9d8c004250 con 0x55f9bb36a310 2023-09-30T14:54:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.200+0000 7f9da9736b80 1 -- 172.21.15.32:0/3803471175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d88021070 con 0x55f9bb36a310 2023-09-30T14:54:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.201+0000 7f9d7aff5700 1 -- 172.21.15.32:0/3803471175 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d8c01cb10 con 0x55f9bb36a310 2023-09-30T14:54:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.201+0000 7f9da9736b80 10 client.16809 fetch_fsmap learned FSMap version 48 2023-09-30T14:54:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.201+0000 7f9da9736b80 1 -- 172.21.15.32:0/3803471175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55f9bb0cfc00 con 0x55f9bb36a310 2023-09-30T14:54:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.201+0000 7f9d7aff5700 1 -- 172.21.15.32:0/3803471175 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d88021070 con 0x55f9bb36a310 2023-09-30T14:54:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.201+0000 7f9d7aff5700 1 -- 172.21.15.32:0/3803471175 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 48) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9d8c02e5e0 con 0x55f9bb36a310 2023-09-30T14:54:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.201+0000 7f9da9736b80 10 client.16809 fetch_fsmap finished waiting for FSMap version 48 2023-09-30T14:54:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.201+0000 7f9da9736b80 10 client.16809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:54:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.201+0000 7f9da9736b80 20 client.16809 populate_metadata read hostname 'smithi032' 2023-09-30T14:54:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.201+0000 7f9da9736b80 1 --2- 172.21.15.32:0/3803471175 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f9bb3d3470 0x55f9bb3d5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.201+0000 7f9da9736b80 4 client.16809 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:54:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.201+0000 7f9da9736b80 1 -- 172.21.15.32:0/3803471175 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f9bb2b4ad0 con 0x55f9bb3d3470 2023-09-30T14:54:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.201+0000 7f9d96ffd700 1 --2- 172.21.15.32:0/3803471175 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f9bb3d3470 0x55f9bb3d5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.202+0000 7f9d96ffd700 1 --2- 172.21.15.32:0/3803471175 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f9bb3d3470 0x55f9bb3d5890 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c879fc78df78441b server_cookie=cabae3703c6b6ed0 in_seq=0 out_seq=0 2023-09-30T14:54:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.202+0000 7f9d7aff5700 10 client.16809 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:54:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.496+0000 7f9d7aff5700 1 -- 172.21.15.32:0/3803471175 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f9bb2b4ad0 con 0x55f9bb3d3470 2023-09-30T14:54:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.496+0000 7f9d7aff5700 10 client.16809 handle_command_reply: tid=0 2023-09-30T14:54:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.496+0000 7f9da9736b80 1 client.16809 shutdown 2023-09-30T14:54:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.496+0000 7f9da9736b80 1 -- 172.21.15.32:0/3803471175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb36a310 msgr2=0x55f9bb3783a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.496+0000 7f9da9736b80 1 --2- 172.21.15.32:0/3803471175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb36a310 0x55f9bb3783a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009de0 tx=0x7f9d8c01df40 comp rx=0 tx=0).stop 2023-09-30T14:54:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.496+0000 7f9da9736b80 1 -- 172.21.15.32:0/3803471175 shutdown_connections 2023-09-30T14:54:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.496+0000 7f9da9736b80 1 --2- 172.21.15.32:0/3803471175 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f9bb3d3470 0x55f9bb3d5890 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.496+0000 7f9da9736b80 1 --2- 172.21.15.32:0/3803471175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9bb36acd0 0x55f9bb3793b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.496+0000 7f9d7aff5700 0 client.16809 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:54:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.496+0000 7f9da9736b80 1 --2- 172.21.15.32:0/3803471175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9bb369200 0x55f9bb377e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.496+0000 7f9da9736b80 1 --2- 172.21.15.32:0/3803471175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f9bb36a310 0x55f9bb3783a0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.496+0000 7f9da9736b80 1 -- 172.21.15.32:0/3803471175 >> 172.21.15.32:0/3803471175 conn(0x55f9bb2dd200 msgr2=0x55f9bb3710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.497+0000 7f9da9736b80 1 -- 172.21.15.32:0/3803471175 shutdown_connections 2023-09-30T14:54:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.497+0000 7f9da9736b80 1 -- 172.21.15.32:0/3803471175 wait complete. 2023-09-30T14:54:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.497+0000 7f9da9736b80 20 client.16809 trim_cache size 0 max 16384 2023-09-30T14:54:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:54:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:54:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:54:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:54:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.498+0000 7f9d9d54e700 1 -- 172.21.15.32:0/2761482603 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9d800a1020 msgr2=0x7f9d800a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.498+0000 7f9d9d54e700 1 --2- 172.21.15.32:0/2761482603 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9d800a1020 0x7f9d800a34e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9d98196af0 tx=0x7f9d88009260 comp rx=0 tx=0).stop 2023-09-30T14:54:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.498+0000 7f9d9d54e700 1 -- 172.21.15.32:0/2761482603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d981018e0 msgr2=0x7f9d98198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.498+0000 7f9d9d54e700 1 --2- 172.21.15.32:0/2761482603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d981018e0 0x7f9d98198080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9d90008eb0 tx=0x7f9d90008ee0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.498+0000 7f9d9d54e700 1 -- 172.21.15.32:0/2761482603 shutdown_connections 2023-09-30T14:54:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.498+0000 7f9d9d54e700 1 --2- 172.21.15.32:0/2761482603 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9d800a1020 0x7f9d800a34e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.498+0000 7f9d9d54e700 1 --2- 172.21.15.32:0/2761482603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d981018e0 0x7f9d98198080 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.498+0000 7f9d9d54e700 1 --2- 172.21.15.32:0/2761482603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d980ffd50 0x7f9d98196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.498+0000 7f9d9d54e700 1 --2- 172.21.15.32:0/2761482603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d98100f40 0x7f9d981972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.498+0000 7f9d9d54e700 1 -- 172.21.15.32:0/2761482603 >> 172.21.15.32:0/2761482603 conn(0x7f9d980fb2a0 msgr2=0x7f9d980fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.498+0000 7f9d9d54e700 1 -- 172.21.15.32:0/2761482603 shutdown_connections 2023-09-30T14:54:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:12.498+0000 7f9d9d54e700 1 -- 172.21.15.32:0/2761482603 wait complete. 2023-09-30T14:54:13.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 111532 38.72 MB/sec execute 46 sec latency 284.902 ms 2023-09-30T14:54:14.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 112284 38.77 MB/sec execute 47 sec latency 131.568 ms 2023-09-30T14:54:14.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:13 smithi156 ceph-mon[103579]: pgmap v765: 97 pgs: 97 active+clean; 696 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.1 MiB/s wr, 754 op/s 2023-09-30T14:54:14.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:13 smithi148 ceph-mon[102355]: pgmap v765: 97 pgs: 97 active+clean; 696 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.1 MiB/s wr, 754 op/s 2023-09-30T14:54:14.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:13 smithi032 ceph-mon[98592]: pgmap v765: 97 pgs: 97 active+clean; 696 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.1 MiB/s wr, 754 op/s 2023-09-30T14:54:15.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 113153 39.35 MB/sec execute 48 sec latency 48.272 ms 2023-09-30T14:54:16.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 113643 38.96 MB/sec execute 49 sec latency 65.036 ms 2023-09-30T14:54:16.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:15 smithi156 ceph-mon[103579]: pgmap v766: 97 pgs: 97 active+clean; 697 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 9.4 MiB/s wr, 846 op/s 2023-09-30T14:54:16.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:15 smithi148 ceph-mon[102355]: pgmap v766: 97 pgs: 97 active+clean; 697 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 9.4 MiB/s wr, 846 op/s 2023-09-30T14:54:16.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:15 smithi032 ceph-mon[98592]: pgmap v766: 97 pgs: 97 active+clean; 697 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 9.4 MiB/s wr, 846 op/s 2023-09-30T14:54:17.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 114241 38.62 MB/sec execute 50 sec latency 111.667 ms 2023-09-30T14:54:18.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 114853 38.44 MB/sec execute 51 sec latency 180.429 ms 2023-09-30T14:54:18.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:17 smithi156 ceph-mon[103579]: pgmap v767: 97 pgs: 97 active+clean; 689 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.8 MiB/s wr, 793 op/s 2023-09-30T14:54:18.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:17 smithi148 ceph-mon[102355]: pgmap v767: 97 pgs: 97 active+clean; 689 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.8 MiB/s wr, 793 op/s 2023-09-30T14:54:18.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:17 smithi032 ceph-mon[98592]: pgmap v767: 97 pgs: 97 active+clean; 689 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.8 MiB/s wr, 793 op/s 2023-09-30T14:54:19.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 115453 38.46 MB/sec execute 52 sec latency 246.561 ms 2023-09-30T14:54:20.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 116251 38.40 MB/sec execute 53 sec latency 74.041 ms 2023-09-30T14:54:20.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:19 smithi156 ceph-mon[103579]: pgmap v768: 97 pgs: 97 active+clean; 686 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 825 op/s 2023-09-30T14:54:20.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:19 smithi148 ceph-mon[102355]: pgmap v768: 97 pgs: 97 active+clean; 686 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 825 op/s 2023-09-30T14:54:20.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:19 smithi032 ceph-mon[98592]: pgmap v768: 97 pgs: 97 active+clean; 686 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 825 op/s 2023-09-30T14:54:21.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 117079 38.53 MB/sec execute 54 sec latency 68.968 ms 2023-09-30T14:54:22.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 117924 38.92 MB/sec execute 55 sec latency 93.668 ms 2023-09-30T14:54:22.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:21 smithi156 ceph-mon[103579]: pgmap v769: 97 pgs: 97 active+clean; 694 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 10 MiB/s wr, 790 op/s 2023-09-30T14:54:22.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:21 smithi148 ceph-mon[102355]: pgmap v769: 97 pgs: 97 active+clean; 694 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 10 MiB/s wr, 790 op/s 2023-09-30T14:54:22.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:21 smithi032 ceph-mon[98592]: pgmap v769: 97 pgs: 97 active+clean; 694 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 10 MiB/s wr, 790 op/s 2023-09-30T14:54:23.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 118473 38.68 MB/sec execute 56 sec latency 69.444 ms 2023-09-30T14:54:24.008 INFO:tasks.workunit.client.0.smithi032.stdout: 10 118812 38.20 MB/sec execute 57 sec latency 431.498 ms 2023-09-30T14:54:24.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:23 smithi156 ceph-mon[103579]: pgmap v770: 97 pgs: 97 active+clean; 695 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 771 op/s 2023-09-30T14:54:24.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:23 smithi148 ceph-mon[102355]: pgmap v770: 97 pgs: 97 active+clean; 695 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 771 op/s 2023-09-30T14:54:24.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:23 smithi032 ceph-mon[98592]: pgmap v770: 97 pgs: 97 active+clean; 695 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 771 op/s 2023-09-30T14:54:25.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 119650 38.27 MB/sec execute 58 sec latency 52.883 ms 2023-09-30T14:54:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.648+0000 7fa22fc46700 1 -- 172.21.15.32:0/3990422070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa228100920 msgr2=0x7fa2281081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.648+0000 7fa22fc46700 1 --2- 172.21.15.32:0/3990422070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa228100920 0x7fa2281081c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa22000b0a0 tx=0x7fa22000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.648+0000 7fa22fc46700 1 -- 172.21.15.32:0/3990422070 shutdown_connections 2023-09-30T14:54:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.648+0000 7fa22fc46700 1 --2- 172.21.15.32:0/3990422070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa228100920 0x7fa2281081c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.648+0000 7fa22fc46700 1 --2- 172.21.15.32:0/3990422070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2281013a0 0x7fa2281017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.648+0000 7fa22fc46700 1 --2- 172.21.15.32:0/3990422070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2280fff60 0x7fa2281003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.648+0000 7fa22fc46700 1 -- 172.21.15.32:0/3990422070 >> 172.21.15.32:0/3990422070 conn(0x7fa2280fa9f0 msgr2=0x7fa2280fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.648+0000 7fa22fc46700 1 -- 172.21.15.32:0/3990422070 shutdown_connections 2023-09-30T14:54:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.648+0000 7fa22fc46700 1 -- 172.21.15.32:0/3990422070 wait complete. 2023-09-30T14:54:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.649+0000 7fa22fc46700 1 Processor -- start 2023-09-30T14:54:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.649+0000 7fa22fc46700 1 -- start start 2023-09-30T14:54:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.649+0000 7fa22fc46700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2280fff60 0x7fa22810eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.649+0000 7fa22fc46700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa228100920 0x7fa22810f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.649+0000 7fa22fc46700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2281013a0 0x7fa2281100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.649+0000 7fa22fc46700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2281106b0 con 0x7fa2281013a0 2023-09-30T14:54:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.649+0000 7fa22fc46700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa228110820 con 0x7fa2280fff60 2023-09-30T14:54:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.649+0000 7fa22fc46700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2281109e0 con 0x7fa228100920 2023-09-30T14:54:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.650+0000 7fa22e1e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2281013a0 0x7fa2281100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.650+0000 7fa22d9e2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2280fff60 0x7fa22810eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.650+0000 7fa22e1e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2281013a0 0x7fa2281100e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38856/0 (socket says 172.21.15.32:38856) 2023-09-30T14:54:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.650+0000 7fa22e1e3700 1 -- 172.21.15.32:0/1780817080 learned_addr learned my addr 172.21.15.32:0/1780817080 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.650+0000 7fa22d1e1700 1 --2- 172.21.15.32:0/1780817080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa228100920 0x7fa22810f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.650+0000 7fa22d9e2700 1 -- 172.21.15.32:0/1780817080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa228100920 msgr2=0x7fa22810f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.650+0000 7fa22d9e2700 1 --2- 172.21.15.32:0/1780817080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa228100920 0x7fa22810f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.650+0000 7fa22d9e2700 1 -- 172.21.15.32:0/1780817080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2281013a0 msgr2=0x7fa2281100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.650+0000 7fa22d9e2700 1 --2- 172.21.15.32:0/1780817080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2281013a0 0x7fa2281100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.650+0000 7fa22d9e2700 1 -- 172.21.15.32:0/1780817080 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa220009d00 con 0x7fa2280fff60 2023-09-30T14:54:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.650+0000 7fa22e1e3700 1 --2- 172.21.15.32:0/1780817080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2281013a0 0x7fa2281100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:54:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.650+0000 7fa22d9e2700 1 --2- 172.21.15.32:0/1780817080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2280fff60 0x7fa22810eb60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa22400b450 tx=0x7fa22400b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.650+0000 7fa21effd700 1 -- 172.21.15.32:0/1780817080 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224011830 con 0x7fa2280fff60 2023-09-30T14:54:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.650+0000 7fa22fc46700 1 -- 172.21.15.32:0/1780817080 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa228110c60 con 0x7fa2280fff60 2023-09-30T14:54:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.650+0000 7fa21effd700 1 -- 172.21.15.32:0/1780817080 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa22400be60 con 0x7fa2280fff60 2023-09-30T14:54:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.651+0000 7fa21effd700 1 -- 172.21.15.32:0/1780817080 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224010d20 con 0x7fa2280fff60 2023-09-30T14:54:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.651+0000 7fa22fc46700 1 -- 172.21.15.32:0/1780817080 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa228111240 con 0x7fa2280fff60 2023-09-30T14:54:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.653+0000 7fa21effd700 1 -- 172.21.15.32:0/1780817080 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fa224010380 con 0x7fa2280fff60 2023-09-30T14:54:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.654+0000 7fa21effd700 1 --2- 172.21.15.32:0/1780817080 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa2140a1000 0x7fa2140a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.654+0000 7fa21effd700 1 -- 172.21.15.32:0/1780817080 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa2240cbfd0 con 0x7fa2280fff60 2023-09-30T14:54:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.654+0000 7fa22d1e1700 1 --2- 172.21.15.32:0/1780817080 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa2140a1000 0x7fa2140a34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.654+0000 7fa23be2eb80 1 Processor -- start 2023-09-30T14:54:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa23be2eb80 1 -- start start 2023-09-30T14:54:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa22d1e1700 1 --2- 172.21.15.32:0/1780817080 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa2140a1000 0x7fa2140a34c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa218009c60 tx=0x7fa218009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa23be2eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fa200 0x55a4e39fa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa23be2eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fb310 0x55a4e39fb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa23be2eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fbcd0 0x55a4e3a02560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa23be2eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4e3962400 con 0x55a4e39fa200 2023-09-30T14:54:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa23be2eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4e3953bc0 con 0x55a4e39fb310 2023-09-30T14:54:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa23be2eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4e39514e0 con 0x55a4e39fbcd0 2023-09-30T14:54:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa22d9e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fa200 0x55a4e39fa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa22d9e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fa200 0x55a4e39fa620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38862/0 (socket says 172.21.15.32:38862) 2023-09-30T14:54:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa22d9e2700 1 -- 172.21.15.32:0/2652457157 learned_addr learned my addr 172.21.15.32:0/2652457157 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa22d1e1700 1 --2- 172.21.15.32:0/2652457157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fb310 0x55a4e39fb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa22e1e3700 1 --2- 172.21.15.32:0/2652457157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fbcd0 0x55a4e3a02560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa22e1e3700 1 -- 172.21.15.32:0/2652457157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fb310 msgr2=0x55a4e39fb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa22e1e3700 1 --2- 172.21.15.32:0/2652457157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fb310 0x55a4e39fb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa22e1e3700 1 -- 172.21.15.32:0/2652457157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fa200 msgr2=0x55a4e39fa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa22e1e3700 1 --2- 172.21.15.32:0/2652457157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fa200 0x55a4e39fa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa22e1e3700 1 -- 172.21.15.32:0/2652457157 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4e39af870 con 0x55a4e39fbcd0 2023-09-30T14:54:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.655+0000 7fa22d9e2700 1 --2- 172.21.15.32:0/2652457157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fa200 0x55a4e39fa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:54:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.656+0000 7fa22e1e3700 1 --2- 172.21.15.32:0/2652457157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fbcd0 0x55a4e3a02560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa2200062a0 tx=0x7fa220009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c10e97bc45e5049 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.656+0000 7fa213fff700 1 -- 172.21.15.32:0/2652457157 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22000e040 con 0x55a4e39fbcd0 2023-09-30T14:54:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.656+0000 7fa213fff700 1 -- 172.21.15.32:0/2652457157 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2200076e0 con 0x55a4e39fbcd0 2023-09-30T14:54:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.656+0000 7fa213fff700 1 -- 172.21.15.32:0/2652457157 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa220005330 con 0x55a4e39fbcd0 2023-09-30T14:54:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.657+0000 7fa23be2eb80 1 -- 172.21.15.32:0/2652457157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fbcd0 msgr2=0x55a4e3a02560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.657+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/2652457157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fbcd0 0x55a4e3a02560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa2200062a0 tx=0x7fa220009450 comp rx=0 tx=0).stop 2023-09-30T14:54:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.657+0000 7fa23be2eb80 1 -- 172.21.15.32:0/2652457157 shutdown_connections 2023-09-30T14:54:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.657+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/2652457157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fbcd0 0x55a4e3a02560 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.657+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/2652457157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fa200 0x55a4e39fa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.657+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/2652457157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fb310 0x55a4e39fb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.657+0000 7fa23be2eb80 1 -- 172.21.15.32:0/2652457157 >> 172.21.15.32:0/2652457157 conn(0x55a4e396e200 msgr2=0x55a4e39f4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.657+0000 7fa23be2eb80 1 -- 172.21.15.32:0/2652457157 shutdown_connections 2023-09-30T14:54:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.657+0000 7fa23be2eb80 1 -- 172.21.15.32:0/2652457157 wait complete. 2023-09-30T14:54:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.657+0000 7fa23be2eb80 1 -- start start 2023-09-30T14:54:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.657+0000 7fa23be2eb80 1 Processor -- start 2023-09-30T14:54:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa23be2eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fa200 0x55a4e3a03d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa23be2eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fb310 0x55a4e3a042c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa23be2eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fbcd0 0x55a4e3a0b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa23be2eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4e394cdd0 con 0x55a4e39fb310 2023-09-30T14:54:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa23be2eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4e38e96b0 con 0x55a4e39fbcd0 2023-09-30T14:54:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa23be2eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4e38e9350 con 0x55a4e39fa200 2023-09-30T14:54:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa23be2eb80 10 client.0 _ll_register_callbacks cb 0x55a4e37467a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:54:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa22d9e2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fa200 0x55a4e3a03d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa22d1e1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fb310 0x55a4e3a042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa22d9e2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fa200 0x55a4e3a03d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53250/0 (socket says 172.21.15.32:53250) 2023-09-30T14:54:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa22e1e3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fbcd0 0x55a4e3a0b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa22d9e2700 1 -- 172.21.15.32:0/3634450696 learned_addr learned my addr 172.21.15.32:0/3634450696 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa22d1e1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fb310 0x55a4e3a042c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38872/0 (socket says 172.21.15.32:38872) 2023-09-30T14:54:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa22d9e2700 1 -- 172.21.15.32:0/3634450696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fbcd0 msgr2=0x55a4e3a0b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa22d9e2700 1 --2- 172.21.15.32:0/3634450696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fbcd0 0x55a4e3a0b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa22d9e2700 1 -- 172.21.15.32:0/3634450696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fb310 msgr2=0x55a4e3a042c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa22d9e2700 1 --2- 172.21.15.32:0/3634450696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fb310 0x55a4e3a042c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa22d9e2700 1 -- 172.21.15.32:0/3634450696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa228111240 con 0x55a4e39fa200 2023-09-30T14:54:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa22d9e2700 1 --2- 172.21.15.32:0/3634450696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fa200 0x55a4e3a03d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa2240c9b30 tx=0x7fa224098e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9298923472ed8297 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.658+0000 7fa2117fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:54:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.659+0000 7fa2117fa700 1 -- 172.21.15.32:0/3634450696 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224098b90 con 0x55a4e39fa200 2023-09-30T14:54:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.659+0000 7fa23be2eb80 1 -- 172.21.15.32:0/3634450696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4e39ed090 con 0x55a4e39fa200 2023-09-30T14:54:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.659+0000 7fa2117fa700 1 -- 172.21.15.32:0/3634450696 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa224099ba0 con 0x55a4e39fa200 2023-09-30T14:54:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.659+0000 7fa2117fa700 1 -- 172.21.15.32:0/3634450696 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2240994a0 con 0x55a4e39fa200 2023-09-30T14:54:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.659+0000 7fa23be2eb80 10 client.26476 fetch_fsmap learned FSMap version 48 2023-09-30T14:54:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.659+0000 7fa23be2eb80 1 -- 172.21.15.32:0/3634450696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x7fa228108b70 con 0x55a4e39fa200 2023-09-30T14:54:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.659+0000 7fa2117fa700 1 -- 172.21.15.32:0/3634450696 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa22407d020 con 0x55a4e39fa200 2023-09-30T14:54:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.659+0000 7fa2117fa700 1 -- 172.21.15.32:0/3634450696 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 48) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa224095290 con 0x55a4e39fa200 2023-09-30T14:54:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.659+0000 7fa23be2eb80 10 client.26476 fetch_fsmap finished waiting for FSMap version 48 2023-09-30T14:54:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.659+0000 7fa23be2eb80 10 client.26476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:54:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.659+0000 7fa23be2eb80 20 client.26476 populate_metadata read hostname 'smithi032' 2023-09-30T14:54:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.660+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/3634450696 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a4e3a20c00 0x55a4e3a23020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.660+0000 7fa23be2eb80 4 client.26476 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:54:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.660+0000 7fa23be2eb80 1 -- 172.21.15.32:0/3634450696 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4e3945db0 con 0x55a4e3a20c00 2023-09-30T14:54:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.660+0000 7fa22d1e1700 1 --2- 172.21.15.32:0/3634450696 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a4e3a20c00 0x55a4e3a23020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.660+0000 7fa2117fa700 10 client.26476 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:54:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.660+0000 7fa22d1e1700 1 --2- 172.21.15.32:0/3634450696 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a4e3a20c00 0x55a4e3a23020 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d57cfba4ac2067 server_cookie=70ad60e84ab6b888 in_seq=0 out_seq=0 2023-09-30T14:54:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.661+0000 7fa2117fa700 1 -- 172.21.15.32:0/3634450696 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a4e3945db0 con 0x55a4e3a20c00 2023-09-30T14:54:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.661+0000 7fa2117fa700 10 client.26476 handle_command_reply: tid=0 2023-09-30T14:54:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.661+0000 7fa23be2eb80 1 client.26476 shutdown 2023-09-30T14:54:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.661+0000 7fa23be2eb80 1 -- 172.21.15.32:0/3634450696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fa200 msgr2=0x55a4e3a03d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.661+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/3634450696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fa200 0x55a4e3a03d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa2240c9b30 tx=0x7fa224098e00 comp rx=0 tx=0).stop 2023-09-30T14:54:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.661+0000 7fa23be2eb80 1 -- 172.21.15.32:0/3634450696 shutdown_connections 2023-09-30T14:54:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.661+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/3634450696 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a4e3a20c00 0x55a4e3a23020 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.661+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/3634450696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fbcd0 0x55a4e3a0b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.661+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/3634450696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fa200 0x55a4e3a03d80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.661+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/3634450696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fb310 0x55a4e3a042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.661+0000 7fa2117fa700 0 client.26476 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:54:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.661+0000 7fa23be2eb80 1 -- 172.21.15.32:0/3634450696 >> 172.21.15.32:0/3634450696 conn(0x55a4e396e200 msgr2=0x55a4e39f4590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.661+0000 7fa23be2eb80 1 -- 172.21.15.32:0/3634450696 shutdown_connections 2023-09-30T14:54:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.661+0000 7fa23be2eb80 1 -- 172.21.15.32:0/3634450696 wait complete. 2023-09-30T14:54:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.662+0000 7fa23be2eb80 20 client.26476 trim_cache size 0 max 16384 2023-09-30T14:54:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.677+0000 7fa23be2eb80 1 Processor -- start 2023-09-30T14:54:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.677+0000 7fa23be2eb80 1 -- start start 2023-09-30T14:54:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa23be2eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fa200 0x55a4e39f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa23be2eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fb310 0x55a4e3a0b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa23be2eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fbcd0 0x55a4e39f4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa23be2eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4e37cd8c0 con 0x55a4e39fa200 2023-09-30T14:54:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa23be2eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4e39fab20 con 0x55a4e39fbcd0 2023-09-30T14:54:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa23be2eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4e39ff730 con 0x55a4e39fb310 2023-09-30T14:54:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa22d9e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fa200 0x55a4e39f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa22e1e3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fbcd0 0x55a4e39f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa22d9e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fa200 0x55a4e39f4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38876/0 (socket says 172.21.15.32:38876) 2023-09-30T14:54:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa22d1e1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fb310 0x55a4e3a0b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa22d9e2700 1 -- 172.21.15.32:0/3088345087 learned_addr learned my addr 172.21.15.32:0/3088345087 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa22d9e2700 1 -- 172.21.15.32:0/3088345087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fb310 msgr2=0x55a4e3a0b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa22d9e2700 1 --2- 172.21.15.32:0/3088345087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fb310 0x55a4e3a0b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa22d9e2700 1 -- 172.21.15.32:0/3088345087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fbcd0 msgr2=0x55a4e39f4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa22d9e2700 1 --2- 172.21.15.32:0/3088345087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fbcd0 0x55a4e39f4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa22d9e2700 1 -- 172.21.15.32:0/3088345087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4e3777d80 con 0x55a4e39fa200 2023-09-30T14:54:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.678+0000 7fa22d9e2700 1 --2- 172.21.15.32:0/3088345087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fa200 0x55a4e39f4ec0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa2240c9e50 tx=0x7fa224097a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11f256d032b5190 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.679+0000 7fa21cd82700 1 -- 172.21.15.32:0/3088345087 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224082b70 con 0x55a4e39fa200 2023-09-30T14:54:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.679+0000 7fa21cd82700 1 -- 172.21.15.32:0/3088345087 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa22407d050 con 0x55a4e39fa200 2023-09-30T14:54:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.679+0000 7fa21cd82700 1 -- 172.21.15.32:0/3088345087 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240a13a0 con 0x55a4e39fa200 2023-09-30T14:54:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.679+0000 7fa23be2eb80 1 -- 172.21.15.32:0/3088345087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fa200 msgr2=0x55a4e39f4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.679+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/3088345087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fa200 0x55a4e39f4ec0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa2240c9e50 tx=0x7fa224097a50 comp rx=0 tx=0).stop 2023-09-30T14:54:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.680+0000 7fa23be2eb80 1 -- 172.21.15.32:0/3088345087 shutdown_connections 2023-09-30T14:54:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.680+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/3088345087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fbcd0 0x55a4e39f4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.680+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/3088345087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fa200 0x55a4e39f4ec0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.680+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/3088345087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fb310 0x55a4e3a0b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.680+0000 7fa23be2eb80 1 -- 172.21.15.32:0/3088345087 >> 172.21.15.32:0/3088345087 conn(0x55a4e396e200 msgr2=0x55a4e3a02b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.680+0000 7fa23be2eb80 1 -- 172.21.15.32:0/3088345087 shutdown_connections 2023-09-30T14:54:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.680+0000 7fa23be2eb80 1 -- 172.21.15.32:0/3088345087 wait complete. 2023-09-30T14:54:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.680+0000 7fa23be2eb80 1 -- start start 2023-09-30T14:54:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.680+0000 7fa23be2eb80 1 Processor -- start 2023-09-30T14:54:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa23be2eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fa200 0x55a4e3a0ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa23be2eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fb310 0x55a4e3a0b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa23be2eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fbcd0 0x55a4e3a09190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa23be2eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4e3a097b0 con 0x55a4e39fbcd0 2023-09-30T14:54:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa23be2eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4e3a09920 con 0x55a4e39fa200 2023-09-30T14:54:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa23be2eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4e3a09ae0 con 0x55a4e39fb310 2023-09-30T14:54:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa23be2eb80 10 client.0 _ll_register_callbacks cb 0x55a4e37467a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:54:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa22e1e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fbcd0 0x55a4e3a09190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa22e1e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fbcd0 0x55a4e3a09190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38886/0 (socket says 172.21.15.32:38886) 2023-09-30T14:54:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa22d9e2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fa200 0x55a4e3a0ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa22e1e3700 1 -- 172.21.15.32:0/2529742384 learned_addr learned my addr 172.21.15.32:0/2529742384 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa22d1e1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fb310 0x55a4e3a0b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa22e1e3700 1 -- 172.21.15.32:0/2529742384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fb310 msgr2=0x55a4e3a0b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa22e1e3700 1 --2- 172.21.15.32:0/2529742384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fb310 0x55a4e3a0b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa22e1e3700 1 -- 172.21.15.32:0/2529742384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fa200 msgr2=0x55a4e3a0ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa22e1e3700 1 --2- 172.21.15.32:0/2529742384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fa200 0x55a4e3a0ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa22e1e3700 1 -- 172.21.15.32:0/2529742384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa228108b70 con 0x55a4e39fbcd0 2023-09-30T14:54:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa22e1e3700 1 --2- 172.21.15.32:0/2529742384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fbcd0 0x55a4e3a09190 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa220007970 tx=0x7fa220006060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2d6d530213d718 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa2117fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:54:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa2117fa700 1 -- 172.21.15.32:0/2529742384 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22002b070 con 0x55a4e39fbcd0 2023-09-30T14:54:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa23be2eb80 1 -- 172.21.15.32:0/2529742384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2200076e0 con 0x55a4e39fbcd0 2023-09-30T14:54:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.681+0000 7fa2117fa700 1 -- 172.21.15.32:0/2529742384 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa220030460 con 0x55a4e39fbcd0 2023-09-30T14:54:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.682+0000 7fa2117fa700 1 -- 172.21.15.32:0/2529742384 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa220036030 con 0x55a4e39fbcd0 2023-09-30T14:54:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.682+0000 7fa23be2eb80 10 client.16827 fetch_fsmap learned FSMap version 48 2023-09-30T14:54:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.682+0000 7fa23be2eb80 1 -- 172.21.15.32:0/2529742384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55a4e3760c00 con 0x55a4e39fbcd0 2023-09-30T14:54:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.682+0000 7fa2117fa700 1 -- 172.21.15.32:0/2529742384 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2200076e0 con 0x55a4e39fbcd0 2023-09-30T14:54:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.682+0000 7fa2117fa700 1 -- 172.21.15.32:0/2529742384 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 48) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa220024030 con 0x55a4e39fbcd0 2023-09-30T14:54:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.682+0000 7fa23be2eb80 10 client.16827 fetch_fsmap finished waiting for FSMap version 48 2023-09-30T14:54:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.682+0000 7fa23be2eb80 10 client.16827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:54:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.682+0000 7fa23be2eb80 20 client.16827 populate_metadata read hostname 'smithi032' 2023-09-30T14:54:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.682+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/2529742384 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a4e3a75b00 0x55a4e3a77f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.682+0000 7fa23be2eb80 4 client.16827 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:54:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.682+0000 7fa23be2eb80 1 -- 172.21.15.32:0/2529742384 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a4e3945ad0 con 0x55a4e3a75b00 2023-09-30T14:54:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.682+0000 7fa22d9e2700 1 --2- 172.21.15.32:0/2529742384 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a4e3a75b00 0x55a4e3a77f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.683+0000 7fa2117fa700 10 client.16827 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:54:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.683+0000 7fa22d9e2700 1 --2- 172.21.15.32:0/2529742384 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a4e3a75b00 0x55a4e3a77f20 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be32a0de62f72c1a server_cookie=bd35c3e6b4cca50c in_seq=0 out_seq=0 2023-09-30T14:54:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.974+0000 7fa2117fa700 1 -- 172.21.15.32:0/2529742384 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a4e3945ad0 con 0x55a4e3a75b00 2023-09-30T14:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.974+0000 7fa2117fa700 10 client.16827 handle_command_reply: tid=0 2023-09-30T14:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.974+0000 7fa23be2eb80 1 client.16827 shutdown 2023-09-30T14:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.974+0000 7fa23be2eb80 1 -- 172.21.15.32:0/2529742384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fbcd0 msgr2=0x55a4e3a09190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.974+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/2529742384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fbcd0 0x55a4e3a09190 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa220007970 tx=0x7fa220006060 comp rx=0 tx=0).stop 2023-09-30T14:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.974+0000 7fa23be2eb80 1 -- 172.21.15.32:0/2529742384 shutdown_connections 2023-09-30T14:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.974+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/2529742384 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a4e3a75b00 0x55a4e3a77f20 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.974+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/2529742384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a4e39fbcd0 0x55a4e3a09190 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.974+0000 7fa2117fa700 0 client.16827 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.974+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/2529742384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a4e39fa200 0x55a4e3a0ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.974+0000 7fa23be2eb80 1 --2- 172.21.15.32:0/2529742384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4e39fb310 0x55a4e3a0b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.974+0000 7fa23be2eb80 1 -- 172.21.15.32:0/2529742384 >> 172.21.15.32:0/2529742384 conn(0x55a4e396e200 msgr2=0x55a4e3a02090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.974+0000 7fa23be2eb80 1 -- 172.21.15.32:0/2529742384 shutdown_connections 2023-09-30T14:54:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.974+0000 7fa23be2eb80 1 -- 172.21.15.32:0/2529742384 wait complete. 2023-09-30T14:54:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.974+0000 7fa23be2eb80 20 client.16827 trim_cache size 0 max 16384 2023-09-30T14:54:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:54:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:54:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:54:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:54:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.975+0000 7fa22fc46700 1 -- 172.21.15.32:0/1780817080 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa2140a1000 msgr2=0x7fa2140a34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.975+0000 7fa22fc46700 1 --2- 172.21.15.32:0/1780817080 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa2140a1000 0x7fa2140a34c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa218009c60 tx=0x7fa218009260 comp rx=0 tx=0).stop 2023-09-30T14:54:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.975+0000 7fa22fc46700 1 -- 172.21.15.32:0/1780817080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2280fff60 msgr2=0x7fa22810eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.975+0000 7fa22fc46700 1 --2- 172.21.15.32:0/1780817080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2280fff60 0x7fa22810eb60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa22400b450 tx=0x7fa22400b760 comp rx=0 tx=0).stop 2023-09-30T14:54:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.975+0000 7fa22fc46700 1 -- 172.21.15.32:0/1780817080 shutdown_connections 2023-09-30T14:54:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.975+0000 7fa22fc46700 1 --2- 172.21.15.32:0/1780817080 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fa2140a1000 0x7fa2140a34c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.975+0000 7fa22fc46700 1 --2- 172.21.15.32:0/1780817080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2281013a0 0x7fa2281100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.975+0000 7fa22fc46700 1 --2- 172.21.15.32:0/1780817080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2280fff60 0x7fa22810eb60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.975+0000 7fa22fc46700 1 --2- 172.21.15.32:0/1780817080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa228100920 0x7fa22810f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.975+0000 7fa22fc46700 1 -- 172.21.15.32:0/1780817080 >> 172.21.15.32:0/1780817080 conn(0x7fa2280fa9f0 msgr2=0x7fa228104450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.975+0000 7fa22fc46700 1 -- 172.21.15.32:0/1780817080 shutdown_connections 2023-09-30T14:54:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:25.975+0000 7fa22fc46700 1 -- 172.21.15.32:0/1780817080 wait complete. 2023-09-30T14:54:26.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 120434 38.34 MB/sec execute 59 sec latency 211.979 ms 2023-09-30T14:54:26.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:25 smithi156 ceph-mon[103579]: pgmap v771: 97 pgs: 97 active+clean; 699 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 776 op/s 2023-09-30T14:54:26.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:25 smithi148 ceph-mon[102355]: pgmap v771: 97 pgs: 97 active+clean; 699 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 776 op/s 2023-09-30T14:54:26.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:25 smithi032 ceph-mon[98592]: pgmap v771: 97 pgs: 97 active+clean; 699 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 776 op/s 2023-09-30T14:54:27.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 121268 38.63 MB/sec execute 60 sec latency 51.546 ms 2023-09-30T14:54:28.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 122009 38.59 MB/sec execute 61 sec latency 63.550 ms 2023-09-30T14:54:28.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:27 smithi156 ceph-mon[103579]: pgmap v772: 97 pgs: 97 active+clean; 707 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 737 op/s 2023-09-30T14:54:28.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:27 smithi148 ceph-mon[102355]: pgmap v772: 97 pgs: 97 active+clean; 707 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 737 op/s 2023-09-30T14:54:28.144 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:27 smithi032 ceph-mon[98592]: pgmap v772: 97 pgs: 97 active+clean; 707 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 737 op/s 2023-09-30T14:54:29.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 122431 38.34 MB/sec execute 62 sec latency 338.255 ms 2023-09-30T14:54:30.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 123051 38.26 MB/sec execute 63 sec latency 104.229 ms 2023-09-30T14:54:30.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:29 smithi156 ceph-mon[103579]: pgmap v773: 97 pgs: 97 active+clean; 704 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 808 op/s 2023-09-30T14:54:30.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:29 smithi148 ceph-mon[102355]: pgmap v773: 97 pgs: 97 active+clean; 704 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 808 op/s 2023-09-30T14:54:30.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:29 smithi032 ceph-mon[98592]: pgmap v773: 97 pgs: 97 active+clean; 704 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 808 op/s 2023-09-30T14:54:31.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 123773 38.32 MB/sec execute 64 sec latency 74.056 ms 2023-09-30T14:54:32.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 124739 38.35 MB/sec execute 65 sec latency 63.590 ms 2023-09-30T14:54:32.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:31 smithi156 ceph-mon[103579]: pgmap v774: 97 pgs: 97 active+clean; 697 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.7 MiB/s wr, 761 op/s 2023-09-30T14:54:32.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:31 smithi148 ceph-mon[102355]: pgmap v774: 97 pgs: 97 active+clean; 697 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.7 MiB/s wr, 761 op/s 2023-09-30T14:54:32.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:31 smithi032 ceph-mon[98592]: pgmap v774: 97 pgs: 97 active+clean; 697 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.7 MiB/s wr, 761 op/s 2023-09-30T14:54:33.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 125609 38.64 MB/sec execute 66 sec latency 70.558 ms 2023-09-30T14:54:34.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 126044 38.49 MB/sec execute 67 sec latency 272.197 ms 2023-09-30T14:54:34.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:33 smithi156 ceph-mon[103579]: pgmap v775: 97 pgs: 97 active+clean; 681 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.5 MiB/s wr, 746 op/s 2023-09-30T14:54:34.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:33 smithi148 ceph-mon[102355]: pgmap v775: 97 pgs: 97 active+clean; 681 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.5 MiB/s wr, 746 op/s 2023-09-30T14:54:34.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:33 smithi032 ceph-mon[98592]: pgmap v775: 97 pgs: 97 active+clean; 681 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.5 MiB/s wr, 746 op/s 2023-09-30T14:54:35.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 126599 38.31 MB/sec execute 68 sec latency 59.339 ms 2023-09-30T14:54:36.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127169 38.08 MB/sec execute 69 sec latency 93.624 ms 2023-09-30T14:54:36.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:35 smithi156 ceph-mon[103579]: pgmap v776: 97 pgs: 97 active+clean; 688 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 10 MiB/s wr, 839 op/s 2023-09-30T14:54:36.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:35 smithi148 ceph-mon[102355]: pgmap v776: 97 pgs: 97 active+clean; 688 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 10 MiB/s wr, 839 op/s 2023-09-30T14:54:36.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:35 smithi032 ceph-mon[98592]: pgmap v776: 97 pgs: 97 active+clean; 688 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 10 MiB/s wr, 839 op/s 2023-09-30T14:54:37.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 127915 38.20 MB/sec execute 70 sec latency 60.433 ms 2023-09-30T14:54:38.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 128775 38.08 MB/sec execute 71 sec latency 45.184 ms 2023-09-30T14:54:38.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:37 smithi156 ceph-mon[103579]: pgmap v777: 97 pgs: 97 active+clean; 695 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 902 op/s 2023-09-30T14:54:38.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:37 smithi148 ceph-mon[102355]: pgmap v777: 97 pgs: 97 active+clean; 695 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 902 op/s 2023-09-30T14:54:38.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:37 smithi032 ceph-mon[98592]: pgmap v777: 97 pgs: 97 active+clean; 695 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 902 op/s 2023-09-30T14:54:39.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 129381 38.14 MB/sec execute 72 sec latency 216.287 ms 2023-09-30T14:54:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.124+0000 7f540a6c3700 1 -- 172.21.15.32:0/2961087355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5404100f50 msgr2=0x7f54041013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.124+0000 7f540a6c3700 1 --2- 172.21.15.32:0/2961087355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5404100f50 0x7f54041013b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f53f80099c0 tx=0x7f53f8009cd0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.124+0000 7f540a6c3700 1 -- 172.21.15.32:0/2961087355 shutdown_connections 2023-09-30T14:54:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.124+0000 7f540a6c3700 1 --2- 172.21.15.32:0/2961087355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54041018f0 0x7f54041081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.124+0000 7f540a6c3700 1 --2- 172.21.15.32:0/2961087355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54040ffd60 0x7f5404100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.124+0000 7f540a6c3700 1 --2- 172.21.15.32:0/2961087355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5404100f50 0x7f54041013b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.124+0000 7f540a6c3700 1 -- 172.21.15.32:0/2961087355 >> 172.21.15.32:0/2961087355 conn(0x7f54040fb2b0 msgr2=0x7f54040fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.124+0000 7f540a6c3700 1 -- 172.21.15.32:0/2961087355 shutdown_connections 2023-09-30T14:54:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.124+0000 7f540a6c3700 1 -- 172.21.15.32:0/2961087355 wait complete. 2023-09-30T14:54:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.124+0000 7f540a6c3700 1 Processor -- start 2023-09-30T14:54:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.124+0000 7f540a6c3700 1 -- start start 2023-09-30T14:54:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f540a6c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54040ffd60 0x7f5404102690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f540a6c3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5404100f50 0x7f5404102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f540a6c3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54041018f0 0x7f5404103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f540a6c3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041a0bc0 con 0x7f54040ffd60 2023-09-30T14:54:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f540a6c3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041a0d00 con 0x7f5404100f50 2023-09-30T14:54:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f540a6c3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041a0e40 con 0x7f54041018f0 2023-09-30T14:54:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f5403fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54040ffd60 0x7f5404102690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f5403fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54040ffd60 0x7f5404102690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38896/0 (socket says 172.21.15.32:38896) 2023-09-30T14:54:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f5403fff700 1 -- 172.21.15.32:0/1423467655 learned_addr learned my addr 172.21.15.32:0/1423467655 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f54037fe700 1 --2- 172.21.15.32:0/1423467655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5404100f50 0x7f5404102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f5408c60700 1 --2- 172.21.15.32:0/1423467655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54041018f0 0x7f5404103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f5403fff700 1 -- 172.21.15.32:0/1423467655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54041018f0 msgr2=0x7f5404103110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f5403fff700 1 --2- 172.21.15.32:0/1423467655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54041018f0 0x7f5404103110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f5403fff700 1 -- 172.21.15.32:0/1423467655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5404100f50 msgr2=0x7f5404102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f5403fff700 1 --2- 172.21.15.32:0/1423467655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5404100f50 0x7f5404102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f5403fff700 1 -- 172.21.15.32:0/1423467655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53fc009c40 con 0x7f54040ffd60 2023-09-30T14:54:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.125+0000 7f5403fff700 1 --2- 172.21.15.32:0/1423467655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54040ffd60 0x7f5404102690 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f53f400d980 tx=0x7f53f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.126+0000 7f54017fa700 1 -- 172.21.15.32:0/1423467655 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4004020 con 0x7f54040ffd60 2023-09-30T14:54:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.126+0000 7f540a6c3700 1 -- 172.21.15.32:0/1423467655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f80095f0 con 0x7f54040ffd60 2023-09-30T14:54:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.126+0000 7f54017fa700 1 -- 172.21.15.32:0/1423467655 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53f4009730 con 0x7f54040ffd60 2023-09-30T14:54:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.126+0000 7f540a6c3700 1 -- 172.21.15.32:0/1423467655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54041a14a0 con 0x7f54040ffd60 2023-09-30T14:54:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.126+0000 7f54017fa700 1 -- 172.21.15.32:0/1423467655 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4003950 con 0x7f54040ffd60 2023-09-30T14:54:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.128+0000 7f54017fa700 1 -- 172.21.15.32:0/1423467655 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f53f40098a0 con 0x7f54040ffd60 2023-09-30T14:54:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.128+0000 7f54017fa700 1 --2- 172.21.15.32:0/1423467655 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f53ec0a0fa0 0x7f53ec0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.129+0000 7f54017fa700 1 -- 172.21.15.32:0/1423467655 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f53f40cc3d0 con 0x7f54040ffd60 2023-09-30T14:54:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.129+0000 7f54037fe700 1 --2- 172.21.15.32:0/1423467655 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f53ec0a0fa0 0x7f53ec0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.129+0000 7f54037fe700 1 --2- 172.21.15.32:0/1423467655 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f53ec0a0fa0 0x7f53ec0a3460 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009990 tx=0x7f53f801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.130+0000 7f54168abb80 1 Processor -- start 2023-09-30T14:54:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.130+0000 7f54168abb80 1 -- start start 2023-09-30T14:54:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.130+0000 7f54168abb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a5a10 0x5651426a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.130+0000 7f54168abb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a6b20 0x5651426a6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.130+0000 7f54168abb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a74e0 0x5651426add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.130+0000 7f54168abb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x565142605400 con 0x5651426a74e0 2023-09-30T14:54:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.130+0000 7f54168abb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651425f6bc0 con 0x5651426a6b20 2023-09-30T14:54:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.131+0000 7f54168abb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651425f44e0 con 0x5651426a5a10 2023-09-30T14:54:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.131+0000 7f54037fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a6b20 0x5651426a6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.131+0000 7f5408c60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a74e0 0x5651426add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.131+0000 7f5408c60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a74e0 0x5651426add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38908/0 (socket says 172.21.15.32:38908) 2023-09-30T14:54:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.131+0000 7f5408c60700 1 -- 172.21.15.32:0/3866633534 learned_addr learned my addr 172.21.15.32:0/3866633534 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.131+0000 7f54037fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a6b20 0x5651426a6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:50382/0 (socket says 172.21.15.32:50382) 2023-09-30T14:54:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.131+0000 7f5403fff700 1 --2- 172.21.15.32:0/3866633534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a5a10 0x5651426a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.131+0000 7f5408c60700 1 -- 172.21.15.32:0/3866633534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a5a10 msgr2=0x5651426a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.131+0000 7f5408c60700 1 --2- 172.21.15.32:0/3866633534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a5a10 0x5651426a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.131+0000 7f5408c60700 1 -- 172.21.15.32:0/3866633534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a6b20 msgr2=0x5651426a6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.131+0000 7f5408c60700 1 --2- 172.21.15.32:0/3866633534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a6b20 0x5651426a6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.131+0000 7f5408c60700 1 -- 172.21.15.32:0/3866633534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565142652870 con 0x5651426a74e0 2023-09-30T14:54:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.131+0000 7f54037fe700 1 --2- 172.21.15.32:0/3866633534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a6b20 0x5651426a6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:54:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.131+0000 7f5408c60700 1 --2- 172.21.15.32:0/3866633534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a74e0 0x5651426add70 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00e730 tx=0x7f53fc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c98b40048415d2a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.131+0000 7f53ea7fc700 1 -- 172.21.15.32:0/3866633534 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc00e040 con 0x5651426a74e0 2023-09-30T14:54:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.132+0000 7f53ea7fc700 1 -- 172.21.15.32:0/3866633534 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53fc00f040 con 0x5651426a74e0 2023-09-30T14:54:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.132+0000 7f53ea7fc700 1 -- 172.21.15.32:0/3866633534 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc007500 con 0x5651426a74e0 2023-09-30T14:54:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.132+0000 7f54168abb80 1 -- 172.21.15.32:0/3866633534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a74e0 msgr2=0x5651426add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.132+0000 7f54168abb80 1 --2- 172.21.15.32:0/3866633534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a74e0 0x5651426add70 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00e730 tx=0x7f53fc009920 comp rx=0 tx=0).stop 2023-09-30T14:54:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.132+0000 7f54168abb80 1 -- 172.21.15.32:0/3866633534 shutdown_connections 2023-09-30T14:54:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.132+0000 7f54168abb80 1 --2- 172.21.15.32:0/3866633534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a74e0 0x5651426add70 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.132+0000 7f54168abb80 1 --2- 172.21.15.32:0/3866633534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a5a10 0x5651426a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.132+0000 7f54168abb80 1 --2- 172.21.15.32:0/3866633534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a6b20 0x5651426a6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.132+0000 7f54168abb80 1 -- 172.21.15.32:0/3866633534 >> 172.21.15.32:0/3866633534 conn(0x565142611200 msgr2=0x565142697ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.132+0000 7f54168abb80 1 -- 172.21.15.32:0/3866633534 shutdown_connections 2023-09-30T14:54:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.132+0000 7f54168abb80 1 -- 172.21.15.32:0/3866633534 wait complete. 2023-09-30T14:54:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.133+0000 7f54168abb80 1 -- start start 2023-09-30T14:54:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.133+0000 7f54168abb80 1 Processor -- start 2023-09-30T14:54:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.133+0000 7f54168abb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a5a10 0x56514269e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f54168abb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a6b20 0x56514269e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f54168abb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a74e0 0x5651426a3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f54168abb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651425efdd0 con 0x5651426a6b20 2023-09-30T14:54:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f54168abb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56514258c6b0 con 0x5651426a74e0 2023-09-30T14:54:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f54168abb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56514258c350 con 0x5651426a5a10 2023-09-30T14:54:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f54168abb80 10 client.0 _ll_register_callbacks cb 0x5651423e97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:54:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f54037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a6b20 0x56514269e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f54037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a6b20 0x56514269e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38912/0 (socket says 172.21.15.32:38912) 2023-09-30T14:54:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f5403fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a5a10 0x56514269e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f54037fe700 1 -- 172.21.15.32:0/4001543593 learned_addr learned my addr 172.21.15.32:0/4001543593 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f5408c60700 1 --2- 172.21.15.32:0/4001543593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a74e0 0x5651426a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f5403fff700 1 -- 172.21.15.32:0/4001543593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a74e0 msgr2=0x5651426a3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f5403fff700 1 --2- 172.21.15.32:0/4001543593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a74e0 0x5651426a3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f5403fff700 1 -- 172.21.15.32:0/4001543593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a6b20 msgr2=0x56514269e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f5403fff700 1 --2- 172.21.15.32:0/4001543593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a6b20 0x56514269e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f5403fff700 1 -- 172.21.15.32:0/4001543593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5404108af0 con 0x5651426a5a10 2023-09-30T14:54:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f54037fe700 1 --2- 172.21.15.32:0/4001543593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a6b20 0x56514269e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:54:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f5403fff700 1 --2- 172.21.15.32:0/4001543593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a5a10 0x56514269e2a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f53f4099540 tx=0x7f53f4099270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38f9a6e1b890a112 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f53e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:54:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.134+0000 7f53e37fe700 1 -- 172.21.15.32:0/4001543593 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f409d990 con 0x5651426a5a10 2023-09-30T14:54:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.135+0000 7f53e37fe700 1 -- 172.21.15.32:0/4001543593 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f53f40a1bd0 con 0x5651426a5a10 2023-09-30T14:54:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.135+0000 7f54168abb80 1 -- 172.21.15.32:0/4001543593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565142690090 con 0x5651426a5a10 2023-09-30T14:54:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.135+0000 7f53e37fe700 1 -- 172.21.15.32:0/4001543593 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53f40840b0 con 0x5651426a5a10 2023-09-30T14:54:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.135+0000 7f54168abb80 10 client.26494 fetch_fsmap learned FSMap version 48 2023-09-30T14:54:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.135+0000 7f54168abb80 1 -- 172.21.15.32:0/4001543593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x7f53fc00e410 con 0x5651426a5a10 2023-09-30T14:54:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.135+0000 7f53e37fe700 1 -- 172.21.15.32:0/4001543593 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565142690090 con 0x5651426a5a10 2023-09-30T14:54:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.135+0000 7f53e37fe700 1 -- 172.21.15.32:0/4001543593 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 48) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f53f4098d30 con 0x5651426a5a10 2023-09-30T14:54:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.135+0000 7f54168abb80 10 client.26494 fetch_fsmap finished waiting for FSMap version 48 2023-09-30T14:54:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.135+0000 7f54168abb80 10 client.26494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:54:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.135+0000 7f54168abb80 20 client.26494 populate_metadata read hostname 'smithi032' 2023-09-30T14:54:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.135+0000 7f54168abb80 1 --2- 172.21.15.32:0/4001543593 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5651426c3bc0 0x5651426c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.135+0000 7f54168abb80 4 client.26494 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:54:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.135+0000 7f54168abb80 1 -- 172.21.15.32:0/4001543593 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651425e8db0 con 0x5651426c3bc0 2023-09-30T14:54:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.135+0000 7f54037fe700 1 --2- 172.21.15.32:0/4001543593 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5651426c3bc0 0x5651426c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.137+0000 7f54037fe700 1 --2- 172.21.15.32:0/4001543593 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5651426c3bc0 0x5651426c5fe0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3abbb1fe06572cf7 server_cookie=be29cb7ddb391cac in_seq=0 out_seq=0 2023-09-30T14:54:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.137+0000 7f53e37fe700 10 client.26494 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:54:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.142+0000 7f53e37fe700 1 -- 172.21.15.32:0/4001543593 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5651425e8db0 con 0x5651426c3bc0 2023-09-30T14:54:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.142+0000 7f53e37fe700 10 client.26494 handle_command_reply: tid=0 2023-09-30T14:54:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.142+0000 7f54168abb80 1 client.26494 shutdown 2023-09-30T14:54:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.142+0000 7f54168abb80 1 -- 172.21.15.32:0/4001543593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a5a10 msgr2=0x56514269e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.142+0000 7f54168abb80 1 --2- 172.21.15.32:0/4001543593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a5a10 0x56514269e2a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f53f4099540 tx=0x7f53f4099270 comp rx=0 tx=0).stop 2023-09-30T14:54:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.142+0000 7f54168abb80 1 -- 172.21.15.32:0/4001543593 shutdown_connections 2023-09-30T14:54:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.142+0000 7f54168abb80 1 --2- 172.21.15.32:0/4001543593 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5651426c3bc0 0x5651426c5fe0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.142+0000 7f54168abb80 1 --2- 172.21.15.32:0/4001543593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a74e0 0x5651426a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.142+0000 7f54168abb80 1 --2- 172.21.15.32:0/4001543593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a5a10 0x56514269e2a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.142+0000 7f54168abb80 1 --2- 172.21.15.32:0/4001543593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a6b20 0x56514269e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.142+0000 7f53e37fe700 0 client.26494 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:54:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.142+0000 7f54168abb80 1 -- 172.21.15.32:0/4001543593 >> 172.21.15.32:0/4001543593 conn(0x565142611200 msgr2=0x565142697590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.142+0000 7f54168abb80 1 -- 172.21.15.32:0/4001543593 shutdown_connections 2023-09-30T14:54:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.142+0000 7f54168abb80 1 -- 172.21.15.32:0/4001543593 wait complete. 2023-09-30T14:54:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.142+0000 7f54168abb80 20 client.26494 trim_cache size 0 max 16384 2023-09-30T14:54:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.161+0000 7f54168abb80 1 Processor -- start 2023-09-30T14:54:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.161+0000 7f54168abb80 1 -- start start 2023-09-30T14:54:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.161+0000 7f54168abb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a5a10 0x565142699630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.161+0000 7f54168abb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a6b20 0x56514269cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.161+0000 7f54168abb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a74e0 0x5651426d37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.161+0000 7f54168abb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651424708c0 con 0x5651426a5a10 2023-09-30T14:54:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.161+0000 7f54168abb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651426c3760 con 0x5651426a6b20 2023-09-30T14:54:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.161+0000 7f54168abb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651426a9fa0 con 0x5651426a74e0 2023-09-30T14:54:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.161+0000 7f5403fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a5a10 0x565142699630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.161+0000 7f5403fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a5a10 0x565142699630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38918/0 (socket says 172.21.15.32:38918) 2023-09-30T14:54:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.161+0000 7f5403fff700 1 -- 172.21.15.32:0/1498095299 learned_addr learned my addr 172.21.15.32:0/1498095299 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.161+0000 7f54037fe700 1 --2- 172.21.15.32:0/1498095299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a6b20 0x56514269cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.161+0000 7f5408c60700 1 --2- 172.21.15.32:0/1498095299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a74e0 0x5651426d37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.164 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.162+0000 7f5403fff700 1 -- 172.21.15.32:0/1498095299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a74e0 msgr2=0x5651426d37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.162+0000 7f5403fff700 1 --2- 172.21.15.32:0/1498095299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a74e0 0x5651426d37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.162+0000 7f5403fff700 1 -- 172.21.15.32:0/1498095299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a6b20 msgr2=0x56514269cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.162+0000 7f5403fff700 1 --2- 172.21.15.32:0/1498095299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a6b20 0x56514269cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.162+0000 7f5403fff700 1 -- 172.21.15.32:0/1498095299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56514241ad80 con 0x5651426a5a10 2023-09-30T14:54:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.162+0000 7f5403fff700 1 --2- 172.21.15.32:0/1498095299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a5a10 0x565142699630 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f53f4084010 tx=0x7f53f4097df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f358eeb4da69a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.162+0000 7f53eaffd700 1 -- 172.21.15.32:0/1498095299 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f40a19c0 con 0x5651426a5a10 2023-09-30T14:54:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.162+0000 7f53eaffd700 1 -- 172.21.15.32:0/1498095299 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53f40a1bd0 con 0x5651426a5a10 2023-09-30T14:54:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.162+0000 7f53eaffd700 1 -- 172.21.15.32:0/1498095299 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f40840b0 con 0x5651426a5a10 2023-09-30T14:54:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.163+0000 7f54168abb80 1 -- 172.21.15.32:0/1498095299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a5a10 msgr2=0x565142699630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.163+0000 7f54168abb80 1 --2- 172.21.15.32:0/1498095299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a5a10 0x565142699630 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f53f4084010 tx=0x7f53f4097df0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.163+0000 7f54168abb80 1 -- 172.21.15.32:0/1498095299 shutdown_connections 2023-09-30T14:54:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.163+0000 7f54168abb80 1 --2- 172.21.15.32:0/1498095299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a74e0 0x5651426d37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.163+0000 7f54168abb80 1 --2- 172.21.15.32:0/1498095299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a5a10 0x565142699630 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.163+0000 7f54168abb80 1 --2- 172.21.15.32:0/1498095299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a6b20 0x56514269cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.163+0000 7f54168abb80 1 -- 172.21.15.32:0/1498095299 >> 172.21.15.32:0/1498095299 conn(0x565142611200 msgr2=0x5651426ca940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.164+0000 7f54168abb80 1 -- 172.21.15.32:0/1498095299 shutdown_connections 2023-09-30T14:54:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.164+0000 7f54168abb80 1 -- 172.21.15.32:0/1498095299 wait complete. 2023-09-30T14:54:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.164+0000 7f54168abb80 1 -- start start 2023-09-30T14:54:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.164+0000 7f54168abb80 1 Processor -- start 2023-09-30T14:54:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f54168abb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a5a10 0x5651426d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f54168abb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a6b20 0x5651426d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f54168abb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a74e0 0x5651427162b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f54168abb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651427167f0 con 0x5651426a6b20 2023-09-30T14:54:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f54168abb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x565142716930 con 0x5651426a5a10 2023-09-30T14:54:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f54168abb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x565142716af0 con 0x5651426a74e0 2023-09-30T14:54:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f54168abb80 10 client.0 _ll_register_callbacks cb 0x7f53f4099380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:54:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f54037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a6b20 0x5651426d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f5408c60700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a74e0 0x5651427162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f54037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a6b20 0x5651426d30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38926/0 (socket says 172.21.15.32:38926) 2023-09-30T14:54:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f5403fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a5a10 0x5651426d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f5408c60700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a74e0 0x5651427162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53310/0 (socket says 172.21.15.32:53310) 2023-09-30T14:54:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f54037fe700 1 -- 172.21.15.32:0/3320283979 learned_addr learned my addr 172.21.15.32:0/3320283979 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f5408c60700 1 -- 172.21.15.32:0/3320283979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a5a10 msgr2=0x5651426d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f5408c60700 1 --2- 172.21.15.32:0/3320283979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a5a10 0x5651426d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f5408c60700 1 -- 172.21.15.32:0/3320283979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a6b20 msgr2=0x5651426d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f5408c60700 1 --2- 172.21.15.32:0/3320283979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a6b20 0x5651426d30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f5408c60700 1 -- 172.21.15.32:0/3320283979 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53fc00e410 con 0x5651426a74e0 2023-09-30T14:54:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f5408c60700 1 --2- 172.21.15.32:0/3320283979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a74e0 0x5651427162b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00ed20 tx=0x7f53fc003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2133166abd3d9fa3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f53e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:54:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.165+0000 7f53e37fe700 1 -- 172.21.15.32:0/3320283979 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc025070 con 0x5651426a74e0 2023-09-30T14:54:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.166+0000 7f54168abb80 1 -- 172.21.15.32:0/3320283979 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53fc00f040 con 0x5651426a74e0 2023-09-30T14:54:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.166+0000 7f53e37fe700 1 -- 172.21.15.32:0/3320283979 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f53fc007790 con 0x5651426a74e0 2023-09-30T14:54:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.166+0000 7f53e37fe700 1 -- 172.21.15.32:0/3320283979 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53fc00e040 con 0x5651426a74e0 2023-09-30T14:54:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.166+0000 7f54168abb80 10 client.26500 fetch_fsmap learned FSMap version 48 2023-09-30T14:54:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.166+0000 7f54168abb80 1 -- 172.21.15.32:0/3320283979 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x565142403c00 con 0x5651426a74e0 2023-09-30T14:54:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.166+0000 7f53e37fe700 1 -- 172.21.15.32:0/3320283979 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53fc00f040 con 0x5651426a74e0 2023-09-30T14:54:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.166+0000 7f53e37fe700 1 -- 172.21.15.32:0/3320283979 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 48) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f53fc015c10 con 0x5651426a74e0 2023-09-30T14:54:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.166+0000 7f54168abb80 10 client.26500 fetch_fsmap finished waiting for FSMap version 48 2023-09-30T14:54:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.166+0000 7f54168abb80 10 client.26500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:54:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.166+0000 7f54168abb80 20 client.26500 populate_metadata read hostname 'smithi032' 2023-09-30T14:54:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.166+0000 7f54168abb80 1 --2- 172.21.15.32:0/3320283979 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x565142718480 0x56514271a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.166+0000 7f54168abb80 4 client.26500 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:54:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.166+0000 7f54168abb80 1 -- 172.21.15.32:0/3320283979 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5651425e8ad0 con 0x565142718480 2023-09-30T14:54:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.166+0000 7f5403fff700 1 --2- 172.21.15.32:0/3320283979 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x565142718480 0x56514271a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.167+0000 7f53e37fe700 10 client.26500 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:54:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.167+0000 7f5403fff700 1 --2- 172.21.15.32:0/3320283979 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x565142718480 0x56514271a870 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e78da41969ee3b86 server_cookie=4fc7125d6df27ff5 in_seq=0 out_seq=0 2023-09-30T14:54:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.467+0000 7f53e37fe700 1 -- 172.21.15.32:0/3320283979 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5651425e8ad0 con 0x565142718480 2023-09-30T14:54:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.467+0000 7f53e37fe700 10 client.26500 handle_command_reply: tid=0 2023-09-30T14:54:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.467+0000 7f54168abb80 1 client.26500 shutdown 2023-09-30T14:54:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.467+0000 7f54168abb80 1 -- 172.21.15.32:0/3320283979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a74e0 msgr2=0x5651427162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.467+0000 7f54168abb80 1 --2- 172.21.15.32:0/3320283979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a74e0 0x5651427162b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00ed20 tx=0x7f53fc003f90 comp rx=0 tx=0).stop 2023-09-30T14:54:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.467+0000 7f54168abb80 1 -- 172.21.15.32:0/3320283979 shutdown_connections 2023-09-30T14:54:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.467+0000 7f54168abb80 1 --2- 172.21.15.32:0/3320283979 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x565142718480 0x56514271a870 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.467+0000 7f54168abb80 1 --2- 172.21.15.32:0/3320283979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651426a74e0 0x5651427162b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.467+0000 7f54168abb80 1 --2- 172.21.15.32:0/3320283979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651426a5a10 0x5651426d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.467+0000 7f54168abb80 1 --2- 172.21.15.32:0/3320283979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5651426a6b20 0x5651426d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.467+0000 7f54168abb80 1 -- 172.21.15.32:0/3320283979 >> 172.21.15.32:0/3320283979 conn(0x565142611200 msgr2=0x56514269cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.468+0000 7f53e37fe700 0 client.26500 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:54:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.468+0000 7f54168abb80 1 -- 172.21.15.32:0/3320283979 shutdown_connections 2023-09-30T14:54:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.468+0000 7f54168abb80 1 -- 172.21.15.32:0/3320283979 wait complete. 2023-09-30T14:54:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.468+0000 7f54168abb80 20 client.26500 trim_cache size 0 max 16384 2023-09-30T14:54:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:54:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:54:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:54:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:54:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.468+0000 7f540a6c3700 1 -- 172.21.15.32:0/1423467655 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f53ec0a0fa0 msgr2=0x7f53ec0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.468+0000 7f540a6c3700 1 --2- 172.21.15.32:0/1423467655 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f53ec0a0fa0 0x7f53ec0a3460 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009990 tx=0x7f53f801a040 comp rx=0 tx=0).stop 2023-09-30T14:54:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.468+0000 7f540a6c3700 1 -- 172.21.15.32:0/1423467655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54040ffd60 msgr2=0x7f5404102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.468+0000 7f540a6c3700 1 --2- 172.21.15.32:0/1423467655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54040ffd60 0x7f5404102690 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f53f400d980 tx=0x7f53f400dc90 comp rx=0 tx=0).stop 2023-09-30T14:54:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.470+0000 7f540a6c3700 1 -- 172.21.15.32:0/1423467655 shutdown_connections 2023-09-30T14:54:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.470+0000 7f540a6c3700 1 --2- 172.21.15.32:0/1423467655 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f53ec0a0fa0 0x7f53ec0a3460 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.470+0000 7f540a6c3700 1 --2- 172.21.15.32:0/1423467655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54041018f0 0x7f5404103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.470+0000 7f540a6c3700 1 --2- 172.21.15.32:0/1423467655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54040ffd60 0x7f5404102690 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.470+0000 7f540a6c3700 1 --2- 172.21.15.32:0/1423467655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5404100f50 0x7f5404102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.470+0000 7f540a6c3700 1 -- 172.21.15.32:0/1423467655 >> 172.21.15.32:0/1423467655 conn(0x7f54040fb2b0 msgr2=0x7f54040fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.470+0000 7f540a6c3700 1 -- 172.21.15.32:0/1423467655 shutdown_connections 2023-09-30T14:54:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:39.470+0000 7f540a6c3700 1 -- 172.21.15.32:0/1423467655 wait complete. 2023-09-30T14:54:40.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 129956 38.24 MB/sec execute 73 sec latency 178.627 ms 2023-09-30T14:54:40.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:39 smithi156 ceph-mon[103579]: pgmap v778: 97 pgs: 97 active+clean; 699 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 861 op/s 2023-09-30T14:54:40.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:39 smithi148 ceph-mon[102355]: pgmap v778: 97 pgs: 97 active+clean; 699 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 861 op/s 2023-09-30T14:54:40.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:39 smithi032 ceph-mon[98592]: pgmap v778: 97 pgs: 97 active+clean; 699 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 861 op/s 2023-09-30T14:54:41.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 130467 38.07 MB/sec execute 74 sec latency 75.830 ms 2023-09-30T14:54:41.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:40 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:54:41.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:40 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:54:41.142 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:40 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:54:42.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 131012 37.80 MB/sec execute 75 sec latency 76.394 ms 2023-09-30T14:54:42.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:41 smithi156 ceph-mon[103579]: pgmap v779: 97 pgs: 97 active+clean; 700 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.6 MiB/s wr, 703 op/s 2023-09-30T14:54:42.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:41 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:54:42.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:41 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:54:42.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:41 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:54:42.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:41 smithi148 ceph-mon[102355]: pgmap v779: 97 pgs: 97 active+clean; 700 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.6 MiB/s wr, 703 op/s 2023-09-30T14:54:42.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:41 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:54:42.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:41 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:54:42.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:41 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:54:42.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:41 smithi032 ceph-mon[98592]: pgmap v779: 97 pgs: 97 active+clean; 700 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.6 MiB/s wr, 703 op/s 2023-09-30T14:54:42.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:41 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:54:42.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:41 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:54:42.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:41 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:54:43.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 131793 37.89 MB/sec execute 76 sec latency 60.618 ms 2023-09-30T14:54:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:43 smithi156 ceph-mon[103579]: pgmap v780: 97 pgs: 97 active+clean; 702 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 759 op/s 2023-09-30T14:54:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:43 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:54:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:43 smithi148 ceph-mon[102355]: pgmap v780: 97 pgs: 97 active+clean; 702 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 759 op/s 2023-09-30T14:54:43.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:43 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:54:43.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:43 smithi032 ceph-mon[98592]: pgmap v780: 97 pgs: 97 active+clean; 702 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 759 op/s 2023-09-30T14:54:43.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:43 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:54:44.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 132462 37.77 MB/sec execute 77 sec latency 291.215 ms 2023-09-30T14:54:45.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 133199 37.81 MB/sec execute 78 sec latency 51.230 ms 2023-09-30T14:54:45.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:45 smithi156 ceph-mon[103579]: pgmap v781: 97 pgs: 97 active+clean; 706 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 873 op/s 2023-09-30T14:54:45.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:45 smithi148 ceph-mon[102355]: pgmap v781: 97 pgs: 97 active+clean; 706 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 873 op/s 2023-09-30T14:54:45.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:45 smithi032 ceph-mon[98592]: pgmap v781: 97 pgs: 97 active+clean; 706 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 873 op/s 2023-09-30T14:54:46.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 134150 38.06 MB/sec execute 79 sec latency 52.444 ms 2023-09-30T14:54:47.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 134783 38.12 MB/sec execute 80 sec latency 67.699 ms 2023-09-30T14:54:47.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:47 smithi156 ceph-mon[103579]: pgmap v782: 97 pgs: 97 active+clean; 707 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 830 op/s 2023-09-30T14:54:48.151 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:47 smithi032 ceph-mon[98592]: pgmap v782: 97 pgs: 97 active+clean; 707 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 830 op/s 2023-09-30T14:54:48.151 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:47 smithi148 ceph-mon[102355]: pgmap v782: 97 pgs: 97 active+clean; 707 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 830 op/s 2023-09-30T14:54:48.151 INFO:tasks.workunit.client.0.smithi032.stdout: 10 135353 37.92 MB/sec execute 81 sec latency 82.704 ms 2023-09-30T14:54:49.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 135943 37.84 MB/sec execute 82 sec latency 212.049 ms 2023-09-30T14:54:49.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:49 smithi156 ceph-mon[103579]: pgmap v783: 97 pgs: 97 active+clean; 714 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 802 op/s 2023-09-30T14:54:49.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:49 smithi148 ceph-mon[102355]: pgmap v783: 97 pgs: 97 active+clean; 714 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 802 op/s 2023-09-30T14:54:49.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:49 smithi032 ceph-mon[98592]: pgmap v783: 97 pgs: 97 active+clean; 714 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 802 op/s 2023-09-30T14:54:50.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 136878 37.92 MB/sec execute 83 sec latency 66.241 ms 2023-09-30T14:54:51.010 INFO:tasks.workunit.client.0.smithi032.stdout: 10 137659 38.08 MB/sec execute 84 sec latency 54.285 ms 2023-09-30T14:54:51.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:51 smithi156 ceph-mon[103579]: pgmap v784: 97 pgs: 97 active+clean; 717 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.5 MiB/s wr, 741 op/s 2023-09-30T14:54:51.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:51 smithi148 ceph-mon[102355]: pgmap v784: 97 pgs: 97 active+clean; 717 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.5 MiB/s wr, 741 op/s 2023-09-30T14:54:51.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:51 smithi032 ceph-mon[98592]: pgmap v784: 97 pgs: 97 active+clean; 717 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.5 MiB/s wr, 741 op/s 2023-09-30T14:54:52.009 INFO:tasks.workunit.client.0.smithi032.stdout: 10 138270 38.07 MB/sec execute 85 sec latency 87.284 ms 2023-09-30T14:54:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.610+0000 7fbaf0a8c700 1 -- 172.21.15.32:0/3023305044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaec0ffd70 msgr2=0x7fbaec100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.610+0000 7fbaf0a8c700 1 --2- 172.21.15.32:0/3023305044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaec0ffd70 0x7fbaec100190 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0099c0 tx=0x7fbadc009cd0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.611+0000 7fbaf0a8c700 1 -- 172.21.15.32:0/3023305044 shutdown_connections 2023-09-30T14:54:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.611+0000 7fbaf0a8c700 1 --2- 172.21.15.32:0/3023305044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaec101900 0x7fbaec1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.611+0000 7fbaf0a8c700 1 --2- 172.21.15.32:0/3023305044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaec0ffd70 0x7fbaec100190 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.611+0000 7fbaf0a8c700 1 --2- 172.21.15.32:0/3023305044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbaec100f60 0x7fbaec1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.611+0000 7fbaf0a8c700 1 -- 172.21.15.32:0/3023305044 >> 172.21.15.32:0/3023305044 conn(0x7fbaec0fb300 msgr2=0x7fbaec0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.611+0000 7fbaf0a8c700 1 -- 172.21.15.32:0/3023305044 shutdown_connections 2023-09-30T14:54:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.611+0000 7fbaf0a8c700 1 -- 172.21.15.32:0/3023305044 wait complete. 2023-09-30T14:54:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.612+0000 7fbaf0a8c700 1 Processor -- start 2023-09-30T14:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.612+0000 7fbaf0a8c700 1 -- start start 2023-09-30T14:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.612+0000 7fbaf0a8c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbaec0ffd70 0x7fbaec195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.612+0000 7fbaf0a8c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaec100f60 0x7fbaec196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.613+0000 7fbaf0a8c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaec101900 0x7fbaec196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.613+0000 7fbaf0a8c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec1975f0 con 0x7fbaec101900 2023-09-30T14:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.613+0000 7fbaf0a8c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec197760 con 0x7fbaec100f60 2023-09-30T14:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.613+0000 7fbaf0a8c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec197920 con 0x7fbaec0ffd70 2023-09-30T14:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.613+0000 7fbaea59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbaec0ffd70 0x7fbaec195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.613+0000 7fbaea59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbaec0ffd70 0x7fbaec195bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53320/0 (socket says 172.21.15.32:53320) 2023-09-30T14:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.613+0000 7fbaea59c700 1 -- 172.21.15.32:0/3433124431 learned_addr learned my addr 172.21.15.32:0/3433124431 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.613+0000 7fbae9d9b700 1 --2- 172.21.15.32:0/3433124431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaec100f60 0x7fbaec196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.613+0000 7fbaea59c700 1 -- 172.21.15.32:0/3433124431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaec100f60 msgr2=0x7fbaec196110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.613+0000 7fbaea59c700 1 --2- 172.21.15.32:0/3433124431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaec100f60 0x7fbaec196110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.613+0000 7fbaea59c700 1 -- 172.21.15.32:0/3433124431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaec101900 msgr2=0x7fbaec196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.613+0000 7fbaea59c700 1 --2- 172.21.15.32:0/3433124431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaec101900 0x7fbaec196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.613+0000 7fbaea59c700 1 -- 172.21.15.32:0/3433124431 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbadc0095f0 con 0x7fbaec0ffd70 2023-09-30T14:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.613+0000 7fbaea59c700 1 --2- 172.21.15.32:0/3433124431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbaec0ffd70 0x7fbaec195bd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009990 tx=0x7fbadc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.614+0000 7fbadb7fe700 1 -- 172.21.15.32:0/3433124431 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc01d070 con 0x7fbaec0ffd70 2023-09-30T14:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.614+0000 7fbadb7fe700 1 -- 172.21.15.32:0/3433124431 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbadc0054f0 con 0x7fbaec0ffd70 2023-09-30T14:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.614+0000 7fbadb7fe700 1 -- 172.21.15.32:0/3433124431 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc004a90 con 0x7fbaec0ffd70 2023-09-30T14:54:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.614+0000 7fbaf0a8c700 1 -- 172.21.15.32:0/3433124431 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaec197ba0 con 0x7fbaec0ffd70 2023-09-30T14:54:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.614+0000 7fbaf0a8c700 1 -- 172.21.15.32:0/3433124431 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaec06aa20 con 0x7fbaec0ffd70 2023-09-30T14:54:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.617+0000 7fbadb7fe700 1 -- 172.21.15.32:0/3433124431 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fbadc00fb80 con 0x7fbaec0ffd70 2023-09-30T14:54:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.618+0000 7fbadb7fe700 1 --2- 172.21.15.32:0/3433124431 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbad40a1020 0x7fbad40a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.618+0000 7fbadb7fe700 1 -- 172.21.15.32:0/3433124431 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbadc0cd450 con 0x7fbaec0ffd70 2023-09-30T14:54:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.618+0000 7fbae9d9b700 1 --2- 172.21.15.32:0/3433124431 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbad40a1020 0x7fbad40a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.619+0000 7fbae9d9b700 1 --2- 172.21.15.32:0/3433124431 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbad40a1020 0x7fbad40a34e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009c40 tx=0x7fbae0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.619+0000 7fbafcc74b80 1 Processor -- start 2023-09-30T14:54:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.619+0000 7fbafcc74b80 1 -- start start 2023-09-30T14:54:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.619+0000 7fbafcc74b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55769329f200 0x55769329f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.619+0000 7fbafcc74b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576932a0310 0x5576932a0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.619+0000 7fbafcc74b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0cd0 0x5576932a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.619+0000 7fbafcc74b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557693207400 con 0x5576932a0cd0 2023-09-30T14:54:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.619+0000 7fbafcc74b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576931f8bc0 con 0x5576932a0310 2023-09-30T14:54:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.619+0000 7fbafcc74b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576931f64e0 con 0x55769329f200 2023-09-30T14:54:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.620+0000 7fbaead9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0cd0 0x5576932a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.620+0000 7fbaead9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0cd0 0x5576932a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38950/0 (socket says 172.21.15.32:38950) 2023-09-30T14:54:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.620+0000 7fbae9d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576932a0310 0x5576932a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.620+0000 7fbaead9d700 1 -- 172.21.15.32:0/3024418475 learned_addr learned my addr 172.21.15.32:0/3024418475 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.620+0000 7fbaea59c700 1 --2- 172.21.15.32:0/3024418475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55769329f200 0x55769329f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.620+0000 7fbaead9d700 1 -- 172.21.15.32:0/3024418475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55769329f200 msgr2=0x55769329f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.620+0000 7fbaead9d700 1 --2- 172.21.15.32:0/3024418475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55769329f200 0x55769329f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.620+0000 7fbaead9d700 1 -- 172.21.15.32:0/3024418475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576932a0310 msgr2=0x5576932a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.620+0000 7fbaead9d700 1 --2- 172.21.15.32:0/3024418475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576932a0310 0x5576932a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.620+0000 7fbaead9d700 1 -- 172.21.15.32:0/3024418475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557693254870 con 0x5576932a0cd0 2023-09-30T14:54:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.620+0000 7fbaead9d700 1 --2- 172.21.15.32:0/3024418475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0cd0 0x5576932a7560 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fbae4030410 tx=0x7fbae4030720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d498a44d10364fd1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.620+0000 7fbad8ff9700 1 -- 172.21.15.32:0/3024418475 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae4030e10 con 0x5576932a0cd0 2023-09-30T14:54:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.621+0000 7fbad8ff9700 1 -- 172.21.15.32:0/3024418475 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbae4004bb0 con 0x5576932a0cd0 2023-09-30T14:54:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.621+0000 7fbad8ff9700 1 -- 172.21.15.32:0/3024418475 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae4004240 con 0x5576932a0cd0 2023-09-30T14:54:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.621+0000 7fbafcc74b80 1 -- 172.21.15.32:0/3024418475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0cd0 msgr2=0x5576932a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.621+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/3024418475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0cd0 0x5576932a7560 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fbae4030410 tx=0x7fbae4030720 comp rx=0 tx=0).stop 2023-09-30T14:54:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.621+0000 7fbafcc74b80 1 -- 172.21.15.32:0/3024418475 shutdown_connections 2023-09-30T14:54:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.621+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/3024418475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0cd0 0x5576932a7560 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.621+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/3024418475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55769329f200 0x55769329f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.621+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/3024418475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576932a0310 0x5576932a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.621+0000 7fbafcc74b80 1 -- 172.21.15.32:0/3024418475 >> 172.21.15.32:0/3024418475 conn(0x557693213200 msgr2=0x557693299ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.622+0000 7fbafcc74b80 1 -- 172.21.15.32:0/3024418475 shutdown_connections 2023-09-30T14:54:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.622+0000 7fbafcc74b80 1 -- 172.21.15.32:0/3024418475 wait complete. 2023-09-30T14:54:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.622+0000 7fbafcc74b80 1 -- start start 2023-09-30T14:54:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.622+0000 7fbafcc74b80 1 Processor -- start 2023-09-30T14:54:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbafcc74b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55769329f200 0x5576932ece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbafcc74b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0310 0x5576932ed390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbafcc74b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576932a0cd0 0x5576932f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbafcc74b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576931f1dd0 con 0x5576932a0310 2023-09-30T14:54:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbafcc74b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769318e6b0 con 0x55769329f200 2023-09-30T14:54:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbafcc74b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769318e350 con 0x5576932a0cd0 2023-09-30T14:54:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbafcc74b80 10 client.0 _ll_register_callbacks cb 0x557692feb7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:54:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbae9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0310 0x5576932ed390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbae9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0310 0x5576932ed390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38954/0 (socket says 172.21.15.32:38954) 2023-09-30T14:54:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbae9d9b700 1 -- 172.21.15.32:0/808320173 learned_addr learned my addr 172.21.15.32:0/808320173 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbaead9d700 1 --2- 172.21.15.32:0/808320173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576932a0cd0 0x5576932f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbaea59c700 1 --2- 172.21.15.32:0/808320173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55769329f200 0x5576932ece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbae9d9b700 1 -- 172.21.15.32:0/808320173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576932a0cd0 msgr2=0x5576932f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbae9d9b700 1 --2- 172.21.15.32:0/808320173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576932a0cd0 0x5576932f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbae9d9b700 1 -- 172.21.15.32:0/808320173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55769329f200 msgr2=0x5576932ece50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbae9d9b700 1 --2- 172.21.15.32:0/808320173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55769329f200 0x5576932ece50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.623+0000 7fbae9d9b700 1 -- 172.21.15.32:0/808320173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae4030150 con 0x5576932a0310 2023-09-30T14:54:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.624+0000 7fbae9d9b700 1 --2- 172.21.15.32:0/808320173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0310 0x5576932ed390 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fbae0004580 tx=0x7fbae00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a11f447d93702a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.624+0000 7fbad1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:54:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.624+0000 7fbad1ffb700 1 -- 172.21.15.32:0/808320173 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae00045b0 con 0x5576932a0310 2023-09-30T14:54:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.624+0000 7fbad1ffb700 1 -- 172.21.15.32:0/808320173 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbae0028640 con 0x5576932a0310 2023-09-30T14:54:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.624+0000 7fbafcc74b80 1 -- 172.21.15.32:0/808320173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557693292090 con 0x5576932a0310 2023-09-30T14:54:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.624+0000 7fbad1ffb700 1 -- 172.21.15.32:0/808320173 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbae0028800 con 0x5576932a0310 2023-09-30T14:54:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.624+0000 7fbafcc74b80 10 client.16854 fetch_fsmap learned FSMap version 49 2023-09-30T14:54:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.624+0000 7fbafcc74b80 1 -- 172.21.15.32:0/808320173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55769301cd80 con 0x5576932a0310 2023-09-30T14:54:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.624+0000 7fbad1ffb700 1 -- 172.21.15.32:0/808320173 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557693292090 con 0x5576932a0310 2023-09-30T14:54:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.624+0000 7fbad1ffb700 1 -- 172.21.15.32:0/808320173 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbae0031020 con 0x5576932a0310 2023-09-30T14:54:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.625+0000 7fbafcc74b80 10 client.16854 fetch_fsmap finished waiting for FSMap version 49 2023-09-30T14:54:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.625+0000 7fbafcc74b80 10 client.16854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:54:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.625+0000 7fbafcc74b80 20 client.16854 populate_metadata read hostname 'smithi032' 2023-09-30T14:54:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.625+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/808320173 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5576932f8d60 0x5576932fb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.625+0000 7fbafcc74b80 4 client.16854 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:54:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.625+0000 7fbafcc74b80 1 -- 172.21.15.32:0/808320173 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576931eadb0 con 0x5576932f8d60 2023-09-30T14:54:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.625+0000 7fbaea59c700 1 --2- 172.21.15.32:0/808320173 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5576932f8d60 0x5576932fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.625+0000 7fbad1ffb700 10 client.16854 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:54:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.625+0000 7fbaea59c700 1 --2- 172.21.15.32:0/808320173 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5576932f8d60 0x5576932fb180 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee10a8d6573a57d6 server_cookie=4ea3051026b7fde2 in_seq=0 out_seq=0 2023-09-30T14:54:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.631+0000 7fbad1ffb700 1 -- 172.21.15.32:0/808320173 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5576931eadb0 con 0x5576932f8d60 2023-09-30T14:54:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.631+0000 7fbad1ffb700 10 client.16854 handle_command_reply: tid=0 2023-09-30T14:54:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.631+0000 7fbafcc74b80 1 client.16854 shutdown 2023-09-30T14:54:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.631+0000 7fbafcc74b80 1 -- 172.21.15.32:0/808320173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0310 msgr2=0x5576932ed390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.631+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/808320173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0310 0x5576932ed390 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fbae0004580 tx=0x7fbae00047c0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.632+0000 7fbafcc74b80 1 -- 172.21.15.32:0/808320173 shutdown_connections 2023-09-30T14:54:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.632+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/808320173 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5576932f8d60 0x5576932fb180 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.632+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/808320173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576932a0cd0 0x5576932f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.632+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/808320173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55769329f200 0x5576932ece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.632+0000 7fbad1ffb700 0 client.16854 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:54:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.632+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/808320173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0310 0x5576932ed390 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.632+0000 7fbafcc74b80 1 -- 172.21.15.32:0/808320173 >> 172.21.15.32:0/808320173 conn(0x557693213200 msgr2=0x557693299590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.632+0000 7fbafcc74b80 1 -- 172.21.15.32:0/808320173 shutdown_connections 2023-09-30T14:54:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.632+0000 7fbafcc74b80 1 -- 172.21.15.32:0/808320173 wait complete. 2023-09-30T14:54:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.632+0000 7fbafcc74b80 20 client.16854 trim_cache size 0 max 16384 2023-09-30T14:54:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.651+0000 7fbafcc74b80 1 Processor -- start 2023-09-30T14:54:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.651+0000 7fbafcc74b80 1 -- start start 2023-09-30T14:54:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.651+0000 7fbafcc74b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55769329f200 0x5576932a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.651+0000 7fbafcc74b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0310 0x5576932f4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.651+0000 7fbafcc74b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576932a0cd0 0x5576932ece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.651+0000 7fbafcc74b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576930728c0 con 0x5576932a0310 2023-09-30T14:54:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.651+0000 7fbafcc74b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576932ec9c0 con 0x55769329f200 2023-09-30T14:54:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.651+0000 7fbafcc74b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557693291bd0 con 0x5576932a0cd0 2023-09-30T14:54:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.651+0000 7fbae9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0310 0x5576932f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.651+0000 7fbae9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0310 0x5576932f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38962/0 (socket says 172.21.15.32:38962) 2023-09-30T14:54:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.651+0000 7fbae9d9b700 1 -- 172.21.15.32:0/2004763624 learned_addr learned my addr 172.21.15.32:0/2004763624 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.651+0000 7fbaea59c700 1 --2- 172.21.15.32:0/2004763624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55769329f200 0x5576932a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.651+0000 7fbaead9d700 1 --2- 172.21.15.32:0/2004763624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576932a0cd0 0x5576932ece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.652+0000 7fbae9d9b700 1 -- 172.21.15.32:0/2004763624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576932a0cd0 msgr2=0x5576932ece50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.652+0000 7fbae9d9b700 1 --2- 172.21.15.32:0/2004763624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576932a0cd0 0x5576932ece50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.652+0000 7fbae9d9b700 1 -- 172.21.15.32:0/2004763624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55769329f200 msgr2=0x5576932a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.652+0000 7fbae9d9b700 1 --2- 172.21.15.32:0/2004763624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55769329f200 0x5576932a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.652+0000 7fbae9d9b700 1 -- 172.21.15.32:0/2004763624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557693005c00 con 0x5576932a0310 2023-09-30T14:54:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.652+0000 7fbae9d9b700 1 --2- 172.21.15.32:0/2004763624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0310 0x5576932f4390 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fbae0013010 tx=0x7fbae0009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f13a4f92919ce90 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.652+0000 7fbad97fa700 1 -- 172.21.15.32:0/2004763624 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae001a3c0 con 0x5576932a0310 2023-09-30T14:54:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.652+0000 7fbad97fa700 1 -- 172.21.15.32:0/2004763624 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbae002e070 con 0x5576932a0310 2023-09-30T14:54:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.652+0000 7fbad97fa700 1 -- 172.21.15.32:0/2004763624 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0005040 con 0x5576932a0310 2023-09-30T14:54:52.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.653+0000 7fbafcc74b80 1 -- 172.21.15.32:0/2004763624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0310 msgr2=0x5576932f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.653+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/2004763624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0310 0x5576932f4390 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fbae0013010 tx=0x7fbae0009350 comp rx=0 tx=0).stop 2023-09-30T14:54:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.653+0000 7fbafcc74b80 1 -- 172.21.15.32:0/2004763624 shutdown_connections 2023-09-30T14:54:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.653+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/2004763624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576932a0cd0 0x5576932ece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.653+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/2004763624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55769329f200 0x5576932a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.653+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/2004763624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576932a0310 0x5576932f4390 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.653+0000 7fbafcc74b80 1 -- 172.21.15.32:0/2004763624 >> 172.21.15.32:0/2004763624 conn(0x557693213200 msgr2=0x5576932ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.653+0000 7fbafcc74b80 1 -- 172.21.15.32:0/2004763624 shutdown_connections 2023-09-30T14:54:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.653+0000 7fbafcc74b80 1 -- 172.21.15.32:0/2004763624 wait complete. 2023-09-30T14:54:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.653+0000 7fbafcc74b80 1 -- start start 2023-09-30T14:54:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.653+0000 7fbafcc74b80 1 Processor -- start 2023-09-30T14:54:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbafcc74b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55769329f200 0x5576932f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbafcc74b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576932a0310 0x5576932f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbafcc74b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576932a0cd0 0x5576932f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbafcc74b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576932f3b30 con 0x55769329f200 2023-09-30T14:54:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbafcc74b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576932f3ca0 con 0x5576932a0cd0 2023-09-30T14:54:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbafcc74b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576932f3e60 con 0x5576932a0310 2023-09-30T14:54:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbafcc74b80 10 client.0 _ll_register_callbacks cb 0x7fbae00043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:54:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbaea59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55769329f200 0x5576932f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbaea59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55769329f200 0x5576932f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38966/0 (socket says 172.21.15.32:38966) 2023-09-30T14:54:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbaea59c700 1 -- 172.21.15.32:0/3277471943 learned_addr learned my addr 172.21.15.32:0/3277471943 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:54:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbae9d9b700 1 --2- 172.21.15.32:0/3277471943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576932a0310 0x5576932f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbaead9d700 1 --2- 172.21.15.32:0/3277471943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576932a0cd0 0x5576932f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbaea59c700 1 -- 172.21.15.32:0/3277471943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576932a0310 msgr2=0x5576932f25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbaea59c700 1 --2- 172.21.15.32:0/3277471943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576932a0310 0x5576932f25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbaea59c700 1 -- 172.21.15.32:0/3277471943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576932a0cd0 msgr2=0x5576932f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:54:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbaea59c700 1 --2- 172.21.15.32:0/3277471943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576932a0cd0 0x5576932f35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbaea59c700 1 -- 172.21.15.32:0/3277471943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55769301cd80 con 0x55769329f200 2023-09-30T14:54:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbaea59c700 1 --2- 172.21.15.32:0/3277471943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55769329f200 0x5576932f20a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fbadc09a7d0 tx=0x7fbadc01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe8128b1a953ece server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:54:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbad1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:54:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbad1ffb700 1 -- 172.21.15.32:0/3277471943 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc09ec20 con 0x55769329f200 2023-09-30T14:54:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbad1ffb700 1 -- 172.21.15.32:0/3277471943 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbadc0a2dd0 con 0x55769329f200 2023-09-30T14:54:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbafcc74b80 1 -- 172.21.15.32:0/3277471943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbae4004bb0 con 0x55769329f200 2023-09-30T14:54:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbad1ffb700 1 -- 172.21.15.32:0/3277471943 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbadc0b8ac0 con 0x55769329f200 2023-09-30T14:54:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbafcc74b80 10 client.16860 fetch_fsmap learned FSMap version 49 2023-09-30T14:54:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.654+0000 7fbafcc74b80 1 -- 172.21.15.32:0/3277471943 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5576930431f0 con 0x55769329f200 2023-09-30T14:54:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.655+0000 7fbad1ffb700 1 -- 172.21.15.32:0/3277471943 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbae4004bb0 con 0x55769329f200 2023-09-30T14:54:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.655+0000 7fbad1ffb700 1 -- 172.21.15.32:0/3277471943 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbadc099fc0 con 0x55769329f200 2023-09-30T14:54:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.655+0000 7fbafcc74b80 10 client.16860 fetch_fsmap finished waiting for FSMap version 49 2023-09-30T14:54:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.655+0000 7fbafcc74b80 10 client.16860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:54:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.655+0000 7fbafcc74b80 20 client.16860 populate_metadata read hostname 'smithi032' 2023-09-30T14:54:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.655+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/3277471943 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55769334d6f0 0x55769334fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:54:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.655+0000 7fbafcc74b80 4 client.16860 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:54:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.655+0000 7fbafcc74b80 1 -- 172.21.15.32:0/3277471943 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5576931eaad0 con 0x55769334d6f0 2023-09-30T14:54:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.655+0000 7fbae9d9b700 1 --2- 172.21.15.32:0/3277471943 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55769334d6f0 0x55769334fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:54:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.655+0000 7fbae9d9b700 1 --2- 172.21.15.32:0/3277471943 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55769334d6f0 0x55769334fb10 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20098c9625b48ce server_cookie=8ae07f4d060b8d41 in_seq=0 out_seq=0 2023-09-30T14:54:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:52.655+0000 7fbad1ffb700 10 client.16860 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:54:53.010 INFO:tasks.workunit.client.0.smithi032.stdout: 10 138775 37.86 MB/sec execute 86 sec latency 154.331 ms 2023-09-30T14:54:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.094+0000 7fbad1ffb700 1 -- 172.21.15.32:0/3277471943 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5576931eaad0 con 0x55769334d6f0 2023-09-30T14:54:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.094+0000 7fbad1ffb700 10 client.16860 handle_command_reply: tid=0 2023-09-30T14:54:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.094+0000 7fbafcc74b80 1 client.16860 shutdown 2023-09-30T14:54:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.096+0000 7fbafcc74b80 1 -- 172.21.15.32:0/3277471943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55769329f200 msgr2=0x5576932f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.096+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/3277471943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55769329f200 0x5576932f20a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fbadc09a7d0 tx=0x7fbadc01a040 comp rx=0 tx=0).stop 2023-09-30T14:54:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.096+0000 7fbafcc74b80 1 -- 172.21.15.32:0/3277471943 shutdown_connections 2023-09-30T14:54:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.096+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/3277471943 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55769334d6f0 0x55769334fb10 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.096+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/3277471943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576932a0cd0 0x5576932f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.096+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/3277471943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55769329f200 0x5576932f20a0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.096+0000 7fbafcc74b80 1 --2- 172.21.15.32:0/3277471943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576932a0310 0x5576932f25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.096+0000 7fbafcc74b80 1 -- 172.21.15.32:0/3277471943 >> 172.21.15.32:0/3277471943 conn(0x557693213200 msgr2=0x5576932a7050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.096+0000 7fbad1ffb700 0 client.16860 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:54:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.096+0000 7fbafcc74b80 1 -- 172.21.15.32:0/3277471943 shutdown_connections 2023-09-30T14:54:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.096+0000 7fbafcc74b80 1 -- 172.21.15.32:0/3277471943 wait complete. 2023-09-30T14:54:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.096+0000 7fbafcc74b80 20 client.16860 trim_cache size 0 max 16384 2023-09-30T14:54:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:54:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:54:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:54:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:54:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.097+0000 7fbaf0a8c700 1 -- 172.21.15.32:0/3433124431 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbad40a1020 msgr2=0x7fbad40a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.097+0000 7fbaf0a8c700 1 --2- 172.21.15.32:0/3433124431 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbad40a1020 0x7fbad40a34e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009c40 tx=0x7fbae0009260 comp rx=0 tx=0).stop 2023-09-30T14:54:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.097+0000 7fbaf0a8c700 1 -- 172.21.15.32:0/3433124431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbaec0ffd70 msgr2=0x7fbaec195bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:54:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.097+0000 7fbaf0a8c700 1 --2- 172.21.15.32:0/3433124431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbaec0ffd70 0x7fbaec195bd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009990 tx=0x7fbadc00f690 comp rx=0 tx=0).stop 2023-09-30T14:54:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.097+0000 7fbaf0a8c700 1 -- 172.21.15.32:0/3433124431 shutdown_connections 2023-09-30T14:54:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.097+0000 7fbaf0a8c700 1 --2- 172.21.15.32:0/3433124431 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbad40a1020 0x7fbad40a34e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.097+0000 7fbaf0a8c700 1 --2- 172.21.15.32:0/3433124431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaec101900 0x7fbaec196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.097+0000 7fbaf0a8c700 1 --2- 172.21.15.32:0/3433124431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbaec0ffd70 0x7fbaec195bd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.097+0000 7fbaf0a8c700 1 --2- 172.21.15.32:0/3433124431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbaec100f60 0x7fbaec196110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:54:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.097+0000 7fbaf0a8c700 1 -- 172.21.15.32:0/3433124431 >> 172.21.15.32:0/3433124431 conn(0x7fbaec0fb300 msgr2=0x7fbaec1044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:54:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.098+0000 7fbaf0a8c700 1 -- 172.21.15.32:0/3433124431 shutdown_connections 2023-09-30T14:54:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:54:53.098+0000 7fbaf0a8c700 1 -- 172.21.15.32:0/3433124431 wait complete. 2023-09-30T14:54:53.808 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:53 smithi156 ceph-mon[103579]: pgmap v785: 97 pgs: 97 active+clean; 722 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.7 MiB/s wr, 782 op/s 2023-09-30T14:54:53.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:53 smithi148 ceph-mon[102355]: pgmap v785: 97 pgs: 97 active+clean; 722 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.7 MiB/s wr, 782 op/s 2023-09-30T14:54:53.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:53 smithi032 ceph-mon[98592]: pgmap v785: 97 pgs: 97 active+clean; 722 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.7 MiB/s wr, 782 op/s 2023-09-30T14:54:54.010 INFO:tasks.workunit.client.0.smithi032.stdout: 10 139304 37.70 MB/sec execute 87 sec latency 271.970 ms 2023-09-30T14:54:55.010 INFO:tasks.workunit.client.0.smithi032.stdout: 10 140036 37.79 MB/sec execute 88 sec latency 96.232 ms 2023-09-30T14:54:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:55 smithi156 ceph-mon[103579]: pgmap v786: 97 pgs: 97 active+clean; 723 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.5 MiB/s wr, 732 op/s 2023-09-30T14:54:55.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:55 smithi148 ceph-mon[102355]: pgmap v786: 97 pgs: 97 active+clean; 723 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.5 MiB/s wr, 732 op/s 2023-09-30T14:54:55.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:55 smithi032 ceph-mon[98592]: pgmap v786: 97 pgs: 97 active+clean; 723 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.5 MiB/s wr, 732 op/s 2023-09-30T14:54:56.010 INFO:tasks.workunit.client.0.smithi032.stdout: 10 140940 37.80 MB/sec execute 89 sec latency 71.173 ms 2023-09-30T14:54:57.010 INFO:tasks.workunit.client.0.smithi032.stdout: 10 141802 38.03 MB/sec execute 90 sec latency 104.524 ms 2023-09-30T14:54:57.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:57 smithi156 ceph-mon[103579]: pgmap v787: 97 pgs: 97 active+clean; 722 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.3 MiB/s wr, 742 op/s 2023-09-30T14:54:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:57 smithi148 ceph-mon[102355]: pgmap v787: 97 pgs: 97 active+clean; 722 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.3 MiB/s wr, 742 op/s 2023-09-30T14:54:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:57 smithi032 ceph-mon[98592]: pgmap v787: 97 pgs: 97 active+clean; 722 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.3 MiB/s wr, 742 op/s 2023-09-30T14:54:58.011 INFO:tasks.workunit.client.0.smithi032.stdout: 10 142561 38.06 MB/sec execute 91 sec latency 108.506 ms 2023-09-30T14:54:59.011 INFO:tasks.workunit.client.0.smithi032.stdout: 10 143124 38.04 MB/sec execute 92 sec latency 223.995 ms 2023-09-30T14:54:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:54:59 smithi156 ceph-mon[103579]: pgmap v788: 97 pgs: 97 active+clean; 709 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.9 MiB/s wr, 765 op/s 2023-09-30T14:54:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:54:59 smithi148 ceph-mon[102355]: pgmap v788: 97 pgs: 97 active+clean; 709 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.9 MiB/s wr, 765 op/s 2023-09-30T14:54:59.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:54:59 smithi032 ceph-mon[98592]: pgmap v788: 97 pgs: 97 active+clean; 709 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.9 MiB/s wr, 765 op/s 2023-09-30T14:55:00.011 INFO:tasks.workunit.client.0.smithi032.stdout: 10 143588 37.79 MB/sec execute 93 sec latency 324.353 ms 2023-09-30T14:55:01.011 INFO:tasks.workunit.client.0.smithi032.stdout: 10 144206 37.73 MB/sec execute 94 sec latency 89.134 ms 2023-09-30T14:55:01.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:01 smithi156 ceph-mon[103579]: pgmap v789: 97 pgs: 97 active+clean; 701 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.3 MiB/s wr, 706 op/s 2023-09-30T14:55:01.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:01 smithi148 ceph-mon[102355]: pgmap v789: 97 pgs: 97 active+clean; 701 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.3 MiB/s wr, 706 op/s 2023-09-30T14:55:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:01 smithi032 ceph-mon[98592]: pgmap v789: 97 pgs: 97 active+clean; 701 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.3 MiB/s wr, 706 op/s 2023-09-30T14:55:02.011 INFO:tasks.workunit.client.0.smithi032.stdout: 10 145145 37.76 MB/sec execute 95 sec latency 59.640 ms 2023-09-30T14:55:03.011 INFO:tasks.workunit.client.0.smithi032.stdout: 10 146181 38.07 MB/sec execute 96 sec latency 40.746 ms 2023-09-30T14:55:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:03 smithi156 ceph-mon[103579]: pgmap v790: 97 pgs: 97 active+clean; 700 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.5 MiB/s wr, 757 op/s 2023-09-30T14:55:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:03 smithi148 ceph-mon[102355]: pgmap v790: 97 pgs: 97 active+clean; 700 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.5 MiB/s wr, 757 op/s 2023-09-30T14:55:03.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:03 smithi032 ceph-mon[98592]: pgmap v790: 97 pgs: 97 active+clean; 700 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.5 MiB/s wr, 757 op/s 2023-09-30T14:55:04.015 INFO:tasks.workunit.client.0.smithi032.stdout: 10 146875 38.18 MB/sec execute 97 sec latency 105.956 ms 2023-09-30T14:55:05.012 INFO:tasks.workunit.client.0.smithi032.stdout: 10 147282 37.97 MB/sec execute 98 sec latency 240.142 ms 2023-09-30T14:55:05.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:05 smithi156 ceph-mon[103579]: pgmap v791: 97 pgs: 97 active+clean; 709 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 817 op/s 2023-09-30T14:55:05.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:05 smithi148 ceph-mon[102355]: pgmap v791: 97 pgs: 97 active+clean; 709 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 817 op/s 2023-09-30T14:55:05.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:05 smithi032 ceph-mon[98592]: pgmap v791: 97 pgs: 97 active+clean; 709 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 817 op/s 2023-09-30T14:55:06.011 INFO:tasks.workunit.client.0.smithi032.stdout: 10 147873 37.84 MB/sec execute 99 sec latency 68.636 ms 2023-09-30T14:55:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.227+0000 7f18ce184700 1 -- 172.21.15.32:0/2727518116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c8102b50 msgr2=0x7f18c8109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.227+0000 7f18ce184700 1 --2- 172.21.15.32:0/2727518116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c8102b50 0x7f18c8109400 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f18c000b0a0 tx=0x7f18c000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.228+0000 7f18ce184700 1 -- 172.21.15.32:0/2727518116 shutdown_connections 2023-09-30T14:55:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.228+0000 7f18ce184700 1 --2- 172.21.15.32:0/2727518116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c8102b50 0x7f18c8109400 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.228+0000 7f18ce184700 1 --2- 172.21.15.32:0/2727518116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c80ff140 0x7f18c80ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.228+0000 7f18ce184700 1 --2- 172.21.15.32:0/2727518116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18c81021b0 0x7f18c8102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.228+0000 7f18ce184700 1 -- 172.21.15.32:0/2727518116 >> 172.21.15.32:0/2727518116 conn(0x7f18c80faa30 msgr2=0x7f18c80fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.228+0000 7f18ce184700 1 -- 172.21.15.32:0/2727518116 shutdown_connections 2023-09-30T14:55:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.228+0000 7f18ce184700 1 -- 172.21.15.32:0/2727518116 wait complete. 2023-09-30T14:55:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.228+0000 7f18ce184700 1 Processor -- start 2023-09-30T14:55:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.228+0000 7f18ce184700 1 -- start start 2023-09-30T14:55:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18ce184700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c80ff140 0x7f18c8100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18ce184700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18c81021b0 0x7f18c81006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18ce184700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c8102b50 0x7f18c8100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18ce184700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c81a0cc0 con 0x7f18c8102b50 2023-09-30T14:55:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18ce184700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c81a0e00 con 0x7f18c80ff140 2023-09-30T14:55:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18ce184700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c81a0f40 con 0x7f18c81021b0 2023-09-30T14:55:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18c7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c8102b50 0x7f18c8100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18c6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18c81021b0 0x7f18c81006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18c7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c8102b50 0x7f18c8100be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38984/0 (socket says 172.21.15.32:38984) 2023-09-30T14:55:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18c7fff700 1 -- 172.21.15.32:0/690972987 learned_addr learned my addr 172.21.15.32:0/690972987 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18c77fe700 1 --2- 172.21.15.32:0/690972987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c80ff140 0x7f18c8100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18c7fff700 1 -- 172.21.15.32:0/690972987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18c81021b0 msgr2=0x7f18c81006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18c7fff700 1 --2- 172.21.15.32:0/690972987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18c81021b0 0x7f18c81006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18c7fff700 1 -- 172.21.15.32:0/690972987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c80ff140 msgr2=0x7f18c8100160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18c7fff700 1 --2- 172.21.15.32:0/690972987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c80ff140 0x7f18c8100160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18c7fff700 1 -- 172.21.15.32:0/690972987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c0009d00 con 0x7f18c8102b50 2023-09-30T14:55:06.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18c7fff700 1 --2- 172.21.15.32:0/690972987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c8102b50 0x7f18c8100be0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009fd0 tx=0x7f18c000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:06.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.229+0000 7f18c4ff9700 1 -- 172.21.15.32:0/690972987 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c000e040 con 0x7f18c8102b50 2023-09-30T14:55:06.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.230+0000 7f18c4ff9700 1 -- 172.21.15.32:0/690972987 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18c00094a0 con 0x7f18c8102b50 2023-09-30T14:55:06.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.230+0000 7f18ce184700 1 -- 172.21.15.32:0/690972987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c8196880 con 0x7f18c8102b50 2023-09-30T14:55:06.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.230+0000 7f18c4ff9700 1 -- 172.21.15.32:0/690972987 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0005840 con 0x7f18c8102b50 2023-09-30T14:55:06.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.230+0000 7f18ce184700 1 -- 172.21.15.32:0/690972987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c8196d90 con 0x7f18c8102b50 2023-09-30T14:55:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.232+0000 7f18c4ff9700 1 -- 172.21.15.32:0/690972987 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f18c0005040 con 0x7f18c8102b50 2023-09-30T14:55:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.232+0000 7f18c4ff9700 1 --2- 172.21.15.32:0/690972987 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f18b00a0fa0 0x7f18b00a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.233+0000 7f18c4ff9700 1 -- 172.21.15.32:0/690972987 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f18c0026030 con 0x7f18c8102b50 2023-09-30T14:55:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.233+0000 7f18c77fe700 1 --2- 172.21.15.32:0/690972987 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f18b00a0fa0 0x7f18b00a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.233+0000 7f18c77fe700 1 --2- 172.21.15.32:0/690972987 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f18b00a0fa0 0x7f18b00a3460 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f18c8101a70 tx=0x7f18b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.234+0000 7f18da36cb80 1 Processor -- start 2023-09-30T14:55:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.234+0000 7f18da36cb80 1 -- start start 2023-09-30T14:55:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.234+0000 7f18da36cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff2200 0x562199ff2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18da36cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff3310 0x562199ff3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18da36cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3cd0 0x562199ffa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18da36cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562199f5a400 con 0x562199ff3310 2023-09-30T14:55:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18da36cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562199f4bbc0 con 0x562199ff3cd0 2023-09-30T14:55:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18da36cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562199f494e0 con 0x562199ff2200 2023-09-30T14:55:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18c77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff2200 0x562199ff2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff3310 0x562199ff3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18c77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff2200 0x562199ff2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53370/0 (socket says 172.21.15.32:53370) 2023-09-30T14:55:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18c77fe700 1 -- 172.21.15.32:0/3723648384 learned_addr learned my addr 172.21.15.32:0/3723648384 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18c7fff700 1 --2- 172.21.15.32:0/3723648384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3cd0 0x562199ffa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18c6ffd700 1 -- 172.21.15.32:0/3723648384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff2200 msgr2=0x562199ff2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18c6ffd700 1 --2- 172.21.15.32:0/3723648384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff2200 0x562199ff2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18c6ffd700 1 -- 172.21.15.32:0/3723648384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3cd0 msgr2=0x562199ffa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18c6ffd700 1 --2- 172.21.15.32:0/3723648384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3cd0 0x562199ffa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18c6ffd700 1 -- 172.21.15.32:0/3723648384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562199fa7870 con 0x562199ff3310 2023-09-30T14:55:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18c7fff700 1 --2- 172.21.15.32:0/3723648384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3cd0 0x562199ffa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:55:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18c77fe700 1 --2- 172.21.15.32:0/3723648384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff2200 0x562199ff2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:55:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18c6ffd700 1 --2- 172.21.15.32:0/3723648384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff3310 0x562199ff3790 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009c80 tx=0x7f18bc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f25e65a65d75473 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.235+0000 7f18adffb700 1 -- 172.21.15.32:0/3723648384 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc011c50 con 0x562199ff3310 2023-09-30T14:55:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.236+0000 7f18adffb700 1 -- 172.21.15.32:0/3723648384 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18bc004510 con 0x562199ff3310 2023-09-30T14:55:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.236+0000 7f18adffb700 1 -- 172.21.15.32:0/3723648384 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc005280 con 0x562199ff3310 2023-09-30T14:55:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.236+0000 7f18da36cb80 1 -- 172.21.15.32:0/3723648384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff3310 msgr2=0x562199ff3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.236+0000 7f18da36cb80 1 --2- 172.21.15.32:0/3723648384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff3310 0x562199ff3790 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009c80 tx=0x7f18bc009f90 comp rx=0 tx=0).stop 2023-09-30T14:55:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.236+0000 7f18da36cb80 1 -- 172.21.15.32:0/3723648384 shutdown_connections 2023-09-30T14:55:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.236+0000 7f18da36cb80 1 --2- 172.21.15.32:0/3723648384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3cd0 0x562199ffa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.236+0000 7f18da36cb80 1 --2- 172.21.15.32:0/3723648384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff2200 0x562199ff2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.236+0000 7f18da36cb80 1 --2- 172.21.15.32:0/3723648384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff3310 0x562199ff3790 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.236+0000 7f18da36cb80 1 -- 172.21.15.32:0/3723648384 >> 172.21.15.32:0/3723648384 conn(0x562199f66200 msgr2=0x562199fecef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.236+0000 7f18da36cb80 1 -- 172.21.15.32:0/3723648384 shutdown_connections 2023-09-30T14:55:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.237+0000 7f18da36cb80 1 -- 172.21.15.32:0/3723648384 wait complete. 2023-09-30T14:55:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.237+0000 7f18da36cb80 1 -- start start 2023-09-30T14:55:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.237+0000 7f18da36cb80 1 Processor -- start 2023-09-30T14:55:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.237+0000 7f18da36cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff2200 0x562199ffbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.237+0000 7f18da36cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 0x562199ffc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.237+0000 7f18da36cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff3cd0 0x56219a0033f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.237+0000 7f18da36cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562199f44dd0 con 0x562199ff2200 2023-09-30T14:55:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.237+0000 7f18da36cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562199ee16b0 con 0x562199ff3310 2023-09-30T14:55:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.237+0000 7f18da36cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562199ee1350 con 0x562199ff3cd0 2023-09-30T14:55:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.238+0000 7f18da36cb80 10 client.0 _ll_register_callbacks cb 0x562199d3e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:55:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.238+0000 7f18c6ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 0x562199ffc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.238+0000 7f18c6ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 0x562199ffc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:50470/0 (socket says 172.21.15.32:50470) 2023-09-30T14:55:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.238+0000 7f18c6ffd700 1 -- 172.21.15.32:0/2957689693 learned_addr learned my addr 172.21.15.32:0/2957689693 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.238+0000 7f18c77fe700 1 --2- 172.21.15.32:0/2957689693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff2200 0x562199ffbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.238+0000 7f18c7fff700 1 --2- 172.21.15.32:0/2957689693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff3cd0 0x56219a0033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.238+0000 7f18c6ffd700 1 -- 172.21.15.32:0/2957689693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff3cd0 msgr2=0x56219a0033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.238+0000 7f18c6ffd700 1 --2- 172.21.15.32:0/2957689693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff3cd0 0x56219a0033f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.238+0000 7f18c6ffd700 1 -- 172.21.15.32:0/2957689693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff2200 msgr2=0x562199ffbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.238+0000 7f18c6ffd700 1 --2- 172.21.15.32:0/2957689693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff2200 0x562199ffbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.238+0000 7f18c6ffd700 1 -- 172.21.15.32:0/2957689693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c8196d90 con 0x562199ff3310 2023-09-30T14:55:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.238+0000 7f18c6ffd700 1 --2- 172.21.15.32:0/2957689693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 0x562199ffc3b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f18bc005d20 tx=0x7f18bc005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cd4d59d5ed70fe8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.238+0000 7f18a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:55:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.239+0000 7f18a6ffd700 1 -- 172.21.15.32:0/2957689693 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc011c60 con 0x562199ff3310 2023-09-30T14:55:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.239+0000 7f18a6ffd700 1 -- 172.21.15.32:0/2957689693 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f18bc011dc0 con 0x562199ff3310 2023-09-30T14:55:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.239+0000 7f18da36cb80 1 -- 172.21.15.32:0/2957689693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562199fe5090 con 0x562199ff3310 2023-09-30T14:55:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.239+0000 7f18a6ffd700 1 -- 172.21.15.32:0/2957689693 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18bc003680 con 0x562199ff3310 2023-09-30T14:55:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.239+0000 7f18da36cb80 10 client.26516 fetch_fsmap learned FSMap version 49 2023-09-30T14:55:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.239+0000 7f18da36cb80 1 -- 172.21.15.32:0/2957689693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f18bc0098e0 con 0x562199ff3310 2023-09-30T14:55:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.239+0000 7f18a6ffd700 1 -- 172.21.15.32:0/2957689693 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562199fe5090 con 0x562199ff3310 2023-09-30T14:55:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.239+0000 7f18a6ffd700 1 -- 172.21.15.32:0/2957689693 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f18bc0053d0 con 0x562199ff3310 2023-09-30T14:55:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.239+0000 7f18da36cb80 10 client.26516 fetch_fsmap finished waiting for FSMap version 49 2023-09-30T14:55:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.239+0000 7f18da36cb80 10 client.26516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:55:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.239+0000 7f18da36cb80 20 client.26516 populate_metadata read hostname 'smithi032' 2023-09-30T14:55:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.239+0000 7f18da36cb80 1 --2- 172.21.15.32:0/2957689693 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56219a007b60 0x56219a009f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.239+0000 7f18da36cb80 4 client.26516 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:55:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.239+0000 7f18da36cb80 1 -- 172.21.15.32:0/2957689693 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562199f3ddb0 con 0x56219a007b60 2023-09-30T14:55:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.240+0000 7f18c77fe700 1 --2- 172.21.15.32:0/2957689693 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56219a007b60 0x56219a009f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.240+0000 7f18c77fe700 1 --2- 172.21.15.32:0/2957689693 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56219a007b60 0x56219a009f80 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=135491a8dcfbeb55 server_cookie=a9512933798dfe41 in_seq=0 out_seq=0 2023-09-30T14:55:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.240+0000 7f18a6ffd700 10 client.26516 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:55:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.241+0000 7f18a6ffd700 1 -- 172.21.15.32:0/2957689693 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562199f3ddb0 con 0x56219a007b60 2023-09-30T14:55:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.241+0000 7f18a6ffd700 10 client.26516 handle_command_reply: tid=0 2023-09-30T14:55:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.241+0000 7f18da36cb80 1 client.26516 shutdown 2023-09-30T14:55:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.241+0000 7f18da36cb80 1 -- 172.21.15.32:0/2957689693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 msgr2=0x562199ffc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.241+0000 7f18da36cb80 1 --2- 172.21.15.32:0/2957689693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 0x562199ffc3b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f18bc005d20 tx=0x7f18bc005d50 comp rx=0 tx=0).stop 2023-09-30T14:55:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.241+0000 7f18da36cb80 1 -- 172.21.15.32:0/2957689693 shutdown_connections 2023-09-30T14:55:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.241+0000 7f18da36cb80 1 --2- 172.21.15.32:0/2957689693 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56219a007b60 0x56219a009f80 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.241+0000 7f18da36cb80 1 --2- 172.21.15.32:0/2957689693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff3cd0 0x56219a0033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.241+0000 7f18da36cb80 1 --2- 172.21.15.32:0/2957689693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff2200 0x562199ffbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.241+0000 7f18da36cb80 1 --2- 172.21.15.32:0/2957689693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 0x562199ffc3b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.241+0000 7f18a6ffd700 0 client.26516 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:55:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.241+0000 7f18da36cb80 1 -- 172.21.15.32:0/2957689693 >> 172.21.15.32:0/2957689693 conn(0x562199f66200 msgr2=0x562199fec590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.242+0000 7f18da36cb80 1 -- 172.21.15.32:0/2957689693 shutdown_connections 2023-09-30T14:55:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.242+0000 7f18da36cb80 1 -- 172.21.15.32:0/2957689693 wait complete. 2023-09-30T14:55:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.242+0000 7f18da36cb80 20 client.26516 trim_cache size 0 max 16384 2023-09-30T14:55:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.260+0000 7f18da36cb80 1 Processor -- start 2023-09-30T14:55:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.260+0000 7f18da36cb80 1 -- start start 2023-09-30T14:55:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.260+0000 7f18da36cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff2200 0x562199ff6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.260+0000 7f18da36cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 0x562199ff70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.260+0000 7f18da36cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff3cd0 0x56219a0033b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.260+0000 7f18da36cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562199dc58c0 con 0x562199ff2200 2023-09-30T14:55:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.260+0000 7f18da36cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562199fe9ba0 con 0x562199ff3310 2023-09-30T14:55:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.260+0000 7f18da36cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562199fee9c0 con 0x562199ff3cd0 2023-09-30T14:55:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.260+0000 7f18c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff2200 0x562199ff6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.260+0000 7f18c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff2200 0x562199ff6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39002/0 (socket says 172.21.15.32:39002) 2023-09-30T14:55:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.261+0000 7f18c77fe700 1 -- 172.21.15.32:0/306997096 learned_addr learned my addr 172.21.15.32:0/306997096 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.261+0000 7f18c6ffd700 1 --2- 172.21.15.32:0/306997096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 0x562199ff70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.261+0000 7f18c7fff700 1 --2- 172.21.15.32:0/306997096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff3cd0 0x56219a0033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.261+0000 7f18c77fe700 1 -- 172.21.15.32:0/306997096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff3cd0 msgr2=0x56219a0033b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.261+0000 7f18c77fe700 1 --2- 172.21.15.32:0/306997096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff3cd0 0x56219a0033b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.261+0000 7f18c77fe700 1 -- 172.21.15.32:0/306997096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 msgr2=0x562199ff70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.261+0000 7f18c77fe700 1 --2- 172.21.15.32:0/306997096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 0x562199ff70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.261+0000 7f18c77fe700 1 -- 172.21.15.32:0/306997096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562199d6fd80 con 0x562199ff2200 2023-09-30T14:55:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.261+0000 7f18c6ffd700 1 --2- 172.21.15.32:0/306997096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 0x562199ff70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:55:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.261+0000 7f18c77fe700 1 --2- 172.21.15.32:0/306997096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff2200 0x562199ff6b90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x562199ff1fa0 tx=0x7f18b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c6645868b30170 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.261+0000 7f18ae7fc700 1 -- 172.21.15.32:0/306997096 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8022070 con 0x562199ff2200 2023-09-30T14:55:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.261+0000 7f18ae7fc700 1 -- 172.21.15.32:0/306997096 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18b8023040 con 0x562199ff2200 2023-09-30T14:55:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.261+0000 7f18ae7fc700 1 -- 172.21.15.32:0/306997096 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8003e30 con 0x562199ff2200 2023-09-30T14:55:06.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.262+0000 7f18da36cb80 1 -- 172.21.15.32:0/306997096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff2200 msgr2=0x562199ff6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:06.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.262+0000 7f18da36cb80 1 --2- 172.21.15.32:0/306997096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff2200 0x562199ff6b90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x562199ff1fa0 tx=0x7f18b8014000 comp rx=0 tx=0).stop 2023-09-30T14:55:06.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.262+0000 7f18da36cb80 1 -- 172.21.15.32:0/306997096 shutdown_connections 2023-09-30T14:55:06.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.262+0000 7f18da36cb80 1 --2- 172.21.15.32:0/306997096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff3cd0 0x56219a0033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.262+0000 7f18da36cb80 1 --2- 172.21.15.32:0/306997096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff2200 0x562199ff6b90 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.262+0000 7f18da36cb80 1 --2- 172.21.15.32:0/306997096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 0x562199ff70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.262+0000 7f18da36cb80 1 -- 172.21.15.32:0/306997096 >> 172.21.15.32:0/306997096 conn(0x562199f66200 msgr2=0x56219a00e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:06.266 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.262+0000 7f18da36cb80 1 -- 172.21.15.32:0/306997096 shutdown_connections 2023-09-30T14:55:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.262+0000 7f18da36cb80 1 -- 172.21.15.32:0/306997096 wait complete. 2023-09-30T14:55:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.263+0000 7f18da36cb80 1 -- start start 2023-09-30T14:55:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.263+0000 7f18da36cb80 1 Processor -- start 2023-09-30T14:55:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.263+0000 7f18da36cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff2200 0x56219a001400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.263+0000 7f18da36cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 0x56219a001940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.263+0000 7f18da36cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff3cd0 0x56219a002950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.263+0000 7f18da36cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56219a002e90 con 0x562199ff3cd0 2023-09-30T14:55:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.263+0000 7f18da36cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56219a003000 con 0x562199ff3310 2023-09-30T14:55:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.263+0000 7f18da36cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56219a0031c0 con 0x562199ff2200 2023-09-30T14:55:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.263+0000 7f18da36cb80 10 client.0 _ll_register_callbacks cb 0x7f18c000bed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:55:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.264+0000 7f18c77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff2200 0x56219a001400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.264+0000 7f18c77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff2200 0x56219a001400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53390/0 (socket says 172.21.15.32:53390) 2023-09-30T14:55:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.264+0000 7f18c77fe700 1 -- 172.21.15.32:0/2728887320 learned_addr learned my addr 172.21.15.32:0/2728887320 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.264+0000 7f18c7fff700 1 --2- 172.21.15.32:0/2728887320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff3cd0 0x56219a002950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.264+0000 7f18c77fe700 1 -- 172.21.15.32:0/2728887320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 msgr2=0x56219a001940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.264+0000 7f18c77fe700 1 --2- 172.21.15.32:0/2728887320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 0x56219a001940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.264+0000 7f18c77fe700 1 -- 172.21.15.32:0/2728887320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff3cd0 msgr2=0x56219a002950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.264+0000 7f18c77fe700 1 --2- 172.21.15.32:0/2728887320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff3cd0 0x56219a002950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.264+0000 7f18c77fe700 1 -- 172.21.15.32:0/2728887320 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18bc0098e0 con 0x562199ff2200 2023-09-30T14:55:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.264+0000 7f18c77fe700 1 --2- 172.21.15.32:0/2728887320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff2200 0x56219a001400 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f18b80097c0 tx=0x7f18b802bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c371ad39aa83d816 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.264+0000 7f18a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:55:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.264+0000 7f18a6ffd700 1 -- 172.21.15.32:0/2728887320 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8022040 con 0x562199ff2200 2023-09-30T14:55:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.265+0000 7f18da36cb80 1 -- 172.21.15.32:0/2728887320 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18bc004510 con 0x562199ff2200 2023-09-30T14:55:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.265+0000 7f18a6ffd700 1 -- 172.21.15.32:0/2728887320 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f18b802b040 con 0x562199ff2200 2023-09-30T14:55:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.265+0000 7f18a6ffd700 1 -- 172.21.15.32:0/2728887320 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18b802b200 con 0x562199ff2200 2023-09-30T14:55:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.265+0000 7f18da36cb80 10 client.26536 fetch_fsmap learned FSMap version 49 2023-09-30T14:55:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.265+0000 7f18da36cb80 1 -- 172.21.15.32:0/2728887320 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f18b8011b70 con 0x562199ff2200 2023-09-30T14:55:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.265+0000 7f18a6ffd700 1 -- 172.21.15.32:0/2728887320 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18bc004510 con 0x562199ff2200 2023-09-30T14:55:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.265+0000 7f18a6ffd700 1 -- 172.21.15.32:0/2728887320 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f18b8035be0 con 0x562199ff2200 2023-09-30T14:55:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.265+0000 7f18da36cb80 10 client.26536 fetch_fsmap finished waiting for FSMap version 49 2023-09-30T14:55:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.265+0000 7f18da36cb80 10 client.26536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:55:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.265+0000 7f18da36cb80 20 client.26536 populate_metadata read hostname 'smithi032' 2023-09-30T14:55:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.265+0000 7f18da36cb80 1 --2- 172.21.15.32:0/2728887320 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56219a05cb30 0x56219a05ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.265+0000 7f18da36cb80 4 client.26536 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:55:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.265+0000 7f18da36cb80 1 -- 172.21.15.32:0/2728887320 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562199f3dad0 con 0x56219a05cb30 2023-09-30T14:55:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.266+0000 7f18c6ffd700 1 --2- 172.21.15.32:0/2728887320 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56219a05cb30 0x56219a05ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.266+0000 7f18a6ffd700 10 client.26536 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:55:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.266+0000 7f18c6ffd700 1 --2- 172.21.15.32:0/2728887320 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56219a05cb30 0x56219a05ef50 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dc587b8af8d46ab server_cookie=14198fa263e402be in_seq=0 out_seq=0 2023-09-30T14:55:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.604+0000 7f18a6ffd700 1 -- 172.21.15.32:0/2728887320 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562199f3dad0 con 0x56219a05cb30 2023-09-30T14:55:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.604+0000 7f18a6ffd700 10 client.26536 handle_command_reply: tid=0 2023-09-30T14:55:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.604+0000 7f18da36cb80 1 client.26536 shutdown 2023-09-30T14:55:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.604+0000 7f18da36cb80 1 -- 172.21.15.32:0/2728887320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff2200 msgr2=0x56219a001400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.604+0000 7f18da36cb80 1 --2- 172.21.15.32:0/2728887320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff2200 0x56219a001400 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f18b80097c0 tx=0x7f18b802bf00 comp rx=0 tx=0).stop 2023-09-30T14:55:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.604+0000 7f18da36cb80 1 -- 172.21.15.32:0/2728887320 shutdown_connections 2023-09-30T14:55:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.604+0000 7f18da36cb80 1 --2- 172.21.15.32:0/2728887320 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56219a05cb30 0x56219a05ef50 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.604+0000 7f18da36cb80 1 --2- 172.21.15.32:0/2728887320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562199ff3cd0 0x56219a002950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.604+0000 7f18da36cb80 1 --2- 172.21.15.32:0/2728887320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562199ff2200 0x56219a001400 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.604+0000 7f18da36cb80 1 --2- 172.21.15.32:0/2728887320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562199ff3310 0x56219a001940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.604+0000 7f18da36cb80 1 -- 172.21.15.32:0/2728887320 >> 172.21.15.32:0/2728887320 conn(0x562199f66200 msgr2=0x562199ff6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.604+0000 7f18a6ffd700 0 client.26536 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:55:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.605+0000 7f18da36cb80 1 -- 172.21.15.32:0/2728887320 shutdown_connections 2023-09-30T14:55:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.605+0000 7f18da36cb80 1 -- 172.21.15.32:0/2728887320 wait complete. 2023-09-30T14:55:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.605+0000 7f18da36cb80 20 client.26536 trim_cache size 0 max 16384 2023-09-30T14:55:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:55:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:55:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:55:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:55:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.605+0000 7f18ce184700 1 -- 172.21.15.32:0/690972987 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f18b00a0fa0 msgr2=0x7f18b00a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.605+0000 7f18ce184700 1 --2- 172.21.15.32:0/690972987 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f18b00a0fa0 0x7f18b00a3460 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f18c8101a70 tx=0x7f18b8009260 comp rx=0 tx=0).stop 2023-09-30T14:55:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.605+0000 7f18ce184700 1 -- 172.21.15.32:0/690972987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c8102b50 msgr2=0x7f18c8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.605+0000 7f18ce184700 1 --2- 172.21.15.32:0/690972987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c8102b50 0x7f18c8100be0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009fd0 tx=0x7f18c000bce0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.605+0000 7f18ce184700 1 -- 172.21.15.32:0/690972987 shutdown_connections 2023-09-30T14:55:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.605+0000 7f18ce184700 1 --2- 172.21.15.32:0/690972987 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f18b00a0fa0 0x7f18b00a3460 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.605+0000 7f18ce184700 1 --2- 172.21.15.32:0/690972987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18c8102b50 0x7f18c8100be0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.605+0000 7f18ce184700 1 --2- 172.21.15.32:0/690972987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18c80ff140 0x7f18c8100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.605+0000 7f18ce184700 1 --2- 172.21.15.32:0/690972987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18c81021b0 0x7f18c81006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.605+0000 7f18ce184700 1 -- 172.21.15.32:0/690972987 >> 172.21.15.32:0/690972987 conn(0x7f18c80faa30 msgr2=0x7f18c80fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.606+0000 7f18ce184700 1 -- 172.21.15.32:0/690972987 shutdown_connections 2023-09-30T14:55:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:06.606+0000 7f18ce184700 1 -- 172.21.15.32:0/690972987 wait complete. 2023-09-30T14:55:07.012 INFO:tasks.workunit.client.0.smithi032.stdout: 10 148435 37.76 MB/sec execute 100 sec latency 223.334 ms 2023-09-30T14:55:07.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:07 smithi156 ceph-mon[103579]: pgmap v792: 97 pgs: 97 active+clean; 709 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 10 MiB/s wr, 811 op/s 2023-09-30T14:55:07.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:07 smithi148 ceph-mon[102355]: pgmap v792: 97 pgs: 97 active+clean; 709 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 10 MiB/s wr, 811 op/s 2023-09-30T14:55:07.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:07 smithi032 ceph-mon[98592]: pgmap v792: 97 pgs: 97 active+clean; 709 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 10 MiB/s wr, 811 op/s 2023-09-30T14:55:08.011 INFO:tasks.workunit.client.0.smithi032.stdout: 10 149234 37.78 MB/sec execute 101 sec latency 57.665 ms 2023-09-30T14:55:09.012 INFO:tasks.workunit.client.0.smithi032.stdout: 10 150069 37.88 MB/sec execute 102 sec latency 65.211 ms 2023-09-30T14:55:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:09 smithi156 ceph-mon[103579]: pgmap v793: 97 pgs: 97 active+clean; 711 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 10 MiB/s wr, 796 op/s 2023-09-30T14:55:09.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:09 smithi148 ceph-mon[102355]: pgmap v793: 97 pgs: 97 active+clean; 711 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 10 MiB/s wr, 796 op/s 2023-09-30T14:55:09.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:09 smithi032 ceph-mon[98592]: pgmap v793: 97 pgs: 97 active+clean; 711 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 10 MiB/s wr, 796 op/s 2023-09-30T14:55:10.012 INFO:tasks.workunit.client.0.smithi032.stdout: 10 150609 37.79 MB/sec execute 103 sec latency 279.702 ms 2023-09-30T14:55:11.012 INFO:tasks.workunit.client.0.smithi032.stdout: 10 151237 37.81 MB/sec execute 104 sec latency 180.171 ms 2023-09-30T14:55:11.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:11 smithi156 ceph-mon[103579]: pgmap v794: 97 pgs: 97 active+clean; 709 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.4 MiB/s wr, 749 op/s 2023-09-30T14:55:11.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:11 smithi148 ceph-mon[102355]: pgmap v794: 97 pgs: 97 active+clean; 709 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.4 MiB/s wr, 749 op/s 2023-09-30T14:55:11.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:11 smithi032 ceph-mon[98592]: pgmap v794: 97 pgs: 97 active+clean; 709 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.4 MiB/s wr, 749 op/s 2023-09-30T14:55:12.012 INFO:tasks.workunit.client.0.smithi032.stdout: 10 151751 37.67 MB/sec execute 105 sec latency 48.518 ms 2023-09-30T14:55:13.012 INFO:tasks.workunit.client.0.smithi032.stdout: 10 152383 37.59 MB/sec execute 106 sec latency 110.228 ms 2023-09-30T14:55:13.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:13 smithi156 ceph-mon[103579]: pgmap v795: 97 pgs: 97 active+clean; 708 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.6 MiB/s wr, 813 op/s 2023-09-30T14:55:13.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:13 smithi148 ceph-mon[102355]: pgmap v795: 97 pgs: 97 active+clean; 708 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.6 MiB/s wr, 813 op/s 2023-09-30T14:55:13.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:13 smithi032 ceph-mon[98592]: pgmap v795: 97 pgs: 97 active+clean; 708 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.6 MiB/s wr, 813 op/s 2023-09-30T14:55:14.012 INFO:tasks.workunit.client.0.smithi032.stdout: 10 153289 37.69 MB/sec execute 107 sec latency 58.916 ms 2023-09-30T14:55:15.012 INFO:tasks.workunit.client.0.smithi032.stdout: 10 153798 37.62 MB/sec execute 108 sec latency 186.916 ms 2023-09-30T14:55:15.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:15 smithi156 ceph-mon[103579]: pgmap v796: 97 pgs: 97 active+clean; 713 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.9 MiB/s wr, 823 op/s 2023-09-30T14:55:15.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:15 smithi148 ceph-mon[102355]: pgmap v796: 97 pgs: 97 active+clean; 713 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.9 MiB/s wr, 823 op/s 2023-09-30T14:55:15.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:15 smithi032 ceph-mon[98592]: pgmap v796: 97 pgs: 97 active+clean; 713 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.9 MiB/s wr, 823 op/s 2023-09-30T14:55:16.012 INFO:tasks.workunit.client.0.smithi032.stdout: 10 154323 37.56 MB/sec execute 109 sec latency 106.201 ms 2023-09-30T14:55:17.012 INFO:tasks.workunit.client.0.smithi032.stdout: 10 154922 37.58 MB/sec execute 110 sec latency 80.306 ms 2023-09-30T14:55:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:17 smithi156 ceph-mon[103579]: pgmap v797: 97 pgs: 97 active+clean; 721 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.8 MiB/s wr, 809 op/s 2023-09-30T14:55:17.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:17 smithi148 ceph-mon[102355]: pgmap v797: 97 pgs: 97 active+clean; 721 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.8 MiB/s wr, 809 op/s 2023-09-30T14:55:17.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:17 smithi032 ceph-mon[98592]: pgmap v797: 97 pgs: 97 active+clean; 721 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.8 MiB/s wr, 809 op/s 2023-09-30T14:55:18.012 INFO:tasks.workunit.client.0.smithi032.stdout: 10 155418 37.45 MB/sec execute 111 sec latency 93.177 ms 2023-09-30T14:55:19.012 INFO:tasks.workunit.client.0.smithi032.stdout: 10 155919 37.32 MB/sec execute 112 sec latency 83.844 ms 2023-09-30T14:55:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.730+0000 7f970d378700 1 -- 172.21.15.32:0/1090823987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9708100fa0 msgr2=0x7f9708101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.730+0000 7f970d378700 1 --2- 172.21.15.32:0/1090823987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9708100fa0 0x7f9708101400 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009990 tx=0x7f96fc009ca0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.730+0000 7f970d378700 1 -- 172.21.15.32:0/1090823987 shutdown_connections 2023-09-30T14:55:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.730+0000 7f970d378700 1 --2- 172.21.15.32:0/1090823987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9708101940 0x7f97081081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.730+0000 7f970d378700 1 --2- 172.21.15.32:0/1090823987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97080ffdb0 0x7f97081001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.730+0000 7f970d378700 1 --2- 172.21.15.32:0/1090823987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9708100fa0 0x7f9708101400 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.730+0000 7f970d378700 1 -- 172.21.15.32:0/1090823987 >> 172.21.15.32:0/1090823987 conn(0x7f97080fb320 msgr2=0x7f97080fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.731+0000 7f970d378700 1 -- 172.21.15.32:0/1090823987 shutdown_connections 2023-09-30T14:55:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.731+0000 7f970d378700 1 -- 172.21.15.32:0/1090823987 wait complete. 2023-09-30T14:55:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.732+0000 7f970d378700 1 Processor -- start 2023-09-30T14:55:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.732+0000 7f970d378700 1 -- start start 2023-09-30T14:55:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.732+0000 7f970d378700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97080ffdb0 0x7f9708108b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.732+0000 7f970d378700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9708100fa0 0x7f97081090b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.732+0000 7f970d378700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9708101940 0x7f97081095f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.732+0000 7f970d378700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970810a710 con 0x7f9708101940 2023-09-30T14:55:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.732+0000 7f970d378700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970810a880 con 0x7f97080ffdb0 2023-09-30T14:55:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.732+0000 7f970d378700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970810aa40 con 0x7f9708100fa0 2023-09-30T14:55:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.733+0000 7f9706ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97080ffdb0 0x7f9708108b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.734+0000 7f9706ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97080ffdb0 0x7f9708108b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:50498/0 (socket says 172.21.15.32:50498) 2023-09-30T14:55:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.734+0000 7f9706ffd700 1 -- 172.21.15.32:0/1859653162 learned_addr learned my addr 172.21.15.32:0/1859653162 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.734+0000 7f97067fc700 1 --2- 172.21.15.32:0/1859653162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9708100fa0 0x7f97081090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.734+0000 7f9706ffd700 1 -- 172.21.15.32:0/1859653162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9708100fa0 msgr2=0x7f97081090b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.734+0000 7f9706ffd700 1 --2- 172.21.15.32:0/1859653162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9708100fa0 0x7f97081090b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.734+0000 7f9706ffd700 1 -- 172.21.15.32:0/1859653162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9708101940 msgr2=0x7f97081095f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.734+0000 7f9706ffd700 1 --2- 172.21.15.32:0/1859653162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9708101940 0x7f97081095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.734+0000 7f9706ffd700 1 -- 172.21.15.32:0/1859653162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96fc0095f0 con 0x7f97080ffdb0 2023-09-30T14:55:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.734+0000 7f9706ffd700 1 --2- 172.21.15.32:0/1859653162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97080ffdb0 0x7f9708108b70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f970000efc0 tx=0x7f970000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.735+0000 7f96effff700 1 -- 172.21.15.32:0/1859653162 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9700018070 con 0x7f97080ffdb0 2023-09-30T14:55:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.735+0000 7f970d378700 1 -- 172.21.15.32:0/1859653162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970819ac80 con 0x7f97080ffdb0 2023-09-30T14:55:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.735+0000 7f96effff700 1 -- 172.21.15.32:0/1859653162 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f970000f040 con 0x7f97080ffdb0 2023-09-30T14:55:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.735+0000 7f96effff700 1 -- 172.21.15.32:0/1859653162 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9700004310 con 0x7f97080ffdb0 2023-09-30T14:55:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.735+0000 7f970d378700 1 -- 172.21.15.32:0/1859653162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970819b1e0 con 0x7f97080ffdb0 2023-09-30T14:55:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.738+0000 7f96effff700 1 -- 172.21.15.32:0/1859653162 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f970000ca70 con 0x7f97080ffdb0 2023-09-30T14:55:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.738+0000 7f96effff700 1 --2- 172.21.15.32:0/1859653162 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f96f00a1020 0x7f96f00a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.739+0000 7f96effff700 1 -- 172.21.15.32:0/1859653162 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f97000cc540 con 0x7f97080ffdb0 2023-09-30T14:55:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.739+0000 7f97067fc700 1 --2- 172.21.15.32:0/1859653162 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f96f00a1020 0x7f96f00a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.739+0000 7f97067fc700 1 --2- 172.21.15.32:0/1859653162 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f96f00a1020 0x7f96f00a34e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00b2e0 tx=0x7f96fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.739+0000 7f9719560b80 1 Processor -- start 2023-09-30T14:55:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.739+0000 7f9719560b80 1 -- start start 2023-09-30T14:55:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f9719560b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32830200 0x562a32830620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f9719560b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32831310 0x562a32831790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f9719560b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a32838560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f9719560b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a32798400 con 0x562a32831cd0 2023-09-30T14:55:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f9719560b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a32789bc0 con 0x562a32830200 2023-09-30T14:55:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f9719560b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a327874e0 con 0x562a32831310 2023-09-30T14:55:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f97077fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a32838560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f97067fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32831310 0x562a32831790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f97067fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32831310 0x562a32831790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53414/0 (socket says 172.21.15.32:53414) 2023-09-30T14:55:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f97077fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a32838560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39034/0 (socket says 172.21.15.32:39034) 2023-09-30T14:55:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f97067fc700 1 -- 172.21.15.32:0/3198587563 learned_addr learned my addr 172.21.15.32:0/3198587563 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f97077fe700 1 -- 172.21.15.32:0/3198587563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32831310 msgr2=0x562a32831790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f97077fe700 1 --2- 172.21.15.32:0/3198587563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32831310 0x562a32831790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f97077fe700 1 -- 172.21.15.32:0/3198587563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32830200 msgr2=0x562a32830620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f97077fe700 1 --2- 172.21.15.32:0/3198587563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32830200 0x562a32830620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f97077fe700 1 -- 172.21.15.32:0/3198587563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a327e5870 con 0x562a32831cd0 2023-09-30T14:55:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.740+0000 7f97077fe700 1 --2- 172.21.15.32:0/3198587563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a32838560 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f96f800ba50 tx=0x7f96f800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec8c25d4180865b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.741+0000 7f96ed7fa700 1 -- 172.21.15.32:0/3198587563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f800d970 con 0x562a32831cd0 2023-09-30T14:55:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.741+0000 7f96ed7fa700 1 -- 172.21.15.32:0/3198587563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96f800b500 con 0x562a32831cd0 2023-09-30T14:55:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.741+0000 7f96ed7fa700 1 -- 172.21.15.32:0/3198587563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f800f3c0 con 0x562a32831cd0 2023-09-30T14:55:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.741+0000 7f9719560b80 1 -- 172.21.15.32:0/3198587563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 msgr2=0x562a32838560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.741+0000 7f9719560b80 1 --2- 172.21.15.32:0/3198587563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a32838560 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f96f800ba50 tx=0x7f96f800bd60 comp rx=0 tx=0).stop 2023-09-30T14:55:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.742+0000 7f9719560b80 1 -- 172.21.15.32:0/3198587563 shutdown_connections 2023-09-30T14:55:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.742+0000 7f9719560b80 1 --2- 172.21.15.32:0/3198587563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a32838560 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.742+0000 7f9719560b80 1 --2- 172.21.15.32:0/3198587563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32830200 0x562a32830620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.742+0000 7f9719560b80 1 --2- 172.21.15.32:0/3198587563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32831310 0x562a32831790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.742+0000 7f9719560b80 1 -- 172.21.15.32:0/3198587563 >> 172.21.15.32:0/3198587563 conn(0x562a327a4200 msgr2=0x562a3282aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.742+0000 7f9719560b80 1 -- 172.21.15.32:0/3198587563 shutdown_connections 2023-09-30T14:55:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.742+0000 7f9719560b80 1 -- 172.21.15.32:0/3198587563 wait complete. 2023-09-30T14:55:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.742+0000 7f9719560b80 1 -- start start 2023-09-30T14:55:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.742+0000 7f9719560b80 1 Processor -- start 2023-09-30T14:55:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f9719560b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32830200 0x562a32839f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f9719560b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32831310 0x562a3283a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f9719560b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a328414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f9719560b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a32782dd0 con 0x562a32831cd0 2023-09-30T14:55:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f9719560b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3271f6b0 con 0x562a32831310 2023-09-30T14:55:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f9719560b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3271f350 con 0x562a32830200 2023-09-30T14:55:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f9719560b80 10 client.0 _ll_register_callbacks cb 0x562a3257c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:55:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f97067fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32831310 0x562a3283a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f97077fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a328414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f9706ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32830200 0x562a32839f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f97077fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a328414d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39040/0 (socket says 172.21.15.32:39040) 2023-09-30T14:55:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f97077fe700 1 -- 172.21.15.32:0/2225130162 learned_addr learned my addr 172.21.15.32:0/2225130162 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f97077fe700 1 -- 172.21.15.32:0/2225130162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32830200 msgr2=0x562a32839f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f97077fe700 1 --2- 172.21.15.32:0/2225130162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32830200 0x562a32839f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f97077fe700 1 -- 172.21.15.32:0/2225130162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32831310 msgr2=0x562a3283a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f97077fe700 1 --2- 172.21.15.32:0/2225130162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32831310 0x562a3283a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f97077fe700 1 -- 172.21.15.32:0/2225130162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f800b680 con 0x562a32831cd0 2023-09-30T14:55:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.743+0000 7f97067fc700 1 --2- 172.21.15.32:0/2225130162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32831310 0x562a3283a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:55:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.744+0000 7f97077fe700 1 --2- 172.21.15.32:0/2225130162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a328414d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f96f8006010 tx=0x7f96f8011ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e9bc711300ade7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.745+0000 7f96eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:55:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.745+0000 7f96eaff5700 1 -- 172.21.15.32:0/2225130162 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8004a80 con 0x562a32831cd0 2023-09-30T14:55:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.745+0000 7f96eaff5700 1 -- 172.21.15.32:0/2225130162 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f96f8004be0 con 0x562a32831cd0 2023-09-30T14:55:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.745+0000 7f9719560b80 1 -- 172.21.15.32:0/2225130162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a3277bdb0 con 0x562a32831cd0 2023-09-30T14:55:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.746+0000 7f96eaff5700 1 -- 172.21.15.32:0/2225130162 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96f8004da0 con 0x562a32831cd0 2023-09-30T14:55:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.746+0000 7f9719560b80 10 client.16887 fetch_fsmap learned FSMap version 49 2023-09-30T14:55:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.746+0000 7f9719560b80 1 -- 172.21.15.32:0/2225130162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x562a325add80 con 0x562a32831cd0 2023-09-30T14:55:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.746+0000 7f96eaff5700 1 -- 172.21.15.32:0/2225130162 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562a3277bdb0 con 0x562a32831cd0 2023-09-30T14:55:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.746+0000 7f96eaff5700 1 -- 172.21.15.32:0/2225130162 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f96f800da20 con 0x562a32831cd0 2023-09-30T14:55:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.746+0000 7f9719560b80 10 client.16887 fetch_fsmap finished waiting for FSMap version 49 2023-09-30T14:55:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.746+0000 7f9719560b80 10 client.16887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:55:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.746+0000 7f9719560b80 20 client.16887 populate_metadata read hostname 'smithi032' 2023-09-30T14:55:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.746+0000 7f9719560b80 1 --2- 172.21.15.32:0/2225130162 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562a32845d40 0x562a32848160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.746+0000 7f9719560b80 4 client.16887 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:55:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.746+0000 7f9719560b80 1 -- 172.21.15.32:0/2225130162 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a3277bad0 con 0x562a32845d40 2023-09-30T14:55:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.747+0000 7f9706ffd700 1 --2- 172.21.15.32:0/2225130162 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562a32845d40 0x562a32848160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.747+0000 7f96eaff5700 10 client.16887 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:55:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.747+0000 7f9706ffd700 1 --2- 172.21.15.32:0/2225130162 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562a32845d40 0x562a32848160 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59f05e9f7fe85b75 server_cookie=f90d8ae1ebffdea in_seq=0 out_seq=0 2023-09-30T14:55:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.750+0000 7f96eaff5700 1 -- 172.21.15.32:0/2225130162 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562a3277bad0 con 0x562a32845d40 2023-09-30T14:55:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.750+0000 7f96eaff5700 10 client.16887 handle_command_reply: tid=0 2023-09-30T14:55:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.750+0000 7f9719560b80 1 client.16887 shutdown 2023-09-30T14:55:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.750+0000 7f9719560b80 1 -- 172.21.15.32:0/2225130162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 msgr2=0x562a328414d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.750+0000 7f9719560b80 1 --2- 172.21.15.32:0/2225130162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a328414d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f96f8006010 tx=0x7f96f8011ff0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.750+0000 7f9719560b80 1 -- 172.21.15.32:0/2225130162 shutdown_connections 2023-09-30T14:55:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.750+0000 7f9719560b80 1 --2- 172.21.15.32:0/2225130162 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562a32845d40 0x562a32848160 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.750+0000 7f9719560b80 1 --2- 172.21.15.32:0/2225130162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a328414d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.750+0000 7f96eaff5700 0 client.16887 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:55:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.750+0000 7f9719560b80 1 --2- 172.21.15.32:0/2225130162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32830200 0x562a32839f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.750+0000 7f9719560b80 1 --2- 172.21.15.32:0/2225130162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32831310 0x562a3283a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.750+0000 7f9719560b80 1 -- 172.21.15.32:0/2225130162 >> 172.21.15.32:0/2225130162 conn(0x562a327a4200 msgr2=0x562a3282a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.750+0000 7f9719560b80 1 -- 172.21.15.32:0/2225130162 shutdown_connections 2023-09-30T14:55:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.750+0000 7f9719560b80 1 -- 172.21.15.32:0/2225130162 wait complete. 2023-09-30T14:55:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.750+0000 7f9719560b80 20 client.16887 trim_cache size 0 max 16384 2023-09-30T14:55:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.775+0000 7f9719560b80 1 Processor -- start 2023-09-30T14:55:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.775+0000 7f9719560b80 1 -- start start 2023-09-30T14:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.775+0000 7f9719560b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32830200 0x562a328380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.775+0000 7f9719560b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32831310 0x562a32838620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.775+0000 7f9719560b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a32841490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.775+0000 7f9719560b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a326038c0 con 0x562a32831cd0 2023-09-30T14:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.775+0000 7f9719560b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a32839af0 con 0x562a32830200 2023-09-30T14:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.775+0000 7f9719560b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a32827bf0 con 0x562a32831310 2023-09-30T14:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.775+0000 7f97077fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a32841490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.775+0000 7f97077fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a32841490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39046/0 (socket says 172.21.15.32:39046) 2023-09-30T14:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.775+0000 7f97077fe700 1 -- 172.21.15.32:0/1015085113 learned_addr learned my addr 172.21.15.32:0/1015085113 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.775+0000 7f9706ffd700 1 --2- 172.21.15.32:0/1015085113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32830200 0x562a328380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.776+0000 7f97077fe700 1 -- 172.21.15.32:0/1015085113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32831310 msgr2=0x562a32838620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.776+0000 7f97077fe700 1 --2- 172.21.15.32:0/1015085113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32831310 0x562a32838620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.776+0000 7f97077fe700 1 -- 172.21.15.32:0/1015085113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32830200 msgr2=0x562a328380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.776+0000 7f97077fe700 1 --2- 172.21.15.32:0/1015085113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32830200 0x562a328380e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.776+0000 7f97077fe700 1 -- 172.21.15.32:0/1015085113 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a32596c00 con 0x562a32831cd0 2023-09-30T14:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.776+0000 7f97077fe700 1 --2- 172.21.15.32:0/1015085113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a32841490 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f96f800b970 tx=0x7f96f80059f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fc6f4d2fed3136 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.776+0000 7f96edffb700 1 -- 172.21.15.32:0/1015085113 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8000ea0 con 0x562a32831cd0 2023-09-30T14:55:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.776+0000 7f96edffb700 1 -- 172.21.15.32:0/1015085113 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96f8014040 con 0x562a32831cd0 2023-09-30T14:55:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.776+0000 7f96edffb700 1 -- 172.21.15.32:0/1015085113 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f801b970 con 0x562a32831cd0 2023-09-30T14:55:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.777+0000 7f9719560b80 1 -- 172.21.15.32:0/1015085113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 msgr2=0x562a32841490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.777+0000 7f9719560b80 1 --2- 172.21.15.32:0/1015085113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a32841490 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f96f800b970 tx=0x7f96f80059f0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.778+0000 7f9719560b80 1 -- 172.21.15.32:0/1015085113 shutdown_connections 2023-09-30T14:55:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.778+0000 7f9719560b80 1 --2- 172.21.15.32:0/1015085113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831cd0 0x562a32841490 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.778+0000 7f9719560b80 1 --2- 172.21.15.32:0/1015085113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32830200 0x562a328380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.778+0000 7f9719560b80 1 --2- 172.21.15.32:0/1015085113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32831310 0x562a32838620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.778+0000 7f9719560b80 1 -- 172.21.15.32:0/1015085113 >> 172.21.15.32:0/1015085113 conn(0x562a327a4200 msgr2=0x562a32838c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.778+0000 7f9719560b80 1 -- 172.21.15.32:0/1015085113 shutdown_connections 2023-09-30T14:55:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.778+0000 7f9719560b80 1 -- 172.21.15.32:0/1015085113 wait complete. 2023-09-30T14:55:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.778+0000 7f9719560b80 1 -- start start 2023-09-30T14:55:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.778+0000 7f9719560b80 1 Processor -- start 2023-09-30T14:55:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.779+0000 7f9719560b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32830200 0x562a3283ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.779+0000 7f9719560b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831310 0x562a3283f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.779+0000 7f9719560b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32831cd0 0x562a328401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.779+0000 7f9719560b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a328406e0 con 0x562a32831310 2023-09-30T14:55:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.779+0000 7f9719560b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a32840850 con 0x562a32831cd0 2023-09-30T14:55:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.779+0000 7f9719560b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a328409c0 con 0x562a32830200 2023-09-30T14:55:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.779+0000 7f9719560b80 10 client.0 _ll_register_callbacks cb 0x7f96fc004500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:55:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.779+0000 7f97077fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32831cd0 0x562a328401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.779+0000 7f97077fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32831cd0 0x562a328401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:50530/0 (socket says 172.21.15.32:50530) 2023-09-30T14:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.779+0000 7f97077fe700 1 -- 172.21.15.32:0/660766746 learned_addr learned my addr 172.21.15.32:0/660766746 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.779+0000 7f9706ffd700 1 --2- 172.21.15.32:0/660766746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32830200 0x562a3283ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.780+0000 7f9706ffd700 1 -- 172.21.15.32:0/660766746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32831cd0 msgr2=0x562a328401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.780+0000 7f9706ffd700 1 --2- 172.21.15.32:0/660766746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32831cd0 0x562a328401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.780+0000 7f9706ffd700 1 -- 172.21.15.32:0/660766746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831310 msgr2=0x562a3283f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.780+0000 7f9706ffd700 1 --2- 172.21.15.32:0/660766746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831310 0x562a3283f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.780+0000 7f9706ffd700 1 -- 172.21.15.32:0/660766746 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a325add80 con 0x562a32830200 2023-09-30T14:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.780+0000 7f9706ffd700 1 --2- 172.21.15.32:0/660766746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32830200 0x562a3283ec50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x562a3282ffa0 tx=0x7f9700099990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9d9dad693f8c783 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.780+0000 7f96eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:55:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.784+0000 7f96eaff5700 1 -- 172.21.15.32:0/660766746 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97000a1da0 con 0x562a32830200 2023-09-30T14:55:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.784+0000 7f96eaff5700 1 -- 172.21.15.32:0/660766746 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f97000b7a90 con 0x562a32830200 2023-09-30T14:55:19.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.784+0000 7f9719560b80 1 -- 172.21.15.32:0/660766746 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a32823090 con 0x562a32830200 2023-09-30T14:55:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.787+0000 7f96eaff5700 1 -- 172.21.15.32:0/660766746 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9700097e40 con 0x562a32830200 2023-09-30T14:55:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.787+0000 7f9719560b80 10 client.26551 fetch_fsmap learned FSMap version 49 2023-09-30T14:55:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.787+0000 7f9719560b80 1 -- 172.21.15.32:0/660766746 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x562a325d41f0 con 0x562a32830200 2023-09-30T14:55:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.787+0000 7f96eaff5700 1 -- 172.21.15.32:0/660766746 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562a32823090 con 0x562a32830200 2023-09-30T14:55:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.787+0000 7f96eaff5700 1 -- 172.21.15.32:0/660766746 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9700099d00 con 0x562a32830200 2023-09-30T14:55:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.788+0000 7f9719560b80 10 client.26551 fetch_fsmap finished waiting for FSMap version 49 2023-09-30T14:55:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.788+0000 7f9719560b80 10 client.26551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:55:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.788+0000 7f9719560b80 20 client.26551 populate_metadata read hostname 'smithi032' 2023-09-30T14:55:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.788+0000 7f9719560b80 1 --2- 172.21.15.32:0/660766746 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562a328ab420 0x562a328ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.788+0000 7f9719560b80 4 client.26551 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:55:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.788+0000 7f9719560b80 1 -- 172.21.15.32:0/660766746 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562a325cef10 con 0x562a328ab420 2023-09-30T14:55:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.789+0000 7f97067fc700 1 --2- 172.21.15.32:0/660766746 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562a328ab420 0x562a328ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.792+0000 7f96eaff5700 10 client.26551 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:55:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:19.792+0000 7f97067fc700 1 --2- 172.21.15.32:0/660766746 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562a328ab420 0x562a328ad840 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8c288caf43e3663 server_cookie=23dd6287276a381c in_seq=0 out_seq=0 2023-09-30T14:55:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:19 smithi156 ceph-mon[103579]: pgmap v798: 97 pgs: 97 active+clean; 725 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.7 MiB/s wr, 828 op/s 2023-09-30T14:55:19.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:19 smithi148 ceph-mon[102355]: pgmap v798: 97 pgs: 97 active+clean; 725 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.7 MiB/s wr, 828 op/s 2023-09-30T14:55:19.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:19 smithi032 ceph-mon[98592]: pgmap v798: 97 pgs: 97 active+clean; 725 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.7 MiB/s wr, 828 op/s 2023-09-30T14:55:20.017 INFO:tasks.workunit.client.0.smithi032.stdout: 10 156194 37.14 MB/sec execute 113 sec latency 213.430 ms 2023-09-30T14:55:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.212+0000 7f96eaff5700 1 -- 172.21.15.32:0/660766746 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562a325cef10 con 0x562a328ab420 2023-09-30T14:55:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.212+0000 7f96eaff5700 10 client.26551 handle_command_reply: tid=0 2023-09-30T14:55:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.213+0000 7f9719560b80 1 client.26551 shutdown 2023-09-30T14:55:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.213+0000 7f9719560b80 1 -- 172.21.15.32:0/660766746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32830200 msgr2=0x562a3283ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.213+0000 7f9719560b80 1 --2- 172.21.15.32:0/660766746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32830200 0x562a3283ec50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x562a3282ffa0 tx=0x7f9700099990 comp rx=0 tx=0).stop 2023-09-30T14:55:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.213+0000 7f9719560b80 1 -- 172.21.15.32:0/660766746 shutdown_connections 2023-09-30T14:55:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.213+0000 7f9719560b80 1 --2- 172.21.15.32:0/660766746 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x562a328ab420 0x562a328ad840 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.213+0000 7f9719560b80 1 --2- 172.21.15.32:0/660766746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562a32831cd0 0x562a328401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.213+0000 7f9719560b80 1 --2- 172.21.15.32:0/660766746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a32830200 0x562a3283ec50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.213+0000 7f9719560b80 1 --2- 172.21.15.32:0/660766746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a32831310 0x562a3283f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.213+0000 7f9719560b80 1 -- 172.21.15.32:0/660766746 >> 172.21.15.32:0/660766746 conn(0x562a327a4200 msgr2=0x562a328380e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.213+0000 7f96eaff5700 0 client.26551 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:55:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.213+0000 7f9719560b80 1 -- 172.21.15.32:0/660766746 shutdown_connections 2023-09-30T14:55:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.213+0000 7f9719560b80 1 -- 172.21.15.32:0/660766746 wait complete. 2023-09-30T14:55:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.213+0000 7f9719560b80 20 client.26551 trim_cache size 0 max 16384 2023-09-30T14:55:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:55:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:55:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:55:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:55:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.214+0000 7f970d378700 1 -- 172.21.15.32:0/1859653162 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f96f00a1020 msgr2=0x7f96f00a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.214+0000 7f970d378700 1 --2- 172.21.15.32:0/1859653162 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f96f00a1020 0x7f96f00a34e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00b2e0 tx=0x7f96fc005fb0 comp rx=0 tx=0).stop 2023-09-30T14:55:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.214+0000 7f970d378700 1 -- 172.21.15.32:0/1859653162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97080ffdb0 msgr2=0x7f9708108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.214+0000 7f970d378700 1 --2- 172.21.15.32:0/1859653162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97080ffdb0 0x7f9708108b70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f970000efc0 tx=0x7f970000c3a0 comp rx=0 tx=0).stop 2023-09-30T14:55:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.214+0000 7f970d378700 1 -- 172.21.15.32:0/1859653162 shutdown_connections 2023-09-30T14:55:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.214+0000 7f970d378700 1 --2- 172.21.15.32:0/1859653162 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f96f00a1020 0x7f96f00a34e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.214+0000 7f970d378700 1 --2- 172.21.15.32:0/1859653162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9708101940 0x7f97081095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.214+0000 7f970d378700 1 --2- 172.21.15.32:0/1859653162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97080ffdb0 0x7f9708108b70 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f970000efc0 tx=0x7f970000c3a0 comp rx=0 tx=0).stop 2023-09-30T14:55:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.214+0000 7f970d378700 1 --2- 172.21.15.32:0/1859653162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9708100fa0 0x7f97081090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.214+0000 7f970d378700 1 -- 172.21.15.32:0/1859653162 >> 172.21.15.32:0/1859653162 conn(0x7f97080fb320 msgr2=0x7f9708104430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.216+0000 7f970d378700 1 -- 172.21.15.32:0/1859653162 shutdown_connections 2023-09-30T14:55:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:20.216+0000 7f970d378700 1 -- 172.21.15.32:0/1859653162 wait complete. 2023-09-30T14:55:21.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 156678 37.00 MB/sec execute 114 sec latency 279.089 ms 2023-09-30T14:55:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:21 smithi156 ceph-mon[103579]: pgmap v799: 97 pgs: 97 active+clean; 727 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.1 MiB/s wr, 644 op/s 2023-09-30T14:55:21.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:21 smithi148 ceph-mon[102355]: pgmap v799: 97 pgs: 97 active+clean; 727 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.1 MiB/s wr, 644 op/s 2023-09-30T14:55:21.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:21 smithi032 ceph-mon[98592]: pgmap v799: 97 pgs: 97 active+clean; 727 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.1 MiB/s wr, 644 op/s 2023-09-30T14:55:22.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 157646 37.09 MB/sec execute 115 sec latency 54.075 ms 2023-09-30T14:55:23.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 158568 37.28 MB/sec execute 116 sec latency 49.760 ms 2023-09-30T14:55:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:23 smithi156 ceph-mon[103579]: pgmap v800: 97 pgs: 97 active+clean; 723 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.8 MiB/s wr, 684 op/s 2023-09-30T14:55:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:23 smithi148 ceph-mon[102355]: pgmap v800: 97 pgs: 97 active+clean; 723 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.8 MiB/s wr, 684 op/s 2023-09-30T14:55:23.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:23 smithi032 ceph-mon[98592]: pgmap v800: 97 pgs: 97 active+clean; 723 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.8 MiB/s wr, 684 op/s 2023-09-30T14:55:24.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 159239 37.35 MB/sec execute 117 sec latency 67.708 ms 2023-09-30T14:55:25.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 159606 37.18 MB/sec execute 118 sec latency 218.883 ms 2023-09-30T14:55:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:25 smithi156 ceph-mon[103579]: pgmap v801: 97 pgs: 97 active+clean; 730 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 726 op/s 2023-09-30T14:55:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:25 smithi148 ceph-mon[102355]: pgmap v801: 97 pgs: 97 active+clean; 730 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 726 op/s 2023-09-30T14:55:25.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:25 smithi032 ceph-mon[98592]: pgmap v801: 97 pgs: 97 active+clean; 730 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 726 op/s 2023-09-30T14:55:26.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 160171 37.03 MB/sec execute 119 sec latency 69.589 ms 2023-09-30T14:55:27.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161033 37.14 MB/sec execute 120 sec latency 98.129 ms 2023-09-30T14:55:27.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:27 smithi156 ceph-mon[103579]: pgmap v802: 97 pgs: 97 active+clean; 729 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.2 MiB/s wr, 740 op/s 2023-09-30T14:55:27.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:27 smithi148 ceph-mon[102355]: pgmap v802: 97 pgs: 97 active+clean; 729 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.2 MiB/s wr, 740 op/s 2023-09-30T14:55:27.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:27 smithi032 ceph-mon[98592]: pgmap v802: 97 pgs: 97 active+clean; 729 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.2 MiB/s wr, 740 op/s 2023-09-30T14:55:28.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 161848 37.16 MB/sec execute 121 sec latency 72.083 ms 2023-09-30T14:55:29.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 162610 37.20 MB/sec execute 122 sec latency 85.901 ms 2023-09-30T14:55:29.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:29 smithi148 ceph-mon[102355]: pgmap v803: 97 pgs: 97 active+clean; 729 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.0 MiB/s wr, 737 op/s 2023-09-30T14:55:29.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:29 smithi032 ceph-mon[98592]: pgmap v803: 97 pgs: 97 active+clean; 729 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.0 MiB/s wr, 737 op/s 2023-09-30T14:55:30.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 163182 37.18 MB/sec execute 123 sec latency 306.477 ms 2023-09-30T14:55:30.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:29 smithi156 ceph-mon[103579]: pgmap v803: 97 pgs: 97 active+clean; 729 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.0 MiB/s wr, 737 op/s 2023-09-30T14:55:31.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 163712 37.15 MB/sec execute 124 sec latency 104.021 ms 2023-09-30T14:55:31.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:31 smithi148 ceph-mon[102355]: pgmap v804: 97 pgs: 97 active+clean; 711 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.6 MiB/s wr, 684 op/s 2023-09-30T14:55:31.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:31 smithi032 ceph-mon[98592]: pgmap v804: 97 pgs: 97 active+clean; 711 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.6 MiB/s wr, 684 op/s 2023-09-30T14:55:32.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 164221 37.03 MB/sec execute 125 sec latency 99.387 ms 2023-09-30T14:55:32.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:31 smithi156 ceph-mon[103579]: pgmap v804: 97 pgs: 97 active+clean; 711 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.6 MiB/s wr, 684 op/s 2023-09-30T14:55:33.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 165127 37.12 MB/sec execute 126 sec latency 50.961 ms 2023-09-30T14:55:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.345+0000 7f479a216700 1 -- 172.21.15.32:0/3528537336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4794101940 msgr2=0x7f47941081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.345+0000 7f479a216700 1 --2- 172.21.15.32:0/3528537336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4794101940 0x7f47941081f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f478c00b0d0 tx=0x7f478c00b3e0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.346+0000 7f479a216700 1 -- 172.21.15.32:0/3528537336 shutdown_connections 2023-09-30T14:55:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.346+0000 7f479a216700 1 --2- 172.21.15.32:0/3528537336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4794101940 0x7f47941081f0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.346+0000 7f479a216700 1 --2- 172.21.15.32:0/3528537336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47940ffdb0 0x7f47941001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.346+0000 7f479a216700 1 --2- 172.21.15.32:0/3528537336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4794100fa0 0x7f4794101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.346+0000 7f479a216700 1 -- 172.21.15.32:0/3528537336 >> 172.21.15.32:0/3528537336 conn(0x7f47940fb320 msgr2=0x7f47940fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.346+0000 7f479a216700 1 -- 172.21.15.32:0/3528537336 shutdown_connections 2023-09-30T14:55:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.346+0000 7f479a216700 1 -- 172.21.15.32:0/3528537336 wait complete. 2023-09-30T14:55:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.346+0000 7f479a216700 1 Processor -- start 2023-09-30T14:55:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.346+0000 7f479a216700 1 -- start start 2023-09-30T14:55:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f479a216700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47940ffdb0 0x7f4794196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f479a216700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4794100fa0 0x7f47941972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f479a216700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4794101940 0x7f47941980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f479a216700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4794198790 con 0x7f47940ffdb0 2023-09-30T14:55:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f479a216700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4794198900 con 0x7f4794101940 2023-09-30T14:55:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f479a216700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4794198ac0 con 0x7f4794100fa0 2023-09-30T14:55:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f47937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47940ffdb0 0x7f4794196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f47937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47940ffdb0 0x7f4794196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39064/0 (socket says 172.21.15.32:39064) 2023-09-30T14:55:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f4792ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4794100fa0 0x7f47941972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f47937fe700 1 -- 172.21.15.32:0/2816219470 learned_addr learned my addr 172.21.15.32:0/2816219470 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f4793fff700 1 --2- 172.21.15.32:0/2816219470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4794101940 0x7f47941980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f47937fe700 1 -- 172.21.15.32:0/2816219470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4794100fa0 msgr2=0x7f47941972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f47937fe700 1 --2- 172.21.15.32:0/2816219470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4794100fa0 0x7f47941972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f47937fe700 1 -- 172.21.15.32:0/2816219470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4794101940 msgr2=0x7f47941980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f47937fe700 1 --2- 172.21.15.32:0/2816219470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4794101940 0x7f47941980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f47937fe700 1 -- 172.21.15.32:0/2816219470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c009d00 con 0x7f47940ffdb0 2023-09-30T14:55:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.347+0000 7f47937fe700 1 --2- 172.21.15.32:0/2816219470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47940ffdb0 0x7f4794196da0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f478400e970 tx=0x7f478400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.348+0000 7f4790ff9700 1 -- 172.21.15.32:0/2816219470 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4784009790 con 0x7f47940ffdb0 2023-09-30T14:55:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.348+0000 7f479a216700 1 -- 172.21.15.32:0/2816219470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4794198da0 con 0x7f47940ffdb0 2023-09-30T14:55:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.348+0000 7f479a216700 1 -- 172.21.15.32:0/2816219470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4794102400 con 0x7f47940ffdb0 2023-09-30T14:55:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.348+0000 7f4790ff9700 1 -- 172.21.15.32:0/2816219470 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f478400c7e0 con 0x7f47940ffdb0 2023-09-30T14:55:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.348+0000 7f4790ff9700 1 -- 172.21.15.32:0/2816219470 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4784003b60 con 0x7f47940ffdb0 2023-09-30T14:55:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.350+0000 7f4790ff9700 1 -- 172.21.15.32:0/2816219470 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f4784004220 con 0x7f47940ffdb0 2023-09-30T14:55:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.351+0000 7f4790ff9700 1 --2- 172.21.15.32:0/2816219470 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f477c0a0fa0 0x7f477c0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.351+0000 7f4790ff9700 1 -- 172.21.15.32:0/2816219470 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4784014070 con 0x7f47940ffdb0 2023-09-30T14:55:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.351+0000 7f4792ffd700 1 --2- 172.21.15.32:0/2816219470 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f477c0a0fa0 0x7f477c0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.351+0000 7f4792ffd700 1 --2- 172.21.15.32:0/2816219470 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f477c0a0fa0 0x7f477c0a3460 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4794196b10 tx=0x7f4788009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.352+0000 7f47a63feb80 1 Processor -- start 2023-09-30T14:55:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.352+0000 7f47a63feb80 1 -- start start 2023-09-30T14:55:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.352+0000 7f47a63feb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1b200 0x55a473c1b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.352+0000 7f47a63feb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1c310 0x55a473c1c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.352+0000 7f47a63feb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1ccd0 0x55a473c23560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.352+0000 7f47a63feb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a473b83400 con 0x55a473c1b200 2023-09-30T14:55:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.352+0000 7f47a63feb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a473b74bc0 con 0x55a473c1ccd0 2023-09-30T14:55:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.352+0000 7f47a63feb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a473b724e0 con 0x55a473c1c310 2023-09-30T14:55:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.353+0000 7f47937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1b200 0x55a473c1b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.353+0000 7f47937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1b200 0x55a473c1b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39076/0 (socket says 172.21.15.32:39076) 2023-09-30T14:55:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.353+0000 7f47937fe700 1 -- 172.21.15.32:0/2112021625 learned_addr learned my addr 172.21.15.32:0/2112021625 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.353+0000 7f4792ffd700 1 --2- 172.21.15.32:0/2112021625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1c310 0x55a473c1c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.353+0000 7f47937fe700 1 -- 172.21.15.32:0/2112021625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1c310 msgr2=0x55a473c1c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.353+0000 7f47937fe700 1 --2- 172.21.15.32:0/2112021625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1c310 0x55a473c1c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.353+0000 7f47937fe700 1 -- 172.21.15.32:0/2112021625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1ccd0 msgr2=0x55a473c23560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.353+0000 7f47937fe700 1 --2- 172.21.15.32:0/2112021625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1ccd0 0x55a473c23560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.353+0000 7f47937fe700 1 -- 172.21.15.32:0/2112021625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a473bd0870 con 0x55a473c1b200 2023-09-30T14:55:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.353+0000 7f47937fe700 1 --2- 172.21.15.32:0/2112021625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1b200 0x55a473c1b620 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4784096820 tx=0x7f4784099790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ce40635695068d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.353+0000 7f4779ffb700 1 -- 172.21.15.32:0/2112021625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478409df00 con 0x55a473c1b200 2023-09-30T14:55:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.353+0000 7f4779ffb700 1 -- 172.21.15.32:0/2112021625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f478409d070 con 0x55a473c1b200 2023-09-30T14:55:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.353+0000 7f4779ffb700 1 -- 172.21.15.32:0/2112021625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478409a270 con 0x55a473c1b200 2023-09-30T14:55:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.354+0000 7f47a63feb80 1 -- 172.21.15.32:0/2112021625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1b200 msgr2=0x55a473c1b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.354+0000 7f47a63feb80 1 --2- 172.21.15.32:0/2112021625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1b200 0x55a473c1b620 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4784096820 tx=0x7f4784099790 comp rx=0 tx=0).stop 2023-09-30T14:55:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.354+0000 7f47a63feb80 1 -- 172.21.15.32:0/2112021625 shutdown_connections 2023-09-30T14:55:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.354+0000 7f47a63feb80 1 --2- 172.21.15.32:0/2112021625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1ccd0 0x55a473c23560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.354+0000 7f47a63feb80 1 --2- 172.21.15.32:0/2112021625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1b200 0x55a473c1b620 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.354+0000 7f47a63feb80 1 --2- 172.21.15.32:0/2112021625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1c310 0x55a473c1c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.354+0000 7f47a63feb80 1 -- 172.21.15.32:0/2112021625 >> 172.21.15.32:0/2112021625 conn(0x55a473b8f200 msgr2=0x55a473c15ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.354+0000 7f47a63feb80 1 -- 172.21.15.32:0/2112021625 shutdown_connections 2023-09-30T14:55:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.354+0000 7f47a63feb80 1 -- 172.21.15.32:0/2112021625 wait complete. 2023-09-30T14:55:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.354+0000 7f47a63feb80 1 -- start start 2023-09-30T14:55:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.355+0000 7f47a63feb80 1 Processor -- start 2023-09-30T14:55:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.355+0000 7f47a63feb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1b200 0x55a473c24ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.355+0000 7f47a63feb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1c310 0x55a473c25420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.355+0000 7f47a63feb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1ccd0 0x55a473c2c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.355+0000 7f47a63feb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a473b6ddd0 con 0x55a473c1ccd0 2023-09-30T14:55:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.355+0000 7f47a63feb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a473b0a6b0 con 0x55a473c1b200 2023-09-30T14:55:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.355+0000 7f47a63feb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a473b0a350 con 0x55a473c1c310 2023-09-30T14:55:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.355+0000 7f47a63feb80 10 client.0 _ll_register_callbacks cb 0x55a4739677a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:55:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.355+0000 7f4792ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1c310 0x55a473c25420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.355+0000 7f4792ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1c310 0x55a473c25420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53462/0 (socket says 172.21.15.32:53462) 2023-09-30T14:55:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.355+0000 7f4792ffd700 1 -- 172.21.15.32:0/229614988 learned_addr learned my addr 172.21.15.32:0/229614988 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.355+0000 7f4793fff700 1 --2- 172.21.15.32:0/229614988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1ccd0 0x55a473c2c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.355+0000 7f47937fe700 1 --2- 172.21.15.32:0/229614988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1b200 0x55a473c24ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.356+0000 7f4792ffd700 1 -- 172.21.15.32:0/229614988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1b200 msgr2=0x55a473c24ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.356+0000 7f4792ffd700 1 --2- 172.21.15.32:0/229614988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1b200 0x55a473c24ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.356+0000 7f4792ffd700 1 -- 172.21.15.32:0/229614988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1ccd0 msgr2=0x55a473c2c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.356+0000 7f4792ffd700 1 --2- 172.21.15.32:0/229614988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1ccd0 0x55a473c2c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.356+0000 7f4792ffd700 1 -- 172.21.15.32:0/229614988 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4794102400 con 0x55a473c1c310 2023-09-30T14:55:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.356+0000 7f4792ffd700 1 --2- 172.21.15.32:0/229614988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1c310 0x55a473c25420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f478801e900 tx=0x7f47880047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e81963e7957a17a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.356+0000 7f4772ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:55:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.356+0000 7f4772ffd700 1 -- 172.21.15.32:0/229614988 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478800e840 con 0x55a473c1c310 2023-09-30T14:55:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.356+0000 7f4772ffd700 1 -- 172.21.15.32:0/229614988 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f478800e9a0 con 0x55a473c1c310 2023-09-30T14:55:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.356+0000 7f47a63feb80 1 -- 172.21.15.32:0/229614988 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a473c0e090 con 0x55a473c1c310 2023-09-30T14:55:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.356+0000 7f4772ffd700 1 -- 172.21.15.32:0/229614988 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f478802f030 con 0x55a473c1c310 2023-09-30T14:55:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.356+0000 7f47a63feb80 10 client.26563 fetch_fsmap learned FSMap version 49 2023-09-30T14:55:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.356+0000 7f47a63feb80 1 -- 172.21.15.32:0/229614988 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55a473998d80 con 0x55a473c1c310 2023-09-30T14:55:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.357+0000 7f4772ffd700 1 -- 172.21.15.32:0/229614988 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a473c0e090 con 0x55a473c1c310 2023-09-30T14:55:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.357+0000 7f4772ffd700 1 -- 172.21.15.32:0/229614988 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f478801cc90 con 0x55a473c1c310 2023-09-30T14:55:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.357+0000 7f47a63feb80 10 client.26563 fetch_fsmap finished waiting for FSMap version 49 2023-09-30T14:55:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.357+0000 7f47a63feb80 10 client.26563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:55:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.357+0000 7f47a63feb80 20 client.26563 populate_metadata read hostname 'smithi032' 2023-09-30T14:55:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.357+0000 7f47a63feb80 1 --2- 172.21.15.32:0/229614988 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a473c30e10 0x55a473c33230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.357+0000 7f47a63feb80 4 client.26563 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:55:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.357+0000 7f47a63feb80 1 -- 172.21.15.32:0/229614988 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a473b66db0 con 0x55a473c30e10 2023-09-30T14:55:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.357+0000 7f47937fe700 1 --2- 172.21.15.32:0/229614988 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a473c30e10 0x55a473c33230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.358+0000 7f4772ffd700 10 client.26563 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:55:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.358+0000 7f47937fe700 1 --2- 172.21.15.32:0/229614988 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a473c30e10 0x55a473c33230 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=536d3a9f8293ecc7 server_cookie=6f2d661bde619261 in_seq=0 out_seq=0 2023-09-30T14:55:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.358+0000 7f4772ffd700 1 -- 172.21.15.32:0/229614988 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a473b66db0 con 0x55a473c30e10 2023-09-30T14:55:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.358+0000 7f4772ffd700 10 client.26563 handle_command_reply: tid=0 2023-09-30T14:55:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.358+0000 7f47a63feb80 1 client.26563 shutdown 2023-09-30T14:55:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.359+0000 7f47a63feb80 1 -- 172.21.15.32:0/229614988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1c310 msgr2=0x55a473c25420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.359+0000 7f47a63feb80 1 --2- 172.21.15.32:0/229614988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1c310 0x55a473c25420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f478801e900 tx=0x7f47880047c0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.359+0000 7f47a63feb80 1 -- 172.21.15.32:0/229614988 shutdown_connections 2023-09-30T14:55:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.359+0000 7f47a63feb80 1 --2- 172.21.15.32:0/229614988 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a473c30e10 0x55a473c33230 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.359+0000 7f47a63feb80 1 --2- 172.21.15.32:0/229614988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1ccd0 0x55a473c2c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.359+0000 7f47a63feb80 1 --2- 172.21.15.32:0/229614988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1b200 0x55a473c24ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.359+0000 7f47a63feb80 1 --2- 172.21.15.32:0/229614988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1c310 0x55a473c25420 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.359+0000 7f4772ffd700 0 client.26563 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:55:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.359+0000 7f47a63feb80 1 -- 172.21.15.32:0/229614988 >> 172.21.15.32:0/229614988 conn(0x55a473b8f200 msgr2=0x55a473c15590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.359+0000 7f47a63feb80 1 -- 172.21.15.32:0/229614988 shutdown_connections 2023-09-30T14:55:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.359+0000 7f47a63feb80 1 -- 172.21.15.32:0/229614988 wait complete. 2023-09-30T14:55:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.359+0000 7f47a63feb80 20 client.26563 trim_cache size 0 max 16384 2023-09-30T14:55:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.375+0000 7f47a63feb80 1 Processor -- start 2023-09-30T14:55:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.375+0000 7f47a63feb80 1 -- start start 2023-09-30T14:55:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.375+0000 7f47a63feb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1b200 0x55a473c16cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.375+0000 7f47a63feb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1c310 0x55a473c2c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.375+0000 7f47a63feb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1ccd0 0x55a473c23c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.375+0000 7f47a63feb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4739ee8c0 con 0x55a473c1ccd0 2023-09-30T14:55:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.375+0000 7f47a63feb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a473c24ab0 con 0x55a473c1c310 2023-09-30T14:55:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.375+0000 7f47a63feb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a473c1f790 con 0x55a473c1b200 2023-09-30T14:55:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.375+0000 7f4793fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1ccd0 0x55a473c23c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.375+0000 7f4793fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1ccd0 0x55a473c23c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39090/0 (socket says 172.21.15.32:39090) 2023-09-30T14:55:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.375+0000 7f4793fff700 1 -- 172.21.15.32:0/2647725727 learned_addr learned my addr 172.21.15.32:0/2647725727 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.375+0000 7f47937fe700 1 --2- 172.21.15.32:0/2647725727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1b200 0x55a473c16cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.375+0000 7f4792ffd700 1 --2- 172.21.15.32:0/2647725727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1c310 0x55a473c2c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.376+0000 7f4793fff700 1 -- 172.21.15.32:0/2647725727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1b200 msgr2=0x55a473c16cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.376+0000 7f4793fff700 1 --2- 172.21.15.32:0/2647725727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1b200 0x55a473c16cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.376+0000 7f4793fff700 1 -- 172.21.15.32:0/2647725727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1c310 msgr2=0x55a473c2c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.376+0000 7f4793fff700 1 --2- 172.21.15.32:0/2647725727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1c310 0x55a473c2c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.376+0000 7f4793fff700 1 -- 172.21.15.32:0/2647725727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a473981c00 con 0x55a473c1ccd0 2023-09-30T14:55:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.376+0000 7f47937fe700 1 --2- 172.21.15.32:0/2647725727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1b200 0x55a473c16cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:55:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.376+0000 7f4793fff700 1 --2- 172.21.15.32:0/2647725727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1ccd0 0x55a473c23c10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f478c015000 tx=0x7f478c005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267e5b43c93a96cb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.376+0000 7f477a7fc700 1 -- 172.21.15.32:0/2647725727 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c00e040 con 0x55a473c1ccd0 2023-09-30T14:55:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.376+0000 7f477a7fc700 1 -- 172.21.15.32:0/2647725727 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f478c00baa0 con 0x55a473c1ccd0 2023-09-30T14:55:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.376+0000 7f477a7fc700 1 -- 172.21.15.32:0/2647725727 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c005110 con 0x55a473c1ccd0 2023-09-30T14:55:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.377+0000 7f47a63feb80 1 -- 172.21.15.32:0/2647725727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1ccd0 msgr2=0x55a473c23c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.377+0000 7f47a63feb80 1 --2- 172.21.15.32:0/2647725727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1ccd0 0x55a473c23c10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f478c015000 tx=0x7f478c005b20 comp rx=0 tx=0).stop 2023-09-30T14:55:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.377+0000 7f47a63feb80 1 -- 172.21.15.32:0/2647725727 shutdown_connections 2023-09-30T14:55:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.377+0000 7f47a63feb80 1 --2- 172.21.15.32:0/2647725727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1ccd0 0x55a473c23c10 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.377+0000 7f47a63feb80 1 --2- 172.21.15.32:0/2647725727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1b200 0x55a473c16cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.377+0000 7f47a63feb80 1 --2- 172.21.15.32:0/2647725727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1c310 0x55a473c2c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.377+0000 7f47a63feb80 1 -- 172.21.15.32:0/2647725727 >> 172.21.15.32:0/2647725727 conn(0x55a473b8f200 msgr2=0x55a473c15590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.377+0000 7f47a63feb80 1 -- 172.21.15.32:0/2647725727 shutdown_connections 2023-09-30T14:55:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.377+0000 7f47a63feb80 1 -- 172.21.15.32:0/2647725727 wait complete. 2023-09-30T14:55:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.377+0000 7f47a63feb80 1 -- start start 2023-09-30T14:55:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f47a63feb80 1 Processor -- start 2023-09-30T14:55:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f47a63feb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1b200 0x55a473c2a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f47a63feb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1c310 0x55a473c2a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f47a63feb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1ccd0 0x55a473c2b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f47a63feb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a473c2bcd0 con 0x55a473c1c310 2023-09-30T14:55:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f47a63feb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a473c2be40 con 0x55a473c1ccd0 2023-09-30T14:55:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f47a63feb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a473c2c000 con 0x55a473c1b200 2023-09-30T14:55:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f47a63feb80 10 client.0 _ll_register_callbacks cb 0x7f478801e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:55:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f4792ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1c310 0x55a473c2a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f4792ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1c310 0x55a473c2a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39092/0 (socket says 172.21.15.32:39092) 2023-09-30T14:55:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f4792ffd700 1 -- 172.21.15.32:0/1551030896 learned_addr learned my addr 172.21.15.32:0/1551030896 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f4793fff700 1 --2- 172.21.15.32:0/1551030896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1ccd0 0x55a473c2b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f47937fe700 1 --2- 172.21.15.32:0/1551030896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1b200 0x55a473c2a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f4792ffd700 1 -- 172.21.15.32:0/1551030896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1b200 msgr2=0x55a473c2a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f4792ffd700 1 --2- 172.21.15.32:0/1551030896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1b200 0x55a473c2a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f4792ffd700 1 -- 172.21.15.32:0/1551030896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1ccd0 msgr2=0x55a473c2b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f4792ffd700 1 --2- 172.21.15.32:0/1551030896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1ccd0 0x55a473c2b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f4792ffd700 1 -- 172.21.15.32:0/1551030896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47840b6040 con 0x55a473c1c310 2023-09-30T14:55:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.378+0000 7f47937fe700 1 --2- 172.21.15.32:0/1551030896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1b200 0x55a473c2a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:55:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.379+0000 7f4792ffd700 1 --2- 172.21.15.32:0/1551030896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1c310 0x55a473c2a780 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f478801b410 tx=0x7f478800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f4afb9d8a752f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.379+0000 7f4772ffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:55:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.379+0000 7f4772ffd700 1 -- 172.21.15.32:0/1551030896 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478802f040 con 0x55a473c1c310 2023-09-30T14:55:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.379+0000 7f4772ffd700 1 -- 172.21.15.32:0/1551030896 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4788030070 con 0x55a473c1c310 2023-09-30T14:55:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.379+0000 7f47a63feb80 1 -- 172.21.15.32:0/1551030896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f478409d070 con 0x55a473c1c310 2023-09-30T14:55:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.379+0000 7f4772ffd700 1 -- 172.21.15.32:0/1551030896 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4788044030 con 0x55a473c1c310 2023-09-30T14:55:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.379+0000 7f47a63feb80 10 client.16908 fetch_fsmap learned FSMap version 49 2023-09-30T14:55:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.379+0000 7f47a63feb80 1 -- 172.21.15.32:0/1551030896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f4794108b40 con 0x55a473c1c310 2023-09-30T14:55:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.379+0000 7f4772ffd700 1 -- 172.21.15.32:0/1551030896 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f478409d070 con 0x55a473c1c310 2023-09-30T14:55:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.379+0000 7f4772ffd700 1 -- 172.21.15.32:0/1551030896 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4788003680 con 0x55a473c1c310 2023-09-30T14:55:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.379+0000 7f47a63feb80 10 client.16908 fetch_fsmap finished waiting for FSMap version 49 2023-09-30T14:55:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.379+0000 7f47a63feb80 10 client.16908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:55:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.379+0000 7f47a63feb80 20 client.16908 populate_metadata read hostname 'smithi032' 2023-09-30T14:55:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.380+0000 7f47a63feb80 1 --2- 172.21.15.32:0/1551030896 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a473c85b60 0x55a473c87f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.380+0000 7f47a63feb80 4 client.16908 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:55:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.380+0000 7f47a63feb80 1 -- 172.21.15.32:0/1551030896 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a473b66ad0 con 0x55a473c85b60 2023-09-30T14:55:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.380+0000 7f47937fe700 1 --2- 172.21.15.32:0/1551030896 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a473c85b60 0x55a473c87f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.380+0000 7f47937fe700 1 --2- 172.21.15.32:0/1551030896 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a473c85b60 0x55a473c87f80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c98a0509627cb077 server_cookie=1e7237b53f87628 in_seq=0 out_seq=0 2023-09-30T14:55:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.380+0000 7f4772ffd700 10 client.16908 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:55:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.679+0000 7f4772ffd700 1 -- 172.21.15.32:0/1551030896 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a473b66ad0 con 0x55a473c85b60 2023-09-30T14:55:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.679+0000 7f4772ffd700 10 client.16908 handle_command_reply: tid=0 2023-09-30T14:55:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.679+0000 7f47a63feb80 1 client.16908 shutdown 2023-09-30T14:55:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.679+0000 7f47a63feb80 1 -- 172.21.15.32:0/1551030896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1c310 msgr2=0x55a473c2a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.679+0000 7f47a63feb80 1 --2- 172.21.15.32:0/1551030896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1c310 0x55a473c2a780 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f478801b410 tx=0x7f478800eda0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.679+0000 7f47a63feb80 1 -- 172.21.15.32:0/1551030896 shutdown_connections 2023-09-30T14:55:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.679+0000 7f47a63feb80 1 --2- 172.21.15.32:0/1551030896 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a473c85b60 0x55a473c87f80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.679+0000 7f47a63feb80 1 --2- 172.21.15.32:0/1551030896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a473c1ccd0 0x55a473c2b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.679+0000 7f4772ffd700 0 client.16908 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:55:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.679+0000 7f47a63feb80 1 --2- 172.21.15.32:0/1551030896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a473c1b200 0x55a473c2a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.679+0000 7f47a63feb80 1 --2- 172.21.15.32:0/1551030896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a473c1c310 0x55a473c2a780 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.679+0000 7f47a63feb80 1 -- 172.21.15.32:0/1551030896 >> 172.21.15.32:0/1551030896 conn(0x55a473b8f200 msgr2=0x55a473c16bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.679+0000 7f47a63feb80 1 -- 172.21.15.32:0/1551030896 shutdown_connections 2023-09-30T14:55:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.679+0000 7f47a63feb80 1 -- 172.21.15.32:0/1551030896 wait complete. 2023-09-30T14:55:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.680+0000 7f47a63feb80 20 client.16908 trim_cache size 0 max 16384 2023-09-30T14:55:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:55:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:55:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:55:33.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:55:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.680+0000 7f479a216700 1 -- 172.21.15.32:0/2816219470 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f477c0a0fa0 msgr2=0x7f477c0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.680+0000 7f479a216700 1 --2- 172.21.15.32:0/2816219470 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f477c0a0fa0 0x7f477c0a3460 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4794196b10 tx=0x7f4788009260 comp rx=0 tx=0).stop 2023-09-30T14:55:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.680+0000 7f479a216700 1 -- 172.21.15.32:0/2816219470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47940ffdb0 msgr2=0x7f4794196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.680+0000 7f479a216700 1 --2- 172.21.15.32:0/2816219470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47940ffdb0 0x7f4794196da0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f478400e970 tx=0x7f478400ec80 comp rx=0 tx=0).stop 2023-09-30T14:55:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.682+0000 7f479a216700 1 -- 172.21.15.32:0/2816219470 shutdown_connections 2023-09-30T14:55:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.682+0000 7f479a216700 1 --2- 172.21.15.32:0/2816219470 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f477c0a0fa0 0x7f477c0a3460 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.682+0000 7f479a216700 1 --2- 172.21.15.32:0/2816219470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4794101940 0x7f47941980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.683 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.682+0000 7f479a216700 1 --2- 172.21.15.32:0/2816219470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47940ffdb0 0x7f4794196da0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.682+0000 7f479a216700 1 --2- 172.21.15.32:0/2816219470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4794100fa0 0x7f47941972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.682+0000 7f479a216700 1 -- 172.21.15.32:0/2816219470 >> 172.21.15.32:0/2816219470 conn(0x7f47940fb320 msgr2=0x7f4794068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.682+0000 7f479a216700 1 -- 172.21.15.32:0/2816219470 shutdown_connections 2023-09-30T14:55:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:33.682+0000 7f479a216700 1 -- 172.21.15.32:0/2816219470 wait complete. 2023-09-30T14:55:33.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:33 smithi148 ceph-mon[102355]: pgmap v805: 97 pgs: 97 active+clean; 701 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.2 MiB/s wr, 724 op/s 2023-09-30T14:55:33.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:33 smithi032 ceph-mon[98592]: pgmap v805: 97 pgs: 97 active+clean; 701 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.2 MiB/s wr, 724 op/s 2023-09-30T14:55:34.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 165847 37.11 MB/sec execute 127 sec latency 235.651 ms 2023-09-30T14:55:34.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:33 smithi156 ceph-mon[103579]: pgmap v805: 97 pgs: 97 active+clean; 701 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.2 MiB/s wr, 724 op/s 2023-09-30T14:55:35.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 166382 37.04 MB/sec execute 128 sec latency 230.177 ms 2023-09-30T14:55:35.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:35 smithi148 ceph-mon[102355]: pgmap v806: 97 pgs: 97 active+clean; 709 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.7 MiB/s wr, 765 op/s 2023-09-30T14:55:35.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:35 smithi032 ceph-mon[98592]: pgmap v806: 97 pgs: 97 active+clean; 709 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.7 MiB/s wr, 765 op/s 2023-09-30T14:55:36.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 167096 37.11 MB/sec execute 129 sec latency 287.228 ms 2023-09-30T14:55:36.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:35 smithi156 ceph-mon[103579]: pgmap v806: 97 pgs: 97 active+clean; 709 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.7 MiB/s wr, 765 op/s 2023-09-30T14:55:37.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 167722 37.07 MB/sec execute 130 sec latency 78.177 ms 2023-09-30T14:55:37.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:37 smithi148 ceph-mon[102355]: pgmap v807: 97 pgs: 97 active+clean; 716 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 768 op/s 2023-09-30T14:55:37.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:37 smithi032 ceph-mon[98592]: pgmap v807: 97 pgs: 97 active+clean; 716 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 768 op/s 2023-09-30T14:55:38.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 168427 37.02 MB/sec execute 131 sec latency 48.882 ms 2023-09-30T14:55:38.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:37 smithi156 ceph-mon[103579]: pgmap v807: 97 pgs: 97 active+clean; 716 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 768 op/s 2023-09-30T14:55:39.014 INFO:tasks.workunit.client.0.smithi032.stdout: 10 169216 37.06 MB/sec execute 132 sec latency 56.287 ms 2023-09-30T14:55:39.862 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:39 smithi148 ceph-mon[102355]: pgmap v808: 97 pgs: 97 active+clean; 711 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.6 MiB/s wr, 783 op/s 2023-09-30T14:55:39.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:39 smithi032 ceph-mon[98592]: pgmap v808: 97 pgs: 97 active+clean; 711 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.6 MiB/s wr, 783 op/s 2023-09-30T14:55:40.018 INFO:tasks.workunit.client.0.smithi032.stdout: 10 169933 37.09 MB/sec execute 133 sec latency 202.942 ms 2023-09-30T14:55:40.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:39 smithi156 ceph-mon[103579]: pgmap v808: 97 pgs: 97 active+clean; 711 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.6 MiB/s wr, 783 op/s 2023-09-30T14:55:41.017 INFO:tasks.workunit.client.0.smithi032.stdout: 10 170625 37.10 MB/sec execute 134 sec latency 309.793 ms 2023-09-30T14:55:41.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:41 smithi148 ceph-mon[102355]: pgmap v809: 97 pgs: 97 active+clean; 716 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.6 MiB/s wr, 746 op/s 2023-09-30T14:55:41.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:41 smithi032 ceph-mon[98592]: pgmap v809: 97 pgs: 97 active+clean; 716 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.6 MiB/s wr, 746 op/s 2023-09-30T14:55:42.016 INFO:tasks.workunit.client.0.smithi032.stdout: 10 171350 37.16 MB/sec execute 135 sec latency 74.693 ms 2023-09-30T14:55:42.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:41 smithi156 ceph-mon[103579]: pgmap v809: 97 pgs: 97 active+clean; 716 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.6 MiB/s wr, 746 op/s 2023-09-30T14:55:42.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:55:42.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:55:42.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:55:42.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:42 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:55:42.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:55:42.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:55:42.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:55:42.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:42 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:55:43.017 INFO:tasks.workunit.client.0.smithi032.stdout: 10 172053 37.22 MB/sec execute 136 sec latency 99.195 ms 2023-09-30T14:55:43.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:55:43.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:55:43.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:55:43.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:42 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:55:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:43 smithi148 ceph-mon[102355]: pgmap v810: 97 pgs: 97 active+clean; 719 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 795 op/s 2023-09-30T14:55:43.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:43 smithi032 ceph-mon[98592]: pgmap v810: 97 pgs: 97 active+clean; 719 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 795 op/s 2023-09-30T14:55:44.017 INFO:tasks.workunit.client.0.smithi032.stdout: 10 172563 37.06 MB/sec execute 137 sec latency 64.247 ms 2023-09-30T14:55:44.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:43 smithi156 ceph-mon[103579]: pgmap v810: 97 pgs: 97 active+clean; 719 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 795 op/s 2023-09-30T14:55:45.017 INFO:tasks.workunit.client.0.smithi032.stdout: 10 173170 36.99 MB/sec execute 138 sec latency 152.504 ms 2023-09-30T14:55:45.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:45 smithi148 ceph-mon[102355]: pgmap v811: 97 pgs: 97 active+clean; 725 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 10 MiB/s wr, 849 op/s 2023-09-30T14:55:45.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:45 smithi032 ceph-mon[98592]: pgmap v811: 97 pgs: 97 active+clean; 725 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 10 MiB/s wr, 849 op/s 2023-09-30T14:55:46.017 INFO:tasks.workunit.client.0.smithi032.stdout: 10 173934 37.00 MB/sec execute 139 sec latency 196.479 ms 2023-09-30T14:55:46.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:45 smithi156 ceph-mon[103579]: pgmap v811: 97 pgs: 97 active+clean; 725 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 10 MiB/s wr, 849 op/s 2023-09-30T14:55:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.881+0000 7f8a4dfa3700 1 -- 172.21.15.32:0/3031505165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a4810d160 msgr2=0x7f8a4810d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.881+0000 7f8a4dfa3700 1 --2- 172.21.15.32:0/3031505165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a4810d160 0x7f8a4810d5e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000b0a0 tx=0x7f8a4000b3b0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.883+0000 7f8a4dfa3700 1 -- 172.21.15.32:0/3031505165 shutdown_connections 2023-09-30T14:55:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.883+0000 7f8a4dfa3700 1 --2- 172.21.15.32:0/3031505165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a4810d160 0x7f8a4810d5e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.883+0000 7f8a4dfa3700 1 --2- 172.21.15.32:0/3031505165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a48107e20 0x7f8a4810a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.883+0000 7f8a4dfa3700 1 --2- 172.21.15.32:0/3031505165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4810a790 0x7f8a4810cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.883+0000 7f8a4dfa3700 1 -- 172.21.15.32:0/3031505165 >> 172.21.15.32:0/3031505165 conn(0x7f8a4806dc80 msgr2=0x7f8a480700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.883+0000 7f8a4dfa3700 1 -- 172.21.15.32:0/3031505165 shutdown_connections 2023-09-30T14:55:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.883+0000 7f8a4dfa3700 1 -- 172.21.15.32:0/3031505165 wait complete. 2023-09-30T14:55:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.884+0000 7f8a4dfa3700 1 Processor -- start 2023-09-30T14:55:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.884+0000 7f8a4dfa3700 1 -- start start 2023-09-30T14:55:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.885+0000 7f8a4dfa3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a48107e20 0x7f8a48118450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.885+0000 7f8a4dfa3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4810a790 0x7f8a48118990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.885+0000 7f8a4dfa3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a4810d160 0x7f8a481b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.885+0000 7f8a4dfa3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a481b6000 con 0x7f8a48107e20 2023-09-30T14:55:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.885+0000 7f8a4dfa3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a481b6140 con 0x7f8a4810d160 2023-09-30T14:55:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.885+0000 7f8a4dfa3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a481b6280 con 0x7f8a4810a790 2023-09-30T14:55:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.885+0000 7f8a477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a48107e20 0x7f8a48118450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.885+0000 7f8a3effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4810a790 0x7f8a48118990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.885+0000 7f8a477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a48107e20 0x7f8a48118450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39106/0 (socket says 172.21.15.32:39106) 2023-09-30T14:55:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.885+0000 7f8a477fe700 1 -- 172.21.15.32:0/2075035433 learned_addr learned my addr 172.21.15.32:0/2075035433 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.885+0000 7f8a47fff700 1 --2- 172.21.15.32:0/2075035433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a4810d160 0x7f8a481b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.885+0000 7f8a477fe700 1 -- 172.21.15.32:0/2075035433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4810a790 msgr2=0x7f8a48118990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.885+0000 7f8a477fe700 1 --2- 172.21.15.32:0/2075035433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4810a790 0x7f8a48118990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.885+0000 7f8a477fe700 1 -- 172.21.15.32:0/2075035433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a4810d160 msgr2=0x7f8a481b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.885+0000 7f8a477fe700 1 --2- 172.21.15.32:0/2075035433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a4810d160 0x7f8a481b5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.885+0000 7f8a477fe700 1 -- 172.21.15.32:0/2075035433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a40009d00 con 0x7f8a48107e20 2023-09-30T14:55:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.886+0000 7f8a477fe700 1 --2- 172.21.15.32:0/2075035433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a48107e20 0x7f8a48118450 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800b530 tx=0x7f8a3800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.886+0000 7f8a457fa700 1 -- 172.21.15.32:0/2075035433 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a38011830 con 0x7f8a48107e20 2023-09-30T14:55:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.886+0000 7f8a457fa700 1 -- 172.21.15.32:0/2075035433 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a38011e50 con 0x7f8a48107e20 2023-09-30T14:55:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.886+0000 7f8a4dfa3700 1 -- 172.21.15.32:0/2075035433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a481b64e0 con 0x7f8a48107e20 2023-09-30T14:55:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.886+0000 7f8a457fa700 1 -- 172.21.15.32:0/2075035433 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3800f2e0 con 0x7f8a48107e20 2023-09-30T14:55:46.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.886+0000 7f8a4dfa3700 1 -- 172.21.15.32:0/2075035433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a481b6ac0 con 0x7f8a48107e20 2023-09-30T14:55:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.890+0000 7f8a457fa700 1 -- 172.21.15.32:0/2075035433 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f8a3800f440 con 0x7f8a48107e20 2023-09-30T14:55:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.892+0000 7f8a5a18bb80 1 Processor -- start 2023-09-30T14:55:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.892+0000 7f8a457fa700 1 --2- 172.21.15.32:0/2075035433 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a340a1070 0x7f8a340a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.892+0000 7f8a5a18bb80 1 -- start start 2023-09-30T14:55:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.892+0000 7f8a5a18bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71ce200 0x558ad71ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.892+0000 7f8a5a18bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71cf310 0x558ad71cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.892+0000 7f8a457fa700 1 -- 172.21.15.32:0/2075035433 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8a380cc140 con 0x7f8a48107e20 2023-09-30T14:55:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.892+0000 7f8a5a18bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cfcd0 0x558ad71d6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.892+0000 7f8a5a18bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad7136400 con 0x558ad71cfcd0 2023-09-30T14:55:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.892+0000 7f8a5a18bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad7127bc0 con 0x558ad71ce200 2023-09-30T14:55:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.892+0000 7f8a5a18bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad71254e0 con 0x558ad71cf310 2023-09-30T14:55:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.892+0000 7f8a47fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cfcd0 0x558ad71d6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.892+0000 7f8a47fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cfcd0 0x558ad71d6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39114/0 (socket says 172.21.15.32:39114) 2023-09-30T14:55:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.892+0000 7f8a47fff700 1 -- 172.21.15.32:0/3022137327 learned_addr learned my addr 172.21.15.32:0/3022137327 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.893+0000 7f8a477fe700 1 --2- 172.21.15.32:0/3022137327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71ce200 0x558ad71ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.893+0000 7f8a3effd700 1 --2- 172.21.15.32:0/3022137327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71cf310 0x558ad71cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.893+0000 7f8a47fff700 1 --2- 172.21.15.32:0/2075035433 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a340a1070 0x7f8a340a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.893+0000 7f8a47fff700 1 -- 172.21.15.32:0/3022137327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71cf310 msgr2=0x558ad71cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.893+0000 7f8a47fff700 1 --2- 172.21.15.32:0/3022137327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71cf310 0x558ad71cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.893+0000 7f8a47fff700 1 -- 172.21.15.32:0/3022137327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71ce200 msgr2=0x558ad71ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:46.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.893+0000 7f8a47fff700 1 --2- 172.21.15.32:0/3022137327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71ce200 0x558ad71ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.893+0000 7f8a47fff700 1 -- 172.21.15.32:0/3022137327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ad7183870 con 0x558ad71cfcd0 2023-09-30T14:55:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.893+0000 7f8a477fe700 1 --2- 172.21.15.32:0/3022137327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71ce200 0x558ad71ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:55:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.893+0000 7f8a47fff700 1 --2- 172.21.15.32:0/3022137327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cfcd0 0x558ad71d6560 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000b780 tx=0x7f8a40004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b4805e0fe09ed1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.893+0000 7f8a47fff700 1 --2- 172.21.15.32:0/2075035433 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a340a1070 0x7f8a340a3530 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8a40015000 tx=0x7f8a40019000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.894+0000 7f8a3dffb700 1 -- 172.21.15.32:0/3022137327 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4000e040 con 0x558ad71cfcd0 2023-09-30T14:55:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.894+0000 7f8a3dffb700 1 -- 172.21.15.32:0/3022137327 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a4000bd10 con 0x558ad71cfcd0 2023-09-30T14:55:46.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.894+0000 7f8a3dffb700 1 -- 172.21.15.32:0/3022137327 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a40025070 con 0x558ad71cfcd0 2023-09-30T14:55:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.895+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/3022137327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cfcd0 msgr2=0x558ad71d6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.895+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/3022137327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cfcd0 0x558ad71d6560 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000b780 tx=0x7f8a40004a50 comp rx=0 tx=0).stop 2023-09-30T14:55:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.895+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/3022137327 shutdown_connections 2023-09-30T14:55:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.895+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/3022137327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cfcd0 0x558ad71d6560 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.895+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/3022137327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71ce200 0x558ad71ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.895+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/3022137327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71cf310 0x558ad71cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.895+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/3022137327 >> 172.21.15.32:0/3022137327 conn(0x558ad7142200 msgr2=0x558ad71c8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:46.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.895+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/3022137327 shutdown_connections 2023-09-30T14:55:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.895+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/3022137327 wait complete. 2023-09-30T14:55:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.896+0000 7f8a5a18bb80 1 -- start start 2023-09-30T14:55:46.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.896+0000 7f8a5a18bb80 1 Processor -- start 2023-09-30T14:55:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a5a18bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71ce200 0x558ad71d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a5a18bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cf310 0x558ad71d84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a5a18bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cfcd0 0x558ad71df500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a5a18bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad7120dd0 con 0x558ad71cfcd0 2023-09-30T14:55:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a5a18bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad70bd6b0 con 0x558ad71cf310 2023-09-30T14:55:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a5a18bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad70bd350 con 0x558ad71ce200 2023-09-30T14:55:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a5a18bb80 10 client.0 _ll_register_callbacks cb 0x558ad6f1a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:55:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a47fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cfcd0 0x558ad71df500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a47fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cfcd0 0x558ad71df500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39122/0 (socket says 172.21.15.32:39122) 2023-09-30T14:55:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a47fff700 1 -- 172.21.15.32:0/229774268 learned_addr learned my addr 172.21.15.32:0/229774268 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a477fe700 1 --2- 172.21.15.32:0/229774268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71ce200 0x558ad71d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a3effd700 1 --2- 172.21.15.32:0/229774268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cf310 0x558ad71d84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a47fff700 1 -- 172.21.15.32:0/229774268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71ce200 msgr2=0x558ad71d7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a47fff700 1 --2- 172.21.15.32:0/229774268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71ce200 0x558ad71d7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a47fff700 1 -- 172.21.15.32:0/229774268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cf310 msgr2=0x558ad71d84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a47fff700 1 --2- 172.21.15.32:0/229774268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cf310 0x558ad71d84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a47fff700 1 -- 172.21.15.32:0/229774268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a481b6ac0 con 0x558ad71cfcd0 2023-09-30T14:55:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a3effd700 1 --2- 172.21.15.32:0/229774268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cf310 0x558ad71d84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:55:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a47fff700 1 --2- 172.21.15.32:0/229774268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cfcd0 0x558ad71df500 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8a40006100 tx=0x7f8a40009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f056dbb3eba12c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.897+0000 7f8a26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:55:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.898+0000 7f8a26ffd700 1 -- 172.21.15.32:0/229774268 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a40037070 con 0x558ad71cfcd0 2023-09-30T14:55:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.898+0000 7f8a26ffd700 1 -- 172.21.15.32:0/229774268 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8a4001f650 con 0x558ad71cfcd0 2023-09-30T14:55:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.898+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/229774268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ad71c1090 con 0x558ad71cfcd0 2023-09-30T14:55:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.898+0000 7f8a26ffd700 1 -- 172.21.15.32:0/229774268 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a40025070 con 0x558ad71cfcd0 2023-09-30T14:55:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.898+0000 7f8a5a18bb80 10 client.16920 fetch_fsmap learned FSMap version 49 2023-09-30T14:55:46.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.898+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/229774268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f8a4810e200 con 0x558ad71cfcd0 2023-09-30T14:55:46.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.898+0000 7f8a26ffd700 1 -- 172.21.15.32:0/229774268 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558ad71c1090 con 0x558ad71cfcd0 2023-09-30T14:55:46.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.899+0000 7f8a26ffd700 1 -- 172.21.15.32:0/229774268 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8a4003a030 con 0x558ad71cfcd0 2023-09-30T14:55:46.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.899+0000 7f8a5a18bb80 10 client.16920 fetch_fsmap finished waiting for FSMap version 49 2023-09-30T14:55:46.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.899+0000 7f8a5a18bb80 10 client.16920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:55:46.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.899+0000 7f8a5a18bb80 20 client.16920 populate_metadata read hostname 'smithi032' 2023-09-30T14:55:46.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.899+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/229774268 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558ad71e3aa0 0x558ad71e5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.899+0000 7f8a5a18bb80 4 client.16920 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:55:46.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.899+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/229774268 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ad7119db0 con 0x558ad71e3aa0 2023-09-30T14:55:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.902+0000 7f8a477fe700 1 --2- 172.21.15.32:0/229774268 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558ad71e3aa0 0x558ad71e5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.902+0000 7f8a26ffd700 10 client.16920 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:55:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.902+0000 7f8a477fe700 1 --2- 172.21.15.32:0/229774268 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558ad71e3aa0 0x558ad71e5ec0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f844f7db0ed8af0 server_cookie=177ca352119a8572 in_seq=0 out_seq=0 2023-09-30T14:55:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.907+0000 7f8a26ffd700 1 -- 172.21.15.32:0/229774268 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558ad7119db0 con 0x558ad71e3aa0 2023-09-30T14:55:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.907+0000 7f8a26ffd700 10 client.16920 handle_command_reply: tid=0 2023-09-30T14:55:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.907+0000 7f8a5a18bb80 1 client.16920 shutdown 2023-09-30T14:55:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.907+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/229774268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cfcd0 msgr2=0x558ad71df500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.907+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/229774268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cfcd0 0x558ad71df500 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8a40006100 tx=0x7f8a40009800 comp rx=0 tx=0).stop 2023-09-30T14:55:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.907+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/229774268 shutdown_connections 2023-09-30T14:55:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.907+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/229774268 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558ad71e3aa0 0x558ad71e5ec0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.907+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/229774268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cfcd0 0x558ad71df500 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.907+0000 7f8a26ffd700 0 client.16920 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:55:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.907+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/229774268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71ce200 0x558ad71d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.907+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/229774268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cf310 0x558ad71d84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.907+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/229774268 >> 172.21.15.32:0/229774268 conn(0x558ad7142200 msgr2=0x558ad71c8590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.908+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/229774268 shutdown_connections 2023-09-30T14:55:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.908+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/229774268 wait complete. 2023-09-30T14:55:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.908+0000 7f8a5a18bb80 20 client.16920 trim_cache size 0 max 16384 2023-09-30T14:55:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.926+0000 7f8a5a18bb80 1 Processor -- start 2023-09-30T14:55:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.926+0000 7f8a5a18bb80 1 -- start start 2023-09-30T14:55:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a5a18bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71ce200 0x558ad71d7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a5a18bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cf310 0x558ad71df4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a5a18bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cfcd0 0x558ad71d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a5a18bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad6fa18c0 con 0x558ad71cf310 2023-09-30T14:55:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a5a18bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad71ca9c0 con 0x558ad71cfcd0 2023-09-30T14:55:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a5a18bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad71c9e40 con 0x558ad71ce200 2023-09-30T14:55:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a3effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cf310 0x558ad71df4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a3effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cf310 0x558ad71df4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39130/0 (socket says 172.21.15.32:39130) 2023-09-30T14:55:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a3effd700 1 -- 172.21.15.32:0/2203836417 learned_addr learned my addr 172.21.15.32:0/2203836417 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a477fe700 1 --2- 172.21.15.32:0/2203836417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71ce200 0x558ad71d7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a47fff700 1 --2- 172.21.15.32:0/2203836417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cfcd0 0x558ad71d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a3effd700 1 -- 172.21.15.32:0/2203836417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71ce200 msgr2=0x558ad71d7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a3effd700 1 --2- 172.21.15.32:0/2203836417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71ce200 0x558ad71d7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a3effd700 1 -- 172.21.15.32:0/2203836417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cfcd0 msgr2=0x558ad71d7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a3effd700 1 --2- 172.21.15.32:0/2203836417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cfcd0 0x558ad71d7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a3effd700 1 -- 172.21.15.32:0/2203836417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ad6f4bd80 con 0x558ad71cf310 2023-09-30T14:55:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a3effd700 1 --2- 172.21.15.32:0/2203836417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cf310 0x558ad71df4c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8a300045b0 tx=0x7f8a30009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77660d01bc53580 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a3e7fc700 1 -- 172.21.15.32:0/2203836417 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a30017070 con 0x558ad71cf310 2023-09-30T14:55:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a3e7fc700 1 -- 172.21.15.32:0/2203836417 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a30003c00 con 0x558ad71cf310 2023-09-30T14:55:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.927+0000 7f8a3e7fc700 1 -- 172.21.15.32:0/2203836417 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a30010c40 con 0x558ad71cf310 2023-09-30T14:55:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.928+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/2203836417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cf310 msgr2=0x558ad71df4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.928+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/2203836417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cf310 0x558ad71df4c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8a300045b0 tx=0x7f8a30009bc0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.928+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/2203836417 shutdown_connections 2023-09-30T14:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.928+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/2203836417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cfcd0 0x558ad71d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.928+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/2203836417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71ce200 0x558ad71d7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.928+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/2203836417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71cf310 0x558ad71df4c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.928+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/2203836417 >> 172.21.15.32:0/2203836417 conn(0x558ad7142200 msgr2=0x558ad71d6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.928+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/2203836417 shutdown_connections 2023-09-30T14:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.928+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/2203836417 wait complete. 2023-09-30T14:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.928+0000 7f8a5a18bb80 1 -- start start 2023-09-30T14:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.929+0000 7f8a5a18bb80 1 Processor -- start 2023-09-30T14:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.929+0000 7f8a5a18bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71ce200 0x558ad71dd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.929+0000 7f8a5a18bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cf310 0x558ad71dd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.929+0000 7f8a5a18bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71cfcd0 0x558ad71de600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.929+0000 7f8a5a18bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad71deb40 con 0x558ad71ce200 2023-09-30T14:55:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.929+0000 7f8a5a18bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad71decb0 con 0x558ad71cf310 2023-09-30T14:55:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.929+0000 7f8a5a18bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad71dee70 con 0x558ad71cfcd0 2023-09-30T14:55:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.929+0000 7f8a5a18bb80 10 client.0 _ll_register_callbacks cb 0x7f8a38095950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:55:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.929+0000 7f8a477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71ce200 0x558ad71dd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.929+0000 7f8a477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71ce200 0x558ad71dd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39134/0 (socket says 172.21.15.32:39134) 2023-09-30T14:55:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.929+0000 7f8a477fe700 1 -- 172.21.15.32:0/1429400844 learned_addr learned my addr 172.21.15.32:0/1429400844 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:55:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.929+0000 7f8a3effd700 1 --2- 172.21.15.32:0/1429400844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cf310 0x558ad71dd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.929+0000 7f8a47fff700 1 --2- 172.21.15.32:0/1429400844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71cfcd0 0x558ad71de600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.930+0000 7f8a3effd700 1 -- 172.21.15.32:0/1429400844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71cfcd0 msgr2=0x558ad71de600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.930+0000 7f8a3effd700 1 --2- 172.21.15.32:0/1429400844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71cfcd0 0x558ad71de600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.930+0000 7f8a3effd700 1 -- 172.21.15.32:0/1429400844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71ce200 msgr2=0x558ad71dd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:55:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.930+0000 7f8a3effd700 1 --2- 172.21.15.32:0/1429400844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71ce200 0x558ad71dd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.930+0000 7f8a3effd700 1 -- 172.21.15.32:0/1429400844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a30004260 con 0x558ad71cf310 2023-09-30T14:55:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.930+0000 7f8a47fff700 1 --2- 172.21.15.32:0/1429400844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71cfcd0 0x558ad71de600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:55:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.930+0000 7f8a3effd700 1 --2- 172.21.15.32:0/1429400844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cf310 0x558ad71dd5f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x558ad71c10e0 tx=0x7f8a30003b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13e5ef346a05d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:55:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.930+0000 7f8a26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:55:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.930+0000 7f8a26ffd700 1 -- 172.21.15.32:0/1429400844 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a30017040 con 0x558ad71cf310 2023-09-30T14:55:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.930+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/1429400844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a4000bd10 con 0x558ad71cf310 2023-09-30T14:55:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.930+0000 7f8a26ffd700 1 -- 172.21.15.32:0/1429400844 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8a30010b80 con 0x558ad71cf310 2023-09-30T14:55:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.930+0000 7f8a26ffd700 1 -- 172.21.15.32:0/1429400844 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a30010d80 con 0x558ad71cf310 2023-09-30T14:55:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.930+0000 7f8a5a18bb80 10 client.26561 fetch_fsmap learned FSMap version 49 2023-09-30T14:55:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.930+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/1429400844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x558ad6f34c00 con 0x558ad71cf310 2023-09-30T14:55:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.930+0000 7f8a26ffd700 1 -- 172.21.15.32:0/1429400844 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a4000bd10 con 0x558ad71cf310 2023-09-30T14:55:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.931+0000 7f8a26ffd700 1 -- 172.21.15.32:0/1429400844 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8a30024400 con 0x558ad71cf310 2023-09-30T14:55:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.931+0000 7f8a5a18bb80 10 client.26561 fetch_fsmap finished waiting for FSMap version 49 2023-09-30T14:55:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.931+0000 7f8a5a18bb80 10 client.26561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:55:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.931+0000 7f8a5a18bb80 20 client.26561 populate_metadata read hostname 'smithi032' 2023-09-30T14:55:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.931+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/1429400844 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558ad7238560 0x558ad723a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:55:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.931+0000 7f8a5a18bb80 4 client.26561 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:55:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.931+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/1429400844 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558ad7119ad0 con 0x558ad7238560 2023-09-30T14:55:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.931+0000 7f8a477fe700 1 --2- 172.21.15.32:0/1429400844 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558ad7238560 0x558ad723a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:55:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.931+0000 7f8a477fe700 1 --2- 172.21.15.32:0/1429400844 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558ad7238560 0x558ad723a9e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0adf43916458c7d server_cookie=281a8d542dff82b7 in_seq=0 out_seq=0 2023-09-30T14:55:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:46.931+0000 7f8a26ffd700 10 client.26561 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:55:47.029 INFO:tasks.workunit.client.0.smithi032.stdout: 10 174567 36.98 MB/sec execute 140 sec latency 188.920 ms 2023-09-30T14:55:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.285+0000 7f8a26ffd700 1 -- 172.21.15.32:0/1429400844 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558ad7119ad0 con 0x558ad7238560 2023-09-30T14:55:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.285+0000 7f8a26ffd700 10 client.26561 handle_command_reply: tid=0 2023-09-30T14:55:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.286+0000 7f8a5a18bb80 1 client.26561 shutdown 2023-09-30T14:55:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.286+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/1429400844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cf310 msgr2=0x558ad71dd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.286+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/1429400844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cf310 0x558ad71dd5f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x558ad71c10e0 tx=0x7f8a30003b60 comp rx=0 tx=0).stop 2023-09-30T14:55:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.286+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/1429400844 shutdown_connections 2023-09-30T14:55:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.286+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/1429400844 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558ad7238560 0x558ad723a9e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.286+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/1429400844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558ad71cfcd0 0x558ad71de600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.286+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/1429400844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558ad71ce200 0x558ad71dd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.286+0000 7f8a5a18bb80 1 --2- 172.21.15.32:0/1429400844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ad71cf310 0x558ad71dd5f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.286+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/1429400844 >> 172.21.15.32:0/1429400844 conn(0x558ad7142200 msgr2=0x558ad71d7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.286+0000 7f8a26ffd700 0 client.26561 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:55:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.287+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/1429400844 shutdown_connections 2023-09-30T14:55:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.287+0000 7f8a5a18bb80 1 -- 172.21.15.32:0/1429400844 wait complete. 2023-09-30T14:55:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.287+0000 7f8a5a18bb80 20 client.26561 trim_cache size 0 max 16384 2023-09-30T14:55:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:55:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:55:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:55:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:55:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.288+0000 7f8a4dfa3700 1 -- 172.21.15.32:0/2075035433 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a340a1070 msgr2=0x7f8a340a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.288+0000 7f8a4dfa3700 1 --2- 172.21.15.32:0/2075035433 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a340a1070 0x7f8a340a3530 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8a40015000 tx=0x7f8a40019000 comp rx=0 tx=0).stop 2023-09-30T14:55:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.288+0000 7f8a4dfa3700 1 -- 172.21.15.32:0/2075035433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a48107e20 msgr2=0x7f8a48118450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:55:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.288+0000 7f8a4dfa3700 1 --2- 172.21.15.32:0/2075035433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a48107e20 0x7f8a48118450 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800b530 tx=0x7f8a3800b840 comp rx=0 tx=0).stop 2023-09-30T14:55:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.288+0000 7f8a4dfa3700 1 -- 172.21.15.32:0/2075035433 shutdown_connections 2023-09-30T14:55:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.288+0000 7f8a4dfa3700 1 --2- 172.21.15.32:0/2075035433 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a340a1070 0x7f8a340a3530 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.288+0000 7f8a4dfa3700 1 --2- 172.21.15.32:0/2075035433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a4810d160 0x7f8a481b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.288+0000 7f8a4dfa3700 1 --2- 172.21.15.32:0/2075035433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a48107e20 0x7f8a48118450 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.288+0000 7f8a4dfa3700 1 --2- 172.21.15.32:0/2075035433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4810a790 0x7f8a48118990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:55:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.288+0000 7f8a4dfa3700 1 -- 172.21.15.32:0/2075035433 >> 172.21.15.32:0/2075035433 conn(0x7f8a4806dc80 msgr2=0x7f8a481087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:55:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.288+0000 7f8a4dfa3700 1 -- 172.21.15.32:0/2075035433 shutdown_connections 2023-09-30T14:55:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:55:47.288+0000 7f8a4dfa3700 1 -- 172.21.15.32:0/2075035433 wait complete. 2023-09-30T14:55:47.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:47 smithi148 ceph-mon[102355]: pgmap v812: 97 pgs: 97 active+clean; 735 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 832 op/s 2023-09-30T14:55:47.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:47 smithi032 ceph-mon[98592]: pgmap v812: 97 pgs: 97 active+clean; 735 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 832 op/s 2023-09-30T14:55:48.026 INFO:tasks.workunit.client.0.smithi032.stdout: 10 175201 37.04 MB/sec execute 141 sec latency 189.728 ms 2023-09-30T14:55:48.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:47 smithi156 ceph-mon[103579]: pgmap v812: 97 pgs: 97 active+clean; 735 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 832 op/s 2023-09-30T14:55:49.026 INFO:tasks.workunit.client.0.smithi032.stdout: 10 175916 37.06 MB/sec execute 142 sec latency 69.469 ms 2023-09-30T14:55:49.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:49 smithi032 ceph-mon[98592]: pgmap v813: 97 pgs: 97 active+clean; 736 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.6 MiB/s wr, 796 op/s 2023-09-30T14:55:50.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 176534 37.02 MB/sec execute 143 sec latency 74.222 ms 2023-09-30T14:55:50.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:49 smithi156 ceph-mon[103579]: pgmap v813: 97 pgs: 97 active+clean; 736 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.6 MiB/s wr, 796 op/s 2023-09-30T14:55:50.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:49 smithi148 ceph-mon[102355]: pgmap v813: 97 pgs: 97 active+clean; 736 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.6 MiB/s wr, 796 op/s 2023-09-30T14:55:51.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177080 36.97 MB/sec execute 144 sec latency 159.673 ms 2023-09-30T14:55:51.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:51 smithi032 ceph-mon[98592]: pgmap v814: 97 pgs: 97 active+clean; 735 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.3 MiB/s wr, 785 op/s 2023-09-30T14:55:52.034 INFO:tasks.workunit.client.0.smithi032.stdout: 10 177786 36.93 MB/sec execute 145 sec latency 122.938 ms 2023-09-30T14:55:52.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:51 smithi156 ceph-mon[103579]: pgmap v814: 97 pgs: 97 active+clean; 735 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.3 MiB/s wr, 785 op/s 2023-09-30T14:55:52.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:51 smithi148 ceph-mon[102355]: pgmap v814: 97 pgs: 97 active+clean; 735 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.3 MiB/s wr, 785 op/s 2023-09-30T14:55:53.034 INFO:tasks.workunit.client.0.smithi032.stdout: 10 178682 36.99 MB/sec execute 146 sec latency 61.181 ms 2023-09-30T14:55:53.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:53 smithi032 ceph-mon[98592]: pgmap v815: 97 pgs: 97 active+clean; 739 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.1 MiB/s wr, 841 op/s 2023-09-30T14:55:54.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 179411 37.09 MB/sec execute 147 sec latency 75.541 ms 2023-09-30T14:55:54.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:53 smithi156 ceph-mon[103579]: pgmap v815: 97 pgs: 97 active+clean; 739 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.1 MiB/s wr, 841 op/s 2023-09-30T14:55:54.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:53 smithi148 ceph-mon[102355]: pgmap v815: 97 pgs: 97 active+clean; 739 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.1 MiB/s wr, 841 op/s 2023-09-30T14:55:55.034 INFO:tasks.workunit.client.0.smithi032.stdout: 10 180120 37.06 MB/sec execute 148 sec latency 52.590 ms 2023-09-30T14:55:55.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:55 smithi032 ceph-mon[98592]: pgmap v816: 97 pgs: 97 active+clean; 741 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.7 MiB/s wr, 793 op/s 2023-09-30T14:55:56.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 180556 36.99 MB/sec execute 149 sec latency 297.845 ms 2023-09-30T14:55:56.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:55 smithi156 ceph-mon[103579]: pgmap v816: 97 pgs: 97 active+clean; 741 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.7 MiB/s wr, 793 op/s 2023-09-30T14:55:56.114 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:55 smithi148 ceph-mon[102355]: pgmap v816: 97 pgs: 97 active+clean; 741 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.7 MiB/s wr, 793 op/s 2023-09-30T14:55:57.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 181103 36.90 MB/sec execute 150 sec latency 108.534 ms 2023-09-30T14:55:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:57 smithi032 ceph-mon[98592]: pgmap v817: 97 pgs: 97 active+clean; 749 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.1 MiB/s wr, 838 op/s 2023-09-30T14:55:58.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 181908 36.95 MB/sec execute 151 sec latency 60.177 ms 2023-09-30T14:55:58.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:57 smithi156 ceph-mon[103579]: pgmap v817: 97 pgs: 97 active+clean; 749 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.1 MiB/s wr, 838 op/s 2023-09-30T14:55:58.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:57 smithi148 ceph-mon[102355]: pgmap v817: 97 pgs: 97 active+clean; 749 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.1 MiB/s wr, 838 op/s 2023-09-30T14:55:59.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 182562 36.95 MB/sec execute 152 sec latency 258.488 ms 2023-09-30T14:55:59.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:55:59 smithi032 ceph-mon[98592]: pgmap v818: 97 pgs: 97 active+clean; 759 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.2 MiB/s wr, 819 op/s 2023-09-30T14:56:00.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 183349 36.98 MB/sec execute 153 sec latency 149.885 ms 2023-09-30T14:56:00.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:55:59 smithi156 ceph-mon[103579]: pgmap v818: 97 pgs: 97 active+clean; 759 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.2 MiB/s wr, 819 op/s 2023-09-30T14:56:00.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:55:59 smithi148 ceph-mon[102355]: pgmap v818: 97 pgs: 97 active+clean; 759 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.2 MiB/s wr, 819 op/s 2023-09-30T14:56:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.467+0000 7f29c0c46700 1 -- 172.21.15.32:0/411153020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29bc100f50 msgr2=0x7f29bc1013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.467+0000 7f29c0c46700 1 --2- 172.21.15.32:0/411153020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29bc100f50 0x7f29bc1013b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f29b00099c0 tx=0x7f29b0009cd0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.468+0000 7f29c0c46700 1 -- 172.21.15.32:0/411153020 shutdown_connections 2023-09-30T14:56:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.468+0000 7f29c0c46700 1 --2- 172.21.15.32:0/411153020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29bc1018f0 0x7f29bc1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.468+0000 7f29c0c46700 1 --2- 172.21.15.32:0/411153020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29bc0ffd60 0x7f29bc100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.468+0000 7f29c0c46700 1 --2- 172.21.15.32:0/411153020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29bc100f50 0x7f29bc1013b0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.468+0000 7f29c0c46700 1 -- 172.21.15.32:0/411153020 >> 172.21.15.32:0/411153020 conn(0x7f29bc0fb2b0 msgr2=0x7f29bc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.468+0000 7f29c0c46700 1 -- 172.21.15.32:0/411153020 shutdown_connections 2023-09-30T14:56:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.468+0000 7f29c0c46700 1 -- 172.21.15.32:0/411153020 wait complete. 2023-09-30T14:56:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29c0c46700 1 Processor -- start 2023-09-30T14:56:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29c0c46700 1 -- start start 2023-09-30T14:56:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29c0c46700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29bc0ffd60 0x7f29bc199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29c0c46700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29bc100f50 0x7f29bc19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29c0c46700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29bc1018f0 0x7f29bc19e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29c0c46700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc19e8b0 con 0x7f29bc100f50 2023-09-30T14:56:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29c0c46700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc19e9f0 con 0x7f29bc1018f0 2023-09-30T14:56:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29c0c46700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc19ebb0 con 0x7f29bc0ffd60 2023-09-30T14:56:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29b9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29bc100f50 0x7f29bc19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29ba59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29bc0ffd60 0x7f29bc199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29b9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29bc100f50 0x7f29bc19bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39150/0 (socket says 172.21.15.32:39150) 2023-09-30T14:56:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29b9d9b700 1 -- 172.21.15.32:0/3917364993 learned_addr learned my addr 172.21.15.32:0/3917364993 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29bad9d700 1 --2- 172.21.15.32:0/3917364993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29bc1018f0 0x7f29bc19e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29b9d9b700 1 -- 172.21.15.32:0/3917364993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29bc0ffd60 msgr2=0x7f29bc199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29b9d9b700 1 --2- 172.21.15.32:0/3917364993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29bc0ffd60 0x7f29bc199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29b9d9b700 1 -- 172.21.15.32:0/3917364993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29bc1018f0 msgr2=0x7f29bc19e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29b9d9b700 1 --2- 172.21.15.32:0/3917364993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29bc1018f0 0x7f29bc19e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29b9d9b700 1 -- 172.21.15.32:0/3917364993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b00095f0 con 0x7f29bc100f50 2023-09-30T14:56:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.469+0000 7f29ba59c700 1 --2- 172.21.15.32:0/3917364993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29bc0ffd60 0x7f29bc199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:56:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.470+0000 7f29b9d9b700 1 --2- 172.21.15.32:0/3917364993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29bc100f50 0x7f29bc19bd90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009fd0 tx=0x7f29b0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.470+0000 7f29ab7fe700 1 -- 172.21.15.32:0/3917364993 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b001d070 con 0x7f29bc100f50 2023-09-30T14:56:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.470+0000 7f29c0c46700 1 -- 172.21.15.32:0/3917364993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc19ee30 con 0x7f29bc100f50 2023-09-30T14:56:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.470+0000 7f29ab7fe700 1 -- 172.21.15.32:0/3917364993 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29b0005030 con 0x7f29bc100f50 2023-09-30T14:56:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.470+0000 7f29ab7fe700 1 -- 172.21.15.32:0/3917364993 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0004a90 con 0x7f29bc100f50 2023-09-30T14:56:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.470+0000 7f29c0c46700 1 -- 172.21.15.32:0/3917364993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29bc19f250 con 0x7f29bc100f50 2023-09-30T14:56:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.473+0000 7f29ab7fe700 1 -- 172.21.15.32:0/3917364993 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f29b00051a0 con 0x7f29bc100f50 2023-09-30T14:56:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.473+0000 7f29ab7fe700 1 --2- 172.21.15.32:0/3917364993 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f29a40a0fa0 0x7f29a40a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.473+0000 7f29ab7fe700 1 -- 172.21.15.32:0/3917364993 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f29b00cd4b0 con 0x7f29bc100f50 2023-09-30T14:56:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.473+0000 7f29ba59c700 1 --2- 172.21.15.32:0/3917364993 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f29a40a0fa0 0x7f29a40a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.474+0000 7f29ba59c700 1 --2- 172.21.15.32:0/3917364993 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f29a40a0fa0 0x7f29a40a3460 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f29bc068d20 tx=0x7f29ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.474+0000 7f29cce2eb80 1 Processor -- start 2023-09-30T14:56:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.474+0000 7f29cce2eb80 1 -- start start 2023-09-30T14:56:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.474+0000 7f29cce2eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d849200 0x561c7d849620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.474+0000 7f29cce2eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d84a310 0x561c7d84a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.474+0000 7f29cce2eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84acd0 0x561c7d851560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.474+0000 7f29cce2eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7d7b1400 con 0x561c7d849200 2023-09-30T14:56:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.474+0000 7f29cce2eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7d7a2bc0 con 0x561c7d84a310 2023-09-30T14:56:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.474+0000 7f29cce2eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7d7a04e0 con 0x561c7d84acd0 2023-09-30T14:56:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.474+0000 7f29ba59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d849200 0x561c7d849620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.474+0000 7f29ba59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d849200 0x561c7d849620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39156/0 (socket says 172.21.15.32:39156) 2023-09-30T14:56:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.474+0000 7f29ba59c700 1 -- 172.21.15.32:0/283060736 learned_addr learned my addr 172.21.15.32:0/283060736 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.474+0000 7f29b9d9b700 1 --2- 172.21.15.32:0/283060736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d84a310 0x561c7d84a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.475+0000 7f29bad9d700 1 --2- 172.21.15.32:0/283060736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84acd0 0x561c7d851560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.475+0000 7f29ba59c700 1 -- 172.21.15.32:0/283060736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84acd0 msgr2=0x561c7d851560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.475+0000 7f29ba59c700 1 --2- 172.21.15.32:0/283060736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84acd0 0x561c7d851560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.475+0000 7f29ba59c700 1 -- 172.21.15.32:0/283060736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d84a310 msgr2=0x561c7d84a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.475+0000 7f29ba59c700 1 --2- 172.21.15.32:0/283060736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d84a310 0x561c7d84a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.475+0000 7f29ba59c700 1 -- 172.21.15.32:0/283060736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c7d7fe870 con 0x561c7d849200 2023-09-30T14:56:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.475+0000 7f29ba59c700 1 --2- 172.21.15.32:0/283060736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d849200 0x561c7d849620 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0043e0 tx=0x7f29ac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adce5b79d6789ebd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.475+0000 7f29a8ff9700 1 -- 172.21.15.32:0/283060736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac003d10 con 0x561c7d849200 2023-09-30T14:56:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.475+0000 7f29a8ff9700 1 -- 172.21.15.32:0/283060736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29ac003e70 con 0x561c7d849200 2023-09-30T14:56:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.475+0000 7f29a8ff9700 1 -- 172.21.15.32:0/283060736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac01e750 con 0x561c7d849200 2023-09-30T14:56:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.476+0000 7f29cce2eb80 1 -- 172.21.15.32:0/283060736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d849200 msgr2=0x561c7d849620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.476+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/283060736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d849200 0x561c7d849620 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0043e0 tx=0x7f29ac014000 comp rx=0 tx=0).stop 2023-09-30T14:56:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.476+0000 7f29cce2eb80 1 -- 172.21.15.32:0/283060736 shutdown_connections 2023-09-30T14:56:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.476+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/283060736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84acd0 0x561c7d851560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.476+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/283060736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d849200 0x561c7d849620 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.476+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/283060736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d84a310 0x561c7d84a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.476+0000 7f29cce2eb80 1 -- 172.21.15.32:0/283060736 >> 172.21.15.32:0/283060736 conn(0x561c7d7bd200 msgr2=0x561c7d843ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.476+0000 7f29cce2eb80 1 -- 172.21.15.32:0/283060736 shutdown_connections 2023-09-30T14:56:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.476+0000 7f29cce2eb80 1 -- 172.21.15.32:0/283060736 wait complete. 2023-09-30T14:56:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.476+0000 7f29cce2eb80 1 -- start start 2023-09-30T14:56:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.476+0000 7f29cce2eb80 1 Processor -- start 2023-09-30T14:56:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29cce2eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d849200 0x561c7d857310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29cce2eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84a310 0x561c7d857850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29cce2eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d84acd0 0x561c7d85e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29cce2eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7d79bdd0 con 0x561c7d84acd0 2023-09-30T14:56:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29cce2eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7d7386b0 con 0x561c7d849200 2023-09-30T14:56:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29cce2eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7d738350 con 0x561c7d84a310 2023-09-30T14:56:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29cce2eb80 10 client.0 _ll_register_callbacks cb 0x561c7d5957a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:56:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29bad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d84acd0 0x561c7d85e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29b9d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84a310 0x561c7d857850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29bad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d84acd0 0x561c7d85e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39166/0 (socket says 172.21.15.32:39166) 2023-09-30T14:56:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29bad9d700 1 -- 172.21.15.32:0/102153084 learned_addr learned my addr 172.21.15.32:0/102153084 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29ba59c700 1 --2- 172.21.15.32:0/102153084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d849200 0x561c7d857310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29b9d9b700 1 -- 172.21.15.32:0/102153084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d849200 msgr2=0x561c7d857310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29b9d9b700 1 --2- 172.21.15.32:0/102153084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d849200 0x561c7d857310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29b9d9b700 1 -- 172.21.15.32:0/102153084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d84acd0 msgr2=0x561c7d85e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29b9d9b700 1 --2- 172.21.15.32:0/102153084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d84acd0 0x561c7d85e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29b9d9b700 1 -- 172.21.15.32:0/102153084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ac017070 con 0x561c7d84a310 2023-09-30T14:56:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29bad9d700 1 --2- 172.21.15.32:0/102153084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d84acd0 0x561c7d85e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:56:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29b9d9b700 1 --2- 172.21.15.32:0/102153084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84a310 0x561c7d857850 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f29b000bf40 tx=0x7f29b0096e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de815297fea91ca9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:56:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.477+0000 7f29a1ffb700 1 -- 172.21.15.32:0/102153084 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b009eb60 con 0x561c7d84a310 2023-09-30T14:56:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.478+0000 7f29cce2eb80 1 -- 172.21.15.32:0/102153084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c7d83c090 con 0x561c7d84a310 2023-09-30T14:56:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.478+0000 7f29a1ffb700 1 -- 172.21.15.32:0/102153084 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f29b00a2d10 con 0x561c7d84a310 2023-09-30T14:56:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.478+0000 7f29a1ffb700 1 -- 172.21.15.32:0/102153084 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29b00cd0e0 con 0x561c7d84a310 2023-09-30T14:56:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.478+0000 7f29cce2eb80 10 client.26599 fetch_fsmap learned FSMap version 49 2023-09-30T14:56:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.478+0000 7f29cce2eb80 1 -- 172.21.15.32:0/102153084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x561c7d5c6d80 con 0x561c7d84a310 2023-09-30T14:56:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.478+0000 7f29a1ffb700 1 -- 172.21.15.32:0/102153084 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c7d83c090 con 0x561c7d84a310 2023-09-30T14:56:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.478+0000 7f29a1ffb700 1 -- 172.21.15.32:0/102153084 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f29b00a2880 con 0x561c7d84a310 2023-09-30T14:56:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.478+0000 7f29cce2eb80 10 client.26599 fetch_fsmap finished waiting for FSMap version 49 2023-09-30T14:56:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.478+0000 7f29cce2eb80 10 client.26599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:56:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.478+0000 7f29cce2eb80 20 client.26599 populate_metadata read hostname 'smithi032' 2023-09-30T14:56:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.478+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/102153084 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x561c7d862e90 0x561c7d865280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.478+0000 7f29cce2eb80 4 client.26599 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:56:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.478+0000 7f29cce2eb80 1 -- 172.21.15.32:0/102153084 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c7d794db0 con 0x561c7d862e90 2023-09-30T14:56:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.478+0000 7f29ba59c700 1 --2- 172.21.15.32:0/102153084 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x561c7d862e90 0x561c7d865280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.479+0000 7f29ba59c700 1 --2- 172.21.15.32:0/102153084 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x561c7d862e90 0x561c7d865280 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9623392e6c73b87d server_cookie=4f2636e56f24a6a0 in_seq=0 out_seq=0 2023-09-30T14:56:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.479+0000 7f29a1ffb700 10 client.26599 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.482+0000 7f29a1ffb700 1 -- 172.21.15.32:0/102153084 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561c7d794db0 con 0x561c7d862e90 2023-09-30T14:56:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.482+0000 7f29a1ffb700 10 client.26599 handle_command_reply: tid=0 2023-09-30T14:56:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.482+0000 7f29cce2eb80 1 client.26599 shutdown 2023-09-30T14:56:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.482+0000 7f29cce2eb80 1 -- 172.21.15.32:0/102153084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84a310 msgr2=0x561c7d857850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.482+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/102153084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84a310 0x561c7d857850 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f29b000bf40 tx=0x7f29b0096e80 comp rx=0 tx=0).stop 2023-09-30T14:56:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.482+0000 7f29cce2eb80 1 -- 172.21.15.32:0/102153084 shutdown_connections 2023-09-30T14:56:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.482+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/102153084 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x561c7d862e90 0x561c7d865280 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.482+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/102153084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d84acd0 0x561c7d85e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.482+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/102153084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d849200 0x561c7d857310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.482+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/102153084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84a310 0x561c7d857850 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.482+0000 7f29a1ffb700 0 client.26599 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.482+0000 7f29cce2eb80 1 -- 172.21.15.32:0/102153084 >> 172.21.15.32:0/102153084 conn(0x561c7d7bd200 msgr2=0x561c7d843590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.482+0000 7f29cce2eb80 1 -- 172.21.15.32:0/102153084 shutdown_connections 2023-09-30T14:56:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.482+0000 7f29cce2eb80 1 -- 172.21.15.32:0/102153084 wait complete. 2023-09-30T14:56:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.483+0000 7f29cce2eb80 20 client.26599 trim_cache size 0 max 16384 2023-09-30T14:56:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29cce2eb80 1 Processor -- start 2023-09-30T14:56:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29cce2eb80 1 -- start start 2023-09-30T14:56:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29cce2eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d849200 0x561c7d8510c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29cce2eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d84a310 0x561c7d851600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29cce2eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84acd0 0x561c7d85e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29cce2eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7d61c8c0 con 0x561c7d84a310 2023-09-30T14:56:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29cce2eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7d858500 con 0x561c7d849200 2023-09-30T14:56:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29cce2eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7d8523b0 con 0x561c7d84acd0 2023-09-30T14:56:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29b9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d84a310 0x561c7d851600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29b9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d84a310 0x561c7d851600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39172/0 (socket says 172.21.15.32:39172) 2023-09-30T14:56:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29b9d9b700 1 -- 172.21.15.32:0/795775370 learned_addr learned my addr 172.21.15.32:0/795775370 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29ba59c700 1 --2- 172.21.15.32:0/795775370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d849200 0x561c7d8510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29bad9d700 1 --2- 172.21.15.32:0/795775370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84acd0 0x561c7d85e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29b9d9b700 1 -- 172.21.15.32:0/795775370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84acd0 msgr2=0x561c7d85e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29b9d9b700 1 --2- 172.21.15.32:0/795775370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84acd0 0x561c7d85e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29b9d9b700 1 -- 172.21.15.32:0/795775370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d849200 msgr2=0x561c7d8510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29b9d9b700 1 --2- 172.21.15.32:0/795775370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d849200 0x561c7d8510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.498+0000 7f29b9d9b700 1 -- 172.21.15.32:0/795775370 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c7d5afc00 con 0x561c7d84a310 2023-09-30T14:56:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.499+0000 7f29b9d9b700 1 --2- 172.21.15.32:0/795775370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d84a310 0x561c7d851600 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f29b00cb010 tx=0x7f29b0009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c8711415d1430b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.499+0000 7f29a97fa700 1 -- 172.21.15.32:0/795775370 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b00a2d10 con 0x561c7d84a310 2023-09-30T14:56:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.499+0000 7f29a97fa700 1 -- 172.21.15.32:0/795775370 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29b009eb60 con 0x561c7d84a310 2023-09-30T14:56:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.499+0000 7f29a97fa700 1 -- 172.21.15.32:0/795775370 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0099f00 con 0x561c7d84a310 2023-09-30T14:56:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.499+0000 7f29cce2eb80 1 -- 172.21.15.32:0/795775370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d84a310 msgr2=0x561c7d851600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.499+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/795775370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d84a310 0x561c7d851600 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f29b00cb010 tx=0x7f29b0009d10 comp rx=0 tx=0).stop 2023-09-30T14:56:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.499+0000 7f29cce2eb80 1 -- 172.21.15.32:0/795775370 shutdown_connections 2023-09-30T14:56:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.499+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/795775370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84acd0 0x561c7d85e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.499+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/795775370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d849200 0x561c7d8510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.499+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/795775370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d84a310 0x561c7d851600 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.499+0000 7f29cce2eb80 1 -- 172.21.15.32:0/795775370 >> 172.21.15.32:0/795775370 conn(0x561c7d7bd200 msgr2=0x561c7d851c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.499+0000 7f29cce2eb80 1 -- 172.21.15.32:0/795775370 shutdown_connections 2023-09-30T14:56:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.499+0000 7f29cce2eb80 1 -- 172.21.15.32:0/795775370 wait complete. 2023-09-30T14:56:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.500+0000 7f29cce2eb80 1 -- start start 2023-09-30T14:56:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.500+0000 7f29cce2eb80 1 Processor -- start 2023-09-30T14:56:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.500+0000 7f29cce2eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d849200 0x561c7d85bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.500+0000 7f29cce2eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84a310 0x561c7d85c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.500+0000 7f29cce2eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d84acd0 0x561c7d85d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.500+0000 7f29cce2eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7d85d9c0 con 0x561c7d849200 2023-09-30T14:56:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.500+0000 7f29cce2eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7d85db30 con 0x561c7d84acd0 2023-09-30T14:56:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.500+0000 7f29cce2eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7d85dcf0 con 0x561c7d84a310 2023-09-30T14:56:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.500+0000 7f29ba59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d849200 0x561c7d85bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.500+0000 7f29cce2eb80 10 client.0 _ll_register_callbacks cb 0x7f29b009b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:56:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.500+0000 7f29ba59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d849200 0x561c7d85bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39176/0 (socket says 172.21.15.32:39176) 2023-09-30T14:56:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.500+0000 7f29ba59c700 1 -- 172.21.15.32:0/2336712303 learned_addr learned my addr 172.21.15.32:0/2336712303 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.500+0000 7f29b9d9b700 1 --2- 172.21.15.32:0/2336712303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84a310 0x561c7d85c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.500+0000 7f29bad9d700 1 --2- 172.21.15.32:0/2336712303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d84acd0 0x561c7d85d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.500+0000 7f29ba59c700 1 -- 172.21.15.32:0/2336712303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84a310 msgr2=0x561c7d85c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.501+0000 7f29ba59c700 1 --2- 172.21.15.32:0/2336712303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84a310 0x561c7d85c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.501+0000 7f29ba59c700 1 -- 172.21.15.32:0/2336712303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d84acd0 msgr2=0x561c7d85d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.501+0000 7f29ba59c700 1 --2- 172.21.15.32:0/2336712303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d84acd0 0x561c7d85d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.501+0000 7f29ba59c700 1 -- 172.21.15.32:0/2336712303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c7d5c6d80 con 0x561c7d849200 2023-09-30T14:56:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.501+0000 7f29b9d9b700 1 --2- 172.21.15.32:0/2336712303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84a310 0x561c7d85c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:56:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.501+0000 7f29ba59c700 1 --2- 172.21.15.32:0/2336712303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d849200 0x561c7d85bf30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f29ac004380 tx=0x7f29ac003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bfa29aef9ca584 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.501+0000 7f29a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:56:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.501+0000 7f29a1ffb700 1 -- 172.21.15.32:0/2336712303 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac02a040 con 0x561c7d849200 2023-09-30T14:56:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.501+0000 7f29a1ffb700 1 -- 172.21.15.32:0/2336712303 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f29ac034ad0 con 0x561c7d849200 2023-09-30T14:56:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.501+0000 7f29cce2eb80 1 -- 172.21.15.32:0/2336712303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29ac003e70 con 0x561c7d849200 2023-09-30T14:56:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.501+0000 7f29a1ffb700 1 -- 172.21.15.32:0/2336712303 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29ac034c90 con 0x561c7d849200 2023-09-30T14:56:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.501+0000 7f29cce2eb80 10 client.16944 fetch_fsmap learned FSMap version 49 2023-09-30T14:56:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.501+0000 7f29cce2eb80 1 -- 172.21.15.32:0/2336712303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x561c7d5ed1f0 con 0x561c7d849200 2023-09-30T14:56:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.501+0000 7f29a1ffb700 1 -- 172.21.15.32:0/2336712303 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29ac003e70 con 0x561c7d849200 2023-09-30T14:56:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.502+0000 7f29a1ffb700 1 -- 172.21.15.32:0/2336712303 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f29ac046070 con 0x561c7d849200 2023-09-30T14:56:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.502+0000 7f29cce2eb80 10 client.16944 fetch_fsmap finished waiting for FSMap version 49 2023-09-30T14:56:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.502+0000 7f29cce2eb80 10 client.16944 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:56:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.502+0000 7f29cce2eb80 20 client.16944 populate_metadata read hostname 'smithi032' 2023-09-30T14:56:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.502+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/2336712303 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x561c7d8b74e0 0x561c7d8b9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.502+0000 7f29cce2eb80 4 client.16944 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:56:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.502+0000 7f29cce2eb80 1 -- 172.21.15.32:0/2336712303 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561c7d794ad0 con 0x561c7d8b74e0 2023-09-30T14:56:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.502+0000 7f29b9d9b700 1 --2- 172.21.15.32:0/2336712303 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x561c7d8b74e0 0x561c7d8b9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.502+0000 7f29b9d9b700 1 --2- 172.21.15.32:0/2336712303 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x561c7d8b74e0 0x561c7d8b9900 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1ebdede013cb974 server_cookie=4f92d0eb2159fd26 in_seq=0 out_seq=0 2023-09-30T14:56:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.502+0000 7f29a1ffb700 10 client.16944 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.840+0000 7f29a1ffb700 1 -- 172.21.15.32:0/2336712303 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561c7d794ad0 con 0x561c7d8b74e0 2023-09-30T14:56:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.840+0000 7f29a1ffb700 10 client.16944 handle_command_reply: tid=0 2023-09-30T14:56:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.841+0000 7f29cce2eb80 1 client.16944 shutdown 2023-09-30T14:56:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.841+0000 7f29cce2eb80 1 -- 172.21.15.32:0/2336712303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d849200 msgr2=0x561c7d85bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.841+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/2336712303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d849200 0x561c7d85bf30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f29ac004380 tx=0x7f29ac003df0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.841+0000 7f29cce2eb80 1 -- 172.21.15.32:0/2336712303 shutdown_connections 2023-09-30T14:56:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.841+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/2336712303 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x561c7d8b74e0 0x561c7d8b9900 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.841+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/2336712303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c7d84acd0 0x561c7d85d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.841+0000 7f29a1ffb700 0 client.16944 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.841+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/2336712303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x561c7d849200 0x561c7d85bf30 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.841+0000 7f29cce2eb80 1 --2- 172.21.15.32:0/2336712303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c7d84a310 0x561c7d85c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.841+0000 7f29cce2eb80 1 -- 172.21.15.32:0/2336712303 >> 172.21.15.32:0/2336712303 conn(0x561c7d7bd200 msgr2=0x561c7d8510c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.841+0000 7f29cce2eb80 1 -- 172.21.15.32:0/2336712303 shutdown_connections 2023-09-30T14:56:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.841+0000 7f29cce2eb80 1 -- 172.21.15.32:0/2336712303 wait complete. 2023-09-30T14:56:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.841+0000 7f29cce2eb80 20 client.16944 trim_cache size 0 max 16384 2023-09-30T14:56:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:56:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:56:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:56:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:56:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.842+0000 7f29c0c46700 1 -- 172.21.15.32:0/3917364993 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f29a40a0fa0 msgr2=0x7f29a40a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.842+0000 7f29c0c46700 1 --2- 172.21.15.32:0/3917364993 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f29a40a0fa0 0x7f29a40a3460 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f29bc068d20 tx=0x7f29ac009260 comp rx=0 tx=0).stop 2023-09-30T14:56:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.842+0000 7f29c0c46700 1 -- 172.21.15.32:0/3917364993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29bc100f50 msgr2=0x7f29bc19bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.842+0000 7f29c0c46700 1 --2- 172.21.15.32:0/3917364993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29bc100f50 0x7f29bc19bd90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009fd0 tx=0x7f29b0009580 comp rx=0 tx=0).stop 2023-09-30T14:56:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.842+0000 7f29c0c46700 1 -- 172.21.15.32:0/3917364993 shutdown_connections 2023-09-30T14:56:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.842+0000 7f29c0c46700 1 --2- 172.21.15.32:0/3917364993 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f29a40a0fa0 0x7f29a40a3460 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.842+0000 7f29c0c46700 1 --2- 172.21.15.32:0/3917364993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29bc1018f0 0x7f29bc19e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.842+0000 7f29c0c46700 1 --2- 172.21.15.32:0/3917364993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29bc0ffd60 0x7f29bc199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.842+0000 7f29c0c46700 1 --2- 172.21.15.32:0/3917364993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29bc100f50 0x7f29bc19bd90 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.842+0000 7f29c0c46700 1 -- 172.21.15.32:0/3917364993 >> 172.21.15.32:0/3917364993 conn(0x7f29bc0fb2b0 msgr2=0x7f29bc104430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.842+0000 7f29c0c46700 1 -- 172.21.15.32:0/3917364993 shutdown_connections 2023-09-30T14:56:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:00.842+0000 7f29c0c46700 1 -- 172.21.15.32:0/3917364993 wait complete. 2023-09-30T14:56:01.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 183621 36.87 MB/sec execute 154 sec latency 317.840 ms 2023-09-30T14:56:02.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 184193 36.82 MB/sec execute 155 sec latency 94.294 ms 2023-09-30T14:56:02.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:01 smithi156 ceph-mon[103579]: pgmap v819: 97 pgs: 97 active+clean; 759 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.6 MiB/s wr, 707 op/s 2023-09-30T14:56:02.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:01 smithi148 ceph-mon[102355]: pgmap v819: 97 pgs: 97 active+clean; 759 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.6 MiB/s wr, 707 op/s 2023-09-30T14:56:02.141 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:01 smithi032 ceph-mon[98592]: pgmap v819: 97 pgs: 97 active+clean; 759 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.6 MiB/s wr, 707 op/s 2023-09-30T14:56:03.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 184947 36.82 MB/sec execute 156 sec latency 55.656 ms 2023-09-30T14:56:04.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 185622 36.84 MB/sec execute 157 sec latency 95.939 ms 2023-09-30T14:56:04.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:03 smithi156 ceph-mon[103579]: pgmap v820: 97 pgs: 97 active+clean; 763 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.2 MiB/s wr, 693 op/s 2023-09-30T14:56:04.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:03 smithi148 ceph-mon[102355]: pgmap v820: 97 pgs: 97 active+clean; 763 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.2 MiB/s wr, 693 op/s 2023-09-30T14:56:04.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:03 smithi032 ceph-mon[98592]: pgmap v820: 97 pgs: 97 active+clean; 763 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.2 MiB/s wr, 693 op/s 2023-09-30T14:56:05.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 186336 36.78 MB/sec execute 158 sec latency 67.405 ms 2023-09-30T14:56:06.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 186931 36.76 MB/sec execute 159 sec latency 203.322 ms 2023-09-30T14:56:06.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:05 smithi156 ceph-mon[103579]: pgmap v821: 97 pgs: 97 active+clean; 752 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.7 MiB/s wr, 714 op/s 2023-09-30T14:56:06.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:05 smithi148 ceph-mon[102355]: pgmap v821: 97 pgs: 97 active+clean; 752 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.7 MiB/s wr, 714 op/s 2023-09-30T14:56:06.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:05 smithi032 ceph-mon[98592]: pgmap v821: 97 pgs: 97 active+clean; 752 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.7 MiB/s wr, 714 op/s 2023-09-30T14:56:07.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 187851 36.89 MB/sec execute 160 sec latency 75.847 ms 2023-09-30T14:56:08.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 188571 36.96 MB/sec execute 161 sec latency 62.584 ms 2023-09-30T14:56:08.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:07 smithi156 ceph-mon[103579]: pgmap v822: 97 pgs: 97 active+clean; 718 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 10 MiB/s wr, 826 op/s 2023-09-30T14:56:08.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:07 smithi148 ceph-mon[102355]: pgmap v822: 97 pgs: 97 active+clean; 718 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 10 MiB/s wr, 826 op/s 2023-09-30T14:56:08.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:07 smithi032 ceph-mon[98592]: pgmap v822: 97 pgs: 97 active+clean; 718 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 10 MiB/s wr, 826 op/s 2023-09-30T14:56:09.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 189066 36.88 MB/sec execute 162 sec latency 57.311 ms 2023-09-30T14:56:10.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 189704 36.80 MB/sec execute 163 sec latency 82.877 ms 2023-09-30T14:56:10.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:09 smithi156 ceph-mon[103579]: pgmap v823: 97 pgs: 97 active+clean; 724 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.9 MiB/s wr, 810 op/s 2023-09-30T14:56:10.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:09 smithi148 ceph-mon[102355]: pgmap v823: 97 pgs: 97 active+clean; 724 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.9 MiB/s wr, 810 op/s 2023-09-30T14:56:10.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:09 smithi032 ceph-mon[98592]: pgmap v823: 97 pgs: 97 active+clean; 724 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.9 MiB/s wr, 810 op/s 2023-09-30T14:56:11.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 190188 36.72 MB/sec execute 164 sec latency 229.576 ms 2023-09-30T14:56:12.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 190956 36.73 MB/sec execute 165 sec latency 139.455 ms 2023-09-30T14:56:12.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:11 smithi156 ceph-mon[103579]: pgmap v824: 97 pgs: 97 active+clean; 724 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.0 MiB/s wr, 724 op/s 2023-09-30T14:56:12.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:11 smithi148 ceph-mon[102355]: pgmap v824: 97 pgs: 97 active+clean; 724 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.0 MiB/s wr, 724 op/s 2023-09-30T14:56:12.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:11 smithi032 ceph-mon[98592]: pgmap v824: 97 pgs: 97 active+clean; 724 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.0 MiB/s wr, 724 op/s 2023-09-30T14:56:13.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 191852 36.85 MB/sec execute 166 sec latency 51.422 ms 2023-09-30T14:56:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.961+0000 7fef8431b700 1 -- 172.21.15.32:0/1373178377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c1018f0 msgr2=0x7fef7c1081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.961+0000 7fef8431b700 1 --2- 172.21.15.32:0/1373178377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c1018f0 0x7fef7c1081a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fef7400b0d0 tx=0x7fef7400b3e0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.961+0000 7fef8431b700 1 -- 172.21.15.32:0/1373178377 shutdown_connections 2023-09-30T14:56:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.961+0000 7fef8431b700 1 --2- 172.21.15.32:0/1373178377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c1018f0 0x7fef7c1081a0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.961+0000 7fef8431b700 1 --2- 172.21.15.32:0/1373178377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef7c0ffd60 0x7fef7c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.961+0000 7fef8431b700 1 --2- 172.21.15.32:0/1373178377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef7c100f50 0x7fef7c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.961+0000 7fef8431b700 1 -- 172.21.15.32:0/1373178377 >> 172.21.15.32:0/1373178377 conn(0x7fef7c0fb2b0 msgr2=0x7fef7c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.961+0000 7fef8431b700 1 -- 172.21.15.32:0/1373178377 shutdown_connections 2023-09-30T14:56:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.962+0000 7fef8431b700 1 -- 172.21.15.32:0/1373178377 wait complete. 2023-09-30T14:56:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.962+0000 7fef8431b700 1 Processor -- start 2023-09-30T14:56:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.962+0000 7fef8431b700 1 -- start start 2023-09-30T14:56:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.962+0000 7fef8431b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef7c0ffd60 0x7fef7c196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.962+0000 7fef8431b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c100f50 0x7fef7c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.962+0000 7fef8431b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef7c1018f0 0x7fef7c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.962+0000 7fef8431b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c1987a0 con 0x7fef7c100f50 2023-09-30T14:56:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.962+0000 7fef8431b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c198910 con 0x7fef7c1018f0 2023-09-30T14:56:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.962+0000 7fef8431b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c198ad0 con 0x7fef7c0ffd60 2023-09-30T14:56:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.963+0000 7fef820b7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef7c0ffd60 0x7fef7c196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.963+0000 7fef818b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c100f50 0x7fef7c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.963+0000 7fef820b7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef7c0ffd60 0x7fef7c196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53572/0 (socket says 172.21.15.32:53572) 2023-09-30T14:56:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.963+0000 7fef820b7700 1 -- 172.21.15.32:0/3688350068 learned_addr learned my addr 172.21.15.32:0/3688350068 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.963+0000 7fef828b8700 1 --2- 172.21.15.32:0/3688350068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef7c1018f0 0x7fef7c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.963+0000 7fef818b6700 1 -- 172.21.15.32:0/3688350068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef7c0ffd60 msgr2=0x7fef7c196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.963+0000 7fef818b6700 1 --2- 172.21.15.32:0/3688350068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef7c0ffd60 0x7fef7c196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.963+0000 7fef818b6700 1 -- 172.21.15.32:0/3688350068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef7c1018f0 msgr2=0x7fef7c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.963+0000 7fef818b6700 1 --2- 172.21.15.32:0/3688350068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef7c1018f0 0x7fef7c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.963+0000 7fef818b6700 1 -- 172.21.15.32:0/3688350068 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef74009d00 con 0x7fef7c100f50 2023-09-30T14:56:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.963+0000 7fef818b6700 1 --2- 172.21.15.32:0/3688350068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c100f50 0x7fef7c1972f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00e9f0 tx=0x7fef6c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.963+0000 7fef737fe700 1 -- 172.21.15.32:0/3688350068 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c00c9c0 con 0x7fef7c100f50 2023-09-30T14:56:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.963+0000 7fef737fe700 1 -- 172.21.15.32:0/3688350068 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef6c010430 con 0x7fef7c100f50 2023-09-30T14:56:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.963+0000 7fef8431b700 1 -- 172.21.15.32:0/3688350068 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7c198db0 con 0x7fef7c100f50 2023-09-30T14:56:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.963+0000 7fef737fe700 1 -- 172.21.15.32:0/3688350068 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c018680 con 0x7fef7c100f50 2023-09-30T14:56:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.963+0000 7fef8431b700 1 -- 172.21.15.32:0/3688350068 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c19f540 con 0x7fef7c100f50 2023-09-30T14:56:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.966+0000 7fef737fe700 1 -- 172.21.15.32:0/3688350068 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fef6c010860 con 0x7fef7c100f50 2023-09-30T14:56:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.967+0000 7fef737fe700 1 --2- 172.21.15.32:0/3688350068 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fef680a0fa0 0x7fef680a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.967+0000 7fef737fe700 1 -- 172.21.15.32:0/3688350068 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fef6c014070 con 0x7fef7c100f50 2023-09-30T14:56:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.967+0000 7fef820b7700 1 --2- 172.21.15.32:0/3688350068 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fef680a0fa0 0x7fef680a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.968+0000 7fef820b7700 1 --2- 172.21.15.32:0/3688350068 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fef680a0fa0 0x7fef680a3460 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fef7c197b50 tx=0x7fef78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.968+0000 7fef90503b80 1 Processor -- start 2023-09-30T14:56:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.968+0000 7fef90503b80 1 -- start start 2023-09-30T14:56:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.968+0000 7fef90503b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e356610 0x558e8e356a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.968+0000 7fef90503b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e357720 0x558e8e357ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.968+0000 7fef90503b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e3580e0 0x558e8e35e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.968+0000 7fef90503b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8e2ba400 con 0x558e8e357720 2023-09-30T14:56:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.968+0000 7fef90503b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8e2abbc0 con 0x558e8e3580e0 2023-09-30T14:56:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.968+0000 7fef90503b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8e2a94e0 con 0x558e8e356610 2023-09-30T14:56:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.968+0000 7fef818b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e357720 0x558e8e357ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.968+0000 7fef818b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e357720 0x558e8e357ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39200/0 (socket says 172.21.15.32:39200) 2023-09-30T14:56:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.968+0000 7fef818b6700 1 -- 172.21.15.32:0/911402909 learned_addr learned my addr 172.21.15.32:0/911402909 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.968+0000 7fef820b7700 1 --2- 172.21.15.32:0/911402909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e356610 0x558e8e356a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.968+0000 7fef828b8700 1 --2- 172.21.15.32:0/911402909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e3580e0 0x558e8e35e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.969+0000 7fef818b6700 1 -- 172.21.15.32:0/911402909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e356610 msgr2=0x558e8e356a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.969+0000 7fef818b6700 1 --2- 172.21.15.32:0/911402909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e356610 0x558e8e356a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.969+0000 7fef818b6700 1 -- 172.21.15.32:0/911402909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e3580e0 msgr2=0x558e8e35e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.969+0000 7fef818b6700 1 --2- 172.21.15.32:0/911402909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e3580e0 0x558e8e35e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.969+0000 7fef818b6700 1 -- 172.21.15.32:0/911402909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e8e307870 con 0x558e8e357720 2023-09-30T14:56:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.969+0000 7fef818b6700 1 --2- 172.21.15.32:0/911402909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e357720 0x558e8e357ba0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0961d0 tx=0x7fef6c0996b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e501b8b51407c05c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.969+0000 7fef70d82700 1 -- 172.21.15.32:0/911402909 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c099cd0 con 0x558e8e357720 2023-09-30T14:56:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.969+0000 7fef70d82700 1 -- 172.21.15.32:0/911402909 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef6c09de20 con 0x558e8e357720 2023-09-30T14:56:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.969+0000 7fef70d82700 1 -- 172.21.15.32:0/911402909 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c099420 con 0x558e8e357720 2023-09-30T14:56:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.970+0000 7fef90503b80 1 -- 172.21.15.32:0/911402909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e357720 msgr2=0x558e8e357ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.970+0000 7fef90503b80 1 --2- 172.21.15.32:0/911402909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e357720 0x558e8e357ba0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0961d0 tx=0x7fef6c0996b0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.970+0000 7fef90503b80 1 -- 172.21.15.32:0/911402909 shutdown_connections 2023-09-30T14:56:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.970+0000 7fef90503b80 1 --2- 172.21.15.32:0/911402909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e3580e0 0x558e8e35e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.970+0000 7fef90503b80 1 --2- 172.21.15.32:0/911402909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e356610 0x558e8e356a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.970+0000 7fef90503b80 1 --2- 172.21.15.32:0/911402909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e357720 0x558e8e357ba0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.970+0000 7fef90503b80 1 -- 172.21.15.32:0/911402909 >> 172.21.15.32:0/911402909 conn(0x558e8e2c6200 msgr2=0x558e8e34cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.970+0000 7fef90503b80 1 -- 172.21.15.32:0/911402909 shutdown_connections 2023-09-30T14:56:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.970+0000 7fef90503b80 1 -- 172.21.15.32:0/911402909 wait complete. 2023-09-30T14:56:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.970+0000 7fef90503b80 1 -- start start 2023-09-30T14:56:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef90503b80 1 Processor -- start 2023-09-30T14:56:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef90503b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e356610 0x558e8e360340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef90503b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e357720 0x558e8e360880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef90503b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e3580e0 0x558e8e3678c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef90503b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8e2a4dd0 con 0x558e8e357720 2023-09-30T14:56:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef90503b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8e2416b0 con 0x558e8e356610 2023-09-30T14:56:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef90503b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8e241350 con 0x558e8e3580e0 2023-09-30T14:56:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef90503b80 10 client.0 _ll_register_callbacks cb 0x558e8e09e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:56:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef818b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e357720 0x558e8e360880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef818b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e357720 0x558e8e360880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39206/0 (socket says 172.21.15.32:39206) 2023-09-30T14:56:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef818b6700 1 -- 172.21.15.32:0/795332858 learned_addr learned my addr 172.21.15.32:0/795332858 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef820b7700 1 --2- 172.21.15.32:0/795332858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e356610 0x558e8e360340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef828b8700 1 --2- 172.21.15.32:0/795332858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e3580e0 0x558e8e3678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef818b6700 1 -- 172.21.15.32:0/795332858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e3580e0 msgr2=0x558e8e3678c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef818b6700 1 --2- 172.21.15.32:0/795332858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e3580e0 0x558e8e3678c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef818b6700 1 -- 172.21.15.32:0/795332858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e356610 msgr2=0x558e8e360340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.971+0000 7fef818b6700 1 --2- 172.21.15.32:0/795332858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e356610 0x558e8e360340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.972+0000 7fef818b6700 1 -- 172.21.15.32:0/795332858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c19f540 con 0x558e8e357720 2023-09-30T14:56:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.972+0000 7fef818b6700 1 --2- 172.21.15.32:0/795332858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e357720 0x558e8e360880 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0a1b30 tx=0x7fef6c010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2ac69a320092dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.972+0000 7fef65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:56:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.972+0000 7fef65ffb700 1 -- 172.21.15.32:0/795332858 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c0b6070 con 0x558e8e357720 2023-09-30T14:56:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.972+0000 7fef65ffb700 1 -- 172.21.15.32:0/795332858 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fef6c0a2060 con 0x558e8e357720 2023-09-30T14:56:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.972+0000 7fef90503b80 1 -- 172.21.15.32:0/795332858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e8e345090 con 0x558e8e357720 2023-09-30T14:56:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.972+0000 7fef65ffb700 1 -- 172.21.15.32:0/795332858 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef6c099cd0 con 0x558e8e357720 2023-09-30T14:56:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.972+0000 7fef90503b80 10 client.16956 fetch_fsmap learned FSMap version 49 2023-09-30T14:56:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.972+0000 7fef90503b80 1 -- 172.21.15.32:0/795332858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x558e8e0cfd80 con 0x558e8e357720 2023-09-30T14:56:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.972+0000 7fef65ffb700 1 -- 172.21.15.32:0/795332858 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e8e345090 con 0x558e8e357720 2023-09-30T14:56:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.972+0000 7fef65ffb700 1 -- 172.21.15.32:0/795332858 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fef6c095ac0 con 0x558e8e357720 2023-09-30T14:56:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.973+0000 7fef90503b80 10 client.16956 fetch_fsmap finished waiting for FSMap version 49 2023-09-30T14:56:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.973+0000 7fef90503b80 10 client.16956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:56:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.973+0000 7fef90503b80 20 client.16956 populate_metadata read hostname 'smithi032' 2023-09-30T14:56:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.973+0000 7fef90503b80 1 --2- 172.21.15.32:0/795332858 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558e8e36beb0 0x558e8e36e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.973+0000 7fef90503b80 4 client.16956 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:56:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.973+0000 7fef90503b80 1 -- 172.21.15.32:0/795332858 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e8e29ddb0 con 0x558e8e36beb0 2023-09-30T14:56:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.973+0000 7fef820b7700 1 --2- 172.21.15.32:0/795332858 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558e8e36beb0 0x558e8e36e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.973+0000 7fef820b7700 1 --2- 172.21.15.32:0/795332858 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558e8e36beb0 0x558e8e36e2d0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e611189b432fe4 server_cookie=cd5da0ebc66c69b7 in_seq=0 out_seq=0 2023-09-30T14:56:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.973+0000 7fef65ffb700 10 client.16956 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.978+0000 7fef65ffb700 1 -- 172.21.15.32:0/795332858 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558e8e29ddb0 con 0x558e8e36beb0 2023-09-30T14:56:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.978+0000 7fef65ffb700 10 client.16956 handle_command_reply: tid=0 2023-09-30T14:56:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.979+0000 7fef90503b80 1 client.16956 shutdown 2023-09-30T14:56:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.979+0000 7fef90503b80 1 -- 172.21.15.32:0/795332858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e357720 msgr2=0x558e8e360880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.979+0000 7fef90503b80 1 --2- 172.21.15.32:0/795332858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e357720 0x558e8e360880 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0a1b30 tx=0x7fef6c010c60 comp rx=0 tx=0).stop 2023-09-30T14:56:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.979+0000 7fef90503b80 1 -- 172.21.15.32:0/795332858 shutdown_connections 2023-09-30T14:56:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.979+0000 7fef90503b80 1 --2- 172.21.15.32:0/795332858 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558e8e36beb0 0x558e8e36e2d0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.979+0000 7fef90503b80 1 --2- 172.21.15.32:0/795332858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e3580e0 0x558e8e3678c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.979+0000 7fef65ffb700 0 client.16956 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.979+0000 7fef90503b80 1 --2- 172.21.15.32:0/795332858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e356610 0x558e8e360340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.979+0000 7fef90503b80 1 --2- 172.21.15.32:0/795332858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e357720 0x558e8e360880 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.979+0000 7fef90503b80 1 -- 172.21.15.32:0/795332858 >> 172.21.15.32:0/795332858 conn(0x558e8e2c6200 msgr2=0x558e8e34c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.979+0000 7fef90503b80 1 -- 172.21.15.32:0/795332858 shutdown_connections 2023-09-30T14:56:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.979+0000 7fef90503b80 1 -- 172.21.15.32:0/795332858 wait complete. 2023-09-30T14:56:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.979+0000 7fef90503b80 20 client.16956 trim_cache size 0 max 16384 2023-09-30T14:56:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.995+0000 7fef90503b80 1 Processor -- start 2023-09-30T14:56:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.995+0000 7fef90503b80 1 -- start start 2023-09-30T14:56:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.995+0000 7fef90503b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e356610 0x558e8e35f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.995+0000 7fef90503b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e357720 0x558e8e34ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.995+0000 7fef90503b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e3580e0 0x558e8e35b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.995+0000 7fef90503b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8e1258c0 con 0x558e8e3580e0 2023-09-30T14:56:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.995+0000 7fef90503b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8e357430 con 0x558e8e356610 2023-09-30T14:56:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.995+0000 7fef90503b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8e344bd0 con 0x558e8e357720 2023-09-30T14:56:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.996+0000 7fef828b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e3580e0 0x558e8e35b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.996+0000 7fef828b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e3580e0 0x558e8e35b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39216/0 (socket says 172.21.15.32:39216) 2023-09-30T14:56:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.996+0000 7fef828b8700 1 -- 172.21.15.32:0/187489882 learned_addr learned my addr 172.21.15.32:0/187489882 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.996+0000 7fef818b6700 1 --2- 172.21.15.32:0/187489882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e357720 0x558e8e34ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.996+0000 7fef820b7700 1 --2- 172.21.15.32:0/187489882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e356610 0x558e8e35f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.996+0000 7fef828b8700 1 -- 172.21.15.32:0/187489882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e357720 msgr2=0x558e8e34ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.996+0000 7fef828b8700 1 --2- 172.21.15.32:0/187489882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e357720 0x558e8e34ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.996+0000 7fef828b8700 1 -- 172.21.15.32:0/187489882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e356610 msgr2=0x558e8e35f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.996+0000 7fef828b8700 1 --2- 172.21.15.32:0/187489882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e356610 0x558e8e35f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.996+0000 7fef828b8700 1 -- 172.21.15.32:0/187489882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e8e0b8c00 con 0x558e8e3580e0 2023-09-30T14:56:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.996+0000 7fef828b8700 1 --2- 172.21.15.32:0/187489882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e3580e0 0x558e8e35b040 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fef74015000 tx=0x7fef74009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e3b47f2730e5c0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.996+0000 7fef71583700 1 -- 172.21.15.32:0/187489882 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74021070 con 0x558e8e3580e0 2023-09-30T14:56:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.996+0000 7fef71583700 1 -- 172.21.15.32:0/187489882 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef740094b0 con 0x558e8e3580e0 2023-09-30T14:56:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.996+0000 7fef71583700 1 -- 172.21.15.32:0/187489882 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7400e040 con 0x558e8e3580e0 2023-09-30T14:56:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.997+0000 7fef90503b80 1 -- 172.21.15.32:0/187489882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e3580e0 msgr2=0x558e8e35b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.997+0000 7fef90503b80 1 --2- 172.21.15.32:0/187489882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e3580e0 0x558e8e35b040 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fef74015000 tx=0x7fef74009390 comp rx=0 tx=0).stop 2023-09-30T14:56:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.997+0000 7fef90503b80 1 -- 172.21.15.32:0/187489882 shutdown_connections 2023-09-30T14:56:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.997+0000 7fef90503b80 1 --2- 172.21.15.32:0/187489882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e3580e0 0x558e8e35b040 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.997+0000 7fef90503b80 1 --2- 172.21.15.32:0/187489882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e356610 0x558e8e35f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.997+0000 7fef90503b80 1 --2- 172.21.15.32:0/187489882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e357720 0x558e8e34ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.997+0000 7fef90503b80 1 -- 172.21.15.32:0/187489882 >> 172.21.15.32:0/187489882 conn(0x558e8e2c6200 msgr2=0x558e8e34c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.997+0000 7fef90503b80 1 -- 172.21.15.32:0/187489882 shutdown_connections 2023-09-30T14:56:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.997+0000 7fef90503b80 1 -- 172.21.15.32:0/187489882 wait complete. 2023-09-30T14:56:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.997+0000 7fef90503b80 1 -- start start 2023-09-30T14:56:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef90503b80 1 Processor -- start 2023-09-30T14:56:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef90503b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e356610 0x558e8e363a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef90503b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e357720 0x558e8e365f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef90503b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e3580e0 0x558e8e3664e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef90503b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8e366bf0 con 0x558e8e3580e0 2023-09-30T14:56:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef90503b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8e366d60 con 0x558e8e356610 2023-09-30T14:56:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef90503b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8e366f20 con 0x558e8e357720 2023-09-30T14:56:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef90503b80 10 client.0 _ll_register_callbacks cb 0x7fef6c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:56:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef818b6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e357720 0x558e8e365f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef828b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e3580e0 0x558e8e3664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef818b6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e357720 0x558e8e365f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53602/0 (socket says 172.21.15.32:53602) 2023-09-30T14:56:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef818b6700 1 -- 172.21.15.32:0/2747211781 learned_addr learned my addr 172.21.15.32:0/2747211781 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef820b7700 1 --2- 172.21.15.32:0/2747211781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e356610 0x558e8e363a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef818b6700 1 -- 172.21.15.32:0/2747211781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e356610 msgr2=0x558e8e363a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef818b6700 1 --2- 172.21.15.32:0/2747211781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e356610 0x558e8e363a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef818b6700 1 -- 172.21.15.32:0/2747211781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e3580e0 msgr2=0x558e8e3664e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef818b6700 1 --2- 172.21.15.32:0/2747211781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e3580e0 0x558e8e3664e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef818b6700 1 -- 172.21.15.32:0/2747211781 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c108af0 con 0x558e8e357720 2023-09-30T14:56:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.998+0000 7fef828b8700 1 --2- 172.21.15.32:0/2747211781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e3580e0 0x558e8e3664e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:56:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.999+0000 7fef818b6700 1 --2- 172.21.15.32:0/2747211781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e357720 0x558e8e365f60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0995c0 tx=0x7fef6c09a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d0e567ebc685d2b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.999+0000 7fef65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:56:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.999+0000 7fef65ffb700 1 -- 172.21.15.32:0/2747211781 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c0b6060 con 0x558e8e357720 2023-09-30T14:56:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.999+0000 7fef90503b80 1 -- 172.21.15.32:0/2747211781 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef6c09de20 con 0x558e8e357720 2023-09-30T14:56:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.999+0000 7fef65ffb700 1 -- 172.21.15.32:0/2747211781 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fef6c010610 con 0x558e8e357720 2023-09-30T14:56:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.999+0000 7fef65ffb700 1 -- 172.21.15.32:0/2747211781 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef6c099630 con 0x558e8e357720 2023-09-30T14:56:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.999+0000 7fef90503b80 10 client.26623 fetch_fsmap learned FSMap version 49 2023-09-30T14:56:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.999+0000 7fef90503b80 1 -- 172.21.15.32:0/2747211781 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x558e8e0f61f0 con 0x558e8e357720 2023-09-30T14:56:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.999+0000 7fef65ffb700 1 -- 172.21.15.32:0/2747211781 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef6c09de20 con 0x558e8e357720 2023-09-30T14:56:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.999+0000 7fef65ffb700 1 -- 172.21.15.32:0/2747211781 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fef6c0b2150 con 0x558e8e357720 2023-09-30T14:56:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.999+0000 7fef90503b80 10 client.26623 fetch_fsmap finished waiting for FSMap version 49 2023-09-30T14:56:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.999+0000 7fef90503b80 10 client.26623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:56:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.999+0000 7fef90503b80 20 client.26623 populate_metadata read hostname 'smithi032' 2023-09-30T14:56:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.999+0000 7fef90503b80 1 --2- 172.21.15.32:0/2747211781 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558e8e3c4a90 0x558e8e3c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.999+0000 7fef90503b80 4 client.26623 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:56:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:13.999+0000 7fef90503b80 1 -- 172.21.15.32:0/2747211781 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558e8e29dad0 con 0x558e8e3c4a90 2023-09-30T14:56:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.002+0000 7fef820b7700 1 --2- 172.21.15.32:0/2747211781 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558e8e3c4a90 0x558e8e3c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.003+0000 7fef820b7700 1 --2- 172.21.15.32:0/2747211781 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558e8e3c4a90 0x558e8e3c6e80 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22be2cbd5bde4c91 server_cookie=c7a1004797b2086d in_seq=0 out_seq=0 2023-09-30T14:56:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.003+0000 7fef65ffb700 10 client.26623 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:14.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 192551 36.91 MB/sec execute 167 sec latency 38.705 ms 2023-09-30T14:56:14.059 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:13 smithi156 ceph-mon[103579]: pgmap v825: 97 pgs: 97 active+clean; 726 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.2 MiB/s wr, 743 op/s 2023-09-30T14:56:14.114 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:13 smithi148 ceph-mon[102355]: pgmap v825: 97 pgs: 97 active+clean; 726 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.2 MiB/s wr, 743 op/s 2023-09-30T14:56:14.146 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:13 smithi032 ceph-mon[98592]: pgmap v825: 97 pgs: 97 active+clean; 726 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.2 MiB/s wr, 743 op/s 2023-09-30T14:56:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.397+0000 7fef65ffb700 1 -- 172.21.15.32:0/2747211781 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558e8e29dad0 con 0x558e8e3c4a90 2023-09-30T14:56:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.397+0000 7fef65ffb700 10 client.26623 handle_command_reply: tid=0 2023-09-30T14:56:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.397+0000 7fef90503b80 1 client.26623 shutdown 2023-09-30T14:56:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.398+0000 7fef90503b80 1 -- 172.21.15.32:0/2747211781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e357720 msgr2=0x558e8e365f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.398+0000 7fef90503b80 1 --2- 172.21.15.32:0/2747211781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e357720 0x558e8e365f60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0995c0 tx=0x7fef6c09a510 comp rx=0 tx=0).stop 2023-09-30T14:56:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.398+0000 7fef90503b80 1 -- 172.21.15.32:0/2747211781 shutdown_connections 2023-09-30T14:56:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.398+0000 7fef90503b80 1 --2- 172.21.15.32:0/2747211781 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x558e8e3c4a90 0x558e8e3c6e80 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.398+0000 7fef90503b80 1 --2- 172.21.15.32:0/2747211781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e8e3580e0 0x558e8e3664e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.398+0000 7fef90503b80 1 --2- 172.21.15.32:0/2747211781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e8e356610 0x558e8e363a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.398+0000 7fef90503b80 1 --2- 172.21.15.32:0/2747211781 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558e8e357720 0x558e8e365f60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.398+0000 7fef90503b80 1 -- 172.21.15.32:0/2747211781 >> 172.21.15.32:0/2747211781 conn(0x558e8e2c6200 msgr2=0x558e8e35b040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.398+0000 7fef65ffb700 0 client.26623 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.398+0000 7fef90503b80 1 -- 172.21.15.32:0/2747211781 shutdown_connections 2023-09-30T14:56:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.398+0000 7fef90503b80 1 -- 172.21.15.32:0/2747211781 wait complete. 2023-09-30T14:56:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.398+0000 7fef90503b80 20 client.26623 trim_cache size 0 max 16384 2023-09-30T14:56:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:56:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:56:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:56:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:56:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.399+0000 7fef8431b700 1 -- 172.21.15.32:0/3688350068 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fef680a0fa0 msgr2=0x7fef680a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.399+0000 7fef8431b700 1 --2- 172.21.15.32:0/3688350068 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fef680a0fa0 0x7fef680a3460 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fef7c197b50 tx=0x7fef78009260 comp rx=0 tx=0).stop 2023-09-30T14:56:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.399+0000 7fef8431b700 1 -- 172.21.15.32:0/3688350068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c100f50 msgr2=0x7fef7c1972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.399+0000 7fef8431b700 1 --2- 172.21.15.32:0/3688350068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c100f50 0x7fef7c1972f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00e9f0 tx=0x7fef6c00ed00 comp rx=0 tx=0).stop 2023-09-30T14:56:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.399+0000 7fef8431b700 1 -- 172.21.15.32:0/3688350068 shutdown_connections 2023-09-30T14:56:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.399+0000 7fef8431b700 1 --2- 172.21.15.32:0/3688350068 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fef680a0fa0 0x7fef680a3460 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.399+0000 7fef8431b700 1 --2- 172.21.15.32:0/3688350068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef7c1018f0 0x7fef7c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.399+0000 7fef8431b700 1 --2- 172.21.15.32:0/3688350068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef7c0ffd60 0x7fef7c196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.399+0000 7fef8431b700 1 --2- 172.21.15.32:0/3688350068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef7c100f50 0x7fef7c1972f0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:14.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.399+0000 7fef8431b700 1 -- 172.21.15.32:0/3688350068 >> 172.21.15.32:0/3688350068 conn(0x7fef7c0fb2b0 msgr2=0x7fef7c068280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.399+0000 7fef8431b700 1 -- 172.21.15.32:0/3688350068 shutdown_connections 2023-09-30T14:56:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:14.399+0000 7fef8431b700 1 -- 172.21.15.32:0/3688350068 wait complete. 2023-09-30T14:56:15.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 192897 36.77 MB/sec execute 168 sec latency 239.237 ms 2023-09-30T14:56:16.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 193348 36.65 MB/sec execute 169 sec latency 259.716 ms 2023-09-30T14:56:16.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:15 smithi156 ceph-mon[103579]: pgmap v826: 97 pgs: 97 active+clean; 737 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.8 MiB/s wr, 821 op/s 2023-09-30T14:56:16.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:15 smithi148 ceph-mon[102355]: pgmap v826: 97 pgs: 97 active+clean; 737 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.8 MiB/s wr, 821 op/s 2023-09-30T14:56:16.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:15 smithi032 ceph-mon[98592]: pgmap v826: 97 pgs: 97 active+clean; 737 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.8 MiB/s wr, 821 op/s 2023-09-30T14:56:17.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 193987 36.64 MB/sec execute 170 sec latency 99.115 ms 2023-09-30T14:56:18.035 INFO:tasks.workunit.client.0.smithi032.stdout: 10 194837 36.73 MB/sec execute 171 sec latency 70.169 ms 2023-09-30T14:56:18.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:17 smithi156 ceph-mon[103579]: pgmap v827: 97 pgs: 97 active+clean; 734 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.3 MiB/s wr, 769 op/s 2023-09-30T14:56:18.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:17 smithi148 ceph-mon[102355]: pgmap v827: 97 pgs: 97 active+clean; 734 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.3 MiB/s wr, 769 op/s 2023-09-30T14:56:18.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:17 smithi032 ceph-mon[98592]: pgmap v827: 97 pgs: 97 active+clean; 734 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.3 MiB/s wr, 769 op/s 2023-09-30T14:56:19.036 INFO:tasks.workunit.client.0.smithi032.stdout: 10 195450 36.69 MB/sec execute 172 sec latency 82.141 ms 2023-09-30T14:56:20.036 INFO:tasks.workunit.client.0.smithi032.stdout: 10 195993 36.67 MB/sec execute 173 sec latency 79.316 ms 2023-09-30T14:56:20.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:19 smithi156 ceph-mon[103579]: pgmap v828: 97 pgs: 97 active+clean; 730 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.5 MiB/s wr, 803 op/s 2023-09-30T14:56:20.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:19 smithi148 ceph-mon[102355]: pgmap v828: 97 pgs: 97 active+clean; 730 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.5 MiB/s wr, 803 op/s 2023-09-30T14:56:20.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:19 smithi032 ceph-mon[98592]: pgmap v828: 97 pgs: 97 active+clean; 730 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.5 MiB/s wr, 803 op/s 2023-09-30T14:56:21.036 INFO:tasks.workunit.client.0.smithi032.stdout: 10 196504 36.60 MB/sec execute 174 sec latency 288.644 ms 2023-09-30T14:56:22.036 INFO:tasks.workunit.client.0.smithi032.stdout: 10 197164 36.57 MB/sec execute 175 sec latency 261.905 ms 2023-09-30T14:56:22.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:21 smithi156 ceph-mon[103579]: pgmap v829: 97 pgs: 97 active+clean; 730 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.6 MiB/s wr, 697 op/s 2023-09-30T14:56:22.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:21 smithi148 ceph-mon[102355]: pgmap v829: 97 pgs: 97 active+clean; 730 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.6 MiB/s wr, 697 op/s 2023-09-30T14:56:22.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:21 smithi032 ceph-mon[98592]: pgmap v829: 97 pgs: 97 active+clean; 730 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.6 MiB/s wr, 697 op/s 2023-09-30T14:56:23.036 INFO:tasks.workunit.client.0.smithi032.stdout: 10 197880 36.61 MB/sec execute 176 sec latency 49.713 ms 2023-09-30T14:56:24.036 INFO:tasks.workunit.client.0.smithi032.stdout: 10 198597 36.60 MB/sec execute 177 sec latency 84.118 ms 2023-09-30T14:56:24.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:23 smithi156 ceph-mon[103579]: pgmap v830: 97 pgs: 97 active+clean; 734 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.9 MiB/s wr, 746 op/s 2023-09-30T14:56:24.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:23 smithi148 ceph-mon[102355]: pgmap v830: 97 pgs: 97 active+clean; 734 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.9 MiB/s wr, 746 op/s 2023-09-30T14:56:24.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:23 smithi032 ceph-mon[98592]: pgmap v830: 97 pgs: 97 active+clean; 734 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.9 MiB/s wr, 746 op/s 2023-09-30T14:56:25.036 INFO:tasks.workunit.client.0.smithi032.stdout: 10 199464 36.65 MB/sec execute 178 sec latency 78.087 ms 2023-09-30T14:56:26.036 INFO:tasks.workunit.client.0.smithi032.stdout: 10 200016 36.68 MB/sec execute 179 sec latency 213.096 ms 2023-09-30T14:56:26.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:25 smithi156 ceph-mon[103579]: pgmap v831: 97 pgs: 97 active+clean; 740 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.5 MiB/s wr, 818 op/s 2023-09-30T14:56:26.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:25 smithi148 ceph-mon[102355]: pgmap v831: 97 pgs: 97 active+clean; 740 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.5 MiB/s wr, 818 op/s 2023-09-30T14:56:26.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:25 smithi032 ceph-mon[98592]: pgmap v831: 97 pgs: 97 active+clean; 740 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.5 MiB/s wr, 818 op/s 2023-09-30T14:56:27.036 INFO:tasks.workunit.client.0.smithi032.stdout: 10 200536 36.60 MB/sec execute 180 sec latency 250.330 ms 2023-09-30T14:56:27.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:26 smithi148 ceph-mon[102355]: pgmap v832: 97 pgs: 97 active+clean; 745 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.2 MiB/s wr, 736 op/s 2023-09-30T14:56:27.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:26 smithi032 ceph-mon[98592]: pgmap v832: 97 pgs: 97 active+clean; 745 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.2 MiB/s wr, 736 op/s 2023-09-30T14:56:27.307 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:26 smithi156 ceph-mon[103579]: pgmap v832: 97 pgs: 97 active+clean; 745 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.2 MiB/s wr, 736 op/s 2023-09-30T14:56:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.537+0000 7fe5b15cd700 1 -- 172.21.15.32:0/1618611200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5ac100f80 msgr2=0x7fe5ac1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.537+0000 7fe5b15cd700 1 --2- 172.21.15.32:0/1618611200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5ac100f80 0x7fe5ac1013e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe594009920 tx=0x7fe594009c30 comp rx=0 tx=0).stop 2023-09-30T14:56:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.537+0000 7fe5b15cd700 1 -- 172.21.15.32:0/1618611200 shutdown_connections 2023-09-30T14:56:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.537+0000 7fe5b15cd700 1 --2- 172.21.15.32:0/1618611200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac101920 0x7fe5ac1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.537+0000 7fe5b15cd700 1 --2- 172.21.15.32:0/1618611200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5ac0ffd90 0x7fe5ac1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.537+0000 7fe5b15cd700 1 --2- 172.21.15.32:0/1618611200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5ac100f80 0x7fe5ac1013e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.537+0000 7fe5b15cd700 1 -- 172.21.15.32:0/1618611200 >> 172.21.15.32:0/1618611200 conn(0x7fe5ac0fb320 msgr2=0x7fe5ac0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.537+0000 7fe5b15cd700 1 -- 172.21.15.32:0/1618611200 shutdown_connections 2023-09-30T14:56:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.537+0000 7fe5b15cd700 1 -- 172.21.15.32:0/1618611200 wait complete. 2023-09-30T14:56:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.538+0000 7fe5b15cd700 1 Processor -- start 2023-09-30T14:56:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.538+0000 7fe5b15cd700 1 -- start start 2023-09-30T14:56:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.538+0000 7fe5b15cd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5ac0ffd90 0x7fe5ac1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.538+0000 7fe5b15cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac100f80 0x7fe5ac19be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.538+0000 7fe5b15cd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5ac101920 0x7fe5ac19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.538+0000 7fe5b15cd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac19e960 con 0x7fe5ac100f80 2023-09-30T14:56:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.538+0000 7fe5b15cd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac19ead0 con 0x7fe5ac101920 2023-09-30T14:56:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.538+0000 7fe5b15cd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac19ec90 con 0x7fe5ac0ffd90 2023-09-30T14:56:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.539+0000 7fe5aa7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac100f80 0x7fe5ac19be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.539+0000 7fe5aa7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac100f80 0x7fe5ac19be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39234/0 (socket says 172.21.15.32:39234) 2023-09-30T14:56:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.539+0000 7fe5aa7fc700 1 -- 172.21.15.32:0/3012622441 learned_addr learned my addr 172.21.15.32:0/3012622441 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.539+0000 7fe5ab7fe700 1 --2- 172.21.15.32:0/3012622441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5ac101920 0x7fe5ac19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.539+0000 7fe5aaffd700 1 --2- 172.21.15.32:0/3012622441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5ac0ffd90 0x7fe5ac1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.539+0000 7fe5ab7fe700 1 -- 172.21.15.32:0/3012622441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5ac0ffd90 msgr2=0x7fe5ac1998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.539+0000 7fe5ab7fe700 1 --2- 172.21.15.32:0/3012622441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5ac0ffd90 0x7fe5ac1998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.539+0000 7fe5ab7fe700 1 -- 172.21.15.32:0/3012622441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac100f80 msgr2=0x7fe5ac19be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.539+0000 7fe5ab7fe700 1 --2- 172.21.15.32:0/3012622441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac100f80 0x7fe5ac19be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.539+0000 7fe5ab7fe700 1 -- 172.21.15.32:0/3012622441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a40095f0 con 0x7fe5ac101920 2023-09-30T14:56:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.539+0000 7fe5aa7fc700 1 --2- 172.21.15.32:0/3012622441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac100f80 0x7fe5ac19be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:56:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.539+0000 7fe5ab7fe700 1 --2- 172.21.15.32:0/3012622441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5ac101920 0x7fe5ac19e360 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400eb10 tx=0x7fe5a400ee20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.539+0000 7fe5a3fff700 1 -- 172.21.15.32:0/3012622441 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a400cb70 con 0x7fe5ac101920 2023-09-30T14:56:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.540+0000 7fe5b15cd700 1 -- 172.21.15.32:0/3012622441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe594009520 con 0x7fe5ac101920 2023-09-30T14:56:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.540+0000 7fe5a3fff700 1 -- 172.21.15.32:0/3012622441 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5a40044f0 con 0x7fe5ac101920 2023-09-30T14:56:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.540+0000 7fe5a3fff700 1 -- 172.21.15.32:0/3012622441 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4003e70 con 0x7fe5ac101920 2023-09-30T14:56:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.540+0000 7fe5b15cd700 1 -- 172.21.15.32:0/3012622441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ac19f210 con 0x7fe5ac101920 2023-09-30T14:56:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.542+0000 7fe5a3fff700 1 -- 172.21.15.32:0/3012622441 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fe5a4004000 con 0x7fe5ac101920 2023-09-30T14:56:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.543+0000 7fe5a3fff700 1 --2- 172.21.15.32:0/3012622441 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe5980a0fa0 0x7fe5980a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.543+0000 7fe5aaffd700 1 --2- 172.21.15.32:0/3012622441 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe5980a0fa0 0x7fe5980a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.543+0000 7fe5a3fff700 1 -- 172.21.15.32:0/3012622441 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe5a4014070 con 0x7fe5ac101920 2023-09-30T14:56:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.543+0000 7fe5aaffd700 1 --2- 172.21.15.32:0/3012622441 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe5980a0fa0 0x7fe5980a3460 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac069880 tx=0x7fe59c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.545+0000 7fe5bd7b5b80 1 Processor -- start 2023-09-30T14:56:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.545+0000 7fe5bd7b5b80 1 -- start start 2023-09-30T14:56:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.545+0000 7fe5bd7b5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc76200 0x563acfc76620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.545+0000 7fe5bd7b5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc77310 0x563acfc77790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.545+0000 7fe5bd7b5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc77cd0 0x563acfc7e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.545+0000 7fe5bd7b5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acfbde400 con 0x563acfc77310 2023-09-30T14:56:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.545+0000 7fe5bd7b5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acfbcfbc0 con 0x563acfc77cd0 2023-09-30T14:56:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.545+0000 7fe5bd7b5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acfbcd4e0 con 0x563acfc76200 2023-09-30T14:56:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.545+0000 7fe5aa7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc77310 0x563acfc77790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.545+0000 7fe5aa7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc77310 0x563acfc77790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39242/0 (socket says 172.21.15.32:39242) 2023-09-30T14:56:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.545+0000 7fe5aa7fc700 1 -- 172.21.15.32:0/3780863717 learned_addr learned my addr 172.21.15.32:0/3780863717 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.545+0000 7fe5aaffd700 1 --2- 172.21.15.32:0/3780863717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc76200 0x563acfc76620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.545+0000 7fe5ab7fe700 1 --2- 172.21.15.32:0/3780863717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc77cd0 0x563acfc7e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.546+0000 7fe5aa7fc700 1 -- 172.21.15.32:0/3780863717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc76200 msgr2=0x563acfc76620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.546+0000 7fe5aa7fc700 1 --2- 172.21.15.32:0/3780863717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc76200 0x563acfc76620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.546+0000 7fe5aa7fc700 1 -- 172.21.15.32:0/3780863717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc77cd0 msgr2=0x563acfc7e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.546+0000 7fe5aa7fc700 1 --2- 172.21.15.32:0/3780863717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc77cd0 0x563acfc7e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.546+0000 7fe5aa7fc700 1 -- 172.21.15.32:0/3780863717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563acfc2b870 con 0x563acfc77310 2023-09-30T14:56:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.546+0000 7fe5aa7fc700 1 --2- 172.21.15.32:0/3780863717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc77310 0x563acfc77790 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe594009840 tx=0x7fe594005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d9c82d8a65e111 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.546+0000 7fe5a17fa700 1 -- 172.21.15.32:0/3780863717 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59401c050 con 0x563acfc77310 2023-09-30T14:56:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.546+0000 7fe5a17fa700 1 -- 172.21.15.32:0/3780863717 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe59401d040 con 0x563acfc77310 2023-09-30T14:56:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.546+0000 7fe5a17fa700 1 -- 172.21.15.32:0/3780863717 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe594005360 con 0x563acfc77310 2023-09-30T14:56:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.547+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/3780863717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc77310 msgr2=0x563acfc77790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.547+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/3780863717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc77310 0x563acfc77790 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe594009840 tx=0x7fe594005e20 comp rx=0 tx=0).stop 2023-09-30T14:56:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.547+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/3780863717 shutdown_connections 2023-09-30T14:56:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.547+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/3780863717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc77cd0 0x563acfc7e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.547+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/3780863717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc76200 0x563acfc76620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.547+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/3780863717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc77310 0x563acfc77790 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.547+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/3780863717 >> 172.21.15.32:0/3780863717 conn(0x563acfbea200 msgr2=0x563acfc70ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.547+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/3780863717 shutdown_connections 2023-09-30T14:56:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.547+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/3780863717 wait complete. 2023-09-30T14:56:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.547+0000 7fe5bd7b5b80 1 -- start start 2023-09-30T14:56:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5bd7b5b80 1 Processor -- start 2023-09-30T14:56:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5bd7b5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc76200 0x563acfc7fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5bd7b5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc77310 0x563acfc80370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5bd7b5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc77cd0 0x563acfc873b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5bd7b5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acfbc8dd0 con 0x563acfc76200 2023-09-30T14:56:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5bd7b5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acfb656b0 con 0x563acfc77cd0 2023-09-30T14:56:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5bd7b5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acfb65350 con 0x563acfc77310 2023-09-30T14:56:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5bd7b5b80 10 client.0 _ll_register_callbacks cb 0x563acf9c27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:56:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5aaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc76200 0x563acfc7fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5aaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc76200 0x563acfc7fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39246/0 (socket says 172.21.15.32:39246) 2023-09-30T14:56:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5aaffd700 1 -- 172.21.15.32:0/1455040373 learned_addr learned my addr 172.21.15.32:0/1455040373 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5aa7fc700 1 --2- 172.21.15.32:0/1455040373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc77310 0x563acfc80370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5ab7fe700 1 --2- 172.21.15.32:0/1455040373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc77cd0 0x563acfc873b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5aaffd700 1 -- 172.21.15.32:0/1455040373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc77310 msgr2=0x563acfc80370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5aaffd700 1 --2- 172.21.15.32:0/1455040373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc77310 0x563acfc80370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5aaffd700 1 -- 172.21.15.32:0/1455040373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc77cd0 msgr2=0x563acfc873b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5aaffd700 1 --2- 172.21.15.32:0/1455040373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc77cd0 0x563acfc873b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.548+0000 7fe5aaffd700 1 -- 172.21.15.32:0/1455040373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ac108b20 con 0x563acfc76200 2023-09-30T14:56:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.549+0000 7fe5aaffd700 1 --2- 172.21.15.32:0/1455040373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc76200 0x563acfc7fe30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0045f0 tx=0x7fe59c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995543ac74493e30 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.549+0000 7fe5927fc700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:56:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.549+0000 7fe5927fc700 1 -- 172.21.15.32:0/1455040373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c00e040 con 0x563acfc76200 2023-09-30T14:56:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.549+0000 7fe5927fc700 1 -- 172.21.15.32:0/1455040373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe59c01c040 con 0x563acfc76200 2023-09-30T14:56:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.549+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/1455040373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563acfc69090 con 0x563acfc76200 2023-09-30T14:56:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.549+0000 7fe5927fc700 1 -- 172.21.15.32:0/1455040373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe59c004900 con 0x563acfc76200 2023-09-30T14:56:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.549+0000 7fe5bd7b5b80 10 client.16974 fetch_fsmap learned FSMap version 49 2023-09-30T14:56:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.549+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/1455040373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x563acf9f3d80 con 0x563acfc76200 2023-09-30T14:56:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.549+0000 7fe5927fc700 1 -- 172.21.15.32:0/1455040373 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563acfc69090 con 0x563acfc76200 2023-09-30T14:56:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.549+0000 7fe5927fc700 1 -- 172.21.15.32:0/1455040373 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe59c015410 con 0x563acfc76200 2023-09-30T14:56:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.549+0000 7fe5bd7b5b80 10 client.16974 fetch_fsmap finished waiting for FSMap version 49 2023-09-30T14:56:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.549+0000 7fe5bd7b5b80 10 client.16974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:56:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.549+0000 7fe5bd7b5b80 20 client.16974 populate_metadata read hostname 'smithi032' 2023-09-30T14:56:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.549+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/1455040373 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563acfc8bd10 0x563acfc8e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.549+0000 7fe5bd7b5b80 4 client.16974 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:56:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.549+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/1455040373 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563acfbc1db0 con 0x563acfc8bd10 2023-09-30T14:56:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.550+0000 7fe5aa7fc700 1 --2- 172.21.15.32:0/1455040373 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563acfc8bd10 0x563acfc8e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.550+0000 7fe5927fc700 10 client.16974 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.550+0000 7fe5aa7fc700 1 --2- 172.21.15.32:0/1455040373 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563acfc8bd10 0x563acfc8e130 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26f849fbd1a1c078 server_cookie=21335cd361597358 in_seq=0 out_seq=0 2023-09-30T14:56:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.552+0000 7fe5927fc700 1 -- 172.21.15.32:0/1455040373 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563acfbc1db0 con 0x563acfc8bd10 2023-09-30T14:56:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.552+0000 7fe5927fc700 10 client.16974 handle_command_reply: tid=0 2023-09-30T14:56:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.552+0000 7fe5bd7b5b80 1 client.16974 shutdown 2023-09-30T14:56:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.553+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/1455040373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc76200 msgr2=0x563acfc7fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.553+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/1455040373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc76200 0x563acfc7fe30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0045f0 tx=0x7fe59c004ad0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.553+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/1455040373 shutdown_connections 2023-09-30T14:56:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.553+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/1455040373 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563acfc8bd10 0x563acfc8e130 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.553+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/1455040373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc77cd0 0x563acfc873b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.553+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/1455040373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc76200 0x563acfc7fe30 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.553+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/1455040373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc77310 0x563acfc80370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.553+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/1455040373 >> 172.21.15.32:0/1455040373 conn(0x563acfbea200 msgr2=0x563acfc70590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.553+0000 7fe5927fc700 0 client.16974 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.553+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/1455040373 shutdown_connections 2023-09-30T14:56:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.553+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/1455040373 wait complete. 2023-09-30T14:56:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.553+0000 7fe5bd7b5b80 20 client.16974 trim_cache size 0 max 16384 2023-09-30T14:56:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.571+0000 7fe5bd7b5b80 1 Processor -- start 2023-09-30T14:56:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.571+0000 7fe5bd7b5b80 1 -- start start 2023-09-30T14:56:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.572+0000 7fe5bd7b5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc76200 0x563acfc87be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.572+0000 7fe5bd7b5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc77310 0x563acfc87370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.572+0000 7fe5bd7b5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc77cd0 0x563acfc8dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.572+0000 7fe5bd7b5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acfa498c0 con 0x563acfc76200 2023-09-30T14:56:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.572+0000 7fe5bd7b5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acfc890a0 con 0x563acfc77310 2023-09-30T14:56:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.572+0000 7fe5bd7b5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acfc729c0 con 0x563acfc77cd0 2023-09-30T14:56:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.572+0000 7fe5aaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc76200 0x563acfc87be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.572+0000 7fe5aaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc76200 0x563acfc87be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39254/0 (socket says 172.21.15.32:39254) 2023-09-30T14:56:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.572+0000 7fe5aaffd700 1 -- 172.21.15.32:0/3261404653 learned_addr learned my addr 172.21.15.32:0/3261404653 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.572+0000 7fe5aa7fc700 1 --2- 172.21.15.32:0/3261404653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc77310 0x563acfc87370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.572+0000 7fe5aaffd700 1 -- 172.21.15.32:0/3261404653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc77cd0 msgr2=0x563acfc8dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.572+0000 7fe5aaffd700 1 --2- 172.21.15.32:0/3261404653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc77cd0 0x563acfc8dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.572+0000 7fe5aaffd700 1 -- 172.21.15.32:0/3261404653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc77310 msgr2=0x563acfc87370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.572+0000 7fe5aaffd700 1 --2- 172.21.15.32:0/3261404653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc77310 0x563acfc87370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.572+0000 7fe5aaffd700 1 -- 172.21.15.32:0/3261404653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563acf9dcc00 con 0x563acfc76200 2023-09-30T14:56:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.572+0000 7fe5aaffd700 1 --2- 172.21.15.32:0/3261404653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc76200 0x563acfc87be0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009950 tx=0x7fe59c0073e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df39e5fd2784eb8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.573+0000 7fe5a1ffb700 1 -- 172.21.15.32:0/3261404653 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c00e070 con 0x563acfc76200 2023-09-30T14:56:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.573+0000 7fe5a1ffb700 1 -- 172.21.15.32:0/3261404653 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe59c00f040 con 0x563acfc76200 2023-09-30T14:56:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.573+0000 7fe5a1ffb700 1 -- 172.21.15.32:0/3261404653 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c023770 con 0x563acfc76200 2023-09-30T14:56:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.573+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/3261404653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc76200 msgr2=0x563acfc87be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.573+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/3261404653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc76200 0x563acfc87be0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009950 tx=0x7fe59c0073e0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.573+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/3261404653 shutdown_connections 2023-09-30T14:56:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.573+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/3261404653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc77cd0 0x563acfc8dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.573+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/3261404653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc76200 0x563acfc87be0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.573+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/3261404653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc77310 0x563acfc87370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.573+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/3261404653 >> 172.21.15.32:0/3261404653 conn(0x563acfbea200 msgr2=0x563acfc7ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.574+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/3261404653 shutdown_connections 2023-09-30T14:56:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.574+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/3261404653 wait complete. 2023-09-30T14:56:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.574+0000 7fe5bd7b5b80 1 -- start start 2023-09-30T14:56:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.574+0000 7fe5bd7b5b80 1 Processor -- start 2023-09-30T14:56:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.574+0000 7fe5bd7b5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc76200 0x563acfcdb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.574+0000 7fe5bd7b5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc77310 0x563acfcdb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.574+0000 7fe5bd7b5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc77cd0 0x563acfce27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.574+0000 7fe5bd7b5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acfce2d10 con 0x563acfc77310 2023-09-30T14:56:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.574+0000 7fe5bd7b5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acfce2e50 con 0x563acfc76200 2023-09-30T14:56:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.574+0000 7fe5bd7b5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acfce2fe0 con 0x563acfc77cd0 2023-09-30T14:56:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.574+0000 7fe5bd7b5b80 10 client.0 _ll_register_callbacks cb 0x563acf9c27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:56:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.574+0000 7fe5aa7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc77310 0x563acfcdb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.574+0000 7fe5aa7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc77310 0x563acfcdb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39262/0 (socket says 172.21.15.32:39262) 2023-09-30T14:56:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.574+0000 7fe5aa7fc700 1 -- 172.21.15.32:0/2276270456 learned_addr learned my addr 172.21.15.32:0/2276270456 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.574+0000 7fe5aaffd700 1 --2- 172.21.15.32:0/2276270456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc76200 0x563acfcdb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.575+0000 7fe5ab7fe700 1 --2- 172.21.15.32:0/2276270456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc77cd0 0x563acfce27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.575+0000 7fe5aa7fc700 1 -- 172.21.15.32:0/2276270456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc77cd0 msgr2=0x563acfce27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.575+0000 7fe5aa7fc700 1 --2- 172.21.15.32:0/2276270456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc77cd0 0x563acfce27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.575+0000 7fe5aa7fc700 1 -- 172.21.15.32:0/2276270456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc76200 msgr2=0x563acfcdb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.575+0000 7fe5aa7fc700 1 --2- 172.21.15.32:0/2276270456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc76200 0x563acfcdb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.575+0000 7fe5aa7fc700 1 -- 172.21.15.32:0/2276270456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563acf9f3d80 con 0x563acfc77310 2023-09-30T14:56:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.575+0000 7fe5ab7fe700 1 --2- 172.21.15.32:0/2276270456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc77cd0 0x563acfce27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:56:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.575+0000 7fe5aa7fc700 1 --2- 172.21.15.32:0/2276270456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc77310 0x563acfcdb790 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe59400fa40 tx=0x7fe59400f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b71409c42d9e29 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.575+0000 7fe5927fc700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:56:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.575+0000 7fe5927fc700 1 -- 172.21.15.32:0/2276270456 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe594034480 con 0x563acfc77310 2023-09-30T14:56:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.575+0000 7fe5927fc700 1 -- 172.21.15.32:0/2276270456 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe594032070 con 0x563acfc77310 2023-09-30T14:56:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.575+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/2276270456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe59401d040 con 0x563acfc77310 2023-09-30T14:56:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.575+0000 7fe5927fc700 1 -- 172.21.15.32:0/2276270456 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe59401c020 con 0x563acfc77310 2023-09-30T14:56:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.575+0000 7fe5bd7b5b80 10 client.16980 fetch_fsmap learned FSMap version 49 2023-09-30T14:56:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.575+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/2276270456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x563acfa1a1f0 con 0x563acfc77310 2023-09-30T14:56:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.576+0000 7fe5927fc700 1 -- 172.21.15.32:0/2276270456 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe59401d040 con 0x563acfc77310 2023-09-30T14:56:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.576+0000 7fe5927fc700 1 -- 172.21.15.32:0/2276270456 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 49) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe594015020 con 0x563acfc77310 2023-09-30T14:56:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.576+0000 7fe5bd7b5b80 10 client.16980 fetch_fsmap finished waiting for FSMap version 49 2023-09-30T14:56:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.576+0000 7fe5bd7b5b80 10 client.16980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:56:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.576+0000 7fe5bd7b5b80 20 client.16980 populate_metadata read hostname 'smithi032' 2023-09-30T14:56:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.576+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/2276270456 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563acfce4650 0x563acfce6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.576+0000 7fe5bd7b5b80 4 client.16980 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:56:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.576+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/2276270456 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563acfbc1ad0 con 0x563acfce4650 2023-09-30T14:56:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.576+0000 7fe5aaffd700 1 --2- 172.21.15.32:0/2276270456 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563acfce4650 0x563acfce6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.577+0000 7fe5927fc700 10 client.16980 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.577+0000 7fe5aaffd700 1 --2- 172.21.15.32:0/2276270456 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563acfce4650 0x563acfce6a70 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61ba62d22073b331 server_cookie=1b8a499639a1a6e8 in_seq=0 out_seq=0 2023-09-30T14:56:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.941+0000 7fe5927fc700 1 -- 172.21.15.32:0/2276270456 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563acfbc1ad0 con 0x563acfce4650 2023-09-30T14:56:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.941+0000 7fe5927fc700 10 client.16980 handle_command_reply: tid=0 2023-09-30T14:56:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.941+0000 7fe5bd7b5b80 1 client.16980 shutdown 2023-09-30T14:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.941+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/2276270456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc77310 msgr2=0x563acfcdb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.941+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/2276270456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc77310 0x563acfcdb790 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe59400fa40 tx=0x7fe59400f820 comp rx=0 tx=0).stop 2023-09-30T14:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.942+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/2276270456 shutdown_connections 2023-09-30T14:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.942+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/2276270456 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563acfce4650 0x563acfce6a70 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.942+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/2276270456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563acfc77cd0 0x563acfce27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.942+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/2276270456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563acfc76200 0x563acfcdb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.942+0000 7fe5927fc700 0 client.16980 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.942+0000 7fe5bd7b5b80 1 --2- 172.21.15.32:0/2276270456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563acfc77310 0x563acfcdb790 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.942+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/2276270456 >> 172.21.15.32:0/2276270456 conn(0x563acfbea200 msgr2=0x563acfc8dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.942+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/2276270456 shutdown_connections 2023-09-30T14:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.942+0000 7fe5bd7b5b80 1 -- 172.21.15.32:0/2276270456 wait complete. 2023-09-30T14:56:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.942+0000 7fe5bd7b5b80 20 client.16980 trim_cache size 0 max 16384 2023-09-30T14:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:56:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:56:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.943+0000 7fe5b15cd700 1 -- 172.21.15.32:0/3012622441 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe5980a0fa0 msgr2=0x7fe5980a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.943+0000 7fe5b15cd700 1 --2- 172.21.15.32:0/3012622441 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe5980a0fa0 0x7fe5980a3460 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac069880 tx=0x7fe59c00b040 comp rx=0 tx=0).stop 2023-09-30T14:56:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.943+0000 7fe5b15cd700 1 -- 172.21.15.32:0/3012622441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5ac101920 msgr2=0x7fe5ac19e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.943+0000 7fe5b15cd700 1 --2- 172.21.15.32:0/3012622441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5ac101920 0x7fe5ac19e360 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400eb10 tx=0x7fe5a400ee20 comp rx=0 tx=0).stop 2023-09-30T14:56:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.943+0000 7fe5b15cd700 1 -- 172.21.15.32:0/3012622441 shutdown_connections 2023-09-30T14:56:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.943+0000 7fe5b15cd700 1 --2- 172.21.15.32:0/3012622441 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe5980a0fa0 0x7fe5980a3460 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.943+0000 7fe5b15cd700 1 --2- 172.21.15.32:0/3012622441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5ac101920 0x7fe5ac19e360 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.943+0000 7fe5b15cd700 1 --2- 172.21.15.32:0/3012622441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5ac0ffd90 0x7fe5ac1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.943+0000 7fe5b15cd700 1 --2- 172.21.15.32:0/3012622441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac100f80 0x7fe5ac19be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.943+0000 7fe5b15cd700 1 -- 172.21.15.32:0/3012622441 >> 172.21.15.32:0/3012622441 conn(0x7fe5ac0fb320 msgr2=0x7fe5ac104590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.946+0000 7fe5b15cd700 1 -- 172.21.15.32:0/3012622441 shutdown_connections 2023-09-30T14:56:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:27.946+0000 7fe5b15cd700 1 -- 172.21.15.32:0/3012622441 wait complete. 2023-09-30T14:56:28.036 INFO:tasks.workunit.client.0.smithi032.stdout: 10 201001 36.54 MB/sec execute 181 sec latency 243.212 ms 2023-09-30T14:56:29.036 INFO:tasks.workunit.client.0.smithi032.stdout: 10 201765 36.56 MB/sec execute 182 sec latency 100.893 ms 2023-09-30T14:56:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:29 smithi156 ceph-mon[103579]: pgmap v833: 97 pgs: 97 active+clean; 754 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.6 MiB/s wr, 756 op/s 2023-09-30T14:56:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:29 smithi148 ceph-mon[102355]: pgmap v833: 97 pgs: 97 active+clean; 754 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.6 MiB/s wr, 756 op/s 2023-09-30T14:56:29.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:29 smithi032 ceph-mon[98592]: pgmap v833: 97 pgs: 97 active+clean; 754 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.6 MiB/s wr, 756 op/s 2023-09-30T14:56:30.036 INFO:tasks.workunit.client.0.smithi032.stdout: 10 202464 36.52 MB/sec execute 183 sec latency 75.494 ms 2023-09-30T14:56:31.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 203302 36.58 MB/sec execute 184 sec latency 96.678 ms 2023-09-30T14:56:31.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:31 smithi156 ceph-mon[103579]: pgmap v834: 97 pgs: 97 active+clean; 758 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.2 MiB/s wr, 671 op/s 2023-09-30T14:56:31.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:31 smithi148 ceph-mon[102355]: pgmap v834: 97 pgs: 97 active+clean; 758 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.2 MiB/s wr, 671 op/s 2023-09-30T14:56:31.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:31 smithi032 ceph-mon[98592]: pgmap v834: 97 pgs: 97 active+clean; 758 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.2 MiB/s wr, 671 op/s 2023-09-30T14:56:32.038 INFO:tasks.workunit.client.0.smithi032.stdout: 10 203875 36.53 MB/sec execute 185 sec latency 256.649 ms 2023-09-30T14:56:33.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 204725 36.63 MB/sec execute 186 sec latency 275.364 ms 2023-09-30T14:56:33.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:33 smithi156 ceph-mon[103579]: pgmap v835: 97 pgs: 97 active+clean; 752 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.5 MiB/s wr, 723 op/s 2023-09-30T14:56:33.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:33 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:56:33.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:33 smithi148 ceph-mon[102355]: pgmap v835: 97 pgs: 97 active+clean; 752 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.5 MiB/s wr, 723 op/s 2023-09-30T14:56:33.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:33 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:56:33.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:33 smithi032 ceph-mon[98592]: pgmap v835: 97 pgs: 97 active+clean; 752 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.5 MiB/s wr, 723 op/s 2023-09-30T14:56:33.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:33 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:56:34.038 INFO:tasks.workunit.client.0.smithi032.stdout: 10 205385 36.65 MB/sec execute 187 sec latency 53.171 ms 2023-09-30T14:56:35.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 206035 36.61 MB/sec execute 188 sec latency 89.613 ms 2023-09-30T14:56:35.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:35 smithi156 ceph-mon[103579]: pgmap v836: 97 pgs: 97 active+clean; 745 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 763 op/s 2023-09-30T14:56:35.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:35 smithi148 ceph-mon[102355]: pgmap v836: 97 pgs: 97 active+clean; 745 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 763 op/s 2023-09-30T14:56:35.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:35 smithi032 ceph-mon[98592]: pgmap v836: 97 pgs: 97 active+clean; 745 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 763 op/s 2023-09-30T14:56:36.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 206891 36.61 MB/sec execute 189 sec latency 68.592 ms 2023-09-30T14:56:37.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 207586 36.63 MB/sec execute 190 sec latency 306.647 ms 2023-09-30T14:56:37.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:37 smithi156 ceph-mon[103579]: pgmap v837: 97 pgs: 97 active+clean; 734 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.4 MiB/s wr, 752 op/s 2023-09-30T14:56:37.614 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:37 smithi148 ceph-mon[102355]: pgmap v837: 97 pgs: 97 active+clean; 734 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.4 MiB/s wr, 752 op/s 2023-09-30T14:56:37.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:37 smithi032 ceph-mon[98592]: pgmap v837: 97 pgs: 97 active+clean; 734 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.4 MiB/s wr, 752 op/s 2023-09-30T14:56:38.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 208279 36.69 MB/sec execute 191 sec latency 62.775 ms 2023-09-30T14:56:38.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:38 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:56:38.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:38 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:56:38.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:38 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:56:39.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 209044 36.73 MB/sec execute 192 sec latency 45.217 ms 2023-09-30T14:56:39.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:39 smithi156 ceph-mon[103579]: pgmap v838: 97 pgs: 97 active+clean; 746 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.5 MiB/s wr, 795 op/s 2023-09-30T14:56:39.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:39 smithi148 ceph-mon[102355]: pgmap v838: 97 pgs: 97 active+clean; 746 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.5 MiB/s wr, 795 op/s 2023-09-30T14:56:39.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:39 smithi032 ceph-mon[98592]: pgmap v838: 97 pgs: 97 active+clean; 746 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.5 MiB/s wr, 795 op/s 2023-09-30T14:56:40.040 INFO:tasks.workunit.client.0.smithi032.stdout: 10 209784 36.70 MB/sec execute 193 sec latency 39.847 ms 2023-09-30T14:56:41.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 210583 36.70 MB/sec execute 194 sec latency 49.816 ms 2023-09-30T14:56:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.088+0000 7fe9d9ebd700 1 -- 172.21.15.32:0/1316949197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9d4100280 msgr2=0x7fe9d41006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.088+0000 7fe9d9ebd700 1 --2- 172.21.15.32:0/1316949197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9d4100280 0x7fe9d41006a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40099c0 tx=0x7fe9c4009cd0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.088+0000 7fe9d9ebd700 1 -- 172.21.15.32:0/1316949197 shutdown_connections 2023-09-30T14:56:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.088+0000 7fe9d9ebd700 1 --2- 172.21.15.32:0/1316949197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9d4101950 0x7fe9d41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.088+0000 7fe9d9ebd700 1 --2- 172.21.15.32:0/1316949197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9d4100280 0x7fe9d41006a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.088+0000 7fe9d9ebd700 1 --2- 172.21.15.32:0/1316949197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9d4100f90 0x7fe9d4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.088+0000 7fe9d9ebd700 1 -- 172.21.15.32:0/1316949197 >> 172.21.15.32:0/1316949197 conn(0x7fe9d40faa30 msgr2=0x7fe9d40fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.088+0000 7fe9d9ebd700 1 -- 172.21.15.32:0/1316949197 shutdown_connections 2023-09-30T14:56:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.088+0000 7fe9d9ebd700 1 -- 172.21.15.32:0/1316949197 wait complete. 2023-09-30T14:56:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.089+0000 7fe9d9ebd700 1 Processor -- start 2023-09-30T14:56:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.089+0000 7fe9d9ebd700 1 -- start start 2023-09-30T14:56:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.089+0000 7fe9d9ebd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9d4100280 0x7fe9d4108cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.089+0000 7fe9d9ebd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9d4100f90 0x7fe9d4109210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.089+0000 7fe9d9ebd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9d4101950 0x7fe9d4109750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.089+0000 7fe9d9ebd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d41a0cd0 con 0x7fe9d4100280 2023-09-30T14:56:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.089+0000 7fe9d9ebd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d41a0e10 con 0x7fe9d4101950 2023-09-30T14:56:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.089+0000 7fe9d9ebd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d41a0f50 con 0x7fe9d4100f90 2023-09-30T14:56:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.089+0000 7fe9d37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9d4100280 0x7fe9d4108cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.090+0000 7fe9d37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9d4100280 0x7fe9d4108cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39274/0 (socket says 172.21.15.32:39274) 2023-09-30T14:56:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.090+0000 7fe9d37fe700 1 -- 172.21.15.32:0/52480281 learned_addr learned my addr 172.21.15.32:0/52480281 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.090+0000 7fe9d2ffd700 1 --2- 172.21.15.32:0/52480281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9d4100f90 0x7fe9d4109210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.090+0000 7fe9d3fff700 1 --2- 172.21.15.32:0/52480281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9d4101950 0x7fe9d4109750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.090+0000 7fe9d2ffd700 1 -- 172.21.15.32:0/52480281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9d4101950 msgr2=0x7fe9d4109750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.090+0000 7fe9d2ffd700 1 --2- 172.21.15.32:0/52480281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9d4101950 0x7fe9d4109750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.090+0000 7fe9d2ffd700 1 -- 172.21.15.32:0/52480281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9d4100280 msgr2=0x7fe9d4108cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.090+0000 7fe9d2ffd700 1 --2- 172.21.15.32:0/52480281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9d4100280 0x7fe9d4108cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.090+0000 7fe9d2ffd700 1 -- 172.21.15.32:0/52480281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc009c40 con 0x7fe9d4100f90 2023-09-30T14:56:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.090+0000 7fe9d37fe700 1 --2- 172.21.15.32:0/52480281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9d4100280 0x7fe9d4108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:56:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.090+0000 7fe9d2ffd700 1 --2- 172.21.15.32:0/52480281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9d4100f90 0x7fe9d4109210 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40fffa0 tx=0x7fe9c800b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.090+0000 7fe9d0ff9700 1 -- 172.21.15.32:0/52480281 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8011830 con 0x7fe9d4100f90 2023-09-30T14:56:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.090+0000 7fe9d9ebd700 1 -- 172.21.15.32:0/52480281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c40095f0 con 0x7fe9d4100f90 2023-09-30T14:56:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.090+0000 7fe9d0ff9700 1 -- 172.21.15.32:0/52480281 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c8011e50 con 0x7fe9d4100f90 2023-09-30T14:56:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.090+0000 7fe9d9ebd700 1 -- 172.21.15.32:0/52480281 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d41a15b0 con 0x7fe9d4100f90 2023-09-30T14:56:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.090+0000 7fe9d0ff9700 1 -- 172.21.15.32:0/52480281 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8010d70 con 0x7fe9d4100f90 2023-09-30T14:56:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.093+0000 7fe9d0ff9700 1 -- 172.21.15.32:0/52480281 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fe9c8011990 con 0x7fe9d4100f90 2023-09-30T14:56:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.093+0000 7fe9d0ff9700 1 --2- 172.21.15.32:0/52480281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe9bc0a1020 0x7fe9bc0a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.094+0000 7fe9d0ff9700 1 -- 172.21.15.32:0/52480281 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe9c80cbf20 con 0x7fe9d4100f90 2023-09-30T14:56:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.094+0000 7fe9d37fe700 1 --2- 172.21.15.32:0/52480281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe9bc0a1020 0x7fe9bc0a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.094+0000 7fe9d37fe700 1 --2- 172.21.15.32:0/52480281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe9bc0a1020 0x7fe9bc0a34e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4005b40 tx=0x7fe9c4005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.095+0000 7fe9e60a5b80 1 Processor -- start 2023-09-30T14:56:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.095+0000 7fe9e60a5b80 1 -- start start 2023-09-30T14:56:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.095+0000 7fe9e60a5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437dee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.095+0000 7fe9e60a5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437def310 0x559437def790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.095+0000 7fe9e60a5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 0x559437df6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.095+0000 7fe9e60a5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559437d56400 con 0x559437dee200 2023-09-30T14:56:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.095+0000 7fe9e60a5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x559437d47bc0 con 0x559437def310 2023-09-30T14:56:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.095+0000 7fe9e60a5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559437d454e0 con 0x559437defcd0 2023-09-30T14:56:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.095+0000 7fe9d37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437dee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.095+0000 7fe9d37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437dee620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39282/0 (socket says 172.21.15.32:39282) 2023-09-30T14:56:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.095+0000 7fe9d37fe700 1 -- 172.21.15.32:0/2992458883 learned_addr learned my addr 172.21.15.32:0/2992458883 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.095+0000 7fe9d3fff700 1 --2- 172.21.15.32:0/2992458883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 0x559437df6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.095+0000 7fe9d2ffd700 1 --2- 172.21.15.32:0/2992458883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437def310 0x559437def790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.096+0000 7fe9d3fff700 1 -- 172.21.15.32:0/2992458883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437def310 msgr2=0x559437def790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.096+0000 7fe9d3fff700 1 --2- 172.21.15.32:0/2992458883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437def310 0x559437def790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.096+0000 7fe9d3fff700 1 -- 172.21.15.32:0/2992458883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 msgr2=0x559437dee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.096+0000 7fe9d3fff700 1 --2- 172.21.15.32:0/2992458883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437dee620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.096+0000 7fe9d3fff700 1 -- 172.21.15.32:0/2992458883 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559437da3870 con 0x559437defcd0 2023-09-30T14:56:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.096+0000 7fe9d2ffd700 1 --2- 172.21.15.32:0/2992458883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437def310 0x559437def790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:56:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.096+0000 7fe9d37fe700 1 --2- 172.21.15.32:0/2992458883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437dee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:56:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.096+0000 7fe9d3fff700 1 --2- 172.21.15.32:0/2992458883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 0x559437df6560 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00e650 tx=0x7fe9cc009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=393f66547db18ff2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.096+0000 7fe9b9ffb700 1 -- 172.21.15.32:0/2992458883 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc00e040 con 0x559437defcd0 2023-09-30T14:56:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.096+0000 7fe9b9ffb700 1 -- 172.21.15.32:0/2992458883 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9cc00f040 con 0x559437defcd0 2023-09-30T14:56:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.096+0000 7fe9b9ffb700 1 -- 172.21.15.32:0/2992458883 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc007500 con 0x559437defcd0 2023-09-30T14:56:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.097+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/2992458883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 msgr2=0x559437df6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.097+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/2992458883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 0x559437df6560 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00e650 tx=0x7fe9cc009920 comp rx=0 tx=0).stop 2023-09-30T14:56:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.097+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/2992458883 shutdown_connections 2023-09-30T14:56:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.097+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/2992458883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 0x559437df6560 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.097+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/2992458883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437dee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.097+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/2992458883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437def310 0x559437def790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.097+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/2992458883 >> 172.21.15.32:0/2992458883 conn(0x559437d62200 msgr2=0x559437de8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.097+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/2992458883 shutdown_connections 2023-09-30T14:56:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.097+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/2992458883 wait complete. 2023-09-30T14:56:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.098+0000 7fe9e60a5b80 1 -- start start 2023-09-30T14:56:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.098+0000 7fe9e60a5b80 1 Processor -- start 2023-09-30T14:56:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.098+0000 7fe9e60a5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437df7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9e60a5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437def310 0x559437df8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9e60a5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 0x559437dff440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9e60a5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559437d40dd0 con 0x559437dee200 2023-09-30T14:56:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9e60a5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x559437cdd6b0 con 0x559437def310 2023-09-30T14:56:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9e60a5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559437cdd350 con 0x559437defcd0 2023-09-30T14:56:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9e60a5b80 10 client.0 _ll_register_callbacks cb 0x559437b3a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:56:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9d37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437df7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9d37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437df7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39288/0 (socket says 172.21.15.32:39288) 2023-09-30T14:56:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9d37fe700 1 -- 172.21.15.32:0/2004447491 learned_addr learned my addr 172.21.15.32:0/2004447491 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9d3fff700 1 --2- 172.21.15.32:0/2004447491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 0x559437dff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9d2ffd700 1 --2- 172.21.15.32:0/2004447491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437def310 0x559437df8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9d3fff700 1 -- 172.21.15.32:0/2004447491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437def310 msgr2=0x559437df8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9d3fff700 1 --2- 172.21.15.32:0/2004447491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437def310 0x559437df8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9d3fff700 1 -- 172.21.15.32:0/2004447491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 msgr2=0x559437df7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9d3fff700 1 --2- 172.21.15.32:0/2004447491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437df7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9d3fff700 1 -- 172.21.15.32:0/2004447491 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d41a15b0 con 0x559437defcd0 2023-09-30T14:56:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9d3fff700 1 --2- 172.21.15.32:0/2004447491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 0x559437dff440 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00e650 tx=0x7fe9cc00e1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4336b0feea62b09b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.099+0000 7fe9b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:56:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.100+0000 7fe9b2ffd700 1 -- 172.21.15.32:0/2004447491 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc00e040 con 0x559437defcd0 2023-09-30T14:56:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.100+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/2004447491 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559437de1090 con 0x559437defcd0 2023-09-30T14:56:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.100+0000 7fe9b2ffd700 1 -- 172.21.15.32:0/2004447491 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe9cc004060 con 0x559437defcd0 2023-09-30T14:56:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.100+0000 7fe9b2ffd700 1 -- 172.21.15.32:0/2004447491 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9cc03f030 con 0x559437defcd0 2023-09-30T14:56:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.100+0000 7fe9e60a5b80 10 client.26650 fetch_fsmap learned FSMap version 51 2023-09-30T14:56:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.100+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/2004447491 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7fe9cc00e330 con 0x559437defcd0 2023-09-30T14:56:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.100+0000 7fe9b2ffd700 1 -- 172.21.15.32:0/2004447491 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559437de1090 con 0x559437defcd0 2023-09-30T14:56:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.100+0000 7fe9b2ffd700 1 -- 172.21.15.32:0/2004447491 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 51) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe9cc039470 con 0x559437defcd0 2023-09-30T14:56:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.100+0000 7fe9e60a5b80 10 client.26650 fetch_fsmap finished waiting for FSMap version 51 2023-09-30T14:56:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.100+0000 7fe9e60a5b80 10 client.26650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:56:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.100+0000 7fe9e60a5b80 20 client.26650 populate_metadata read hostname 'smithi032' 2023-09-30T14:56:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.100+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/2004447491 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x559437e03cf0 0x559437e06110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.100+0000 7fe9e60a5b80 4 client.26650 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:56:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.101+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/2004447491 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559437d39db0 con 0x559437e03cf0 2023-09-30T14:56:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.101+0000 7fe9d37fe700 1 --2- 172.21.15.32:0/2004447491 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x559437e03cf0 0x559437e06110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.101+0000 7fe9b2ffd700 10 client.26650 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.101+0000 7fe9d37fe700 1 --2- 172.21.15.32:0/2004447491 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x559437e03cf0 0x559437e06110 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=646796eb082d902a server_cookie=1767793136edb678 in_seq=0 out_seq=0 2023-09-30T14:56:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.104+0000 7fe9b2ffd700 1 -- 172.21.15.32:0/2004447491 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559437d39db0 con 0x559437e03cf0 2023-09-30T14:56:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.104+0000 7fe9b2ffd700 10 client.26650 handle_command_reply: tid=0 2023-09-30T14:56:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.104+0000 7fe9e60a5b80 1 client.26650 shutdown 2023-09-30T14:56:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.105+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/2004447491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 msgr2=0x559437dff440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.105+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/2004447491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 0x559437dff440 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00e650 tx=0x7fe9cc00e1c0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.105+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/2004447491 shutdown_connections 2023-09-30T14:56:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.105+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/2004447491 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x559437e03cf0 0x559437e06110 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.105+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/2004447491 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 0x559437dff440 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.105+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/2004447491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437df7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.105+0000 7fe9b2ffd700 0 client.26650 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.105+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/2004447491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437def310 0x559437df8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.105+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/2004447491 >> 172.21.15.32:0/2004447491 conn(0x559437d62200 msgr2=0x559437de8590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.105+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/2004447491 shutdown_connections 2023-09-30T14:56:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.105+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/2004447491 wait complete. 2023-09-30T14:56:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.105+0000 7fe9e60a5b80 20 client.26650 trim_cache size 0 max 16384 2023-09-30T14:56:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.123+0000 7fe9e60a5b80 1 Processor -- start 2023-09-30T14:56:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.123+0000 7fe9e60a5b80 1 -- start start 2023-09-30T14:56:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.123+0000 7fe9e60a5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437dee200 0x559437df7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.123+0000 7fe9e60a5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437def310 0x559437dff360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.123+0000 7fe9e60a5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437defcd0 0x559437df83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.123+0000 7fe9e60a5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559437bc18c0 con 0x559437defcd0 2023-09-30T14:56:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.123+0000 7fe9e60a5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x559437df7370 con 0x559437dee200 2023-09-30T14:56:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.123+0000 7fe9e60a5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559437de0f80 con 0x559437def310 2023-09-30T14:56:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.123+0000 7fe9d3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437defcd0 0x559437df83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.123+0000 7fe9d3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437defcd0 0x559437df83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39300/0 (socket says 172.21.15.32:39300) 2023-09-30T14:56:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.123+0000 7fe9d3fff700 1 -- 172.21.15.32:0/3659371728 learned_addr learned my addr 172.21.15.32:0/3659371728 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.123+0000 7fe9d37fe700 1 --2- 172.21.15.32:0/3659371728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437dee200 0x559437df7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.123+0000 7fe9d2ffd700 1 --2- 172.21.15.32:0/3659371728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437def310 0x559437dff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.124+0000 7fe9d3fff700 1 -- 172.21.15.32:0/3659371728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437def310 msgr2=0x559437dff360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.124+0000 7fe9d3fff700 1 --2- 172.21.15.32:0/3659371728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437def310 0x559437dff360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.124+0000 7fe9d3fff700 1 -- 172.21.15.32:0/3659371728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437dee200 msgr2=0x559437df7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.124+0000 7fe9d3fff700 1 --2- 172.21.15.32:0/3659371728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437dee200 0x559437df7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.124+0000 7fe9d3fff700 1 -- 172.21.15.32:0/3659371728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559437b6bd80 con 0x559437defcd0 2023-09-30T14:56:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.124+0000 7fe9d37fe700 1 --2- 172.21.15.32:0/3659371728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437dee200 0x559437df7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:56:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.124+0000 7fe9d3fff700 1 --2- 172.21.15.32:0/3659371728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437defcd0 0x559437df83c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00e620 tx=0x7fe9cc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982f4589ae421f8a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.124+0000 7fe9ba7fc700 1 -- 172.21.15.32:0/3659371728 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc03f070 con 0x559437defcd0 2023-09-30T14:56:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.124+0000 7fe9ba7fc700 1 -- 172.21.15.32:0/3659371728 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9cc00e040 con 0x559437defcd0 2023-09-30T14:56:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.124+0000 7fe9ba7fc700 1 -- 172.21.15.32:0/3659371728 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc0049b0 con 0x559437defcd0 2023-09-30T14:56:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.125+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/3659371728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437defcd0 msgr2=0x559437df83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.125+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/3659371728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437defcd0 0x559437df83c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00e620 tx=0x7fe9cc009920 comp rx=0 tx=0).stop 2023-09-30T14:56:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.125+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/3659371728 shutdown_connections 2023-09-30T14:56:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.125+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/3659371728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437defcd0 0x559437df83c0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.125+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/3659371728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437dee200 0x559437df7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.125+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/3659371728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437def310 0x559437dff360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.125+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/3659371728 >> 172.21.15.32:0/3659371728 conn(0x559437d62200 msgr2=0x559437df6050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.125+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/3659371728 shutdown_connections 2023-09-30T14:56:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.125+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/3659371728 wait complete. 2023-09-30T14:56:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.125+0000 7fe9e60a5b80 1 -- start start 2023-09-30T14:56:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.126+0000 7fe9e60a5b80 1 Processor -- start 2023-09-30T14:56:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.126+0000 7fe9e60a5b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437dfcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.126+0000 7fe9e60a5b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437def310 0x559437dfd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.126+0000 7fe9e60a5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 0x559437dfe340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.126+0000 7fe9e60a5b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559437dfe880 con 0x559437dee200 2023-09-30T14:56:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.126+0000 7fe9e60a5b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x559437dfe9f0 con 0x559437def310 2023-09-30T14:56:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.126+0000 7fe9e60a5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559437dfebb0 con 0x559437defcd0 2023-09-30T14:56:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.126+0000 7fe9e60a5b80 10 client.0 _ll_register_callbacks cb 0x7fe9c400bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:56:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.126+0000 7fe9d37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437dfcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.127+0000 7fe9d37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437dfcdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39302/0 (socket says 172.21.15.32:39302) 2023-09-30T14:56:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.127+0000 7fe9d37fe700 1 -- 172.21.15.32:0/1024672563 learned_addr learned my addr 172.21.15.32:0/1024672563 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.127+0000 7fe9d2ffd700 1 --2- 172.21.15.32:0/1024672563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437def310 0x559437dfd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.127+0000 7fe9d3fff700 1 --2- 172.21.15.32:0/1024672563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 0x559437dfe340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.127+0000 7fe9d37fe700 1 -- 172.21.15.32:0/1024672563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 msgr2=0x559437dfe340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.127+0000 7fe9d37fe700 1 --2- 172.21.15.32:0/1024672563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 0x559437dfe340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.127+0000 7fe9d37fe700 1 -- 172.21.15.32:0/1024672563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437def310 msgr2=0x559437dfd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.127+0000 7fe9d37fe700 1 --2- 172.21.15.32:0/1024672563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437def310 0x559437dfd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.127+0000 7fe9d37fe700 1 -- 172.21.15.32:0/1024672563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c400bbf0 con 0x559437dee200 2023-09-30T14:56:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.127+0000 7fe9d37fe700 1 --2- 172.21.15.32:0/1024672563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437dfcdf0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40098e0 tx=0x7fe9c4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff987a80e8ff6da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.127+0000 7fe9b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:56:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.128+0000 7fe9b2ffd700 1 -- 172.21.15.32:0/1024672563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c402b040 con 0x559437dee200 2023-09-30T14:56:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.128+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/1024672563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559437d39ad0 con 0x559437dee200 2023-09-30T14:56:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.128+0000 7fe9b2ffd700 1 -- 172.21.15.32:0/1024672563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe9c40360b0 con 0x559437dee200 2023-09-30T14:56:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.128+0000 7fe9b2ffd700 1 -- 172.21.15.32:0/1024672563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9c4055030 con 0x559437dee200 2023-09-30T14:56:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.128+0000 7fe9e60a5b80 10 client.16995 fetch_fsmap learned FSMap version 51 2023-09-30T14:56:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.128+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/1024672563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7fe9cc00e330 con 0x559437dee200 2023-09-30T14:56:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.128+0000 7fe9b2ffd700 1 -- 172.21.15.32:0/1024672563 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559437d39ad0 con 0x559437dee200 2023-09-30T14:56:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.128+0000 7fe9b2ffd700 1 -- 172.21.15.32:0/1024672563 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 51) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe9c40347f0 con 0x559437dee200 2023-09-30T14:56:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.128+0000 7fe9e60a5b80 10 client.16995 fetch_fsmap finished waiting for FSMap version 51 2023-09-30T14:56:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.128+0000 7fe9e60a5b80 10 client.16995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:56:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.128+0000 7fe9e60a5b80 20 client.16995 populate_metadata read hostname 'smithi032' 2023-09-30T14:56:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.128+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/1024672563 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x559437e587a0 0x559437e5ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.128+0000 7fe9e60a5b80 4 client.16995 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:56:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.128+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/1024672563 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559437b8cf10 con 0x559437e587a0 2023-09-30T14:56:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.128+0000 7fe9d2ffd700 1 --2- 172.21.15.32:0/1024672563 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x559437e587a0 0x559437e5ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.129+0000 7fe9b2ffd700 10 client.16995 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.129+0000 7fe9d2ffd700 1 --2- 172.21.15.32:0/1024672563 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x559437e587a0 0x559437e5ac60 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b1303f00e845764 server_cookie=d09083cb3d3a601a in_seq=0 out_seq=0 2023-09-30T14:56:41.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:41 smithi156 ceph-mon[103579]: pgmap v839: 97 pgs: 97 active+clean; 746 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.7 MiB/s wr, 680 op/s 2023-09-30T14:56:41.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:41 smithi148 ceph-mon[102355]: pgmap v839: 97 pgs: 97 active+clean; 746 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.7 MiB/s wr, 680 op/s 2023-09-30T14:56:41.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:41 smithi032 ceph-mon[98592]: pgmap v839: 97 pgs: 97 active+clean; 746 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.7 MiB/s wr, 680 op/s 2023-09-30T14:56:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.817+0000 7fe9b2ffd700 1 -- 172.21.15.32:0/1024672563 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559437b8cf10 con 0x559437e587a0 2023-09-30T14:56:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.817+0000 7fe9b2ffd700 10 client.16995 handle_command_reply: tid=0 2023-09-30T14:56:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.817+0000 7fe9e60a5b80 1 client.16995 shutdown 2023-09-30T14:56:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.817+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/1024672563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 msgr2=0x559437dfcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.817+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/1024672563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437dfcdf0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40098e0 tx=0x7fe9c4009cd0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.817+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/1024672563 shutdown_connections 2023-09-30T14:56:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.817+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/1024672563 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x559437e587a0 0x559437e5ac60 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.817+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/1024672563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559437defcd0 0x559437dfe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.817+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/1024672563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559437dee200 0x559437dfcdf0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.817+0000 7fe9e60a5b80 1 --2- 172.21.15.32:0/1024672563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559437def310 0x559437dfd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.817+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/1024672563 >> 172.21.15.32:0/1024672563 conn(0x559437d62200 msgr2=0x559437df6050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.817+0000 7fe9b2ffd700 0 client.16995 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.817+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/1024672563 shutdown_connections 2023-09-30T14:56:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.817+0000 7fe9e60a5b80 1 -- 172.21.15.32:0/1024672563 wait complete. 2023-09-30T14:56:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.818+0000 7fe9e60a5b80 20 client.16995 trim_cache size 0 max 16384 2023-09-30T14:56:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:56:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:56:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:56:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:56:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.823+0000 7fe9d9ebd700 1 -- 172.21.15.32:0/52480281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe9bc0a1020 msgr2=0x7fe9bc0a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.823+0000 7fe9d9ebd700 1 --2- 172.21.15.32:0/52480281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe9bc0a1020 0x7fe9bc0a34e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4005b40 tx=0x7fe9c4005ad0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.823+0000 7fe9d9ebd700 1 -- 172.21.15.32:0/52480281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9d4100f90 msgr2=0x7fe9d4109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.823+0000 7fe9d9ebd700 1 --2- 172.21.15.32:0/52480281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9d4100f90 0x7fe9d4109210 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40fffa0 tx=0x7fe9c800b730 comp rx=0 tx=0).stop 2023-09-30T14:56:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.823+0000 7fe9d9ebd700 1 -- 172.21.15.32:0/52480281 shutdown_connections 2023-09-30T14:56:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.823+0000 7fe9d9ebd700 1 --2- 172.21.15.32:0/52480281 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe9bc0a1020 0x7fe9bc0a34e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.823+0000 7fe9d9ebd700 1 --2- 172.21.15.32:0/52480281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9d4101950 0x7fe9d4109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.823+0000 7fe9d9ebd700 1 --2- 172.21.15.32:0/52480281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9d4100280 0x7fe9d4108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.823+0000 7fe9d9ebd700 1 --2- 172.21.15.32:0/52480281 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9d4100f90 0x7fe9d4109210 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.823+0000 7fe9d9ebd700 1 -- 172.21.15.32:0/52480281 >> 172.21.15.32:0/52480281 conn(0x7fe9d40faa30 msgr2=0x7fe9d41044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.823+0000 7fe9d9ebd700 1 -- 172.21.15.32:0/52480281 shutdown_connections 2023-09-30T14:56:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:41.823+0000 7fe9d9ebd700 1 -- 172.21.15.32:0/52480281 wait complete. 2023-09-30T14:56:42.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 210933 36.61 MB/sec execute 195 sec latency 552.342 ms 2023-09-30T14:56:43.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 211662 36.65 MB/sec execute 196 sec latency 55.482 ms 2023-09-30T14:56:43.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:43 smithi156 ceph-mon[103579]: pgmap v840: 97 pgs: 97 active+clean; 753 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.8 MiB/s wr, 727 op/s 2023-09-30T14:56:43.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:43 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:56:43.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:43 smithi148 ceph-mon[102355]: pgmap v840: 97 pgs: 97 active+clean; 753 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.8 MiB/s wr, 727 op/s 2023-09-30T14:56:43.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:43 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:56:43.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:43 smithi032 ceph-mon[98592]: pgmap v840: 97 pgs: 97 active+clean; 753 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.8 MiB/s wr, 727 op/s 2023-09-30T14:56:43.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:43 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:56:44.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 212400 36.70 MB/sec execute 197 sec latency 38.782 ms 2023-09-30T14:56:44.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.556 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:44 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi032 to 197.2M 2023-09-30T14:56:44.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:44 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi148 to 196.2M 2023-09-30T14:56:44.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:44 smithi156 ceph-mon[103579]: Unable to set osd_memory_target on smithi032 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-30T14:56:44.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:44 smithi156 ceph-mon[103579]: Unable to set osd_memory_target on smithi148 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-30T14:56:44.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:56:44.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:56:44.558 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:56:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:44 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi032 to 197.2M 2023-09-30T14:56:44.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:44 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi148 to 196.2M 2023-09-30T14:56:44.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:44 smithi148 ceph-mon[102355]: Unable to set osd_memory_target on smithi032 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-30T14:56:44.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:44 smithi148 ceph-mon[102355]: Unable to set osd_memory_target on smithi148 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-30T14:56:44.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:56:44.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:56:44.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:56:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:56:44.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:44 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi032 to 197.2M 2023-09-30T14:56:44.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:44 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi148 to 196.2M 2023-09-30T14:56:44.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:44 smithi032 ceph-mon[98592]: Unable to set osd_memory_target on smithi032 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-09-30T14:56:44.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:44 smithi032 ceph-mon[98592]: Unable to set osd_memory_target on smithi148 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-30T14:56:44.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:56:44.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:56:44.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:56:45.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 213081 36.69 MB/sec execute 198 sec latency 65.915 ms 2023-09-30T14:56:45.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:45 smithi148 ceph-mon[102355]: pgmap v841: 97 pgs: 97 active+clean; 751 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.3 MiB/s wr, 778 op/s 2023-09-30T14:56:45.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:45 smithi032 ceph-mon[98592]: pgmap v841: 97 pgs: 97 active+clean; 751 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.3 MiB/s wr, 778 op/s 2023-09-30T14:56:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:45 smithi156 ceph-mon[103579]: pgmap v841: 97 pgs: 97 active+clean; 751 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.3 MiB/s wr, 778 op/s 2023-09-30T14:56:46.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 213787 36.67 MB/sec execute 199 sec latency 47.339 ms 2023-09-30T14:56:47.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 214282 36.59 MB/sec execute 200 sec latency 316.678 ms 2023-09-30T14:56:47.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:47 smithi156 ceph-mon[103579]: pgmap v842: 97 pgs: 97 active+clean; 749 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.7 MiB/s wr, 750 op/s 2023-09-30T14:56:47.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:47 smithi148 ceph-mon[102355]: pgmap v842: 97 pgs: 97 active+clean; 749 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.7 MiB/s wr, 750 op/s 2023-09-30T14:56:47.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:47 smithi032 ceph-mon[98592]: pgmap v842: 97 pgs: 97 active+clean; 749 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.7 MiB/s wr, 750 op/s 2023-09-30T14:56:48.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 214984 36.61 MB/sec execute 201 sec latency 69.366 ms 2023-09-30T14:56:49.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 215779 36.66 MB/sec execute 202 sec latency 69.299 ms 2023-09-30T14:56:49.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:49 smithi156 ceph-mon[103579]: pgmap v843: 97 pgs: 97 active+clean; 752 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 849 op/s 2023-09-30T14:56:49.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:49 smithi148 ceph-mon[102355]: pgmap v843: 97 pgs: 97 active+clean; 752 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 849 op/s 2023-09-30T14:56:49.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:49 smithi032 ceph-mon[98592]: pgmap v843: 97 pgs: 97 active+clean; 752 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 849 op/s 2023-09-30T14:56:50.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 216518 36.70 MB/sec execute 203 sec latency 34.828 ms 2023-09-30T14:56:51.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 217080 36.66 MB/sec execute 204 sec latency 49.069 ms 2023-09-30T14:56:51.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:51 smithi156 ceph-mon[103579]: pgmap v844: 97 pgs: 97 active+clean; 753 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.1 MiB/s wr, 748 op/s 2023-09-30T14:56:51.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:51 smithi148 ceph-mon[102355]: pgmap v844: 97 pgs: 97 active+clean; 753 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.1 MiB/s wr, 748 op/s 2023-09-30T14:56:51.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:51 smithi032 ceph-mon[98592]: pgmap v844: 97 pgs: 97 active+clean; 753 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.1 MiB/s wr, 748 op/s 2023-09-30T14:56:52.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 217554 36.58 MB/sec execute 205 sec latency 282.547 ms 2023-09-30T14:56:53.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 218209 36.58 MB/sec execute 206 sec latency 66.709 ms 2023-09-30T14:56:53.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:53 smithi156 ceph-mon[103579]: pgmap v845: 97 pgs: 97 active+clean; 754 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.1 MiB/s wr, 762 op/s 2023-09-30T14:56:53.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:53 smithi148 ceph-mon[102355]: pgmap v845: 97 pgs: 97 active+clean; 754 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.1 MiB/s wr, 762 op/s 2023-09-30T14:56:53.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:53 smithi032 ceph-mon[98592]: pgmap v845: 97 pgs: 97 active+clean; 754 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.1 MiB/s wr, 762 op/s 2023-09-30T14:56:54.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 219148 36.67 MB/sec execute 207 sec latency 45.717 ms 2023-09-30T14:56:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.958+0000 7f39cae20700 1 -- 172.21.15.32:0/3869662855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c4101900 msgr2=0x7f39c41081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.958+0000 7f39cae20700 1 --2- 172.21.15.32:0/3869662855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c4101900 0x7f39c41081b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00b0d0 tx=0x7f39bc00b3e0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.958+0000 7f39cae20700 1 -- 172.21.15.32:0/3869662855 shutdown_connections 2023-09-30T14:56:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.958+0000 7f39cae20700 1 --2- 172.21.15.32:0/3869662855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c4101900 0x7f39c41081b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.959+0000 7f39cae20700 1 --2- 172.21.15.32:0/3869662855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c40ffd70 0x7f39c4100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.959+0000 7f39cae20700 1 --2- 172.21.15.32:0/3869662855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39c4100f60 0x7f39c41013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.959+0000 7f39cae20700 1 -- 172.21.15.32:0/3869662855 >> 172.21.15.32:0/3869662855 conn(0x7f39c40fb300 msgr2=0x7f39c40fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.959+0000 7f39cae20700 1 -- 172.21.15.32:0/3869662855 shutdown_connections 2023-09-30T14:56:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.959+0000 7f39cae20700 1 -- 172.21.15.32:0/3869662855 wait complete. 2023-09-30T14:56:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.959+0000 7f39cae20700 1 Processor -- start 2023-09-30T14:56:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.959+0000 7f39cae20700 1 -- start start 2023-09-30T14:56:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.959+0000 7f39cae20700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39c40ffd70 0x7f39c4195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.959+0000 7f39cae20700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c4100f60 0x7f39c4196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.959+0000 7f39cae20700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c4101900 0x7f39c4196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.959+0000 7f39cae20700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c4197690 con 0x7f39c4100f60 2023-09-30T14:56:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.959+0000 7f39cae20700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c4197800 con 0x7f39c4101900 2023-09-30T14:56:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.959+0000 7f39cae20700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c41979c0 con 0x7f39c40ffd70 2023-09-30T14:56:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39c3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c4100f60 0x7f39c4196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39c8bbc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39c40ffd70 0x7f39c4195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39c93bd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c4101900 0x7f39c4196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39c3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c4100f60 0x7f39c4196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39318/0 (socket says 172.21.15.32:39318) 2023-09-30T14:56:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39c93bd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c4101900 0x7f39c4196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:50796/0 (socket says 172.21.15.32:50796) 2023-09-30T14:56:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39c8bbc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39c40ffd70 0x7f39c4195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53698/0 (socket says 172.21.15.32:53698) 2023-09-30T14:56:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39c93bd700 1 -- 172.21.15.32:0/105634486 learned_addr learned my addr 172.21.15.32:0/105634486 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39c3fff700 1 -- 172.21.15.32:0/105634486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39c40ffd70 msgr2=0x7f39c4195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39c3fff700 1 --2- 172.21.15.32:0/105634486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39c40ffd70 0x7f39c4195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39c3fff700 1 -- 172.21.15.32:0/105634486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c4101900 msgr2=0x7f39c4196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39c3fff700 1 --2- 172.21.15.32:0/105634486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c4101900 0x7f39c4196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39c3fff700 1 -- 172.21.15.32:0/105634486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc009d00 con 0x7f39c4100f60 2023-09-30T14:56:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39c3fff700 1 --2- 172.21.15.32:0/105634486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c4100f60 0x7f39c4196250 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f39b000e9f0 tx=0x7f39b000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39c1ffb700 1 -- 172.21.15.32:0/105634486 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b000c9c0 con 0x7f39c4100f60 2023-09-30T14:56:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39c1ffb700 1 -- 172.21.15.32:0/105634486 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39b0010430 con 0x7f39c4100f60 2023-09-30T14:56:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39cae20700 1 -- 172.21.15.32:0/105634486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c4197ca0 con 0x7f39c4100f60 2023-09-30T14:56:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.960+0000 7f39c1ffb700 1 -- 172.21.15.32:0/105634486 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b0018680 con 0x7f39c4100f60 2023-09-30T14:56:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.961+0000 7f39cae20700 1 -- 172.21.15.32:0/105634486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c40693a0 con 0x7f39c4100f60 2023-09-30T14:56:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.963+0000 7f39c1ffb700 1 -- 172.21.15.32:0/105634486 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f39b0010860 con 0x7f39c4100f60 2023-09-30T14:56:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.964+0000 7f39c1ffb700 1 --2- 172.21.15.32:0/105634486 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f39b40a1020 0x7f39b40a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.964+0000 7f39c8bbc700 1 --2- 172.21.15.32:0/105634486 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f39b40a1020 0x7f39b40a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.964+0000 7f39c1ffb700 1 -- 172.21.15.32:0/105634486 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f39b0014070 con 0x7f39c4100f60 2023-09-30T14:56:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.965+0000 7f39c8bbc700 1 --2- 172.21.15.32:0/105634486 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f39b40a1020 0x7f39b40a34e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f39c4100dc0 tx=0x7f39b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.965+0000 7f39d7008b80 1 Processor -- start 2023-09-30T14:56:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.965+0000 7f39d7008b80 1 -- start start 2023-09-30T14:56:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.965+0000 7f39d7008b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bc200 0x5576378bc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.965+0000 7f39d7008b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bd310 0x5576378bd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.965+0000 7f39d7008b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bdcd0 0x5576378c4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.965+0000 7f39d7008b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557637824400 con 0x5576378bc200 2023-09-30T14:56:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.965+0000 7f39d7008b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557637815bc0 con 0x5576378bdcd0 2023-09-30T14:56:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.965+0000 7f39d7008b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576378134e0 con 0x5576378bd310 2023-09-30T14:56:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.965+0000 7f39c8bbc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bc200 0x5576378bc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.965+0000 7f39c8bbc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bc200 0x5576378bc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39324/0 (socket says 172.21.15.32:39324) 2023-09-30T14:56:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.965+0000 7f39c8bbc700 1 -- 172.21.15.32:0/1614264213 learned_addr learned my addr 172.21.15.32:0/1614264213 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.965+0000 7f39c93bd700 1 --2- 172.21.15.32:0/1614264213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bdcd0 0x5576378c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.966+0000 7f39c8bbc700 1 -- 172.21.15.32:0/1614264213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bd310 msgr2=0x5576378bd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.966+0000 7f39c8bbc700 1 --2- 172.21.15.32:0/1614264213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bd310 0x5576378bd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.966+0000 7f39c8bbc700 1 -- 172.21.15.32:0/1614264213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bdcd0 msgr2=0x5576378c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.966+0000 7f39c8bbc700 1 --2- 172.21.15.32:0/1614264213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bdcd0 0x5576378c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.966+0000 7f39c8bbc700 1 -- 172.21.15.32:0/1614264213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557637871870 con 0x5576378bc200 2023-09-30T14:56:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.966+0000 7f39c93bd700 1 --2- 172.21.15.32:0/1614264213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bdcd0 0x5576378c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:56:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.966+0000 7f39c8bbc700 1 --2- 172.21.15.32:0/1614264213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bc200 0x5576378bc620 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f39b8011d50 tx=0x7f39b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68085ed1daf25003 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.966+0000 7f39aeffd700 1 -- 172.21.15.32:0/1614264213 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b800ed20 con 0x5576378bc200 2023-09-30T14:56:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.966+0000 7f39aeffd700 1 -- 172.21.15.32:0/1614264213 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39b8021070 con 0x5576378bc200 2023-09-30T14:56:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.966+0000 7f39aeffd700 1 -- 172.21.15.32:0/1614264213 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b801d670 con 0x5576378bc200 2023-09-30T14:56:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.967+0000 7f39d7008b80 1 -- 172.21.15.32:0/1614264213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bc200 msgr2=0x5576378bc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.967+0000 7f39d7008b80 1 --2- 172.21.15.32:0/1614264213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bc200 0x5576378bc620 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f39b8011d50 tx=0x7f39b8014000 comp rx=0 tx=0).stop 2023-09-30T14:56:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.967+0000 7f39d7008b80 1 -- 172.21.15.32:0/1614264213 shutdown_connections 2023-09-30T14:56:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.967+0000 7f39d7008b80 1 --2- 172.21.15.32:0/1614264213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bdcd0 0x5576378c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.967+0000 7f39d7008b80 1 --2- 172.21.15.32:0/1614264213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bc200 0x5576378bc620 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.967+0000 7f39d7008b80 1 --2- 172.21.15.32:0/1614264213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bd310 0x5576378bd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.967+0000 7f39d7008b80 1 -- 172.21.15.32:0/1614264213 >> 172.21.15.32:0/1614264213 conn(0x557637830200 msgr2=0x5576378b6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.967+0000 7f39d7008b80 1 -- 172.21.15.32:0/1614264213 shutdown_connections 2023-09-30T14:56:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.967+0000 7f39d7008b80 1 -- 172.21.15.32:0/1614264213 wait complete. 2023-09-30T14:56:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.967+0000 7f39d7008b80 1 -- start start 2023-09-30T14:56:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.967+0000 7f39d7008b80 1 Processor -- start 2023-09-30T14:56:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.968+0000 7f39d7008b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bc200 0x5576378c9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.968+0000 7f39d7008b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bd310 0x5576378c9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.968+0000 7f39d7008b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bdcd0 0x5576378c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.968+0000 7f39d7008b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55763780edd0 con 0x5576378bd310 2023-09-30T14:56:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.968+0000 7f39d7008b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576377ab6b0 con 0x5576378bdcd0 2023-09-30T14:56:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.968+0000 7f39d7008b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576377ab350 con 0x5576378bc200 2023-09-30T14:56:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.968+0000 7f39d7008b80 10 client.0 _ll_register_callbacks cb 0x5576376087a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:56:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.968+0000 7f39c3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bd310 0x5576378c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.968+0000 7f39c3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bd310 0x5576378c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39332/0 (socket says 172.21.15.32:39332) 2023-09-30T14:56:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.968+0000 7f39c3fff700 1 -- 172.21.15.32:0/2905330401 learned_addr learned my addr 172.21.15.32:0/2905330401 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.968+0000 7f39c8bbc700 1 --2- 172.21.15.32:0/2905330401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bc200 0x5576378c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.968+0000 7f39c93bd700 1 --2- 172.21.15.32:0/2905330401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bdcd0 0x5576378c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.969+0000 7f39c3fff700 1 -- 172.21.15.32:0/2905330401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bc200 msgr2=0x5576378c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.969+0000 7f39c3fff700 1 --2- 172.21.15.32:0/2905330401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bc200 0x5576378c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.969+0000 7f39c3fff700 1 -- 172.21.15.32:0/2905330401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bdcd0 msgr2=0x5576378c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.969+0000 7f39c3fff700 1 --2- 172.21.15.32:0/2905330401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bdcd0 0x5576378c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.969+0000 7f39c3fff700 1 -- 172.21.15.32:0/2905330401 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c4108b00 con 0x5576378bd310 2023-09-30T14:56:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.969+0000 7f39c93bd700 1 --2- 172.21.15.32:0/2905330401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bdcd0 0x5576378c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:56:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.969+0000 7f39c3fff700 1 --2- 172.21.15.32:0/2905330401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bd310 0x5576378c9980 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f39b00a1b00 tx=0x7f39b009a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769413fc8d09be5d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.969+0000 7f39a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:56:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.969+0000 7f39a3fff700 1 -- 172.21.15.32:0/2905330401 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b0099ca0 con 0x5576378bd310 2023-09-30T14:56:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.969+0000 7f39a3fff700 1 -- 172.21.15.32:0/2905330401 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f39b009ddf0 con 0x5576378bd310 2023-09-30T14:56:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.969+0000 7f39d7008b80 1 -- 172.21.15.32:0/2905330401 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576378af090 con 0x5576378bd310 2023-09-30T14:56:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.969+0000 7f39a3fff700 1 -- 172.21.15.32:0/2905330401 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39b00b2020 con 0x5576378bd310 2023-09-30T14:56:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.969+0000 7f39d7008b80 10 client.17007 fetch_fsmap learned FSMap version 51 2023-09-30T14:56:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.969+0000 7f39d7008b80 1 -- 172.21.15.32:0/2905330401 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f39b8017070 con 0x5576378bd310 2023-09-30T14:56:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.969+0000 7f39a3fff700 1 -- 172.21.15.32:0/2905330401 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39b00b6070 con 0x5576378bd310 2023-09-30T14:56:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.970+0000 7f39a3fff700 1 -- 172.21.15.32:0/2905330401 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 51) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f39b00a1cc0 con 0x5576378bd310 2023-09-30T14:56:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.970+0000 7f39d7008b80 10 client.17007 fetch_fsmap finished waiting for FSMap version 51 2023-09-30T14:56:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.970+0000 7f39d7008b80 10 client.17007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:56:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.970+0000 7f39d7008b80 20 client.17007 populate_metadata read hostname 'smithi032' 2023-09-30T14:56:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.970+0000 7f39d7008b80 1 --2- 172.21.15.32:0/2905330401 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5576378da660 0x5576378dca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.970+0000 7f39d7008b80 4 client.17007 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:56:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.970+0000 7f39d7008b80 1 -- 172.21.15.32:0/2905330401 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557637807db0 con 0x5576378da660 2023-09-30T14:56:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.970+0000 7f39c8bbc700 1 --2- 172.21.15.32:0/2905330401 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5576378da660 0x5576378dca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.970+0000 7f39c8bbc700 1 --2- 172.21.15.32:0/2905330401 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5576378da660 0x5576378dca80 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15e75126b364c265 server_cookie=ba572667f5db1e64 in_seq=0 out_seq=0 2023-09-30T14:56:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.970+0000 7f39a3fff700 10 client.17007 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.971+0000 7f39a3fff700 1 -- 172.21.15.32:0/2905330401 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557637807db0 con 0x5576378da660 2023-09-30T14:56:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.971+0000 7f39a3fff700 10 client.17007 handle_command_reply: tid=0 2023-09-30T14:56:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.971+0000 7f39d7008b80 1 client.17007 shutdown 2023-09-30T14:56:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.971+0000 7f39d7008b80 1 -- 172.21.15.32:0/2905330401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bd310 msgr2=0x5576378c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.971+0000 7f39d7008b80 1 --2- 172.21.15.32:0/2905330401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bd310 0x5576378c9980 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f39b00a1b00 tx=0x7f39b009a1e0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.971+0000 7f39d7008b80 1 -- 172.21.15.32:0/2905330401 shutdown_connections 2023-09-30T14:56:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.971+0000 7f39d7008b80 1 --2- 172.21.15.32:0/2905330401 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5576378da660 0x5576378dca80 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.971+0000 7f39d7008b80 1 --2- 172.21.15.32:0/2905330401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bdcd0 0x5576378c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.971+0000 7f39d7008b80 1 --2- 172.21.15.32:0/2905330401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bc200 0x5576378c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.971+0000 7f39d7008b80 1 --2- 172.21.15.32:0/2905330401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bd310 0x5576378c9980 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.971+0000 7f39d7008b80 1 -- 172.21.15.32:0/2905330401 >> 172.21.15.32:0/2905330401 conn(0x557637830200 msgr2=0x5576378b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.971+0000 7f39a3fff700 0 client.17007 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.971+0000 7f39d7008b80 1 -- 172.21.15.32:0/2905330401 shutdown_connections 2023-09-30T14:56:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.971+0000 7f39d7008b80 1 -- 172.21.15.32:0/2905330401 wait complete. 2023-09-30T14:56:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.971+0000 7f39d7008b80 20 client.17007 trim_cache size 0 max 16384 2023-09-30T14:56:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39d7008b80 1 Processor -- start 2023-09-30T14:56:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39d7008b80 1 -- start start 2023-09-30T14:56:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39d7008b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bc200 0x5576378ea220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39d7008b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bd310 0x5576378b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39d7008b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bdcd0 0x5576378c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39d7008b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55763768f8c0 con 0x5576378bdcd0 2023-09-30T14:56:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39d7008b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576378da1e0 con 0x5576378bd310 2023-09-30T14:56:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39d7008b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576378b86b0 con 0x5576378bc200 2023-09-30T14:56:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39c93bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bdcd0 0x5576378c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39c3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bd310 0x5576378b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39c93bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bdcd0 0x5576378c4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39342/0 (socket says 172.21.15.32:39342) 2023-09-30T14:56:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39c93bd700 1 -- 172.21.15.32:0/947883899 learned_addr learned my addr 172.21.15.32:0/947883899 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39c8bbc700 1 --2- 172.21.15.32:0/947883899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bc200 0x5576378ea220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39c93bd700 1 -- 172.21.15.32:0/947883899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bc200 msgr2=0x5576378ea220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39c93bd700 1 --2- 172.21.15.32:0/947883899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bc200 0x5576378ea220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39c93bd700 1 -- 172.21.15.32:0/947883899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bd310 msgr2=0x5576378b6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39c93bd700 1 --2- 172.21.15.32:0/947883899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bd310 0x5576378b6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39c93bd700 1 -- 172.21.15.32:0/947883899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557637639d80 con 0x5576378bdcd0 2023-09-30T14:56:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.987+0000 7f39c3fff700 1 --2- 172.21.15.32:0/947883899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bd310 0x5576378b6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:56:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.988+0000 7f39c93bd700 1 --2- 172.21.15.32:0/947883899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bdcd0 0x5576378c4c50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0094a0 tx=0x7f39bc0094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13098a61629239dd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.988+0000 7f39af7fe700 1 -- 172.21.15.32:0/947883899 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc021070 con 0x5576378bdcd0 2023-09-30T14:56:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.988+0000 7f39af7fe700 1 -- 172.21.15.32:0/947883899 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39bc00baa0 con 0x5576378bdcd0 2023-09-30T14:56:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.988+0000 7f39af7fe700 1 -- 172.21.15.32:0/947883899 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc00e040 con 0x5576378bdcd0 2023-09-30T14:56:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.989+0000 7f39d7008b80 1 -- 172.21.15.32:0/947883899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bdcd0 msgr2=0x5576378c4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.989+0000 7f39d7008b80 1 --2- 172.21.15.32:0/947883899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bdcd0 0x5576378c4c50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0094a0 tx=0x7f39bc0094d0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.989+0000 7f39d7008b80 1 -- 172.21.15.32:0/947883899 shutdown_connections 2023-09-30T14:56:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.989+0000 7f39d7008b80 1 --2- 172.21.15.32:0/947883899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bdcd0 0x5576378c4c50 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.989+0000 7f39d7008b80 1 --2- 172.21.15.32:0/947883899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bc200 0x5576378ea220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.989+0000 7f39d7008b80 1 --2- 172.21.15.32:0/947883899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bd310 0x5576378b6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.989+0000 7f39d7008b80 1 -- 172.21.15.32:0/947883899 >> 172.21.15.32:0/947883899 conn(0x557637830200 msgr2=0x5576378e13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.989+0000 7f39d7008b80 1 -- 172.21.15.32:0/947883899 shutdown_connections 2023-09-30T14:56:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.989+0000 7f39d7008b80 1 -- 172.21.15.32:0/947883899 wait complete. 2023-09-30T14:56:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.989+0000 7f39d7008b80 1 -- start start 2023-09-30T14:56:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.989+0000 7f39d7008b80 1 Processor -- start 2023-09-30T14:56:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39d7008b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bc200 0x5576378e9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39d7008b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bd310 0x5576378e9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39d7008b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bdcd0 0x55763792ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39d7008b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55763792d3c0 con 0x5576378bd310 2023-09-30T14:56:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39d7008b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55763792d500 con 0x5576378bc200 2023-09-30T14:56:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39d7008b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55763792d6c0 con 0x5576378bdcd0 2023-09-30T14:56:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39d7008b80 10 client.0 _ll_register_callbacks cb 0x7f39b0099a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:56:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39c3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bd310 0x5576378e9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39c3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bd310 0x5576378e9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39346/0 (socket says 172.21.15.32:39346) 2023-09-30T14:56:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39c3fff700 1 -- 172.21.15.32:0/181484168 learned_addr learned my addr 172.21.15.32:0/181484168 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:56:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39c8bbc700 1 --2- 172.21.15.32:0/181484168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bc200 0x5576378e9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39c93bd700 1 --2- 172.21.15.32:0/181484168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bdcd0 0x55763792ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39c3fff700 1 -- 172.21.15.32:0/181484168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bdcd0 msgr2=0x55763792ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39c3fff700 1 --2- 172.21.15.32:0/181484168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bdcd0 0x55763792ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39c3fff700 1 -- 172.21.15.32:0/181484168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bc200 msgr2=0x5576378e9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:56:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39c3fff700 1 --2- 172.21.15.32:0/181484168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bc200 0x5576378e9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39c3fff700 1 -- 172.21.15.32:0/181484168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b8017070 con 0x5576378bd310 2023-09-30T14:56:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39c3fff700 1 --2- 172.21.15.32:0/181484168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bd310 0x5576378e9c90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f39b00999a0 tx=0x7f39b0095bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ee5dfdd301b26 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:56:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:56:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39a3fff700 1 -- 172.21.15.32:0/181484168 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b000cd20 con 0x5576378bd310 2023-09-30T14:56:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39a3fff700 1 -- 172.21.15.32:0/181484168 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f39b009ddf0 con 0x5576378bd310 2023-09-30T14:56:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.990+0000 7f39d7008b80 1 -- 172.21.15.32:0/181484168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39b8021070 con 0x5576378bd310 2023-09-30T14:56:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.991+0000 7f39a3fff700 1 -- 172.21.15.32:0/181484168 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39b000ce80 con 0x5576378bd310 2023-09-30T14:56:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.991+0000 7f39d7008b80 10 client.17013 fetch_fsmap learned FSMap version 51 2023-09-30T14:56:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.991+0000 7f39d7008b80 1 -- 172.21.15.32:0/181484168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f39bc0092e0 con 0x5576378bd310 2023-09-30T14:56:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.991+0000 7f39a3fff700 1 -- 172.21.15.32:0/181484168 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39b8021070 con 0x5576378bd310 2023-09-30T14:56:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.991+0000 7f39a3fff700 1 -- 172.21.15.32:0/181484168 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 51) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f39b009eb30 con 0x5576378bd310 2023-09-30T14:56:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.991+0000 7f39d7008b80 10 client.17013 fetch_fsmap finished waiting for FSMap version 51 2023-09-30T14:56:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.991+0000 7f39d7008b80 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:56:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.991+0000 7f39d7008b80 20 client.17013 populate_metadata read hostname 'smithi032' 2023-09-30T14:56:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.991+0000 7f39d7008b80 1 --2- 172.21.15.32:0/181484168 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55763792ee50 0x557637931270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:56:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.991+0000 7f39d7008b80 4 client.17013 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:56:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.991+0000 7f39d7008b80 1 -- 172.21.15.32:0/181484168 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557637807ad0 con 0x55763792ee50 2023-09-30T14:56:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.991+0000 7f39c8bbc700 1 --2- 172.21.15.32:0/181484168 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55763792ee50 0x557637931270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:56:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.992+0000 7f39a3fff700 10 client.17013 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:54.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:54.992+0000 7f39c8bbc700 1 --2- 172.21.15.32:0/181484168 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55763792ee50 0x557637931270 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd0059704fe3485 server_cookie=85b0e293a9154dd0 in_seq=0 out_seq=0 2023-09-30T14:56:55.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 219886 36.69 MB/sec execute 208 sec latency 61.877 ms 2023-09-30T14:56:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.368+0000 7f39a3fff700 1 -- 172.21.15.32:0/181484168 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557637807ad0 con 0x55763792ee50 2023-09-30T14:56:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.368+0000 7f39a3fff700 10 client.17013 handle_command_reply: tid=0 2023-09-30T14:56:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.368+0000 7f39d7008b80 1 client.17013 shutdown 2023-09-30T14:56:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.369+0000 7f39d7008b80 1 -- 172.21.15.32:0/181484168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bd310 msgr2=0x5576378e9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.369+0000 7f39d7008b80 1 --2- 172.21.15.32:0/181484168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bd310 0x5576378e9c90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f39b00999a0 tx=0x7f39b0095bb0 comp rx=0 tx=0).stop 2023-09-30T14:56:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.369+0000 7f39d7008b80 1 -- 172.21.15.32:0/181484168 shutdown_connections 2023-09-30T14:56:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.369+0000 7f39d7008b80 1 --2- 172.21.15.32:0/181484168 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55763792ee50 0x557637931270 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.369+0000 7f39d7008b80 1 --2- 172.21.15.32:0/181484168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5576378bdcd0 0x55763792ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.369+0000 7f39a3fff700 0 client.17013 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:56:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.369+0000 7f39d7008b80 1 --2- 172.21.15.32:0/181484168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576378bc200 0x5576378e9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.369+0000 7f39d7008b80 1 --2- 172.21.15.32:0/181484168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5576378bd310 0x5576378e9c90 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.369+0000 7f39d7008b80 1 -- 172.21.15.32:0/181484168 >> 172.21.15.32:0/181484168 conn(0x557637830200 msgr2=0x5576378b6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.369+0000 7f39d7008b80 1 -- 172.21.15.32:0/181484168 shutdown_connections 2023-09-30T14:56:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.369+0000 7f39d7008b80 1 -- 172.21.15.32:0/181484168 wait complete. 2023-09-30T14:56:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.369+0000 7f39d7008b80 20 client.17013 trim_cache size 0 max 16384 2023-09-30T14:56:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:56:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:56:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:56:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:56:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.370+0000 7f39cae20700 1 -- 172.21.15.32:0/105634486 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f39b40a1020 msgr2=0x7f39b40a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.370+0000 7f39cae20700 1 --2- 172.21.15.32:0/105634486 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f39b40a1020 0x7f39b40a34e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f39c4100dc0 tx=0x7f39b8009260 comp rx=0 tx=0).stop 2023-09-30T14:56:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.370+0000 7f39cae20700 1 -- 172.21.15.32:0/105634486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c4100f60 msgr2=0x7f39c4196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:56:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.370+0000 7f39cae20700 1 --2- 172.21.15.32:0/105634486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c4100f60 0x7f39c4196250 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f39b000e9f0 tx=0x7f39b000ed00 comp rx=0 tx=0).stop 2023-09-30T14:56:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.370+0000 7f39cae20700 1 -- 172.21.15.32:0/105634486 shutdown_connections 2023-09-30T14:56:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.370+0000 7f39cae20700 1 --2- 172.21.15.32:0/105634486 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f39b40a1020 0x7f39b40a34e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.370+0000 7f39cae20700 1 --2- 172.21.15.32:0/105634486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c4101900 0x7f39c4196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.370+0000 7f39cae20700 1 --2- 172.21.15.32:0/105634486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39c40ffd70 0x7f39c4195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.370+0000 7f39cae20700 1 --2- 172.21.15.32:0/105634486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c4100f60 0x7f39c4196250 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:56:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.370+0000 7f39cae20700 1 -- 172.21.15.32:0/105634486 >> 172.21.15.32:0/105634486 conn(0x7f39c40fb300 msgr2=0x7f39c4104410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:56:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.370+0000 7f39cae20700 1 -- 172.21.15.32:0/105634486 shutdown_connections 2023-09-30T14:56:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:56:55.370+0000 7f39cae20700 1 -- 172.21.15.32:0/105634486 wait complete. 2023-09-30T14:56:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:55 smithi156 ceph-mon[103579]: pgmap v846: 97 pgs: 97 active+clean; 759 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.4 MiB/s wr, 743 op/s 2023-09-30T14:56:55.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:55 smithi148 ceph-mon[102355]: pgmap v846: 97 pgs: 97 active+clean; 759 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.4 MiB/s wr, 743 op/s 2023-09-30T14:56:55.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:55 smithi032 ceph-mon[98592]: pgmap v846: 97 pgs: 97 active+clean; 759 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.4 MiB/s wr, 743 op/s 2023-09-30T14:56:56.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 220349 36.64 MB/sec execute 209 sec latency 237.869 ms 2023-09-30T14:56:57.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 220720 36.55 MB/sec execute 210 sec latency 216.325 ms 2023-09-30T14:56:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:57 smithi156 ceph-mon[103579]: pgmap v847: 97 pgs: 97 active+clean; 764 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.1 MiB/s wr, 716 op/s 2023-09-30T14:56:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:57 smithi148 ceph-mon[102355]: pgmap v847: 97 pgs: 97 active+clean; 764 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.1 MiB/s wr, 716 op/s 2023-09-30T14:56:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:57 smithi032 ceph-mon[98592]: pgmap v847: 97 pgs: 97 active+clean; 764 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.1 MiB/s wr, 716 op/s 2023-09-30T14:56:58.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 221408 36.52 MB/sec execute 211 sec latency 62.245 ms 2023-09-30T14:56:59.039 INFO:tasks.workunit.client.0.smithi032.stdout: 10 222325 36.62 MB/sec execute 212 sec latency 34.389 ms 2023-09-30T14:56:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:56:59 smithi156 ceph-mon[103579]: pgmap v848: 97 pgs: 97 active+clean; 765 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 826 op/s 2023-09-30T14:56:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:56:59 smithi148 ceph-mon[102355]: pgmap v848: 97 pgs: 97 active+clean; 765 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 826 op/s 2023-09-30T14:56:59.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:56:59 smithi032 ceph-mon[98592]: pgmap v848: 97 pgs: 97 active+clean; 765 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 826 op/s 2023-09-30T14:57:00.040 INFO:tasks.workunit.client.0.smithi032.stdout: 10 222800 36.57 MB/sec execute 213 sec latency 291.162 ms 2023-09-30T14:57:01.040 INFO:tasks.workunit.client.0.smithi032.stdout: 10 223501 36.56 MB/sec execute 214 sec latency 164.180 ms 2023-09-30T14:57:01.811 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:01 smithi156 ceph-mon[103579]: pgmap v849: 97 pgs: 97 active+clean; 766 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.6 MiB/s wr, 683 op/s 2023-09-30T14:57:01.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:01 smithi148 ceph-mon[102355]: pgmap v849: 97 pgs: 97 active+clean; 766 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.6 MiB/s wr, 683 op/s 2023-09-30T14:57:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:01 smithi032 ceph-mon[98592]: pgmap v849: 97 pgs: 97 active+clean; 766 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.6 MiB/s wr, 683 op/s 2023-09-30T14:57:02.040 INFO:tasks.workunit.client.0.smithi032.stdout: 10 224032 36.52 MB/sec execute 215 sec latency 283.570 ms 2023-09-30T14:57:03.040 INFO:tasks.workunit.client.0.smithi032.stdout: 10 224695 36.50 MB/sec execute 216 sec latency 86.309 ms 2023-09-30T14:57:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:03 smithi156 ceph-mon[103579]: pgmap v850: 97 pgs: 97 active+clean; 766 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.2 MiB/s wr, 721 op/s 2023-09-30T14:57:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:03 smithi148 ceph-mon[102355]: pgmap v850: 97 pgs: 97 active+clean; 766 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.2 MiB/s wr, 721 op/s 2023-09-30T14:57:03.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:03 smithi032 ceph-mon[98592]: pgmap v850: 97 pgs: 97 active+clean; 766 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.2 MiB/s wr, 721 op/s 2023-09-30T14:57:04.040 INFO:tasks.workunit.client.0.smithi032.stdout: 10 225565 36.57 MB/sec execute 217 sec latency 43.623 ms 2023-09-30T14:57:05.040 INFO:tasks.workunit.client.0.smithi032.stdout: 10 226209 36.57 MB/sec execute 218 sec latency 101.810 ms 2023-09-30T14:57:05.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:05 smithi156 ceph-mon[103579]: pgmap v851: 97 pgs: 97 active+clean; 777 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 837 op/s 2023-09-30T14:57:05.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:05 smithi148 ceph-mon[102355]: pgmap v851: 97 pgs: 97 active+clean; 777 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 837 op/s 2023-09-30T14:57:05.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:05 smithi032 ceph-mon[98592]: pgmap v851: 97 pgs: 97 active+clean; 777 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 837 op/s 2023-09-30T14:57:06.040 INFO:tasks.workunit.client.0.smithi032.stdout: 10 226932 36.57 MB/sec execute 219 sec latency 71.282 ms 2023-09-30T14:57:07.040 INFO:tasks.workunit.client.0.smithi032.stdout: 10 227545 36.54 MB/sec execute 220 sec latency 247.709 ms 2023-09-30T14:57:07.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:07 smithi156 ceph-mon[103579]: pgmap v852: 97 pgs: 97 active+clean; 775 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.5 MiB/s wr, 752 op/s 2023-09-30T14:57:07.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:07 smithi148 ceph-mon[102355]: pgmap v852: 97 pgs: 97 active+clean; 775 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.5 MiB/s wr, 752 op/s 2023-09-30T14:57:07.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:07 smithi032 ceph-mon[98592]: pgmap v852: 97 pgs: 97 active+clean; 775 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.5 MiB/s wr, 752 op/s 2023-09-30T14:57:08.040 INFO:tasks.workunit.client.0.smithi032.stdout: 10 228057 36.51 MB/sec execute 221 sec latency 270.073 ms 2023-09-30T14:57:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.510+0000 7f12ef495700 1 -- 172.21.15.32:0/3155979573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e8101900 msgr2=0x7f12e81081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.510+0000 7f12ef495700 1 --2- 172.21.15.32:0/3155979573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e8101900 0x7f12e81081b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f12e000b0d0 tx=0x7f12e000b3e0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.510+0000 7f12ef495700 1 -- 172.21.15.32:0/3155979573 shutdown_connections 2023-09-30T14:57:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.510+0000 7f12ef495700 1 --2- 172.21.15.32:0/3155979573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e8101900 0x7f12e81081b0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.510+0000 7f12ef495700 1 --2- 172.21.15.32:0/3155979573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e80ffd70 0x7f12e8100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.510+0000 7f12ef495700 1 --2- 172.21.15.32:0/3155979573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12e8100f60 0x7f12e81013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.510+0000 7f12ef495700 1 -- 172.21.15.32:0/3155979573 >> 172.21.15.32:0/3155979573 conn(0x7f12e80fb300 msgr2=0x7f12e80fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.510+0000 7f12ef495700 1 -- 172.21.15.32:0/3155979573 shutdown_connections 2023-09-30T14:57:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.510+0000 7f12ef495700 1 -- 172.21.15.32:0/3155979573 wait complete. 2023-09-30T14:57:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.511+0000 7f12ef495700 1 Processor -- start 2023-09-30T14:57:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.511+0000 7f12ef495700 1 -- start start 2023-09-30T14:57:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.511+0000 7f12ef495700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e80ffd70 0x7f12e819da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.511+0000 7f12ef495700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e8100f60 0x7f12e819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.511+0000 7f12ef495700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12e8101900 0x7f12e819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.511+0000 7f12ef495700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e819ed90 con 0x7f12e8100f60 2023-09-30T14:57:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.511+0000 7f12ef495700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e819ef00 con 0x7f12e80ffd70 2023-09-30T14:57:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.511+0000 7f12ef495700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e819f0c0 con 0x7f12e8101900 2023-09-30T14:57:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.511+0000 7f12eca30700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e8100f60 0x7f12e819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.511+0000 7f12eca30700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e8100f60 0x7f12e819df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39360/0 (socket says 172.21.15.32:39360) 2023-09-30T14:57:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.511+0000 7f12eca30700 1 -- 172.21.15.32:0/1518435824 learned_addr learned my addr 172.21.15.32:0/1518435824 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.511+0000 7f12eda32700 1 --2- 172.21.15.32:0/1518435824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12e8101900 0x7f12e819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.511+0000 7f12ed231700 1 --2- 172.21.15.32:0/1518435824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e80ffd70 0x7f12e819da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.512+0000 7f12eca30700 1 -- 172.21.15.32:0/1518435824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12e8101900 msgr2=0x7f12e819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.512+0000 7f12eca30700 1 --2- 172.21.15.32:0/1518435824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12e8101900 0x7f12e819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.512+0000 7f12eca30700 1 -- 172.21.15.32:0/1518435824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e80ffd70 msgr2=0x7f12e819da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.512+0000 7f12eca30700 1 --2- 172.21.15.32:0/1518435824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e80ffd70 0x7f12e819da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.512+0000 7f12eca30700 1 -- 172.21.15.32:0/1518435824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e0009d00 con 0x7f12e8100f60 2023-09-30T14:57:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.512+0000 7f12eca30700 1 --2- 172.21.15.32:0/1518435824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e8100f60 0x7f12e819df60 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f12d800e9a0 tx=0x7f12d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.512+0000 7f12de7fc700 1 -- 172.21.15.32:0/1518435824 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d8009790 con 0x7f12e8100f60 2023-09-30T14:57:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.512+0000 7f12de7fc700 1 -- 172.21.15.32:0/1518435824 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12d8010430 con 0x7f12e8100f60 2023-09-30T14:57:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.512+0000 7f12de7fc700 1 -- 172.21.15.32:0/1518435824 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d8018680 con 0x7f12e8100f60 2023-09-30T14:57:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.512+0000 7f12ef495700 1 -- 172.21.15.32:0/1518435824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e819f3a0 con 0x7f12e8100f60 2023-09-30T14:57:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.512+0000 7f12ef495700 1 -- 172.21.15.32:0/1518435824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e81a5c40 con 0x7f12e8100f60 2023-09-30T14:57:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.515+0000 7f12de7fc700 1 -- 172.21.15.32:0/1518435824 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f12d80105a0 con 0x7f12e8100f60 2023-09-30T14:57:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.515+0000 7f12de7fc700 1 --2- 172.21.15.32:0/1518435824 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f12d40a1020 0x7f12d40a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.515+0000 7f12de7fc700 1 -- 172.21.15.32:0/1518435824 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f12d8014070 con 0x7f12e8100f60 2023-09-30T14:57:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.515+0000 7f12ed231700 1 --2- 172.21.15.32:0/1518435824 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f12d40a1020 0x7f12d40a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.516+0000 7f12ed231700 1 --2- 172.21.15.32:0/1518435824 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f12d40a1020 0x7f12d40a34e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f12e81086c0 tx=0x7f12e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.516+0000 7f12fb67db80 1 Processor -- start 2023-09-30T14:57:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.516+0000 7f12fb67db80 1 -- start start 2023-09-30T14:57:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.516+0000 7f12fb67db80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d019620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.516+0000 7f12fb67db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01a310 0x55a01d01a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.516+0000 7f12fb67db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d01acd0 0x55a01d021560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.516+0000 7f12fb67db80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01cf81400 con 0x55a01d019200 2023-09-30T14:57:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.516+0000 7f12fb67db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01cf72bc0 con 0x55a01d01acd0 2023-09-30T14:57:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.516+0000 7f12fb67db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01cf704e0 con 0x55a01d01a310 2023-09-30T14:57:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.516+0000 7f12ed231700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d019620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.517+0000 7f12eca30700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01a310 0x55a01d01a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.517+0000 7f12ed231700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d019620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39366/0 (socket says 172.21.15.32:39366) 2023-09-30T14:57:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.517+0000 7f12ed231700 1 -- 172.21.15.32:0/1727465979 learned_addr learned my addr 172.21.15.32:0/1727465979 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.517+0000 7f12eda32700 1 --2- 172.21.15.32:0/1727465979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d01acd0 0x55a01d021560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.517+0000 7f12eca30700 1 -- 172.21.15.32:0/1727465979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d01acd0 msgr2=0x55a01d021560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.517+0000 7f12eca30700 1 --2- 172.21.15.32:0/1727465979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d01acd0 0x55a01d021560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.517+0000 7f12eca30700 1 -- 172.21.15.32:0/1727465979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 msgr2=0x55a01d019620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.517+0000 7f12eca30700 1 --2- 172.21.15.32:0/1727465979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d019620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.517+0000 7f12eca30700 1 -- 172.21.15.32:0/1727465979 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a01cfce870 con 0x55a01d01a310 2023-09-30T14:57:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.517+0000 7f12ed231700 1 --2- 172.21.15.32:0/1727465979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d019620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:57:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.517+0000 7f12eca30700 1 --2- 172.21.15.32:0/1727465979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01a310 0x55a01d01a790 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f12d80ca380 tx=0x7f12d8099950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f6cbb4a3946ab84 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.517+0000 7f12d37fe700 1 -- 172.21.15.32:0/1727465979 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d8099d40 con 0x55a01d01a310 2023-09-30T14:57:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.517+0000 7f12d37fe700 1 -- 172.21.15.32:0/1727465979 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12d809de90 con 0x55a01d01a310 2023-09-30T14:57:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.517+0000 7f12d37fe700 1 -- 172.21.15.32:0/1727465979 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d8099490 con 0x55a01d01a310 2023-09-30T14:57:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.518+0000 7f12fb67db80 1 -- 172.21.15.32:0/1727465979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01a310 msgr2=0x55a01d01a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.518+0000 7f12fb67db80 1 --2- 172.21.15.32:0/1727465979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01a310 0x55a01d01a790 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f12d80ca380 tx=0x7f12d8099950 comp rx=0 tx=0).stop 2023-09-30T14:57:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.518+0000 7f12fb67db80 1 -- 172.21.15.32:0/1727465979 shutdown_connections 2023-09-30T14:57:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.518+0000 7f12fb67db80 1 --2- 172.21.15.32:0/1727465979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d01acd0 0x55a01d021560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.518+0000 7f12fb67db80 1 --2- 172.21.15.32:0/1727465979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d019620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.518+0000 7f12fb67db80 1 --2- 172.21.15.32:0/1727465979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01a310 0x55a01d01a790 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.518+0000 7f12fb67db80 1 -- 172.21.15.32:0/1727465979 >> 172.21.15.32:0/1727465979 conn(0x55a01cf8d200 msgr2=0x55a01d013ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.518+0000 7f12fb67db80 1 -- 172.21.15.32:0/1727465979 shutdown_connections 2023-09-30T14:57:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.518+0000 7f12fb67db80 1 -- 172.21.15.32:0/1727465979 wait complete. 2023-09-30T14:57:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.519+0000 7f12fb67db80 1 -- start start 2023-09-30T14:57:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.519+0000 7f12fb67db80 1 Processor -- start 2023-09-30T14:57:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.519+0000 7f12fb67db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d019200 0x55a01d022df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.519+0000 7f12fb67db80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d01a310 0x55a01d023330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.519+0000 7f12fb67db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01acd0 0x55a01d02a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.519+0000 7f12fb67db80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01cf6bdd0 con 0x55a01d01a310 2023-09-30T14:57:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.519+0000 7f12fb67db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01cf086b0 con 0x55a01d019200 2023-09-30T14:57:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.519+0000 7f12fb67db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01cf08350 con 0x55a01d01acd0 2023-09-30T14:57:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.519+0000 7f12fb67db80 10 client.0 _ll_register_callbacks cb 0x55a01cd657a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:57:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.519+0000 7f12eca30700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d01a310 0x55a01d023330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.519+0000 7f12eca30700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d01a310 0x55a01d023330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39372/0 (socket says 172.21.15.32:39372) 2023-09-30T14:57:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.519+0000 7f12eca30700 1 -- 172.21.15.32:0/2431043537 learned_addr learned my addr 172.21.15.32:0/2431043537 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.520+0000 7f12eda32700 1 --2- 172.21.15.32:0/2431043537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01acd0 0x55a01d02a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.520+0000 7f12ed231700 1 --2- 172.21.15.32:0/2431043537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d019200 0x55a01d022df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.520+0000 7f12eca30700 1 -- 172.21.15.32:0/2431043537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01acd0 msgr2=0x55a01d02a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.520+0000 7f12eca30700 1 --2- 172.21.15.32:0/2431043537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01acd0 0x55a01d02a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.520+0000 7f12eca30700 1 -- 172.21.15.32:0/2431043537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d019200 msgr2=0x55a01d022df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.520+0000 7f12eca30700 1 --2- 172.21.15.32:0/2431043537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d019200 0x55a01d022df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.520+0000 7f12eca30700 1 -- 172.21.15.32:0/2431043537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e4017070 con 0x55a01d01a310 2023-09-30T14:57:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.520+0000 7f12ed231700 1 --2- 172.21.15.32:0/2431043537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d019200 0x55a01d022df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:57:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.520+0000 7f12eca30700 1 --2- 172.21.15.32:0/2431043537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d01a310 0x55a01d023330 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f12d8099cd0 tx=0x7f12d809a500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b215dec2ab6f269 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.520+0000 7f12d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:57:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.520+0000 7f12d0ff9700 1 -- 172.21.15.32:0/2431043537 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80980e0 con 0x55a01d01a310 2023-09-30T14:57:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.520+0000 7f12d0ff9700 1 -- 172.21.15.32:0/2431043537 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f12d8099d40 con 0x55a01d01a310 2023-09-30T14:57:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.520+0000 7f12fb67db80 1 -- 172.21.15.32:0/2431043537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a01d00c090 con 0x55a01d01a310 2023-09-30T14:57:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.520+0000 7f12d0ff9700 1 -- 172.21.15.32:0/2431043537 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12d80a1c40 con 0x55a01d01a310 2023-09-30T14:57:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.520+0000 7f12fb67db80 10 client.17025 fetch_fsmap learned FSMap version 51 2023-09-30T14:57:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.520+0000 7f12fb67db80 1 -- 172.21.15.32:0/2431043537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f12e81a5c40 con 0x55a01d01a310 2023-09-30T14:57:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.521+0000 7f12d0ff9700 1 -- 172.21.15.32:0/2431043537 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a01d00c090 con 0x55a01d01a310 2023-09-30T14:57:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.521+0000 7f12d0ff9700 1 -- 172.21.15.32:0/2431043537 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 51) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f12d809ebd0 con 0x55a01d01a310 2023-09-30T14:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.521+0000 7f12fb67db80 10 client.17025 fetch_fsmap finished waiting for FSMap version 51 2023-09-30T14:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.521+0000 7f12fb67db80 10 client.17025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.521+0000 7f12fb67db80 20 client.17025 populate_metadata read hostname 'smithi032' 2023-09-30T14:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.521+0000 7f12fb67db80 1 --2- 172.21.15.32:0/2431043537 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a01d02e760 0x55a01d030b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.521+0000 7f12fb67db80 4 client.17025 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.521+0000 7f12fb67db80 1 -- 172.21.15.32:0/2431043537 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a01cf64db0 con 0x55a01d02e760 2023-09-30T14:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.521+0000 7f12ed231700 1 --2- 172.21.15.32:0/2431043537 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a01d02e760 0x55a01d030b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.522+0000 7f12ed231700 1 --2- 172.21.15.32:0/2431043537 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a01d02e760 0x55a01d030b50 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61d4ca502b0b17b8 server_cookie=aa2307581ae4e325 in_seq=0 out_seq=0 2023-09-30T14:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.522+0000 7f12d0ff9700 10 client.17025 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.523+0000 7f12d0ff9700 1 -- 172.21.15.32:0/2431043537 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a01cf64db0 con 0x55a01d02e760 2023-09-30T14:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.523+0000 7f12d0ff9700 10 client.17025 handle_command_reply: tid=0 2023-09-30T14:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.523+0000 7f12fb67db80 1 client.17025 shutdown 2023-09-30T14:57:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.523+0000 7f12fb67db80 1 -- 172.21.15.32:0/2431043537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d01a310 msgr2=0x55a01d023330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.523+0000 7f12fb67db80 1 --2- 172.21.15.32:0/2431043537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d01a310 0x55a01d023330 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f12d8099cd0 tx=0x7f12d809a500 comp rx=0 tx=0).stop 2023-09-30T14:57:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.523+0000 7f12fb67db80 1 -- 172.21.15.32:0/2431043537 shutdown_connections 2023-09-30T14:57:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.523+0000 7f12fb67db80 1 --2- 172.21.15.32:0/2431043537 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a01d02e760 0x55a01d030b50 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.523+0000 7f12fb67db80 1 --2- 172.21.15.32:0/2431043537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01acd0 0x55a01d02a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.523+0000 7f12fb67db80 1 --2- 172.21.15.32:0/2431043537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d019200 0x55a01d022df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.523+0000 7f12d0ff9700 0 client.17025 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:57:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.523+0000 7f12fb67db80 1 --2- 172.21.15.32:0/2431043537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d01a310 0x55a01d023330 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.523+0000 7f12fb67db80 1 -- 172.21.15.32:0/2431043537 >> 172.21.15.32:0/2431043537 conn(0x55a01cf8d200 msgr2=0x55a01d013590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.523+0000 7f12fb67db80 1 -- 172.21.15.32:0/2431043537 shutdown_connections 2023-09-30T14:57:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.523+0000 7f12fb67db80 1 -- 172.21.15.32:0/2431043537 wait complete. 2023-09-30T14:57:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.523+0000 7f12fb67db80 20 client.17025 trim_cache size 0 max 16384 2023-09-30T14:57:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.540+0000 7f12fb67db80 1 Processor -- start 2023-09-30T14:57:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.540+0000 7f12fb67db80 1 -- start start 2023-09-30T14:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.541+0000 7f12fb67db80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d013cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.541+0000 7f12fb67db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01a310 0x55a01d02a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.541+0000 7f12fb67db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d01acd0 0x55a01d0210e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.541+0000 7f12fb67db80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01cdec8c0 con 0x55a01d019200 2023-09-30T14:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.541+0000 7f12fb67db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01d0152c0 con 0x55a01d01acd0 2023-09-30T14:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.541+0000 7f12fb67db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01d010ba0 con 0x55a01d01a310 2023-09-30T14:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.541+0000 7f12ed231700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d013cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.541+0000 7f12ed231700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d013cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39380/0 (socket says 172.21.15.32:39380) 2023-09-30T14:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.541+0000 7f12ed231700 1 -- 172.21.15.32:0/1574920778 learned_addr learned my addr 172.21.15.32:0/1574920778 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.541+0000 7f12eca30700 1 --2- 172.21.15.32:0/1574920778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01a310 0x55a01d02a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.541+0000 7f12ed231700 1 -- 172.21.15.32:0/1574920778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01a310 msgr2=0x55a01d02a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.541+0000 7f12ed231700 1 --2- 172.21.15.32:0/1574920778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01a310 0x55a01d02a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.541+0000 7f12ed231700 1 -- 172.21.15.32:0/1574920778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d01acd0 msgr2=0x55a01d0210e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.541+0000 7f12ed231700 1 --2- 172.21.15.32:0/1574920778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d01acd0 0x55a01d0210e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.541+0000 7f12ed231700 1 -- 172.21.15.32:0/1574920778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a01cd96d80 con 0x55a01d019200 2023-09-30T14:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.541+0000 7f12ed231700 1 --2- 172.21.15.32:0/1574920778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d013cc0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009860 tx=0x7f12e4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe31ae18669f799 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.542+0000 7f12d3fff700 1 -- 172.21.15.32:0/1574920778 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e4023040 con 0x55a01d019200 2023-09-30T14:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.542+0000 7f12d3fff700 1 -- 172.21.15.32:0/1574920778 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12e4011ac0 con 0x55a01d019200 2023-09-30T14:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.542+0000 7f12d3fff700 1 -- 172.21.15.32:0/1574920778 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e401d410 con 0x55a01d019200 2023-09-30T14:57:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.542+0000 7f12fb67db80 1 -- 172.21.15.32:0/1574920778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 msgr2=0x55a01d013cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.542+0000 7f12fb67db80 1 --2- 172.21.15.32:0/1574920778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d013cc0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009860 tx=0x7f12e4014000 comp rx=0 tx=0).stop 2023-09-30T14:57:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.542+0000 7f12fb67db80 1 -- 172.21.15.32:0/1574920778 shutdown_connections 2023-09-30T14:57:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.542+0000 7f12fb67db80 1 --2- 172.21.15.32:0/1574920778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d01acd0 0x55a01d0210e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.542+0000 7f12fb67db80 1 --2- 172.21.15.32:0/1574920778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d013cc0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.542+0000 7f12fb67db80 1 --2- 172.21.15.32:0/1574920778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01a310 0x55a01d02a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.542+0000 7f12fb67db80 1 -- 172.21.15.32:0/1574920778 >> 172.21.15.32:0/1574920778 conn(0x55a01cf8d200 msgr2=0x55a01d021b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.542+0000 7f12fb67db80 1 -- 172.21.15.32:0/1574920778 shutdown_connections 2023-09-30T14:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.542+0000 7f12fb67db80 1 -- 172.21.15.32:0/1574920778 wait complete. 2023-09-30T14:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.543+0000 7f12fb67db80 1 -- start start 2023-09-30T14:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.543+0000 7f12fb67db80 1 Processor -- start 2023-09-30T14:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.543+0000 7f12fb67db80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d029de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.543+0000 7f12fb67db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01a310 0x55a01d02a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.543+0000 7f12fb67db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d01acd0 0x55a01d028530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.543+0000 7f12fb67db80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01d028c10 con 0x55a01d019200 2023-09-30T14:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.543+0000 7f12fb67db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01d028d80 con 0x55a01d01acd0 2023-09-30T14:57:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.543+0000 7f12fb67db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01d028f40 con 0x55a01d01a310 2023-09-30T14:57:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.543+0000 7f12fb67db80 10 client.0 _ll_register_callbacks cb 0x7f12e4011a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:57:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.543+0000 7f12ed231700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d029de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.543+0000 7f12ed231700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d029de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39386/0 (socket says 172.21.15.32:39386) 2023-09-30T14:57:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.543+0000 7f12ed231700 1 -- 172.21.15.32:0/818474222 learned_addr learned my addr 172.21.15.32:0/818474222 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.543+0000 7f12eca30700 1 --2- 172.21.15.32:0/818474222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01a310 0x55a01d02a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.544+0000 7f12ed231700 1 -- 172.21.15.32:0/818474222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01a310 msgr2=0x55a01d02a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.544+0000 7f12ed231700 1 --2- 172.21.15.32:0/818474222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01a310 0x55a01d02a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.544+0000 7f12ed231700 1 -- 172.21.15.32:0/818474222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d01acd0 msgr2=0x55a01d028530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.544+0000 7f12ed231700 1 --2- 172.21.15.32:0/818474222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d01acd0 0x55a01d028530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.544+0000 7f12ed231700 1 -- 172.21.15.32:0/818474222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e4004040 con 0x55a01d019200 2023-09-30T14:57:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.544+0000 7f12ed231700 1 --2- 172.21.15.32:0/818474222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d029de0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009340 tx=0x7f12e4011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d92ff79653d907 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.544+0000 7f12d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:57:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.544+0000 7f12d0ff9700 1 -- 172.21.15.32:0/818474222 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e4023030 con 0x55a01d019200 2023-09-30T14:57:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.544+0000 7f12d0ff9700 1 -- 172.21.15.32:0/818474222 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f12e4003c10 con 0x55a01d019200 2023-09-30T14:57:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.544+0000 7f12fb67db80 1 -- 172.21.15.32:0/818474222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a01cf64ad0 con 0x55a01d019200 2023-09-30T14:57:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.544+0000 7f12d0ff9700 1 -- 172.21.15.32:0/818474222 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12e4028980 con 0x55a01d019200 2023-09-30T14:57:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.544+0000 7f12fb67db80 10 client.17031 fetch_fsmap learned FSMap version 51 2023-09-30T14:57:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.544+0000 7f12fb67db80 1 -- 172.21.15.32:0/818474222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55a01cd7fc00 con 0x55a01d019200 2023-09-30T14:57:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.544+0000 7f12d0ff9700 1 -- 172.21.15.32:0/818474222 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12e4041020 con 0x55a01d019200 2023-09-30T14:57:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.545+0000 7f12d0ff9700 1 -- 172.21.15.32:0/818474222 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 51) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f12e4004ab0 con 0x55a01d019200 2023-09-30T14:57:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.545+0000 7f12fb67db80 10 client.17031 fetch_fsmap finished waiting for FSMap version 51 2023-09-30T14:57:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.545+0000 7f12fb67db80 10 client.17031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:57:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.545+0000 7f12fb67db80 20 client.17031 populate_metadata read hostname 'smithi032' 2023-09-30T14:57:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.545+0000 7f12fb67db80 1 --2- 172.21.15.32:0/818474222 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a01d0839f0 0x55a01d085e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.545+0000 7f12fb67db80 4 client.17031 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:57:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.545+0000 7f12fb67db80 1 -- 172.21.15.32:0/818474222 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a01cdb7f10 con 0x55a01d0839f0 2023-09-30T14:57:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.545+0000 7f12eca30700 1 --2- 172.21.15.32:0/818474222 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a01d0839f0 0x55a01d085e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.545+0000 7f12eca30700 1 --2- 172.21.15.32:0/818474222 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a01d0839f0 0x55a01d085e10 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c583062724114c2 server_cookie=43f57653c0823af1 in_seq=0 out_seq=0 2023-09-30T14:57:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.545+0000 7f12d0ff9700 10 client.17031 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:57:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.886+0000 7f12d0ff9700 1 -- 172.21.15.32:0/818474222 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a01cdb7f10 con 0x55a01d0839f0 2023-09-30T14:57:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.886+0000 7f12d0ff9700 10 client.17031 handle_command_reply: tid=0 2023-09-30T14:57:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.886+0000 7f12fb67db80 1 client.17031 shutdown 2023-09-30T14:57:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.887+0000 7f12fb67db80 1 -- 172.21.15.32:0/818474222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 msgr2=0x55a01d029de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.887+0000 7f12fb67db80 1 --2- 172.21.15.32:0/818474222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d029de0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009340 tx=0x7f12e4011bd0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.887+0000 7f12fb67db80 1 -- 172.21.15.32:0/818474222 shutdown_connections 2023-09-30T14:57:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.887+0000 7f12fb67db80 1 --2- 172.21.15.32:0/818474222 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55a01d0839f0 0x55a01d085e10 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.887+0000 7f12fb67db80 1 --2- 172.21.15.32:0/818474222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a01d01acd0 0x55a01d028530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.887+0000 7f12fb67db80 1 --2- 172.21.15.32:0/818474222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a01d019200 0x55a01d029de0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.887+0000 7f12fb67db80 1 --2- 172.21.15.32:0/818474222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a01d01a310 0x55a01d02a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.887+0000 7f12d0ff9700 0 client.17031 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:57:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.887+0000 7f12fb67db80 1 -- 172.21.15.32:0/818474222 >> 172.21.15.32:0/818474222 conn(0x55a01cf8d200 msgr2=0x55a01d013c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.887+0000 7f12fb67db80 1 -- 172.21.15.32:0/818474222 shutdown_connections 2023-09-30T14:57:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.887+0000 7f12fb67db80 1 -- 172.21.15.32:0/818474222 wait complete. 2023-09-30T14:57:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.887+0000 7f12fb67db80 20 client.17031 trim_cache size 0 max 16384 2023-09-30T14:57:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:57:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:57:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:57:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.888+0000 7f12ef495700 1 -- 172.21.15.32:0/1518435824 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f12d40a1020 msgr2=0x7f12d40a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.888+0000 7f12ef495700 1 --2- 172.21.15.32:0/1518435824 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f12d40a1020 0x7f12d40a34e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f12e81086c0 tx=0x7f12e4009260 comp rx=0 tx=0).stop 2023-09-30T14:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.888+0000 7f12ef495700 1 -- 172.21.15.32:0/1518435824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e8100f60 msgr2=0x7f12e819df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.888+0000 7f12ef495700 1 --2- 172.21.15.32:0/1518435824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e8100f60 0x7f12e819df60 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f12d800e9a0 tx=0x7f12d800ecb0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.888+0000 7f12ef495700 1 -- 172.21.15.32:0/1518435824 shutdown_connections 2023-09-30T14:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.888+0000 7f12ef495700 1 --2- 172.21.15.32:0/1518435824 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f12d40a1020 0x7f12d40a34e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.888+0000 7f12ef495700 1 --2- 172.21.15.32:0/1518435824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12e8101900 0x7f12e819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.888+0000 7f12ef495700 1 --2- 172.21.15.32:0/1518435824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12e80ffd70 0x7f12e819da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.888+0000 7f12ef495700 1 --2- 172.21.15.32:0/1518435824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e8100f60 0x7f12e819df60 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.888+0000 7f12ef495700 1 -- 172.21.15.32:0/1518435824 >> 172.21.15.32:0/1518435824 conn(0x7f12e80fb300 msgr2=0x7f12e80fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.888+0000 7f12ef495700 1 -- 172.21.15.32:0/1518435824 shutdown_connections 2023-09-30T14:57:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:08.888+0000 7f12ef495700 1 -- 172.21.15.32:0/1518435824 wait complete. 2023-09-30T14:57:09.040 INFO:tasks.workunit.client.0.smithi032.stdout: 10 228664 36.49 MB/sec execute 222 sec latency 225.126 ms 2023-09-30T14:57:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:09 smithi156 ceph-mon[103579]: pgmap v853: 97 pgs: 97 active+clean; 787 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 832 op/s 2023-09-30T14:57:09.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:09 smithi148 ceph-mon[102355]: pgmap v853: 97 pgs: 97 active+clean; 787 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 832 op/s 2023-09-30T14:57:09.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:09 smithi032 ceph-mon[98592]: pgmap v853: 97 pgs: 97 active+clean; 787 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 832 op/s 2023-09-30T14:57:10.040 INFO:tasks.workunit.client.0.smithi032.stdout: 10 229279 36.50 MB/sec execute 223 sec latency 74.078 ms 2023-09-30T14:57:11.040 INFO:tasks.workunit.client.0.smithi032.stdout: 10 229993 36.48 MB/sec execute 224 sec latency 100.735 ms 2023-09-30T14:57:11.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:11 smithi156 ceph-mon[103579]: pgmap v854: 97 pgs: 97 active+clean; 786 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.5 MiB/s wr, 690 op/s 2023-09-30T14:57:11.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:11 smithi148 ceph-mon[102355]: pgmap v854: 97 pgs: 97 active+clean; 786 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.5 MiB/s wr, 690 op/s 2023-09-30T14:57:11.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:11 smithi032 ceph-mon[98592]: pgmap v854: 97 pgs: 97 active+clean; 786 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.5 MiB/s wr, 690 op/s 2023-09-30T14:57:12.042 INFO:tasks.workunit.client.0.smithi032.stdout: 10 230739 36.49 MB/sec execute 225 sec latency 146.552 ms 2023-09-30T14:57:13.042 INFO:tasks.workunit.client.0.smithi032.stdout: 10 231242 36.45 MB/sec execute 226 sec latency 195.764 ms 2023-09-30T14:57:13.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:13 smithi156 ceph-mon[103579]: pgmap v855: 97 pgs: 97 active+clean; 779 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.5 MiB/s wr, 714 op/s 2023-09-30T14:57:13.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:13 smithi148 ceph-mon[102355]: pgmap v855: 97 pgs: 97 active+clean; 779 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.5 MiB/s wr, 714 op/s 2023-09-30T14:57:13.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:13 smithi032 ceph-mon[98592]: pgmap v855: 97 pgs: 97 active+clean; 779 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.5 MiB/s wr, 714 op/s 2023-09-30T14:57:14.042 INFO:tasks.workunit.client.0.smithi032.stdout: 10 232068 36.51 MB/sec execute 227 sec latency 52.541 ms 2023-09-30T14:57:15.043 INFO:tasks.workunit.client.0.smithi032.stdout: 10 232751 36.49 MB/sec execute 228 sec latency 123.214 ms 2023-09-30T14:57:15.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:15 smithi156 ceph-mon[103579]: pgmap v856: 97 pgs: 97 active+clean; 761 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.6 MiB/s wr, 803 op/s 2023-09-30T14:57:15.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:15 smithi148 ceph-mon[102355]: pgmap v856: 97 pgs: 97 active+clean; 761 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.6 MiB/s wr, 803 op/s 2023-09-30T14:57:15.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:15 smithi032 ceph-mon[98592]: pgmap v856: 97 pgs: 97 active+clean; 761 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.6 MiB/s wr, 803 op/s 2023-09-30T14:57:16.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 233507 36.52 MB/sec execute 229 sec latency 41.597 ms 2023-09-30T14:57:17.044 INFO:tasks.workunit.client.0.smithi032.stdout: 10 234197 36.52 MB/sec execute 230 sec latency 53.083 ms 2023-09-30T14:57:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:17 smithi156 ceph-mon[103579]: pgmap v857: 97 pgs: 97 active+clean; 760 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.9 MiB/s wr, 728 op/s 2023-09-30T14:57:17.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:17 smithi148 ceph-mon[102355]: pgmap v857: 97 pgs: 97 active+clean; 760 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.9 MiB/s wr, 728 op/s 2023-09-30T14:57:17.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:17 smithi032 ceph-mon[98592]: pgmap v857: 97 pgs: 97 active+clean; 760 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.9 MiB/s wr, 728 op/s 2023-09-30T14:57:18.044 INFO:tasks.workunit.client.0.smithi032.stdout: 10 234633 36.45 MB/sec execute 231 sec latency 256.533 ms 2023-09-30T14:57:19.044 INFO:tasks.workunit.client.0.smithi032.stdout: 10 235368 36.48 MB/sec execute 232 sec latency 56.227 ms 2023-09-30T14:57:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:19 smithi156 ceph-mon[103579]: pgmap v858: 97 pgs: 97 active+clean; 770 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 10 MiB/s wr, 799 op/s 2023-09-30T14:57:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:19 smithi148 ceph-mon[102355]: pgmap v858: 97 pgs: 97 active+clean; 770 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 10 MiB/s wr, 799 op/s 2023-09-30T14:57:19.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:19 smithi032 ceph-mon[98592]: pgmap v858: 97 pgs: 97 active+clean; 770 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 10 MiB/s wr, 799 op/s 2023-09-30T14:57:20.044 INFO:tasks.workunit.client.0.smithi032.stdout: 10 236111 36.48 MB/sec execute 233 sec latency 47.065 ms 2023-09-30T14:57:21.044 INFO:tasks.workunit.client.0.smithi032.stdout: 10 236860 36.52 MB/sec execute 234 sec latency 49.944 ms 2023-09-30T14:57:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:21 smithi156 ceph-mon[103579]: pgmap v859: 97 pgs: 97 active+clean; 774 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.6 MiB/s wr, 678 op/s 2023-09-30T14:57:21.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:21 smithi148 ceph-mon[102355]: pgmap v859: 97 pgs: 97 active+clean; 774 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.6 MiB/s wr, 678 op/s 2023-09-30T14:57:21.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:21 smithi032 ceph-mon[98592]: pgmap v859: 97 pgs: 97 active+clean; 774 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.6 MiB/s wr, 678 op/s 2023-09-30T14:57:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.023+0000 7faa3ac7d700 1 -- 172.21.15.32:0/2741013235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa34106bf0 msgr2=0x7faa34109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.023+0000 7faa3ac7d700 1 --2- 172.21.15.32:0/2741013235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa34106bf0 0x7faa34109000 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009f90 tx=0x7faa2c00b330 comp rx=0 tx=0).stop 2023-09-30T14:57:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.023+0000 7faa3ac7d700 1 -- 172.21.15.32:0/2741013235 shutdown_connections 2023-09-30T14:57:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.023+0000 7faa3ac7d700 1 --2- 172.21.15.32:0/2741013235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa34106bf0 0x7faa34109000 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.023+0000 7faa3ac7d700 1 --2- 172.21.15.32:0/2741013235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa34101990 0x7faa34103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.023+0000 7faa3ac7d700 1 --2- 172.21.15.32:0/2741013235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa341042c0 0x7faa341066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.023+0000 7faa3ac7d700 1 -- 172.21.15.32:0/2741013235 >> 172.21.15.32:0/2741013235 conn(0x7faa340fb320 msgr2=0x7faa340fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.023+0000 7faa3ac7d700 1 -- 172.21.15.32:0/2741013235 shutdown_connections 2023-09-30T14:57:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.023+0000 7faa3ac7d700 1 -- 172.21.15.32:0/2741013235 wait complete. 2023-09-30T14:57:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.024+0000 7faa3ac7d700 1 Processor -- start 2023-09-30T14:57:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.024+0000 7faa3ac7d700 1 -- start start 2023-09-30T14:57:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.024+0000 7faa3ac7d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa34101990 0x7faa34196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.024+0000 7faa3ac7d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa341042c0 0x7faa34197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.024+0000 7faa3ac7d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa34106bf0 0x7faa34198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.024+0000 7faa3ac7d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa34198860 con 0x7faa341042c0 2023-09-30T14:57:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.024+0000 7faa3ac7d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa341989d0 con 0x7faa34101990 2023-09-30T14:57:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.024+0000 7faa3ac7d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa34198b90 con 0x7faa34106bf0 2023-09-30T14:57:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.024+0000 7faa33fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa341042c0 0x7faa34197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.024+0000 7faa33fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa341042c0 0x7faa34197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39402/0 (socket says 172.21.15.32:39402) 2023-09-30T14:57:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.024+0000 7faa33fff700 1 -- 172.21.15.32:0/657579600 learned_addr learned my addr 172.21.15.32:0/657579600 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.024+0000 7faa38a19700 1 --2- 172.21.15.32:0/657579600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa34101990 0x7faa34196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.024+0000 7faa3921a700 1 --2- 172.21.15.32:0/657579600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa34106bf0 0x7faa34198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.025+0000 7faa33fff700 1 -- 172.21.15.32:0/657579600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa34106bf0 msgr2=0x7faa34198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.025+0000 7faa33fff700 1 --2- 172.21.15.32:0/657579600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa34106bf0 0x7faa34198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.025+0000 7faa33fff700 1 -- 172.21.15.32:0/657579600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa34101990 msgr2=0x7faa34196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.025+0000 7faa33fff700 1 --2- 172.21.15.32:0/657579600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa34101990 0x7faa34196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.025+0000 7faa33fff700 1 -- 172.21.15.32:0/657579600 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c009c40 con 0x7faa341042c0 2023-09-30T14:57:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.025+0000 7faa33fff700 1 --2- 172.21.15.32:0/657579600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa341042c0 0x7faa34197420 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7faa2000b530 tx=0x7faa2000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.025+0000 7faa31ffb700 1 -- 172.21.15.32:0/657579600 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20011830 con 0x7faa341042c0 2023-09-30T14:57:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.025+0000 7faa31ffb700 1 -- 172.21.15.32:0/657579600 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa20011e50 con 0x7faa341042c0 2023-09-30T14:57:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.025+0000 7faa3ac7d700 1 -- 172.21.15.32:0/657579600 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa34198e70 con 0x7faa341042c0 2023-09-30T14:57:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.025+0000 7faa31ffb700 1 -- 172.21.15.32:0/657579600 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20010e50 con 0x7faa341042c0 2023-09-30T14:57:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.025+0000 7faa3ac7d700 1 -- 172.21.15.32:0/657579600 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa340ff9e0 con 0x7faa341042c0 2023-09-30T14:57:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.028+0000 7faa31ffb700 1 -- 172.21.15.32:0/657579600 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7faa200104f0 con 0x7faa341042c0 2023-09-30T14:57:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.028+0000 7faa31ffb700 1 --2- 172.21.15.32:0/657579600 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7faa240a0fa0 0x7faa240a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.028+0000 7faa31ffb700 1 -- 172.21.15.32:0/657579600 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7faa200cc0d0 con 0x7faa341042c0 2023-09-30T14:57:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.028+0000 7faa38a19700 1 --2- 172.21.15.32:0/657579600 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7faa240a0fa0 0x7faa240a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.029+0000 7faa38a19700 1 --2- 172.21.15.32:0/657579600 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7faa240a0fa0 0x7faa240a3460 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7faa28005fd0 tx=0x7faa28009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.029+0000 7faa46e65b80 1 Processor -- start 2023-09-30T14:57:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.029+0000 7faa46e65b80 1 -- start start 2023-09-30T14:57:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.029+0000 7faa46e65b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7d200 0x55f6ceb7d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.029+0000 7faa46e65b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7e310 0x55f6ceb7e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.029+0000 7faa46e65b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb85560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.029+0000 7faa46e65b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ceae5400 con 0x55f6ceb7ecd0 2023-09-30T14:57:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.029+0000 7faa46e65b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6cead6bc0 con 0x55f6ceb7d200 2023-09-30T14:57:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.029+0000 7faa46e65b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6cead44e0 con 0x55f6ceb7e310 2023-09-30T14:57:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.029+0000 7faa3921a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb85560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.029+0000 7faa3921a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb85560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39412/0 (socket says 172.21.15.32:39412) 2023-09-30T14:57:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.029+0000 7faa38a19700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7d200 0x55f6ceb7d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.029+0000 7faa3921a700 1 -- 172.21.15.32:0/3225383278 learned_addr learned my addr 172.21.15.32:0/3225383278 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.029+0000 7faa33fff700 1 --2- 172.21.15.32:0/3225383278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7e310 0x55f6ceb7e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.030+0000 7faa3921a700 1 -- 172.21.15.32:0/3225383278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7e310 msgr2=0x55f6ceb7e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.030+0000 7faa3921a700 1 --2- 172.21.15.32:0/3225383278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7e310 0x55f6ceb7e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.030+0000 7faa3921a700 1 -- 172.21.15.32:0/3225383278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7d200 msgr2=0x55f6ceb7d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.030+0000 7faa3921a700 1 --2- 172.21.15.32:0/3225383278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7d200 0x55f6ceb7d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.030+0000 7faa3921a700 1 -- 172.21.15.32:0/3225383278 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6ceb32870 con 0x55f6ceb7ecd0 2023-09-30T14:57:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.030+0000 7faa3921a700 1 --2- 172.21.15.32:0/3225383278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb85560 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009470 tx=0x7faa2c0094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daada0af8f6db207 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.030+0000 7faa1effd700 1 -- 172.21.15.32:0/3225383278 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c00e040 con 0x55f6ceb7ecd0 2023-09-30T14:57:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.030+0000 7faa1effd700 1 -- 172.21.15.32:0/3225383278 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa2c0076a0 con 0x55f6ceb7ecd0 2023-09-30T14:57:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.030+0000 7faa1effd700 1 -- 172.21.15.32:0/3225383278 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c00bd70 con 0x55f6ceb7ecd0 2023-09-30T14:57:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.031+0000 7faa46e65b80 1 -- 172.21.15.32:0/3225383278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7ecd0 msgr2=0x55f6ceb85560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.031+0000 7faa46e65b80 1 --2- 172.21.15.32:0/3225383278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb85560 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009470 tx=0x7faa2c0094a0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.031+0000 7faa46e65b80 1 -- 172.21.15.32:0/3225383278 shutdown_connections 2023-09-30T14:57:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.031+0000 7faa46e65b80 1 --2- 172.21.15.32:0/3225383278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb85560 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.031+0000 7faa46e65b80 1 --2- 172.21.15.32:0/3225383278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7d200 0x55f6ceb7d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.031+0000 7faa46e65b80 1 --2- 172.21.15.32:0/3225383278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7e310 0x55f6ceb7e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.031+0000 7faa46e65b80 1 -- 172.21.15.32:0/3225383278 >> 172.21.15.32:0/3225383278 conn(0x55f6ceaf1200 msgr2=0x55f6ceb77ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.031+0000 7faa46e65b80 1 -- 172.21.15.32:0/3225383278 shutdown_connections 2023-09-30T14:57:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.031+0000 7faa46e65b80 1 -- 172.21.15.32:0/3225383278 wait complete. 2023-09-30T14:57:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.031+0000 7faa46e65b80 1 -- start start 2023-09-30T14:57:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.031+0000 7faa46e65b80 1 Processor -- start 2023-09-30T14:57:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa46e65b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7d200 0x55f6ceb86e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa46e65b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb87370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa46e65b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb8e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa46e65b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ceacfdd0 con 0x55f6ceb7e310 2023-09-30T14:57:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa46e65b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6cea6c6b0 con 0x55f6ceb7ecd0 2023-09-30T14:57:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa46e65b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6cea6c350 con 0x55f6ceb7d200 2023-09-30T14:57:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa46e65b80 10 client.0 _ll_register_callbacks cb 0x55f6ce8c97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:57:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa33fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb87370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa38a19700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7d200 0x55f6ceb86e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa38a19700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7d200 0x55f6ceb86e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53796/0 (socket says 172.21.15.32:53796) 2023-09-30T14:57:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa38a19700 1 -- 172.21.15.32:0/4146404055 learned_addr learned my addr 172.21.15.32:0/4146404055 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa3921a700 1 --2- 172.21.15.32:0/4146404055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb8e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa33fff700 1 -- 172.21.15.32:0/4146404055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7d200 msgr2=0x55f6ceb86e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa33fff700 1 --2- 172.21.15.32:0/4146404055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7d200 0x55f6ceb86e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa33fff700 1 -- 172.21.15.32:0/4146404055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7ecd0 msgr2=0x55f6ceb8e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa33fff700 1 --2- 172.21.15.32:0/4146404055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb8e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa33fff700 1 -- 172.21.15.32:0/4146404055 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa34109bf0 con 0x55f6ceb7e310 2023-09-30T14:57:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.032+0000 7faa38a19700 1 --2- 172.21.15.32:0/4146404055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7d200 0x55f6ceb86e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:57:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.033+0000 7faa33fff700 1 --2- 172.21.15.32:0/4146404055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb87370 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7faa20000c00 tx=0x7faa20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f456473847664c99 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.033+0000 7faa13fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:57:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.033+0000 7faa13fff700 1 -- 172.21.15.32:0/4146404055 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20098f00 con 0x55f6ceb7e310 2023-09-30T14:57:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.033+0000 7faa46e65b80 1 -- 172.21.15.32:0/4146404055 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6ceb70090 con 0x55f6ceb7e310 2023-09-30T14:57:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.033+0000 7faa13fff700 1 -- 172.21.15.32:0/4146404055 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7faa20099ca0 con 0x55f6ceb7e310 2023-09-30T14:57:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.033+0000 7faa13fff700 1 -- 172.21.15.32:0/4146404055 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa200995a0 con 0x55f6ceb7e310 2023-09-30T14:57:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.033+0000 7faa46e65b80 10 client.17043 fetch_fsmap learned FSMap version 51 2023-09-30T14:57:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.033+0000 7faa46e65b80 1 -- 172.21.15.32:0/4146404055 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55f6ce8fad80 con 0x55f6ceb7e310 2023-09-30T14:57:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.033+0000 7faa13fff700 1 -- 172.21.15.32:0/4146404055 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6ceb70090 con 0x55f6ceb7e310 2023-09-30T14:57:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.033+0000 7faa13fff700 1 -- 172.21.15.32:0/4146404055 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 51) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faa20098a90 con 0x55f6ceb7e310 2023-09-30T14:57:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.033+0000 7faa46e65b80 10 client.17043 fetch_fsmap finished waiting for FSMap version 51 2023-09-30T14:57:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.034+0000 7faa46e65b80 10 client.17043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:57:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.034+0000 7faa46e65b80 20 client.17043 populate_metadata read hostname 'smithi032' 2023-09-30T14:57:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.034+0000 7faa46e65b80 1 --2- 172.21.15.32:0/4146404055 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f6ceb92c00 0x55f6ceb95020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.034+0000 7faa46e65b80 4 client.17043 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:57:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.034+0000 7faa46e65b80 1 -- 172.21.15.32:0/4146404055 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6ceac8db0 con 0x55f6ceb92c00 2023-09-30T14:57:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.034+0000 7faa38a19700 1 --2- 172.21.15.32:0/4146404055 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f6ceb92c00 0x55f6ceb95020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.034+0000 7faa13fff700 10 client.17043 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:57:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.034+0000 7faa38a19700 1 --2- 172.21.15.32:0/4146404055 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f6ceb92c00 0x55f6ceb95020 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a549ab807a18fe8 server_cookie=a10adde91cb512bb in_seq=0 out_seq=0 2023-09-30T14:57:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.035+0000 7faa13fff700 1 -- 172.21.15.32:0/4146404055 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f6ceac8db0 con 0x55f6ceb92c00 2023-09-30T14:57:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.035+0000 7faa13fff700 10 client.17043 handle_command_reply: tid=0 2023-09-30T14:57:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.035+0000 7faa46e65b80 1 client.17043 shutdown 2023-09-30T14:57:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.035+0000 7faa46e65b80 1 -- 172.21.15.32:0/4146404055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 msgr2=0x55f6ceb87370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.035+0000 7faa46e65b80 1 --2- 172.21.15.32:0/4146404055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb87370 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7faa20000c00 tx=0x7faa20004750 comp rx=0 tx=0).stop 2023-09-30T14:57:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.035+0000 7faa46e65b80 1 -- 172.21.15.32:0/4146404055 shutdown_connections 2023-09-30T14:57:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.035+0000 7faa46e65b80 1 --2- 172.21.15.32:0/4146404055 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f6ceb92c00 0x55f6ceb95020 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.035+0000 7faa46e65b80 1 --2- 172.21.15.32:0/4146404055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb8e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.035+0000 7faa46e65b80 1 --2- 172.21.15.32:0/4146404055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7d200 0x55f6ceb86e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.035+0000 7faa46e65b80 1 --2- 172.21.15.32:0/4146404055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb87370 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.035+0000 7faa46e65b80 1 -- 172.21.15.32:0/4146404055 >> 172.21.15.32:0/4146404055 conn(0x55f6ceaf1200 msgr2=0x55f6ceb77590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.036+0000 7faa13fff700 0 client.17043 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:57:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.036+0000 7faa46e65b80 1 -- 172.21.15.32:0/4146404055 shutdown_connections 2023-09-30T14:57:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.038+0000 7faa46e65b80 1 -- 172.21.15.32:0/4146404055 wait complete. 2023-09-30T14:57:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.038+0000 7faa46e65b80 20 client.17043 trim_cache size 0 max 16384 2023-09-30T14:57:22.044 INFO:tasks.workunit.client.0.smithi032.stdout: 10 237607 36.51 MB/sec execute 235 sec latency 86.600 ms 2023-09-30T14:57:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa46e65b80 1 Processor -- start 2023-09-30T14:57:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa46e65b80 1 -- start start 2023-09-30T14:57:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa46e65b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7d200 0x55f6ceb77ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa46e65b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb8e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa46e65b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb8ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa46e65b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ce9508c0 con 0x55f6ceb7e310 2023-09-30T14:57:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa46e65b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ceb79290 con 0x55f6ceb7d200 2023-09-30T14:57:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa46e65b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ceb92a50 con 0x55f6ceb7ecd0 2023-09-30T14:57:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa33fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb8e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa33fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb8e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39422/0 (socket says 172.21.15.32:39422) 2023-09-30T14:57:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa33fff700 1 -- 172.21.15.32:0/849190930 learned_addr learned my addr 172.21.15.32:0/849190930 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa3921a700 1 --2- 172.21.15.32:0/849190930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb8ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa38a19700 1 --2- 172.21.15.32:0/849190930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7d200 0x55f6ceb77ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa33fff700 1 -- 172.21.15.32:0/849190930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7ecd0 msgr2=0x55f6ceb8ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa33fff700 1 --2- 172.21.15.32:0/849190930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb8ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa33fff700 1 -- 172.21.15.32:0/849190930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7d200 msgr2=0x55f6ceb77ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa33fff700 1 --2- 172.21.15.32:0/849190930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7d200 0x55f6ceb77ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.055+0000 7faa33fff700 1 -- 172.21.15.32:0/849190930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6ce8e3c00 con 0x55f6ceb7e310 2023-09-30T14:57:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.056+0000 7faa33fff700 1 --2- 172.21.15.32:0/849190930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb8e370 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7faa200094b0 tx=0x7faa200c9b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850669adbbf0a8a5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.056+0000 7faa1f7fe700 1 -- 172.21.15.32:0/849190930 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2000bdf0 con 0x55f6ceb7e310 2023-09-30T14:57:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.056+0000 7faa1f7fe700 1 -- 172.21.15.32:0/849190930 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa20098f00 con 0x55f6ceb7e310 2023-09-30T14:57:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.056+0000 7faa1f7fe700 1 -- 172.21.15.32:0/849190930 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20097940 con 0x55f6ceb7e310 2023-09-30T14:57:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.056+0000 7faa46e65b80 1 -- 172.21.15.32:0/849190930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 msgr2=0x55f6ceb8e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.056+0000 7faa46e65b80 1 --2- 172.21.15.32:0/849190930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb8e370 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7faa200094b0 tx=0x7faa200c9b20 comp rx=0 tx=0).stop 2023-09-30T14:57:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.057+0000 7faa46e65b80 1 -- 172.21.15.32:0/849190930 shutdown_connections 2023-09-30T14:57:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.057+0000 7faa46e65b80 1 --2- 172.21.15.32:0/849190930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb8ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.057+0000 7faa46e65b80 1 --2- 172.21.15.32:0/849190930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7d200 0x55f6ceb77ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.057+0000 7faa46e65b80 1 --2- 172.21.15.32:0/849190930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb8e370 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.057+0000 7faa46e65b80 1 -- 172.21.15.32:0/849190930 >> 172.21.15.32:0/849190930 conn(0x55f6ceaf1200 msgr2=0x55f6ceb85ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.057+0000 7faa46e65b80 1 -- 172.21.15.32:0/849190930 shutdown_connections 2023-09-30T14:57:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.057+0000 7faa46e65b80 1 -- 172.21.15.32:0/849190930 wait complete. 2023-09-30T14:57:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.057+0000 7faa46e65b80 1 -- start start 2023-09-30T14:57:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.057+0000 7faa46e65b80 1 Processor -- start 2023-09-30T14:57:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.057+0000 7faa46e65b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7d200 0x55f6ceb8b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.057+0000 7faa46e65b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb8bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.057+0000 7faa46e65b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb8cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.057+0000 7faa46e65b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ceb8d020 con 0x55f6ceb7e310 2023-09-30T14:57:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.057+0000 7faa46e65b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ceb8d190 con 0x55f6ceb7d200 2023-09-30T14:57:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.057+0000 7faa46e65b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ceb8d350 con 0x55f6ceb7ecd0 2023-09-30T14:57:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.057+0000 7faa46e65b80 10 client.0 _ll_register_callbacks cb 0x7faa20084160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:57:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa33fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb8bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa38a19700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7d200 0x55f6ceb8b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa33fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb8bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39430/0 (socket says 172.21.15.32:39430) 2023-09-30T14:57:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa3921a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb8cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa33fff700 1 -- 172.21.15.32:0/1624655711 learned_addr learned my addr 172.21.15.32:0/1624655711 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa33fff700 1 -- 172.21.15.32:0/1624655711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7ecd0 msgr2=0x55f6ceb8cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa33fff700 1 --2- 172.21.15.32:0/1624655711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb8cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa33fff700 1 -- 172.21.15.32:0/1624655711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7d200 msgr2=0x55f6ceb8b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa33fff700 1 --2- 172.21.15.32:0/1624655711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7d200 0x55f6ceb8b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa33fff700 1 -- 172.21.15.32:0/1624655711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6ce8fad80 con 0x55f6ceb7e310 2023-09-30T14:57:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa3921a700 1 --2- 172.21.15.32:0/1624655711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb8cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:57:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa33fff700 1 --2- 172.21.15.32:0/1624655711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb8bad0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7faa200c9f50 tx=0x7faa20095220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d649d30bb85991c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa13fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:57:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa13fff700 1 -- 172.21.15.32:0/1624655711 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20098ca0 con 0x55f6ceb7e310 2023-09-30T14:57:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa13fff700 1 -- 172.21.15.32:0/1624655711 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7faa200995a0 con 0x55f6ceb7e310 2023-09-30T14:57:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa46e65b80 1 -- 172.21.15.32:0/1624655711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa2c0076a0 con 0x55f6ceb7e310 2023-09-30T14:57:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa13fff700 1 -- 172.21.15.32:0/1624655711 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa200a19a0 con 0x55f6ceb7e310 2023-09-30T14:57:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa46e65b80 10 client.17049 fetch_fsmap learned FSMap version 51 2023-09-30T14:57:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.058+0000 7faa46e65b80 1 -- 172.21.15.32:0/1624655711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55f6ce9211f0 con 0x55f6ceb7e310 2023-09-30T14:57:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.059+0000 7faa13fff700 1 -- 172.21.15.32:0/1624655711 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa20053080 con 0x55f6ceb7e310 2023-09-30T14:57:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.059+0000 7faa13fff700 1 -- 172.21.15.32:0/1624655711 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 51) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faa200a1b80 con 0x55f6ceb7e310 2023-09-30T14:57:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.059+0000 7faa46e65b80 10 client.17049 fetch_fsmap finished waiting for FSMap version 51 2023-09-30T14:57:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.059+0000 7faa46e65b80 10 client.17049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:57:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.059+0000 7faa46e65b80 20 client.17049 populate_metadata read hostname 'smithi032' 2023-09-30T14:57:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.059+0000 7faa46e65b80 1 --2- 172.21.15.32:0/1624655711 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f6cebe6a40 0x55f6cebe8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.059+0000 7faa46e65b80 4 client.17049 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:57:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.059+0000 7faa46e65b80 1 -- 172.21.15.32:0/1624655711 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f6ceac8ad0 con 0x55f6cebe6a40 2023-09-30T14:57:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.060+0000 7faa38a19700 1 --2- 172.21.15.32:0/1624655711 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f6cebe6a40 0x55f6cebe8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.061+0000 7faa13fff700 10 client.17049 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:57:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.061+0000 7faa38a19700 1 --2- 172.21.15.32:0/1624655711 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f6cebe6a40 0x55f6cebe8e30 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce8b998b241c031 server_cookie=50527047e74aa5cd in_seq=0 out_seq=0 2023-09-30T14:57:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:57:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:57:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:57:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:57:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.778+0000 7faa13fff700 1 -- 172.21.15.32:0/1624655711 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f6ceac8ad0 con 0x55f6cebe6a40 2023-09-30T14:57:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.778+0000 7faa13fff700 10 client.17049 handle_command_reply: tid=0 2023-09-30T14:57:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.778+0000 7faa46e65b80 1 client.17049 shutdown 2023-09-30T14:57:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.779+0000 7faa46e65b80 1 -- 172.21.15.32:0/1624655711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 msgr2=0x55f6ceb8bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.779+0000 7faa46e65b80 1 --2- 172.21.15.32:0/1624655711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb8bad0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7faa200c9f50 tx=0x7faa20095220 comp rx=0 tx=0).stop 2023-09-30T14:57:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.779+0000 7faa46e65b80 1 -- 172.21.15.32:0/1624655711 shutdown_connections 2023-09-30T14:57:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.779+0000 7faa46e65b80 1 --2- 172.21.15.32:0/1624655711 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55f6cebe6a40 0x55f6cebe8e30 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.779+0000 7faa46e65b80 1 --2- 172.21.15.32:0/1624655711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f6ceb7ecd0 0x55f6ceb8cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.779+0000 7faa46e65b80 1 --2- 172.21.15.32:0/1624655711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f6ceb7d200 0x55f6ceb8b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.779+0000 7faa46e65b80 1 --2- 172.21.15.32:0/1624655711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55f6ceb7e310 0x55f6ceb8bad0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.779+0000 7faa46e65b80 1 -- 172.21.15.32:0/1624655711 >> 172.21.15.32:0/1624655711 conn(0x55f6ceaf1200 msgr2=0x55f6ceb77e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.779+0000 7faa13fff700 0 client.17049 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:57:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.779+0000 7faa46e65b80 1 -- 172.21.15.32:0/1624655711 shutdown_connections 2023-09-30T14:57:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.779+0000 7faa46e65b80 1 -- 172.21.15.32:0/1624655711 wait complete. 2023-09-30T14:57:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.782+0000 7faa46e65b80 20 client.17049 trim_cache size 0 max 16384 2023-09-30T14:57:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.786+0000 7faa3ac7d700 1 -- 172.21.15.32:0/657579600 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7faa240a0fa0 msgr2=0x7faa240a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.786+0000 7faa3ac7d700 1 --2- 172.21.15.32:0/657579600 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7faa240a0fa0 0x7faa240a3460 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7faa28005fd0 tx=0x7faa28009310 comp rx=0 tx=0).stop 2023-09-30T14:57:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.786+0000 7faa3ac7d700 1 -- 172.21.15.32:0/657579600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa341042c0 msgr2=0x7faa34197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.786+0000 7faa3ac7d700 1 --2- 172.21.15.32:0/657579600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa341042c0 0x7faa34197420 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7faa2000b530 tx=0x7faa2000b840 comp rx=0 tx=0).stop 2023-09-30T14:57:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.788+0000 7faa3ac7d700 1 -- 172.21.15.32:0/657579600 shutdown_connections 2023-09-30T14:57:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.788+0000 7faa3ac7d700 1 --2- 172.21.15.32:0/657579600 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7faa240a0fa0 0x7faa240a3460 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.788+0000 7faa3ac7d700 1 --2- 172.21.15.32:0/657579600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa34106bf0 0x7faa34198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.788+0000 7faa3ac7d700 1 --2- 172.21.15.32:0/657579600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa34101990 0x7faa34196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.788+0000 7faa3ac7d700 1 --2- 172.21.15.32:0/657579600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa341042c0 0x7faa34197420 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.788+0000 7faa3ac7d700 1 -- 172.21.15.32:0/657579600 >> 172.21.15.32:0/657579600 conn(0x7faa340fb320 msgr2=0x7faa34102600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.789+0000 7faa3ac7d700 1 -- 172.21.15.32:0/657579600 shutdown_connections 2023-09-30T14:57:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:22.789+0000 7faa3ac7d700 1 -- 172.21.15.32:0/657579600 wait complete. 2023-09-30T14:57:23.044 INFO:tasks.workunit.client.0.smithi032.stdout: 10 237995 36.46 MB/sec execute 236 sec latency 527.086 ms 2023-09-30T14:57:23.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:23 smithi148 ceph-mon[102355]: pgmap v860: 97 pgs: 97 active+clean; 777 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.0 MiB/s wr, 698 op/s 2023-09-30T14:57:23.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:23 smithi032 ceph-mon[98592]: pgmap v860: 97 pgs: 97 active+clean; 777 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.0 MiB/s wr, 698 op/s 2023-09-30T14:57:24.044 INFO:tasks.workunit.client.0.smithi032.stdout: 10 238619 36.45 MB/sec execute 237 sec latency 62.457 ms 2023-09-30T14:57:24.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:23 smithi156 ceph-mon[103579]: pgmap v860: 97 pgs: 97 active+clean; 777 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.0 MiB/s wr, 698 op/s 2023-09-30T14:57:25.044 INFO:tasks.workunit.client.0.smithi032.stdout: 10 239330 36.46 MB/sec execute 238 sec latency 46.938 ms 2023-09-30T14:57:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:25 smithi148 ceph-mon[102355]: pgmap v861: 97 pgs: 97 active+clean; 780 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.6 MiB/s wr, 722 op/s 2023-09-30T14:57:25.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:25 smithi032 ceph-mon[98592]: pgmap v861: 97 pgs: 97 active+clean; 780 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.6 MiB/s wr, 722 op/s 2023-09-30T14:57:26.044 INFO:tasks.workunit.client.0.smithi032.stdout: 10 240230 36.49 MB/sec execute 239 sec latency 55.329 ms 2023-09-30T14:57:26.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:25 smithi156 ceph-mon[103579]: pgmap v861: 97 pgs: 97 active+clean; 780 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.6 MiB/s wr, 722 op/s 2023-09-30T14:57:27.044 INFO:tasks.workunit.client.0.smithi032.stdout: 10 241042 36.54 MB/sec execute 240 sec latency 84.787 ms 2023-09-30T14:57:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:27 smithi148 ceph-mon[102355]: pgmap v862: 97 pgs: 97 active+clean; 781 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.5 MiB/s wr, 720 op/s 2023-09-30T14:57:27.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:27 smithi032 ceph-mon[98592]: pgmap v862: 97 pgs: 97 active+clean; 781 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.5 MiB/s wr, 720 op/s 2023-09-30T14:57:28.044 INFO:tasks.workunit.client.0.smithi032.stdout: 10 241510 36.51 MB/sec execute 241 sec latency 271.136 ms 2023-09-30T14:57:28.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:27 smithi156 ceph-mon[103579]: pgmap v862: 97 pgs: 97 active+clean; 781 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.5 MiB/s wr, 720 op/s 2023-09-30T14:57:29.044 INFO:tasks.workunit.client.0.smithi032.stdout: 10 242061 36.48 MB/sec execute 242 sec latency 63.549 ms 2023-09-30T14:57:29.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:29 smithi148 ceph-mon[102355]: pgmap v863: 97 pgs: 97 active+clean; 764 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 10 MiB/s wr, 779 op/s 2023-09-30T14:57:29.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:29 smithi032 ceph-mon[98592]: pgmap v863: 97 pgs: 97 active+clean; 764 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 10 MiB/s wr, 779 op/s 2023-09-30T14:57:30.044 INFO:tasks.workunit.client.0.smithi032.stdout: 10 242718 36.44 MB/sec execute 243 sec latency 102.347 ms 2023-09-30T14:57:30.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:29 smithi156 ceph-mon[103579]: pgmap v863: 97 pgs: 97 active+clean; 764 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 10 MiB/s wr, 779 op/s 2023-09-30T14:57:31.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 243549 36.47 MB/sec execute 244 sec latency 50.253 ms 2023-09-30T14:57:31.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:31 smithi148 ceph-mon[102355]: pgmap v864: 97 pgs: 97 active+clean; 765 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.2 MiB/s wr, 720 op/s 2023-09-30T14:57:31.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:31 smithi032 ceph-mon[98592]: pgmap v864: 97 pgs: 97 active+clean; 765 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.2 MiB/s wr, 720 op/s 2023-09-30T14:57:32.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 244402 36.52 MB/sec execute 245 sec latency 89.020 ms 2023-09-30T14:57:32.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:31 smithi156 ceph-mon[103579]: pgmap v864: 97 pgs: 97 active+clean; 765 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.2 MiB/s wr, 720 op/s 2023-09-30T14:57:33.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 244869 36.52 MB/sec execute 246 sec latency 293.830 ms 2023-09-30T14:57:33.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:33 smithi148 ceph-mon[102355]: pgmap v865: 97 pgs: 97 active+clean; 772 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.7 MiB/s wr, 784 op/s 2023-09-30T14:57:33.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:33 smithi032 ceph-mon[98592]: pgmap v865: 97 pgs: 97 active+clean; 772 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.7 MiB/s wr, 784 op/s 2023-09-30T14:57:34.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 245320 36.47 MB/sec execute 247 sec latency 67.551 ms 2023-09-30T14:57:34.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:33 smithi156 ceph-mon[103579]: pgmap v865: 97 pgs: 97 active+clean; 772 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.7 MiB/s wr, 784 op/s 2023-09-30T14:57:35.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 245797 36.40 MB/sec execute 248 sec latency 66.736 ms 2023-09-30T14:57:35.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:35 smithi148 ceph-mon[102355]: pgmap v866: 97 pgs: 97 active+clean; 777 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.9 MiB/s wr, 800 op/s 2023-09-30T14:57:35.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:35 smithi032 ceph-mon[98592]: pgmap v866: 97 pgs: 97 active+clean; 777 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.9 MiB/s wr, 800 op/s 2023-09-30T14:57:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.942+0000 7f318f4d7700 1 -- 172.21.15.32:0/2822602493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31880ffd70 msgr2=0x7f3188100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.942+0000 7f318f4d7700 1 --2- 172.21.15.32:0/2822602493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31880ffd70 0x7f3188100190 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f3184009990 tx=0x7f3184009ca0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.942+0000 7f318f4d7700 1 -- 172.21.15.32:0/2822602493 shutdown_connections 2023-09-30T14:57:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.942+0000 7f318f4d7700 1 --2- 172.21.15.32:0/2822602493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3188101900 0x7f31881081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.942+0000 7f318f4d7700 1 --2- 172.21.15.32:0/2822602493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31880ffd70 0x7f3188100190 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.942+0000 7f318f4d7700 1 --2- 172.21.15.32:0/2822602493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3188100f60 0x7f31881013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.942+0000 7f318f4d7700 1 -- 172.21.15.32:0/2822602493 >> 172.21.15.32:0/2822602493 conn(0x7f31880fb2e0 msgr2=0x7f31880fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.942+0000 7f318f4d7700 1 -- 172.21.15.32:0/2822602493 shutdown_connections 2023-09-30T14:57:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.942+0000 7f318f4d7700 1 -- 172.21.15.32:0/2822602493 wait complete. 2023-09-30T14:57:35.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.943+0000 7f318f4d7700 1 Processor -- start 2023-09-30T14:57:35.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.943+0000 7f318f4d7700 1 -- start start 2023-09-30T14:57:35.946 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.943+0000 7f318f4d7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31880ffd70 0x7f3188196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.943+0000 7f318f4d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3188100f60 0x7f31881972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.943+0000 7f318f4d7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3188101900 0x7f3188197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.943+0000 7f318f4d7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31881986f0 con 0x7f3188100f60 2023-09-30T14:57:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.943+0000 7f318f4d7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3188198860 con 0x7f31880ffd70 2023-09-30T14:57:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.943+0000 7f318f4d7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3188198a20 con 0x7f3188101900 2023-09-30T14:57:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.943+0000 7f318ca72700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3188100f60 0x7f31881972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.944+0000 7f318da74700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3188101900 0x7f3188197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.944+0000 7f318ca72700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3188100f60 0x7f31881972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39444/0 (socket says 172.21.15.32:39444) 2023-09-30T14:57:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.944+0000 7f318ca72700 1 -- 172.21.15.32:0/3951607170 learned_addr learned my addr 172.21.15.32:0/3951607170 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.944+0000 7f318d273700 1 --2- 172.21.15.32:0/3951607170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31880ffd70 0x7f3188196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.944+0000 7f318da74700 1 -- 172.21.15.32:0/3951607170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31880ffd70 msgr2=0x7f3188196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.944+0000 7f318da74700 1 --2- 172.21.15.32:0/3951607170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31880ffd70 0x7f3188196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.944+0000 7f318da74700 1 -- 172.21.15.32:0/3951607170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3188100f60 msgr2=0x7f31881972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.944+0000 7f318da74700 1 --2- 172.21.15.32:0/3951607170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3188100f60 0x7f31881972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.944+0000 7f318da74700 1 -- 172.21.15.32:0/3951607170 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3180009c40 con 0x7f3188101900 2023-09-30T14:57:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.944+0000 7f318ca72700 1 --2- 172.21.15.32:0/3951607170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3188100f60 0x7f31881972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:57:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.944+0000 7f318da74700 1 --2- 172.21.15.32:0/3951607170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3188101900 0x7f3188197ff0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f318000efb0 tx=0x7f318000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.944+0000 7f317e7fc700 1 -- 172.21.15.32:0/3951607170 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180003a00 con 0x7f3188101900 2023-09-30T14:57:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.944+0000 7f317e7fc700 1 -- 172.21.15.32:0/3951607170 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f318000f040 con 0x7f3188101900 2023-09-30T14:57:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.944+0000 7f317e7fc700 1 -- 172.21.15.32:0/3951607170 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180004540 con 0x7f3188101900 2023-09-30T14:57:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.944+0000 7f318f4d7700 1 -- 172.21.15.32:0/3951607170 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31840095f0 con 0x7f3188101900 2023-09-30T14:57:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.944+0000 7f318f4d7700 1 -- 172.21.15.32:0/3951607170 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3188198e50 con 0x7f3188101900 2023-09-30T14:57:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.947+0000 7f317e7fc700 1 -- 172.21.15.32:0/3951607170 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f3180003bf0 con 0x7f3188101900 2023-09-30T14:57:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.948+0000 7f317e7fc700 1 --2- 172.21.15.32:0/3951607170 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31740a0fa0 0x7f31740a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.948+0000 7f318d273700 1 --2- 172.21.15.32:0/3951607170 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31740a0fa0 0x7f31740a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.948+0000 7f317e7fc700 1 -- 172.21.15.32:0/3951607170 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f31800cc360 con 0x7f3188101900 2023-09-30T14:57:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.949+0000 7f319b6bfb80 1 Processor -- start 2023-09-30T14:57:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.949+0000 7f318d273700 1 --2- 172.21.15.32:0/3951607170 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31740a0fa0 0x7f31740a3460 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f318400b2e0 tx=0x7f3184005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.949+0000 7f319b6bfb80 1 -- start start 2023-09-30T14:57:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.950+0000 7f319b6bfb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fa610 0x563cb13faa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.950+0000 7f319b6bfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fb720 0x563cb13fbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.950+0000 7f319b6bfb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fc0e0 0x563cb1402970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.950+0000 7f319b6bfb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb135e400 con 0x563cb13fa610 2023-09-30T14:57:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.950+0000 7f319b6bfb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb134fbc0 con 0x563cb13fc0e0 2023-09-30T14:57:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.950+0000 7f319b6bfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb134d4e0 con 0x563cb13fb720 2023-09-30T14:57:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.950+0000 7f318d273700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fa610 0x563cb13faa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.950+0000 7f318d273700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fa610 0x563cb13faa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39452/0 (socket says 172.21.15.32:39452) 2023-09-30T14:57:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.950+0000 7f318d273700 1 -- 172.21.15.32:0/2634006418 learned_addr learned my addr 172.21.15.32:0/2634006418 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.950+0000 7f318ca72700 1 --2- 172.21.15.32:0/2634006418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fb720 0x563cb13fbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.950+0000 7f318da74700 1 --2- 172.21.15.32:0/2634006418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fc0e0 0x563cb1402970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.950+0000 7f318ca72700 1 -- 172.21.15.32:0/2634006418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fc0e0 msgr2=0x563cb1402970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.950+0000 7f318ca72700 1 --2- 172.21.15.32:0/2634006418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fc0e0 0x563cb1402970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.950+0000 7f318ca72700 1 -- 172.21.15.32:0/2634006418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fa610 msgr2=0x563cb13faa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.950+0000 7f318ca72700 1 --2- 172.21.15.32:0/2634006418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fa610 0x563cb13faa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.950+0000 7f318ca72700 1 -- 172.21.15.32:0/2634006418 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cb13ab870 con 0x563cb13fb720 2023-09-30T14:57:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.951+0000 7f318ca72700 1 --2- 172.21.15.32:0/2634006418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fb720 0x563cb13fbba0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3178011570 tx=0x7f3178011880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdddba28585e5fbf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.951+0000 7f31737fe700 1 -- 172.21.15.32:0/2634006418 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178011ea0 con 0x563cb13fb720 2023-09-30T14:57:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.951+0000 7f31737fe700 1 -- 172.21.15.32:0/2634006418 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3178003680 con 0x563cb13fb720 2023-09-30T14:57:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.951+0000 7f31737fe700 1 -- 172.21.15.32:0/2634006418 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31780197e0 con 0x563cb13fb720 2023-09-30T14:57:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.952+0000 7f319b6bfb80 1 -- 172.21.15.32:0/2634006418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fb720 msgr2=0x563cb13fbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.952+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/2634006418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fb720 0x563cb13fbba0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3178011570 tx=0x7f3178011880 comp rx=0 tx=0).stop 2023-09-30T14:57:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.952+0000 7f319b6bfb80 1 -- 172.21.15.32:0/2634006418 shutdown_connections 2023-09-30T14:57:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.952+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/2634006418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fc0e0 0x563cb1402970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.952+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/2634006418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fa610 0x563cb13faa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.952+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/2634006418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fb720 0x563cb13fbba0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.952+0000 7f319b6bfb80 1 -- 172.21.15.32:0/2634006418 >> 172.21.15.32:0/2634006418 conn(0x563cb136a200 msgr2=0x563cb13f0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.952+0000 7f319b6bfb80 1 -- 172.21.15.32:0/2634006418 shutdown_connections 2023-09-30T14:57:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.952+0000 7f319b6bfb80 1 -- 172.21.15.32:0/2634006418 wait complete. 2023-09-30T14:57:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.952+0000 7f319b6bfb80 1 -- start start 2023-09-30T14:57:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.952+0000 7f319b6bfb80 1 Processor -- start 2023-09-30T14:57:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.953+0000 7f319b6bfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fa610 0x563cb1404290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.953+0000 7f319b6bfb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fb720 0x563cb14047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.953+0000 7f319b6bfb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fc0e0 0x563cb140b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.953+0000 7f319b6bfb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb1348dd0 con 0x563cb13fb720 2023-09-30T14:57:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.953+0000 7f319b6bfb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb12e56b0 con 0x563cb13fc0e0 2023-09-30T14:57:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.953+0000 7f319b6bfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb12e5350 con 0x563cb13fa610 2023-09-30T14:57:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.953+0000 7f319b6bfb80 10 client.0 _ll_register_callbacks cb 0x563cb11427a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:57:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.953+0000 7f318d273700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fa610 0x563cb1404290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.953+0000 7f318ca72700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fb720 0x563cb14047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.953+0000 7f318d273700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fa610 0x563cb1404290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53838/0 (socket says 172.21.15.32:53838) 2023-09-30T14:57:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.953+0000 7f318d273700 1 -- 172.21.15.32:0/245358780 learned_addr learned my addr 172.21.15.32:0/245358780 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.953+0000 7f318da74700 1 --2- 172.21.15.32:0/245358780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fc0e0 0x563cb140b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.954+0000 7f318ca72700 1 -- 172.21.15.32:0/245358780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fa610 msgr2=0x563cb1404290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.954+0000 7f318ca72700 1 --2- 172.21.15.32:0/245358780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fa610 0x563cb1404290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.954+0000 7f318ca72700 1 -- 172.21.15.32:0/245358780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fc0e0 msgr2=0x563cb140b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.954+0000 7f318ca72700 1 --2- 172.21.15.32:0/245358780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fc0e0 0x563cb140b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.954+0000 7f318ca72700 1 -- 172.21.15.32:0/245358780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3178009d60 con 0x563cb13fb720 2023-09-30T14:57:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.954+0000 7f318da74700 1 --2- 172.21.15.32:0/245358780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fc0e0 0x563cb140b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:57:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.954+0000 7f318ca72700 1 --2- 172.21.15.32:0/245358780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fb720 0x563cb14047d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3178000c00 tx=0x7f3178004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26eb8f95ca9271f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.954+0000 7f3170ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:57:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.954+0000 7f3170ff9700 1 -- 172.21.15.32:0/245358780 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178005450 con 0x563cb13fb720 2023-09-30T14:57:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.954+0000 7f3170ff9700 1 -- 172.21.15.32:0/245358780 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3178005ce0 con 0x563cb13fb720 2023-09-30T14:57:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.954+0000 7f319b6bfb80 1 -- 172.21.15.32:0/245358780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cb13e9090 con 0x563cb13fb720 2023-09-30T14:57:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.954+0000 7f3170ff9700 1 -- 172.21.15.32:0/245358780 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3178003d60 con 0x563cb13fb720 2023-09-30T14:57:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.954+0000 7f319b6bfb80 10 client.17061 fetch_fsmap learned FSMap version 51 2023-09-30T14:57:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.954+0000 7f319b6bfb80 1 -- 172.21.15.32:0/245358780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x563cb1173d80 con 0x563cb13fb720 2023-09-30T14:57:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.955+0000 7f3170ff9700 1 -- 172.21.15.32:0/245358780 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563cb13e9090 con 0x563cb13fb720 2023-09-30T14:57:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.957+0000 7f3170ff9700 1 -- 172.21.15.32:0/245358780 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 51) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3178021040 con 0x563cb13fb720 2023-09-30T14:57:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.957+0000 7f319b6bfb80 10 client.17061 fetch_fsmap finished waiting for FSMap version 51 2023-09-30T14:57:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.957+0000 7f319b6bfb80 10 client.17061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:57:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.957+0000 7f319b6bfb80 20 client.17061 populate_metadata read hostname 'smithi032' 2023-09-30T14:57:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.957+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/245358780 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563cb140ff30 0x563cb1412350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.957+0000 7f319b6bfb80 4 client.17061 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:57:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.957+0000 7f319b6bfb80 1 -- 172.21.15.32:0/245358780 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cb1341db0 con 0x563cb140ff30 2023-09-30T14:57:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.957+0000 7f318d273700 1 --2- 172.21.15.32:0/245358780 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563cb140ff30 0x563cb1412350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.958+0000 7f318d273700 1 --2- 172.21.15.32:0/245358780 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563cb140ff30 0x563cb1412350 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9b628a0e575f3d1 server_cookie=7ae9fd41c43dc167 in_seq=0 out_seq=0 2023-09-30T14:57:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.958+0000 7f3170ff9700 10 client.17061 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:57:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.958+0000 7f3170ff9700 1 -- 172.21.15.32:0/245358780 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563cb1341db0 con 0x563cb140ff30 2023-09-30T14:57:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.958+0000 7f3170ff9700 10 client.17061 handle_command_reply: tid=0 2023-09-30T14:57:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.958+0000 7f319b6bfb80 1 client.17061 shutdown 2023-09-30T14:57:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.959+0000 7f319b6bfb80 1 -- 172.21.15.32:0/245358780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fb720 msgr2=0x563cb14047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.959+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/245358780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fb720 0x563cb14047d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3178000c00 tx=0x7f3178004280 comp rx=0 tx=0).stop 2023-09-30T14:57:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.959+0000 7f319b6bfb80 1 -- 172.21.15.32:0/245358780 shutdown_connections 2023-09-30T14:57:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.959+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/245358780 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563cb140ff30 0x563cb1412350 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.959+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/245358780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fc0e0 0x563cb140b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.959+0000 7f3170ff9700 0 client.17061 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:57:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.959+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/245358780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fa610 0x563cb1404290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.959+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/245358780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fb720 0x563cb14047d0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.959+0000 7f319b6bfb80 1 -- 172.21.15.32:0/245358780 >> 172.21.15.32:0/245358780 conn(0x563cb136a200 msgr2=0x563cb13f0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.959+0000 7f319b6bfb80 1 -- 172.21.15.32:0/245358780 shutdown_connections 2023-09-30T14:57:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.959+0000 7f319b6bfb80 1 -- 172.21.15.32:0/245358780 wait complete. 2023-09-30T14:57:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.961+0000 7f319b6bfb80 20 client.17061 trim_cache size 0 max 16384 2023-09-30T14:57:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.986+0000 7f319b6bfb80 1 Processor -- start 2023-09-30T14:57:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.986+0000 7f319b6bfb80 1 -- start start 2023-09-30T14:57:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.986+0000 7f319b6bfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fa610 0x563cb13fbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.986+0000 7f319b6bfb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fb720 0x563cb140b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.986+0000 7f319b6bfb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fc0e0 0x563cb14024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.986+0000 7f319b6bfb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb11c98c0 con 0x563cb13fc0e0 2023-09-30T14:57:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.986+0000 7f319b6bfb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb13f1bb0 con 0x563cb13fb720 2023-09-30T14:57:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.986+0000 7f319b6bfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb140fad0 con 0x563cb13fa610 2023-09-30T14:57:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.986+0000 7f318da74700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fc0e0 0x563cb14024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.986+0000 7f318da74700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fc0e0 0x563cb14024d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39466/0 (socket says 172.21.15.32:39466) 2023-09-30T14:57:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.986+0000 7f318da74700 1 -- 172.21.15.32:0/1008127857 learned_addr learned my addr 172.21.15.32:0/1008127857 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.986+0000 7f318d273700 1 --2- 172.21.15.32:0/1008127857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fa610 0x563cb13fbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.987+0000 7f318ca72700 1 --2- 172.21.15.32:0/1008127857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fb720 0x563cb140b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.987+0000 7f318da74700 1 -- 172.21.15.32:0/1008127857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fa610 msgr2=0x563cb13fbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.987+0000 7f318da74700 1 --2- 172.21.15.32:0/1008127857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fa610 0x563cb13fbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.987+0000 7f318da74700 1 -- 172.21.15.32:0/1008127857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fb720 msgr2=0x563cb140b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.987+0000 7f318da74700 1 --2- 172.21.15.32:0/1008127857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fb720 0x563cb140b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.987+0000 7f318da74700 1 -- 172.21.15.32:0/1008127857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cb115cc00 con 0x563cb13fc0e0 2023-09-30T14:57:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.987+0000 7f318da74700 1 --2- 172.21.15.32:0/1008127857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fc0e0 0x563cb14024d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f31800090a0 tx=0x7f3180004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad2f909b66f4a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.987+0000 7f3173fff700 1 -- 172.21.15.32:0/1008127857 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31800998c0 con 0x563cb13fc0e0 2023-09-30T14:57:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.988+0000 7f3173fff700 1 -- 172.21.15.32:0/1008127857 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f318009da10 con 0x563cb13fc0e0 2023-09-30T14:57:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.988+0000 7f3173fff700 1 -- 172.21.15.32:0/1008127857 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180099b20 con 0x563cb13fc0e0 2023-09-30T14:57:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.988+0000 7f319b6bfb80 1 -- 172.21.15.32:0/1008127857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fc0e0 msgr2=0x563cb14024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.988+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/1008127857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fc0e0 0x563cb14024d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f31800090a0 tx=0x7f3180004ad0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.988+0000 7f319b6bfb80 1 -- 172.21.15.32:0/1008127857 shutdown_connections 2023-09-30T14:57:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.988+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/1008127857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fc0e0 0x563cb14024d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.988+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/1008127857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fa610 0x563cb13fbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.988+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/1008127857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fb720 0x563cb140b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.988+0000 7f319b6bfb80 1 -- 172.21.15.32:0/1008127857 >> 172.21.15.32:0/1008127857 conn(0x563cb136a200 msgr2=0x563cb1416cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.989+0000 7f319b6bfb80 1 -- 172.21.15.32:0/1008127857 shutdown_connections 2023-09-30T14:57:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.989+0000 7f319b6bfb80 1 -- 172.21.15.32:0/1008127857 wait complete. 2023-09-30T14:57:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.989+0000 7f319b6bfb80 1 -- start start 2023-09-30T14:57:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.989+0000 7f319b6bfb80 1 Processor -- start 2023-09-30T14:57:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.989+0000 7f319b6bfb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fa610 0x563cb14077f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.989+0000 7f319b6bfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fb720 0x563cb1407d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.989+0000 7f319b6bfb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fc0e0 0x563cb140ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.989+0000 7f319b6bfb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb140b290 con 0x563cb13fc0e0 2023-09-30T14:57:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.989+0000 7f319b6bfb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb140b400 con 0x563cb13fa610 2023-09-30T14:57:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.989+0000 7f319b6bfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb140b5c0 con 0x563cb13fb720 2023-09-30T14:57:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.989+0000 7f319b6bfb80 10 client.0 _ll_register_callbacks cb 0x7f318400b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:57:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.990+0000 7f318d273700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fa610 0x563cb14077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.990+0000 7f318d273700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fa610 0x563cb14077f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:50946/0 (socket says 172.21.15.32:50946) 2023-09-30T14:57:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.990+0000 7f318d273700 1 -- 172.21.15.32:0/4060064453 learned_addr learned my addr 172.21.15.32:0/4060064453 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.990+0000 7f318da74700 1 --2- 172.21.15.32:0/4060064453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fc0e0 0x563cb140ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.990+0000 7f318ca72700 1 --2- 172.21.15.32:0/4060064453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fb720 0x563cb1407d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.990+0000 7f318d273700 1 -- 172.21.15.32:0/4060064453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fb720 msgr2=0x563cb1407d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.990+0000 7f318d273700 1 --2- 172.21.15.32:0/4060064453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fb720 0x563cb1407d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.990+0000 7f318d273700 1 -- 172.21.15.32:0/4060064453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fc0e0 msgr2=0x563cb140ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.990+0000 7f318d273700 1 --2- 172.21.15.32:0/4060064453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fc0e0 0x563cb140ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.990+0000 7f318d273700 1 -- 172.21.15.32:0/4060064453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3188198e50 con 0x563cb13fa610 2023-09-30T14:57:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.990+0000 7f318ca72700 1 --2- 172.21.15.32:0/4060064453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fb720 0x563cb1407d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:57:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.990+0000 7f318da74700 1 --2- 172.21.15.32:0/4060064453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fc0e0 0x563cb140ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:57:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.990+0000 7f318d273700 1 --2- 172.21.15.32:0/4060064453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fa610 0x563cb14077f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f318400b960 tx=0x7f3184009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bacf8463b0be3db8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.990+0000 7f3170ff9700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:57:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.991+0000 7f3170ff9700 1 -- 172.21.15.32:0/4060064453 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318402b040 con 0x563cb13fa610 2023-09-30T14:57:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.991+0000 7f3170ff9700 1 -- 172.21.15.32:0/4060064453 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3184035040 con 0x563cb13fa610 2023-09-30T14:57:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.991+0000 7f319b6bfb80 1 -- 172.21.15.32:0/4060064453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3178003680 con 0x563cb13fa610 2023-09-30T14:57:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.991+0000 7f3170ff9700 1 -- 172.21.15.32:0/4060064453 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3184043030 con 0x563cb13fa610 2023-09-30T14:57:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.991+0000 7f319b6bfb80 10 client.26693 fetch_fsmap learned FSMap version 51 2023-09-30T14:57:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.991+0000 7f319b6bfb80 1 -- 172.21.15.32:0/4060064453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x563cb119a1f0 con 0x563cb13fa610 2023-09-30T14:57:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.991+0000 7f3170ff9700 1 -- 172.21.15.32:0/4060064453 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3178003680 con 0x563cb13fa610 2023-09-30T14:57:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.991+0000 7f3170ff9700 1 -- 172.21.15.32:0/4060064453 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 51) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f318403b020 con 0x563cb13fa610 2023-09-30T14:57:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.991+0000 7f319b6bfb80 10 client.26693 fetch_fsmap finished waiting for FSMap version 51 2023-09-30T14:57:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.991+0000 7f319b6bfb80 10 client.26693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:57:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.991+0000 7f319b6bfb80 20 client.26693 populate_metadata read hostname 'smithi032' 2023-09-30T14:57:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.991+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/4060064453 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563cb14760e0 0x563cb1478500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.991+0000 7f319b6bfb80 4 client.26693 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:57:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.991+0000 7f319b6bfb80 1 -- 172.21.15.32:0/4060064453 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563cb1341ad0 con 0x563cb14760e0 2023-09-30T14:57:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.992+0000 7f318ca72700 1 --2- 172.21.15.32:0/4060064453 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563cb14760e0 0x563cb1478500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.992+0000 7f318ca72700 1 --2- 172.21.15.32:0/4060064453 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563cb14760e0 0x563cb1478500 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e6bbb2a6bec413c server_cookie=3f591385e4bdfb03 in_seq=0 out_seq=0 2023-09-30T14:57:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:35.992+0000 7f3170ff9700 10 client.26693 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:57:36.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 246482 36.37 MB/sec execute 249 sec latency 75.211 ms 2023-09-30T14:57:36.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:35 smithi156 ceph-mon[103579]: pgmap v866: 97 pgs: 97 active+clean; 777 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.9 MiB/s wr, 800 op/s 2023-09-30T14:57:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.348+0000 7f3170ff9700 1 -- 172.21.15.32:0/4060064453 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563cb1341ad0 con 0x563cb14760e0 2023-09-30T14:57:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.348+0000 7f3170ff9700 10 client.26693 handle_command_reply: tid=0 2023-09-30T14:57:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.348+0000 7f319b6bfb80 1 client.26693 shutdown 2023-09-30T14:57:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.349+0000 7f319b6bfb80 1 -- 172.21.15.32:0/4060064453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fa610 msgr2=0x563cb14077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.349+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/4060064453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fa610 0x563cb14077f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f318400b960 tx=0x7f3184009ca0 comp rx=0 tx=0).stop 2023-09-30T14:57:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.352+0000 7f319b6bfb80 1 -- 172.21.15.32:0/4060064453 shutdown_connections 2023-09-30T14:57:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.352+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/4060064453 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563cb14760e0 0x563cb1478500 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.353+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/4060064453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563cb13fc0e0 0x563cb140ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.353+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/4060064453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563cb13fa610 0x563cb14077f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.353+0000 7f319b6bfb80 1 --2- 172.21.15.32:0/4060064453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563cb13fb720 0x563cb1407d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.353+0000 7f319b6bfb80 1 -- 172.21.15.32:0/4060064453 >> 172.21.15.32:0/4060064453 conn(0x563cb136a200 msgr2=0x563cb14024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.353+0000 7f3170ff9700 0 client.26693 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:57:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.353+0000 7f319b6bfb80 1 -- 172.21.15.32:0/4060064453 shutdown_connections 2023-09-30T14:57:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.353+0000 7f319b6bfb80 1 -- 172.21.15.32:0/4060064453 wait complete. 2023-09-30T14:57:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.353+0000 7f319b6bfb80 20 client.26693 trim_cache size 0 max 16384 2023-09-30T14:57:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:57:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:57:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:57:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:57:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.354+0000 7f318f4d7700 1 -- 172.21.15.32:0/3951607170 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31740a0fa0 msgr2=0x7f31740a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.354+0000 7f318f4d7700 1 --2- 172.21.15.32:0/3951607170 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31740a0fa0 0x7f31740a3460 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f318400b2e0 tx=0x7f3184005fb0 comp rx=0 tx=0).stop 2023-09-30T14:57:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.354+0000 7f318f4d7700 1 -- 172.21.15.32:0/3951607170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3188101900 msgr2=0x7f3188197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.354+0000 7f318f4d7700 1 --2- 172.21.15.32:0/3951607170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3188101900 0x7f3188197ff0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f318000efb0 tx=0x7f318000c3a0 comp rx=0 tx=0).stop 2023-09-30T14:57:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.355+0000 7f318f4d7700 1 -- 172.21.15.32:0/3951607170 shutdown_connections 2023-09-30T14:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.355+0000 7f318f4d7700 1 --2- 172.21.15.32:0/3951607170 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31740a0fa0 0x7f31740a3460 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.355+0000 7f318f4d7700 1 --2- 172.21.15.32:0/3951607170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3188101900 0x7f3188197ff0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.355+0000 7f318f4d7700 1 --2- 172.21.15.32:0/3951607170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31880ffd70 0x7f3188196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.355+0000 7f318f4d7700 1 --2- 172.21.15.32:0/3951607170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3188100f60 0x7f31881972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.355+0000 7f318f4d7700 1 -- 172.21.15.32:0/3951607170 >> 172.21.15.32:0/3951607170 conn(0x7f31880fb2e0 msgr2=0x7f3188104490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.355+0000 7f318f4d7700 1 -- 172.21.15.32:0/3951607170 shutdown_connections 2023-09-30T14:57:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:36.355+0000 7f318f4d7700 1 -- 172.21.15.32:0/3951607170 wait complete. 2023-09-30T14:57:37.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 247152 36.36 MB/sec execute 250 sec latency 219.603 ms 2023-09-30T14:57:37.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:37 smithi032 ceph-mon[98592]: pgmap v867: 97 pgs: 97 active+clean; 782 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.9 MiB/s wr, 800 op/s 2023-09-30T14:57:38.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 247762 36.37 MB/sec execute 251 sec latency 243.628 ms 2023-09-30T14:57:38.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:37 smithi156 ceph-mon[103579]: pgmap v867: 97 pgs: 97 active+clean; 782 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.9 MiB/s wr, 800 op/s 2023-09-30T14:57:38.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:37 smithi148 ceph-mon[102355]: pgmap v867: 97 pgs: 97 active+clean; 782 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.9 MiB/s wr, 800 op/s 2023-09-30T14:57:39.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 248336 36.40 MB/sec execute 252 sec latency 113.979 ms 2023-09-30T14:57:39.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:39 smithi032 ceph-mon[98592]: pgmap v868: 97 pgs: 97 active+clean; 778 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 788 op/s 2023-09-30T14:57:40.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 248852 36.36 MB/sec execute 253 sec latency 61.066 ms 2023-09-30T14:57:40.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:39 smithi156 ceph-mon[103579]: pgmap v868: 97 pgs: 97 active+clean; 778 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 788 op/s 2023-09-30T14:57:40.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:39 smithi148 ceph-mon[102355]: pgmap v868: 97 pgs: 97 active+clean; 778 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 788 op/s 2023-09-30T14:57:41.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 249427 36.29 MB/sec execute 254 sec latency 67.607 ms 2023-09-30T14:57:42.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 250246 36.29 MB/sec execute 255 sec latency 47.146 ms 2023-09-30T14:57:42.494 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:41 smithi148 ceph-mon[102355]: pgmap v869: 97 pgs: 97 active+clean; 779 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.0 MiB/s wr, 704 op/s 2023-09-30T14:57:42.494 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:41 smithi156 ceph-mon[103579]: pgmap v869: 97 pgs: 97 active+clean; 779 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.0 MiB/s wr, 704 op/s 2023-09-30T14:57:42.495 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:41 smithi032 ceph-mon[98592]: pgmap v869: 97 pgs: 97 active+clean; 779 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.0 MiB/s wr, 704 op/s 2023-09-30T14:57:43.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 250965 36.32 MB/sec execute 256 sec latency 234.481 ms 2023-09-30T14:57:44.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 251616 36.34 MB/sec execute 257 sec latency 84.313 ms 2023-09-30T14:57:44.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:43 smithi156 ceph-mon[103579]: pgmap v870: 97 pgs: 97 active+clean; 786 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.4 MiB/s wr, 785 op/s 2023-09-30T14:57:44.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:43 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:57:44.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:43 smithi148 ceph-mon[102355]: pgmap v870: 97 pgs: 97 active+clean; 786 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.4 MiB/s wr, 785 op/s 2023-09-30T14:57:44.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:43 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:57:44.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:43 smithi032 ceph-mon[98592]: pgmap v870: 97 pgs: 97 active+clean; 786 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.4 MiB/s wr, 785 op/s 2023-09-30T14:57:44.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:43 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:57:45.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 252156 36.35 MB/sec execute 258 sec latency 57.629 ms 2023-09-30T14:57:45.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:57:45.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:57:45.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:57:45.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:44 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:57:45.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:57:45.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:57:45.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:57:45.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:44 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:57:45.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:57:45.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:57:45.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:57:45.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:44 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-30T14:57:46.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 252713 36.30 MB/sec execute 259 sec latency 44.938 ms 2023-09-30T14:57:46.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:45 smithi156 ceph-mon[103579]: pgmap v871: 97 pgs: 97 active+clean; 789 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.1 MiB/s wr, 728 op/s 2023-09-30T14:57:46.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:45 smithi156 ceph-mon[103579]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-09-30T14:57:46.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:45 smithi156 ceph-mon[103579]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-30T14:57:46.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:45 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:57:46.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:45 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:57:46.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:45 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:57:46.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:45 smithi148 ceph-mon[102355]: pgmap v871: 97 pgs: 97 active+clean; 789 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.1 MiB/s wr, 728 op/s 2023-09-30T14:57:46.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:45 smithi148 ceph-mon[102355]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-09-30T14:57:46.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:45 smithi148 ceph-mon[102355]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-30T14:57:46.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:45 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:57:46.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:45 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:57:46.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:45 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:57:46.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:45 smithi032 ceph-mon[98592]: pgmap v871: 97 pgs: 97 active+clean; 789 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.1 MiB/s wr, 728 op/s 2023-09-30T14:57:46.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:45 smithi032 ceph-mon[98592]: Adjusting osd_memory_target on smithi156 to 196.2M 2023-09-30T14:57:46.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:45 smithi032 ceph-mon[98592]: Unable to set osd_memory_target on smithi156 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-09-30T14:57:46.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:45 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:57:46.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:45 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:57:46.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:45 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:57:47.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 253377 36.26 MB/sec execute 260 sec latency 62.804 ms 2023-09-30T14:57:48.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 253988 36.26 MB/sec execute 261 sec latency 200.894 ms 2023-09-30T14:57:48.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:47 smithi156 ceph-mon[103579]: pgmap v872: 97 pgs: 97 active+clean; 793 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.8 MiB/s wr, 719 op/s 2023-09-30T14:57:48.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:47 smithi148 ceph-mon[102355]: pgmap v872: 97 pgs: 97 active+clean; 793 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.8 MiB/s wr, 719 op/s 2023-09-30T14:57:48.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:47 smithi032 ceph-mon[98592]: pgmap v872: 97 pgs: 97 active+clean; 793 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.8 MiB/s wr, 719 op/s 2023-09-30T14:57:49.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 254753 36.26 MB/sec execute 262 sec latency 139.160 ms 2023-09-30T14:57:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.495+0000 7f4b0a881700 1 -- 172.21.15.32:0/1860334995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b04100e70 msgr2=0x7f4b04101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.495+0000 7f4b0a881700 1 --2- 172.21.15.32:0/1860334995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b04100e70 0x7f4b04101290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4af40098f0 tx=0x7f4af4009c00 comp rx=0 tx=0).stop 2023-09-30T14:57:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.495+0000 7f4b0a881700 1 -- 172.21.15.32:0/1860334995 shutdown_connections 2023-09-30T14:57:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.495+0000 7f4b0a881700 1 --2- 172.21.15.32:0/1860334995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b04102980 0x7f4b041092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.495+0000 7f4b0a881700 1 --2- 172.21.15.32:0/1860334995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b04100e70 0x7f4b04101290 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.495+0000 7f4b0a881700 1 --2- 172.21.15.32:0/1860334995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b04101fc0 0x7f4b04102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.495+0000 7f4b0a881700 1 -- 172.21.15.32:0/1860334995 >> 172.21.15.32:0/1860334995 conn(0x7f4b040fc400 msgr2=0x7f4b040fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.495+0000 7f4b0a881700 1 -- 172.21.15.32:0/1860334995 shutdown_connections 2023-09-30T14:57:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.496+0000 7f4b0a881700 1 -- 172.21.15.32:0/1860334995 wait complete. 2023-09-30T14:57:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.496+0000 7f4b0a881700 1 Processor -- start 2023-09-30T14:57:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.496+0000 7f4b0a881700 1 -- start start 2023-09-30T14:57:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.496+0000 7f4b0a881700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b04100e70 0x7f4b04196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.496+0000 7f4b0a881700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b04101fc0 0x7f4b041972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.496+0000 7f4b0a881700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b04102980 0x7f4b04198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.497+0000 7f4b0a881700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b04198770 con 0x7f4b04100e70 2023-09-30T14:57:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.497+0000 7f4b0a881700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b041988e0 con 0x7f4b04102980 2023-09-30T14:57:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.497+0000 7f4b0a881700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b04198aa0 con 0x7f4b04101fc0 2023-09-30T14:57:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.497+0000 7f4b03fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b04100e70 0x7f4b04196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.497+0000 7f4b03fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b04100e70 0x7f4b04196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39484/0 (socket says 172.21.15.32:39484) 2023-09-30T14:57:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.497+0000 7f4b03fff700 1 -- 172.21.15.32:0/2144119131 learned_addr learned my addr 172.21.15.32:0/2144119131 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.497+0000 7f4b08e1e700 1 --2- 172.21.15.32:0/2144119131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b04102980 0x7f4b04198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.497+0000 7f4b037fe700 1 --2- 172.21.15.32:0/2144119131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b04101fc0 0x7f4b041972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.497+0000 7f4b03fff700 1 -- 172.21.15.32:0/2144119131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b04101fc0 msgr2=0x7f4b041972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.497+0000 7f4b03fff700 1 --2- 172.21.15.32:0/2144119131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b04101fc0 0x7f4b041972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.497+0000 7f4b03fff700 1 -- 172.21.15.32:0/2144119131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b04102980 msgr2=0x7f4b04198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.497+0000 7f4b03fff700 1 --2- 172.21.15.32:0/2144119131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b04102980 0x7f4b04198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.497+0000 7f4b03fff700 1 -- 172.21.15.32:0/2144119131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc009d00 con 0x7f4b04100e70 2023-09-30T14:57:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.497+0000 7f4b037fe700 1 --2- 172.21.15.32:0/2144119131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b04101fc0 0x7f4b041972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:57:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.497+0000 7f4b03fff700 1 --2- 172.21.15.32:0/2144119131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b04100e70 0x7f4b04196d80 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009fd0 tx=0x7f4af400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.497+0000 7f4b017fa700 1 -- 172.21.15.32:0/2144119131 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af401c070 con 0x7f4b04100e70 2023-09-30T14:57:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.498+0000 7f4b017fa700 1 -- 172.21.15.32:0/2144119131 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4af400f820 con 0x7f4b04100e70 2023-09-30T14:57:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.498+0000 7f4b0a881700 1 -- 172.21.15.32:0/2144119131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af4009520 con 0x7f4b04100e70 2023-09-30T14:57:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.498+0000 7f4b017fa700 1 -- 172.21.15.32:0/2144119131 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4004b00 con 0x7f4b04100e70 2023-09-30T14:57:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.498+0000 7f4b0a881700 1 -- 172.21.15.32:0/2144119131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b04073050 con 0x7f4b04100e70 2023-09-30T14:57:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.501+0000 7f4b017fa700 1 -- 172.21.15.32:0/2144119131 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f4af4020070 con 0x7f4b04100e70 2023-09-30T14:57:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.501+0000 7f4b017fa700 1 --2- 172.21.15.32:0/2144119131 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4aec0a0fa0 0x7f4aec0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.502+0000 7f4b017fa700 1 -- 172.21.15.32:0/2144119131 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4af40cd4b0 con 0x7f4b04100e70 2023-09-30T14:57:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.502+0000 7f4b16a69b80 1 Processor -- start 2023-09-30T14:57:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.502+0000 7f4b037fe700 1 --2- 172.21.15.32:0/2144119131 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4aec0a0fa0 0x7f4aec0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.502+0000 7f4b16a69b80 1 -- start start 2023-09-30T14:57:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.502+0000 7f4b16a69b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b23a10 0x563c90b23e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.502+0000 7f4b16a69b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b24b20 0x563c90b24fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.502+0000 7f4b16a69b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b254e0 0x563c90b2bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.502+0000 7f4b16a69b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c90a83400 con 0x563c90b254e0 2023-09-30T14:57:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.502+0000 7f4b16a69b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c90a74bc0 con 0x563c90b24b20 2023-09-30T14:57:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.502+0000 7f4b16a69b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c90a724e0 con 0x563c90b23a10 2023-09-30T14:57:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.502+0000 7f4b037fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b24b20 0x563c90b24fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.502+0000 7f4b08e1e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b254e0 0x563c90b2bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.502+0000 7f4b03fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b23a10 0x563c90b23e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.502+0000 7f4b08e1e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b254e0 0x563c90b2bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39496/0 (socket says 172.21.15.32:39496) 2023-09-30T14:57:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.502+0000 7f4b08e1e700 1 -- 172.21.15.32:0/3979770609 learned_addr learned my addr 172.21.15.32:0/3979770609 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.503+0000 7f4b037fe700 1 --2- 172.21.15.32:0/2144119131 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4aec0a0fa0 0x7f4aec0a3460 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4af800ffd0 tx=0x7f4af800f820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.503+0000 7f4b08e1e700 1 -- 172.21.15.32:0/3979770609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b23a10 msgr2=0x563c90b23e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.503+0000 7f4b08e1e700 1 --2- 172.21.15.32:0/3979770609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b23a10 0x563c90b23e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.503+0000 7f4b08e1e700 1 -- 172.21.15.32:0/3979770609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b24b20 msgr2=0x563c90b24fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.503+0000 7f4b08e1e700 1 --2- 172.21.15.32:0/3979770609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b24b20 0x563c90b24fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.503+0000 7f4b08e1e700 1 -- 172.21.15.32:0/3979770609 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c90ad0870 con 0x563c90b254e0 2023-09-30T14:57:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.503+0000 7f4b03fff700 1 --2- 172.21.15.32:0/3979770609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b23a10 0x563c90b23e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:57:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.503+0000 7f4b08e1e700 1 --2- 172.21.15.32:0/3979770609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b254e0 0x563c90b2bd70 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009480 tx=0x7f4afc0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1243e69206d7e6d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.503+0000 7f4aea7fc700 1 -- 172.21.15.32:0/3979770609 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc00e070 con 0x563c90b254e0 2023-09-30T14:57:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.503+0000 7f4aea7fc700 1 -- 172.21.15.32:0/3979770609 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4afc00f040 con 0x563c90b254e0 2023-09-30T14:57:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.503+0000 7f4aea7fc700 1 -- 172.21.15.32:0/3979770609 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc015610 con 0x563c90b254e0 2023-09-30T14:57:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.504+0000 7f4b16a69b80 1 -- 172.21.15.32:0/3979770609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b254e0 msgr2=0x563c90b2bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.504+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/3979770609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b254e0 0x563c90b2bd70 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009480 tx=0x7f4afc0099e0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.504+0000 7f4b16a69b80 1 -- 172.21.15.32:0/3979770609 shutdown_connections 2023-09-30T14:57:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.504+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/3979770609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b254e0 0x563c90b2bd70 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.504+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/3979770609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b23a10 0x563c90b23e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.504+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/3979770609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b24b20 0x563c90b24fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.504+0000 7f4b16a69b80 1 -- 172.21.15.32:0/3979770609 >> 172.21.15.32:0/3979770609 conn(0x563c90a8f200 msgr2=0x563c90b15ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.504+0000 7f4b16a69b80 1 -- 172.21.15.32:0/3979770609 shutdown_connections 2023-09-30T14:57:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.504+0000 7f4b16a69b80 1 -- 172.21.15.32:0/3979770609 wait complete. 2023-09-30T14:57:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.505+0000 7f4b16a69b80 1 -- start start 2023-09-30T14:57:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.505+0000 7f4b16a69b80 1 Processor -- start 2023-09-30T14:57:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.505+0000 7f4b16a69b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b23a10 0x563c90b1c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.505+0000 7f4b16a69b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b24b20 0x563c90b1c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.505+0000 7f4b16a69b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b254e0 0x563c90b21910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.505+0000 7f4b16a69b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c90a6ddd0 con 0x563c90b254e0 2023-09-30T14:57:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.505+0000 7f4b16a69b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c90a0a6b0 con 0x563c90b24b20 2023-09-30T14:57:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.505+0000 7f4b16a69b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c90a0a350 con 0x563c90b23a10 2023-09-30T14:57:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.505+0000 7f4b16a69b80 10 client.0 _ll_register_callbacks cb 0x563c908677a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:57:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.505+0000 7f4b03fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b23a10 0x563c90b1c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.505+0000 7f4b03fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b23a10 0x563c90b1c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53880/0 (socket says 172.21.15.32:53880) 2023-09-30T14:57:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.505+0000 7f4b03fff700 1 -- 172.21.15.32:0/544059225 learned_addr learned my addr 172.21.15.32:0/544059225 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.505+0000 7f4b037fe700 1 --2- 172.21.15.32:0/544059225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b24b20 0x563c90b1c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.505+0000 7f4b08e1e700 1 --2- 172.21.15.32:0/544059225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b254e0 0x563c90b21910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.506+0000 7f4b03fff700 1 -- 172.21.15.32:0/544059225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b24b20 msgr2=0x563c90b1c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.506+0000 7f4b03fff700 1 --2- 172.21.15.32:0/544059225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b24b20 0x563c90b1c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.506+0000 7f4b03fff700 1 -- 172.21.15.32:0/544059225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b254e0 msgr2=0x563c90b21910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.506+0000 7f4b03fff700 1 --2- 172.21.15.32:0/544059225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b254e0 0x563c90b21910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.506+0000 7f4b03fff700 1 -- 172.21.15.32:0/544059225 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b04073050 con 0x563c90b23a10 2023-09-30T14:57:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.506+0000 7f4b03fff700 1 --2- 172.21.15.32:0/544059225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b23a10 0x563c90b1c3a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4af40cafc0 tx=0x7f4af4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18dac72c2329b383 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.506+0000 7f4ae37fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:57:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.506+0000 7f4ae37fe700 1 -- 172.21.15.32:0/544059225 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af409eb60 con 0x563c90b23a10 2023-09-30T14:57:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.506+0000 7f4ae37fe700 1 -- 172.21.15.32:0/544059225 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4af40a2d10 con 0x563c90b23a10 2023-09-30T14:57:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.506+0000 7f4b16a69b80 1 -- 172.21.15.32:0/544059225 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c90b0e090 con 0x563c90b23a10 2023-09-30T14:57:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.507+0000 7f4ae37fe700 1 -- 172.21.15.32:0/544059225 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4af40ba070 con 0x563c90b23a10 2023-09-30T14:57:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.507+0000 7f4b16a69b80 10 client.26737 fetch_fsmap learned FSMap version 51 2023-09-30T14:57:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.507+0000 7f4b16a69b80 1 -- 172.21.15.32:0/544059225 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f4afc009130 con 0x563c90b23a10 2023-09-30T14:57:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.507+0000 7f4ae37fe700 1 -- 172.21.15.32:0/544059225 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563c90b0e090 con 0x563c90b23a10 2023-09-30T14:57:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.507+0000 7f4ae37fe700 1 -- 172.21.15.32:0/544059225 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 51) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4af409ad00 con 0x563c90b23a10 2023-09-30T14:57:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.507+0000 7f4b16a69b80 10 client.26737 fetch_fsmap finished waiting for FSMap version 51 2023-09-30T14:57:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.507+0000 7f4b16a69b80 10 client.26737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:57:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.507+0000 7f4b16a69b80 20 client.26737 populate_metadata read hostname 'smithi032' 2023-09-30T14:57:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.507+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/544059225 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563c90b418c0 0x563c90b43cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.507+0000 7f4b16a69b80 4 client.26737 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:57:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.507+0000 7f4b16a69b80 1 -- 172.21.15.32:0/544059225 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c90a66db0 con 0x563c90b418c0 2023-09-30T14:57:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.507+0000 7f4b037fe700 1 --2- 172.21.15.32:0/544059225 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563c90b418c0 0x563c90b43cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.508+0000 7f4b037fe700 1 --2- 172.21.15.32:0/544059225 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563c90b418c0 0x563c90b43cb0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c51c6db0865c6132 server_cookie=90fef6f4e2152218 in_seq=0 out_seq=0 2023-09-30T14:57:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.508+0000 7f4ae37fe700 10 client.26737 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:57:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.510+0000 7f4ae37fe700 1 -- 172.21.15.32:0/544059225 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563c90a66db0 con 0x563c90b418c0 2023-09-30T14:57:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.510+0000 7f4ae37fe700 10 client.26737 handle_command_reply: tid=0 2023-09-30T14:57:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.510+0000 7f4b16a69b80 1 client.26737 shutdown 2023-09-30T14:57:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.510+0000 7f4b16a69b80 1 -- 172.21.15.32:0/544059225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b23a10 msgr2=0x563c90b1c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.510+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/544059225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b23a10 0x563c90b1c3a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4af40cafc0 tx=0x7f4af4009c00 comp rx=0 tx=0).stop 2023-09-30T14:57:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.510+0000 7f4b16a69b80 1 -- 172.21.15.32:0/544059225 shutdown_connections 2023-09-30T14:57:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.510+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/544059225 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563c90b418c0 0x563c90b43cb0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.510+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/544059225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b254e0 0x563c90b21910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.510+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/544059225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b23a10 0x563c90b1c3a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.510+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/544059225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b24b20 0x563c90b1c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.510+0000 7f4b16a69b80 1 -- 172.21.15.32:0/544059225 >> 172.21.15.32:0/544059225 conn(0x563c90a8f200 msgr2=0x563c90b15590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.510+0000 7f4ae37fe700 0 client.26737 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:57:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.511+0000 7f4b16a69b80 1 -- 172.21.15.32:0/544059225 shutdown_connections 2023-09-30T14:57:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.511+0000 7f4b16a69b80 1 -- 172.21.15.32:0/544059225 wait complete. 2023-09-30T14:57:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.511+0000 7f4b16a69b80 20 client.26737 trim_cache size 0 max 16384 2023-09-30T14:57:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.531+0000 7f4b16a69b80 1 Processor -- start 2023-09-30T14:57:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.534+0000 7f4b16a69b80 1 -- start start 2023-09-30T14:57:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.534+0000 7f4b16a69b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b23a10 0x563c90b218d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.534+0000 7f4b16a69b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b24b20 0x563c90b514b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.534+0000 7f4b16a69b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b254e0 0x563c90b438d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.534+0000 7f4b16a69b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c908ee8c0 con 0x563c90b23a10 2023-09-30T14:57:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.534+0000 7f4b16a69b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c90b24290 con 0x563c90b254e0 2023-09-30T14:57:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.534+0000 7f4b16a69b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c90b1bf10 con 0x563c90b24b20 2023-09-30T14:57:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.534+0000 7f4b037fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b24b20 0x563c90b514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.534+0000 7f4b037fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b24b20 0x563c90b514b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53890/0 (socket says 172.21.15.32:53890) 2023-09-30T14:57:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.534+0000 7f4b037fe700 1 -- 172.21.15.32:0/2070568129 learned_addr learned my addr 172.21.15.32:0/2070568129 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.534+0000 7f4b08e1e700 1 --2- 172.21.15.32:0/2070568129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b254e0 0x563c90b438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.535+0000 7f4b037fe700 1 -- 172.21.15.32:0/2070568129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b254e0 msgr2=0x563c90b438d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.535+0000 7f4b03fff700 1 --2- 172.21.15.32:0/2070568129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b23a10 0x563c90b218d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.535+0000 7f4b037fe700 1 --2- 172.21.15.32:0/2070568129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b254e0 0x563c90b438d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.535+0000 7f4b037fe700 1 -- 172.21.15.32:0/2070568129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b23a10 msgr2=0x563c90b218d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.535+0000 7f4b037fe700 1 --2- 172.21.15.32:0/2070568129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b23a10 0x563c90b218d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.535+0000 7f4b037fe700 1 -- 172.21.15.32:0/2070568129 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c90898d80 con 0x563c90b24b20 2023-09-30T14:57:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.535+0000 7f4b03fff700 1 --2- 172.21.15.32:0/2070568129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b23a10 0x563c90b218d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T14:57:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.535+0000 7f4b037fe700 1 --2- 172.21.15.32:0/2070568129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b24b20 0x563c90b514b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4af801c760 tx=0x7f4af801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c0b08db211896 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.537+0000 7f4aeaffd700 1 -- 172.21.15.32:0/2070568129 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8021030 con 0x563c90b24b20 2023-09-30T14:57:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.538+0000 7f4aeaffd700 1 -- 172.21.15.32:0/2070568129 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4af8022040 con 0x563c90b24b20 2023-09-30T14:57:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.538+0000 7f4aeaffd700 1 -- 172.21.15.32:0/2070568129 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8014040 con 0x563c90b24b20 2023-09-30T14:57:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.539+0000 7f4b16a69b80 1 -- 172.21.15.32:0/2070568129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b24b20 msgr2=0x563c90b514b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.539+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/2070568129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b24b20 0x563c90b514b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4af801c760 tx=0x7f4af801b000 comp rx=0 tx=0).stop 2023-09-30T14:57:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.539+0000 7f4b16a69b80 1 -- 172.21.15.32:0/2070568129 shutdown_connections 2023-09-30T14:57:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.539+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/2070568129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b254e0 0x563c90b438d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.539+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/2070568129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b23a10 0x563c90b218d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.539+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/2070568129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b24b20 0x563c90b514b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.539+0000 7f4b16a69b80 1 -- 172.21.15.32:0/2070568129 >> 172.21.15.32:0/2070568129 conn(0x563c90a8f200 msgr2=0x563c90b1b050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.539+0000 7f4b16a69b80 1 -- 172.21.15.32:0/2070568129 shutdown_connections 2023-09-30T14:57:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.539+0000 7f4b16a69b80 1 -- 172.21.15.32:0/2070568129 wait complete. 2023-09-30T14:57:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.539+0000 7f4b16a69b80 1 -- start start 2023-09-30T14:57:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.540+0000 7f4b16a69b80 1 Processor -- start 2023-09-30T14:57:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.540+0000 7f4b16a69b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b23a10 0x563c90b90f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.540+0000 7f4b16a69b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b24b20 0x563c90b914c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.540+0000 7f4b16a69b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b254e0 0x563c90b4fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.540+0000 7f4b16a69b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c90b50090 con 0x563c90b23a10 2023-09-30T14:57:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.540+0000 7f4b16a69b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c90b50200 con 0x563c90b254e0 2023-09-30T14:57:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.540+0000 7f4b16a69b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c90b503c0 con 0x563c90b24b20 2023-09-30T14:57:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.540+0000 7f4b16a69b80 10 client.0 _ll_register_callbacks cb 0x7f4af409a640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:57:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.540+0000 7f4b03fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b23a10 0x563c90b90f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.540+0000 7f4b03fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b23a10 0x563c90b90f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39512/0 (socket says 172.21.15.32:39512) 2023-09-30T14:57:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.540+0000 7f4b03fff700 1 -- 172.21.15.32:0/4188827632 learned_addr learned my addr 172.21.15.32:0/4188827632 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:57:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.540+0000 7f4b037fe700 1 --2- 172.21.15.32:0/4188827632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b24b20 0x563c90b914c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.541+0000 7f4b08e1e700 1 --2- 172.21.15.32:0/4188827632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b254e0 0x563c90b4fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.541+0000 7f4b03fff700 1 -- 172.21.15.32:0/4188827632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b24b20 msgr2=0x563c90b914c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.541+0000 7f4b03fff700 1 --2- 172.21.15.32:0/4188827632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b24b20 0x563c90b914c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.541+0000 7f4b03fff700 1 -- 172.21.15.32:0/4188827632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b254e0 msgr2=0x563c90b4fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:57:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.541+0000 7f4b03fff700 1 --2- 172.21.15.32:0/4188827632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b254e0 0x563c90b4fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.541+0000 7f4b03fff700 1 -- 172.21.15.32:0/4188827632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af801c410 con 0x563c90b23a10 2023-09-30T14:57:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.541+0000 7f4b03fff700 1 --2- 172.21.15.32:0/4188827632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b23a10 0x563c90b90f80 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4af400fa40 tx=0x7f4af4098fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95071d71b8223d5d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:57:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.541+0000 7f4ae37fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:57:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.541+0000 7f4ae37fe700 1 -- 172.21.15.32:0/4188827632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4096d50 con 0x563c90b23a10 2023-09-30T14:57:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.541+0000 7f4ae37fe700 1 -- 172.21.15.32:0/4188827632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4af40a2d10 con 0x563c90b23a10 2023-09-30T14:57:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.541+0000 7f4b16a69b80 1 -- 172.21.15.32:0/4188827632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4afc00f040 con 0x563c90b23a10 2023-09-30T14:57:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.542+0000 7f4ae37fe700 1 -- 172.21.15.32:0/4188827632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4af409a140 con 0x563c90b23a10 2023-09-30T14:57:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.542+0000 7f4b16a69b80 10 client.17082 fetch_fsmap learned FSMap version 51 2023-09-30T14:57:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.542+0000 7f4b16a69b80 1 -- 172.21.15.32:0/4188827632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x563c90881c00 con 0x563c90b23a10 2023-09-30T14:57:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.542+0000 7f4ae37fe700 1 -- 172.21.15.32:0/4188827632 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4afc00f040 con 0x563c90b23a10 2023-09-30T14:57:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.542+0000 7f4ae37fe700 1 -- 172.21.15.32:0/4188827632 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 51) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4af40a2980 con 0x563c90b23a10 2023-09-30T14:57:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.542+0000 7f4b16a69b80 10 client.17082 fetch_fsmap finished waiting for FSMap version 51 2023-09-30T14:57:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.542+0000 7f4b16a69b80 10 client.17082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:57:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.542+0000 7f4b16a69b80 20 client.17082 populate_metadata read hostname 'smithi032' 2023-09-30T14:57:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.542+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/4188827632 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563c90ba6ec0 0x563c90ba92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:57:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.542+0000 7f4b16a69b80 4 client.17082 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:57:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.542+0000 7f4b16a69b80 1 -- 172.21.15.32:0/4188827632 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563c90a66ad0 con 0x563c90ba6ec0 2023-09-30T14:57:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.542+0000 7f4b037fe700 1 --2- 172.21.15.32:0/4188827632 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563c90ba6ec0 0x563c90ba92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:57:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.543+0000 7f4ae37fe700 10 client.17082 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:57:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.543+0000 7f4b037fe700 1 --2- 172.21.15.32:0/4188827632 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563c90ba6ec0 0x563c90ba92b0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa55a4b8300fd9c0 server_cookie=5f41182e5832313f in_seq=0 out_seq=0 2023-09-30T14:57:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.865+0000 7f4ae37fe700 1 -- 172.21.15.32:0/4188827632 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563c90a66ad0 con 0x563c90ba6ec0 2023-09-30T14:57:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.865+0000 7f4ae37fe700 10 client.17082 handle_command_reply: tid=0 2023-09-30T14:57:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.866+0000 7f4b16a69b80 1 client.17082 shutdown 2023-09-30T14:57:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.866+0000 7f4b16a69b80 1 -- 172.21.15.32:0/4188827632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b23a10 msgr2=0x563c90b90f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.866+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/4188827632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b23a10 0x563c90b90f80 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4af400fa40 tx=0x7f4af4098fc0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.866+0000 7f4b16a69b80 1 -- 172.21.15.32:0/4188827632 shutdown_connections 2023-09-30T14:57:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.866+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/4188827632 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x563c90ba6ec0 0x563c90ba92b0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.866+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/4188827632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c90b254e0 0x563c90b4fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.866+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/4188827632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563c90b23a10 0x563c90b90f80 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.866+0000 7f4b16a69b80 1 --2- 172.21.15.32:0/4188827632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563c90b24b20 0x563c90b914c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.866+0000 7f4b16a69b80 1 -- 172.21.15.32:0/4188827632 >> 172.21.15.32:0/4188827632 conn(0x563c90a8f200 msgr2=0x563c90b438d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.866+0000 7f4ae37fe700 0 client.17082 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:57:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.866+0000 7f4b16a69b80 1 -- 172.21.15.32:0/4188827632 shutdown_connections 2023-09-30T14:57:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.866+0000 7f4b16a69b80 1 -- 172.21.15.32:0/4188827632 wait complete. 2023-09-30T14:57:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.867+0000 7f4b16a69b80 20 client.17082 trim_cache size 0 max 16384 2023-09-30T14:57:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:57:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:57:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:57:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:57:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.870+0000 7f4b0a881700 1 -- 172.21.15.32:0/2144119131 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4aec0a0fa0 msgr2=0x7f4aec0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.870+0000 7f4b0a881700 1 --2- 172.21.15.32:0/2144119131 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4aec0a0fa0 0x7f4aec0a3460 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4af800ffd0 tx=0x7f4af800f820 comp rx=0 tx=0).stop 2023-09-30T14:57:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.870+0000 7f4b0a881700 1 -- 172.21.15.32:0/2144119131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b04100e70 msgr2=0x7f4b04196d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:57:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.870+0000 7f4b0a881700 1 --2- 172.21.15.32:0/2144119131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b04100e70 0x7f4b04196d80 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009fd0 tx=0x7f4af400b790 comp rx=0 tx=0).stop 2023-09-30T14:57:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.874+0000 7f4b0a881700 1 -- 172.21.15.32:0/2144119131 shutdown_connections 2023-09-30T14:57:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.874+0000 7f4b0a881700 1 --2- 172.21.15.32:0/2144119131 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4aec0a0fa0 0x7f4aec0a3460 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.874+0000 7f4b0a881700 1 --2- 172.21.15.32:0/2144119131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b04102980 0x7f4b04198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.874+0000 7f4b0a881700 1 --2- 172.21.15.32:0/2144119131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b04100e70 0x7f4b04196d80 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.874+0000 7f4b0a881700 1 --2- 172.21.15.32:0/2144119131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b04101fc0 0x7f4b041972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:57:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.874+0000 7f4b0a881700 1 -- 172.21.15.32:0/2144119131 >> 172.21.15.32:0/2144119131 conn(0x7f4b040fc400 msgr2=0x7f4b04105630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:57:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.874+0000 7f4b0a881700 1 -- 172.21.15.32:0/2144119131 shutdown_connections 2023-09-30T14:57:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:57:49.874+0000 7f4b0a881700 1 -- 172.21.15.32:0/2144119131 wait complete. 2023-09-30T14:57:50.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 255343 36.28 MB/sec execute 263 sec latency 255.357 ms 2023-09-30T14:57:50.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:49 smithi156 ceph-mon[103579]: pgmap v873: 97 pgs: 97 active+clean; 788 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.1 MiB/s wr, 742 op/s 2023-09-30T14:57:50.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:49 smithi148 ceph-mon[102355]: pgmap v873: 97 pgs: 97 active+clean; 788 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.1 MiB/s wr, 742 op/s 2023-09-30T14:57:50.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:49 smithi032 ceph-mon[98592]: pgmap v873: 97 pgs: 97 active+clean; 788 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.1 MiB/s wr, 742 op/s 2023-09-30T14:57:51.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 256010 36.28 MB/sec execute 264 sec latency 49.565 ms 2023-09-30T14:57:52.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 256626 36.29 MB/sec execute 265 sec latency 91.287 ms 2023-09-30T14:57:52.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:51 smithi156 ceph-mon[103579]: pgmap v874: 97 pgs: 97 active+clean; 784 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.5 MiB/s wr, 627 op/s 2023-09-30T14:57:52.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:51 smithi148 ceph-mon[102355]: pgmap v874: 97 pgs: 97 active+clean; 784 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.5 MiB/s wr, 627 op/s 2023-09-30T14:57:52.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:51 smithi032 ceph-mon[98592]: pgmap v874: 97 pgs: 97 active+clean; 784 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.5 MiB/s wr, 627 op/s 2023-09-30T14:57:53.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 257234 36.21 MB/sec execute 266 sec latency 186.237 ms 2023-09-30T14:57:54.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 257845 36.21 MB/sec execute 267 sec latency 234.068 ms 2023-09-30T14:57:54.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:53 smithi156 ceph-mon[103579]: pgmap v875: 97 pgs: 97 active+clean; 771 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.8 MiB/s wr, 680 op/s 2023-09-30T14:57:54.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:53 smithi148 ceph-mon[102355]: pgmap v875: 97 pgs: 97 active+clean; 771 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.8 MiB/s wr, 680 op/s 2023-09-30T14:57:54.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:53 smithi032 ceph-mon[98592]: pgmap v875: 97 pgs: 97 active+clean; 771 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.8 MiB/s wr, 680 op/s 2023-09-30T14:57:55.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 258804 36.29 MB/sec execute 268 sec latency 38.050 ms 2023-09-30T14:57:56.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 259525 36.31 MB/sec execute 269 sec latency 58.097 ms 2023-09-30T14:57:56.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:55 smithi156 ceph-mon[103579]: pgmap v876: 97 pgs: 97 active+clean; 776 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.0 MiB/s wr, 696 op/s 2023-09-30T14:57:56.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:55 smithi148 ceph-mon[102355]: pgmap v876: 97 pgs: 97 active+clean; 776 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.0 MiB/s wr, 696 op/s 2023-09-30T14:57:56.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:55 smithi032 ceph-mon[98592]: pgmap v876: 97 pgs: 97 active+clean; 776 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.0 MiB/s wr, 696 op/s 2023-09-30T14:57:57.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 260181 36.30 MB/sec execute 270 sec latency 78.109 ms 2023-09-30T14:57:58.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 260932 36.31 MB/sec execute 271 sec latency 80.648 ms 2023-09-30T14:57:58.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:57 smithi156 ceph-mon[103579]: pgmap v877: 97 pgs: 97 active+clean; 780 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.2 MiB/s wr, 760 op/s 2023-09-30T14:57:58.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:57 smithi148 ceph-mon[102355]: pgmap v877: 97 pgs: 97 active+clean; 780 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.2 MiB/s wr, 760 op/s 2023-09-30T14:57:58.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:57 smithi032 ceph-mon[98592]: pgmap v877: 97 pgs: 97 active+clean; 780 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.2 MiB/s wr, 760 op/s 2023-09-30T14:57:59.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 261411 36.25 MB/sec execute 272 sec latency 256.156 ms 2023-09-30T14:58:00.045 INFO:tasks.workunit.client.0.smithi032.stdout: 10 262233 36.31 MB/sec execute 273 sec latency 40.925 ms 2023-09-30T14:58:00.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:57:59 smithi156 ceph-mon[103579]: pgmap v878: 97 pgs: 97 active+clean; 781 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.2 MiB/s wr, 731 op/s 2023-09-30T14:58:00.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:57:59 smithi148 ceph-mon[102355]: pgmap v878: 97 pgs: 97 active+clean; 781 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.2 MiB/s wr, 731 op/s 2023-09-30T14:58:00.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:57:59 smithi032 ceph-mon[98592]: pgmap v878: 97 pgs: 97 active+clean; 781 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.2 MiB/s wr, 731 op/s 2023-09-30T14:58:01.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 262913 36.31 MB/sec execute 274 sec latency 41.103 ms 2023-09-30T14:58:02.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 263690 36.32 MB/sec execute 275 sec latency 44.271 ms 2023-09-30T14:58:02.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:01 smithi156 ceph-mon[103579]: pgmap v879: 97 pgs: 97 active+clean; 787 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 712 op/s 2023-09-30T14:58:02.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:01 smithi148 ceph-mon[102355]: pgmap v879: 97 pgs: 97 active+clean; 787 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 712 op/s 2023-09-30T14:58:02.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:01 smithi032 ceph-mon[98592]: pgmap v879: 97 pgs: 97 active+clean; 787 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 712 op/s 2023-09-30T14:58:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.002+0000 7f2b6043a700 1 -- 172.21.15.32:0/3645861357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b580ff4e0 msgr2=0x7f2b580ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.002+0000 7f2b6043a700 1 --2- 172.21.15.32:0/3645861357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b580ff4e0 0x7f2b580ff900 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2b540099c0 tx=0x7f2b54009cd0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.002+0000 7f2b6043a700 1 -- 172.21.15.32:0/3645861357 shutdown_connections 2023-09-30T14:58:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.002+0000 7f2b6043a700 1 --2- 172.21.15.32:0/3645861357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b58101070 0x7f2b58107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.002+0000 7f2b6043a700 1 --2- 172.21.15.32:0/3645861357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b580ff4e0 0x7f2b580ff900 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.002+0000 7f2b6043a700 1 --2- 172.21.15.32:0/3645861357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b581006d0 0x7f2b58100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.002+0000 7f2b6043a700 1 -- 172.21.15.32:0/3645861357 >> 172.21.15.32:0/3645861357 conn(0x7f2b580faa30 msgr2=0x7f2b580fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.002+0000 7f2b6043a700 1 -- 172.21.15.32:0/3645861357 shutdown_connections 2023-09-30T14:58:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.002+0000 7f2b6043a700 1 -- 172.21.15.32:0/3645861357 wait complete. 2023-09-30T14:58:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.003+0000 7f2b6043a700 1 Processor -- start 2023-09-30T14:58:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.003+0000 7f2b6043a700 1 -- start start 2023-09-30T14:58:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.003+0000 7f2b6043a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b580ff4e0 0x7f2b581952b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.003+0000 7f2b6043a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b581006d0 0x7f2b581957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.003+0000 7f2b6043a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b58101070 0x7f2b58196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.003+0000 7f2b6043a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58196cd0 con 0x7f2b581006d0 2023-09-30T14:58:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.003+0000 7f2b6043a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58196e40 con 0x7f2b58101070 2023-09-30T14:58:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.003+0000 7f2b6043a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58197000 con 0x7f2b580ff4e0 2023-09-30T14:58:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.003+0000 7f2b5d9d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b581006d0 0x7f2b581957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.003+0000 7f2b5d9d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b581006d0 0x7f2b581957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39528/0 (socket says 172.21.15.32:39528) 2023-09-30T14:58:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.003+0000 7f2b5d9d5700 1 -- 172.21.15.32:0/2673017792 learned_addr learned my addr 172.21.15.32:0/2673017792 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.003+0000 7f2b5e1d6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b580ff4e0 0x7f2b581952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.003+0000 7f2b5e9d7700 1 --2- 172.21.15.32:0/2673017792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b58101070 0x7f2b58196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.004+0000 7f2b5d9d5700 1 -- 172.21.15.32:0/2673017792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b580ff4e0 msgr2=0x7f2b581952b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.004+0000 7f2b5d9d5700 1 --2- 172.21.15.32:0/2673017792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b580ff4e0 0x7f2b581952b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.004+0000 7f2b5d9d5700 1 -- 172.21.15.32:0/2673017792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b58101070 msgr2=0x7f2b58196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.004+0000 7f2b5d9d5700 1 --2- 172.21.15.32:0/2673017792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b58101070 0x7f2b58196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.004+0000 7f2b5d9d5700 1 -- 172.21.15.32:0/2673017792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b540095f0 con 0x7f2b581006d0 2023-09-30T14:58:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.004+0000 7f2b5d9d5700 1 --2- 172.21.15.32:0/2673017792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b581006d0 0x7f2b581957f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800e9a0 tx=0x7f2b4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.004+0000 7f2b4f7fe700 1 -- 172.21.15.32:0/2673017792 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48009790 con 0x7f2b581006d0 2023-09-30T14:58:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.004+0000 7f2b4f7fe700 1 -- 172.21.15.32:0/2673017792 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b480104d0 con 0x7f2b581006d0 2023-09-30T14:58:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.004+0000 7f2b4f7fe700 1 -- 172.21.15.32:0/2673017792 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b480186a0 con 0x7f2b581006d0 2023-09-30T14:58:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.004+0000 7f2b6043a700 1 -- 172.21.15.32:0/2673017792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b581972e0 con 0x7f2b581006d0 2023-09-30T14:58:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.004+0000 7f2b6043a700 1 -- 172.21.15.32:0/2673017792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b58067b90 con 0x7f2b581006d0 2023-09-30T14:58:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.007+0000 7f2b4f7fe700 1 -- 172.21.15.32:0/2673017792 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f2b48010640 con 0x7f2b581006d0 2023-09-30T14:58:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.008+0000 7f2b4f7fe700 1 --2- 172.21.15.32:0/2673017792 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2b440a1020 0x7f2b440a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.008+0000 7f2b5e1d6700 1 --2- 172.21.15.32:0/2673017792 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2b440a1020 0x7f2b440a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.008+0000 7f2b4f7fe700 1 -- 172.21.15.32:0/2673017792 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2b480cd580 con 0x7f2b581006d0 2023-09-30T14:58:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.009+0000 7f2b5e1d6700 1 --2- 172.21.15.32:0/2673017792 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2b440a1020 0x7f2b440a34e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009990 tx=0x7f2b54005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.009+0000 7f2b6c622b80 1 Processor -- start 2023-09-30T14:58:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.009+0000 7f2b6c622b80 1 -- start start 2023-09-30T14:58:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.009+0000 7f2b6c622b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e3410 0x55b8538e3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.009+0000 7f2b6c622b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e4520 0x55b8538e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.009+0000 7f2b6c622b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e4ee0 0x55b8538eb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.009+0000 7f2b6c622b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b853849400 con 0x55b8538e4ee0 2023-09-30T14:58:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.009+0000 7f2b6c622b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85383abc0 con 0x55b8538e3410 2023-09-30T14:58:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.009+0000 7f2b6c622b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8538384e0 con 0x55b8538e4520 2023-09-30T14:58:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.010+0000 7f2b5e1d6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e3410 0x55b8538e3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.010+0000 7f2b5e9d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e4ee0 0x55b8538eb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.010+0000 7f2b5e1d6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e3410 0x55b8538e3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51010/0 (socket says 172.21.15.32:51010) 2023-09-30T14:58:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.010+0000 7f2b5e1d6700 1 -- 172.21.15.32:0/372748944 learned_addr learned my addr 172.21.15.32:0/372748944 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.010+0000 7f2b5d9d5700 1 --2- 172.21.15.32:0/372748944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e4520 0x55b8538e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.010+0000 7f2b5e1d6700 1 -- 172.21.15.32:0/372748944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e4520 msgr2=0x55b8538e49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.010+0000 7f2b5e1d6700 1 --2- 172.21.15.32:0/372748944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e4520 0x55b8538e49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.010+0000 7f2b5e1d6700 1 -- 172.21.15.32:0/372748944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e4ee0 msgr2=0x55b8538eb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.010+0000 7f2b5e1d6700 1 --2- 172.21.15.32:0/372748944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e4ee0 0x55b8538eb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.010+0000 7f2b5e1d6700 1 -- 172.21.15.32:0/372748944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b853896870 con 0x55b8538e3410 2023-09-30T14:58:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.010+0000 7f2b5e1d6700 1 --2- 172.21.15.32:0/372748944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e3410 0x55b8538e3830 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2b58100530 tx=0x7f2b5401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71e058823825d8a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.010+0000 7f2b4cd82700 1 -- 172.21.15.32:0/372748944 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b54029070 con 0x55b8538e3410 2023-09-30T14:58:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.010+0000 7f2b4cd82700 1 -- 172.21.15.32:0/372748944 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b5401e040 con 0x55b8538e3410 2023-09-30T14:58:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.010+0000 7f2b4cd82700 1 -- 172.21.15.32:0/372748944 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5400f750 con 0x55b8538e3410 2023-09-30T14:58:03.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.011+0000 7f2b6c622b80 1 -- 172.21.15.32:0/372748944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e3410 msgr2=0x55b8538e3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:03.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.011+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/372748944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e3410 0x55b8538e3830 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2b58100530 tx=0x7f2b5401f000 comp rx=0 tx=0).stop 2023-09-30T14:58:03.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.011+0000 7f2b6c622b80 1 -- 172.21.15.32:0/372748944 shutdown_connections 2023-09-30T14:58:03.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.011+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/372748944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e4ee0 0x55b8538eb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.011+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/372748944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e3410 0x55b8538e3830 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.011+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/372748944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e4520 0x55b8538e49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.017 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.011+0000 7f2b6c622b80 1 -- 172.21.15.32:0/372748944 >> 172.21.15.32:0/372748944 conn(0x55b853855200 msgr2=0x55b8538dbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.011+0000 7f2b6c622b80 1 -- 172.21.15.32:0/372748944 shutdown_connections 2023-09-30T14:58:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.011+0000 7f2b6c622b80 1 -- 172.21.15.32:0/372748944 wait complete. 2023-09-30T14:58:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b6c622b80 1 -- start start 2023-09-30T14:58:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b6c622b80 1 Processor -- start 2023-09-30T14:58:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b6c622b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e3410 0x55b8538ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b6c622b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e4520 0x55b8538ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b6c622b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e4ee0 0x55b8538f6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b6c622b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b853833dd0 con 0x55b8538e3410 2023-09-30T14:58:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b6c622b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8537d06b0 con 0x55b8538e4ee0 2023-09-30T14:58:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b6c622b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8537d0350 con 0x55b8538e4520 2023-09-30T14:58:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b6c622b80 10 client.0 _ll_register_callbacks cb 0x55b85362d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:58:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b5e1d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e3410 0x55b8538ef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b5e1d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e3410 0x55b8538ef1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39540/0 (socket says 172.21.15.32:39540) 2023-09-30T14:58:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b5e1d6700 1 -- 172.21.15.32:0/2423500792 learned_addr learned my addr 172.21.15.32:0/2423500792 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b5e9d7700 1 --2- 172.21.15.32:0/2423500792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e4ee0 0x55b8538f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b5d9d5700 1 --2- 172.21.15.32:0/2423500792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e4520 0x55b8538ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b5e1d6700 1 -- 172.21.15.32:0/2423500792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e4520 msgr2=0x55b8538ef6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b5e1d6700 1 --2- 172.21.15.32:0/2423500792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e4520 0x55b8538ef6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b5e1d6700 1 -- 172.21.15.32:0/2423500792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e4ee0 msgr2=0x55b8538f6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b5e1d6700 1 --2- 172.21.15.32:0/2423500792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e4ee0 0x55b8538f6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.012+0000 7f2b5e1d6700 1 -- 172.21.15.32:0/2423500792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b58108270 con 0x55b8538e3410 2023-09-30T14:58:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.013+0000 7f2b5e1d6700 1 --2- 172.21.15.32:0/2423500792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e3410 0x55b8538ef1a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2b54005040 tx=0x7f2b54009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb029060feba577 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.013+0000 7f2b41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:58:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.013+0000 7f2b41ffb700 1 -- 172.21.15.32:0/2423500792 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b54029040 con 0x55b8538e3410 2023-09-30T14:58:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.013+0000 7f2b41ffb700 1 -- 172.21.15.32:0/2423500792 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2b540039d0 con 0x55b8538e3410 2023-09-30T14:58:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.013+0000 7f2b6c622b80 1 -- 172.21.15.32:0/2423500792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8538d4090 con 0x55b8538e3410 2023-09-30T14:58:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.013+0000 7f2b41ffb700 1 -- 172.21.15.32:0/2423500792 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b54003b90 con 0x55b8538e3410 2023-09-30T14:58:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.013+0000 7f2b6c622b80 10 client.17094 fetch_fsmap learned FSMap version 51 2023-09-30T14:58:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.013+0000 7f2b6c622b80 1 -- 172.21.15.32:0/2423500792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55b85365ed80 con 0x55b8538e3410 2023-09-30T14:58:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.013+0000 7f2b41ffb700 1 -- 172.21.15.32:0/2423500792 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b8538d4090 con 0x55b8538e3410 2023-09-30T14:58:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.013+0000 7f2b41ffb700 1 -- 172.21.15.32:0/2423500792 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 51) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2b5402d400 con 0x55b8538e3410 2023-09-30T14:58:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.013+0000 7f2b6c622b80 10 client.17094 fetch_fsmap finished waiting for FSMap version 51 2023-09-30T14:58:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.013+0000 7f2b6c622b80 10 client.17094 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:58:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.013+0000 7f2b6c622b80 20 client.17094 populate_metadata read hostname 'smithi032' 2023-09-30T14:58:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.013+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/2423500792 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b8538fab20 0x55b8538fcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.013+0000 7f2b6c622b80 4 client.17094 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:58:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.013+0000 7f2b6c622b80 1 -- 172.21.15.32:0/2423500792 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b85382cdb0 con 0x55b8538fab20 2023-09-30T14:58:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.014+0000 7f2b5d9d5700 1 --2- 172.21.15.32:0/2423500792 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b8538fab20 0x55b8538fcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.014+0000 7f2b41ffb700 10 client.17094 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.014+0000 7f2b5d9d5700 1 --2- 172.21.15.32:0/2423500792 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b8538fab20 0x55b8538fcf40 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c483f3aa53dd6b8 server_cookie=dd61086f2560f24d in_seq=0 out_seq=0 2023-09-30T14:58:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.015+0000 7f2b41ffb700 1 -- 172.21.15.32:0/2423500792 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b85382cdb0 con 0x55b8538fab20 2023-09-30T14:58:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.015+0000 7f2b41ffb700 10 client.17094 handle_command_reply: tid=0 2023-09-30T14:58:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.015+0000 7f2b6c622b80 1 client.17094 shutdown 2023-09-30T14:58:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.016+0000 7f2b6c622b80 1 -- 172.21.15.32:0/2423500792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e3410 msgr2=0x55b8538ef1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.016+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/2423500792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e3410 0x55b8538ef1a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2b54005040 tx=0x7f2b54009300 comp rx=0 tx=0).stop 2023-09-30T14:58:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.016+0000 7f2b6c622b80 1 -- 172.21.15.32:0/2423500792 shutdown_connections 2023-09-30T14:58:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.016+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/2423500792 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b8538fab20 0x55b8538fcf40 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.016+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/2423500792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e4ee0 0x55b8538f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.016+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/2423500792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e3410 0x55b8538ef1a0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.016+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/2423500792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e4520 0x55b8538ef6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.016+0000 7f2b41ffb700 0 client.17094 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.016+0000 7f2b6c622b80 1 -- 172.21.15.32:0/2423500792 >> 172.21.15.32:0/2423500792 conn(0x55b853855200 msgr2=0x55b8538db590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.016+0000 7f2b6c622b80 1 -- 172.21.15.32:0/2423500792 shutdown_connections 2023-09-30T14:58:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.016+0000 7f2b6c622b80 1 -- 172.21.15.32:0/2423500792 wait complete. 2023-09-30T14:58:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.016+0000 7f2b6c622b80 20 client.17094 trim_cache size 0 max 16384 2023-09-30T14:58:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.035+0000 7f2b6c622b80 1 Processor -- start 2023-09-30T14:58:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.035+0000 7f2b6c622b80 1 -- start start 2023-09-30T14:58:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.035+0000 7f2b6c622b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e3410 0x55b8538ef660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.035+0000 7f2b6c622b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e4520 0x55b8538ed560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.035+0000 7f2b6c622b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e4ee0 0x55b8538edaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.035+0000 7f2b6c622b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8536b48c0 con 0x55b8538e4520 2023-09-30T14:58:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.035+0000 7f2b6c622b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8538d8bd0 con 0x55b8538e4ee0 2023-09-30T14:58:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.035+0000 7f2b6c622b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8538dd6b0 con 0x55b8538e3410 2023-09-30T14:58:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.035+0000 7f2b5d9d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e4520 0x55b8538ed560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.035+0000 7f2b5d9d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e4520 0x55b8538ed560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39550/0 (socket says 172.21.15.32:39550) 2023-09-30T14:58:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.035+0000 7f2b5d9d5700 1 -- 172.21.15.32:0/1100074784 learned_addr learned my addr 172.21.15.32:0/1100074784 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.035+0000 7f2b5e1d6700 1 --2- 172.21.15.32:0/1100074784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e3410 0x55b8538ef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.035+0000 7f2b5e9d7700 1 --2- 172.21.15.32:0/1100074784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e4ee0 0x55b8538edaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.036+0000 7f2b5d9d5700 1 -- 172.21.15.32:0/1100074784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e3410 msgr2=0x55b8538ef660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.036+0000 7f2b5d9d5700 1 --2- 172.21.15.32:0/1100074784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e3410 0x55b8538ef660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.036+0000 7f2b5d9d5700 1 -- 172.21.15.32:0/1100074784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e4ee0 msgr2=0x55b8538edaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.036+0000 7f2b5d9d5700 1 --2- 172.21.15.32:0/1100074784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e4ee0 0x55b8538edaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.036+0000 7f2b5d9d5700 1 -- 172.21.15.32:0/1100074784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b853647c00 con 0x55b8538e4520 2023-09-30T14:58:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.036+0000 7f2b5d9d5700 1 --2- 172.21.15.32:0/1100074784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e4520 0x55b8538ed560 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2b48096f00 tx=0x7f2b480caf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a547a779718946a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.036+0000 7f2b4d583700 1 -- 172.21.15.32:0/1100074784 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b480b2070 con 0x55b8538e4520 2023-09-30T14:58:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.036+0000 7f2b4d583700 1 -- 172.21.15.32:0/1100074784 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b4809eba0 con 0x55b8538e4520 2023-09-30T14:58:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.036+0000 7f2b4d583700 1 -- 172.21.15.32:0/1100074784 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48010920 con 0x55b8538e4520 2023-09-30T14:58:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.037+0000 7f2b6c622b80 1 -- 172.21.15.32:0/1100074784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e4520 msgr2=0x55b8538ed560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.037+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/1100074784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e4520 0x55b8538ed560 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2b48096f00 tx=0x7f2b480caf60 comp rx=0 tx=0).stop 2023-09-30T14:58:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.037+0000 7f2b6c622b80 1 -- 172.21.15.32:0/1100074784 shutdown_connections 2023-09-30T14:58:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.037+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/1100074784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e4ee0 0x55b8538edaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.037+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/1100074784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e3410 0x55b8538ef660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.037+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/1100074784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e4520 0x55b8538ed560 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.037+0000 7f2b6c622b80 1 -- 172.21.15.32:0/1100074784 >> 172.21.15.32:0/1100074784 conn(0x55b853855200 msgr2=0x55b8538eef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.037+0000 7f2b6c622b80 1 -- 172.21.15.32:0/1100074784 shutdown_connections 2023-09-30T14:58:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.037+0000 7f2b6c622b80 1 -- 172.21.15.32:0/1100074784 wait complete. 2023-09-30T14:58:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.037+0000 7f2b6c622b80 1 -- start start 2023-09-30T14:58:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.037+0000 7f2b6c622b80 1 Processor -- start 2023-09-30T14:58:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.037+0000 7f2b6c622b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e3410 0x55b853946b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b6c622b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e4520 0x55b853947050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b6c622b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e4ee0 0x55b85394dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b6c622b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85394e290 con 0x55b8538e3410 2023-09-30T14:58:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b6c622b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85394e3d0 con 0x55b8538e4ee0 2023-09-30T14:58:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b6c622b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85394e590 con 0x55b8538e4520 2023-09-30T14:58:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b6c622b80 10 client.0 _ll_register_callbacks cb 0x7f2b50009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:58:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b5e1d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e3410 0x55b853946b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b5e1d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e3410 0x55b853946b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39554/0 (socket says 172.21.15.32:39554) 2023-09-30T14:58:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b5e1d6700 1 -- 172.21.15.32:0/1771512903 learned_addr learned my addr 172.21.15.32:0/1771512903 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b5d9d5700 1 --2- 172.21.15.32:0/1771512903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e4520 0x55b853947050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b5e9d7700 1 --2- 172.21.15.32:0/1771512903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e4ee0 0x55b85394dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b5e1d6700 1 -- 172.21.15.32:0/1771512903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e4520 msgr2=0x55b853947050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b5e1d6700 1 --2- 172.21.15.32:0/1771512903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e4520 0x55b853947050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b5e1d6700 1 -- 172.21.15.32:0/1771512903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e4ee0 msgr2=0x55b85394dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b5e1d6700 1 --2- 172.21.15.32:0/1771512903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e4ee0 0x55b85394dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b5e1d6700 1 -- 172.21.15.32:0/1771512903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b58067b90 con 0x55b8538e3410 2023-09-30T14:58:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b5e1d6700 1 --2- 172.21.15.32:0/1771512903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e3410 0x55b853946b10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2b540098e0 tx=0x7f2b5402d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71df08978bdfbf57 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:58:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b41ffb700 1 -- 172.21.15.32:0/1771512903 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b54029050 con 0x55b8538e3410 2023-09-30T14:58:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b41ffb700 1 -- 172.21.15.32:0/1771512903 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2b5402dbd0 con 0x55b8538e3410 2023-09-30T14:58:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.038+0000 7f2b6c622b80 1 -- 172.21.15.32:0/1771512903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b5401e040 con 0x55b8538e3410 2023-09-30T14:58:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.039+0000 7f2b41ffb700 1 -- 172.21.15.32:0/1771512903 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b5402dd90 con 0x55b8538e3410 2023-09-30T14:58:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.039+0000 7f2b6c622b80 10 client.17100 fetch_fsmap learned FSMap version 51 2023-09-30T14:58:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.039+0000 7f2b6c622b80 1 -- 172.21.15.32:0/1771512903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55b8536851f0 con 0x55b8538e3410 2023-09-30T14:58:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.039+0000 7f2b41ffb700 1 -- 172.21.15.32:0/1771512903 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b5404f020 con 0x55b8538e3410 2023-09-30T14:58:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.039+0000 7f2b41ffb700 1 -- 172.21.15.32:0/1771512903 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 51) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2b54041680 con 0x55b8538e3410 2023-09-30T14:58:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.039+0000 7f2b6c622b80 10 client.17100 fetch_fsmap finished waiting for FSMap version 51 2023-09-30T14:58:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.039+0000 7f2b6c622b80 10 client.17100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:58:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.039+0000 7f2b6c622b80 20 client.17100 populate_metadata read hostname 'smithi032' 2023-09-30T14:58:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.039+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/1771512903 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b85394fb60 0x55b853951f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.039+0000 7f2b6c622b80 4 client.17100 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:58:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.039+0000 7f2b6c622b80 1 -- 172.21.15.32:0/1771512903 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b85382cad0 con 0x55b85394fb60 2023-09-30T14:58:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.040+0000 7f2b5d9d5700 1 --2- 172.21.15.32:0/1771512903 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b85394fb60 0x55b853951f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.040+0000 7f2b41ffb700 10 client.17100 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.040+0000 7f2b5d9d5700 1 --2- 172.21.15.32:0/1771512903 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b85394fb60 0x55b853951f80 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1bffb007ef282c0 server_cookie=bb7c62bf587f575c in_seq=0 out_seq=0 2023-09-30T14:58:03.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 264549 36.37 MB/sec execute 276 sec latency 48.198 ms 2023-09-30T14:58:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.581+0000 7f2b41ffb700 1 -- 172.21.15.32:0/1771512903 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b85382cad0 con 0x55b85394fb60 2023-09-30T14:58:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.581+0000 7f2b41ffb700 10 client.17100 handle_command_reply: tid=0 2023-09-30T14:58:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.581+0000 7f2b6c622b80 1 client.17100 shutdown 2023-09-30T14:58:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.582+0000 7f2b6c622b80 1 -- 172.21.15.32:0/1771512903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e3410 msgr2=0x55b853946b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.582+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/1771512903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e3410 0x55b853946b10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2b540098e0 tx=0x7f2b5402d630 comp rx=0 tx=0).stop 2023-09-30T14:58:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.582+0000 7f2b6c622b80 1 -- 172.21.15.32:0/1771512903 shutdown_connections 2023-09-30T14:58:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.582+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/1771512903 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55b85394fb60 0x55b853951f80 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.582+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/1771512903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8538e4ee0 0x55b85394dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.582+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/1771512903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b8538e3410 0x55b853946b10 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.582+0000 7f2b41ffb700 0 client.17100 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.582+0000 7f2b6c622b80 1 --2- 172.21.15.32:0/1771512903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8538e4520 0x55b853947050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.582+0000 7f2b6c622b80 1 -- 172.21.15.32:0/1771512903 >> 172.21.15.32:0/1771512903 conn(0x55b853855200 msgr2=0x55b8538db590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.582+0000 7f2b6c622b80 1 -- 172.21.15.32:0/1771512903 shutdown_connections 2023-09-30T14:58:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.582+0000 7f2b6c622b80 1 -- 172.21.15.32:0/1771512903 wait complete. 2023-09-30T14:58:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.582+0000 7f2b6c622b80 20 client.17100 trim_cache size 0 max 16384 2023-09-30T14:58:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:58:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:58:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:58:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:58:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.582+0000 7f2b6043a700 1 -- 172.21.15.32:0/2673017792 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2b440a1020 msgr2=0x7f2b440a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.583+0000 7f2b6043a700 1 --2- 172.21.15.32:0/2673017792 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2b440a1020 0x7f2b440a34e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009990 tx=0x7f2b54005fb0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.583+0000 7f2b6043a700 1 -- 172.21.15.32:0/2673017792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b581006d0 msgr2=0x7f2b581957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.583+0000 7f2b6043a700 1 --2- 172.21.15.32:0/2673017792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b581006d0 0x7f2b581957f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800e9a0 tx=0x7f2b4800ecb0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.583+0000 7f2b6043a700 1 -- 172.21.15.32:0/2673017792 shutdown_connections 2023-09-30T14:58:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.583+0000 7f2b6043a700 1 --2- 172.21.15.32:0/2673017792 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2b440a1020 0x7f2b440a34e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.583+0000 7f2b6043a700 1 --2- 172.21.15.32:0/2673017792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b58101070 0x7f2b58196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.583+0000 7f2b6043a700 1 --2- 172.21.15.32:0/2673017792 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b580ff4e0 0x7f2b581952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.583+0000 7f2b6043a700 1 --2- 172.21.15.32:0/2673017792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b581006d0 0x7f2b581957f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.583+0000 7f2b6043a700 1 -- 172.21.15.32:0/2673017792 >> 172.21.15.32:0/2673017792 conn(0x7f2b580faa30 msgr2=0x7f2b58103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.583+0000 7f2b6043a700 1 -- 172.21.15.32:0/2673017792 shutdown_connections 2023-09-30T14:58:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:03.583+0000 7f2b6043a700 1 -- 172.21.15.32:0/2673017792 wait complete. 2023-09-30T14:58:04.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 264832 36.29 MB/sec execute 277 sec latency 473.858 ms 2023-09-30T14:58:04.102 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:03 smithi156 ceph-mon[103579]: pgmap v880: 97 pgs: 97 active+clean; 791 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.8 MiB/s wr, 761 op/s 2023-09-30T14:58:04.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:03 smithi148 ceph-mon[102355]: pgmap v880: 97 pgs: 97 active+clean; 791 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.8 MiB/s wr, 761 op/s 2023-09-30T14:58:04.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:03 smithi032 ceph-mon[98592]: pgmap v880: 97 pgs: 97 active+clean; 791 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.8 MiB/s wr, 761 op/s 2023-09-30T14:58:05.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 265504 36.29 MB/sec execute 278 sec latency 89.066 ms 2023-09-30T14:58:06.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 266049 36.27 MB/sec execute 279 sec latency 98.016 ms 2023-09-30T14:58:06.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:05 smithi156 ceph-mon[103579]: pgmap v881: 97 pgs: 97 active+clean; 798 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 10 MiB/s wr, 820 op/s 2023-09-30T14:58:06.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:05 smithi148 ceph-mon[102355]: pgmap v881: 97 pgs: 97 active+clean; 798 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 10 MiB/s wr, 820 op/s 2023-09-30T14:58:06.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:05 smithi032 ceph-mon[98592]: pgmap v881: 97 pgs: 97 active+clean; 798 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 10 MiB/s wr, 820 op/s 2023-09-30T14:58:07.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 266715 36.23 MB/sec execute 280 sec latency 99.300 ms 2023-09-30T14:58:08.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 267637 36.27 MB/sec execute 281 sec latency 85.884 ms 2023-09-30T14:58:08.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:07 smithi148 ceph-mon[102355]: pgmap v882: 97 pgs: 97 active+clean; 785 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 772 op/s 2023-09-30T14:58:08.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:07 smithi032 ceph-mon[98592]: pgmap v882: 97 pgs: 97 active+clean; 785 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 772 op/s 2023-09-30T14:58:08.310 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:07 smithi156 ceph-mon[103579]: pgmap v882: 97 pgs: 97 active+clean; 785 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 772 op/s 2023-09-30T14:58:09.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 268110 36.25 MB/sec execute 282 sec latency 282.650 ms 2023-09-30T14:58:10.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 268795 36.31 MB/sec execute 283 sec latency 54.078 ms 2023-09-30T14:58:10.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:09 smithi148 ceph-mon[102355]: pgmap v883: 97 pgs: 97 active+clean; 786 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.6 MiB/s wr, 713 op/s 2023-09-30T14:58:10.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:09 smithi032 ceph-mon[98592]: pgmap v883: 97 pgs: 97 active+clean; 786 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.6 MiB/s wr, 713 op/s 2023-09-30T14:58:10.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:09 smithi156 ceph-mon[103579]: pgmap v883: 97 pgs: 97 active+clean; 786 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.6 MiB/s wr, 713 op/s 2023-09-30T14:58:11.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 269178 36.24 MB/sec execute 284 sec latency 194.406 ms 2023-09-30T14:58:12.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 269807 36.19 MB/sec execute 285 sec latency 136.814 ms 2023-09-30T14:58:12.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:11 smithi148 ceph-mon[102355]: pgmap v884: 97 pgs: 97 active+clean; 784 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 11 MiB/s wr, 824 op/s 2023-09-30T14:58:12.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:11 smithi032 ceph-mon[98592]: pgmap v884: 97 pgs: 97 active+clean; 784 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 11 MiB/s wr, 824 op/s 2023-09-30T14:58:12.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:11 smithi156 ceph-mon[103579]: pgmap v884: 97 pgs: 97 active+clean; 784 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 11 MiB/s wr, 824 op/s 2023-09-30T14:58:13.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 270792 36.23 MB/sec execute 286 sec latency 45.704 ms 2023-09-30T14:58:14.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 271401 36.23 MB/sec execute 287 sec latency 212.484 ms 2023-09-30T14:58:14.659 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:13 smithi156 ceph-mon[103579]: pgmap v885: 97 pgs: 97 active+clean; 787 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.9 MiB/s wr, 789 op/s 2023-09-30T14:58:15.326 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:13 smithi148 ceph-mon[102355]: pgmap v885: 97 pgs: 97 active+clean; 787 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.9 MiB/s wr, 789 op/s 2023-09-30T14:58:15.326 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:13 smithi032 ceph-mon[98592]: pgmap v885: 97 pgs: 97 active+clean; 787 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.9 MiB/s wr, 789 op/s 2023-09-30T14:58:15.327 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:14 smithi156 ceph-mon[103579]: pgmap v886: 97 pgs: 97 active+clean; 784 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 737 op/s 2023-09-30T14:58:15.327 INFO:tasks.workunit.client.0.smithi032.stdout: 10 272096 36.28 MB/sec execute 288 sec latency 79.064 ms 2023-09-30T14:58:15.327 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:14 smithi032 ceph-mon[98592]: pgmap v886: 97 pgs: 97 active+clean; 784 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 737 op/s 2023-09-30T14:58:15.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:14 smithi148 ceph-mon[102355]: pgmap v886: 97 pgs: 97 active+clean; 784 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 737 op/s 2023-09-30T14:58:16.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 272567 36.24 MB/sec execute 289 sec latency 68.690 ms 2023-09-30T14:58:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.726+0000 7fb79e944700 1 -- 172.21.15.32:0/4126480598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb798100f50 msgr2=0x7fb7981013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.726+0000 7fb79e944700 1 --2- 172.21.15.32:0/4126480598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb798100f50 0x7fb7981013b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0099c0 tx=0x7fb78c009cd0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.727+0000 7fb79e944700 1 -- 172.21.15.32:0/4126480598 shutdown_connections 2023-09-30T14:58:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.727+0000 7fb79e944700 1 --2- 172.21.15.32:0/4126480598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7981018f0 0x7fb7981081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.727+0000 7fb79e944700 1 --2- 172.21.15.32:0/4126480598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7980ffd60 0x7fb798100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.727+0000 7fb79e944700 1 --2- 172.21.15.32:0/4126480598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb798100f50 0x7fb7981013b0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.727+0000 7fb79e944700 1 -- 172.21.15.32:0/4126480598 >> 172.21.15.32:0/4126480598 conn(0x7fb7980fb2b0 msgr2=0x7fb7980fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.728+0000 7fb79e944700 1 -- 172.21.15.32:0/4126480598 shutdown_connections 2023-09-30T14:58:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.728+0000 7fb79e944700 1 -- 172.21.15.32:0/4126480598 wait complete. 2023-09-30T14:58:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.728+0000 7fb79e944700 1 Processor -- start 2023-09-30T14:58:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.728+0000 7fb79e944700 1 -- start start 2023-09-30T14:58:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb79e944700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7980ffd60 0x7fb7981949e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb79e944700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb798100f50 0x7fb798194f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb79e944700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7981018f0 0x7fb798195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb79e944700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb798196400 con 0x7fb798100f50 2023-09-30T14:58:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb79e944700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb798196570 con 0x7fb7981018f0 2023-09-30T14:58:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb79e944700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb798196730 con 0x7fb7980ffd60 2023-09-30T14:58:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb7977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb798100f50 0x7fb798194f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb797fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7980ffd60 0x7fb7981949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb7977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb798100f50 0x7fb798194f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39572/0 (socket says 172.21.15.32:39572) 2023-09-30T14:58:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb79cee1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7981018f0 0x7fb798195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb7977fe700 1 -- 172.21.15.32:0/1061084842 learned_addr learned my addr 172.21.15.32:0/1061084842 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb7977fe700 1 -- 172.21.15.32:0/1061084842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7980ffd60 msgr2=0x7fb7981949e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb7977fe700 1 --2- 172.21.15.32:0/1061084842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7980ffd60 0x7fb7981949e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb7977fe700 1 -- 172.21.15.32:0/1061084842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7981018f0 msgr2=0x7fb798195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb7977fe700 1 --2- 172.21.15.32:0/1061084842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7981018f0 0x7fb798195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb7977fe700 1 -- 172.21.15.32:0/1061084842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78800b730 con 0x7fb798100f50 2023-09-30T14:58:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.729+0000 7fb7977fe700 1 --2- 172.21.15.32:0/1061084842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb798100f50 0x7fb798194f20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00c010 tx=0x7fb78c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.730+0000 7fb7957fa700 1 -- 172.21.15.32:0/1061084842 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c01d070 con 0x7fb798100f50 2023-09-30T14:58:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.730+0000 7fb79e944700 1 -- 172.21.15.32:0/1061084842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78c0095f0 con 0x7fb798100f50 2023-09-30T14:58:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.730+0000 7fb7957fa700 1 -- 172.21.15.32:0/1061084842 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb78c005050 con 0x7fb798100f50 2023-09-30T14:58:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.730+0000 7fb7957fa700 1 -- 172.21.15.32:0/1061084842 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c004ad0 con 0x7fb798100f50 2023-09-30T14:58:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.730+0000 7fb79e944700 1 -- 172.21.15.32:0/1061084842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7981a1400 con 0x7fb798100f50 2023-09-30T14:58:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.732+0000 7fb7957fa700 1 -- 172.21.15.32:0/1061084842 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fb78c0051c0 con 0x7fb798100f50 2023-09-30T14:58:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.732+0000 7fb7957fa700 1 --2- 172.21.15.32:0/1061084842 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb7800a0fa0 0x7fb7800a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.733+0000 7fb7957fa700 1 -- 172.21.15.32:0/1061084842 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb78c0cd5b0 con 0x7fb798100f50 2023-09-30T14:58:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.733+0000 7fb797fff700 1 --2- 172.21.15.32:0/1061084842 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb7800a0fa0 0x7fb7800a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.733+0000 7fb797fff700 1 --2- 172.21.15.32:0/1061084842 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb7800a0fa0 0x7fb7800a3460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb798100db0 tx=0x7fb78800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.734+0000 7fb7aab2cb80 1 Processor -- start 2023-09-30T14:58:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.734+0000 7fb7aab2cb80 1 -- start start 2023-09-30T14:58:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.734+0000 7fb7aab2cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb52200 0x564f9eb52620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.734+0000 7fb7aab2cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53310 0x564f9eb53790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.734+0000 7fb7aab2cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb53cd0 0x564f9eb5a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.734+0000 7fb7aab2cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9eaba400 con 0x564f9eb53310 2023-09-30T14:58:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.734+0000 7fb7aab2cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9eaabbc0 con 0x564f9eb52200 2023-09-30T14:58:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.734+0000 7fb7aab2cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9eaa94e0 con 0x564f9eb53cd0 2023-09-30T14:58:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.734+0000 7fb7977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53310 0x564f9eb53790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.734+0000 7fb7977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53310 0x564f9eb53790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39578/0 (socket says 172.21.15.32:39578) 2023-09-30T14:58:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.734+0000 7fb7977fe700 1 -- 172.21.15.32:0/4204524925 learned_addr learned my addr 172.21.15.32:0/4204524925 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.734+0000 7fb797fff700 1 --2- 172.21.15.32:0/4204524925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb52200 0x564f9eb52620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.735+0000 7fb79cee1700 1 --2- 172.21.15.32:0/4204524925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb53cd0 0x564f9eb5a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.735+0000 7fb7977fe700 1 -- 172.21.15.32:0/4204524925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb53cd0 msgr2=0x564f9eb5a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.735+0000 7fb7977fe700 1 --2- 172.21.15.32:0/4204524925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb53cd0 0x564f9eb5a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.735+0000 7fb7977fe700 1 -- 172.21.15.32:0/4204524925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb52200 msgr2=0x564f9eb52620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.735+0000 7fb7977fe700 1 --2- 172.21.15.32:0/4204524925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb52200 0x564f9eb52620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.735+0000 7fb7977fe700 1 -- 172.21.15.32:0/4204524925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f9eb07870 con 0x564f9eb53310 2023-09-30T14:58:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.735+0000 7fb7977fe700 1 --2- 172.21.15.32:0/4204524925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53310 0x564f9eb53790 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb78c097010 tx=0x7fb78c09a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e008b7b938936af server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.735+0000 7fb77e7fc700 1 -- 172.21.15.32:0/4204524925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c0a2e10 con 0x564f9eb53310 2023-09-30T14:58:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.735+0000 7fb77e7fc700 1 -- 172.21.15.32:0/4204524925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb78c098eb0 con 0x564f9eb53310 2023-09-30T14:58:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.735+0000 7fb77e7fc700 1 -- 172.21.15.32:0/4204524925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c09ad70 con 0x564f9eb53310 2023-09-30T14:58:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.736+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/4204524925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53310 msgr2=0x564f9eb53790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.736+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/4204524925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53310 0x564f9eb53790 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb78c097010 tx=0x7fb78c09a780 comp rx=0 tx=0).stop 2023-09-30T14:58:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.736+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/4204524925 shutdown_connections 2023-09-30T14:58:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.736+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/4204524925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb53cd0 0x564f9eb5a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.736+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/4204524925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb52200 0x564f9eb52620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.736+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/4204524925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53310 0x564f9eb53790 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.736+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/4204524925 >> 172.21.15.32:0/4204524925 conn(0x564f9eac6200 msgr2=0x564f9eb4cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.736+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/4204524925 shutdown_connections 2023-09-30T14:58:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.736+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/4204524925 wait complete. 2023-09-30T14:58:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.737+0000 7fb7aab2cb80 1 -- start start 2023-09-30T14:58:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.737+0000 7fb7aab2cb80 1 Processor -- start 2023-09-30T14:58:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.737+0000 7fb7aab2cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb52200 0x564f9eb5bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.737+0000 7fb7aab2cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb53310 0x564f9eb5c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.737+0000 7fb7aab2cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53cd0 0x564f9eb630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.737+0000 7fb7aab2cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9eaa4dd0 con 0x564f9eb53cd0 2023-09-30T14:58:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.737+0000 7fb7aab2cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9ea416b0 con 0x564f9eb53310 2023-09-30T14:58:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.737+0000 7fb7aab2cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9ea41350 con 0x564f9eb52200 2023-09-30T14:58:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.737+0000 7fb7aab2cb80 10 client.0 _ll_register_callbacks cb 0x564f9e89e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:58:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.737+0000 7fb79cee1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53cd0 0x564f9eb630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.737+0000 7fb79cee1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53cd0 0x564f9eb630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39586/0 (socket says 172.21.15.32:39586) 2023-09-30T14:58:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.737+0000 7fb79cee1700 1 -- 172.21.15.32:0/2931007988 learned_addr learned my addr 172.21.15.32:0/2931007988 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.737+0000 7fb797fff700 1 --2- 172.21.15.32:0/2931007988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb52200 0x564f9eb5bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.737+0000 7fb7977fe700 1 --2- 172.21.15.32:0/2931007988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb53310 0x564f9eb5c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.738+0000 7fb79cee1700 1 -- 172.21.15.32:0/2931007988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb52200 msgr2=0x564f9eb5bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.738+0000 7fb79cee1700 1 --2- 172.21.15.32:0/2931007988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb52200 0x564f9eb5bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.738+0000 7fb79cee1700 1 -- 172.21.15.32:0/2931007988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb53310 msgr2=0x564f9eb5c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.738+0000 7fb79cee1700 1 --2- 172.21.15.32:0/2931007988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb53310 0x564f9eb5c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.738+0000 7fb79cee1700 1 -- 172.21.15.32:0/2931007988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7981a1400 con 0x564f9eb53cd0 2023-09-30T14:58:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.738+0000 7fb79cee1700 1 --2- 172.21.15.32:0/2931007988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53cd0 0x564f9eb630c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb790009fd0 tx=0x7fb790007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee122dc91eb2dd70 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.738+0000 7fb7777fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:58:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.738+0000 7fb7777fe700 1 -- 172.21.15.32:0/2931007988 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79000e040 con 0x564f9eb53cd0 2023-09-30T14:58:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.738+0000 7fb7777fe700 1 -- 172.21.15.32:0/2931007988 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb790016840 con 0x564f9eb53cd0 2023-09-30T14:58:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.738+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/2931007988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f9eb45090 con 0x564f9eb53cd0 2023-09-30T14:58:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.738+0000 7fb7777fe700 1 -- 172.21.15.32:0/2931007988 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb790016a00 con 0x564f9eb53cd0 2023-09-30T14:58:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.738+0000 7fb7aab2cb80 10 client.17112 fetch_fsmap learned FSMap version 51 2023-09-30T14:58:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.738+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/2931007988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x564f9e8cfd80 con 0x564f9eb53cd0 2023-09-30T14:58:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.738+0000 7fb7777fe700 1 -- 172.21.15.32:0/2931007988 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f9eb45090 con 0x564f9eb53cd0 2023-09-30T14:58:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.738+0000 7fb7777fe700 1 -- 172.21.15.32:0/2931007988 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 51) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb79001ed20 con 0x564f9eb53cd0 2023-09-30T14:58:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.739+0000 7fb7aab2cb80 10 client.17112 fetch_fsmap finished waiting for FSMap version 51 2023-09-30T14:58:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.739+0000 7fb7aab2cb80 10 client.17112 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:58:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.739+0000 7fb7aab2cb80 20 client.17112 populate_metadata read hostname 'smithi032' 2023-09-30T14:58:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.739+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/2931007988 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564f9eb67a70 0x564f9eb69e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.739+0000 7fb7aab2cb80 4 client.17112 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:58:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.739+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/2931007988 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f9ea9ddb0 con 0x564f9eb67a70 2023-09-30T14:58:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.739+0000 7fb797fff700 1 --2- 172.21.15.32:0/2931007988 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564f9eb67a70 0x564f9eb69e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.739+0000 7fb7777fe700 10 client.17112 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.739+0000 7fb797fff700 1 --2- 172.21.15.32:0/2931007988 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564f9eb67a70 0x564f9eb69e90 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68c3b2a0ecfbb93f server_cookie=7635e4c9bfa941e4 in_seq=0 out_seq=0 2023-09-30T14:58:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.742+0000 7fb7777fe700 1 -- 172.21.15.32:0/2931007988 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564f9ea9ddb0 con 0x564f9eb67a70 2023-09-30T14:58:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.742+0000 7fb7777fe700 10 client.17112 handle_command_reply: tid=0 2023-09-30T14:58:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.742+0000 7fb7aab2cb80 1 client.17112 shutdown 2023-09-30T14:58:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.742+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/2931007988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53cd0 msgr2=0x564f9eb630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.742+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/2931007988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53cd0 0x564f9eb630c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb790009fd0 tx=0x7fb790007b10 comp rx=0 tx=0).stop 2023-09-30T14:58:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.742+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/2931007988 shutdown_connections 2023-09-30T14:58:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.742+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/2931007988 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564f9eb67a70 0x564f9eb69e90 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.742+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/2931007988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53cd0 0x564f9eb630c0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.742+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/2931007988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb52200 0x564f9eb5bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.742+0000 7fb7777fe700 0 client.17112 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.742+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/2931007988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb53310 0x564f9eb5c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.742+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/2931007988 >> 172.21.15.32:0/2931007988 conn(0x564f9eac6200 msgr2=0x564f9eb4c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.743+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/2931007988 shutdown_connections 2023-09-30T14:58:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.743+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/2931007988 wait complete. 2023-09-30T14:58:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.743+0000 7fb7aab2cb80 20 client.17112 trim_cache size 0 max 16384 2023-09-30T14:58:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.761+0000 7fb7aab2cb80 1 Processor -- start 2023-09-30T14:58:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.762+0000 7fb7aab2cb80 1 -- start start 2023-09-30T14:58:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.762+0000 7fb7aab2cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb52200 0x564f9eb53710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.762+0000 7fb7aab2cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb53310 0x564f9eb62fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.762+0000 7fb7aab2cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb53cd0 0x564f9eb56aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.762+0000 7fb7aab2cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9e9258c0 con 0x564f9eb52200 2023-09-30T14:58:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.762+0000 7fb7aab2cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9eb5b6b0 con 0x564f9eb53cd0 2023-09-30T14:58:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.762+0000 7fb7aab2cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9eb53100 con 0x564f9eb53310 2023-09-30T14:58:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.762+0000 7fb797fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb52200 0x564f9eb53710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.762+0000 7fb797fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb52200 0x564f9eb53710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39590/0 (socket says 172.21.15.32:39590) 2023-09-30T14:58:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.762+0000 7fb797fff700 1 -- 172.21.15.32:0/1755133102 learned_addr learned my addr 172.21.15.32:0/1755133102 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.762+0000 7fb79cee1700 1 --2- 172.21.15.32:0/1755133102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb53cd0 0x564f9eb56aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.762+0000 7fb797fff700 1 -- 172.21.15.32:0/1755133102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb53310 msgr2=0x564f9eb62fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.762+0000 7fb797fff700 1 --2- 172.21.15.32:0/1755133102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb53310 0x564f9eb62fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.762+0000 7fb797fff700 1 -- 172.21.15.32:0/1755133102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb53cd0 msgr2=0x564f9eb56aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.762+0000 7fb797fff700 1 --2- 172.21.15.32:0/1755133102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb53cd0 0x564f9eb56aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.762+0000 7fb797fff700 1 -- 172.21.15.32:0/1755133102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f9e8b8c00 con 0x564f9eb52200 2023-09-30T14:58:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.763+0000 7fb797fff700 1 --2- 172.21.15.32:0/1755133102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb52200 0x564f9eb53710 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb78800a9c0 tx=0x7fb788014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c83605f3886f9f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.764+0000 7fb77effd700 1 -- 172.21.15.32:0/1755133102 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78802c070 con 0x564f9eb52200 2023-09-30T14:58:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.764+0000 7fb77effd700 1 -- 172.21.15.32:0/1755133102 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7880043f0 con 0x564f9eb52200 2023-09-30T14:58:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.764+0000 7fb77effd700 1 -- 172.21.15.32:0/1755133102 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb788010040 con 0x564f9eb52200 2023-09-30T14:58:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.765+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/1755133102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb52200 msgr2=0x564f9eb53710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.765+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/1755133102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb52200 0x564f9eb53710 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb78800a9c0 tx=0x7fb788014000 comp rx=0 tx=0).stop 2023-09-30T14:58:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.765+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/1755133102 shutdown_connections 2023-09-30T14:58:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.765+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/1755133102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb53cd0 0x564f9eb56aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.765+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/1755133102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb52200 0x564f9eb53710 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.765+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/1755133102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb53310 0x564f9eb62fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.766+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/1755133102 >> 172.21.15.32:0/1755133102 conn(0x564f9eac6200 msgr2=0x564f9eb6e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.767+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/1755133102 shutdown_connections 2023-09-30T14:58:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.769+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/1755133102 wait complete. 2023-09-30T14:58:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.769+0000 7fb7aab2cb80 1 -- start start 2023-09-30T14:58:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.769+0000 7fb7aab2cb80 1 Processor -- start 2023-09-30T14:58:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb7aab2cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb52200 0x564f9eb60c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb7aab2cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb53310 0x564f9eb611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb7aab2cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53cd0 0x564f9eb621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb7aab2cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9eb626f0 con 0x564f9eb53cd0 2023-09-30T14:58:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb7aab2cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9eb62860 con 0x564f9eb53310 2023-09-30T14:58:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb7aab2cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9eb62a20 con 0x564f9eb52200 2023-09-30T14:58:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb7aab2cb80 10 client.0 _ll_register_callbacks cb 0x7fb7900094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:58:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb79cee1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53cd0 0x564f9eb621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb7977fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb53310 0x564f9eb611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb79cee1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53cd0 0x564f9eb621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39600/0 (socket says 172.21.15.32:39600) 2023-09-30T14:58:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb7977fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb53310 0x564f9eb611a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51074/0 (socket says 172.21.15.32:51074) 2023-09-30T14:58:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb79cee1700 1 -- 172.21.15.32:0/1642958468 learned_addr learned my addr 172.21.15.32:0/1642958468 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb797fff700 1 --2- 172.21.15.32:0/1642958468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb52200 0x564f9eb60c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb79cee1700 1 -- 172.21.15.32:0/1642958468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb52200 msgr2=0x564f9eb60c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb79cee1700 1 --2- 172.21.15.32:0/1642958468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb52200 0x564f9eb60c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb79cee1700 1 -- 172.21.15.32:0/1642958468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb53310 msgr2=0x564f9eb611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb79cee1700 1 --2- 172.21.15.32:0/1642958468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb53310 0x564f9eb611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb79cee1700 1 -- 172.21.15.32:0/1642958468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f9e8f61f0 con 0x564f9eb53cd0 2023-09-30T14:58:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb79cee1700 1 --2- 172.21.15.32:0/1642958468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53cd0 0x564f9eb621b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb790002230 tx=0x7fb790015830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e6ebbaad623f0d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.770+0000 7fb7777fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:58:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.771+0000 7fb7777fe700 1 -- 172.21.15.32:0/1642958468 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79000e040 con 0x564f9eb53cd0 2023-09-30T14:58:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.771+0000 7fb7777fe700 1 -- 172.21.15.32:0/1642958468 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb790009a70 con 0x564f9eb53cd0 2023-09-30T14:58:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.771+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/1642958468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb78c098eb0 con 0x564f9eb53cd0 2023-09-30T14:58:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.771+0000 7fb7777fe700 1 -- 172.21.15.32:0/1642958468 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb790003c10 con 0x564f9eb53cd0 2023-09-30T14:58:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.771+0000 7fb7aab2cb80 10 client.17118 fetch_fsmap learned FSMap version 51 2023-09-30T14:58:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.771+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/1642958468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x564f9e8ce250 con 0x564f9eb53cd0 2023-09-30T14:58:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.771+0000 7fb7777fe700 1 -- 172.21.15.32:0/1642958468 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb78c098eb0 con 0x564f9eb53cd0 2023-09-30T14:58:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.771+0000 7fb7777fe700 1 -- 172.21.15.32:0/1642958468 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 51) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb79003c070 con 0x564f9eb53cd0 2023-09-30T14:58:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.771+0000 7fb7aab2cb80 10 client.17118 fetch_fsmap finished waiting for FSMap version 51 2023-09-30T14:58:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.771+0000 7fb7aab2cb80 10 client.17118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:58:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.771+0000 7fb7aab2cb80 20 client.17118 populate_metadata read hostname 'smithi032' 2023-09-30T14:58:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.771+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/1642958468 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564f9ebbc6d0 0x564f9ebbeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.771+0000 7fb7aab2cb80 4 client.17118 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:58:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.771+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/1642958468 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564f9ea9dad0 con 0x564f9ebbc6d0 2023-09-30T14:58:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.772+0000 7fb797fff700 1 --2- 172.21.15.32:0/1642958468 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564f9ebbc6d0 0x564f9ebbeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.772+0000 7fb797fff700 1 --2- 172.21.15.32:0/1642958468 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564f9ebbc6d0 0x564f9ebbeb70 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e89391fdbe60ac7a server_cookie=40e9e39e7316eeda in_seq=0 out_seq=0 2023-09-30T14:58:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:16.772+0000 7fb7777fe700 10 client.17118 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:17.050 INFO:tasks.workunit.client.0.smithi032.stdout: 10 273032 36.18 MB/sec execute 290 sec latency 119.913 ms 2023-09-30T14:58:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.279+0000 7fb7777fe700 1 -- 172.21.15.32:0/1642958468 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564f9ea9dad0 con 0x564f9ebbc6d0 2023-09-30T14:58:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.279+0000 7fb7777fe700 10 client.17118 handle_command_reply: tid=0 2023-09-30T14:58:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.279+0000 7fb7aab2cb80 1 client.17118 shutdown 2023-09-30T14:58:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.280+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/1642958468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53cd0 msgr2=0x564f9eb621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.280+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/1642958468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53cd0 0x564f9eb621b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb790002230 tx=0x7fb790015830 comp rx=0 tx=0).stop 2023-09-30T14:58:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.280+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/1642958468 shutdown_connections 2023-09-30T14:58:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.280+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/1642958468 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564f9ebbc6d0 0x564f9ebbeb70 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.280+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/1642958468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564f9eb53cd0 0x564f9eb621b0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.280+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/1642958468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564f9eb52200 0x564f9eb60c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.280+0000 7fb7777fe700 0 client.17118 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.280+0000 7fb7aab2cb80 1 --2- 172.21.15.32:0/1642958468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f9eb53310 0x564f9eb611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.280+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/1642958468 >> 172.21.15.32:0/1642958468 conn(0x564f9eac6200 msgr2=0x564f9eb56aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.280+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/1642958468 shutdown_connections 2023-09-30T14:58:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.280+0000 7fb7aab2cb80 1 -- 172.21.15.32:0/1642958468 wait complete. 2023-09-30T14:58:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.280+0000 7fb7aab2cb80 20 client.17118 trim_cache size 0 max 16384 2023-09-30T14:58:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:58:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:58:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:58:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:58:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.280+0000 7fb79e944700 1 -- 172.21.15.32:0/1061084842 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb7800a0fa0 msgr2=0x7fb7800a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.280+0000 7fb79e944700 1 --2- 172.21.15.32:0/1061084842 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb7800a0fa0 0x7fb7800a3460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb798100db0 tx=0x7fb78800d040 comp rx=0 tx=0).stop 2023-09-30T14:58:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.280+0000 7fb79e944700 1 -- 172.21.15.32:0/1061084842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb798100f50 msgr2=0x7fb798194f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.280+0000 7fb79e944700 1 --2- 172.21.15.32:0/1061084842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb798100f50 0x7fb798194f20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00c010 tx=0x7fb78c00f690 comp rx=0 tx=0).stop 2023-09-30T14:58:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.281+0000 7fb79e944700 1 -- 172.21.15.32:0/1061084842 shutdown_connections 2023-09-30T14:58:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.281+0000 7fb79e944700 1 --2- 172.21.15.32:0/1061084842 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb7800a0fa0 0x7fb7800a3460 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.281+0000 7fb79e944700 1 --2- 172.21.15.32:0/1061084842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7981018f0 0x7fb798195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.281+0000 7fb79e944700 1 --2- 172.21.15.32:0/1061084842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7980ffd60 0x7fb7981949e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.281+0000 7fb79e944700 1 --2- 172.21.15.32:0/1061084842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb798100f50 0x7fb798194f20 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.281+0000 7fb79e944700 1 -- 172.21.15.32:0/1061084842 >> 172.21.15.32:0/1061084842 conn(0x7fb7980fb2b0 msgr2=0x7fb798104490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.281+0000 7fb79e944700 1 -- 172.21.15.32:0/1061084842 shutdown_connections 2023-09-30T14:58:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:17.281+0000 7fb79e944700 1 -- 172.21.15.32:0/1061084842 wait complete. 2023-09-30T14:58:17.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:17 smithi156 ceph-mon[103579]: pgmap v887: 97 pgs: 97 active+clean; 791 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.8 MiB/s wr, 699 op/s 2023-09-30T14:58:17.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:17 smithi148 ceph-mon[102355]: pgmap v887: 97 pgs: 97 active+clean; 791 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.8 MiB/s wr, 699 op/s 2023-09-30T14:58:17.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:17 smithi032 ceph-mon[98592]: pgmap v887: 97 pgs: 97 active+clean; 791 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.8 MiB/s wr, 699 op/s 2023-09-30T14:58:18.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 273597 36.15 MB/sec execute 291 sec latency 215.046 ms 2023-09-30T14:58:19.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 274108 36.13 MB/sec execute 292 sec latency 258.857 ms 2023-09-30T14:58:19.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:19 smithi156 ceph-mon[103579]: pgmap v888: 97 pgs: 97 active+clean; 791 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.5 MiB/s wr, 658 op/s 2023-09-30T14:58:19.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:19 smithi148 ceph-mon[102355]: pgmap v888: 97 pgs: 97 active+clean; 791 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.5 MiB/s wr, 658 op/s 2023-09-30T14:58:19.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:19 smithi032 ceph-mon[98592]: pgmap v888: 97 pgs: 97 active+clean; 791 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.5 MiB/s wr, 658 op/s 2023-09-30T14:58:20.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 274774 36.12 MB/sec execute 293 sec latency 92.634 ms 2023-09-30T14:58:21.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 275400 36.13 MB/sec execute 294 sec latency 78.777 ms 2023-09-30T14:58:21.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:21 smithi156 ceph-mon[103579]: pgmap v889: 97 pgs: 97 active+clean; 794 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.9 MiB/s wr, 704 op/s 2023-09-30T14:58:21.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:21 smithi148 ceph-mon[102355]: pgmap v889: 97 pgs: 97 active+clean; 794 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.9 MiB/s wr, 704 op/s 2023-09-30T14:58:21.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:21 smithi032 ceph-mon[98592]: pgmap v889: 97 pgs: 97 active+clean; 794 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.9 MiB/s wr, 704 op/s 2023-09-30T14:58:22.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 276059 36.12 MB/sec execute 295 sec latency 69.586 ms 2023-09-30T14:58:23.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 276708 36.13 MB/sec execute 296 sec latency 67.432 ms 2023-09-30T14:58:23.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:23 smithi156 ceph-mon[103579]: pgmap v890: 97 pgs: 97 active+clean; 797 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.1 MiB/s wr, 684 op/s 2023-09-30T14:58:23.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:23 smithi148 ceph-mon[102355]: pgmap v890: 97 pgs: 97 active+clean; 797 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.1 MiB/s wr, 684 op/s 2023-09-30T14:58:23.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:23 smithi032 ceph-mon[98592]: pgmap v890: 97 pgs: 97 active+clean; 797 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.1 MiB/s wr, 684 op/s 2023-09-30T14:58:24.046 INFO:tasks.workunit.client.0.smithi032.stdout: 10 277142 36.09 MB/sec execute 297 sec latency 288.902 ms 2023-09-30T14:58:25.047 INFO:tasks.workunit.client.0.smithi032.stdout: 10 277716 36.05 MB/sec execute 298 sec latency 64.816 ms 2023-09-30T14:58:25.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:25 smithi156 ceph-mon[103579]: pgmap v891: 97 pgs: 97 active+clean; 778 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.2 MiB/s wr, 676 op/s 2023-09-30T14:58:25.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:25 smithi148 ceph-mon[102355]: pgmap v891: 97 pgs: 97 active+clean; 778 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.2 MiB/s wr, 676 op/s 2023-09-30T14:58:25.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:25 smithi032 ceph-mon[98592]: pgmap v891: 97 pgs: 97 active+clean; 778 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.2 MiB/s wr, 676 op/s 2023-09-30T14:58:26.047 INFO:tasks.workunit.client.0.smithi032.stdout: 10 278447 36.06 MB/sec execute 299 sec latency 67.053 ms 2023-09-30T14:58:27.047 INFO:tasks.workunit.client.0.smithi032.stdout: 10 279236 36.08 MB/sec execute 300 sec latency 87.587 ms 2023-09-30T14:58:27.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:27 smithi156 ceph-mon[103579]: pgmap v892: 97 pgs: 97 active+clean; 776 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.6 MiB/s wr, 707 op/s 2023-09-30T14:58:27.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:27 smithi148 ceph-mon[102355]: pgmap v892: 97 pgs: 97 active+clean; 776 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.6 MiB/s wr, 707 op/s 2023-09-30T14:58:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:27 smithi032 ceph-mon[98592]: pgmap v892: 97 pgs: 97 active+clean; 776 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.6 MiB/s wr, 707 op/s 2023-09-30T14:58:28.047 INFO:tasks.workunit.client.0.smithi032.stdout: 10 279871 36.09 MB/sec execute 301 sec latency 125.158 ms 2023-09-30T14:58:28.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:28 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:58:28.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:28 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:58:28.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:28 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:58:29.047 INFO:tasks.workunit.client.0.smithi032.stdout: 10 280416 36.08 MB/sec execute 302 sec latency 285.311 ms 2023-09-30T14:58:29.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:29 smithi156 ceph-mon[103579]: pgmap v893: 97 pgs: 97 active+clean; 783 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.6 MiB/s wr, 686 op/s 2023-09-30T14:58:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:29 smithi148 ceph-mon[102355]: pgmap v893: 97 pgs: 97 active+clean; 783 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.6 MiB/s wr, 686 op/s 2023-09-30T14:58:29.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:29 smithi032 ceph-mon[98592]: pgmap v893: 97 pgs: 97 active+clean; 783 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.6 MiB/s wr, 686 op/s 2023-09-30T14:58:30.047 INFO:tasks.workunit.client.0.smithi032.stdout: 10 280921 36.05 MB/sec execute 303 sec latency 225.902 ms 2023-09-30T14:58:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.409+0000 7fc1d19c8700 1 -- 172.21.15.32:0/3528757015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1cc104220 msgr2=0x7fc1cc106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.409+0000 7fc1d19c8700 1 --2- 172.21.15.32:0/3528757015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1cc104220 0x7fc1cc106610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00098f0 tx=0x7fc1c0009c00 comp rx=0 tx=0).stop 2023-09-30T14:58:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.410+0000 7fc1d19c8700 1 -- 172.21.15.32:0/3528757015 shutdown_connections 2023-09-30T14:58:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.410+0000 7fc1d19c8700 1 --2- 172.21.15.32:0/3528757015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1cc106b50 0x7fc1cc108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.410+0000 7fc1d19c8700 1 --2- 172.21.15.32:0/3528757015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1cc1018f0 0x7fc1cc103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.410+0000 7fc1d19c8700 1 --2- 172.21.15.32:0/3528757015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1cc104220 0x7fc1cc106610 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.410+0000 7fc1d19c8700 1 -- 172.21.15.32:0/3528757015 >> 172.21.15.32:0/3528757015 conn(0x7fc1cc0fb300 msgr2=0x7fc1cc0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.410+0000 7fc1d19c8700 1 -- 172.21.15.32:0/3528757015 shutdown_connections 2023-09-30T14:58:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.410+0000 7fc1d19c8700 1 -- 172.21.15.32:0/3528757015 wait complete. 2023-09-30T14:58:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.410+0000 7fc1d19c8700 1 Processor -- start 2023-09-30T14:58:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1d19c8700 1 -- start start 2023-09-30T14:58:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1d19c8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1cc1018f0 0x7fc1cc0ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1d19c8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1cc104220 0x7fc1cc0ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1d19c8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1cc106b50 0x7fc1cc100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1d19c8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc1a0c10 con 0x7fc1cc104220 2023-09-30T14:58:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1d19c8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc1a0d50 con 0x7fc1cc106b50 2023-09-30T14:58:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1d19c8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc1a0e90 con 0x7fc1cc1018f0 2023-09-30T14:58:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1caffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1cc1018f0 0x7fc1cc0ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1caffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1cc1018f0 0x7fc1cc0ff8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:53992/0 (socket says 172.21.15.32:53992) 2023-09-30T14:58:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1caffd700 1 -- 172.21.15.32:0/1478723286 learned_addr learned my addr 172.21.15.32:0/1478723286 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1cb7fe700 1 --2- 172.21.15.32:0/1478723286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1cc106b50 0x7fc1cc100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1ca7fc700 1 --2- 172.21.15.32:0/1478723286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1cc104220 0x7fc1cc0ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1caffd700 1 -- 172.21.15.32:0/1478723286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1cc106b50 msgr2=0x7fc1cc100320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1caffd700 1 --2- 172.21.15.32:0/1478723286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1cc106b50 0x7fc1cc100320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1caffd700 1 -- 172.21.15.32:0/1478723286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1cc104220 msgr2=0x7fc1cc0ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1caffd700 1 --2- 172.21.15.32:0/1478723286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1cc104220 0x7fc1cc0ffde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.411+0000 7fc1caffd700 1 -- 172.21.15.32:0/1478723286 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc0095f0 con 0x7fc1cc1018f0 2023-09-30T14:58:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.412+0000 7fc1caffd700 1 --2- 172.21.15.32:0/1478723286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1cc1018f0 0x7fc1cc0ff8a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00ec10 tx=0x7fc1bc00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.412+0000 7fc1d09c6700 1 -- 172.21.15.32:0/1478723286 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc009a10 con 0x7fc1cc1018f0 2023-09-30T14:58:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.412+0000 7fc1d19c8700 1 -- 172.21.15.32:0/1478723286 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c4009c40 con 0x7fc1cc1018f0 2023-09-30T14:58:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.412+0000 7fc1d09c6700 1 -- 172.21.15.32:0/1478723286 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1bc0044e0 con 0x7fc1cc1018f0 2023-09-30T14:58:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.412+0000 7fc1d09c6700 1 -- 172.21.15.32:0/1478723286 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc010430 con 0x7fc1cc1018f0 2023-09-30T14:58:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.412+0000 7fc1d19c8700 1 -- 172.21.15.32:0/1478723286 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c0009520 con 0x7fc1cc1018f0 2023-09-30T14:58:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.415+0000 7fc1d09c6700 1 -- 172.21.15.32:0/1478723286 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fc1bc004020 con 0x7fc1cc1018f0 2023-09-30T14:58:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.415+0000 7fc1d09c6700 1 --2- 172.21.15.32:0/1478723286 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1b40a0fa0 0x7fc1b40a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.416+0000 7fc1ca7fc700 1 --2- 172.21.15.32:0/1478723286 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1b40a0fa0 0x7fc1b40a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.416+0000 7fc1d09c6700 1 -- 172.21.15.32:0/1478723286 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc1bc014070 con 0x7fc1cc1018f0 2023-09-30T14:58:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.416+0000 7fc1ca7fc700 1 --2- 172.21.15.32:0/1478723286 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1b40a0fa0 0x7fc1b40a3460 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00098c0 tx=0x7fc1c0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.417+0000 7fc1ddbb0b80 1 Processor -- start 2023-09-30T14:58:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.417+0000 7fc1ddbb0b80 1 -- start start 2023-09-30T14:58:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.417+0000 7fc1ddbb0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d796200 0x559d3d796620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.417+0000 7fc1ddbb0b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797310 0x559d3d797790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.417+0000 7fc1ddbb0b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d797cd0 0x559d3d79e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.417+0000 7fc1ddbb0b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3d6fe400 con 0x559d3d797310 2023-09-30T14:58:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.417+0000 7fc1ddbb0b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3d6efbc0 con 0x559d3d797cd0 2023-09-30T14:58:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.417+0000 7fc1ddbb0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3d6ed4e0 con 0x559d3d796200 2023-09-30T14:58:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.417+0000 7fc1ca7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797310 0x559d3d797790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.417+0000 7fc1ca7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797310 0x559d3d797790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39620/0 (socket says 172.21.15.32:39620) 2023-09-30T14:58:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.417+0000 7fc1ca7fc700 1 -- 172.21.15.32:0/2686632545 learned_addr learned my addr 172.21.15.32:0/2686632545 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.417+0000 7fc1caffd700 1 --2- 172.21.15.32:0/2686632545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d796200 0x559d3d796620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.417+0000 7fc1cb7fe700 1 --2- 172.21.15.32:0/2686632545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d797cd0 0x559d3d79e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.418+0000 7fc1ca7fc700 1 -- 172.21.15.32:0/2686632545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d796200 msgr2=0x559d3d796620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.418+0000 7fc1ca7fc700 1 --2- 172.21.15.32:0/2686632545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d796200 0x559d3d796620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.418+0000 7fc1ca7fc700 1 -- 172.21.15.32:0/2686632545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d797cd0 msgr2=0x559d3d79e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.418+0000 7fc1ca7fc700 1 --2- 172.21.15.32:0/2686632545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d797cd0 0x559d3d79e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.418+0000 7fc1ca7fc700 1 -- 172.21.15.32:0/2686632545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d3d74b870 con 0x559d3d797310 2023-09-30T14:58:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.418+0000 7fc1ca7fc700 1 --2- 172.21.15.32:0/2686632545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797310 0x559d3d797790 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00098f0 tx=0x7fc1c001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fa2349c4608ece server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.418+0000 7fc1b1ffb700 1 -- 172.21.15.32:0/2686632545 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0028070 con 0x559d3d797310 2023-09-30T14:58:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.418+0000 7fc1b1ffb700 1 -- 172.21.15.32:0/2686632545 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1c000f820 con 0x559d3d797310 2023-09-30T14:58:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.418+0000 7fc1b1ffb700 1 -- 172.21.15.32:0/2686632545 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c001c040 con 0x559d3d797310 2023-09-30T14:58:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.419+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2686632545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797310 msgr2=0x559d3d797790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.419+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/2686632545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797310 0x559d3d797790 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00098f0 tx=0x7fc1c001d000 comp rx=0 tx=0).stop 2023-09-30T14:58:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.419+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2686632545 shutdown_connections 2023-09-30T14:58:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.419+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/2686632545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d797cd0 0x559d3d79e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.419+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/2686632545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d796200 0x559d3d796620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.419+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/2686632545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797310 0x559d3d797790 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.419+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2686632545 >> 172.21.15.32:0/2686632545 conn(0x559d3d70a200 msgr2=0x559d3d790ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.419+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2686632545 shutdown_connections 2023-09-30T14:58:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.419+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2686632545 wait complete. 2023-09-30T14:58:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.420+0000 7fc1ddbb0b80 1 -- start start 2023-09-30T14:58:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.420+0000 7fc1ddbb0b80 1 Processor -- start 2023-09-30T14:58:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.420+0000 7fc1ddbb0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d796200 0x559d3d79fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.420+0000 7fc1ddbb0b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797310 0x559d3d7a0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.420+0000 7fc1ddbb0b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d797cd0 0x559d3d7a7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.420+0000 7fc1ddbb0b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3d6e8dd0 con 0x559d3d797310 2023-09-30T14:58:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.420+0000 7fc1ddbb0b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3d6856b0 con 0x559d3d797cd0 2023-09-30T14:58:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.420+0000 7fc1ddbb0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3d685350 con 0x559d3d796200 2023-09-30T14:58:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.420+0000 7fc1ddbb0b80 10 client.0 _ll_register_callbacks cb 0x559d3d4e27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:58:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.420+0000 7fc1caffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d796200 0x559d3d79fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.420+0000 7fc1ca7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797310 0x559d3d7a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.420+0000 7fc1ca7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797310 0x559d3d7a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39626/0 (socket says 172.21.15.32:39626) 2023-09-30T14:58:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.420+0000 7fc1caffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d796200 0x559d3d79fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54006/0 (socket says 172.21.15.32:54006) 2023-09-30T14:58:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.420+0000 7fc1ca7fc700 1 -- 172.21.15.32:0/2637826943 learned_addr learned my addr 172.21.15.32:0/2637826943 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.420+0000 7fc1cb7fe700 1 --2- 172.21.15.32:0/2637826943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d797cd0 0x559d3d7a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.421+0000 7fc1caffd700 1 -- 172.21.15.32:0/2637826943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d797cd0 msgr2=0x559d3d7a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.421+0000 7fc1caffd700 1 --2- 172.21.15.32:0/2637826943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d797cd0 0x559d3d7a7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.421+0000 7fc1caffd700 1 -- 172.21.15.32:0/2637826943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797310 msgr2=0x559d3d7a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.421+0000 7fc1caffd700 1 --2- 172.21.15.32:0/2637826943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797310 0x559d3d7a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.421+0000 7fc1caffd700 1 -- 172.21.15.32:0/2637826943 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc109bb0 con 0x559d3d796200 2023-09-30T14:58:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.421+0000 7fc1ca7fc700 1 --2- 172.21.15.32:0/2637826943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797310 0x559d3d7a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:58:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.421+0000 7fc1caffd700 1 --2- 172.21.15.32:0/2637826943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d796200 0x559d3d79fed0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0ca770 tx=0x7fc1bc09a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b430c2f309f3871 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.421+0000 7fc1aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:58:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.421+0000 7fc1aaffd700 1 -- 172.21.15.32:0/2637826943 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0a24c0 con 0x559d3d796200 2023-09-30T14:58:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.421+0000 7fc1aaffd700 1 -- 172.21.15.32:0/2637826943 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc1bc003680 con 0x559d3d796200 2023-09-30T14:58:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.421+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2637826943 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d3d789090 con 0x559d3d796200 2023-09-30T14:58:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.421+0000 7fc1aaffd700 1 -- 172.21.15.32:0/2637826943 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1bc0b81e0 con 0x559d3d796200 2023-09-30T14:58:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.421+0000 7fc1ddbb0b80 10 client.26788 fetch_fsmap learned FSMap version 52 2023-09-30T14:58:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.421+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2637826943 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x559d3d513d80 con 0x559d3d796200 2023-09-30T14:58:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.422+0000 7fc1aaffd700 1 -- 172.21.15.32:0/2637826943 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559d3d789090 con 0x559d3d796200 2023-09-30T14:58:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.422+0000 7fc1aaffd700 1 -- 172.21.15.32:0/2637826943 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 52) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc1bc099620 con 0x559d3d796200 2023-09-30T14:58:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.422+0000 7fc1ddbb0b80 10 client.26788 fetch_fsmap finished waiting for FSMap version 52 2023-09-30T14:58:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.422+0000 7fc1ddbb0b80 10 client.26788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:58:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.422+0000 7fc1ddbb0b80 20 client.26788 populate_metadata read hostname 'smithi032' 2023-09-30T14:58:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.422+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/2637826943 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x559d3d7abd30 0x559d3d7ae150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.422+0000 7fc1ddbb0b80 4 client.26788 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:58:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.422+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2637826943 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d3d6e1db0 con 0x559d3d7abd30 2023-09-30T14:58:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.422+0000 7fc1ca7fc700 1 --2- 172.21.15.32:0/2637826943 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x559d3d7abd30 0x559d3d7ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.422+0000 7fc1ca7fc700 1 --2- 172.21.15.32:0/2637826943 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x559d3d7abd30 0x559d3d7ae150 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fe9f1f37070f3de server_cookie=d39e035d87693b15 in_seq=0 out_seq=0 2023-09-30T14:58:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.422+0000 7fc1aaffd700 10 client.26788 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.424+0000 7fc1aaffd700 1 -- 172.21.15.32:0/2637826943 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559d3d6e1db0 con 0x559d3d7abd30 2023-09-30T14:58:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.424+0000 7fc1aaffd700 10 client.26788 handle_command_reply: tid=0 2023-09-30T14:58:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.424+0000 7fc1ddbb0b80 1 client.26788 shutdown 2023-09-30T14:58:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.424+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2637826943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d796200 msgr2=0x559d3d79fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.424+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/2637826943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d796200 0x559d3d79fed0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0ca770 tx=0x7fc1bc09a130 comp rx=0 tx=0).stop 2023-09-30T14:58:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.426+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2637826943 shutdown_connections 2023-09-30T14:58:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.426+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/2637826943 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x559d3d7abd30 0x559d3d7ae150 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.426+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/2637826943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d797cd0 0x559d3d7a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.426+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/2637826943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d796200 0x559d3d79fed0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.426+0000 7fc1aaffd700 0 client.26788 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.426+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/2637826943 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797310 0x559d3d7a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.426+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2637826943 >> 172.21.15.32:0/2637826943 conn(0x559d3d70a200 msgr2=0x559d3d790590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.426+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2637826943 shutdown_connections 2023-09-30T14:58:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.426+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2637826943 wait complete. 2023-09-30T14:58:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.426+0000 7fc1ddbb0b80 20 client.26788 trim_cache size 0 max 16384 2023-09-30T14:58:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.442+0000 7fc1ddbb0b80 1 Processor -- start 2023-09-30T14:58:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.442+0000 7fc1ddbb0b80 1 -- start start 2023-09-30T14:58:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1ddbb0b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d796200 0x559d3d79ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1ddbb0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d797310 0x559d3d7a73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1ddbb0b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797cd0 0x559d3d790590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1ddbb0b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3d5698c0 con 0x559d3d797cd0 2023-09-30T14:58:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1ddbb0b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3d79fa40 con 0x559d3d796200 2023-09-30T14:58:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1ddbb0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3d796ef0 con 0x559d3d797310 2023-09-30T14:58:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1cb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797cd0 0x559d3d790590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1cb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797cd0 0x559d3d790590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39634/0 (socket says 172.21.15.32:39634) 2023-09-30T14:58:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1cb7fe700 1 -- 172.21.15.32:0/2581251743 learned_addr learned my addr 172.21.15.32:0/2581251743 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1ca7fc700 1 --2- 172.21.15.32:0/2581251743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d797310 0x559d3d7a73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1caffd700 1 --2- 172.21.15.32:0/2581251743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d796200 0x559d3d79ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1cb7fe700 1 -- 172.21.15.32:0/2581251743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d797310 msgr2=0x559d3d7a73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1cb7fe700 1 --2- 172.21.15.32:0/2581251743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d797310 0x559d3d7a73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1cb7fe700 1 -- 172.21.15.32:0/2581251743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d796200 msgr2=0x559d3d79ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1cb7fe700 1 --2- 172.21.15.32:0/2581251743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d796200 0x559d3d79ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1cb7fe700 1 -- 172.21.15.32:0/2581251743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d3d4fcc00 con 0x559d3d797cd0 2023-09-30T14:58:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.443+0000 7fc1caffd700 1 --2- 172.21.15.32:0/2581251743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d796200 0x559d3d79ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:58:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.444+0000 7fc1cb7fe700 1 --2- 172.21.15.32:0/2581251743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797cd0 0x559d3d790590 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40158c0 tx=0x7fc1c4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c4d54b720d584e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.444+0000 7fc1b27fc700 1 -- 172.21.15.32:0/2581251743 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c400e040 con 0x559d3d797cd0 2023-09-30T14:58:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.444+0000 7fc1b27fc700 1 -- 172.21.15.32:0/2581251743 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1c400f040 con 0x559d3d797cd0 2023-09-30T14:58:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.444+0000 7fc1b27fc700 1 -- 172.21.15.32:0/2581251743 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c40179d0 con 0x559d3d797cd0 2023-09-30T14:58:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.445+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2581251743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797cd0 msgr2=0x559d3d790590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.445+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/2581251743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797cd0 0x559d3d790590 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40158c0 tx=0x7fc1c4009920 comp rx=0 tx=0).stop 2023-09-30T14:58:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.445+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2581251743 shutdown_connections 2023-09-30T14:58:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.445+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/2581251743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d797cd0 0x559d3d790590 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.445+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/2581251743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d796200 0x559d3d79ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.445+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/2581251743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d797310 0x559d3d7a73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.445+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2581251743 >> 172.21.15.32:0/2581251743 conn(0x559d3d70a200 msgr2=0x559d3d790ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.445+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2581251743 shutdown_connections 2023-09-30T14:58:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.445+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/2581251743 wait complete. 2023-09-30T14:58:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.445+0000 7fc1ddbb0b80 1 -- start start 2023-09-30T14:58:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.445+0000 7fc1ddbb0b80 1 Processor -- start 2023-09-30T14:58:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.446+0000 7fc1ddbb0b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d796200 0x559d3d7a5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.446+0000 7fc1ddbb0b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d797310 0x559d3d7a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.446+0000 7fc1ddbb0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d797cd0 0x559d3d7a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.446+0000 7fc1ddbb0b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3d7a6e00 con 0x559d3d796200 2023-09-30T14:58:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.446+0000 7fc1ddbb0b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3d7a6f70 con 0x559d3d797310 2023-09-30T14:58:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.446+0000 7fc1ddbb0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3d7a7130 con 0x559d3d797cd0 2023-09-30T14:58:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.446+0000 7fc1ddbb0b80 10 client.0 _ll_register_callbacks cb 0x559d3d4e27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:58:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.446+0000 7fc1caffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d796200 0x559d3d7a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.446+0000 7fc1caffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d796200 0x559d3d7a5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39638/0 (socket says 172.21.15.32:39638) 2023-09-30T14:58:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.446+0000 7fc1caffd700 1 -- 172.21.15.32:0/3488259827 learned_addr learned my addr 172.21.15.32:0/3488259827 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.446+0000 7fc1ca7fc700 1 --2- 172.21.15.32:0/3488259827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d797310 0x559d3d7a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.446+0000 7fc1cb7fe700 1 --2- 172.21.15.32:0/3488259827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d797cd0 0x559d3d7a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.447+0000 7fc1ca7fc700 1 -- 172.21.15.32:0/3488259827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d797cd0 msgr2=0x559d3d7a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.447+0000 7fc1ca7fc700 1 --2- 172.21.15.32:0/3488259827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d797cd0 0x559d3d7a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.447+0000 7fc1ca7fc700 1 -- 172.21.15.32:0/3488259827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d796200 msgr2=0x559d3d7a5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.447+0000 7fc1ca7fc700 1 --2- 172.21.15.32:0/3488259827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d796200 0x559d3d7a5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.447+0000 7fc1ca7fc700 1 -- 172.21.15.32:0/3488259827 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c4015540 con 0x559d3d797310 2023-09-30T14:58:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.447+0000 7fc1ca7fc700 1 --2- 172.21.15.32:0/3488259827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d797310 0x559d3d7a58b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x559d3d7ac110 tx=0x7fc1c0005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83d68f7d6774c3e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.447+0000 7fc1aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:58:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.447+0000 7fc1aaffd700 1 -- 172.21.15.32:0/3488259827 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0028040 con 0x559d3d797310 2023-09-30T14:58:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.447+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/3488259827 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1c000f820 con 0x559d3d797310 2023-09-30T14:58:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.447+0000 7fc1aaffd700 1 -- 172.21.15.32:0/3488259827 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc1c0039c50 con 0x559d3d797310 2023-09-30T14:58:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.447+0000 7fc1aaffd700 1 -- 172.21.15.32:0/3488259827 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1c001c040 con 0x559d3d797310 2023-09-30T14:58:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.447+0000 7fc1ddbb0b80 10 client.26765 fetch_fsmap learned FSMap version 52 2023-09-30T14:58:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.447+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/3488259827 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x559d3d53a1f0 con 0x559d3d797310 2023-09-30T14:58:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.448+0000 7fc1aaffd700 1 -- 172.21.15.32:0/3488259827 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1c000f820 con 0x559d3d797310 2023-09-30T14:58:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.448+0000 7fc1aaffd700 1 -- 172.21.15.32:0/3488259827 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 52) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc1c0004eb0 con 0x559d3d797310 2023-09-30T14:58:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.448+0000 7fc1ddbb0b80 10 client.26765 fetch_fsmap finished waiting for FSMap version 52 2023-09-30T14:58:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.448+0000 7fc1ddbb0b80 10 client.26765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:58:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.448+0000 7fc1ddbb0b80 20 client.26765 populate_metadata read hostname 'smithi032' 2023-09-30T14:58:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.448+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/3488259827 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x559d3d8090d0 0x559d3d80b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.448+0000 7fc1ddbb0b80 4 client.26765 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:58:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.448+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/3488259827 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559d3d6e1ad0 con 0x559d3d8090d0 2023-09-30T14:58:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.448+0000 7fc1caffd700 1 --2- 172.21.15.32:0/3488259827 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x559d3d8090d0 0x559d3d80b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.449+0000 7fc1aaffd700 10 client.26765 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.449+0000 7fc1caffd700 1 --2- 172.21.15.32:0/3488259827 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x559d3d8090d0 0x559d3d80b4c0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebe83b71078fc5c8 server_cookie=8686b8b153764bb2 in_seq=0 out_seq=0 2023-09-30T14:58:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.813+0000 7fc1aaffd700 1 -- 172.21.15.32:0/3488259827 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559d3d6e1ad0 con 0x559d3d8090d0 2023-09-30T14:58:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.813+0000 7fc1aaffd700 10 client.26765 handle_command_reply: tid=0 2023-09-30T14:58:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.813+0000 7fc1ddbb0b80 1 client.26765 shutdown 2023-09-30T14:58:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.814+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/3488259827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d797310 msgr2=0x559d3d7a58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.814+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/3488259827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d797310 0x559d3d7a58b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x559d3d7ac110 tx=0x7fc1c0005de0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.814+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/3488259827 shutdown_connections 2023-09-30T14:58:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.814+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/3488259827 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x559d3d8090d0 0x559d3d80b4c0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.814+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/3488259827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d3d797cd0 0x559d3d7a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.814+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/3488259827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559d3d796200 0x559d3d7a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.814+0000 7fc1aaffd700 0 client.26765 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.814+0000 7fc1ddbb0b80 1 --2- 172.21.15.32:0/3488259827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d3d797310 0x559d3d7a58b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.814+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/3488259827 >> 172.21.15.32:0/3488259827 conn(0x559d3d70a200 msgr2=0x559d3d79e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.814+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/3488259827 shutdown_connections 2023-09-30T14:58:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.814+0000 7fc1ddbb0b80 1 -- 172.21.15.32:0/3488259827 wait complete. 2023-09-30T14:58:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.814+0000 7fc1ddbb0b80 20 client.26765 trim_cache size 0 max 16384 2023-09-30T14:58:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:58:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:58:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:58:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.818+0000 7fc1d19c8700 1 -- 172.21.15.32:0/1478723286 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1b40a0fa0 msgr2=0x7fc1b40a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.818+0000 7fc1d19c8700 1 --2- 172.21.15.32:0/1478723286 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1b40a0fa0 0x7fc1b40a3460 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00098c0 tx=0x7fc1c0019040 comp rx=0 tx=0).stop 2023-09-30T14:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.818+0000 7fc1d19c8700 1 -- 172.21.15.32:0/1478723286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1cc1018f0 msgr2=0x7fc1cc0ff8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.818+0000 7fc1d19c8700 1 --2- 172.21.15.32:0/1478723286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1cc1018f0 0x7fc1cc0ff8a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00ec10 tx=0x7fc1bc00c2d0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.818+0000 7fc1d19c8700 1 -- 172.21.15.32:0/1478723286 shutdown_connections 2023-09-30T14:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.818+0000 7fc1d19c8700 1 --2- 172.21.15.32:0/1478723286 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1b40a0fa0 0x7fc1b40a3460 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.818+0000 7fc1d19c8700 1 --2- 172.21.15.32:0/1478723286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1cc106b50 0x7fc1cc100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.818+0000 7fc1d19c8700 1 --2- 172.21.15.32:0/1478723286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1cc1018f0 0x7fc1cc0ff8a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.818+0000 7fc1d19c8700 1 --2- 172.21.15.32:0/1478723286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1cc104220 0x7fc1cc0ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.818+0000 7fc1d19c8700 1 -- 172.21.15.32:0/1478723286 >> 172.21.15.32:0/1478723286 conn(0x7fc1cc0fb300 msgr2=0x7fc1cc102560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.819+0000 7fc1d19c8700 1 -- 172.21.15.32:0/1478723286 shutdown_connections 2023-09-30T14:58:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:30.819+0000 7fc1d19c8700 1 -- 172.21.15.32:0/1478723286 wait complete. 2023-09-30T14:58:31.047 INFO:tasks.workunit.client.0.smithi032.stdout: 10 281416 35.99 MB/sec execute 304 sec latency 197.923 ms 2023-09-30T14:58:31.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:31 smithi156 ceph-mon[103579]: pgmap v894: 97 pgs: 97 active+clean; 790 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.5 MiB/s wr, 723 op/s 2023-09-30T14:58:31.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:31 smithi148 ceph-mon[102355]: pgmap v894: 97 pgs: 97 active+clean; 790 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.5 MiB/s wr, 723 op/s 2023-09-30T14:58:31.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:31 smithi032 ceph-mon[98592]: pgmap v894: 97 pgs: 97 active+clean; 790 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.5 MiB/s wr, 723 op/s 2023-09-30T14:58:32.047 INFO:tasks.workunit.client.0.smithi032.stdout: 10 282222 36.03 MB/sec execute 305 sec latency 48.339 ms 2023-09-30T14:58:33.047 INFO:tasks.workunit.client.0.smithi032.stdout: 10 282965 36.03 MB/sec execute 306 sec latency 128.375 ms 2023-09-30T14:58:33.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:33 smithi156 ceph-mon[103579]: pgmap v895: 97 pgs: 97 active+clean; 786 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.1 MiB/s wr, 665 op/s 2023-09-30T14:58:33.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:33 smithi148 ceph-mon[102355]: pgmap v895: 97 pgs: 97 active+clean; 786 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.1 MiB/s wr, 665 op/s 2023-09-30T14:58:33.640 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:33 smithi032 ceph-mon[98592]: pgmap v895: 97 pgs: 97 active+clean; 786 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.1 MiB/s wr, 665 op/s 2023-09-30T14:58:34.047 INFO:tasks.workunit.client.0.smithi032.stdout: 10 283609 36.03 MB/sec execute 307 sec latency 236.167 ms 2023-09-30T14:58:35.047 INFO:tasks.workunit.client.0.smithi032.stdout: 10 284139 36.02 MB/sec execute 308 sec latency 292.555 ms 2023-09-30T14:58:35.557 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:35 smithi156 ceph-mon[103579]: pgmap v896: 97 pgs: 97 active+clean; 789 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.5 MiB/s wr, 686 op/s 2023-09-30T14:58:35.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:35 smithi148 ceph-mon[102355]: pgmap v896: 97 pgs: 97 active+clean; 789 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.5 MiB/s wr, 686 op/s 2023-09-30T14:58:35.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:35 smithi032 ceph-mon[98592]: pgmap v896: 97 pgs: 97 active+clean; 789 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.5 MiB/s wr, 686 op/s 2023-09-30T14:58:36.047 INFO:tasks.workunit.client.0.smithi032.stdout: 10 284845 36.04 MB/sec execute 309 sec latency 89.732 ms 2023-09-30T14:58:37.047 INFO:tasks.workunit.client.0.smithi032.stdout: 10 285486 36.02 MB/sec execute 310 sec latency 72.562 ms 2023-09-30T14:58:37.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:37 smithi156 ceph-mon[103579]: pgmap v897: 97 pgs: 97 active+clean; 790 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.5 MiB/s wr, 742 op/s 2023-09-30T14:58:37.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:37 smithi148 ceph-mon[102355]: pgmap v897: 97 pgs: 97 active+clean; 790 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.5 MiB/s wr, 742 op/s 2023-09-30T14:58:37.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:37 smithi032 ceph-mon[98592]: pgmap v897: 97 pgs: 97 active+clean; 790 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.5 MiB/s wr, 742 op/s 2023-09-30T14:58:38.047 INFO:tasks.workunit.client.0.smithi032.stdout: 10 286222 36.01 MB/sec execute 311 sec latency 55.130 ms 2023-09-30T14:58:39.049 INFO:tasks.workunit.client.0.smithi032.stdout: 10 286925 36.02 MB/sec execute 312 sec latency 101.287 ms 2023-09-30T14:58:39.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:39 smithi148 ceph-mon[102355]: pgmap v898: 97 pgs: 97 active+clean; 791 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.7 MiB/s wr, 730 op/s 2023-09-30T14:58:39.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:39 smithi032 ceph-mon[98592]: pgmap v898: 97 pgs: 97 active+clean; 791 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.7 MiB/s wr, 730 op/s 2023-09-30T14:58:39.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:39 smithi156 ceph-mon[103579]: pgmap v898: 97 pgs: 97 active+clean; 791 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.7 MiB/s wr, 730 op/s 2023-09-30T14:58:40.049 INFO:tasks.workunit.client.0.smithi032.stdout: 10 287428 36.00 MB/sec execute 313 sec latency 332.391 ms 2023-09-30T14:58:41.049 INFO:tasks.workunit.client.0.smithi032.stdout: 10 288127 36.02 MB/sec execute 314 sec latency 60.209 ms 2023-09-30T14:58:41.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:41 smithi148 ceph-mon[102355]: pgmap v899: 97 pgs: 97 active+clean; 783 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 777 op/s 2023-09-30T14:58:41.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:41 smithi032 ceph-mon[98592]: pgmap v899: 97 pgs: 97 active+clean; 783 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 777 op/s 2023-09-30T14:58:41.810 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:41 smithi156 ceph-mon[103579]: pgmap v899: 97 pgs: 97 active+clean; 783 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 777 op/s 2023-09-30T14:58:42.049 INFO:tasks.workunit.client.0.smithi032.stdout: 10 288742 36.01 MB/sec execute 315 sec latency 74.986 ms 2023-09-30T14:58:43.049 INFO:tasks.workunit.client.0.smithi032.stdout: 10 289466 36.00 MB/sec execute 316 sec latency 92.441 ms 2023-09-30T14:58:43.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:43 smithi148 ceph-mon[102355]: pgmap v900: 97 pgs: 97 active+clean; 783 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.4 MiB/s wr, 718 op/s 2023-09-30T14:58:43.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:43 smithi032 ceph-mon[98592]: pgmap v900: 97 pgs: 97 active+clean; 783 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.4 MiB/s wr, 718 op/s 2023-09-30T14:58:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:43 smithi156 ceph-mon[103579]: pgmap v900: 97 pgs: 97 active+clean; 783 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.4 MiB/s wr, 718 op/s 2023-09-30T14:58:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.977+0000 7f2302a38700 1 -- 172.21.15.32:0/744557243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22fc10e270 msgr2=0x7f22fc110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.977+0000 7f2302a38700 1 --2- 172.21.15.32:0/744557243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22fc10e270 0x7f22fc110710 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009f90 tx=0x7f22f400b330 comp rx=0 tx=0).stop 2023-09-30T14:58:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.978+0000 7f2302a38700 1 -- 172.21.15.32:0/744557243 shutdown_connections 2023-09-30T14:58:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.978+0000 7f2302a38700 1 --2- 172.21.15.32:0/744557243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22fc10e270 0x7f22fc110710 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.978+0000 7f2302a38700 1 --2- 172.21.15.32:0/744557243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22fc108f30 0x7f22fc10b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.978+0000 7f2302a38700 1 --2- 172.21.15.32:0/744557243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22fc10b860 0x7f22fc10dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.978+0000 7f2302a38700 1 -- 172.21.15.32:0/744557243 >> 172.21.15.32:0/744557243 conn(0x7f22fc06dc80 msgr2=0x7f22fc0700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.978+0000 7f2302a38700 1 -- 172.21.15.32:0/744557243 shutdown_connections 2023-09-30T14:58:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.978+0000 7f2302a38700 1 -- 172.21.15.32:0/744557243 wait complete. 2023-09-30T14:58:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.979+0000 7f2302a38700 1 Processor -- start 2023-09-30T14:58:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.979+0000 7f2302a38700 1 -- start start 2023-09-30T14:58:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.979+0000 7f2302a38700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22fc108f30 0x7f22fc111300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.979+0000 7f2302a38700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22fc10b860 0x7f22fc111840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.979+0000 7f2302a38700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22fc10e270 0x7f22fc111d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.979+0000 7f2302a38700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc1a94b0 con 0x7f22fc10b860 2023-09-30T14:58:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.979+0000 7f2302a38700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc1a95f0 con 0x7f22fc10e270 2023-09-30T14:58:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.979+0000 7f2302a38700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc1a9730 con 0x7f22fc108f30 2023-09-30T14:58:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.979+0000 7f22fb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22fc10b860 0x7f22fc111840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.979+0000 7f22fbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22fc108f30 0x7f22fc111300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.979+0000 7f22fb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22fc10b860 0x7f22fc111840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39654/0 (socket says 172.21.15.32:39654) 2023-09-30T14:58:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.979+0000 7f22fb7fe700 1 -- 172.21.15.32:0/189034954 learned_addr learned my addr 172.21.15.32:0/189034954 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.980+0000 7f22fb7fe700 1 -- 172.21.15.32:0/189034954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22fc108f30 msgr2=0x7f22fc111300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.980+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/189034954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22fc108f30 0x7f22fc111300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.980+0000 7f22fb7fe700 1 -- 172.21.15.32:0/189034954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22fc10e270 msgr2=0x7f22fc111d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.980+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/189034954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22fc10e270 0x7f22fc111d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.980+0000 7f22fb7fe700 1 -- 172.21.15.32:0/189034954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f4009c40 con 0x7f22fc10b860 2023-09-30T14:58:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.980+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/189034954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22fc10b860 0x7f22fc111840 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00e9c0 tx=0x7f22ec00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.981+0000 7f22f97fa700 1 -- 172.21.15.32:0/189034954 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec00cbd0 con 0x7f22fc10b860 2023-09-30T14:58:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.981+0000 7f22f97fa700 1 -- 172.21.15.32:0/189034954 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22ec00eeb0 con 0x7f22fc10b860 2023-09-30T14:58:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.981+0000 7f22f97fa700 1 -- 172.21.15.32:0/189034954 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec005550 con 0x7f22fc10b860 2023-09-30T14:58:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.981+0000 7f2302a38700 1 -- 172.21.15.32:0/189034954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22fc1a99c0 con 0x7f22fc10b860 2023-09-30T14:58:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.981+0000 7f2302a38700 1 -- 172.21.15.32:0/189034954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22fc1a9ed0 con 0x7f22fc10b860 2023-09-30T14:58:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.984+0000 7f22f97fa700 1 -- 172.21.15.32:0/189034954 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f22ec021440 con 0x7f22fc10b860 2023-09-30T14:58:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.985+0000 7f22f97fa700 1 --2- 172.21.15.32:0/189034954 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f22e40a1020 0x7f22e40a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.985+0000 7f22f97fa700 1 -- 172.21.15.32:0/189034954 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f22ec014070 con 0x7f22fc10b860 2023-09-30T14:58:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.985+0000 7f22fbfff700 1 --2- 172.21.15.32:0/189034954 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f22e40a1020 0x7f22e40a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.985+0000 7f22fbfff700 1 --2- 172.21.15.32:0/189034954 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f22e40a1020 0x7f22e40a34e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f22fc112c10 tx=0x7f22f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.986+0000 7f230ec20b80 1 Processor -- start 2023-09-30T14:58:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.986+0000 7f230ec20b80 1 -- start start 2023-09-30T14:58:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.986+0000 7f230ec20b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d37a10 0x5619e9d37e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.986+0000 7f230ec20b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d38b20 0x5619e9d38fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.986+0000 7f230ec20b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d394e0 0x5619e9d3fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.986+0000 7f230ec20b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e9c97400 con 0x5619e9d38b20 2023-09-30T14:58:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.986+0000 7f230ec20b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e9c88bc0 con 0x5619e9d37a10 2023-09-30T14:58:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.986+0000 7f230ec20b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e9c864e0 con 0x5619e9d394e0 2023-09-30T14:58:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.987+0000 7f22fb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d38b20 0x5619e9d38fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.987+0000 7f22fb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d38b20 0x5619e9d38fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39662/0 (socket says 172.21.15.32:39662) 2023-09-30T14:58:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.987+0000 7f22fb7fe700 1 -- 172.21.15.32:0/3277145277 learned_addr learned my addr 172.21.15.32:0/3277145277 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.987+0000 7f22fb7fe700 1 -- 172.21.15.32:0/3277145277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d394e0 msgr2=0x5619e9d3fd70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-30T14:58:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.987+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/3277145277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d394e0 0x5619e9d3fd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.987+0000 7f22fb7fe700 1 -- 172.21.15.32:0/3277145277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d37a10 msgr2=0x5619e9d37e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.987+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/3277145277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d37a10 0x5619e9d37e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.987+0000 7f22fb7fe700 1 -- 172.21.15.32:0/3277145277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e9ce4870 con 0x5619e9d38b20 2023-09-30T14:58:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.990+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/3277145277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d38b20 0x5619e9d38fa0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f22ec096670 tx=0x7f22ec09a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfb5ec32227dece server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.994+0000 7f22e27fc700 1 -- 172.21.15.32:0/3277145277 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec097d40 con 0x5619e9d38b20 2023-09-30T14:58:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.994+0000 7f22e27fc700 1 -- 172.21.15.32:0/3277145277 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22ec097fa0 con 0x5619e9d38b20 2023-09-30T14:58:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.994+0000 7f22e27fc700 1 -- 172.21.15.32:0/3277145277 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec016590 con 0x5619e9d38b20 2023-09-30T14:58:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.996+0000 7f230ec20b80 1 -- 172.21.15.32:0/3277145277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d38b20 msgr2=0x5619e9d38fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.996+0000 7f230ec20b80 1 --2- 172.21.15.32:0/3277145277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d38b20 0x5619e9d38fa0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f22ec096670 tx=0x7f22ec09a3d0 comp rx=0 tx=0).stop 2023-09-30T14:58:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.996+0000 7f230ec20b80 1 -- 172.21.15.32:0/3277145277 shutdown_connections 2023-09-30T14:58:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.996+0000 7f230ec20b80 1 --2- 172.21.15.32:0/3277145277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d394e0 0x5619e9d3fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.996+0000 7f230ec20b80 1 --2- 172.21.15.32:0/3277145277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d37a10 0x5619e9d37e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.996+0000 7f230ec20b80 1 --2- 172.21.15.32:0/3277145277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d38b20 0x5619e9d38fa0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.996+0000 7f230ec20b80 1 -- 172.21.15.32:0/3277145277 >> 172.21.15.32:0/3277145277 conn(0x5619e9ca3200 msgr2=0x5619e9d29ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.996+0000 7f230ec20b80 1 -- 172.21.15.32:0/3277145277 shutdown_connections 2023-09-30T14:58:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.996+0000 7f230ec20b80 1 -- 172.21.15.32:0/3277145277 wait complete. 2023-09-30T14:58:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.997+0000 7f230ec20b80 1 -- start start 2023-09-30T14:58:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.997+0000 7f230ec20b80 1 Processor -- start 2023-09-30T14:58:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f230ec20b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d37a10 0x5619e9d41770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f230ec20b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d38b20 0x5619e9d41cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f230ec20b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d394e0 0x5619e9d48cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f230ec20b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e9c81dd0 con 0x5619e9d37a10 2023-09-30T14:58:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f230ec20b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e9c1e6b0 con 0x5619e9d38b20 2023-09-30T14:58:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f230ec20b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e9c1e350 con 0x5619e9d394e0 2023-09-30T14:58:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f230ec20b80 10 client.0 _ll_register_callbacks cb 0x5619e9a7b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:58:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f22fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d37a10 0x5619e9d41770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f22fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d37a10 0x5619e9d41770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39666/0 (socket says 172.21.15.32:39666) 2023-09-30T14:58:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f22fbfff700 1 -- 172.21.15.32:0/3475228039 learned_addr learned my addr 172.21.15.32:0/3475228039 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f2300fd5700 1 --2- 172.21.15.32:0/3475228039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d394e0 0x5619e9d48cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f22fbfff700 1 -- 172.21.15.32:0/3475228039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d394e0 msgr2=0x5619e9d48cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f22fbfff700 1 --2- 172.21.15.32:0/3475228039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d394e0 0x5619e9d48cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f22fbfff700 1 -- 172.21.15.32:0/3475228039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d38b20 msgr2=0x5619e9d41cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f22fbfff700 1 --2- 172.21.15.32:0/3475228039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d38b20 0x5619e9d41cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f22fbfff700 1 -- 172.21.15.32:0/3475228039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22fc1a9ed0 con 0x5619e9d37a10 2023-09-30T14:58:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f22fbfff700 1 --2- 172.21.15.32:0/3475228039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d37a10 0x5619e9d41770 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f22f000e700 tx=0x7f22f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296a928881cae2bd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.998+0000 7f22db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:58:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.999+0000 7f22db7fe700 1 -- 172.21.15.32:0/3475228039 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0004a30 con 0x5619e9d37a10 2023-09-30T14:58:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.999+0000 7f22db7fe700 1 -- 172.21.15.32:0/3475228039 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f22f0004b90 con 0x5619e9d37a10 2023-09-30T14:58:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.999+0000 7f230ec20b80 1 -- 172.21.15.32:0/3475228039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619e9d22090 con 0x5619e9d37a10 2023-09-30T14:58:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.999+0000 7f22db7fe700 1 -- 172.21.15.32:0/3475228039 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22f0004d50 con 0x5619e9d37a10 2023-09-30T14:58:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.999+0000 7f230ec20b80 10 client.17148 fetch_fsmap learned FSMap version 52 2023-09-30T14:58:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.999+0000 7f230ec20b80 1 -- 172.21.15.32:0/3475228039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5619e9aacd80 con 0x5619e9d37a10 2023-09-30T14:58:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:43.999+0000 7f22db7fe700 1 -- 172.21.15.32:0/3475228039 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5619e9d22090 con 0x5619e9d37a10 2023-09-30T14:58:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.000+0000 7f22db7fe700 1 -- 172.21.15.32:0/3475228039 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 52) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f22f002f070 con 0x5619e9d37a10 2023-09-30T14:58:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.000+0000 7f230ec20b80 10 client.17148 fetch_fsmap finished waiting for FSMap version 52 2023-09-30T14:58:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.000+0000 7f230ec20b80 10 client.17148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:58:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.000+0000 7f230ec20b80 20 client.17148 populate_metadata read hostname 'smithi032' 2023-09-30T14:58:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.000+0000 7f230ec20b80 1 --2- 172.21.15.32:0/3475228039 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5619e9d4d600 0x5619e9d4fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.000+0000 7f230ec20b80 4 client.17148 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:58:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.000+0000 7f230ec20b80 1 -- 172.21.15.32:0/3475228039 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619e9c7adb0 con 0x5619e9d4d600 2023-09-30T14:58:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.001+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/3475228039 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5619e9d4d600 0x5619e9d4fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.002+0000 7f22db7fe700 10 client.17148 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.002+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/3475228039 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5619e9d4d600 0x5619e9d4fa20 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24d81755ba68284e server_cookie=eaf96bc68e437e0a in_seq=0 out_seq=0 2023-09-30T14:58:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.006+0000 7f22db7fe700 1 -- 172.21.15.32:0/3475228039 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5619e9c7adb0 con 0x5619e9d4d600 2023-09-30T14:58:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.006+0000 7f22db7fe700 10 client.17148 handle_command_reply: tid=0 2023-09-30T14:58:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.006+0000 7f230ec20b80 1 client.17148 shutdown 2023-09-30T14:58:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.007+0000 7f230ec20b80 1 -- 172.21.15.32:0/3475228039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d37a10 msgr2=0x5619e9d41770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.007+0000 7f230ec20b80 1 --2- 172.21.15.32:0/3475228039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d37a10 0x5619e9d41770 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f22f000e700 tx=0x7f22f0004750 comp rx=0 tx=0).stop 2023-09-30T14:58:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.010+0000 7f230ec20b80 1 -- 172.21.15.32:0/3475228039 shutdown_connections 2023-09-30T14:58:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.010+0000 7f230ec20b80 1 --2- 172.21.15.32:0/3475228039 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5619e9d4d600 0x5619e9d4fa20 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.010+0000 7f230ec20b80 1 --2- 172.21.15.32:0/3475228039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d394e0 0x5619e9d48cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.010+0000 7f230ec20b80 1 --2- 172.21.15.32:0/3475228039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d37a10 0x5619e9d41770 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.010+0000 7f230ec20b80 1 --2- 172.21.15.32:0/3475228039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d38b20 0x5619e9d41cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.010+0000 7f230ec20b80 1 -- 172.21.15.32:0/3475228039 >> 172.21.15.32:0/3475228039 conn(0x5619e9ca3200 msgr2=0x5619e9d29590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.012+0000 7f22db7fe700 0 client.17148 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.012+0000 7f230ec20b80 1 -- 172.21.15.32:0/3475228039 shutdown_connections 2023-09-30T14:58:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.012+0000 7f230ec20b80 1 -- 172.21.15.32:0/3475228039 wait complete. 2023-09-30T14:58:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.012+0000 7f230ec20b80 20 client.17148 trim_cache size 0 max 16384 2023-09-30T14:58:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f230ec20b80 1 Processor -- start 2023-09-30T14:58:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f230ec20b80 1 -- start start 2023-09-30T14:58:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f230ec20b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d37a10 0x5619e9d40420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f230ec20b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d38b20 0x5619e9d48c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f230ec20b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d394e0 0x5619e9d29590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f230ec20b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e9b028c0 con 0x5619e9d394e0 2023-09-30T14:58:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f230ec20b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e9d412e0 con 0x5619e9d38b20 2023-09-30T14:58:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f230ec20b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e9d26ba0 con 0x5619e9d37a10 2023-09-30T14:58:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f2300fd5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d394e0 0x5619e9d29590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f2300fd5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d394e0 0x5619e9d29590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39676/0 (socket says 172.21.15.32:39676) 2023-09-30T14:58:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f2300fd5700 1 -- 172.21.15.32:0/1714013975 learned_addr learned my addr 172.21.15.32:0/1714013975 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f22fbfff700 1 --2- 172.21.15.32:0/1714013975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d37a10 0x5619e9d40420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/1714013975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d38b20 0x5619e9d48c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f2300fd5700 1 -- 172.21.15.32:0/1714013975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d37a10 msgr2=0x5619e9d40420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f2300fd5700 1 --2- 172.21.15.32:0/1714013975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d37a10 0x5619e9d40420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f2300fd5700 1 -- 172.21.15.32:0/1714013975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d38b20 msgr2=0x5619e9d48c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f2300fd5700 1 --2- 172.21.15.32:0/1714013975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d38b20 0x5619e9d48c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.032+0000 7f2300fd5700 1 -- 172.21.15.32:0/1714013975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e9a95c00 con 0x5619e9d394e0 2023-09-30T14:58:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.033+0000 7f2300fd5700 1 --2- 172.21.15.32:0/1714013975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d394e0 0x5619e9d29590 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f22f4006060 tx=0x7f22f40092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af66274979a4b8da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.033+0000 7f22e2ffd700 1 -- 172.21.15.32:0/1714013975 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f400e040 con 0x5619e9d394e0 2023-09-30T14:58:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.033+0000 7f22e2ffd700 1 -- 172.21.15.32:0/1714013975 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22f4007620 con 0x5619e9d394e0 2023-09-30T14:58:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.033+0000 7f22e2ffd700 1 -- 172.21.15.32:0/1714013975 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f400bc10 con 0x5619e9d394e0 2023-09-30T14:58:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.034+0000 7f230ec20b80 1 -- 172.21.15.32:0/1714013975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d394e0 msgr2=0x5619e9d29590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.034+0000 7f230ec20b80 1 --2- 172.21.15.32:0/1714013975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d394e0 0x5619e9d29590 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f22f4006060 tx=0x7f22f40092e0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.034+0000 7f230ec20b80 1 -- 172.21.15.32:0/1714013975 shutdown_connections 2023-09-30T14:58:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.034+0000 7f230ec20b80 1 --2- 172.21.15.32:0/1714013975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d394e0 0x5619e9d29590 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.034+0000 7f230ec20b80 1 --2- 172.21.15.32:0/1714013975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d37a10 0x5619e9d40420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.034+0000 7f230ec20b80 1 --2- 172.21.15.32:0/1714013975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d38b20 0x5619e9d48c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.034+0000 7f230ec20b80 1 -- 172.21.15.32:0/1714013975 >> 172.21.15.32:0/1714013975 conn(0x5619e9ca3200 msgr2=0x5619e9d29ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.034+0000 7f230ec20b80 1 -- 172.21.15.32:0/1714013975 shutdown_connections 2023-09-30T14:58:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.034+0000 7f230ec20b80 1 -- 172.21.15.32:0/1714013975 wait complete. 2023-09-30T14:58:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.034+0000 7f230ec20b80 1 -- start start 2023-09-30T14:58:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f230ec20b80 1 Processor -- start 2023-09-30T14:58:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f230ec20b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d37a10 0x5619e9d46510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f230ec20b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d38b20 0x5619e9d46a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f230ec20b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d394e0 0x5619e9d47a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f230ec20b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e9d47fa0 con 0x5619e9d37a10 2023-09-30T14:58:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f230ec20b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e9d48110 con 0x5619e9d38b20 2023-09-30T14:58:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f230ec20b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e9d482d0 con 0x5619e9d394e0 2023-09-30T14:58:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f230ec20b80 10 client.0 _ll_register_callbacks cb 0x7f22f0011ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:58:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f22fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d37a10 0x5619e9d46510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f22fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d37a10 0x5619e9d46510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39680/0 (socket says 172.21.15.32:39680) 2023-09-30T14:58:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f22fbfff700 1 -- 172.21.15.32:0/677432275 learned_addr learned my addr 172.21.15.32:0/677432275 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f2300fd5700 1 --2- 172.21.15.32:0/677432275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d394e0 0x5619e9d47a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/677432275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d38b20 0x5619e9d46a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f22fbfff700 1 -- 172.21.15.32:0/677432275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d394e0 msgr2=0x5619e9d47a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f22fbfff700 1 --2- 172.21.15.32:0/677432275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d394e0 0x5619e9d47a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f22fbfff700 1 -- 172.21.15.32:0/677432275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d38b20 msgr2=0x5619e9d46a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f22fbfff700 1 --2- 172.21.15.32:0/677432275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d38b20 0x5619e9d46a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.035+0000 7f22fbfff700 1 -- 172.21.15.32:0/677432275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22fc107fe0 con 0x5619e9d37a10 2023-09-30T14:58:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.036+0000 7f22fbfff700 1 --2- 172.21.15.32:0/677432275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d37a10 0x5619e9d46510 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f22f0020000 tx=0x7f22f0005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577bc16b1dc10a43 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.036+0000 7f22db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:58:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.036+0000 7f22db7fe700 1 -- 172.21.15.32:0/677432275 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0036070 con 0x5619e9d37a10 2023-09-30T14:58:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.036+0000 7f230ec20b80 1 -- 172.21.15.32:0/677432275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22ec097fa0 con 0x5619e9d37a10 2023-09-30T14:58:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.036+0000 7f22db7fe700 1 -- 172.21.15.32:0/677432275 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f22f0004b90 con 0x5619e9d37a10 2023-09-30T14:58:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.036+0000 7f22db7fe700 1 -- 172.21.15.32:0/677432275 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22f0041030 con 0x5619e9d37a10 2023-09-30T14:58:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.036+0000 7f230ec20b80 10 client.17154 fetch_fsmap learned FSMap version 52 2023-09-30T14:58:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.036+0000 7f230ec20b80 1 -- 172.21.15.32:0/677432275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5619e9ad31f0 con 0x5619e9d37a10 2023-09-30T14:58:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.036+0000 7f22db7fe700 1 -- 172.21.15.32:0/677432275 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22ec097fa0 con 0x5619e9d37a10 2023-09-30T14:58:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.036+0000 7f22db7fe700 1 -- 172.21.15.32:0/677432275 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 52) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f22f000e730 con 0x5619e9d37a10 2023-09-30T14:58:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.036+0000 7f230ec20b80 10 client.17154 fetch_fsmap finished waiting for FSMap version 52 2023-09-30T14:58:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.036+0000 7f230ec20b80 10 client.17154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:58:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.036+0000 7f230ec20b80 20 client.17154 populate_metadata read hostname 'smithi032' 2023-09-30T14:58:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.036+0000 7f230ec20b80 1 --2- 172.21.15.32:0/677432275 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5619e9da1de0 0x5619e9da4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.036+0000 7f230ec20b80 4 client.17154 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:58:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.036+0000 7f230ec20b80 1 -- 172.21.15.32:0/677432275 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5619e9c7aad0 con 0x5619e9da1de0 2023-09-30T14:58:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.037+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/677432275 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5619e9da1de0 0x5619e9da4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.037+0000 7f22db7fe700 10 client.17154 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.037+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/677432275 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5619e9da1de0 0x5619e9da4200 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c39715aed9ca37f server_cookie=c239aea5a90e840e in_seq=0 out_seq=0 2023-09-30T14:58:44.049 INFO:tasks.workunit.client.0.smithi032.stdout: 10 290216 36.02 MB/sec execute 317 sec latency 99.181 ms 2023-09-30T14:58:44.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.661+0000 7f22db7fe700 1 -- 172.21.15.32:0/677432275 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5619e9c7aad0 con 0x5619e9da1de0 2023-09-30T14:58:44.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.661+0000 7f22db7fe700 10 client.17154 handle_command_reply: tid=0 2023-09-30T14:58:44.661 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.661+0000 7f230ec20b80 1 client.17154 shutdown 2023-09-30T14:58:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.663+0000 7f230ec20b80 1 -- 172.21.15.32:0/677432275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d37a10 msgr2=0x5619e9d46510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.663+0000 7f230ec20b80 1 --2- 172.21.15.32:0/677432275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d37a10 0x5619e9d46510 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f22f0020000 tx=0x7f22f0005cd0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.663+0000 7f230ec20b80 1 -- 172.21.15.32:0/677432275 shutdown_connections 2023-09-30T14:58:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.663+0000 7f230ec20b80 1 --2- 172.21.15.32:0/677432275 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5619e9da1de0 0x5619e9da4200 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.663+0000 7f230ec20b80 1 --2- 172.21.15.32:0/677432275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5619e9d394e0 0x5619e9d47a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.663+0000 7f230ec20b80 1 --2- 172.21.15.32:0/677432275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5619e9d37a10 0x5619e9d46510 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.663+0000 7f230ec20b80 1 --2- 172.21.15.32:0/677432275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e9d38b20 0x5619e9d46a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.663+0000 7f230ec20b80 1 -- 172.21.15.32:0/677432275 >> 172.21.15.32:0/677432275 conn(0x5619e9ca3200 msgr2=0x5619e9d4f610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.663+0000 7f22db7fe700 0 client.17154 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.663+0000 7f230ec20b80 1 -- 172.21.15.32:0/677432275 shutdown_connections 2023-09-30T14:58:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.663+0000 7f230ec20b80 1 -- 172.21.15.32:0/677432275 wait complete. 2023-09-30T14:58:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.663+0000 7f230ec20b80 20 client.17154 trim_cache size 0 max 16384 2023-09-30T14:58:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:58:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:58:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:58:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:58:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.664+0000 7f2302a38700 1 -- 172.21.15.32:0/189034954 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f22e40a1020 msgr2=0x7f22e40a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.664+0000 7f2302a38700 1 --2- 172.21.15.32:0/189034954 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f22e40a1020 0x7f22e40a34e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f22fc112c10 tx=0x7f22f0009260 comp rx=0 tx=0).stop 2023-09-30T14:58:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.664+0000 7f2302a38700 1 -- 172.21.15.32:0/189034954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22fc10b860 msgr2=0x7f22fc111840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.664+0000 7f2302a38700 1 --2- 172.21.15.32:0/189034954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22fc10b860 0x7f22fc111840 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00e9c0 tx=0x7f22ec00ecd0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.664+0000 7f2302a38700 1 -- 172.21.15.32:0/189034954 shutdown_connections 2023-09-30T14:58:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.664+0000 7f2302a38700 1 --2- 172.21.15.32:0/189034954 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f22e40a1020 0x7f22e40a34e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.664+0000 7f2302a38700 1 --2- 172.21.15.32:0/189034954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22fc10e270 0x7f22fc111d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.664+0000 7f2302a38700 1 --2- 172.21.15.32:0/189034954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22fc108f30 0x7f22fc111300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.664+0000 7f2302a38700 1 --2- 172.21.15.32:0/189034954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22fc10b860 0x7f22fc111840 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.664+0000 7f2302a38700 1 -- 172.21.15.32:0/189034954 >> 172.21.15.32:0/189034954 conn(0x7f22fc06dc80 msgr2=0x7f22fc10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.664+0000 7f2302a38700 1 -- 172.21.15.32:0/189034954 shutdown_connections 2023-09-30T14:58:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:44.664+0000 7f2302a38700 1 -- 172.21.15.32:0/189034954 wait complete. 2023-09-30T14:58:45.049 INFO:tasks.workunit.client.0.smithi032.stdout: 10 290512 35.97 MB/sec execute 318 sec latency 406.178 ms 2023-09-30T14:58:45.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:45 smithi032 ceph-mon[98592]: pgmap v901: 97 pgs: 97 active+clean; 784 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.1 MiB/s wr, 713 op/s 2023-09-30T14:58:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:45 smithi156 ceph-mon[103579]: pgmap v901: 97 pgs: 97 active+clean; 784 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.1 MiB/s wr, 713 op/s 2023-09-30T14:58:45.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:45 smithi148 ceph-mon[102355]: pgmap v901: 97 pgs: 97 active+clean; 784 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.1 MiB/s wr, 713 op/s 2023-09-30T14:58:46.049 INFO:tasks.workunit.client.0.smithi032.stdout: 10 291156 35.96 MB/sec execute 319 sec latency 80.220 ms 2023-09-30T14:58:46.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:46 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:58:46.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:46 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:58:46.862 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:46 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:58:47.049 INFO:tasks.workunit.client.0.smithi032.stdout: 10 291861 35.95 MB/sec execute 320 sec latency 54.741 ms 2023-09-30T14:58:47.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:47 smithi032 ceph-mon[98592]: pgmap v902: 97 pgs: 97 active+clean; 785 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 708 op/s 2023-09-30T14:58:47.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:47 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:58:47.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:47 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:58:47.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:47 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:58:47.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:47 smithi156 ceph-mon[103579]: pgmap v902: 97 pgs: 97 active+clean; 785 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 708 op/s 2023-09-30T14:58:47.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:47 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:58:47.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:47 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:58:47.808 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:47 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:58:47.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:47 smithi148 ceph-mon[102355]: pgmap v902: 97 pgs: 97 active+clean; 785 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 708 op/s 2023-09-30T14:58:47.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:47 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:58:47.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:47 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:58:47.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:47 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:58:48.049 INFO:tasks.workunit.client.0.smithi032.stdout: 10 292682 35.99 MB/sec execute 321 sec latency 47.048 ms 2023-09-30T14:58:49.049 INFO:tasks.workunit.client.0.smithi032.stdout: 10 293540 36.02 MB/sec execute 322 sec latency 51.736 ms 2023-09-30T14:58:49.375 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:49 smithi032 ceph-mon[98592]: pgmap v903: 97 pgs: 97 active+clean; 785 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.0 MiB/s wr, 701 op/s 2023-09-30T14:58:49.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:49 smithi156 ceph-mon[103579]: pgmap v903: 97 pgs: 97 active+clean; 785 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.0 MiB/s wr, 701 op/s 2023-09-30T14:58:49.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:49 smithi148 ceph-mon[102355]: pgmap v903: 97 pgs: 97 active+clean; 785 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.0 MiB/s wr, 701 op/s 2023-09-30T14:58:50.050 INFO:tasks.workunit.client.0.smithi032.stdout: 10 293928 35.99 MB/sec execute 323 sec latency 222.983 ms 2023-09-30T14:58:51.055 INFO:tasks.workunit.client.0.smithi032.stdout: 10 294529 35.97 MB/sec execute 324 sec latency 89.671 ms 2023-09-30T14:58:51.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:51 smithi032 ceph-mon[98592]: pgmap v904: 97 pgs: 97 active+clean; 790 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.7 MiB/s wr, 835 op/s 2023-09-30T14:58:51.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:51 smithi156 ceph-mon[103579]: pgmap v904: 97 pgs: 97 active+clean; 790 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.7 MiB/s wr, 835 op/s 2023-09-30T14:58:51.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:51 smithi148 ceph-mon[102355]: pgmap v904: 97 pgs: 97 active+clean; 790 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.7 MiB/s wr, 835 op/s 2023-09-30T14:58:52.052 INFO:tasks.workunit.client.0.smithi032.stdout: 10 295221 35.95 MB/sec execute 325 sec latency 73.112 ms 2023-09-30T14:58:53.052 INFO:tasks.workunit.client.0.smithi032.stdout: 10 296057 35.97 MB/sec execute 326 sec latency 76.835 ms 2023-09-30T14:58:53.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:53 smithi032 ceph-mon[98592]: pgmap v905: 97 pgs: 97 active+clean; 791 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.5 MiB/s wr, 717 op/s 2023-09-30T14:58:53.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:53 smithi156 ceph-mon[103579]: pgmap v905: 97 pgs: 97 active+clean; 791 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.5 MiB/s wr, 717 op/s 2023-09-30T14:58:53.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:53 smithi148 ceph-mon[102355]: pgmap v905: 97 pgs: 97 active+clean; 791 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.5 MiB/s wr, 717 op/s 2023-09-30T14:58:54.052 INFO:tasks.workunit.client.0.smithi032.stdout: 10 296875 36.05 MB/sec execute 327 sec latency 70.819 ms 2023-09-30T14:58:55.052 INFO:tasks.workunit.client.0.smithi032.stdout: 10 297289 35.99 MB/sec execute 328 sec latency 267.606 ms 2023-09-30T14:58:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:55 smithi156 ceph-mon[103579]: pgmap v906: 97 pgs: 97 active+clean; 793 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.9 MiB/s wr, 709 op/s 2023-09-30T14:58:55.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:55 smithi148 ceph-mon[102355]: pgmap v906: 97 pgs: 97 active+clean; 793 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.9 MiB/s wr, 709 op/s 2023-09-30T14:58:55.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:55 smithi032 ceph-mon[98592]: pgmap v906: 97 pgs: 97 active+clean; 793 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.9 MiB/s wr, 709 op/s 2023-09-30T14:58:56.052 INFO:tasks.workunit.client.0.smithi032.stdout: 10 297795 35.95 MB/sec execute 329 sec latency 55.305 ms 2023-09-30T14:58:57.052 INFO:tasks.workunit.client.0.smithi032.stdout: 10 298646 35.98 MB/sec execute 330 sec latency 44.967 ms 2023-09-30T14:58:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.792+0000 7f9caf741700 1 -- 172.21.15.32:0/2791062611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca81000b0 msgr2=0x7f9ca8100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.792+0000 7f9caf741700 1 --2- 172.21.15.32:0/2791062611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca81000b0 0x7f9ca8100530 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009990 tx=0x7f9c98009ca0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.792+0000 7f9caf741700 1 -- 172.21.15.32:0/2791062611 shutdown_connections 2023-09-30T14:58:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.792+0000 7f9caf741700 1 --2- 172.21.15.32:0/2791062611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca8100a70 0x7f9ca8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.792+0000 7f9caf741700 1 --2- 172.21.15.32:0/2791062611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca80ff750 0x7f9ca80ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.792+0000 7f9caf741700 1 --2- 172.21.15.32:0/2791062611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca81000b0 0x7f9ca8100530 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.792+0000 7f9caf741700 1 -- 172.21.15.32:0/2791062611 >> 172.21.15.32:0/2791062611 conn(0x7f9ca80fb2b0 msgr2=0x7f9ca80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.792+0000 7f9caf741700 1 -- 172.21.15.32:0/2791062611 shutdown_connections 2023-09-30T14:58:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.792+0000 7f9caf741700 1 -- 172.21.15.32:0/2791062611 wait complete. 2023-09-30T14:58:57.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.793+0000 7f9caf741700 1 Processor -- start 2023-09-30T14:58:57.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.793+0000 7f9caf741700 1 -- start start 2023-09-30T14:58:57.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.793+0000 7f9caf741700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca80ff750 0x7f9ca8198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.796 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.793+0000 7f9caf741700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca81000b0 0x7f9ca8199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.793+0000 7f9caf741700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca8100a70 0x7f9ca819a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.793+0000 7f9caf741700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca819a840 con 0x7f9ca80ff750 2023-09-30T14:58:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.793+0000 7f9caf741700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca819a9b0 con 0x7f9ca81000b0 2023-09-30T14:58:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.793+0000 7f9caf741700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca819ab70 con 0x7f9ca8100a70 2023-09-30T14:58:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.793+0000 7f9cad4dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca80ff750 0x7f9ca8198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.793+0000 7f9cad4dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca80ff750 0x7f9ca8198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39692/0 (socket says 172.21.15.32:39692) 2023-09-30T14:58:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.793+0000 7f9cad4dd700 1 -- 172.21.15.32:0/4166021753 learned_addr learned my addr 172.21.15.32:0/4166021753 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.793+0000 7f9caccdc700 1 --2- 172.21.15.32:0/4166021753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca81000b0 0x7f9ca8199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.793+0000 7f9cadcde700 1 --2- 172.21.15.32:0/4166021753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca8100a70 0x7f9ca819a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.794+0000 7f9cad4dd700 1 -- 172.21.15.32:0/4166021753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca8100a70 msgr2=0x7f9ca819a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.794+0000 7f9cad4dd700 1 --2- 172.21.15.32:0/4166021753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca8100a70 0x7f9ca819a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.794+0000 7f9cad4dd700 1 -- 172.21.15.32:0/4166021753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca81000b0 msgr2=0x7f9ca8199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.794+0000 7f9cad4dd700 1 --2- 172.21.15.32:0/4166021753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca81000b0 0x7f9ca8199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.794+0000 7f9cad4dd700 1 -- 172.21.15.32:0/4166021753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c980095f0 con 0x7f9ca80ff750 2023-09-30T14:58:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.794+0000 7f9cad4dd700 1 --2- 172.21.15.32:0/4166021753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca80ff750 0x7f9ca8198ec0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400e9c0 tx=0x7f9ca400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.794+0000 7f9c9e7fc700 1 -- 172.21.15.32:0/4166021753 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca40097b0 con 0x7f9ca80ff750 2023-09-30T14:58:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.794+0000 7f9caf741700 1 -- 172.21.15.32:0/4166021753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca819ae50 con 0x7f9ca80ff750 2023-09-30T14:58:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.794+0000 7f9c9e7fc700 1 -- 172.21.15.32:0/4166021753 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ca400ee60 con 0x7f9ca80ff750 2023-09-30T14:58:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.794+0000 7f9c9e7fc700 1 -- 172.21.15.32:0/4166021753 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca40054f0 con 0x7f9ca80ff750 2023-09-30T14:58:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.794+0000 7f9caf741700 1 -- 172.21.15.32:0/4166021753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca81a1570 con 0x7f9ca80ff750 2023-09-30T14:58:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.796+0000 7f9c9e7fc700 1 -- 172.21.15.32:0/4166021753 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f9ca4004750 con 0x7f9ca80ff750 2023-09-30T14:58:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.797+0000 7f9c9e7fc700 1 --2- 172.21.15.32:0/4166021753 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9c940a0fa0 0x7f9c940a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.797+0000 7f9c9e7fc700 1 -- 172.21.15.32:0/4166021753 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9ca4014070 con 0x7f9ca80ff750 2023-09-30T14:58:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.797+0000 7f9caccdc700 1 --2- 172.21.15.32:0/4166021753 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9c940a0fa0 0x7f9c940a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.798+0000 7f9caccdc700 1 --2- 172.21.15.32:0/4166021753 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9c940a0fa0 0x7f9c940a3460 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800b2e0 tx=0x7f9c98005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.798+0000 7f9cbb929b80 1 Processor -- start 2023-09-30T14:58:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.798+0000 7f9cbb929b80 1 -- start start 2023-09-30T14:58:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.798+0000 7f9cbb929b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c314620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.798+0000 7f9cbb929b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315310 0x56142c315790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.798+0000 7f9cbb929b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315cd0 0x56142c31c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.798+0000 7f9cbb929b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142c27c400 con 0x56142c314200 2023-09-30T14:58:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.798+0000 7f9cbb929b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142c26dbc0 con 0x56142c315cd0 2023-09-30T14:58:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.798+0000 7f9cbb929b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142c26b4e0 con 0x56142c315310 2023-09-30T14:58:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.799+0000 7f9caccdc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315310 0x56142c315790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.799+0000 7f9cad4dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c314620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.799+0000 7f9caccdc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315310 0x56142c315790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54084/0 (socket says 172.21.15.32:54084) 2023-09-30T14:58:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.799+0000 7f9caccdc700 1 -- 172.21.15.32:0/201594817 learned_addr learned my addr 172.21.15.32:0/201594817 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.799+0000 7f9cad4dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c314620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39700/0 (socket says 172.21.15.32:39700) 2023-09-30T14:58:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.799+0000 7f9cadcde700 1 --2- 172.21.15.32:0/201594817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315cd0 0x56142c31c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.799+0000 7f9cad4dd700 1 -- 172.21.15.32:0/201594817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315310 msgr2=0x56142c315790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.799+0000 7f9cad4dd700 1 --2- 172.21.15.32:0/201594817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315310 0x56142c315790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.799+0000 7f9cad4dd700 1 -- 172.21.15.32:0/201594817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315cd0 msgr2=0x56142c31c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.799+0000 7f9cad4dd700 1 --2- 172.21.15.32:0/201594817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315cd0 0x56142c31c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.799+0000 7f9cad4dd700 1 -- 172.21.15.32:0/201594817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56142c2c9870 con 0x56142c314200 2023-09-30T14:58:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.799+0000 7f9caccdc700 1 --2- 172.21.15.32:0/201594817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315310 0x56142c315790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:58:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.799+0000 7f9cad4dd700 1 --2- 172.21.15.32:0/201594817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c314620 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40ca420 tx=0x7f9ca40999f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea71eb621080e6a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.799+0000 7f9c937fe700 1 -- 172.21.15.32:0/201594817 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4099de0 con 0x56142c314200 2023-09-30T14:58:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.799+0000 7f9c937fe700 1 -- 172.21.15.32:0/201594817 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ca40b9050 con 0x56142c314200 2023-09-30T14:58:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.799+0000 7f9c937fe700 1 -- 172.21.15.32:0/201594817 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca409a2a0 con 0x56142c314200 2023-09-30T14:58:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.800+0000 7f9cbb929b80 1 -- 172.21.15.32:0/201594817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 msgr2=0x56142c314620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.800+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/201594817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c314620 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40ca420 tx=0x7f9ca40999f0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.802+0000 7f9cbb929b80 1 -- 172.21.15.32:0/201594817 shutdown_connections 2023-09-30T14:58:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.802+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/201594817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315cd0 0x56142c31c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.802+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/201594817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c314620 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.802+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/201594817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315310 0x56142c315790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.802+0000 7f9cbb929b80 1 -- 172.21.15.32:0/201594817 >> 172.21.15.32:0/201594817 conn(0x56142c288200 msgr2=0x56142c30eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.802+0000 7f9cbb929b80 1 -- 172.21.15.32:0/201594817 shutdown_connections 2023-09-30T14:58:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.802+0000 7f9cbb929b80 1 -- 172.21.15.32:0/201594817 wait complete. 2023-09-30T14:58:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.802+0000 7f9cbb929b80 1 -- start start 2023-09-30T14:58:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.802+0000 7f9cbb929b80 1 Processor -- start 2023-09-30T14:58:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9cbb929b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c314200 0x56142c31dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9cbb929b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c315310 0x56142c31e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9cbb929b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315cd0 0x56142c325540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9cbb929b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142c266dd0 con 0x56142c315310 2023-09-30T14:58:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9cbb929b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142c2036b0 con 0x56142c315cd0 2023-09-30T14:58:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9cbb929b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142c203350 con 0x56142c314200 2023-09-30T14:58:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9cbb929b80 10 client.0 _ll_register_callbacks cb 0x56142c0607a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:58:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9caccdc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c315310 0x56142c31e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9caccdc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c315310 0x56142c31e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39706/0 (socket says 172.21.15.32:39706) 2023-09-30T14:58:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9caccdc700 1 -- 172.21.15.32:0/2305665690 learned_addr learned my addr 172.21.15.32:0/2305665690 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9cad4dd700 1 --2- 172.21.15.32:0/2305665690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c314200 0x56142c31dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9cadcde700 1 --2- 172.21.15.32:0/2305665690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315cd0 0x56142c325540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9caccdc700 1 -- 172.21.15.32:0/2305665690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c314200 msgr2=0x56142c31dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9caccdc700 1 --2- 172.21.15.32:0/2305665690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c314200 0x56142c31dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9caccdc700 1 -- 172.21.15.32:0/2305665690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315cd0 msgr2=0x56142c325540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9caccdc700 1 --2- 172.21.15.32:0/2305665690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315cd0 0x56142c325540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9caccdc700 1 -- 172.21.15.32:0/2305665690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca810bdd0 con 0x56142c315310 2023-09-30T14:58:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.803+0000 7f9cad4dd700 1 --2- 172.21.15.32:0/2305665690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c314200 0x56142c31dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:58:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.804+0000 7f9caccdc700 1 --2- 172.21.15.32:0/2305665690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c315310 0x56142c31e500 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9c98000c00 tx=0x7f9c98009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60aede186e34265a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.804+0000 7f9c90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:58:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.804+0000 7f9c90ff9700 1 -- 172.21.15.32:0/2305665690 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c98004ec0 con 0x56142c315310 2023-09-30T14:58:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.804+0000 7f9c90ff9700 1 -- 172.21.15.32:0/2305665690 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9c9800f460 con 0x56142c315310 2023-09-30T14:58:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.804+0000 7f9cbb929b80 1 -- 172.21.15.32:0/2305665690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56142c307090 con 0x56142c315310 2023-09-30T14:58:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.804+0000 7f9c90ff9700 1 -- 172.21.15.32:0/2305665690 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c9800f620 con 0x56142c315310 2023-09-30T14:58:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.804+0000 7f9cbb929b80 10 client.17166 fetch_fsmap learned FSMap version 53 2023-09-30T14:58:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.804+0000 7f9cbb929b80 1 -- 172.21.15.32:0/2305665690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7f9ca81a1570 con 0x56142c315310 2023-09-30T14:58:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.804+0000 7f9c90ff9700 1 -- 172.21.15.32:0/2305665690 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c98040020 con 0x56142c315310 2023-09-30T14:58:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.805+0000 7f9c90ff9700 1 -- 172.21.15.32:0/2305665690 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 53) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9c9802f070 con 0x56142c315310 2023-09-30T14:58:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.805+0000 7f9cbb929b80 10 client.17166 fetch_fsmap finished waiting for FSMap version 53 2023-09-30T14:58:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.805+0000 7f9cbb929b80 10 client.17166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:58:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.805+0000 7f9cbb929b80 20 client.17166 populate_metadata read hostname 'smithi032' 2023-09-30T14:58:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.805+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/2305665690 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56142c329dc0 0x56142c32c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.805+0000 7f9cbb929b80 4 client.17166 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:58:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.805+0000 7f9cbb929b80 1 -- 172.21.15.32:0/2305665690 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56142c25fdb0 con 0x56142c329dc0 2023-09-30T14:58:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.805+0000 7f9cad4dd700 1 --2- 172.21.15.32:0/2305665690 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56142c329dc0 0x56142c32c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.805+0000 7f9c90ff9700 10 client.17166 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.805+0000 7f9cad4dd700 1 --2- 172.21.15.32:0/2305665690 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56142c329dc0 0x56142c32c1e0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52cae18dc0123892 server_cookie=c2e3143f8903b140 in_seq=0 out_seq=0 2023-09-30T14:58:57.811 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:57 smithi156 ceph-mon[103579]: pgmap v907: 97 pgs: 97 active+clean; 805 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.4 MiB/s wr, 808 op/s 2023-09-30T14:58:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.809+0000 7f9c90ff9700 1 -- 172.21.15.32:0/2305665690 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56142c25fdb0 con 0x56142c329dc0 2023-09-30T14:58:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.809+0000 7f9c90ff9700 10 client.17166 handle_command_reply: tid=0 2023-09-30T14:58:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.809+0000 7f9cbb929b80 1 client.17166 shutdown 2023-09-30T14:58:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.810+0000 7f9cbb929b80 1 -- 172.21.15.32:0/2305665690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c315310 msgr2=0x56142c31e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.810+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/2305665690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c315310 0x56142c31e500 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9c98000c00 tx=0x7f9c98009ca0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.810+0000 7f9cbb929b80 1 -- 172.21.15.32:0/2305665690 shutdown_connections 2023-09-30T14:58:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.810+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/2305665690 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56142c329dc0 0x56142c32c1e0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.810+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/2305665690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315cd0 0x56142c325540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.810+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/2305665690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c314200 0x56142c31dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.810+0000 7f9c90ff9700 0 client.17166 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.810+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/2305665690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c315310 0x56142c31e500 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.810+0000 7f9cbb929b80 1 -- 172.21.15.32:0/2305665690 >> 172.21.15.32:0/2305665690 conn(0x56142c288200 msgr2=0x56142c30e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.810+0000 7f9cbb929b80 1 -- 172.21.15.32:0/2305665690 shutdown_connections 2023-09-30T14:58:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.810+0000 7f9cbb929b80 1 -- 172.21.15.32:0/2305665690 wait complete. 2023-09-30T14:58:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.810+0000 7f9cbb929b80 20 client.17166 trim_cache size 0 max 16384 2023-09-30T14:58:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.828+0000 7f9cbb929b80 1 Processor -- start 2023-09-30T14:58:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cbb929b80 1 -- start start 2023-09-30T14:58:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cbb929b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c30fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cbb929b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315310 0x56142c325500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cbb929b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315cd0 0x56142c31cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cbb929b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142c0e78c0 con 0x56142c314200 2023-09-30T14:58:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cbb929b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142c31db90 con 0x56142c315cd0 2023-09-30T14:58:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cbb929b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142c3095c0 con 0x56142c315310 2023-09-30T14:58:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cad4dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c30fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cad4dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c30fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39714/0 (socket says 172.21.15.32:39714) 2023-09-30T14:58:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cad4dd700 1 -- 172.21.15.32:0/70548424 learned_addr learned my addr 172.21.15.32:0/70548424 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9caccdc700 1 --2- 172.21.15.32:0/70548424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315310 0x56142c325500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cadcde700 1 --2- 172.21.15.32:0/70548424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315cd0 0x56142c31cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cad4dd700 1 -- 172.21.15.32:0/70548424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315310 msgr2=0x56142c325500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cad4dd700 1 --2- 172.21.15.32:0/70548424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315310 0x56142c325500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cad4dd700 1 -- 172.21.15.32:0/70548424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315cd0 msgr2=0x56142c31cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cad4dd700 1 --2- 172.21.15.32:0/70548424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315cd0 0x56142c31cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cad4dd700 1 -- 172.21.15.32:0/70548424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56142c091d80 con 0x56142c314200 2023-09-30T14:58:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.829+0000 7f9cad4dd700 1 --2- 172.21.15.32:0/70548424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c30fdb0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40ca470 tx=0x7f9ca400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce15a8d6a7abf562 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.830+0000 7f9c93fff700 1 -- 172.21.15.32:0/70548424 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca409df30 con 0x56142c314200 2023-09-30T14:58:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.830+0000 7f9c93fff700 1 -- 172.21.15.32:0/70548424 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ca4099de0 con 0x56142c314200 2023-09-30T14:58:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.830+0000 7f9c93fff700 1 -- 172.21.15.32:0/70548424 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca409a2a0 con 0x56142c314200 2023-09-30T14:58:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.830+0000 7f9cbb929b80 1 -- 172.21.15.32:0/70548424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 msgr2=0x56142c30fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.830+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/70548424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c30fdb0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40ca470 tx=0x7f9ca400efd0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.830+0000 7f9cbb929b80 1 -- 172.21.15.32:0/70548424 shutdown_connections 2023-09-30T14:58:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.830+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/70548424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315cd0 0x56142c31cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.830+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/70548424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c30fdb0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.830+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/70548424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315310 0x56142c325500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.830+0000 7f9cbb929b80 1 -- 172.21.15.32:0/70548424 >> 172.21.15.32:0/70548424 conn(0x56142c288200 msgr2=0x56142c30eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.830+0000 7f9cbb929b80 1 -- 172.21.15.32:0/70548424 shutdown_connections 2023-09-30T14:58:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.830+0000 7f9cbb929b80 1 -- 172.21.15.32:0/70548424 wait complete. 2023-09-30T14:58:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.831+0000 7f9cbb929b80 1 -- start start 2023-09-30T14:58:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.831+0000 7f9cbb929b80 1 Processor -- start 2023-09-30T14:58:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.831+0000 7f9cbb929b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c322bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.831+0000 7f9cbb929b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315310 0x56142c323110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.831+0000 7f9cbb929b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315cd0 0x56142c324120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.831+0000 7f9cbb929b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142c324660 con 0x56142c314200 2023-09-30T14:58:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.831+0000 7f9cbb929b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142c3247d0 con 0x56142c315310 2023-09-30T14:58:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.831+0000 7f9cbb929b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142c324990 con 0x56142c315cd0 2023-09-30T14:58:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.831+0000 7f9cbb929b80 10 client.0 _ll_register_callbacks cb 0x7f9c9800bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:58:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.831+0000 7f9cad4dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c322bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.831+0000 7f9cad4dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c322bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39720/0 (socket says 172.21.15.32:39720) 2023-09-30T14:58:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.831+0000 7f9cad4dd700 1 -- 172.21.15.32:0/1635650152 learned_addr learned my addr 172.21.15.32:0/1635650152 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:58:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.831+0000 7f9caccdc700 1 --2- 172.21.15.32:0/1635650152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315310 0x56142c323110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.832+0000 7f9cadcde700 1 --2- 172.21.15.32:0/1635650152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315cd0 0x56142c324120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.832+0000 7f9cad4dd700 1 -- 172.21.15.32:0/1635650152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315cd0 msgr2=0x56142c324120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.832+0000 7f9cad4dd700 1 --2- 172.21.15.32:0/1635650152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315cd0 0x56142c324120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.832+0000 7f9cad4dd700 1 -- 172.21.15.32:0/1635650152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315310 msgr2=0x56142c323110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:58:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.832+0000 7f9cad4dd700 1 --2- 172.21.15.32:0/1635650152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315310 0x56142c323110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.832+0000 7f9cad4dd700 1 -- 172.21.15.32:0/1635650152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca40b6040 con 0x56142c314200 2023-09-30T14:58:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.832+0000 7f9cadcde700 1 --2- 172.21.15.32:0/1635650152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315cd0 0x56142c324120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:58:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.832+0000 7f9cad4dd700 1 --2- 172.21.15.32:0/1635650152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c322bd0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009230 tx=0x7f9ca4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fb9ca865e8ef37 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:58:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.832+0000 7f9c90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:58:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.832+0000 7f9c90ff9700 1 -- 172.21.15.32:0/1635650152 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4099920 con 0x56142c314200 2023-09-30T14:58:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.832+0000 7f9c90ff9700 1 -- 172.21.15.32:0/1635650152 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9ca4098180 con 0x56142c314200 2023-09-30T14:58:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.832+0000 7f9cbb929b80 1 -- 172.21.15.32:0/1635650152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ca40b9050 con 0x56142c314200 2023-09-30T14:58:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.832+0000 7f9c90ff9700 1 -- 172.21.15.32:0/1635650152 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ca409df30 con 0x56142c314200 2023-09-30T14:58:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.832+0000 7f9cbb929b80 10 client.17172 fetch_fsmap learned FSMap version 53 2023-09-30T14:58:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.832+0000 7f9cbb929b80 1 -- 172.21.15.32:0/1635650152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x56142c07ac00 con 0x56142c314200 2023-09-30T14:58:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.832+0000 7f9c90ff9700 1 -- 172.21.15.32:0/1635650152 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ca40b9050 con 0x56142c314200 2023-09-30T14:58:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.833+0000 7f9c90ff9700 1 -- 172.21.15.32:0/1635650152 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 53) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9ca40b2220 con 0x56142c314200 2023-09-30T14:58:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.833+0000 7f9cbb929b80 10 client.17172 fetch_fsmap finished waiting for FSMap version 53 2023-09-30T14:58:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.833+0000 7f9cbb929b80 10 client.17172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:58:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.833+0000 7f9cbb929b80 20 client.17172 populate_metadata read hostname 'smithi032' 2023-09-30T14:58:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.833+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/1635650152 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56142c37dfd0 0x56142c3803f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:58:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.833+0000 7f9cbb929b80 4 client.17172 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:58:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.833+0000 7f9cbb929b80 1 -- 172.21.15.32:0/1635650152 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56142c25fad0 con 0x56142c37dfd0 2023-09-30T14:58:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.833+0000 7f9caccdc700 1 --2- 172.21.15.32:0/1635650152 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56142c37dfd0 0x56142c3803f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:58:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.833+0000 7f9c90ff9700 10 client.17172 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:57.833+0000 7f9caccdc700 1 --2- 172.21.15.32:0/1635650152 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56142c37dfd0 0x56142c3803f0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e45fab4ef854ae3a server_cookie=45be175feeea2fe3 in_seq=0 out_seq=0 2023-09-30T14:58:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:57 smithi148 ceph-mon[102355]: pgmap v907: 97 pgs: 97 active+clean; 805 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.4 MiB/s wr, 808 op/s 2023-09-30T14:58:57.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:57 smithi032 ceph-mon[98592]: pgmap v907: 97 pgs: 97 active+clean; 805 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.4 MiB/s wr, 808 op/s 2023-09-30T14:58:58.054 INFO:tasks.workunit.client.0.smithi032.stdout: 10 299418 35.98 MB/sec execute 331 sec latency 142.098 ms 2023-09-30T14:58:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.256+0000 7f9c90ff9700 1 -- 172.21.15.32:0/1635650152 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56142c25fad0 con 0x56142c37dfd0 2023-09-30T14:58:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.256+0000 7f9c90ff9700 10 client.17172 handle_command_reply: tid=0 2023-09-30T14:58:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.256+0000 7f9cbb929b80 1 client.17172 shutdown 2023-09-30T14:58:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.256+0000 7f9cbb929b80 1 -- 172.21.15.32:0/1635650152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 msgr2=0x56142c322bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.256+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/1635650152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c322bd0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009230 tx=0x7f9ca4005bd0 comp rx=0 tx=0).stop 2023-09-30T14:58:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.256+0000 7f9cbb929b80 1 -- 172.21.15.32:0/1635650152 shutdown_connections 2023-09-30T14:58:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.256+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/1635650152 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56142c37dfd0 0x56142c3803f0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.256+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/1635650152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56142c315cd0 0x56142c324120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.256+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/1635650152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56142c314200 0x56142c322bd0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.256+0000 7f9cbb929b80 1 --2- 172.21.15.32:0/1635650152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56142c315310 0x56142c323110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.256+0000 7f9cbb929b80 1 -- 172.21.15.32:0/1635650152 >> 172.21.15.32:0/1635650152 conn(0x56142c288200 msgr2=0x56142c31c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.256+0000 7f9c90ff9700 0 client.17172 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:58:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.256+0000 7f9cbb929b80 1 -- 172.21.15.32:0/1635650152 shutdown_connections 2023-09-30T14:58:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.256+0000 7f9cbb929b80 1 -- 172.21.15.32:0/1635650152 wait complete. 2023-09-30T14:58:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.257+0000 7f9cbb929b80 20 client.17172 trim_cache size 0 max 16384 2023-09-30T14:58:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:58:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:58:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:58:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:58:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.257+0000 7f9caf741700 1 -- 172.21.15.32:0/4166021753 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9c940a0fa0 msgr2=0x7f9c940a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.257+0000 7f9caf741700 1 --2- 172.21.15.32:0/4166021753 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9c940a0fa0 0x7f9c940a3460 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800b2e0 tx=0x7f9c98005960 comp rx=0 tx=0).stop 2023-09-30T14:58:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.257+0000 7f9caf741700 1 -- 172.21.15.32:0/4166021753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca80ff750 msgr2=0x7f9ca8198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:58:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.257+0000 7f9caf741700 1 --2- 172.21.15.32:0/4166021753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca80ff750 0x7f9ca8198ec0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400e9c0 tx=0x7f9ca400ecd0 comp rx=0 tx=0).stop 2023-09-30T14:58:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.257+0000 7f9caf741700 1 -- 172.21.15.32:0/4166021753 shutdown_connections 2023-09-30T14:58:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.257+0000 7f9caf741700 1 --2- 172.21.15.32:0/4166021753 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f9c940a0fa0 0x7f9c940a3460 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.257+0000 7f9caf741700 1 --2- 172.21.15.32:0/4166021753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ca8100a70 0x7f9ca819a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.257+0000 7f9caf741700 1 --2- 172.21.15.32:0/4166021753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ca80ff750 0x7f9ca8198ec0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.257+0000 7f9caf741700 1 --2- 172.21.15.32:0/4166021753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca81000b0 0x7f9ca8199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:58:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.257+0000 7f9caf741700 1 -- 172.21.15.32:0/4166021753 >> 172.21.15.32:0/4166021753 conn(0x7f9ca80fb2b0 msgr2=0x7f9ca80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:58:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.257+0000 7f9caf741700 1 -- 172.21.15.32:0/4166021753 shutdown_connections 2023-09-30T14:58:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:58:58.257+0000 7f9caf741700 1 -- 172.21.15.32:0/4166021753 wait complete. 2023-09-30T14:58:58.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:58 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:58:58.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:58 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:58:58.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:58 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T14:58:59.054 INFO:tasks.workunit.client.0.smithi032.stdout: 10 300140 36.02 MB/sec execute 332 sec latency 264.737 ms 2023-09-30T14:58:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:58:59 smithi156 ceph-mon[103579]: pgmap v908: 97 pgs: 97 active+clean; 807 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.6 MiB/s wr, 684 op/s 2023-09-30T14:58:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:58:59 smithi148 ceph-mon[102355]: pgmap v908: 97 pgs: 97 active+clean; 807 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.6 MiB/s wr, 684 op/s 2023-09-30T14:58:59.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:58:59 smithi032 ceph-mon[98592]: pgmap v908: 97 pgs: 97 active+clean; 807 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.6 MiB/s wr, 684 op/s 2023-09-30T14:59:00.056 INFO:tasks.workunit.client.0.smithi032.stdout: 10 300546 36.00 MB/sec execute 333 sec latency 259.945 ms 2023-09-30T14:59:01.056 INFO:tasks.workunit.client.0.smithi032.stdout: 10 301047 35.94 MB/sec execute 334 sec latency 61.376 ms 2023-09-30T14:59:01.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:01 smithi156 ceph-mon[103579]: pgmap v909: 97 pgs: 97 active+clean; 807 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.3 MiB/s wr, 727 op/s 2023-09-30T14:59:01.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:01 smithi148 ceph-mon[102355]: pgmap v909: 97 pgs: 97 active+clean; 807 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.3 MiB/s wr, 727 op/s 2023-09-30T14:59:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:01 smithi032 ceph-mon[98592]: pgmap v909: 97 pgs: 97 active+clean; 807 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.3 MiB/s wr, 727 op/s 2023-09-30T14:59:02.056 INFO:tasks.workunit.client.0.smithi032.stdout: 10 301685 35.93 MB/sec execute 335 sec latency 47.949 ms 2023-09-30T14:59:03.056 INFO:tasks.workunit.client.0.smithi032.stdout: 10 302458 35.93 MB/sec execute 336 sec latency 66.235 ms 2023-09-30T14:59:03.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:03 smithi156 ceph-mon[103579]: pgmap v910: 97 pgs: 97 active+clean; 814 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.0 MiB/s wr, 699 op/s 2023-09-30T14:59:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:03 smithi148 ceph-mon[102355]: pgmap v910: 97 pgs: 97 active+clean; 814 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.0 MiB/s wr, 699 op/s 2023-09-30T14:59:03.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:03 smithi032 ceph-mon[98592]: pgmap v910: 97 pgs: 97 active+clean; 814 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 9.0 MiB/s wr, 699 op/s 2023-09-30T14:59:04.056 INFO:tasks.workunit.client.0.smithi032.stdout: 10 303248 35.96 MB/sec execute 337 sec latency 42.348 ms 2023-09-30T14:59:05.056 INFO:tasks.workunit.client.0.smithi032.stdout: 10 303744 35.94 MB/sec execute 338 sec latency 271.170 ms 2023-09-30T14:59:05.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:05 smithi156 ceph-mon[103579]: pgmap v911: 97 pgs: 97 active+clean; 803 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.7 MiB/s wr, 753 op/s 2023-09-30T14:59:05.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:05 smithi148 ceph-mon[102355]: pgmap v911: 97 pgs: 97 active+clean; 803 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.7 MiB/s wr, 753 op/s 2023-09-30T14:59:05.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:05 smithi032 ceph-mon[98592]: pgmap v911: 97 pgs: 97 active+clean; 803 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.7 MiB/s wr, 753 op/s 2023-09-30T14:59:06.057 INFO:tasks.workunit.client.0.smithi032.stdout: 10 304321 35.95 MB/sec execute 339 sec latency 62.324 ms 2023-09-30T14:59:07.057 INFO:tasks.workunit.client.0.smithi032.stdout: 10 304890 35.90 MB/sec execute 340 sec latency 73.589 ms 2023-09-30T14:59:07.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:07 smithi156 ceph-mon[103579]: pgmap v912: 97 pgs: 97 active+clean; 791 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.6 MiB/s wr, 791 op/s 2023-09-30T14:59:07.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:07 smithi148 ceph-mon[102355]: pgmap v912: 97 pgs: 97 active+clean; 791 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.6 MiB/s wr, 791 op/s 2023-09-30T14:59:07.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:07 smithi032 ceph-mon[98592]: pgmap v912: 97 pgs: 97 active+clean; 791 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 9.6 MiB/s wr, 791 op/s 2023-09-30T14:59:08.057 INFO:tasks.workunit.client.0.smithi032.stdout: 10 305568 35.92 MB/sec execute 341 sec latency 68.329 ms 2023-09-30T14:59:09.057 INFO:tasks.workunit.client.0.smithi032.stdout: 10 306394 35.93 MB/sec execute 342 sec latency 61.723 ms 2023-09-30T14:59:09.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:09 smithi156 ceph-mon[103579]: pgmap v913: 97 pgs: 97 active+clean; 796 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.7 MiB/s wr, 717 op/s 2023-09-30T14:59:09.865 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:09 smithi148 ceph-mon[102355]: pgmap v913: 97 pgs: 97 active+clean; 796 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.7 MiB/s wr, 717 op/s 2023-09-30T14:59:09.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:09 smithi032 ceph-mon[98592]: pgmap v913: 97 pgs: 97 active+clean; 796 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.7 MiB/s wr, 717 op/s 2023-09-30T14:59:10.057 INFO:tasks.workunit.client.0.smithi032.stdout: 10 306929 35.91 MB/sec execute 343 sec latency 290.357 ms 2023-09-30T14:59:11.057 INFO:tasks.workunit.client.0.smithi032.stdout: 10 307576 35.92 MB/sec execute 344 sec latency 108.687 ms 2023-09-30T14:59:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.403+0000 7f2f76c9d700 1 -- 172.21.15.32:0/1232976629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f70101900 msgr2=0x7f2f701081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.403+0000 7f2f76c9d700 1 --2- 172.21.15.32:0/1232976629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f70101900 0x7f2f701081b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800b0d0 tx=0x7f2f6800b3e0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.403+0000 7f2f76c9d700 1 -- 172.21.15.32:0/1232976629 shutdown_connections 2023-09-30T14:59:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.403+0000 7f2f76c9d700 1 --2- 172.21.15.32:0/1232976629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f70101900 0x7f2f701081b0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.403+0000 7f2f76c9d700 1 --2- 172.21.15.32:0/1232976629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f700ffd70 0x7f2f70100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.403+0000 7f2f76c9d700 1 --2- 172.21.15.32:0/1232976629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f70100f60 0x7f2f701013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.403+0000 7f2f76c9d700 1 -- 172.21.15.32:0/1232976629 >> 172.21.15.32:0/1232976629 conn(0x7f2f700fb2a0 msgr2=0x7f2f700fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.403+0000 7f2f76c9d700 1 -- 172.21.15.32:0/1232976629 shutdown_connections 2023-09-30T14:59:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.403+0000 7f2f76c9d700 1 -- 172.21.15.32:0/1232976629 wait complete. 2023-09-30T14:59:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.403+0000 7f2f76c9d700 1 Processor -- start 2023-09-30T14:59:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f76c9d700 1 -- start start 2023-09-30T14:59:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f76c9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f700ffd70 0x7f2f70108cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f76c9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f70100f60 0x7f2f70109200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f76c9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f70101900 0x7f2f70109740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f76c9d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f701a0d50 con 0x7f2f70101900 2023-09-30T14:59:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f76c9d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f701a0e90 con 0x7f2f700ffd70 2023-09-30T14:59:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f76c9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f701a0fd0 con 0x7f2f70100f60 2023-09-30T14:59:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f7523a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f70101900 0x7f2f70109740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f6ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f70100f60 0x7f2f70109200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f7523a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f70101900 0x7f2f70109740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39738/0 (socket says 172.21.15.32:39738) 2023-09-30T14:59:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f7523a700 1 -- 172.21.15.32:0/459653962 learned_addr learned my addr 172.21.15.32:0/459653962 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f74a39700 1 --2- 172.21.15.32:0/459653962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f700ffd70 0x7f2f70108cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f7523a700 1 -- 172.21.15.32:0/459653962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f70100f60 msgr2=0x7f2f70109200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f7523a700 1 --2- 172.21.15.32:0/459653962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f70100f60 0x7f2f70109200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f7523a700 1 -- 172.21.15.32:0/459653962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f700ffd70 msgr2=0x7f2f70108cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f7523a700 1 --2- 172.21.15.32:0/459653962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f700ffd70 0x7f2f70108cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f7523a700 1 -- 172.21.15.32:0/459653962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f68009d00 con 0x7f2f70101900 2023-09-30T14:59:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f74a39700 1 --2- 172.21.15.32:0/459653962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f700ffd70 0x7f2f70108cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:59:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.404+0000 7f2f7523a700 1 --2- 172.21.15.32:0/459653962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f70101900 0x7f2f70109740 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2f68008f10 tx=0x7f2f68008f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.405+0000 7f2f6dffb700 1 -- 172.21.15.32:0/459653962 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6800e040 con 0x7f2f70101900 2023-09-30T14:59:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.405+0000 7f2f6dffb700 1 -- 172.21.15.32:0/459653962 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f6800be60 con 0x7f2f70101900 2023-09-30T14:59:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.405+0000 7f2f76c9d700 1 -- 172.21.15.32:0/459653962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f701a1200 con 0x7f2f70101900 2023-09-30T14:59:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.405+0000 7f2f6dffb700 1 -- 172.21.15.32:0/459653962 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6801b6a0 con 0x7f2f70101900 2023-09-30T14:59:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.405+0000 7f2f76c9d700 1 -- 172.21.15.32:0/459653962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f701a1700 con 0x7f2f70101900 2023-09-30T14:59:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.407+0000 7f2f6dffb700 1 -- 172.21.15.32:0/459653962 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f2f68024410 con 0x7f2f70101900 2023-09-30T14:59:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.408+0000 7f2f6dffb700 1 --2- 172.21.15.32:0/459653962 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2f600a1020 0x7f2f600a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.408+0000 7f2f6dffb700 1 -- 172.21.15.32:0/459653962 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2f680cd8d0 con 0x7f2f70101900 2023-09-30T14:59:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.408+0000 7f2f74a39700 1 --2- 172.21.15.32:0/459653962 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2f600a1020 0x7f2f600a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f74a39700 1 --2- 172.21.15.32:0/459653962 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2f600a1020 0x7f2f600a34e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2f7010a5d0 tx=0x7f2f64008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f82e85b80 1 Processor -- start 2023-09-30T14:59:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f82e85b80 1 -- start start 2023-09-30T14:59:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f82e85b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x56310896f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f82e85b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970310 0x563108970790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f82e85b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x563108977560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f82e85b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631088d7400 con 0x563108970cd0 2023-09-30T14:59:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f82e85b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631088c8bc0 con 0x563108970310 2023-09-30T14:59:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f82e85b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631088c64e0 con 0x56310896f200 2023-09-30T14:59:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f7523a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x563108977560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f74a39700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x56310896f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f74a39700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x56310896f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54124/0 (socket says 172.21.15.32:54124) 2023-09-30T14:59:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f74a39700 1 -- 172.21.15.32:0/2578560619 learned_addr learned my addr 172.21.15.32:0/2578560619 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f6ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970310 0x563108970790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f7523a700 1 -- 172.21.15.32:0/2578560619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 msgr2=0x56310896f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f7523a700 1 --2- 172.21.15.32:0/2578560619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x56310896f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f7523a700 1 -- 172.21.15.32:0/2578560619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970310 msgr2=0x563108970790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.409+0000 7f2f7523a700 1 --2- 172.21.15.32:0/2578560619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970310 0x563108970790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.410+0000 7f2f7523a700 1 -- 172.21.15.32:0/2578560619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563108924870 con 0x563108970cd0 2023-09-30T14:59:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.410+0000 7f2f7523a700 1 --2- 172.21.15.32:0/2578560619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x563108977560 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800bfd0 tx=0x7f2f6800b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8842172d353033c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.410+0000 7f2f5affd700 1 -- 172.21.15.32:0/2578560619 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680a10d0 con 0x563108970cd0 2023-09-30T14:59:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.410+0000 7f2f5affd700 1 -- 172.21.15.32:0/2578560619 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f68003e20 con 0x563108970cd0 2023-09-30T14:59:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.410+0000 7f2f5affd700 1 -- 172.21.15.32:0/2578560619 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680991d0 con 0x563108970cd0 2023-09-30T14:59:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.411+0000 7f2f82e85b80 1 -- 172.21.15.32:0/2578560619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 msgr2=0x563108977560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.411+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/2578560619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x563108977560 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800bfd0 tx=0x7f2f6800b3e0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.411+0000 7f2f82e85b80 1 -- 172.21.15.32:0/2578560619 shutdown_connections 2023-09-30T14:59:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.411+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/2578560619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x563108977560 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.411+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/2578560619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x56310896f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.411+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/2578560619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970310 0x563108970790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.411+0000 7f2f82e85b80 1 -- 172.21.15.32:0/2578560619 >> 172.21.15.32:0/2578560619 conn(0x5631088e3200 msgr2=0x563108969ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.411+0000 7f2f82e85b80 1 -- 172.21.15.32:0/2578560619 shutdown_connections 2023-09-30T14:59:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.411+0000 7f2f82e85b80 1 -- 172.21.15.32:0/2578560619 wait complete. 2023-09-30T14:59:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.411+0000 7f2f82e85b80 1 -- start start 2023-09-30T14:59:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.411+0000 7f2f82e85b80 1 Processor -- start 2023-09-30T14:59:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f82e85b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x563108978cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f82e85b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970310 0x5631089791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f82e85b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x563108980230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f82e85b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631088c1dd0 con 0x563108970cd0 2023-09-30T14:59:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f82e85b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310885e6b0 con 0x563108970310 2023-09-30T14:59:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f82e85b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310885e350 con 0x56310896f200 2023-09-30T14:59:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f82e85b80 10 client.0 _ll_register_callbacks cb 0x5631086bb7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:59:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f7523a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x563108980230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f7523a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x563108980230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39752/0 (socket says 172.21.15.32:39752) 2023-09-30T14:59:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f7523a700 1 -- 172.21.15.32:0/3737576775 learned_addr learned my addr 172.21.15.32:0/3737576775 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f74a39700 1 --2- 172.21.15.32:0/3737576775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x563108978cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f6ffff700 1 --2- 172.21.15.32:0/3737576775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970310 0x5631089791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f7523a700 1 -- 172.21.15.32:0/3737576775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 msgr2=0x563108978cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f7523a700 1 --2- 172.21.15.32:0/3737576775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x563108978cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f7523a700 1 -- 172.21.15.32:0/3737576775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970310 msgr2=0x5631089791f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f7523a700 1 --2- 172.21.15.32:0/3737576775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970310 0x5631089791f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f7523a700 1 -- 172.21.15.32:0/3737576775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f701a1700 con 0x563108970cd0 2023-09-30T14:59:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f74a39700 1 --2- 172.21.15.32:0/3737576775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x563108978cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:59:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f7523a700 1 --2- 172.21.15.32:0/3737576775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x563108980230 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2f680cb750 tx=0x7f2f680074e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773b0b05adb6c2b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.412+0000 7f2f4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:59:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.413+0000 7f2f4ffff700 1 -- 172.21.15.32:0/3737576775 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680a10d0 con 0x563108970cd0 2023-09-30T14:59:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.413+0000 7f2f4ffff700 1 -- 172.21.15.32:0/3737576775 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2f6809b530 con 0x563108970cd0 2023-09-30T14:59:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.413+0000 7f2f82e85b80 1 -- 172.21.15.32:0/3737576775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563108962090 con 0x563108970cd0 2023-09-30T14:59:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.413+0000 7f2f4ffff700 1 -- 172.21.15.32:0/3737576775 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f68009b80 con 0x563108970cd0 2023-09-30T14:59:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.413+0000 7f2f82e85b80 10 client.17184 fetch_fsmap learned FSMap version 53 2023-09-30T14:59:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.413+0000 7f2f82e85b80 1 -- 172.21.15.32:0/3737576775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5631086ecd80 con 0x563108970cd0 2023-09-30T14:59:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.413+0000 7f2f4ffff700 1 -- 172.21.15.32:0/3737576775 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563108962090 con 0x563108970cd0 2023-09-30T14:59:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.413+0000 7f2f4ffff700 1 -- 172.21.15.32:0/3737576775 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 53) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2f680a2ca0 con 0x563108970cd0 2023-09-30T14:59:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.413+0000 7f2f82e85b80 10 client.17184 fetch_fsmap finished waiting for FSMap version 53 2023-09-30T14:59:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.413+0000 7f2f82e85b80 10 client.17184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:59:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.413+0000 7f2f82e85b80 20 client.17184 populate_metadata read hostname 'smithi032' 2023-09-30T14:59:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.413+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/3737576775 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5631089845d0 0x5631089869f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.413+0000 7f2f82e85b80 4 client.17184 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:59:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.413+0000 7f2f82e85b80 1 -- 172.21.15.32:0/3737576775 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631088badb0 con 0x5631089845d0 2023-09-30T14:59:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.413+0000 7f2f74a39700 1 --2- 172.21.15.32:0/3737576775 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5631089845d0 0x5631089869f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.414+0000 7f2f4ffff700 10 client.17184 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:59:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.414+0000 7f2f74a39700 1 --2- 172.21.15.32:0/3737576775 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5631089845d0 0x5631089869f0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f89a2692c09484 server_cookie=c1984f88e6bd2e06 in_seq=0 out_seq=0 2023-09-30T14:59:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.416+0000 7f2f4ffff700 1 -- 172.21.15.32:0/3737576775 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5631088badb0 con 0x5631089845d0 2023-09-30T14:59:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.416+0000 7f2f4ffff700 10 client.17184 handle_command_reply: tid=0 2023-09-30T14:59:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.417+0000 7f2f82e85b80 1 client.17184 shutdown 2023-09-30T14:59:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.417+0000 7f2f82e85b80 1 -- 172.21.15.32:0/3737576775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 msgr2=0x563108980230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.417+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/3737576775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x563108980230 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2f680cb750 tx=0x7f2f680074e0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.417+0000 7f2f82e85b80 1 -- 172.21.15.32:0/3737576775 shutdown_connections 2023-09-30T14:59:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.417+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/3737576775 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5631089845d0 0x5631089869f0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.417+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/3737576775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x563108980230 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.417+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/3737576775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x563108978cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.417+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/3737576775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970310 0x5631089791f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.417+0000 7f2f82e85b80 1 -- 172.21.15.32:0/3737576775 >> 172.21.15.32:0/3737576775 conn(0x5631088e3200 msgr2=0x563108969590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.417+0000 7f2f4ffff700 0 client.17184 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:59:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.417+0000 7f2f82e85b80 1 -- 172.21.15.32:0/3737576775 shutdown_connections 2023-09-30T14:59:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.417+0000 7f2f82e85b80 1 -- 172.21.15.32:0/3737576775 wait complete. 2023-09-30T14:59:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.417+0000 7f2f82e85b80 20 client.17184 trim_cache size 0 max 16384 2023-09-30T14:59:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.434+0000 7f2f82e85b80 1 Processor -- start 2023-09-30T14:59:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.434+0000 7f2f82e85b80 1 -- start start 2023-09-30T14:59:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.434+0000 7f2f82e85b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x563108973b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.434+0000 7f2f82e85b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970310 0x5631089801f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.434+0000 7f2f82e85b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970cd0 0x5631089740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.434+0000 7f2f82e85b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631087428c0 con 0x563108970310 2023-09-30T14:59:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.434+0000 7f2f82e85b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563108978190 con 0x563108970cd0 2023-09-30T14:59:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.434+0000 7f2f82e85b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563108961c60 con 0x56310896f200 2023-09-30T14:59:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.435+0000 7f2f74a39700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x563108973b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.435+0000 7f2f6ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970310 0x5631089801f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.435+0000 7f2f74a39700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x563108973b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54138/0 (socket says 172.21.15.32:54138) 2023-09-30T14:59:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.435+0000 7f2f6ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970310 0x5631089801f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39760/0 (socket says 172.21.15.32:39760) 2023-09-30T14:59:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.435+0000 7f2f7523a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970cd0 0x5631089740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.435+0000 7f2f74a39700 1 -- 172.21.15.32:0/4227253665 learned_addr learned my addr 172.21.15.32:0/4227253665 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.435+0000 7f2f74a39700 1 -- 172.21.15.32:0/4227253665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970cd0 msgr2=0x5631089740c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.435+0000 7f2f74a39700 1 --2- 172.21.15.32:0/4227253665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970cd0 0x5631089740c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.435+0000 7f2f74a39700 1 -- 172.21.15.32:0/4227253665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970310 msgr2=0x5631089801f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.435+0000 7f2f74a39700 1 --2- 172.21.15.32:0/4227253665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970310 0x5631089801f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.435+0000 7f2f74a39700 1 -- 172.21.15.32:0/4227253665 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631086d5c00 con 0x56310896f200 2023-09-30T14:59:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.435+0000 7f2f6ffff700 1 --2- 172.21.15.32:0/4227253665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970310 0x5631089801f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:59:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.435+0000 7f2f74a39700 1 --2- 172.21.15.32:0/4227253665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x563108973b80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2f640097f0 tx=0x7f2f64014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7299a05fbdb04ed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.435+0000 7f2f5b7fe700 1 -- 172.21.15.32:0/4227253665 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6400d980 con 0x56310896f200 2023-09-30T14:59:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.436+0000 7f2f5b7fe700 1 -- 172.21.15.32:0/4227253665 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f64014d50 con 0x56310896f200 2023-09-30T14:59:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.436+0000 7f2f5b7fe700 1 -- 172.21.15.32:0/4227253665 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64022990 con 0x56310896f200 2023-09-30T14:59:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.436+0000 7f2f82e85b80 1 -- 172.21.15.32:0/4227253665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 msgr2=0x563108973b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.436+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/4227253665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x563108973b80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2f640097f0 tx=0x7f2f64014000 comp rx=0 tx=0).stop 2023-09-30T14:59:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.436+0000 7f2f82e85b80 1 -- 172.21.15.32:0/4227253665 shutdown_connections 2023-09-30T14:59:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.436+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/4227253665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970cd0 0x5631089740c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.436+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/4227253665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x563108973b80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.436+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/4227253665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970310 0x5631089801f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.436+0000 7f2f82e85b80 1 -- 172.21.15.32:0/4227253665 >> 172.21.15.32:0/4227253665 conn(0x5631088e3200 msgr2=0x563108978c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.436+0000 7f2f82e85b80 1 -- 172.21.15.32:0/4227253665 shutdown_connections 2023-09-30T14:59:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.436+0000 7f2f82e85b80 1 -- 172.21.15.32:0/4227253665 wait complete. 2023-09-30T14:59:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.437+0000 7f2f82e85b80 1 -- start start 2023-09-30T14:59:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.437+0000 7f2f82e85b80 1 Processor -- start 2023-09-30T14:59:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.437+0000 7f2f82e85b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x56310897dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.437+0000 7f2f82e85b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970310 0x56310897e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.437+0000 7f2f82e85b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x56310897f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.437+0000 7f2f82e85b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310897f6d0 con 0x563108970cd0 2023-09-30T14:59:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.437+0000 7f2f82e85b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310897f840 con 0x563108970310 2023-09-30T14:59:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.437+0000 7f2f82e85b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310897fa00 con 0x56310896f200 2023-09-30T14:59:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.437+0000 7f2f82e85b80 10 client.0 _ll_register_callbacks cb 0x7f2f5c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:59:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.437+0000 7f2f7523a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x56310897f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.437+0000 7f2f74a39700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x56310897dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.437+0000 7f2f7523a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x56310897f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39766/0 (socket says 172.21.15.32:39766) 2023-09-30T14:59:11.445 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.437+0000 7f2f74a39700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x56310897dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54144/0 (socket says 172.21.15.32:54144) 2023-09-30T14:59:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.437+0000 7f2f74a39700 1 -- 172.21.15.32:0/2042276046 learned_addr learned my addr 172.21.15.32:0/2042276046 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.438+0000 7f2f7523a700 1 -- 172.21.15.32:0/2042276046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 msgr2=0x56310897dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.438+0000 7f2f7523a700 1 --2- 172.21.15.32:0/2042276046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x56310897dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.438+0000 7f2f7523a700 1 -- 172.21.15.32:0/2042276046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970310 msgr2=0x56310897e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.438+0000 7f2f7523a700 1 --2- 172.21.15.32:0/2042276046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970310 0x56310897e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.438+0000 7f2f7523a700 1 -- 172.21.15.32:0/2042276046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f5c009b20 con 0x563108970cd0 2023-09-30T14:59:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.438+0000 7f2f7523a700 1 --2- 172.21.15.32:0/2042276046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x56310897f190 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800bfd0 tx=0x7f2f68006230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58204f3181de99cd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.438+0000 7f2f4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:59:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.438+0000 7f2f4ffff700 1 -- 172.21.15.32:0/2042276046 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680a10d0 con 0x563108970cd0 2023-09-30T14:59:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.438+0000 7f2f4ffff700 1 -- 172.21.15.32:0/2042276046 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2f6802bcd0 con 0x563108970cd0 2023-09-30T14:59:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.438+0000 7f2f82e85b80 1 -- 172.21.15.32:0/2042276046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f68003e20 con 0x563108970cd0 2023-09-30T14:59:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.438+0000 7f2f4ffff700 1 -- 172.21.15.32:0/2042276046 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f68009b80 con 0x563108970cd0 2023-09-30T14:59:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.438+0000 7f2f82e85b80 10 client.17190 fetch_fsmap learned FSMap version 53 2023-09-30T14:59:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.438+0000 7f2f82e85b80 1 -- 172.21.15.32:0/2042276046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7f2f6401c040 con 0x563108970cd0 2023-09-30T14:59:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.438+0000 7f2f4ffff700 1 -- 172.21.15.32:0/2042276046 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f68003e20 con 0x563108970cd0 2023-09-30T14:59:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.439+0000 7f2f4ffff700 1 -- 172.21.15.32:0/2042276046 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 53) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2f6809a320 con 0x563108970cd0 2023-09-30T14:59:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.439+0000 7f2f82e85b80 10 client.17190 fetch_fsmap finished waiting for FSMap version 53 2023-09-30T14:59:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.439+0000 7f2f82e85b80 10 client.17190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:59:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.439+0000 7f2f82e85b80 20 client.17190 populate_metadata read hostname 'smithi032' 2023-09-30T14:59:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.439+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/2042276046 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5631089d8ae0 0x5631089daed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.439+0000 7f2f82e85b80 4 client.17190 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:59:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.439+0000 7f2f82e85b80 1 -- 172.21.15.32:0/2042276046 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5631088baad0 con 0x5631089d8ae0 2023-09-30T14:59:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.439+0000 7f2f74a39700 1 --2- 172.21.15.32:0/2042276046 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5631089d8ae0 0x5631089daed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.439+0000 7f2f74a39700 1 --2- 172.21.15.32:0/2042276046 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5631089d8ae0 0x5631089daed0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=850c55fcc5936481 server_cookie=67702876cb220819 in_seq=0 out_seq=0 2023-09-30T14:59:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.439+0000 7f2f4ffff700 10 client.17190 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:59:11.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:11 smithi156 ceph-mon[103579]: pgmap v914: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.6 MiB/s wr, 795 op/s 2023-09-30T14:59:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.830+0000 7f2f4ffff700 1 -- 172.21.15.32:0/2042276046 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5631088baad0 con 0x5631089d8ae0 2023-09-30T14:59:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.830+0000 7f2f4ffff700 10 client.17190 handle_command_reply: tid=0 2023-09-30T14:59:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.830+0000 7f2f82e85b80 1 client.17190 shutdown 2023-09-30T14:59:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.830+0000 7f2f82e85b80 1 -- 172.21.15.32:0/2042276046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 msgr2=0x56310897f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.830+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/2042276046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x56310897f190 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800bfd0 tx=0x7f2f68006230 comp rx=0 tx=0).stop 2023-09-30T14:59:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.830+0000 7f2f82e85b80 1 -- 172.21.15.32:0/2042276046 shutdown_connections 2023-09-30T14:59:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.830+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/2042276046 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5631089d8ae0 0x5631089daed0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.830+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/2042276046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563108970cd0 0x56310897f190 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.830+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/2042276046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56310896f200 0x56310897dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.830+0000 7f2f4ffff700 0 client.17190 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:59:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.830+0000 7f2f82e85b80 1 --2- 172.21.15.32:0/2042276046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563108970310 0x56310897e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.830+0000 7f2f82e85b80 1 -- 172.21.15.32:0/2042276046 >> 172.21.15.32:0/2042276046 conn(0x5631088e3200 msgr2=0x563108978c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.830+0000 7f2f82e85b80 1 -- 172.21.15.32:0/2042276046 shutdown_connections 2023-09-30T14:59:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.830+0000 7f2f82e85b80 1 -- 172.21.15.32:0/2042276046 wait complete. 2023-09-30T14:59:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.830+0000 7f2f82e85b80 20 client.17190 trim_cache size 0 max 16384 2023-09-30T14:59:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:59:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:59:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:59:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:59:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.831+0000 7f2f76c9d700 1 -- 172.21.15.32:0/459653962 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2f600a1020 msgr2=0x7f2f600a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.831+0000 7f2f76c9d700 1 --2- 172.21.15.32:0/459653962 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2f600a1020 0x7f2f600a34e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2f7010a5d0 tx=0x7f2f64008040 comp rx=0 tx=0).stop 2023-09-30T14:59:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.831+0000 7f2f76c9d700 1 -- 172.21.15.32:0/459653962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f70101900 msgr2=0x7f2f70109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.831+0000 7f2f76c9d700 1 --2- 172.21.15.32:0/459653962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f70101900 0x7f2f70109740 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2f68008f10 tx=0x7f2f68008f40 comp rx=0 tx=0).stop 2023-09-30T14:59:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.831+0000 7f2f76c9d700 1 -- 172.21.15.32:0/459653962 shutdown_connections 2023-09-30T14:59:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.831+0000 7f2f76c9d700 1 --2- 172.21.15.32:0/459653962 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f2f600a1020 0x7f2f600a34e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.831+0000 7f2f76c9d700 1 --2- 172.21.15.32:0/459653962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f70101900 0x7f2f70109740 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.831+0000 7f2f76c9d700 1 --2- 172.21.15.32:0/459653962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f700ffd70 0x7f2f70108cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.831+0000 7f2f76c9d700 1 --2- 172.21.15.32:0/459653962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f70100f60 0x7f2f70109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.831+0000 7f2f76c9d700 1 -- 172.21.15.32:0/459653962 >> 172.21.15.32:0/459653962 conn(0x7f2f700fb2a0 msgr2=0x7f2f700fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.831+0000 7f2f76c9d700 1 -- 172.21.15.32:0/459653962 shutdown_connections 2023-09-30T14:59:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:11.831+0000 7f2f76c9d700 1 -- 172.21.15.32:0/459653962 wait complete. 2023-09-30T14:59:11.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:11 smithi148 ceph-mon[102355]: pgmap v914: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.6 MiB/s wr, 795 op/s 2023-09-30T14:59:11.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:11 smithi032 ceph-mon[98592]: pgmap v914: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.6 MiB/s wr, 795 op/s 2023-09-30T14:59:12.058 INFO:tasks.workunit.client.0.smithi032.stdout: 10 307967 35.89 MB/sec execute 345 sec latency 302.382 ms 2023-09-30T14:59:13.057 INFO:tasks.workunit.client.0.smithi032.stdout: 10 308602 35.88 MB/sec execute 346 sec latency 64.398 ms 2023-09-30T14:59:13.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:13 smithi156 ceph-mon[103579]: pgmap v915: 97 pgs: 97 active+clean; 801 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.0 MiB/s wr, 724 op/s 2023-09-30T14:59:13.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:13 smithi148 ceph-mon[102355]: pgmap v915: 97 pgs: 97 active+clean; 801 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.0 MiB/s wr, 724 op/s 2023-09-30T14:59:13.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:13 smithi032 ceph-mon[98592]: pgmap v915: 97 pgs: 97 active+clean; 801 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.0 MiB/s wr, 724 op/s 2023-09-30T14:59:14.057 INFO:tasks.workunit.client.0.smithi032.stdout: 10 309357 35.87 MB/sec execute 347 sec latency 104.426 ms 2023-09-30T14:59:15.057 INFO:tasks.workunit.client.0.smithi032.stdout: 10 310008 35.87 MB/sec execute 348 sec latency 273.875 ms 2023-09-30T14:59:15.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:15 smithi156 ceph-mon[103579]: pgmap v916: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.0 MiB/s wr, 632 op/s 2023-09-30T14:59:15.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:15 smithi148 ceph-mon[102355]: pgmap v916: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.0 MiB/s wr, 632 op/s 2023-09-30T14:59:15.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:15 smithi032 ceph-mon[98592]: pgmap v916: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.0 MiB/s wr, 632 op/s 2023-09-30T14:59:16.057 INFO:tasks.workunit.client.0.smithi032.stdout: 10 310717 35.89 MB/sec execute 349 sec latency 296.292 ms 2023-09-30T14:59:17.057 INFO:tasks.workunit.client.0.smithi032.stdout: 10 311326 35.89 MB/sec execute 350 sec latency 65.004 ms 2023-09-30T14:59:17.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:17 smithi156 ceph-mon[103579]: pgmap v917: 97 pgs: 97 active+clean; 791 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 740 op/s 2023-09-30T14:59:17.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:17 smithi148 ceph-mon[102355]: pgmap v917: 97 pgs: 97 active+clean; 791 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 740 op/s 2023-09-30T14:59:17.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:17 smithi032 ceph-mon[98592]: pgmap v917: 97 pgs: 97 active+clean; 791 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 740 op/s 2023-09-30T14:59:18.057 INFO:tasks.workunit.client.0.smithi032.stdout: 10 311987 35.87 MB/sec execute 351 sec latency 54.866 ms 2023-09-30T14:59:19.057 INFO:tasks.workunit.client.0.smithi032.stdout: 10 312778 35.90 MB/sec execute 352 sec latency 66.325 ms 2023-09-30T14:59:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:19 smithi156 ceph-mon[103579]: pgmap v918: 97 pgs: 97 active+clean; 788 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.6 MiB/s wr, 700 op/s 2023-09-30T14:59:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:19 smithi148 ceph-mon[102355]: pgmap v918: 97 pgs: 97 active+clean; 788 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.6 MiB/s wr, 700 op/s 2023-09-30T14:59:19.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:19 smithi032 ceph-mon[98592]: pgmap v918: 97 pgs: 97 active+clean; 788 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.6 MiB/s wr, 700 op/s 2023-09-30T14:59:20.068 INFO:tasks.workunit.client.0.smithi032.stdout: 10 313454 35.89 MB/sec execute 353 sec latency 156.698 ms 2023-09-30T14:59:21.060 INFO:tasks.workunit.client.0.smithi032.stdout: 10 313927 35.88 MB/sec execute 354 sec latency 389.495 ms 2023-09-30T14:59:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:21 smithi156 ceph-mon[103579]: pgmap v919: 97 pgs: 97 active+clean; 793 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.5 MiB/s wr, 742 op/s 2023-09-30T14:59:21.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:21 smithi148 ceph-mon[102355]: pgmap v919: 97 pgs: 97 active+clean; 793 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.5 MiB/s wr, 742 op/s 2023-09-30T14:59:21.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:21 smithi032 ceph-mon[98592]: pgmap v919: 97 pgs: 97 active+clean; 793 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.5 MiB/s wr, 742 op/s 2023-09-30T14:59:22.060 INFO:tasks.workunit.client.0.smithi032.stdout: 10 314441 35.86 MB/sec execute 355 sec latency 112.332 ms 2023-09-30T14:59:23.061 INFO:tasks.workunit.client.0.smithi032.stdout: 10 315114 35.84 MB/sec execute 356 sec latency 55.769 ms 2023-09-30T14:59:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:23 smithi156 ceph-mon[103579]: pgmap v920: 97 pgs: 97 active+clean; 799 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.4 MiB/s wr, 723 op/s 2023-09-30T14:59:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:23 smithi148 ceph-mon[102355]: pgmap v920: 97 pgs: 97 active+clean; 799 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.4 MiB/s wr, 723 op/s 2023-09-30T14:59:23.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:23 smithi032 ceph-mon[98592]: pgmap v920: 97 pgs: 97 active+clean; 799 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.4 MiB/s wr, 723 op/s 2023-09-30T14:59:24.061 INFO:tasks.workunit.client.0.smithi032.stdout: 10 315847 35.83 MB/sec execute 357 sec latency 69.594 ms 2023-09-30T14:59:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.975+0000 7f4bdcc04700 1 -- 172.21.15.32:0/631623625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bd80fff60 msgr2=0x7f4bd81003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.975+0000 7f4bdcc04700 1 --2- 172.21.15.32:0/631623625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bd80fff60 0x7f4bd81003e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009990 tx=0x7f4bcc009ca0 comp rx=0 tx=0).stop 2023-09-30T14:59:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.975+0000 7f4bdcc04700 1 -- 172.21.15.32:0/631623625 shutdown_connections 2023-09-30T14:59:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.975+0000 7f4bdcc04700 1 --2- 172.21.15.32:0/631623625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd8100920 0x7f4bd81081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.975+0000 7f4bdcc04700 1 --2- 172.21.15.32:0/631623625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd8101310 0x7f4bd8101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.975+0000 7f4bdcc04700 1 --2- 172.21.15.32:0/631623625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bd80fff60 0x7f4bd81003e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.975+0000 7f4bdcc04700 1 -- 172.21.15.32:0/631623625 >> 172.21.15.32:0/631623625 conn(0x7f4bd80faa30 msgr2=0x7f4bd80fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.976+0000 7f4bdcc04700 1 -- 172.21.15.32:0/631623625 shutdown_connections 2023-09-30T14:59:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.976+0000 7f4bdcc04700 1 -- 172.21.15.32:0/631623625 wait complete. 2023-09-30T14:59:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bdcc04700 1 Processor -- start 2023-09-30T14:59:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bdcc04700 1 -- start start 2023-09-30T14:59:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bdcc04700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd80fff60 0x7f4bd8195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bdcc04700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bd8100920 0x7f4bd8196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bdcc04700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd8101310 0x7f4bd8196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bdcc04700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd8197540 con 0x7f4bd8100920 2023-09-30T14:59:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bdcc04700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd81976b0 con 0x7f4bd8101310 2023-09-30T14:59:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bdcc04700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd8197870 con 0x7f4bd80fff60 2023-09-30T14:59:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bd5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bd8100920 0x7f4bd8196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bd659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd80fff60 0x7f4bd8195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bd5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bd8100920 0x7f4bd8196060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39778/0 (socket says 172.21.15.32:39778) 2023-09-30T14:59:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bd659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd80fff60 0x7f4bd8195b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54158/0 (socket says 172.21.15.32:54158) 2023-09-30T14:59:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bd5d9b700 1 -- 172.21.15.32:0/2352880438 learned_addr learned my addr 172.21.15.32:0/2352880438 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bd6d9d700 1 --2- 172.21.15.32:0/2352880438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd8101310 0x7f4bd8196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bd659c700 1 -- 172.21.15.32:0/2352880438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd8101310 msgr2=0x7f4bd8196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bd659c700 1 --2- 172.21.15.32:0/2352880438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd8101310 0x7f4bd8196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bd659c700 1 -- 172.21.15.32:0/2352880438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bd8100920 msgr2=0x7f4bd8196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bd659c700 1 --2- 172.21.15.32:0/2352880438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bd8100920 0x7f4bd8196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.977+0000 7f4bd659c700 1 -- 172.21.15.32:0/2352880438 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc0095f0 con 0x7f4bd80fff60 2023-09-30T14:59:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.978+0000 7f4bd659c700 1 --2- 172.21.15.32:0/2352880438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd80fff60 0x7f4bd8195b20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800e9f0 tx=0x7f4bc800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.978+0000 7f4bc77fe700 1 -- 172.21.15.32:0/2352880438 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc800c9c0 con 0x7f4bd80fff60 2023-09-30T14:59:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.978+0000 7f4bc77fe700 1 -- 172.21.15.32:0/2352880438 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bc8010430 con 0x7f4bd80fff60 2023-09-30T14:59:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.978+0000 7f4bc77fe700 1 -- 172.21.15.32:0/2352880438 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8018680 con 0x7f4bd80fff60 2023-09-30T14:59:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.978+0000 7f4bdcc04700 1 -- 172.21.15.32:0/2352880438 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd8197b50 con 0x7f4bd80fff60 2023-09-30T14:59:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.978+0000 7f4bdcc04700 1 -- 172.21.15.32:0/2352880438 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd819e310 con 0x7f4bd80fff60 2023-09-30T14:59:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.980+0000 7f4bc77fe700 1 -- 172.21.15.32:0/2352880438 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f4bc8010860 con 0x7f4bd80fff60 2023-09-30T14:59:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.981+0000 7f4bc77fe700 1 --2- 172.21.15.32:0/2352880438 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4bc00a1020 0x7f4bc00a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.981+0000 7f4bc77fe700 1 -- 172.21.15.32:0/2352880438 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4bc8014070 con 0x7f4bd80fff60 2023-09-30T14:59:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.981+0000 7f4bd5d9b700 1 --2- 172.21.15.32:0/2352880438 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4bc00a1020 0x7f4bc00a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.982+0000 7f4bd5d9b700 1 --2- 172.21.15.32:0/2352880438 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4bc00a1020 0x7f4bc00a34e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc00b2e0 tx=0x7f4bcc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.982+0000 7f4be8decb80 1 Processor -- start 2023-09-30T14:59:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.982+0000 7f4be8decb80 1 -- start start 2023-09-30T14:59:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.983+0000 7f4be8decb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 0x55697484d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.983+0000 7f4be8decb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484e310 0x55697484e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.983+0000 7f4be8decb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484ecd0 0x556974855560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.983+0000 7f4be8decb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569747b5400 con 0x55697484ecd0 2023-09-30T14:59:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.983+0000 7f4be8decb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569747a6bc0 con 0x55697484d200 2023-09-30T14:59:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.983+0000 7f4be8decb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569747a44e0 con 0x55697484e310 2023-09-30T14:59:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.983+0000 7f4bd659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 0x55697484d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.983+0000 7f4bd659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 0x55697484d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51260/0 (socket says 172.21.15.32:51260) 2023-09-30T14:59:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.983+0000 7f4bd659c700 1 -- 172.21.15.32:0/302522348 learned_addr learned my addr 172.21.15.32:0/302522348 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.983+0000 7f4bd5d9b700 1 --2- 172.21.15.32:0/302522348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484e310 0x55697484e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.983+0000 7f4bd5d9b700 1 -- 172.21.15.32:0/302522348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 msgr2=0x55697484d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.983+0000 7f4bd5d9b700 1 --2- 172.21.15.32:0/302522348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 0x55697484d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:24.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.983+0000 7f4bd6d9d700 1 --2- 172.21.15.32:0/302522348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484ecd0 0x556974855560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:24.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.983+0000 7f4bd5d9b700 1 -- 172.21.15.32:0/302522348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484ecd0 msgr2=0x556974855560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:24.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.983+0000 7f4bd5d9b700 1 --2- 172.21.15.32:0/302522348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484ecd0 0x556974855560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:24.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.983+0000 7f4bd5d9b700 1 -- 172.21.15.32:0/302522348 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556974802870 con 0x55697484e310 2023-09-30T14:59:24.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.984+0000 7f4bd5d9b700 1 --2- 172.21.15.32:0/302522348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484e310 0x55697484e790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc005830 tx=0x7f4bcc020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4bf8b35439d648f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:24.992 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.984+0000 7f4bc4ff9700 1 -- 172.21.15.32:0/302522348 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc00f660 con 0x55697484e310 2023-09-30T14:59:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.984+0000 7f4bc4ff9700 1 -- 172.21.15.32:0/302522348 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bcc01e040 con 0x55697484e310 2023-09-30T14:59:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.984+0000 7f4bc4ff9700 1 -- 172.21.15.32:0/302522348 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc02fa20 con 0x55697484e310 2023-09-30T14:59:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.985+0000 7f4be8decb80 1 -- 172.21.15.32:0/302522348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484e310 msgr2=0x55697484e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.985+0000 7f4be8decb80 1 --2- 172.21.15.32:0/302522348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484e310 0x55697484e790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc005830 tx=0x7f4bcc020000 comp rx=0 tx=0).stop 2023-09-30T14:59:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.987+0000 7f4be8decb80 1 -- 172.21.15.32:0/302522348 shutdown_connections 2023-09-30T14:59:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.987+0000 7f4be8decb80 1 --2- 172.21.15.32:0/302522348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484ecd0 0x556974855560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.988+0000 7f4be8decb80 1 --2- 172.21.15.32:0/302522348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 0x55697484d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.988+0000 7f4be8decb80 1 --2- 172.21.15.32:0/302522348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484e310 0x55697484e790 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.988+0000 7f4be8decb80 1 -- 172.21.15.32:0/302522348 >> 172.21.15.32:0/302522348 conn(0x5569747c1200 msgr2=0x556974847ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.991+0000 7f4be8decb80 1 -- 172.21.15.32:0/302522348 shutdown_connections 2023-09-30T14:59:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.991+0000 7f4be8decb80 1 -- 172.21.15.32:0/302522348 wait complete. 2023-09-30T14:59:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.991+0000 7f4be8decb80 1 -- start start 2023-09-30T14:59:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.991+0000 7f4be8decb80 1 Processor -- start 2023-09-30T14:59:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4be8decb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484d200 0x556974856e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4be8decb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484e310 0x5569748573b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4be8decb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484ecd0 0x55697485e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4be8decb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697479fdd0 con 0x55697484ecd0 2023-09-30T14:59:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4be8decb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697473c6b0 con 0x55697484e310 2023-09-30T14:59:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4be8decb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697473c350 con 0x55697484d200 2023-09-30T14:59:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4be8decb80 10 client.0 _ll_register_callbacks cb 0x5569745997a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:59:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4bd659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484d200 0x556974856e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4bd659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484d200 0x556974856e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54172/0 (socket says 172.21.15.32:54172) 2023-09-30T14:59:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4bd659c700 1 -- 172.21.15.32:0/4156597817 learned_addr learned my addr 172.21.15.32:0/4156597817 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4bd6d9d700 1 --2- 172.21.15.32:0/4156597817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484ecd0 0x55697485e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4bd5d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484e310 0x5569748573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4bd659c700 1 -- 172.21.15.32:0/4156597817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484e310 msgr2=0x5569748573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4bd659c700 1 --2- 172.21.15.32:0/4156597817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484e310 0x5569748573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4bd659c700 1 -- 172.21.15.32:0/4156597817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484ecd0 msgr2=0x55697485e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4bd659c700 1 --2- 172.21.15.32:0/4156597817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484ecd0 0x55697485e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4bd659c700 1 -- 172.21.15.32:0/4156597817 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd8108af0 con 0x55697484d200 2023-09-30T14:59:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4bd659c700 1 --2- 172.21.15.32:0/4156597817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484d200 0x556974856e70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80ca650 tx=0x7f4bc809a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca7dba2b5e9f7345 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.992+0000 7f4bbdffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T14:59:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.993+0000 7f4bbdffb700 1 -- 172.21.15.32:0/4156597817 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8099ca0 con 0x55697484d200 2023-09-30T14:59:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.993+0000 7f4be8decb80 1 -- 172.21.15.32:0/4156597817 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556974840090 con 0x55697484d200 2023-09-30T14:59:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.993+0000 7f4bbdffb700 1 -- 172.21.15.32:0/4156597817 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4bc809ddf0 con 0x55697484d200 2023-09-30T14:59:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.993+0000 7f4bbdffb700 1 -- 172.21.15.32:0/4156597817 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bc80b2020 con 0x55697484d200 2023-09-30T14:59:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.993+0000 7f4be8decb80 10 client.26857 fetch_fsmap learned FSMap version 53 2023-09-30T14:59:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.993+0000 7f4be8decb80 1 -- 172.21.15.32:0/4156597817 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5569745cad80 con 0x55697484d200 2023-09-30T14:59:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.993+0000 7f4bbdffb700 1 -- 172.21.15.32:0/4156597817 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bc80b6070 con 0x55697484d200 2023-09-30T14:59:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.993+0000 7f4bbdffb700 1 -- 172.21.15.32:0/4156597817 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 53) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4bc80a1ba0 con 0x55697484d200 2023-09-30T14:59:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.993+0000 7f4be8decb80 10 client.26857 fetch_fsmap finished waiting for FSMap version 53 2023-09-30T14:59:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.993+0000 7f4be8decb80 10 client.26857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:59:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.993+0000 7f4be8decb80 20 client.26857 populate_metadata read hostname 'smithi032' 2023-09-30T14:59:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.993+0000 7f4be8decb80 1 --2- 172.21.15.32:0/4156597817 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556974862c70 0x556974865090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.993+0000 7f4be8decb80 4 client.26857 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:59:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.993+0000 7f4be8decb80 1 -- 172.21.15.32:0/4156597817 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556974798db0 con 0x556974862c70 2023-09-30T14:59:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.994+0000 7f4bd5d9b700 1 --2- 172.21.15.32:0/4156597817 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556974862c70 0x556974865090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.994+0000 7f4bbdffb700 10 client.26857 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:59:25.002 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.994+0000 7f4bd5d9b700 1 --2- 172.21.15.32:0/4156597817 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556974862c70 0x556974865090 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=720504b986e2149 server_cookie=ce27cf26f6863a3a in_seq=0 out_seq=0 2023-09-30T14:59:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.999+0000 7f4bbdffb700 1 -- 172.21.15.32:0/4156597817 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556974798db0 con 0x556974862c70 2023-09-30T14:59:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.999+0000 7f4bbdffb700 10 client.26857 handle_command_reply: tid=0 2023-09-30T14:59:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.999+0000 7f4be8decb80 1 client.26857 shutdown 2023-09-30T14:59:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.999+0000 7f4be8decb80 1 -- 172.21.15.32:0/4156597817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484d200 msgr2=0x556974856e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.999+0000 7f4be8decb80 1 --2- 172.21.15.32:0/4156597817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484d200 0x556974856e70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80ca650 tx=0x7f4bc809a1e0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.999+0000 7f4be8decb80 1 -- 172.21.15.32:0/4156597817 shutdown_connections 2023-09-30T14:59:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.999+0000 7f4be8decb80 1 --2- 172.21.15.32:0/4156597817 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x556974862c70 0x556974865090 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.999+0000 7f4be8decb80 1 --2- 172.21.15.32:0/4156597817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484ecd0 0x55697485e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.999+0000 7f4bbdffb700 0 client.26857 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:59:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.999+0000 7f4be8decb80 1 --2- 172.21.15.32:0/4156597817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484d200 0x556974856e70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.999+0000 7f4be8decb80 1 --2- 172.21.15.32:0/4156597817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484e310 0x5569748573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.999+0000 7f4be8decb80 1 -- 172.21.15.32:0/4156597817 >> 172.21.15.32:0/4156597817 conn(0x5569747c1200 msgr2=0x556974847590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.999+0000 7f4be8decb80 1 -- 172.21.15.32:0/4156597817 shutdown_connections 2023-09-30T14:59:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:24.999+0000 7f4be8decb80 1 -- 172.21.15.32:0/4156597817 wait complete. 2023-09-30T14:59:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.000+0000 7f4be8decb80 20 client.26857 trim_cache size 0 max 16384 2023-09-30T14:59:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.025+0000 7f4be8decb80 1 Processor -- start 2023-09-30T14:59:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.025+0000 7f4be8decb80 1 -- start start 2023-09-30T14:59:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4be8decb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 0x556974847590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4be8decb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484e310 0x55697485e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4be8decb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484ecd0 0x556974855ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4be8decb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569746208c0 con 0x55697484e310 2023-09-30T14:59:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4be8decb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697483fbd0 con 0x55697484d200 2023-09-30T14:59:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4be8decb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556974851790 con 0x55697484ecd0 2023-09-30T14:59:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4bd5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484e310 0x55697485e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4bd659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 0x556974847590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4bd5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484e310 0x55697485e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39800/0 (socket says 172.21.15.32:39800) 2023-09-30T14:59:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4bd659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 0x556974847590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51274/0 (socket says 172.21.15.32:51274) 2023-09-30T14:59:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4bd5d9b700 1 -- 172.21.15.32:0/1561296796 learned_addr learned my addr 172.21.15.32:0/1561296796 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4bd6d9d700 1 --2- 172.21.15.32:0/1561296796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484ecd0 0x556974855ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4bd5d9b700 1 -- 172.21.15.32:0/1561296796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484ecd0 msgr2=0x556974855ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4bd5d9b700 1 --2- 172.21.15.32:0/1561296796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484ecd0 0x556974855ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4bd5d9b700 1 -- 172.21.15.32:0/1561296796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 msgr2=0x556974847590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4bd5d9b700 1 --2- 172.21.15.32:0/1561296796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 0x556974847590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4bd5d9b700 1 -- 172.21.15.32:0/1561296796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569745b3c00 con 0x55697484e310 2023-09-30T14:59:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.026+0000 7f4bd6d9d700 1 --2- 172.21.15.32:0/1561296796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484ecd0 0x556974855ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T14:59:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.027+0000 7f4bd5d9b700 1 --2- 172.21.15.32:0/1561296796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484e310 0x55697485e3b0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0095c0 tx=0x7f4bcc0092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6e6a1ef63dbad server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.027+0000 7f4bc57fa700 1 -- 172.21.15.32:0/1561296796 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc02fb60 con 0x55697484e310 2023-09-30T14:59:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.027+0000 7f4bc57fa700 1 -- 172.21.15.32:0/1561296796 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bcc03e1d0 con 0x55697484e310 2023-09-30T14:59:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.027+0000 7f4bc57fa700 1 -- 172.21.15.32:0/1561296796 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc00f750 con 0x55697484e310 2023-09-30T14:59:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.028+0000 7f4be8decb80 1 -- 172.21.15.32:0/1561296796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484e310 msgr2=0x55697485e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.028+0000 7f4be8decb80 1 --2- 172.21.15.32:0/1561296796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484e310 0x55697485e3b0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0095c0 tx=0x7f4bcc0092f0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.028+0000 7f4be8decb80 1 -- 172.21.15.32:0/1561296796 shutdown_connections 2023-09-30T14:59:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.028+0000 7f4be8decb80 1 --2- 172.21.15.32:0/1561296796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484ecd0 0x556974855ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.028+0000 7f4be8decb80 1 --2- 172.21.15.32:0/1561296796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 0x556974847590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.028+0000 7f4be8decb80 1 --2- 172.21.15.32:0/1561296796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484e310 0x55697485e3b0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.028+0000 7f4be8decb80 1 -- 172.21.15.32:0/1561296796 >> 172.21.15.32:0/1561296796 conn(0x5569747c1200 msgr2=0x5569748699f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.044+0000 7f4be8decb80 1 -- 172.21.15.32:0/1561296796 shutdown_connections 2023-09-30T14:59:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.044+0000 7f4be8decb80 1 -- 172.21.15.32:0/1561296796 wait complete. 2023-09-30T14:59:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.044+0000 7f4be8decb80 1 -- start start 2023-09-30T14:59:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.044+0000 7f4be8decb80 1 Processor -- start 2023-09-30T14:59:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.045+0000 7f4be8decb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 0x55697485c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.045+0000 7f4be8decb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484e310 0x55697485c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.045+0000 7f4be8decb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484ecd0 0x55697485d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.045+0000 7f4be8decb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697485da90 con 0x55697484ecd0 2023-09-30T14:59:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.045+0000 7f4be8decb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697485dc00 con 0x55697484d200 2023-09-30T14:59:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.045+0000 7f4be8decb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55697485dd70 con 0x55697484e310 2023-09-30T14:59:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.045+0000 7f4be8decb80 10 client.0 _ll_register_callbacks cb 0x7f4bc8099b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:59:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.045+0000 7f4bd5d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484e310 0x55697485c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.045+0000 7f4bd5d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484e310 0x55697485c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54186/0 (socket says 172.21.15.32:54186) 2023-09-30T14:59:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.045+0000 7f4bd5d9b700 1 -- 172.21.15.32:0/59270217 learned_addr learned my addr 172.21.15.32:0/59270217 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.045+0000 7f4bd659c700 1 --2- 172.21.15.32:0/59270217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 0x55697485c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.046+0000 7f4bd659c700 1 -- 172.21.15.32:0/59270217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484e310 msgr2=0x55697485c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.046+0000 7f4bd659c700 1 --2- 172.21.15.32:0/59270217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484e310 0x55697485c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.046+0000 7f4bd659c700 1 -- 172.21.15.32:0/59270217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484ecd0 msgr2=0x55697485d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.046+0000 7f4bd659c700 1 --2- 172.21.15.32:0/59270217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484ecd0 0x55697485d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.046+0000 7f4bd659c700 1 -- 172.21.15.32:0/59270217 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556974802870 con 0x55697484d200 2023-09-30T14:59:25.060 INFO:tasks.workunit.client.0.smithi032.stdout: 10 316640 35.90 MB/sec execute 358 sec latency 60.125 ms 2023-09-30T14:59:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.063+0000 7f4bd659c700 1 --2- 172.21.15.32:0/59270217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 0x55697485c000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80961a0 tx=0x7f4bc8010c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f786390d3c023670 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.064+0000 7f4bbdffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:59:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.070+0000 7f4bbdffb700 1 -- 172.21.15.32:0/59270217 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc809ddf0 con 0x55697484d200 2023-09-30T14:59:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.070+0000 7f4bbdffb700 1 -- 172.21.15.32:0/59270217 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4bc8099ca0 con 0x55697484d200 2023-09-30T14:59:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.070+0000 7f4be8decb80 1 -- 172.21.15.32:0/59270217 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4bcc01e040 con 0x55697484d200 2023-09-30T14:59:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.082+0000 7f4bbdffb700 1 -- 172.21.15.32:0/59270217 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bc8095a90 con 0x55697484d200 2023-09-30T14:59:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.087+0000 7f4be8decb80 10 client.26822 fetch_fsmap learned FSMap version 53 2023-09-30T14:59:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.087+0000 7f4be8decb80 1 -- 172.21.15.32:0/59270217 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5569745f11f0 con 0x55697484d200 2023-09-30T14:59:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.091+0000 7f4bbdffb700 1 -- 172.21.15.32:0/59270217 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bcc01e040 con 0x55697484d200 2023-09-30T14:59:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.091+0000 7f4bbdffb700 1 -- 172.21.15.32:0/59270217 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 53) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4bc809eb30 con 0x55697484d200 2023-09-30T14:59:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.091+0000 7f4be8decb80 10 client.26822 fetch_fsmap finished waiting for FSMap version 53 2023-09-30T14:59:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.091+0000 7f4be8decb80 10 client.26822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:59:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.091+0000 7f4be8decb80 20 client.26822 populate_metadata read hostname 'smithi032' 2023-09-30T14:59:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.091+0000 7f4be8decb80 1 --2- 172.21.15.32:0/59270217 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5569748d0e20 0x5569748d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.091+0000 7f4be8decb80 4 client.26822 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:59:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.091+0000 7f4be8decb80 1 -- 172.21.15.32:0/59270217 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556974798ad0 con 0x5569748d0e20 2023-09-30T14:59:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.093+0000 7f4bd5d9b700 1 --2- 172.21.15.32:0/59270217 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5569748d0e20 0x5569748d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.122+0000 7f4bd5d9b700 1 --2- 172.21.15.32:0/59270217 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5569748d0e20 0x5569748d3240 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1100d4765722ffb server_cookie=b329daf5b527f214 in_seq=0 out_seq=0 2023-09-30T14:59:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.122+0000 7f4bbdffb700 10 client.26822 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:59:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.749+0000 7f4bbdffb700 1 -- 172.21.15.32:0/59270217 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556974798ad0 con 0x5569748d0e20 2023-09-30T14:59:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.749+0000 7f4bbdffb700 10 client.26822 handle_command_reply: tid=0 2023-09-30T14:59:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.749+0000 7f4be8decb80 1 client.26822 shutdown 2023-09-30T14:59:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.749+0000 7f4be8decb80 1 -- 172.21.15.32:0/59270217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 msgr2=0x55697485c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.749+0000 7f4be8decb80 1 --2- 172.21.15.32:0/59270217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 0x55697485c000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80961a0 tx=0x7f4bc8010c60 comp rx=0 tx=0).stop 2023-09-30T14:59:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.749+0000 7f4be8decb80 1 -- 172.21.15.32:0/59270217 shutdown_connections 2023-09-30T14:59:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.749+0000 7f4be8decb80 1 --2- 172.21.15.32:0/59270217 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5569748d0e20 0x5569748d3240 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.749+0000 7f4be8decb80 1 --2- 172.21.15.32:0/59270217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55697484ecd0 0x55697485d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.749+0000 7f4be8decb80 1 --2- 172.21.15.32:0/59270217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55697484d200 0x55697485c000 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.749+0000 7f4be8decb80 1 --2- 172.21.15.32:0/59270217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55697484e310 0x55697485c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.749+0000 7f4be8decb80 1 -- 172.21.15.32:0/59270217 >> 172.21.15.32:0/59270217 conn(0x5569747c1200 msgr2=0x556974855ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.749+0000 7f4bbdffb700 0 client.26822 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:59:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.749+0000 7f4be8decb80 1 -- 172.21.15.32:0/59270217 shutdown_connections 2023-09-30T14:59:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.749+0000 7f4be8decb80 1 -- 172.21.15.32:0/59270217 wait complete. 2023-09-30T14:59:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.750+0000 7f4be8decb80 20 client.26822 trim_cache size 0 max 16384 2023-09-30T14:59:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:59:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:59:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:59:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:59:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.750+0000 7f4bdcc04700 1 -- 172.21.15.32:0/2352880438 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4bc00a1020 msgr2=0x7f4bc00a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.750+0000 7f4bdcc04700 1 --2- 172.21.15.32:0/2352880438 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4bc00a1020 0x7f4bc00a34e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc00b2e0 tx=0x7f4bcc005fb0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.750+0000 7f4bdcc04700 1 -- 172.21.15.32:0/2352880438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd80fff60 msgr2=0x7f4bd8195b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.750+0000 7f4bdcc04700 1 --2- 172.21.15.32:0/2352880438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd80fff60 0x7f4bd8195b20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800e9f0 tx=0x7f4bc800ed00 comp rx=0 tx=0).stop 2023-09-30T14:59:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.752+0000 7f4bdcc04700 1 -- 172.21.15.32:0/2352880438 shutdown_connections 2023-09-30T14:59:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.752+0000 7f4bdcc04700 1 --2- 172.21.15.32:0/2352880438 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f4bc00a1020 0x7f4bc00a34e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.752+0000 7f4bdcc04700 1 --2- 172.21.15.32:0/2352880438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd8101310 0x7f4bd8196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.752+0000 7f4bdcc04700 1 --2- 172.21.15.32:0/2352880438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bd80fff60 0x7f4bd8195b20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.752+0000 7f4bdcc04700 1 --2- 172.21.15.32:0/2352880438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bd8100920 0x7f4bd8196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.752+0000 7f4bdcc04700 1 -- 172.21.15.32:0/2352880438 >> 172.21.15.32:0/2352880438 conn(0x7f4bd80faa30 msgr2=0x7f4bd8104440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.752+0000 7f4bdcc04700 1 -- 172.21.15.32:0/2352880438 shutdown_connections 2023-09-30T14:59:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:25.752+0000 7f4bdcc04700 1 -- 172.21.15.32:0/2352880438 wait complete. 2023-09-30T14:59:25.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:25 smithi148 ceph-mon[102355]: pgmap v921: 97 pgs: 97 active+clean; 802 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.0 MiB/s wr, 680 op/s 2023-09-30T14:59:25.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:25 smithi032 ceph-mon[98592]: pgmap v921: 97 pgs: 97 active+clean; 802 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.0 MiB/s wr, 680 op/s 2023-09-30T14:59:26.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:25 smithi156 ceph-mon[103579]: pgmap v921: 97 pgs: 97 active+clean; 802 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.0 MiB/s wr, 680 op/s 2023-09-30T14:59:26.064 INFO:tasks.workunit.client.0.smithi032.stdout: 10 316907 35.83 MB/sec execute 359 sec latency 308.658 ms 2023-09-30T14:59:27.064 INFO:tasks.workunit.client.0.smithi032.stdout: 10 317325 35.78 MB/sec execute 360 sec latency 91.227 ms 2023-09-30T14:59:27.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:27 smithi148 ceph-mon[102355]: pgmap v922: 97 pgs: 97 active+clean; 806 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.9 MiB/s wr, 758 op/s 2023-09-30T14:59:27.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:27 smithi032 ceph-mon[98592]: pgmap v922: 97 pgs: 97 active+clean; 806 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.9 MiB/s wr, 758 op/s 2023-09-30T14:59:28.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:27 smithi156 ceph-mon[103579]: pgmap v922: 97 pgs: 97 active+clean; 806 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.9 MiB/s wr, 758 op/s 2023-09-30T14:59:28.064 INFO:tasks.workunit.client.0.smithi032.stdout: 10 317949 35.77 MB/sec execute 361 sec latency 57.373 ms 2023-09-30T14:59:29.064 INFO:tasks.workunit.client.0.smithi032.stdout: 10 318628 35.76 MB/sec execute 362 sec latency 120.768 ms 2023-09-30T14:59:29.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:29 smithi148 ceph-mon[102355]: pgmap v923: 97 pgs: 97 active+clean; 807 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.5 MiB/s wr, 609 op/s 2023-09-30T14:59:29.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:29 smithi032 ceph-mon[98592]: pgmap v923: 97 pgs: 97 active+clean; 807 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.5 MiB/s wr, 609 op/s 2023-09-30T14:59:30.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:29 smithi156 ceph-mon[103579]: pgmap v923: 97 pgs: 97 active+clean; 807 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.5 MiB/s wr, 609 op/s 2023-09-30T14:59:30.064 INFO:tasks.workunit.client.0.smithi032.stdout: 10 319356 35.77 MB/sec execute 363 sec latency 118.615 ms 2023-09-30T14:59:31.068 INFO:tasks.workunit.client.0.smithi032.stdout: 10 319843 35.76 MB/sec execute 364 sec latency 283.115 ms 2023-09-30T14:59:31.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:31 smithi148 ceph-mon[102355]: pgmap v924: 97 pgs: 97 active+clean; 812 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.2 MiB/s wr, 725 op/s 2023-09-30T14:59:31.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:31 smithi032 ceph-mon[98592]: pgmap v924: 97 pgs: 97 active+clean; 812 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.2 MiB/s wr, 725 op/s 2023-09-30T14:59:32.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:31 smithi156 ceph-mon[103579]: pgmap v924: 97 pgs: 97 active+clean; 812 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.2 MiB/s wr, 725 op/s 2023-09-30T14:59:32.068 INFO:tasks.workunit.client.0.smithi032.stdout: 10 320358 35.75 MB/sec execute 365 sec latency 77.504 ms 2023-09-30T14:59:33.068 INFO:tasks.workunit.client.0.smithi032.stdout: 10 320994 35.74 MB/sec execute 366 sec latency 42.912 ms 2023-09-30T14:59:33.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:33 smithi148 ceph-mon[102355]: pgmap v925: 97 pgs: 97 active+clean; 814 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.4 MiB/s wr, 737 op/s 2023-09-30T14:59:33.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:33 smithi032 ceph-mon[98592]: pgmap v925: 97 pgs: 97 active+clean; 814 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.4 MiB/s wr, 737 op/s 2023-09-30T14:59:34.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:33 smithi156 ceph-mon[103579]: pgmap v925: 97 pgs: 97 active+clean; 814 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.4 MiB/s wr, 737 op/s 2023-09-30T14:59:34.068 INFO:tasks.workunit.client.0.smithi032.stdout: 10 321599 35.72 MB/sec execute 367 sec latency 76.359 ms 2023-09-30T14:59:35.068 INFO:tasks.workunit.client.0.smithi032.stdout: 10 322468 35.75 MB/sec execute 368 sec latency 57.287 ms 2023-09-30T14:59:35.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:35 smithi148 ceph-mon[102355]: pgmap v926: 97 pgs: 97 active+clean; 804 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.0 MiB/s wr, 673 op/s 2023-09-30T14:59:35.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:35 smithi032 ceph-mon[98592]: pgmap v926: 97 pgs: 97 active+clean; 804 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.0 MiB/s wr, 673 op/s 2023-09-30T14:59:36.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:35 smithi156 ceph-mon[103579]: pgmap v926: 97 pgs: 97 active+clean; 804 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.0 MiB/s wr, 673 op/s 2023-09-30T14:59:36.068 INFO:tasks.workunit.client.0.smithi032.stdout: 10 323011 35.74 MB/sec execute 369 sec latency 267.272 ms 2023-09-30T14:59:37.068 INFO:tasks.workunit.client.0.smithi032.stdout: 10 323619 35.74 MB/sec execute 370 sec latency 74.224 ms 2023-09-30T14:59:37.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:37 smithi148 ceph-mon[102355]: pgmap v927: 97 pgs: 97 active+clean; 793 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.3 MiB/s wr, 779 op/s 2023-09-30T14:59:37.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:37 smithi032 ceph-mon[98592]: pgmap v927: 97 pgs: 97 active+clean; 793 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.3 MiB/s wr, 779 op/s 2023-09-30T14:59:38.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:37 smithi156 ceph-mon[103579]: pgmap v927: 97 pgs: 97 active+clean; 793 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.3 MiB/s wr, 779 op/s 2023-09-30T14:59:38.068 INFO:tasks.workunit.client.0.smithi032.stdout: 10 324291 35.74 MB/sec execute 371 sec latency 39.538 ms 2023-09-30T14:59:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.879+0000 7f657bc96700 1 -- 172.21.15.32:0/3173721510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6574102970 msgr2=0x7f6574109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.879+0000 7f657bc96700 1 --2- 172.21.15.32:0/3173721510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6574102970 0x7f6574109290 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f656c00b0a0 tx=0x7f656c00b3b0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.879+0000 7f657bc96700 1 -- 172.21.15.32:0/3173721510 shutdown_connections 2023-09-30T14:59:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.879+0000 7f657bc96700 1 --2- 172.21.15.32:0/3173721510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6574102970 0x7f6574109290 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.879+0000 7f657bc96700 1 --2- 172.21.15.32:0/3173721510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6574100e60 0x7f6574101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.879+0000 7f657bc96700 1 --2- 172.21.15.32:0/3173721510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6574101fb0 0x7f6574102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.879+0000 7f657bc96700 1 -- 172.21.15.32:0/3173721510 >> 172.21.15.32:0/3173721510 conn(0x7f65740fc3d0 msgr2=0x7f65740fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.880+0000 7f657bc96700 1 -- 172.21.15.32:0/3173721510 shutdown_connections 2023-09-30T14:59:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.880+0000 7f657bc96700 1 -- 172.21.15.32:0/3173721510 wait complete. 2023-09-30T14:59:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.880+0000 7f657bc96700 1 Processor -- start 2023-09-30T14:59:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.880+0000 7f657bc96700 1 -- start start 2023-09-30T14:59:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f657bc96700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6574100e60 0x7f6574073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f657bc96700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6574101fb0 0x7f6574073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f657bc96700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6574102970 0x7f6574073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f657bc96700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6574074db0 con 0x7f6574102970 2023-09-30T14:59:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f657bc96700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6574074f20 con 0x7f6574100e60 2023-09-30T14:59:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f657bc96700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65740750e0 con 0x7f6574101fb0 2023-09-30T14:59:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f657a233700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6574102970 0x7f6574073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f6579a32700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6574100e60 0x7f6574073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f6579231700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6574101fb0 0x7f6574073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f6579a32700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6574100e60 0x7f6574073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51294/0 (socket says 172.21.15.32:51294) 2023-09-30T14:59:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f6579a32700 1 -- 172.21.15.32:0/3981523599 learned_addr learned my addr 172.21.15.32:0/3981523599 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f6579a32700 1 -- 172.21.15.32:0/3981523599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6574101fb0 msgr2=0x7f6574073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f6579a32700 1 --2- 172.21.15.32:0/3981523599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6574101fb0 0x7f6574073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f6579a32700 1 -- 172.21.15.32:0/3981523599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6574102970 msgr2=0x7f6574073c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f6579a32700 1 --2- 172.21.15.32:0/3981523599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6574102970 0x7f6574073c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f6579a32700 1 -- 172.21.15.32:0/3981523599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c009d00 con 0x7f6574100e60 2023-09-30T14:59:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f6579a32700 1 --2- 172.21.15.32:0/3981523599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6574100e60 0x7f6574073210 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f657000e9a0 tx=0x7f657000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.881+0000 7f656affd700 1 -- 172.21.15.32:0/3981523599 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570009790 con 0x7f6574100e60 2023-09-30T14:59:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.882+0000 7f657bc96700 1 -- 172.21.15.32:0/3981523599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657419ae00 con 0x7f6574100e60 2023-09-30T14:59:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.882+0000 7f656affd700 1 -- 172.21.15.32:0/3981523599 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6570010430 con 0x7f6574100e60 2023-09-30T14:59:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.882+0000 7f656affd700 1 -- 172.21.15.32:0/3981523599 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570018680 con 0x7f6574100e60 2023-09-30T14:59:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.882+0000 7f657bc96700 1 -- 172.21.15.32:0/3981523599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657419b360 con 0x7f6574100e60 2023-09-30T14:59:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.884+0000 7f656affd700 1 -- 172.21.15.32:0/3981523599 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f65700105a0 con 0x7f6574100e60 2023-09-30T14:59:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.885+0000 7f656affd700 1 --2- 172.21.15.32:0/3981523599 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f65600a0fa0 0x7f65600a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.885+0000 7f6579231700 1 --2- 172.21.15.32:0/3981523599 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f65600a0fa0 0x7f65600a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.885+0000 7f656affd700 1 -- 172.21.15.32:0/3981523599 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6570014070 con 0x7f6574100e60 2023-09-30T14:59:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.885+0000 7f6587e7eb80 1 Processor -- start 2023-09-30T14:59:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.885+0000 7f6579231700 1 --2- 172.21.15.32:0/3981523599 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f65600a0fa0 0x7f65600a3460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6574074510 tx=0x7f6564009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6587e7eb80 1 -- start start 2023-09-30T14:59:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6587e7eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37887200 0x55ee37887620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6587e7eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37888310 0x55ee37888790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6587e7eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888cd0 0x55ee3788f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6587e7eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee377ef400 con 0x55ee37887200 2023-09-30T14:59:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6587e7eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee377e0bc0 con 0x55ee37888310 2023-09-30T14:59:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6587e7eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee377de4e0 con 0x55ee37888cd0 2023-09-30T14:59:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6579a32700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37887200 0x55ee37887620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6579a32700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37887200 0x55ee37887620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39826/0 (socket says 172.21.15.32:39826) 2023-09-30T14:59:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6579a32700 1 -- 172.21.15.32:0/139049152 learned_addr learned my addr 172.21.15.32:0/139049152 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6579231700 1 --2- 172.21.15.32:0/139049152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37888310 0x55ee37888790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f657a233700 1 --2- 172.21.15.32:0/139049152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888cd0 0x55ee3788f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6579a32700 1 -- 172.21.15.32:0/139049152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888cd0 msgr2=0x55ee3788f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6579a32700 1 --2- 172.21.15.32:0/139049152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888cd0 0x55ee3788f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6579a32700 1 -- 172.21.15.32:0/139049152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37888310 msgr2=0x55ee37888790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6579a32700 1 --2- 172.21.15.32:0/139049152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37888310 0x55ee37888790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6579a32700 1 -- 172.21.15.32:0/139049152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee3783c870 con 0x55ee37887200 2023-09-30T14:59:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.886+0000 7f6579a32700 1 --2- 172.21.15.32:0/139049152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37887200 0x55ee37887620 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6570099d10 tx=0x7f65700997e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b581917dc4ed63a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.887+0000 7f655ffff700 1 -- 172.21.15.32:0/139049152 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570099d80 con 0x55ee37887200 2023-09-30T14:59:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.887+0000 7f655ffff700 1 -- 172.21.15.32:0/139049152 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f657009ded0 con 0x55ee37887200 2023-09-30T14:59:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.887+0000 7f655ffff700 1 -- 172.21.15.32:0/139049152 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65700b2210 con 0x55ee37887200 2023-09-30T14:59:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.887+0000 7f6587e7eb80 1 -- 172.21.15.32:0/139049152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37887200 msgr2=0x55ee37887620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.887+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/139049152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37887200 0x55ee37887620 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6570099d10 tx=0x7f65700997e0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.887+0000 7f6587e7eb80 1 -- 172.21.15.32:0/139049152 shutdown_connections 2023-09-30T14:59:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.887+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/139049152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888cd0 0x55ee3788f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.887+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/139049152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37887200 0x55ee37887620 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.887+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/139049152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37888310 0x55ee37888790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.887+0000 7f6587e7eb80 1 -- 172.21.15.32:0/139049152 >> 172.21.15.32:0/139049152 conn(0x55ee377fb200 msgr2=0x55ee37881ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.888+0000 7f6587e7eb80 1 -- 172.21.15.32:0/139049152 shutdown_connections 2023-09-30T14:59:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.888+0000 7f6587e7eb80 1 -- 172.21.15.32:0/139049152 wait complete. 2023-09-30T14:59:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.888+0000 7f6587e7eb80 1 -- start start 2023-09-30T14:59:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.888+0000 7f6587e7eb80 1 Processor -- start 2023-09-30T14:59:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.888+0000 7f6587e7eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37887200 0x55ee37890fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.888+0000 7f6587e7eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888310 0x55ee378914e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.888+0000 7f6587e7eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee37898520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.888+0000 7f6587e7eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee377d9dd0 con 0x55ee37888cd0 2023-09-30T14:59:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.888+0000 7f6587e7eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee377766b0 con 0x55ee37887200 2023-09-30T14:59:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.888+0000 7f6587e7eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee37776350 con 0x55ee37888310 2023-09-30T14:59:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.888+0000 7f6587e7eb80 10 client.0 _ll_register_callbacks cb 0x55ee375d37a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:59:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f657a233700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee37898520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f6579a32700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37887200 0x55ee37890fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f6579231700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888310 0x55ee378914e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f6579a32700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37887200 0x55ee37890fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51308/0 (socket says 172.21.15.32:51308) 2023-09-30T14:59:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f6579231700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888310 0x55ee378914e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54216/0 (socket says 172.21.15.32:54216) 2023-09-30T14:59:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f6579a32700 1 -- 172.21.15.32:0/3740600683 learned_addr learned my addr 172.21.15.32:0/3740600683 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f657a233700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee37898520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39836/0 (socket says 172.21.15.32:39836) 2023-09-30T14:59:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f6579a32700 1 -- 172.21.15.32:0/3740600683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888310 msgr2=0x55ee378914e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f6579a32700 1 --2- 172.21.15.32:0/3740600683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888310 0x55ee378914e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f6579a32700 1 -- 172.21.15.32:0/3740600683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 msgr2=0x55ee37898520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f6579a32700 1 --2- 172.21.15.32:0/3740600683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee37898520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f6579a32700 1 -- 172.21.15.32:0/3740600683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657419b360 con 0x55ee37887200 2023-09-30T14:59:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f6579231700 1 --2- 172.21.15.32:0/3740600683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888310 0x55ee378914e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:59:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f657a233700 1 --2- 172.21.15.32:0/3740600683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee37898520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:59:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f6579a32700 1 --2- 172.21.15.32:0/3740600683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37887200 0x55ee37890fa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f65700a1be0 tx=0x7f65700a2180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69415922dd826154 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f655d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T14:59:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f655d7fa700 1 -- 172.21.15.32:0/3740600683 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65700b2410 con 0x55ee37887200 2023-09-30T14:59:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f655d7fa700 1 -- 172.21.15.32:0/3740600683 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6570099d80 con 0x55ee37887200 2023-09-30T14:59:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f6587e7eb80 1 -- 172.21.15.32:0/3740600683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee3787a090 con 0x55ee37887200 2023-09-30T14:59:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f655d7fa700 1 -- 172.21.15.32:0/3740600683 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65700a1c80 con 0x55ee37887200 2023-09-30T14:59:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f6587e7eb80 10 client.26834 fetch_fsmap learned FSMap version 53 2023-09-30T14:59:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.889+0000 7f6587e7eb80 1 -- 172.21.15.32:0/3740600683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55ee37604d80 con 0x55ee37887200 2023-09-30T14:59:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.890+0000 7f655d7fa700 1 -- 172.21.15.32:0/3740600683 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ee3787a090 con 0x55ee37887200 2023-09-30T14:59:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.890+0000 7f655d7fa700 1 -- 172.21.15.32:0/3740600683 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 53) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f657009ec10 con 0x55ee37887200 2023-09-30T14:59:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.890+0000 7f6587e7eb80 10 client.26834 fetch_fsmap finished waiting for FSMap version 53 2023-09-30T14:59:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.890+0000 7f6587e7eb80 10 client.26834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:59:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.890+0000 7f6587e7eb80 20 client.26834 populate_metadata read hostname 'smithi032' 2023-09-30T14:59:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.890+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/3740600683 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ee3789c8c0 0x55ee3789ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.890+0000 7f6587e7eb80 4 client.26834 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:59:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.890+0000 7f6587e7eb80 1 -- 172.21.15.32:0/3740600683 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee377d2db0 con 0x55ee3789c8c0 2023-09-30T14:59:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.890+0000 7f6579231700 1 --2- 172.21.15.32:0/3740600683 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ee3789c8c0 0x55ee3789ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.890+0000 7f655d7fa700 10 client.26834 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:59:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.890+0000 7f6579231700 1 --2- 172.21.15.32:0/3740600683 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ee3789c8c0 0x55ee3789ece0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19582a0e7101ef57 server_cookie=cdb111e7115bc540 in_seq=0 out_seq=0 2023-09-30T14:59:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.892+0000 7f655d7fa700 1 -- 172.21.15.32:0/3740600683 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ee377d2db0 con 0x55ee3789c8c0 2023-09-30T14:59:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.892+0000 7f655d7fa700 10 client.26834 handle_command_reply: tid=0 2023-09-30T14:59:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.892+0000 7f6587e7eb80 1 client.26834 shutdown 2023-09-30T14:59:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.892+0000 7f6587e7eb80 1 -- 172.21.15.32:0/3740600683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37887200 msgr2=0x55ee37890fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.892+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/3740600683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37887200 0x55ee37890fa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f65700a1be0 tx=0x7f65700a2180 comp rx=0 tx=0).stop 2023-09-30T14:59:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.892+0000 7f6587e7eb80 1 -- 172.21.15.32:0/3740600683 shutdown_connections 2023-09-30T14:59:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.892+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/3740600683 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ee3789c8c0 0x55ee3789ece0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.892+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/3740600683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee37898520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.892+0000 7f655d7fa700 0 client.26834 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:59:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.892+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/3740600683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37887200 0x55ee37890fa0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.892+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/3740600683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888310 0x55ee378914e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.892+0000 7f6587e7eb80 1 -- 172.21.15.32:0/3740600683 >> 172.21.15.32:0/3740600683 conn(0x55ee377fb200 msgr2=0x55ee37881590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.892+0000 7f6587e7eb80 1 -- 172.21.15.32:0/3740600683 shutdown_connections 2023-09-30T14:59:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.892+0000 7f6587e7eb80 1 -- 172.21.15.32:0/3740600683 wait complete. 2023-09-30T14:59:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.893+0000 7f6587e7eb80 20 client.26834 trim_cache size 0 max 16384 2023-09-30T14:59:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f6587e7eb80 1 Processor -- start 2023-09-30T14:59:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f6587e7eb80 1 -- start start 2023-09-30T14:59:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f6587e7eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37887200 0x55ee378886f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f6587e7eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37888310 0x55ee378984e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f6587e7eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee3788f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f6587e7eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3765a8c0 con 0x55ee37888cd0 2023-09-30T14:59:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f6587e7eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee378836d0 con 0x55ee37888310 2023-09-30T14:59:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f6587e7eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3788efa0 con 0x55ee37887200 2023-09-30T14:59:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f657a233700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee3788f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f6579231700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37888310 0x55ee378984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f657a233700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee3788f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39844/0 (socket says 172.21.15.32:39844) 2023-09-30T14:59:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f657a233700 1 -- 172.21.15.32:0/2130172992 learned_addr learned my addr 172.21.15.32:0/2130172992 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f6579a32700 1 --2- 172.21.15.32:0/2130172992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37887200 0x55ee378886f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f657a233700 1 -- 172.21.15.32:0/2130172992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37887200 msgr2=0x55ee378886f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f657a233700 1 --2- 172.21.15.32:0/2130172992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37887200 0x55ee378886f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f657a233700 1 -- 172.21.15.32:0/2130172992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37888310 msgr2=0x55ee378984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f657a233700 1 --2- 172.21.15.32:0/2130172992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37888310 0x55ee378984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f657a233700 1 -- 172.21.15.32:0/2130172992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee375edc00 con 0x55ee37888cd0 2023-09-30T14:59:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.908+0000 7f6579231700 1 --2- 172.21.15.32:0/2130172992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37888310 0x55ee378984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T14:59:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.909+0000 7f657a233700 1 --2- 172.21.15.32:0/2130172992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee3788f170 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f656c015000 tx=0x7f656c007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906303ece7a3c928 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.909+0000 7f6568d82700 1 -- 172.21.15.32:0/2130172992 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c00e040 con 0x55ee37888cd0 2023-09-30T14:59:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.909+0000 7f6568d82700 1 -- 172.21.15.32:0/2130172992 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f656c0076e0 con 0x55ee37888cd0 2023-09-30T14:59:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.909+0000 7f6568d82700 1 -- 172.21.15.32:0/2130172992 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c0052a0 con 0x55ee37888cd0 2023-09-30T14:59:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 -- 172.21.15.32:0/2130172992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 msgr2=0x55ee3788f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/2130172992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee3788f170 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f656c015000 tx=0x7f656c007d20 comp rx=0 tx=0).stop 2023-09-30T14:59:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 -- 172.21.15.32:0/2130172992 shutdown_connections 2023-09-30T14:59:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/2130172992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee3788f170 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/2130172992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37887200 0x55ee378886f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/2130172992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37888310 0x55ee378984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 -- 172.21.15.32:0/2130172992 >> 172.21.15.32:0/2130172992 conn(0x55ee377fb200 msgr2=0x55ee3788fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 -- 172.21.15.32:0/2130172992 shutdown_connections 2023-09-30T14:59:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 -- 172.21.15.32:0/2130172992 wait complete. 2023-09-30T14:59:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 -- start start 2023-09-30T14:59:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 Processor -- start 2023-09-30T14:59:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37887200 0x55ee37896140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888310 0x55ee37896680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee37897690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee37897bd0 con 0x55ee37888cd0 2023-09-30T14:59:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee37897d40 con 0x55ee37887200 2023-09-30T14:59:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.910+0000 7f6587e7eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee37897f00 con 0x55ee37888310 2023-09-30T14:59:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f6587e7eb80 10 client.0 _ll_register_callbacks cb 0x7f65640096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:59:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f657a233700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee37897690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f6579a32700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37887200 0x55ee37896140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f6579231700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888310 0x55ee37896680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f657a233700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee37897690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39848/0 (socket says 172.21.15.32:39848) 2023-09-30T14:59:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f6579a32700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37887200 0x55ee37896140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51322/0 (socket says 172.21.15.32:51322) 2023-09-30T14:59:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f657a233700 1 -- 172.21.15.32:0/848824514 learned_addr learned my addr 172.21.15.32:0/848824514 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f657a233700 1 -- 172.21.15.32:0/848824514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888310 msgr2=0x55ee37896680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f657a233700 1 --2- 172.21.15.32:0/848824514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888310 0x55ee37896680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f657a233700 1 -- 172.21.15.32:0/848824514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37887200 msgr2=0x55ee37896140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f657a233700 1 --2- 172.21.15.32:0/848824514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37887200 0x55ee37896140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f657a233700 1 -- 172.21.15.32:0/848824514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6564017070 con 0x55ee37888cd0 2023-09-30T14:59:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f657a233700 1 --2- 172.21.15.32:0/848824514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee37897690 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f656c009fd0 tx=0x7f656c006080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f6452e6a97317b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f655d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:59:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f655d7fa700 1 -- 172.21.15.32:0/848824514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c00e040 con 0x55ee37888cd0 2023-09-30T14:59:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f6587e7eb80 1 -- 172.21.15.32:0/848824514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f657009ded0 con 0x55ee37888cd0 2023-09-30T14:59:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f655d7fa700 1 -- 172.21.15.32:0/848824514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f656c003ab0 con 0x55ee37888cd0 2023-09-30T14:59:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f655d7fa700 1 -- 172.21.15.32:0/848824514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f656c0054d0 con 0x55ee37888cd0 2023-09-30T14:59:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f6587e7eb80 10 client.17220 fetch_fsmap learned FSMap version 53 2023-09-30T14:59:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.911+0000 7f6587e7eb80 1 -- 172.21.15.32:0/848824514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7f6574109bc0 con 0x55ee37888cd0 2023-09-30T14:59:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.912+0000 7f655d7fa700 1 -- 172.21.15.32:0/848824514 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f657009ded0 con 0x55ee37888cd0 2023-09-30T14:59:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.912+0000 7f655d7fa700 1 -- 172.21.15.32:0/848824514 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 53) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f656c00b0d0 con 0x55ee37888cd0 2023-09-30T14:59:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.912+0000 7f6587e7eb80 10 client.17220 fetch_fsmap finished waiting for FSMap version 53 2023-09-30T14:59:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.912+0000 7f6587e7eb80 10 client.17220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:59:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.912+0000 7f6587e7eb80 20 client.17220 populate_metadata read hostname 'smithi032' 2023-09-30T14:59:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.912+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/848824514 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ee378f1250 0x55ee378f3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.912+0000 7f6587e7eb80 4 client.17220 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:59:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.912+0000 7f6587e7eb80 1 -- 172.21.15.32:0/848824514 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ee377d2ad0 con 0x55ee378f1250 2023-09-30T14:59:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.912+0000 7f6579a32700 1 --2- 172.21.15.32:0/848824514 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ee378f1250 0x55ee378f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.913+0000 7f655d7fa700 10 client.17220 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:59:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:38.913+0000 7f6579a32700 1 --2- 172.21.15.32:0/848824514 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ee378f1250 0x55ee378f3670 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a90981e0ba5ac59 server_cookie=7e42b764a3237bf4 in_seq=0 out_seq=0 2023-09-30T14:59:39.073 INFO:tasks.workunit.client.0.smithi032.stdout: 10 324825 35.70 MB/sec execute 372 sec latency 148.189 ms 2023-09-30T14:59:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.331+0000 7f655d7fa700 1 -- 172.21.15.32:0/848824514 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ee377d2ad0 con 0x55ee378f1250 2023-09-30T14:59:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.331+0000 7f655d7fa700 10 client.17220 handle_command_reply: tid=0 2023-09-30T14:59:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.331+0000 7f6587e7eb80 1 client.17220 shutdown 2023-09-30T14:59:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.332+0000 7f6587e7eb80 1 -- 172.21.15.32:0/848824514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 msgr2=0x55ee37897690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.332+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/848824514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee37897690 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f656c009fd0 tx=0x7f656c006080 comp rx=0 tx=0).stop 2023-09-30T14:59:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.335+0000 7f6587e7eb80 1 -- 172.21.15.32:0/848824514 shutdown_connections 2023-09-30T14:59:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.335+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/848824514 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ee378f1250 0x55ee378f3670 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.335+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/848824514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ee37888cd0 0x55ee37897690 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.335+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/848824514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee37887200 0x55ee37896140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.335+0000 7f6587e7eb80 1 --2- 172.21.15.32:0/848824514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ee37888310 0x55ee37896680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.335+0000 7f6587e7eb80 1 -- 172.21.15.32:0/848824514 >> 172.21.15.32:0/848824514 conn(0x55ee377fb200 msgr2=0x55ee3788f170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.335+0000 7f655d7fa700 0 client.17220 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:59:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.335+0000 7f6587e7eb80 1 -- 172.21.15.32:0/848824514 shutdown_connections 2023-09-30T14:59:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.336+0000 7f6587e7eb80 1 -- 172.21.15.32:0/848824514 wait complete. 2023-09-30T14:59:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.337+0000 7f6587e7eb80 20 client.17220 trim_cache size 0 max 16384 2023-09-30T14:59:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:59:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:59:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:59:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:59:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.337+0000 7f657bc96700 1 -- 172.21.15.32:0/3981523599 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f65600a0fa0 msgr2=0x7f65600a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.337+0000 7f657bc96700 1 --2- 172.21.15.32:0/3981523599 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f65600a0fa0 0x7f65600a3460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6574074510 tx=0x7f6564009260 comp rx=0 tx=0).stop 2023-09-30T14:59:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.337+0000 7f657bc96700 1 -- 172.21.15.32:0/3981523599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6574100e60 msgr2=0x7f6574073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.337+0000 7f657bc96700 1 --2- 172.21.15.32:0/3981523599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6574100e60 0x7f6574073210 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f657000e9a0 tx=0x7f657000ecb0 comp rx=0 tx=0).stop 2023-09-30T14:59:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.345+0000 7f657bc96700 1 -- 172.21.15.32:0/3981523599 shutdown_connections 2023-09-30T14:59:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.345+0000 7f657bc96700 1 --2- 172.21.15.32:0/3981523599 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f65600a0fa0 0x7f65600a3460 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.345+0000 7f657bc96700 1 --2- 172.21.15.32:0/3981523599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6574102970 0x7f6574073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.345+0000 7f657bc96700 1 --2- 172.21.15.32:0/3981523599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6574100e60 0x7f6574073210 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.345+0000 7f657bc96700 1 --2- 172.21.15.32:0/3981523599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6574101fb0 0x7f6574073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.345+0000 7f657bc96700 1 -- 172.21.15.32:0/3981523599 >> 172.21.15.32:0/3981523599 conn(0x7f65740fc3d0 msgr2=0x7f65740fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.346+0000 7f657bc96700 1 -- 172.21.15.32:0/3981523599 shutdown_connections 2023-09-30T14:59:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:39.346+0000 7f657bc96700 1 -- 172.21.15.32:0/3981523599 wait complete. 2023-09-30T14:59:39.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:39 smithi148 ceph-mon[102355]: pgmap v928: 97 pgs: 97 active+clean; 794 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.5 MiB/s wr, 673 op/s 2023-09-30T14:59:39.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:39 smithi032 ceph-mon[98592]: pgmap v928: 97 pgs: 97 active+clean; 794 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.5 MiB/s wr, 673 op/s 2023-09-30T14:59:40.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:39 smithi156 ceph-mon[103579]: pgmap v928: 97 pgs: 97 active+clean; 794 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.5 MiB/s wr, 673 op/s 2023-09-30T14:59:40.072 INFO:tasks.workunit.client.0.smithi032.stdout: 10 325442 35.69 MB/sec execute 373 sec latency 292.349 ms 2023-09-30T14:59:41.073 INFO:tasks.workunit.client.0.smithi032.stdout: 10 325994 35.68 MB/sec execute 374 sec latency 345.275 ms 2023-09-30T14:59:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:41 smithi148 ceph-mon[102355]: pgmap v929: 97 pgs: 97 active+clean; 794 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.6 MiB/s wr, 657 op/s 2023-09-30T14:59:41.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:41 smithi032 ceph-mon[98592]: pgmap v929: 97 pgs: 97 active+clean; 794 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.6 MiB/s wr, 657 op/s 2023-09-30T14:59:42.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:41 smithi156 ceph-mon[103579]: pgmap v929: 97 pgs: 97 active+clean; 794 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.6 MiB/s wr, 657 op/s 2023-09-30T14:59:42.073 INFO:tasks.workunit.client.0.smithi032.stdout: 10 326583 35.66 MB/sec execute 375 sec latency 70.340 ms 2023-09-30T14:59:43.073 INFO:tasks.workunit.client.0.smithi032.stdout: 10 327336 35.69 MB/sec execute 376 sec latency 55.010 ms 2023-09-30T14:59:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:43 smithi148 ceph-mon[102355]: pgmap v930: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.9 MiB/s wr, 759 op/s 2023-09-30T14:59:43.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:43 smithi032 ceph-mon[98592]: pgmap v930: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.9 MiB/s wr, 759 op/s 2023-09-30T14:59:44.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:43 smithi156 ceph-mon[103579]: pgmap v930: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.9 MiB/s wr, 759 op/s 2023-09-30T14:59:44.073 INFO:tasks.workunit.client.0.smithi032.stdout: 10 327969 35.69 MB/sec execute 377 sec latency 57.468 ms 2023-09-30T14:59:45.073 INFO:tasks.workunit.client.0.smithi032.stdout: 10 328631 35.70 MB/sec execute 378 sec latency 83.458 ms 2023-09-30T14:59:45.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:45 smithi032 ceph-mon[98592]: pgmap v931: 97 pgs: 97 active+clean; 795 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.0 MiB/s wr, 650 op/s 2023-09-30T14:59:46.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:45 smithi156 ceph-mon[103579]: pgmap v931: 97 pgs: 97 active+clean; 795 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.0 MiB/s wr, 650 op/s 2023-09-30T14:59:46.073 INFO:tasks.workunit.client.0.smithi032.stdout: 10 328992 35.68 MB/sec execute 379 sec latency 320.717 ms 2023-09-30T14:59:46.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:45 smithi148 ceph-mon[102355]: pgmap v931: 97 pgs: 97 active+clean; 795 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.0 MiB/s wr, 650 op/s 2023-09-30T14:59:47.073 INFO:tasks.workunit.client.0.smithi032.stdout: 10 329500 35.66 MB/sec execute 380 sec latency 100.412 ms 2023-09-30T14:59:47.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:47 smithi032 ceph-mon[98592]: pgmap v932: 97 pgs: 97 active+clean; 799 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.9 MiB/s wr, 784 op/s 2023-09-30T14:59:47.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:47 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:59:48.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:47 smithi156 ceph-mon[103579]: pgmap v932: 97 pgs: 97 active+clean; 799 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.9 MiB/s wr, 784 op/s 2023-09-30T14:59:48.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:47 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:59:48.073 INFO:tasks.workunit.client.0.smithi032.stdout: 10 330290 35.67 MB/sec execute 381 sec latency 78.683 ms 2023-09-30T14:59:48.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:47 smithi148 ceph-mon[102355]: pgmap v932: 97 pgs: 97 active+clean; 799 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.9 MiB/s wr, 784 op/s 2023-09-30T14:59:48.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:47 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T14:59:49.073 INFO:tasks.workunit.client.0.smithi032.stdout: 10 330974 35.69 MB/sec execute 382 sec latency 55.164 ms 2023-09-30T14:59:49.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:49 smithi032 ceph-mon[98592]: pgmap v933: 97 pgs: 97 active+clean; 797 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.7 MiB/s wr, 689 op/s 2023-09-30T14:59:50.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:49 smithi156 ceph-mon[103579]: pgmap v933: 97 pgs: 97 active+clean; 797 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.7 MiB/s wr, 689 op/s 2023-09-30T14:59:50.073 INFO:tasks.workunit.client.0.smithi032.stdout: 10 331634 35.69 MB/sec execute 383 sec latency 95.522 ms 2023-09-30T14:59:50.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:49 smithi148 ceph-mon[102355]: pgmap v933: 97 pgs: 97 active+clean; 797 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.7 MiB/s wr, 689 op/s 2023-09-30T14:59:51.073 INFO:tasks.workunit.client.0.smithi032.stdout: 10 332009 35.65 MB/sec execute 384 sec latency 322.823 ms 2023-09-30T14:59:51.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:50 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:51.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:50 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:51.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:50 smithi032 ceph-mon[98592]: pgmap v934: 97 pgs: 97 active+clean; 793 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.4 MiB/s wr, 662 op/s 2023-09-30T14:59:51.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:50 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:51.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:50 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:51.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:50 smithi156 ceph-mon[103579]: pgmap v934: 97 pgs: 97 active+clean; 793 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.4 MiB/s wr, 662 op/s 2023-09-30T14:59:51.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:50 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:51.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:50 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:51.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:50 smithi148 ceph-mon[102355]: pgmap v934: 97 pgs: 97 active+clean; 793 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.4 MiB/s wr, 662 op/s 2023-09-30T14:59:52.074 INFO:tasks.workunit.client.0.smithi032.stdout: 10 332450 35.63 MB/sec execute 385 sec latency 135.768 ms 2023-09-30T14:59:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.489+0000 7ff4d409b700 1 -- 172.21.15.32:0/704935694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4cc0ffd90 msgr2=0x7ff4cc1001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.489+0000 7ff4d409b700 1 --2- 172.21.15.32:0/704935694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4cc0ffd90 0x7ff4cc1001b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80099c0 tx=0x7ff4c8009cd0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.490+0000 7ff4d409b700 1 -- 172.21.15.32:0/704935694 shutdown_connections 2023-09-30T14:59:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.490+0000 7ff4d409b700 1 --2- 172.21.15.32:0/704935694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4cc101920 0x7ff4cc1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.490+0000 7ff4d409b700 1 --2- 172.21.15.32:0/704935694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4cc0ffd90 0x7ff4cc1001b0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.490+0000 7ff4d409b700 1 --2- 172.21.15.32:0/704935694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4cc100f80 0x7ff4cc1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.490+0000 7ff4d409b700 1 -- 172.21.15.32:0/704935694 >> 172.21.15.32:0/704935694 conn(0x7ff4cc0fb320 msgr2=0x7ff4cc0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.490+0000 7ff4d409b700 1 -- 172.21.15.32:0/704935694 shutdown_connections 2023-09-30T14:59:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.490+0000 7ff4d409b700 1 -- 172.21.15.32:0/704935694 wait complete. 2023-09-30T14:59:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.490+0000 7ff4d409b700 1 Processor -- start 2023-09-30T14:59:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.490+0000 7ff4d409b700 1 -- start start 2023-09-30T14:59:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d409b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4cc0ffd90 0x7ff4cc196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d409b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4cc100f80 0x7ff4cc197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d409b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4cc101920 0x7ff4cc197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d409b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc198670 con 0x7ff4cc100f80 2023-09-30T14:59:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d409b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc1987e0 con 0x7ff4cc101920 2023-09-30T14:59:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d409b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc1989a0 con 0x7ff4cc0ffd90 2023-09-30T14:59:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d1636700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4cc100f80 0x7ff4cc197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d1636700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4cc100f80 0x7ff4cc197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39860/0 (socket says 172.21.15.32:39860) 2023-09-30T14:59:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d1636700 1 -- 172.21.15.32:0/936397436 learned_addr learned my addr 172.21.15.32:0/936397436 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d1e37700 1 --2- 172.21.15.32:0/936397436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4cc0ffd90 0x7ff4cc196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d2638700 1 --2- 172.21.15.32:0/936397436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4cc101920 0x7ff4cc197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d1636700 1 -- 172.21.15.32:0/936397436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4cc0ffd90 msgr2=0x7ff4cc196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d1636700 1 --2- 172.21.15.32:0/936397436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4cc0ffd90 0x7ff4cc196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d1636700 1 -- 172.21.15.32:0/936397436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4cc101920 msgr2=0x7ff4cc197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d1636700 1 --2- 172.21.15.32:0/936397436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4cc101920 0x7ff4cc197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d1636700 1 -- 172.21.15.32:0/936397436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c80095f0 con 0x7ff4cc100f80 2023-09-30T14:59:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.491+0000 7ff4d1636700 1 --2- 172.21.15.32:0/936397436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4cc100f80 0x7ff4cc197230 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00e9a0 tx=0x7ff4bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.492+0000 7ff4c2ffd700 1 -- 172.21.15.32:0/936397436 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc009790 con 0x7ff4cc100f80 2023-09-30T14:59:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.492+0000 7ff4d409b700 1 -- 172.21.15.32:0/936397436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc198c80 con 0x7ff4cc100f80 2023-09-30T14:59:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.492+0000 7ff4c2ffd700 1 -- 172.21.15.32:0/936397436 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4bc010430 con 0x7ff4cc100f80 2023-09-30T14:59:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.492+0000 7ff4c2ffd700 1 -- 172.21.15.32:0/936397436 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc018680 con 0x7ff4cc100f80 2023-09-30T14:59:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.492+0000 7ff4d409b700 1 -- 172.21.15.32:0/936397436 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc102430 con 0x7ff4cc100f80 2023-09-30T14:59:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.495+0000 7ff4c2ffd700 1 -- 172.21.15.32:0/936397436 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7ff4bc0105a0 con 0x7ff4cc100f80 2023-09-30T14:59:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.495+0000 7ff4c2ffd700 1 --2- 172.21.15.32:0/936397436 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff4b80a0fa0 0x7ff4b80a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.496+0000 7ff4c2ffd700 1 -- 172.21.15.32:0/936397436 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff4bc014070 con 0x7ff4cc100f80 2023-09-30T14:59:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.496+0000 7ff4e0283b80 1 Processor -- start 2023-09-30T14:59:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.496+0000 7ff4d1e37700 1 --2- 172.21.15.32:0/936397436 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff4b80a0fa0 0x7ff4b80a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.496+0000 7ff4e0283b80 1 -- start start 2023-09-30T14:59:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.496+0000 7ff4e0283b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3ee200 0x55c70a3ee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.496+0000 7ff4e0283b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ef310 0x55c70a3ef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.496+0000 7ff4e0283b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3efcd0 0x55c70a3f6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.496+0000 7ff4e0283b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70a356400 con 0x55c70a3efcd0 2023-09-30T14:59:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.496+0000 7ff4e0283b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70a347bc0 con 0x55c70a3ee200 2023-09-30T14:59:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.496+0000 7ff4e0283b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70a3454e0 con 0x55c70a3ef310 2023-09-30T14:59:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.496+0000 7ff4d1e37700 1 --2- 172.21.15.32:0/936397436 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff4b80a0fa0 0x7ff4b80a3460 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009990 tx=0x7ff4c8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.497+0000 7ff4d1e37700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3ee200 0x55c70a3ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.497+0000 7ff4d2638700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3efcd0 0x55c70a3f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.497+0000 7ff4d2638700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3efcd0 0x55c70a3f6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39868/0 (socket says 172.21.15.32:39868) 2023-09-30T14:59:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.497+0000 7ff4d2638700 1 -- 172.21.15.32:0/2633651294 learned_addr learned my addr 172.21.15.32:0/2633651294 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.497+0000 7ff4d2638700 1 -- 172.21.15.32:0/2633651294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ef310 msgr2=0x55c70a3ef790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.497+0000 7ff4d2638700 1 --2- 172.21.15.32:0/2633651294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ef310 0x55c70a3ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.497+0000 7ff4d2638700 1 -- 172.21.15.32:0/2633651294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3ee200 msgr2=0x55c70a3ee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.497+0000 7ff4d2638700 1 --2- 172.21.15.32:0/2633651294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3ee200 0x55c70a3ee620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.497+0000 7ff4d2638700 1 -- 172.21.15.32:0/2633651294 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c70a3a3870 con 0x55c70a3efcd0 2023-09-30T14:59:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.497+0000 7ff4d2638700 1 --2- 172.21.15.32:0/2633651294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3efcd0 0x55c70a3f6560 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40115e0 tx=0x7ff4c4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313a0c5c6f1b9e08 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.497+0000 7ff4b7fff700 1 -- 172.21.15.32:0/2633651294 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400e040 con 0x55c70a3efcd0 2023-09-30T14:59:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.497+0000 7ff4b7fff700 1 -- 172.21.15.32:0/2633651294 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4c400f040 con 0x55c70a3efcd0 2023-09-30T14:59:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.497+0000 7ff4b7fff700 1 -- 172.21.15.32:0/2633651294 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4004710 con 0x55c70a3efcd0 2023-09-30T14:59:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.498+0000 7ff4e0283b80 1 -- 172.21.15.32:0/2633651294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3efcd0 msgr2=0x55c70a3f6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.498+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/2633651294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3efcd0 0x55c70a3f6560 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40115e0 tx=0x7ff4c4009920 comp rx=0 tx=0).stop 2023-09-30T14:59:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.499+0000 7ff4e0283b80 1 -- 172.21.15.32:0/2633651294 shutdown_connections 2023-09-30T14:59:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.499+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/2633651294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3efcd0 0x55c70a3f6560 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.499+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/2633651294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3ee200 0x55c70a3ee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.499+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/2633651294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ef310 0x55c70a3ef790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.499+0000 7ff4e0283b80 1 -- 172.21.15.32:0/2633651294 >> 172.21.15.32:0/2633651294 conn(0x55c70a362200 msgr2=0x55c70a3e8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.499+0000 7ff4e0283b80 1 -- 172.21.15.32:0/2633651294 shutdown_connections 2023-09-30T14:59:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.499+0000 7ff4e0283b80 1 -- 172.21.15.32:0/2633651294 wait complete. 2023-09-30T14:59:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.499+0000 7ff4e0283b80 1 -- start start 2023-09-30T14:59:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.499+0000 7ff4e0283b80 1 Processor -- start 2023-09-30T14:59:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4e0283b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ee200 0x55c70a3f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4e0283b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3ef310 0x55c70a3f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4e0283b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3efcd0 0x55c70a3ff420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4e0283b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70a340dd0 con 0x55c70a3ef310 2023-09-30T14:59:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4e0283b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70a2dd6b0 con 0x55c70a3efcd0 2023-09-30T14:59:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4e0283b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70a2dd350 con 0x55c70a3ee200 2023-09-30T14:59:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4e0283b80 10 client.0 _ll_register_callbacks cb 0x55c70a13a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:59:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4d1636700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3ef310 0x55c70a3f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4d1636700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3ef310 0x55c70a3f83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39874/0 (socket says 172.21.15.32:39874) 2023-09-30T14:59:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4d1636700 1 -- 172.21.15.32:0/2712269537 learned_addr learned my addr 172.21.15.32:0/2712269537 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4d1e37700 1 --2- 172.21.15.32:0/2712269537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ee200 0x55c70a3f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4d2638700 1 --2- 172.21.15.32:0/2712269537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3efcd0 0x55c70a3ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4d1636700 1 -- 172.21.15.32:0/2712269537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ee200 msgr2=0x55c70a3f7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4d1636700 1 --2- 172.21.15.32:0/2712269537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ee200 0x55c70a3f7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4d1636700 1 -- 172.21.15.32:0/2712269537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3efcd0 msgr2=0x55c70a3ff420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4d1636700 1 --2- 172.21.15.32:0/2712269537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3efcd0 0x55c70a3ff420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.500+0000 7ff4d1636700 1 -- 172.21.15.32:0/2712269537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c4011290 con 0x55c70a3ef310 2023-09-30T14:59:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.501+0000 7ff4d1636700 1 --2- 172.21.15.32:0/2712269537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3ef310 0x55c70a3f83e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc099d10 tx=0x7ff4bc09a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da45ff1de9cb890e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.501+0000 7ff4b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:59:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.501+0000 7ff4b57fa700 1 -- 172.21.15.32:0/2712269537 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc09ded0 con 0x55c70a3ef310 2023-09-30T14:59:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.501+0000 7ff4b57fa700 1 -- 172.21.15.32:0/2712269537 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff4bc0a2110 con 0x55c70a3ef310 2023-09-30T14:59:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.501+0000 7ff4e0283b80 1 -- 172.21.15.32:0/2712269537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c70a3e1090 con 0x55c70a3ef310 2023-09-30T14:59:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.501+0000 7ff4b57fa700 1 -- 172.21.15.32:0/2712269537 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4bc0a1c80 con 0x55c70a3ef310 2023-09-30T14:59:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.501+0000 7ff4e0283b80 10 client.17232 fetch_fsmap learned FSMap version 53 2023-09-30T14:59:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.501+0000 7ff4e0283b80 1 -- 172.21.15.32:0/2712269537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55c70a16bd80 con 0x55c70a3ef310 2023-09-30T14:59:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.501+0000 7ff4b57fa700 1 -- 172.21.15.32:0/2712269537 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4bc063020 con 0x55c70a3ef310 2023-09-30T14:59:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.501+0000 7ff4b57fa700 1 -- 172.21.15.32:0/2712269537 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 53) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff4bc098ce0 con 0x55c70a3ef310 2023-09-30T14:59:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.501+0000 7ff4e0283b80 10 client.17232 fetch_fsmap finished waiting for FSMap version 53 2023-09-30T14:59:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.501+0000 7ff4e0283b80 10 client.17232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:59:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.501+0000 7ff4e0283b80 20 client.17232 populate_metadata read hostname 'smithi032' 2023-09-30T14:59:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.501+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/2712269537 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c70a403cd0 0x55c70a4060f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.501+0000 7ff4e0283b80 4 client.17232 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T14:59:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.501+0000 7ff4e0283b80 1 -- 172.21.15.32:0/2712269537 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c70a339db0 con 0x55c70a403cd0 2023-09-30T14:59:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.504+0000 7ff4d1e37700 1 --2- 172.21.15.32:0/2712269537 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c70a403cd0 0x55c70a4060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.504+0000 7ff4d1e37700 1 --2- 172.21.15.32:0/2712269537 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c70a403cd0 0x55c70a4060f0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96a57c6a3a82dc5b server_cookie=642ab828d097a432 in_seq=0 out_seq=0 2023-09-30T14:59:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.504+0000 7ff4b57fa700 10 client.17232 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:59:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.506+0000 7ff4b57fa700 1 -- 172.21.15.32:0/2712269537 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c70a339db0 con 0x55c70a403cd0 2023-09-30T14:59:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.506+0000 7ff4b57fa700 10 client.17232 handle_command_reply: tid=0 2023-09-30T14:59:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.506+0000 7ff4e0283b80 1 client.17232 shutdown 2023-09-30T14:59:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.506+0000 7ff4e0283b80 1 -- 172.21.15.32:0/2712269537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3ef310 msgr2=0x55c70a3f83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.506+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/2712269537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3ef310 0x55c70a3f83e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc099d10 tx=0x7ff4bc09a2c0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.506+0000 7ff4e0283b80 1 -- 172.21.15.32:0/2712269537 shutdown_connections 2023-09-30T14:59:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.506+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/2712269537 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c70a403cd0 0x55c70a4060f0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.506+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/2712269537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3efcd0 0x55c70a3ff420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.506+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/2712269537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ee200 0x55c70a3f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.506+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/2712269537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3ef310 0x55c70a3f83e0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.506+0000 7ff4b57fa700 0 client.17232 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:59:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.506+0000 7ff4e0283b80 1 -- 172.21.15.32:0/2712269537 >> 172.21.15.32:0/2712269537 conn(0x55c70a362200 msgr2=0x55c70a3e8590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.506+0000 7ff4e0283b80 1 -- 172.21.15.32:0/2712269537 shutdown_connections 2023-09-30T14:59:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.506+0000 7ff4e0283b80 1 -- 172.21.15.32:0/2712269537 wait complete. 2023-09-30T14:59:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.506+0000 7ff4e0283b80 20 client.17232 trim_cache size 0 max 16384 2023-09-30T14:59:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4e0283b80 1 Processor -- start 2023-09-30T14:59:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4e0283b80 1 -- start start 2023-09-30T14:59:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4e0283b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ee200 0x55c70a3e8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4e0283b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3ef310 0x55c70a3ff3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4e0283b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3efcd0 0x55c70a3ffb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4e0283b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70a1c18c0 con 0x55c70a3ef310 2023-09-30T14:59:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4e0283b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70a3e0bd0 con 0x55c70a3efcd0 2023-09-30T14:59:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4e0283b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70a4037a0 con 0x55c70a3ee200 2023-09-30T14:59:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4d1636700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3ef310 0x55c70a3ff3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4d1636700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3ef310 0x55c70a3ff3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39882/0 (socket says 172.21.15.32:39882) 2023-09-30T14:59:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4d1636700 1 -- 172.21.15.32:0/1875715201 learned_addr learned my addr 172.21.15.32:0/1875715201 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4d1e37700 1 --2- 172.21.15.32:0/1875715201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ee200 0x55c70a3e8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4d2638700 1 --2- 172.21.15.32:0/1875715201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3efcd0 0x55c70a3ffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4d1636700 1 -- 172.21.15.32:0/1875715201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ee200 msgr2=0x55c70a3e8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4d1636700 1 --2- 172.21.15.32:0/1875715201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ee200 0x55c70a3e8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4d1636700 1 -- 172.21.15.32:0/1875715201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3efcd0 msgr2=0x55c70a3ffb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4d1636700 1 --2- 172.21.15.32:0/1875715201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3efcd0 0x55c70a3ffb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.526+0000 7ff4d1636700 1 -- 172.21.15.32:0/1875715201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c70a154c00 con 0x55c70a3ef310 2023-09-30T14:59:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.527+0000 7ff4d1636700 1 --2- 172.21.15.32:0/1875715201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3ef310 0x55c70a3ff3e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c2d0 tx=0x7ff4bc098330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e7f9689daff279 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.527+0000 7ff4c0ff9700 1 -- 172.21.15.32:0/1875715201 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0a2110 con 0x55c70a3ef310 2023-09-30T14:59:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.527+0000 7ff4c0ff9700 1 -- 172.21.15.32:0/1875715201 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4bc063070 con 0x55c70a3ef310 2023-09-30T14:59:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.527+0000 7ff4c0ff9700 1 -- 172.21.15.32:0/1875715201 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0a1c80 con 0x55c70a3ef310 2023-09-30T14:59:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.527+0000 7ff4e0283b80 1 -- 172.21.15.32:0/1875715201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3ef310 msgr2=0x55c70a3ff3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.527+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/1875715201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3ef310 0x55c70a3ff3e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c2d0 tx=0x7ff4bc098330 comp rx=0 tx=0).stop 2023-09-30T14:59:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.527+0000 7ff4e0283b80 1 -- 172.21.15.32:0/1875715201 shutdown_connections 2023-09-30T14:59:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.528+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/1875715201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3efcd0 0x55c70a3ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.528+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/1875715201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ee200 0x55c70a3e8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.528+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/1875715201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3ef310 0x55c70a3ff3e0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.528+0000 7ff4e0283b80 1 -- 172.21.15.32:0/1875715201 >> 172.21.15.32:0/1875715201 conn(0x55c70a362200 msgr2=0x55c70a3f6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.528+0000 7ff4e0283b80 1 -- 172.21.15.32:0/1875715201 shutdown_connections 2023-09-30T14:59:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.528+0000 7ff4e0283b80 1 -- 172.21.15.32:0/1875715201 wait complete. 2023-09-30T14:59:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.528+0000 7ff4e0283b80 1 -- start start 2023-09-30T14:59:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.528+0000 7ff4e0283b80 1 Processor -- start 2023-09-30T14:59:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4e0283b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ee200 0x55c70a3fcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4e0283b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3ef310 0x55c70a3fd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4e0283b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3efcd0 0x55c70a3fe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4e0283b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70a3fe990 con 0x55c70a3efcd0 2023-09-30T14:59:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4e0283b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70a3feb00 con 0x55c70a3ef310 2023-09-30T14:59:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4e0283b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70a3fecc0 con 0x55c70a3ee200 2023-09-30T14:59:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4e0283b80 10 client.0 _ll_register_callbacks cb 0x7ff4bc0998c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T14:59:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4d2638700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3efcd0 0x55c70a3fe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4d2638700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3efcd0 0x55c70a3fe450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39890/0 (socket says 172.21.15.32:39890) 2023-09-30T14:59:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4d1e37700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ee200 0x55c70a3fcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4d2638700 1 -- 172.21.15.32:0/3191306664 learned_addr learned my addr 172.21.15.32:0/3191306664 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T14:59:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4d1636700 1 --2- 172.21.15.32:0/3191306664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3ef310 0x55c70a3fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4d2638700 1 -- 172.21.15.32:0/3191306664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ee200 msgr2=0x55c70a3fcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4d2638700 1 --2- 172.21.15.32:0/3191306664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ee200 0x55c70a3fcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4d2638700 1 -- 172.21.15.32:0/3191306664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3ef310 msgr2=0x55c70a3fd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T14:59:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4d2638700 1 --2- 172.21.15.32:0/3191306664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3ef310 0x55c70a3fd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4d2638700 1 -- 172.21.15.32:0/3191306664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c70a16bd80 con 0x55c70a3efcd0 2023-09-30T14:59:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4d2638700 1 --2- 172.21.15.32:0/3191306664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3efcd0 0x55c70a3fe450 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40115b0 tx=0x7ff4c4004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d99d9b8e8ed8e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T14:59:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T14:59:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4b57fa700 1 -- 172.21.15.32:0/3191306664 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400e040 con 0x55c70a3efcd0 2023-09-30T14:59:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4b57fa700 1 -- 172.21.15.32:0/3191306664 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff4c4005590 con 0x55c70a3efcd0 2023-09-30T14:59:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.529+0000 7ff4e0283b80 1 -- 172.21.15.32:0/3191306664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4c400f040 con 0x55c70a3efcd0 2023-09-30T14:59:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.530+0000 7ff4b57fa700 1 -- 172.21.15.32:0/3191306664 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4c4005750 con 0x55c70a3efcd0 2023-09-30T14:59:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.530+0000 7ff4e0283b80 10 client.17238 fetch_fsmap learned FSMap version 53 2023-09-30T14:59:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.530+0000 7ff4e0283b80 1 -- 172.21.15.32:0/3191306664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55c70a1921f0 con 0x55c70a3efcd0 2023-09-30T14:59:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.530+0000 7ff4b57fa700 1 -- 172.21.15.32:0/3191306664 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4c400f040 con 0x55c70a3efcd0 2023-09-30T14:59:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.530+0000 7ff4b57fa700 1 -- 172.21.15.32:0/3191306664 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 53) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff4c4031070 con 0x55c70a3efcd0 2023-09-30T14:59:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.530+0000 7ff4e0283b80 10 client.17238 fetch_fsmap finished waiting for FSMap version 53 2023-09-30T14:59:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.530+0000 7ff4e0283b80 10 client.17238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T14:59:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.530+0000 7ff4e0283b80 20 client.17238 populate_metadata read hostname 'smithi032' 2023-09-30T14:59:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.530+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/3191306664 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c70a4587e0 0x55c70a45ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T14:59:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.530+0000 7ff4e0283b80 4 client.17238 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T14:59:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.530+0000 7ff4e0283b80 1 -- 172.21.15.32:0/3191306664 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c70a339ad0 con 0x55c70a4587e0 2023-09-30T14:59:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.530+0000 7ff4d1e37700 1 --2- 172.21.15.32:0/3191306664 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c70a4587e0 0x55c70a45ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T14:59:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.531+0000 7ff4b57fa700 10 client.17238 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T14:59:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.531+0000 7ff4d1e37700 1 --2- 172.21.15.32:0/3191306664 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c70a4587e0 0x55c70a45ac00 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0cf26621c8800e1 server_cookie=ee5fc6e2ea796d44 in_seq=0 out_seq=0 2023-09-30T14:59:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.926+0000 7ff4b57fa700 1 -- 172.21.15.32:0/3191306664 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c70a339ad0 con 0x55c70a4587e0 2023-09-30T14:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.926+0000 7ff4b57fa700 10 client.17238 handle_command_reply: tid=0 2023-09-30T14:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.926+0000 7ff4e0283b80 1 client.17238 shutdown 2023-09-30T14:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.927+0000 7ff4e0283b80 1 -- 172.21.15.32:0/3191306664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3efcd0 msgr2=0x55c70a3fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.927+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/3191306664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3efcd0 0x55c70a3fe450 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40115b0 tx=0x7ff4c4004200 comp rx=0 tx=0).stop 2023-09-30T14:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.927+0000 7ff4e0283b80 1 -- 172.21.15.32:0/3191306664 shutdown_connections 2023-09-30T14:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.927+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/3191306664 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c70a4587e0 0x55c70a45ac00 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.927+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/3191306664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c70a3efcd0 0x55c70a3fe450 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.927+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/3191306664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c70a3ee200 0x55c70a3fcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.927+0000 7ff4e0283b80 1 --2- 172.21.15.32:0/3191306664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c70a3ef310 0x55c70a3fd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.927+0000 7ff4e0283b80 1 -- 172.21.15.32:0/3191306664 >> 172.21.15.32:0/3191306664 conn(0x55c70a362200 msgr2=0x55c70a3e8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.927+0000 7ff4b57fa700 0 client.17238 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T14:59:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.927+0000 7ff4e0283b80 1 -- 172.21.15.32:0/3191306664 shutdown_connections 2023-09-30T14:59:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.927+0000 7ff4e0283b80 1 -- 172.21.15.32:0/3191306664 wait complete. 2023-09-30T14:59:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.927+0000 7ff4e0283b80 20 client.17238 trim_cache size 0 max 16384 2023-09-30T14:59:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T14:59:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T14:59:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T14:59:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T14:59:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.928+0000 7ff4d409b700 1 -- 172.21.15.32:0/936397436 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff4b80a0fa0 msgr2=0x7ff4b80a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.928+0000 7ff4d409b700 1 --2- 172.21.15.32:0/936397436 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff4b80a0fa0 0x7ff4b80a3460 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009990 tx=0x7ff4c8005fb0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.928+0000 7ff4d409b700 1 -- 172.21.15.32:0/936397436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4cc100f80 msgr2=0x7ff4cc197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T14:59:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.928+0000 7ff4d409b700 1 --2- 172.21.15.32:0/936397436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4cc100f80 0x7ff4cc197230 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00e9a0 tx=0x7ff4bc00ecb0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.928+0000 7ff4d409b700 1 -- 172.21.15.32:0/936397436 shutdown_connections 2023-09-30T14:59:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.928+0000 7ff4d409b700 1 --2- 172.21.15.32:0/936397436 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff4b80a0fa0 0x7ff4b80a3460 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.928+0000 7ff4d409b700 1 --2- 172.21.15.32:0/936397436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4cc101920 0x7ff4cc197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.928+0000 7ff4d409b700 1 --2- 172.21.15.32:0/936397436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4cc0ffd90 0x7ff4cc196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.928+0000 7ff4d409b700 1 --2- 172.21.15.32:0/936397436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4cc100f80 0x7ff4cc197230 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T14:59:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.928+0000 7ff4d409b700 1 -- 172.21.15.32:0/936397436 >> 172.21.15.32:0/936397436 conn(0x7ff4cc0fb320 msgr2=0x7ff4cc068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T14:59:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.928+0000 7ff4d409b700 1 -- 172.21.15.32:0/936397436 shutdown_connections 2023-09-30T14:59:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T14:59:52.928+0000 7ff4d409b700 1 -- 172.21.15.32:0/936397436 wait complete. 2023-09-30T14:59:53.074 INFO:tasks.workunit.client.0.smithi032.stdout: 10 332963 35.59 MB/sec execute 386 sec latency 241.776 ms 2023-09-30T14:59:54.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:53 smithi156 ceph-mon[103579]: pgmap v935: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 780 op/s 2023-09-30T14:59:54.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:53 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:54.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:53 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:54.074 INFO:tasks.workunit.client.0.smithi032.stdout: 10 333637 35.60 MB/sec execute 387 sec latency 49.679 ms 2023-09-30T14:59:54.113 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:53 smithi148 ceph-mon[102355]: pgmap v935: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 780 op/s 2023-09-30T14:59:54.114 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:53 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:54.114 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:53 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:54.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:53 smithi032 ceph-mon[98592]: pgmap v935: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 780 op/s 2023-09-30T14:59:54.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:53 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:54.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:53 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:55.074 INFO:tasks.workunit.client.0.smithi032.stdout: 10 334297 35.59 MB/sec execute 388 sec latency 89.547 ms 2023-09-30T14:59:56.074 INFO:tasks.workunit.client.0.smithi032.stdout: 10 334650 35.57 MB/sec execute 389 sec latency 218.474 ms 2023-09-30T14:59:56.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:55 smithi148 ceph-mon[102355]: pgmap v936: 97 pgs: 97 active+clean; 803 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.1 MiB/s wr, 586 op/s 2023-09-30T14:59:56.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:55 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:56.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:55 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:56.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:55 smithi032 ceph-mon[98592]: pgmap v936: 97 pgs: 97 active+clean; 803 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.1 MiB/s wr, 586 op/s 2023-09-30T14:59:56.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:55 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:56.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:55 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:56.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:55 smithi156 ceph-mon[103579]: pgmap v936: 97 pgs: 97 active+clean; 803 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.1 MiB/s wr, 586 op/s 2023-09-30T14:59:56.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:55 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:56.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:55 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:57.074 INFO:tasks.workunit.client.0.smithi032.stdout: 10 335165 35.52 MB/sec execute 390 sec latency 226.626 ms 2023-09-30T14:59:57.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:56 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:59:57.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:56 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:59:57.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:56 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:57.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:59:57.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:59:57.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:57.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T14:59:57.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T14:59:57.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T14:59:58.074 INFO:tasks.workunit.client.0.smithi032.stdout: 10 335870 35.55 MB/sec execute 391 sec latency 63.903 ms 2023-09-30T14:59:58.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:57 smithi148 ceph-mon[102355]: pgmap v937: 97 pgs: 97 active+clean; 786 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.3 MiB/s wr, 701 op/s 2023-09-30T14:59:58.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:57 smithi032 ceph-mon[98592]: pgmap v937: 97 pgs: 97 active+clean; 786 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.3 MiB/s wr, 701 op/s 2023-09-30T14:59:58.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:57 smithi156 ceph-mon[103579]: pgmap v937: 97 pgs: 97 active+clean; 786 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.3 MiB/s wr, 701 op/s 2023-09-30T14:59:59.074 INFO:tasks.workunit.client.0.smithi032.stdout: 10 336631 35.55 MB/sec execute 392 sec latency 73.995 ms 2023-09-30T15:00:00.074 INFO:tasks.workunit.client.0.smithi032.stdout: 10 337260 35.56 MB/sec execute 393 sec latency 53.101 ms 2023-09-30T15:00:00.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 14:59:59 smithi148 ceph-mon[102355]: pgmap v938: 97 pgs: 97 active+clean; 784 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 652 op/s 2023-09-30T15:00:00.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 14:59:59 smithi032 ceph-mon[98592]: pgmap v938: 97 pgs: 97 active+clean; 784 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 652 op/s 2023-09-30T15:00:00.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:00 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-a[98568]: 2023-09-30T14:59:59.998+0000 7f49afbf4700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2023-09-30T15:00:00.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 14:59:59 smithi156 ceph-mon[103579]: pgmap v938: 97 pgs: 97 active+clean; 784 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 652 op/s 2023-09-30T15:00:01.075 INFO:tasks.workunit.client.0.smithi032.stdout: 10 337762 35.53 MB/sec execute 394 sec latency 212.370 ms 2023-09-30T15:00:01.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:00 smithi148 ceph-mon[102355]: overall HEALTH_ERR 1 MDSs report damaged metadata 2023-09-30T15:00:01.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:00 smithi032 ceph-mon[98592]: overall HEALTH_ERR 1 MDSs report damaged metadata 2023-09-30T15:00:01.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:00 smithi156 ceph-mon[103579]: overall HEALTH_ERR 1 MDSs report damaged metadata 2023-09-30T15:00:02.075 INFO:tasks.workunit.client.0.smithi032.stdout: 10 338268 35.50 MB/sec execute 395 sec latency 247.634 ms 2023-09-30T15:00:02.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:01 smithi148 ceph-mon[102355]: pgmap v939: 97 pgs: 97 active+clean; 780 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.1 MiB/s wr, 609 op/s 2023-09-30T15:00:02.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:01 smithi032 ceph-mon[98592]: pgmap v939: 97 pgs: 97 active+clean; 780 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.1 MiB/s wr, 609 op/s 2023-09-30T15:00:02.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:01 smithi156 ceph-mon[103579]: pgmap v939: 97 pgs: 97 active+clean; 780 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.1 MiB/s wr, 609 op/s 2023-09-30T15:00:03.075 INFO:tasks.workunit.client.0.smithi032.stdout: 10 339115 35.52 MB/sec execute 396 sec latency 68.623 ms 2023-09-30T15:00:04.075 INFO:tasks.workunit.client.0.smithi032.stdout: 10 339934 35.56 MB/sec execute 397 sec latency 41.749 ms 2023-09-30T15:00:04.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:03 smithi148 ceph-mon[102355]: pgmap v940: 97 pgs: 97 active+clean; 784 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 10 MiB/s wr, 758 op/s 2023-09-30T15:00:04.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:03 smithi032 ceph-mon[98592]: pgmap v940: 97 pgs: 97 active+clean; 784 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 10 MiB/s wr, 758 op/s 2023-09-30T15:00:04.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:03 smithi156 ceph-mon[103579]: pgmap v940: 97 pgs: 97 active+clean; 784 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 10 MiB/s wr, 758 op/s 2023-09-30T15:00:05.075 INFO:tasks.workunit.client.0.smithi032.stdout: 10 340541 35.56 MB/sec execute 398 sec latency 46.154 ms 2023-09-30T15:00:06.080 INFO:tasks.workunit.client.0.smithi032.stdout: 10 341140 35.55 MB/sec execute 399 sec latency 101.473 ms 2023-09-30T15:00:06.113 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:05 smithi148 ceph-mon[102355]: pgmap v941: 97 pgs: 97 active+clean; 785 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.7 MiB/s wr, 633 op/s 2023-09-30T15:00:06.145 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:05 smithi032 ceph-mon[98592]: pgmap v941: 97 pgs: 97 active+clean; 785 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.7 MiB/s wr, 633 op/s 2023-09-30T15:00:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.188+0000 7f693afdf700 1 -- 172.21.15.32:0/1453265973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f693410b860 msgr2=0x7f693410dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.188+0000 7f693afdf700 1 --2- 172.21.15.32:0/1453265973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f693410b860 0x7f693410dc50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f692c00b0a0 tx=0x7f692c00b3b0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.188+0000 7f693afdf700 1 -- 172.21.15.32:0/1453265973 shutdown_connections 2023-09-30T15:00:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.188+0000 7f693afdf700 1 --2- 172.21.15.32:0/1453265973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f693410e270 0x7f6934110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.188+0000 7f693afdf700 1 --2- 172.21.15.32:0/1453265973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6934108f30 0x7f693410b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.188+0000 7f693afdf700 1 --2- 172.21.15.32:0/1453265973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f693410b860 0x7f693410dc50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.188+0000 7f693afdf700 1 -- 172.21.15.32:0/1453265973 >> 172.21.15.32:0/1453265973 conn(0x7f693406dc80 msgr2=0x7f69340700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.188+0000 7f693afdf700 1 -- 172.21.15.32:0/1453265973 shutdown_connections 2023-09-30T15:00:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.188+0000 7f693afdf700 1 -- 172.21.15.32:0/1453265973 wait complete. 2023-09-30T15:00:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.189+0000 7f693afdf700 1 Processor -- start 2023-09-30T15:00:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.189+0000 7f693afdf700 1 -- start start 2023-09-30T15:00:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.190+0000 7f693afdf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6934108f30 0x7f693419d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.190+0000 7f693afdf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f693410b860 0x7f693419d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.190+0000 7f693afdf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f693410e270 0x7f693419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.190+0000 7f693afdf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693419edc0 con 0x7f6934108f30 2023-09-30T15:00:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.190+0000 7f693afdf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693419ef30 con 0x7f693410e270 2023-09-30T15:00:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.190+0000 7f693afdf700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693419f0f0 con 0x7f693410b860 2023-09-30T15:00:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.193+0000 7f693957c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f693410e270 0x7f693419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.193+0000 7f693957c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f693410e270 0x7f693419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51380/0 (socket says 172.21.15.32:51380) 2023-09-30T15:00:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.193+0000 7f693957c700 1 -- 172.21.15.32:0/1067143677 learned_addr learned my addr 172.21.15.32:0/1067143677 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.194+0000 7f6933fff700 1 --2- 172.21.15.32:0/1067143677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f693410b860 0x7f693419d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.195+0000 7f6938d7b700 1 --2- 172.21.15.32:0/1067143677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6934108f30 0x7f693419d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.195+0000 7f6933fff700 1 -- 172.21.15.32:0/1067143677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f693410e270 msgr2=0x7f693419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.195+0000 7f6933fff700 1 --2- 172.21.15.32:0/1067143677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f693410e270 0x7f693419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.195+0000 7f6933fff700 1 -- 172.21.15.32:0/1067143677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6934108f30 msgr2=0x7f693419d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.195+0000 7f6933fff700 1 --2- 172.21.15.32:0/1067143677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6934108f30 0x7f693419d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.195+0000 7f6933fff700 1 -- 172.21.15.32:0/1067143677 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c009d00 con 0x7f693410b860 2023-09-30T15:00:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.197+0000 7f6933fff700 1 --2- 172.21.15.32:0/1067143677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f693410b860 0x7f693419d890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f692c00b070 tx=0x7f692c00bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.202+0000 7f6931ffb700 1 -- 172.21.15.32:0/1067143677 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c00e040 con 0x7f693410b860 2023-09-30T15:00:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.202+0000 7f6931ffb700 1 -- 172.21.15.32:0/1067143677 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f692c003e10 con 0x7f693410b860 2023-09-30T15:00:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.202+0000 7f6931ffb700 1 -- 172.21.15.32:0/1067143677 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c01b7e0 con 0x7f693410b860 2023-09-30T15:00:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.203+0000 7f693afdf700 1 -- 172.21.15.32:0/1067143677 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693419f370 con 0x7f693410b860 2023-09-30T15:00:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.203+0000 7f693afdf700 1 -- 172.21.15.32:0/1067143677 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69341a3ba0 con 0x7f693410b860 2023-09-30T15:00:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.208+0000 7f69471c7b80 1 Processor -- start 2023-09-30T15:00:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.208+0000 7f69471c7b80 1 -- start start 2023-09-30T15:00:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.208+0000 7f69471c7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62307200 0x55ab62307620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.208+0000 7f69471c7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62308310 0x55ab62308790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.208+0000 7f69471c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62308cd0 0x55ab6230f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.208+0000 7f69471c7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab6226f400 con 0x55ab62308310 2023-09-30T15:00:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.208+0000 7f69471c7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab62260bc0 con 0x55ab62307200 2023-09-30T15:00:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.208+0000 7f69471c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab6225e4e0 con 0x55ab62308cd0 2023-09-30T15:00:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.212+0000 7f6938d7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62308310 0x55ab62308790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.212+0000 7f6938d7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62308310 0x55ab62308790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39906/0 (socket says 172.21.15.32:39906) 2023-09-30T15:00:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.212+0000 7f6938d7b700 1 -- 172.21.15.32:0/139795125 learned_addr learned my addr 172.21.15.32:0/139795125 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.212+0000 7f6931ffb700 1 -- 172.21.15.32:0/1067143677 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f692c01da30 con 0x7f693410b860 2023-09-30T15:00:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.213+0000 7f6931ffb700 1 --2- 172.21.15.32:0/1067143677 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f691c0a1070 0x7f691c0a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.214+0000 7f6931ffb700 1 -- 172.21.15.32:0/1067143677 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f692c0cd140 con 0x7f693410b860 2023-09-30T15:00:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.214+0000 7f693957c700 1 --2- 172.21.15.32:0/139795125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62307200 0x55ab62307620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.215+0000 7f693957c700 1 -- 172.21.15.32:0/139795125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62308cd0 msgr2=0x55ab6230f560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-30T15:00:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.215+0000 7f693957c700 1 --2- 172.21.15.32:0/139795125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62308cd0 0x55ab6230f560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.215+0000 7f693957c700 1 -- 172.21.15.32:0/139795125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62308310 msgr2=0x55ab62308790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.215+0000 7f693957c700 1 --2- 172.21.15.32:0/139795125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62308310 0x55ab62308790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.215+0000 7f693957c700 1 -- 172.21.15.32:0/139795125 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab622bc870 con 0x55ab62307200 2023-09-30T15:00:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.217+0000 7f693957c700 1 --2- 172.21.15.32:0/139795125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62307200 0x55ab62307620 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6924005950 tx=0x7f69240126d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2d4b5c86a849806 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.219+0000 7f693957c700 1 --2- 172.21.15.32:0/1067143677 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f691c0a1070 0x7f691c0a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.221+0000 7f693957c700 1 --2- 172.21.15.32:0/1067143677 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f691c0a1070 0x7f691c0a3530 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f692401c000 tx=0x7f6924017000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.223+0000 7f691affd700 1 -- 172.21.15.32:0/139795125 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924012e50 con 0x55ab62307200 2023-09-30T15:00:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.223+0000 7f691affd700 1 -- 172.21.15.32:0/139795125 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f692400f880 con 0x55ab62307200 2023-09-30T15:00:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.223+0000 7f691affd700 1 -- 172.21.15.32:0/139795125 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692401b840 con 0x55ab62307200 2023-09-30T15:00:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.224+0000 7f69471c7b80 1 -- 172.21.15.32:0/139795125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62307200 msgr2=0x55ab62307620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.224+0000 7f69471c7b80 1 --2- 172.21.15.32:0/139795125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62307200 0x55ab62307620 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6924005950 tx=0x7f69240126d0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.224+0000 7f69471c7b80 1 -- 172.21.15.32:0/139795125 shutdown_connections 2023-09-30T15:00:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.224+0000 7f69471c7b80 1 --2- 172.21.15.32:0/139795125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62308cd0 0x55ab6230f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.224+0000 7f69471c7b80 1 --2- 172.21.15.32:0/139795125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62307200 0x55ab62307620 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.224+0000 7f69471c7b80 1 --2- 172.21.15.32:0/139795125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62308310 0x55ab62308790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.224+0000 7f69471c7b80 1 -- 172.21.15.32:0/139795125 >> 172.21.15.32:0/139795125 conn(0x55ab6227b200 msgr2=0x55ab62301ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.231+0000 7f69471c7b80 1 -- 172.21.15.32:0/139795125 shutdown_connections 2023-09-30T15:00:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.231+0000 7f69471c7b80 1 -- 172.21.15.32:0/139795125 wait complete. 2023-09-30T15:00:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.231+0000 7f69471c7b80 1 -- start start 2023-09-30T15:00:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.231+0000 7f69471c7b80 1 Processor -- start 2023-09-30T15:00:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.232+0000 7f69471c7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62307200 0x55ab62310fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.232+0000 7f69471c7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62308310 0x55ab623114e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.232+0000 7f69471c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62308cd0 0x55ab62318520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.232+0000 7f69471c7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab62259dd0 con 0x55ab62307200 2023-09-30T15:00:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.232+0000 7f69471c7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab621f66b0 con 0x55ab62308310 2023-09-30T15:00:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.232+0000 7f69471c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab621f6350 con 0x55ab62308cd0 2023-09-30T15:00:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.232+0000 7f69471c7b80 10 client.0 _ll_register_callbacks cb 0x55ab620537a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:00:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.237+0000 7f693957c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62308cd0 0x55ab62318520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.237+0000 7f6933fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62308310 0x55ab623114e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.237+0000 7f693957c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62308cd0 0x55ab62318520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54298/0 (socket says 172.21.15.32:54298) 2023-09-30T15:00:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.237+0000 7f693957c700 1 -- 172.21.15.32:0/1383088472 learned_addr learned my addr 172.21.15.32:0/1383088472 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.240+0000 7f6938d7b700 1 --2- 172.21.15.32:0/1383088472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62307200 0x55ab62310fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.240+0000 7f6938d7b700 1 -- 172.21.15.32:0/1383088472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62308cd0 msgr2=0x55ab62318520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.240+0000 7f6938d7b700 1 --2- 172.21.15.32:0/1383088472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62308cd0 0x55ab62318520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.240+0000 7f6938d7b700 1 -- 172.21.15.32:0/1383088472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62308310 msgr2=0x55ab623114e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.240+0000 7f6938d7b700 1 --2- 172.21.15.32:0/1383088472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62308310 0x55ab623114e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.240+0000 7f6938d7b700 1 -- 172.21.15.32:0/1383088472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692400ba70 con 0x55ab62307200 2023-09-30T15:00:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.241+0000 7f6938d7b700 1 --2- 172.21.15.32:0/1383088472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62307200 0x55ab62310fa0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f692800b3c0 tx=0x7f692800a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777a8dc7bffc169d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.244+0000 7f6913fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T15:00:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.244+0000 7f6913fff700 1 -- 172.21.15.32:0/1383088472 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928010040 con 0x55ab62307200 2023-09-30T15:00:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.244+0000 7f6913fff700 1 -- 172.21.15.32:0/1383088472 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6928017860 con 0x55ab62307200 2023-09-30T15:00:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.244+0000 7f69471c7b80 1 -- 172.21.15.32:0/1383088472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab622fa090 con 0x55ab62307200 2023-09-30T15:00:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.244+0000 7f6913fff700 1 -- 172.21.15.32:0/1383088472 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6928017a20 con 0x55ab62307200 2023-09-30T15:00:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.244+0000 7f69471c7b80 10 client.17250 fetch_fsmap learned FSMap version 53 2023-09-30T15:00:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.244+0000 7f69471c7b80 1 -- 172.21.15.32:0/1383088472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55ab62084d80 con 0x55ab62307200 2023-09-30T15:00:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.245+0000 7f6913fff700 1 -- 172.21.15.32:0/1383088472 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab622fa090 con 0x55ab62307200 2023-09-30T15:00:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.245+0000 7f6913fff700 1 -- 172.21.15.32:0/1383088472 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 53) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6928016cf0 con 0x55ab62307200 2023-09-30T15:00:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.245+0000 7f69471c7b80 10 client.17250 fetch_fsmap finished waiting for FSMap version 53 2023-09-30T15:00:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.245+0000 7f69471c7b80 10 client.17250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:00:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.245+0000 7f69471c7b80 20 client.17250 populate_metadata read hostname 'smithi032' 2023-09-30T15:00:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.245+0000 7f69471c7b80 1 --2- 172.21.15.32:0/1383088472 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ab6233edc0 0x55ab623411e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.245+0000 7f69471c7b80 4 client.17250 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T15:00:06.247 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.245+0000 7f69471c7b80 1 -- 172.21.15.32:0/1383088472 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab62252db0 con 0x55ab6233edc0 2023-09-30T15:00:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.247+0000 7f6933fff700 1 --2- 172.21.15.32:0/1383088472 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ab6233edc0 0x55ab623411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.250+0000 7f6913fff700 10 client.17250 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:00:06.252 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.251+0000 7f6933fff700 1 --2- 172.21.15.32:0/1383088472 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ab6233edc0 0x55ab623411e0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a7a804af710258d server_cookie=3d253515b6ec5c61 in_seq=0 out_seq=0 2023-09-30T15:00:06.253 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.252+0000 7f6913fff700 1 -- 172.21.15.32:0/1383088472 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ab62252db0 con 0x55ab6233edc0 2023-09-30T15:00:06.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.253+0000 7f6913fff700 10 client.17250 handle_command_reply: tid=0 2023-09-30T15:00:06.254 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.253+0000 7f69471c7b80 1 client.17250 shutdown 2023-09-30T15:00:06.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.254+0000 7f69471c7b80 1 -- 172.21.15.32:0/1383088472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62307200 msgr2=0x55ab62310fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:06.255 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.254+0000 7f69471c7b80 1 --2- 172.21.15.32:0/1383088472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62307200 0x55ab62310fa0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f692800b3c0 tx=0x7f692800a8d0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.254+0000 7f69471c7b80 1 -- 172.21.15.32:0/1383088472 shutdown_connections 2023-09-30T15:00:06.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.254+0000 7f69471c7b80 1 --2- 172.21.15.32:0/1383088472 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ab6233edc0 0x55ab623411e0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.254+0000 7f69471c7b80 1 --2- 172.21.15.32:0/1383088472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62308cd0 0x55ab62318520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.254+0000 7f69471c7b80 1 --2- 172.21.15.32:0/1383088472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62307200 0x55ab62310fa0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.254+0000 7f69471c7b80 1 --2- 172.21.15.32:0/1383088472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62308310 0x55ab623114e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.254+0000 7f69471c7b80 1 -- 172.21.15.32:0/1383088472 >> 172.21.15.32:0/1383088472 conn(0x55ab6227b200 msgr2=0x55ab62301590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:06.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.254+0000 7f6913fff700 0 client.17250 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:00:06.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.254+0000 7f69471c7b80 1 -- 172.21.15.32:0/1383088472 shutdown_connections 2023-09-30T15:00:06.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.254+0000 7f69471c7b80 1 -- 172.21.15.32:0/1383088472 wait complete. 2023-09-30T15:00:06.256 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.254+0000 7f69471c7b80 20 client.17250 trim_cache size 0 max 16384 2023-09-30T15:00:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f69471c7b80 1 Processor -- start 2023-09-30T15:00:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f69471c7b80 1 -- start start 2023-09-30T15:00:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f69471c7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62307200 0x55ab62318cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f69471c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62308310 0x55ab623184e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f69471c7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62308cd0 0x55ab6230f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f69471c7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab620da8c0 con 0x55ab62307200 2023-09-30T15:00:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f69471c7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab62307a80 con 0x55ab62308cd0 2023-09-30T15:00:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f69471c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab62310290 con 0x55ab62308310 2023-09-30T15:00:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f6938d7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62307200 0x55ab62318cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f6938d7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62307200 0x55ab62318cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39922/0 (socket says 172.21.15.32:39922) 2023-09-30T15:00:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f6938d7b700 1 -- 172.21.15.32:0/2096296789 learned_addr learned my addr 172.21.15.32:0/2096296789 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f6933fff700 1 --2- 172.21.15.32:0/2096296789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62308310 0x55ab623184e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f693957c700 1 --2- 172.21.15.32:0/2096296789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62308cd0 0x55ab6230f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f6938d7b700 1 -- 172.21.15.32:0/2096296789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62308310 msgr2=0x55ab623184e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f6938d7b700 1 --2- 172.21.15.32:0/2096296789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62308310 0x55ab623184e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f6938d7b700 1 -- 172.21.15.32:0/2096296789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62308cd0 msgr2=0x55ab6230f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f6938d7b700 1 --2- 172.21.15.32:0/2096296789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62308cd0 0x55ab6230f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.275+0000 7f6938d7b700 1 -- 172.21.15.32:0/2096296789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab6206dc00 con 0x55ab62307200 2023-09-30T15:00:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.276+0000 7f6938d7b700 1 --2- 172.21.15.32:0/2096296789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62307200 0x55ab62318cf0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f692801f470 tx=0x7f692800af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d56f42c5a28207 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.276+0000 7f691b7fe700 1 -- 172.21.15.32:0/2096296789 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928010070 con 0x55ab62307200 2023-09-30T15:00:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.276+0000 7f691b7fe700 1 -- 172.21.15.32:0/2096296789 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6928011040 con 0x55ab62307200 2023-09-30T15:00:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.276+0000 7f691b7fe700 1 -- 172.21.15.32:0/2096296789 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69280038e0 con 0x55ab62307200 2023-09-30T15:00:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.276+0000 7f69471c7b80 1 -- 172.21.15.32:0/2096296789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62307200 msgr2=0x55ab62318cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.276+0000 7f69471c7b80 1 --2- 172.21.15.32:0/2096296789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62307200 0x55ab62318cf0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f692801f470 tx=0x7f692800af50 comp rx=0 tx=0).stop 2023-09-30T15:00:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.276+0000 7f69471c7b80 1 -- 172.21.15.32:0/2096296789 shutdown_connections 2023-09-30T15:00:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.276+0000 7f69471c7b80 1 --2- 172.21.15.32:0/2096296789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62308cd0 0x55ab6230f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.277+0000 7f69471c7b80 1 --2- 172.21.15.32:0/2096296789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62307200 0x55ab62318cf0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.277+0000 7f69471c7b80 1 --2- 172.21.15.32:0/2096296789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62308310 0x55ab623184e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.277+0000 7f69471c7b80 1 -- 172.21.15.32:0/2096296789 >> 172.21.15.32:0/2096296789 conn(0x55ab6227b200 msgr2=0x55ab62340dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.277+0000 7f69471c7b80 1 -- 172.21.15.32:0/2096296789 shutdown_connections 2023-09-30T15:00:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.277+0000 7f69471c7b80 1 -- 172.21.15.32:0/2096296789 wait complete. 2023-09-30T15:00:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.277+0000 7f69471c7b80 1 -- start start 2023-09-30T15:00:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.277+0000 7f69471c7b80 1 Processor -- start 2023-09-30T15:00:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.277+0000 7f69471c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62307200 0x55ab62316250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.277+0000 7f69471c7b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62308310 0x55ab62316790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.277+0000 7f69471c7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62308cd0 0x55ab623177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.277+0000 7f69471c7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab62317ce0 con 0x55ab62308cd0 2023-09-30T15:00:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.277+0000 7f69471c7b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab62317e50 con 0x55ab62308310 2023-09-30T15:00:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.277+0000 7f69471c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab62318010 con 0x55ab62307200 2023-09-30T15:00:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.277+0000 7f69471c7b80 10 client.0 _ll_register_callbacks cb 0x7f692800be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:00:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f6938d7b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62307200 0x55ab62316250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f693957c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62308cd0 0x55ab623177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f6938d7b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62307200 0x55ab62316250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54310/0 (socket says 172.21.15.32:54310) 2023-09-30T15:00:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f6933fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62308310 0x55ab62316790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f6938d7b700 1 -- 172.21.15.32:0/290182291 learned_addr learned my addr 172.21.15.32:0/290182291 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f693957c700 1 -- 172.21.15.32:0/290182291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62307200 msgr2=0x55ab62316250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f693957c700 1 --2- 172.21.15.32:0/290182291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62307200 0x55ab62316250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f693957c700 1 -- 172.21.15.32:0/290182291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62308310 msgr2=0x55ab62316790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f693957c700 1 --2- 172.21.15.32:0/290182291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62308310 0x55ab62316790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f693957c700 1 -- 172.21.15.32:0/290182291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab62084d80 con 0x55ab62308cd0 2023-09-30T15:00:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f6933fff700 1 --2- 172.21.15.32:0/290182291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62308310 0x55ab62316790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:00:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f693957c700 1 --2- 172.21.15.32:0/290182291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62308cd0 0x55ab623177a0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f6924005950 tx=0x7f692401cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24c4e75163750e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f6913fff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T15:00:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f6913fff700 1 -- 172.21.15.32:0/290182291 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692400f3c0 con 0x55ab62308cd0 2023-09-30T15:00:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f6913fff700 1 -- 172.21.15.32:0/290182291 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f692400f520 con 0x55ab62308cd0 2023-09-30T15:00:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f69471c7b80 1 -- 172.21.15.32:0/290182291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f692400f880 con 0x55ab62308cd0 2023-09-30T15:00:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f6913fff700 1 -- 172.21.15.32:0/290182291 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f692400f6e0 con 0x55ab62308cd0 2023-09-30T15:00:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f69471c7b80 10 client.17256 fetch_fsmap learned FSMap version 53 2023-09-30T15:00:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.278+0000 7f69471c7b80 1 -- 172.21.15.32:0/290182291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55ab620ab1f0 con 0x55ab62308cd0 2023-09-30T15:00:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.279+0000 7f6913fff700 1 -- 172.21.15.32:0/290182291 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f692400f880 con 0x55ab62308cd0 2023-09-30T15:00:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.279+0000 7f6913fff700 1 -- 172.21.15.32:0/290182291 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 53) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6924004a20 con 0x55ab62308cd0 2023-09-30T15:00:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.279+0000 7f69471c7b80 10 client.17256 fetch_fsmap finished waiting for FSMap version 53 2023-09-30T15:00:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.279+0000 7f69471c7b80 10 client.17256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:00:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.279+0000 7f69471c7b80 20 client.17256 populate_metadata read hostname 'smithi032' 2023-09-30T15:00:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.279+0000 7f69471c7b80 1 --2- 172.21.15.32:0/290182291 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ab62393c20 0x55ab62396040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.279+0000 7f69471c7b80 4 client.17256 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T15:00:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.279+0000 7f69471c7b80 1 -- 172.21.15.32:0/290182291 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ab62252ad0 con 0x55ab62393c20 2023-09-30T15:00:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.279+0000 7f6938d7b700 1 --2- 172.21.15.32:0/290182291 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ab62393c20 0x55ab62396040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.279+0000 7f6913fff700 10 client.17256 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:00:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.279+0000 7f6938d7b700 1 --2- 172.21.15.32:0/290182291 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ab62393c20 0x55ab62396040 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=816608bd9df9cf8d server_cookie=55da1a116c63dc38 in_seq=0 out_seq=0 2023-09-30T15:00:06.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:05 smithi156 ceph-mon[103579]: pgmap v941: 97 pgs: 97 active+clean; 785 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.7 MiB/s wr, 633 op/s 2023-09-30T15:00:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.745+0000 7f6913fff700 1 -- 172.21.15.32:0/290182291 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ab62252ad0 con 0x55ab62393c20 2023-09-30T15:00:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.745+0000 7f6913fff700 10 client.17256 handle_command_reply: tid=0 2023-09-30T15:00:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.745+0000 7f69471c7b80 1 client.17256 shutdown 2023-09-30T15:00:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.746+0000 7f69471c7b80 1 -- 172.21.15.32:0/290182291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62308cd0 msgr2=0x55ab623177a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.746+0000 7f69471c7b80 1 --2- 172.21.15.32:0/290182291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62308cd0 0x55ab623177a0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f6924005950 tx=0x7f692401cf20 comp rx=0 tx=0).stop 2023-09-30T15:00:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.746+0000 7f69471c7b80 1 -- 172.21.15.32:0/290182291 shutdown_connections 2023-09-30T15:00:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.746+0000 7f69471c7b80 1 --2- 172.21.15.32:0/290182291 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55ab62393c20 0x55ab62396040 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.746+0000 7f69471c7b80 1 --2- 172.21.15.32:0/290182291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ab62308cd0 0x55ab623177a0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.746+0000 7f69471c7b80 1 --2- 172.21.15.32:0/290182291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ab62307200 0x55ab62316250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.746+0000 7f69471c7b80 1 --2- 172.21.15.32:0/290182291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab62308310 0x55ab62316790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.746+0000 7f69471c7b80 1 -- 172.21.15.32:0/290182291 >> 172.21.15.32:0/290182291 conn(0x55ab6227b200 msgr2=0x55ab6230f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.746+0000 7f6913fff700 0 client.17256 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:00:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.746+0000 7f69471c7b80 1 -- 172.21.15.32:0/290182291 shutdown_connections 2023-09-30T15:00:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.746+0000 7f69471c7b80 1 -- 172.21.15.32:0/290182291 wait complete. 2023-09-30T15:00:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.746+0000 7f69471c7b80 20 client.17256 trim_cache size 0 max 16384 2023-09-30T15:00:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T15:00:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T15:00:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T15:00:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T15:00:06.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.747+0000 7f693afdf700 1 -- 172.21.15.32:0/1067143677 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f691c0a1070 msgr2=0x7f691c0a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:06.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.747+0000 7f693afdf700 1 --2- 172.21.15.32:0/1067143677 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f691c0a1070 0x7f691c0a3530 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f692401c000 tx=0x7f6924017000 comp rx=0 tx=0).stop 2023-09-30T15:00:06.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.747+0000 7f693afdf700 1 -- 172.21.15.32:0/1067143677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f693410b860 msgr2=0x7f693419d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:06.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.747+0000 7f693afdf700 1 --2- 172.21.15.32:0/1067143677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f693410b860 0x7f693419d890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f692c00b070 tx=0x7f692c00bce0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.747+0000 7f693afdf700 1 -- 172.21.15.32:0/1067143677 shutdown_connections 2023-09-30T15:00:06.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.747+0000 7f693afdf700 1 --2- 172.21.15.32:0/1067143677 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f691c0a1070 0x7f691c0a3530 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.747+0000 7f693afdf700 1 --2- 172.21.15.32:0/1067143677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f693410e270 0x7f693419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.750 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.747+0000 7f693afdf700 1 --2- 172.21.15.32:0/1067143677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6934108f30 0x7f693419d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.747+0000 7f693afdf700 1 --2- 172.21.15.32:0/1067143677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f693410b860 0x7f693419d890 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.747+0000 7f693afdf700 1 -- 172.21.15.32:0/1067143677 >> 172.21.15.32:0/1067143677 conn(0x7f693406dc80 msgr2=0x7f693410a780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.747+0000 7f693afdf700 1 -- 172.21.15.32:0/1067143677 shutdown_connections 2023-09-30T15:00:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:06.747+0000 7f693afdf700 1 -- 172.21.15.32:0/1067143677 wait complete. 2023-09-30T15:00:07.080 INFO:tasks.workunit.client.0.smithi032.stdout: 10 341347 35.48 MB/sec execute 400 sec latency 330.998 ms 2023-09-30T15:00:08.080 INFO:tasks.workunit.client.0.smithi032.stdout: 10 342025 35.46 MB/sec execute 401 sec latency 111.212 ms 2023-09-30T15:00:08.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:07 smithi148 ceph-mon[102355]: pgmap v942: 97 pgs: 97 active+clean; 794 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.6 MiB/s wr, 701 op/s 2023-09-30T15:00:08.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:07 smithi032 ceph-mon[98592]: pgmap v942: 97 pgs: 97 active+clean; 794 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.6 MiB/s wr, 701 op/s 2023-09-30T15:00:08.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:07 smithi156 ceph-mon[103579]: pgmap v942: 97 pgs: 97 active+clean; 794 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.6 MiB/s wr, 701 op/s 2023-09-30T15:00:09.081 INFO:tasks.workunit.client.0.smithi032.stdout: 10 342935 35.52 MB/sec execute 402 sec latency 39.832 ms 2023-09-30T15:00:10.081 INFO:tasks.workunit.client.0.smithi032.stdout: 10 343639 35.53 MB/sec execute 403 sec latency 43.356 ms 2023-09-30T15:00:10.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:09 smithi148 ceph-mon[102355]: pgmap v943: 97 pgs: 97 active+clean; 784 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.4 MiB/s wr, 652 op/s 2023-09-30T15:00:10.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:09 smithi032 ceph-mon[98592]: pgmap v943: 97 pgs: 97 active+clean; 784 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.4 MiB/s wr, 652 op/s 2023-09-30T15:00:10.308 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:09 smithi156 ceph-mon[103579]: pgmap v943: 97 pgs: 97 active+clean; 784 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.4 MiB/s wr, 652 op/s 2023-09-30T15:00:11.081 INFO:tasks.workunit.client.0.smithi032.stdout: 10 344254 35.53 MB/sec execute 404 sec latency 69.922 ms 2023-09-30T15:00:11.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:10 smithi156 ceph-mon[103579]: pgmap v944: 97 pgs: 97 active+clean; 776 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 588 op/s 2023-09-30T15:00:11.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:10 smithi148 ceph-mon[102355]: pgmap v944: 97 pgs: 97 active+clean; 776 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 588 op/s 2023-09-30T15:00:11.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:10 smithi032 ceph-mon[98592]: pgmap v944: 97 pgs: 97 active+clean; 776 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 588 op/s 2023-09-30T15:00:12.081 INFO:tasks.workunit.client.0.smithi032.stdout: 10 344540 35.46 MB/sec execute 405 sec latency 260.895 ms 2023-09-30T15:00:13.081 INFO:tasks.workunit.client.0.smithi032.stdout: 10 345179 35.46 MB/sec execute 406 sec latency 68.486 ms 2023-09-30T15:00:13.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:12 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:00:13.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:12 smithi156 ceph-mon[103579]: pgmap v945: 97 pgs: 97 active+clean; 768 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.2 MiB/s wr, 755 op/s 2023-09-30T15:00:13.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:12 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:00:13.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:12 smithi148 ceph-mon[102355]: pgmap v945: 97 pgs: 97 active+clean; 768 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.2 MiB/s wr, 755 op/s 2023-09-30T15:00:13.387 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:12 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:00:13.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:12 smithi032 ceph-mon[98592]: pgmap v945: 97 pgs: 97 active+clean; 768 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.2 MiB/s wr, 755 op/s 2023-09-30T15:00:14.081 INFO:tasks.workunit.client.0.smithi032.stdout: 10 345886 35.46 MB/sec execute 407 sec latency 82.489 ms 2023-09-30T15:00:15.081 INFO:tasks.workunit.client.0.smithi032.stdout: 10 346687 35.48 MB/sec execute 408 sec latency 52.920 ms 2023-09-30T15:00:15.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:15 smithi156 ceph-mon[103579]: pgmap v946: 97 pgs: 97 active+clean; 774 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.8 MiB/s wr, 679 op/s 2023-09-30T15:00:15.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:15 smithi148 ceph-mon[102355]: pgmap v946: 97 pgs: 97 active+clean; 774 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.8 MiB/s wr, 679 op/s 2023-09-30T15:00:15.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:15 smithi032 ceph-mon[98592]: pgmap v946: 97 pgs: 97 active+clean; 774 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.8 MiB/s wr, 679 op/s 2023-09-30T15:00:16.081 INFO:tasks.workunit.client.0.smithi032.stdout: 10 347269 35.49 MB/sec execute 409 sec latency 53.568 ms 2023-09-30T15:00:17.081 INFO:tasks.workunit.client.0.smithi032.stdout: 10 347644 35.45 MB/sec execute 410 sec latency 300.910 ms 2023-09-30T15:00:17.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:17 smithi156 ceph-mon[103579]: pgmap v947: 97 pgs: 97 active+clean; 774 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 680 op/s 2023-09-30T15:00:17.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:17 smithi148 ceph-mon[102355]: pgmap v947: 97 pgs: 97 active+clean; 774 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 680 op/s 2023-09-30T15:00:17.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:17 smithi032 ceph-mon[98592]: pgmap v947: 97 pgs: 97 active+clean; 774 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 680 op/s 2023-09-30T15:00:18.081 INFO:tasks.workunit.client.0.smithi032.stdout: 10 348136 35.43 MB/sec execute 411 sec latency 96.636 ms 2023-09-30T15:00:18.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:18 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:00:18.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:18 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:00:18.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:18 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:00:19.081 INFO:tasks.workunit.client.0.smithi032.stdout: 10 348742 35.41 MB/sec execute 412 sec latency 54.630 ms 2023-09-30T15:00:19.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:19 smithi148 ceph-mon[102355]: pgmap v948: 97 pgs: 97 active+clean; 794 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.7 MiB/s wr, 692 op/s 2023-09-30T15:00:19.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:19 smithi032 ceph-mon[98592]: pgmap v948: 97 pgs: 97 active+clean; 794 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.7 MiB/s wr, 692 op/s 2023-09-30T15:00:19.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:19 smithi156 ceph-mon[103579]: pgmap v948: 97 pgs: 97 active+clean; 794 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.7 MiB/s wr, 692 op/s 2023-09-30T15:00:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.872+0000 7fd64341a700 1 -- 172.21.15.32:0/3668161736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c101fd0 msgr2=0x7fd63c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.872+0000 7fd64341a700 1 --2- 172.21.15.32:0/3668161736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c101fd0 0x7fd63c102450 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009990 tx=0x7fd62c009ca0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.872+0000 7fd64341a700 1 -- 172.21.15.32:0/3668161736 shutdown_connections 2023-09-30T15:00:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.872+0000 7fd64341a700 1 --2- 172.21.15.32:0/3668161736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd63c102990 0x7fd63c1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.872+0000 7fd64341a700 1 --2- 172.21.15.32:0/3668161736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd63c100e80 0x7fd63c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.872+0000 7fd64341a700 1 --2- 172.21.15.32:0/3668161736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c101fd0 0x7fd63c102450 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.872+0000 7fd64341a700 1 -- 172.21.15.32:0/3668161736 >> 172.21.15.32:0/3668161736 conn(0x7fd63c0fc3d0 msgr2=0x7fd63c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.872+0000 7fd64341a700 1 -- 172.21.15.32:0/3668161736 shutdown_connections 2023-09-30T15:00:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.872+0000 7fd64341a700 1 -- 172.21.15.32:0/3668161736 wait complete. 2023-09-30T15:00:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.873+0000 7fd64341a700 1 Processor -- start 2023-09-30T15:00:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.873+0000 7fd64341a700 1 -- start start 2023-09-30T15:00:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.873+0000 7fd64341a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd63c100e80 0x7fd63c073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.873+0000 7fd64341a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c101fd0 0x7fd63c073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.873+0000 7fd64341a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd63c102990 0x7fd63c073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.873+0000 7fd64341a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c1a0c10 con 0x7fd63c101fd0 2023-09-30T15:00:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.873+0000 7fd64341a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c1a0d50 con 0x7fd63c100e80 2023-09-30T15:00:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.873+0000 7fd64341a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c1a0e90 con 0x7fd63c102990 2023-09-30T15:00:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.873+0000 7fd6409b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c101fd0 0x7fd63c073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.873+0000 7fd6409b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c101fd0 0x7fd63c073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39944/0 (socket says 172.21.15.32:39944) 2023-09-30T15:00:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.873+0000 7fd6409b5700 1 -- 172.21.15.32:0/1308477228 learned_addr learned my addr 172.21.15.32:0/1308477228 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.873+0000 7fd6419b7700 1 --2- 172.21.15.32:0/1308477228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd63c102990 0x7fd63c073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.873+0000 7fd6411b6700 1 --2- 172.21.15.32:0/1308477228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd63c100e80 0x7fd63c073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.874+0000 7fd6419b7700 1 -- 172.21.15.32:0/1308477228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd63c100e80 msgr2=0x7fd63c073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.874+0000 7fd6419b7700 1 --2- 172.21.15.32:0/1308477228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd63c100e80 0x7fd63c073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.874+0000 7fd6419b7700 1 -- 172.21.15.32:0/1308477228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c101fd0 msgr2=0x7fd63c073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.874+0000 7fd6419b7700 1 --2- 172.21.15.32:0/1308477228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c101fd0 0x7fd63c073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.874+0000 7fd6419b7700 1 -- 172.21.15.32:0/1308477228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd634009c40 con 0x7fd63c102990 2023-09-30T15:00:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.874+0000 7fd6409b5700 1 --2- 172.21.15.32:0/1308477228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c101fd0 0x7fd63c073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:00:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.874+0000 7fd6419b7700 1 --2- 172.21.15.32:0/1308477228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd63c102990 0x7fd63c073c90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd63400efb0 tx=0x7fd63400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.874+0000 7fd6327fc700 1 -- 172.21.15.32:0/1308477228 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634003a00 con 0x7fd63c102990 2023-09-30T15:00:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.874+0000 7fd64341a700 1 -- 172.21.15.32:0/1308477228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62c0095f0 con 0x7fd63c102990 2023-09-30T15:00:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.874+0000 7fd6327fc700 1 -- 172.21.15.32:0/1308477228 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd63400f040 con 0x7fd63c102990 2023-09-30T15:00:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.874+0000 7fd6327fc700 1 -- 172.21.15.32:0/1308477228 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634004550 con 0x7fd63c102990 2023-09-30T15:00:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.874+0000 7fd64341a700 1 -- 172.21.15.32:0/1308477228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c1a14a0 con 0x7fd63c102990 2023-09-30T15:00:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.876+0000 7fd6327fc700 1 -- 172.21.15.32:0/1308477228 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fd634003bf0 con 0x7fd63c102990 2023-09-30T15:00:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.877+0000 7fd6327fc700 1 --2- 172.21.15.32:0/1308477228 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd6280a0fa0 0x7fd6280a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.877+0000 7fd6327fc700 1 -- 172.21.15.32:0/1308477228 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd6340cc2c0 con 0x7fd63c102990 2023-09-30T15:00:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.877+0000 7fd6411b6700 1 --2- 172.21.15.32:0/1308477228 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd6280a0fa0 0x7fd6280a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.878+0000 7fd6411b6700 1 --2- 172.21.15.32:0/1308477228 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd6280a0fa0 0x7fd6280a3460 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd63c074b20 tx=0x7fd638009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.878+0000 7fd64f602b80 1 Processor -- start 2023-09-30T15:00:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.878+0000 7fd64f602b80 1 -- start start 2023-09-30T15:00:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.878+0000 7fd64f602b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6c9200 0x557fff6c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.878+0000 7fd64f602b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 0x557fff6ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.878+0000 7fd64f602b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6cacd0 0x557fff6d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.878+0000 7fd64f602b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fff631400 con 0x557fff6cacd0 2023-09-30T15:00:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.878+0000 7fd64f602b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fff622bc0 con 0x557fff6ca310 2023-09-30T15:00:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.878+0000 7fd64f602b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fff6204e0 con 0x557fff6c9200 2023-09-30T15:00:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.878+0000 7fd6411b6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6c9200 0x557fff6c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.878+0000 7fd6419b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6cacd0 0x557fff6d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.878+0000 7fd6411b6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6c9200 0x557fff6c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54332/0 (socket says 172.21.15.32:54332) 2023-09-30T15:00:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.879+0000 7fd6409b5700 1 --2- 172.21.15.32:0/2820857076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 0x557fff6ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.879+0000 7fd6411b6700 1 -- 172.21.15.32:0/2820857076 learned_addr learned my addr 172.21.15.32:0/2820857076 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.879+0000 7fd6419b7700 1 -- 172.21.15.32:0/2820857076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6c9200 msgr2=0x557fff6c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.879+0000 7fd6419b7700 1 --2- 172.21.15.32:0/2820857076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6c9200 0x557fff6c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.879+0000 7fd6419b7700 1 -- 172.21.15.32:0/2820857076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 msgr2=0x557fff6ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.879+0000 7fd6419b7700 1 --2- 172.21.15.32:0/2820857076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 0x557fff6ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.879+0000 7fd6419b7700 1 -- 172.21.15.32:0/2820857076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fff67e870 con 0x557fff6cacd0 2023-09-30T15:00:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.879+0000 7fd6419b7700 1 --2- 172.21.15.32:0/2820857076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6cacd0 0x557fff6d1560 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd634017000 tx=0x7fd634099670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b511ecfe03774258 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.879+0000 7fd6277fe700 1 -- 172.21.15.32:0/2820857076 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634099f20 con 0x557fff6cacd0 2023-09-30T15:00:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.879+0000 7fd6277fe700 1 -- 172.21.15.32:0/2820857076 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd634099820 con 0x557fff6cacd0 2023-09-30T15:00:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.879+0000 7fd6277fe700 1 -- 172.21.15.32:0/2820857076 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634004910 con 0x557fff6cacd0 2023-09-30T15:00:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.880+0000 7fd64f602b80 1 -- 172.21.15.32:0/2820857076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6cacd0 msgr2=0x557fff6d1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.880+0000 7fd64f602b80 1 --2- 172.21.15.32:0/2820857076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6cacd0 0x557fff6d1560 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd634017000 tx=0x7fd634099670 comp rx=0 tx=0).stop 2023-09-30T15:00:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.880+0000 7fd64f602b80 1 -- 172.21.15.32:0/2820857076 shutdown_connections 2023-09-30T15:00:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.880+0000 7fd64f602b80 1 --2- 172.21.15.32:0/2820857076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6cacd0 0x557fff6d1560 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.880+0000 7fd64f602b80 1 --2- 172.21.15.32:0/2820857076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6c9200 0x557fff6c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.880+0000 7fd64f602b80 1 --2- 172.21.15.32:0/2820857076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 0x557fff6ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.880+0000 7fd64f602b80 1 -- 172.21.15.32:0/2820857076 >> 172.21.15.32:0/2820857076 conn(0x557fff63d200 msgr2=0x557fff6c3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.880+0000 7fd64f602b80 1 -- 172.21.15.32:0/2820857076 shutdown_connections 2023-09-30T15:00:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.880+0000 7fd64f602b80 1 -- 172.21.15.32:0/2820857076 wait complete. 2023-09-30T15:00:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.881+0000 7fd64f602b80 1 -- start start 2023-09-30T15:00:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.881+0000 7fd64f602b80 1 Processor -- start 2023-09-30T15:00:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.881+0000 7fd64f602b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.881+0000 7fd64f602b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 0x557fff6d33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.881+0000 7fd64f602b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 0x557fff6da3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.881+0000 7fd64f602b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fff61bdd0 con 0x557fff6c9200 2023-09-30T15:00:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.881+0000 7fd64f602b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fff5b86b0 con 0x557fff6ca310 2023-09-30T15:00:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.881+0000 7fd64f602b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fff5b8350 con 0x557fff6cacd0 2023-09-30T15:00:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.881+0000 7fd64f602b80 10 client.0 _ll_register_callbacks cb 0x557fff4157a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:00:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.881+0000 7fd6411b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.881+0000 7fd6409b5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 0x557fff6d33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.881+0000 7fd6411b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39956/0 (socket says 172.21.15.32:39956) 2023-09-30T15:00:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.881+0000 7fd6419b7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 0x557fff6da3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.881+0000 7fd6411b6700 1 -- 172.21.15.32:0/1047718794 learned_addr learned my addr 172.21.15.32:0/1047718794 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.882+0000 7fd6411b6700 1 -- 172.21.15.32:0/1047718794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 msgr2=0x557fff6da3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.882+0000 7fd6411b6700 1 --2- 172.21.15.32:0/1047718794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 0x557fff6da3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.882+0000 7fd6411b6700 1 -- 172.21.15.32:0/1047718794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 msgr2=0x557fff6d33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.882+0000 7fd6411b6700 1 --2- 172.21.15.32:0/1047718794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 0x557fff6d33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.882+0000 7fd6411b6700 1 -- 172.21.15.32:0/1047718794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c1a14a0 con 0x557fff6c9200 2023-09-30T15:00:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.882+0000 7fd6411b6700 1 --2- 172.21.15.32:0/1047718794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d2e70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd6380045b0 tx=0x7fd6380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6925affab0c24d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.882+0000 7fd624ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T15:00:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.882+0000 7fd624ff9700 1 -- 172.21.15.32:0/1047718794 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638022070 con 0x557fff6c9200 2023-09-30T15:00:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.882+0000 7fd624ff9700 1 -- 172.21.15.32:0/1047718794 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd638004830 con 0x557fff6c9200 2023-09-30T15:00:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.882+0000 7fd64f602b80 1 -- 172.21.15.32:0/1047718794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fff6bc090 con 0x557fff6c9200 2023-09-30T15:00:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.882+0000 7fd624ff9700 1 -- 172.21.15.32:0/1047718794 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd638004a30 con 0x557fff6c9200 2023-09-30T15:00:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.882+0000 7fd64f602b80 10 client.17268 fetch_fsmap learned FSMap version 55 2023-09-30T15:00:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.882+0000 7fd64f602b80 1 -- 172.21.15.32:0/1047718794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x557fff446d80 con 0x557fff6c9200 2023-09-30T15:00:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.883+0000 7fd624ff9700 1 -- 172.21.15.32:0/1047718794 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557fff6bc090 con 0x557fff6c9200 2023-09-30T15:00:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.883+0000 7fd624ff9700 1 -- 172.21.15.32:0/1047718794 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd638030020 con 0x557fff6c9200 2023-09-30T15:00:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.883+0000 7fd64f602b80 10 client.17268 fetch_fsmap finished waiting for FSMap version 55 2023-09-30T15:00:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.883+0000 7fd64f602b80 10 client.17268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:00:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.883+0000 7fd64f602b80 20 client.17268 populate_metadata read hostname 'smithi032' 2023-09-30T15:00:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.883+0000 7fd64f602b80 1 --2- 172.21.15.32:0/1047718794 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557fff6decd0 0x557fff6e10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.883+0000 7fd64f602b80 4 client.17268 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T15:00:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.883+0000 7fd64f602b80 1 -- 172.21.15.32:0/1047718794 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fff614db0 con 0x557fff6decd0 2023-09-30T15:00:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.883+0000 7fd6409b5700 1 --2- 172.21.15.32:0/1047718794 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557fff6decd0 0x557fff6e10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.883+0000 7fd624ff9700 10 client.17268 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:00:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.883+0000 7fd6409b5700 1 --2- 172.21.15.32:0/1047718794 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557fff6decd0 0x557fff6e10f0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a50fbb3ec35fd211 server_cookie=43479d61183ffd3 in_seq=0 out_seq=0 2023-09-30T15:00:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.888+0000 7fd624ff9700 1 -- 172.21.15.32:0/1047718794 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557fff614db0 con 0x557fff6decd0 2023-09-30T15:00:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.888+0000 7fd624ff9700 10 client.17268 handle_command_reply: tid=0 2023-09-30T15:00:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.888+0000 7fd64f602b80 1 client.17268 shutdown 2023-09-30T15:00:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.888+0000 7fd64f602b80 1 -- 172.21.15.32:0/1047718794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 msgr2=0x557fff6d2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.888+0000 7fd64f602b80 1 --2- 172.21.15.32:0/1047718794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d2e70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd6380045b0 tx=0x7fd6380047c0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.889+0000 7fd64f602b80 1 -- 172.21.15.32:0/1047718794 shutdown_connections 2023-09-30T15:00:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.889+0000 7fd64f602b80 1 --2- 172.21.15.32:0/1047718794 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557fff6decd0 0x557fff6e10f0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.889+0000 7fd64f602b80 1 --2- 172.21.15.32:0/1047718794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 0x557fff6da3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.889+0000 7fd64f602b80 1 --2- 172.21.15.32:0/1047718794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d2e70 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.889+0000 7fd624ff9700 0 client.17268 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:00:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.889+0000 7fd64f602b80 1 --2- 172.21.15.32:0/1047718794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 0x557fff6d33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.889+0000 7fd64f602b80 1 -- 172.21.15.32:0/1047718794 >> 172.21.15.32:0/1047718794 conn(0x557fff63d200 msgr2=0x557fff6c3590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.889+0000 7fd64f602b80 1 -- 172.21.15.32:0/1047718794 shutdown_connections 2023-09-30T15:00:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.889+0000 7fd64f602b80 1 -- 172.21.15.32:0/1047718794 wait complete. 2023-09-30T15:00:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.889+0000 7fd64f602b80 20 client.17268 trim_cache size 0 max 16384 2023-09-30T15:00:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.910+0000 7fd64f602b80 1 Processor -- start 2023-09-30T15:00:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.910+0000 7fd64f602b80 1 -- start start 2023-09-30T15:00:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.910+0000 7fd64f602b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.910+0000 7fd64f602b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 0x557fff6da3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.910+0000 7fd64f602b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 0x557fff6c3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.910+0000 7fd64f602b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fff49c8c0 con 0x557fff6c9200 2023-09-30T15:00:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.910+0000 7fd64f602b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fff6d2a40 con 0x557fff6ca310 2023-09-30T15:00:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.910+0000 7fd64f602b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fff6ca0e0 con 0x557fff6cacd0 2023-09-30T15:00:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.910+0000 7fd6419b7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 0x557fff6c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.910+0000 7fd6419b7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 0x557fff6c3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54348/0 (socket says 172.21.15.32:54348) 2023-09-30T15:00:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.910+0000 7fd6419b7700 1 -- 172.21.15.32:0/1237126278 learned_addr learned my addr 172.21.15.32:0/1237126278 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.910+0000 7fd6411b6700 1 --2- 172.21.15.32:0/1237126278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.910+0000 7fd6409b5700 1 --2- 172.21.15.32:0/1237126278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 0x557fff6da3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.911+0000 7fd6419b7700 1 -- 172.21.15.32:0/1237126278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 msgr2=0x557fff6da3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.911+0000 7fd6419b7700 1 --2- 172.21.15.32:0/1237126278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 0x557fff6da3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.911+0000 7fd6419b7700 1 -- 172.21.15.32:0/1237126278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 msgr2=0x557fff6d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.911+0000 7fd6419b7700 1 --2- 172.21.15.32:0/1237126278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.911+0000 7fd6419b7700 1 -- 172.21.15.32:0/1237126278 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fff42fc00 con 0x557fff6cacd0 2023-09-30T15:00:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.911+0000 7fd6409b5700 1 --2- 172.21.15.32:0/1237126278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 0x557fff6da3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T15:00:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.911+0000 7fd6411b6700 1 --2- 172.21.15.32:0/1237126278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:00:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.911+0000 7fd6419b7700 1 --2- 172.21.15.32:0/1237126278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 0x557fff6c3590 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd634099760 tx=0x7fd634003b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce1a73a11cd43370 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.911+0000 7fd627fff700 1 -- 172.21.15.32:0/1237126278 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634099f20 con 0x557fff6cacd0 2023-09-30T15:00:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.911+0000 7fd627fff700 1 -- 172.21.15.32:0/1237126278 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd63409d970 con 0x557fff6cacd0 2023-09-30T15:00:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.911+0000 7fd627fff700 1 -- 172.21.15.32:0/1237126278 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634099a80 con 0x557fff6cacd0 2023-09-30T15:00:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.912+0000 7fd64f602b80 1 -- 172.21.15.32:0/1237126278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 msgr2=0x557fff6c3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.912+0000 7fd64f602b80 1 --2- 172.21.15.32:0/1237126278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 0x557fff6c3590 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd634099760 tx=0x7fd634003b60 comp rx=0 tx=0).stop 2023-09-30T15:00:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.912+0000 7fd64f602b80 1 -- 172.21.15.32:0/1237126278 shutdown_connections 2023-09-30T15:00:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.912+0000 7fd64f602b80 1 --2- 172.21.15.32:0/1237126278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 0x557fff6c3590 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.912+0000 7fd64f602b80 1 --2- 172.21.15.32:0/1237126278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.912+0000 7fd64f602b80 1 --2- 172.21.15.32:0/1237126278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 0x557fff6da3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.912+0000 7fd64f602b80 1 -- 172.21.15.32:0/1237126278 >> 172.21.15.32:0/1237126278 conn(0x557fff63d200 msgr2=0x557fff6c3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.912+0000 7fd64f602b80 1 -- 172.21.15.32:0/1237126278 shutdown_connections 2023-09-30T15:00:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.912+0000 7fd64f602b80 1 -- 172.21.15.32:0/1237126278 wait complete. 2023-09-30T15:00:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.912+0000 7fd64f602b80 1 -- start start 2023-09-30T15:00:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.913+0000 7fd64f602b80 1 Processor -- start 2023-09-30T15:00:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.913+0000 7fd64f602b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.913+0000 7fd64f602b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 0x557fff6d8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.913+0000 7fd64f602b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 0x557fff6d9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.913+0000 7fd64f602b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fff6d95a0 con 0x557fff6c9200 2023-09-30T15:00:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.913+0000 7fd64f602b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fff6d9710 con 0x557fff6ca310 2023-09-30T15:00:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.913+0000 7fd64f602b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fff6d9880 con 0x557fff6cacd0 2023-09-30T15:00:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.913+0000 7fd64f602b80 10 client.0 _ll_register_callbacks cb 0x7fd6380043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:00:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.913+0000 7fd6411b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.913+0000 7fd6411b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39970/0 (socket says 172.21.15.32:39970) 2023-09-30T15:00:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.913+0000 7fd6411b6700 1 -- 172.21.15.32:0/752693705 learned_addr learned my addr 172.21.15.32:0/752693705 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.914+0000 7fd6419b7700 1 --2- 172.21.15.32:0/752693705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 0x557fff6d9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.914+0000 7fd6411b6700 1 -- 172.21.15.32:0/752693705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 msgr2=0x557fff6d9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.914+0000 7fd6411b6700 1 --2- 172.21.15.32:0/752693705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 0x557fff6d9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.914+0000 7fd6411b6700 1 -- 172.21.15.32:0/752693705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 msgr2=0x557fff6d8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.914+0000 7fd6411b6700 1 --2- 172.21.15.32:0/752693705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 0x557fff6d8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.914+0000 7fd6411b6700 1 -- 172.21.15.32:0/752693705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fff446d80 con 0x557fff6c9200 2023-09-30T15:00:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.914+0000 7fd6411b6700 1 --2- 172.21.15.32:0/752693705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d7b10 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd6380058b0 tx=0x7fd638011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9b9e3f1cf7a16c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.914+0000 7fd624ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T15:00:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.915+0000 7fd624ff9700 1 -- 172.21.15.32:0/752693705 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638022040 con 0x557fff6c9200 2023-09-30T15:00:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.915+0000 7fd624ff9700 1 -- 172.21.15.32:0/752693705 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd63801dcf0 con 0x557fff6c9200 2023-09-30T15:00:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.916+0000 7fd64f602b80 1 -- 172.21.15.32:0/752693705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd634099820 con 0x557fff6c9200 2023-09-30T15:00:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.916+0000 7fd624ff9700 1 -- 172.21.15.32:0/752693705 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd638027470 con 0x557fff6c9200 2023-09-30T15:00:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.916+0000 7fd64f602b80 10 client.17274 fetch_fsmap learned FSMap version 55 2023-09-30T15:00:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.916+0000 7fd64f602b80 1 -- 172.21.15.32:0/752693705 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x557fff67e870 con 0x557fff6c9200 2023-09-30T15:00:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.916+0000 7fd624ff9700 1 -- 172.21.15.32:0/752693705 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd638041030 con 0x557fff6c9200 2023-09-30T15:00:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.916+0000 7fd624ff9700 1 -- 172.21.15.32:0/752693705 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd638003680 con 0x557fff6c9200 2023-09-30T15:00:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.916+0000 7fd64f602b80 10 client.17274 fetch_fsmap finished waiting for FSMap version 55 2023-09-30T15:00:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.916+0000 7fd64f602b80 10 client.17274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:00:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.916+0000 7fd64f602b80 20 client.17274 populate_metadata read hostname 'smithi032' 2023-09-30T15:00:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.916+0000 7fd64f602b80 1 --2- 172.21.15.32:0/752693705 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557fff72d4c0 0x557fff72f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.916+0000 7fd64f602b80 4 client.17274 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T15:00:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.916+0000 7fd64f602b80 1 -- 172.21.15.32:0/752693705 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557fff614ad0 con 0x557fff72d4c0 2023-09-30T15:00:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.917+0000 7fd6409b5700 1 --2- 172.21.15.32:0/752693705 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557fff72d4c0 0x557fff72f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.917+0000 7fd624ff9700 10 client.17274 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:00:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:19.917+0000 7fd6409b5700 1 --2- 172.21.15.32:0/752693705 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557fff72d4c0 0x557fff72f8e0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a849967a4d603ec server_cookie=9dda48e25467666f in_seq=0 out_seq=0 2023-09-30T15:00:20.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 349323 35.41 MB/sec execute 413 sec latency 130.673 ms 2023-09-30T15:00:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.337+0000 7fd624ff9700 1 -- 172.21.15.32:0/752693705 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557fff614ad0 con 0x557fff72d4c0 2023-09-30T15:00:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.337+0000 7fd624ff9700 10 client.17274 handle_command_reply: tid=0 2023-09-30T15:00:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.337+0000 7fd64f602b80 1 client.17274 shutdown 2023-09-30T15:00:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.337+0000 7fd64f602b80 1 -- 172.21.15.32:0/752693705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 msgr2=0x557fff6d7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.337+0000 7fd64f602b80 1 --2- 172.21.15.32:0/752693705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d7b10 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd6380058b0 tx=0x7fd638011b70 comp rx=0 tx=0).stop 2023-09-30T15:00:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.337+0000 7fd64f602b80 1 -- 172.21.15.32:0/752693705 shutdown_connections 2023-09-30T15:00:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.337+0000 7fd64f602b80 1 --2- 172.21.15.32:0/752693705 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557fff72d4c0 0x557fff72f8e0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.337+0000 7fd64f602b80 1 --2- 172.21.15.32:0/752693705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557fff6cacd0 0x557fff6d9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.337+0000 7fd64f602b80 1 --2- 172.21.15.32:0/752693705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557fff6c9200 0x557fff6d7b10 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.337+0000 7fd64f602b80 1 --2- 172.21.15.32:0/752693705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557fff6ca310 0x557fff6d8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.337+0000 7fd64f602b80 1 -- 172.21.15.32:0/752693705 >> 172.21.15.32:0/752693705 conn(0x557fff63d200 msgr2=0x557fff6d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T15:00:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T15:00:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T15:00:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T15:00:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.337+0000 7fd624ff9700 0 client.17274 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:00:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.337+0000 7fd64f602b80 1 -- 172.21.15.32:0/752693705 shutdown_connections 2023-09-30T15:00:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.337+0000 7fd64f602b80 1 -- 172.21.15.32:0/752693705 wait complete. 2023-09-30T15:00:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.337+0000 7fd64f602b80 20 client.17274 trim_cache size 0 max 16384 2023-09-30T15:00:20.341 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.341+0000 7fd64341a700 1 -- 172.21.15.32:0/1308477228 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd6280a0fa0 msgr2=0x7fd6280a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.341+0000 7fd64341a700 1 --2- 172.21.15.32:0/1308477228 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd6280a0fa0 0x7fd6280a3460 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd63c074b20 tx=0x7fd638009260 comp rx=0 tx=0).stop 2023-09-30T15:00:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.341+0000 7fd64341a700 1 -- 172.21.15.32:0/1308477228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd63c102990 msgr2=0x7fd63c073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.341+0000 7fd64341a700 1 --2- 172.21.15.32:0/1308477228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd63c102990 0x7fd63c073c90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd63400efb0 tx=0x7fd63400c3a0 comp rx=0 tx=0).stop 2023-09-30T15:00:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.341+0000 7fd64341a700 1 -- 172.21.15.32:0/1308477228 shutdown_connections 2023-09-30T15:00:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.341+0000 7fd64341a700 1 --2- 172.21.15.32:0/1308477228 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd6280a0fa0 0x7fd6280a3460 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.341+0000 7fd64341a700 1 --2- 172.21.15.32:0/1308477228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd63c102990 0x7fd63c073c90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.341+0000 7fd64341a700 1 --2- 172.21.15.32:0/1308477228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd63c100e80 0x7fd63c073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.341+0000 7fd64341a700 1 --2- 172.21.15.32:0/1308477228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd63c101fd0 0x7fd63c073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.341+0000 7fd64341a700 1 -- 172.21.15.32:0/1308477228 >> 172.21.15.32:0/1308477228 conn(0x7fd63c0fc3d0 msgr2=0x7fd63c105590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.341+0000 7fd64341a700 1 -- 172.21.15.32:0/1308477228 shutdown_connections 2023-09-30T15:00:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:20.341+0000 7fd64341a700 1 -- 172.21.15.32:0/1308477228 wait complete. 2023-09-30T15:00:21.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 349831 35.38 MB/sec execute 414 sec latency 252.508 ms 2023-09-30T15:00:21.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:21 smithi148 ceph-mon[102355]: pgmap v949: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 9.2 MiB/s wr, 619 op/s 2023-09-30T15:00:21.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:21 smithi032 ceph-mon[98592]: pgmap v949: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 9.2 MiB/s wr, 619 op/s 2023-09-30T15:00:21.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:21 smithi156 ceph-mon[103579]: pgmap v949: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 9.2 MiB/s wr, 619 op/s 2023-09-30T15:00:22.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 350227 35.35 MB/sec execute 415 sec latency 332.463 ms 2023-09-30T15:00:23.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 350786 35.33 MB/sec execute 416 sec latency 114.285 ms 2023-09-30T15:00:23.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:23 smithi148 ceph-mon[102355]: pgmap v950: 97 pgs: 97 active+clean; 804 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 11 MiB/s wr, 807 op/s 2023-09-30T15:00:23.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:23 smithi032 ceph-mon[98592]: pgmap v950: 97 pgs: 97 active+clean; 804 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 11 MiB/s wr, 807 op/s 2023-09-30T15:00:23.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:23 smithi156 ceph-mon[103579]: pgmap v950: 97 pgs: 97 active+clean; 804 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 11 MiB/s wr, 807 op/s 2023-09-30T15:00:24.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 351500 35.36 MB/sec execute 417 sec latency 89.631 ms 2023-09-30T15:00:25.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 352155 35.34 MB/sec execute 418 sec latency 41.079 ms 2023-09-30T15:00:25.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:25 smithi148 ceph-mon[102355]: pgmap v951: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.6 MiB/s wr, 648 op/s 2023-09-30T15:00:25.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:25 smithi032 ceph-mon[98592]: pgmap v951: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.6 MiB/s wr, 648 op/s 2023-09-30T15:00:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:25 smithi156 ceph-mon[103579]: pgmap v951: 97 pgs: 97 active+clean; 800 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.6 MiB/s wr, 648 op/s 2023-09-30T15:00:26.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 352888 35.36 MB/sec execute 419 sec latency 233.772 ms 2023-09-30T15:00:27.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 353450 35.34 MB/sec execute 420 sec latency 256.980 ms 2023-09-30T15:00:27.612 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:27 smithi148 ceph-mon[102355]: pgmap v952: 97 pgs: 97 active+clean; 806 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.8 MiB/s wr, 634 op/s 2023-09-30T15:00:27.640 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:27 smithi032 ceph-mon[98592]: pgmap v952: 97 pgs: 97 active+clean; 806 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.8 MiB/s wr, 634 op/s 2023-09-30T15:00:27.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:27 smithi156 ceph-mon[103579]: pgmap v952: 97 pgs: 97 active+clean; 806 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.8 MiB/s wr, 634 op/s 2023-09-30T15:00:28.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 354034 35.33 MB/sec execute 421 sec latency 88.036 ms 2023-09-30T15:00:29.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 354750 35.34 MB/sec execute 422 sec latency 87.090 ms 2023-09-30T15:00:29.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:29 smithi148 ceph-mon[102355]: pgmap v953: 97 pgs: 97 active+clean; 807 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 721 op/s 2023-09-30T15:00:29.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:29 smithi032 ceph-mon[98592]: pgmap v953: 97 pgs: 97 active+clean; 807 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 721 op/s 2023-09-30T15:00:29.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:29 smithi156 ceph-mon[103579]: pgmap v953: 97 pgs: 97 active+clean; 807 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 721 op/s 2023-09-30T15:00:30.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 355469 35.34 MB/sec execute 423 sec latency 80.589 ms 2023-09-30T15:00:31.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 356261 35.37 MB/sec execute 424 sec latency 33.302 ms 2023-09-30T15:00:31.612 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:31 smithi148 ceph-mon[102355]: pgmap v954: 97 pgs: 97 active+clean; 813 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.7 MiB/s wr, 670 op/s 2023-09-30T15:00:31.640 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:31 smithi032 ceph-mon[98592]: pgmap v954: 97 pgs: 97 active+clean; 813 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.7 MiB/s wr, 670 op/s 2023-09-30T15:00:31.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:31 smithi156 ceph-mon[103579]: pgmap v954: 97 pgs: 97 active+clean; 813 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.7 MiB/s wr, 670 op/s 2023-09-30T15:00:32.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 356701 35.33 MB/sec execute 425 sec latency 308.039 ms 2023-09-30T15:00:33.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 357338 35.33 MB/sec execute 426 sec latency 69.203 ms 2023-09-30T15:00:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.486+0000 7fac10855700 1 -- 172.21.15.32:0/870853633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac0c101fb0 msgr2=0x7fac0c102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.486+0000 7fac10855700 1 --2- 172.21.15.32:0/870853633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac0c101fb0 0x7fac0c102430 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fac00009990 tx=0x7fac00009ca0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.486+0000 7fac10855700 1 -- 172.21.15.32:0/870853633 shutdown_connections 2023-09-30T15:00:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.486+0000 7fac10855700 1 --2- 172.21.15.32:0/870853633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac0c102970 0x7fac0c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.486+0000 7fac10855700 1 --2- 172.21.15.32:0/870853633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac0c100e60 0x7fac0c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.486+0000 7fac10855700 1 --2- 172.21.15.32:0/870853633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac0c101fb0 0x7fac0c102430 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.486+0000 7fac10855700 1 -- 172.21.15.32:0/870853633 >> 172.21.15.32:0/870853633 conn(0x7fac0c0fc3f0 msgr2=0x7fac0c0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.486+0000 7fac10855700 1 -- 172.21.15.32:0/870853633 shutdown_connections 2023-09-30T15:00:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.486+0000 7fac10855700 1 -- 172.21.15.32:0/870853633 wait complete. 2023-09-30T15:00:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.487+0000 7fac10855700 1 Processor -- start 2023-09-30T15:00:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.487+0000 7fac10855700 1 -- start start 2023-09-30T15:00:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.487+0000 7fac10855700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac0c100e60 0x7fac0c196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.487+0000 7fac10855700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac0c101fb0 0x7fac0c197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.488+0000 7fac10855700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac0c102970 0x7fac0c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.488+0000 7fac10855700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c198670 con 0x7fac0c101fb0 2023-09-30T15:00:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.488+0000 7fac10855700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c1987e0 con 0x7fac0c102970 2023-09-30T15:00:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.488+0000 7fac10855700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c1989a0 con 0x7fac0c100e60 2023-09-30T15:00:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.488+0000 7fac0a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac0c100e60 0x7fac0c196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.488+0000 7fac0a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac0c100e60 0x7fac0c196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54366/0 (socket says 172.21.15.32:54366) 2023-09-30T15:00:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.488+0000 7fac0a59c700 1 -- 172.21.15.32:0/792863234 learned_addr learned my addr 172.21.15.32:0/792863234 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.488+0000 7fac0ad9d700 1 --2- 172.21.15.32:0/792863234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac0c102970 0x7fac0c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.488+0000 7fac09d9b700 1 --2- 172.21.15.32:0/792863234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac0c101fb0 0x7fac0c197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.488+0000 7fac0ad9d700 1 -- 172.21.15.32:0/792863234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac0c100e60 msgr2=0x7fac0c196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.488+0000 7fac0ad9d700 1 --2- 172.21.15.32:0/792863234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac0c100e60 0x7fac0c196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.488+0000 7fac0ad9d700 1 -- 172.21.15.32:0/792863234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac0c101fb0 msgr2=0x7fac0c197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.488+0000 7fac0ad9d700 1 --2- 172.21.15.32:0/792863234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac0c101fb0 0x7fac0c197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.488+0000 7fac0ad9d700 1 -- 172.21.15.32:0/792863234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc009520 con 0x7fac0c102970 2023-09-30T15:00:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.488+0000 7fac0a59c700 1 --2- 172.21.15.32:0/792863234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac0c100e60 0x7fac0c196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:00:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.488+0000 7fac0ad9d700 1 --2- 172.21.15.32:0/792863234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac0c102970 0x7fac0c197f70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fac0400efc0 tx=0x7fac0400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.489+0000 7fabfb7fe700 1 -- 172.21.15.32:0/792863234 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac04018070 con 0x7fac0c102970 2023-09-30T15:00:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.489+0000 7fac10855700 1 -- 172.21.15.32:0/792863234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac000095f0 con 0x7fac0c102970 2023-09-30T15:00:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.489+0000 7fabfb7fe700 1 -- 172.21.15.32:0/792863234 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac0400f040 con 0x7fac0c102970 2023-09-30T15:00:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.489+0000 7fabfb7fe700 1 -- 172.21.15.32:0/792863234 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0400cd40 con 0x7fac0c102970 2023-09-30T15:00:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.489+0000 7fac10855700 1 -- 172.21.15.32:0/792863234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac0c073050 con 0x7fac0c102970 2023-09-30T15:00:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.491+0000 7fabfb7fe700 1 -- 172.21.15.32:0/792863234 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fac04003a00 con 0x7fac0c102970 2023-09-30T15:00:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.492+0000 7fabfb7fe700 1 --2- 172.21.15.32:0/792863234 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fabf40a0fa0 0x7fabf40a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.492+0000 7fabfb7fe700 1 -- 172.21.15.32:0/792863234 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fac040cc310 con 0x7fac0c102970 2023-09-30T15:00:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.492+0000 7fac0a59c700 1 --2- 172.21.15.32:0/792863234 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fabf40a0fa0 0x7fabf40a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.492+0000 7fac0a59c700 1 --2- 172.21.15.32:0/792863234 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fabf40a0fa0 0x7fabf40a3460 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009fd0 tx=0x7fabfc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.493+0000 7fac1ca3db80 1 Processor -- start 2023-09-30T15:00:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.493+0000 7fac1ca3db80 1 -- start start 2023-09-30T15:00:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.493+0000 7fac1ca3db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c83200 0x55e0b7c83620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.493+0000 7fac1ca3db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c84310 0x55e0b7c84790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.493+0000 7fac1ca3db80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c8b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.493+0000 7fac1ca3db80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b7beb400 con 0x55e0b7c84cd0 2023-09-30T15:00:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.493+0000 7fac1ca3db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b7bdcbc0 con 0x55e0b7c84310 2023-09-30T15:00:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.493+0000 7fac1ca3db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b7bda4e0 con 0x55e0b7c83200 2023-09-30T15:00:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.493+0000 7fac0ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c8b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.493+0000 7fac09d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c84310 0x55e0b7c84790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.493+0000 7fac0ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c8b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39994/0 (socket says 172.21.15.32:39994) 2023-09-30T15:00:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.493+0000 7fac0ad9d700 1 -- 172.21.15.32:0/3605645878 learned_addr learned my addr 172.21.15.32:0/3605645878 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.493+0000 7fac0a59c700 1 --2- 172.21.15.32:0/3605645878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c83200 0x55e0b7c83620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.494+0000 7fac0ad9d700 1 -- 172.21.15.32:0/3605645878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c83200 msgr2=0x55e0b7c83620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.494+0000 7fac0ad9d700 1 --2- 172.21.15.32:0/3605645878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c83200 0x55e0b7c83620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.494+0000 7fac0ad9d700 1 -- 172.21.15.32:0/3605645878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c84310 msgr2=0x55e0b7c84790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.494+0000 7fac0ad9d700 1 --2- 172.21.15.32:0/3605645878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c84310 0x55e0b7c84790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.494+0000 7fac0ad9d700 1 -- 172.21.15.32:0/3605645878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0b7c38870 con 0x55e0b7c84cd0 2023-09-30T15:00:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.494+0000 7fac0ad9d700 1 --2- 172.21.15.32:0/3605645878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c8b560 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fac040090a0 tx=0x7fac04095cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451e392d72001d26 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.494+0000 7fabf8ff9700 1 -- 172.21.15.32:0/3605645878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac04082ea0 con 0x55e0b7c84cd0 2023-09-30T15:00:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.494+0000 7fabf8ff9700 1 -- 172.21.15.32:0/3605645878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac040991d0 con 0x55e0b7c84cd0 2023-09-30T15:00:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.494+0000 7fabf8ff9700 1 -- 172.21.15.32:0/3605645878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0401b320 con 0x55e0b7c84cd0 2023-09-30T15:00:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.495+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3605645878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84cd0 msgr2=0x55e0b7c8b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.495+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/3605645878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c8b560 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fac040090a0 tx=0x7fac04095cb0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.495+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3605645878 shutdown_connections 2023-09-30T15:00:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.495+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/3605645878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c8b560 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.495+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/3605645878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c83200 0x55e0b7c83620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.495+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/3605645878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c84310 0x55e0b7c84790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.495+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3605645878 >> 172.21.15.32:0/3605645878 conn(0x55e0b7bf7200 msgr2=0x55e0b7c7def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.495+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3605645878 shutdown_connections 2023-09-30T15:00:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.495+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3605645878 wait complete. 2023-09-30T15:00:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.496+0000 7fac1ca3db80 1 -- start start 2023-09-30T15:00:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.496+0000 7fac1ca3db80 1 Processor -- start 2023-09-30T15:00:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.496+0000 7fac1ca3db80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c83200 0x55e0b7c8cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.496+0000 7fac1ca3db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c84310 0x55e0b7c8d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.496+0000 7fac1ca3db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c944e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.496+0000 7fac1ca3db80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b7bd5dd0 con 0x55e0b7c83200 2023-09-30T15:00:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.496+0000 7fac1ca3db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b7b726b0 con 0x55e0b7c84310 2023-09-30T15:00:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.496+0000 7fac1ca3db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b7b72350 con 0x55e0b7c84cd0 2023-09-30T15:00:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.496+0000 7fac1ca3db80 10 client.0 _ll_register_callbacks cb 0x55e0b79cf7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:00:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.496+0000 7fac0a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c83200 0x55e0b7c8cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.496+0000 7fac09d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c84310 0x55e0b7c8d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.496+0000 7fac09d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c84310 0x55e0b7c8d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51476/0 (socket says 172.21.15.32:51476) 2023-09-30T15:00:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.496+0000 7fac09d9b700 1 -- 172.21.15.32:0/1782807902 learned_addr learned my addr 172.21.15.32:0/1782807902 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.496+0000 7fac0ad9d700 1 --2- 172.21.15.32:0/1782807902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c944e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.496+0000 7fac0a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c83200 0x55e0b7c8cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39998/0 (socket says 172.21.15.32:39998) 2023-09-30T15:00:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.497+0000 7fac0a59c700 1 -- 172.21.15.32:0/1782807902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 msgr2=0x55e0b7c944e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.497+0000 7fac0a59c700 1 --2- 172.21.15.32:0/1782807902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c944e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.497+0000 7fac0a59c700 1 -- 172.21.15.32:0/1782807902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c84310 msgr2=0x55e0b7c8d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.497+0000 7fac0a59c700 1 --2- 172.21.15.32:0/1782807902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c84310 0x55e0b7c8d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.497+0000 7fac0a59c700 1 -- 172.21.15.32:0/1782807902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac0c073050 con 0x55e0b7c83200 2023-09-30T15:00:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.497+0000 7fac09d9b700 1 --2- 172.21.15.32:0/1782807902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c84310 0x55e0b7c8d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:00:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.497+0000 7fac0a59c700 1 --2- 172.21.15.32:0/1782807902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c83200 0x55e0b7c8cf60 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x55e0b7c82fa0 tx=0x7fabfc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b0461016d13b42 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.497+0000 7fabf1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T15:00:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.497+0000 7fabf1ffb700 1 -- 172.21.15.32:0/1782807902 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc022070 con 0x55e0b7c83200 2023-09-30T15:00:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.497+0000 7fabf1ffb700 1 -- 172.21.15.32:0/1782807902 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fabfc01c7f0 con 0x55e0b7c83200 2023-09-30T15:00:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.497+0000 7fac1ca3db80 1 -- 172.21.15.32:0/1782807902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0b7c76090 con 0x55e0b7c83200 2023-09-30T15:00:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.497+0000 7fabf1ffb700 1 -- 172.21.15.32:0/1782807902 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabfc01c9b0 con 0x55e0b7c83200 2023-09-30T15:00:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.497+0000 7fac1ca3db80 10 client.17286 fetch_fsmap learned FSMap version 55 2023-09-30T15:00:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.497+0000 7fac1ca3db80 1 -- 172.21.15.32:0/1782807902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55e0b7a00d80 con 0x55e0b7c83200 2023-09-30T15:00:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.498+0000 7fabf1ffb700 1 -- 172.21.15.32:0/1782807902 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e0b7c76090 con 0x55e0b7c83200 2023-09-30T15:00:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.499+0000 7fabf1ffb700 1 -- 172.21.15.32:0/1782807902 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fabfc01cb10 con 0x55e0b7c83200 2023-09-30T15:00:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.499+0000 7fac1ca3db80 10 client.17286 fetch_fsmap finished waiting for FSMap version 55 2023-09-30T15:00:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.499+0000 7fac1ca3db80 10 client.17286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:00:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.499+0000 7fac1ca3db80 20 client.17286 populate_metadata read hostname 'smithi032' 2023-09-30T15:00:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.499+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/1782807902 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e0b7c98d90 0x55e0b7c9b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.499+0000 7fac1ca3db80 4 client.17286 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T15:00:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.499+0000 7fac1ca3db80 1 -- 172.21.15.32:0/1782807902 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0b7bcedb0 con 0x55e0b7c98d90 2023-09-30T15:00:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.499+0000 7fac09d9b700 1 --2- 172.21.15.32:0/1782807902 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e0b7c98d90 0x55e0b7c9b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.499+0000 7fabf1ffb700 10 client.17286 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:00:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.499+0000 7fac09d9b700 1 --2- 172.21.15.32:0/1782807902 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e0b7c98d90 0x55e0b7c9b1b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f78383f7d8a18f8 server_cookie=5314b5629ccf31ca in_seq=0 out_seq=0 2023-09-30T15:00:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.502+0000 7fabf1ffb700 1 -- 172.21.15.32:0/1782807902 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e0b7bcedb0 con 0x55e0b7c98d90 2023-09-30T15:00:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.502+0000 7fabf1ffb700 10 client.17286 handle_command_reply: tid=0 2023-09-30T15:00:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.502+0000 7fac1ca3db80 1 client.17286 shutdown 2023-09-30T15:00:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.503+0000 7fac1ca3db80 1 -- 172.21.15.32:0/1782807902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c83200 msgr2=0x55e0b7c8cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.503+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/1782807902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c83200 0x55e0b7c8cf60 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x55e0b7c82fa0 tx=0x7fabfc0047c0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.503+0000 7fac1ca3db80 1 -- 172.21.15.32:0/1782807902 shutdown_connections 2023-09-30T15:00:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.503+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/1782807902 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e0b7c98d90 0x55e0b7c9b1b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.503+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/1782807902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c944e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.503+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/1782807902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c83200 0x55e0b7c8cf60 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.503+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/1782807902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c84310 0x55e0b7c8d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.503+0000 7fac1ca3db80 1 -- 172.21.15.32:0/1782807902 >> 172.21.15.32:0/1782807902 conn(0x55e0b7bf7200 msgr2=0x55e0b7c7d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.503+0000 7fabf1ffb700 0 client.17286 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:00:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.503+0000 7fac1ca3db80 1 -- 172.21.15.32:0/1782807902 shutdown_connections 2023-09-30T15:00:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.503+0000 7fac1ca3db80 1 -- 172.21.15.32:0/1782807902 wait complete. 2023-09-30T15:00:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.504+0000 7fac1ca3db80 20 client.17286 trim_cache size 0 max 16384 2023-09-30T15:00:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.522+0000 7fac1ca3db80 1 Processor -- start 2023-09-30T15:00:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.522+0000 7fac1ca3db80 1 -- start start 2023-09-30T15:00:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.522+0000 7fac1ca3db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c83200 0x55e0b7c87c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.522+0000 7fac1ca3db80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84310 0x55e0b7c944a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.522+0000 7fac1ca3db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c881c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.522+0000 7fac1ca3db80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b7c8cb30 con 0x55e0b7c84310 2023-09-30T15:00:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.522+0000 7fac1ca3db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b7a568c0 con 0x55e0b7c83200 2023-09-30T15:00:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.522+0000 7fac1ca3db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b7c961d0 con 0x55e0b7c84cd0 2023-09-30T15:00:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.523+0000 7fac0a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c83200 0x55e0b7c87c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.523+0000 7fac0a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c83200 0x55e0b7c87c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51482/0 (socket says 172.21.15.32:51482) 2023-09-30T15:00:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.523+0000 7fac0a59c700 1 -- 172.21.15.32:0/3059164798 learned_addr learned my addr 172.21.15.32:0/3059164798 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.523+0000 7fac0ad9d700 1 --2- 172.21.15.32:0/3059164798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c881c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.524+0000 7fac0ad9d700 1 -- 172.21.15.32:0/3059164798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c83200 msgr2=0x55e0b7c87c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.524+0000 7fac0ad9d700 1 --2- 172.21.15.32:0/3059164798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c83200 0x55e0b7c87c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.524+0000 7fac0ad9d700 1 -- 172.21.15.32:0/3059164798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84310 msgr2=0x55e0b7c944a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.524+0000 7fac0ad9d700 1 --2- 172.21.15.32:0/3059164798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84310 0x55e0b7c944a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.524+0000 7fac0ad9d700 1 -- 172.21.15.32:0/3059164798 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0b79e9c00 con 0x55e0b7c84cd0 2023-09-30T15:00:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.524+0000 7fac0ad9d700 1 --2- 172.21.15.32:0/3059164798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c881c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55e0b7c85290 tx=0x7fac040b9040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=973d664a231a17b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.524+0000 7fabf97fa700 1 -- 172.21.15.32:0/3059164798 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac04082ea0 con 0x55e0b7c84cd0 2023-09-30T15:00:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.524+0000 7fabf97fa700 1 -- 172.21.15.32:0/3059164798 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac0409d9c0 con 0x55e0b7c84cd0 2023-09-30T15:00:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.524+0000 7fabf97fa700 1 -- 172.21.15.32:0/3059164798 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0401b580 con 0x55e0b7c84cd0 2023-09-30T15:00:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.525+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3059164798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 msgr2=0x55e0b7c881c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.525+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/3059164798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c881c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55e0b7c85290 tx=0x7fac040b9040 comp rx=0 tx=0).stop 2023-09-30T15:00:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.525+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3059164798 shutdown_connections 2023-09-30T15:00:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.525+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/3059164798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c881c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.525+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/3059164798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c83200 0x55e0b7c87c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.525+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/3059164798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84310 0x55e0b7c944a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.525+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3059164798 >> 172.21.15.32:0/3059164798 conn(0x55e0b7bf7200 msgr2=0x55e0b7c8b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.525+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3059164798 shutdown_connections 2023-09-30T15:00:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.525+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3059164798 wait complete. 2023-09-30T15:00:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.525+0000 7fac1ca3db80 1 -- start start 2023-09-30T15:00:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.525+0000 7fac1ca3db80 1 Processor -- start 2023-09-30T15:00:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac1ca3db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c83200 0x55e0b7c92310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac1ca3db80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84310 0x55e0b7c92850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac1ca3db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c93860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac1ca3db80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b7c93da0 con 0x55e0b7c84310 2023-09-30T15:00:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac1ca3db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b7c93f10 con 0x55e0b7c83200 2023-09-30T15:00:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac1ca3db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b7c940d0 con 0x55e0b7c84cd0 2023-09-30T15:00:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac1ca3db80 10 client.0 _ll_register_callbacks cb 0x7fabfc00fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:00:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac0a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c83200 0x55e0b7c92310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac09d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84310 0x55e0b7c92850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac0a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c83200 0x55e0b7c92310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51488/0 (socket says 172.21.15.32:51488) 2023-09-30T15:00:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac0ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c93860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac0a59c700 1 -- 172.21.15.32:0/3234999244 learned_addr learned my addr 172.21.15.32:0/3234999244 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac0a59c700 1 -- 172.21.15.32:0/3234999244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 msgr2=0x55e0b7c93860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac0a59c700 1 --2- 172.21.15.32:0/3234999244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c93860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac0a59c700 1 -- 172.21.15.32:0/3234999244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84310 msgr2=0x55e0b7c92850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac0a59c700 1 --2- 172.21.15.32:0/3234999244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84310 0x55e0b7c92850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.526+0000 7fac0a59c700 1 -- 172.21.15.32:0/3234999244 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0b7c38870 con 0x55e0b7c83200 2023-09-30T15:00:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.527+0000 7fac09d9b700 1 --2- 172.21.15.32:0/3234999244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84310 0x55e0b7c92850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:00:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.527+0000 7fac0ad9d700 1 --2- 172.21.15.32:0/3234999244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c93860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:00:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.527+0000 7fac0a59c700 1 --2- 172.21.15.32:0/3234999244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c83200 0x55e0b7c92310 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fabfc01ce90 tx=0x7fabfc011e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc0fffca0c871c7c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.527+0000 7fabf1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T15:00:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.527+0000 7fabf1ffb700 1 -- 172.21.15.32:0/3234999244 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc022040 con 0x55e0b7c83200 2023-09-30T15:00:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.527+0000 7fabf1ffb700 1 -- 172.21.15.32:0/3234999244 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fabfc01c6f0 con 0x55e0b7c83200 2023-09-30T15:00:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.527+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3234999244 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac040991d0 con 0x55e0b7c83200 2023-09-30T15:00:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.527+0000 7fabf1ffb700 1 -- 172.21.15.32:0/3234999244 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabfc028e90 con 0x55e0b7c83200 2023-09-30T15:00:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.527+0000 7fac1ca3db80 10 client.26909 fetch_fsmap learned FSMap version 55 2023-09-30T15:00:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.527+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3234999244 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55e0b7a271f0 con 0x55e0b7c83200 2023-09-30T15:00:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.527+0000 7fabf1ffb700 1 -- 172.21.15.32:0/3234999244 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac040991d0 con 0x55e0b7c83200 2023-09-30T15:00:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.528+0000 7fabf1ffb700 1 -- 172.21.15.32:0/3234999244 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fabfc033b40 con 0x55e0b7c83200 2023-09-30T15:00:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.528+0000 7fac1ca3db80 10 client.26909 fetch_fsmap finished waiting for FSMap version 55 2023-09-30T15:00:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.528+0000 7fac1ca3db80 10 client.26909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:00:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.528+0000 7fac1ca3db80 20 client.26909 populate_metadata read hostname 'smithi032' 2023-09-30T15:00:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.528+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/3234999244 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e0b7cedbc0 0x55e0b7ceffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.528+0000 7fac1ca3db80 4 client.26909 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T15:00:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.528+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3234999244 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e0b7bcead0 con 0x55e0b7cedbc0 2023-09-30T15:00:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.528+0000 7fac09d9b700 1 --2- 172.21.15.32:0/3234999244 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e0b7cedbc0 0x55e0b7ceffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.528+0000 7fabf1ffb700 10 client.26909 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:00:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.528+0000 7fac09d9b700 1 --2- 172.21.15.32:0/3234999244 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e0b7cedbc0 0x55e0b7ceffe0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ccc978d84b3a3c8 server_cookie=b00b8802c9f19ba3 in_seq=0 out_seq=0 2023-09-30T15:00:33.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:33 smithi148 ceph-mon[102355]: pgmap v955: 97 pgs: 97 active+clean; 818 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.6 MiB/s wr, 760 op/s 2023-09-30T15:00:33.644 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:33 smithi032 ceph-mon[98592]: pgmap v955: 97 pgs: 97 active+clean; 818 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.6 MiB/s wr, 760 op/s 2023-09-30T15:00:33.807 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:33 smithi156 ceph-mon[103579]: pgmap v955: 97 pgs: 97 active+clean; 818 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.6 MiB/s wr, 760 op/s 2023-09-30T15:00:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.896+0000 7fabf1ffb700 1 -- 172.21.15.32:0/3234999244 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e0b7bcead0 con 0x55e0b7cedbc0 2023-09-30T15:00:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.896+0000 7fabf1ffb700 10 client.26909 handle_command_reply: tid=0 2023-09-30T15:00:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.896+0000 7fac1ca3db80 1 client.26909 shutdown 2023-09-30T15:00:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.897+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3234999244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c83200 msgr2=0x55e0b7c92310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.897+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/3234999244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c83200 0x55e0b7c92310 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fabfc01ce90 tx=0x7fabfc011e50 comp rx=0 tx=0).stop 2023-09-30T15:00:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.897+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3234999244 shutdown_connections 2023-09-30T15:00:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.897+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/3234999244 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e0b7cedbc0 0x55e0b7ceffe0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.897+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/3234999244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0b7c84cd0 0x55e0b7c93860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.897+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/3234999244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0b7c83200 0x55e0b7c92310 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.897+0000 7fac1ca3db80 1 --2- 172.21.15.32:0/3234999244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e0b7c84310 0x55e0b7c92850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.897+0000 7fabf1ffb700 0 client.26909 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:00:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.897+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3234999244 >> 172.21.15.32:0/3234999244 conn(0x55e0b7bf7200 msgr2=0x55e0b7c87c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.897+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3234999244 shutdown_connections 2023-09-30T15:00:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.897+0000 7fac1ca3db80 1 -- 172.21.15.32:0/3234999244 wait complete. 2023-09-30T15:00:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.898+0000 7fac1ca3db80 20 client.26909 trim_cache size 0 max 16384 2023-09-30T15:00:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T15:00:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T15:00:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T15:00:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T15:00:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.901+0000 7fac10855700 1 -- 172.21.15.32:0/792863234 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fabf40a0fa0 msgr2=0x7fabf40a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.901+0000 7fac10855700 1 --2- 172.21.15.32:0/792863234 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fabf40a0fa0 0x7fabf40a3460 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009fd0 tx=0x7fabfc009260 comp rx=0 tx=0).stop 2023-09-30T15:00:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.901+0000 7fac10855700 1 -- 172.21.15.32:0/792863234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac0c102970 msgr2=0x7fac0c197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.901+0000 7fac10855700 1 --2- 172.21.15.32:0/792863234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac0c102970 0x7fac0c197f70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fac0400efc0 tx=0x7fac0400c3a0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.901+0000 7fac10855700 1 -- 172.21.15.32:0/792863234 shutdown_connections 2023-09-30T15:00:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.901+0000 7fac10855700 1 --2- 172.21.15.32:0/792863234 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fabf40a0fa0 0x7fabf40a3460 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.901+0000 7fac10855700 1 --2- 172.21.15.32:0/792863234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac0c102970 0x7fac0c197f70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.901+0000 7fac10855700 1 --2- 172.21.15.32:0/792863234 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac0c100e60 0x7fac0c196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.901+0000 7fac10855700 1 --2- 172.21.15.32:0/792863234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac0c101fb0 0x7fac0c197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.901+0000 7fac10855700 1 -- 172.21.15.32:0/792863234 >> 172.21.15.32:0/792863234 conn(0x7fac0c0fc3f0 msgr2=0x7fac0c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.901+0000 7fac10855700 1 -- 172.21.15.32:0/792863234 shutdown_connections 2023-09-30T15:00:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:33.901+0000 7fac10855700 1 -- 172.21.15.32:0/792863234 wait complete. 2023-09-30T15:00:34.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 357845 35.32 MB/sec execute 427 sec latency 266.259 ms 2023-09-30T15:00:35.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 358423 35.33 MB/sec execute 428 sec latency 72.676 ms 2023-09-30T15:00:35.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:35 smithi148 ceph-mon[102355]: pgmap v956: 97 pgs: 97 active+clean; 821 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.3 MiB/s wr, 569 op/s 2023-09-30T15:00:35.639 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:35 smithi032 ceph-mon[98592]: pgmap v956: 97 pgs: 97 active+clean; 821 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.3 MiB/s wr, 569 op/s 2023-09-30T15:00:35.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:35 smithi156 ceph-mon[103579]: pgmap v956: 97 pgs: 97 active+clean; 821 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.3 MiB/s wr, 569 op/s 2023-09-30T15:00:36.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 359023 35.29 MB/sec execute 429 sec latency 74.923 ms 2023-09-30T15:00:37.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 359475 35.27 MB/sec execute 430 sec latency 267.108 ms 2023-09-30T15:00:37.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:37 smithi148 ceph-mon[102355]: pgmap v957: 97 pgs: 97 active+clean; 821 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.5 MiB/s wr, 653 op/s 2023-09-30T15:00:37.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:37 smithi032 ceph-mon[98592]: pgmap v957: 97 pgs: 97 active+clean; 821 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.5 MiB/s wr, 653 op/s 2023-09-30T15:00:37.810 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:37 smithi156 ceph-mon[103579]: pgmap v957: 97 pgs: 97 active+clean; 821 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.5 MiB/s wr, 653 op/s 2023-09-30T15:00:38.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 360068 35.27 MB/sec execute 431 sec latency 137.451 ms 2023-09-30T15:00:39.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 360783 35.28 MB/sec execute 432 sec latency 52.643 ms 2023-09-30T15:00:39.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:39 smithi032 ceph-mon[98592]: pgmap v958: 97 pgs: 97 active+clean; 826 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.1 MiB/s wr, 756 op/s 2023-09-30T15:00:39.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:39 smithi156 ceph-mon[103579]: pgmap v958: 97 pgs: 97 active+clean; 826 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.1 MiB/s wr, 756 op/s 2023-09-30T15:00:39.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:39 smithi148 ceph-mon[102355]: pgmap v958: 97 pgs: 97 active+clean; 826 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 9.1 MiB/s wr, 756 op/s 2023-09-30T15:00:40.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 361387 35.28 MB/sec execute 433 sec latency 61.684 ms 2023-09-30T15:00:41.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 361897 35.26 MB/sec execute 434 sec latency 77.714 ms 2023-09-30T15:00:41.611 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:41 smithi148 ceph-mon[102355]: pgmap v959: 97 pgs: 97 active+clean; 827 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.2 MiB/s wr, 689 op/s 2023-09-30T15:00:41.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:41 smithi032 ceph-mon[98592]: pgmap v959: 97 pgs: 97 active+clean; 827 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.2 MiB/s wr, 689 op/s 2023-09-30T15:00:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:41 smithi156 ceph-mon[103579]: pgmap v959: 97 pgs: 97 active+clean; 827 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.2 MiB/s wr, 689 op/s 2023-09-30T15:00:42.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 362299 35.21 MB/sec execute 435 sec latency 254.630 ms 2023-09-30T15:00:43.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 362711 35.18 MB/sec execute 436 sec latency 281.337 ms 2023-09-30T15:00:43.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:43 smithi032 ceph-mon[98592]: pgmap v960: 97 pgs: 97 active+clean; 827 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.4 MiB/s wr, 745 op/s 2023-09-30T15:00:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:43 smithi156 ceph-mon[103579]: pgmap v960: 97 pgs: 97 active+clean; 827 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.4 MiB/s wr, 745 op/s 2023-09-30T15:00:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:43 smithi148 ceph-mon[102355]: pgmap v960: 97 pgs: 97 active+clean; 827 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.4 MiB/s wr, 745 op/s 2023-09-30T15:00:44.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 363581 35.21 MB/sec execute 437 sec latency 85.300 ms 2023-09-30T15:00:45.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 364324 35.24 MB/sec execute 438 sec latency 180.096 ms 2023-09-30T15:00:45.640 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:45 smithi032 ceph-mon[98592]: pgmap v961: 97 pgs: 97 active+clean; 822 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.4 MiB/s wr, 676 op/s 2023-09-30T15:00:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:45 smithi156 ceph-mon[103579]: pgmap v961: 97 pgs: 97 active+clean; 822 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.4 MiB/s wr, 676 op/s 2023-09-30T15:00:45.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:45 smithi148 ceph-mon[102355]: pgmap v961: 97 pgs: 97 active+clean; 822 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.4 MiB/s wr, 676 op/s 2023-09-30T15:00:46.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 365017 35.25 MB/sec execute 439 sec latency 57.045 ms 2023-09-30T15:00:47.082 INFO:tasks.workunit.client.0.smithi032.stdout: 10 365487 35.22 MB/sec execute 440 sec latency 140.887 ms 2023-09-30T15:00:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.112+0000 7fd7d8200700 1 -- 172.21.15.32:0/455386664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d010a790 msgr2=0x7fd7d010cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.112+0000 7fd7d8200700 1 --2- 172.21.15.32:0/455386664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d010a790 0x7fd7d010cc20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc01a9a0 tx=0x7fd7cc01acb0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.112+0000 7fd7d8200700 1 -- 172.21.15.32:0/455386664 shutdown_connections 2023-09-30T15:00:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.112+0000 7fd7d8200700 1 --2- 172.21.15.32:0/455386664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d010d160 0x7fd7d010d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.112+0000 7fd7d8200700 1 --2- 172.21.15.32:0/455386664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d0107e20 0x7fd7d010a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.112+0000 7fd7d8200700 1 --2- 172.21.15.32:0/455386664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d010a790 0x7fd7d010cc20 secure :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc01a9a0 tx=0x7fd7cc01acb0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.112+0000 7fd7d8200700 1 -- 172.21.15.32:0/455386664 >> 172.21.15.32:0/455386664 conn(0x7fd7d006dc80 msgr2=0x7fd7d00700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.122+0000 7fd7d8200700 1 -- 172.21.15.32:0/455386664 shutdown_connections 2023-09-30T15:00:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.122+0000 7fd7d8200700 1 -- 172.21.15.32:0/455386664 wait complete. 2023-09-30T15:00:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.124+0000 7fd7d8200700 1 Processor -- start 2023-09-30T15:00:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.124+0000 7fd7d8200700 1 -- start start 2023-09-30T15:00:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.124+0000 7fd7d8200700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d0107e20 0x7fd7d019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.124+0000 7fd7d8200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d010d160 0x7fd7d019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.124+0000 7fd7d8200700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d019e990 0x7fd7d019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.124+0000 7fd7d8200700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d019f3e0 con 0x7fd7d019e990 2023-09-30T15:00:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.124+0000 7fd7d8200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d019f550 con 0x7fd7d010d160 2023-09-30T15:00:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.124+0000 7fd7d8200700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d019f710 con 0x7fd7d0107e20 2023-09-30T15:00:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.125+0000 7fd7d5f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d0107e20 0x7fd7d019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.127+0000 7fd7d579b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d010d160 0x7fd7d019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.127+0000 7fd7d679d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d019e990 0x7fd7d019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.127+0000 7fd7d679d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d019e990 0x7fd7d019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40026/0 (socket says 172.21.15.32:40026) 2023-09-30T15:00:47.131 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.127+0000 7fd7d679d700 1 -- 172.21.15.32:0/3095403129 learned_addr learned my addr 172.21.15.32:0/3095403129 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.131+0000 7fd7d679d700 1 -- 172.21.15.32:0/3095403129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d0107e20 msgr2=0x7fd7d019d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.131+0000 7fd7d679d700 1 --2- 172.21.15.32:0/3095403129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d0107e20 0x7fd7d019d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.131+0000 7fd7d679d700 1 -- 172.21.15.32:0/3095403129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d010d160 msgr2=0x7fd7d019db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.131+0000 7fd7d679d700 1 --2- 172.21.15.32:0/3095403129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d010d160 0x7fd7d019db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.131+0000 7fd7d679d700 1 -- 172.21.15.32:0/3095403129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc01a600 con 0x7fd7d019e990 2023-09-30T15:00:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.131+0000 7fd7d679d700 1 --2- 172.21.15.32:0/3095403129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d019e990 0x7fd7d019ee10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800b780 tx=0x7fd7c800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.132+0000 7fd7c6ffd700 1 -- 172.21.15.32:0/3095403129 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8010860 con 0x7fd7d019e990 2023-09-30T15:00:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.132+0000 7fd7c6ffd700 1 -- 172.21.15.32:0/3095403129 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7c8010e80 con 0x7fd7d019e990 2023-09-30T15:00:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.132+0000 7fd7c6ffd700 1 -- 172.21.15.32:0/3095403129 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8007370 con 0x7fd7d019e990 2023-09-30T15:00:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.132+0000 7fd7d8200700 1 -- 172.21.15.32:0/3095403129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d01a5c40 con 0x7fd7d019e990 2023-09-30T15:00:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.132+0000 7fd7d8200700 1 -- 172.21.15.32:0/3095403129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d01a6100 con 0x7fd7d019e990 2023-09-30T15:00:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.138+0000 7fd7e43e8b80 1 Processor -- start 2023-09-30T15:00:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.139+0000 7fd7e43e8b80 1 -- start start 2023-09-30T15:00:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.139+0000 7fd7e43e8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0ae200 0x5558dd0ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.139+0000 7fd7e43e8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0af310 0x5558dd0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.139+0000 7fd7e43e8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 0x5558dd0b6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.139+0000 7fd7e43e8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558dd016400 con 0x5558dd0af310 2023-09-30T15:00:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.139+0000 7fd7e43e8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558dd007bc0 con 0x5558dd0ae200 2023-09-30T15:00:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.139+0000 7fd7e43e8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558dd0054e0 con 0x5558dd0afcd0 2023-09-30T15:00:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.140+0000 7fd7d579b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0ae200 0x5558dd0ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.140+0000 7fd7d579b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 0x5558dd0b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.146+0000 7fd7d579b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 0x5558dd0b6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54418/0 (socket says 172.21.15.32:54418) 2023-09-30T15:00:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.146+0000 7fd7d579b700 1 -- 172.21.15.32:0/2594452419 learned_addr learned my addr 172.21.15.32:0/2594452419 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.147+0000 7fd7d579b700 1 -- 172.21.15.32:0/2594452419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0ae200 msgr2=0x5558dd0ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.147+0000 7fd7d579b700 1 --2- 172.21.15.32:0/2594452419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0ae200 0x5558dd0ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.147+0000 7fd7d579b700 1 -- 172.21.15.32:0/2594452419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0af310 msgr2=0x5558dd0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.147+0000 7fd7d579b700 1 --2- 172.21.15.32:0/2594452419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0af310 0x5558dd0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.147+0000 7fd7d579b700 1 -- 172.21.15.32:0/2594452419 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558dd063870 con 0x5558dd0afcd0 2023-09-30T15:00:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.147+0000 7fd7d579b700 1 --2- 172.21.15.32:0/2594452419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 0x5558dd0b6560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc01af50 tx=0x7fd7cc01acb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50e5ca2af8e1cb02 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.152+0000 7fd7bbfff700 1 -- 172.21.15.32:0/2594452419 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc01cc70 con 0x5558dd0afcd0 2023-09-30T15:00:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.152+0000 7fd7bbfff700 1 -- 172.21.15.32:0/2594452419 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7cc02e070 con 0x5558dd0afcd0 2023-09-30T15:00:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.152+0000 7fd7bbfff700 1 -- 172.21.15.32:0/2594452419 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc004a80 con 0x5558dd0afcd0 2023-09-30T15:00:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.153+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/2594452419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 msgr2=0x5558dd0b6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.153+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/2594452419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 0x5558dd0b6560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc01af50 tx=0x7fd7cc01acb0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.153+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/2594452419 shutdown_connections 2023-09-30T15:00:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.153+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/2594452419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 0x5558dd0b6560 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.153+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/2594452419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0ae200 0x5558dd0ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.153+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/2594452419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0af310 0x5558dd0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.153+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/2594452419 >> 172.21.15.32:0/2594452419 conn(0x5558dd022200 msgr2=0x5558dd0a8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.161+0000 7fd7c6ffd700 1 -- 172.21.15.32:0/3095403129 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fd7c80109c0 con 0x7fd7d019e990 2023-09-30T15:00:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.162+0000 7fd7c6ffd700 1 --2- 172.21.15.32:0/3095403129 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd7bc0a10b0 0x7fd7bc0a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.162+0000 7fd7c6ffd700 1 -- 172.21.15.32:0/3095403129 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd7c80cc470 con 0x7fd7d019e990 2023-09-30T15:00:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.164+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/2594452419 shutdown_connections 2023-09-30T15:00:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.165+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/2594452419 wait complete. 2023-09-30T15:00:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.165+0000 7fd7e43e8b80 1 -- start start 2023-09-30T15:00:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.165+0000 7fd7e43e8b80 1 Processor -- start 2023-09-30T15:00:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.166+0000 7fd7e43e8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 0x5558dd0b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.166+0000 7fd7e43e8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 0x5558dd0b8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.166+0000 7fd7e43e8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0bd5c0 0x5558dd0bf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.166+0000 7fd7e43e8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558dd000dd0 con 0x5558dd0ae200 2023-09-30T15:00:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.166+0000 7fd7e43e8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558dcf9d6b0 con 0x5558dd0bd5c0 2023-09-30T15:00:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.166+0000 7fd7e43e8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558dcf9d350 con 0x5558dd0afcd0 2023-09-30T15:00:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.166+0000 7fd7e43e8b80 10 client.0 _ll_register_callbacks cb 0x5558dcdfa7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:00:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.167+0000 7fd7d679d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0bd5c0 0x5558dd0bf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.167+0000 7fd7d679d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0bd5c0 0x5558dd0bf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51520/0 (socket says 172.21.15.32:51520) 2023-09-30T15:00:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.167+0000 7fd7d679d700 1 -- 172.21.15.32:0/1431340389 learned_addr learned my addr 172.21.15.32:0/1431340389 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.167+0000 7fd7d5f9c700 1 --2- 172.21.15.32:0/1431340389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 0x5558dd0b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.167+0000 7fd7d579b700 1 --2- 172.21.15.32:0/1431340389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 0x5558dd0b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.168+0000 7fd7d679d700 1 -- 172.21.15.32:0/1431340389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 msgr2=0x5558dd0b8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.168+0000 7fd7d679d700 1 --2- 172.21.15.32:0/1431340389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 0x5558dd0b8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.168+0000 7fd7d679d700 1 -- 172.21.15.32:0/1431340389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 msgr2=0x5558dd0b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.168+0000 7fd7d679d700 1 --2- 172.21.15.32:0/1431340389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 0x5558dd0b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.168+0000 7fd7d679d700 1 -- 172.21.15.32:0/1431340389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc030040 con 0x5558dd0bd5c0 2023-09-30T15:00:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.171+0000 7fd7d679d700 1 --2- 172.21.15.32:0/1431340389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0bd5c0 0x5558dd0bf9b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8095de0 tx=0x7fd7c809a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cd5fa2594dd8948 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.171+0000 7fd7b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T15:00:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.172+0000 7fd7b97fa700 1 -- 172.21.15.32:0/1431340389 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c809da30 con 0x5558dd0bd5c0 2023-09-30T15:00:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.172+0000 7fd7b97fa700 1 -- 172.21.15.32:0/1431340389 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd7c80a1c70 con 0x5558dd0bd5c0 2023-09-30T15:00:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.172+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/1431340389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558dd0a1090 con 0x5558dd0bd5c0 2023-09-30T15:00:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.172+0000 7fd7b97fa700 1 -- 172.21.15.32:0/1431340389 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7c80ca030 con 0x5558dd0bd5c0 2023-09-30T15:00:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.172+0000 7fd7e43e8b80 10 client.26918 fetch_fsmap learned FSMap version 55 2023-09-30T15:00:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.172+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/1431340389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x5558dce2bd80 con 0x5558dd0bd5c0 2023-09-30T15:00:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.174+0000 7fd7b97fa700 1 -- 172.21.15.32:0/1431340389 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5558dd0a1090 con 0x5558dd0bd5c0 2023-09-30T15:00:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.174+0000 7fd7b97fa700 1 -- 172.21.15.32:0/1431340389 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd7c800f3b0 con 0x5558dd0bd5c0 2023-09-30T15:00:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.174+0000 7fd7e43e8b80 10 client.26918 fetch_fsmap finished waiting for FSMap version 55 2023-09-30T15:00:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.174+0000 7fd7e43e8b80 10 client.26918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:00:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.174+0000 7fd7e43e8b80 20 client.26918 populate_metadata read hostname 'smithi032' 2023-09-30T15:00:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.174+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/1431340389 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5558dd0c4140 0x5558dd0c6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.174+0000 7fd7e43e8b80 4 client.26918 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T15:00:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.174+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/1431340389 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558dcff9db0 con 0x5558dd0c4140 2023-09-30T15:00:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.175+0000 7fd7d5f9c700 1 --2- 172.21.15.32:0/3095403129 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd7bc0a10b0 0x7fd7bc0a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.176+0000 7fd7d5f9c700 1 --2- 172.21.15.32:0/1431340389 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5558dd0c4140 0x5558dd0c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.186+0000 7fd7d5f9c700 1 --2- 172.21.15.32:0/3095403129 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd7bc0a10b0 0x7fd7bc0a3570 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x5558dd0aaad0 tx=0x7fd7c000c1a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.197+0000 7fd7b97fa700 10 client.26918 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:00:47.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.197+0000 7fd7d5f9c700 1 --2- 172.21.15.32:0/1431340389 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5558dd0c4140 0x5558dd0c6560 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78553ebe67c1623e server_cookie=5b3146291326a6fd in_seq=0 out_seq=0 2023-09-30T15:00:47.207 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.206+0000 7fd7b97fa700 1 -- 172.21.15.32:0/1431340389 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5558dcff9db0 con 0x5558dd0c4140 2023-09-30T15:00:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.206+0000 7fd7b97fa700 10 client.26918 handle_command_reply: tid=0 2023-09-30T15:00:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.206+0000 7fd7e43e8b80 1 client.26918 shutdown 2023-09-30T15:00:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.206+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/1431340389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0bd5c0 msgr2=0x5558dd0bf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.206+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/1431340389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0bd5c0 0x5558dd0bf9b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8095de0 tx=0x7fd7c809a110 comp rx=0 tx=0).stop 2023-09-30T15:00:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.206+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/1431340389 shutdown_connections 2023-09-30T15:00:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.206+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/1431340389 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5558dd0c4140 0x5558dd0c6560 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.206+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/1431340389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0bd5c0 0x5558dd0bf9b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.206+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/1431340389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 0x5558dd0b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.206+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/1431340389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 0x5558dd0b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.206+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/1431340389 >> 172.21.15.32:0/1431340389 conn(0x5558dd022200 msgr2=0x5558dd0a8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.206+0000 7fd7b97fa700 0 client.26918 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:00:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.206+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/1431340389 shutdown_connections 2023-09-30T15:00:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.206+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/1431340389 wait complete. 2023-09-30T15:00:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.206+0000 7fd7e43e8b80 20 client.26918 trim_cache size 0 max 16384 2023-09-30T15:00:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.228+0000 7fd7e43e8b80 1 Processor -- start 2023-09-30T15:00:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.228+0000 7fd7e43e8b80 1 -- start start 2023-09-30T15:00:47.230 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7e43e8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 0x5558dd0d3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7e43e8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 0x5558dd0b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7e43e8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0bd5c0 0x5558dd0b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7e43e8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558dce818c0 con 0x5558dd0ae200 2023-09-30T15:00:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7e43e8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558dd0a3680 con 0x5558dd0bd5c0 2023-09-30T15:00:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7e43e8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558dd0b0170 con 0x5558dd0afcd0 2023-09-30T15:00:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7d5f9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 0x5558dd0d3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7d579b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 0x5558dd0b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7d5f9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 0x5558dd0d3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40050/0 (socket says 172.21.15.32:40050) 2023-09-30T15:00:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7d5f9c700 1 -- 172.21.15.32:0/478104346 learned_addr learned my addr 172.21.15.32:0/478104346 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7d679d700 1 --2- 172.21.15.32:0/478104346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0bd5c0 0x5558dd0b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7d5f9c700 1 -- 172.21.15.32:0/478104346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 msgr2=0x5558dd0b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7d5f9c700 1 --2- 172.21.15.32:0/478104346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 0x5558dd0b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7d5f9c700 1 -- 172.21.15.32:0/478104346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0bd5c0 msgr2=0x5558dd0b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7d5f9c700 1 --2- 172.21.15.32:0/478104346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0bd5c0 0x5558dd0b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7d5f9c700 1 -- 172.21.15.32:0/478104346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558dce14c00 con 0x5558dd0ae200 2023-09-30T15:00:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7d579b700 1 --2- 172.21.15.32:0/478104346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 0x5558dd0b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:00:47.232 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.229+0000 7fd7d5f9c700 1 --2- 172.21.15.32:0/478104346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 0x5558dd0d3d60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000c670 tx=0x7fd7c001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e8782c08b2921a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.230+0000 7fd7c4ff9700 1 -- 172.21.15.32:0/478104346 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0025d50 con 0x5558dd0ae200 2023-09-30T15:00:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.230+0000 7fd7c4ff9700 1 -- 172.21.15.32:0/478104346 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7c0004020 con 0x5558dd0ae200 2023-09-30T15:00:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.230+0000 7fd7c4ff9700 1 -- 172.21.15.32:0/478104346 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0003980 con 0x5558dd0ae200 2023-09-30T15:00:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.230+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/478104346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 msgr2=0x5558dd0d3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.230+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/478104346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 0x5558dd0d3d60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000c670 tx=0x7fd7c001b000 comp rx=0 tx=0).stop 2023-09-30T15:00:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.231+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/478104346 shutdown_connections 2023-09-30T15:00:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.231+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/478104346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0bd5c0 0x5558dd0b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.231+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/478104346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 0x5558dd0d3d60 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.231+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/478104346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0afcd0 0x5558dd0b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.231+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/478104346 >> 172.21.15.32:0/478104346 conn(0x5558dd022200 msgr2=0x5558dd0b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.231+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/478104346 shutdown_connections 2023-09-30T15:00:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.231+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/478104346 wait complete. 2023-09-30T15:00:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.231+0000 7fd7e43e8b80 1 -- start start 2023-09-30T15:00:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.231+0000 7fd7e43e8b80 1 Processor -- start 2023-09-30T15:00:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.232+0000 7fd7e43e8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 0x5558dd0d3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.232+0000 7fd7e43e8b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0afcd0 0x5558dd0d38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.236 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.232+0000 7fd7e43e8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0bd5c0 0x5558dd116cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.232+0000 7fd7e43e8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558dd1171f0 con 0x5558dd0ae200 2023-09-30T15:00:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.232+0000 7fd7e43e8b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558dd117360 con 0x5558dd0afcd0 2023-09-30T15:00:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.232+0000 7fd7e43e8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558dd117520 con 0x5558dd0bd5c0 2023-09-30T15:00:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.232+0000 7fd7e43e8b80 10 client.0 _ll_register_callbacks cb 0x7fd7c8099420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:00:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.232+0000 7fd7d679d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0bd5c0 0x5558dd116cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.232+0000 7fd7d679d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0bd5c0 0x5558dd116cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54438/0 (socket says 172.21.15.32:54438) 2023-09-30T15:00:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.233+0000 7fd7d679d700 1 -- 172.21.15.32:0/3042057638 learned_addr learned my addr 172.21.15.32:0/3042057638 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:00:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.233+0000 7fd7d579b700 1 --2- 172.21.15.32:0/3042057638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0afcd0 0x5558dd0d38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.233+0000 7fd7d5f9c700 1 --2- 172.21.15.32:0/3042057638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 0x5558dd0d3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.233+0000 7fd7d679d700 1 -- 172.21.15.32:0/3042057638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0afcd0 msgr2=0x5558dd0d38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:47.237 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.233+0000 7fd7d679d700 1 --2- 172.21.15.32:0/3042057638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0afcd0 0x5558dd0d38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.233+0000 7fd7d679d700 1 -- 172.21.15.32:0/3042057638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 msgr2=0x5558dd0d3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:00:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.233+0000 7fd7d679d700 1 --2- 172.21.15.32:0/3042057638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 0x5558dd0d3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.233+0000 7fd7d679d700 1 -- 172.21.15.32:0/3042057638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc03c040 con 0x5558dd0bd5c0 2023-09-30T15:00:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.233+0000 7fd7d5f9c700 1 --2- 172.21.15.32:0/3042057638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 0x5558dd0d3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T15:00:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.233+0000 7fd7d679d700 1 --2- 172.21.15.32:0/3042057638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0bd5c0 0x5558dd116cb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800f6e0 tx=0x7fd7c80996b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db6f87628087fb8c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:00:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.233+0000 7fd7b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T15:00:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.234+0000 7fd7b97fa700 1 -- 172.21.15.32:0/3042057638 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80a1c70 con 0x5558dd0bd5c0 2023-09-30T15:00:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.234+0000 7fd7b97fa700 1 -- 172.21.15.32:0/3042057638 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd7c80956d0 con 0x5558dd0bd5c0 2023-09-30T15:00:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.234+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/3042057638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7cc02e070 con 0x5558dd0bd5c0 2023-09-30T15:00:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.234+0000 7fd7b97fa700 1 -- 172.21.15.32:0/3042057638 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7c809da30 con 0x5558dd0bd5c0 2023-09-30T15:00:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.234+0000 7fd7e43e8b80 10 client.26959 fetch_fsmap learned FSMap version 55 2023-09-30T15:00:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.234+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/3042057638 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7fd7c000e040 con 0x5558dd0bd5c0 2023-09-30T15:00:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.234+0000 7fd7b97fa700 1 -- 172.21.15.32:0/3042057638 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7cc02e070 con 0x5558dd0bd5c0 2023-09-30T15:00:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.234+0000 7fd7b97fa700 1 -- 172.21.15.32:0/3042057638 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd7c8098dd0 con 0x5558dd0bd5c0 2023-09-30T15:00:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.234+0000 7fd7e43e8b80 10 client.26959 fetch_fsmap finished waiting for FSMap version 55 2023-09-30T15:00:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.234+0000 7fd7e43e8b80 10 client.26959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:00:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.234+0000 7fd7e43e8b80 20 client.26959 populate_metadata read hostname 'smithi032' 2023-09-30T15:00:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.234+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/3042057638 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5558dd118b20 0x5558dd11af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:00:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.234+0000 7fd7e43e8b80 4 client.26959 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T15:00:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.234+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/3042057638 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5558dcff9ad0 con 0x5558dd118b20 2023-09-30T15:00:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.235+0000 7fd7d5f9c700 1 --2- 172.21.15.32:0/3042057638 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5558dd118b20 0x5558dd11af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:00:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.236+0000 7fd7d5f9c700 1 --2- 172.21.15.32:0/3042057638 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5558dd118b20 0x5558dd11af40 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddc5bfe9cc61ab9d server_cookie=f58e2ea33cfaf3c7 in_seq=0 out_seq=0 2023-09-30T15:00:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.236+0000 7fd7b97fa700 10 client.26959 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:00:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.653+0000 7fd7b97fa700 1 -- 172.21.15.32:0/3042057638 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5558dcff9ad0 con 0x5558dd118b20 2023-09-30T15:00:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.653+0000 7fd7b97fa700 10 client.26959 handle_command_reply: tid=0 2023-09-30T15:00:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.653+0000 7fd7e43e8b80 1 client.26959 shutdown 2023-09-30T15:00:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.653+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/3042057638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0bd5c0 msgr2=0x5558dd116cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.653+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/3042057638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0bd5c0 0x5558dd116cb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800f6e0 tx=0x7fd7c80996b0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.653+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/3042057638 shutdown_connections 2023-09-30T15:00:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.653+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/3042057638 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5558dd118b20 0x5558dd11af40 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.653+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/3042057638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558dd0bd5c0 0x5558dd116cb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.653+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/3042057638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5558dd0ae200 0x5558dd0d3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.653+0000 7fd7e43e8b80 1 --2- 172.21.15.32:0/3042057638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5558dd0afcd0 0x5558dd0d38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.653+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/3042057638 >> 172.21.15.32:0/3042057638 conn(0x5558dd022200 msgr2=0x5558dd0b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.654+0000 7fd7b97fa700 0 client.26959 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:00:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.654+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/3042057638 shutdown_connections 2023-09-30T15:00:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.654+0000 7fd7e43e8b80 1 -- 172.21.15.32:0/3042057638 wait complete. 2023-09-30T15:00:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.656+0000 7fd7e43e8b80 20 client.26959 trim_cache size 0 max 16384 2023-09-30T15:00:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T15:00:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T15:00:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T15:00:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T15:00:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.656+0000 7fd7d8200700 1 -- 172.21.15.32:0/3095403129 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd7bc0a10b0 msgr2=0x7fd7bc0a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.656+0000 7fd7d8200700 1 --2- 172.21.15.32:0/3095403129 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd7bc0a10b0 0x7fd7bc0a3570 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x5558dd0aaad0 tx=0x7fd7c000c1a0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.656+0000 7fd7d8200700 1 -- 172.21.15.32:0/3095403129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d019e990 msgr2=0x7fd7d019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:00:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.656+0000 7fd7d8200700 1 --2- 172.21.15.32:0/3095403129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d019e990 0x7fd7d019ee10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800b780 tx=0x7fd7c800ba90 comp rx=0 tx=0).stop 2023-09-30T15:00:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.656+0000 7fd7d8200700 1 -- 172.21.15.32:0/3095403129 shutdown_connections 2023-09-30T15:00:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.656+0000 7fd7d8200700 1 --2- 172.21.15.32:0/3095403129 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd7bc0a10b0 0x7fd7bc0a3570 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.656+0000 7fd7d8200700 1 --2- 172.21.15.32:0/3095403129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d019e990 0x7fd7d019ee10 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.656+0000 7fd7d8200700 1 --2- 172.21.15.32:0/3095403129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d0107e20 0x7fd7d019d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.656+0000 7fd7d8200700 1 --2- 172.21.15.32:0/3095403129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d010d160 0x7fd7d019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:00:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.656+0000 7fd7d8200700 1 -- 172.21.15.32:0/3095403129 >> 172.21.15.32:0/3095403129 conn(0x7fd7d006dc80 msgr2=0x7fd7d010b350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:00:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.657+0000 7fd7d8200700 1 -- 172.21.15.32:0/3095403129 shutdown_connections 2023-09-30T15:00:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:00:47.657+0000 7fd7d8200700 1 -- 172.21.15.32:0/3095403129 wait complete. 2023-09-30T15:00:47.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:47 smithi156 ceph-mon[103579]: pgmap v962: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.7 MiB/s wr, 688 op/s 2023-09-30T15:00:47.862 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:47 smithi148 ceph-mon[102355]: pgmap v962: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.7 MiB/s wr, 688 op/s 2023-09-30T15:00:47.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:47 smithi032 ceph-mon[98592]: pgmap v962: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.7 MiB/s wr, 688 op/s 2023-09-30T15:00:48.083 INFO:tasks.workunit.client.0.smithi032.stdout: 10 365688 35.16 MB/sec execute 441 sec latency 255.828 ms 2023-09-30T15:00:49.083 INFO:tasks.workunit.client.0.smithi032.stdout: 10 366515 35.18 MB/sec execute 442 sec latency 65.185 ms 2023-09-30T15:00:49.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:49 smithi156 ceph-mon[103579]: pgmap v963: 97 pgs: 97 active+clean; 815 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 688 op/s 2023-09-30T15:00:49.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:49 smithi148 ceph-mon[102355]: pgmap v963: 97 pgs: 97 active+clean; 815 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 688 op/s 2023-09-30T15:00:49.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:49 smithi032 ceph-mon[98592]: pgmap v963: 97 pgs: 97 active+clean; 815 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 688 op/s 2023-09-30T15:00:50.083 INFO:tasks.workunit.client.0.smithi032.stdout: 10 367283 35.19 MB/sec execute 443 sec latency 54.107 ms 2023-09-30T15:00:51.083 INFO:tasks.workunit.client.0.smithi032.stdout: 10 367972 35.19 MB/sec execute 444 sec latency 83.216 ms 2023-09-30T15:00:51.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:51 smithi156 ceph-mon[103579]: pgmap v964: 97 pgs: 97 active+clean; 811 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.4 MiB/s wr, 604 op/s 2023-09-30T15:00:51.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:51 smithi148 ceph-mon[102355]: pgmap v964: 97 pgs: 97 active+clean; 811 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.4 MiB/s wr, 604 op/s 2023-09-30T15:00:51.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:51 smithi032 ceph-mon[98592]: pgmap v964: 97 pgs: 97 active+clean; 811 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.4 MiB/s wr, 604 op/s 2023-09-30T15:00:52.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 368633 35.19 MB/sec execute 445 sec latency 71.540 ms 2023-09-30T15:00:53.086 INFO:tasks.workunit.client.0.smithi032.stdout: 10 369014 35.16 MB/sec execute 446 sec latency 215.147 ms 2023-09-30T15:00:53.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:53 smithi156 ceph-mon[103579]: pgmap v965: 97 pgs: 97 active+clean; 816 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.7 MiB/s wr, 655 op/s 2023-09-30T15:00:53.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:53 smithi148 ceph-mon[102355]: pgmap v965: 97 pgs: 97 active+clean; 816 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.7 MiB/s wr, 655 op/s 2023-09-30T15:00:53.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:53 smithi032 ceph-mon[98592]: pgmap v965: 97 pgs: 97 active+clean; 816 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.7 MiB/s wr, 655 op/s 2023-09-30T15:00:54.086 INFO:tasks.workunit.client.0.smithi032.stdout: 10 369835 35.19 MB/sec execute 447 sec latency 53.734 ms 2023-09-30T15:00:55.086 INFO:tasks.workunit.client.0.smithi032.stdout: 10 370541 35.19 MB/sec execute 448 sec latency 58.212 ms 2023-09-30T15:00:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:55 smithi156 ceph-mon[103579]: pgmap v966: 97 pgs: 97 active+clean; 793 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.9 MiB/s wr, 652 op/s 2023-09-30T15:00:55.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:55 smithi148 ceph-mon[102355]: pgmap v966: 97 pgs: 97 active+clean; 793 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.9 MiB/s wr, 652 op/s 2023-09-30T15:00:55.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:55 smithi032 ceph-mon[98592]: pgmap v966: 97 pgs: 97 active+clean; 793 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.9 MiB/s wr, 652 op/s 2023-09-30T15:00:56.086 INFO:tasks.workunit.client.0.smithi032.stdout: 10 371187 35.19 MB/sec execute 449 sec latency 77.662 ms 2023-09-30T15:00:56.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:56 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T15:00:56.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:56 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T15:00:56.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:56 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T15:00:57.086 INFO:tasks.workunit.client.0.smithi032.stdout: 10 371759 35.17 MB/sec execute 450 sec latency 146.623 ms 2023-09-30T15:00:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:57 smithi156 ceph-mon[103579]: pgmap v967: 97 pgs: 97 active+clean; 798 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.2 MiB/s wr, 682 op/s 2023-09-30T15:00:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:57 smithi148 ceph-mon[102355]: pgmap v967: 97 pgs: 97 active+clean; 798 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.2 MiB/s wr, 682 op/s 2023-09-30T15:00:57.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:57 smithi032 ceph-mon[98592]: pgmap v967: 97 pgs: 97 active+clean; 798 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.2 MiB/s wr, 682 op/s 2023-09-30T15:00:58.086 INFO:tasks.workunit.client.0.smithi032.stdout: 10 372071 35.14 MB/sec execute 451 sec latency 284.724 ms 2023-09-30T15:00:59.086 INFO:tasks.workunit.client.0.smithi032.stdout: 10 372675 35.13 MB/sec execute 452 sec latency 95.095 ms 2023-09-30T15:00:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:59 smithi156 ceph-mon[103579]: pgmap v968: 97 pgs: 97 active+clean; 802 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.7 MiB/s wr, 709 op/s 2023-09-30T15:00:59.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:59 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T15:00:59.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:59 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T15:00:59.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:00:59 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:00:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:59 smithi148 ceph-mon[102355]: pgmap v968: 97 pgs: 97 active+clean; 802 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.7 MiB/s wr, 709 op/s 2023-09-30T15:00:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T15:00:59.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T15:00:59.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:00:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:00:59.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:59 smithi032 ceph-mon[98592]: pgmap v968: 97 pgs: 97 active+clean; 802 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.7 MiB/s wr, 709 op/s 2023-09-30T15:00:59.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:59 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T15:00:59.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:59 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T15:00:59.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:00:59 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:01:00.086 INFO:tasks.workunit.client.0.smithi032.stdout: 10 373318 35.13 MB/sec execute 453 sec latency 52.403 ms 2023-09-30T15:01:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.807+0000 7f097a5b3700 1 -- 172.21.15.32:0/2490788935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09740ffd60 msgr2=0x7f0974100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.807+0000 7f097a5b3700 1 --2- 172.21.15.32:0/2490788935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09740ffd60 0x7f0974100180 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0964009990 tx=0x7f0964009ca0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.807+0000 7f097a5b3700 1 -- 172.21.15.32:0/2490788935 shutdown_connections 2023-09-30T15:01:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.807+0000 7f097a5b3700 1 --2- 172.21.15.32:0/2490788935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09741018f0 0x7f09741081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.807+0000 7f097a5b3700 1 --2- 172.21.15.32:0/2490788935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09740ffd60 0x7f0974100180 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.807+0000 7f097a5b3700 1 --2- 172.21.15.32:0/2490788935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0974100f50 0x7f09741013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.807+0000 7f097a5b3700 1 -- 172.21.15.32:0/2490788935 >> 172.21.15.32:0/2490788935 conn(0x7f09740fb2b0 msgr2=0x7f09740fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.807+0000 7f097a5b3700 1 -- 172.21.15.32:0/2490788935 shutdown_connections 2023-09-30T15:01:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.807+0000 7f097a5b3700 1 -- 172.21.15.32:0/2490788935 wait complete. 2023-09-30T15:01:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.808+0000 7f097a5b3700 1 Processor -- start 2023-09-30T15:01:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.808+0000 7f097a5b3700 1 -- start start 2023-09-30T15:01:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.808+0000 7f097a5b3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09740ffd60 0x7f0974194c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.808+0000 7f097a5b3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0974100f50 0x7f0974195150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.808+0000 7f097a5b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09741018f0 0x7f0974195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.808+0000 7f097a5b3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0974196630 con 0x7f09741018f0 2023-09-30T15:01:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.808+0000 7f097a5b3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09741967a0 con 0x7f09740ffd60 2023-09-30T15:01:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.808+0000 7f097a5b3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0974196960 con 0x7f0974100f50 2023-09-30T15:01:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.808+0000 7f0978b50700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09741018f0 0x7f0974195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.808+0000 7f09737fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0974100f50 0x7f0974195150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.808+0000 7f09737fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0974100f50 0x7f0974195150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54450/0 (socket says 172.21.15.32:54450) 2023-09-30T15:01:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.808+0000 7f0978b50700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09741018f0 0x7f0974195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40072/0 (socket says 172.21.15.32:40072) 2023-09-30T15:01:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.809+0000 7f09737fe700 1 -- 172.21.15.32:0/1038700706 learned_addr learned my addr 172.21.15.32:0/1038700706 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.809+0000 7f0978b50700 1 -- 172.21.15.32:0/1038700706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0974100f50 msgr2=0x7f0974195150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.809+0000 7f0978b50700 1 --2- 172.21.15.32:0/1038700706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0974100f50 0x7f0974195150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.809+0000 7f0978b50700 1 -- 172.21.15.32:0/1038700706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09740ffd60 msgr2=0x7f0974194c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.809+0000 7f0978b50700 1 --2- 172.21.15.32:0/1038700706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09740ffd60 0x7f0974194c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.809+0000 7f0978b50700 1 -- 172.21.15.32:0/1038700706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09640095f0 con 0x7f09741018f0 2023-09-30T15:01:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.809+0000 7f0978b50700 1 --2- 172.21.15.32:0/1038700706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09741018f0 0x7f0974195fd0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f096c00be20 tx=0x7f096c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.809+0000 7f09717fa700 1 -- 172.21.15.32:0/1038700706 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c00cc10 con 0x7f09741018f0 2023-09-30T15:01:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.809+0000 7f09717fa700 1 -- 172.21.15.32:0/1038700706 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f096c00cd70 con 0x7f09741018f0 2023-09-30T15:01:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.809+0000 7f097a5b3700 1 -- 172.21.15.32:0/1038700706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0974196c40 con 0x7f09741018f0 2023-09-30T15:01:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.809+0000 7f09717fa700 1 -- 172.21.15.32:0/1038700706 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c0126c0 con 0x7f09741018f0 2023-09-30T15:01:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.809+0000 7f097a5b3700 1 -- 172.21.15.32:0/1038700706 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097419d400 con 0x7f09741018f0 2023-09-30T15:01:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.812+0000 7f09717fa700 1 -- 172.21.15.32:0/1038700706 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f096c012860 con 0x7f09741018f0 2023-09-30T15:01:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.813+0000 7f09717fa700 1 --2- 172.21.15.32:0/1038700706 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f095c0a0fa0 0x7f095c0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.813+0000 7f09717fa700 1 -- 172.21.15.32:0/1038700706 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f096c006e40 con 0x7f09741018f0 2023-09-30T15:01:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.814+0000 7f098679bb80 1 Processor -- start 2023-09-30T15:01:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.814+0000 7f0973fff700 1 --2- 172.21.15.32:0/1038700706 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f095c0a0fa0 0x7f095c0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.815+0000 7f0973fff700 1 --2- 172.21.15.32:0/1038700706 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f095c0a0fa0 0x7f095c0a3460 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0964004500 tx=0x7f0964009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.816+0000 7f098679bb80 1 -- start start 2023-09-30T15:01:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.817+0000 7f098679bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd6a10 0x557a12fd6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.817+0000 7f098679bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd7b20 0x557a12fd7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.817+0000 7f098679bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd84e0 0x557a12fded70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.817+0000 7f098679bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a12f36400 con 0x557a12fd84e0 2023-09-30T15:01:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.817+0000 7f098679bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a12f27bc0 con 0x557a12fd6a10 2023-09-30T15:01:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.817+0000 7f098679bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a12f254e0 con 0x557a12fd7b20 2023-09-30T15:01:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.817+0000 7f09737fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd7b20 0x557a12fd7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.817+0000 7f09737fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd7b20 0x557a12fd7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54458/0 (socket says 172.21.15.32:54458) 2023-09-30T15:01:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.817+0000 7f09737fe700 1 -- 172.21.15.32:0/2178677210 learned_addr learned my addr 172.21.15.32:0/2178677210 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.817+0000 7f0978b50700 1 --2- 172.21.15.32:0/2178677210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd84e0 0x557a12fded70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.818+0000 7f0978b50700 1 -- 172.21.15.32:0/2178677210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd7b20 msgr2=0x557a12fd7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.818+0000 7f0978b50700 1 --2- 172.21.15.32:0/2178677210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd7b20 0x557a12fd7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.818+0000 7f0978b50700 1 -- 172.21.15.32:0/2178677210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd6a10 msgr2=0x557a12fd6e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T15:01:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.818+0000 7f0978b50700 1 --2- 172.21.15.32:0/2178677210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd6a10 0x557a12fd6e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.818+0000 7f0978b50700 1 -- 172.21.15.32:0/2178677210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a12f83870 con 0x557a12fd84e0 2023-09-30T15:01:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.818+0000 7f0978b50700 1 --2- 172.21.15.32:0/2178677210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd84e0 0x557a12fded70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f096c0c9e80 tx=0x7f096c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e30c61f2f7ccad2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.818+0000 7f095a7fc700 1 -- 172.21.15.32:0/2178677210 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c0994d0 con 0x557a12fd84e0 2023-09-30T15:01:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.818+0000 7f095a7fc700 1 -- 172.21.15.32:0/2178677210 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f096c09d620 con 0x557a12fd84e0 2023-09-30T15:01:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.818+0000 7f095a7fc700 1 -- 172.21.15.32:0/2178677210 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c0989c0 con 0x557a12fd84e0 2023-09-30T15:01:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.819+0000 7f098679bb80 1 -- 172.21.15.32:0/2178677210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd84e0 msgr2=0x557a12fded70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.819+0000 7f098679bb80 1 --2- 172.21.15.32:0/2178677210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd84e0 0x557a12fded70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f096c0c9e80 tx=0x7f096c004ad0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.819+0000 7f098679bb80 1 -- 172.21.15.32:0/2178677210 shutdown_connections 2023-09-30T15:01:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.819+0000 7f098679bb80 1 --2- 172.21.15.32:0/2178677210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd84e0 0x557a12fded70 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.819+0000 7f098679bb80 1 --2- 172.21.15.32:0/2178677210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd6a10 0x557a12fd6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.819+0000 7f098679bb80 1 --2- 172.21.15.32:0/2178677210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd7b20 0x557a12fd7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.819+0000 7f098679bb80 1 -- 172.21.15.32:0/2178677210 >> 172.21.15.32:0/2178677210 conn(0x557a12f42200 msgr2=0x557a12fc8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.819+0000 7f098679bb80 1 -- 172.21.15.32:0/2178677210 shutdown_connections 2023-09-30T15:01:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.819+0000 7f098679bb80 1 -- 172.21.15.32:0/2178677210 wait complete. 2023-09-30T15:01:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.820+0000 7f098679bb80 1 -- start start 2023-09-30T15:01:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.820+0000 7f098679bb80 1 Processor -- start 2023-09-30T15:01:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.820+0000 7f098679bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd6a10 0x557a12fd5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.820+0000 7f098679bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd7b20 0x557a12fd5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.820+0000 7f098679bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd84e0 0x557a12fdf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.820+0000 7f098679bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a12f20dd0 con 0x557a12fd7b20 2023-09-30T15:01:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.820+0000 7f098679bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a12ebd6b0 con 0x557a12fd84e0 2023-09-30T15:01:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.820+0000 7f098679bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a12ebd350 con 0x557a12fd6a10 2023-09-30T15:01:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.820+0000 7f098679bb80 10 client.0 _ll_register_callbacks cb 0x557a12d1a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:01:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.821+0000 7f0973fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd6a10 0x557a12fd5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.821+0000 7f0973fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd6a10 0x557a12fd5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54462/0 (socket says 172.21.15.32:54462) 2023-09-30T15:01:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.821+0000 7f0973fff700 1 -- 172.21.15.32:0/1476167664 learned_addr learned my addr 172.21.15.32:0/1476167664 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.821+0000 7f0978b50700 1 --2- 172.21.15.32:0/1476167664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd84e0 0x557a12fdf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.821+0000 7f09737fe700 1 --2- 172.21.15.32:0/1476167664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd7b20 0x557a12fd5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.821+0000 7f09737fe700 1 -- 172.21.15.32:0/1476167664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd6a10 msgr2=0x557a12fd5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.821+0000 7f09737fe700 1 --2- 172.21.15.32:0/1476167664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd6a10 0x557a12fd5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.821+0000 7f09737fe700 1 -- 172.21.15.32:0/1476167664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd84e0 msgr2=0x557a12fdf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.821+0000 7f09737fe700 1 --2- 172.21.15.32:0/1476167664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd84e0 0x557a12fdf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.821+0000 7f09737fe700 1 -- 172.21.15.32:0/1476167664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097419d400 con 0x557a12fd7b20 2023-09-30T15:01:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.821+0000 7f0973fff700 1 --2- 172.21.15.32:0/1476167664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd6a10 0x557a12fd5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:01:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.821+0000 7f09737fe700 1 --2- 172.21.15.32:0/1476167664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd7b20 0x557a12fd5cb0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0968011540 tx=0x7f0968011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50566d1b73047a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.821+0000 7f09537fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T15:01:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.821+0000 7f09537fe700 1 -- 172.21.15.32:0/1476167664 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968014650 con 0x557a12fd7b20 2023-09-30T15:01:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.821+0000 7f09537fe700 1 -- 172.21.15.32:0/1476167664 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f09680147b0 con 0x557a12fd7b20 2023-09-30T15:01:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.821+0000 7f098679bb80 1 -- 172.21.15.32:0/1476167664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a12fc1090 con 0x557a12fd7b20 2023-09-30T15:01:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.822+0000 7f09537fe700 1 -- 172.21.15.32:0/1476167664 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0968023030 con 0x557a12fd7b20 2023-09-30T15:01:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.822+0000 7f098679bb80 10 client.17313 fetch_fsmap learned FSMap version 55 2023-09-30T15:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.822+0000 7f098679bb80 1 -- 172.21.15.32:0/1476167664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x557a12d4bd80 con 0x557a12fd7b20 2023-09-30T15:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.822+0000 7f09537fe700 1 -- 172.21.15.32:0/1476167664 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a12fc1090 con 0x557a12fd7b20 2023-09-30T15:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.822+0000 7f09537fe700 1 -- 172.21.15.32:0/1476167664 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f096801b020 con 0x557a12fd7b20 2023-09-30T15:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.822+0000 7f098679bb80 10 client.17313 fetch_fsmap finished waiting for FSMap version 55 2023-09-30T15:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.822+0000 7f098679bb80 10 client.17313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.822+0000 7f098679bb80 20 client.17313 populate_metadata read hostname 'smithi032' 2023-09-30T15:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.822+0000 7f098679bb80 1 --2- 172.21.15.32:0/1476167664 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557a12fe3bd0 0x557a12fe5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.822+0000 7f098679bb80 4 client.17313 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T15:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.822+0000 7f098679bb80 1 -- 172.21.15.32:0/1476167664 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a12f19db0 con 0x557a12fe3bd0 2023-09-30T15:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.822+0000 7f0973fff700 1 --2- 172.21.15.32:0/1476167664 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557a12fe3bd0 0x557a12fe5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.823+0000 7f09537fe700 10 client.17313 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.823+0000 7f0973fff700 1 --2- 172.21.15.32:0/1476167664 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557a12fe3bd0 0x557a12fe5ff0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c9f12e12ca538b2 server_cookie=6139191c525c4210 in_seq=0 out_seq=0 2023-09-30T15:01:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.824+0000 7f09537fe700 1 -- 172.21.15.32:0/1476167664 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557a12f19db0 con 0x557a12fe3bd0 2023-09-30T15:01:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.824+0000 7f09537fe700 10 client.17313 handle_command_reply: tid=0 2023-09-30T15:01:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.824+0000 7f098679bb80 1 client.17313 shutdown 2023-09-30T15:01:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.824+0000 7f098679bb80 1 -- 172.21.15.32:0/1476167664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd7b20 msgr2=0x557a12fd5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.824+0000 7f098679bb80 1 --2- 172.21.15.32:0/1476167664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd7b20 0x557a12fd5cb0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0968011540 tx=0x7f0968011850 comp rx=0 tx=0).stop 2023-09-30T15:01:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.824+0000 7f098679bb80 1 -- 172.21.15.32:0/1476167664 shutdown_connections 2023-09-30T15:01:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.824+0000 7f098679bb80 1 --2- 172.21.15.32:0/1476167664 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557a12fe3bd0 0x557a12fe5ff0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.824+0000 7f098679bb80 1 --2- 172.21.15.32:0/1476167664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd84e0 0x557a12fdf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.824+0000 7f098679bb80 1 --2- 172.21.15.32:0/1476167664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd6a10 0x557a12fd5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.824+0000 7f098679bb80 1 --2- 172.21.15.32:0/1476167664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd7b20 0x557a12fd5cb0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.824+0000 7f098679bb80 1 -- 172.21.15.32:0/1476167664 >> 172.21.15.32:0/1476167664 conn(0x557a12f42200 msgr2=0x557a12fc8590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.824+0000 7f09537fe700 0 client.17313 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.824+0000 7f098679bb80 1 -- 172.21.15.32:0/1476167664 shutdown_connections 2023-09-30T15:01:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.824+0000 7f098679bb80 1 -- 172.21.15.32:0/1476167664 wait complete. 2023-09-30T15:01:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.824+0000 7f098679bb80 20 client.17313 trim_cache size 0 max 16384 2023-09-30T15:01:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.847+0000 7f098679bb80 1 Processor -- start 2023-09-30T15:01:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.847+0000 7f098679bb80 1 -- start start 2023-09-30T15:01:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.847+0000 7f098679bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd6a10 0x557a12fd6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.847+0000 7f098679bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd7b20 0x557a12fd5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.847+0000 7f098679bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd84e0 0x557a12fd5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.847+0000 7f098679bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a12da18c0 con 0x557a12fd7b20 2023-09-30T15:01:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.847+0000 7f098679bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a12fd88c0 con 0x557a12fd6a10 2023-09-30T15:01:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.847+0000 7f098679bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a12fc9bf0 con 0x557a12fd84e0 2023-09-30T15:01:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.847+0000 7f0978b50700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd84e0 0x557a12fd5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.847+0000 7f0973fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd6a10 0x557a12fd6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.847+0000 7f0978b50700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd84e0 0x557a12fd5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54474/0 (socket says 172.21.15.32:54474) 2023-09-30T15:01:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.847+0000 7f0978b50700 1 -- 172.21.15.32:0/3993028762 learned_addr learned my addr 172.21.15.32:0/3993028762 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.848+0000 7f09737fe700 1 --2- 172.21.15.32:0/3993028762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd7b20 0x557a12fd5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.848+0000 7f0978b50700 1 -- 172.21.15.32:0/3993028762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd6a10 msgr2=0x557a12fd6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.848+0000 7f0978b50700 1 --2- 172.21.15.32:0/3993028762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd6a10 0x557a12fd6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.848+0000 7f0978b50700 1 -- 172.21.15.32:0/3993028762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd7b20 msgr2=0x557a12fd5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.848+0000 7f0978b50700 1 --2- 172.21.15.32:0/3993028762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd7b20 0x557a12fd5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.848+0000 7f0978b50700 1 -- 172.21.15.32:0/3993028762 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a12d34c00 con 0x557a12fd84e0 2023-09-30T15:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.848+0000 7f0978b50700 1 --2- 172.21.15.32:0/3993028762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd84e0 0x557a12fd5c90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f096c00be50 tx=0x7f096c084040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f719d46125291849 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.848+0000 7f095affd700 1 -- 172.21.15.32:0/3993028762 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c0994d0 con 0x557a12fd84e0 2023-09-30T15:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.849+0000 7f095affd700 1 -- 172.21.15.32:0/3993028762 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f096c0b1070 con 0x557a12fd84e0 2023-09-30T15:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.849+0000 7f095affd700 1 -- 172.21.15.32:0/3993028762 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c099990 con 0x557a12fd84e0 2023-09-30T15:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.849+0000 7f098679bb80 1 -- 172.21.15.32:0/3993028762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd84e0 msgr2=0x557a12fd5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.849+0000 7f098679bb80 1 --2- 172.21.15.32:0/3993028762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd84e0 0x557a12fd5c90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f096c00be50 tx=0x7f096c084040 comp rx=0 tx=0).stop 2023-09-30T15:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.850+0000 7f098679bb80 1 -- 172.21.15.32:0/3993028762 shutdown_connections 2023-09-30T15:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.850+0000 7f098679bb80 1 --2- 172.21.15.32:0/3993028762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd84e0 0x557a12fd5c90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.850+0000 7f098679bb80 1 --2- 172.21.15.32:0/3993028762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd6a10 0x557a12fd6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.850+0000 7f098679bb80 1 --2- 172.21.15.32:0/3993028762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a12fd7b20 0x557a12fd5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.850+0000 7f098679bb80 1 -- 172.21.15.32:0/3993028762 >> 172.21.15.32:0/3993028762 conn(0x557a12f42200 msgr2=0x557a12fd0660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.851+0000 7f098679bb80 1 -- 172.21.15.32:0/3993028762 shutdown_connections 2023-09-30T15:01:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.851+0000 7f098679bb80 1 -- 172.21.15.32:0/3993028762 wait complete. 2023-09-30T15:01:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.851+0000 7f098679bb80 1 -- start start 2023-09-30T15:01:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.852+0000 7f098679bb80 1 Processor -- start 2023-09-30T15:01:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.852+0000 7f098679bb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd6a10 0x557a12fcf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.852+0000 7f098679bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd84e0 0x557a12fcf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.852+0000 7f098679bb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a13033ee0 0x557a130362d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.852+0000 7f098679bb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a13036810 con 0x557a13033ee0 2023-09-30T15:01:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.852+0000 7f098679bb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a13036950 con 0x557a12fd6a10 2023-09-30T15:01:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.852+0000 7f098679bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a13036b10 con 0x557a12fd84e0 2023-09-30T15:01:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.852+0000 7f098679bb80 10 client.0 _ll_register_callbacks cb 0x7f0968009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:01:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.852+0000 7f0978b50700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a13033ee0 0x557a130362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.852+0000 7f09737fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd84e0 0x557a12fcf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.852+0000 7f0973fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd6a10 0x557a12fcf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.852+0000 7f0978b50700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a13033ee0 0x557a130362d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40098/0 (socket says 172.21.15.32:40098) 2023-09-30T15:01:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.852+0000 7f09737fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd84e0 0x557a12fcf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54476/0 (socket says 172.21.15.32:54476) 2023-09-30T15:01:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.852+0000 7f0978b50700 1 -- 172.21.15.32:0/2481115631 learned_addr learned my addr 172.21.15.32:0/2481115631 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.852+0000 7f0973fff700 1 --2- 172.21.15.32:0/2481115631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd6a10 0x557a12fcf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51570/0 (socket says 172.21.15.32:51570) 2023-09-30T15:01:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.853+0000 7f0973fff700 1 -- 172.21.15.32:0/2481115631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd84e0 msgr2=0x557a12fcf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.853+0000 7f0973fff700 1 --2- 172.21.15.32:0/2481115631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd84e0 0x557a12fcf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.853+0000 7f0973fff700 1 -- 172.21.15.32:0/2481115631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a13033ee0 msgr2=0x557a130362d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.853+0000 7f0973fff700 1 --2- 172.21.15.32:0/2481115631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a13033ee0 0x557a130362d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.853+0000 7f0973fff700 1 -- 172.21.15.32:0/2481115631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a12f83870 con 0x557a12fd6a10 2023-09-30T15:01:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.853+0000 7f0973fff700 1 --2- 172.21.15.32:0/2481115631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd6a10 0x557a12fcf320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x557a12fe68a0 tx=0x7f0964000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e0f58db8aeab145 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.853+0000 7f09537fe700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T15:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.855+0000 7f09537fe700 1 -- 172.21.15.32:0/2481115631 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096402b040 con 0x557a12fd6a10 2023-09-30T15:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.855+0000 7f09537fe700 1 -- 172.21.15.32:0/2481115631 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f096400dac0 con 0x557a12fd6a10 2023-09-30T15:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.855+0000 7f098679bb80 1 -- 172.21.15.32:0/2481115631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f096c09d620 con 0x557a12fd6a10 2023-09-30T15:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.855+0000 7f09537fe700 1 -- 172.21.15.32:0/2481115631 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0964004e80 con 0x557a12fd6a10 2023-09-30T15:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.855+0000 7f098679bb80 10 client.26933 fetch_fsmap learned FSMap version 55 2023-09-30T15:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.855+0000 7f098679bb80 1 -- 172.21.15.32:0/2481115631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x557a12d721f0 con 0x557a12fd6a10 2023-09-30T15:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.855+0000 7f09537fe700 1 -- 172.21.15.32:0/2481115631 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f096c09d620 con 0x557a12fd6a10 2023-09-30T15:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.855+0000 7f09537fe700 1 -- 172.21.15.32:0/2481115631 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f09640053c0 con 0x557a12fd6a10 2023-09-30T15:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.855+0000 7f098679bb80 10 client.26933 fetch_fsmap finished waiting for FSMap version 55 2023-09-30T15:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.855+0000 7f098679bb80 10 client.26933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.855+0000 7f098679bb80 20 client.26933 populate_metadata read hostname 'smithi032' 2023-09-30T15:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.856+0000 7f098679bb80 1 --2- 172.21.15.32:0/2481115631 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557a130384f0 0x557a1303a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.856+0000 7f098679bb80 4 client.26933 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T15:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.856+0000 7f098679bb80 1 -- 172.21.15.32:0/2481115631 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557a12f19ad0 con 0x557a130384f0 2023-09-30T15:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.856+0000 7f09737fe700 1 --2- 172.21.15.32:0/2481115631 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557a130384f0 0x557a1303a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.856+0000 7f09537fe700 10 client.26933 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:00.856+0000 7f09737fe700 1 --2- 172.21.15.32:0/2481115631 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557a130384f0 0x557a1303a910 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0c24481e139ca9d server_cookie=f48a9e961f3c6655 in_seq=0 out_seq=0 2023-09-30T15:01:01.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 373891 35.13 MB/sec execute 454 sec latency 178.686 ms 2023-09-30T15:01:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.239+0000 7f09537fe700 1 -- 172.21.15.32:0/2481115631 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557a12f19ad0 con 0x557a130384f0 2023-09-30T15:01:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.239+0000 7f09537fe700 10 client.26933 handle_command_reply: tid=0 2023-09-30T15:01:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.239+0000 7f098679bb80 1 client.26933 shutdown 2023-09-30T15:01:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.239+0000 7f098679bb80 1 -- 172.21.15.32:0/2481115631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd6a10 msgr2=0x557a12fcf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.239+0000 7f098679bb80 1 --2- 172.21.15.32:0/2481115631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd6a10 0x557a12fcf320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x557a12fe68a0 tx=0x7f0964000bc0 comp rx=0 tx=0).stop 2023-09-30T15:01:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.239+0000 7f098679bb80 1 -- 172.21.15.32:0/2481115631 shutdown_connections 2023-09-30T15:01:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.239+0000 7f098679bb80 1 --2- 172.21.15.32:0/2481115631 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x557a130384f0 0x557a1303a910 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.239+0000 7f098679bb80 1 --2- 172.21.15.32:0/2481115631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557a13033ee0 0x557a130362d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.239+0000 7f09537fe700 0 client.26933 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.239+0000 7f098679bb80 1 --2- 172.21.15.32:0/2481115631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a12fd6a10 0x557a12fcf320 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.239+0000 7f098679bb80 1 --2- 172.21.15.32:0/2481115631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557a12fd84e0 0x557a12fcf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.239+0000 7f098679bb80 1 -- 172.21.15.32:0/2481115631 >> 172.21.15.32:0/2481115631 conn(0x557a12f42200 msgr2=0x557a12fd0660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.239+0000 7f098679bb80 1 -- 172.21.15.32:0/2481115631 shutdown_connections 2023-09-30T15:01:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.239+0000 7f098679bb80 1 -- 172.21.15.32:0/2481115631 wait complete. 2023-09-30T15:01:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.239+0000 7f098679bb80 20 client.26933 trim_cache size 0 max 16384 2023-09-30T15:01:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T15:01:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T15:01:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T15:01:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T15:01:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.240+0000 7f097a5b3700 1 -- 172.21.15.32:0/1038700706 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f095c0a0fa0 msgr2=0x7f095c0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.240+0000 7f097a5b3700 1 --2- 172.21.15.32:0/1038700706 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f095c0a0fa0 0x7f095c0a3460 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0964004500 tx=0x7f0964009f90 comp rx=0 tx=0).stop 2023-09-30T15:01:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.240+0000 7f097a5b3700 1 -- 172.21.15.32:0/1038700706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09741018f0 msgr2=0x7f0974195fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.240+0000 7f097a5b3700 1 --2- 172.21.15.32:0/1038700706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09741018f0 0x7f0974195fd0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f096c00be20 tx=0x7f096c009ed0 comp rx=0 tx=0).stop 2023-09-30T15:01:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.240+0000 7f097a5b3700 1 -- 172.21.15.32:0/1038700706 shutdown_connections 2023-09-30T15:01:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.240+0000 7f097a5b3700 1 --2- 172.21.15.32:0/1038700706 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f095c0a0fa0 0x7f095c0a3460 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.240+0000 7f097a5b3700 1 --2- 172.21.15.32:0/1038700706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09741018f0 0x7f0974195fd0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.240+0000 7f097a5b3700 1 --2- 172.21.15.32:0/1038700706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09740ffd60 0x7f0974194c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.240+0000 7f097a5b3700 1 --2- 172.21.15.32:0/1038700706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0974100f50 0x7f0974195150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.240+0000 7f097a5b3700 1 -- 172.21.15.32:0/1038700706 >> 172.21.15.32:0/1038700706 conn(0x7f09740fb2b0 msgr2=0x7f09741044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.240+0000 7f097a5b3700 1 -- 172.21.15.32:0/1038700706 shutdown_connections 2023-09-30T15:01:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:01.240+0000 7f097a5b3700 1 -- 172.21.15.32:0/1038700706 wait complete. 2023-09-30T15:01:01.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:01 smithi148 ceph-mon[102355]: pgmap v969: 97 pgs: 97 active+clean; 809 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.0 MiB/s wr, 678 op/s 2023-09-30T15:01:01.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:01 smithi156 ceph-mon[103579]: pgmap v969: 97 pgs: 97 active+clean; 809 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.0 MiB/s wr, 678 op/s 2023-09-30T15:01:01.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:01 smithi032 ceph-mon[98592]: pgmap v969: 97 pgs: 97 active+clean; 809 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.0 MiB/s wr, 678 op/s 2023-09-30T15:01:02.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 374466 35.11 MB/sec execute 455 sec latency 292.521 ms 2023-09-30T15:01:03.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 374747 35.07 MB/sec execute 456 sec latency 321.000 ms 2023-09-30T15:01:03.814 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:03 smithi156 ceph-mon[103579]: pgmap v970: 97 pgs: 97 active+clean; 811 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.0 MiB/s wr, 656 op/s 2023-09-30T15:01:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:03 smithi148 ceph-mon[102355]: pgmap v970: 97 pgs: 97 active+clean; 811 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.0 MiB/s wr, 656 op/s 2023-09-30T15:01:03.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:03 smithi032 ceph-mon[98592]: pgmap v970: 97 pgs: 97 active+clean; 811 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.0 MiB/s wr, 656 op/s 2023-09-30T15:01:04.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 375272 35.05 MB/sec execute 457 sec latency 119.113 ms 2023-09-30T15:01:05.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 375891 35.06 MB/sec execute 458 sec latency 56.853 ms 2023-09-30T15:01:05.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:05 smithi156 ceph-mon[103579]: pgmap v971: 97 pgs: 97 active+clean; 812 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.8 MiB/s wr, 743 op/s 2023-09-30T15:01:05.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:05 smithi148 ceph-mon[102355]: pgmap v971: 97 pgs: 97 active+clean; 812 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.8 MiB/s wr, 743 op/s 2023-09-30T15:01:05.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:05 smithi032 ceph-mon[98592]: pgmap v971: 97 pgs: 97 active+clean; 812 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.8 MiB/s wr, 743 op/s 2023-09-30T15:01:06.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 376592 35.05 MB/sec execute 459 sec latency 82.587 ms 2023-09-30T15:01:07.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 377225 35.06 MB/sec execute 460 sec latency 115.569 ms 2023-09-30T15:01:07.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:07 smithi156 ceph-mon[103579]: pgmap v972: 97 pgs: 97 active+clean; 808 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.5 MiB/s wr, 671 op/s 2023-09-30T15:01:07.862 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:07 smithi148 ceph-mon[102355]: pgmap v972: 97 pgs: 97 active+clean; 808 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.5 MiB/s wr, 671 op/s 2023-09-30T15:01:07.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:07 smithi032 ceph-mon[98592]: pgmap v972: 97 pgs: 97 active+clean; 808 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.5 MiB/s wr, 671 op/s 2023-09-30T15:01:08.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 377719 35.04 MB/sec execute 461 sec latency 229.776 ms 2023-09-30T15:01:09.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 378371 35.04 MB/sec execute 462 sec latency 93.182 ms 2023-09-30T15:01:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:09 smithi156 ceph-mon[103579]: pgmap v973: 97 pgs: 97 active+clean; 815 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.4 MiB/s wr, 723 op/s 2023-09-30T15:01:09.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:09 smithi148 ceph-mon[102355]: pgmap v973: 97 pgs: 97 active+clean; 815 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.4 MiB/s wr, 723 op/s 2023-09-30T15:01:09.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:09 smithi032 ceph-mon[98592]: pgmap v973: 97 pgs: 97 active+clean; 815 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.4 MiB/s wr, 723 op/s 2023-09-30T15:01:10.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 379073 35.05 MB/sec execute 463 sec latency 54.951 ms 2023-09-30T15:01:11.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 379728 35.05 MB/sec execute 464 sec latency 47.812 ms 2023-09-30T15:01:11.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:11 smithi156 ceph-mon[103579]: pgmap v974: 97 pgs: 97 active+clean; 819 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.8 MiB/s wr, 688 op/s 2023-09-30T15:01:11.865 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:11 smithi148 ceph-mon[102355]: pgmap v974: 97 pgs: 97 active+clean; 819 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.8 MiB/s wr, 688 op/s 2023-09-30T15:01:11.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:11 smithi032 ceph-mon[98592]: pgmap v974: 97 pgs: 97 active+clean; 819 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.8 MiB/s wr, 688 op/s 2023-09-30T15:01:12.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 380354 35.04 MB/sec execute 465 sec latency 82.233 ms 2023-09-30T15:01:13.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 380786 35.01 MB/sec execute 466 sec latency 290.185 ms 2023-09-30T15:01:13.810 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:13 smithi156 ceph-mon[103579]: pgmap v975: 97 pgs: 97 active+clean; 821 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.5 MiB/s wr, 654 op/s 2023-09-30T15:01:13.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:13 smithi148 ceph-mon[102355]: pgmap v975: 97 pgs: 97 active+clean; 821 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.5 MiB/s wr, 654 op/s 2023-09-30T15:01:13.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:13 smithi032 ceph-mon[98592]: pgmap v975: 97 pgs: 97 active+clean; 821 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.5 MiB/s wr, 654 op/s 2023-09-30T15:01:14.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 381345 35.01 MB/sec execute 467 sec latency 161.298 ms 2023-09-30T15:01:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.393+0000 7f366c974700 1 -- 172.21.15.32:0/1228486650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3668102050 msgr2=0x7f36681024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.393+0000 7f366c974700 1 --2- 172.21.15.32:0/1228486650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3668102050 0x7f36681024d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f365c009990 tx=0x7f365c009ca0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.393+0000 7f366c974700 1 -- 172.21.15.32:0/1228486650 shutdown_connections 2023-09-30T15:01:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.393+0000 7f366c974700 1 --2- 172.21.15.32:0/1228486650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3668102a10 0x7f3668109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.393+0000 7f366c974700 1 --2- 172.21.15.32:0/1228486650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3668100e70 0x7f3668101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.393+0000 7f366c974700 1 --2- 172.21.15.32:0/1228486650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3668102050 0x7f36681024d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.393+0000 7f366c974700 1 -- 172.21.15.32:0/1228486650 >> 172.21.15.32:0/1228486650 conn(0x7f36680fc430 msgr2=0x7f36680fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.393+0000 7f366c974700 1 -- 172.21.15.32:0/1228486650 shutdown_connections 2023-09-30T15:01:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.394+0000 7f366c974700 1 -- 172.21.15.32:0/1228486650 wait complete. 2023-09-30T15:01:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.395+0000 7f366c974700 1 Processor -- start 2023-09-30T15:01:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.395+0000 7f366c974700 1 -- start start 2023-09-30T15:01:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.395+0000 7f366c974700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3668100e70 0x7f3668073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.395+0000 7f366c974700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3668102050 0x7f3668073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.395+0000 7f366c974700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3668102a10 0x7f3668073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.395+0000 7f366c974700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3668074db0 con 0x7f3668102a10 2023-09-30T15:01:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.395+0000 7f366c974700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3668074f20 con 0x7f3668102050 2023-09-30T15:01:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.395+0000 7f366c974700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36680750e0 con 0x7f3668100e70 2023-09-30T15:01:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.395+0000 7f366659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3668100e70 0x7f3668073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.395+0000 7f3666d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3668102a10 0x7f3668073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.395+0000 7f3666d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3668102a10 0x7f3668073c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40118/0 (socket says 172.21.15.32:40118) 2023-09-30T15:01:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.395+0000 7f366659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3668100e70 0x7f3668073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54496/0 (socket says 172.21.15.32:54496) 2023-09-30T15:01:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.395+0000 7f3666d9d700 1 -- 172.21.15.32:0/3267669125 learned_addr learned my addr 172.21.15.32:0/3267669125 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.396+0000 7f3665d9b700 1 --2- 172.21.15.32:0/3267669125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3668102050 0x7f3668073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.396+0000 7f3666d9d700 1 -- 172.21.15.32:0/3267669125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3668100e70 msgr2=0x7f3668073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.396+0000 7f3666d9d700 1 --2- 172.21.15.32:0/3267669125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3668100e70 0x7f3668073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.396+0000 7f3666d9d700 1 -- 172.21.15.32:0/3267669125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3668102050 msgr2=0x7f3668073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.396+0000 7f3666d9d700 1 --2- 172.21.15.32:0/3267669125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3668102050 0x7f3668073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.396+0000 7f3666d9d700 1 -- 172.21.15.32:0/3267669125 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c0095f0 con 0x7f3668102a10 2023-09-30T15:01:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.396+0000 7f3666d9d700 1 --2- 172.21.15.32:0/3267669125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3668102a10 0x7f3668073c90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f366000ef30 tx=0x7f366000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.396+0000 7f36577fe700 1 -- 172.21.15.32:0/3267669125 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660018070 con 0x7f3668102a10 2023-09-30T15:01:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.396+0000 7f366c974700 1 -- 172.21.15.32:0/3267669125 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366819aca0 con 0x7f3668102a10 2023-09-30T15:01:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.396+0000 7f36577fe700 1 -- 172.21.15.32:0/3267669125 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f366000f040 con 0x7f3668102a10 2023-09-30T15:01:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.396+0000 7f36577fe700 1 -- 172.21.15.32:0/3267669125 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366000cd00 con 0x7f3668102a10 2023-09-30T15:01:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.396+0000 7f366c974700 1 -- 172.21.15.32:0/3267669125 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366819b1b0 con 0x7f3668102a10 2023-09-30T15:01:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.399+0000 7f36577fe700 1 -- 172.21.15.32:0/3267669125 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f3660003a00 con 0x7f3668102a10 2023-09-30T15:01:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.399+0000 7f36577fe700 1 --2- 172.21.15.32:0/3267669125 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f36500a0fa0 0x7f36500a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.399+0000 7f36577fe700 1 -- 172.21.15.32:0/3267669125 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f36600cc280 con 0x7f3668102a10 2023-09-30T15:01:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.399+0000 7f366659c700 1 --2- 172.21.15.32:0/3267669125 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f36500a0fa0 0x7f36500a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.400+0000 7f366659c700 1 --2- 172.21.15.32:0/3267669125 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f36500a0fa0 0x7f36500a3460 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3668074b20 tx=0x7f3658009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:14.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.401+0000 7f3678b5cb80 1 Processor -- start 2023-09-30T15:01:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.401+0000 7f3678b5cb80 1 -- start start 2023-09-30T15:01:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3678b5cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90ad200 0x564db90ad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3678b5cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 0x564db90ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3678b5cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90aecd0 0x564db90b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3678b5cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9015400 con 0x564db90aecd0 2023-09-30T15:01:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3678b5cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9006bc0 con 0x564db90ad200 2023-09-30T15:01:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3678b5cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db90044e0 con 0x564db90ae310 2023-09-30T15:01:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3665d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 0x564db90ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f366659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90ad200 0x564db90ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3665d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 0x564db90ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54506/0 (socket says 172.21.15.32:54506) 2023-09-30T15:01:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f366659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90ad200 0x564db90ad620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51598/0 (socket says 172.21.15.32:51598) 2023-09-30T15:01:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3665d9b700 1 -- 172.21.15.32:0/2323565516 learned_addr learned my addr 172.21.15.32:0/2323565516 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3666d9d700 1 --2- 172.21.15.32:0/2323565516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90aecd0 0x564db90b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3665d9b700 1 -- 172.21.15.32:0/2323565516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90ad200 msgr2=0x564db90ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3665d9b700 1 --2- 172.21.15.32:0/2323565516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90ad200 0x564db90ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3665d9b700 1 -- 172.21.15.32:0/2323565516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90aecd0 msgr2=0x564db90b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3665d9b700 1 --2- 172.21.15.32:0/2323565516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90aecd0 0x564db90b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3665d9b700 1 -- 172.21.15.32:0/2323565516 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564db9062870 con 0x564db90ae310 2023-09-30T15:01:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3666d9d700 1 --2- 172.21.15.32:0/2323565516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90aecd0 0x564db90b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T15:01:14.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.402+0000 7f3665d9b700 1 --2- 172.21.15.32:0/2323565516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 0x564db90ae790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f365c009990 tx=0x7f365c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6927832c9d8d6478 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.403+0000 7f3654ff9700 1 -- 172.21.15.32:0/2323565516 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c01d050 con 0x564db90ae310 2023-09-30T15:01:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.403+0000 7f3654ff9700 1 -- 172.21.15.32:0/2323565516 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f365c01e040 con 0x564db90ae310 2023-09-30T15:01:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.403+0000 7f3654ff9700 1 -- 172.21.15.32:0/2323565516 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c023d30 con 0x564db90ae310 2023-09-30T15:01:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.403+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2323565516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 msgr2=0x564db90ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.403+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/2323565516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 0x564db90ae790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f365c009990 tx=0x7f365c005e80 comp rx=0 tx=0).stop 2023-09-30T15:01:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.403+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2323565516 shutdown_connections 2023-09-30T15:01:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.403+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/2323565516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90aecd0 0x564db90b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.403+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/2323565516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90ad200 0x564db90ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.403+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/2323565516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 0x564db90ae790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.403+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2323565516 >> 172.21.15.32:0/2323565516 conn(0x564db9021200 msgr2=0x564db90a7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.404+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2323565516 shutdown_connections 2023-09-30T15:01:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.404+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2323565516 wait complete. 2023-09-30T15:01:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.404+0000 7f3678b5cb80 1 -- start start 2023-09-30T15:01:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.404+0000 7f3678b5cb80 1 Processor -- start 2023-09-30T15:01:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.404+0000 7f3678b5cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ad200 0x564db90b6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.404+0000 7f3678b5cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90ae310 0x564db90b7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.404+0000 7f3678b5cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90aecd0 0x564db90be380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.404+0000 7f3678b5cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db8fffdd0 con 0x564db90ae310 2023-09-30T15:01:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.404+0000 7f3678b5cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db8f9c6b0 con 0x564db90aecd0 2023-09-30T15:01:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.404+0000 7f3678b5cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db8f9c350 con 0x564db90ad200 2023-09-30T15:01:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.404+0000 7f3678b5cb80 10 client.0 _ll_register_callbacks cb 0x564db8df97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:01:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.405+0000 7f366659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ad200 0x564db90b6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.405+0000 7f366659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ad200 0x564db90b6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54510/0 (socket says 172.21.15.32:54510) 2023-09-30T15:01:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.405+0000 7f366659c700 1 -- 172.21.15.32:0/2807190127 learned_addr learned my addr 172.21.15.32:0/2807190127 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.405+0000 7f3666d9d700 1 --2- 172.21.15.32:0/2807190127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90aecd0 0x564db90be380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.405+0000 7f366659c700 1 -- 172.21.15.32:0/2807190127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90aecd0 msgr2=0x564db90be380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.405+0000 7f366659c700 1 --2- 172.21.15.32:0/2807190127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90aecd0 0x564db90be380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.405+0000 7f366659c700 1 -- 172.21.15.32:0/2807190127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90ae310 msgr2=0x564db90b7340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T15:01:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.405+0000 7f366659c700 1 --2- 172.21.15.32:0/2807190127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90ae310 0x564db90b7340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.405+0000 7f366659c700 1 -- 172.21.15.32:0/2807190127 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3668109c60 con 0x564db90ad200 2023-09-30T15:01:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.405+0000 7f366659c700 1 --2- 172.21.15.32:0/2807190127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ad200 0x564db90b6e00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3658011fd0 tx=0x7f36580047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62ac7742c6e4b82 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.405+0000 7f364dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T15:01:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.405+0000 7f364dffb700 1 -- 172.21.15.32:0/2807190127 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658022070 con 0x564db90ad200 2023-09-30T15:01:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.405+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2807190127 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564db90a0090 con 0x564db90ad200 2023-09-30T15:01:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.405+0000 7f364dffb700 1 -- 172.21.15.32:0/2807190127 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3658003e20 con 0x564db90ad200 2023-09-30T15:01:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.406+0000 7f364dffb700 1 -- 172.21.15.32:0/2807190127 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3658027420 con 0x564db90ad200 2023-09-30T15:01:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.406+0000 7f3678b5cb80 10 client.26989 fetch_fsmap learned FSMap version 55 2023-09-30T15:01:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.406+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2807190127 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x564db8e2ad80 con 0x564db90ad200 2023-09-30T15:01:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.406+0000 7f364dffb700 1 -- 172.21.15.32:0/2807190127 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564db90a0090 con 0x564db90ad200 2023-09-30T15:01:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.406+0000 7f364dffb700 1 -- 172.21.15.32:0/2807190127 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f365801d710 con 0x564db90ad200 2023-09-30T15:01:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.406+0000 7f3678b5cb80 10 client.26989 fetch_fsmap finished waiting for FSMap version 55 2023-09-30T15:01:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.406+0000 7f3678b5cb80 10 client.26989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:01:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.406+0000 7f3678b5cb80 20 client.26989 populate_metadata read hostname 'smithi032' 2023-09-30T15:01:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.406+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/2807190127 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564db90c2d30 0x564db90c5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.406+0000 7f3678b5cb80 4 client.26989 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T15:01:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.406+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2807190127 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564db8ff8db0 con 0x564db90c2d30 2023-09-30T15:01:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.406+0000 7f3665d9b700 1 --2- 172.21.15.32:0/2807190127 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564db90c2d30 0x564db90c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.407+0000 7f364dffb700 10 client.26989 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.407+0000 7f3665d9b700 1 --2- 172.21.15.32:0/2807190127 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564db90c2d30 0x564db90c5150 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64af4db4cf718332 server_cookie=16004b2c828dd895 in_seq=0 out_seq=0 2023-09-30T15:01:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.408+0000 7f364dffb700 1 -- 172.21.15.32:0/2807190127 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564db8ff8db0 con 0x564db90c2d30 2023-09-30T15:01:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.408+0000 7f364dffb700 10 client.26989 handle_command_reply: tid=0 2023-09-30T15:01:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.408+0000 7f3678b5cb80 1 client.26989 shutdown 2023-09-30T15:01:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.409+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2807190127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ad200 msgr2=0x564db90b6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.409+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/2807190127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ad200 0x564db90b6e00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3658011fd0 tx=0x7f36580047c0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.409+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2807190127 shutdown_connections 2023-09-30T15:01:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.409+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/2807190127 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564db90c2d30 0x564db90c5150 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.409+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/2807190127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90aecd0 0x564db90be380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.409+0000 7f364dffb700 0 client.26989 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.409+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/2807190127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ad200 0x564db90b6e00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.409+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/2807190127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90ae310 0x564db90b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.409+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2807190127 >> 172.21.15.32:0/2807190127 conn(0x564db9021200 msgr2=0x564db90a7590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.409+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2807190127 shutdown_connections 2023-09-30T15:01:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.409+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2807190127 wait complete. 2023-09-30T15:01:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.409+0000 7f3678b5cb80 20 client.26989 trim_cache size 0 max 16384 2023-09-30T15:01:14.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.431+0000 7f3678b5cb80 1 Processor -- start 2023-09-30T15:01:14.431 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.431+0000 7f3678b5cb80 1 -- start start 2023-09-30T15:01:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.432+0000 7f3678b5cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90ad200 0x564db90c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.432+0000 7f3678b5cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 0x564db90be340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.432+0000 7f3678b5cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90aecd0 0x564db90a7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.432+0000 7f3678b5cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db8e808c0 con 0x564db90ad200 2023-09-30T15:01:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.432+0000 7f3678b5cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db90b6970 con 0x564db90aecd0 2023-09-30T15:01:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.432+0000 7f3678b5cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db90af0b0 con 0x564db90ae310 2023-09-30T15:01:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.432+0000 7f366659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90ad200 0x564db90c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.432+0000 7f366659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90ad200 0x564db90c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40134/0 (socket says 172.21.15.32:40134) 2023-09-30T15:01:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.432+0000 7f366659c700 1 -- 172.21.15.32:0/2378843399 learned_addr learned my addr 172.21.15.32:0/2378843399 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.432+0000 7f3666d9d700 1 --2- 172.21.15.32:0/2378843399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90aecd0 0x564db90a7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.432+0000 7f366659c700 1 -- 172.21.15.32:0/2378843399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 msgr2=0x564db90be340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T15:01:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.432+0000 7f366659c700 1 --2- 172.21.15.32:0/2378843399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 0x564db90be340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.432+0000 7f366659c700 1 -- 172.21.15.32:0/2378843399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90aecd0 msgr2=0x564db90a7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.432+0000 7f366659c700 1 --2- 172.21.15.32:0/2378843399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90aecd0 0x564db90a7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.432+0000 7f366659c700 1 -- 172.21.15.32:0/2378843399 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564db8e13c00 con 0x564db90ad200 2023-09-30T15:01:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.432+0000 7f366659c700 1 --2- 172.21.15.32:0/2378843399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90ad200 0x564db90c5120 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3658005bf0 tx=0x7f3658011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b59b82df1307da8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.433+0000 7f36557fa700 1 -- 172.21.15.32:0/2378843399 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658022040 con 0x564db90ad200 2023-09-30T15:01:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.433+0000 7f36557fa700 1 -- 172.21.15.32:0/2378843399 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3658003e70 con 0x564db90ad200 2023-09-30T15:01:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.433+0000 7f36557fa700 1 -- 172.21.15.32:0/2378843399 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658005480 con 0x564db90ad200 2023-09-30T15:01:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.434+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2378843399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90ad200 msgr2=0x564db90c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:14.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.434+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/2378843399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90ad200 0x564db90c5120 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3658005bf0 tx=0x7f3658011850 comp rx=0 tx=0).stop 2023-09-30T15:01:14.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.435+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2378843399 shutdown_connections 2023-09-30T15:01:14.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.435+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/2378843399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90aecd0 0x564db90a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.435+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/2378843399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90ad200 0x564db90c5120 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.435+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/2378843399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 0x564db90be340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.435+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2378843399 >> 172.21.15.32:0/2378843399 conn(0x564db9021200 msgr2=0x564db90a7590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.435+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2378843399 shutdown_connections 2023-09-30T15:01:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.436+0000 7f3678b5cb80 1 -- 172.21.15.32:0/2378843399 wait complete. 2023-09-30T15:01:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.436+0000 7f3678b5cb80 1 -- start start 2023-09-30T15:01:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.436+0000 7f3678b5cb80 1 Processor -- start 2023-09-30T15:01:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.436+0000 7f3678b5cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90ad200 0x564db90bdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.436+0000 7f3678b5cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 0x564db90be1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.436+0000 7f3678b5cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90aecd0 0x564db90bc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.436+0000 7f3678b5cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db90bc980 con 0x564db90aecd0 2023-09-30T15:01:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.436+0000 7f3678b5cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db90bcaf0 con 0x564db90ad200 2023-09-30T15:01:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.436+0000 7f3678b5cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db90bccb0 con 0x564db90ae310 2023-09-30T15:01:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.436+0000 7f3678b5cb80 10 client.0 _ll_register_callbacks cb 0x7f3658004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:01:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.437+0000 7f366659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90ad200 0x564db90bdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.437+0000 7f3665d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 0x564db90be1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.437+0000 7f366659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90ad200 0x564db90bdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51614/0 (socket says 172.21.15.32:51614) 2023-09-30T15:01:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.437+0000 7f366659c700 1 -- 172.21.15.32:0/3047904323 learned_addr learned my addr 172.21.15.32:0/3047904323 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.437+0000 7f3666d9d700 1 --2- 172.21.15.32:0/3047904323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90aecd0 0x564db90bc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.437+0000 7f366659c700 1 -- 172.21.15.32:0/3047904323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 msgr2=0x564db90be1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.437+0000 7f366659c700 1 --2- 172.21.15.32:0/3047904323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 0x564db90be1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.437+0000 7f366659c700 1 -- 172.21.15.32:0/3047904323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90aecd0 msgr2=0x564db90bc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.437+0000 7f366659c700 1 --2- 172.21.15.32:0/3047904323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90aecd0 0x564db90bc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.437+0000 7f366659c700 1 -- 172.21.15.32:0/3047904323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564db8e2ad80 con 0x564db90ad200 2023-09-30T15:01:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.437+0000 7f3665d9b700 1 --2- 172.21.15.32:0/3047904323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 0x564db90be1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:01:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.437+0000 7f3666d9d700 1 --2- 172.21.15.32:0/3047904323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90aecd0 0x564db90bc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:01:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.437+0000 7f366659c700 1 --2- 172.21.15.32:0/3047904323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90ad200 0x564db90bdca0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f36580097c0 tx=0x7f3658005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b9592de4365103c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.437+0000 7f364dffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T15:01:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.437+0000 7f364dffb700 1 -- 172.21.15.32:0/3047904323 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658022070 con 0x564db90ad200 2023-09-30T15:01:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.438+0000 7f364dffb700 1 -- 172.21.15.32:0/3047904323 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f36580047c0 con 0x564db90ad200 2023-09-30T15:01:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.438+0000 7f3678b5cb80 1 -- 172.21.15.32:0/3047904323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f365c01e040 con 0x564db90ad200 2023-09-30T15:01:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.438+0000 7f364dffb700 1 -- 172.21.15.32:0/3047904323 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3658005800 con 0x564db90ad200 2023-09-30T15:01:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.438+0000 7f3678b5cb80 10 client.26951 fetch_fsmap learned FSMap version 55 2023-09-30T15:01:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.438+0000 7f3678b5cb80 1 -- 172.21.15.32:0/3047904323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x564db8e511f0 con 0x564db90ad200 2023-09-30T15:01:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.438+0000 7f364dffb700 1 -- 172.21.15.32:0/3047904323 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f365c01e040 con 0x564db90ad200 2023-09-30T15:01:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.438+0000 7f364dffb700 1 -- 172.21.15.32:0/3047904323 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3658027a80 con 0x564db90ad200 2023-09-30T15:01:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.438+0000 7f3678b5cb80 10 client.26951 fetch_fsmap finished waiting for FSMap version 55 2023-09-30T15:01:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.438+0000 7f3678b5cb80 10 client.26951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:01:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.438+0000 7f3678b5cb80 20 client.26951 populate_metadata read hostname 'smithi032' 2023-09-30T15:01:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.438+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/3047904323 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564db9117c40 0x564db911a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.438+0000 7f3678b5cb80 4 client.26951 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T15:01:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.438+0000 7f3678b5cb80 1 -- 172.21.15.32:0/3047904323 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564db8ff8ad0 con 0x564db9117c40 2023-09-30T15:01:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.439+0000 7f3665d9b700 1 --2- 172.21.15.32:0/3047904323 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564db9117c40 0x564db911a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.439+0000 7f364dffb700 10 client.26951 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.439+0000 7f3665d9b700 1 --2- 172.21.15.32:0/3047904323 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564db9117c40 0x564db911a060 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12b77c45cda21316 server_cookie=c68cbfa16ac26556 in_seq=0 out_seq=0 2023-09-30T15:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.837+0000 7f364dffb700 1 -- 172.21.15.32:0/3047904323 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564db8ff8ad0 con 0x564db9117c40 2023-09-30T15:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.837+0000 7f364dffb700 10 client.26951 handle_command_reply: tid=0 2023-09-30T15:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.837+0000 7f3678b5cb80 1 client.26951 shutdown 2023-09-30T15:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.837+0000 7f3678b5cb80 1 -- 172.21.15.32:0/3047904323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90ad200 msgr2=0x564db90bdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.837+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/3047904323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90ad200 0x564db90bdca0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f36580097c0 tx=0x7f3658005a20 comp rx=0 tx=0).stop 2023-09-30T15:01:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.837+0000 7f3678b5cb80 1 -- 172.21.15.32:0/3047904323 shutdown_connections 2023-09-30T15:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.837+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/3047904323 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x564db9117c40 0x564db911a060 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.837+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/3047904323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564db90aecd0 0x564db90bc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.837+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/3047904323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564db90ad200 0x564db90bdca0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.837+0000 7f3678b5cb80 1 --2- 172.21.15.32:0/3047904323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564db90ae310 0x564db90be1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.837+0000 7f3678b5cb80 1 -- 172.21.15.32:0/3047904323 >> 172.21.15.32:0/3047904323 conn(0x564db9021200 msgr2=0x564db90b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.837+0000 7f364dffb700 0 client.26951 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.837+0000 7f3678b5cb80 1 -- 172.21.15.32:0/3047904323 shutdown_connections 2023-09-30T15:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.837+0000 7f3678b5cb80 1 -- 172.21.15.32:0/3047904323 wait complete. 2023-09-30T15:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.838+0000 7f3678b5cb80 20 client.26951 trim_cache size 0 max 16384 2023-09-30T15:01:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T15:01:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T15:01:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T15:01:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T15:01:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.838+0000 7f366c974700 1 -- 172.21.15.32:0/3267669125 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f36500a0fa0 msgr2=0x7f36500a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.838+0000 7f366c974700 1 --2- 172.21.15.32:0/3267669125 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f36500a0fa0 0x7f36500a3460 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3668074b20 tx=0x7f3658009260 comp rx=0 tx=0).stop 2023-09-30T15:01:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.838+0000 7f366c974700 1 -- 172.21.15.32:0/3267669125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3668102a10 msgr2=0x7f3668073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.838+0000 7f366c974700 1 --2- 172.21.15.32:0/3267669125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3668102a10 0x7f3668073c90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f366000ef30 tx=0x7f366000c3a0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.838+0000 7f366c974700 1 -- 172.21.15.32:0/3267669125 shutdown_connections 2023-09-30T15:01:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.838+0000 7f366c974700 1 --2- 172.21.15.32:0/3267669125 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f36500a0fa0 0x7f36500a3460 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.838+0000 7f366c974700 1 --2- 172.21.15.32:0/3267669125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3668102a10 0x7f3668073c90 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.838+0000 7f366c974700 1 --2- 172.21.15.32:0/3267669125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3668100e70 0x7f3668073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.838+0000 7f366c974700 1 --2- 172.21.15.32:0/3267669125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3668102050 0x7f3668073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.838+0000 7f366c974700 1 -- 172.21.15.32:0/3267669125 >> 172.21.15.32:0/3267669125 conn(0x7f36680fc430 msgr2=0x7f36680fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.838+0000 7f366c974700 1 -- 172.21.15.32:0/3267669125 shutdown_connections 2023-09-30T15:01:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:14.839+0000 7f366c974700 1 -- 172.21.15.32:0/3267669125 wait complete. 2023-09-30T15:01:15.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 381786 34.98 MB/sec execute 468 sec latency 305.561 ms 2023-09-30T15:01:15.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:15 smithi156 ceph-mon[103579]: pgmap v976: 97 pgs: 97 active+clean; 826 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.9 MiB/s wr, 711 op/s 2023-09-30T15:01:15.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:15 smithi148 ceph-mon[102355]: pgmap v976: 97 pgs: 97 active+clean; 826 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.9 MiB/s wr, 711 op/s 2023-09-30T15:01:15.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:15 smithi032 ceph-mon[98592]: pgmap v976: 97 pgs: 97 active+clean; 826 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.9 MiB/s wr, 711 op/s 2023-09-30T15:01:16.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 382366 34.97 MB/sec execute 469 sec latency 55.944 ms 2023-09-30T15:01:17.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 383070 34.98 MB/sec execute 470 sec latency 66.664 ms 2023-09-30T15:01:17.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:17 smithi156 ceph-mon[103579]: pgmap v977: 97 pgs: 97 active+clean; 831 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.1 MiB/s wr, 639 op/s 2023-09-30T15:01:17.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:17 smithi148 ceph-mon[102355]: pgmap v977: 97 pgs: 97 active+clean; 831 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.1 MiB/s wr, 639 op/s 2023-09-30T15:01:17.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:17 smithi032 ceph-mon[98592]: pgmap v977: 97 pgs: 97 active+clean; 831 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.1 MiB/s wr, 639 op/s 2023-09-30T15:01:18.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 383547 34.96 MB/sec execute 471 sec latency 310.328 ms 2023-09-30T15:01:19.090 INFO:tasks.workunit.client.0.smithi032.stdout: 10 384111 34.94 MB/sec execute 472 sec latency 103.519 ms 2023-09-30T15:01:19.808 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:19 smithi156 ceph-mon[103579]: pgmap v978: 97 pgs: 97 active+clean; 818 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.7 MiB/s wr, 709 op/s 2023-09-30T15:01:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:19 smithi148 ceph-mon[102355]: pgmap v978: 97 pgs: 97 active+clean; 818 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.7 MiB/s wr, 709 op/s 2023-09-30T15:01:19.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:19 smithi032 ceph-mon[98592]: pgmap v978: 97 pgs: 97 active+clean; 818 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.7 MiB/s wr, 709 op/s 2023-09-30T15:01:20.089 INFO:tasks.workunit.client.0.smithi032.stdout: 10 384782 34.96 MB/sec execute 473 sec latency 60.218 ms 2023-09-30T15:01:21.090 INFO:tasks.workunit.client.0.smithi032.stdout: 10 385448 34.96 MB/sec execute 474 sec latency 75.062 ms 2023-09-30T15:01:21.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:21 smithi156 ceph-mon[103579]: pgmap v979: 97 pgs: 97 active+clean; 814 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.6 MiB/s wr, 648 op/s 2023-09-30T15:01:21.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:21 smithi148 ceph-mon[102355]: pgmap v979: 97 pgs: 97 active+clean; 814 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.6 MiB/s wr, 648 op/s 2023-09-30T15:01:21.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:21 smithi032 ceph-mon[98592]: pgmap v979: 97 pgs: 97 active+clean; 814 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.6 MiB/s wr, 648 op/s 2023-09-30T15:01:22.090 INFO:tasks.workunit.client.0.smithi032.stdout: 10 386011 34.94 MB/sec execute 475 sec latency 89.386 ms 2023-09-30T15:01:23.090 INFO:tasks.workunit.client.0.smithi032.stdout: 10 386490 34.92 MB/sec execute 476 sec latency 285.674 ms 2023-09-30T15:01:23.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:23 smithi156 ceph-mon[103579]: pgmap v980: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.8 MiB/s wr, 652 op/s 2023-09-30T15:01:23.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:23 smithi148 ceph-mon[102355]: pgmap v980: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.8 MiB/s wr, 652 op/s 2023-09-30T15:01:23.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:23 smithi032 ceph-mon[98592]: pgmap v980: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.8 MiB/s wr, 652 op/s 2023-09-30T15:01:24.090 INFO:tasks.workunit.client.0.smithi032.stdout: 10 387015 34.90 MB/sec execute 477 sec latency 307.643 ms 2023-09-30T15:01:25.090 INFO:tasks.workunit.client.0.smithi032.stdout: 10 387725 34.92 MB/sec execute 478 sec latency 45.449 ms 2023-09-30T15:01:25.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:25 smithi156 ceph-mon[103579]: pgmap v981: 97 pgs: 97 active+clean; 814 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.5 MiB/s wr, 708 op/s 2023-09-30T15:01:25.862 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:25 smithi148 ceph-mon[102355]: pgmap v981: 97 pgs: 97 active+clean; 814 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.5 MiB/s wr, 708 op/s 2023-09-30T15:01:25.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:25 smithi032 ceph-mon[98592]: pgmap v981: 97 pgs: 97 active+clean; 814 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.5 MiB/s wr, 708 op/s 2023-09-30T15:01:26.090 INFO:tasks.workunit.client.0.smithi032.stdout: 10 388322 34.92 MB/sec execute 479 sec latency 87.551 ms 2023-09-30T15:01:27.090 INFO:tasks.workunit.client.0.smithi032.stdout: 10 388906 34.92 MB/sec execute 480 sec latency 80.959 ms 2023-09-30T15:01:27.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:27 smithi156 ceph-mon[103579]: pgmap v982: 97 pgs: 97 active+clean; 818 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.3 MiB/s wr, 667 op/s 2023-09-30T15:01:27.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:27 smithi148 ceph-mon[102355]: pgmap v982: 97 pgs: 97 active+clean; 818 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.3 MiB/s wr, 667 op/s 2023-09-30T15:01:27.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:27 smithi032 ceph-mon[98592]: pgmap v982: 97 pgs: 97 active+clean; 818 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.3 MiB/s wr, 667 op/s 2023-09-30T15:01:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.035+0000 7f6ca5f50700 1 -- 172.21.15.32:0/2102016982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca010d160 msgr2=0x7f6ca010d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.035+0000 7f6ca5f50700 1 --2- 172.21.15.32:0/2102016982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca010d160 0x7f6ca010d5e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009f90 tx=0x7f6c9800b330 comp rx=0 tx=0).stop 2023-09-30T15:01:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.035+0000 7f6ca5f50700 1 -- 172.21.15.32:0/2102016982 shutdown_connections 2023-09-30T15:01:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.035+0000 7f6ca5f50700 1 --2- 172.21.15.32:0/2102016982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca010d160 0x7f6ca010d5e0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.035+0000 7f6ca5f50700 1 --2- 172.21.15.32:0/2102016982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca0107e20 0x7f6ca010a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.035+0000 7f6ca5f50700 1 --2- 172.21.15.32:0/2102016982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ca010a790 0x7f6ca010cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.036+0000 7f6ca5f50700 1 -- 172.21.15.32:0/2102016982 >> 172.21.15.32:0/2102016982 conn(0x7f6ca006dc80 msgr2=0x7f6ca00700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.039+0000 7f6ca5f50700 1 -- 172.21.15.32:0/2102016982 shutdown_connections 2023-09-30T15:01:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.039+0000 7f6ca5f50700 1 -- 172.21.15.32:0/2102016982 wait complete. 2023-09-30T15:01:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.040+0000 7f6ca5f50700 1 Processor -- start 2023-09-30T15:01:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.040+0000 7f6ca5f50700 1 -- start start 2023-09-30T15:01:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.040+0000 7f6ca5f50700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca0107e20 0x7f6ca01a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.040+0000 7f6ca5f50700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ca010a790 0x7f6ca01a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.040+0000 7f6ca5f50700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca010d160 0x7f6ca01a6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.040+0000 7f6ca5f50700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca01a75d0 con 0x7f6ca010d160 2023-09-30T15:01:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.040+0000 7f6ca5f50700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca01a7740 con 0x7f6ca0107e20 2023-09-30T15:01:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.040+0000 7f6ca5f50700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca01a7900 con 0x7f6ca010a790 2023-09-30T15:01:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.041+0000 7f6c9ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca010d160 0x7f6ca01a6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.041+0000 7f6c9ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca010d160 0x7f6ca01a6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40152/0 (socket says 172.21.15.32:40152) 2023-09-30T15:01:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.041+0000 7f6c9ffff700 1 -- 172.21.15.32:0/3504441671 learned_addr learned my addr 172.21.15.32:0/3504441671 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.041+0000 7f6c9effd700 1 --2- 172.21.15.32:0/3504441671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ca010a790 0x7f6ca01a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.041+0000 7f6c9ffff700 1 -- 172.21.15.32:0/3504441671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ca010a790 msgr2=0x7f6ca01a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.041+0000 7f6c9ffff700 1 --2- 172.21.15.32:0/3504441671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ca010a790 0x7f6ca01a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.041+0000 7f6c9ffff700 1 -- 172.21.15.32:0/3504441671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca0107e20 msgr2=0x7f6ca01a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.041+0000 7f6c9ffff700 1 --2- 172.21.15.32:0/3504441671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca0107e20 0x7f6ca01a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.041+0000 7f6c9ffff700 1 -- 172.21.15.32:0/3504441671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c98009c40 con 0x7f6ca010d160 2023-09-30T15:01:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.042+0000 7f6c9ffff700 1 --2- 172.21.15.32:0/3504441671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca010d160 0x7f6ca01a6f70 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6c98007a10 tx=0x7f6c9800bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.042+0000 7f6c9cff9700 1 -- 172.21.15.32:0/3504441671 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9800e040 con 0x7f6ca010d160 2023-09-30T15:01:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.042+0000 7f6c9cff9700 1 -- 172.21.15.32:0/3504441671 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c98004e40 con 0x7f6ca010d160 2023-09-30T15:01:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.042+0000 7f6c9cff9700 1 -- 172.21.15.32:0/3504441671 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c980054e0 con 0x7f6ca010d160 2023-09-30T15:01:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.042+0000 7f6ca5f50700 1 -- 172.21.15.32:0/3504441671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca01a7b80 con 0x7f6ca010d160 2023-09-30T15:01:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.042+0000 7f6ca5f50700 1 -- 172.21.15.32:0/3504441671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca01ae3a0 con 0x7f6ca010d160 2023-09-30T15:01:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.046+0000 7f6c9cff9700 1 -- 172.21.15.32:0/3504441671 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f6c98005640 con 0x7f6ca010d160 2023-09-30T15:01:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.047+0000 7f6c9cff9700 1 --2- 172.21.15.32:0/3504441671 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c880a10f0 0x7f6c880a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.047+0000 7f6c9cff9700 1 -- 172.21.15.32:0/3504441671 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6c980cdcb0 con 0x7f6ca010d160 2023-09-30T15:01:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.047+0000 7f6cb2138b80 1 Processor -- start 2023-09-30T15:01:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.047+0000 7f6cb2138b80 1 -- start start 2023-09-30T15:01:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.048+0000 7f6cb2138b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70ee200 0x5598b70ee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.048+0000 7f6cb2138b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70ef310 0x5598b70ef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.048+0000 7f6cb2138b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70efcd0 0x5598b70f6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.048+0000 7f6cb2138b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b7056400 con 0x5598b70efcd0 2023-09-30T15:01:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.048+0000 7f6cb2138b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b7047bc0 con 0x5598b70ee200 2023-09-30T15:01:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.048+0000 7f6cb2138b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b70454e0 con 0x5598b70ef310 2023-09-30T15:01:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.048+0000 7f6c9ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70efcd0 0x5598b70f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.048+0000 7f6c9f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70ee200 0x5598b70ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.048+0000 7f6c9ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70efcd0 0x5598b70f6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40160/0 (socket says 172.21.15.32:40160) 2023-09-30T15:01:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.048+0000 7f6c9ffff700 1 -- 172.21.15.32:0/4181991322 learned_addr learned my addr 172.21.15.32:0/4181991322 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.048+0000 7f6c9f7fe700 1 --2- 172.21.15.32:0/3504441671 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c880a10f0 0x7f6c880a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.049+0000 7f6c9ffff700 1 -- 172.21.15.32:0/4181991322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70ef310 msgr2=0x5598b70ef790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.049+0000 7f6c9ffff700 1 --2- 172.21.15.32:0/4181991322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70ef310 0x5598b70ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.049+0000 7f6c9ffff700 1 -- 172.21.15.32:0/4181991322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70ee200 msgr2=0x5598b70ee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.049+0000 7f6c9ffff700 1 --2- 172.21.15.32:0/4181991322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70ee200 0x5598b70ee620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.049+0000 7f6c9ffff700 1 -- 172.21.15.32:0/4181991322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598b70a3870 con 0x5598b70efcd0 2023-09-30T15:01:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.049+0000 7f6c9f7fe700 1 --2- 172.21.15.32:0/3504441671 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c880a10f0 0x7f6c880a35b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6c90012550 tx=0x7f6c9000a430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.049+0000 7f6c9ffff700 1 --2- 172.21.15.32:0/4181991322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70efcd0 0x5598b70f6560 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6c9809af10 tx=0x7f6c9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e2ccb3785260d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.049+0000 7f6c85ffb700 1 -- 172.21.15.32:0/4181991322 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9809f170 con 0x5598b70efcd0 2023-09-30T15:01:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.049+0000 7f6c85ffb700 1 -- 172.21.15.32:0/4181991322 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c9809b210 con 0x5598b70efcd0 2023-09-30T15:01:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.049+0000 7f6c85ffb700 1 -- 172.21.15.32:0/4181991322 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c980058f0 con 0x5598b70efcd0 2023-09-30T15:01:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.050+0000 7f6cb2138b80 1 -- 172.21.15.32:0/4181991322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70efcd0 msgr2=0x5598b70f6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.050+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/4181991322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70efcd0 0x5598b70f6560 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6c9809af10 tx=0x7f6c9800b330 comp rx=0 tx=0).stop 2023-09-30T15:01:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.053+0000 7f6cb2138b80 1 -- 172.21.15.32:0/4181991322 shutdown_connections 2023-09-30T15:01:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.053+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/4181991322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70efcd0 0x5598b70f6560 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.053+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/4181991322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70ee200 0x5598b70ee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.053+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/4181991322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70ef310 0x5598b70ef790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.054 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.053+0000 7f6cb2138b80 1 -- 172.21.15.32:0/4181991322 >> 172.21.15.32:0/4181991322 conn(0x5598b7062200 msgr2=0x5598b70e8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:28.056 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.054+0000 7f6cb2138b80 1 -- 172.21.15.32:0/4181991322 shutdown_connections 2023-09-30T15:01:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.057+0000 7f6cb2138b80 1 -- 172.21.15.32:0/4181991322 wait complete. 2023-09-30T15:01:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.057+0000 7f6cb2138b80 1 -- start start 2023-09-30T15:01:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.057+0000 7f6cb2138b80 1 Processor -- start 2023-09-30T15:01:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.074+0000 7f6cb2138b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70ee200 0x5598b70f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.074+0000 7f6cb2138b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70ef310 0x5598b70f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.074+0000 7f6cb2138b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70efcd0 0x5598b70ff230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.074+0000 7f6cb2138b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b7040dd0 con 0x5598b70ee200 2023-09-30T15:01:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.074+0000 7f6cb2138b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b6fdd6b0 con 0x5598b70ef310 2023-09-30T15:01:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.074+0000 7f6cb2138b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b6fdd350 con 0x5598b70efcd0 2023-09-30T15:01:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.074+0000 7f6cb2138b80 10 client.0 _ll_register_callbacks cb 0x5598b6e3a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:01:28.097 INFO:tasks.workunit.client.0.smithi032.stdout: 10 389321 34.89 MB/sec execute 481 sec latency 186.814 ms 2023-09-30T15:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.084+0000 7f6c9f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70ee200 0x5598b70f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.084+0000 7f6c9f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70ee200 0x5598b70f7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40168/0 (socket says 172.21.15.32:40168) 2023-09-30T15:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.084+0000 7f6c9f7fe700 1 -- 172.21.15.32:0/3467472322 learned_addr learned my addr 172.21.15.32:0/3467472322 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.086+0000 7f6c9effd700 1 --2- 172.21.15.32:0/3467472322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70ef310 0x5598b70f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.086+0000 7f6c9ffff700 1 --2- 172.21.15.32:0/3467472322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70efcd0 0x5598b70ff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.086+0000 7f6c9f7fe700 1 -- 172.21.15.32:0/3467472322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70efcd0 msgr2=0x5598b70ff230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.086+0000 7f6c9f7fe700 1 --2- 172.21.15.32:0/3467472322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70efcd0 0x5598b70ff230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.086+0000 7f6c9f7fe700 1 -- 172.21.15.32:0/3467472322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70ef310 msgr2=0x5598b70f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.086+0000 7f6c9f7fe700 1 --2- 172.21.15.32:0/3467472322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70ef310 0x5598b70f81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.086+0000 7f6c9f7fe700 1 -- 172.21.15.32:0/3467472322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca01ae3a0 con 0x5598b70ee200 2023-09-30T15:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.086+0000 7f6c9f7fe700 1 --2- 172.21.15.32:0/3467472322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70ee200 0x5598b70f7cb0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6c90004b80 tx=0x7f6c900090e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58cb444baeed67c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.086+0000 7f6c7effd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T15:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.087+0000 7f6c7effd700 1 -- 172.21.15.32:0/3467472322 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90003880 con 0x5598b70ee200 2023-09-30T15:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.087+0000 7f6c7effd700 1 -- 172.21.15.32:0/3467472322 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6c9001a050 con 0x5598b70ee200 2023-09-30T15:01:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.099+0000 7f6cb2138b80 1 -- 172.21.15.32:0/3467472322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598b70e1090 con 0x5598b70ee200 2023-09-30T15:01:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.110+0000 7f6c7effd700 1 -- 172.21.15.32:0/3467472322 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c900067b0 con 0x5598b70ee200 2023-09-30T15:01:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.115+0000 7f6cb2138b80 10 client.17346 fetch_fsmap learned FSMap version 55 2023-09-30T15:01:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.116+0000 7f6cb2138b80 1 -- 172.21.15.32:0/3467472322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x5598b6e6bd80 con 0x5598b70ee200 2023-09-30T15:01:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.116+0000 7f6c7effd700 1 -- 172.21.15.32:0/3467472322 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5598b70e1090 con 0x5598b70ee200 2023-09-30T15:01:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.116+0000 7f6c7effd700 1 -- 172.21.15.32:0/3467472322 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6c9001fba0 con 0x5598b70ee200 2023-09-30T15:01:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.116+0000 7f6cb2138b80 10 client.17346 fetch_fsmap finished waiting for FSMap version 55 2023-09-30T15:01:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.116+0000 7f6cb2138b80 10 client.17346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:01:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.116+0000 7f6cb2138b80 20 client.17346 populate_metadata read hostname 'smithi032' 2023-09-30T15:01:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.116+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/3467472322 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5598b7114ac0 0x5598b7116ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.116+0000 7f6cb2138b80 4 client.17346 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T15:01:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.116+0000 7f6cb2138b80 1 -- 172.21.15.32:0/3467472322 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598b7039db0 con 0x5598b7114ac0 2023-09-30T15:01:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.120+0000 7f6c9effd700 1 --2- 172.21.15.32:0/3467472322 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5598b7114ac0 0x5598b7116ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.134+0000 7f6c7effd700 10 client.17346 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.134+0000 7f6c9effd700 1 --2- 172.21.15.32:0/3467472322 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5598b7114ac0 0x5598b7116ee0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c18a6579c41370a server_cookie=241e4b36f13f8b3f in_seq=0 out_seq=0 2023-09-30T15:01:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.148+0000 7f6c7effd700 1 -- 172.21.15.32:0/3467472322 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5598b7039db0 con 0x5598b7114ac0 2023-09-30T15:01:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.148+0000 7f6c7effd700 10 client.17346 handle_command_reply: tid=0 2023-09-30T15:01:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.149+0000 7f6cb2138b80 1 client.17346 shutdown 2023-09-30T15:01:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.151+0000 7f6cb2138b80 1 -- 172.21.15.32:0/3467472322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70ee200 msgr2=0x5598b70f7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.151+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/3467472322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70ee200 0x5598b70f7cb0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6c90004b80 tx=0x7f6c900090e0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.152+0000 7f6cb2138b80 1 -- 172.21.15.32:0/3467472322 shutdown_connections 2023-09-30T15:01:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.152+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/3467472322 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5598b7114ac0 0x5598b7116ee0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.152+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/3467472322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70efcd0 0x5598b70ff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.152+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/3467472322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70ee200 0x5598b70f7cb0 secure :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6c90004b80 tx=0x7f6c900090e0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.152+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/3467472322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70ef310 0x5598b70f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.152+0000 7f6cb2138b80 1 -- 172.21.15.32:0/3467472322 >> 172.21.15.32:0/3467472322 conn(0x5598b7062200 msgr2=0x5598b70e8590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.152+0000 7f6c7effd700 0 client.17346 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.158+0000 7f6cb2138b80 1 -- 172.21.15.32:0/3467472322 shutdown_connections 2023-09-30T15:01:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.158+0000 7f6cb2138b80 1 -- 172.21.15.32:0/3467472322 wait complete. 2023-09-30T15:01:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.159+0000 7f6cb2138b80 20 client.17346 trim_cache size 0 max 16384 2023-09-30T15:01:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.185+0000 7f6cb2138b80 1 Processor -- start 2023-09-30T15:01:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.185+0000 7f6cb2138b80 1 -- start start 2023-09-30T15:01:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.186+0000 7f6cb2138b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70ef310 0x5598b70f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.186+0000 7f6cb2138b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70efcd0 0x5598b70ff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.186+0000 7f6cb2138b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70e5240 0x5598b70f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.186+0000 7f6cb2138b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b6ec18c0 con 0x5598b70ef310 2023-09-30T15:01:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.186+0000 7f6cb2138b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b70f7820 con 0x5598b70efcd0 2023-09-30T15:01:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.186+0000 7f6cb2138b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b70f00f0 con 0x5598b70e5240 2023-09-30T15:01:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.186+0000 7f6c9f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70ef310 0x5598b70f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.186+0000 7f6c9f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70ef310 0x5598b70f7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40176/0 (socket says 172.21.15.32:40176) 2023-09-30T15:01:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.186+0000 7f6c9f7fe700 1 -- 172.21.15.32:0/1622448506 learned_addr learned my addr 172.21.15.32:0/1622448506 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.186+0000 7f6c9effd700 1 --2- 172.21.15.32:0/1622448506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70efcd0 0x5598b70ff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.186+0000 7f6c9ffff700 1 --2- 172.21.15.32:0/1622448506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70e5240 0x5598b70f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.186+0000 7f6c9f7fe700 1 -- 172.21.15.32:0/1622448506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70e5240 msgr2=0x5598b70f7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.186+0000 7f6c9f7fe700 1 --2- 172.21.15.32:0/1622448506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70e5240 0x5598b70f7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.186+0000 7f6c9f7fe700 1 -- 172.21.15.32:0/1622448506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70efcd0 msgr2=0x5598b70ff1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.186+0000 7f6c9f7fe700 1 --2- 172.21.15.32:0/1622448506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70efcd0 0x5598b70ff1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.186+0000 7f6c9f7fe700 1 -- 172.21.15.32:0/1622448506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598b6e54c00 con 0x5598b70ef310 2023-09-30T15:01:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.187+0000 7f6c9f7fe700 1 --2- 172.21.15.32:0/1622448506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70ef310 0x5598b70f7a60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f6c90006a70 tx=0x7f6c90012ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f214d4eca5be08 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.187+0000 7f6c867fc700 1 -- 172.21.15.32:0/1622448506 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c900067b0 con 0x5598b70ef310 2023-09-30T15:01:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.187+0000 7f6c867fc700 1 -- 172.21.15.32:0/1622448506 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c9000ab30 con 0x5598b70ef310 2023-09-30T15:01:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.187+0000 7f6c867fc700 1 -- 172.21.15.32:0/1622448506 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90014850 con 0x5598b70ef310 2023-09-30T15:01:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.188+0000 7f6cb2138b80 1 -- 172.21.15.32:0/1622448506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70ef310 msgr2=0x5598b70f7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.188+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/1622448506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70ef310 0x5598b70f7a60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f6c90006a70 tx=0x7f6c90012ca0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.188+0000 7f6cb2138b80 1 -- 172.21.15.32:0/1622448506 shutdown_connections 2023-09-30T15:01:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.188+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/1622448506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70e5240 0x5598b70f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.188+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/1622448506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70ef310 0x5598b70f7a60 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.188+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/1622448506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70efcd0 0x5598b70ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.188+0000 7f6cb2138b80 1 -- 172.21.15.32:0/1622448506 >> 172.21.15.32:0/1622448506 conn(0x5598b7062200 msgr2=0x5598b70f2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.188+0000 7f6cb2138b80 1 -- 172.21.15.32:0/1622448506 shutdown_connections 2023-09-30T15:01:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.188+0000 7f6cb2138b80 1 -- 172.21.15.32:0/1622448506 wait complete. 2023-09-30T15:01:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.189+0000 7f6cb2138b80 1 -- start start 2023-09-30T15:01:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.189+0000 7f6cb2138b80 1 Processor -- start 2023-09-30T15:01:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.189+0000 7f6cb2138b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70e5240 0x5598b70fcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.189+0000 7f6cb2138b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70ef310 0x5598b70fd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.189+0000 7f6cb2138b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70efcd0 0x5598b70fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.189+0000 7f6cb2138b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b70fe830 con 0x5598b70efcd0 2023-09-30T15:01:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.189+0000 7f6cb2138b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b70fe9a0 con 0x5598b70e5240 2023-09-30T15:01:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.189+0000 7f6cb2138b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b70feb60 con 0x5598b70ef310 2023-09-30T15:01:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.189+0000 7f6cb2138b80 10 client.0 _ll_register_callbacks cb 0x5598b6e3a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:01:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.190+0000 7f6c9f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70e5240 0x5598b70fcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.190+0000 7f6c9ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70efcd0 0x5598b70fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.190+0000 7f6c9f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70e5240 0x5598b70fcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51658/0 (socket says 172.21.15.32:51658) 2023-09-30T15:01:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.190+0000 7f6c9f7fe700 1 -- 172.21.15.32:0/1408213684 learned_addr learned my addr 172.21.15.32:0/1408213684 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.190+0000 7f6c9effd700 1 --2- 172.21.15.32:0/1408213684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70ef310 0x5598b70fd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.190+0000 7f6c9ffff700 1 -- 172.21.15.32:0/1408213684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70ef310 msgr2=0x5598b70fd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.190+0000 7f6c9ffff700 1 --2- 172.21.15.32:0/1408213684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70ef310 0x5598b70fd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.190+0000 7f6c9ffff700 1 -- 172.21.15.32:0/1408213684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70e5240 msgr2=0x5598b70fcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.190+0000 7f6c9ffff700 1 --2- 172.21.15.32:0/1408213684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70e5240 0x5598b70fcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.190+0000 7f6c9ffff700 1 -- 172.21.15.32:0/1408213684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598b6e6bd80 con 0x5598b70efcd0 2023-09-30T15:01:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.191+0000 7f6c9ffff700 1 --2- 172.21.15.32:0/1408213684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70efcd0 0x5598b70fe2f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x5598b70f0290 tx=0x7f6c9809b380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8413ad993f87cc3f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.191+0000 7f6c7effd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T15:01:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.193+0000 7f6c7effd700 1 -- 172.21.15.32:0/1408213684 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9809f360 con 0x5598b70efcd0 2023-09-30T15:01:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.193+0000 7f6c7effd700 1 -- 172.21.15.32:0/1408213684 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6c980a3510 con 0x5598b70efcd0 2023-09-30T15:01:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.195+0000 7f6cb2138b80 1 -- 172.21.15.32:0/1408213684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c9809b210 con 0x5598b70efcd0 2023-09-30T15:01:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.198+0000 7f6c7effd700 1 -- 172.21.15.32:0/1408213684 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c9809b470 con 0x5598b70efcd0 2023-09-30T15:01:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.198+0000 7f6cb2138b80 10 client.17352 fetch_fsmap learned FSMap version 55 2023-09-30T15:01:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.198+0000 7f6cb2138b80 1 -- 172.21.15.32:0/1408213684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x5598b6e921f0 con 0x5598b70efcd0 2023-09-30T15:01:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.199+0000 7f6c7effd700 1 -- 172.21.15.32:0/1408213684 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c9809b210 con 0x5598b70efcd0 2023-09-30T15:01:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.199+0000 7f6c7effd700 1 -- 172.21.15.32:0/1408213684 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6c980a3080 con 0x5598b70efcd0 2023-09-30T15:01:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.199+0000 7f6cb2138b80 10 client.17352 fetch_fsmap finished waiting for FSMap version 55 2023-09-30T15:01:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.199+0000 7f6cb2138b80 10 client.17352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:01:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.199+0000 7f6cb2138b80 20 client.17352 populate_metadata read hostname 'smithi032' 2023-09-30T15:01:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.199+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/1408213684 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5598b7169200 0x5598b716b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.199+0000 7f6cb2138b80 4 client.17352 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T15:01:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.199+0000 7f6cb2138b80 1 -- 172.21.15.32:0/1408213684 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5598b7039ad0 con 0x5598b7169200 2023-09-30T15:01:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.201+0000 7f6c9f7fe700 1 --2- 172.21.15.32:0/1408213684 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5598b7169200 0x5598b716b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.202+0000 7f6c7effd700 10 client.17352 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.202+0000 7f6c9f7fe700 1 --2- 172.21.15.32:0/1408213684 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5598b7169200 0x5598b716b620 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83b0678155f90c8 server_cookie=1889eef6ad27d339 in_seq=0 out_seq=0 2023-09-30T15:01:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.744+0000 7f6c7effd700 1 -- 172.21.15.32:0/1408213684 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5598b7039ad0 con 0x5598b7169200 2023-09-30T15:01:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.744+0000 7f6c7effd700 10 client.17352 handle_command_reply: tid=0 2023-09-30T15:01:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.744+0000 7f6cb2138b80 1 client.17352 shutdown 2023-09-30T15:01:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.745+0000 7f6cb2138b80 1 -- 172.21.15.32:0/1408213684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70efcd0 msgr2=0x5598b70fe2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.745+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/1408213684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70efcd0 0x5598b70fe2f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x5598b70f0290 tx=0x7f6c9809b380 comp rx=0 tx=0).stop 2023-09-30T15:01:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.745+0000 7f6cb2138b80 1 -- 172.21.15.32:0/1408213684 shutdown_connections 2023-09-30T15:01:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.745+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/1408213684 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5598b7169200 0x5598b716b620 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.745+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/1408213684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5598b70efcd0 0x5598b70fe2f0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.745+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/1408213684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598b70e5240 0x5598b70fcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.745+0000 7f6cb2138b80 1 --2- 172.21.15.32:0/1408213684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598b70ef310 0x5598b70fd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.745+0000 7f6cb2138b80 1 -- 172.21.15.32:0/1408213684 >> 172.21.15.32:0/1408213684 conn(0x5598b7062200 msgr2=0x5598b70f7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.745+0000 7f6c7effd700 0 client.17352 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.745+0000 7f6cb2138b80 1 -- 172.21.15.32:0/1408213684 shutdown_connections 2023-09-30T15:01:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.745+0000 7f6cb2138b80 1 -- 172.21.15.32:0/1408213684 wait complete. 2023-09-30T15:01:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.745+0000 7f6cb2138b80 20 client.17352 trim_cache size 0 max 16384 2023-09-30T15:01:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T15:01:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T15:01:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T15:01:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T15:01:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.746+0000 7f6ca5f50700 1 -- 172.21.15.32:0/3504441671 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c880a10f0 msgr2=0x7f6c880a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.746+0000 7f6ca5f50700 1 --2- 172.21.15.32:0/3504441671 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c880a10f0 0x7f6c880a35b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6c90012550 tx=0x7f6c9000a430 comp rx=0 tx=0).stop 2023-09-30T15:01:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.746+0000 7f6ca5f50700 1 -- 172.21.15.32:0/3504441671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca010d160 msgr2=0x7f6ca01a6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.746+0000 7f6ca5f50700 1 --2- 172.21.15.32:0/3504441671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca010d160 0x7f6ca01a6f70 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6c98007a10 tx=0x7f6c9800bcd0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.746+0000 7f6ca5f50700 1 -- 172.21.15.32:0/3504441671 shutdown_connections 2023-09-30T15:01:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.746+0000 7f6ca5f50700 1 --2- 172.21.15.32:0/3504441671 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c880a10f0 0x7f6c880a35b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.746+0000 7f6ca5f50700 1 --2- 172.21.15.32:0/3504441671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca010d160 0x7f6ca01a6f70 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.746+0000 7f6ca5f50700 1 --2- 172.21.15.32:0/3504441671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca0107e20 0x7f6ca01a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.746+0000 7f6ca5f50700 1 --2- 172.21.15.32:0/3504441671 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ca010a790 0x7f6ca01a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.746+0000 7f6ca5f50700 1 -- 172.21.15.32:0/3504441671 >> 172.21.15.32:0/3504441671 conn(0x7f6ca006dc80 msgr2=0x7f6ca010bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.746+0000 7f6ca5f50700 1 -- 172.21.15.32:0/3504441671 shutdown_connections 2023-09-30T15:01:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:28.746+0000 7f6ca5f50700 1 -- 172.21.15.32:0/3504441671 wait complete. 2023-09-30T15:01:29.093 INFO:tasks.workunit.client.0.smithi032.stdout: 10 389527 34.84 MB/sec execute 482 sec latency 304.568 ms 2023-09-30T15:01:29.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:29 smithi148 ceph-mon[102355]: pgmap v983: 97 pgs: 97 active+clean; 822 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.7 MiB/s wr, 671 op/s 2023-09-30T15:01:29.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:29 smithi032 ceph-mon[98592]: pgmap v983: 97 pgs: 97 active+clean; 822 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.7 MiB/s wr, 671 op/s 2023-09-30T15:01:30.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:29 smithi156 ceph-mon[103579]: pgmap v983: 97 pgs: 97 active+clean; 822 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.7 MiB/s wr, 671 op/s 2023-09-30T15:01:30.093 INFO:tasks.workunit.client.0.smithi032.stdout: 10 390130 34.82 MB/sec execute 483 sec latency 153.000 ms 2023-09-30T15:01:31.094 INFO:tasks.workunit.client.0.smithi032.stdout: 10 390935 34.83 MB/sec execute 484 sec latency 56.035 ms 2023-09-30T15:01:31.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:31 smithi148 ceph-mon[102355]: pgmap v984: 97 pgs: 97 active+clean; 823 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 606 op/s 2023-09-30T15:01:31.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:31 smithi032 ceph-mon[98592]: pgmap v984: 97 pgs: 97 active+clean; 823 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 606 op/s 2023-09-30T15:01:32.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:31 smithi156 ceph-mon[103579]: pgmap v984: 97 pgs: 97 active+clean; 823 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 606 op/s 2023-09-30T15:01:32.093 INFO:tasks.workunit.client.0.smithi032.stdout: 10 391695 34.86 MB/sec execute 485 sec latency 43.072 ms 2023-09-30T15:01:33.097 INFO:tasks.workunit.client.0.smithi032.stdout: 10 392327 34.87 MB/sec execute 486 sec latency 81.575 ms 2023-09-30T15:01:33.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:33 smithi148 ceph-mon[102355]: pgmap v985: 97 pgs: 97 active+clean; 820 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.1 MiB/s wr, 586 op/s 2023-09-30T15:01:33.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:33 smithi032 ceph-mon[98592]: pgmap v985: 97 pgs: 97 active+clean; 820 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.1 MiB/s wr, 586 op/s 2023-09-30T15:01:34.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:33 smithi156 ceph-mon[103579]: pgmap v985: 97 pgs: 97 active+clean; 820 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.1 MiB/s wr, 586 op/s 2023-09-30T15:01:34.095 INFO:tasks.workunit.client.0.smithi032.stdout: 10 392634 34.83 MB/sec execute 487 sec latency 293.051 ms 2023-09-30T15:01:35.095 INFO:tasks.workunit.client.0.smithi032.stdout: 10 393181 34.80 MB/sec execute 488 sec latency 62.064 ms 2023-09-30T15:01:35.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:35 smithi032 ceph-mon[98592]: pgmap v986: 97 pgs: 97 active+clean; 798 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.7 MiB/s wr, 695 op/s 2023-09-30T15:01:36.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:35 smithi156 ceph-mon[103579]: pgmap v986: 97 pgs: 97 active+clean; 798 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.7 MiB/s wr, 695 op/s 2023-09-30T15:01:36.095 INFO:tasks.workunit.client.0.smithi032.stdout: 10 393940 34.82 MB/sec execute 489 sec latency 75.304 ms 2023-09-30T15:01:36.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:35 smithi148 ceph-mon[102355]: pgmap v986: 97 pgs: 97 active+clean; 798 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.7 MiB/s wr, 695 op/s 2023-09-30T15:01:37.095 INFO:tasks.workunit.client.0.smithi032.stdout: 10 394654 34.82 MB/sec execute 490 sec latency 75.923 ms 2023-09-30T15:01:37.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:37 smithi032 ceph-mon[98592]: pgmap v987: 97 pgs: 97 active+clean; 804 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.3 MiB/s wr, 696 op/s 2023-09-30T15:01:38.058 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:37 smithi156 ceph-mon[103579]: pgmap v987: 97 pgs: 97 active+clean; 804 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.3 MiB/s wr, 696 op/s 2023-09-30T15:01:38.095 INFO:tasks.workunit.client.0.smithi032.stdout: 10 395454 34.85 MB/sec execute 491 sec latency 47.497 ms 2023-09-30T15:01:38.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:37 smithi148 ceph-mon[102355]: pgmap v987: 97 pgs: 97 active+clean; 804 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.3 MiB/s wr, 696 op/s 2023-09-30T15:01:39.095 INFO:tasks.workunit.client.0.smithi032.stdout: 10 395747 34.81 MB/sec execute 492 sec latency 299.789 ms 2023-09-30T15:01:39.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:39 smithi032 ceph-mon[98592]: pgmap v988: 97 pgs: 97 active+clean; 804 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.6 MiB/s wr, 632 op/s 2023-09-30T15:01:40.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:39 smithi156 ceph-mon[103579]: pgmap v988: 97 pgs: 97 active+clean; 804 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.6 MiB/s wr, 632 op/s 2023-09-30T15:01:40.095 INFO:tasks.workunit.client.0.smithi032.stdout: 10 396216 34.79 MB/sec execute 493 sec latency 112.402 ms 2023-09-30T15:01:40.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:39 smithi148 ceph-mon[102355]: pgmap v988: 97 pgs: 97 active+clean; 804 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.6 MiB/s wr, 632 op/s 2023-09-30T15:01:41.095 INFO:tasks.workunit.client.0.smithi032.stdout: 10 396768 34.78 MB/sec execute 494 sec latency 69.767 ms 2023-09-30T15:01:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.877+0000 7f0cdb49c700 1 -- 172.21.15.32:0/526320967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd4101900 msgr2=0x7f0cd41081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.877+0000 7f0cdb49c700 1 --2- 172.21.15.32:0/526320967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd4101900 0x7f0cd41081b0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00b0d0 tx=0x7f0ccc00b3e0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.877+0000 7f0cdb49c700 1 -- 172.21.15.32:0/526320967 shutdown_connections 2023-09-30T15:01:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.877+0000 7f0cdb49c700 1 --2- 172.21.15.32:0/526320967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd4101900 0x7f0cd41081b0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.877+0000 7f0cdb49c700 1 --2- 172.21.15.32:0/526320967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cd40ffd70 0x7f0cd4100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.877+0000 7f0cdb49c700 1 --2- 172.21.15.32:0/526320967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd4100f60 0x7f0cd41013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.877+0000 7f0cdb49c700 1 -- 172.21.15.32:0/526320967 >> 172.21.15.32:0/526320967 conn(0x7f0cd40fb2a0 msgr2=0x7f0cd40fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.877+0000 7f0cdb49c700 1 -- 172.21.15.32:0/526320967 shutdown_connections 2023-09-30T15:01:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.877+0000 7f0cdb49c700 1 -- 172.21.15.32:0/526320967 wait complete. 2023-09-30T15:01:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.878+0000 7f0cdb49c700 1 Processor -- start 2023-09-30T15:01:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.878+0000 7f0cdb49c700 1 -- start start 2023-09-30T15:01:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.878+0000 7f0cdb49c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd40ffd70 0x7f0cd4102340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.878+0000 7f0cdb49c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cd4100f60 0x7f0cd4102880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.878+0000 7f0cdb49c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd4101900 0x7f0cd4102dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.878+0000 7f0cdb49c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd41a0ca0 con 0x7f0cd40ffd70 2023-09-30T15:01:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.878+0000 7f0cdb49c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd41a0de0 con 0x7f0cd4101900 2023-09-30T15:01:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.878+0000 7f0cdb49c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd41a0f20 con 0x7f0cd4100f60 2023-09-30T15:01:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.878+0000 7f0cd9238700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd40ffd70 0x7f0cd4102340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.879+0000 7f0cd9238700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd40ffd70 0x7f0cd4102340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40194/0 (socket says 172.21.15.32:40194) 2023-09-30T15:01:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.879+0000 7f0cd9238700 1 -- 172.21.15.32:0/2249679296 learned_addr learned my addr 172.21.15.32:0/2249679296 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.879+0000 7f0cd9a39700 1 --2- 172.21.15.32:0/2249679296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd4101900 0x7f0cd4102dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.879+0000 7f0cd8a37700 1 --2- 172.21.15.32:0/2249679296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cd4100f60 0x7f0cd4102880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.879+0000 7f0cd9238700 1 -- 172.21.15.32:0/2249679296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cd4100f60 msgr2=0x7f0cd4102880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.879+0000 7f0cd9238700 1 --2- 172.21.15.32:0/2249679296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cd4100f60 0x7f0cd4102880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.879+0000 7f0cd9238700 1 -- 172.21.15.32:0/2249679296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd4101900 msgr2=0x7f0cd4102dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.879+0000 7f0cd9238700 1 --2- 172.21.15.32:0/2249679296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd4101900 0x7f0cd4102dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.879+0000 7f0cd9238700 1 -- 172.21.15.32:0/2249679296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc009d00 con 0x7f0cd40ffd70 2023-09-30T15:01:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.879+0000 7f0cd9a39700 1 --2- 172.21.15.32:0/2249679296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd4101900 0x7f0cd4102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:01:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.879+0000 7f0cd9238700 1 --2- 172.21.15.32:0/2249679296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd40ffd70 0x7f0cd4102340 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000e970 tx=0x7f0cd000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.879+0000 7f0cca7fc700 1 -- 172.21.15.32:0/2249679296 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0009790 con 0x7f0cd40ffd70 2023-09-30T15:01:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.879+0000 7f0cca7fc700 1 -- 172.21.15.32:0/2249679296 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd000c7e0 con 0x7f0cd40ffd70 2023-09-30T15:01:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.879+0000 7f0cdb49c700 1 -- 172.21.15.32:0/2249679296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd41a11b0 con 0x7f0cd40ffd70 2023-09-30T15:01:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.879+0000 7f0cca7fc700 1 -- 172.21.15.32:0/2249679296 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0003c70 con 0x7f0cd40ffd70 2023-09-30T15:01:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.879+0000 7f0cdb49c700 1 -- 172.21.15.32:0/2249679296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd41a1710 con 0x7f0cd40ffd70 2023-09-30T15:01:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.882+0000 7f0cca7fc700 1 -- 172.21.15.32:0/2249679296 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f0cd000cc50 con 0x7f0cd40ffd70 2023-09-30T15:01:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.882+0000 7f0cca7fc700 1 --2- 172.21.15.32:0/2249679296 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0cc00a1020 0x7f0cc00a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.882+0000 7f0cca7fc700 1 -- 172.21.15.32:0/2249679296 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0cd0014070 con 0x7f0cd40ffd70 2023-09-30T15:01:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.882+0000 7f0cd8a37700 1 --2- 172.21.15.32:0/2249679296 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0cc00a1020 0x7f0cc00a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.883+0000 7f0cd8a37700 1 --2- 172.21.15.32:0/2249679296 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0cc00a1020 0x7f0cc00a34e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4103640 tx=0x7f0cc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.884+0000 7f0ce7684b80 1 Processor -- start 2023-09-30T15:01:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.884+0000 7f0ce7684b80 1 -- start start 2023-09-30T15:01:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.884+0000 7f0ce7684b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87bf200 0x5643a87bf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.884+0000 7f0ce7684b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87c0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.884+0000 7f0ce7684b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87c0cd0 0x5643a87c7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.884+0000 7f0ce7684b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a8727400 con 0x5643a87c0310 2023-09-30T15:01:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.884+0000 7f0ce7684b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a8718bc0 con 0x5643a87bf200 2023-09-30T15:01:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.884+0000 7f0ce7684b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a87164e0 con 0x5643a87c0cd0 2023-09-30T15:01:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.884+0000 7f0cd8a37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87c0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.884+0000 7f0cd8a37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87c0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40204/0 (socket says 172.21.15.32:40204) 2023-09-30T15:01:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.884+0000 7f0cd8a37700 1 -- 172.21.15.32:0/3016368501 learned_addr learned my addr 172.21.15.32:0/3016368501 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.884+0000 7f0cd8a37700 1 -- 172.21.15.32:0/3016368501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87c0cd0 msgr2=0x5643a87c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.884+0000 7f0cd8a37700 1 --2- 172.21.15.32:0/3016368501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87c0cd0 0x5643a87c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.884+0000 7f0cd8a37700 1 -- 172.21.15.32:0/3016368501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87bf200 msgr2=0x5643a87bf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.884+0000 7f0cd8a37700 1 --2- 172.21.15.32:0/3016368501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87bf200 0x5643a87bf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.884+0000 7f0cd8a37700 1 -- 172.21.15.32:0/3016368501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643a8774870 con 0x5643a87c0310 2023-09-30T15:01:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.885+0000 7f0cd8a37700 1 --2- 172.21.15.32:0/3016368501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87c0790 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4011d50 tx=0x7f0cc4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cea5fc94ac42c0f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.885+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/3016368501 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc400ebd0 con 0x5643a87c0310 2023-09-30T15:01:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.885+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/3016368501 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cc4021070 con 0x5643a87c0310 2023-09-30T15:01:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.885+0000 7f0cbf7fe700 1 -- 172.21.15.32:0/3016368501 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4003cf0 con 0x5643a87c0310 2023-09-30T15:01:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.886+0000 7f0ce7684b80 1 -- 172.21.15.32:0/3016368501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 msgr2=0x5643a87c0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.886+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/3016368501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87c0790 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4011d50 tx=0x7f0cc4014000 comp rx=0 tx=0).stop 2023-09-30T15:01:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.886+0000 7f0ce7684b80 1 -- 172.21.15.32:0/3016368501 shutdown_connections 2023-09-30T15:01:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.886+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/3016368501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87c0cd0 0x5643a87c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.886+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/3016368501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87bf200 0x5643a87bf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.886+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/3016368501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87c0790 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.886+0000 7f0ce7684b80 1 -- 172.21.15.32:0/3016368501 >> 172.21.15.32:0/3016368501 conn(0x5643a8733200 msgr2=0x5643a87b9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.886+0000 7f0ce7684b80 1 -- 172.21.15.32:0/3016368501 shutdown_connections 2023-09-30T15:01:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.886+0000 7f0ce7684b80 1 -- 172.21.15.32:0/3016368501 wait complete. 2023-09-30T15:01:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.886+0000 7f0ce7684b80 1 -- start start 2023-09-30T15:01:41.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:41 smithi032 ceph-mon[98592]: pgmap v989: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.0 MiB/s wr, 622 op/s 2023-09-30T15:01:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.886+0000 7f0ce7684b80 1 Processor -- start 2023-09-30T15:01:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0ce7684b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87bf200 0x5643a87c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0ce7684b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0ce7684b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87c0cd0 0x5643a87d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0ce7684b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a8711dd0 con 0x5643a87c0310 2023-09-30T15:01:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0ce7684b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a86ae6b0 con 0x5643a87c0cd0 2023-09-30T15:01:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0ce7684b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a86ae350 con 0x5643a87bf200 2023-09-30T15:01:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0ce7684b80 10 client.0 _ll_register_callbacks cb 0x5643a850b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:01:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0cd8a37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0cd8a37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87c94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40210/0 (socket says 172.21.15.32:40210) 2023-09-30T15:01:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0cd8a37700 1 -- 172.21.15.32:0/3700059721 learned_addr learned my addr 172.21.15.32:0/3700059721 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0cd9238700 1 --2- 172.21.15.32:0/3700059721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87bf200 0x5643a87c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0cd9a39700 1 --2- 172.21.15.32:0/3700059721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87c0cd0 0x5643a87d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0cd8a37700 1 -- 172.21.15.32:0/3700059721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87bf200 msgr2=0x5643a87c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0cd8a37700 1 --2- 172.21.15.32:0/3700059721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87bf200 0x5643a87c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0cd8a37700 1 -- 172.21.15.32:0/3700059721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87c0cd0 msgr2=0x5643a87d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0cd8a37700 1 --2- 172.21.15.32:0/3700059721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87c0cd0 0x5643a87d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0cd8a37700 1 -- 172.21.15.32:0/3700059721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc4017070 con 0x5643a87c0310 2023-09-30T15:01:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0cd8a37700 1 --2- 172.21.15.32:0/3700059721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87c94b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40037b0 tx=0x7f0cc400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdab2c5f6b8c02 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0cbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T15:01:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.887+0000 7f0cbcff9700 1 -- 172.21.15.32:0/3700059721 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc402a040 con 0x5643a87c0310 2023-09-30T15:01:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.888+0000 7f0cbcff9700 1 -- 172.21.15.32:0/3700059721 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0cc4003e40 con 0x5643a87c0310 2023-09-30T15:01:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.888+0000 7f0ce7684b80 1 -- 172.21.15.32:0/3700059721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643a87b2090 con 0x5643a87c0310 2023-09-30T15:01:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.888+0000 7f0cbcff9700 1 -- 172.21.15.32:0/3700059721 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cc40050c0 con 0x5643a87c0310 2023-09-30T15:01:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.888+0000 7f0ce7684b80 10 client.17364 fetch_fsmap learned FSMap version 55 2023-09-30T15:01:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.888+0000 7f0ce7684b80 1 -- 172.21.15.32:0/3700059721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x5643a853cd80 con 0x5643a87c0310 2023-09-30T15:01:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.888+0000 7f0cbcff9700 1 -- 172.21.15.32:0/3700059721 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5643a87b2090 con 0x5643a87c0310 2023-09-30T15:01:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.888+0000 7f0cbcff9700 1 -- 172.21.15.32:0/3700059721 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0cc4026c00 con 0x5643a87c0310 2023-09-30T15:01:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.888+0000 7f0ce7684b80 10 client.17364 fetch_fsmap finished waiting for FSMap version 55 2023-09-30T15:01:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.888+0000 7f0ce7684b80 10 client.17364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:01:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.888+0000 7f0ce7684b80 20 client.17364 populate_metadata read hostname 'smithi032' 2023-09-30T15:01:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.888+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/3700059721 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5643a87d4ce0 0x5643a87d7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.888+0000 7f0ce7684b80 4 client.17364 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T15:01:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.888+0000 7f0ce7684b80 1 -- 172.21.15.32:0/3700059721 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643a870adb0 con 0x5643a87d4ce0 2023-09-30T15:01:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.888+0000 7f0cd9238700 1 --2- 172.21.15.32:0/3700059721 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5643a87d4ce0 0x5643a87d7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.889+0000 7f0cd9238700 1 --2- 172.21.15.32:0/3700059721 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5643a87d4ce0 0x5643a87d7100 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7abf1c9f470fb92 server_cookie=6ef103586662fb20 in_seq=0 out_seq=0 2023-09-30T15:01:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.889+0000 7f0cbcff9700 10 client.17364 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.890+0000 7f0cbcff9700 1 -- 172.21.15.32:0/3700059721 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5643a870adb0 con 0x5643a87d4ce0 2023-09-30T15:01:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.890+0000 7f0cbcff9700 10 client.17364 handle_command_reply: tid=0 2023-09-30T15:01:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.890+0000 7f0ce7684b80 1 client.17364 shutdown 2023-09-30T15:01:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.891+0000 7f0ce7684b80 1 -- 172.21.15.32:0/3700059721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 msgr2=0x5643a87c94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.891+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/3700059721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87c94b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40037b0 tx=0x7f0cc400ebd0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.891+0000 7f0ce7684b80 1 -- 172.21.15.32:0/3700059721 shutdown_connections 2023-09-30T15:01:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.891+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/3700059721 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5643a87d4ce0 0x5643a87d7100 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.891+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/3700059721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87c0cd0 0x5643a87d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.891+0000 7f0cbcff9700 0 client.17364 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.891+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/3700059721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87bf200 0x5643a87c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.891+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/3700059721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87c94b0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.891+0000 7f0ce7684b80 1 -- 172.21.15.32:0/3700059721 >> 172.21.15.32:0/3700059721 conn(0x5643a8733200 msgr2=0x5643a87b9590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.891+0000 7f0ce7684b80 1 -- 172.21.15.32:0/3700059721 shutdown_connections 2023-09-30T15:01:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.891+0000 7f0ce7684b80 1 -- 172.21.15.32:0/3700059721 wait complete. 2023-09-30T15:01:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.891+0000 7f0ce7684b80 20 client.17364 trim_cache size 0 max 16384 2023-09-30T15:01:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.908+0000 7f0ce7684b80 1 Processor -- start 2023-09-30T15:01:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.908+0000 7f0ce7684b80 1 -- start start 2023-09-30T15:01:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0ce7684b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87bf200 0x5643a87c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0ce7684b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87c0310 0x5643a87c7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0ce7684b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0cd0 0x5643a87d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0ce7684b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a85928c0 con 0x5643a87c0cd0 2023-09-30T15:01:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0ce7684b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a87bb630 con 0x5643a87bf200 2023-09-30T15:01:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0ce7684b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a87c6eb0 con 0x5643a87c0310 2023-09-30T15:01:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0cd9a39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0cd0 0x5643a87d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0cd8a37700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87c0310 0x5643a87c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0cd9a39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0cd0 0x5643a87d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40220/0 (socket says 172.21.15.32:40220) 2023-09-30T15:01:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0cd9a39700 1 -- 172.21.15.32:0/2921122766 learned_addr learned my addr 172.21.15.32:0/2921122766 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0cd9238700 1 --2- 172.21.15.32:0/2921122766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87bf200 0x5643a87c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0cd9a39700 1 -- 172.21.15.32:0/2921122766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87c0310 msgr2=0x5643a87c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0cd9a39700 1 --2- 172.21.15.32:0/2921122766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87c0310 0x5643a87c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0cd9a39700 1 -- 172.21.15.32:0/2921122766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87bf200 msgr2=0x5643a87c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0cd9a39700 1 --2- 172.21.15.32:0/2921122766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87bf200 0x5643a87c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0cd9a39700 1 -- 172.21.15.32:0/2921122766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643a8525c00 con 0x5643a87c0cd0 2023-09-30T15:01:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0cd8a37700 1 --2- 172.21.15.32:0/2921122766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87c0310 0x5643a87c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:01:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0cd9a39700 1 --2- 172.21.15.32:0/2921122766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0cd0 0x5643a87d04b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc007a30 tx=0x7f0ccc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694f6a439456f647 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.909+0000 7f0cbffff700 1 -- 172.21.15.32:0/2921122766 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc00e040 con 0x5643a87c0cd0 2023-09-30T15:01:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.910+0000 7f0cbffff700 1 -- 172.21.15.32:0/2921122766 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ccc003a00 con 0x5643a87c0cd0 2023-09-30T15:01:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.910+0000 7f0cbffff700 1 -- 172.21.15.32:0/2921122766 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc005610 con 0x5643a87c0cd0 2023-09-30T15:01:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.910+0000 7f0ce7684b80 1 -- 172.21.15.32:0/2921122766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0cd0 msgr2=0x5643a87d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.910+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/2921122766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0cd0 0x5643a87d04b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc007a30 tx=0x7f0ccc005a20 comp rx=0 tx=0).stop 2023-09-30T15:01:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.910+0000 7f0ce7684b80 1 -- 172.21.15.32:0/2921122766 shutdown_connections 2023-09-30T15:01:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.910+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/2921122766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0cd0 0x5643a87d04b0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.910+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/2921122766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87bf200 0x5643a87c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.910+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/2921122766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87c0310 0x5643a87c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.910+0000 7f0ce7684b80 1 -- 172.21.15.32:0/2921122766 >> 172.21.15.32:0/2921122766 conn(0x5643a8733200 msgr2=0x5643a87c7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.910+0000 7f0ce7684b80 1 -- 172.21.15.32:0/2921122766 shutdown_connections 2023-09-30T15:01:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.910+0000 7f0ce7684b80 1 -- 172.21.15.32:0/2921122766 wait complete. 2023-09-30T15:01:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.911+0000 7f0ce7684b80 1 -- start start 2023-09-30T15:01:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.911+0000 7f0ce7684b80 1 Processor -- start 2023-09-30T15:01:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.911+0000 7f0ce7684b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87bf200 0x5643a87ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.911+0000 7f0ce7684b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.911+0000 7f0ce7684b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87c0cd0 0x5643a87cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.911+0000 7f0ce7684b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a87cfb10 con 0x5643a87c0310 2023-09-30T15:01:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.911+0000 7f0ce7684b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a87cfc80 con 0x5643a87bf200 2023-09-30T15:01:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.911+0000 7f0ce7684b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643a87cfe40 con 0x5643a87c0cd0 2023-09-30T15:01:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.911+0000 7f0ce7684b80 10 client.0 _ll_register_callbacks cb 0x7f0cc4011c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:01:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.912+0000 7f0cd9238700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87bf200 0x5643a87ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.912+0000 7f0cd9238700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87bf200 0x5643a87ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51698/0 (socket says 172.21.15.32:51698) 2023-09-30T15:01:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.912+0000 7f0cd9238700 1 -- 172.21.15.32:0/2616973896 learned_addr learned my addr 172.21.15.32:0/2616973896 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.912+0000 7f0cd8a37700 1 --2- 172.21.15.32:0/2616973896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.912+0000 7f0cd9238700 1 -- 172.21.15.32:0/2616973896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87c0cd0 msgr2=0x5643a87cf5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T15:01:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.912+0000 7f0cd9238700 1 --2- 172.21.15.32:0/2616973896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87c0cd0 0x5643a87cf5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.912+0000 7f0cd9238700 1 -- 172.21.15.32:0/2616973896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 msgr2=0x5643a87ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.912+0000 7f0cd9238700 1 --2- 172.21.15.32:0/2616973896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.912+0000 7f0cd9238700 1 -- 172.21.15.32:0/2616973896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643a853cd80 con 0x5643a87bf200 2023-09-30T15:01:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.912+0000 7f0cd8a37700 1 --2- 172.21.15.32:0/2616973896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:01:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.912+0000 7f0cd9238700 1 --2- 172.21.15.32:0/2616973896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87bf200 0x5643a87ce080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0005950 tx=0x7f0cd0099660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fd52fb24b0a9097 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.912+0000 7f0cbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T15:01:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.913+0000 7f0cbcff9700 1 -- 172.21.15.32:0/2616973896 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd00a2090 con 0x5643a87bf200 2023-09-30T15:01:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.913+0000 7f0ce7684b80 1 -- 172.21.15.32:0/2616973896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cc4021070 con 0x5643a87bf200 2023-09-30T15:01:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.913+0000 7f0cbcff9700 1 -- 172.21.15.32:0/2616973896 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0cd00b2290 con 0x5643a87bf200 2023-09-30T15:01:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.913+0000 7f0cbcff9700 1 -- 172.21.15.32:0/2616973896 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cd00b2450 con 0x5643a87bf200 2023-09-30T15:01:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.913+0000 7f0ce7684b80 10 client.26978 fetch_fsmap learned FSMap version 55 2023-09-30T15:01:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.913+0000 7f0ce7684b80 1 -- 172.21.15.32:0/2616973896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f0cd4108b00 con 0x5643a87bf200 2023-09-30T15:01:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.913+0000 7f0cbcff9700 1 -- 172.21.15.32:0/2616973896 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cc4021070 con 0x5643a87bf200 2023-09-30T15:01:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.913+0000 7f0cbcff9700 1 -- 172.21.15.32:0/2616973896 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0cd00991f0 con 0x5643a87bf200 2023-09-30T15:01:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.913+0000 7f0ce7684b80 10 client.26978 fetch_fsmap finished waiting for FSMap version 55 2023-09-30T15:01:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.913+0000 7f0ce7684b80 10 client.26978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:01:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.913+0000 7f0ce7684b80 20 client.26978 populate_metadata read hostname 'smithi032' 2023-09-30T15:01:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.913+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/2616973896 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5643a8829910 0x5643a882bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.913+0000 7f0ce7684b80 4 client.26978 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T15:01:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.913+0000 7f0ce7684b80 1 -- 172.21.15.32:0/2616973896 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5643a870aad0 con 0x5643a8829910 2023-09-30T15:01:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.914+0000 7f0cd8a37700 1 --2- 172.21.15.32:0/2616973896 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5643a8829910 0x5643a882bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.914+0000 7f0cbcff9700 10 client.26978 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:41.914+0000 7f0cd8a37700 1 --2- 172.21.15.32:0/2616973896 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5643a8829910 0x5643a882bd30 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f66007105b8003c server_cookie=526df10816201f76 in_seq=0 out_seq=0 2023-09-30T15:01:42.061 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:41 smithi156 ceph-mon[103579]: pgmap v989: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.0 MiB/s wr, 622 op/s 2023-09-30T15:01:42.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 397430 34.77 MB/sec execute 495 sec latency 122.524 ms 2023-09-30T15:01:42.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:41 smithi148 ceph-mon[102355]: pgmap v989: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.0 MiB/s wr, 622 op/s 2023-09-30T15:01:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.343+0000 7f0cbcff9700 1 -- 172.21.15.32:0/2616973896 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5643a870aad0 con 0x5643a8829910 2023-09-30T15:01:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.343+0000 7f0cbcff9700 10 client.26978 handle_command_reply: tid=0 2023-09-30T15:01:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.343+0000 7f0ce7684b80 1 client.26978 shutdown 2023-09-30T15:01:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.343+0000 7f0ce7684b80 1 -- 172.21.15.32:0/2616973896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87bf200 msgr2=0x5643a87ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.343+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/2616973896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87bf200 0x5643a87ce080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0005950 tx=0x7f0cd0099660 comp rx=0 tx=0).stop 2023-09-30T15:01:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.343+0000 7f0ce7684b80 1 -- 172.21.15.32:0/2616973896 shutdown_connections 2023-09-30T15:01:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.343+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/2616973896 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x5643a8829910 0x5643a882bd30 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.343+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/2616973896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643a87c0cd0 0x5643a87cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.343+0000 7f0cbcff9700 0 client.26978 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.343+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/2616973896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643a87bf200 0x5643a87ce080 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.343+0000 7f0ce7684b80 1 --2- 172.21.15.32:0/2616973896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5643a87c0310 0x5643a87ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.343+0000 7f0ce7684b80 1 -- 172.21.15.32:0/2616973896 >> 172.21.15.32:0/2616973896 conn(0x5643a8733200 msgr2=0x5643a87c7100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.343+0000 7f0ce7684b80 1 -- 172.21.15.32:0/2616973896 shutdown_connections 2023-09-30T15:01:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.343+0000 7f0ce7684b80 1 -- 172.21.15.32:0/2616973896 wait complete. 2023-09-30T15:01:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.343+0000 7f0ce7684b80 20 client.26978 trim_cache size 0 max 16384 2023-09-30T15:01:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T15:01:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T15:01:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T15:01:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T15:01:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.344+0000 7f0cdb49c700 1 -- 172.21.15.32:0/2249679296 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0cc00a1020 msgr2=0x7f0cc00a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.344+0000 7f0cdb49c700 1 --2- 172.21.15.32:0/2249679296 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0cc00a1020 0x7f0cc00a34e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4103640 tx=0x7f0cc4009260 comp rx=0 tx=0).stop 2023-09-30T15:01:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.344+0000 7f0cdb49c700 1 -- 172.21.15.32:0/2249679296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd40ffd70 msgr2=0x7f0cd4102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.344+0000 7f0cdb49c700 1 --2- 172.21.15.32:0/2249679296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd40ffd70 0x7f0cd4102340 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000e970 tx=0x7f0cd000ec80 comp rx=0 tx=0).stop 2023-09-30T15:01:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.344+0000 7f0cdb49c700 1 -- 172.21.15.32:0/2249679296 shutdown_connections 2023-09-30T15:01:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.344+0000 7f0cdb49c700 1 --2- 172.21.15.32:0/2249679296 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f0cc00a1020 0x7f0cc00a34e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.344+0000 7f0cdb49c700 1 --2- 172.21.15.32:0/2249679296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd4101900 0x7f0cd4102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.344+0000 7f0cdb49c700 1 --2- 172.21.15.32:0/2249679296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cd40ffd70 0x7f0cd4102340 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.344+0000 7f0cdb49c700 1 --2- 172.21.15.32:0/2249679296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cd4100f60 0x7f0cd4102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.344+0000 7f0cdb49c700 1 -- 172.21.15.32:0/2249679296 >> 172.21.15.32:0/2249679296 conn(0x7f0cd40fb2a0 msgr2=0x7f0cd4068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.344+0000 7f0cdb49c700 1 -- 172.21.15.32:0/2249679296 shutdown_connections 2023-09-30T15:01:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:42.344+0000 7f0cdb49c700 1 -- 172.21.15.32:0/2249679296 wait complete. 2023-09-30T15:01:43.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 398038 34.76 MB/sec execute 496 sec latency 328.896 ms 2023-09-30T15:01:43.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:43 smithi032 ceph-mon[98592]: pgmap v990: 97 pgs: 97 active+clean; 815 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.3 MiB/s wr, 592 op/s 2023-09-30T15:01:44.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:43 smithi156 ceph-mon[103579]: pgmap v990: 97 pgs: 97 active+clean; 815 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.3 MiB/s wr, 592 op/s 2023-09-30T15:01:44.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 398423 34.76 MB/sec execute 497 sec latency 260.433 ms 2023-09-30T15:01:44.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:43 smithi148 ceph-mon[102355]: pgmap v990: 97 pgs: 97 active+clean; 815 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.3 MiB/s wr, 592 op/s 2023-09-30T15:01:45.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 398995 34.74 MB/sec execute 498 sec latency 124.559 ms 2023-09-30T15:01:45.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:45 smithi032 ceph-mon[98592]: pgmap v991: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.7 MiB/s wr, 765 op/s 2023-09-30T15:01:46.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:45 smithi156 ceph-mon[103579]: pgmap v991: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.7 MiB/s wr, 765 op/s 2023-09-30T15:01:46.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 399557 34.73 MB/sec execute 499 sec latency 75.199 ms 2023-09-30T15:01:46.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:45 smithi148 ceph-mon[102355]: pgmap v991: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.7 MiB/s wr, 765 op/s 2023-09-30T15:01:47.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 400168 34.72 MB/sec execute 500 sec latency 121.581 ms 2023-09-30T15:01:47.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:47 smithi032 ceph-mon[98592]: pgmap v992: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 649 op/s 2023-09-30T15:01:48.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:47 smithi156 ceph-mon[103579]: pgmap v992: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 649 op/s 2023-09-30T15:01:48.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 400932 34.77 MB/sec execute 501 sec latency 53.691 ms 2023-09-30T15:01:48.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:47 smithi148 ceph-mon[102355]: pgmap v992: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 649 op/s 2023-09-30T15:01:49.098 INFO:tasks.workunit.client.0.smithi032.stdout: 10 401359 34.75 MB/sec execute 502 sec latency 259.392 ms 2023-09-30T15:01:49.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:49 smithi032 ceph-mon[98592]: pgmap v993: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.7 MiB/s wr, 576 op/s 2023-09-30T15:01:50.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:49 smithi156 ceph-mon[103579]: pgmap v993: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.7 MiB/s wr, 576 op/s 2023-09-30T15:01:50.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 401957 34.75 MB/sec execute 503 sec latency 82.266 ms 2023-09-30T15:01:50.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:49 smithi148 ceph-mon[102355]: pgmap v993: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.7 MiB/s wr, 576 op/s 2023-09-30T15:01:51.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 402669 34.75 MB/sec execute 504 sec latency 69.391 ms 2023-09-30T15:01:51.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:51 smithi032 ceph-mon[98592]: pgmap v994: 97 pgs: 97 active+clean; 821 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.1 MiB/s wr, 715 op/s 2023-09-30T15:01:52.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:51 smithi156 ceph-mon[103579]: pgmap v994: 97 pgs: 97 active+clean; 821 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.1 MiB/s wr, 715 op/s 2023-09-30T15:01:52.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 403347 34.76 MB/sec execute 505 sec latency 98.577 ms 2023-09-30T15:01:52.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:51 smithi148 ceph-mon[102355]: pgmap v994: 97 pgs: 97 active+clean; 821 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.1 MiB/s wr, 715 op/s 2023-09-30T15:01:53.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 404070 34.76 MB/sec execute 506 sec latency 59.293 ms 2023-09-30T15:01:54.061 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:53 smithi156 ceph-mon[103579]: pgmap v995: 97 pgs: 97 active+clean; 818 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.0 MiB/s wr, 656 op/s 2023-09-30T15:01:54.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 404548 34.74 MB/sec execute 507 sec latency 328.262 ms 2023-09-30T15:01:54.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:53 smithi148 ceph-mon[102355]: pgmap v995: 97 pgs: 97 active+clean; 818 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.0 MiB/s wr, 656 op/s 2023-09-30T15:01:54.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:53 smithi032 ceph-mon[98592]: pgmap v995: 97 pgs: 97 active+clean; 818 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.0 MiB/s wr, 656 op/s 2023-09-30T15:01:55.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 405149 34.74 MB/sec execute 508 sec latency 60.856 ms 2023-09-30T15:01:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.477+0000 7f8099e9c700 1 -- 172.21.15.32:0/949402189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80940ff4e0 msgr2=0x7f80940ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.477+0000 7f8099e9c700 1 --2- 172.21.15.32:0/949402189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80940ff4e0 0x7f80940ff900 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f808400a9d0 tx=0x7f808400ace0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.477+0000 7f8099e9c700 1 -- 172.21.15.32:0/949402189 shutdown_connections 2023-09-30T15:01:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.477+0000 7f8099e9c700 1 --2- 172.21.15.32:0/949402189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8094101070 0x7f8094107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.477+0000 7f8099e9c700 1 --2- 172.21.15.32:0/949402189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80940ff4e0 0x7f80940ff900 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.477+0000 7f8099e9c700 1 --2- 172.21.15.32:0/949402189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80941006d0 0x7f8094100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.477+0000 7f8099e9c700 1 -- 172.21.15.32:0/949402189 >> 172.21.15.32:0/949402189 conn(0x7f80940faa30 msgr2=0x7f80940fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.477+0000 7f8099e9c700 1 -- 172.21.15.32:0/949402189 shutdown_connections 2023-09-30T15:01:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.477+0000 7f8099e9c700 1 -- 172.21.15.32:0/949402189 wait complete. 2023-09-30T15:01:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.478+0000 7f8099e9c700 1 Processor -- start 2023-09-30T15:01:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.478+0000 7f8099e9c700 1 -- start start 2023-09-30T15:01:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.478+0000 7f8099e9c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940ff4e0 0x7f8094194a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.478+0000 7f8099e9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80941006d0 0x7f8094194f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.478+0000 7f8099e9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8094101070 0x7f8094195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.478+0000 7f8099e9c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8094196450 con 0x7f8094101070 2023-09-30T15:01:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.478+0000 7f8099e9c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80941965c0 con 0x7f80940ff4e0 2023-09-30T15:01:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.478+0000 7f8099e9c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8094196780 con 0x7f80941006d0 2023-09-30T15:01:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.478+0000 7f80937fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940ff4e0 0x7f8094194a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.478+0000 7f8092ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80941006d0 0x7f8094194f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.478+0000 7f80937fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940ff4e0 0x7f8094194a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51710/0 (socket says 172.21.15.32:51710) 2023-09-30T15:01:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.478+0000 7f80937fe700 1 -- 172.21.15.32:0/2863756040 learned_addr learned my addr 172.21.15.32:0/2863756040 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.478+0000 7f8093fff700 1 --2- 172.21.15.32:0/2863756040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8094101070 0x7f8094195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.479+0000 7f80937fe700 1 -- 172.21.15.32:0/2863756040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80941006d0 msgr2=0x7f8094194f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.479+0000 7f80937fe700 1 --2- 172.21.15.32:0/2863756040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80941006d0 0x7f8094194f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.479+0000 7f80937fe700 1 -- 172.21.15.32:0/2863756040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8094101070 msgr2=0x7f8094195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.479+0000 7f80937fe700 1 --2- 172.21.15.32:0/2863756040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8094101070 0x7f8094195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.479+0000 7f80937fe700 1 -- 172.21.15.32:0/2863756040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808800b730 con 0x7f80940ff4e0 2023-09-30T15:01:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.479+0000 7f8092ffd700 1 --2- 172.21.15.32:0/2863756040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80941006d0 0x7f8094194f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:01:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.479+0000 7f80937fe700 1 --2- 172.21.15.32:0/2863756040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940ff4e0 0x7f8094194a30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f808400b680 tx=0x7f808400d670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.479+0000 7f8090ff9700 1 -- 172.21.15.32:0/2863756040 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084003d40 con 0x7f80940ff4e0 2023-09-30T15:01:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.479+0000 7f8090ff9700 1 -- 172.21.15.32:0/2863756040 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f808400dd90 con 0x7f80940ff4e0 2023-09-30T15:01:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.479+0000 7f8099e9c700 1 -- 172.21.15.32:0/2863756040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808400a610 con 0x7f80940ff4e0 2023-09-30T15:01:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.479+0000 7f8090ff9700 1 -- 172.21.15.32:0/2863756040 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084015680 con 0x7f80940ff4e0 2023-09-30T15:01:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.479+0000 7f8099e9c700 1 -- 172.21.15.32:0/2863756040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809406a840 con 0x7f80940ff4e0 2023-09-30T15:01:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.482+0000 7f8090ff9700 1 -- 172.21.15.32:0/2863756040 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f808400d8d0 con 0x7f80940ff4e0 2023-09-30T15:01:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.483+0000 7f8090ff9700 1 --2- 172.21.15.32:0/2863756040 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f807c0a1020 0x7f807c0a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.483+0000 7f8090ff9700 1 -- 172.21.15.32:0/2863756040 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8084011070 con 0x7f80940ff4e0 2023-09-30T15:01:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.483+0000 7f8092ffd700 1 --2- 172.21.15.32:0/2863756040 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f807c0a1020 0x7f807c0a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.483+0000 7f80a6084b80 1 Processor -- start 2023-09-30T15:01:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f80a6084b80 1 -- start start 2023-09-30T15:01:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f8092ffd700 1 --2- 172.21.15.32:0/2863756040 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f807c0a1020 0x7f807c0a34e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f80880127a0 tx=0x7f8088014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f80a6084b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a41292190 0x555a412925b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f80a6084b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a41292af0 0x555a412932f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f80a6084b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a412938d0 0x555a41293d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f80a6084b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a411fd400 con 0x555a41292af0 2023-09-30T15:01:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f80a6084b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a411eebc0 con 0x555a41292190 2023-09-30T15:01:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f80a6084b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a411ec4e0 con 0x555a412938d0 2023-09-30T15:01:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f8092ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a41292af0 0x555a412932f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f80937fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a41292190 0x555a412925b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f80937fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a41292190 0x555a412925b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51718/0 (socket says 172.21.15.32:51718) 2023-09-30T15:01:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f80937fe700 1 -- 172.21.15.32:0/3431485995 learned_addr learned my addr 172.21.15.32:0/3431485995 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f8093fff700 1 --2- 172.21.15.32:0/3431485995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a412938d0 0x555a41293d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f8092ffd700 1 -- 172.21.15.32:0/3431485995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a412938d0 msgr2=0x555a41293d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f8092ffd700 1 --2- 172.21.15.32:0/3431485995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a412938d0 0x555a41293d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f8092ffd700 1 -- 172.21.15.32:0/3431485995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a41292190 msgr2=0x555a412925b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f8092ffd700 1 --2- 172.21.15.32:0/3431485995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a41292190 0x555a412925b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.484+0000 7f8092ffd700 1 -- 172.21.15.32:0/3431485995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a4124a870 con 0x555a41292af0 2023-09-30T15:01:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.485+0000 7f8093fff700 1 --2- 172.21.15.32:0/3431485995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a412938d0 0x555a41293d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:01:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.485+0000 7f8092ffd700 1 --2- 172.21.15.32:0/3431485995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a41292af0 0x555a412932f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8088012ef0 tx=0x7f80880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2ab670ffb5f4cd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.485+0000 7f8079ffb700 1 -- 172.21.15.32:0/3431485995 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088017070 con 0x555a41292af0 2023-09-30T15:01:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.485+0000 7f8079ffb700 1 -- 172.21.15.32:0/3431485995 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80880044e0 con 0x555a41292af0 2023-09-30T15:01:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.485+0000 7f8079ffb700 1 -- 172.21.15.32:0/3431485995 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088010070 con 0x555a41292af0 2023-09-30T15:01:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.486+0000 7f80a6084b80 1 -- 172.21.15.32:0/3431485995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a41292af0 msgr2=0x555a412932f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.486+0000 7f80a6084b80 1 --2- 172.21.15.32:0/3431485995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a41292af0 0x555a412932f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8088012ef0 tx=0x7f80880047c0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.486+0000 7f80a6084b80 1 -- 172.21.15.32:0/3431485995 shutdown_connections 2023-09-30T15:01:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.486+0000 7f80a6084b80 1 --2- 172.21.15.32:0/3431485995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a412938d0 0x555a41293d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.486+0000 7f80a6084b80 1 --2- 172.21.15.32:0/3431485995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a41292190 0x555a412925b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.486+0000 7f80a6084b80 1 --2- 172.21.15.32:0/3431485995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a41292af0 0x555a412932f0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.486+0000 7f80a6084b80 1 -- 172.21.15.32:0/3431485995 >> 172.21.15.32:0/3431485995 conn(0x555a41209200 msgr2=0x555a4128fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.486+0000 7f80a6084b80 1 -- 172.21.15.32:0/3431485995 shutdown_connections 2023-09-30T15:01:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.486+0000 7f80a6084b80 1 -- 172.21.15.32:0/3431485995 wait complete. 2023-09-30T15:01:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.486+0000 7f80a6084b80 1 -- start start 2023-09-30T15:01:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f80a6084b80 1 Processor -- start 2023-09-30T15:01:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f80a6084b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a41292190 0x555a412afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f80a6084b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a41292af0 0x555a412b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f80a6084b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a412938d0 0x555a412b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f80a6084b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a411e7dd0 con 0x555a412938d0 2023-09-30T15:01:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f80a6084b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a411846b0 con 0x555a41292af0 2023-09-30T15:01:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f80a6084b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a41184350 con 0x555a41292190 2023-09-30T15:01:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f80a6084b80 10 client.0 _ll_register_callbacks cb 0x555a40fe17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:01:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f80937fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a41292190 0x555a412afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f8093fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a412938d0 0x555a412b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f8092ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a41292af0 0x555a412b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f80937fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a41292190 0x555a412afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54630/0 (socket says 172.21.15.32:54630) 2023-09-30T15:01:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f80937fe700 1 -- 172.21.15.32:0/3372584250 learned_addr learned my addr 172.21.15.32:0/3372584250 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f80937fe700 1 -- 172.21.15.32:0/3372584250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a41292af0 msgr2=0x555a412b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f80937fe700 1 --2- 172.21.15.32:0/3372584250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a41292af0 0x555a412b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f80937fe700 1 -- 172.21.15.32:0/3372584250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a412938d0 msgr2=0x555a412b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f80937fe700 1 --2- 172.21.15.32:0/3372584250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a412938d0 0x555a412b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.487+0000 7f80937fe700 1 -- 172.21.15.32:0/3372584250 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808c011230 con 0x555a41292190 2023-09-30T15:01:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.488+0000 7f80937fe700 1 --2- 172.21.15.32:0/3372584250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a41292190 0x555a412afea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f808400a5e0 tx=0x7f808409ab20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12f6706496c03fa7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.488+0000 7f8072ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T15:01:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.488+0000 7f8072ffd700 1 -- 172.21.15.32:0/3372584250 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084013e30 con 0x555a41292190 2023-09-30T15:01:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.488+0000 7f80a6084b80 1 -- 172.21.15.32:0/3372584250 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a41288090 con 0x555a41292190 2023-09-30T15:01:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.488+0000 7f8072ffd700 1 -- 172.21.15.32:0/3372584250 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f808409b1c0 con 0x555a41292190 2023-09-30T15:01:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.488+0000 7f8072ffd700 1 -- 172.21.15.32:0/3372584250 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f808409f310 con 0x555a41292190 2023-09-30T15:01:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.488+0000 7f80a6084b80 10 client.27031 fetch_fsmap learned FSMap version 55 2023-09-30T15:01:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.488+0000 7f80a6084b80 1 -- 172.21.15.32:0/3372584250 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f808801d380 con 0x555a41292190 2023-09-30T15:01:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.488+0000 7f8072ffd700 1 -- 172.21.15.32:0/3372584250 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555a41288090 con 0x555a41292190 2023-09-30T15:01:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.489+0000 7f8072ffd700 1 -- 172.21.15.32:0/3372584250 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f808409a6b0 con 0x555a41292190 2023-09-30T15:01:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.489+0000 7f80a6084b80 10 client.27031 fetch_fsmap finished waiting for FSMap version 55 2023-09-30T15:01:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.489+0000 7f80a6084b80 10 client.27031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:01:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.489+0000 7f80a6084b80 20 client.27031 populate_metadata read hostname 'smithi032' 2023-09-30T15:01:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.489+0000 7f80a6084b80 1 --2- 172.21.15.32:0/3372584250 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x555a412bba20 0x555a412bde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.489+0000 7f80a6084b80 4 client.27031 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T15:01:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.489+0000 7f80a6084b80 1 -- 172.21.15.32:0/3372584250 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a411e0db0 con 0x555a412bba20 2023-09-30T15:01:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.489+0000 7f8092ffd700 1 --2- 172.21.15.32:0/3372584250 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x555a412bba20 0x555a412bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.489+0000 7f8072ffd700 10 client.27031 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.489+0000 7f8092ffd700 1 --2- 172.21.15.32:0/3372584250 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x555a412bba20 0x555a412bde10 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10847c0891c4868d server_cookie=c992c6b96021bd60 in_seq=0 out_seq=0 2023-09-30T15:01:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.492+0000 7f8072ffd700 1 -- 172.21.15.32:0/3372584250 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555a411e0db0 con 0x555a412bba20 2023-09-30T15:01:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.492+0000 7f8072ffd700 10 client.27031 handle_command_reply: tid=0 2023-09-30T15:01:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.492+0000 7f80a6084b80 1 client.27031 shutdown 2023-09-30T15:01:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.492+0000 7f80a6084b80 1 -- 172.21.15.32:0/3372584250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a41292190 msgr2=0x555a412afea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.492+0000 7f80a6084b80 1 --2- 172.21.15.32:0/3372584250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a41292190 0x555a412afea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f808400a5e0 tx=0x7f808409ab20 comp rx=0 tx=0).stop 2023-09-30T15:01:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.492+0000 7f80a6084b80 1 -- 172.21.15.32:0/3372584250 shutdown_connections 2023-09-30T15:01:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.492+0000 7f80a6084b80 1 --2- 172.21.15.32:0/3372584250 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x555a412bba20 0x555a412bde10 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.492+0000 7f80a6084b80 1 --2- 172.21.15.32:0/3372584250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a412938d0 0x555a412b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.492+0000 7f80a6084b80 1 --2- 172.21.15.32:0/3372584250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a41292190 0x555a412afea0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.492+0000 7f80a6084b80 1 --2- 172.21.15.32:0/3372584250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a41292af0 0x555a412b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.492+0000 7f80a6084b80 1 -- 172.21.15.32:0/3372584250 >> 172.21.15.32:0/3372584250 conn(0x555a41209200 msgr2=0x555a4128f590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.492+0000 7f8072ffd700 0 client.27031 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.492+0000 7f80a6084b80 1 -- 172.21.15.32:0/3372584250 shutdown_connections 2023-09-30T15:01:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.492+0000 7f80a6084b80 1 -- 172.21.15.32:0/3372584250 wait complete. 2023-09-30T15:01:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.493+0000 7f80a6084b80 20 client.27031 trim_cache size 0 max 16384 2023-09-30T15:01:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.512+0000 7f80a6084b80 1 Processor -- start 2023-09-30T15:01:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.512+0000 7f80a6084b80 1 -- start start 2023-09-30T15:01:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.512+0000 7f80a6084b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a41292190 0x555a41293cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.512+0000 7f80a6084b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a41292af0 0x555a412bda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.512+0000 7f80a6084b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a412938d0 0x555a4129cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.512+0000 7f80a6084b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a410688c0 con 0x555a41292190 2023-09-30T15:01:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.512+0000 7f80a6084b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a41293110 con 0x555a412938d0 2023-09-30T15:01:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.512+0000 7f80a6084b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a41291990 con 0x555a41292af0 2023-09-30T15:01:55.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.513+0000 7f80937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a41292190 0x555a41293cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:55.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.513+0000 7f80937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a41292190 0x555a41293cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40256/0 (socket says 172.21.15.32:40256) 2023-09-30T15:01:55.515 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.513+0000 7f80937fe700 1 -- 172.21.15.32:0/1779669525 learned_addr learned my addr 172.21.15.32:0/1779669525 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.513+0000 7f80937fe700 1 -- 172.21.15.32:0/1779669525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a41292af0 msgr2=0x555a412bda30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T15:01:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.513+0000 7f80937fe700 1 --2- 172.21.15.32:0/1779669525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a41292af0 0x555a412bda30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.513+0000 7f80937fe700 1 -- 172.21.15.32:0/1779669525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a412938d0 msgr2=0x555a4129cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.513+0000 7f80937fe700 1 --2- 172.21.15.32:0/1779669525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a412938d0 0x555a4129cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.513+0000 7f80937fe700 1 -- 172.21.15.32:0/1779669525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a41012d80 con 0x555a41292190 2023-09-30T15:01:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.513+0000 7f80937fe700 1 --2- 172.21.15.32:0/1779669525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a41292190 0x555a41293cb0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f8084099510 tx=0x7f8084005390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faf93184106d335 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.513+0000 7f807a7fc700 1 -- 172.21.15.32:0/1779669525 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80840b9290 con 0x555a41292190 2023-09-30T15:01:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.514+0000 7f807a7fc700 1 -- 172.21.15.32:0/1779669525 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f808409b1c0 con 0x555a41292190 2023-09-30T15:01:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.514+0000 7f807a7fc700 1 -- 172.21.15.32:0/1779669525 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084005760 con 0x555a41292190 2023-09-30T15:01:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.514+0000 7f80a6084b80 1 -- 172.21.15.32:0/1779669525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a41292190 msgr2=0x555a41293cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.514+0000 7f80a6084b80 1 --2- 172.21.15.32:0/1779669525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a41292190 0x555a41293cb0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f8084099510 tx=0x7f8084005390 comp rx=0 tx=0).stop 2023-09-30T15:01:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.514+0000 7f80a6084b80 1 -- 172.21.15.32:0/1779669525 shutdown_connections 2023-09-30T15:01:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.514+0000 7f80a6084b80 1 --2- 172.21.15.32:0/1779669525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a412938d0 0x555a4129cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.514+0000 7f80a6084b80 1 --2- 172.21.15.32:0/1779669525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a41292190 0x555a41293cb0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.514+0000 7f80a6084b80 1 --2- 172.21.15.32:0/1779669525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a41292af0 0x555a412bda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.514+0000 7f80a6084b80 1 -- 172.21.15.32:0/1779669525 >> 172.21.15.32:0/1779669525 conn(0x555a41209200 msgr2=0x555a412aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.515+0000 7f80a6084b80 1 -- 172.21.15.32:0/1779669525 shutdown_connections 2023-09-30T15:01:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.515+0000 7f80a6084b80 1 -- 172.21.15.32:0/1779669525 wait complete. 2023-09-30T15:01:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.515+0000 7f80a6084b80 1 -- start start 2023-09-30T15:01:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.515+0000 7f80a6084b80 1 Processor -- start 2023-09-30T15:01:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f80a6084b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a41292af0 0x555a412990a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f80a6084b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a412938d0 0x555a4130b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f80a6084b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4130bdd0 0x555a4130e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f80a6084b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4130e950 con 0x555a412938d0 2023-09-30T15:01:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f80a6084b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4130eac0 con 0x555a41292af0 2023-09-30T15:01:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f80a6084b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a4130ec80 con 0x555a4130bdd0 2023-09-30T15:01:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f80a6084b80 10 client.0 _ll_register_callbacks cb 0x555a40fe17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:01:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f8092ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a412938d0 0x555a4130b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f8092ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a412938d0 0x555a4130b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40262/0 (socket says 172.21.15.32:40262) 2023-09-30T15:01:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f8092ffd700 1 -- 172.21.15.32:0/702433531 learned_addr learned my addr 172.21.15.32:0/702433531 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:01:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f80937fe700 1 --2- 172.21.15.32:0/702433531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a41292af0 0x555a412990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f8093fff700 1 --2- 172.21.15.32:0/702433531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4130bdd0 0x555a4130e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f8092ffd700 1 -- 172.21.15.32:0/702433531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4130bdd0 msgr2=0x555a4130e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f8092ffd700 1 --2- 172.21.15.32:0/702433531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4130bdd0 0x555a4130e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f8092ffd700 1 -- 172.21.15.32:0/702433531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a41292af0 msgr2=0x555a412990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:01:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f8092ffd700 1 --2- 172.21.15.32:0/702433531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a41292af0 0x555a412990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f8092ffd700 1 -- 172.21.15.32:0/702433531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808c011230 con 0x555a412938d0 2023-09-30T15:01:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f8092ffd700 1 --2- 172.21.15.32:0/702433531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a412938d0 0x555a4130b7f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f808801e010 tx=0x7f8088004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f46e9a1b416b9e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:01:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f8072ffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T15:01:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f8072ffd700 1 -- 172.21.15.32:0/702433531 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088010050 con 0x555a412938d0 2023-09-30T15:01:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f8072ffd700 1 -- 172.21.15.32:0/702433531 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8088012440 con 0x555a412938d0 2023-09-30T15:01:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.516+0000 7f80a6084b80 1 -- 172.21.15.32:0/702433531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80880044e0 con 0x555a412938d0 2023-09-30T15:01:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.517+0000 7f8072ffd700 1 -- 172.21.15.32:0/702433531 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8088017040 con 0x555a412938d0 2023-09-30T15:01:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.517+0000 7f80a6084b80 10 client.17388 fetch_fsmap learned FSMap version 55 2023-09-30T15:01:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.517+0000 7f80a6084b80 1 -- 172.21.15.32:0/702433531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x555a40ffbc00 con 0x555a412938d0 2023-09-30T15:01:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.517+0000 7f8072ffd700 1 -- 172.21.15.32:0/702433531 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80880044e0 con 0x555a412938d0 2023-09-30T15:01:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.517+0000 7f8072ffd700 1 -- 172.21.15.32:0/702433531 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 55) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8088003ac0 con 0x555a412938d0 2023-09-30T15:01:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.517+0000 7f80a6084b80 10 client.17388 fetch_fsmap finished waiting for FSMap version 55 2023-09-30T15:01:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.517+0000 7f80a6084b80 10 client.17388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:01:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.517+0000 7f80a6084b80 20 client.17388 populate_metadata read hostname 'smithi032' 2023-09-30T15:01:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.517+0000 7f80a6084b80 1 --2- 172.21.15.32:0/702433531 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x555a41310650 0x555a41312a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:01:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.517+0000 7f80a6084b80 4 client.17388 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T15:01:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.517+0000 7f80a6084b80 1 -- 172.21.15.32:0/702433531 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555a411e0ad0 con 0x555a41310650 2023-09-30T15:01:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.517+0000 7f80937fe700 1 --2- 172.21.15.32:0/702433531 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x555a41310650 0x555a41312a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:01:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.518+0000 7f8072ffd700 10 client.17388 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.518+0000 7f80937fe700 1 --2- 172.21.15.32:0/702433531 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x555a41310650 0x555a41312a70 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef974287a8fb200 server_cookie=d56f4a8f255625c1 in_seq=0 out_seq=0 2023-09-30T15:01:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.896+0000 7f8072ffd700 1 -- 172.21.15.32:0/702433531 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555a411e0ad0 con 0x555a41310650 2023-09-30T15:01:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.896+0000 7f8072ffd700 10 client.17388 handle_command_reply: tid=0 2023-09-30T15:01:55.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.896+0000 7f80a6084b80 1 client.17388 shutdown 2023-09-30T15:01:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.898+0000 7f80a6084b80 1 -- 172.21.15.32:0/702433531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a412938d0 msgr2=0x555a4130b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.898+0000 7f80a6084b80 1 --2- 172.21.15.32:0/702433531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a412938d0 0x555a4130b7f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f808801e010 tx=0x7f8088004740 comp rx=0 tx=0).stop 2023-09-30T15:01:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.898+0000 7f80a6084b80 1 -- 172.21.15.32:0/702433531 shutdown_connections 2023-09-30T15:01:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.898+0000 7f80a6084b80 1 --2- 172.21.15.32:0/702433531 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x555a41310650 0x555a41312a70 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.898+0000 7f80a6084b80 1 --2- 172.21.15.32:0/702433531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555a4130bdd0 0x555a4130e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.898+0000 7f80a6084b80 1 --2- 172.21.15.32:0/702433531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a41292af0 0x555a412990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.898+0000 7f80a6084b80 1 --2- 172.21.15.32:0/702433531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555a412938d0 0x555a4130b7f0 secure :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f808801e010 tx=0x7f8088004740 comp rx=0 tx=0).stop 2023-09-30T15:01:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.898+0000 7f80a6084b80 1 -- 172.21.15.32:0/702433531 >> 172.21.15.32:0/702433531 conn(0x555a41209200 msgr2=0x555a412bda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.898+0000 7f8072ffd700 0 client.17388 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:01:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.900+0000 7f80a6084b80 1 -- 172.21.15.32:0/702433531 shutdown_connections 2023-09-30T15:01:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.900+0000 7f80a6084b80 1 -- 172.21.15.32:0/702433531 wait complete. 2023-09-30T15:01:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.905+0000 7f80a6084b80 20 client.17388 trim_cache size 0 max 16384 2023-09-30T15:01:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T15:01:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T15:01:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T15:01:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T15:01:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.905+0000 7f8099e9c700 1 -- 172.21.15.32:0/2863756040 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f807c0a1020 msgr2=0x7f807c0a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.905+0000 7f8099e9c700 1 --2- 172.21.15.32:0/2863756040 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f807c0a1020 0x7f807c0a34e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f80880127a0 tx=0x7f8088014040 comp rx=0 tx=0).stop 2023-09-30T15:01:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.905+0000 7f8099e9c700 1 -- 172.21.15.32:0/2863756040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940ff4e0 msgr2=0x7f8094194a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:01:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.905+0000 7f8099e9c700 1 --2- 172.21.15.32:0/2863756040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940ff4e0 0x7f8094194a30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f808400b680 tx=0x7f808400d670 comp rx=0 tx=0).stop 2023-09-30T15:01:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.906+0000 7f8099e9c700 1 -- 172.21.15.32:0/2863756040 shutdown_connections 2023-09-30T15:01:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.906+0000 7f8099e9c700 1 --2- 172.21.15.32:0/2863756040 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f807c0a1020 0x7f807c0a34e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.906+0000 7f8099e9c700 1 --2- 172.21.15.32:0/2863756040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8094101070 0x7f8094195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.906+0000 7f8099e9c700 1 --2- 172.21.15.32:0/2863756040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940ff4e0 0x7f8094194a30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.906+0000 7f8099e9c700 1 --2- 172.21.15.32:0/2863756040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80941006d0 0x7f8094194f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:01:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.906+0000 7f8099e9c700 1 -- 172.21.15.32:0/2863756040 >> 172.21.15.32:0/2863756040 conn(0x7f80940faa30 msgr2=0x7f80940fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:01:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.908+0000 7f8099e9c700 1 -- 172.21.15.32:0/2863756040 shutdown_connections 2023-09-30T15:01:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:01:55.909+0000 7f8099e9c700 1 -- 172.21.15.32:0/2863756040 wait complete. 2023-09-30T15:01:56.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:55 smithi156 ceph-mon[103579]: pgmap v996: 97 pgs: 97 active+clean; 814 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 724 op/s 2023-09-30T15:01:56.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 405593 34.72 MB/sec execute 509 sec latency 305.559 ms 2023-09-30T15:01:56.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:55 smithi148 ceph-mon[102355]: pgmap v996: 97 pgs: 97 active+clean; 814 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 724 op/s 2023-09-30T15:01:56.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:55 smithi032 ceph-mon[98592]: pgmap v996: 97 pgs: 97 active+clean; 814 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 724 op/s 2023-09-30T15:01:57.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 406165 34.71 MB/sec execute 510 sec latency 89.658 ms 2023-09-30T15:01:58.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:57 smithi156 ceph-mon[103579]: pgmap v997: 97 pgs: 97 active+clean; 822 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.1 MiB/s wr, 605 op/s 2023-09-30T15:01:58.097 INFO:tasks.workunit.client.0.smithi032.stdout: 10 406730 34.71 MB/sec execute 511 sec latency 57.242 ms 2023-09-30T15:01:58.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:57 smithi148 ceph-mon[102355]: pgmap v997: 97 pgs: 97 active+clean; 822 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.1 MiB/s wr, 605 op/s 2023-09-30T15:01:58.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:57 smithi032 ceph-mon[98592]: pgmap v997: 97 pgs: 97 active+clean; 822 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.1 MiB/s wr, 605 op/s 2023-09-30T15:01:59.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 407228 34.69 MB/sec execute 512 sec latency 272.389 ms 2023-09-30T15:02:00.064 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:59 smithi156 ceph-mon[103579]: pgmap v998: 97 pgs: 97 active+clean; 818 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.0 MiB/s wr, 597 op/s 2023-09-30T15:02:01.113 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:01:59 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T15:02:01.114 INFO:tasks.workunit.client.0.smithi032.stdout: 10 407719 34.67 MB/sec execute 513 sec latency 119.050 ms 2023-09-30T15:02:01.115 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:00 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:01.115 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:00 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:01.115 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:59 smithi148 ceph-mon[102355]: pgmap v998: 97 pgs: 97 active+clean; 818 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.0 MiB/s wr, 597 op/s 2023-09-30T15:02:01.115 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:01:59 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T15:02:01.115 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:00 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:01.115 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:00 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:01.115 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:59 smithi032 ceph-mon[98592]: pgmap v998: 97 pgs: 97 active+clean; 818 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.0 MiB/s wr, 597 op/s 2023-09-30T15:02:01.115 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:01:59 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T15:02:01.116 INFO:tasks.workunit.client.0.smithi032.stdout: 10 408421 34.67 MB/sec execute 514 sec latency 70.648 ms 2023-09-30T15:02:01.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:00 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:01.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:00 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 409112 34.68 MB/sec execute 515 sec latency 58.420 ms 2023-09-30T15:02:02.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:01 smithi148 ceph-mon[102355]: pgmap v999: 97 pgs: 97 active+clean; 796 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.8 MiB/s wr, 721 op/s 2023-09-30T15:02:02.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T15:02:02.113 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T15:02:02.113 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:01 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:01 smithi032 ceph-mon[98592]: pgmap v999: 97 pgs: 97 active+clean; 796 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.8 MiB/s wr, 721 op/s 2023-09-30T15:02:02.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T15:02:02.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T15:02:02.139 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:01 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.309 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:01 smithi156 ceph-mon[103579]: pgmap v999: 97 pgs: 97 active+clean; 796 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.8 MiB/s wr, 721 op/s 2023-09-30T15:02:02.309 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.309 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.309 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.309 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.310 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.310 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:02.310 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T15:02:02.310 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T15:02:02.310 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:01 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:02:03.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 409665 34.68 MB/sec execute 516 sec latency 101.355 ms 2023-09-30T15:02:04.097 INFO:tasks.workunit.client.0.smithi032.stdout: 10 410083 34.65 MB/sec execute 517 sec latency 298.151 ms 2023-09-30T15:02:04.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:03 smithi148 ceph-mon[102355]: pgmap v1000: 97 pgs: 97 active+clean; 796 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 640 op/s 2023-09-30T15:02:04.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:03 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:02:04.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:03 smithi032 ceph-mon[98592]: pgmap v1000: 97 pgs: 97 active+clean; 796 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 640 op/s 2023-09-30T15:02:04.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:03 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:02:04.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:03 smithi156 ceph-mon[103579]: pgmap v1000: 97 pgs: 97 active+clean; 796 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 640 op/s 2023-09-30T15:02:04.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:03 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:02:05.097 INFO:tasks.workunit.client.0.smithi032.stdout: 10 410620 34.64 MB/sec execute 518 sec latency 193.169 ms 2023-09-30T15:02:06.096 INFO:tasks.workunit.client.0.smithi032.stdout: 10 411200 34.63 MB/sec execute 519 sec latency 88.547 ms 2023-09-30T15:02:08.128 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:05 smithi156 ceph-mon[103579]: pgmap v1001: 97 pgs: 97 active+clean; 804 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.7 MiB/s wr, 724 op/s 2023-09-30T15:02:08.129 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:05 smithi032 ceph-mon[98592]: pgmap v1001: 97 pgs: 97 active+clean; 804 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.7 MiB/s wr, 724 op/s 2023-09-30T15:02:08.143 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:05 smithi148 ceph-mon[102355]: pgmap v1001: 97 pgs: 97 active+clean; 804 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.7 MiB/s wr, 724 op/s 2023-09-30T15:02:08.144 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:07 smithi148 ceph-mon[102355]: pgmap v1002: 97 pgs: 97 active+clean; 804 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 645 op/s 2023-09-30T15:02:08.144 INFO:tasks.workunit.client.0.smithi032.stdout: 10 411901 34.63 MB/sec execute 520 sec latency 105.066 ms 2023-09-30T15:02:08.144 INFO:tasks.workunit.client.0.smithi032.stdout: 10 412555 34.63 MB/sec execute 521 sec latency 105.678 ms 2023-09-30T15:02:08.144 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:07 smithi032 ceph-mon[98592]: pgmap v1002: 97 pgs: 97 active+clean; 804 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 645 op/s 2023-09-30T15:02:08.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:07 smithi156 ceph-mon[103579]: pgmap v1002: 97 pgs: 97 active+clean; 804 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.8 MiB/s wr, 645 op/s 2023-09-30T15:02:09.103 INFO:tasks.workunit.client.0.smithi032.stdout: 10 413137 34.63 MB/sec execute 522 sec latency 240.632 ms 2023-09-30T15:02:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.124+0000 7f31d1952700 1 -- 172.21.15.32:0/4233534444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31cc10d160 msgr2=0x7f31cc10d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.124+0000 7f31d1952700 1 --2- 172.21.15.32:0/4233534444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31cc10d160 0x7f31cc10d5e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f31c400b0a0 tx=0x7f31c400b3b0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.129+0000 7f31d1952700 1 -- 172.21.15.32:0/4233534444 shutdown_connections 2023-09-30T15:02:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.129+0000 7f31d1952700 1 --2- 172.21.15.32:0/4233534444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31cc10d160 0x7f31cc10d5e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.129+0000 7f31d1952700 1 --2- 172.21.15.32:0/4233534444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31cc107e20 0x7f31cc10a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.129+0000 7f31d1952700 1 --2- 172.21.15.32:0/4233534444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31cc10a790 0x7f31cc10cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.129+0000 7f31d1952700 1 -- 172.21.15.32:0/4233534444 >> 172.21.15.32:0/4233534444 conn(0x7f31cc06dc80 msgr2=0x7f31cc0700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.130+0000 7f31d1952700 1 -- 172.21.15.32:0/4233534444 shutdown_connections 2023-09-30T15:02:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.130+0000 7f31d1952700 1 -- 172.21.15.32:0/4233534444 wait complete. 2023-09-30T15:02:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.131+0000 7f31d1952700 1 Processor -- start 2023-09-30T15:02:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.134+0000 7f31d1952700 1 -- start start 2023-09-30T15:02:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.134+0000 7f31d1952700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31cc107e20 0x7f31cc118840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.134+0000 7f31d1952700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31cc10a790 0x7f31cc118d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.134+0000 7f31d1952700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31cc10d160 0x7f31cc117180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.134+0000 7f31d1952700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc119670 con 0x7f31cc107e20 2023-09-30T15:02:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.134+0000 7f31d1952700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc1176c0 con 0x7f31cc10a790 2023-09-30T15:02:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.134+0000 7f31d1952700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc117880 con 0x7f31cc10d160 2023-09-30T15:02:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.135+0000 7f31cb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31cc10d160 0x7f31cc117180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.135+0000 7f31caffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31cc107e20 0x7f31cc118840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.135+0000 7f31caffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31cc107e20 0x7f31cc118840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40272/0 (socket says 172.21.15.32:40272) 2023-09-30T15:02:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.135+0000 7f31caffd700 1 -- 172.21.15.32:0/1202525487 learned_addr learned my addr 172.21.15.32:0/1202525487 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.135+0000 7f31c3fff700 1 --2- 172.21.15.32:0/1202525487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31cc10a790 0x7f31cc118d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.136+0000 7f31cb7fe700 1 -- 172.21.15.32:0/1202525487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31cc10a790 msgr2=0x7f31cc118d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.136+0000 7f31cb7fe700 1 --2- 172.21.15.32:0/1202525487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31cc10a790 0x7f31cc118d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.136+0000 7f31cb7fe700 1 -- 172.21.15.32:0/1202525487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31cc107e20 msgr2=0x7f31cc118840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.136+0000 7f31cb7fe700 1 --2- 172.21.15.32:0/1202525487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31cc107e20 0x7f31cc118840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.136+0000 7f31cb7fe700 1 -- 172.21.15.32:0/1202525487 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c4009d00 con 0x7f31cc10d160 2023-09-30T15:02:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.136+0000 7f31cb7fe700 1 --2- 172.21.15.32:0/1202525487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31cc10d160 0x7f31cc117180 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009fd0 tx=0x7f31c40123f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.142+0000 7f31d0950700 1 -- 172.21.15.32:0/1202525487 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c400e040 con 0x7f31cc10d160 2023-09-30T15:02:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.142+0000 7f31d0950700 1 -- 172.21.15.32:0/1202525487 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31c401d480 con 0x7f31cc10d160 2023-09-30T15:02:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.142+0000 7f31d0950700 1 -- 172.21.15.32:0/1202525487 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c401b6b0 con 0x7f31cc10d160 2023-09-30T15:02:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.142+0000 7f31d1952700 1 -- 172.21.15.32:0/1202525487 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31cc117b00 con 0x7f31cc10d160 2023-09-30T15:02:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.142+0000 7f31d1952700 1 -- 172.21.15.32:0/1202525487 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31cc1b6b90 con 0x7f31cc10d160 2023-09-30T15:02:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.147+0000 7f31ddb3ab80 1 Processor -- start 2023-09-30T15:02:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.149+0000 7f31ddb3ab80 1 -- start start 2023-09-30T15:02:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.149+0000 7f31ddb3ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf7200 0x55e355cf7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.149+0000 7f31ddb3ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf8310 0x55e355cf8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.149+0000 7f31ddb3ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 0x55e355cff560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.149+0000 7f31ddb3ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e355c5f400 con 0x55e355cf7200 2023-09-30T15:02:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.149+0000 7f31ddb3ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e355c50bc0 con 0x55e355cf8310 2023-09-30T15:02:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.149+0000 7f31ddb3ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e355c4e4e0 con 0x55e355cf8cd0 2023-09-30T15:02:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.150+0000 7f31c3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 0x55e355cff560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.150+0000 7f31c3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 0x55e355cff560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54662/0 (socket says 172.21.15.32:54662) 2023-09-30T15:02:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.150+0000 7f31c3fff700 1 -- 172.21.15.32:0/1962659184 learned_addr learned my addr 172.21.15.32:0/1962659184 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.150+0000 7f31c3fff700 1 --2- 172.21.15.32:0/1962659184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf7200 0x55e355cf7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.150+0000 7f31caffd700 1 --2- 172.21.15.32:0/1962659184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf8310 0x55e355cf8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.150+0000 7f31c3fff700 1 -- 172.21.15.32:0/1962659184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf8310 msgr2=0x55e355cf8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.150+0000 7f31c3fff700 1 --2- 172.21.15.32:0/1962659184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf8310 0x55e355cf8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.150+0000 7f31c3fff700 1 -- 172.21.15.32:0/1962659184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf7200 msgr2=0x55e355cf7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.150+0000 7f31c3fff700 1 --2- 172.21.15.32:0/1962659184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf7200 0x55e355cf7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.150+0000 7f31c3fff700 1 -- 172.21.15.32:0/1962659184 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e355cac870 con 0x55e355cf8cd0 2023-09-30T15:02:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.150+0000 7f31d0950700 1 -- 172.21.15.32:0/1202525487 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f31c401d5f0 con 0x7f31cc10d160 2023-09-30T15:02:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.151+0000 7f31d0950700 1 --2- 172.21.15.32:0/1202525487 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31b80a1120 0x7f31b80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.151+0000 7f31d0950700 1 -- 172.21.15.32:0/1202525487 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f31c40cdf50 con 0x7f31cc10d160 2023-09-30T15:02:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.151+0000 7f31caffd700 1 --2- 172.21.15.32:0/1202525487 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31b80a1120 0x7f31b80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.152+0000 7f31caffd700 1 --2- 172.21.15.32:0/1202525487 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31b80a1120 0x7f31b80a35e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f31b4005950 tx=0x7f31b400a400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.153+0000 7f31c3fff700 1 --2- 172.21.15.32:0/1962659184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 0x55e355cff560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00cf30 tx=0x7f31bc00c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5af7edfa2a735dd2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.153+0000 7f31c1ffb700 1 -- 172.21.15.32:0/1962659184 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc00ebd0 con 0x55e355cf8cd0 2023-09-30T15:02:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.153+0000 7f31c1ffb700 1 -- 172.21.15.32:0/1962659184 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31bc0044e0 con 0x55e355cf8cd0 2023-09-30T15:02:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.153+0000 7f31c1ffb700 1 -- 172.21.15.32:0/1962659184 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc0163f0 con 0x55e355cf8cd0 2023-09-30T15:02:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.154+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/1962659184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 msgr2=0x55e355cff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.154+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/1962659184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 0x55e355cff560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00cf30 tx=0x7f31bc00c3b0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.154+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/1962659184 shutdown_connections 2023-09-30T15:02:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.154+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/1962659184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 0x55e355cff560 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.154+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/1962659184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf7200 0x55e355cf7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.154+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/1962659184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf8310 0x55e355cf8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.154+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/1962659184 >> 172.21.15.32:0/1962659184 conn(0x55e355c6b200 msgr2=0x55e355cf1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.154+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/1962659184 shutdown_connections 2023-09-30T15:02:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.154+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/1962659184 wait complete. 2023-09-30T15:02:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.155+0000 7f31ddb3ab80 1 -- start start 2023-09-30T15:02:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.155+0000 7f31ddb3ab80 1 Processor -- start 2023-09-30T15:02:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.155+0000 7f31ddb3ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf7200 0x55e355d00eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.155+0000 7f31ddb3ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf8310 0x55e355d013f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.155+0000 7f31ddb3ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 0x55e355d08430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.155+0000 7f31ddb3ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e355c49dd0 con 0x55e355cf7200 2023-09-30T15:02:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.155+0000 7f31ddb3ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e355be66b0 con 0x55e355cf8310 2023-09-30T15:02:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.155+0000 7f31ddb3ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e355be6350 con 0x55e355cf8cd0 2023-09-30T15:02:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.155+0000 7f31ddb3ab80 10 client.0 _ll_register_callbacks cb 0x55e355a437a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:02:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.155+0000 7f31caffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf7200 0x55e355d00eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.156+0000 7f31cb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 0x55e355d08430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.156+0000 7f31caffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf7200 0x55e355d00eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40286/0 (socket says 172.21.15.32:40286) 2023-09-30T15:02:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.156+0000 7f31caffd700 1 -- 172.21.15.32:0/3760847680 learned_addr learned my addr 172.21.15.32:0/3760847680 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.156+0000 7f31c3fff700 1 --2- 172.21.15.32:0/3760847680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf8310 0x55e355d013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.156+0000 7f31cb7fe700 1 -- 172.21.15.32:0/3760847680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf8310 msgr2=0x55e355d013f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.156+0000 7f31cb7fe700 1 --2- 172.21.15.32:0/3760847680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf8310 0x55e355d013f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.156+0000 7f31cb7fe700 1 -- 172.21.15.32:0/3760847680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf7200 msgr2=0x55e355d00eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.156+0000 7f31cb7fe700 1 --2- 172.21.15.32:0/3760847680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf7200 0x55e355d00eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.156+0000 7f31cb7fe700 1 -- 172.21.15.32:0/3760847680 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31bc00cac0 con 0x55e355cf8cd0 2023-09-30T15:02:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.156+0000 7f31cb7fe700 1 --2- 172.21.15.32:0/3760847680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 0x55e355d08430 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f31c401d8a0 tx=0x7f31c409b0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7507b435c57379e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.156+0000 7f31aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T15:02:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.156+0000 7f31aaffd700 1 -- 172.21.15.32:0/3760847680 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c409b910 con 0x55e355cf8cd0 2023-09-30T15:02:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.156+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/3760847680 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e355cea090 con 0x55e355cf8cd0 2023-09-30T15:02:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.156+0000 7f31aaffd700 1 -- 172.21.15.32:0/3760847680 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f31c409b4b0 con 0x55e355cf8cd0 2023-09-30T15:02:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.156+0000 7f31aaffd700 1 -- 172.21.15.32:0/3760847680 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31c409f600 con 0x55e355cf8cd0 2023-09-30T15:02:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.156+0000 7f31ddb3ab80 10 client.27046 fetch_fsmap learned FSMap version 56 2023-09-30T15:02:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.157+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/3760847680 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55e355a74d80 con 0x55e355cf8cd0 2023-09-30T15:02:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.157+0000 7f31aaffd700 1 -- 172.21.15.32:0/3760847680 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31c40a7080 con 0x55e355cf8cd0 2023-09-30T15:02:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.157+0000 7f31aaffd700 1 -- 172.21.15.32:0/3760847680 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f31c409f290 con 0x55e355cf8cd0 2023-09-30T15:02:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.157+0000 7f31ddb3ab80 10 client.27046 fetch_fsmap finished waiting for FSMap version 56 2023-09-30T15:02:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.157+0000 7f31ddb3ab80 10 client.27046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:02:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.157+0000 7f31ddb3ab80 20 client.27046 populate_metadata read hostname 'smithi032' 2023-09-30T15:02:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.157+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/3760847680 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e355d0ca30 0x55e355d0ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.157+0000 7f31ddb3ab80 4 client.27046 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T15:02:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.157+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/3760847680 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e355c42db0 con 0x55e355d0ca30 2023-09-30T15:02:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.159+0000 7f31caffd700 1 --2- 172.21.15.32:0/3760847680 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e355d0ca30 0x55e355d0ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.159+0000 7f31aaffd700 10 client.27046 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:02:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.159+0000 7f31caffd700 1 --2- 172.21.15.32:0/3760847680 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e355d0ca30 0x55e355d0ee50 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34930992f067b6b8 server_cookie=628364b53685689d in_seq=0 out_seq=0 2023-09-30T15:02:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.165+0000 7f31aaffd700 1 -- 172.21.15.32:0/3760847680 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e355c42db0 con 0x55e355d0ca30 2023-09-30T15:02:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.165+0000 7f31aaffd700 10 client.27046 handle_command_reply: tid=0 2023-09-30T15:02:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.165+0000 7f31ddb3ab80 1 client.27046 shutdown 2023-09-30T15:02:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.168+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/3760847680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 msgr2=0x55e355d08430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.168+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/3760847680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 0x55e355d08430 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f31c401d8a0 tx=0x7f31c409b0e0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.168+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/3760847680 shutdown_connections 2023-09-30T15:02:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.168+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/3760847680 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e355d0ca30 0x55e355d0ee50 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.168+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/3760847680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 0x55e355d08430 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.168+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/3760847680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf7200 0x55e355d00eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.168+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/3760847680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf8310 0x55e355d013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.168+0000 7f31aaffd700 0 client.27046 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:02:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.168+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/3760847680 >> 172.21.15.32:0/3760847680 conn(0x55e355c6b200 msgr2=0x55e355cf1590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.168+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/3760847680 shutdown_connections 2023-09-30T15:02:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.168+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/3760847680 wait complete. 2023-09-30T15:02:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.168+0000 7f31ddb3ab80 20 client.27046 trim_cache size 0 max 16384 2023-09-30T15:02:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.186+0000 7f31ddb3ab80 1 Processor -- start 2023-09-30T15:02:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.186+0000 7f31ddb3ab80 1 -- start start 2023-09-30T15:02:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.186+0000 7f31ddb3ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf7200 0x55e355cf86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.186+0000 7f31ddb3ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8310 0x55e355d083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.186+0000 7f31ddb3ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf8cd0 0x55e355cff130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.186+0000 7f31ddb3ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e355aca8c0 con 0x55e355cf8cd0 2023-09-30T15:02:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.186+0000 7f31ddb3ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e355d00a50 con 0x55e355cf7200 2023-09-30T15:02:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.186+0000 7f31ddb3ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e355d0c620 con 0x55e355cf8310 2023-09-30T15:02:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.187+0000 7f31cb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf8cd0 0x55e355cff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.187+0000 7f31cb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf8cd0 0x55e355cff130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40296/0 (socket says 172.21.15.32:40296) 2023-09-30T15:02:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.187+0000 7f31cb7fe700 1 -- 172.21.15.32:0/1468032640 learned_addr learned my addr 172.21.15.32:0/1468032640 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.187+0000 7f31caffd700 1 --2- 172.21.15.32:0/1468032640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf7200 0x55e355cf86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.187+0000 7f31c3fff700 1 --2- 172.21.15.32:0/1468032640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8310 0x55e355d083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.187+0000 7f31cb7fe700 1 -- 172.21.15.32:0/1468032640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8310 msgr2=0x55e355d083f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.187+0000 7f31cb7fe700 1 --2- 172.21.15.32:0/1468032640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8310 0x55e355d083f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.187+0000 7f31cb7fe700 1 -- 172.21.15.32:0/1468032640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf7200 msgr2=0x55e355cf86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.187+0000 7f31cb7fe700 1 --2- 172.21.15.32:0/1468032640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf7200 0x55e355cf86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.187+0000 7f31cb7fe700 1 -- 172.21.15.32:0/1468032640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e355a5dc00 con 0x55e355cf8cd0 2023-09-30T15:02:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.187+0000 7f31cb7fe700 1 --2- 172.21.15.32:0/1468032640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf8cd0 0x55e355cff130 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f31c409b1b0 tx=0x7f31c4099a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6fdced4a673c47 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.187+0000 7f31c27fc700 1 -- 172.21.15.32:0/1468032640 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c409b910 con 0x55e355cf8cd0 2023-09-30T15:02:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.187+0000 7f31c27fc700 1 -- 172.21.15.32:0/1468032640 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e355cea090 con 0x55e355cf8cd0 2023-09-30T15:02:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.187+0000 7f31c27fc700 1 -- 172.21.15.32:0/1468032640 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c409ac00 con 0x55e355cf8cd0 2023-09-30T15:02:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.188+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/1468032640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf8cd0 msgr2=0x55e355cff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.188+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/1468032640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf8cd0 0x55e355cff130 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f31c409b1b0 tx=0x7f31c4099a60 comp rx=0 tx=0).stop 2023-09-30T15:02:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.188+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/1468032640 shutdown_connections 2023-09-30T15:02:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.188+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/1468032640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf8cd0 0x55e355cff130 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.188+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/1468032640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf7200 0x55e355cf86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.188+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/1468032640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8310 0x55e355d083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.188+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/1468032640 >> 172.21.15.32:0/1468032640 conn(0x55e355c6b200 msgr2=0x55e355cffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.188+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/1468032640 shutdown_connections 2023-09-30T15:02:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.188+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/1468032640 wait complete. 2023-09-30T15:02:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.188+0000 7f31ddb3ab80 1 -- start start 2023-09-30T15:02:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31ddb3ab80 1 Processor -- start 2023-09-30T15:02:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31ddb3ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf7200 0x55e355d06000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31ddb3ab80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf8310 0x55e355d06540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31ddb3ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 0x55e355d07550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31ddb3ab80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e355d07a90 con 0x55e355cf8310 2023-09-30T15:02:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31ddb3ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e355d07c00 con 0x55e355cf7200 2023-09-30T15:02:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31ddb3ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e355d07dc0 con 0x55e355cf8cd0 2023-09-30T15:02:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31ddb3ab80 10 client.0 _ll_register_callbacks cb 0x7f31c409b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:02:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31c3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf8310 0x55e355d06540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31c3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf8310 0x55e355d06540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40302/0 (socket says 172.21.15.32:40302) 2023-09-30T15:02:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31c3fff700 1 -- 172.21.15.32:0/824573608 learned_addr learned my addr 172.21.15.32:0/824573608 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31caffd700 1 --2- 172.21.15.32:0/824573608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf7200 0x55e355d06000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31cb7fe700 1 --2- 172.21.15.32:0/824573608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 0x55e355d07550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31c3fff700 1 -- 172.21.15.32:0/824573608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 msgr2=0x55e355d07550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31c3fff700 1 --2- 172.21.15.32:0/824573608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 0x55e355d07550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31c3fff700 1 -- 172.21.15.32:0/824573608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf7200 msgr2=0x55e355d06000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31c3fff700 1 --2- 172.21.15.32:0/824573608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf7200 0x55e355d06000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.189+0000 7f31c3fff700 1 -- 172.21.15.32:0/824573608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e355a74d80 con 0x55e355cf8310 2023-09-30T15:02:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.190+0000 7f31c3fff700 1 --2- 172.21.15.32:0/824573608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf8310 0x55e355d06540 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00cf30 tx=0x7f31bc006e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8325309af0a16e39 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.190+0000 7f31aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T15:02:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.190+0000 7f31aaffd700 1 -- 172.21.15.32:0/824573608 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc003b30 con 0x55e355cf8310 2023-09-30T15:02:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.190+0000 7f31aaffd700 1 -- 172.21.15.32:0/824573608 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f31bc003c90 con 0x55e355cf8310 2023-09-30T15:02:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.190+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/824573608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31bc0044e0 con 0x55e355cf8310 2023-09-30T15:02:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.190+0000 7f31aaffd700 1 -- 172.21.15.32:0/824573608 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31bc003e70 con 0x55e355cf8310 2023-09-30T15:02:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.190+0000 7f31ddb3ab80 10 client.17403 fetch_fsmap learned FSMap version 56 2023-09-30T15:02:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.190+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/824573608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55e355a9b1f0 con 0x55e355cf8310 2023-09-30T15:02:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.190+0000 7f31aaffd700 1 -- 172.21.15.32:0/824573608 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31bc0044e0 con 0x55e355cf8310 2023-09-30T15:02:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.190+0000 7f31aaffd700 1 -- 172.21.15.32:0/824573608 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 56) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f31bc033070 con 0x55e355cf8310 2023-09-30T15:02:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.190+0000 7f31ddb3ab80 10 client.17403 fetch_fsmap finished waiting for FSMap version 56 2023-09-30T15:02:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.190+0000 7f31ddb3ab80 10 client.17403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:02:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.190+0000 7f31ddb3ab80 20 client.17403 populate_metadata read hostname 'smithi032' 2023-09-30T15:02:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.190+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/824573608 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e355d61580 0x55e355d639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.190+0000 7f31ddb3ab80 4 client.17403 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T15:02:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.190+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/824573608 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e355c42ad0 con 0x55e355d61580 2023-09-30T15:02:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.191+0000 7f31caffd700 1 --2- 172.21.15.32:0/824573608 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e355d61580 0x55e355d639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.194+0000 7f31aaffd700 10 client.17403 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:02:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.194+0000 7f31caffd700 1 --2- 172.21.15.32:0/824573608 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e355d61580 0x55e355d639c0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34705ab826e71a46 server_cookie=2d53e2b8074462fb in_seq=0 out_seq=0 2023-09-30T15:02:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.678+0000 7f31aaffd700 1 -- 172.21.15.32:0/824573608 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e355c42ad0 con 0x55e355d61580 2023-09-30T15:02:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.678+0000 7f31aaffd700 10 client.17403 handle_command_reply: tid=0 2023-09-30T15:02:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.678+0000 7f31ddb3ab80 1 client.17403 shutdown 2023-09-30T15:02:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.678+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/824573608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf8310 msgr2=0x55e355d06540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.678+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/824573608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf8310 0x55e355d06540 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00cf30 tx=0x7f31bc006e70 comp rx=0 tx=0).stop 2023-09-30T15:02:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.678+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/824573608 shutdown_connections 2023-09-30T15:02:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.678+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/824573608 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55e355d61580 0x55e355d639c0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.678+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/824573608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e355cf8cd0 0x55e355d07550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.678+0000 7f31aaffd700 0 client.17403 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:02:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.678+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/824573608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e355cf7200 0x55e355d06000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.678+0000 7f31ddb3ab80 1 --2- 172.21.15.32:0/824573608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55e355cf8310 0x55e355d06540 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.678+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/824573608 >> 172.21.15.32:0/824573608 conn(0x55e355c6b200 msgr2=0x55e355cff130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.678+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/824573608 shutdown_connections 2023-09-30T15:02:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.678+0000 7f31ddb3ab80 1 -- 172.21.15.32:0/824573608 wait complete. 2023-09-30T15:02:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.679+0000 7f31ddb3ab80 20 client.17403 trim_cache size 0 max 16384 2023-09-30T15:02:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T15:02:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T15:02:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T15:02:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T15:02:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.679+0000 7f31d1952700 1 -- 172.21.15.32:0/1202525487 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31b80a1120 msgr2=0x7f31b80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.679+0000 7f31d1952700 1 --2- 172.21.15.32:0/1202525487 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31b80a1120 0x7f31b80a35e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f31b4005950 tx=0x7f31b400a400 comp rx=0 tx=0).stop 2023-09-30T15:02:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.679+0000 7f31d1952700 1 -- 172.21.15.32:0/1202525487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31cc10d160 msgr2=0x7f31cc117180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.679+0000 7f31d1952700 1 --2- 172.21.15.32:0/1202525487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31cc10d160 0x7f31cc117180 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009fd0 tx=0x7f31c40123f0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.679+0000 7f31d1952700 1 -- 172.21.15.32:0/1202525487 shutdown_connections 2023-09-30T15:02:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.679+0000 7f31d1952700 1 --2- 172.21.15.32:0/1202525487 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f31b80a1120 0x7f31b80a35e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.679+0000 7f31d1952700 1 --2- 172.21.15.32:0/1202525487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31cc10d160 0x7f31cc117180 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.679+0000 7f31d1952700 1 --2- 172.21.15.32:0/1202525487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31cc107e20 0x7f31cc118840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.679+0000 7f31d1952700 1 --2- 172.21.15.32:0/1202525487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31cc10a790 0x7f31cc118d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.679+0000 7f31d1952700 1 -- 172.21.15.32:0/1202525487 >> 172.21.15.32:0/1202525487 conn(0x7f31cc06dc80 msgr2=0x7f31cc1087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.679+0000 7f31d1952700 1 -- 172.21.15.32:0/1202525487 shutdown_connections 2023-09-30T15:02:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:09.680+0000 7f31d1952700 1 -- 172.21.15.32:0/1202525487 wait complete. 2023-09-30T15:02:10.104 INFO:tasks.workunit.client.0.smithi032.stdout: 10 413440 34.61 MB/sec execute 523 sec latency 327.335 ms 2023-09-30T15:02:10.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:09 smithi032 ceph-mon[98592]: pgmap v1003: 97 pgs: 97 active+clean; 809 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 615 op/s 2023-09-30T15:02:10.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:09 smithi156 ceph-mon[103579]: pgmap v1003: 97 pgs: 97 active+clean; 809 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 615 op/s 2023-09-30T15:02:10.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:09 smithi148 ceph-mon[102355]: pgmap v1003: 97 pgs: 97 active+clean; 809 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 615 op/s 2023-09-30T15:02:11.104 INFO:tasks.workunit.client.0.smithi032.stdout: 10 414028 34.59 MB/sec execute 524 sec latency 69.779 ms 2023-09-30T15:02:12.104 INFO:tasks.workunit.client.0.smithi032.stdout: 10 414742 34.60 MB/sec execute 525 sec latency 50.166 ms 2023-09-30T15:02:12.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:11 smithi148 ceph-mon[102355]: pgmap v1004: 97 pgs: 97 active+clean; 817 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.1 MiB/s wr, 704 op/s 2023-09-30T15:02:12.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:11 smithi032 ceph-mon[98592]: pgmap v1004: 97 pgs: 97 active+clean; 817 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.1 MiB/s wr, 704 op/s 2023-09-30T15:02:12.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:11 smithi156 ceph-mon[103579]: pgmap v1004: 97 pgs: 97 active+clean; 817 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.1 MiB/s wr, 704 op/s 2023-09-30T15:02:13.104 INFO:tasks.workunit.client.0.smithi032.stdout: 10 415442 34.60 MB/sec execute 526 sec latency 92.431 ms 2023-09-30T15:02:14.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 416111 34.61 MB/sec execute 527 sec latency 137.489 ms 2023-09-30T15:02:14.142 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:13 smithi032 ceph-mon[98592]: pgmap v1005: 97 pgs: 97 active+clean; 817 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.2 MiB/s wr, 599 op/s 2023-09-30T15:02:14.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:13 smithi156 ceph-mon[103579]: pgmap v1005: 97 pgs: 97 active+clean; 817 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.2 MiB/s wr, 599 op/s 2023-09-30T15:02:14.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:13 smithi148 ceph-mon[102355]: pgmap v1005: 97 pgs: 97 active+clean; 817 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.2 MiB/s wr, 599 op/s 2023-09-30T15:02:15.109 INFO:tasks.workunit.client.0.smithi032.stdout: 10 416563 34.60 MB/sec execute 528 sec latency 263.445 ms 2023-09-30T15:02:16.109 INFO:tasks.workunit.client.0.smithi032.stdout: 10 417083 34.58 MB/sec execute 529 sec latency 119.457 ms 2023-09-30T15:02:16.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:15 smithi032 ceph-mon[98592]: pgmap v1006: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.1 MiB/s wr, 696 op/s 2023-09-30T15:02:16.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:15 smithi156 ceph-mon[103579]: pgmap v1006: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.1 MiB/s wr, 696 op/s 2023-09-30T15:02:16.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:15 smithi148 ceph-mon[102355]: pgmap v1006: 97 pgs: 97 active+clean; 813 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.1 MiB/s wr, 696 op/s 2023-09-30T15:02:17.109 INFO:tasks.workunit.client.0.smithi032.stdout: 10 417572 34.55 MB/sec execute 530 sec latency 240.865 ms 2023-09-30T15:02:18.109 INFO:tasks.workunit.client.0.smithi032.stdout: 10 418307 34.58 MB/sec execute 531 sec latency 52.345 ms 2023-09-30T15:02:18.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:17 smithi032 ceph-mon[98592]: pgmap v1007: 97 pgs: 97 active+clean; 816 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.4 MiB/s wr, 644 op/s 2023-09-30T15:02:18.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:17 smithi156 ceph-mon[103579]: pgmap v1007: 97 pgs: 97 active+clean; 816 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.4 MiB/s wr, 644 op/s 2023-09-30T15:02:18.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:17 smithi148 ceph-mon[102355]: pgmap v1007: 97 pgs: 97 active+clean; 816 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.4 MiB/s wr, 644 op/s 2023-09-30T15:02:19.109 INFO:tasks.workunit.client.0.smithi032.stdout: 10 419050 34.58 MB/sec execute 532 sec latency 112.451 ms 2023-09-30T15:02:20.109 INFO:tasks.workunit.client.0.smithi032.stdout: 10 419337 34.55 MB/sec execute 533 sec latency 368.569 ms 2023-09-30T15:02:20.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:19 smithi032 ceph-mon[98592]: pgmap v1008: 97 pgs: 97 active+clean; 819 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.4 MiB/s wr, 662 op/s 2023-09-30T15:02:20.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:19 smithi156 ceph-mon[103579]: pgmap v1008: 97 pgs: 97 active+clean; 819 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.4 MiB/s wr, 662 op/s 2023-09-30T15:02:20.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:19 smithi148 ceph-mon[102355]: pgmap v1008: 97 pgs: 97 active+clean; 819 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.4 MiB/s wr, 662 op/s 2023-09-30T15:02:21.109 INFO:tasks.workunit.client.0.smithi032.stdout: 10 419773 34.53 MB/sec execute 534 sec latency 94.219 ms 2023-09-30T15:02:22.109 INFO:tasks.workunit.client.0.smithi032.stdout: 10 420339 34.52 MB/sec execute 535 sec latency 49.401 ms 2023-09-30T15:02:22.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:21 smithi032 ceph-mon[98592]: pgmap v1009: 97 pgs: 97 active+clean; 825 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 674 op/s 2023-09-30T15:02:22.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:21 smithi156 ceph-mon[103579]: pgmap v1009: 97 pgs: 97 active+clean; 825 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 674 op/s 2023-09-30T15:02:22.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:21 smithi148 ceph-mon[102355]: pgmap v1009: 97 pgs: 97 active+clean; 825 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 674 op/s 2023-09-30T15:02:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.817+0000 7f8c2f454700 1 -- 172.21.15.32:0/2183660438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c28106b90 msgr2=0x7f8c28108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.817+0000 7f8c2f454700 1 --2- 172.21.15.32:0/2183660438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c28106b90 0x7f8c28108fc0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000b0a0 tx=0x7f8c2000b3b0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.817+0000 7f8c2f454700 1 -- 172.21.15.32:0/2183660438 shutdown_connections 2023-09-30T15:02:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.817+0000 7f8c2f454700 1 --2- 172.21.15.32:0/2183660438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c28106b90 0x7f8c28108fc0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.817+0000 7f8c2f454700 1 --2- 172.21.15.32:0/2183660438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c28101930 0x7f8c28103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.817+0000 7f8c2f454700 1 --2- 172.21.15.32:0/2183660438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c28104260 0x7f8c28106650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.817+0000 7f8c2f454700 1 -- 172.21.15.32:0/2183660438 >> 172.21.15.32:0/2183660438 conn(0x7f8c280fb2a0 msgr2=0x7f8c280fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.817+0000 7f8c2f454700 1 -- 172.21.15.32:0/2183660438 shutdown_connections 2023-09-30T15:02:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.817+0000 7f8c2f454700 1 -- 172.21.15.32:0/2183660438 wait complete. 2023-09-30T15:02:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.818+0000 7f8c2f454700 1 Processor -- start 2023-09-30T15:02:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.818+0000 7f8c2f454700 1 -- start start 2023-09-30T15:02:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.818+0000 7f8c2f454700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c28101930 0x7f8c28196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.818+0000 7f8c2f454700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c28104260 0x7f8c28197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.818+0000 7f8c2f454700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c28106b90 0x7f8c28198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.818+0000 7f8c2f454700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c28198840 con 0x7f8c28104260 2023-09-30T15:02:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.818+0000 7f8c2f454700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c281989b0 con 0x7f8c28106b90 2023-09-30T15:02:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.818+0000 7f8c2f454700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c28198b70 con 0x7f8c28101930 2023-09-30T15:02:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c2d1f0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c28101930 0x7f8c28196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c2c9ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c28104260 0x7f8c28197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c2d1f0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c28101930 0x7f8c28196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54700/0 (socket says 172.21.15.32:54700) 2023-09-30T15:02:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c2c9ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c28104260 0x7f8c28197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40316/0 (socket says 172.21.15.32:40316) 2023-09-30T15:02:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c2d1f0700 1 -- 172.21.15.32:0/3009066362 learned_addr learned my addr 172.21.15.32:0/3009066362 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c2d9f1700 1 --2- 172.21.15.32:0/3009066362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c28106b90 0x7f8c28198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c2d1f0700 1 -- 172.21.15.32:0/3009066362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c28106b90 msgr2=0x7f8c28198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c2d1f0700 1 --2- 172.21.15.32:0/3009066362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c28106b90 0x7f8c28198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c2d1f0700 1 -- 172.21.15.32:0/3009066362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c28104260 msgr2=0x7f8c28197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c2d1f0700 1 --2- 172.21.15.32:0/3009066362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c28104260 0x7f8c28197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c2d1f0700 1 -- 172.21.15.32:0/3009066362 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c20009d00 con 0x7f8c28101930 2023-09-30T15:02:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c2c9ef700 1 --2- 172.21.15.32:0/3009066362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c28104260 0x7f8c28197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:02:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c2d1f0700 1 --2- 172.21.15.32:0/3009066362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c28101930 0x7f8c28196e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400b570 tx=0x7f8c2400b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c1e7fc700 1 -- 172.21.15.32:0/3009066362 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c24019070 con 0x7f8c28101930 2023-09-30T15:02:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c2f454700 1 -- 172.21.15.32:0/3009066362 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c28198e50 con 0x7f8c28101930 2023-09-30T15:02:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c1e7fc700 1 -- 172.21.15.32:0/3009066362 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c2401d8d0 con 0x7f8c28101930 2023-09-30T15:02:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.819+0000 7f8c1e7fc700 1 -- 172.21.15.32:0/3009066362 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2401c910 con 0x7f8c28101930 2023-09-30T15:02:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.820+0000 7f8c2f454700 1 -- 172.21.15.32:0/3009066362 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c280ff9d0 con 0x7f8c28101930 2023-09-30T15:02:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.824+0000 7f8c3b63cb80 1 Processor -- start 2023-09-30T15:02:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.825+0000 7f8c3b63cb80 1 -- start start 2023-09-30T15:02:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.825+0000 7f8c3b63cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f5200 0x55c6134f5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.825+0000 7f8c3b63cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 0x55c6134f6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.825+0000 7f8c3b63cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f6cd0 0x55c6134fd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.825+0000 7f8c3b63cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c61345d400 con 0x55c6134f6cd0 2023-09-30T15:02:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.825+0000 7f8c3b63cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c61344ebc0 con 0x55c6134f5200 2023-09-30T15:02:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.825+0000 7f8c3b63cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c61344c4e0 con 0x55c6134f6310 2023-09-30T15:02:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.825+0000 7f8c2d9f1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f5200 0x55c6134f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.825+0000 7f8c2d9f1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f5200 0x55c6134f5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51796/0 (socket says 172.21.15.32:51796) 2023-09-30T15:02:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.825+0000 7f8c2d9f1700 1 -- 172.21.15.32:0/2555828469 learned_addr learned my addr 172.21.15.32:0/2555828469 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.826+0000 7f8c2d9f1700 1 -- 172.21.15.32:0/2555828469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 msgr2=0x55c6134f6790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T15:02:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.826+0000 7f8c2d9f1700 1 --2- 172.21.15.32:0/2555828469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 0x55c6134f6790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.826+0000 7f8c2d9f1700 1 -- 172.21.15.32:0/2555828469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f6cd0 msgr2=0x55c6134fd560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T15:02:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.826+0000 7f8c2d9f1700 1 --2- 172.21.15.32:0/2555828469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f6cd0 0x55c6134fd560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.826+0000 7f8c2d9f1700 1 -- 172.21.15.32:0/2555828469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6134aa870 con 0x55c6134f5200 2023-09-30T15:02:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.826+0000 7f8c2d9f1700 1 --2- 172.21.15.32:0/2555828469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f5200 0x55c6134f5620 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000b0a0 tx=0x7f8c20009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a640830a91b6f842 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.826+0000 7f8c137fe700 1 -- 172.21.15.32:0/2555828469 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2000e040 con 0x55c6134f5200 2023-09-30T15:02:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.826+0000 7f8c137fe700 1 -- 172.21.15.32:0/2555828469 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c200076e0 con 0x55c6134f5200 2023-09-30T15:02:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.826+0000 7f8c137fe700 1 -- 172.21.15.32:0/2555828469 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2001c7a0 con 0x55c6134f5200 2023-09-30T15:02:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.826+0000 7f8c1e7fc700 1 -- 172.21.15.32:0/3009066362 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f8c2401d410 con 0x7f8c28101930 2023-09-30T15:02:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.827+0000 7f8c1e7fc700 1 --2- 172.21.15.32:0/3009066362 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8c140a0fa0 0x7f8c140a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.827+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/2555828469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f5200 msgr2=0x55c6134f5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.827+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/2555828469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f5200 0x55c6134f5620 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000b0a0 tx=0x7f8c20009360 comp rx=0 tx=0).stop 2023-09-30T15:02:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.827+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/2555828469 shutdown_connections 2023-09-30T15:02:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.827+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/2555828469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f6cd0 0x55c6134fd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.827+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/2555828469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f5200 0x55c6134f5620 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.827+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/2555828469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 0x55c6134f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.827+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/2555828469 >> 172.21.15.32:0/2555828469 conn(0x55c613469200 msgr2=0x55c6134efef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.827+0000 7f8c2d1f0700 1 --2- 172.21.15.32:0/3009066362 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8c140a0fa0 0x7f8c140a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.827+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/2555828469 shutdown_connections 2023-09-30T15:02:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.827+0000 7f8c1e7fc700 1 -- 172.21.15.32:0/3009066362 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8c240d46f0 con 0x7f8c28101930 2023-09-30T15:02:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.827+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/2555828469 wait complete. 2023-09-30T15:02:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.828+0000 7f8c3b63cb80 1 -- start start 2023-09-30T15:02:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.828+0000 7f8c2d1f0700 1 --2- 172.21.15.32:0/3009066362 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8c140a0fa0 0x7f8c140a3460 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8c240a1b50 tx=0x7f8c24097380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.828+0000 7f8c3b63cb80 1 Processor -- start 2023-09-30T15:02:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.828+0000 7f8c3b63cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f5200 0x55c6134fede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.828+0000 7f8c3b63cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 0x55c6134ff320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.828+0000 7f8c3b63cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f6cd0 0x55c613506360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.828+0000 7f8c3b63cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c613447dd0 con 0x55c6134f5200 2023-09-30T15:02:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.828+0000 7f8c3b63cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6133e46b0 con 0x55c6134f6cd0 2023-09-30T15:02:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.828+0000 7f8c3b63cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6133e4350 con 0x55c6134f6310 2023-09-30T15:02:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.828+0000 7f8c3b63cb80 10 client.0 _ll_register_callbacks cb 0x55c6132417a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:02:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.828+0000 7f8c2d9f1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f5200 0x55c6134fede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.828+0000 7f8c2d9f1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f5200 0x55c6134fede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40324/0 (socket says 172.21.15.32:40324) 2023-09-30T15:02:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.828+0000 7f8c2d9f1700 1 -- 172.21.15.32:0/3135119247 learned_addr learned my addr 172.21.15.32:0/3135119247 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.828+0000 7f8c2c9ef700 1 --2- 172.21.15.32:0/3135119247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 0x55c6134ff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.828+0000 7f8c2d9f1700 1 --2- 172.21.15.32:0/3135119247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f6cd0 0x55c613506360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.829+0000 7f8c2c9ef700 1 -- 172.21.15.32:0/3135119247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f6cd0 msgr2=0x55c613506360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.829+0000 7f8c2c9ef700 1 --2- 172.21.15.32:0/3135119247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f6cd0 0x55c613506360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.829+0000 7f8c2c9ef700 1 -- 172.21.15.32:0/3135119247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f5200 msgr2=0x55c6134fede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.829+0000 7f8c2c9ef700 1 --2- 172.21.15.32:0/3135119247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f5200 0x55c6134fede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.829+0000 7f8c2c9ef700 1 -- 172.21.15.32:0/3135119247 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c28109bb0 con 0x55c6134f6310 2023-09-30T15:02:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.829+0000 7f8c2c9ef700 1 --2- 172.21.15.32:0/3135119247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 0x55c6134ff320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800e7f0 tx=0x7f8c1800eb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9fd9f97ebfeac17 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.829+0000 7f8c10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T15:02:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.829+0000 7f8c10ff9700 1 -- 172.21.15.32:0/3135119247 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18013680 con 0x55c6134f6310 2023-09-30T15:02:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.829+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/3135119247 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6134e8090 con 0x55c6134f6310 2023-09-30T15:02:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.829+0000 7f8c10ff9700 1 -- 172.21.15.32:0/3135119247 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8c180137e0 con 0x55c6134f6310 2023-09-30T15:02:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.829+0000 7f8c10ff9700 1 -- 172.21.15.32:0/3135119247 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c180139a0 con 0x55c6134f6310 2023-09-30T15:02:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.829+0000 7f8c3b63cb80 10 client.27061 fetch_fsmap learned FSMap version 57 2023-09-30T15:02:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.829+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/3135119247 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55c613272d80 con 0x55c6134f6310 2023-09-30T15:02:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.830+0000 7f8c10ff9700 1 -- 172.21.15.32:0/3135119247 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6134e8090 con 0x55c6134f6310 2023-09-30T15:02:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.830+0000 7f8c10ff9700 1 -- 172.21.15.32:0/3135119247 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 57) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8c18011070 con 0x55c6134f6310 2023-09-30T15:02:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.830+0000 7f8c3b63cb80 10 client.27061 fetch_fsmap finished waiting for FSMap version 57 2023-09-30T15:02:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.830+0000 7f8c3b63cb80 10 client.27061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:02:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.830+0000 7f8c3b63cb80 20 client.27061 populate_metadata read hostname 'smithi032' 2023-09-30T15:02:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.830+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/3135119247 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c61350ac90 0x55c61350d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.830+0000 7f8c3b63cb80 4 client.27061 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T15:02:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.830+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/3135119247 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c613440db0 con 0x55c61350ac90 2023-09-30T15:02:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.830+0000 7f8c2d9f1700 1 --2- 172.21.15.32:0/3135119247 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c61350ac90 0x55c61350d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.830+0000 7f8c10ff9700 10 client.27061 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:02:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.831+0000 7f8c2d9f1700 1 --2- 172.21.15.32:0/3135119247 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c61350ac90 0x55c61350d0b0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11a4aa34dcb2a612 server_cookie=1cee6ffee03c3c0a in_seq=0 out_seq=0 2023-09-30T15:02:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.838+0000 7f8c10ff9700 1 -- 172.21.15.32:0/3135119247 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c613440db0 con 0x55c61350ac90 2023-09-30T15:02:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.838+0000 7f8c10ff9700 10 client.27061 handle_command_reply: tid=0 2023-09-30T15:02:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.838+0000 7f8c3b63cb80 1 client.27061 shutdown 2023-09-30T15:02:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.838+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/3135119247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 msgr2=0x55c6134ff320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.838+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/3135119247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 0x55c6134ff320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800e7f0 tx=0x7f8c1800eb00 comp rx=0 tx=0).stop 2023-09-30T15:02:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.838+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/3135119247 shutdown_connections 2023-09-30T15:02:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.838+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/3135119247 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c61350ac90 0x55c61350d0b0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.838+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/3135119247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f6cd0 0x55c613506360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.838+0000 7f8c10ff9700 0 client.27061 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:02:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.838+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/3135119247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f5200 0x55c6134fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.838+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/3135119247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 0x55c6134ff320 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.838+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/3135119247 >> 172.21.15.32:0/3135119247 conn(0x55c613469200 msgr2=0x55c6134ef590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.838+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/3135119247 shutdown_connections 2023-09-30T15:02:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.838+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/3135119247 wait complete. 2023-09-30T15:02:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.838+0000 7f8c3b63cb80 20 client.27061 trim_cache size 0 max 16384 2023-09-30T15:02:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.856+0000 7f8c3b63cb80 1 Processor -- start 2023-09-30T15:02:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.856+0000 7f8c3b63cb80 1 -- start start 2023-09-30T15:02:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.856+0000 7f8c3b63cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f5200 0x55c6134fe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.856+0000 7f8c3b63cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f6310 0x55c613506320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.856+0000 7f8c3b63cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6cd0 0x55c6134fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.856+0000 7f8c3b63cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6132c88c0 con 0x55c6134f6310 2023-09-30T15:02:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.856+0000 7f8c3b63cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6134f0d90 con 0x55c6134f5200 2023-09-30T15:02:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.856+0000 7f8c3b63cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6134ecba0 con 0x55c6134f6cd0 2023-09-30T15:02:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.856+0000 7f8c2d9f1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f5200 0x55c6134fe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.856+0000 7f8c2d9f1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f5200 0x55c6134fe490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51808/0 (socket says 172.21.15.32:51808) 2023-09-30T15:02:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.856+0000 7f8c2d9f1700 1 -- 172.21.15.32:0/1464629156 learned_addr learned my addr 172.21.15.32:0/1464629156 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.856+0000 7f8c2d9f1700 1 --2- 172.21.15.32:0/1464629156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6cd0 0x55c6134fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.856+0000 7f8c2c9ef700 1 --2- 172.21.15.32:0/1464629156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f6310 0x55c613506320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.857+0000 7f8c2d9f1700 1 -- 172.21.15.32:0/1464629156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6cd0 msgr2=0x55c6134fd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.857+0000 7f8c2d9f1700 1 --2- 172.21.15.32:0/1464629156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6cd0 0x55c6134fd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.857+0000 7f8c2d9f1700 1 -- 172.21.15.32:0/1464629156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f6310 msgr2=0x55c613506320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.857+0000 7f8c2d9f1700 1 --2- 172.21.15.32:0/1464629156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f6310 0x55c613506320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.857+0000 7f8c2d9f1700 1 -- 172.21.15.32:0/1464629156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c61325bc00 con 0x55c6134f5200 2023-09-30T15:02:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.857+0000 7f8c2c9ef700 1 --2- 172.21.15.32:0/1464629156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f6310 0x55c613506320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T15:02:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.857+0000 7f8c2d9f1700 1 --2- 172.21.15.32:0/1464629156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f5200 0x55c6134fe490 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8c20005010 tx=0x7f8c200078d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3b1fa3d648241bb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.857+0000 7f8c13fff700 1 -- 172.21.15.32:0/1464629156 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2000e070 con 0x55c6134f5200 2023-09-30T15:02:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.857+0000 7f8c13fff700 1 -- 172.21.15.32:0/1464629156 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c20032400 con 0x55c6134f5200 2023-09-30T15:02:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.857+0000 7f8c13fff700 1 -- 172.21.15.32:0/1464629156 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20005700 con 0x55c6134f5200 2023-09-30T15:02:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.858+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/1464629156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f5200 msgr2=0x55c6134fe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.858+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/1464629156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f5200 0x55c6134fe490 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8c20005010 tx=0x7f8c200078d0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.858+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/1464629156 shutdown_connections 2023-09-30T15:02:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.858+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/1464629156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6cd0 0x55c6134fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.858+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/1464629156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f5200 0x55c6134fe490 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.858+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/1464629156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f6310 0x55c613506320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.858+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/1464629156 >> 172.21.15.32:0/1464629156 conn(0x55c613469200 msgr2=0x55c6134fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.858+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/1464629156 shutdown_connections 2023-09-30T15:02:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.858+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/1464629156 wait complete. 2023-09-30T15:02:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.858+0000 7f8c3b63cb80 1 -- start start 2023-09-30T15:02:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.859+0000 7f8c3b63cb80 1 Processor -- start 2023-09-30T15:02:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.859+0000 7f8c3b63cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f5200 0x55c613503b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.859+0000 7f8c3b63cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 0x55c613504060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.859+0000 7f8c3b63cb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f6cd0 0x55c613505070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.859+0000 7f8c3b63cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6135055b0 con 0x55c6134f5200 2023-09-30T15:02:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.859+0000 7f8c3b63cb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c613505720 con 0x55c6134f6cd0 2023-09-30T15:02:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.859+0000 7f8c3b63cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6135058e0 con 0x55c6134f6310 2023-09-30T15:02:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.859+0000 7f8c3b63cb80 10 client.0 _ll_register_callbacks cb 0x7f8c18009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:02:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.859+0000 7f8c2d9f1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f5200 0x55c613503b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.859+0000 7f8c2c9ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 0x55c613504060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.859+0000 7f8c2c9ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 0x55c613504060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54722/0 (socket says 172.21.15.32:54722) 2023-09-30T15:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.859+0000 7f8c2d9f1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f5200 0x55c613503b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40338/0 (socket says 172.21.15.32:40338) 2023-09-30T15:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.859+0000 7f8c2c9ef700 1 -- 172.21.15.32:0/1583006990 learned_addr learned my addr 172.21.15.32:0/1583006990 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.859+0000 7f8c2d9f1700 1 --2- 172.21.15.32:0/1583006990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f6cd0 0x55c613505070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.860+0000 7f8c2c9ef700 1 -- 172.21.15.32:0/1583006990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f6cd0 msgr2=0x55c613505070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.860+0000 7f8c2c9ef700 1 --2- 172.21.15.32:0/1583006990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f6cd0 0x55c613505070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.860+0000 7f8c2c9ef700 1 -- 172.21.15.32:0/1583006990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f5200 msgr2=0x55c613503b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.860+0000 7f8c2c9ef700 1 --2- 172.21.15.32:0/1583006990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f5200 0x55c613503b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.860+0000 7f8c2c9ef700 1 -- 172.21.15.32:0/1583006990 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2002d040 con 0x55c6134f6310 2023-09-30T15:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.860+0000 7f8c2d9f1700 1 --2- 172.21.15.32:0/1583006990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f5200 0x55c613503b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.860+0000 7f8c2c9ef700 1 --2- 172.21.15.32:0/1583006990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 0x55c613504060 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c18012cf0 tx=0x7f8c1800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69e31793d0b6fffe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.860+0000 7f8c10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T15:02:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.860+0000 7f8c10ff9700 1 -- 172.21.15.32:0/1583006990 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c180123d0 con 0x55c6134f6310 2023-09-30T15:02:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.860+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/1583006990 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c200076e0 con 0x55c6134f6310 2023-09-30T15:02:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.860+0000 7f8c10ff9700 1 -- 172.21.15.32:0/1583006990 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8c18000e40 con 0x55c6134f6310 2023-09-30T15:02:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.860+0000 7f8c10ff9700 1 -- 172.21.15.32:0/1583006990 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c18012530 con 0x55c6134f6310 2023-09-30T15:02:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.860+0000 7f8c3b63cb80 10 client.27067 fetch_fsmap learned FSMap version 57 2023-09-30T15:02:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.860+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/1583006990 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55c6132991f0 con 0x55c6134f6310 2023-09-30T15:02:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.861+0000 7f8c10ff9700 1 -- 172.21.15.32:0/1583006990 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c200076e0 con 0x55c6134f6310 2023-09-30T15:02:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.861+0000 7f8c10ff9700 1 -- 172.21.15.32:0/1583006990 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 57) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8c18011070 con 0x55c6134f6310 2023-09-30T15:02:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.861+0000 7f8c3b63cb80 10 client.27067 fetch_fsmap finished waiting for FSMap version 57 2023-09-30T15:02:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.861+0000 7f8c3b63cb80 10 client.27067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:02:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.861+0000 7f8c3b63cb80 20 client.27067 populate_metadata read hostname 'smithi032' 2023-09-30T15:02:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.861+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/1583006990 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c61355f3f0 0x55c613561810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.861+0000 7f8c3b63cb80 4 client.27067 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T15:02:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.861+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/1583006990 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c613440ad0 con 0x55c61355f3f0 2023-09-30T15:02:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.861+0000 7f8c2d9f1700 1 --2- 172.21.15.32:0/1583006990 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c61355f3f0 0x55c613561810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.861+0000 7f8c10ff9700 10 client.27067 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:02:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:22.861+0000 7f8c2d9f1700 1 --2- 172.21.15.32:0/1583006990 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c61355f3f0 0x55c613561810 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=556c0276f67cf99e server_cookie=aedd71835d1bdb12 in_seq=0 out_seq=0 2023-09-30T15:02:23.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 420874 34.50 MB/sec execute 536 sec latency 149.482 ms 2023-09-30T15:02:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.289+0000 7f8c10ff9700 1 -- 172.21.15.32:0/1583006990 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c613440ad0 con 0x55c61355f3f0 2023-09-30T15:02:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.289+0000 7f8c10ff9700 10 client.27067 handle_command_reply: tid=0 2023-09-30T15:02:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.289+0000 7f8c3b63cb80 1 client.27067 shutdown 2023-09-30T15:02:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.289+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/1583006990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 msgr2=0x55c613504060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.289+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/1583006990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 0x55c613504060 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c18012cf0 tx=0x7f8c1800ee50 comp rx=0 tx=0).stop 2023-09-30T15:02:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/1583006990 shutdown_connections 2023-09-30T15:02:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/1583006990 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55c61355f3f0 0x55c613561810 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/1583006990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6134f6cd0 0x55c613505070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/1583006990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c6134f5200 0x55c613503b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c3b63cb80 1 --2- 172.21.15.32:0/1583006990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c6134f6310 0x55c613504060 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c10ff9700 0 client.27067 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:02:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/1583006990 >> 172.21.15.32:0/1583006990 conn(0x55c613469200 msgr2=0x55c6134fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/1583006990 shutdown_connections 2023-09-30T15:02:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c3b63cb80 1 -- 172.21.15.32:0/1583006990 wait complete. 2023-09-30T15:02:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c3b63cb80 20 client.27067 trim_cache size 0 max 16384 2023-09-30T15:02:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T15:02:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T15:02:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T15:02:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T15:02:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c2f454700 1 -- 172.21.15.32:0/3009066362 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8c140a0fa0 msgr2=0x7f8c140a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c2f454700 1 --2- 172.21.15.32:0/3009066362 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8c140a0fa0 0x7f8c140a3460 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8c240a1b50 tx=0x7f8c24097380 comp rx=0 tx=0).stop 2023-09-30T15:02:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c2f454700 1 -- 172.21.15.32:0/3009066362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c28101930 msgr2=0x7f8c28196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c2f454700 1 --2- 172.21.15.32:0/3009066362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c28101930 0x7f8c28196e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400b570 tx=0x7f8c2400b880 comp rx=0 tx=0).stop 2023-09-30T15:02:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c2f454700 1 -- 172.21.15.32:0/3009066362 shutdown_connections 2023-09-30T15:02:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c2f454700 1 --2- 172.21.15.32:0/3009066362 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8c140a0fa0 0x7f8c140a3460 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c2f454700 1 --2- 172.21.15.32:0/3009066362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c28106b90 0x7f8c28198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c2f454700 1 --2- 172.21.15.32:0/3009066362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c28101930 0x7f8c28196e50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c2f454700 1 --2- 172.21.15.32:0/3009066362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c28104260 0x7f8c28197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.290+0000 7f8c2f454700 1 -- 172.21.15.32:0/3009066362 >> 172.21.15.32:0/3009066362 conn(0x7f8c280fb2a0 msgr2=0x7f8c281025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.291+0000 7f8c2f454700 1 -- 172.21.15.32:0/3009066362 shutdown_connections 2023-09-30T15:02:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:23.291+0000 7f8c2f454700 1 -- 172.21.15.32:0/3009066362 wait complete. 2023-09-30T15:02:23.308 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:22 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:02:23.360 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:22 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:02:23.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:22 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:02:24.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 421416 34.48 MB/sec execute 537 sec latency 308.523 ms 2023-09-30T15:02:24.306 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:23 smithi156 ceph-mon[103579]: pgmap v1010: 97 pgs: 97 active+clean; 832 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.4 MiB/s wr, 595 op/s 2023-09-30T15:02:24.361 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:23 smithi148 ceph-mon[102355]: pgmap v1010: 97 pgs: 97 active+clean; 832 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.4 MiB/s wr, 595 op/s 2023-09-30T15:02:24.394 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:23 smithi032 ceph-mon[98592]: pgmap v1010: 97 pgs: 97 active+clean; 832 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.4 MiB/s wr, 595 op/s 2023-09-30T15:02:25.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 421816 34.48 MB/sec execute 538 sec latency 450.566 ms 2023-09-30T15:02:25.305 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:24 smithi156 ceph-mon[103579]: pgmap v1011: 97 pgs: 97 active+clean; 824 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.4 MiB/s wr, 650 op/s 2023-09-30T15:02:25.359 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:24 smithi148 ceph-mon[102355]: pgmap v1011: 97 pgs: 97 active+clean; 824 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.4 MiB/s wr, 650 op/s 2023-09-30T15:02:25.388 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:24 smithi032 ceph-mon[98592]: pgmap v1011: 97 pgs: 97 active+clean; 824 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.4 MiB/s wr, 650 op/s 2023-09-30T15:02:26.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 422385 34.49 MB/sec execute 539 sec latency 116.231 ms 2023-09-30T15:02:27.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 422991 34.50 MB/sec execute 540 sec latency 44.023 ms 2023-09-30T15:02:27.555 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:27 smithi156 ceph-mon[103579]: pgmap v1012: 97 pgs: 97 active+clean; 828 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.6 MiB/s wr, 663 op/s 2023-09-30T15:02:27.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:27 smithi148 ceph-mon[102355]: pgmap v1012: 97 pgs: 97 active+clean; 828 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.6 MiB/s wr, 663 op/s 2023-09-30T15:02:27.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:27 smithi032 ceph-mon[98592]: pgmap v1012: 97 pgs: 97 active+clean; 828 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.6 MiB/s wr, 663 op/s 2023-09-30T15:02:28.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 423542 34.48 MB/sec execute 541 sec latency 67.299 ms 2023-09-30T15:02:29.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 424059 34.47 MB/sec execute 542 sec latency 89.982 ms 2023-09-30T15:02:29.613 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:29 smithi148 ceph-mon[102355]: pgmap v1013: 97 pgs: 97 active+clean; 825 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.7 MiB/s wr, 598 op/s 2023-09-30T15:02:29.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:29 smithi032 ceph-mon[98592]: pgmap v1013: 97 pgs: 97 active+clean; 825 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.7 MiB/s wr, 598 op/s 2023-09-30T15:02:29.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:29 smithi156 ceph-mon[103579]: pgmap v1013: 97 pgs: 97 active+clean; 825 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.7 MiB/s wr, 598 op/s 2023-09-30T15:02:30.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 424466 34.44 MB/sec execute 543 sec latency 198.804 ms 2023-09-30T15:02:31.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 425062 34.44 MB/sec execute 544 sec latency 100.614 ms 2023-09-30T15:02:31.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:31 smithi148 ceph-mon[102355]: pgmap v1014: 97 pgs: 97 active+clean; 814 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 10 MiB/s wr, 672 op/s 2023-09-30T15:02:31.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:31 smithi032 ceph-mon[98592]: pgmap v1014: 97 pgs: 97 active+clean; 814 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 10 MiB/s wr, 672 op/s 2023-09-30T15:02:31.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:31 smithi156 ceph-mon[103579]: pgmap v1014: 97 pgs: 97 active+clean; 814 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 10 MiB/s wr, 672 op/s 2023-09-30T15:02:32.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 425630 34.45 MB/sec execute 545 sec latency 100.712 ms 2023-09-30T15:02:33.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 426297 34.44 MB/sec execute 546 sec latency 73.789 ms 2023-09-30T15:02:33.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:33 smithi148 ceph-mon[102355]: pgmap v1015: 97 pgs: 97 active+clean; 818 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.9 MiB/s wr, 642 op/s 2023-09-30T15:02:33.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:33 smithi032 ceph-mon[98592]: pgmap v1015: 97 pgs: 97 active+clean; 818 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.9 MiB/s wr, 642 op/s 2023-09-30T15:02:33.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:33 smithi156 ceph-mon[103579]: pgmap v1015: 97 pgs: 97 active+clean; 818 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.9 MiB/s wr, 642 op/s 2023-09-30T15:02:34.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 426908 34.44 MB/sec execute 547 sec latency 44.630 ms 2023-09-30T15:02:35.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 427381 34.42 MB/sec execute 548 sec latency 256.584 ms 2023-09-30T15:02:35.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:35 smithi032 ceph-mon[98592]: pgmap v1016: 97 pgs: 97 active+clean; 824 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.9 MiB/s wr, 672 op/s 2023-09-30T15:02:35.808 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:35 smithi156 ceph-mon[103579]: pgmap v1016: 97 pgs: 97 active+clean; 824 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.9 MiB/s wr, 672 op/s 2023-09-30T15:02:35.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:35 smithi148 ceph-mon[102355]: pgmap v1016: 97 pgs: 97 active+clean; 824 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.9 MiB/s wr, 672 op/s 2023-09-30T15:02:36.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 427876 34.41 MB/sec execute 549 sec latency 70.784 ms 2023-09-30T15:02:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.418+0000 7fe475b3d700 1 -- 172.21.15.32:0/3172111428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4701019a0 msgr2=0x7fe470103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.418+0000 7fe475b3d700 1 --2- 172.21.15.32:0/3172111428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4701019a0 0x7fe470103d90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe460009990 tx=0x7fe460009ca0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.418+0000 7fe475b3d700 1 -- 172.21.15.32:0/3172111428 shutdown_connections 2023-09-30T15:02:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.418+0000 7fe475b3d700 1 --2- 172.21.15.32:0/3172111428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe470106c00 0x7fe470108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.418+0000 7fe475b3d700 1 --2- 172.21.15.32:0/3172111428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4701019a0 0x7fe470103d90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.418+0000 7fe475b3d700 1 --2- 172.21.15.32:0/3172111428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4701042d0 0x7fe4701066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.418+0000 7fe475b3d700 1 -- 172.21.15.32:0/3172111428 >> 172.21.15.32:0/3172111428 conn(0x7fe4700fb2b0 msgr2=0x7fe4700fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.418+0000 7fe475b3d700 1 -- 172.21.15.32:0/3172111428 shutdown_connections 2023-09-30T15:02:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.418+0000 7fe475b3d700 1 -- 172.21.15.32:0/3172111428 wait complete. 2023-09-30T15:02:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.419+0000 7fe475b3d700 1 Processor -- start 2023-09-30T15:02:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.419+0000 7fe475b3d700 1 -- start start 2023-09-30T15:02:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe475b3d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4701019a0 0x7fe470196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe475b3d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4701042d0 0x7fe470197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe475b3d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe470106c00 0x7fe470197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe475b3d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe470198650 con 0x7fe4701019a0 2023-09-30T15:02:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe475b3d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4701987c0 con 0x7fe4701042d0 2023-09-30T15:02:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe475b3d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe470198980 con 0x7fe470106c00 2023-09-30T15:02:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe46f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4701019a0 0x7fe470196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe46f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4701019a0 0x7fe470196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40352/0 (socket says 172.21.15.32:40352) 2023-09-30T15:02:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe46f7fe700 1 -- 172.21.15.32:0/3323540982 learned_addr learned my addr 172.21.15.32:0/3323540982 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe46ffff700 1 --2- 172.21.15.32:0/3323540982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe470106c00 0x7fe470197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe46effd700 1 --2- 172.21.15.32:0/3323540982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4701042d0 0x7fe470197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe46f7fe700 1 -- 172.21.15.32:0/3323540982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe470106c00 msgr2=0x7fe470197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/3323540982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe470106c00 0x7fe470197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe46f7fe700 1 -- 172.21.15.32:0/3323540982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4701042d0 msgr2=0x7fe470197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/3323540982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4701042d0 0x7fe470197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe46f7fe700 1 -- 172.21.15.32:0/3323540982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4600095f0 con 0x7fe4701019a0 2023-09-30T15:02:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe46effd700 1 --2- 172.21.15.32:0/3323540982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4701042d0 0x7fe470197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T15:02:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/3323540982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4701019a0 0x7fe470196c30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe460004740 tx=0x7fe460009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.420+0000 7fe46cff9700 1 -- 172.21.15.32:0/3323540982 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46001d070 con 0x7fe4701019a0 2023-09-30T15:02:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.421+0000 7fe46cff9700 1 -- 172.21.15.32:0/3323540982 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe460004830 con 0x7fe4701019a0 2023-09-30T15:02:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.421+0000 7fe46cff9700 1 -- 172.21.15.32:0/3323540982 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460005450 con 0x7fe4701019a0 2023-09-30T15:02:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.421+0000 7fe475b3d700 1 -- 172.21.15.32:0/3323540982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe470198c00 con 0x7fe4701019a0 2023-09-30T15:02:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.421+0000 7fe475b3d700 1 -- 172.21.15.32:0/3323540982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe470198e70 con 0x7fe4701019a0 2023-09-30T15:02:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.423+0000 7fe46cff9700 1 -- 172.21.15.32:0/3323540982 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fe46000fb00 con 0x7fe4701019a0 2023-09-30T15:02:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.424+0000 7fe46cff9700 1 --2- 172.21.15.32:0/3323540982 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe4580a1020 0x7fe4580a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.424+0000 7fe46cff9700 1 -- 172.21.15.32:0/3323540982 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe4600cda40 con 0x7fe4701019a0 2023-09-30T15:02:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.424+0000 7fe481d25b80 1 Processor -- start 2023-09-30T15:02:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.424+0000 7fe46effd700 1 --2- 172.21.15.32:0/3323540982 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe4580a1020 0x7fe4580a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe481d25b80 1 -- start start 2023-09-30T15:02:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe481d25b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 0x55aab3828620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe481d25b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829310 0x55aab3829790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe481d25b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829cd0 0x55aab3830560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe46effd700 1 --2- 172.21.15.32:0/3323540982 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe4580a1020 0x7fe4580a34e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe464009cb0 tx=0x7fe464009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe481d25b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab3790400 con 0x55aab3829cd0 2023-09-30T15:02:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe481d25b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab3781bc0 con 0x55aab3829310 2023-09-30T15:02:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe481d25b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab377f4e0 con 0x55aab3828200 2023-09-30T15:02:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe46ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829cd0 0x55aab3830560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe46f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 0x55aab3828620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe46ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829cd0 0x55aab3830560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40364/0 (socket says 172.21.15.32:40364) 2023-09-30T15:02:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe46ffff700 1 -- 172.21.15.32:0/4127010700 learned_addr learned my addr 172.21.15.32:0/4127010700 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe46effd700 1 --2- 172.21.15.32:0/4127010700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829310 0x55aab3829790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe46ffff700 1 -- 172.21.15.32:0/4127010700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 msgr2=0x55aab3828620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe46ffff700 1 --2- 172.21.15.32:0/4127010700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 0x55aab3828620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe46ffff700 1 -- 172.21.15.32:0/4127010700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829310 msgr2=0x55aab3829790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe46ffff700 1 --2- 172.21.15.32:0/4127010700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829310 0x55aab3829790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.425+0000 7fe46ffff700 1 -- 172.21.15.32:0/4127010700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aab37dd870 con 0x55aab3829cd0 2023-09-30T15:02:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.426+0000 7fe46ffff700 1 --2- 172.21.15.32:0/4127010700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829cd0 0x55aab3830560 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe468011680 tx=0x7fe468009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c877db6f47f29934 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.426+0000 7fe455ffb700 1 -- 172.21.15.32:0/4127010700 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46800e040 con 0x55aab3829cd0 2023-09-30T15:02:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.426+0000 7fe455ffb700 1 -- 172.21.15.32:0/4127010700 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe46800f040 con 0x55aab3829cd0 2023-09-30T15:02:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.426+0000 7fe455ffb700 1 -- 172.21.15.32:0/4127010700 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4680046a0 con 0x55aab3829cd0 2023-09-30T15:02:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.427+0000 7fe481d25b80 1 -- 172.21.15.32:0/4127010700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829cd0 msgr2=0x55aab3830560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.427+0000 7fe481d25b80 1 --2- 172.21.15.32:0/4127010700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829cd0 0x55aab3830560 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe468011680 tx=0x7fe468009920 comp rx=0 tx=0).stop 2023-09-30T15:02:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.427+0000 7fe481d25b80 1 -- 172.21.15.32:0/4127010700 shutdown_connections 2023-09-30T15:02:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.427+0000 7fe481d25b80 1 --2- 172.21.15.32:0/4127010700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829cd0 0x55aab3830560 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.427+0000 7fe481d25b80 1 --2- 172.21.15.32:0/4127010700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 0x55aab3828620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.427+0000 7fe481d25b80 1 --2- 172.21.15.32:0/4127010700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829310 0x55aab3829790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.427+0000 7fe481d25b80 1 -- 172.21.15.32:0/4127010700 >> 172.21.15.32:0/4127010700 conn(0x55aab379c200 msgr2=0x55aab3822ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.427+0000 7fe481d25b80 1 -- 172.21.15.32:0/4127010700 shutdown_connections 2023-09-30T15:02:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.427+0000 7fe481d25b80 1 -- 172.21.15.32:0/4127010700 wait complete. 2023-09-30T15:02:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.427+0000 7fe481d25b80 1 -- start start 2023-09-30T15:02:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe481d25b80 1 Processor -- start 2023-09-30T15:02:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe481d25b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 0x55aab3831e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe481d25b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829310 0x55aab38323c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe481d25b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829cd0 0x55aab3839400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe481d25b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab377add0 con 0x55aab3829cd0 2023-09-30T15:02:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe481d25b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab37176b0 con 0x55aab3829310 2023-09-30T15:02:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe481d25b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab3717350 con 0x55aab3828200 2023-09-30T15:02:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe481d25b80 10 client.0 _ll_register_callbacks cb 0x55aab35747a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:02:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe46f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 0x55aab3831e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe46ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829cd0 0x55aab3839400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe46f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 0x55aab3831e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54748/0 (socket says 172.21.15.32:54748) 2023-09-30T15:02:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe46f7fe700 1 -- 172.21.15.32:0/3502560944 learned_addr learned my addr 172.21.15.32:0/3502560944 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe46ffff700 1 -- 172.21.15.32:0/3502560944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 msgr2=0x55aab3831e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe46ffff700 1 --2- 172.21.15.32:0/3502560944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 0x55aab3831e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe46ffff700 1 -- 172.21.15.32:0/3502560944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829310 msgr2=0x55aab38323c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe46ffff700 1 --2- 172.21.15.32:0/3502560944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829310 0x55aab38323c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.428+0000 7fe46ffff700 1 -- 172.21.15.32:0/3502560944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe468011330 con 0x55aab3829cd0 2023-09-30T15:02:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.429+0000 7fe46ffff700 1 --2- 172.21.15.32:0/3502560944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829cd0 0x55aab3839400 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe4680042b0 tx=0x7fe468003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7e0e985dec064d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.429+0000 7fe44effd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T15:02:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.429+0000 7fe44effd700 1 -- 172.21.15.32:0/3502560944 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46800e030 con 0x55aab3829cd0 2023-09-30T15:02:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.429+0000 7fe44effd700 1 -- 172.21.15.32:0/3502560944 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe4680092e0 con 0x55aab3829cd0 2023-09-30T15:02:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.429+0000 7fe481d25b80 1 -- 172.21.15.32:0/3502560944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aab381b090 con 0x55aab3829cd0 2023-09-30T15:02:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.429+0000 7fe44effd700 1 -- 172.21.15.32:0/3502560944 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe468004930 con 0x55aab3829cd0 2023-09-30T15:02:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.429+0000 7fe481d25b80 10 client.17430 fetch_fsmap learned FSMap version 57 2023-09-30T15:02:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.429+0000 7fe481d25b80 1 -- 172.21.15.32:0/3502560944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55aab35a5d80 con 0x55aab3829cd0 2023-09-30T15:02:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.429+0000 7fe44effd700 1 -- 172.21.15.32:0/3502560944 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aab381b090 con 0x55aab3829cd0 2023-09-30T15:02:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.429+0000 7fe44effd700 1 -- 172.21.15.32:0/3502560944 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 57) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe46802f070 con 0x55aab3829cd0 2023-09-30T15:02:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.429+0000 7fe481d25b80 10 client.17430 fetch_fsmap finished waiting for FSMap version 57 2023-09-30T15:02:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.429+0000 7fe481d25b80 10 client.17430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:02:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.429+0000 7fe481d25b80 20 client.17430 populate_metadata read hostname 'smithi032' 2023-09-30T15:02:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.429+0000 7fe481d25b80 1 --2- 172.21.15.32:0/3502560944 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55aab3841ff0 0x55aab3844410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.430+0000 7fe481d25b80 4 client.17430 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T15:02:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.430+0000 7fe481d25b80 1 -- 172.21.15.32:0/3502560944 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aab3773db0 con 0x55aab3841ff0 2023-09-30T15:02:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.430+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/3502560944 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55aab3841ff0 0x55aab3844410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.430+0000 7fe44effd700 10 client.17430 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:02:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.430+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/3502560944 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55aab3841ff0 0x55aab3844410 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdda2fdd8792fe75 server_cookie=3df9c7babb692fed in_seq=0 out_seq=0 2023-09-30T15:02:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.431+0000 7fe44effd700 1 -- 172.21.15.32:0/3502560944 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aab3773db0 con 0x55aab3841ff0 2023-09-30T15:02:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.431+0000 7fe44effd700 10 client.17430 handle_command_reply: tid=0 2023-09-30T15:02:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.431+0000 7fe481d25b80 1 client.17430 shutdown 2023-09-30T15:02:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.432+0000 7fe481d25b80 1 -- 172.21.15.32:0/3502560944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829cd0 msgr2=0x55aab3839400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.432+0000 7fe481d25b80 1 --2- 172.21.15.32:0/3502560944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829cd0 0x55aab3839400 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe4680042b0 tx=0x7fe468003ce0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.432+0000 7fe481d25b80 1 -- 172.21.15.32:0/3502560944 shutdown_connections 2023-09-30T15:02:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.432+0000 7fe481d25b80 1 --2- 172.21.15.32:0/3502560944 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55aab3841ff0 0x55aab3844410 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.432+0000 7fe481d25b80 1 --2- 172.21.15.32:0/3502560944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829cd0 0x55aab3839400 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.432+0000 7fe481d25b80 1 --2- 172.21.15.32:0/3502560944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 0x55aab3831e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.432+0000 7fe481d25b80 1 --2- 172.21.15.32:0/3502560944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829310 0x55aab38323c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.432+0000 7fe481d25b80 1 -- 172.21.15.32:0/3502560944 >> 172.21.15.32:0/3502560944 conn(0x55aab379c200 msgr2=0x55aab3822590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.432+0000 7fe44effd700 0 client.17430 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:02:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.432+0000 7fe481d25b80 1 -- 172.21.15.32:0/3502560944 shutdown_connections 2023-09-30T15:02:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.432+0000 7fe481d25b80 1 -- 172.21.15.32:0/3502560944 wait complete. 2023-09-30T15:02:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.432+0000 7fe481d25b80 20 client.17430 trim_cache size 0 max 16384 2023-09-30T15:02:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.449+0000 7fe481d25b80 1 Processor -- start 2023-09-30T15:02:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.449+0000 7fe481d25b80 1 -- start start 2023-09-30T15:02:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.449+0000 7fe481d25b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3828200 0x55aab3830050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.449+0000 7fe481d25b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3829310 0x55aab38393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.449+0000 7fe481d25b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829cd0 0x55aab3830590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.449+0000 7fe481d25b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab35fb8c0 con 0x55aab3828200 2023-09-30T15:02:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.449+0000 7fe481d25b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab3828a80 con 0x55aab3829cd0 2023-09-30T15:02:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.449+0000 7fe481d25b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab3841e10 con 0x55aab3829310 2023-09-30T15:02:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.450+0000 7fe46f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3828200 0x55aab3830050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.450+0000 7fe46f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3828200 0x55aab3830050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40374/0 (socket says 172.21.15.32:40374) 2023-09-30T15:02:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.450+0000 7fe46f7fe700 1 -- 172.21.15.32:0/4159901463 learned_addr learned my addr 172.21.15.32:0/4159901463 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.450+0000 7fe46effd700 1 --2- 172.21.15.32:0/4159901463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3829310 0x55aab38393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.450+0000 7fe46ffff700 1 --2- 172.21.15.32:0/4159901463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829cd0 0x55aab3830590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.450+0000 7fe46f7fe700 1 -- 172.21.15.32:0/4159901463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3829310 msgr2=0x55aab38393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.450+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/4159901463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3829310 0x55aab38393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.450+0000 7fe46f7fe700 1 -- 172.21.15.32:0/4159901463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829cd0 msgr2=0x55aab3830590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.450+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/4159901463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829cd0 0x55aab3830590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.450+0000 7fe46f7fe700 1 -- 172.21.15.32:0/4159901463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aab358ec00 con 0x55aab3828200 2023-09-30T15:02:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.450+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/4159901463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3828200 0x55aab3830050 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fe460097a10 tx=0x7fe460023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5216545da7a19653 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.450+0000 7fe4567fc700 1 -- 172.21.15.32:0/4159901463 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4600a32a0 con 0x55aab3828200 2023-09-30T15:02:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.450+0000 7fe4567fc700 1 -- 172.21.15.32:0/4159901463 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4600cb660 con 0x55aab3828200 2023-09-30T15:02:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.450+0000 7fe4567fc700 1 -- 172.21.15.32:0/4159901463 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46009b200 con 0x55aab3828200 2023-09-30T15:02:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.451+0000 7fe481d25b80 1 -- 172.21.15.32:0/4159901463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3828200 msgr2=0x55aab3830050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.451+0000 7fe481d25b80 1 --2- 172.21.15.32:0/4159901463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3828200 0x55aab3830050 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fe460097a10 tx=0x7fe460023040 comp rx=0 tx=0).stop 2023-09-30T15:02:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.451+0000 7fe481d25b80 1 -- 172.21.15.32:0/4159901463 shutdown_connections 2023-09-30T15:02:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.451+0000 7fe481d25b80 1 --2- 172.21.15.32:0/4159901463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829cd0 0x55aab3830590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.451+0000 7fe481d25b80 1 --2- 172.21.15.32:0/4159901463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3828200 0x55aab3830050 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.451+0000 7fe481d25b80 1 --2- 172.21.15.32:0/4159901463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3829310 0x55aab38393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.451+0000 7fe481d25b80 1 -- 172.21.15.32:0/4159901463 >> 172.21.15.32:0/4159901463 conn(0x55aab379c200 msgr2=0x55aab3844000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.451+0000 7fe481d25b80 1 -- 172.21.15.32:0/4159901463 shutdown_connections 2023-09-30T15:02:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.451+0000 7fe481d25b80 1 -- 172.21.15.32:0/4159901463 wait complete. 2023-09-30T15:02:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.452+0000 7fe481d25b80 1 -- start start 2023-09-30T15:02:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.452+0000 7fe481d25b80 1 Processor -- start 2023-09-30T15:02:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.452+0000 7fe481d25b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 0x55aab3836e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.452+0000 7fe481d25b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829310 0x55aab3837360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.452+0000 7fe481d25b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829cd0 0x55aab3838370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.452+0000 7fe481d25b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab38388b0 con 0x55aab3829310 2023-09-30T15:02:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.452+0000 7fe481d25b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab3838a20 con 0x55aab3829cd0 2023-09-30T15:02:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.452+0000 7fe481d25b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab3838be0 con 0x55aab3828200 2023-09-30T15:02:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.452+0000 7fe481d25b80 10 client.0 _ll_register_callbacks cb 0x7fe464009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:02:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.452+0000 7fe46f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 0x55aab3836e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.452+0000 7fe46f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 0x55aab3836e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54760/0 (socket says 172.21.15.32:54760) 2023-09-30T15:02:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.452+0000 7fe46f7fe700 1 -- 172.21.15.32:0/2587422556 learned_addr learned my addr 172.21.15.32:0/2587422556 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.452+0000 7fe46effd700 1 --2- 172.21.15.32:0/2587422556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829310 0x55aab3837360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.453+0000 7fe46ffff700 1 --2- 172.21.15.32:0/2587422556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829cd0 0x55aab3838370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.453+0000 7fe46effd700 1 -- 172.21.15.32:0/2587422556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 msgr2=0x55aab3836e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.453+0000 7fe46effd700 1 --2- 172.21.15.32:0/2587422556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 0x55aab3836e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.453+0000 7fe46effd700 1 -- 172.21.15.32:0/2587422556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829cd0 msgr2=0x55aab3838370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.453+0000 7fe46effd700 1 --2- 172.21.15.32:0/2587422556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829cd0 0x55aab3838370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.453+0000 7fe46effd700 1 -- 172.21.15.32:0/2587422556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe470198e70 con 0x55aab3829310 2023-09-30T15:02:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.453+0000 7fe46effd700 1 --2- 172.21.15.32:0/2587422556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829310 0x55aab3837360 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe464009ad0 tx=0x7fe464004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d42c546004bb1b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.453+0000 7fe44effd700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T15:02:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.453+0000 7fe44effd700 1 -- 172.21.15.32:0/2587422556 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464003e70 con 0x55aab3829310 2023-09-30T15:02:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.453+0000 7fe44effd700 1 -- 172.21.15.32:0/2587422556 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe464018070 con 0x55aab3829310 2023-09-30T15:02:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.453+0000 7fe481d25b80 1 -- 172.21.15.32:0/2587422556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe46800f040 con 0x55aab3829310 2023-09-30T15:02:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.454+0000 7fe44effd700 1 -- 172.21.15.32:0/2587422556 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe464012450 con 0x55aab3829310 2023-09-30T15:02:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.454+0000 7fe481d25b80 10 client.17436 fetch_fsmap learned FSMap version 57 2023-09-30T15:02:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.454+0000 7fe481d25b80 1 -- 172.21.15.32:0/2587422556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55aab35cc1f0 con 0x55aab3829310 2023-09-30T15:02:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.454+0000 7fe44effd700 1 -- 172.21.15.32:0/2587422556 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe46800f040 con 0x55aab3829310 2023-09-30T15:02:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.454+0000 7fe44effd700 1 -- 172.21.15.32:0/2587422556 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 57) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe464033070 con 0x55aab3829310 2023-09-30T15:02:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.454+0000 7fe481d25b80 10 client.17436 fetch_fsmap finished waiting for FSMap version 57 2023-09-30T15:02:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.454+0000 7fe481d25b80 10 client.17436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:02:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.454+0000 7fe481d25b80 20 client.17436 populate_metadata read hostname 'smithi032' 2023-09-30T15:02:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.454+0000 7fe481d25b80 1 --2- 172.21.15.32:0/2587422556 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55aab3896b60 0x55aab3898f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.454+0000 7fe481d25b80 4 client.17436 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T15:02:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.454+0000 7fe481d25b80 1 -- 172.21.15.32:0/2587422556 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aab3773ad0 con 0x55aab3896b60 2023-09-30T15:02:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.454+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/2587422556 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55aab3896b60 0x55aab3898f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.455+0000 7fe44effd700 10 client.17436 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:02:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.455+0000 7fe46f7fe700 1 --2- 172.21.15.32:0/2587422556 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55aab3896b60 0x55aab3898f80 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e8b8fc17f13293 server_cookie=d124e2eed4cd4d1b in_seq=0 out_seq=0 2023-09-30T15:02:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.899+0000 7fe44effd700 1 -- 172.21.15.32:0/2587422556 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aab3773ad0 con 0x55aab3896b60 2023-09-30T15:02:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.899+0000 7fe44effd700 10 client.17436 handle_command_reply: tid=0 2023-09-30T15:02:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.899+0000 7fe481d25b80 1 client.17436 shutdown 2023-09-30T15:02:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.900+0000 7fe481d25b80 1 -- 172.21.15.32:0/2587422556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829310 msgr2=0x55aab3837360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.900+0000 7fe481d25b80 1 --2- 172.21.15.32:0/2587422556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829310 0x55aab3837360 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe464009ad0 tx=0x7fe464004750 comp rx=0 tx=0).stop 2023-09-30T15:02:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.900+0000 7fe481d25b80 1 -- 172.21.15.32:0/2587422556 shutdown_connections 2023-09-30T15:02:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.900+0000 7fe481d25b80 1 --2- 172.21.15.32:0/2587422556 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55aab3896b60 0x55aab3898f80 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.900+0000 7fe481d25b80 1 --2- 172.21.15.32:0/2587422556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aab3829cd0 0x55aab3838370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.900+0000 7fe481d25b80 1 --2- 172.21.15.32:0/2587422556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aab3828200 0x55aab3836e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.900+0000 7fe481d25b80 1 --2- 172.21.15.32:0/2587422556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55aab3829310 0x55aab3837360 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.900+0000 7fe481d25b80 1 -- 172.21.15.32:0/2587422556 >> 172.21.15.32:0/2587422556 conn(0x55aab379c200 msgr2=0x55aab3830050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.900+0000 7fe44effd700 0 client.17436 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:02:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.900+0000 7fe481d25b80 1 -- 172.21.15.32:0/2587422556 shutdown_connections 2023-09-30T15:02:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.900+0000 7fe481d25b80 1 -- 172.21.15.32:0/2587422556 wait complete. 2023-09-30T15:02:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.900+0000 7fe481d25b80 20 client.17436 trim_cache size 0 max 16384 2023-09-30T15:02:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T15:02:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T15:02:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T15:02:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T15:02:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.900+0000 7fe475b3d700 1 -- 172.21.15.32:0/3323540982 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe4580a1020 msgr2=0x7fe4580a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.901+0000 7fe475b3d700 1 --2- 172.21.15.32:0/3323540982 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe4580a1020 0x7fe4580a34e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe464009cb0 tx=0x7fe464009260 comp rx=0 tx=0).stop 2023-09-30T15:02:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.901+0000 7fe475b3d700 1 -- 172.21.15.32:0/3323540982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4701019a0 msgr2=0x7fe470196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.901+0000 7fe475b3d700 1 --2- 172.21.15.32:0/3323540982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4701019a0 0x7fe470196c30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe460004740 tx=0x7fe460009580 comp rx=0 tx=0).stop 2023-09-30T15:02:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.901+0000 7fe475b3d700 1 -- 172.21.15.32:0/3323540982 shutdown_connections 2023-09-30T15:02:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.901+0000 7fe475b3d700 1 --2- 172.21.15.32:0/3323540982 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fe4580a1020 0x7fe4580a34e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.901+0000 7fe475b3d700 1 --2- 172.21.15.32:0/3323540982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe470106c00 0x7fe470197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.901+0000 7fe475b3d700 1 --2- 172.21.15.32:0/3323540982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4701019a0 0x7fe470196c30 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.901+0000 7fe475b3d700 1 --2- 172.21.15.32:0/3323540982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4701042d0 0x7fe470197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.901+0000 7fe475b3d700 1 -- 172.21.15.32:0/3323540982 >> 172.21.15.32:0/3323540982 conn(0x7fe4700fb2b0 msgr2=0x7fe470102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.901+0000 7fe475b3d700 1 -- 172.21.15.32:0/3323540982 shutdown_connections 2023-09-30T15:02:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:36.901+0000 7fe475b3d700 1 -- 172.21.15.32:0/3323540982 wait complete. 2023-09-30T15:02:37.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 428329 34.40 MB/sec execute 550 sec latency 315.310 ms 2023-09-30T15:02:37.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:37 smithi032 ceph-mon[98592]: pgmap v1017: 97 pgs: 97 active+clean; 823 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.6 MiB/s wr, 657 op/s 2023-09-30T15:02:37.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:37 smithi156 ceph-mon[103579]: pgmap v1017: 97 pgs: 97 active+clean; 823 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.6 MiB/s wr, 657 op/s 2023-09-30T15:02:37.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:37 smithi148 ceph-mon[102355]: pgmap v1017: 97 pgs: 97 active+clean; 823 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.6 MiB/s wr, 657 op/s 2023-09-30T15:02:38.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 428957 34.39 MB/sec execute 551 sec latency 46.177 ms 2023-09-30T15:02:39.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 429597 34.39 MB/sec execute 552 sec latency 54.239 ms 2023-09-30T15:02:39.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:39 smithi032 ceph-mon[98592]: pgmap v1018: 97 pgs: 97 active+clean; 826 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.1 MiB/s wr, 516 op/s 2023-09-30T15:02:39.806 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:39 smithi156 ceph-mon[103579]: pgmap v1018: 97 pgs: 97 active+clean; 826 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.1 MiB/s wr, 516 op/s 2023-09-30T15:02:39.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:39 smithi148 ceph-mon[102355]: pgmap v1018: 97 pgs: 97 active+clean; 826 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.1 MiB/s wr, 516 op/s 2023-09-30T15:02:40.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 430023 34.37 MB/sec execute 553 sec latency 337.336 ms 2023-09-30T15:02:41.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 430605 34.35 MB/sec execute 554 sec latency 108.328 ms 2023-09-30T15:02:41.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:41 smithi032 ceph-mon[98592]: pgmap v1019: 97 pgs: 97 active+clean; 807 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.6 MiB/s wr, 650 op/s 2023-09-30T15:02:41.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:41 smithi156 ceph-mon[103579]: pgmap v1019: 97 pgs: 97 active+clean; 807 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.6 MiB/s wr, 650 op/s 2023-09-30T15:02:41.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:41 smithi148 ceph-mon[102355]: pgmap v1019: 97 pgs: 97 active+clean; 807 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.6 MiB/s wr, 650 op/s 2023-09-30T15:02:42.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 431209 34.36 MB/sec execute 555 sec latency 82.118 ms 2023-09-30T15:02:43.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 431933 34.37 MB/sec execute 556 sec latency 85.452 ms 2023-09-30T15:02:43.638 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:43 smithi032 ceph-mon[98592]: pgmap v1020: 97 pgs: 97 active+clean; 803 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.4 MiB/s wr, 619 op/s 2023-09-30T15:02:43.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:43 smithi156 ceph-mon[103579]: pgmap v1020: 97 pgs: 97 active+clean; 803 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.4 MiB/s wr, 619 op/s 2023-09-30T15:02:43.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:43 smithi148 ceph-mon[102355]: pgmap v1020: 97 pgs: 97 active+clean; 803 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.4 MiB/s wr, 619 op/s 2023-09-30T15:02:44.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 432552 34.37 MB/sec execute 557 sec latency 77.734 ms 2023-09-30T15:02:45.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 432980 34.34 MB/sec execute 558 sec latency 323.874 ms 2023-09-30T15:02:45.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:45 smithi032 ceph-mon[98592]: pgmap v1021: 97 pgs: 97 active+clean; 807 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.9 MiB/s wr, 654 op/s 2023-09-30T15:02:45.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:45 smithi156 ceph-mon[103579]: pgmap v1021: 97 pgs: 97 active+clean; 807 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.9 MiB/s wr, 654 op/s 2023-09-30T15:02:45.862 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:45 smithi148 ceph-mon[102355]: pgmap v1021: 97 pgs: 97 active+clean; 807 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.9 MiB/s wr, 654 op/s 2023-09-30T15:02:46.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 433502 34.33 MB/sec execute 559 sec latency 91.622 ms 2023-09-30T15:02:47.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 434125 34.33 MB/sec execute 560 sec latency 71.033 ms 2023-09-30T15:02:47.810 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:47 smithi156 ceph-mon[103579]: pgmap v1022: 97 pgs: 97 active+clean; 815 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.6 MiB/s wr, 682 op/s 2023-09-30T15:02:47.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:47 smithi148 ceph-mon[102355]: pgmap v1022: 97 pgs: 97 active+clean; 815 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.6 MiB/s wr, 682 op/s 2023-09-30T15:02:47.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:47 smithi032 ceph-mon[98592]: pgmap v1022: 97 pgs: 97 active+clean; 815 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.6 MiB/s wr, 682 op/s 2023-09-30T15:02:48.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 434768 34.35 MB/sec execute 561 sec latency 88.516 ms 2023-09-30T15:02:49.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 435339 34.36 MB/sec execute 562 sec latency 75.643 ms 2023-09-30T15:02:49.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:49 smithi156 ceph-mon[103579]: pgmap v1023: 97 pgs: 97 active+clean; 815 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.4 MiB/s wr, 602 op/s 2023-09-30T15:02:49.865 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:49 smithi148 ceph-mon[102355]: pgmap v1023: 97 pgs: 97 active+clean; 815 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.4 MiB/s wr, 602 op/s 2023-09-30T15:02:49.891 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:49 smithi032 ceph-mon[98592]: pgmap v1023: 97 pgs: 97 active+clean; 815 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.4 MiB/s wr, 602 op/s 2023-09-30T15:02:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.087+0000 7f476455c700 1 -- 172.21.15.32:0/964611753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c102b50 msgr2=0x7f475c109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.087+0000 7f476455c700 1 --2- 172.21.15.32:0/964611753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c102b50 0x7f475c109400 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f475400b0a0 tx=0x7f475400b3b0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.087+0000 7f476455c700 1 -- 172.21.15.32:0/964611753 shutdown_connections 2023-09-30T15:02:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.087+0000 7f476455c700 1 --2- 172.21.15.32:0/964611753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c102b50 0x7f475c109400 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.087+0000 7f476455c700 1 --2- 172.21.15.32:0/964611753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f475c0ff140 0x7f475c0ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.087+0000 7f476455c700 1 --2- 172.21.15.32:0/964611753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f475c1021b0 0x7f475c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.087+0000 7f476455c700 1 -- 172.21.15.32:0/964611753 >> 172.21.15.32:0/964611753 conn(0x7f475c0faa30 msgr2=0x7f475c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.087+0000 7f476455c700 1 -- 172.21.15.32:0/964611753 shutdown_connections 2023-09-30T15:02:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.087+0000 7f476455c700 1 -- 172.21.15.32:0/964611753 wait complete. 2023-09-30T15:02:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.088+0000 7f476455c700 1 Processor -- start 2023-09-30T15:02:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.088+0000 7f476455c700 1 -- start start 2023-09-30T15:02:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.088+0000 7f476455c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f475c0ff140 0x7f475c100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.088+0000 7f476455c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c1021b0 0x7f475c1006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.088+0000 7f476455c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f475c102b50 0x7f475c100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.088+0000 7f476455c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c1a0d00 con 0x7f475c1021b0 2023-09-30T15:02:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.088+0000 7f476455c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c1a0e40 con 0x7f475c102b50 2023-09-30T15:02:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.088+0000 7f476455c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c1a0f80 con 0x7f475c0ff140 2023-09-30T15:02:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.088+0000 7f4761af7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c1021b0 0x7f475c1006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.088+0000 7f47622f8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f475c0ff140 0x7f475c100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.088+0000 7f4761af7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c1021b0 0x7f475c1006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40396/0 (socket says 172.21.15.32:40396) 2023-09-30T15:02:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.088+0000 7f4761af7700 1 -- 172.21.15.32:0/3475128713 learned_addr learned my addr 172.21.15.32:0/3475128713 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.088+0000 7f47622f8700 1 --2- 172.21.15.32:0/3475128713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f475c0ff140 0x7f475c100160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54776/0 (socket says 172.21.15.32:54776) 2023-09-30T15:02:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.088+0000 7f4762af9700 1 --2- 172.21.15.32:0/3475128713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f475c102b50 0x7f475c100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.089+0000 7f4761af7700 1 -- 172.21.15.32:0/3475128713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f475c0ff140 msgr2=0x7f475c100160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.089+0000 7f4761af7700 1 --2- 172.21.15.32:0/3475128713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f475c0ff140 0x7f475c100160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.089+0000 7f4761af7700 1 -- 172.21.15.32:0/3475128713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f475c102b50 msgr2=0x7f475c100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.089+0000 7f4761af7700 1 --2- 172.21.15.32:0/3475128713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f475c102b50 0x7f475c100be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.089+0000 7f4761af7700 1 -- 172.21.15.32:0/3475128713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4754009d00 con 0x7f475c1021b0 2023-09-30T15:02:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.089+0000 7f4761af7700 1 --2- 172.21.15.32:0/3475128713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c1021b0 0x7f475c1006a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f474c00e9a0 tx=0x7f474c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.089+0000 7f47537fe700 1 -- 172.21.15.32:0/3475128713 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c009790 con 0x7f475c1021b0 2023-09-30T15:02:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.089+0000 7f47537fe700 1 -- 172.21.15.32:0/3475128713 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f474c00cbd0 con 0x7f475c1021b0 2023-09-30T15:02:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.089+0000 7f476455c700 1 -- 172.21.15.32:0/3475128713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475c1a1210 con 0x7f475c1021b0 2023-09-30T15:02:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.089+0000 7f47537fe700 1 -- 172.21.15.32:0/3475128713 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c018680 con 0x7f475c1021b0 2023-09-30T15:02:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.089+0000 7f476455c700 1 -- 172.21.15.32:0/3475128713 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475c1a17f0 con 0x7f475c1021b0 2023-09-30T15:02:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.091+0000 7f47537fe700 1 -- 172.21.15.32:0/3475128713 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f474c00cd40 con 0x7f475c1021b0 2023-09-30T15:02:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.092+0000 7f47537fe700 1 --2- 172.21.15.32:0/3475128713 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f47480a1020 0x7f47480a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.092+0000 7f47622f8700 1 --2- 172.21.15.32:0/3475128713 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f47480a1020 0x7f47480a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.092+0000 7f47537fe700 1 -- 172.21.15.32:0/3475128713 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f474c014070 con 0x7f475c1021b0 2023-09-30T15:02:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.093+0000 7f47622f8700 1 --2- 172.21.15.32:0/3475128713 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f47480a1020 0x7f47480a34e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f475c101a70 tx=0x7f4758009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.093+0000 7f4770744b80 1 Processor -- start 2023-09-30T15:02:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4770744b80 1 -- start start 2023-09-30T15:02:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4770744b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad65200 0x55d6bad65620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4770744b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad66310 0x55d6bad66790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4770744b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad6d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4770744b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6baccd400 con 0x55d6bad66cd0 2023-09-30T15:02:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4770744b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6bacbebc0 con 0x55d6bad65200 2023-09-30T15:02:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4770744b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6bacbc4e0 con 0x55d6bad66310 2023-09-30T15:02:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4762af9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad6d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4761af7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad66310 0x55d6bad66790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4762af9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad6d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40406/0 (socket says 172.21.15.32:40406) 2023-09-30T15:02:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4762af9700 1 -- 172.21.15.32:0/3856018608 learned_addr learned my addr 172.21.15.32:0/3856018608 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4761af7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad66310 0x55d6bad66790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54786/0 (socket says 172.21.15.32:54786) 2023-09-30T15:02:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4762af9700 1 -- 172.21.15.32:0/3856018608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad66310 msgr2=0x55d6bad66790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4762af9700 1 --2- 172.21.15.32:0/3856018608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad66310 0x55d6bad66790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4762af9700 1 -- 172.21.15.32:0/3856018608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad65200 msgr2=0x55d6bad65620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4762af9700 1 --2- 172.21.15.32:0/3856018608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad65200 0x55d6bad65620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4762af9700 1 -- 172.21.15.32:0/3856018608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6bad1a870 con 0x55d6bad66cd0 2023-09-30T15:02:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.094+0000 7f4762af9700 1 --2- 172.21.15.32:0/3856018608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad6d560 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f475400b780 tx=0x7f4754009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7bdd735223e4a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.095+0000 7f4750ff9700 1 -- 172.21.15.32:0/3856018608 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475400e050 con 0x55d6bad66cd0 2023-09-30T15:02:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.095+0000 7f4750ff9700 1 -- 172.21.15.32:0/3856018608 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47540076e0 con 0x55d6bad66cd0 2023-09-30T15:02:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.095+0000 7f4750ff9700 1 -- 172.21.15.32:0/3856018608 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754005280 con 0x55d6bad66cd0 2023-09-30T15:02:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.095+0000 7f4770744b80 1 -- 172.21.15.32:0/3856018608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 msgr2=0x55d6bad6d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.095+0000 7f4770744b80 1 --2- 172.21.15.32:0/3856018608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad6d560 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f475400b780 tx=0x7f4754009360 comp rx=0 tx=0).stop 2023-09-30T15:02:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.095+0000 7f4770744b80 1 -- 172.21.15.32:0/3856018608 shutdown_connections 2023-09-30T15:02:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.095+0000 7f4770744b80 1 --2- 172.21.15.32:0/3856018608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad6d560 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.095+0000 7f4770744b80 1 --2- 172.21.15.32:0/3856018608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad65200 0x55d6bad65620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.095+0000 7f4770744b80 1 --2- 172.21.15.32:0/3856018608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad66310 0x55d6bad66790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.095+0000 7f4770744b80 1 -- 172.21.15.32:0/3856018608 >> 172.21.15.32:0/3856018608 conn(0x55d6bacd9200 msgr2=0x55d6bad5fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.096+0000 7f4770744b80 1 -- 172.21.15.32:0/3856018608 shutdown_connections 2023-09-30T15:02:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.096+0000 7f4770744b80 1 -- 172.21.15.32:0/3856018608 wait complete. 2023-09-30T15:02:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.096+0000 7f4770744b80 1 -- start start 2023-09-30T15:02:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.096+0000 7f4770744b80 1 Processor -- start 2023-09-30T15:02:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.096+0000 7f4770744b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad65200 0x55d6bad6ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.096+0000 7f4770744b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad66310 0x55d6bad6f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.096+0000 7f4770744b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad66cd0 0x55d6bad76480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.096+0000 7f4770744b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6bacb7dd0 con 0x55d6bad65200 2023-09-30T15:02:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.096+0000 7f4770744b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6bac546b0 con 0x55d6bad66cd0 2023-09-30T15:02:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.096+0000 7f4770744b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6bac54350 con 0x55d6bad66310 2023-09-30T15:02:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.097+0000 7f4770744b80 10 client.0 _ll_register_callbacks cb 0x55d6baab17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:02:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.097+0000 7f47622f8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad65200 0x55d6bad6ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.097+0000 7f47622f8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad65200 0x55d6bad6ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40408/0 (socket says 172.21.15.32:40408) 2023-09-30T15:02:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.097+0000 7f47622f8700 1 -- 172.21.15.32:0/435015164 learned_addr learned my addr 172.21.15.32:0/435015164 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.097+0000 7f47622f8700 1 -- 172.21.15.32:0/435015164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad66310 msgr2=0x55d6bad6f440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T15:02:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.097+0000 7f47622f8700 1 --2- 172.21.15.32:0/435015164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad66310 0x55d6bad6f440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.097+0000 7f47622f8700 1 -- 172.21.15.32:0/435015164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad66cd0 msgr2=0x55d6bad76480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.097+0000 7f47622f8700 1 --2- 172.21.15.32:0/435015164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad66cd0 0x55d6bad76480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.097+0000 7f47622f8700 1 -- 172.21.15.32:0/435015164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475c109bf0 con 0x55d6bad65200 2023-09-30T15:02:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.097+0000 7f47622f8700 1 --2- 172.21.15.32:0/435015164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad65200 0x55d6bad6ef00 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4758015520 tx=0x7f4758015600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f022504b7905a0c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.097+0000 7f4745ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T15:02:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.097+0000 7f4745ffb700 1 -- 172.21.15.32:0/435015164 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758015d80 con 0x55d6bad65200 2023-09-30T15:02:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.097+0000 7f4745ffb700 1 -- 172.21.15.32:0/435015164 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4758013450 con 0x55d6bad65200 2023-09-30T15:02:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.097+0000 7f4770744b80 1 -- 172.21.15.32:0/435015164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6bad58090 con 0x55d6bad65200 2023-09-30T15:02:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.097+0000 7f4745ffb700 1 -- 172.21.15.32:0/435015164 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4758013610 con 0x55d6bad65200 2023-09-30T15:02:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.098+0000 7f4770744b80 10 client.17448 fetch_fsmap learned FSMap version 57 2023-09-30T15:02:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.098+0000 7f4770744b80 1 -- 172.21.15.32:0/435015164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55d6baae2d80 con 0x55d6bad65200 2023-09-30T15:02:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.098+0000 7f4745ffb700 1 -- 172.21.15.32:0/435015164 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d6bad58090 con 0x55d6bad65200 2023-09-30T15:02:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.098+0000 7f4745ffb700 1 -- 172.21.15.32:0/435015164 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 57) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f475801e070 con 0x55d6bad65200 2023-09-30T15:02:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.098+0000 7f4770744b80 10 client.17448 fetch_fsmap finished waiting for FSMap version 57 2023-09-30T15:02:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.098+0000 7f4770744b80 10 client.17448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:02:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.098+0000 7f4770744b80 20 client.17448 populate_metadata read hostname 'smithi032' 2023-09-30T15:02:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.098+0000 7f4770744b80 1 --2- 172.21.15.32:0/435015164 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d6bad7ae80 0x55d6bad7d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.098+0000 7f4770744b80 4 client.17448 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T15:02:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.098+0000 7f4770744b80 1 -- 172.21.15.32:0/435015164 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6bacb0db0 con 0x55d6bad7ae80 2023-09-30T15:02:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.099+0000 7f4761af7700 1 --2- 172.21.15.32:0/435015164 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d6bad7ae80 0x55d6bad7d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.100+0000 7f4745ffb700 10 client.17448 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:02:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.100+0000 7f4761af7700 1 --2- 172.21.15.32:0/435015164 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d6bad7ae80 0x55d6bad7d2a0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0b1156251252dd server_cookie=6b341831a5720904 in_seq=0 out_seq=0 2023-09-30T15:02:50.110 INFO:tasks.workunit.client.0.smithi032.stdout: 10 435825 34.35 MB/sec execute 563 sec latency 258.017 ms 2023-09-30T15:02:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.111+0000 7f4745ffb700 1 -- 172.21.15.32:0/435015164 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d6bacb0db0 con 0x55d6bad7ae80 2023-09-30T15:02:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.111+0000 7f4745ffb700 10 client.17448 handle_command_reply: tid=0 2023-09-30T15:02:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.111+0000 7f4770744b80 1 client.17448 shutdown 2023-09-30T15:02:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.111+0000 7f4770744b80 1 -- 172.21.15.32:0/435015164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad65200 msgr2=0x55d6bad6ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.111+0000 7f4770744b80 1 --2- 172.21.15.32:0/435015164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad65200 0x55d6bad6ef00 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4758015520 tx=0x7f4758015600 comp rx=0 tx=0).stop 2023-09-30T15:02:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.111+0000 7f4770744b80 1 -- 172.21.15.32:0/435015164 shutdown_connections 2023-09-30T15:02:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.111+0000 7f4770744b80 1 --2- 172.21.15.32:0/435015164 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d6bad7ae80 0x55d6bad7d2a0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.111+0000 7f4770744b80 1 --2- 172.21.15.32:0/435015164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad66cd0 0x55d6bad76480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.111+0000 7f4745ffb700 0 client.17448 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:02:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.111+0000 7f4770744b80 1 --2- 172.21.15.32:0/435015164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad65200 0x55d6bad6ef00 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.111+0000 7f4770744b80 1 --2- 172.21.15.32:0/435015164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad66310 0x55d6bad6f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.111+0000 7f4770744b80 1 -- 172.21.15.32:0/435015164 >> 172.21.15.32:0/435015164 conn(0x55d6bacd9200 msgr2=0x55d6bad5f590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.111+0000 7f4770744b80 1 -- 172.21.15.32:0/435015164 shutdown_connections 2023-09-30T15:02:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.111+0000 7f4770744b80 1 -- 172.21.15.32:0/435015164 wait complete. 2023-09-30T15:02:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.111+0000 7f4770744b80 20 client.17448 trim_cache size 0 max 16384 2023-09-30T15:02:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.126+0000 7f4770744b80 1 Processor -- start 2023-09-30T15:02:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.126+0000 7f4770744b80 1 -- start start 2023-09-30T15:02:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.126+0000 7f4770744b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad65200 0x55d6bad666f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.126+0000 7f4770744b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad66310 0x55d6bad76440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.126+0000 7f4770744b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad6dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.127+0000 7f4770744b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6bab388c0 con 0x55d6bad66cd0 2023-09-30T15:02:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.127+0000 7f4770744b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6bad67100 con 0x55d6bad66310 2023-09-30T15:02:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.127+0000 7f4770744b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6bad57bd0 con 0x55d6bad65200 2023-09-30T15:02:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.127+0000 7f4762af9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad6dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.127+0000 7f4762af9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad6dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40418/0 (socket says 172.21.15.32:40418) 2023-09-30T15:02:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.127+0000 7f4762af9700 1 -- 172.21.15.32:0/807279075 learned_addr learned my addr 172.21.15.32:0/807279075 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.127+0000 7f47622f8700 1 --2- 172.21.15.32:0/807279075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad65200 0x55d6bad666f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.127+0000 7f4761af7700 1 --2- 172.21.15.32:0/807279075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad66310 0x55d6bad76440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.127+0000 7f4762af9700 1 -- 172.21.15.32:0/807279075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad65200 msgr2=0x55d6bad666f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.127+0000 7f4762af9700 1 --2- 172.21.15.32:0/807279075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad65200 0x55d6bad666f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.127+0000 7f4762af9700 1 -- 172.21.15.32:0/807279075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad66310 msgr2=0x55d6bad76440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.127+0000 7f4762af9700 1 --2- 172.21.15.32:0/807279075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad66310 0x55d6bad76440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.127+0000 7f4762af9700 1 -- 172.21.15.32:0/807279075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6baacbc00 con 0x55d6bad66cd0 2023-09-30T15:02:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.127+0000 7f4762af9700 1 --2- 172.21.15.32:0/807279075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad6dc10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f4754006270 tx=0x7f4754009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6bee6192329154 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.127+0000 7f47517fa700 1 -- 172.21.15.32:0/807279075 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475400b7b0 con 0x55d6bad66cd0 2023-09-30T15:02:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.127+0000 7f47517fa700 1 -- 172.21.15.32:0/807279075 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f475401f070 con 0x55d6bad66cd0 2023-09-30T15:02:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.128+0000 7f47517fa700 1 -- 172.21.15.32:0/807279075 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754005500 con 0x55d6bad66cd0 2023-09-30T15:02:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.128+0000 7f4770744b80 1 -- 172.21.15.32:0/807279075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 msgr2=0x55d6bad6dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.128+0000 7f4770744b80 1 --2- 172.21.15.32:0/807279075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad6dc10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f4754006270 tx=0x7f4754009810 comp rx=0 tx=0).stop 2023-09-30T15:02:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.128+0000 7f4770744b80 1 -- 172.21.15.32:0/807279075 shutdown_connections 2023-09-30T15:02:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.128+0000 7f4770744b80 1 --2- 172.21.15.32:0/807279075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad6dc10 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.128+0000 7f4770744b80 1 --2- 172.21.15.32:0/807279075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad65200 0x55d6bad666f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.128+0000 7f4770744b80 1 --2- 172.21.15.32:0/807279075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad66310 0x55d6bad76440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.128+0000 7f4770744b80 1 -- 172.21.15.32:0/807279075 >> 172.21.15.32:0/807279075 conn(0x55d6bacd9200 msgr2=0x55d6bad5fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.128+0000 7f4770744b80 1 -- 172.21.15.32:0/807279075 shutdown_connections 2023-09-30T15:02:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.128+0000 7f4770744b80 1 -- 172.21.15.32:0/807279075 wait complete. 2023-09-30T15:02:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4770744b80 1 -- start start 2023-09-30T15:02:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4770744b80 1 Processor -- start 2023-09-30T15:02:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4770744b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad65200 0x55d6bad73ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4770744b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad66310 0x55d6bad74400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4770744b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad75410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4770744b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6bad75950 con 0x55d6bad66cd0 2023-09-30T15:02:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4770744b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6bad75ac0 con 0x55d6bad65200 2023-09-30T15:02:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4770744b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6bad75c80 con 0x55d6bad66310 2023-09-30T15:02:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4770744b80 10 client.0 _ll_register_callbacks cb 0x7f475800ea70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:02:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4762af9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad75410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4762af9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad75410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40424/0 (socket says 172.21.15.32:40424) 2023-09-30T15:02:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4762af9700 1 -- 172.21.15.32:0/547661702 learned_addr learned my addr 172.21.15.32:0/547661702 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:02:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4761af7700 1 --2- 172.21.15.32:0/547661702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad66310 0x55d6bad74400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4762af9700 1 -- 172.21.15.32:0/547661702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad66310 msgr2=0x55d6bad74400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4762af9700 1 --2- 172.21.15.32:0/547661702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad66310 0x55d6bad74400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4762af9700 1 -- 172.21.15.32:0/547661702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad65200 msgr2=0x55d6bad73ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:02:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4762af9700 1 --2- 172.21.15.32:0/547661702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad65200 0x55d6bad73ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4762af9700 1 -- 172.21.15.32:0/547661702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6bad1a870 con 0x55d6bad66cd0 2023-09-30T15:02:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.129+0000 7f4761af7700 1 --2- 172.21.15.32:0/547661702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad66310 0x55d6bad74400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:02:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.130+0000 7f4745ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T15:02:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.130+0000 7f4762af9700 1 --2- 172.21.15.32:0/547661702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad75410 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4754007a60 tx=0x7f4754004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb843bc03ed04937 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:02:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.130+0000 7f4745ffb700 1 -- 172.21.15.32:0/547661702 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754005480 con 0x55d6bad66cd0 2023-09-30T15:02:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.130+0000 7f4770744b80 1 -- 172.21.15.32:0/547661702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47540076e0 con 0x55d6bad66cd0 2023-09-30T15:02:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.130+0000 7f4745ffb700 1 -- 172.21.15.32:0/547661702 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f475400be40 con 0x55d6bad66cd0 2023-09-30T15:02:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.130+0000 7f4745ffb700 1 -- 172.21.15.32:0/547661702 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47540055e0 con 0x55d6bad66cd0 2023-09-30T15:02:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.130+0000 7f4770744b80 10 client.17454 fetch_fsmap learned FSMap version 57 2023-09-30T15:02:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.130+0000 7f4770744b80 1 -- 172.21.15.32:0/547661702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55d6bab091f0 con 0x55d6bad66cd0 2023-09-30T15:02:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.130+0000 7f4745ffb700 1 -- 172.21.15.32:0/547661702 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47540076e0 con 0x55d6bad66cd0 2023-09-30T15:02:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.130+0000 7f4745ffb700 1 -- 172.21.15.32:0/547661702 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 57) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f475400b0d0 con 0x55d6bad66cd0 2023-09-30T15:02:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.130+0000 7f4770744b80 10 client.17454 fetch_fsmap finished waiting for FSMap version 57 2023-09-30T15:02:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.130+0000 7f4770744b80 10 client.17454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:02:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.130+0000 7f4770744b80 20 client.17454 populate_metadata read hostname 'smithi032' 2023-09-30T15:02:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.131+0000 7f4770744b80 1 --2- 172.21.15.32:0/547661702 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d6badcf410 0x55d6badd1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:02:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.131+0000 7f4770744b80 4 client.17454 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T15:02:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.131+0000 7f4770744b80 1 -- 172.21.15.32:0/547661702 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d6bacb0ad0 con 0x55d6badcf410 2023-09-30T15:02:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.135+0000 7f47622f8700 1 --2- 172.21.15.32:0/547661702 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d6badcf410 0x55d6badd1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:02:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.135+0000 7f47622f8700 1 --2- 172.21.15.32:0/547661702 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d6badcf410 0x55d6badd1800 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85ae16e33a0b1b48 server_cookie=3d985b7844d9dcd0 in_seq=0 out_seq=0 2023-09-30T15:02:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.135+0000 7f4745ffb700 10 client.17454 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:02:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.739+0000 7f4745ffb700 1 -- 172.21.15.32:0/547661702 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d6bacb0ad0 con 0x55d6badcf410 2023-09-30T15:02:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.739+0000 7f4745ffb700 10 client.17454 handle_command_reply: tid=0 2023-09-30T15:02:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.739+0000 7f4770744b80 1 client.17454 shutdown 2023-09-30T15:02:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.739+0000 7f4770744b80 1 -- 172.21.15.32:0/547661702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 msgr2=0x55d6bad75410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.739+0000 7f4770744b80 1 --2- 172.21.15.32:0/547661702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad75410 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4754007a60 tx=0x7f4754004e90 comp rx=0 tx=0).stop 2023-09-30T15:02:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.739+0000 7f4770744b80 1 -- 172.21.15.32:0/547661702 shutdown_connections 2023-09-30T15:02:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.739+0000 7f4770744b80 1 --2- 172.21.15.32:0/547661702 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x55d6badcf410 0x55d6badd1800 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.739+0000 7f4770744b80 1 --2- 172.21.15.32:0/547661702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6bad66cd0 0x55d6bad75410 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.739+0000 7f4770744b80 1 --2- 172.21.15.32:0/547661702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6bad65200 0x55d6bad73ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.739+0000 7f4770744b80 1 --2- 172.21.15.32:0/547661702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6bad66310 0x55d6bad74400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.739+0000 7f4770744b80 1 -- 172.21.15.32:0/547661702 >> 172.21.15.32:0/547661702 conn(0x55d6bacd9200 msgr2=0x55d6bad5fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.739+0000 7f4745ffb700 0 client.17454 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:02:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.739+0000 7f4770744b80 1 -- 172.21.15.32:0/547661702 shutdown_connections 2023-09-30T15:02:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.739+0000 7f4770744b80 1 -- 172.21.15.32:0/547661702 wait complete. 2023-09-30T15:02:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T15:02:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T15:02:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T15:02:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T15:02:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.739+0000 7f4770744b80 20 client.17454 trim_cache size 0 max 16384 2023-09-30T15:02:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.740+0000 7f476455c700 1 -- 172.21.15.32:0/3475128713 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f47480a1020 msgr2=0x7f47480a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.740+0000 7f476455c700 1 --2- 172.21.15.32:0/3475128713 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f47480a1020 0x7f47480a34e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f475c101a70 tx=0x7f4758009190 comp rx=0 tx=0).stop 2023-09-30T15:02:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.740+0000 7f476455c700 1 -- 172.21.15.32:0/3475128713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c1021b0 msgr2=0x7f475c1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:02:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.740+0000 7f476455c700 1 --2- 172.21.15.32:0/3475128713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c1021b0 0x7f475c1006a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f474c00e9a0 tx=0x7f474c00ecb0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.740+0000 7f476455c700 1 -- 172.21.15.32:0/3475128713 shutdown_connections 2023-09-30T15:02:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.740+0000 7f476455c700 1 --2- 172.21.15.32:0/3475128713 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f47480a1020 0x7f47480a34e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.740+0000 7f476455c700 1 --2- 172.21.15.32:0/3475128713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f475c102b50 0x7f475c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.740+0000 7f476455c700 1 --2- 172.21.15.32:0/3475128713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f475c0ff140 0x7f475c100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.740+0000 7f476455c700 1 --2- 172.21.15.32:0/3475128713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c1021b0 0x7f475c1006a0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:02:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.740+0000 7f476455c700 1 -- 172.21.15.32:0/3475128713 >> 172.21.15.32:0/3475128713 conn(0x7f475c0faa30 msgr2=0x7f475c0fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:02:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.741+0000 7f476455c700 1 -- 172.21.15.32:0/3475128713 shutdown_connections 2023-09-30T15:02:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:02:50.741+0000 7f476455c700 1 -- 172.21.15.32:0/3475128713 wait complete. 2023-09-30T15:02:51.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 436187 34.32 MB/sec execute 564 sec latency 360.482 ms 2023-09-30T15:02:51.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:51 smithi156 ceph-mon[103579]: pgmap v1024: 97 pgs: 97 active+clean; 810 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 694 op/s 2023-09-30T15:02:51.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:51 smithi148 ceph-mon[102355]: pgmap v1024: 97 pgs: 97 active+clean; 810 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 694 op/s 2023-09-30T15:02:51.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:51 smithi032 ceph-mon[98592]: pgmap v1024: 97 pgs: 97 active+clean; 810 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 694 op/s 2023-09-30T15:02:52.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 436717 34.30 MB/sec execute 565 sec latency 129.569 ms 2023-09-30T15:02:53.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 437472 34.32 MB/sec execute 566 sec latency 83.739 ms 2023-09-30T15:02:53.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:53 smithi156 ceph-mon[103579]: pgmap v1025: 97 pgs: 97 active+clean; 809 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 618 op/s 2023-09-30T15:02:53.862 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:53 smithi148 ceph-mon[102355]: pgmap v1025: 97 pgs: 97 active+clean; 809 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 618 op/s 2023-09-30T15:02:53.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:53 smithi032 ceph-mon[98592]: pgmap v1025: 97 pgs: 97 active+clean; 809 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 618 op/s 2023-09-30T15:02:54.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 438085 34.33 MB/sec execute 567 sec latency 61.116 ms 2023-09-30T15:02:55.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 438542 34.30 MB/sec execute 568 sec latency 171.496 ms 2023-09-30T15:02:55.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:55 smithi156 ceph-mon[103579]: pgmap v1026: 97 pgs: 97 active+clean; 804 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.4 MiB/s wr, 644 op/s 2023-09-30T15:02:55.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:55 smithi148 ceph-mon[102355]: pgmap v1026: 97 pgs: 97 active+clean; 804 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.4 MiB/s wr, 644 op/s 2023-09-30T15:02:55.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:55 smithi032 ceph-mon[98592]: pgmap v1026: 97 pgs: 97 active+clean; 804 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.4 MiB/s wr, 644 op/s 2023-09-30T15:02:56.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 438932 34.27 MB/sec execute 569 sec latency 263.926 ms 2023-09-30T15:02:57.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 439541 34.27 MB/sec execute 570 sec latency 99.945 ms 2023-09-30T15:02:57.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:57 smithi156 ceph-mon[103579]: pgmap v1027: 97 pgs: 97 active+clean; 805 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.5 MiB/s wr, 679 op/s 2023-09-30T15:02:57.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:57 smithi148 ceph-mon[102355]: pgmap v1027: 97 pgs: 97 active+clean; 805 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.5 MiB/s wr, 679 op/s 2023-09-30T15:02:57.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:57 smithi032 ceph-mon[98592]: pgmap v1027: 97 pgs: 97 active+clean; 805 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.5 MiB/s wr, 679 op/s 2023-09-30T15:02:58.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 440272 34.27 MB/sec execute 571 sec latency 78.936 ms 2023-09-30T15:02:59.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 440918 34.29 MB/sec execute 572 sec latency 62.440 ms 2023-09-30T15:02:59.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:02:59 smithi156 ceph-mon[103579]: pgmap v1028: 97 pgs: 97 active+clean; 811 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.4 MiB/s wr, 628 op/s 2023-09-30T15:02:59.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:02:59 smithi148 ceph-mon[102355]: pgmap v1028: 97 pgs: 97 active+clean; 811 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.4 MiB/s wr, 628 op/s 2023-09-30T15:02:59.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:02:59 smithi032 ceph-mon[98592]: pgmap v1028: 97 pgs: 97 active+clean; 811 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.4 MiB/s wr, 628 op/s 2023-09-30T15:03:00.115 INFO:tasks.workunit.client.0.smithi032.stdout: 10 441451 34.28 MB/sec execute 573 sec latency 64.109 ms 2023-09-30T15:03:01.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 441770 34.25 MB/sec execute 574 sec latency 285.133 ms 2023-09-30T15:03:01.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:01 smithi156 ceph-mon[103579]: pgmap v1029: 97 pgs: 97 active+clean; 814 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.6 MiB/s wr, 693 op/s 2023-09-30T15:03:01.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:01 smithi148 ceph-mon[102355]: pgmap v1029: 97 pgs: 97 active+clean; 814 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.6 MiB/s wr, 693 op/s 2023-09-30T15:03:01.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:01 smithi032 ceph-mon[98592]: pgmap v1029: 97 pgs: 97 active+clean; 814 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.6 MiB/s wr, 693 op/s 2023-09-30T15:03:02.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 442243 34.22 MB/sec execute 575 sec latency 99.950 ms 2023-09-30T15:03:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T15:03:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:03:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:03:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:03:02.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:02 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:03:02.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T15:03:02.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:03:02.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:03:02.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:03:02.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:02 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:03:02.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T15:03:02.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:03:02.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:03:02.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:03:02.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:02 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:03:03.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 442885 34.22 MB/sec execute 576 sec latency 94.363 ms 2023-09-30T15:03:03.808 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:03 smithi156 ceph-mon[103579]: pgmap v1030: 97 pgs: 97 active+clean; 816 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.6 MiB/s wr, 590 op/s 2023-09-30T15:03:03.808 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:03 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T15:03:03.808 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:03 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T15:03:03.808 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:03 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:03:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:03 smithi148 ceph-mon[102355]: pgmap v1030: 97 pgs: 97 active+clean; 816 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.6 MiB/s wr, 590 op/s 2023-09-30T15:03:03.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:03 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T15:03:03.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:03 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T15:03:03.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:03 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:03:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.876+0000 7fbf32d66700 1 -- 172.21.15.32:0/1511093403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2c104260 msgr2=0x7fbf2c106650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.876+0000 7fbf32d66700 1 --2- 172.21.15.32:0/1511093403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2c104260 0x7fbf2c106650 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009990 tx=0x7fbf20009ca0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.876+0000 7fbf32d66700 1 -- 172.21.15.32:0/1511093403 shutdown_connections 2023-09-30T15:03:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.876+0000 7fbf32d66700 1 --2- 172.21.15.32:0/1511093403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf2c106b90 0x7fbf2c108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.876+0000 7fbf32d66700 1 --2- 172.21.15.32:0/1511093403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf2c101930 0x7fbf2c103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.876+0000 7fbf32d66700 1 --2- 172.21.15.32:0/1511093403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2c104260 0x7fbf2c106650 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.876+0000 7fbf32d66700 1 -- 172.21.15.32:0/1511093403 >> 172.21.15.32:0/1511093403 conn(0x7fbf2c0fb300 msgr2=0x7fbf2c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.876+0000 7fbf32d66700 1 -- 172.21.15.32:0/1511093403 shutdown_connections 2023-09-30T15:03:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.876+0000 7fbf32d66700 1 -- 172.21.15.32:0/1511093403 wait complete. 2023-09-30T15:03:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.876+0000 7fbf32d66700 1 Processor -- start 2023-09-30T15:03:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.876+0000 7fbf32d66700 1 -- start start 2023-09-30T15:03:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf32d66700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf2c101930 0x7fbf2c196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf32d66700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf2c104260 0x7fbf2c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf32d66700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2c106b90 0x7fbf2c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf32d66700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c198690 con 0x7fbf2c106b90 2023-09-30T15:03:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf32d66700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c198800 con 0x7fbf2c104260 2023-09-30T15:03:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf32d66700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c1989c0 con 0x7fbf2c101930 2023-09-30T15:03:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf30b02700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf2c101930 0x7fbf2c196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf31303700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2c106b90 0x7fbf2c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf30b02700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf2c101930 0x7fbf2c196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54816/0 (socket says 172.21.15.32:54816) 2023-09-30T15:03:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf30b02700 1 -- 172.21.15.32:0/1852371250 learned_addr learned my addr 172.21.15.32:0/1852371250 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:03:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf31303700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2c106b90 0x7fbf2c197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40438/0 (socket says 172.21.15.32:40438) 2023-09-30T15:03:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf31303700 1 -- 172.21.15.32:0/1852371250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf2c101930 msgr2=0x7fbf2c196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf31303700 1 --2- 172.21.15.32:0/1852371250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf2c101930 0x7fbf2c196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf31303700 1 -- 172.21.15.32:0/1852371250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf2c104260 msgr2=0x7fbf2c197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf31303700 1 --2- 172.21.15.32:0/1852371250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf2c104260 0x7fbf2c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf31303700 1 -- 172.21.15.32:0/1852371250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf24009c40 con 0x7fbf2c106b90 2023-09-30T15:03:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.877+0000 7fbf31303700 1 --2- 172.21.15.32:0/1852371250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2c106b90 0x7fbf2c197f90 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400efb0 tx=0x7fbf2400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.878+0000 7fbf29ffb700 1 -- 172.21.15.32:0/1852371250 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2400ca50 con 0x7fbf2c106b90 2023-09-30T15:03:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.878+0000 7fbf29ffb700 1 -- 172.21.15.32:0/1852371250 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf2400f040 con 0x7fbf2c106b90 2023-09-30T15:03:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.878+0000 7fbf29ffb700 1 -- 172.21.15.32:0/1852371250 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24004660 con 0x7fbf2c106b90 2023-09-30T15:03:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.878+0000 7fbf32d66700 1 -- 172.21.15.32:0/1852371250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf200095f0 con 0x7fbf2c106b90 2023-09-30T15:03:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.878+0000 7fbf32d66700 1 -- 172.21.15.32:0/1852371250 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2c198df0 con 0x7fbf2c106b90 2023-09-30T15:03:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.880+0000 7fbf29ffb700 1 -- 172.21.15.32:0/1852371250 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fbf240047c0 con 0x7fbf2c106b90 2023-09-30T15:03:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.881+0000 7fbf29ffb700 1 --2- 172.21.15.32:0/1852371250 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbf140a0fa0 0x7fbf140a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.881+0000 7fbf29ffb700 1 -- 172.21.15.32:0/1852371250 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbf240cc3a0 con 0x7fbf2c106b90 2023-09-30T15:03:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.881+0000 7fbf30b02700 1 --2- 172.21.15.32:0/1852371250 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbf140a0fa0 0x7fbf140a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.881+0000 7fbf30b02700 1 --2- 172.21.15.32:0/1852371250 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbf140a0fa0 0x7fbf140a3460 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009d20 tx=0x7fbf1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.882+0000 7fbf3ef4eb80 1 Processor -- start 2023-09-30T15:03:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.882+0000 7fbf3ef4eb80 1 -- start start 2023-09-30T15:03:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.882+0000 7fbf3ef4eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc2200 0x555732dc2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.882+0000 7fbf3ef4eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc3310 0x555732dc3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.882+0000 7fbf3ef4eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 0x555732dca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.882+0000 7fbf3ef4eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x555732d2a400 con 0x555732dc2200 2023-09-30T15:03:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.882+0000 7fbf3ef4eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555732d1bbc0 con 0x555732dc3310 2023-09-30T15:03:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.882+0000 7fbf3ef4eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555732d194e0 con 0x555732dc3cd0 2023-09-30T15:03:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.882+0000 7fbf30b02700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc2200 0x555732dc2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.882+0000 7fbf30b02700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc2200 0x555732dc2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40442/0 (socket says 172.21.15.32:40442) 2023-09-30T15:03:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.882+0000 7fbf30b02700 1 -- 172.21.15.32:0/3252712137 learned_addr learned my addr 172.21.15.32:0/3252712137 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:03:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.882+0000 7fbf31303700 1 --2- 172.21.15.32:0/3252712137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 0x555732dca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.882+0000 7fbf2bfff700 1 --2- 172.21.15.32:0/3252712137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc3310 0x555732dc3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.883+0000 7fbf30b02700 1 -- 172.21.15.32:0/3252712137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 msgr2=0x555732dca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:03:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.883+0000 7fbf30b02700 1 --2- 172.21.15.32:0/3252712137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 0x555732dca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.883+0000 7fbf30b02700 1 -- 172.21.15.32:0/3252712137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc3310 msgr2=0x555732dc3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:03:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.883+0000 7fbf30b02700 1 --2- 172.21.15.32:0/3252712137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc3310 0x555732dc3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.883+0000 7fbf30b02700 1 -- 172.21.15.32:0/3252712137 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555732d77870 con 0x555732dc2200 2023-09-30T15:03:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.883+0000 7fbf31303700 1 --2- 172.21.15.32:0/3252712137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 0x555732dca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:03:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.883+0000 7fbf30b02700 1 --2- 172.21.15.32:0/3252712137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc2200 0x555732dc2620 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00e540 tx=0x7fbf1c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b29ae768e587fe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.883+0000 7fbf12ffd700 1 -- 172.21.15.32:0/3252712137 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c026430 con 0x555732dc2200 2023-09-30T15:03:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.883+0000 7fbf12ffd700 1 -- 172.21.15.32:0/3252712137 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf1c021070 con 0x555732dc2200 2023-09-30T15:03:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.883+0000 7fbf12ffd700 1 -- 172.21.15.32:0/3252712137 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c01d6e0 con 0x555732dc2200 2023-09-30T15:03:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.884+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/3252712137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc2200 msgr2=0x555732dc2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.884+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/3252712137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc2200 0x555732dc2620 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00e540 tx=0x7fbf1c014000 comp rx=0 tx=0).stop 2023-09-30T15:03:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.884+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/3252712137 shutdown_connections 2023-09-30T15:03:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.884+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/3252712137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 0x555732dca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.884+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/3252712137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc2200 0x555732dc2620 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.884+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/3252712137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc3310 0x555732dc3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.884+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/3252712137 >> 172.21.15.32:0/3252712137 conn(0x555732d36200 msgr2=0x555732dbcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.884+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/3252712137 shutdown_connections 2023-09-30T15:03:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.884+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/3252712137 wait complete. 2023-09-30T15:03:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.884+0000 7fbf3ef4eb80 1 -- start start 2023-09-30T15:03:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.884+0000 7fbf3ef4eb80 1 Processor -- start 2023-09-30T15:03:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf3ef4eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc2200 0x555732dcbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf3ef4eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 0x555732dcc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf3ef4eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 0x555732dd3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf3ef4eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x555732d14dd0 con 0x555732dc3310 2023-09-30T15:03:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf3ef4eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555732cb16b0 con 0x555732dc2200 2023-09-30T15:03:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf3ef4eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555732cb1350 con 0x555732dc3cd0 2023-09-30T15:03:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf3ef4eb80 10 client.0 _ll_register_callbacks cb 0x555732b0e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:03:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf2bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 0x555732dcc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf2bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 0x555732dcc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40450/0 (socket says 172.21.15.32:40450) 2023-09-30T15:03:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf2bfff700 1 -- 172.21.15.32:0/2139629892 learned_addr learned my addr 172.21.15.32:0/2139629892 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:03:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf31303700 1 --2- 172.21.15.32:0/2139629892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 0x555732dd3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf30b02700 1 --2- 172.21.15.32:0/2139629892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc2200 0x555732dcbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf2bfff700 1 -- 172.21.15.32:0/2139629892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 msgr2=0x555732dd3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:03:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf2bfff700 1 --2- 172.21.15.32:0/2139629892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 0x555732dd3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf2bfff700 1 -- 172.21.15.32:0/2139629892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc2200 msgr2=0x555732dcbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:03:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf2bfff700 1 --2- 172.21.15.32:0/2139629892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc2200 0x555732dcbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.885+0000 7fbf2bfff700 1 -- 172.21.15.32:0/2139629892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2c198df0 con 0x555732dc3310 2023-09-30T15:03:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.886+0000 7fbf2bfff700 1 --2- 172.21.15.32:0/2139629892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 0x555732dcc400 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000bef0 tx=0x7fbf2000bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1cb000825b5d40e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.886+0000 7fbf0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-09-30T15:03:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.886+0000 7fbf0bfff700 1 -- 172.21.15.32:0/2139629892 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2001f040 con 0x555732dc3310 2023-09-30T15:03:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.886+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/2139629892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555732db5090 con 0x555732dc3310 2023-09-30T15:03:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.886+0000 7fbf0bfff700 1 -- 172.21.15.32:0/2139629892 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbf20004500 con 0x555732dc3310 2023-09-30T15:03:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.886+0000 7fbf0bfff700 1 -- 172.21.15.32:0/2139629892 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf200046c0 con 0x555732dc3310 2023-09-30T15:03:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.886+0000 7fbf3ef4eb80 10 client.17466 fetch_fsmap learned FSMap version 57 2023-09-30T15:03:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.886+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/2139629892 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x7fbf1c017070 con 0x555732dc3310 2023-09-30T15:03:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.886+0000 7fbf0bfff700 1 -- 172.21.15.32:0/2139629892 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555732db5090 con 0x555732dc3310 2023-09-30T15:03:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.886+0000 7fbf0bfff700 1 -- 172.21.15.32:0/2139629892 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 57) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbf2000f7f0 con 0x555732dc3310 2023-09-30T15:03:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.887+0000 7fbf3ef4eb80 10 client.17466 fetch_fsmap finished waiting for FSMap version 57 2023-09-30T15:03:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.887+0000 7fbf3ef4eb80 10 client.17466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:03:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.887+0000 7fbf3ef4eb80 20 client.17466 populate_metadata read hostname 'smithi032' 2023-09-30T15:03:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.887+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/2139629892 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x555732dd7d10 0x555732dda130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.887+0000 7fbf3ef4eb80 4 client.17466 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T15:03:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.887+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/2139629892 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555732d0ddb0 con 0x555732dd7d10 2023-09-30T15:03:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.887+0000 7fbf30b02700 1 --2- 172.21.15.32:0/2139629892 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x555732dd7d10 0x555732dda130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:03.901 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:03 smithi032 ceph-mon[98592]: pgmap v1030: 97 pgs: 97 active+clean; 816 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.6 MiB/s wr, 590 op/s 2023-09-30T15:03:03.901 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:03 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T15:03:03.901 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:03 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T15:03:03.901 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:03 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:03:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.887+0000 7fbf0bfff700 10 client.17466 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:03:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.887+0000 7fbf30b02700 1 --2- 172.21.15.32:0/2139629892 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x555732dd7d10 0x555732dda130 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6ca1e5aa3f7cdef server_cookie=7600539546df6b00 in_seq=0 out_seq=0 2023-09-30T15:03:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.888+0000 7fbf0bfff700 1 -- 172.21.15.32:0/2139629892 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555732d0ddb0 con 0x555732dd7d10 2023-09-30T15:03:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.888+0000 7fbf0bfff700 10 client.17466 handle_command_reply: tid=0 2023-09-30T15:03:03.902 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.888+0000 7fbf3ef4eb80 1 client.17466 shutdown 2023-09-30T15:03:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.889+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/2139629892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 msgr2=0x555732dcc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.889+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/2139629892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 0x555732dcc400 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000bef0 tx=0x7fbf2000bf20 comp rx=0 tx=0).stop 2023-09-30T15:03:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.889+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/2139629892 shutdown_connections 2023-09-30T15:03:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.889+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/2139629892 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x555732dd7d10 0x555732dda130 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.889+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/2139629892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 0x555732dd3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.889+0000 7fbf0bfff700 0 client.17466 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:03:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.889+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/2139629892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc2200 0x555732dcbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.889+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/2139629892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 0x555732dcc400 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.889+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/2139629892 >> 172.21.15.32:0/2139629892 conn(0x555732d36200 msgr2=0x555732dbc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.889+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/2139629892 shutdown_connections 2023-09-30T15:03:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.889+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/2139629892 wait complete. 2023-09-30T15:03:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.889+0000 7fbf3ef4eb80 20 client.17466 trim_cache size 0 max 16384 2023-09-30T15:03:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.906+0000 7fbf3ef4eb80 1 Processor -- start 2023-09-30T15:03:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.906+0000 7fbf3ef4eb80 1 -- start start 2023-09-30T15:03:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.906+0000 7fbf3ef4eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc2200 0x555732dc36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.906+0000 7fbf3ef4eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 0x555732dd3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.906+0000 7fbf3ef4eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 0x555732dca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.906+0000 7fbf3ef4eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x555732b958c0 con 0x555732dc3310 2023-09-30T15:03:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.906+0000 7fbf3ef4eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555732dcba90 con 0x555732dc2200 2023-09-30T15:03:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.906+0000 7fbf3ef4eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555732dbea40 con 0x555732dc3cd0 2023-09-30T15:03:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.906+0000 7fbf30b02700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc2200 0x555732dc36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.906+0000 7fbf2bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 0x555732dd3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.906+0000 7fbf30b02700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc2200 0x555732dc36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:51932/0 (socket says 172.21.15.32:51932) 2023-09-30T15:03:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.906+0000 7fbf30b02700 1 -- 172.21.15.32:0/2548196937 learned_addr learned my addr 172.21.15.32:0/2548196937 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:03:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.906+0000 7fbf2bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 0x555732dd3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40460/0 (socket says 172.21.15.32:40460) 2023-09-30T15:03:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.906+0000 7fbf31303700 1 --2- 172.21.15.32:0/2548196937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 0x555732dca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.907+0000 7fbf2bfff700 1 -- 172.21.15.32:0/2548196937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 msgr2=0x555732dca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:03:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.907+0000 7fbf2bfff700 1 --2- 172.21.15.32:0/2548196937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 0x555732dca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.907+0000 7fbf2bfff700 1 -- 172.21.15.32:0/2548196937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc2200 msgr2=0x555732dc36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:03:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.907+0000 7fbf2bfff700 1 --2- 172.21.15.32:0/2548196937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc2200 0x555732dc36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.907+0000 7fbf2bfff700 1 -- 172.21.15.32:0/2548196937 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555732b3fd80 con 0x555732dc3310 2023-09-30T15:03:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.907+0000 7fbf31303700 1 --2- 172.21.15.32:0/2548196937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 0x555732dca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:03:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.907+0000 7fbf2bfff700 1 --2- 172.21.15.32:0/2548196937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 0x555732dd3400 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009990 tx=0x7fbf20005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706508feb0d09099 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.907+0000 7fbf137fe700 1 -- 172.21.15.32:0/2548196937 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2001f070 con 0x555732dc3310 2023-09-30T15:03:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.907+0000 7fbf137fe700 1 -- 172.21.15.32:0/2548196937 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf2000fd70 con 0x555732dc3310 2023-09-30T15:03:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.907+0000 7fbf137fe700 1 -- 172.21.15.32:0/2548196937 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf200179a0 con 0x555732dc3310 2023-09-30T15:03:03.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.908+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/2548196937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 msgr2=0x555732dd3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:03.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.908+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/2548196937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 0x555732dd3400 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009990 tx=0x7fbf20005fb0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.912 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.908+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/2548196937 shutdown_connections 2023-09-30T15:03:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.908+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/2548196937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc3cd0 0x555732dca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.908+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/2548196937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc2200 0x555732dc36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.908+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/2548196937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 0x555732dd3400 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.908+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/2548196937 >> 172.21.15.32:0/2548196937 conn(0x555732d36200 msgr2=0x555732dcac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.908+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/2548196937 shutdown_connections 2023-09-30T15:03:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.908+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/2548196937 wait complete. 2023-09-30T15:03:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.909+0000 7fbf3ef4eb80 1 -- start start 2023-09-30T15:03:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.909+0000 7fbf3ef4eb80 1 Processor -- start 2023-09-30T15:03:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.909+0000 7fbf3ef4eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc2200 0x555732dd1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.909+0000 7fbf3ef4eb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 0x555732dd1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.909+0000 7fbf3ef4eb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc3cd0 0x555732dd2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.909+0000 7fbf3ef4eb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x555732dd2bc0 con 0x555732dc3310 2023-09-30T15:03:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.909+0000 7fbf3ef4eb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555732dd2d30 con 0x555732dc3cd0 2023-09-30T15:03:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.909+0000 7fbf3ef4eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555732dd2ea0 con 0x555732dc2200 2023-09-30T15:03:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.909+0000 7fbf3ef4eb80 10 client.0 _ll_register_callbacks cb 0x7fbf20009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:03:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.910+0000 7fbf30b02700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc2200 0x555732dd1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.910+0000 7fbf30b02700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc2200 0x555732dd1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54844/0 (socket says 172.21.15.32:54844) 2023-09-30T15:03:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.910+0000 7fbf30b02700 1 -- 172.21.15.32:0/1715754041 learned_addr learned my addr 172.21.15.32:0/1715754041 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:03:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.910+0000 7fbf30b02700 1 -- 172.21.15.32:0/1715754041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc3cd0 msgr2=0x555732dd2680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-30T15:03:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.910+0000 7fbf30b02700 1 --2- 172.21.15.32:0/1715754041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc3cd0 0x555732dd2680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.910+0000 7fbf30b02700 1 -- 172.21.15.32:0/1715754041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 msgr2=0x555732dd1670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T15:03:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.910+0000 7fbf30b02700 1 --2- 172.21.15.32:0/1715754041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 0x555732dd1670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.910+0000 7fbf30b02700 1 -- 172.21.15.32:0/1715754041 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf24021020 con 0x555732dc2200 2023-09-30T15:03:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.910+0000 7fbf30b02700 1 --2- 172.21.15.32:0/1715754041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc2200 0x555732dd1130 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00eb30 tx=0x7fbf1c005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c67979e8888292c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.910+0000 7fbf0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T15:03:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.910+0000 7fbf0bfff700 1 -- 172.21.15.32:0/1715754041 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c01cd00 con 0x555732dc2200 2023-09-30T15:03:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.910+0000 7fbf0bfff700 1 -- 172.21.15.32:0/1715754041 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbf1c01ce60 con 0x555732dc2200 2023-09-30T15:03:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.910+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/1715754041 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf1c021070 con 0x555732dc2200 2023-09-30T15:03:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.911+0000 7fbf0bfff700 1 -- 172.21.15.32:0/1715754041 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf1c033a90 con 0x555732dc2200 2023-09-30T15:03:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.911+0000 7fbf3ef4eb80 10 client.27118 fetch_fsmap learned FSMap version 57 2023-09-30T15:03:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.911+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/1715754041 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x7fbf1c017070 con 0x555732dc2200 2023-09-30T15:03:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.911+0000 7fbf0bfff700 1 -- 172.21.15.32:0/1715754041 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf1c021070 con 0x555732dc2200 2023-09-30T15:03:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.911+0000 7fbf0bfff700 1 -- 172.21.15.32:0/1715754041 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 57) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbf1c01c670 con 0x555732dc2200 2023-09-30T15:03:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.911+0000 7fbf3ef4eb80 10 client.27118 fetch_fsmap finished waiting for FSMap version 57 2023-09-30T15:03:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.911+0000 7fbf3ef4eb80 10 client.27118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:03:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.911+0000 7fbf3ef4eb80 20 client.27118 populate_metadata read hostname 'smithi032' 2023-09-30T15:03:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.911+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/1715754041 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x555732e3da40 0x555732e3fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.911+0000 7fbf3ef4eb80 4 client.27118 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T15:03:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.911+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/1715754041 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555732d0dad0 con 0x555732e3da40 2023-09-30T15:03:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.911+0000 7fbf2bfff700 1 --2- 172.21.15.32:0/1715754041 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x555732e3da40 0x555732e3fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.912+0000 7fbf0bfff700 10 client.27118 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:03:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:03.912+0000 7fbf2bfff700 1 --2- 172.21.15.32:0/1715754041 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x555732e3da40 0x555732e3fe60 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c555cd13f2d1c4f server_cookie=99428e55dc158f9c in_seq=0 out_seq=0 2023-09-30T15:03:04.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 443535 34.21 MB/sec execute 577 sec latency 106.036 ms 2023-09-30T15:03:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.400+0000 7fbf0bfff700 1 -- 172.21.15.32:0/1715754041 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555732d0dad0 con 0x555732e3da40 2023-09-30T15:03:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.400+0000 7fbf0bfff700 10 client.27118 handle_command_reply: tid=0 2023-09-30T15:03:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.401+0000 7fbf3ef4eb80 1 client.27118 shutdown 2023-09-30T15:03:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.401+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/1715754041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc2200 msgr2=0x555732dd1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.401+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/1715754041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc2200 0x555732dd1130 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00eb30 tx=0x7fbf1c005d30 comp rx=0 tx=0).stop 2023-09-30T15:03:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.401+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/1715754041 shutdown_connections 2023-09-30T15:03:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.401+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/1715754041 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x555732e3da40 0x555732e3fe60 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.401+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/1715754041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555732dc3cd0 0x555732dd2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.401+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/1715754041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555732dc2200 0x555732dd1130 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.401+0000 7fbf3ef4eb80 1 --2- 172.21.15.32:0/1715754041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x555732dc3310 0x555732dd1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.401+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/1715754041 >> 172.21.15.32:0/1715754041 conn(0x555732d36200 msgr2=0x555732dca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.401+0000 7fbf0bfff700 0 client.27118 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:03:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.401+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/1715754041 shutdown_connections 2023-09-30T15:03:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.401+0000 7fbf3ef4eb80 1 -- 172.21.15.32:0/1715754041 wait complete. 2023-09-30T15:03:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.402+0000 7fbf3ef4eb80 20 client.27118 trim_cache size 0 max 16384 2023-09-30T15:03:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T15:03:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T15:03:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T15:03:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T15:03:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.402+0000 7fbf32d66700 1 -- 172.21.15.32:0/1852371250 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbf140a0fa0 msgr2=0x7fbf140a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.402+0000 7fbf32d66700 1 --2- 172.21.15.32:0/1852371250 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbf140a0fa0 0x7fbf140a3460 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009d20 tx=0x7fbf1c009260 comp rx=0 tx=0).stop 2023-09-30T15:03:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.402+0000 7fbf32d66700 1 -- 172.21.15.32:0/1852371250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2c106b90 msgr2=0x7fbf2c197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.402+0000 7fbf32d66700 1 --2- 172.21.15.32:0/1852371250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2c106b90 0x7fbf2c197f90 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400efb0 tx=0x7fbf2400c3a0 comp rx=0 tx=0).stop 2023-09-30T15:03:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.402+0000 7fbf32d66700 1 -- 172.21.15.32:0/1852371250 shutdown_connections 2023-09-30T15:03:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.402+0000 7fbf32d66700 1 --2- 172.21.15.32:0/1852371250 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbf140a0fa0 0x7fbf140a3460 secure :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009d20 tx=0x7fbf1c009260 comp rx=0 tx=0).stop 2023-09-30T15:03:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.402+0000 7fbf32d66700 1 --2- 172.21.15.32:0/1852371250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2c106b90 0x7fbf2c197f90 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.402+0000 7fbf32d66700 1 --2- 172.21.15.32:0/1852371250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf2c101930 0x7fbf2c196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.402+0000 7fbf32d66700 1 --2- 172.21.15.32:0/1852371250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf2c104260 0x7fbf2c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.402+0000 7fbf32d66700 1 -- 172.21.15.32:0/1852371250 >> 172.21.15.32:0/1852371250 conn(0x7fbf2c0fb300 msgr2=0x7fbf2c1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.404+0000 7fbf32d66700 1 -- 172.21.15.32:0/1852371250 shutdown_connections 2023-09-30T15:03:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:04.404+0000 7fbf32d66700 1 -- 172.21.15.32:0/1852371250 wait complete. 2023-09-30T15:03:05.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 444124 34.22 MB/sec execute 578 sec latency 313.475 ms 2023-09-30T15:03:05.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:05 smithi156 ceph-mon[103579]: pgmap v1031: 97 pgs: 97 active+clean; 812 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.1 MiB/s wr, 629 op/s 2023-09-30T15:03:05.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:05 smithi148 ceph-mon[102355]: pgmap v1031: 97 pgs: 97 active+clean; 812 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.1 MiB/s wr, 629 op/s 2023-09-30T15:03:05.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:05 smithi032 ceph-mon[98592]: pgmap v1031: 97 pgs: 97 active+clean; 812 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.1 MiB/s wr, 629 op/s 2023-09-30T15:03:06.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 444575 34.22 MB/sec execute 579 sec latency 296.549 ms 2023-09-30T15:03:07.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 445050 34.20 MB/sec execute 580 sec latency 71.349 ms 2023-09-30T15:03:07.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:07 smithi156 ceph-mon[103579]: pgmap v1032: 97 pgs: 97 active+clean; 811 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.2 MiB/s wr, 641 op/s 2023-09-30T15:03:07.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:07 smithi148 ceph-mon[102355]: pgmap v1032: 97 pgs: 97 active+clean; 811 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.2 MiB/s wr, 641 op/s 2023-09-30T15:03:07.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:07 smithi032 ceph-mon[98592]: pgmap v1032: 97 pgs: 97 active+clean; 811 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.2 MiB/s wr, 641 op/s 2023-09-30T15:03:08.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 445617 34.19 MB/sec execute 581 sec latency 81.283 ms 2023-09-30T15:03:09.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 446159 34.17 MB/sec execute 582 sec latency 107.282 ms 2023-09-30T15:03:09.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:09 smithi156 ceph-mon[103579]: pgmap v1033: 97 pgs: 97 active+clean; 809 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 569 op/s 2023-09-30T15:03:09.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:09 smithi148 ceph-mon[102355]: pgmap v1033: 97 pgs: 97 active+clean; 809 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 569 op/s 2023-09-30T15:03:09.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:09 smithi032 ceph-mon[98592]: pgmap v1033: 97 pgs: 97 active+clean; 809 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.2 MiB/s wr, 569 op/s 2023-09-30T15:03:10.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 446903 34.18 MB/sec execute 583 sec latency 85.970 ms 2023-09-30T15:03:11.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 447349 34.17 MB/sec execute 584 sec latency 348.410 ms 2023-09-30T15:03:11.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:11 smithi156 ceph-mon[103579]: pgmap v1034: 97 pgs: 97 active+clean; 811 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.0 MiB/s wr, 625 op/s 2023-09-30T15:03:11.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:11 smithi148 ceph-mon[102355]: pgmap v1034: 97 pgs: 97 active+clean; 811 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.0 MiB/s wr, 625 op/s 2023-09-30T15:03:11.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:11 smithi032 ceph-mon[98592]: pgmap v1034: 97 pgs: 97 active+clean; 811 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.0 MiB/s wr, 625 op/s 2023-09-30T15:03:12.111 INFO:tasks.workunit.client.0.smithi032.stdout: 10 447876 34.16 MB/sec execute 585 sec latency 112.404 ms 2023-09-30T15:03:13.112 INFO:tasks.workunit.client.0.smithi032.stdout: 10 448467 34.16 MB/sec execute 586 sec latency 81.490 ms 2023-09-30T15:03:13.808 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:13 smithi156 ceph-mon[103579]: pgmap v1035: 97 pgs: 97 active+clean; 819 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.0 MiB/s wr, 643 op/s 2023-09-30T15:03:13.865 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:13 smithi148 ceph-mon[102355]: pgmap v1035: 97 pgs: 97 active+clean; 819 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.0 MiB/s wr, 643 op/s 2023-09-30T15:03:13.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:13 smithi032 ceph-mon[98592]: pgmap v1035: 97 pgs: 97 active+clean; 819 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.0 MiB/s wr, 643 op/s 2023-09-30T15:03:14.112 INFO:tasks.workunit.client.0.smithi032.stdout: 10 449172 34.16 MB/sec execute 587 sec latency 84.668 ms 2023-09-30T15:03:15.112 INFO:tasks.workunit.client.0.smithi032.stdout: 10 449833 34.19 MB/sec execute 588 sec latency 97.767 ms 2023-09-30T15:03:15.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:15 smithi148 ceph-mon[102355]: pgmap v1036: 97 pgs: 97 active+clean; 817 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.0 MiB/s wr, 688 op/s 2023-09-30T15:03:15.921 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:15 smithi032 ceph-mon[98592]: pgmap v1036: 97 pgs: 97 active+clean; 817 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.0 MiB/s wr, 688 op/s 2023-09-30T15:03:16.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:15 smithi156 ceph-mon[103579]: pgmap v1036: 97 pgs: 97 active+clean; 817 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.0 MiB/s wr, 688 op/s 2023-09-30T15:03:16.533 INFO:tasks.workunit.client.0.smithi032.stdout: 10 450267 34.18 MB/sec execute 589 sec latency 392.794 ms 2023-09-30T15:03:17.112 INFO:tasks.workunit.client.0.smithi032.stdout: 10 450747 34.16 MB/sec execute 590 sec latency 103.369 ms 2023-09-30T15:03:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.538+0000 7fbb092f9700 1 -- 172.21.15.32:0/4236913725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb04101340 msgr2=0x7fbb04101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.538+0000 7fbb092f9700 1 --2- 172.21.15.32:0/4236913725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb04101340 0x7fbb04101760 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009990 tx=0x7fbaf4009ca0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.538+0000 7fbb092f9700 1 -- 172.21.15.32:0/4236913725 shutdown_connections 2023-09-30T15:03:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.538+0000 7fbb092f9700 1 --2- 172.21.15.32:0/4236913725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb04100980 0x7fbb04108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.538+0000 7fbb092f9700 1 --2- 172.21.15.32:0/4236913725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb04101340 0x7fbb04101760 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.538+0000 7fbb092f9700 1 --2- 172.21.15.32:0/4236913725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb040fffe0 0x7fbb04100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.538+0000 7fbb092f9700 1 -- 172.21.15.32:0/4236913725 >> 172.21.15.32:0/4236913725 conn(0x7fbb040faa30 msgr2=0x7fbb040fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.538+0000 7fbb092f9700 1 -- 172.21.15.32:0/4236913725 shutdown_connections 2023-09-30T15:03:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.538+0000 7fbb092f9700 1 -- 172.21.15.32:0/4236913725 wait complete. 2023-09-30T15:03:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.538+0000 7fbb092f9700 1 Processor -- start 2023-09-30T15:03:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.539+0000 7fbb092f9700 1 -- start start 2023-09-30T15:03:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.539+0000 7fbb092f9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb040fffe0 0x7fbb04195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.539+0000 7fbb092f9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb04100980 0x7fbb041960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.539+0000 7fbb092f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb04101340 0x7fbb04196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.539+0000 7fbb092f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb041975a0 con 0x7fbb04101340 2023-09-30T15:03:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.539+0000 7fbb092f9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb04197710 con 0x7fbb04100980 2023-09-30T15:03:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.539+0000 7fbb092f9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb041978d0 con 0x7fbb040fffe0 2023-09-30T15:03:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.539+0000 7fbb02ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb040fffe0 0x7fbb04195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.539+0000 7fbb02ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb040fffe0 0x7fbb04195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54856/0 (socket says 172.21.15.32:54856) 2023-09-30T15:03:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.539+0000 7fbb02ffd700 1 -- 172.21.15.32:0/1927747659 learned_addr learned my addr 172.21.15.32:0/1927747659 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:03:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.539+0000 7fbb037fe700 1 --2- 172.21.15.32:0/1927747659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb04101340 0x7fbb04196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.540+0000 7fbb02ffd700 1 -- 172.21.15.32:0/1927747659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb04100980 msgr2=0x7fbb041960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.540+0000 7fbb02ffd700 1 --2- 172.21.15.32:0/1927747659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb04100980 0x7fbb041960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.540+0000 7fbb02ffd700 1 -- 172.21.15.32:0/1927747659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb04101340 msgr2=0x7fbb04196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.540+0000 7fbb02ffd700 1 --2- 172.21.15.32:0/1927747659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb04101340 0x7fbb04196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.540+0000 7fbb02ffd700 1 -- 172.21.15.32:0/1927747659 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf40095f0 con 0x7fbb040fffe0 2023-09-30T15:03:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.540+0000 7fbb037fe700 1 --2- 172.21.15.32:0/1927747659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb04101340 0x7fbb04196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:03:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.540+0000 7fbb02ffd700 1 --2- 172.21.15.32:0/1927747659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb040fffe0 0x7fbb04195b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4005720 tx=0x7fbaf4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.540+0000 7fbaebfff700 1 -- 172.21.15.32:0/1927747659 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf401d070 con 0x7fbb040fffe0 2023-09-30T15:03:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.540+0000 7fbaebfff700 1 -- 172.21.15.32:0/1927747659 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbaf4021410 con 0x7fbb040fffe0 2023-09-30T15:03:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.540+0000 7fbb092f9700 1 -- 172.21.15.32:0/1927747659 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb04197b50 con 0x7fbb040fffe0 2023-09-30T15:03:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.540+0000 7fbaebfff700 1 -- 172.21.15.32:0/1927747659 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4017650 con 0x7fbb040fffe0 2023-09-30T15:03:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.540+0000 7fbb092f9700 1 -- 172.21.15.32:0/1927747659 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0419e370 con 0x7fbb040fffe0 2023-09-30T15:03:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.543+0000 7fbaebfff700 1 -- 172.21.15.32:0/1927747659 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fbaf4021580 con 0x7fbb040fffe0 2023-09-30T15:03:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.544+0000 7fbb154e1b80 1 Processor -- start 2023-09-30T15:03:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.544+0000 7fbaebfff700 1 --2- 172.21.15.32:0/1927747659 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbaec0a0fa0 0x7fbaec0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.544+0000 7fbb154e1b80 1 -- start start 2023-09-30T15:03:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.544+0000 7fbb037fe700 1 --2- 172.21.15.32:0/1927747659 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbaec0a0fa0 0x7fbaec0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.544+0000 7fbaebfff700 1 -- 172.21.15.32:0/1927747659 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbaf40cd970 con 0x7fbb040fffe0 2023-09-30T15:03:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.544+0000 7fbb154e1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd88410 0x56456fd88830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.545+0000 7fbb154e1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd89520 0x56456fd899a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.545+0000 7fbb154e1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89ee0 0x56456fd90770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.545+0000 7fbb154e1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456fcee400 con 0x56456fd89ee0 2023-09-30T15:03:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.545+0000 7fbb154e1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456fcdfbc0 con 0x56456fd88410 2023-09-30T15:03:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.545+0000 7fbb154e1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456fcdd4e0 con 0x56456fd89520 2023-09-30T15:03:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.545+0000 7fbb037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89ee0 0x56456fd90770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.545+0000 7fbb037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89ee0 0x56456fd90770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40484/0 (socket says 172.21.15.32:40484) 2023-09-30T15:03:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.545+0000 7fbb037fe700 1 -- 172.21.15.32:0/2766092917 learned_addr learned my addr 172.21.15.32:0/2766092917 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:03:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.545+0000 7fbb02ffd700 1 --2- 172.21.15.32:0/2766092917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd88410 0x56456fd88830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.545+0000 7fbb037fe700 1 -- 172.21.15.32:0/2766092917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd89520 msgr2=0x56456fd899a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T15:03:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.545+0000 7fbb037fe700 1 --2- 172.21.15.32:0/2766092917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd89520 0x56456fd899a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.545+0000 7fbb037fe700 1 -- 172.21.15.32:0/2766092917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd88410 msgr2=0x56456fd88830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:03:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.545+0000 7fbb037fe700 1 --2- 172.21.15.32:0/2766092917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd88410 0x56456fd88830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.545+0000 7fbb037fe700 1 -- 172.21.15.32:0/2766092917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56456fd3b870 con 0x56456fd89ee0 2023-09-30T15:03:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.545+0000 7fbb037fe700 1 --2- 172.21.15.32:0/1927747659 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbaec0a0fa0 0x7fbaec0a3460 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbafc003fd0 tx=0x7fbafc003dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.545+0000 7fbb037fe700 1 --2- 172.21.15.32:0/2766092917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89ee0 0x56456fd90770 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00eb60 tx=0x7fbafc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a44f0929f443d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.546+0000 7fbae97fa700 1 -- 172.21.15.32:0/2766092917 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc009220 con 0x56456fd89ee0 2023-09-30T15:03:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.546+0000 7fbae97fa700 1 -- 172.21.15.32:0/2766092917 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbafc00f040 con 0x56456fd89ee0 2023-09-30T15:03:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.546+0000 7fbae97fa700 1 -- 172.21.15.32:0/2766092917 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0089f0 con 0x56456fd89ee0 2023-09-30T15:03:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.546+0000 7fbb154e1b80 1 -- 172.21.15.32:0/2766092917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89ee0 msgr2=0x56456fd90770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.546+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/2766092917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89ee0 0x56456fd90770 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00eb60 tx=0x7fbafc01d000 comp rx=0 tx=0).stop 2023-09-30T15:03:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.547+0000 7fbb154e1b80 1 -- 172.21.15.32:0/2766092917 shutdown_connections 2023-09-30T15:03:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.547+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/2766092917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89ee0 0x56456fd90770 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.547+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/2766092917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd88410 0x56456fd88830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.547+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/2766092917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd89520 0x56456fd899a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.547+0000 7fbb154e1b80 1 -- 172.21.15.32:0/2766092917 >> 172.21.15.32:0/2766092917 conn(0x56456fcfa200 msgr2=0x56456fd80ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.547+0000 7fbb154e1b80 1 -- 172.21.15.32:0/2766092917 shutdown_connections 2023-09-30T15:03:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.547+0000 7fbb154e1b80 1 -- 172.21.15.32:0/2766092917 wait complete. 2023-09-30T15:03:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.547+0000 7fbb154e1b80 1 -- start start 2023-09-30T15:03:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.547+0000 7fbb154e1b80 1 Processor -- start 2023-09-30T15:03:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.547+0000 7fbb154e1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd88410 0x56456fd8fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.547+0000 7fbb154e1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89520 0x56456fd90330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.547+0000 7fbb154e1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd89ee0 0x56456fd9b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.547+0000 7fbb154e1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456fcd8dd0 con 0x56456fd89520 2023-09-30T15:03:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.547+0000 7fbb154e1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456fc756b0 con 0x56456fd88410 2023-09-30T15:03:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.547+0000 7fbb154e1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456fc75350 con 0x56456fd89ee0 2023-09-30T15:03:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.548+0000 7fbb154e1b80 10 client.0 _ll_register_callbacks cb 0x56456fad27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:03:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.548+0000 7fbb027fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89520 0x56456fd90330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.548+0000 7fbb027fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89520 0x56456fd90330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40488/0 (socket says 172.21.15.32:40488) 2023-09-30T15:03:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.548+0000 7fbb02ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd88410 0x56456fd8fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.548+0000 7fbb027fc700 1 -- 172.21.15.32:0/4155909873 learned_addr learned my addr 172.21.15.32:0/4155909873 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:03:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.548+0000 7fbb037fe700 1 --2- 172.21.15.32:0/4155909873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd89ee0 0x56456fd9b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.548+0000 7fbb02ffd700 1 -- 172.21.15.32:0/4155909873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd89ee0 msgr2=0x56456fd9b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:03:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.548+0000 7fbb02ffd700 1 --2- 172.21.15.32:0/4155909873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd89ee0 0x56456fd9b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.548+0000 7fbb02ffd700 1 -- 172.21.15.32:0/4155909873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89520 msgr2=0x56456fd90330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:03:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.548+0000 7fbb02ffd700 1 --2- 172.21.15.32:0/4155909873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89520 0x56456fd90330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.548+0000 7fbb02ffd700 1 -- 172.21.15.32:0/4155909873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbafc0048c0 con 0x56456fd88410 2023-09-30T15:03:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.548+0000 7fbb027fc700 1 --2- 172.21.15.32:0/4155909873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89520 0x56456fd90330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:03:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.548+0000 7fbb02ffd700 1 --2- 172.21.15.32:0/4155909873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd88410 0x56456fd8fdf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400b2e0 tx=0x7fbaf409a950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a15fea84babe4a85 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.548+0000 7fbae6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-09-30T15:03:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.548+0000 7fbae6ff5700 1 -- 172.21.15.32:0/4155909873 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf409b450 con 0x56456fd88410 2023-09-30T15:03:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.549+0000 7fbae6ff5700 1 -- 172.21.15.32:0/4155909873 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbaf409aff0 con 0x56456fd88410 2023-09-30T15:03:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.550+0000 7fbb154e1b80 1 -- 172.21.15.32:0/4155909873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56456fd79090 con 0x56456fd88410 2023-09-30T15:03:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.550+0000 7fbae6ff5700 1 -- 172.21.15.32:0/4155909873 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaf409f140 con 0x56456fd88410 2023-09-30T15:03:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.550+0000 7fbb154e1b80 10 client.27062 fetch_fsmap learned FSMap version 57 2023-09-30T15:03:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.550+0000 7fbb154e1b80 1 -- 172.21.15.32:0/4155909873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x56456fb03d80 con 0x56456fd88410 2023-09-30T15:03:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.550+0000 7fbae6ff5700 1 -- 172.21.15.32:0/4155909873 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56456fd79090 con 0x56456fd88410 2023-09-30T15:03:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.550+0000 7fbae6ff5700 1 -- 172.21.15.32:0/4155909873 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 57) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbaf40a2ed0 con 0x56456fd88410 2023-09-30T15:03:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.550+0000 7fbb154e1b80 10 client.27062 fetch_fsmap finished waiting for FSMap version 57 2023-09-30T15:03:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.550+0000 7fbb154e1b80 10 client.27062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:03:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.550+0000 7fbb154e1b80 20 client.27062 populate_metadata read hostname 'smithi032' 2023-09-30T15:03:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.550+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/4155909873 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56456fd9fc80 0x56456fda20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.550+0000 7fbb154e1b80 4 client.27062 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-09-30T15:03:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.550+0000 7fbb154e1b80 1 -- 172.21.15.32:0/4155909873 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56456fcd1db0 con 0x56456fd9fc80 2023-09-30T15:03:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.551+0000 7fbb027fc700 1 --2- 172.21.15.32:0/4155909873 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56456fd9fc80 0x56456fda20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.551+0000 7fbae6ff5700 10 client.27062 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:03:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.551+0000 7fbb027fc700 1 --2- 172.21.15.32:0/4155909873 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56456fd9fc80 0x56456fda20a0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d539c7f4340db17 server_cookie=26d722bd50f76cc5 in_seq=0 out_seq=0 2023-09-30T15:03:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.552+0000 7fbae6ff5700 1 -- 172.21.15.32:0/4155909873 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56456fcd1db0 con 0x56456fd9fc80 2023-09-30T15:03:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.552+0000 7fbae6ff5700 10 client.27062 handle_command_reply: tid=0 2023-09-30T15:03:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.552+0000 7fbb154e1b80 1 client.27062 shutdown 2023-09-30T15:03:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.552+0000 7fbb154e1b80 1 -- 172.21.15.32:0/4155909873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd88410 msgr2=0x56456fd8fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.552+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/4155909873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd88410 0x56456fd8fdf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400b2e0 tx=0x7fbaf409a950 comp rx=0 tx=0).stop 2023-09-30T15:03:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.553+0000 7fbb154e1b80 1 -- 172.21.15.32:0/4155909873 shutdown_connections 2023-09-30T15:03:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.553+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/4155909873 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56456fd9fc80 0x56456fda20a0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.553+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/4155909873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd89ee0 0x56456fd9b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.553+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/4155909873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd88410 0x56456fd8fdf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.553+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/4155909873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89520 0x56456fd90330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.553+0000 7fbb154e1b80 1 -- 172.21.15.32:0/4155909873 >> 172.21.15.32:0/4155909873 conn(0x56456fcfa200 msgr2=0x56456fd80590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.553+0000 7fbae6ff5700 0 client.27062 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:03:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.553+0000 7fbb154e1b80 1 -- 172.21.15.32:0/4155909873 shutdown_connections 2023-09-30T15:03:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.553+0000 7fbb154e1b80 1 -- 172.21.15.32:0/4155909873 wait complete. 2023-09-30T15:03:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.553+0000 7fbb154e1b80 20 client.27062 trim_cache size 0 max 16384 2023-09-30T15:03:17.920 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:17 smithi148 ceph-mon[102355]: pgmap v1037: 97 pgs: 97 active+clean; 808 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.6 MiB/s wr, 693 op/s 2023-09-30T15:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.571+0000 7fbb154e1b80 1 Processor -- start 2023-09-30T15:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.571+0000 7fbb154e1b80 1 -- start start 2023-09-30T15:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb154e1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd88410 0x56456fd8ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb154e1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89520 0x56456fd8b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb154e1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd89ee0 0x56456fd9b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb154e1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456fb598c0 con 0x56456fd89520 2023-09-30T15:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb154e1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456fd8f960 con 0x56456fd89ee0 2023-09-30T15:03:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb154e1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456fd9c770 con 0x56456fd88410 2023-09-30T15:03:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb027fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89520 0x56456fd8b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb02ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd88410 0x56456fd8ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb027fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89520 0x56456fd8b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40496/0 (socket says 172.21.15.32:40496) 2023-09-30T15:03:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb027fc700 1 -- 172.21.15.32:0/3672263910 learned_addr learned my addr 172.21.15.32:0/3672263910 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:03:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb037fe700 1 --2- 172.21.15.32:0/3672263910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd89ee0 0x56456fd9b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb027fc700 1 -- 172.21.15.32:0/3672263910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd88410 msgr2=0x56456fd8ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:03:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb027fc700 1 --2- 172.21.15.32:0/3672263910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd88410 0x56456fd8ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb027fc700 1 -- 172.21.15.32:0/3672263910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd89ee0 msgr2=0x56456fd9b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:03:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb027fc700 1 --2- 172.21.15.32:0/3672263910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd89ee0 0x56456fd9b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb027fc700 1 -- 172.21.15.32:0/3672263910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56456faecc00 con 0x56456fd89520 2023-09-30T15:03:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.572+0000 7fbb027fc700 1 --2- 172.21.15.32:0/3672263910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89520 0x56456fd8b2c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8005940 tx=0x7fbaf80097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce032041266bf998 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.573+0000 7fbae9ffb700 1 -- 172.21.15.32:0/3672263910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8019520 con 0x56456fd89520 2023-09-30T15:03:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.573+0000 7fbae9ffb700 1 -- 172.21.15.32:0/3672263910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbaf8004050 con 0x56456fd89520 2023-09-30T15:03:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.573+0000 7fbae9ffb700 1 -- 172.21.15.32:0/3672263910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8019c80 con 0x56456fd89520 2023-09-30T15:03:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.573+0000 7fbb154e1b80 1 -- 172.21.15.32:0/3672263910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89520 msgr2=0x56456fd8b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.573+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/3672263910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89520 0x56456fd8b2c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8005940 tx=0x7fbaf80097b0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.573+0000 7fbb154e1b80 1 -- 172.21.15.32:0/3672263910 shutdown_connections 2023-09-30T15:03:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.573+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/3672263910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd89ee0 0x56456fd9b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.573+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/3672263910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd88410 0x56456fd8ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.573+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/3672263910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89520 0x56456fd8b2c0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.573+0000 7fbb154e1b80 1 -- 172.21.15.32:0/3672263910 >> 172.21.15.32:0/3672263910 conn(0x56456fcfa200 msgr2=0x56456fd9c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.573+0000 7fbb154e1b80 1 -- 172.21.15.32:0/3672263910 shutdown_connections 2023-09-30T15:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.574+0000 7fbb154e1b80 1 -- 172.21.15.32:0/3672263910 wait complete. 2023-09-30T15:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.574+0000 7fbb154e1b80 1 -- start start 2023-09-30T15:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.574+0000 7fbb154e1b80 1 Processor -- start 2023-09-30T15:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.574+0000 7fbb154e1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd88410 0x56456fd98e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.574+0000 7fbb154e1b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd89520 0x56456fd99340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.574+0000 7fbb154e1b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89ee0 0x56456fd9a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.574+0000 7fbb154e1b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456fd9a890 con 0x56456fd89ee0 2023-09-30T15:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.574+0000 7fbb154e1b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456fd9aa00 con 0x56456fd89520 2023-09-30T15:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.574+0000 7fbb154e1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456fd9abc0 con 0x56456fd88410 2023-09-30T15:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.574+0000 7fbb154e1b80 10 client.0 _ll_register_callbacks cb 0x7fbaf409adc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T15:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.574+0000 7fbb037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89ee0 0x56456fd9a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.574+0000 7fbb02ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd88410 0x56456fd98e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.574+0000 7fbb02ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd88410 0x56456fd98e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.32:54882/0 (socket says 172.21.15.32:54882) 2023-09-30T15:03:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.574+0000 7fbb02ffd700 1 -- 172.21.15.32:0/1789881750 learned_addr learned my addr 172.21.15.32:0/1789881750 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:03:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.574+0000 7fbb027fc700 1 --2- 172.21.15.32:0/1789881750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd89520 0x56456fd99340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.574+0000 7fbb037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89ee0 0x56456fd9a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40504/0 (socket says 172.21.15.32:40504) 2023-09-30T15:03:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.575+0000 7fbb02ffd700 1 -- 172.21.15.32:0/1789881750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd89520 msgr2=0x56456fd99340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:03:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.575+0000 7fbb02ffd700 1 --2- 172.21.15.32:0/1789881750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd89520 0x56456fd99340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.575+0000 7fbb02ffd700 1 -- 172.21.15.32:0/1789881750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89ee0 msgr2=0x56456fd9a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T15:03:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.575+0000 7fbb02ffd700 1 --2- 172.21.15.32:0/1789881750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89ee0 0x56456fd9a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.575+0000 7fbb02ffd700 1 -- 172.21.15.32:0/1789881750 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf8009e70 con 0x56456fd88410 2023-09-30T15:03:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.575+0000 7fbb027fc700 1 --2- 172.21.15.32:0/1789881750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd89520 0x56456fd99340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:03:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.575+0000 7fbb02ffd700 1 --2- 172.21.15.32:0/1789881750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd88410 0x56456fd98e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4097a10 tx=0x7fbaf400b370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a5281b03437dfb5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.575+0000 7fbae6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-09-30T15:03:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.575+0000 7fbae6ff5700 1 -- 172.21.15.32:0/1789881750 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4009290 con 0x56456fd88410 2023-09-30T15:03:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.575+0000 7fbb154e1b80 1 -- 172.21.15.32:0/1789881750 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbafc00f040 con 0x56456fd88410 2023-09-30T15:03:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.575+0000 7fbae6ff5700 1 -- 172.21.15.32:0/1789881750 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbaf409aff0 con 0x56456fd88410 2023-09-30T15:03:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.575+0000 7fbae6ff5700 1 -- 172.21.15.32:0/1789881750 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaf409f140 con 0x56456fd88410 2023-09-30T15:03:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.575+0000 7fbb154e1b80 10 client.27133 fetch_fsmap learned FSMap version 57 2023-09-30T15:03:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.575+0000 7fbb154e1b80 1 -- 172.21.15.32:0/1789881750 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x56456fb2a1f0 con 0x56456fd88410 2023-09-30T15:03:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.576+0000 7fbae6ff5700 1 -- 172.21.15.32:0/1789881750 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbaf40a7020 con 0x56456fd88410 2023-09-30T15:03:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.576+0000 7fbae6ff5700 1 -- 172.21.15.32:0/1789881750 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 57) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbaf400fac0 con 0x56456fd88410 2023-09-30T15:03:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.576+0000 7fbb154e1b80 10 client.27133 fetch_fsmap finished waiting for FSMap version 57 2023-09-30T15:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.576+0000 7fbb154e1b80 10 client.27133 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-09-30T15:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.576+0000 7fbb154e1b80 20 client.27133 populate_metadata read hostname 'smithi032' 2023-09-30T15:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.576+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/1789881750 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56456fdf3e10 0x56456fdf6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.576+0000 7fbb154e1b80 4 client.27133 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-09-30T15:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.576+0000 7fbb154e1b80 1 -- 172.21.15.32:0/1789881750 --> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56456fcd1ad0 con 0x56456fdf3e10 2023-09-30T15:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.576+0000 7fbb027fc700 1 --2- 172.21.15.32:0/1789881750 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56456fdf3e10 0x56456fdf6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.576+0000 7fbae6ff5700 10 client.27133 ms_handle_connect on v2:172.21.15.148:6838/494821686 2023-09-30T15:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:17.576+0000 7fbb027fc700 1 --2- 172.21.15.32:0/1789881750 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56456fdf3e10 0x56456fdf6200 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72ac8a00f0802529 server_cookie=b624d6f600a8cd22 in_seq=0 out_seq=0 2023-09-30T15:03:17.931 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:17 smithi032 ceph-mon[98592]: pgmap v1037: 97 pgs: 97 active+clean; 808 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.6 MiB/s wr, 693 op/s 2023-09-30T15:03:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.036+0000 7fbae6ff5700 1 -- 172.21.15.32:0/1789881750 <== mds.0 v2:172.21.15.148:6838/494821686 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56456fcd1ad0 con 0x56456fdf3e10 2023-09-30T15:03:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.036+0000 7fbae6ff5700 10 client.27133 handle_command_reply: tid=0 2023-09-30T15:03:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.036+0000 7fbb154e1b80 1 client.27133 shutdown 2023-09-30T15:03:18.077 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:17 smithi156 ceph-mon[103579]: pgmap v1037: 97 pgs: 97 active+clean; 808 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.6 MiB/s wr, 693 op/s 2023-09-30T15:03:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.045+0000 7fbb154e1b80 1 -- 172.21.15.32:0/1789881750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd88410 msgr2=0x56456fd98e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.045+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/1789881750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd88410 0x56456fd98e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4097a10 tx=0x7fbaf400b370 comp rx=0 tx=0).stop 2023-09-30T15:03:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.045+0000 7fbb154e1b80 1 -- 172.21.15.32:0/1789881750 shutdown_connections 2023-09-30T15:03:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.045+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/1789881750 >> [v2:172.21.15.148:6838/494821686,v1:172.21.15.148:6839/494821686] conn(0x56456fdf3e10 0x56456fdf6200 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.045+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/1789881750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56456fd89ee0 0x56456fd9a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.045+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/1789881750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56456fd88410 0x56456fd98e00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.045+0000 7fbb154e1b80 1 --2- 172.21.15.32:0/1789881750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56456fd89520 0x56456fd99340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.045+0000 7fbb154e1b80 1 -- 172.21.15.32:0/1789881750 >> 172.21.15.32:0/1789881750 conn(0x56456fcfa200 msgr2=0x56456fd8ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.045+0000 7fbae6ff5700 0 client.27133 ms_handle_reset on v2:172.21.15.148:6838/494821686 2023-09-30T15:03:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.045+0000 7fbb154e1b80 1 -- 172.21.15.32:0/1789881750 shutdown_connections 2023-09-30T15:03:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.048+0000 7fbb154e1b80 1 -- 172.21.15.32:0/1789881750 wait complete. 2023-09-30T15:03:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.049+0000 7fbb154e1b80 20 client.27133 trim_cache size 0 max 16384 2023-09-30T15:03:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:{ 2023-09-30T15:03:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "message": "", 2023-09-30T15:03:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout: "return_code": 0 2023-09-30T15:03:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stdout:} 2023-09-30T15:03:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.049+0000 7fbb092f9700 1 -- 172.21.15.32:0/1927747659 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbaec0a0fa0 msgr2=0x7fbaec0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.049+0000 7fbb092f9700 1 --2- 172.21.15.32:0/1927747659 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbaec0a0fa0 0x7fbaec0a3460 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbafc003fd0 tx=0x7fbafc003dc0 comp rx=0 tx=0).stop 2023-09-30T15:03:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.049+0000 7fbb092f9700 1 -- 172.21.15.32:0/1927747659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb040fffe0 msgr2=0x7fbb04195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.049+0000 7fbb092f9700 1 --2- 172.21.15.32:0/1927747659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb040fffe0 0x7fbb04195b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4005720 tx=0x7fbaf4009580 comp rx=0 tx=0).stop 2023-09-30T15:03:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.050+0000 7fbb092f9700 1 -- 172.21.15.32:0/1927747659 shutdown_connections 2023-09-30T15:03:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.050+0000 7fbb092f9700 1 --2- 172.21.15.32:0/1927747659 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fbaec0a0fa0 0x7fbaec0a3460 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.050+0000 7fbb092f9700 1 --2- 172.21.15.32:0/1927747659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb04101340 0x7fbb04196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.050+0000 7fbb092f9700 1 --2- 172.21.15.32:0/1927747659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb040fffe0 0x7fbb04195b80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.050+0000 7fbb092f9700 1 --2- 172.21.15.32:0/1927747659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb04100980 0x7fbb041960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.050+0000 7fbb092f9700 1 -- 172.21.15.32:0/1927747659 >> 172.21.15.32:0/1927747659 conn(0x7fbb040faa30 msgr2=0x7fbb04104550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.050+0000 7fbb092f9700 1 -- 172.21.15.32:0/1927747659 shutdown_connections 2023-09-30T15:03:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi032.front.sepia.ceph.com.smithi032.stderr:2023-09-30T15:03:18.050+0000 7fbb092f9700 1 -- 172.21.15.32:0/1927747659 wait complete. 2023-09-30T15:03:18.112 INFO:tasks.workunit.client.0.smithi032.stdout: 10 451205 34.15 MB/sec execute 591 sec latency 302.546 ms 2023-09-30T15:03:19.112 INFO:tasks.workunit.client.0.smithi032.stdout: 10 451732 34.13 MB/sec execute 592 sec latency 358.381 ms 2023-09-30T15:03:19.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:19 smithi148 ceph-mon[102355]: pgmap v1038: 97 pgs: 97 active+clean; 803 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.8 MiB/s wr, 588 op/s 2023-09-30T15:03:19.926 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:19 smithi032 ceph-mon[98592]: pgmap v1038: 97 pgs: 97 active+clean; 803 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.8 MiB/s wr, 588 op/s 2023-09-30T15:03:20.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:19 smithi156 ceph-mon[103579]: pgmap v1038: 97 pgs: 97 active+clean; 803 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.8 MiB/s wr, 588 op/s 2023-09-30T15:03:20.112 INFO:tasks.workunit.client.0.smithi032.stdout: 10 452503 34.15 MB/sec execute 593 sec latency 63.277 ms 2023-09-30T15:03:21.113 INFO:tasks.workunit.client.0.smithi032.stdout: 10 453010 34.13 MB/sec execute 594 sec latency 323.934 ms 2023-09-30T15:03:21.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:21 smithi032 ceph-mon[98592]: pgmap v1039: 97 pgs: 97 active+clean; 795 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.1 MiB/s wr, 709 op/s 2023-09-30T15:03:22.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:21 smithi156 ceph-mon[103579]: pgmap v1039: 97 pgs: 97 active+clean; 795 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.1 MiB/s wr, 709 op/s 2023-09-30T15:03:22.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:21 smithi148 ceph-mon[102355]: pgmap v1039: 97 pgs: 97 active+clean; 795 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.1 MiB/s wr, 709 op/s 2023-09-30T15:03:22.113 INFO:tasks.workunit.client.0.smithi032.stdout: 10 453525 34.13 MB/sec execute 595 sec latency 72.667 ms 2023-09-30T15:03:23.113 INFO:tasks.workunit.client.0.smithi032.stdout: 10 454230 34.13 MB/sec execute 596 sec latency 73.758 ms 2023-09-30T15:03:23.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:23 smithi032 ceph-mon[98592]: pgmap v1040: 97 pgs: 97 active+clean; 792 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.1 MiB/s wr, 666 op/s 2023-09-30T15:03:24.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:23 smithi156 ceph-mon[103579]: pgmap v1040: 97 pgs: 97 active+clean; 792 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.1 MiB/s wr, 666 op/s 2023-09-30T15:03:24.275 INFO:tasks.workunit.client.0.smithi032.stdout: 10 454868 34.13 MB/sec execute 597 sec latency 76.937 ms 2023-09-30T15:03:24.275 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:23 smithi148 ceph-mon[102355]: pgmap v1040: 97 pgs: 97 active+clean; 792 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.1 MiB/s wr, 666 op/s 2023-09-30T15:03:25.113 INFO:tasks.workunit.client.0.smithi032.stdout: 10 455579 34.14 MB/sec execute 598 sec latency 63.955 ms 2023-09-30T15:03:25.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:25 smithi032 ceph-mon[98592]: pgmap v1041: 97 pgs: 97 active+clean; 805 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.5 MiB/s wr, 675 op/s 2023-09-30T15:03:26.059 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:25 smithi156 ceph-mon[103579]: pgmap v1041: 97 pgs: 97 active+clean; 805 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.5 MiB/s wr, 675 op/s 2023-09-30T15:03:26.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:25 smithi148 ceph-mon[102355]: pgmap v1041: 97 pgs: 97 active+clean; 805 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.5 MiB/s wr, 675 op/s 2023-09-30T15:03:26.113 INFO:tasks.workunit.client.0.smithi032.stdout: 10 455995 34.13 MB/sec execute 599 sec latency 325.910 ms 2023-09-30T15:03:27.114 INFO:tasks.workunit.client.0.smithi032.stdout: 10 cleanup 600 sec 2023-09-30T15:03:27.935 INFO:tasks.workunit.client.0.smithi032.stdout: 0 cleanup 601 sec 2023-09-30T15:03:27.935 INFO:tasks.workunit.client.0.smithi032.stdout: 2023-09-30T15:03:27.935 INFO:tasks.workunit.client.0.smithi032.stdout: Operation Count AvgLat MaxLat 2023-09-30T15:03:27.935 INFO:tasks.workunit.client.0.smithi032.stdout: ---------------------------------------- 2023-09-30T15:03:27.935 INFO:tasks.workunit.client.0.smithi032.stdout: NTCreateX 653901 3.661 552.338 2023-09-30T15:03:27.935 INFO:tasks.workunit.client.0.smithi032.stdout: Close 480277 0.002 0.148 2023-09-30T15:03:27.935 INFO:tasks.workunit.client.0.smithi032.stdout: Rename 27695 16.585 527.079 2023-09-30T15:03:27.935 INFO:tasks.workunit.client.0.smithi032.stdout: Unlink 132067 4.665 341.317 2023-09-30T15:03:27.935 INFO:tasks.workunit.client.0.smithi032.stdout: Deltree 10 161.056 272.376 2023-09-30T15:03:27.936 INFO:tasks.workunit.client.0.smithi032.stdout: Mkdir 5 0.008 0.014 2023-09-30T15:03:27.936 INFO:tasks.workunit.client.0.smithi032.stdout: Qpathinfo 592759 2.986 551.463 2023-09-30T15:03:27.936 INFO:tasks.workunit.client.0.smithi032.stdout: Qfileinfo 103660 0.001 0.036 2023-09-30T15:03:27.936 INFO:tasks.workunit.client.0.smithi032.stdout: Qfsinfo 108692 0.708 61.890 2023-09-30T15:03:27.936 INFO:tasks.workunit.client.0.smithi032.stdout: Sfileinfo 53261 0.082 74.768 2023-09-30T15:03:27.936 INFO:tasks.workunit.client.0.smithi032.stdout: Find 229118 0.274 194.508 2023-09-30T15:03:27.936 INFO:tasks.workunit.client.0.smithi032.stdout: WriteX 324815 0.035 355.618 2023-09-30T15:03:27.936 INFO:tasks.workunit.client.0.smithi032.stdout: ReadX 1025177 0.027 244.816 2023-09-30T15:03:27.936 INFO:tasks.workunit.client.0.smithi032.stdout: LockX 2128 3.669 170.841 2023-09-30T15:03:27.936 INFO:tasks.workunit.client.0.smithi032.stdout: UnlockX 2128 3.142 220.577 2023-09-30T15:03:27.936 INFO:tasks.workunit.client.0.smithi032.stdout: Flush 45829 12.062 397.341 2023-09-30T15:03:27.936 INFO:tasks.workunit.client.0.smithi032.stdout: 2023-09-30T15:03:27.936 INFO:tasks.workunit.client.0.smithi032.stdout:Throughput 34.1277 MB/sec 10 clients 10 procs max_latency=552.342 ms 2023-09-30T15:03:27.937 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-09-30T15:03:27.937 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-09-30T15:03:27.977 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:27 smithi032 ceph-mon[98592]: pgmap v1042: 97 pgs: 97 active+clean; 814 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 10 MiB/s wr, 700 op/s 2023-09-30T15:03:28.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:27 smithi156 ceph-mon[103579]: pgmap v1042: 97 pgs: 97 active+clean; 814 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 10 MiB/s wr, 700 op/s 2023-09-30T15:03:28.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:27 smithi148 ceph-mon[102355]: pgmap v1042: 97 pgs: 97 active+clean; 814 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 10 MiB/s wr, 700 op/s 2023-09-30T15:03:29.370 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2023-09-30T15:03:29.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-09-30T15:03:29.713 DEBUG:teuthology.parallel:result is None 2023-09-30T15:03:29.713 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-09-30T15:03:29.747 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-09-30T15:03:29.747 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-09-30T15:03:29.759 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T15:03:29.795 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-09-30T15:03:29.805 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-09-30T15:03:29.805 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_1d10b71792f3be8887a7631e69851ac2df3585af/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_ceph_ceph-c_1d10b71792f3be8887a7631e69851ac2df3585af/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-09-30T15:03:29.806 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-09-30T15:03:29.814 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T15:03:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.960+0000 7f7e49d19700 1 -- 172.21.15.32:0/1306000302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e44102960 msgr2=0x7f7e44109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.960+0000 7f7e49d19700 1 --2- 172.21.15.32:0/1306000302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e44102960 0x7f7e44109280 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00b0a0 tx=0x7f7e3c00b3b0 comp rx=0 tx=0).stop 2023-09-30T15:03:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.960+0000 7f7e49d19700 1 -- 172.21.15.32:0/1306000302 shutdown_connections 2023-09-30T15:03:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.960+0000 7f7e49d19700 1 --2- 172.21.15.32:0/1306000302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e44102960 0x7f7e44109280 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.960+0000 7f7e49d19700 1 --2- 172.21.15.32:0/1306000302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e44100e50 0x7f7e44101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.960+0000 7f7e49d19700 1 --2- 172.21.15.32:0/1306000302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e44101fa0 0x7f7e44102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.960+0000 7f7e49d19700 1 -- 172.21.15.32:0/1306000302 >> 172.21.15.32:0/1306000302 conn(0x7f7e440fc3a0 msgr2=0x7f7e440fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.960+0000 7f7e49d19700 1 -- 172.21.15.32:0/1306000302 shutdown_connections 2023-09-30T15:03:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.960+0000 7f7e49d19700 1 -- 172.21.15.32:0/1306000302 wait complete. 2023-09-30T15:03:29.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.961+0000 7f7e49d19700 1 Processor -- start 2023-09-30T15:03:29.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.961+0000 7f7e49d19700 1 -- start start 2023-09-30T15:03:29.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.961+0000 7f7e49d19700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e44100e50 0x7f7e44196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:29.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.961+0000 7f7e49d19700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e44101fa0 0x7f7e44197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:29.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.961+0000 7f7e49d19700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e44102960 0x7f7e4419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:29.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.961+0000 7f7e49d19700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4419e740 con 0x7f7e44100e50 2023-09-30T15:03:29.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.961+0000 7f7e49d19700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4419e880 con 0x7f7e44102960 2023-09-30T15:03:29.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e49d19700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4419ea10 con 0x7f7e44101fa0 2023-09-30T15:03:29.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e437fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e44100e50 0x7f7e44196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:29.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e437fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e44100e50 0x7f7e44196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40526/0 (socket says 172.21.15.32:40526) 2023-09-30T15:03:29.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e437fe700 1 -- 172.21.15.32:0/432705270 learned_addr learned my addr 172.21.15.32:0/432705270 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:03:29.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e43fff700 1 --2- 172.21.15.32:0/432705270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e44102960 0x7f7e4419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:29.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e42ffd700 1 --2- 172.21.15.32:0/432705270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e44101fa0 0x7f7e44197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:29.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e437fe700 1 -- 172.21.15.32:0/432705270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e44101fa0 msgr2=0x7f7e44197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:29.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e437fe700 1 --2- 172.21.15.32:0/432705270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e44101fa0 0x7f7e44197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:29.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e437fe700 1 -- 172.21.15.32:0/432705270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e44102960 msgr2=0x7f7e4419e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:29.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e437fe700 1 --2- 172.21.15.32:0/432705270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e44102960 0x7f7e4419e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:29.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e437fe700 1 -- 172.21.15.32:0/432705270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c009d00 con 0x7f7e44100e50 2023-09-30T15:03:29.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e43fff700 1 --2- 172.21.15.32:0/432705270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e44102960 0x7f7e4419e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:03:29.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e437fe700 1 --2- 172.21.15.32:0/432705270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e44100e50 0x7f7e44196bc0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400e9a0 tx=0x7f7e3400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:29.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e40ff9700 1 -- 172.21.15.32:0/432705270 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e34009790 con 0x7f7e44100e50 2023-09-30T15:03:29.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e40ff9700 1 -- 172.21.15.32:0/432705270 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e3400cbd0 con 0x7f7e44100e50 2023-09-30T15:03:29.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e49d19700 1 -- 172.21.15.32:0/432705270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e4419ecf0 con 0x7f7e44100e50 2023-09-30T15:03:29.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e40ff9700 1 -- 172.21.15.32:0/432705270 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e34018680 con 0x7f7e44100e50 2023-09-30T15:03:29.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.962+0000 7f7e49d19700 1 -- 172.21.15.32:0/432705270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4419f1a0 con 0x7f7e44100e50 2023-09-30T15:03:29.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.963+0000 7f7e49d19700 1 -- 172.21.15.32:0/432705270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e44108390 con 0x7f7e44100e50 2023-09-30T15:03:29.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.965+0000 7f7e40ff9700 1 -- 172.21.15.32:0/432705270 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f7e3400cd40 con 0x7f7e44100e50 2023-09-30T15:03:29.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.965+0000 7f7e40ff9700 1 --2- 172.21.15.32:0/432705270 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7e2c0a1070 0x7f7e2c0a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:29.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.966+0000 7f7e42ffd700 1 --2- 172.21.15.32:0/432705270 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7e2c0a1070 0x7f7e2c0a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:29.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.966+0000 7f7e40ff9700 1 -- 172.21.15.32:0/432705270 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7e34014070 con 0x7f7e44100e50 2023-09-30T15:03:29.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.966+0000 7f7e42ffd700 1 --2- 172.21.15.32:0/432705270 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7e2c0a1070 0x7f7e2c0a3530 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009bd0 tx=0x7f7e38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:29.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:29.968+0000 7f7e40ff9700 1 -- 172.21.15.32:0/432705270 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f7e34096410 con 0x7f7e44100e50 2023-09-30T15:03:30.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:29 smithi156 ceph-mon[103579]: pgmap v1043: 97 pgs: 97 active+clean; 816 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.6 MiB/s wr, 590 op/s 2023-09-30T15:03:30.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:29 smithi148 ceph-mon[102355]: pgmap v1043: 97 pgs: 97 active+clean; 816 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.6 MiB/s wr, 590 op/s 2023-09-30T15:03:30.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:29 smithi032 ceph-mon[98592]: pgmap v1043: 97 pgs: 97 active+clean; 816 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.6 MiB/s wr, 590 op/s 2023-09-30T15:03:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.170+0000 7f7e49d19700 1 -- 172.21.15.32:0/432705270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e4410c6c0 con 0x7f7e44100e50 2023-09-30T15:03:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.172+0000 7f7e40ff9700 1 -- 172.21.15.32:0/432705270 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22569 (secure 0 0 0) 0x7f7e34099a30 con 0x7f7e44100e50 2023-09-30T15:03:30.173 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T15:03:30.173 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":85,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:38:13.192089+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T15:03:30.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.174+0000 7f7e49d19700 1 -- 172.21.15.32:0/432705270 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7e2c0a1070 msgr2=0x7f7e2c0a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:30.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.174+0000 7f7e49d19700 1 --2- 172.21.15.32:0/432705270 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7e2c0a1070 0x7f7e2c0a3530 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009bd0 tx=0x7f7e38009260 comp rx=0 tx=0).stop 2023-09-30T15:03:30.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.174+0000 7f7e49d19700 1 -- 172.21.15.32:0/432705270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e44100e50 msgr2=0x7f7e44196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:30.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.174+0000 7f7e49d19700 1 --2- 172.21.15.32:0/432705270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e44100e50 0x7f7e44196bc0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400e9a0 tx=0x7f7e3400ecb0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.174+0000 7f7e49d19700 1 -- 172.21.15.32:0/432705270 shutdown_connections 2023-09-30T15:03:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.174+0000 7f7e49d19700 1 --2- 172.21.15.32:0/432705270 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f7e2c0a1070 0x7f7e2c0a3530 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.174+0000 7f7e49d19700 1 --2- 172.21.15.32:0/432705270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e44102960 0x7f7e4419e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.174+0000 7f7e49d19700 1 --2- 172.21.15.32:0/432705270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e44100e50 0x7f7e44196bc0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.174+0000 7f7e49d19700 1 --2- 172.21.15.32:0/432705270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e44101fa0 0x7f7e44197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.174+0000 7f7e49d19700 1 -- 172.21.15.32:0/432705270 >> 172.21.15.32:0/432705270 conn(0x7f7e440fc3a0 msgr2=0x7f7e441054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.174+0000 7f7e49d19700 1 -- 172.21.15.32:0/432705270 shutdown_connections 2023-09-30T15:03:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.174+0000 7f7e49d19700 1 -- 172.21.15.32:0/432705270 wait complete. 2023-09-30T15:03:30.188 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T14:33:20.224529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T14:37:09.573219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T14:37:10.844458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T15:03:30.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T15:03:30.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.316+0000 7fc26e5d5700 1 -- 172.21.15.32:0/3631135113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2681018f0 msgr2=0x7fc2681081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.316+0000 7fc26e5d5700 1 --2- 172.21.15.32:0/3631135113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2681018f0 0x7fc2681081a0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc26000b0d0 tx=0x7fc26000b3e0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.316+0000 7fc26e5d5700 1 -- 172.21.15.32:0/3631135113 shutdown_connections 2023-09-30T15:03:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.316+0000 7fc26e5d5700 1 --2- 172.21.15.32:0/3631135113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2681018f0 0x7fc2681081a0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.316+0000 7fc26e5d5700 1 --2- 172.21.15.32:0/3631135113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2680ffd60 0x7fc268100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.316+0000 7fc26e5d5700 1 --2- 172.21.15.32:0/3631135113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc268100f50 0x7fc2681013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.316+0000 7fc26e5d5700 1 -- 172.21.15.32:0/3631135113 >> 172.21.15.32:0/3631135113 conn(0x7fc2680fb2f0 msgr2=0x7fc2680fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.316+0000 7fc26e5d5700 1 -- 172.21.15.32:0/3631135113 shutdown_connections 2023-09-30T15:03:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.316+0000 7fc26e5d5700 1 -- 172.21.15.32:0/3631135113 wait complete. 2023-09-30T15:03:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.316+0000 7fc26e5d5700 1 Processor -- start 2023-09-30T15:03:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.316+0000 7fc26e5d5700 1 -- start start 2023-09-30T15:03:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc26e5d5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2680ffd60 0x7fc268196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc26e5d5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc268100f50 0x7fc268197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc26e5d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2681018f0 0x7fc26819e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc26e5d5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26819e750 con 0x7fc2681018f0 2023-09-30T15:03:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc26e5d5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26819e890 con 0x7fc2680ffd60 2023-09-30T15:03:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc26e5d5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26819ea20 con 0x7fc268100f50 2023-09-30T15:03:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc26cb72700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2681018f0 0x7fc26819e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc26cb72700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2681018f0 0x7fc26819e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40544/0 (socket says 172.21.15.32:40544) 2023-09-30T15:03:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc26cb72700 1 -- 172.21.15.32:0/1032623404 learned_addr learned my addr 172.21.15.32:0/1032623404 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:03:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc267fff700 1 --2- 172.21.15.32:0/1032623404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2680ffd60 0x7fc268196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc2677fe700 1 --2- 172.21.15.32:0/1032623404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc268100f50 0x7fc268197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc26cb72700 1 -- 172.21.15.32:0/1032623404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc268100f50 msgr2=0x7fc268197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc26cb72700 1 --2- 172.21.15.32:0/1032623404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc268100f50 0x7fc268197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc26cb72700 1 -- 172.21.15.32:0/1032623404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2680ffd60 msgr2=0x7fc268196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:30.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc26cb72700 1 --2- 172.21.15.32:0/1032623404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2680ffd60 0x7fc268196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc26cb72700 1 -- 172.21.15.32:0/1032623404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc260009d00 con 0x7fc2681018f0 2023-09-30T15:03:30.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.317+0000 7fc26cb72700 1 --2- 172.21.15.32:0/1032623404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2681018f0 0x7fc26819e180 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc260007a00 tx=0x7fc2600094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:30.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.318+0000 7fc2657fa700 1 -- 172.21.15.32:0/1032623404 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26000e040 con 0x7fc2681018f0 2023-09-30T15:03:30.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.318+0000 7fc2657fa700 1 -- 172.21.15.32:0/1032623404 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc260003e10 con 0x7fc2681018f0 2023-09-30T15:03:30.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.318+0000 7fc26e5d5700 1 -- 172.21.15.32:0/1032623404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26819eca0 con 0x7fc2681018f0 2023-09-30T15:03:30.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.318+0000 7fc2657fa700 1 -- 172.21.15.32:0/1032623404 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26001b9e0 con 0x7fc2681018f0 2023-09-30T15:03:30.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.318+0000 7fc26e5d5700 1 -- 172.21.15.32:0/1032623404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26819f120 con 0x7fc2681018f0 2023-09-30T15:03:30.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.318+0000 7fc26e5d5700 1 -- 172.21.15.32:0/1032623404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26810b580 con 0x7fc2681018f0 2023-09-30T15:03:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.320+0000 7fc2657fa700 1 -- 172.21.15.32:0/1032623404 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fc260005630 con 0x7fc2681018f0 2023-09-30T15:03:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.320+0000 7fc2657fa700 1 --2- 172.21.15.32:0/1032623404 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2540a1020 0x7fc2540a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.321+0000 7fc2657fa700 1 -- 172.21.15.32:0/1032623404 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc2600ce4e0 con 0x7fc2681018f0 2023-09-30T15:03:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.321+0000 7fc267fff700 1 --2- 172.21.15.32:0/1032623404 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2540a1020 0x7fc2540a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.321+0000 7fc267fff700 1 --2- 172.21.15.32:0/1032623404 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2540a1020 0x7fc2540a34e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc268100db0 tx=0x7fc25000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:30.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.324+0000 7fc2657fa700 1 -- 172.21.15.32:0/1032623404 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fc260097d20 con 0x7fc2681018f0 2023-09-30T15:03:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.527+0000 7fc26e5d5700 1 -- 172.21.15.32:0/1032623404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc26819f2b0 con 0x7fc2681018f0 2023-09-30T15:03:30.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.527+0000 7fc2657fa700 1 -- 172.21.15.32:0/1032623404 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc26009b340 con 0x7fc2681018f0 2023-09-30T15:03:30.529 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T15:03:30.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.530+0000 7fc26e5d5700 1 -- 172.21.15.32:0/1032623404 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2540a1020 msgr2=0x7fc2540a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:30.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.530+0000 7fc26e5d5700 1 --2- 172.21.15.32:0/1032623404 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2540a1020 0x7fc2540a34e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc268100db0 tx=0x7fc25000a3b0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.530+0000 7fc26e5d5700 1 -- 172.21.15.32:0/1032623404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2681018f0 msgr2=0x7fc26819e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:30.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.530+0000 7fc26e5d5700 1 --2- 172.21.15.32:0/1032623404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2681018f0 0x7fc26819e180 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc260007a00 tx=0x7fc2600094c0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.530+0000 7fc26e5d5700 1 -- 172.21.15.32:0/1032623404 shutdown_connections 2023-09-30T15:03:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.530+0000 7fc26e5d5700 1 --2- 172.21.15.32:0/1032623404 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2540a1020 0x7fc2540a34e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.530+0000 7fc26e5d5700 1 --2- 172.21.15.32:0/1032623404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2681018f0 0x7fc26819e180 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.530+0000 7fc26e5d5700 1 --2- 172.21.15.32:0/1032623404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2680ffd60 0x7fc268196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.530+0000 7fc26e5d5700 1 --2- 172.21.15.32:0/1032623404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc268100f50 0x7fc268197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.530+0000 7fc26e5d5700 1 -- 172.21.15.32:0/1032623404 >> 172.21.15.32:0/1032623404 conn(0x7fc2680fb2f0 msgr2=0x7fc2680691c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.530+0000 7fc26e5d5700 1 -- 172.21.15.32:0/1032623404 shutdown_connections 2023-09-30T15:03:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.530+0000 7fc26e5d5700 1 -- 172.21.15.32:0/1032623404 wait complete. 2023-09-30T15:03:30.544 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T15:03:30.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.673+0000 7f8a8ee06700 1 -- 172.21.15.32:0/64707747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a881041e0 msgr2=0x7f8a881065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:30.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.673+0000 7f8a8ee06700 1 --2- 172.21.15.32:0/64707747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a881041e0 0x7f8a881065d0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009990 tx=0x7f8a74009ca0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.673+0000 7f8a8ee06700 1 -- 172.21.15.32:0/64707747 shutdown_connections 2023-09-30T15:03:30.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.673+0000 7f8a8ee06700 1 --2- 172.21.15.32:0/64707747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a88106b10 0x7f8a88108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.673+0000 7f8a8ee06700 1 --2- 172.21.15.32:0/64707747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a881018b0 0x7f8a88103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.673+0000 7f8a8ee06700 1 --2- 172.21.15.32:0/64707747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a881041e0 0x7f8a881065d0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.673+0000 7f8a8ee06700 1 -- 172.21.15.32:0/64707747 >> 172.21.15.32:0/64707747 conn(0x7f8a880fb2a0 msgr2=0x7f8a880fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:30.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.674+0000 7f8a8ee06700 1 -- 172.21.15.32:0/64707747 shutdown_connections 2023-09-30T15:03:30.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.674+0000 7f8a8ee06700 1 -- 172.21.15.32:0/64707747 wait complete. 2023-09-30T15:03:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.674+0000 7f8a8ee06700 1 Processor -- start 2023-09-30T15:03:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.674+0000 7f8a8ee06700 1 -- start start 2023-09-30T15:03:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.674+0000 7f8a8ee06700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a881018b0 0x7f8a88196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.674+0000 7f8a8ee06700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a881041e0 0x7f8a88197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.674+0000 7f8a8ee06700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a88106b10 0x7f8a8819e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.674+0000 7f8a8ee06700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8819e760 con 0x7f8a881041e0 2023-09-30T15:03:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.674+0000 7f8a8ee06700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8819e8a0 con 0x7f8a88106b10 2023-09-30T15:03:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.674+0000 7f8a8ee06700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8819ea30 con 0x7f8a881018b0 2023-09-30T15:03:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.675+0000 7f8a87fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a881041e0 0x7f8a88197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.675+0000 7f8a87fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a881041e0 0x7f8a88197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40556/0 (socket says 172.21.15.32:40556) 2023-09-30T15:03:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.675+0000 7f8a87fff700 1 -- 172.21.15.32:0/2571784561 learned_addr learned my addr 172.21.15.32:0/2571784561 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:03:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.675+0000 7f8a8cba2700 1 --2- 172.21.15.32:0/2571784561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a881018b0 0x7f8a88196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.675+0000 7f8a8d3a3700 1 --2- 172.21.15.32:0/2571784561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a88106b10 0x7f8a8819e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.675+0000 7f8a87fff700 1 -- 172.21.15.32:0/2571784561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a881018b0 msgr2=0x7f8a88196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.675+0000 7f8a87fff700 1 --2- 172.21.15.32:0/2571784561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a881018b0 0x7f8a88196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.675+0000 7f8a87fff700 1 -- 172.21.15.32:0/2571784561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a88106b10 msgr2=0x7f8a8819e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.675+0000 7f8a87fff700 1 --2- 172.21.15.32:0/2571784561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a88106b10 0x7f8a8819e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.675+0000 7f8a87fff700 1 -- 172.21.15.32:0/2571784561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a740095f0 con 0x7f8a881041e0 2023-09-30T15:03:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.675+0000 7f8a87fff700 1 --2- 172.21.15.32:0/2571784561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a881041e0 0x7f8a88197120 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8a74005910 tx=0x7f8a7400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.675+0000 7f8a85ffb700 1 -- 172.21.15.32:0/2571784561 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7401d070 con 0x7f8a881041e0 2023-09-30T15:03:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.675+0000 7f8a8ee06700 1 -- 172.21.15.32:0/2571784561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8819ecb0 con 0x7f8a881041e0 2023-09-30T15:03:30.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.675+0000 7f8a85ffb700 1 -- 172.21.15.32:0/2571784561 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a7400fb80 con 0x7f8a881041e0 2023-09-30T15:03:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.675+0000 7f8a85ffb700 1 -- 172.21.15.32:0/2571784561 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74017650 con 0x7f8a881041e0 2023-09-30T15:03:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.675+0000 7f8a8ee06700 1 -- 172.21.15.32:0/2571784561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8819f0d0 con 0x7f8a881041e0 2023-09-30T15:03:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.676+0000 7f8a8ee06700 1 -- 172.21.15.32:0/2571784561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a880fce20 con 0x7f8a881041e0 2023-09-30T15:03:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.678+0000 7f8a85ffb700 1 -- 172.21.15.32:0/2571784561 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f8a74022470 con 0x7f8a881041e0 2023-09-30T15:03:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.678+0000 7f8a85ffb700 1 --2- 172.21.15.32:0/2571784561 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a780a0fa0 0x7f8a780a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.678+0000 7f8a85ffb700 1 -- 172.21.15.32:0/2571784561 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8a740ce110 con 0x7f8a881041e0 2023-09-30T15:03:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.678+0000 7f8a8cba2700 1 --2- 172.21.15.32:0/2571784561 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a780a0fa0 0x7f8a780a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:30.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.679+0000 7f8a8cba2700 1 --2- 172.21.15.32:0/2571784561 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a780a0fa0 0x7f8a780a3460 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009a10 tx=0x7f8a7c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.682+0000 7f8a85ffb700 1 -- 172.21.15.32:0/2571784561 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f8a740979b0 con 0x7f8a881041e0 2023-09-30T15:03:30.900 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:30 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:03:30.901 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:30 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/432705270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T15:03:30.901 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:30 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/1032623404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T15:03:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.988+0000 7f8a8ee06700 1 -- 172.21.15.32:0/2571784561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a8810c960 con 0x7f8a881041e0 2023-09-30T15:03:30.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:30.994+0000 7f8a85ffb700 1 -- 172.21.15.32:0/2571784561 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a7409afd0 con 0x7f8a881041e0 2023-09-30T15:03:30.998 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-09-30T15:03:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.019+0000 7f8a8ee06700 1 -- 172.21.15.32:0/2571784561 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a780a0fa0 msgr2=0x7f8a780a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.019+0000 7f8a8ee06700 1 --2- 172.21.15.32:0/2571784561 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a780a0fa0 0x7f8a780a3460 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009a10 tx=0x7f8a7c009190 comp rx=0 tx=0).stop 2023-09-30T15:03:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.019+0000 7f8a8ee06700 1 -- 172.21.15.32:0/2571784561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a881041e0 msgr2=0x7f8a88197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.019+0000 7f8a8ee06700 1 --2- 172.21.15.32:0/2571784561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a881041e0 0x7f8a88197120 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8a74005910 tx=0x7f8a7400f690 comp rx=0 tx=0).stop 2023-09-30T15:03:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.020+0000 7f8a8ee06700 1 -- 172.21.15.32:0/2571784561 shutdown_connections 2023-09-30T15:03:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.020+0000 7f8a8ee06700 1 --2- 172.21.15.32:0/2571784561 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f8a780a0fa0 0x7f8a780a3460 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.020+0000 7f8a8ee06700 1 --2- 172.21.15.32:0/2571784561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a88106b10 0x7f8a8819e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.020+0000 7f8a8ee06700 1 --2- 172.21.15.32:0/2571784561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a881018b0 0x7f8a88196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.020+0000 7f8a8ee06700 1 --2- 172.21.15.32:0/2571784561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a881041e0 0x7f8a88197120 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.020+0000 7f8a8ee06700 1 -- 172.21.15.32:0/2571784561 >> 172.21.15.32:0/2571784561 conn(0x7f8a880fb2a0 msgr2=0x7f8a88102520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:31.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.020+0000 7f8a8ee06700 1 -- 172.21.15.32:0/2571784561 shutdown_connections 2023-09-30T15:03:31.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.020+0000 7f8a8ee06700 1 -- 172.21.15.32:0/2571784561 wait complete. 2023-09-30T15:03:31.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T15:03:31.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:30 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:03:31.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:30 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/432705270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T15:03:31.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:30 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/1032623404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T15:03:31.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:30 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:03:31.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:30 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/432705270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T15:03:31.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:30 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/1032623404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T15:03:31.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.194+0000 7ff6dd8a4700 1 -- 172.21.15.32:0/754800760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d81018c0 msgr2=0x7ff6d8108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:31.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.194+0000 7ff6dd8a4700 1 --2- 172.21.15.32:0/754800760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d81018c0 0x7ff6d8108170 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000b0d0 tx=0x7ff6d000b3e0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.194+0000 7ff6dd8a4700 1 -- 172.21.15.32:0/754800760 shutdown_connections 2023-09-30T15:03:31.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.194+0000 7ff6dd8a4700 1 --2- 172.21.15.32:0/754800760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d81018c0 0x7ff6d8108170 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.194+0000 7ff6dd8a4700 1 --2- 172.21.15.32:0/754800760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6d80ffd30 0x7ff6d8100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.194+0000 7ff6dd8a4700 1 --2- 172.21.15.32:0/754800760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d8100f20 0x7ff6d8101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.194+0000 7ff6dd8a4700 1 -- 172.21.15.32:0/754800760 >> 172.21.15.32:0/754800760 conn(0x7ff6d80fb2a0 msgr2=0x7ff6d80fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:31.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.194+0000 7ff6dd8a4700 1 -- 172.21.15.32:0/754800760 shutdown_connections 2023-09-30T15:03:31.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.194+0000 7ff6dd8a4700 1 -- 172.21.15.32:0/754800760 wait complete. 2023-09-30T15:03:31.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.195+0000 7ff6dd8a4700 1 Processor -- start 2023-09-30T15:03:31.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.195+0000 7ff6dd8a4700 1 -- start start 2023-09-30T15:03:31.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.195+0000 7ff6dd8a4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d80ffd30 0x7ff6d8195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:31.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.195+0000 7ff6dd8a4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d8100f20 0x7ff6d8196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:31.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.195+0000 7ff6dd8a4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6d81018c0 0x7ff6d819d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:31.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.195+0000 7ff6dd8a4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d819d6d0 con 0x7ff6d80ffd30 2023-09-30T15:03:31.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.195+0000 7ff6dd8a4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d819d810 con 0x7ff6d8100f20 2023-09-30T15:03:31.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.195+0000 7ff6dd8a4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d819d9a0 con 0x7ff6d81018c0 2023-09-30T15:03:31.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.195+0000 7ff6d6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d80ffd30 0x7ff6d8195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:31.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.196+0000 7ff6d6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d80ffd30 0x7ff6d8195b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40568/0 (socket says 172.21.15.32:40568) 2023-09-30T15:03:31.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.196+0000 7ff6d6ffd700 1 -- 172.21.15.32:0/4025203489 learned_addr learned my addr 172.21.15.32:0/4025203489 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:03:31.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.196+0000 7ff6d67fc700 1 --2- 172.21.15.32:0/4025203489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d8100f20 0x7ff6d8196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:31.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.196+0000 7ff6d77fe700 1 --2- 172.21.15.32:0/4025203489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6d81018c0 0x7ff6d819d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:31.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.196+0000 7ff6d6ffd700 1 -- 172.21.15.32:0/4025203489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6d81018c0 msgr2=0x7ff6d819d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:31.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.196+0000 7ff6d6ffd700 1 --2- 172.21.15.32:0/4025203489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6d81018c0 0x7ff6d819d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.196+0000 7ff6d6ffd700 1 -- 172.21.15.32:0/4025203489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d8100f20 msgr2=0x7ff6d8196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:31.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.196+0000 7ff6d6ffd700 1 --2- 172.21.15.32:0/4025203489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d8100f20 0x7ff6d8196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.196+0000 7ff6d6ffd700 1 -- 172.21.15.32:0/4025203489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d0009d00 con 0x7ff6d80ffd30 2023-09-30T15:03:31.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.196+0000 7ff6d6ffd700 1 --2- 172.21.15.32:0/4025203489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d80ffd30 0x7ff6d8195b50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff6c800e970 tx=0x7ff6c800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:31.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.196+0000 7ff6dc8a2700 1 -- 172.21.15.32:0/4025203489 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8009790 con 0x7ff6d80ffd30 2023-09-30T15:03:31.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.196+0000 7ff6dc8a2700 1 -- 172.21.15.32:0/4025203489 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6c800c7e0 con 0x7ff6d80ffd30 2023-09-30T15:03:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.196+0000 7ff6dd8a4700 1 -- 172.21.15.32:0/4025203489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d819dc80 con 0x7ff6d80ffd30 2023-09-30T15:03:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.196+0000 7ff6dc8a2700 1 -- 172.21.15.32:0/4025203489 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8003c70 con 0x7ff6d80ffd30 2023-09-30T15:03:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.196+0000 7ff6dd8a4700 1 -- 172.21.15.32:0/4025203489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d819e190 con 0x7ff6d80ffd30 2023-09-30T15:03:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.197+0000 7ff6dd8a4700 1 -- 172.21.15.32:0/4025203489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d8107270 con 0x7ff6d80ffd30 2023-09-30T15:03:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.198+0000 7ff6dc8a2700 1 -- 172.21.15.32:0/4025203489 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7ff6c800cc50 con 0x7ff6d80ffd30 2023-09-30T15:03:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.199+0000 7ff6dc8a2700 1 --2- 172.21.15.32:0/4025203489 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff6c40a0fa0 0x7ff6c40a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.199+0000 7ff6dc8a2700 1 -- 172.21.15.32:0/4025203489 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff6c8014070 con 0x7ff6d80ffd30 2023-09-30T15:03:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.199+0000 7ff6d67fc700 1 --2- 172.21.15.32:0/4025203489 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff6c40a0fa0 0x7ff6c40a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.199+0000 7ff6d67fc700 1 --2- 172.21.15.32:0/4025203489 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff6c40a0fa0 0x7ff6c40a3460 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009c60 tx=0x7ff6c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.203+0000 7ff6dc8a2700 1 -- 172.21.15.32:0/4025203489 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7ff6c8096100 con 0x7ff6d80ffd30 2023-09-30T15:03:31.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.410+0000 7ff6dd8a4700 1 -- 172.21.15.32:0/4025203489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff6d802cfe0 con 0x7ff6d80ffd30 2023-09-30T15:03:31.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.410+0000 7ff6dc8a2700 1 -- 172.21.15.32:0/4025203489 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff6c8003dd0 con 0x7ff6d80ffd30 2023-09-30T15:03:31.412 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-09-30T15:03:31.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.412+0000 7ff6dd8a4700 1 -- 172.21.15.32:0/4025203489 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff6c40a0fa0 msgr2=0x7ff6c40a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:31.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.412+0000 7ff6dd8a4700 1 --2- 172.21.15.32:0/4025203489 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff6c40a0fa0 0x7ff6c40a3460 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009c60 tx=0x7ff6c0009260 comp rx=0 tx=0).stop 2023-09-30T15:03:31.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.412+0000 7ff6dd8a4700 1 -- 172.21.15.32:0/4025203489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d80ffd30 msgr2=0x7ff6d8195b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:31.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.412+0000 7ff6dd8a4700 1 --2- 172.21.15.32:0/4025203489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d80ffd30 0x7ff6d8195b50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff6c800e970 tx=0x7ff6c800ec80 comp rx=0 tx=0).stop 2023-09-30T15:03:31.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.412+0000 7ff6dd8a4700 1 -- 172.21.15.32:0/4025203489 shutdown_connections 2023-09-30T15:03:31.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.412+0000 7ff6dd8a4700 1 --2- 172.21.15.32:0/4025203489 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ff6c40a0fa0 0x7ff6c40a3460 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.412+0000 7ff6dd8a4700 1 --2- 172.21.15.32:0/4025203489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6d81018c0 0x7ff6d819d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.412+0000 7ff6dd8a4700 1 --2- 172.21.15.32:0/4025203489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d80ffd30 0x7ff6d8195b50 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.412+0000 7ff6dd8a4700 1 --2- 172.21.15.32:0/4025203489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d8100f20 0x7ff6d8196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.412+0000 7ff6dd8a4700 1 -- 172.21.15.32:0/4025203489 >> 172.21.15.32:0/4025203489 conn(0x7ff6d80fb2a0 msgr2=0x7ff6d81043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:31.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.413+0000 7ff6dd8a4700 1 -- 172.21.15.32:0/4025203489 shutdown_connections 2023-09-30T15:03:31.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.413+0000 7ff6dd8a4700 1 -- 172.21.15.32:0/4025203489 wait complete. 2023-09-30T15:03:31.426 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-09-30T15:03:31.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.554+0000 7fc82aff0700 1 -- 172.21.15.32:0/434471315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc824106c40 msgr2=0x7fc8241090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.554+0000 7fc82aff0700 1 --2- 172.21.15.32:0/434471315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc824106c40 0x7fc8241090d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00b0a0 tx=0x7fc81c00b3b0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.558+0000 7fc82aff0700 1 -- 172.21.15.32:0/434471315 shutdown_connections 2023-09-30T15:03:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.558+0000 7fc82aff0700 1 --2- 172.21.15.32:0/434471315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc824106c40 0x7fc8241090d0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.559+0000 7fc82aff0700 1 --2- 172.21.15.32:0/434471315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc824101990 0x7fc824103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.559+0000 7fc82aff0700 1 --2- 172.21.15.32:0/434471315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8241042c0 0x7fc8241066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.559+0000 7fc82aff0700 1 -- 172.21.15.32:0/434471315 >> 172.21.15.32:0/434471315 conn(0x7fc8240fb2e0 msgr2=0x7fc8240fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.560+0000 7fc82aff0700 1 -- 172.21.15.32:0/434471315 shutdown_connections 2023-09-30T15:03:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.560+0000 7fc82aff0700 1 -- 172.21.15.32:0/434471315 wait complete. 2023-09-30T15:03:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.561+0000 7fc82aff0700 1 Processor -- start 2023-09-30T15:03:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.561+0000 7fc82aff0700 1 -- start start 2023-09-30T15:03:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.561+0000 7fc82aff0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc824101990 0x7fc8240707b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.561+0000 7fc82aff0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8241042c0 0x7fc824070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.561+0000 7fc82aff0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc824106c40 0x7fc824071d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.561+0000 7fc82aff0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824072300 con 0x7fc824106c40 2023-09-30T15:03:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.561+0000 7fc82aff0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824072470 con 0x7fc824101990 2023-09-30T15:03:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.561+0000 7fc82aff0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824072630 con 0x7fc8241042c0 2023-09-30T15:03:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.561+0000 7fc828d8c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc824101990 0x7fc8240707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.561+0000 7fc823fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8241042c0 0x7fc824070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.561+0000 7fc828d8c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc824101990 0x7fc8240707b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.32:52058/0 (socket says 172.21.15.32:52058) 2023-09-30T15:03:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.561+0000 7fc828d8c700 1 -- 172.21.15.32:0/856892262 learned_addr learned my addr 172.21.15.32:0/856892262 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:03:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.561+0000 7fc82958d700 1 --2- 172.21.15.32:0/856892262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc824106c40 0x7fc824071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:31.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.562+0000 7fc823fff700 1 -- 172.21.15.32:0/856892262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc824101990 msgr2=0x7fc8240707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:31.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.562+0000 7fc823fff700 1 --2- 172.21.15.32:0/856892262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc824101990 0x7fc8240707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.562+0000 7fc823fff700 1 -- 172.21.15.32:0/856892262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc824106c40 msgr2=0x7fc824071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:31.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.562+0000 7fc823fff700 1 --2- 172.21.15.32:0/856892262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc824106c40 0x7fc824071d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:31.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.562+0000 7fc823fff700 1 -- 172.21.15.32:0/856892262 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c009d00 con 0x7fc8241042c0 2023-09-30T15:03:31.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.562+0000 7fc823fff700 1 --2- 172.21.15.32:0/856892262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8241042c0 0x7fc824070cf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc81000e9f0 tx=0x7fc81000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:31.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.563+0000 7fc821ffb700 1 -- 172.21.15.32:0/856892262 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81000c9c0 con 0x7fc8241042c0 2023-09-30T15:03:31.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.563+0000 7fc821ffb700 1 -- 172.21.15.32:0/856892262 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc810010430 con 0x7fc8241042c0 2023-09-30T15:03:31.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.563+0000 7fc821ffb700 1 -- 172.21.15.32:0/856892262 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810018680 con 0x7fc8241042c0 2023-09-30T15:03:31.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.563+0000 7fc82aff0700 1 -- 172.21.15.32:0/856892262 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8240728b0 con 0x7fc8241042c0 2023-09-30T15:03:31.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.563+0000 7fc82aff0700 1 -- 172.21.15.32:0/856892262 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc824072d70 con 0x7fc8241042c0 2023-09-30T15:03:31.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.567+0000 7fc821ffb700 1 -- 172.21.15.32:0/856892262 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fc810010860 con 0x7fc8241042c0 2023-09-30T15:03:31.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.567+0000 7fc82aff0700 1 -- 172.21.15.32:0/856892262 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8240fce60 con 0x7fc8241042c0 2023-09-30T15:03:31.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.568+0000 7fc821ffb700 1 --2- 172.21.15.32:0/856892262 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc8140a10a0 0x7fc8140a3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:03:31.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.568+0000 7fc828d8c700 1 --2- 172.21.15.32:0/856892262 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc8140a10a0 0x7fc8140a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:03:31.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.568+0000 7fc821ffb700 1 -- 172.21.15.32:0/856892262 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc810014070 con 0x7fc8241042c0 2023-09-30T15:03:31.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.568+0000 7fc828d8c700 1 --2- 172.21.15.32:0/856892262 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc8140a10a0 0x7fc8140a3560 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc818009d20 tx=0x7fc818009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:03:31.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.573+0000 7fc821ffb700 1 -- 172.21.15.32:0/856892262 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fc810095fc0 con 0x7fc8241042c0 2023-09-30T15:03:31.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:31 smithi032 ceph-mon[98592]: pgmap v1044: 97 pgs: 97 active+clean; 785 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.8 MiB/s wr, 668 op/s 2023-09-30T15:03:31.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:31 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2571784561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T15:03:31.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:31 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/4025203489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T15:03:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.902+0000 7fc82aff0700 1 -- 172.21.15.32:0/856892262 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fc82402cf30 con 0x7fc8241042c0 2023-09-30T15:03:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.911+0000 7fc821ffb700 1 -- 172.21.15.32:0/856892262 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+732166 (secure 0 0 0) 0x7fc8100995e0 con 0x7fc8241042c0 2023-09-30T15:03:31.920 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T15:03:31.920 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T15:03:31.920 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 18, 2023-09-30T15:03:31.920 INFO:teuthology.orchestra.run.smithi032.stdout: "active_gid": 14152, 2023-09-30T15:03:31.920 INFO:teuthology.orchestra.run.smithi032.stdout: "active_name": "x", 2023-09-30T15:03:31.920 INFO:teuthology.orchestra.run.smithi032.stdout: "active_addrs": { 2023-09-30T15:03:31.921 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T15:03:31.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:31.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-09-30T15:03:31.921 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2023-09-30T15:03:31.921 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3273633815 2023-09-30T15:03:31.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.921 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:31.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T15:03:31.921 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2023-09-30T15:03:31.921 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3273633815 2023-09-30T15:03:31.921 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:31.922 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T15:03:31.922 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.922 INFO:teuthology.orchestra.run.smithi032.stdout: "active_addr": "172.21.15.32:6801/3273633815", 2023-09-30T15:03:31.922 INFO:teuthology.orchestra.run.smithi032.stdout: "active_change": "2023-09-30T14:31:11.693659+0000", 2023-09-30T15:03:31.922 INFO:teuthology.orchestra.run.smithi032.stdout: "active_mgr_features": 4540138322906710015, 2023-09-30T15:03:31.922 INFO:teuthology.orchestra.run.smithi032.stdout: "available": true, 2023-09-30T15:03:31.922 INFO:teuthology.orchestra.run.smithi032.stdout: "standbys": [ 2023-09-30T15:03:31.922 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:31.922 INFO:teuthology.orchestra.run.smithi032.stdout: "gid": 24106, 2023-09-30T15:03:31.922 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "z", 2023-09-30T15:03:31.922 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_features": 4540138322906710015, 2023-09-30T15:03:31.923 INFO:teuthology.orchestra.run.smithi032.stdout: "available_modules": [ 2023-09-30T15:03:31.923 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:31.923 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "alerts", 2023-09-30T15:03:31.923 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:31.923 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:31.923 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:31.923 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-09-30T15:03:31.923 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-09-30T15:03:31.923 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:31.923 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.923 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.923 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-09-30T15:03:31.924 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.924 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.924 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.924 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How frequently to reexamine health status", 2023-09-30T15:03:31.924 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.924 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.924 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.924 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:31.924 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:31.924 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.924 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.925 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.925 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:31.925 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.925 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.925 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:31.925 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:31.925 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:31.925 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:31.925 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:31.925 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:31.925 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:31.925 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:31.926 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:31.926 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.926 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.926 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:31.926 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:31.926 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:31.926 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.926 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.926 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:31.926 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.926 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.927 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.927 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:31.927 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.927 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.927 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.927 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.927 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:31.927 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:31.927 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.927 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.927 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.928 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:31.928 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.928 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.928 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:31.928 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:31.928 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:31.928 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:31.928 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:31.928 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:31.928 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:31.928 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:31.928 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:31.928 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.929 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.929 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.929 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:31.929 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:31.929 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:31.929 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.929 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.929 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:31.929 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.929 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.929 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.929 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:31.930 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.930 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.930 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.930 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.930 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_destination": { 2023-09-30T15:03:31.930 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_destination", 2023-09-30T15:03:31.930 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.930 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.930 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.930 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:31.930 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.930 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.931 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.931 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email address to send alerts to", 2023-09-30T15:03:31.931 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.931 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.931 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.931 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_from_name": { 2023-09-30T15:03:31.931 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_from_name", 2023-09-30T15:03:31.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.931 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.931 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.931 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "Ceph", 2023-09-30T15:03:31.932 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.932 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.932 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.932 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email From: name", 2023-09-30T15:03:31.932 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.932 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.932 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.932 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_host": { 2023-09-30T15:03:31.932 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_host", 2023-09-30T15:03:31.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.932 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.932 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.933 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:31.933 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.933 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.933 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.933 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP server", 2023-09-30T15:03:31.933 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.933 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.933 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.933 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_password": { 2023-09-30T15:03:31.933 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_password", 2023-09-30T15:03:31.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.934 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.934 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.934 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:31.934 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.934 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.934 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.934 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Password to authenticate with", 2023-09-30T15:03:31.934 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.934 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.934 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.934 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_port": { 2023-09-30T15:03:31.935 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_port", 2023-09-30T15:03:31.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:31.935 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.935 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.935 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "465", 2023-09-30T15:03:31.935 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.935 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.935 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.935 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP port", 2023-09-30T15:03:31.935 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.935 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.935 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.936 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_sender": { 2023-09-30T15:03:31.936 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_sender", 2023-09-30T15:03:31.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.936 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.936 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.936 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:31.936 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.936 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.936 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.936 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP envelope sender", 2023-09-30T15:03:31.936 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.936 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.937 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.937 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_ssl": { 2023-09-30T15:03:31.937 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_ssl", 2023-09-30T15:03:31.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:31.937 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.937 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.937 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:31.937 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.937 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.937 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.938 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-30T15:03:31.938 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.938 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.938 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.938 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_user": { 2023-09-30T15:03:31.938 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_user", 2023-09-30T15:03:31.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.938 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.938 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.938 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:31.938 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.939 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.939 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.939 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "User to authenticate as", 2023-09-30T15:03:31.939 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.939 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.939 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.939 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:31.939 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:31.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:31.939 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "balancer", 2023-09-30T15:03:31.939 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:31.940 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:31.940 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:31.940 INFO:teuthology.orchestra.run.smithi032.stdout: "active": { 2023-09-30T15:03:31.940 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "active", 2023-09-30T15:03:31.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:31.940 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.940 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.940 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:31.940 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.940 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.940 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.940 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically balance PGs across cluster", 2023-09-30T15:03:31.940 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.941 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.941 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.941 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_time": { 2023-09-30T15:03:31.941 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_time", 2023-09-30T15:03:31.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.941 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.941 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.941 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0000", 2023-09-30T15:03:31.941 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.941 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.941 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.942 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "beginning time of day to automatically balance", 2023-09-30T15:03:31.942 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-30T15:03:31.942 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.942 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.942 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_weekday": { 2023-09-30T15:03:31.942 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_weekday", 2023-09-30T15:03:31.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-09-30T15:03:31.942 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.942 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.942 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-09-30T15:03:31.942 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2023-09-30T15:03:31.943 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2023-09-30T15:03:31.943 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.943 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-30T15:03:31.943 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-30T15:03:31.943 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.943 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.943 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_max_iterations": { 2023-09-30T15:03:31.943 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_max_iterations", 2023-09-30T15:03:31.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-09-30T15:03:31.943 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.943 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.944 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "25", 2023-09-30T15:03:31.944 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-09-30T15:03:31.944 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "250", 2023-09-30T15:03:31.944 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.944 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-30T15:03:31.944 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.944 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.944 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.944 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_metrics": { 2023-09-30T15:03:31.944 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_metrics", 2023-09-30T15:03:31.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.944 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.945 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.945 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "pgs,objects,bytes", 2023-09-30T15:03:31.945 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.945 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.945 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.945 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-30T15:03:31.945 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-30T15:03:31.945 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.945 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.945 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_step": { 2023-09-30T15:03:31.945 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_step", 2023-09-30T15:03:31.946 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:31.946 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.946 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.946 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.5", 2023-09-30T15:03:31.946 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0.001", 2023-09-30T15:03:31.946 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "0.999", 2023-09-30T15:03:31.946 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.946 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "aggressiveness of optimization", 2023-09-30T15:03:31.946 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-30T15:03:31.946 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.946 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.947 INFO:teuthology.orchestra.run.smithi032.stdout: "end_time": { 2023-09-30T15:03:31.947 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_time", 2023-09-30T15:03:31.947 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.947 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.947 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.947 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2359", 2023-09-30T15:03:31.947 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.947 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.947 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.947 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ending time of day to automatically balance", 2023-09-30T15:03:31.947 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-30T15:03:31.947 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.947 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.948 INFO:teuthology.orchestra.run.smithi032.stdout: "end_weekday": { 2023-09-30T15:03:31.948 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_weekday", 2023-09-30T15:03:31.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-09-30T15:03:31.948 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.948 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.948 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-09-30T15:03:31.948 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2023-09-30T15:03:31.948 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2023-09-30T15:03:31.948 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.948 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-30T15:03:31.948 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-30T15:03:31.949 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.949 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.949 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:31.949 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:31.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.949 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.949 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.949 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:31.949 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.949 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.949 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:31.949 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:31.950 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:31.950 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:31.950 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:31.950 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:31.950 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:31.950 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:31.950 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:31.950 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.950 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.950 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.950 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:31.951 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:31.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:31.951 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.951 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.951 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:31.951 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.951 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.951 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.951 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:31.951 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.951 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.952 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.952 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.952 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:31.952 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:31.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.952 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.952 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.952 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:31.952 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.952 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.952 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:31.952 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:31.953 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:31.953 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:31.953 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:31.953 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:31.953 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:31.953 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:31.953 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:31.953 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.953 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.953 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.953 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:31.953 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:31.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:31.954 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.954 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.954 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:31.954 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.954 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.954 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.954 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:31.954 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.954 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.954 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.955 INFO:teuthology.orchestra.run.smithi032.stdout: "min_score": { 2023-09-30T15:03:31.955 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_score", 2023-09-30T15:03:31.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:31.955 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.955 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.955 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-09-30T15:03:31.955 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.955 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.955 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.955 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-30T15:03:31.955 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.955 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.956 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.956 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2023-09-30T15:03:31.956 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2023-09-30T15:03:31.956 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.956 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.956 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.956 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "upmap", 2023-09-30T15:03:31.956 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.956 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.956 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:31.956 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-compat", 2023-09-30T15:03:31.957 INFO:teuthology.orchestra.run.smithi032.stdout: "none", 2023-09-30T15:03:31.957 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap" 2023-09-30T15:03:31.957 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:31.957 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Balancer mode", 2023-09-30T15:03:31.957 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.957 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.957 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.957 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_ids": { 2023-09-30T15:03:31.957 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_ids", 2023-09-30T15:03:31.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.957 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.957 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.958 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:31.958 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.958 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.958 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.958 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-30T15:03:31.958 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.958 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.958 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.958 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-09-30T15:03:31.958 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-09-30T15:03:31.958 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:31.959 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.959 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-09-30T15:03:31.959 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.959 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.959 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.959 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-30T15:03:31.959 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.959 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.959 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.959 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_deviation": { 2023-09-30T15:03:31.960 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_deviation", 2023-09-30T15:03:31.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:31.960 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.960 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.960 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-09-30T15:03:31.960 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-09-30T15:03:31.960 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.960 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.960 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-30T15:03:31.960 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-30T15:03:31.960 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.960 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.960 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.961 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_optimizations": { 2023-09-30T15:03:31.961 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_optimizations", 2023-09-30T15:03:31.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-09-30T15:03:31.961 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.961 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.961 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:31.961 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.961 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.961 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.961 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-30T15:03:31.961 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.961 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.962 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:31.962 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:31.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.962 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:31.962 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm", 2023-09-30T15:03:31.962 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:31.962 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:31.962 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:31.962 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_down_multiplier": { 2023-09-30T15:03:31.962 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_down_multiplier", 2023-09-30T15:03:31.962 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:31.963 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.963 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2023-09-30T15:03:31.963 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.963 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.963 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.963 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-30T15:03:31.963 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.963 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.963 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.963 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_refresh_rate": { 2023-09-30T15:03:31.963 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_refresh_rate", 2023-09-30T15:03:31.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:31.964 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.964 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.964 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "20", 2023-09-30T15:03:31.964 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.964 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.964 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.964 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-30T15:03:31.964 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.964 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.964 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.965 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_starting_port": { 2023-09-30T15:03:31.965 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_starting_port", 2023-09-30T15:03:31.965 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:31.965 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.965 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.965 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4721", 2023-09-30T15:03:31.965 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.965 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.965 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.965 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-30T15:03:31.965 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.965 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.966 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.966 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_ptrace": { 2023-09-30T15:03:31.966 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_ptrace", 2023-09-30T15:03:31.966 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:31.966 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.966 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.966 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:31.966 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.966 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.966 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.967 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-30T15:03:31.967 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-30T15:03:31.967 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.967 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.967 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_interval": { 2023-09-30T15:03:31.967 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_interval", 2023-09-30T15:03:31.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:31.967 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.967 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.967 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-09-30T15:03:31.967 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.968 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.968 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.968 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-30T15:03:31.968 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.968 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.968 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.968 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_memory_target_ratio": { 2023-09-30T15:03:31.968 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_memory_target_ratio", 2023-09-30T15:03:31.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:31.968 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.968 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.969 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.7", 2023-09-30T15:03:31.969 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.969 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.969 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.969 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-30T15:03:31.969 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.969 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.969 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.969 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm_log_destination": { 2023-09-30T15:03:31.969 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm_log_destination", 2023-09-30T15:03:31.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.970 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.970 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:31.970 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.970 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.970 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:31.970 INFO:teuthology.orchestra.run.smithi032.stdout: "file", 2023-09-30T15:03:31.970 INFO:teuthology.orchestra.run.smithi032.stdout: "file,syslog", 2023-09-30T15:03:31.970 INFO:teuthology.orchestra.run.smithi032.stdout: "syslog" 2023-09-30T15:03:31.970 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:31.970 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-09-30T15:03:31.970 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.971 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.971 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.971 INFO:teuthology.orchestra.run.smithi032.stdout: "cgroups_split": { 2023-09-30T15:03:31.971 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cgroups_split", 2023-09-30T15:03:31.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:31.971 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.971 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.971 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:31.971 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.971 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.971 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.972 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-30T15:03:31.972 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.972 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.972 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.972 INFO:teuthology.orchestra.run.smithi032.stdout: "config_checks_enabled": { 2023-09-30T15:03:31.972 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_checks_enabled", 2023-09-30T15:03:31.972 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:31.972 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.972 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.972 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:31.972 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.973 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.973 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.973 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-30T15:03:31.973 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.973 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.973 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.973 INFO:teuthology.orchestra.run.smithi032.stdout: "config_dashboard": { 2023-09-30T15:03:31.973 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_dashboard", 2023-09-30T15:03:31.973 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:31.973 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.973 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.974 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:31.974 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.974 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.974 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.974 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-30T15:03:31.974 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.974 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.974 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.974 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_alertmanager": { 2023-09-30T15:03:31.974 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_alertmanager", 2023-09-30T15:03:31.974 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.975 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.975 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.975 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-30T15:03:31.975 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.975 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.975 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.975 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-09-30T15:03:31.975 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.975 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.975 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.975 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_base": { 2023-09-30T15:03:31.976 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_base", 2023-09-30T15:03:31.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.976 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.976 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:31.976 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-30T15:03:31.976 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.976 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.976 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.976 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Container image name, without the tag", 2023-09-30T15:03:31.976 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.976 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.977 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_elasticsearch": { 2023-09-30T15:03:31.977 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_elasticsearch", 2023-09-30T15:03:31.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.977 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.977 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.977 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-30T15:03:31.977 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.977 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.977 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.977 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "elasticsearch container image", 2023-09-30T15:03:31.977 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.978 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.978 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.978 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_grafana": { 2023-09-30T15:03:31.978 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_grafana", 2023-09-30T15:03:31.978 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.978 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.978 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.978 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-30T15:03:31.978 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.978 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.978 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.979 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-09-30T15:03:31.979 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.979 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.979 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.979 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_haproxy": { 2023-09-30T15:03:31.979 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_haproxy", 2023-09-30T15:03:31.979 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.979 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.979 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.979 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-30T15:03:31.979 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.980 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.980 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.980 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "HAproxy container image", 2023-09-30T15:03:31.980 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.980 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.980 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.980 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.980 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_agent": { 2023-09-30T15:03:31.980 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_agent", 2023-09-30T15:03:31.980 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.980 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.981 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.981 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-30T15:03:31.981 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.981 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.981 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.981 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger agent container image", 2023-09-30T15:03:31.981 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.981 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.981 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.981 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_collector": { 2023-09-30T15:03:31.981 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_collector", 2023-09-30T15:03:31.982 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.982 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.982 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.982 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-30T15:03:31.982 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.982 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.982 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.982 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger collector container image", 2023-09-30T15:03:31.982 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.982 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.982 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_query": { 2023-09-30T15:03:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_query", 2023-09-30T15:03:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-30T15:03:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger query container image", 2023-09-30T15:03:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.983 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_keepalived": { 2023-09-30T15:03:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_keepalived", 2023-09-30T15:03:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-30T15:03:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.984 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Keepalived container image", 2023-09-30T15:03:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_loki": { 2023-09-30T15:03:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_loki", 2023-09-30T15:03:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-30T15:03:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Loki container image", 2023-09-30T15:03:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_node_exporter": { 2023-09-30T15:03:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_node_exporter", 2023-09-30T15:03:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.986 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-30T15:03:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-09-30T15:03:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nvmeof": { 2023-09-30T15:03:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nvmeof", 2023-09-30T15:03:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.987 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-30T15:03:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nvme-of container image", 2023-09-30T15:03:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_prometheus": { 2023-09-30T15:03:31.988 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_prometheus", 2023-09-30T15:03:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-30T15:03:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-09-30T15:03:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.989 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_promtail": { 2023-09-30T15:03:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_promtail", 2023-09-30T15:03:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-30T15:03:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Promtail container image", 2023-09-30T15:03:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.990 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_snmp_gateway": { 2023-09-30T15:03:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_snmp_gateway", 2023-09-30T15:03:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-30T15:03:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SNMP Gateway container image", 2023-09-30T15:03:31.991 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "container_init": { 2023-09-30T15:03:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_init", 2023-09-30T15:03:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.992 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Run podman/docker with `--init`", 2023-09-30T15:03:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "daemon_cache_timeout": { 2023-09-30T15:03:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "daemon_cache_timeout", 2023-09-30T15:03:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.993 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-09-30T15:03:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-30T15:03:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.994 INFO:teuthology.orchestra.run.smithi032.stdout: "default_cephadm_command_timeout": { 2023-09-30T15:03:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_cephadm_command_timeout", 2023-09-30T15:03:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "900", 2023-09-30T15:03:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-30T15:03:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.995 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "default_registry": { 2023-09-30T15:03:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_registry", 2023-09-30T15:03:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io", 2023-09-30T15:03:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-30T15:03:31.996 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "device_cache_timeout": { 2023-09-30T15:03:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_cache_timeout", 2023-09-30T15:03:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1800", 2023-09-30T15:03:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.997 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache device inventory", 2023-09-30T15:03:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "device_enhanced_scan": { 2023-09-30T15:03:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_enhanced_scan", 2023-09-30T15:03:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:31.998 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-30T15:03:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "facts_cache_timeout": { 2023-09-30T15:03:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "facts_cache_timeout", 2023-09-30T15:03:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:31.999 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-09-30T15:03:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache host facts data", 2023-09-30T15:03:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "host_check_interval": { 2023-09-30T15:03:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "host_check_interval", 2023-09-30T15:03:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-09-30T15:03:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to perform a host check", 2023-09-30T15:03:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.002 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "inventory_list_all": { 2023-09-30T15:03:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "inventory_list_all", 2023-09-30T15:03:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-30T15:03:32.003 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.004 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.005 INFO:teuthology.orchestra.run.smithi032.stdout: "log_refresh_metadata": { 2023-09-30T15:03:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_refresh_metadata", 2023-09-30T15:03:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-30T15:03:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-30T15:03:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.007 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.008 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.009 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.010 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-30T15:03:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-30T15:03:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-30T15:03:32.011 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-30T15:03:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-30T15:03:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "*", 2023-09-30T15:03:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.012 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-30T15:03:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "max_count_per_host": { 2023-09-30T15:03:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_count_per_host", 2023-09-30T15:03:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.013 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of daemons per service per host", 2023-09-30T15:03:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd_draining_count": { 2023-09-30T15:03:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_osd_draining_count", 2023-09-30T15:03:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.014 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-30T15:03:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "migration_current": { 2023-09-30T15:03:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "migration_current", 2023-09-30T15:03:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.015 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "internal - do not modify", 2023-09-30T15:03:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2023-09-30T15:03:32.016 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2023-09-30T15:03:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "root", 2023-09-30T15:03:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm-package", 2023-09-30T15:03:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "root" 2023-09-30T15:03:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "mode for remote execution of cephadm", 2023-09-30T15:03:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.017 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "prometheus_alerts_path": { 2023-09-30T15:03:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus_alerts_path", 2023-09-30T15:03:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-30T15:03:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.018 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-30T15:03:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_insecure": { 2023-09-30T15:03:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_insecure", 2023-09-30T15:03:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.019 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-30T15:03:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_password": { 2023-09-30T15:03:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_password", 2023-09-30T15:03:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.020 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-30T15:03:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_url": { 2023-09-30T15:03:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_url", 2023-09-30T15:03:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.021 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-30T15:03:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_username": { 2023-09-30T15:03:32.022 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_username", 2023-09-30T15:03:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-30T15:03:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.023 INFO:teuthology.orchestra.run.smithi032.stdout: "secure_monitoring_stack": { 2023-09-30T15:03:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secure_monitoring_stack", 2023-09-30T15:03:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-30T15:03:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.024 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "service_discovery_port": { 2023-09-30T15:03:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "service_discovery_port", 2023-09-30T15:03:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8765", 2023-09-30T15:03:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "cephadm service discovery port", 2023-09-30T15:03:32.025 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "ssh_config_file": { 2023-09-30T15:03:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssh_config_file", 2023-09-30T15:03:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.026 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-30T15:03:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "use_agent": { 2023-09-30T15:03:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_agent", 2023-09-30T15:03:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.027 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-30T15:03:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "use_repo_digest": { 2023-09-30T15:03:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_repo_digest", 2023-09-30T15:03:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.028 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-30T15:03:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_failed_host_check": { 2023-09-30T15:03:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_failed_host_check", 2023-09-30T15:03:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.029 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if the host check fails", 2023-09-30T15:03:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_daemons": { 2023-09-30T15:03:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_daemons", 2023-09-30T15:03:32.030 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-30T15:03:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.031 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_hosts": { 2023-09-30T15:03:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_hosts", 2023-09-30T15:03:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-30T15:03:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.032 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crash", 2023-09-30T15:03:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.033 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.034 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.035 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.036 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.037 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.038 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "retain_interval": { 2023-09-30T15:03:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retain_interval", 2023-09-30T15:03:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "31536000", 2023-09-30T15:03:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-30T15:03:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_recent_interval": { 2023-09-30T15:03:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_recent_interval", 2023-09-30T15:03:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1209600", 2023-09-30T15:03:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-30T15:03:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dashboard", 2023-09-30T15:03:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-30T15:03:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-30T15:03:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.042 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_HOST": { 2023-09-30T15:03:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-30T15:03:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.043 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-30T15:03:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-30T15:03:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.044 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_ENABLED": { 2023-09-30T15:03:32.045 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_ENABLED", 2023-09-30T15:03:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-30T15:03:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-30T15:03:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.047 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "ENABLE_BROWSABLE_API": { 2023-09-30T15:03:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-30T15:03:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.048 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-30T15:03:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-30T15:03:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.049 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-30T15:03:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-30T15:03:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.050 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-30T15:03:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-30T15:03:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.051 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-30T15:03:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-30T15:03:32.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.053 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-30T15:03:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-30T15:03:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.054 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-30T15:03:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-30T15:03:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.055 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-30T15:03:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-30T15:03:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.056 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-30T15:03:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-30T15:03:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.057 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.058 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_PASSWORD": { 2023-09-30T15:03:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-30T15:03:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-09-30T15:03:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.059 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-30T15:03:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-30T15:03:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.060 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_URL": { 2023-09-30T15:03:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_URL", 2023-09-30T15:03:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.061 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_USERNAME": { 2023-09-30T15:03:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-30T15:03:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-09-30T15:03:32.062 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-30T15:03:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-30T15:03:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.063 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.064 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.064 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.064 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.064 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.064 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.064 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.064 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.064 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.064 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-30T15:03:32.064 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-30T15:03:32.065 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.065 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.065 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.065 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.065 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.065 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.065 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.065 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.065 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.065 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.065 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.065 INFO:teuthology.orchestra.run.smithi032.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-30T15:03:32.066 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-30T15:03:32.066 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.066 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.066 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.066 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.066 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.066 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.066 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.066 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.066 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.066 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.067 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.067 INFO:teuthology.orchestra.run.smithi032.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-30T15:03:32.067 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-30T15:03:32.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.067 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.067 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.067 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.067 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.067 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.067 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.067 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.068 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.068 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.068 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.068 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_HOST": { 2023-09-30T15:03:32.068 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-30T15:03:32.068 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.068 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.068 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.068 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.068 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.068 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.069 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.069 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.069 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.069 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.069 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.069 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.069 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-30T15:03:32.069 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-30T15:03:32.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.069 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.069 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.069 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.070 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.070 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.070 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.070 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.070 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.070 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.070 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.070 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-30T15:03:32.070 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-30T15:03:32.070 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.071 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.071 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.071 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.071 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.071 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.071 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.071 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.071 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.071 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.071 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-30T15:03:32.072 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-30T15:03:32.072 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.072 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.072 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.072 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.072 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.072 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.072 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.072 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.072 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.072 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.073 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.073 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-30T15:03:32.073 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-30T15:03:32.073 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.073 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.073 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.073 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.073 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.073 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.073 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-30T15:03:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-30T15:03:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.074 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.075 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.075 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.075 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.075 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.075 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.075 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.075 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-30T15:03:32.075 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-30T15:03:32.075 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.075 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.075 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.076 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.076 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.076 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.076 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.076 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.076 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.076 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.076 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.076 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-30T15:03:32.076 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-30T15:03:32.076 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.077 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.077 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.077 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.077 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.077 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.077 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.077 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.077 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.077 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.077 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.077 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-30T15:03:32.078 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-30T15:03:32.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.078 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.078 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.078 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.078 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.078 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.078 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.078 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.078 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.078 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.079 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.079 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_ENABLED": { 2023-09-30T15:03:32.079 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-30T15:03:32.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.079 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.079 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.079 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.079 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.079 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.079 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.079 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.079 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.080 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.080 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.080 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-30T15:03:32.080 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-30T15:03:32.080 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.080 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.080 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.080 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-30T15:03:32.080 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.080 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.080 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.080 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.081 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.081 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.081 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.081 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-30T15:03:32.081 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-30T15:03:32.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.081 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.081 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.081 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.081 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.081 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.082 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.082 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.082 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.082 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.082 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.082 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-30T15:03:32.082 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-30T15:03:32.082 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.082 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.082 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.082 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8", 2023-09-30T15:03:32.083 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.083 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.083 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.083 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.083 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.083 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.083 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.083 INFO:teuthology.orchestra.run.smithi032.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-30T15:03:32.083 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-30T15:03:32.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.083 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.084 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.084 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "45", 2023-09-30T15:03:32.084 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.084 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.084 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.084 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.084 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.084 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.084 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.084 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ACCESS_KEY": { 2023-09-30T15:03:32.084 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-30T15:03:32.085 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.085 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.085 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.085 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.085 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.085 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.085 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.085 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.085 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.085 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.085 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.086 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.086 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-30T15:03:32.086 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-30T15:03:32.086 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.086 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.086 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.086 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-09-30T15:03:32.086 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.086 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.086 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.086 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.086 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.087 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.087 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.087 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SECRET_KEY": { 2023-09-30T15:03:32.087 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-30T15:03:32.087 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.087 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.087 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.087 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.087 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.087 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.087 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.088 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.088 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.088 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.088 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.088 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SSL_VERIFY": { 2023-09-30T15:03:32.088 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-30T15:03:32.088 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.088 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.088 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.088 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.088 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.089 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.089 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.089 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.089 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.089 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.089 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.089 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-30T15:03:32.089 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-30T15:03:32.089 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.089 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.089 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.089 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-09-30T15:03:32.090 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.090 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.090 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.090 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.090 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.090 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.090 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.090 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.090 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-30T15:03:32.090 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-30T15:03:32.090 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.090 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.091 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.091 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.091 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.091 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.091 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.091 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.091 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.091 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.091 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.091 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-30T15:03:32.091 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-30T15:03:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-09-30T15:03:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.092 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "cross_origin_url": { 2023-09-30T15:03:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cross_origin_url", 2023-09-30T15:03:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.093 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: "crt_file": { 2023-09-30T15:03:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crt_file", 2023-09-30T15:03:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.094 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: "debug": { 2023-09-30T15:03:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "debug", 2023-09-30T15:03:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.095 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.096 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable/disable debug options", 2023-09-30T15:03:32.096 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.096 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.096 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.096 INFO:teuthology.orchestra.run.smithi032.stdout: "jwt_token_ttl": { 2023-09-30T15:03:32.096 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "jwt_token_ttl", 2023-09-30T15:03:32.096 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.096 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.096 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.096 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "28800", 2023-09-30T15:03:32.096 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.097 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.097 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.097 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.097 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.097 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.097 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.097 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2023-09-30T15:03:32.097 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2023-09-30T15:03:32.097 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.097 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.097 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.098 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.098 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.098 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.098 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.098 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.098 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.098 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.098 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.098 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.098 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.098 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.099 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.099 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.099 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.099 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.099 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.099 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.099 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.099 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.099 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.099 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.099 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.099 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.100 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.101 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.102 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.103 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "motd": { 2023-09-30T15:03:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "motd", 2023-09-30T15:03:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.104 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "The message of the day", 2023-09-30T15:03:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.105 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect_resolve_ip_addr": { 2023-09-30T15:03:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "redirect_resolve_ip_addr", 2023-09-30T15:03:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.106 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-09-30T15:03:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-09-30T15:03:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2023-09-30T15:03:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.107 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-09-30T15:03:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-09-30T15:03:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8080", 2023-09-30T15:03:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.108 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2023-09-30T15:03:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2023-09-30T15:03:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.109 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl_server_port": { 2023-09-30T15:03:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl_server_port", 2023-09-30T15:03:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.110 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8443", 2023-09-30T15:03:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2023-09-30T15:03:32.111 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2023-09-30T15:03:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "redirect", 2023-09-30T15:03:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect" 2023-09-30T15:03:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.112 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2023-09-30T15:03:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2023-09-30T15:03:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2023-09-30T15:03:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2023-09-30T15:03:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2023-09-30T15:03:32.113 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "url_prefix": { 2023-09-30T15:03:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url_prefix", 2023-09-30T15:03:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.114 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.115 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2023-09-30T15:03:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_monitoring": { 2023-09-30T15:03:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_monitoring", 2023-09-30T15:03:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.116 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "monitor device health metrics", 2023-09-30T15:03:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.117 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.118 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.119 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.120 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.121 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.122 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mark_out_threshold": { 2023-09-30T15:03:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mark_out_threshold", 2023-09-30T15:03:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2419200", 2023-09-30T15:03:32.123 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-30T15:03:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": { 2023-09-30T15:03:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_name", 2023-09-30T15:03:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.124 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "device_health_metrics", 2023-09-30T15:03:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-30T15:03:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "retention_period": { 2023-09-30T15:03:32.125 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retention_period", 2023-09-30T15:03:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15552000", 2023-09-30T15:03:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain device health metrics", 2023-09-30T15:03:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_frequency": { 2023-09-30T15:03:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_frequency", 2023-09-30T15:03:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2023-09-30T15:03:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-30T15:03:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.127 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "self_heal": { 2023-09-30T15:03:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "self_heal", 2023-09-30T15:03:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-30T15:03:32.128 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-09-30T15:03:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-09-30T15:03:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-09-30T15:03:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.129 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and check device health", 2023-09-30T15:03:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_threshold": { 2023-09-30T15:03:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_threshold", 2023-09-30T15:03:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7257600", 2023-09-30T15:03:32.130 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-30T15:03:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "diskprediction_local", 2023-09-30T15:03:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.131 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.132 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.133 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.134 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.135 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.136 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.137 INFO:teuthology.orchestra.run.smithi032.stdout: "predict_interval": { 2023-09-30T15:03:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predict_interval", 2023-09-30T15:03:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2023-09-30T15:03:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.138 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "predictor_model": { 2023-09-30T15:03:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predictor_model", 2023-09-30T15:03:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "prophetstor", 2023-09-30T15:03:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.139 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-09-30T15:03:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-09-30T15:03:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-09-30T15:03:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "influx", 2023-09-30T15:03:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": false, 2023-09-30T15:03:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "influxdb python module not found", 2023-09-30T15:03:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "batch_size": { 2023-09-30T15:03:32.141 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "batch_size", 2023-09-30T15:03:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5000", 2023-09-30T15:03:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-30T15:03:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.142 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "database": { 2023-09-30T15:03:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "database", 2023-09-30T15:03:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "ceph", 2023-09-30T15:03:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-30T15:03:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.143 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": { 2023-09-30T15:03:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hostname", 2023-09-30T15:03:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server hostname", 2023-09-30T15:03:32.144 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-09-30T15:03:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-09-30T15:03:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "30", 2023-09-30T15:03:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "5", 2023-09-30T15:03:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.145 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-30T15:03:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.146 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.147 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.148 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.149 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.150 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.151 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "password": { 2023-09-30T15:03:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "password", 2023-09-30T15:03:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.152 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "password of InfluxDB server user", 2023-09-30T15:03:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "port": { 2023-09-30T15:03:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "port", 2023-09-30T15:03:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8086", 2023-09-30T15:03:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.153 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server port", 2023-09-30T15:03:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2023-09-30T15:03:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2023-09-30T15:03:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.154 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "false", 2023-09-30T15:03:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-30T15:03:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "threads": { 2023-09-30T15:03:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threads", 2023-09-30T15:03:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.155 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-09-30T15:03:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-09-30T15:03:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "32", 2023-09-30T15:03:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-30T15:03:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "username": { 2023-09-30T15:03:32.156 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "username", 2023-09-30T15:03:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "username of InfluxDB server user", 2023-09-30T15:03:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "verify_ssl": { 2023-09-30T15:03:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "verify_ssl", 2023-09-30T15:03:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "true", 2023-09-30T15:03:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-30T15:03:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.158 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "insights", 2023-09-30T15:03:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.159 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.160 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.161 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.162 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.163 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.164 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.165 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "iostat", 2023-09-30T15:03:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.166 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.167 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.168 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.169 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.170 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.171 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "k8sevents", 2023-09-30T15:03:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.172 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_event_retention_days": { 2023-09-30T15:03:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ceph_event_retention_days", 2023-09-30T15:03:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7", 2023-09-30T15:03:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-30T15:03:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.173 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "config_check_secs": { 2023-09-30T15:03:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_check_secs", 2023-09-30T15:03:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "10", 2023-09-30T15:03:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.174 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-30T15:03:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.175 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.176 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.177 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.178 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.179 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.180 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "localpool", 2023-09-30T15:03:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.181 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "failure_domain": { 2023-09-30T15:03:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "failure_domain", 2023-09-30T15:03:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "host", 2023-09-30T15:03:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "failure domain for any created local pool", 2023-09-30T15:03:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-30T15:03:32.182 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.183 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.184 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.186 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.186 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.186 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.186 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.186 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.186 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.186 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.186 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.186 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.186 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.186 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.186 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.187 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.187 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.187 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.187 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.187 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.187 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.187 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.187 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.187 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.187 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.187 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.188 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.188 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.188 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.188 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.188 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.188 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.188 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.188 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.188 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.188 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.188 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": { 2023-09-30T15:03:32.188 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_size", 2023-09-30T15:03:32.189 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.189 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.189 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.189 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.189 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.189 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.189 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.189 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default min_size for any created local pool", 2023-09-30T15:03:32.189 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-30T15:03:32.189 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.189 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.190 INFO:teuthology.orchestra.run.smithi032.stdout: "num_rep": { 2023-09-30T15:03:32.190 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "num_rep", 2023-09-30T15:03:32.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.190 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.190 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.190 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3", 2023-09-30T15:03:32.190 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.190 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.190 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.190 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default replica count for any created local pool", 2023-09-30T15:03:32.190 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.190 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.191 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.191 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.191 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": { 2023-09-30T15:03:32.191 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_num", 2023-09-30T15:03:32.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.191 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.191 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.191 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "128", 2023-09-30T15:03:32.191 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.191 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.191 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.191 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default pg_num for any created local pool", 2023-09-30T15:03:32.192 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.192 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.192 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.192 INFO:teuthology.orchestra.run.smithi032.stdout: "prefix": { 2023-09-30T15:03:32.192 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prefix", 2023-09-30T15:03:32.192 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.192 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.192 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.192 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.192 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.192 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.193 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.193 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name prefix for any created local pool", 2023-09-30T15:03:32.193 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.193 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.193 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.193 INFO:teuthology.orchestra.run.smithi032.stdout: "subtree": { 2023-09-30T15:03:32.193 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "subtree", 2023-09-30T15:03:32.193 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.193 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.193 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.193 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "rack", 2023-09-30T15:03:32.193 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.194 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.194 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.194 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-30T15:03:32.194 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-30T15:03:32.194 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.194 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.194 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.194 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.194 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.194 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mds_autoscaler", 2023-09-30T15:03:32.195 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.195 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.195 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.195 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.195 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.195 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.195 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.195 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.195 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.195 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.195 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.196 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.196 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.196 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.196 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.196 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.196 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.196 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.196 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.196 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.196 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.196 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.196 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.197 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.197 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.197 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.197 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.197 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.197 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.197 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.197 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.197 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.197 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.198 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.198 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.198 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.198 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.198 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.198 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.198 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.198 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.198 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.198 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.199 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.199 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.199 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.199 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.199 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.199 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.199 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.199 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.199 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.199 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.199 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.199 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.200 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.200 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.200 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.200 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.200 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.200 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.200 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.200 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.200 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.200 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.201 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.201 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.201 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.201 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.201 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirroring", 2023-09-30T15:03:32.201 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.201 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.201 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.201 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.202 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.202 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.202 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.202 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.202 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.202 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.202 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.202 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.202 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.202 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.202 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.202 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.203 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.203 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.203 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.203 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.203 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.203 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.203 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.203 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.203 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.203 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.204 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.204 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.204 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.204 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.204 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.204 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.204 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.204 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.204 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.204 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.204 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.205 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.205 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.205 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.205 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.205 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.205 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.205 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.205 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.205 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.205 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.205 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.205 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.206 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.206 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.206 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.206 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.206 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.206 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.206 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.206 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.206 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.207 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.207 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.207 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.207 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.207 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.207 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.207 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.207 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.209 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.209 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.209 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.209 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.209 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.209 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "nfs", 2023-09-30T15:03:32.209 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.209 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.209 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.210 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.210 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.210 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.210 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.210 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.210 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.210 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.210 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.210 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.210 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.210 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.211 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.211 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.211 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.211 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.211 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.211 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.211 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.211 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.211 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.211 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.211 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.212 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.212 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.212 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.212 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.212 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.212 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.212 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.212 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.212 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.212 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.213 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.213 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.213 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.213 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.213 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.213 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.213 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.213 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.213 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.213 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.213 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.213 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.214 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.214 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.214 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.214 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.214 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.214 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.214 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.214 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.214 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.215 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.215 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.215 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.215 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.215 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.215 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.215 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.215 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.215 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.215 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.216 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.216 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2023-09-30T15:03:32.216 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.216 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.216 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.216 INFO:teuthology.orchestra.run.smithi032.stdout: "fail_fs": { 2023-09-30T15:03:32.216 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "fail_fs", 2023-09-30T15:03:32.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.216 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.216 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.217 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.217 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.217 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.217 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.217 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-30T15:03:32.217 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.217 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.217 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.217 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.217 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.217 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.218 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.218 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.218 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.218 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.218 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.218 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.218 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.218 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.218 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.218 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.218 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.218 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.219 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.219 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.219 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.219 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.219 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.219 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.219 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.219 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.219 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.219 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.219 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.220 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.220 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.220 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.220 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.220 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.220 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.220 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.220 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.220 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.220 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.221 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.221 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.221 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.221 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.221 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.221 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.221 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.221 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.221 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.221 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.221 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.221 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.221 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.222 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.222 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.222 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.222 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.222 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.222 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.222 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.222 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.222 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.222 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.223 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.223 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.223 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.223 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.223 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.223 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.223 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.223 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator": { 2023-09-30T15:03:32.223 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2023-09-30T15:03:32.223 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.223 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.224 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.224 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.224 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.224 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.224 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.224 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2023-09-30T15:03:32.224 INFO:teuthology.orchestra.run.smithi032.stdout: "rook", 2023-09-30T15:03:32.224 INFO:teuthology.orchestra.run.smithi032.stdout: "test_orchestrator" 2023-09-30T15:03:32.224 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.224 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Orchestrator backend", 2023-09-30T15:03:32.224 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.224 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.225 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.225 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.225 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.225 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_perf_query", 2023-09-30T15:03:32.225 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.225 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.225 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.225 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.225 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.225 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.226 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.226 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.226 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.226 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.226 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.226 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.226 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.226 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.226 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.226 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.226 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.226 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.227 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.227 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.227 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.227 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.227 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.227 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.227 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.227 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.227 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.228 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.228 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.228 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.228 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.228 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.228 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.228 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.228 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.228 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.228 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.228 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.228 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.229 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.229 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.229 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.229 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.229 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.229 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.229 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.229 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.229 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.229 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.229 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.229 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.230 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.230 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.230 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.230 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.230 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.230 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.230 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.230 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.230 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.230 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.231 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.231 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.231 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.231 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.231 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.231 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.231 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.231 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.231 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_support", 2023-09-30T15:03:32.231 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.231 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.232 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.232 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.232 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.232 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.232 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.232 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.232 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.232 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.232 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.232 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.232 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.233 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.233 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.233 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.233 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.233 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.233 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.233 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.233 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.233 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.233 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.233 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.234 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.234 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.234 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.234 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.234 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.234 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.234 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.234 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.234 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.234 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.234 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.235 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.235 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.235 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.235 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.235 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.235 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.235 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.235 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.235 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.235 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.235 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.235 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.236 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.236 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.236 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.236 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.236 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.236 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.236 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.236 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.236 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.236 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.236 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.236 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.237 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.237 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.237 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.237 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.237 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.237 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.237 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.237 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.238 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_autoscaler", 2023-09-30T15:03:32.238 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.238 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.238 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.238 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.238 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.238 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.238 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.238 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.238 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.238 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.238 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.239 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.239 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.239 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.239 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.239 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.239 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.239 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.239 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.239 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.239 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.239 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.239 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.240 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.240 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.240 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.240 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.240 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.240 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.240 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.240 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.240 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.240 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.240 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.240 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.240 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.241 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.241 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.241 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.241 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.241 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.241 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.241 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.241 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.241 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.241 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.241 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.242 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.242 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.242 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.242 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.242 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.242 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.242 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.242 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.242 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.242 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.242 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.243 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.243 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.243 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.243 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.243 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.243 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.243 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.243 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.243 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.243 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.243 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.243 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-09-30T15:03:32.244 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-09-30T15:03:32.244 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.244 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.244 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.244 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-09-30T15:03:32.244 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.244 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.244 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.244 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.244 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.244 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.244 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.245 INFO:teuthology.orchestra.run.smithi032.stdout: "threshold": { 2023-09-30T15:03:32.245 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threshold", 2023-09-30T15:03:32.245 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:32.245 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.245 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.245 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2023-09-30T15:03:32.245 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1.0", 2023-09-30T15:03:32.245 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.245 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.245 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "scaling threshold", 2023-09-30T15:03:32.245 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-30T15:03:32.246 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.246 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.246 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.246 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.246 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "progress", 2023-09-30T15:03:32.246 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.247 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.247 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.247 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_pg_recovery_event": { 2023-09-30T15:03:32.247 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_pg_recovery_event", 2023-09-30T15:03:32.247 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.247 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.247 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.247 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.247 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.247 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.247 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.247 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-30T15:03:32.248 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.248 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.248 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.248 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2023-09-30T15:03:32.248 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2023-09-30T15:03:32.248 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.248 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.248 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.248 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.248 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.248 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.249 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.249 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.249 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.249 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.249 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.249 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.249 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.249 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.249 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.249 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.250 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.250 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.250 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.250 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.250 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.250 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.250 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.250 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.250 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.250 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.250 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.250 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.251 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.251 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.251 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.251 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.251 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.251 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.251 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.251 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.251 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.251 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.251 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.252 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.252 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.252 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.252 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.252 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.252 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.252 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.252 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.252 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.252 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.252 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.253 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.253 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.253 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.253 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.253 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.253 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.253 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.253 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.253 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.253 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.253 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.253 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.253 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.254 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.254 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.254 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.254 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.254 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.254 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.254 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.254 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.254 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.254 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.254 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.255 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.255 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.255 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.255 INFO:teuthology.orchestra.run.smithi032.stdout: "max_completed_events": { 2023-09-30T15:03:32.255 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_completed_events", 2023-09-30T15:03:32.255 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.255 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.255 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.255 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "50", 2023-09-30T15:03:32.255 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.255 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.256 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.256 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "number of past completed events to remember", 2023-09-30T15:03:32.256 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.256 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.256 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.256 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.256 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-09-30T15:03:32.256 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-09-30T15:03:32.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.256 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.256 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.257 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-09-30T15:03:32.257 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.257 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.257 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.257 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long the module is going to sleep", 2023-09-30T15:03:32.257 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.257 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.257 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.257 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.257 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.257 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.258 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus", 2023-09-30T15:03:32.258 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.258 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.258 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.258 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2023-09-30T15:03:32.258 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cache", 2023-09-30T15:03:32.258 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.258 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.258 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.258 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.258 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.258 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.259 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.259 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.259 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.259 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.259 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.259 INFO:teuthology.orchestra.run.smithi032.stdout: "exclude_perf_counters": { 2023-09-30T15:03:32.259 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "exclude_perf_counters", 2023-09-30T15:03:32.259 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.259 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.259 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.259 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.259 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.260 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.260 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.260 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-30T15:03:32.260 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-30T15:03:32.260 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.260 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.260 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.260 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.260 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.260 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.260 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.260 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.261 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.261 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.261 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.261 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.261 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.261 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.261 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.261 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.261 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.261 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.261 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.261 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.261 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.262 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.262 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.262 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.262 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.262 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.262 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.262 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.262 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.262 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.263 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.263 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.263 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.263 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.263 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.263 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.263 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.263 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.263 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.263 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.263 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.263 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.265 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.265 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.265 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.265 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.265 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.265 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.265 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.265 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.265 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.265 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.265 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.265 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.266 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.266 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.266 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.266 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools": { 2023-09-30T15:03:32.266 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools", 2023-09-30T15:03:32.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.266 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.266 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.266 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.266 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.266 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.267 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.267 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.267 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.267 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.267 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.267 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.267 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-30T15:03:32.267 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-30T15:03:32.267 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.267 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.267 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.267 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300", 2023-09-30T15:03:32.268 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.268 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.268 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.268 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.268 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.268 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.268 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.268 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_interval": { 2023-09-30T15:03:32.268 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_interval", 2023-09-30T15:03:32.268 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:32.268 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.268 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.269 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15.0", 2023-09-30T15:03:32.269 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.269 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.269 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.269 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.269 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.269 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.269 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.269 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-09-30T15:03:32.269 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-09-30T15:03:32.269 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.270 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.270 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.270 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2023-09-30T15:03:32.270 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.270 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.270 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.270 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-30T15:03:32.270 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.270 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.270 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.270 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.270 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-09-30T15:03:32.271 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-09-30T15:03:32.271 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.271 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.271 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.271 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "9283", 2023-09-30T15:03:32.271 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.271 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.271 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.271 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-30T15:03:32.271 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.271 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.271 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.272 INFO:teuthology.orchestra.run.smithi032.stdout: "stale_cache_strategy": { 2023-09-30T15:03:32.272 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stale_cache_strategy", 2023-09-30T15:03:32.272 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.272 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.272 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.272 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "log", 2023-09-30T15:03:32.272 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.272 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.272 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.272 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.272 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.273 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.273 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.273 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2023-09-30T15:03:32.273 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2023-09-30T15:03:32.273 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.273 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.273 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.273 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "default", 2023-09-30T15:03:32.273 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.273 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.273 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.274 INFO:teuthology.orchestra.run.smithi032.stdout: "default", 2023-09-30T15:03:32.274 INFO:teuthology.orchestra.run.smithi032.stdout: "error" 2023-09-30T15:03:32.274 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.274 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.274 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.274 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.274 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.274 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.274 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2023-09-30T15:03:32.274 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2023-09-30T15:03:32.274 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.274 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.274 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.275 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2023-09-30T15:03:32.275 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2023-09-30T15:03:32.275 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2023-09-30T15:03:32.275 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.275 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.275 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.275 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.275 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.275 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.275 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.275 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.276 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2023-09-30T15:03:32.276 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.276 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.276 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.276 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.276 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.276 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.276 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.276 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.276 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.276 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.276 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.276 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.277 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.277 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.277 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.277 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.277 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.277 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.277 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.277 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.277 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.277 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.277 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.277 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.277 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.278 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.278 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.278 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.278 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.278 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.278 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.278 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.278 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.278 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.278 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.278 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.279 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.279 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.279 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.279 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.279 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.279 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.279 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.279 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.279 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.279 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.279 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.279 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.280 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.280 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.280 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.280 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.280 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.280 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.280 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.280 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.280 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.280 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.280 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.281 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.281 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.281 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.281 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.281 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.281 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.281 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.281 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.281 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.281 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.281 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.281 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_snap_create": { 2023-09-30T15:03:32.282 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_snap_create", 2023-09-30T15:03:32.282 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.282 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.282 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.282 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.282 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.282 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.282 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.282 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.282 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.282 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.282 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.283 INFO:teuthology.orchestra.run.smithi032.stdout: "mirror_snapshot_schedule": { 2023-09-30T15:03:32.283 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirror_snapshot_schedule", 2023-09-30T15:03:32.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.283 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.283 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.283 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.283 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.283 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.283 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.283 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.283 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.283 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.284 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.284 INFO:teuthology.orchestra.run.smithi032.stdout: "trash_purge_schedule": { 2023-09-30T15:03:32.284 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "trash_purge_schedule", 2023-09-30T15:03:32.284 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.284 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.284 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.284 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.284 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.284 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.284 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.284 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.284 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.285 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.285 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.285 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.285 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.285 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.285 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "restful", 2023-09-30T15:03:32.285 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.285 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.285 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.285 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_auth": { 2023-09-30T15:03:32.285 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_auth", 2023-09-30T15:03:32.286 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.286 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.286 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.286 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.286 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.286 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.286 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.286 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.286 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.286 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.286 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.286 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2023-09-30T15:03:32.287 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2023-09-30T15:03:32.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.287 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.287 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.287 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.287 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.287 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.287 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.287 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.287 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.287 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.288 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.288 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.288 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.288 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.288 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.288 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.288 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.288 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.288 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.288 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.288 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.289 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.289 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.289 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.289 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.289 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.289 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.289 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.289 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.289 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.289 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.289 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.289 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.290 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.290 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.290 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.290 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.290 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.290 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.290 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.290 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.290 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.290 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.290 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.291 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.291 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.291 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.291 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.291 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.291 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.291 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.291 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.291 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.291 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.291 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.292 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.292 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.292 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.292 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.292 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.292 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.292 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.292 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.292 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.292 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.292 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.292 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.293 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.293 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.293 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.293 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.293 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.293 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.293 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.293 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-09-30T15:03:32.294 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-09-30T15:03:32.294 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.294 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.294 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.294 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.294 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.294 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.294 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.294 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.294 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.294 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.294 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.295 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.295 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-09-30T15:03:32.295 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-09-30T15:03:32.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.295 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.295 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.295 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.295 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.295 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.295 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.295 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.295 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rgw", 2023-09-30T15:03:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.297 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.298 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.300 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rook", 2023-09-30T15:03:32.302 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: "drive_group_interval": { 2023-09-30T15:03:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "drive_group_interval", 2023-09-30T15:03:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300.0", 2023-09-30T15:03:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.303 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-30T15:03:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.304 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.305 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.306 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.307 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.308 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "storage_class": { 2023-09-30T15:03:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "storage_class", 2023-09-30T15:03:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "local", 2023-09-30T15:03:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-30T15:03:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.310 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "selftest", 2023-09-30T15:03:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.311 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.312 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.313 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.314 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.315 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.316 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.317 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.317 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.317 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.317 INFO:teuthology.orchestra.run.smithi032.stdout: "roption1": { 2023-09-30T15:03:32.317 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption1", 2023-09-30T15:03:32.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.317 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.317 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.317 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.317 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.317 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.317 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: "roption2": { 2023-09-30T15:03:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption2", 2023-09-30T15:03:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "xyz", 2023-09-30T15:03:32.318 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption1": { 2023-09-30T15:03:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption1", 2023-09-30T15:03:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.319 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption2": { 2023-09-30T15:03:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption2", 2023-09-30T15:03:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.320 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption3": { 2023-09-30T15:03:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption3", 2023-09-30T15:03:32.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.322 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption4": { 2023-09-30T15:03:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption4", 2023-09-30T15:03:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption5": { 2023-09-30T15:03:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption5", 2023-09-30T15:03:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.324 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption6": { 2023-09-30T15:03:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption6", 2023-09-30T15:03:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.325 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption7": { 2023-09-30T15:03:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption7", 2023-09-30T15:03:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-09-30T15:03:32.326 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "42", 2023-09-30T15:03:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "testkey": { 2023-09-30T15:03:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testkey", 2023-09-30T15:03:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.327 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "testlkey": { 2023-09-30T15:03:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testlkey", 2023-09-30T15:03:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.328 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "testnewline": { 2023-09-30T15:03:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testnewline", 2023-09-30T15:03:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.329 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.330 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2023-09-30T15:03:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_m_granularity": { 2023-09-30T15:03:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_m_granularity", 2023-09-30T15:03:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.331 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow minute scheduled snapshots", 2023-09-30T15:03:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "dump_on_update": { 2023-09-30T15:03:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dump_on_update", 2023-09-30T15:03:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.332 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "dump database to debug log on update", 2023-09-30T15:03:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.333 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.334 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.335 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.336 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.337 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.338 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stats", 2023-09-30T15:03:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.339 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.340 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.341 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.342 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.343 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.344 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "status", 2023-09-30T15:03:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.345 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.346 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.347 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.348 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.349 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.350 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telegraf", 2023-09-30T15:03:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.351 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "address": { 2023-09-30T15:03:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "address", 2023-09-30T15:03:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-30T15:03:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.352 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-09-30T15:03:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-09-30T15:03:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15", 2023-09-30T15:03:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.353 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.354 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.355 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.356 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.357 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.358 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.359 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telemetry", 2023-09-30T15:03:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_basic": { 2023-09-30T15:03:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_basic", 2023-09-30T15:03:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.360 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-30T15:03:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_crash": { 2023-09-30T15:03:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_crash", 2023-09-30T15:03:32.361 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-30T15:03:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.362 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_device": { 2023-09-30T15:03:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_device", 2023-09-30T15:03:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-30T15:03:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_ident": { 2023-09-30T15:03:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_ident", 2023-09-30T15:03:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-30T15:03:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.364 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_perf": { 2023-09-30T15:03:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_perf", 2023-09-30T15:03:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-30T15:03:32.365 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "contact": { 2023-09-30T15:03:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "contact", 2023-09-30T15:03:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.366 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "description": { 2023-09-30T15:03:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "description", 2023-09-30T15:03:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.367 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "device_url": { 2023-09-30T15:03:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_url", 2023-09-30T15:03:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-30T15:03:32.368 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2023-09-30T15:03:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2023-09-30T15:03:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.369 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-09-30T15:03:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-09-30T15:03:32.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "24", 2023-09-30T15:03:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "8", 2023-09-30T15:03:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.371 INFO:teuthology.orchestra.run.smithi032.stdout: "last_opt_revision": { 2023-09-30T15:03:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "last_opt_revision", 2023-09-30T15:03:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1", 2023-09-30T15:03:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard": { 2023-09-30T15:03:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard", 2023-09-30T15:03:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.373 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard_description": { 2023-09-30T15:03:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard_description", 2023-09-30T15:03:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.374 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.375 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.376 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.377 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.378 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.380 INFO:teuthology.orchestra.run.smithi032.stdout: "organization": { 2023-09-30T15:03:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "organization", 2023-09-30T15:03:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "proxy": { 2023-09-30T15:03:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "proxy", 2023-09-30T15:03:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.382 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "url": { 2023-09-30T15:03:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url", 2023-09-30T15:03:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-30T15:03:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.383 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "test_orchestrator", 2023-09-30T15:03:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.384 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.385 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.386 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.387 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.388 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.389 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2023-09-30T15:03:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.390 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.391 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.392 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.393 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.394 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.395 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.396 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.396 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.396 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.396 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.396 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.396 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.396 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.396 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.396 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_clones": { 2023-09-30T15:03:32.396 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_clones", 2023-09-30T15:03:32.396 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.396 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.396 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.397 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4", 2023-09-30T15:03:32.397 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.397 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.397 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.397 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-30T15:03:32.397 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.397 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.397 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.397 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.397 INFO:teuthology.orchestra.run.smithi032.stdout: "periodic_async_work": { 2023-09-30T15:03:32.397 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "periodic_async_work", 2023-09-30T15:03:32.397 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.398 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.398 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.398 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.398 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.398 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.398 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.398 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Periodically check for async work", 2023-09-30T15:03:32.398 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.398 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.398 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.398 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.398 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_delay": { 2023-09-30T15:03:32.398 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_delay", 2023-09-30T15:03:32.399 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.399 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.399 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.399 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-09-30T15:03:32.399 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.399 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.399 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.399 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-30T15:03:32.399 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.399 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.399 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.399 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.400 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.400 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.400 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix", 2023-09-30T15:03:32.400 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.400 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.400 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.400 INFO:teuthology.orchestra.run.smithi032.stdout: "discovery_interval": { 2023-09-30T15:03:32.400 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "discovery_interval", 2023-09-30T15:03:32.400 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-09-30T15:03:32.400 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.400 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.400 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "100", 2023-09-30T15:03:32.401 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.401 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.401 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.401 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.401 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.401 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.401 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.401 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.401 INFO:teuthology.orchestra.run.smithi032.stdout: "identifier": { 2023-09-30T15:03:32.401 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "identifier", 2023-09-30T15:03:32.401 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.401 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.402 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.402 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.402 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.402 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.402 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.402 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.402 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.402 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.402 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.402 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.402 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-09-30T15:03:32.402 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-09-30T15:03:32.403 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.403 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.403 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.403 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-09-30T15:03:32.403 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.403 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.403 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.403 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.403 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.403 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.403 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.403 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.403 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.404 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.404 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.404 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.404 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.404 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.404 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.404 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.404 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.404 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.404 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.404 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.404 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.405 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.405 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.405 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.405 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.405 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.405 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.405 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.405 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.405 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.405 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.405 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.405 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.405 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.406 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.406 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.406 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.406 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.406 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.406 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.406 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.406 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.406 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.406 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.406 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.406 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.406 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.407 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.407 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.407 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.407 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.407 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.407 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.407 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.407 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.407 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.407 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.407 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.407 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.407 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.407 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.409 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.409 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.409 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.409 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.409 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.409 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.409 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.409 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.410 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.410 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.410 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.410 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.410 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.410 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.410 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.410 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.410 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.410 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_host": { 2023-09-30T15:03:32.410 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_host", 2023-09-30T15:03:32.411 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.411 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.411 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.411 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.411 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.411 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.411 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.411 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.411 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.411 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.411 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.411 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.411 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_port": { 2023-09-30T15:03:32.412 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_port", 2023-09-30T15:03:32.412 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.412 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.412 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.412 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10051", 2023-09-30T15:03:32.412 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.412 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.412 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.412 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.412 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.412 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.412 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.412 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.413 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_sender": { 2023-09-30T15:03:32.413 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_sender", 2023-09-30T15:03:32.413 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.413 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.413 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.413 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-30T15:03:32.413 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.413 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.413 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.413 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.413 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.413 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.414 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.414 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.414 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.414 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.414 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T15:03:32.414 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.414 INFO:teuthology.orchestra.run.smithi032.stdout: "gid": 24113, 2023-09-30T15:03:32.414 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "y", 2023-09-30T15:03:32.414 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_features": 4540138322906710015, 2023-09-30T15:03:32.414 INFO:teuthology.orchestra.run.smithi032.stdout: "available_modules": [ 2023-09-30T15:03:32.414 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.414 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "alerts", 2023-09-30T15:03:32.415 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.415 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.415 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.415 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-09-30T15:03:32.415 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-09-30T15:03:32.415 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.415 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.415 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.415 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-09-30T15:03:32.415 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.415 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.415 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.416 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How frequently to reexamine health status", 2023-09-30T15:03:32.416 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.416 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.416 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.416 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.416 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.416 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.416 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.416 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.416 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.416 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.416 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.416 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.417 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.417 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.417 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.417 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.417 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.417 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.417 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.417 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.417 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.417 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.417 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.417 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.417 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.418 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.418 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.418 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.418 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.418 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.418 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.418 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.418 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.418 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.418 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.418 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.418 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.419 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.419 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.419 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.419 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.419 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.419 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.419 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.419 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.419 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.419 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.419 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.419 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.420 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.420 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.420 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.420 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.420 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.420 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.420 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.420 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.420 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.420 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.420 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.420 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.420 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.421 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.421 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.421 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.421 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.421 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.421 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.421 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.421 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.421 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.421 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.421 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.421 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.422 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_destination": { 2023-09-30T15:03:32.422 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_destination", 2023-09-30T15:03:32.422 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.422 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.422 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.422 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.422 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.422 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.422 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.422 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email address to send alerts to", 2023-09-30T15:03:32.422 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.422 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.422 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.423 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.423 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_from_name": { 2023-09-30T15:03:32.423 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_from_name", 2023-09-30T15:03:32.423 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.423 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.423 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.423 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "Ceph", 2023-09-30T15:03:32.423 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.423 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.423 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.423 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email From: name", 2023-09-30T15:03:32.423 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.424 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.424 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.424 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.424 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_host": { 2023-09-30T15:03:32.424 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_host", 2023-09-30T15:03:32.424 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.424 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.424 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.424 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.424 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.424 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.424 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.424 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP server", 2023-09-30T15:03:32.425 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.425 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.425 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.425 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.425 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_password": { 2023-09-30T15:03:32.425 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_password", 2023-09-30T15:03:32.425 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.425 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.425 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.425 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.425 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.425 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.426 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.426 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Password to authenticate with", 2023-09-30T15:03:32.426 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.426 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.426 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.426 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.426 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_port": { 2023-09-30T15:03:32.426 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_port", 2023-09-30T15:03:32.426 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.426 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.426 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.426 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "465", 2023-09-30T15:03:32.427 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.427 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.427 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.427 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP port", 2023-09-30T15:03:32.427 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.427 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.427 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.427 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.427 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_sender": { 2023-09-30T15:03:32.427 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_sender", 2023-09-30T15:03:32.427 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.427 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.427 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.428 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.428 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.428 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.428 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.428 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP envelope sender", 2023-09-30T15:03:32.428 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.428 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.428 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.428 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.428 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_ssl": { 2023-09-30T15:03:32.428 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_ssl", 2023-09-30T15:03:32.428 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.429 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.429 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.429 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.429 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.429 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.429 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.429 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-30T15:03:32.429 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.429 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.429 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.429 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.429 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_user": { 2023-09-30T15:03:32.429 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_user", 2023-09-30T15:03:32.430 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.430 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.430 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.430 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.430 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.430 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.430 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.430 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "User to authenticate as", 2023-09-30T15:03:32.430 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.430 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.430 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.430 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.431 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.431 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.431 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.431 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "balancer", 2023-09-30T15:03:32.431 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.431 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.431 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.431 INFO:teuthology.orchestra.run.smithi032.stdout: "active": { 2023-09-30T15:03:32.431 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "active", 2023-09-30T15:03:32.431 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.431 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.431 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.431 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.432 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.432 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.432 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.432 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically balance PGs across cluster", 2023-09-30T15:03:32.432 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.432 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.432 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.432 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.432 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_time": { 2023-09-30T15:03:32.432 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_time", 2023-09-30T15:03:32.432 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.432 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.432 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.433 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0000", 2023-09-30T15:03:32.433 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.433 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.433 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.433 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "beginning time of day to automatically balance", 2023-09-30T15:03:32.433 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-30T15:03:32.433 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.433 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.433 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.433 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_weekday": { 2023-09-30T15:03:32.433 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_weekday", 2023-09-30T15:03:32.433 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-09-30T15:03:32.434 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.434 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.434 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-09-30T15:03:32.434 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2023-09-30T15:03:32.434 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2023-09-30T15:03:32.434 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.434 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-30T15:03:32.434 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-30T15:03:32.434 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.434 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.434 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.434 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_max_iterations": { 2023-09-30T15:03:32.435 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_max_iterations", 2023-09-30T15:03:32.435 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-09-30T15:03:32.435 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.435 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.435 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "25", 2023-09-30T15:03:32.435 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-09-30T15:03:32.435 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "250", 2023-09-30T15:03:32.435 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.435 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-30T15:03:32.435 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.435 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.435 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.435 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.436 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_metrics": { 2023-09-30T15:03:32.436 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_metrics", 2023-09-30T15:03:32.436 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.436 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.436 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.436 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "pgs,objects,bytes", 2023-09-30T15:03:32.436 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.436 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.436 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.436 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-30T15:03:32.436 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-30T15:03:32.436 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.437 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.437 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.437 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_step": { 2023-09-30T15:03:32.437 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_step", 2023-09-30T15:03:32.437 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:32.437 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.437 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.437 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.5", 2023-09-30T15:03:32.437 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0.001", 2023-09-30T15:03:32.437 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "0.999", 2023-09-30T15:03:32.437 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.437 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "aggressiveness of optimization", 2023-09-30T15:03:32.438 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-30T15:03:32.438 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.438 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.438 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.438 INFO:teuthology.orchestra.run.smithi032.stdout: "end_time": { 2023-09-30T15:03:32.438 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_time", 2023-09-30T15:03:32.438 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.438 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.438 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.438 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2359", 2023-09-30T15:03:32.438 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.438 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.439 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.439 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ending time of day to automatically balance", 2023-09-30T15:03:32.439 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-30T15:03:32.439 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.439 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.439 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.439 INFO:teuthology.orchestra.run.smithi032.stdout: "end_weekday": { 2023-09-30T15:03:32.439 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_weekday", 2023-09-30T15:03:32.439 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-09-30T15:03:32.439 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.439 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.440 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-09-30T15:03:32.440 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2023-09-30T15:03:32.440 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2023-09-30T15:03:32.440 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.440 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-30T15:03:32.440 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-30T15:03:32.440 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.440 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.440 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.440 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.441 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.441 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.441 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.441 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.441 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.441 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.441 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.442 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.442 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.442 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.442 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.442 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.442 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.443 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.443 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.443 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.443 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.443 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.443 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.443 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.443 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.443 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.443 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.444 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.444 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.444 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.444 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.444 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.444 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.444 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.444 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.444 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.444 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.444 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.444 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.444 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.445 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.445 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.445 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.445 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.445 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.445 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.445 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.445 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.445 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.445 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.445 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.445 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.446 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.446 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.446 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.446 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.446 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.446 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.446 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.446 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.446 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.446 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.446 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.446 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.446 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.447 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.447 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.447 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.447 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.447 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.447 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.447 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.447 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.447 INFO:teuthology.orchestra.run.smithi032.stdout: "min_score": { 2023-09-30T15:03:32.447 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_score", 2023-09-30T15:03:32.447 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:32.447 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.447 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.448 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-09-30T15:03:32.448 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.448 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.448 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.448 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-30T15:03:32.448 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.448 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.448 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.448 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.448 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2023-09-30T15:03:32.448 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2023-09-30T15:03:32.448 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.449 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.449 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.449 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "upmap", 2023-09-30T15:03:32.449 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.449 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.449 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.449 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-compat", 2023-09-30T15:03:32.449 INFO:teuthology.orchestra.run.smithi032.stdout: "none", 2023-09-30T15:03:32.449 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap" 2023-09-30T15:03:32.449 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.449 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Balancer mode", 2023-09-30T15:03:32.449 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.450 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.450 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.450 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_ids": { 2023-09-30T15:03:32.450 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_ids", 2023-09-30T15:03:32.450 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.450 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.450 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.450 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.450 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.450 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.450 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.451 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-30T15:03:32.451 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.451 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.451 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.451 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.451 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-09-30T15:03:32.451 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-09-30T15:03:32.451 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.451 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.451 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.451 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-09-30T15:03:32.451 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.452 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.452 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.452 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-30T15:03:32.452 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.452 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.452 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.452 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.452 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_deviation": { 2023-09-30T15:03:32.452 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_deviation", 2023-09-30T15:03:32.452 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.452 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.452 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.452 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-09-30T15:03:32.453 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-09-30T15:03:32.453 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.453 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.453 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-30T15:03:32.453 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-30T15:03:32.453 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.453 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.453 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.453 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_optimizations": { 2023-09-30T15:03:32.453 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_optimizations", 2023-09-30T15:03:32.453 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-09-30T15:03:32.453 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.454 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.454 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.454 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.454 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.454 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.454 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-30T15:03:32.454 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.454 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.454 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.454 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.454 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.454 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.455 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.455 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm", 2023-09-30T15:03:32.455 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.455 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.455 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.455 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_down_multiplier": { 2023-09-30T15:03:32.455 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_down_multiplier", 2023-09-30T15:03:32.455 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:32.455 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.455 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.455 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2023-09-30T15:03:32.455 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.455 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-30T15:03:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_refresh_rate": { 2023-09-30T15:03:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_refresh_rate", 2023-09-30T15:03:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "20", 2023-09-30T15:03:32.456 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.457 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.457 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.457 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-30T15:03:32.457 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.457 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.457 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.457 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.457 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_starting_port": { 2023-09-30T15:03:32.457 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_starting_port", 2023-09-30T15:03:32.457 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.457 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.457 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.458 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4721", 2023-09-30T15:03:32.458 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.458 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.458 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.458 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-30T15:03:32.458 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.458 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.458 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.458 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.458 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_ptrace": { 2023-09-30T15:03:32.458 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_ptrace", 2023-09-30T15:03:32.458 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.459 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.459 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.459 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.459 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.459 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.459 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.459 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-30T15:03:32.459 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-30T15:03:32.459 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.459 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.459 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.459 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_interval": { 2023-09-30T15:03:32.459 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_interval", 2023-09-30T15:03:32.460 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.460 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.460 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.460 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-09-30T15:03:32.460 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.460 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.460 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.460 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-30T15:03:32.460 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.460 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.460 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.460 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.460 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_memory_target_ratio": { 2023-09-30T15:03:32.461 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_memory_target_ratio", 2023-09-30T15:03:32.461 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:32.461 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.461 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.461 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.7", 2023-09-30T15:03:32.461 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.461 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.461 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.461 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-30T15:03:32.461 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.461 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.461 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.462 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.462 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm_log_destination": { 2023-09-30T15:03:32.462 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm_log_destination", 2023-09-30T15:03:32.462 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.462 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.462 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.462 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.462 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.462 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.462 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.462 INFO:teuthology.orchestra.run.smithi032.stdout: "file", 2023-09-30T15:03:32.462 INFO:teuthology.orchestra.run.smithi032.stdout: "file,syslog", 2023-09-30T15:03:32.462 INFO:teuthology.orchestra.run.smithi032.stdout: "syslog" 2023-09-30T15:03:32.463 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.463 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-09-30T15:03:32.463 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.463 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.463 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.463 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.463 INFO:teuthology.orchestra.run.smithi032.stdout: "cgroups_split": { 2023-09-30T15:03:32.463 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cgroups_split", 2023-09-30T15:03:32.463 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.463 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.463 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.463 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.464 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.464 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.464 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.464 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-30T15:03:32.464 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.464 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.464 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.464 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.464 INFO:teuthology.orchestra.run.smithi032.stdout: "config_checks_enabled": { 2023-09-30T15:03:32.464 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_checks_enabled", 2023-09-30T15:03:32.464 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.464 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.465 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.465 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.465 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.465 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.465 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.465 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-30T15:03:32.465 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.465 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.465 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.465 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.465 INFO:teuthology.orchestra.run.smithi032.stdout: "config_dashboard": { 2023-09-30T15:03:32.465 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_dashboard", 2023-09-30T15:03:32.465 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.466 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.466 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.466 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.466 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.466 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.466 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.466 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-30T15:03:32.466 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.466 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.466 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.466 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.466 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_alertmanager": { 2023-09-30T15:03:32.467 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_alertmanager", 2023-09-30T15:03:32.467 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.467 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.467 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.467 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-30T15:03:32.467 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.467 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.467 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.467 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-09-30T15:03:32.467 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.467 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.467 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.467 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.468 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_base": { 2023-09-30T15:03:32.468 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_base", 2023-09-30T15:03:32.468 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.468 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.468 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.468 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-30T15:03:32.468 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.468 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.468 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.468 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Container image name, without the tag", 2023-09-30T15:03:32.468 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.468 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.469 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.469 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.469 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_elasticsearch": { 2023-09-30T15:03:32.469 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_elasticsearch", 2023-09-30T15:03:32.469 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.469 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.469 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.469 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-30T15:03:32.469 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.469 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.469 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.469 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "elasticsearch container image", 2023-09-30T15:03:32.469 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.470 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.470 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.470 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.470 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_grafana": { 2023-09-30T15:03:32.470 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_grafana", 2023-09-30T15:03:32.470 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.470 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.470 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.470 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-30T15:03:32.470 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.470 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.470 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.471 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-09-30T15:03:32.471 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.471 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.471 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.471 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.471 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_haproxy": { 2023-09-30T15:03:32.471 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_haproxy", 2023-09-30T15:03:32.471 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.471 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.471 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.471 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-30T15:03:32.471 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.471 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.472 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.472 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "HAproxy container image", 2023-09-30T15:03:32.472 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.472 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.472 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.472 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.472 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_agent": { 2023-09-30T15:03:32.472 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_agent", 2023-09-30T15:03:32.472 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.472 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.472 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.472 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-30T15:03:32.473 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.473 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.473 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.473 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger agent container image", 2023-09-30T15:03:32.473 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.473 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.473 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.473 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.473 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_collector": { 2023-09-30T15:03:32.473 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_collector", 2023-09-30T15:03:32.473 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.473 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.473 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.474 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-30T15:03:32.474 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.474 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.474 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.474 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger collector container image", 2023-09-30T15:03:32.474 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.474 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.474 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.474 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.474 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_query": { 2023-09-30T15:03:32.474 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_query", 2023-09-30T15:03:32.474 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.475 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.475 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.475 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-30T15:03:32.475 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.475 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.475 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.475 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger query container image", 2023-09-30T15:03:32.475 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.475 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.475 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.475 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.475 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_keepalived": { 2023-09-30T15:03:32.476 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_keepalived", 2023-09-30T15:03:32.476 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.476 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.476 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.476 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-30T15:03:32.476 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.476 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.476 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.476 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Keepalived container image", 2023-09-30T15:03:32.476 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.476 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.476 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.476 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.477 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_loki": { 2023-09-30T15:03:32.477 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_loki", 2023-09-30T15:03:32.477 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.477 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.477 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.477 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-30T15:03:32.477 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.477 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.477 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.477 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Loki container image", 2023-09-30T15:03:32.477 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.477 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.478 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.478 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.478 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_node_exporter": { 2023-09-30T15:03:32.478 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_node_exporter", 2023-09-30T15:03:32.478 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.478 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.478 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.478 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-30T15:03:32.478 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.478 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.478 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.478 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-09-30T15:03:32.479 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.479 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.479 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.479 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.479 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nvmeof": { 2023-09-30T15:03:32.479 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nvmeof", 2023-09-30T15:03:32.479 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.479 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.479 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.479 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-30T15:03:32.479 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.479 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.479 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.480 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nvme-of container image", 2023-09-30T15:03:32.480 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.480 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.480 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.480 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.480 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_prometheus": { 2023-09-30T15:03:32.480 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_prometheus", 2023-09-30T15:03:32.480 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.480 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.480 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.480 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-30T15:03:32.480 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.481 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.481 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.481 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-09-30T15:03:32.481 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.481 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.481 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.481 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.481 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_promtail": { 2023-09-30T15:03:32.481 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_promtail", 2023-09-30T15:03:32.481 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.481 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.481 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.481 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-30T15:03:32.482 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.482 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.482 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.482 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Promtail container image", 2023-09-30T15:03:32.482 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.482 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.482 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.482 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.482 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_snmp_gateway": { 2023-09-30T15:03:32.482 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_snmp_gateway", 2023-09-30T15:03:32.482 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.482 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.483 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.483 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-30T15:03:32.483 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.483 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.483 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.483 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SNMP Gateway container image", 2023-09-30T15:03:32.483 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.483 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.483 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.483 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.483 INFO:teuthology.orchestra.run.smithi032.stdout: "container_init": { 2023-09-30T15:03:32.483 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_init", 2023-09-30T15:03:32.483 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.484 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.484 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.484 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.484 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.484 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.484 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.484 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Run podman/docker with `--init`", 2023-09-30T15:03:32.484 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.484 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.484 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.484 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.484 INFO:teuthology.orchestra.run.smithi032.stdout: "daemon_cache_timeout": { 2023-09-30T15:03:32.485 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "daemon_cache_timeout", 2023-09-30T15:03:32.485 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.485 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.485 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.485 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-09-30T15:03:32.485 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.485 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.485 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.485 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-30T15:03:32.485 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.485 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.485 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.485 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.486 INFO:teuthology.orchestra.run.smithi032.stdout: "default_cephadm_command_timeout": { 2023-09-30T15:03:32.486 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_cephadm_command_timeout", 2023-09-30T15:03:32.486 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.486 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.486 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.486 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "900", 2023-09-30T15:03:32.486 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.486 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.486 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.486 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-30T15:03:32.486 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.487 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.487 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.487 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.487 INFO:teuthology.orchestra.run.smithi032.stdout: "default_registry": { 2023-09-30T15:03:32.487 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_registry", 2023-09-30T15:03:32.487 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.487 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.487 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.487 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io", 2023-09-30T15:03:32.487 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.487 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.487 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.487 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-30T15:03:32.488 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.488 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.488 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.488 INFO:teuthology.orchestra.run.smithi032.stdout: "device_cache_timeout": { 2023-09-30T15:03:32.488 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_cache_timeout", 2023-09-30T15:03:32.488 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.488 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.488 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.488 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1800", 2023-09-30T15:03:32.488 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.488 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.489 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.489 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache device inventory", 2023-09-30T15:03:32.489 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.489 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.489 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.489 INFO:teuthology.orchestra.run.smithi032.stdout: "device_enhanced_scan": { 2023-09-30T15:03:32.489 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_enhanced_scan", 2023-09-30T15:03:32.489 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.489 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.489 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.489 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.489 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.490 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.490 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.490 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-30T15:03:32.490 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.490 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.490 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.490 INFO:teuthology.orchestra.run.smithi032.stdout: "facts_cache_timeout": { 2023-09-30T15:03:32.490 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "facts_cache_timeout", 2023-09-30T15:03:32.490 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.490 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.490 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.491 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-09-30T15:03:32.491 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.491 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.491 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.491 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache host facts data", 2023-09-30T15:03:32.491 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.491 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.491 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.491 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.491 INFO:teuthology.orchestra.run.smithi032.stdout: "host_check_interval": { 2023-09-30T15:03:32.491 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "host_check_interval", 2023-09-30T15:03:32.492 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.492 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.492 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.492 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-09-30T15:03:32.492 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.492 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.492 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.492 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to perform a host check", 2023-09-30T15:03:32.492 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.492 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.492 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.492 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.492 INFO:teuthology.orchestra.run.smithi032.stdout: "inventory_list_all": { 2023-09-30T15:03:32.493 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "inventory_list_all", 2023-09-30T15:03:32.493 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.493 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.493 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.493 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.493 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.493 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.493 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.493 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-30T15:03:32.493 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.493 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.493 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.494 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.494 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.494 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.494 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.494 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.494 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.494 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.494 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.494 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.494 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.494 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.494 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.495 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.495 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.495 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.495 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.495 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.495 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.495 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.495 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.495 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.495 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.495 INFO:teuthology.orchestra.run.smithi032.stdout: "log_refresh_metadata": { 2023-09-30T15:03:32.495 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_refresh_metadata", 2023-09-30T15:03:32.496 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.496 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.496 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.496 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.496 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.496 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.496 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.496 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-30T15:03:32.496 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.496 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.496 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.496 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.496 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.497 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.497 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.497 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.497 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.497 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.497 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.497 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.497 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.497 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-30T15:03:32.497 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.497 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.497 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.497 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.499 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.499 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.499 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.499 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.499 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.499 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.499 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.499 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.499 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.499 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.499 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.499 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.500 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.500 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.500 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.500 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.500 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.500 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.500 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.500 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.500 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.500 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.500 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.500 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-30T15:03:32.500 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-30T15:03:32.501 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.501 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.501 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.501 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.501 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.501 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.501 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.501 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-30T15:03:32.501 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.501 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.501 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.501 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.502 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-30T15:03:32.502 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-30T15:03:32.502 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.502 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.502 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.502 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "*", 2023-09-30T15:03:32.502 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.502 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.502 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.502 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-30T15:03:32.502 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.502 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.502 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.503 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.503 INFO:teuthology.orchestra.run.smithi032.stdout: "max_count_per_host": { 2023-09-30T15:03:32.503 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_count_per_host", 2023-09-30T15:03:32.503 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.503 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.503 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.503 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.503 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.503 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.503 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.503 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of daemons per service per host", 2023-09-30T15:03:32.503 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.504 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.504 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.504 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.504 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd_draining_count": { 2023-09-30T15:03:32.504 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_osd_draining_count", 2023-09-30T15:03:32.504 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.504 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.504 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.504 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.504 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.504 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.504 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.504 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-30T15:03:32.505 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.505 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.505 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.505 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.505 INFO:teuthology.orchestra.run.smithi032.stdout: "migration_current": { 2023-09-30T15:03:32.505 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "migration_current", 2023-09-30T15:03:32.505 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.505 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.505 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.505 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.505 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.506 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.506 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.506 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "internal - do not modify", 2023-09-30T15:03:32.506 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.506 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.506 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.506 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.506 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2023-09-30T15:03:32.506 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2023-09-30T15:03:32.506 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.506 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.506 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.507 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "root", 2023-09-30T15:03:32.507 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.507 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.507 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.507 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm-package", 2023-09-30T15:03:32.507 INFO:teuthology.orchestra.run.smithi032.stdout: "root" 2023-09-30T15:03:32.507 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.507 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "mode for remote execution of cephadm", 2023-09-30T15:03:32.507 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.507 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.507 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.507 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.507 INFO:teuthology.orchestra.run.smithi032.stdout: "prometheus_alerts_path": { 2023-09-30T15:03:32.508 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus_alerts_path", 2023-09-30T15:03:32.508 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.508 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.508 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.508 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-30T15:03:32.508 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.508 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.508 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.508 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-30T15:03:32.508 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.508 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.508 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.509 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.509 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_insecure": { 2023-09-30T15:03:32.509 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_insecure", 2023-09-30T15:03:32.509 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.509 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.509 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.509 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.509 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.509 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.509 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.509 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-30T15:03:32.509 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.509 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.510 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.510 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.510 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_password": { 2023-09-30T15:03:32.510 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_password", 2023-09-30T15:03:32.510 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.510 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.510 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.510 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.510 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.510 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.510 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.510 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-30T15:03:32.511 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.511 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.511 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.511 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.511 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_url": { 2023-09-30T15:03:32.511 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_url", 2023-09-30T15:03:32.511 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.511 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.511 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.511 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.511 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.511 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.512 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.512 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-30T15:03:32.512 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.512 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.512 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.512 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.512 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_username": { 2023-09-30T15:03:32.512 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_username", 2023-09-30T15:03:32.512 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.512 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.512 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.512 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.512 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.513 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.513 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.513 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-30T15:03:32.513 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.513 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.513 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.513 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.513 INFO:teuthology.orchestra.run.smithi032.stdout: "secure_monitoring_stack": { 2023-09-30T15:03:32.513 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secure_monitoring_stack", 2023-09-30T15:03:32.513 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.513 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.513 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.513 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.514 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.514 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.514 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.514 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-30T15:03:32.514 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.514 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.514 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.514 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.514 INFO:teuthology.orchestra.run.smithi032.stdout: "service_discovery_port": { 2023-09-30T15:03:32.514 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "service_discovery_port", 2023-09-30T15:03:32.514 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.514 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.515 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.515 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8765", 2023-09-30T15:03:32.515 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.515 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.515 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.515 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "cephadm service discovery port", 2023-09-30T15:03:32.515 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.515 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.515 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.515 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.515 INFO:teuthology.orchestra.run.smithi032.stdout: "ssh_config_file": { 2023-09-30T15:03:32.515 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssh_config_file", 2023-09-30T15:03:32.515 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.516 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.516 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.516 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.516 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.516 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.516 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.516 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-30T15:03:32.516 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.516 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.516 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.516 INFO:teuthology.orchestra.run.smithi032.stdout: "use_agent": { 2023-09-30T15:03:32.516 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_agent", 2023-09-30T15:03:32.517 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.517 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.517 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.517 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.517 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.517 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.517 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.517 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-30T15:03:32.517 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.517 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.517 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.517 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.518 INFO:teuthology.orchestra.run.smithi032.stdout: "use_repo_digest": { 2023-09-30T15:03:32.518 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_repo_digest", 2023-09-30T15:03:32.518 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.518 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.518 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.518 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.518 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.518 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.518 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.518 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-30T15:03:32.518 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.518 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.518 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.519 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_failed_host_check": { 2023-09-30T15:03:32.519 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_failed_host_check", 2023-09-30T15:03:32.519 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.519 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.519 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.519 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.519 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.519 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.519 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.519 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if the host check fails", 2023-09-30T15:03:32.520 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.520 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.520 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.520 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.520 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_daemons": { 2023-09-30T15:03:32.520 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_daemons", 2023-09-30T15:03:32.520 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.520 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.520 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.520 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.520 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.520 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.520 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.521 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-30T15:03:32.521 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.521 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.521 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.521 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.521 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_hosts": { 2023-09-30T15:03:32.521 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_hosts", 2023-09-30T15:03:32.521 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.521 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.521 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.521 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.521 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.521 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.522 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.522 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-30T15:03:32.522 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.522 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.522 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.522 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.522 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.522 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.522 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.522 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crash", 2023-09-30T15:03:32.522 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.522 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.523 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.523 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.523 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.523 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.523 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.523 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.523 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.523 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.523 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.523 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.523 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.523 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.524 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.524 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.524 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.524 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.524 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.524 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.524 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.524 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.524 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.524 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.524 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.524 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.525 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.525 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.525 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.525 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.525 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.525 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.525 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.525 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.525 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.525 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.525 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.525 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.525 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.526 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.526 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.526 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.526 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.526 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.526 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.526 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.526 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.526 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.526 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.526 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.526 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.527 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.527 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.527 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.527 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.527 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.527 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.527 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.527 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.527 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.527 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.527 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.527 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.528 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.528 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.528 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.528 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.528 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.528 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.528 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.528 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.528 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.528 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.528 INFO:teuthology.orchestra.run.smithi032.stdout: "retain_interval": { 2023-09-30T15:03:32.528 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retain_interval", 2023-09-30T15:03:32.528 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.529 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.529 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.529 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "31536000", 2023-09-30T15:03:32.529 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.529 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.529 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.529 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-30T15:03:32.529 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.529 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.529 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.529 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.529 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_recent_interval": { 2023-09-30T15:03:32.530 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_recent_interval", 2023-09-30T15:03:32.530 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.530 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.530 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.530 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1209600", 2023-09-30T15:03:32.530 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.530 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.530 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.530 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-30T15:03:32.530 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.530 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.530 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.531 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.531 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.531 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.531 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.531 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dashboard", 2023-09-30T15:03:32.531 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.531 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.531 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.531 INFO:teuthology.orchestra.run.smithi032.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-30T15:03:32.531 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-30T15:03:32.531 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.531 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.532 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.532 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.532 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.532 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.532 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.532 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.532 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.532 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.532 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.532 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.532 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_HOST": { 2023-09-30T15:03:32.532 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-30T15:03:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.533 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-30T15:03:32.534 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-30T15:03:32.534 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.534 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.534 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.534 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.534 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.534 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.534 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.534 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.534 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.534 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_ENABLED": { 2023-09-30T15:03:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_ENABLED", 2023-09-30T15:03:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.535 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.536 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.536 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.536 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.536 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-30T15:03:32.536 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-30T15:03:32.536 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.536 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.536 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.536 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.536 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.536 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.536 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: "ENABLE_BROWSABLE_API": { 2023-09-30T15:03:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-30T15:03:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.537 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-30T15:03:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-30T15:03:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.538 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.539 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.539 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.539 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.539 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.539 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.539 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.539 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.539 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.539 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-30T15:03:32.539 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-30T15:03:32.539 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.539 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.539 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.540 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.540 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.540 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.540 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.540 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.540 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.540 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.540 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.540 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.540 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-30T15:03:32.540 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-30T15:03:32.540 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.541 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.541 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.541 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.541 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.541 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.541 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.541 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.541 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.541 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.541 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.541 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.541 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-30T15:03:32.541 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-30T15:03:32.542 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.542 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.542 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.542 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.542 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.542 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.542 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.542 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.542 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.542 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.542 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.542 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.543 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-30T15:03:32.543 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-30T15:03:32.543 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.543 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.543 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.543 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.543 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.543 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.543 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.543 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.543 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.543 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.543 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.544 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.544 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-30T15:03:32.544 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-30T15:03:32.544 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.544 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.544 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.544 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.544 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.544 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.544 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.544 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.544 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-30T15:03:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-30T15:03:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.545 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.546 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.546 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.546 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.546 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.546 INFO:teuthology.orchestra.run.smithi032.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-30T15:03:32.546 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-30T15:03:32.546 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.546 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.546 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.546 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.546 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.546 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_PASSWORD": { 2023-09-30T15:03:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-30T15:03:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-09-30T15:03:32.547 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-30T15:03:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-30T15:03:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.548 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_URL": { 2023-09-30T15:03:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_URL", 2023-09-30T15:03:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.550 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.550 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.550 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.550 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.550 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.550 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.550 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.550 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.550 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.550 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.550 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.550 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_USERNAME": { 2023-09-30T15:03:32.551 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-30T15:03:32.551 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.551 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.551 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.551 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-09-30T15:03:32.551 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.551 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.551 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.551 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.551 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.551 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.551 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.551 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.552 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-30T15:03:32.552 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-30T15:03:32.552 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.552 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.552 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.552 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.552 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.552 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.552 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.552 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.552 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.552 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.553 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.553 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.553 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-30T15:03:32.553 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-30T15:03:32.553 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.553 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.553 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.553 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.553 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.553 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.553 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.553 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.553 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.554 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.554 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.554 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.554 INFO:teuthology.orchestra.run.smithi032.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-30T15:03:32.554 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-30T15:03:32.554 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.554 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.554 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.554 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.554 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.554 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.554 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.555 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.555 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.555 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.555 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.555 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.555 INFO:teuthology.orchestra.run.smithi032.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-30T15:03:32.555 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-30T15:03:32.555 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.555 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.555 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.555 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.555 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.555 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.556 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.556 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.556 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.556 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.556 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.556 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.556 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_HOST": { 2023-09-30T15:03:32.556 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-30T15:03:32.556 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.556 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.556 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.556 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.557 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.557 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.557 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.557 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.557 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.557 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.557 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.557 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.557 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-30T15:03:32.557 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-30T15:03:32.557 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.557 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.557 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-30T15:03:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-30T15:03:32.558 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-30T15:03:32.559 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-30T15:03:32.560 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.560 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.560 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.560 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.560 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.560 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.560 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.560 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.560 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.560 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.560 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.560 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-30T15:03:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-30T15:03:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.561 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-30T15:03:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-30T15:03:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.562 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.563 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.563 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.563 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-30T15:03:32.563 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-30T15:03:32.563 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.563 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.563 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.563 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.563 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.563 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.563 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.563 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.564 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.564 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.564 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.564 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.564 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-30T15:03:32.564 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-30T15:03:32.564 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.564 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.564 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.564 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.564 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.564 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-30T15:03:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-30T15:03:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.565 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.566 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.566 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.566 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.566 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.566 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.566 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.566 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.566 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_ENABLED": { 2023-09-30T15:03:32.566 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-30T15:03:32.566 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.566 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.566 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-30T15:03:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-30T15:03:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.567 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-30T15:03:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-30T15:03:32.568 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-30T15:03:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.569 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-30T15:03:32.570 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-30T15:03:32.570 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.570 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.570 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.570 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8", 2023-09-30T15:03:32.570 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.570 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.570 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.570 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.570 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.570 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.570 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-30T15:03:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-30T15:03:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "45", 2023-09-30T15:03:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.571 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.572 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.572 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.572 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ACCESS_KEY": { 2023-09-30T15:03:32.572 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-30T15:03:32.572 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.572 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.572 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.572 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.572 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.572 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.572 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.572 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-30T15:03:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-30T15:03:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-09-30T15:03:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.573 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.574 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.574 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.574 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.574 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.574 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.574 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SECRET_KEY": { 2023-09-30T15:03:32.574 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-30T15:03:32.574 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.574 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.574 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.574 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.574 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SSL_VERIFY": { 2023-09-30T15:03:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-30T15:03:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.575 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-30T15:03:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-30T15:03:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.576 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-09-30T15:03:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-30T15:03:32.577 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-30T15:03:32.578 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.578 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.578 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.578 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.578 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.578 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.578 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.578 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.578 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.578 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.578 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-30T15:03:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-30T15:03:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-09-30T15:03:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.579 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: "cross_origin_url": { 2023-09-30T15:03:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cross_origin_url", 2023-09-30T15:03:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.580 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: "crt_file": { 2023-09-30T15:03:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crt_file", 2023-09-30T15:03:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.581 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: "debug": { 2023-09-30T15:03:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "debug", 2023-09-30T15:03:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.582 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable/disable debug options", 2023-09-30T15:03:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: "jwt_token_ttl": { 2023-09-30T15:03:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "jwt_token_ttl", 2023-09-30T15:03:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.583 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "28800", 2023-09-30T15:03:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2023-09-30T15:03:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2023-09-30T15:03:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.584 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.585 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.586 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.586 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.586 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.586 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.586 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.586 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.586 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.586 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.586 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.586 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.586 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.586 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.587 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.587 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.587 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.587 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.587 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.587 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.587 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.587 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.587 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.587 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.587 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.587 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.587 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.588 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.588 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.588 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.588 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.588 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.588 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.588 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.588 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.588 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.588 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.588 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.588 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.589 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.590 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: "motd": { 2023-09-30T15:03:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "motd", 2023-09-30T15:03:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.591 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "The message of the day", 2023-09-30T15:03:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect_resolve_ip_addr": { 2023-09-30T15:03:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "redirect_resolve_ip_addr", 2023-09-30T15:03:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.592 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-09-30T15:03:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-09-30T15:03:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.593 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.594 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2023-09-30T15:03:32.594 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.594 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.594 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.594 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.594 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.594 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.594 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.594 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.594 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-09-30T15:03:32.594 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-09-30T15:03:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8080", 2023-09-30T15:03:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.595 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.596 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2023-09-30T15:03:32.596 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2023-09-30T15:03:32.596 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.596 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.596 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.596 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.596 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.596 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.596 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.596 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.596 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.596 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.596 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.597 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.597 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl_server_port": { 2023-09-30T15:03:32.597 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl_server_port", 2023-09-30T15:03:32.597 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.597 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.597 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.597 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8443", 2023-09-30T15:03:32.597 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.597 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.597 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.597 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.597 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.598 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.598 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.598 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.598 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2023-09-30T15:03:32.598 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2023-09-30T15:03:32.598 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.598 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.598 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.598 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "redirect", 2023-09-30T15:03:32.598 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.598 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.598 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect" 2023-09-30T15:03:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2023-09-30T15:03:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2023-09-30T15:03:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.599 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.600 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2023-09-30T15:03:32.600 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2023-09-30T15:03:32.600 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2023-09-30T15:03:32.600 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.600 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.600 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.600 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.600 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.600 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.600 INFO:teuthology.orchestra.run.smithi032.stdout: "url_prefix": { 2023-09-30T15:03:32.600 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url_prefix", 2023-09-30T15:03:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.601 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2023-09-30T15:03:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_monitoring": { 2023-09-30T15:03:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_monitoring", 2023-09-30T15:03:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.602 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "monitor device health metrics", 2023-09-30T15:03:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.603 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.604 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.605 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.605 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.605 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.605 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.605 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.605 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.605 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.605 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.605 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.605 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.605 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.605 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.606 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.607 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.607 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.607 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.607 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.607 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.607 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.607 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.607 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.607 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.607 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.607 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.607 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.608 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.608 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.608 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.608 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.608 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.608 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.608 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.608 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.608 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.608 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.608 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.608 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.609 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.609 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.609 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.609 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.609 INFO:teuthology.orchestra.run.smithi032.stdout: "mark_out_threshold": { 2023-09-30T15:03:32.609 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mark_out_threshold", 2023-09-30T15:03:32.609 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.609 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.609 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.609 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2419200", 2023-09-30T15:03:32.609 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-30T15:03:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": { 2023-09-30T15:03:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_name", 2023-09-30T15:03:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.610 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "device_health_metrics", 2023-09-30T15:03:32.611 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.611 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.611 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.611 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-30T15:03:32.611 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.611 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.611 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.613 INFO:teuthology.orchestra.run.smithi032.stdout: "retention_period": { 2023-09-30T15:03:32.613 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retention_period", 2023-09-30T15:03:32.613 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.613 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.613 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.613 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15552000", 2023-09-30T15:03:32.613 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.614 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.614 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.614 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain device health metrics", 2023-09-30T15:03:32.614 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.614 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.614 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.614 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_frequency": { 2023-09-30T15:03:32.614 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_frequency", 2023-09-30T15:03:32.614 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.614 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.614 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.615 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2023-09-30T15:03:32.615 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.615 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.615 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.615 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-30T15:03:32.615 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.615 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.615 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.615 INFO:teuthology.orchestra.run.smithi032.stdout: "self_heal": { 2023-09-30T15:03:32.615 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "self_heal", 2023-09-30T15:03:32.615 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.616 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.616 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.616 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.616 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.616 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.616 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.616 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-30T15:03:32.616 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.616 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.616 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.616 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-09-30T15:03:32.616 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-09-30T15:03:32.617 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.617 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.617 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.617 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-09-30T15:03:32.617 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.617 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.617 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.617 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and check device health", 2023-09-30T15:03:32.617 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.617 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.617 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.618 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_threshold": { 2023-09-30T15:03:32.618 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_threshold", 2023-09-30T15:03:32.618 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.618 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.618 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.618 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7257600", 2023-09-30T15:03:32.618 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.618 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.618 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.618 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-30T15:03:32.618 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.618 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.619 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.619 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "diskprediction_local", 2023-09-30T15:03:32.619 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.619 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.619 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.619 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.619 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.619 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.619 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.620 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.620 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.620 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.620 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.620 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.620 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.620 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.620 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.620 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.620 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.620 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.620 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.621 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.622 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.622 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.622 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.622 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.622 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.622 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.622 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.622 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.622 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.622 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.622 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.623 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.623 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.623 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.623 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.623 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.623 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.623 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.623 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.623 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.623 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.623 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.623 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.623 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.624 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.624 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.624 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.624 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.624 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.624 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.624 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.624 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.624 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.624 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.624 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.624 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.625 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.625 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.625 INFO:teuthology.orchestra.run.smithi032.stdout: "predict_interval": { 2023-09-30T15:03:32.625 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predict_interval", 2023-09-30T15:03:32.625 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.625 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.625 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.625 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2023-09-30T15:03:32.625 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.625 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.625 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.626 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.626 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.626 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.626 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.626 INFO:teuthology.orchestra.run.smithi032.stdout: "predictor_model": { 2023-09-30T15:03:32.626 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predictor_model", 2023-09-30T15:03:32.626 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.626 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.626 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.626 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "prophetstor", 2023-09-30T15:03:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-09-30T15:03:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-09-30T15:03:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.627 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-09-30T15:03:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.629 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "influx", 2023-09-30T15:03:32.629 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": false, 2023-09-30T15:03:32.629 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "influxdb python module not found", 2023-09-30T15:03:32.629 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.629 INFO:teuthology.orchestra.run.smithi032.stdout: "batch_size": { 2023-09-30T15:03:32.629 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "batch_size", 2023-09-30T15:03:32.629 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.629 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.629 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.629 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5000", 2023-09-30T15:03:32.629 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.629 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.630 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.630 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-30T15:03:32.630 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.630 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.630 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.630 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.630 INFO:teuthology.orchestra.run.smithi032.stdout: "database": { 2023-09-30T15:03:32.630 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "database", 2023-09-30T15:03:32.630 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.630 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.630 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.630 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "ceph", 2023-09-30T15:03:32.630 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.631 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.631 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.631 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-30T15:03:32.631 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.631 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.631 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.631 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.631 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": { 2023-09-30T15:03:32.631 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hostname", 2023-09-30T15:03:32.631 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.631 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.631 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server hostname", 2023-09-30T15:03:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-09-30T15:03:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-09-30T15:03:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.632 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.633 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.633 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "30", 2023-09-30T15:03:32.633 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "5", 2023-09-30T15:03:32.633 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.633 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.633 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-30T15:03:32.633 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.633 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.633 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.633 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.633 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.633 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.634 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.634 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.634 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.634 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.634 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.634 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.634 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.634 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.634 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.634 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.634 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.634 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.634 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.635 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.635 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.635 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.635 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.635 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.635 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.635 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.635 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.635 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.635 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.635 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.635 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.636 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.636 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.636 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.636 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.636 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.636 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.636 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.636 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.636 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.636 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.636 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.636 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.637 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.637 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.637 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.637 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.637 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.637 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.637 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.637 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.637 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.637 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.637 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.637 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.637 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.638 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.638 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.638 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.638 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.638 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.638 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.638 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.638 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.638 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.638 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.638 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.638 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.639 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.639 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.639 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.639 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.639 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.639 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.639 INFO:teuthology.orchestra.run.smithi032.stdout: "password": { 2023-09-30T15:03:32.639 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "password", 2023-09-30T15:03:32.639 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.639 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.639 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.639 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.639 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.640 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.640 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.640 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "password of InfluxDB server user", 2023-09-30T15:03:32.640 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.640 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.640 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.640 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.640 INFO:teuthology.orchestra.run.smithi032.stdout: "port": { 2023-09-30T15:03:32.640 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "port", 2023-09-30T15:03:32.640 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.640 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.640 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.641 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8086", 2023-09-30T15:03:32.641 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.641 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.641 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.641 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server port", 2023-09-30T15:03:32.641 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.641 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.641 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.641 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.641 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2023-09-30T15:03:32.641 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2023-09-30T15:03:32.642 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.642 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.642 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.642 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "false", 2023-09-30T15:03:32.642 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.642 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.642 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.642 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-30T15:03:32.642 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.642 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.642 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.642 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.643 INFO:teuthology.orchestra.run.smithi032.stdout: "threads": { 2023-09-30T15:03:32.643 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threads", 2023-09-30T15:03:32.643 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.643 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.643 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.643 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-09-30T15:03:32.643 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-09-30T15:03:32.643 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "32", 2023-09-30T15:03:32.643 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.643 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-30T15:03:32.643 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.643 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.644 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.644 INFO:teuthology.orchestra.run.smithi032.stdout: "username": { 2023-09-30T15:03:32.644 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "username", 2023-09-30T15:03:32.644 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.644 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.644 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.644 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.644 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.644 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.644 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.644 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "username of InfluxDB server user", 2023-09-30T15:03:32.645 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.645 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.645 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.645 INFO:teuthology.orchestra.run.smithi032.stdout: "verify_ssl": { 2023-09-30T15:03:32.645 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "verify_ssl", 2023-09-30T15:03:32.645 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.645 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.645 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.645 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "true", 2023-09-30T15:03:32.645 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.645 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.645 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-30T15:03:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "insights", 2023-09-30T15:03:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.646 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.647 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.648 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.648 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.648 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.648 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.648 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.648 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.648 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.648 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.648 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.648 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.648 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.649 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.649 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.649 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.649 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.649 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.649 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.649 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.649 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.649 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.649 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.649 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.649 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.650 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.650 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.650 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.650 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.650 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.650 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.650 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.650 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.650 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.650 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.650 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.650 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.651 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.651 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.651 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.651 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.651 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.651 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.651 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.651 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.651 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.651 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.651 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.651 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.651 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.652 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.652 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.652 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.652 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.652 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.652 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.652 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.652 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.652 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.652 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.652 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "iostat", 2023-09-30T15:03:32.653 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.653 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.653 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.653 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.653 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.653 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.653 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.653 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.653 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.653 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.653 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.653 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.654 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.654 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.654 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.654 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.654 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.654 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.654 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.654 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.654 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.654 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.654 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.654 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.655 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.655 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.655 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.655 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.655 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.655 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.655 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.655 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.655 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.655 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.655 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.655 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.656 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.656 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.656 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.656 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.656 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.656 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.656 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.656 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.656 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.656 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.656 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.656 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.656 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.657 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.657 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.657 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.657 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.657 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.657 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.657 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.657 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.657 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.657 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.657 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.658 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.658 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.658 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.658 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.658 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.658 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.658 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.658 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.658 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.658 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.658 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.658 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.659 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.659 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.659 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.659 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.659 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "k8sevents", 2023-09-30T15:03:32.659 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.659 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.659 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.659 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_event_retention_days": { 2023-09-30T15:03:32.659 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ceph_event_retention_days", 2023-09-30T15:03:32.659 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.659 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7", 2023-09-30T15:03:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-30T15:03:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: "config_check_secs": { 2023-09-30T15:03:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_check_secs", 2023-09-30T15:03:32.660 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.661 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.661 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.661 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.661 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "10", 2023-09-30T15:03:32.661 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.661 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.661 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-30T15:03:32.661 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.661 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.661 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.661 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.661 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.662 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.662 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.662 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.662 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.662 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.662 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.662 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.662 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.662 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.662 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.662 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.662 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.663 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.663 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.663 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.663 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.663 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.663 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.663 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.663 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.663 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.663 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.663 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.663 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.664 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.664 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.664 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.664 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.664 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.664 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.664 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.664 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.664 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.664 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.664 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.664 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.665 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.665 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.665 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.665 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.665 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.665 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.665 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.665 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.665 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.665 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.665 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.665 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.665 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.666 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.666 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.666 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.666 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.666 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.666 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.666 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.666 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.666 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.666 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.666 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.667 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "localpool", 2023-09-30T15:03:32.668 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.668 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.668 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.668 INFO:teuthology.orchestra.run.smithi032.stdout: "failure_domain": { 2023-09-30T15:03:32.668 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "failure_domain", 2023-09-30T15:03:32.668 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.668 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.668 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.668 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "host", 2023-09-30T15:03:32.668 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.668 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.668 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.668 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "failure domain for any created local pool", 2023-09-30T15:03:32.669 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-30T15:03:32.669 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.669 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.669 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.669 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.669 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.669 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.669 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.669 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.669 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.669 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.670 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.670 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.670 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.670 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.670 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.670 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.670 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.670 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.670 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.670 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.670 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.670 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.671 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.671 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.671 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.671 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.671 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.671 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.671 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.671 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.671 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.671 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.671 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.671 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.671 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.672 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.672 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.672 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.672 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.672 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.672 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.672 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.672 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.672 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.672 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.672 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.672 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.673 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.673 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.673 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.673 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.673 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.673 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.673 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.673 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.673 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.673 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.673 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.673 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.673 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.674 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.674 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.674 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.674 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.674 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.674 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.674 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.674 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.674 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.674 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.674 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.674 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.675 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": { 2023-09-30T15:03:32.675 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_size", 2023-09-30T15:03:32.675 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.675 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.675 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.675 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.675 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.675 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.675 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.675 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default min_size for any created local pool", 2023-09-30T15:03:32.675 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-30T15:03:32.675 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.676 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.676 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.676 INFO:teuthology.orchestra.run.smithi032.stdout: "num_rep": { 2023-09-30T15:03:32.676 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "num_rep", 2023-09-30T15:03:32.676 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.676 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.676 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.676 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3", 2023-09-30T15:03:32.676 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.676 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.676 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.676 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default replica count for any created local pool", 2023-09-30T15:03:32.676 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.677 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.677 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.677 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.677 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": { 2023-09-30T15:03:32.677 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_num", 2023-09-30T15:03:32.677 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.677 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.677 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.677 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "128", 2023-09-30T15:03:32.677 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.677 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.677 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.678 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default pg_num for any created local pool", 2023-09-30T15:03:32.678 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.678 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.678 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.678 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.678 INFO:teuthology.orchestra.run.smithi032.stdout: "prefix": { 2023-09-30T15:03:32.678 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prefix", 2023-09-30T15:03:32.678 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.678 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.678 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.678 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.678 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.679 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.679 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.679 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name prefix for any created local pool", 2023-09-30T15:03:32.679 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.679 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.679 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.679 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.679 INFO:teuthology.orchestra.run.smithi032.stdout: "subtree": { 2023-09-30T15:03:32.679 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "subtree", 2023-09-30T15:03:32.679 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.679 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.679 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.679 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "rack", 2023-09-30T15:03:32.680 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.680 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.680 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.680 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-30T15:03:32.680 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-30T15:03:32.680 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.680 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.680 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.680 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.680 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.680 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.680 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mds_autoscaler", 2023-09-30T15:03:32.681 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.681 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.681 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.681 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.681 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.681 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.681 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.681 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.681 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.681 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.681 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.681 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.681 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.682 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.682 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.682 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.682 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.682 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.682 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.682 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.682 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.682 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.682 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.682 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.682 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.683 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.683 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.683 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.683 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.683 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.683 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.683 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.683 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.683 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.683 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.683 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.683 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.683 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.684 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.684 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.684 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.684 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.684 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.684 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.684 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.684 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.684 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.684 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.684 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.684 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.685 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.685 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.685 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.685 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.685 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.685 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.685 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.685 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.685 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.685 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.685 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.685 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.685 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.686 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.686 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.686 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.686 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.686 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.686 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.686 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.686 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.686 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.686 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.686 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.686 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.687 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.687 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirroring", 2023-09-30T15:03:32.687 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.687 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.687 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.687 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.687 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.687 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.687 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.687 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.687 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.688 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.688 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.688 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.688 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.688 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.688 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.688 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.688 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.688 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.689 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.689 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.689 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.689 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.689 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.689 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.689 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.689 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.689 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.689 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.689 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.689 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.690 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.690 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.690 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.690 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.690 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.690 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.690 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.690 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.690 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.690 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.690 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.690 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.691 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.692 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.692 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.692 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.692 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.692 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.692 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.692 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.692 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.692 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.692 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.692 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.692 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.693 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.693 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.693 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.693 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.693 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.693 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.693 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.693 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.693 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.693 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "nfs", 2023-09-30T15:03:32.693 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.693 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.694 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.694 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.694 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.694 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.694 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.694 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.694 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.694 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.694 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.694 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.694 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.694 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.694 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.695 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.695 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.695 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.695 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.695 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.695 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.695 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.695 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.695 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.695 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.695 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.695 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.696 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.696 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.696 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.696 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.696 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.696 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.696 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.696 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.696 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.696 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.696 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.696 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.696 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.697 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.697 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.697 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.697 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.697 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.697 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.697 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.697 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.697 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.697 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.697 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.697 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.698 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.698 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.698 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.698 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.698 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.698 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.698 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.698 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.698 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.698 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.698 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.698 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.699 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.699 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.699 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.699 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.699 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.699 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.699 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.699 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.699 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.699 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.699 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2023-09-30T15:03:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "fail_fs": { 2023-09-30T15:03:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "fail_fs", 2023-09-30T15:03:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.700 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-30T15:03:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.701 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.702 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.702 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.702 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.702 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.702 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.702 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.702 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.702 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.702 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.702 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.702 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.702 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.703 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.703 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.703 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.703 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.703 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.703 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.703 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.703 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.703 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.703 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.703 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.703 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.703 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.704 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.704 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.704 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.704 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.704 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.704 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.704 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.704 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.704 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.704 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.705 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.706 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.706 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.706 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.706 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.706 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.706 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.706 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.706 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.706 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.706 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.706 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.706 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.707 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.707 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.707 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.707 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator": { 2023-09-30T15:03:32.707 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2023-09-30T15:03:32.707 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.707 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.707 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.707 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.707 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.707 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.707 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2023-09-30T15:03:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: "rook", 2023-09-30T15:03:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: "test_orchestrator" 2023-09-30T15:03:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Orchestrator backend", 2023-09-30T15:03:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.708 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_perf_query", 2023-09-30T15:03:32.709 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.709 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.709 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.709 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.709 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.709 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.709 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.709 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.709 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.709 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.709 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.709 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.710 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.710 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.710 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.710 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.710 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.710 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.710 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.710 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.710 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.710 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.710 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.710 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.711 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.712 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.713 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.714 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.714 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.714 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.714 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.714 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.714 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.714 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.714 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.714 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.714 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.714 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.715 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.715 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.715 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.715 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_support", 2023-09-30T15:03:32.715 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.715 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.715 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.715 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.715 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.715 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.715 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.716 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.717 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.718 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.718 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.718 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.718 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.718 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.718 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.718 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.718 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.718 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.718 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.718 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.718 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.719 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.720 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.720 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.720 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.720 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.720 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.720 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.720 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.720 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.720 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.720 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.720 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.720 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.721 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.721 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.721 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.721 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.721 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.721 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.721 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_autoscaler", 2023-09-30T15:03:32.721 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.721 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.721 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.721 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.722 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.723 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.723 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.723 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.723 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.723 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.723 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.723 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.723 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.723 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.723 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.723 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.723 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.724 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.725 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.726 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.726 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.726 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.726 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.726 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.726 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.726 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.726 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.726 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.726 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.726 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.726 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-09-30T15:03:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-09-30T15:03:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.727 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-09-30T15:03:32.728 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.728 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.728 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.728 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.728 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.728 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.728 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.728 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.728 INFO:teuthology.orchestra.run.smithi032.stdout: "threshold": { 2023-09-30T15:03:32.728 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threshold", 2023-09-30T15:03:32.728 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:32.728 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.729 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.729 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2023-09-30T15:03:32.729 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1.0", 2023-09-30T15:03:32.729 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.729 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.729 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "scaling threshold", 2023-09-30T15:03:32.729 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-30T15:03:32.729 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.729 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.729 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.729 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.730 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.730 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "progress", 2023-09-30T15:03:32.730 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.730 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.730 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.730 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_pg_recovery_event": { 2023-09-30T15:03:32.730 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_pg_recovery_event", 2023-09-30T15:03:32.730 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.730 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.730 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.730 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.730 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-30T15:03:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2023-09-30T15:03:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2023-09-30T15:03:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.731 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.732 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.733 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.734 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.735 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.735 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.735 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.735 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.735 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.735 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.735 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.735 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.735 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.735 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.735 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.735 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.736 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.737 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.738 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.738 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.738 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.738 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.738 INFO:teuthology.orchestra.run.smithi032.stdout: "max_completed_events": { 2023-09-30T15:03:32.738 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_completed_events", 2023-09-30T15:03:32.738 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.738 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.738 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.738 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "50", 2023-09-30T15:03:32.738 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "number of past completed events to remember", 2023-09-30T15:03:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-09-30T15:03:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-09-30T15:03:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.739 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-09-30T15:03:32.740 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.740 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.740 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.740 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long the module is going to sleep", 2023-09-30T15:03:32.740 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.740 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.740 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.740 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus", 2023-09-30T15:03:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2023-09-30T15:03:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cache", 2023-09-30T15:03:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "exclude_perf_counters": { 2023-09-30T15:03:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "exclude_perf_counters", 2023-09-30T15:03:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-30T15:03:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-30T15:03:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.743 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.744 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.744 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.744 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.744 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.744 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.744 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.744 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.744 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.744 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.744 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.744 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.744 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.745 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.746 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.746 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.746 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.746 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.746 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.746 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.746 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.746 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.746 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.746 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.746 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.747 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.747 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.747 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.747 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.747 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.747 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.747 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.747 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.747 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.747 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.747 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.748 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools": { 2023-09-30T15:03:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools", 2023-09-30T15:03:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.749 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.750 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.750 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-30T15:03:32.750 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-30T15:03:32.750 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.750 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.750 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.750 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300", 2023-09-30T15:03:32.750 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.750 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.750 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.750 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_interval": { 2023-09-30T15:03:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_interval", 2023-09-30T15:03:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15.0", 2023-09-30T15:03:32.751 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-09-30T15:03:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-09-30T15:03:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.752 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2023-09-30T15:03:32.753 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.753 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.753 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.753 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-30T15:03:32.753 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.753 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.753 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.753 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-09-30T15:03:32.753 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-09-30T15:03:32.753 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.753 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.754 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.754 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "9283", 2023-09-30T15:03:32.754 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.754 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.754 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.754 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-30T15:03:32.754 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.754 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.754 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.754 INFO:teuthology.orchestra.run.smithi032.stdout: "stale_cache_strategy": { 2023-09-30T15:03:32.754 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stale_cache_strategy", 2023-09-30T15:03:32.754 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.755 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.755 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.755 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "log", 2023-09-30T15:03:32.755 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.755 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.755 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.755 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.755 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.755 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.755 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.755 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.755 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2023-09-30T15:03:32.756 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2023-09-30T15:03:32.756 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.756 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.756 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.756 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "default", 2023-09-30T15:03:32.756 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.756 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.756 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.756 INFO:teuthology.orchestra.run.smithi032.stdout: "default", 2023-09-30T15:03:32.756 INFO:teuthology.orchestra.run.smithi032.stdout: "error" 2023-09-30T15:03:32.756 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.756 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2023-09-30T15:03:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2023-09-30T15:03:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2023-09-30T15:03:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2023-09-30T15:03:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2023-09-30T15:03:32.757 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.758 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.758 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.758 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.758 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.758 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.758 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.758 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.758 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2023-09-30T15:03:32.758 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.758 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.758 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.759 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.759 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.759 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.759 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.759 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.759 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.759 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.759 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.759 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.759 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.759 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.759 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.760 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.760 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.760 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.760 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.760 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.760 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.760 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.760 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.760 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.760 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.760 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.760 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.760 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.761 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.761 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.761 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.761 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.761 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.761 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.761 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.761 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.761 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.761 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.761 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.761 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.762 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.762 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.762 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.762 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.762 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.762 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.762 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.762 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.762 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.762 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.762 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.762 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.763 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.763 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.763 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.763 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.763 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.763 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.763 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.763 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.763 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.763 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.763 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.763 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.764 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.764 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.764 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.764 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.764 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.764 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.764 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.764 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.764 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.764 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_snap_create": { 2023-09-30T15:03:32.764 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_snap_create", 2023-09-30T15:03:32.764 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.765 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.765 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.765 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.765 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.765 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.765 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.765 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.765 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.765 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.765 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.765 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.765 INFO:teuthology.orchestra.run.smithi032.stdout: "mirror_snapshot_schedule": { 2023-09-30T15:03:32.766 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirror_snapshot_schedule", 2023-09-30T15:03:32.766 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.766 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.766 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.766 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.766 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.766 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.766 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.766 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.766 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.766 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.766 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.767 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.767 INFO:teuthology.orchestra.run.smithi032.stdout: "trash_purge_schedule": { 2023-09-30T15:03:32.767 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "trash_purge_schedule", 2023-09-30T15:03:32.767 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.767 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.767 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.767 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.767 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.767 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.767 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.767 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.767 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.767 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.768 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.768 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.768 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.768 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.768 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "restful", 2023-09-30T15:03:32.768 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.768 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.768 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.768 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_auth": { 2023-09-30T15:03:32.768 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_auth", 2023-09-30T15:03:32.768 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.769 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2023-09-30T15:03:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2023-09-30T15:03:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.770 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.771 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.772 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.772 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.772 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.772 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.772 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.772 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.772 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.772 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.772 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.772 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.772 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.772 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.773 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.774 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.775 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-09-30T15:03:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-09-30T15:03:32.776 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.777 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-09-30T15:03:32.778 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-09-30T15:03:32.778 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.778 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.778 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.778 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.778 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.778 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.778 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.778 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.778 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.778 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.778 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rgw", 2023-09-30T15:03:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.779 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.780 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.781 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.782 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.782 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.782 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.782 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.782 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.782 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.782 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.782 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.782 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.782 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.782 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.783 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.784 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.784 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.784 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.784 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.784 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.784 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.784 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.784 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.784 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.784 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.784 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rook", 2023-09-30T15:03:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.785 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: "drive_group_interval": { 2023-09-30T15:03:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "drive_group_interval", 2023-09-30T15:03:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300.0", 2023-09-30T15:03:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-30T15:03:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.787 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.787 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.787 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.787 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.787 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.787 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.787 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.787 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.787 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.787 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.787 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.788 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.788 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.788 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.788 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.788 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.788 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.788 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.788 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.788 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.788 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.788 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.788 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.789 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.790 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.790 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.790 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.790 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.790 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.790 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.790 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.790 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.790 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.790 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.790 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.790 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.791 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.792 INFO:teuthology.orchestra.run.smithi032.stdout: "storage_class": { 2023-09-30T15:03:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "storage_class", 2023-09-30T15:03:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "local", 2023-09-30T15:03:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-30T15:03:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.793 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.794 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.794 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.794 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.794 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "selftest", 2023-09-30T15:03:32.794 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.794 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.794 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.794 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.794 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.794 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.794 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.794 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.795 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.796 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.796 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.796 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.796 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.796 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.796 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.796 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.796 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.796 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.796 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.796 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.796 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.797 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.797 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.797 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.797 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.797 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.797 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.797 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.797 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.797 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.797 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.797 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.798 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.798 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.798 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.798 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.798 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.798 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.798 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.798 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.798 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.798 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.798 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.799 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.799 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.799 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.799 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.799 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.799 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.799 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.799 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.799 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.799 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: "roption1": { 2023-09-30T15:03:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption1", 2023-09-30T15:03:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.800 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.801 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.801 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.801 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.801 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.801 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.801 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.801 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.801 INFO:teuthology.orchestra.run.smithi032.stdout: "roption2": { 2023-09-30T15:03:32.801 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption2", 2023-09-30T15:03:32.801 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.801 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "xyz", 2023-09-30T15:03:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption1": { 2023-09-30T15:03:32.802 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption1", 2023-09-30T15:03:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.803 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption2": { 2023-09-30T15:03:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption2", 2023-09-30T15:03:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.804 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption3": { 2023-09-30T15:03:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption3", 2023-09-30T15:03:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.805 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption4": { 2023-09-30T15:03:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption4", 2023-09-30T15:03:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.806 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.807 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.807 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.807 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.807 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption5": { 2023-09-30T15:03:32.807 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption5", 2023-09-30T15:03:32.807 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.807 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.807 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.807 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.807 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.807 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption6": { 2023-09-30T15:03:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption6", 2023-09-30T15:03:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.808 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption7": { 2023-09-30T15:03:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption7", 2023-09-30T15:03:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.809 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-09-30T15:03:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "42", 2023-09-30T15:03:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: "testkey": { 2023-09-30T15:03:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testkey", 2023-09-30T15:03:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.810 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: "testlkey": { 2023-09-30T15:03:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testlkey", 2023-09-30T15:03:32.811 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.812 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.812 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.812 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.812 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.812 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.812 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.812 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.812 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.812 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.812 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.812 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.812 INFO:teuthology.orchestra.run.smithi032.stdout: "testnewline": { 2023-09-30T15:03:32.813 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testnewline", 2023-09-30T15:03:32.813 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.813 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.813 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.813 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.813 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.813 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.813 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.813 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.813 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.813 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.813 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.813 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.814 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.814 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.814 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.814 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2023-09-30T15:03:32.814 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.814 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.814 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.814 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_m_granularity": { 2023-09-30T15:03:32.814 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_m_granularity", 2023-09-30T15:03:32.814 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.814 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.815 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.815 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.815 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.815 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.815 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.815 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow minute scheduled snapshots", 2023-09-30T15:03:32.815 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.815 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.815 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.815 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.815 INFO:teuthology.orchestra.run.smithi032.stdout: "dump_on_update": { 2023-09-30T15:03:32.815 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dump_on_update", 2023-09-30T15:03:32.815 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.816 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.816 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.816 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.816 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.816 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.816 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.816 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "dump database to debug log on update", 2023-09-30T15:03:32.816 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.816 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.816 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.816 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.816 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.816 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.817 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.817 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.817 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.817 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.817 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.817 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.817 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.817 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.817 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.817 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.817 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.817 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.818 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.818 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.818 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.818 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.818 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.818 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.818 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.818 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.818 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.818 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.818 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.818 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.819 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.819 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.819 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.819 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.819 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.819 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.819 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.819 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.819 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.819 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.819 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.819 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.820 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.821 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.821 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.821 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.821 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.821 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.821 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.821 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.821 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.821 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.821 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.821 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.821 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.822 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.822 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.822 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.822 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.822 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.822 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.822 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.822 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.822 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.822 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stats", 2023-09-30T15:03:32.822 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.823 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.823 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.823 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.823 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.823 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.823 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.823 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.823 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.823 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.823 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.823 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.823 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.823 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.824 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.824 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.824 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.824 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.824 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.824 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.824 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.824 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.824 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.824 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.824 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.824 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.824 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.825 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.825 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.825 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.825 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.825 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.825 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.825 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.825 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.825 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.825 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.825 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.825 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.826 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.826 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.826 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.826 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.826 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.826 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.826 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.826 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.826 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.826 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.826 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.827 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.827 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.827 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.827 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.827 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.827 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.827 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.827 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.827 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.827 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.827 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.827 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.827 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.828 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.828 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.828 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.828 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.828 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.828 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.828 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.828 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.828 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.828 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.828 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.828 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.829 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.829 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "status", 2023-09-30T15:03:32.829 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.829 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.829 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.829 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.829 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.829 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.829 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.829 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.829 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.830 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.830 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.830 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.830 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.830 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.830 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.830 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.830 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.830 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.830 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.831 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.831 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.831 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.831 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.831 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.831 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.831 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.832 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.832 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.832 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.832 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.832 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.832 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.832 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.832 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.832 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.832 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.832 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.833 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.833 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.833 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.833 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.833 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.833 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.833 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.833 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.833 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.833 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.833 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.833 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.834 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.835 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telegraf", 2023-09-30T15:03:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: "address": { 2023-09-30T15:03:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "address", 2023-09-30T15:03:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.836 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-30T15:03:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-09-30T15:03:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-09-30T15:03:32.837 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.838 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.838 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.838 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15", 2023-09-30T15:03:32.838 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.838 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.838 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.838 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.838 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.838 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.838 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.838 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.839 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.839 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.839 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.839 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.839 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.839 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.839 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.839 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.839 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.839 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.839 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.839 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.840 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.840 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.840 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.840 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.840 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.840 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.840 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.840 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.840 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.840 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.840 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.840 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.840 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.841 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.841 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.841 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.841 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.841 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.841 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.841 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.841 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.841 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.841 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.841 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.842 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.842 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.842 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.842 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.842 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.842 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.842 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.842 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.842 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.842 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.842 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.842 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.843 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.843 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.843 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.843 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.843 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.843 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.843 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.843 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.843 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.843 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.843 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.844 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.844 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.844 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.844 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.844 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.844 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.844 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.844 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.844 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.844 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.845 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telemetry", 2023-09-30T15:03:32.845 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.845 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.845 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.845 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_basic": { 2023-09-30T15:03:32.845 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_basic", 2023-09-30T15:03:32.845 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.845 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.845 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.845 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.845 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.845 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.846 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.846 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-30T15:03:32.846 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.846 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.846 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.846 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_crash": { 2023-09-30T15:03:32.846 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_crash", 2023-09-30T15:03:32.846 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.846 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.846 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.846 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.847 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.847 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.847 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.847 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-30T15:03:32.847 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.847 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.847 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.847 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.847 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_device": { 2023-09-30T15:03:32.847 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_device", 2023-09-30T15:03:32.847 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.847 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.848 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.848 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.848 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.848 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.848 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.848 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-30T15:03:32.848 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.848 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.848 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.848 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.848 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_ident": { 2023-09-30T15:03:32.848 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_ident", 2023-09-30T15:03:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-30T15:03:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_perf": { 2023-09-30T15:03:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_perf", 2023-09-30T15:03:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-30T15:03:32.850 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: "contact": { 2023-09-30T15:03:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "contact", 2023-09-30T15:03:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.851 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: "description": { 2023-09-30T15:03:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "description", 2023-09-30T15:03:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.852 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: "device_url": { 2023-09-30T15:03:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_url", 2023-09-30T15:03:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.853 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-30T15:03:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2023-09-30T15:03:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2023-09-30T15:03:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.854 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-09-30T15:03:32.855 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-09-30T15:03:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "24", 2023-09-30T15:03:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "8", 2023-09-30T15:03:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: "last_opt_revision": { 2023-09-30T15:03:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "last_opt_revision", 2023-09-30T15:03:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1", 2023-09-30T15:03:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.857 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard": { 2023-09-30T15:03:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard", 2023-09-30T15:03:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.858 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard_description": { 2023-09-30T15:03:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard_description", 2023-09-30T15:03:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.859 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.860 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.861 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.862 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.863 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.864 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.864 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.864 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.864 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.864 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.864 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.864 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.864 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.864 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.864 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.864 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.864 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.865 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.865 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.865 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.865 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.865 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.865 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.865 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.865 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.865 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.865 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.865 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.865 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.866 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.866 INFO:teuthology.orchestra.run.smithi032.stdout: "organization": { 2023-09-30T15:03:32.866 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "organization", 2023-09-30T15:03:32.866 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.866 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.866 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.866 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.866 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.866 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.867 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.867 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.867 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.867 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.867 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.867 INFO:teuthology.orchestra.run.smithi032.stdout: "proxy": { 2023-09-30T15:03:32.867 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "proxy", 2023-09-30T15:03:32.867 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.867 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.867 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.867 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.868 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.868 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.868 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.868 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.868 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.868 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.868 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.868 INFO:teuthology.orchestra.run.smithi032.stdout: "url": { 2023-09-30T15:03:32.868 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url", 2023-09-30T15:03:32.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.869 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.869 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.869 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-30T15:03:32.869 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.869 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.869 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.869 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.869 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.869 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.869 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.870 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.870 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "test_orchestrator", 2023-09-30T15:03:32.870 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.870 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.870 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.870 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.870 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.870 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.870 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.870 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.871 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.871 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.871 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.871 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.871 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.871 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.871 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.871 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.871 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.871 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.871 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.871 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.872 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.872 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.872 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.872 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.872 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.872 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.872 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.872 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.872 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.874 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.874 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.874 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.874 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.874 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.874 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.874 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.874 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.874 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.874 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.874 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.875 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.875 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.875 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.875 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.875 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.875 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.875 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.875 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.875 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.875 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.875 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.876 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.876 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.876 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.876 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.876 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.876 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.876 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.876 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.876 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.876 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.876 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2023-09-30T15:03:32.876 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.877 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.877 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.877 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.877 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.877 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.877 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.877 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.877 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.877 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.877 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.877 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.877 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.878 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.878 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.878 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.878 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.878 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.878 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.878 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.878 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.878 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.878 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.878 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.879 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.879 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.879 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.879 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.879 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.879 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.879 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.879 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.879 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.879 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.879 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.880 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.880 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.880 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.880 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.880 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.880 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.880 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.880 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.880 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.880 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.880 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.881 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.881 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.881 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.881 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.881 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.881 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.881 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.881 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.881 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.881 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.881 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.882 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.882 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.882 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.882 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.882 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.882 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.882 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.882 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.882 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.882 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.882 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.883 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_clones": { 2023-09-30T15:03:32.883 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_clones", 2023-09-30T15:03:32.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.883 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.883 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.883 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4", 2023-09-30T15:03:32.883 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.883 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.883 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.883 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-30T15:03:32.883 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.883 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.884 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.884 INFO:teuthology.orchestra.run.smithi032.stdout: "periodic_async_work": { 2023-09-30T15:03:32.884 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "periodic_async_work", 2023-09-30T15:03:32.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.884 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.884 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.884 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.884 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.884 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.884 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.884 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Periodically check for async work", 2023-09-30T15:03:32.885 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.885 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.885 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.885 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_delay": { 2023-09-30T15:03:32.885 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_delay", 2023-09-30T15:03:32.885 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.885 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.885 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.885 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-09-30T15:03:32.885 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.885 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.885 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.886 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-30T15:03:32.886 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.886 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.886 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.886 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.886 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix", 2023-09-30T15:03:32.886 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.886 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.886 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.887 INFO:teuthology.orchestra.run.smithi032.stdout: "discovery_interval": { 2023-09-30T15:03:32.887 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "discovery_interval", 2023-09-30T15:03:32.887 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-09-30T15:03:32.887 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.887 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.887 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "100", 2023-09-30T15:03:32.887 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.887 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.887 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.887 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.887 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.887 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.888 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.888 INFO:teuthology.orchestra.run.smithi032.stdout: "identifier": { 2023-09-30T15:03:32.888 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "identifier", 2023-09-30T15:03:32.888 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.888 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.888 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.888 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.888 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.888 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.888 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.888 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.888 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.889 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.889 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.889 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.889 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-09-30T15:03:32.889 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-09-30T15:03:32.889 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.889 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.889 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.889 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-09-30T15:03:32.889 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.889 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.889 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.890 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.890 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.890 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.890 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.890 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.890 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.890 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.890 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.890 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.890 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.890 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.891 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.891 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.891 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.891 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.891 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.891 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.891 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.891 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.891 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.891 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.891 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.891 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.892 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.892 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.892 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.892 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.892 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.892 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.892 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.892 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.892 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.892 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.892 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.892 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.893 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.893 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.893 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.893 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.893 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.893 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.893 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.893 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.893 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.893 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.893 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.893 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.894 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.894 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.894 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.894 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.894 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.894 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.894 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.894 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.894 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.894 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.895 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.895 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.895 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.895 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.895 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.895 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.895 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.895 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.895 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.895 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.895 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.895 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.896 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.896 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_host": { 2023-09-30T15:03:32.896 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_host", 2023-09-30T15:03:32.896 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.896 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.896 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.896 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.896 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.896 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.896 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.896 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.897 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.897 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.897 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.897 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.897 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_port": { 2023-09-30T15:03:32.897 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_port", 2023-09-30T15:03:32.897 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.897 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.897 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.897 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10051", 2023-09-30T15:03:32.897 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.897 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.897 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.898 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.898 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.898 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.898 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.898 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_sender": { 2023-09-30T15:03:32.898 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_sender", 2023-09-30T15:03:32.898 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.898 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.898 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.898 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-30T15:03:32.899 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.899 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.899 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.899 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.899 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.899 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.899 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.899 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T15:03:32.899 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.900 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.900 INFO:teuthology.orchestra.run.smithi032.stdout: "modules": [ 2023-09-30T15:03:32.900 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2023-09-30T15:03:32.900 INFO:teuthology.orchestra.run.smithi032.stdout: "dashboard", 2023-09-30T15:03:32.900 INFO:teuthology.orchestra.run.smithi032.stdout: "iostat", 2023-09-30T15:03:32.900 INFO:teuthology.orchestra.run.smithi032.stdout: "nfs", 2023-09-30T15:03:32.900 INFO:teuthology.orchestra.run.smithi032.stdout: "restful" 2023-09-30T15:03:32.900 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.900 INFO:teuthology.orchestra.run.smithi032.stdout: "available_modules": [ 2023-09-30T15:03:32.900 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.900 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "alerts", 2023-09-30T15:03:32.900 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.900 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.901 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.901 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-09-30T15:03:32.901 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-09-30T15:03:32.901 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.901 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.901 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.901 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-09-30T15:03:32.901 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.901 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.901 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.901 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How frequently to reexamine health status", 2023-09-30T15:03:32.901 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.902 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.902 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.902 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.902 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.902 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.902 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.902 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.902 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.902 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.902 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.902 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.902 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.903 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.903 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.903 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.903 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.903 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.903 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.903 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.903 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.903 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.903 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.903 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.903 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.903 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.904 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.904 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.904 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.904 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.904 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.904 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.904 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.904 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.904 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.904 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.904 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.904 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.905 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.905 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.905 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.905 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.905 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.905 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.905 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.905 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.905 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.905 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.905 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.905 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.906 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.906 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.906 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.906 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.906 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.906 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.906 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.906 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.906 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.906 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.906 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.906 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.907 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.907 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.907 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.907 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.907 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.907 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.907 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.907 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.907 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.907 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.907 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.907 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.907 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_destination": { 2023-09-30T15:03:32.908 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_destination", 2023-09-30T15:03:32.908 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.908 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.908 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.908 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.908 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.908 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.908 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.908 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email address to send alerts to", 2023-09-30T15:03:32.908 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.908 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.908 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.909 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.909 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_from_name": { 2023-09-30T15:03:32.909 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_from_name", 2023-09-30T15:03:32.909 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.909 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.909 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.909 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "Ceph", 2023-09-30T15:03:32.909 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.909 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.909 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.909 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Email From: name", 2023-09-30T15:03:32.909 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.909 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.910 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.910 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.910 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_host": { 2023-09-30T15:03:32.910 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_host", 2023-09-30T15:03:32.910 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.910 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.910 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.910 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.910 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.910 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.910 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.910 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP server", 2023-09-30T15:03:32.911 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.911 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.911 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.911 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.911 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_password": { 2023-09-30T15:03:32.911 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_password", 2023-09-30T15:03:32.911 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.911 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.911 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.911 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.911 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.911 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.912 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.912 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Password to authenticate with", 2023-09-30T15:03:32.912 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.912 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.912 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.912 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.912 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_port": { 2023-09-30T15:03:32.912 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_port", 2023-09-30T15:03:32.912 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.912 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.912 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.912 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "465", 2023-09-30T15:03:32.912 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.913 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.913 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.913 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP port", 2023-09-30T15:03:32.913 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.913 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.913 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.913 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.913 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_sender": { 2023-09-30T15:03:32.913 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_sender", 2023-09-30T15:03:32.913 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.913 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.913 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.914 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.914 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.914 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.914 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.914 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SMTP envelope sender", 2023-09-30T15:03:32.914 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.914 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.914 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.914 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.914 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_ssl": { 2023-09-30T15:03:32.914 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_ssl", 2023-09-30T15:03:32.914 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.915 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.915 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.915 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.915 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.915 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.915 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.915 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-30T15:03:32.915 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.915 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.915 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.915 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.915 INFO:teuthology.orchestra.run.smithi032.stdout: "smtp_user": { 2023-09-30T15:03:32.915 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "smtp_user", 2023-09-30T15:03:32.916 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.916 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.916 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.916 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.916 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.916 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.916 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.916 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "User to authenticate as", 2023-09-30T15:03:32.916 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.916 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.916 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.916 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.917 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.917 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.917 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.917 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "balancer", 2023-09-30T15:03:32.917 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.917 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.917 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.917 INFO:teuthology.orchestra.run.smithi032.stdout: "active": { 2023-09-30T15:03:32.917 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "active", 2023-09-30T15:03:32.917 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.917 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.917 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.917 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.918 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.918 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.918 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.918 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically balance PGs across cluster", 2023-09-30T15:03:32.918 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.918 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.918 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.918 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.918 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_time": { 2023-09-30T15:03:32.918 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_time", 2023-09-30T15:03:32.918 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.918 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.919 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.919 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0000", 2023-09-30T15:03:32.919 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.919 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.919 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.919 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "beginning time of day to automatically balance", 2023-09-30T15:03:32.919 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-30T15:03:32.919 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.919 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.919 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.919 INFO:teuthology.orchestra.run.smithi032.stdout: "begin_weekday": { 2023-09-30T15:03:32.919 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "begin_weekday", 2023-09-30T15:03:32.920 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-09-30T15:03:32.920 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.920 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.920 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-09-30T15:03:32.920 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2023-09-30T15:03:32.920 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2023-09-30T15:03:32.920 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.920 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-30T15:03:32.920 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-30T15:03:32.920 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.920 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.920 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.920 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_max_iterations": { 2023-09-30T15:03:32.921 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_max_iterations", 2023-09-30T15:03:32.921 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-09-30T15:03:32.921 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.921 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.921 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "25", 2023-09-30T15:03:32.921 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-09-30T15:03:32.921 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "250", 2023-09-30T15:03:32.921 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.921 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-30T15:03:32.921 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.921 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.921 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.921 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.922 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_metrics": { 2023-09-30T15:03:32.922 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_metrics", 2023-09-30T15:03:32.922 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.922 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.922 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.922 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "pgs,objects,bytes", 2023-09-30T15:03:32.922 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.922 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.922 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.922 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-30T15:03:32.922 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-30T15:03:32.923 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.923 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.923 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.923 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_compat_step": { 2023-09-30T15:03:32.923 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crush_compat_step", 2023-09-30T15:03:32.923 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:32.923 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.923 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.923 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.5", 2023-09-30T15:03:32.923 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0.001", 2023-09-30T15:03:32.923 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "0.999", 2023-09-30T15:03:32.923 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.924 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "aggressiveness of optimization", 2023-09-30T15:03:32.924 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-30T15:03:32.924 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.924 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.924 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.924 INFO:teuthology.orchestra.run.smithi032.stdout: "end_time": { 2023-09-30T15:03:32.924 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_time", 2023-09-30T15:03:32.924 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.924 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.924 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.924 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2359", 2023-09-30T15:03:32.924 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.925 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.925 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.925 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ending time of day to automatically balance", 2023-09-30T15:03:32.925 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-30T15:03:32.925 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.925 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.925 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.925 INFO:teuthology.orchestra.run.smithi032.stdout: "end_weekday": { 2023-09-30T15:03:32.925 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "end_weekday", 2023-09-30T15:03:32.925 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-09-30T15:03:32.925 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.925 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.926 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-09-30T15:03:32.926 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "0", 2023-09-30T15:03:32.926 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "6", 2023-09-30T15:03:32.926 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.926 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-30T15:03:32.926 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-30T15:03:32.926 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.926 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.926 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.926 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.926 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.927 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.927 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.927 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.927 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.927 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.927 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.927 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.927 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.927 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.927 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.927 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.927 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.927 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.928 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.928 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.928 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.928 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.928 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.928 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.928 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.928 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.928 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.928 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.928 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.928 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.928 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.929 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.929 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.929 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.929 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.929 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.929 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.929 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.929 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.929 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.929 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.929 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.929 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.930 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.930 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.930 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.930 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.930 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.930 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.930 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.930 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.930 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.930 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.930 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.930 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.930 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.931 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.931 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.931 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.931 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.931 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.931 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.931 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.931 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.931 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.931 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.931 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.931 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.932 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.932 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.932 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.932 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.932 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.932 INFO:teuthology.orchestra.run.smithi032.stdout: "min_score": { 2023-09-30T15:03:32.932 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_score", 2023-09-30T15:03:32.932 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:32.932 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.932 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.932 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-09-30T15:03:32.932 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.932 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.933 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.933 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-30T15:03:32.933 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.933 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.933 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.933 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.933 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2023-09-30T15:03:32.933 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2023-09-30T15:03:32.933 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.933 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.933 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.933 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "upmap", 2023-09-30T15:03:32.933 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.934 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.934 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.934 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-compat", 2023-09-30T15:03:32.934 INFO:teuthology.orchestra.run.smithi032.stdout: "none", 2023-09-30T15:03:32.934 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap" 2023-09-30T15:03:32.934 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.934 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Balancer mode", 2023-09-30T15:03:32.934 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.934 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.934 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.934 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.934 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_ids": { 2023-09-30T15:03:32.935 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_ids", 2023-09-30T15:03:32.935 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.935 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.935 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.935 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.935 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.935 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.935 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.935 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-30T15:03:32.935 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.935 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.935 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.936 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.936 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-09-30T15:03:32.936 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-09-30T15:03:32.936 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.936 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.936 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.936 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-09-30T15:03:32.936 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.936 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.936 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.936 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-30T15:03:32.936 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.937 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.937 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.937 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.937 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_deviation": { 2023-09-30T15:03:32.937 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_deviation", 2023-09-30T15:03:32.937 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.937 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.937 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.937 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-09-30T15:03:32.937 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-09-30T15:03:32.937 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.937 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.938 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-30T15:03:32.938 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-30T15:03:32.938 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.938 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.938 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.938 INFO:teuthology.orchestra.run.smithi032.stdout: "upmap_max_optimizations": { 2023-09-30T15:03:32.938 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "upmap_max_optimizations", 2023-09-30T15:03:32.938 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-09-30T15:03:32.938 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.938 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.938 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.938 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.939 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.939 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.939 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-30T15:03:32.939 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.939 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.939 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.939 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.939 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:32.939 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.939 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:32.939 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm", 2023-09-30T15:03:32.939 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:32.940 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:32.940 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:32.940 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_down_multiplier": { 2023-09-30T15:03:32.940 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_down_multiplier", 2023-09-30T15:03:32.940 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:32.940 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.940 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.940 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2023-09-30T15:03:32.940 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.940 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.940 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.940 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-30T15:03:32.940 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.941 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.941 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.941 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.941 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_refresh_rate": { 2023-09-30T15:03:32.941 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_refresh_rate", 2023-09-30T15:03:32.941 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.941 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.941 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.941 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "20", 2023-09-30T15:03:32.941 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.941 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.941 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.942 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-30T15:03:32.942 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.942 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.942 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.942 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.942 INFO:teuthology.orchestra.run.smithi032.stdout: "agent_starting_port": { 2023-09-30T15:03:32.942 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "agent_starting_port", 2023-09-30T15:03:32.942 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.942 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.942 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.942 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4721", 2023-09-30T15:03:32.942 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.943 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.943 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.943 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-30T15:03:32.943 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.943 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.943 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.943 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.943 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_ptrace": { 2023-09-30T15:03:32.943 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_ptrace", 2023-09-30T15:03:32.943 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.943 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.943 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.943 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.944 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.944 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.944 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.944 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-30T15:03:32.944 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-30T15:03:32.944 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.944 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.944 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.944 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_interval": { 2023-09-30T15:03:32.944 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_interval", 2023-09-30T15:03:32.944 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.944 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.945 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.945 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-09-30T15:03:32.945 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.945 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.945 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.945 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-30T15:03:32.945 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.945 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.945 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.945 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.945 INFO:teuthology.orchestra.run.smithi032.stdout: "autotune_memory_target_ratio": { 2023-09-30T15:03:32.945 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "autotune_memory_target_ratio", 2023-09-30T15:03:32.946 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:32.946 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.946 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.946 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0.7", 2023-09-30T15:03:32.946 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.946 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.946 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.946 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-30T15:03:32.946 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.946 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.946 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.946 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.946 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm_log_destination": { 2023-09-30T15:03:32.947 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephadm_log_destination", 2023-09-30T15:03:32.947 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.947 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.947 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.947 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.947 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.947 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.947 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.947 INFO:teuthology.orchestra.run.smithi032.stdout: "file", 2023-09-30T15:03:32.947 INFO:teuthology.orchestra.run.smithi032.stdout: "file,syslog", 2023-09-30T15:03:32.947 INFO:teuthology.orchestra.run.smithi032.stdout: "syslog" 2023-09-30T15:03:32.947 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.948 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-09-30T15:03:32.948 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.948 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.948 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.948 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.948 INFO:teuthology.orchestra.run.smithi032.stdout: "cgroups_split": { 2023-09-30T15:03:32.948 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cgroups_split", 2023-09-30T15:03:32.948 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.948 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.948 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.948 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.948 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.949 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.949 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.949 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-30T15:03:32.949 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.949 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.949 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.949 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.949 INFO:teuthology.orchestra.run.smithi032.stdout: "config_checks_enabled": { 2023-09-30T15:03:32.949 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_checks_enabled", 2023-09-30T15:03:32.949 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.949 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.950 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.950 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.950 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.950 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.950 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.950 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-30T15:03:32.950 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.950 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.950 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.950 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.950 INFO:teuthology.orchestra.run.smithi032.stdout: "config_dashboard": { 2023-09-30T15:03:32.950 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_dashboard", 2023-09-30T15:03:32.951 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.951 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.951 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.951 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.951 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.951 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.951 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.951 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-30T15:03:32.951 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.951 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.951 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.951 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.952 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_alertmanager": { 2023-09-30T15:03:32.952 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_alertmanager", 2023-09-30T15:03:32.952 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.952 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.952 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.952 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-30T15:03:32.952 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.952 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.952 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.952 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-09-30T15:03:32.952 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.952 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.952 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.953 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.953 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_base": { 2023-09-30T15:03:32.953 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_base", 2023-09-30T15:03:32.953 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.953 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.953 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.953 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-30T15:03:32.953 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.953 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.953 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.953 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Container image name, without the tag", 2023-09-30T15:03:32.953 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.954 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.954 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.954 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.954 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_elasticsearch": { 2023-09-30T15:03:32.954 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_elasticsearch", 2023-09-30T15:03:32.954 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.954 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.954 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.954 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-30T15:03:32.954 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.954 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.954 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.955 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "elasticsearch container image", 2023-09-30T15:03:32.955 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.955 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.955 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.955 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.955 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_grafana": { 2023-09-30T15:03:32.955 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_grafana", 2023-09-30T15:03:32.955 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.955 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.955 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.955 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-30T15:03:32.955 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.956 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.956 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.956 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-09-30T15:03:32.956 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.956 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.956 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.956 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.956 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_haproxy": { 2023-09-30T15:03:32.956 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_haproxy", 2023-09-30T15:03:32.956 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.956 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.956 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.956 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-30T15:03:32.957 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.957 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.957 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.957 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "HAproxy container image", 2023-09-30T15:03:32.957 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.957 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.957 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.957 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.957 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_agent": { 2023-09-30T15:03:32.957 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_agent", 2023-09-30T15:03:32.957 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.957 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.958 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.958 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-30T15:03:32.958 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.958 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.958 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.958 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger agent container image", 2023-09-30T15:03:32.958 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.958 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.958 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.958 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.958 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_collector": { 2023-09-30T15:03:32.958 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_collector", 2023-09-30T15:03:32.959 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.959 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.959 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.959 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-30T15:03:32.959 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.959 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.959 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.959 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger collector container image", 2023-09-30T15:03:32.959 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.959 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.959 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.959 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.959 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_jaeger_query": { 2023-09-30T15:03:32.960 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_jaeger_query", 2023-09-30T15:03:32.960 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.960 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.960 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.960 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-30T15:03:32.960 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.960 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.960 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.960 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Jaeger query container image", 2023-09-30T15:03:32.960 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.960 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.960 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.961 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.961 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_keepalived": { 2023-09-30T15:03:32.961 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_keepalived", 2023-09-30T15:03:32.961 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.961 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.961 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.961 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-30T15:03:32.961 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.961 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.961 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.961 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Keepalived container image", 2023-09-30T15:03:32.961 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.962 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.962 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.962 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.962 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_loki": { 2023-09-30T15:03:32.962 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_loki", 2023-09-30T15:03:32.962 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.962 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.962 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.962 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-30T15:03:32.962 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.962 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.962 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.963 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Loki container image", 2023-09-30T15:03:32.963 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.963 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.963 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.963 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.963 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_node_exporter": { 2023-09-30T15:03:32.963 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_node_exporter", 2023-09-30T15:03:32.963 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.963 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.963 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.963 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-30T15:03:32.963 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.964 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.964 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.964 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-09-30T15:03:32.964 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.964 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.964 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.964 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.964 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_nvmeof": { 2023-09-30T15:03:32.964 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_nvmeof", 2023-09-30T15:03:32.964 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.964 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.964 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.964 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-30T15:03:32.965 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.965 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.965 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.965 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Nvme-of container image", 2023-09-30T15:03:32.965 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.965 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.965 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.965 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.965 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_prometheus": { 2023-09-30T15:03:32.965 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_prometheus", 2023-09-30T15:03:32.965 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.965 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.966 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.966 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-30T15:03:32.966 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.966 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.966 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.966 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Prometheus container image", 2023-09-30T15:03:32.966 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.966 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.966 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.966 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_promtail": { 2023-09-30T15:03:32.966 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_promtail", 2023-09-30T15:03:32.967 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.967 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.967 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.967 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-30T15:03:32.967 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.967 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.967 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.967 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Promtail container image", 2023-09-30T15:03:32.967 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.967 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.967 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.968 INFO:teuthology.orchestra.run.smithi032.stdout: "container_image_snmp_gateway": { 2023-09-30T15:03:32.968 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_image_snmp_gateway", 2023-09-30T15:03:32.968 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.968 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.968 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.968 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-30T15:03:32.968 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.968 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.968 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.968 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "SNMP Gateway container image", 2023-09-30T15:03:32.968 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.968 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.968 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.969 INFO:teuthology.orchestra.run.smithi032.stdout: "container_init": { 2023-09-30T15:03:32.969 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "container_init", 2023-09-30T15:03:32.969 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.969 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.969 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.969 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.969 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.969 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.969 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.969 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Run podman/docker with `--init`", 2023-09-30T15:03:32.969 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.970 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.970 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.970 INFO:teuthology.orchestra.run.smithi032.stdout: "daemon_cache_timeout": { 2023-09-30T15:03:32.970 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "daemon_cache_timeout", 2023-09-30T15:03:32.970 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.970 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.970 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.970 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-09-30T15:03:32.970 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.970 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.970 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-30T15:03:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "default_cephadm_command_timeout": { 2023-09-30T15:03:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_cephadm_command_timeout", 2023-09-30T15:03:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "900", 2023-09-30T15:03:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.972 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.972 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.972 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-30T15:03:32.972 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.972 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.972 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.972 INFO:teuthology.orchestra.run.smithi032.stdout: "default_registry": { 2023-09-30T15:03:32.972 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "default_registry", 2023-09-30T15:03:32.972 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.972 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.972 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.973 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "docker.io", 2023-09-30T15:03:32.973 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.973 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.973 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.973 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-30T15:03:32.973 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.973 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.973 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.973 INFO:teuthology.orchestra.run.smithi032.stdout: "device_cache_timeout": { 2023-09-30T15:03:32.973 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_cache_timeout", 2023-09-30T15:03:32.973 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.973 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.974 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.974 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1800", 2023-09-30T15:03:32.974 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.974 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.974 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.974 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache device inventory", 2023-09-30T15:03:32.974 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.974 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.974 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.974 INFO:teuthology.orchestra.run.smithi032.stdout: "device_enhanced_scan": { 2023-09-30T15:03:32.974 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_enhanced_scan", 2023-09-30T15:03:32.975 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.975 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.975 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.975 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.975 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.975 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.975 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.975 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-30T15:03:32.975 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.975 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.975 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.976 INFO:teuthology.orchestra.run.smithi032.stdout: "facts_cache_timeout": { 2023-09-30T15:03:32.976 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "facts_cache_timeout", 2023-09-30T15:03:32.976 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.976 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.976 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.976 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-09-30T15:03:32.976 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.976 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.976 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.976 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "seconds to cache host facts data", 2023-09-30T15:03:32.976 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.976 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.977 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.977 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.977 INFO:teuthology.orchestra.run.smithi032.stdout: "host_check_interval": { 2023-09-30T15:03:32.977 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "host_check_interval", 2023-09-30T15:03:32.977 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:32.977 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.977 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.977 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-09-30T15:03:32.977 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.977 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.977 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.977 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to perform a host check", 2023-09-30T15:03:32.978 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.978 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.978 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.978 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.978 INFO:teuthology.orchestra.run.smithi032.stdout: "inventory_list_all": { 2023-09-30T15:03:32.978 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "inventory_list_all", 2023-09-30T15:03:32.978 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.978 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.978 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.978 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.978 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.979 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.979 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.979 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-30T15:03:32.979 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.979 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.979 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.979 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.979 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:32.979 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:32.979 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.979 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.979 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.980 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.980 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.980 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.980 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.980 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.980 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.980 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.980 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.980 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.980 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.980 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.980 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.980 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.981 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.981 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.981 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.981 INFO:teuthology.orchestra.run.smithi032.stdout: "log_refresh_metadata": { 2023-09-30T15:03:32.981 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_refresh_metadata", 2023-09-30T15:03:32.981 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.981 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.981 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.981 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.981 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.981 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.981 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.982 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-30T15:03:32.982 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.982 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.982 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.982 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.982 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:32.982 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:32.982 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.982 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.982 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.982 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:32.982 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.983 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.983 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.983 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-30T15:03:32.983 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.983 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.983 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.983 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.983 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:32.983 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:32.983 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.983 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.983 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.984 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:32.984 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.984 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.984 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.984 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:32.984 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:32.984 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:32.984 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:32.984 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:32.984 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:32.984 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.984 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.985 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.985 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.985 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.985 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.985 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:32.985 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:32.985 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.985 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.985 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:32.985 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.985 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.985 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.985 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.986 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:32.986 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.986 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.986 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.986 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.986 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-30T15:03:32.986 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-30T15:03:32.986 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.986 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.986 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.986 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.986 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.987 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.987 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.987 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-30T15:03:32.987 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.987 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.987 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.987 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.987 INFO:teuthology.orchestra.run.smithi032.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-30T15:03:32.987 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-30T15:03:32.987 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.987 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.987 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.988 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "*", 2023-09-30T15:03:32.988 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.988 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.988 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.988 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-30T15:03:32.988 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.988 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.988 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.988 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.988 INFO:teuthology.orchestra.run.smithi032.stdout: "max_count_per_host": { 2023-09-30T15:03:32.988 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_count_per_host", 2023-09-30T15:03:32.988 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.988 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.989 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.989 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.989 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.989 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.989 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.989 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of daemons per service per host", 2023-09-30T15:03:32.989 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.989 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.989 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.989 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.989 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd_draining_count": { 2023-09-30T15:03:32.989 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_osd_draining_count", 2023-09-30T15:03:32.990 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.990 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.990 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.990 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:32.990 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.990 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.990 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.990 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-30T15:03:32.990 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.990 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.990 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.990 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.991 INFO:teuthology.orchestra.run.smithi032.stdout: "migration_current": { 2023-09-30T15:03:32.991 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "migration_current", 2023-09-30T15:03:32.991 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:32.991 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.991 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.991 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.991 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.991 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.991 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.991 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "internal - do not modify", 2023-09-30T15:03:32.991 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.992 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.992 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.992 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.992 INFO:teuthology.orchestra.run.smithi032.stdout: "mode": { 2023-09-30T15:03:32.992 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mode", 2023-09-30T15:03:32.992 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.992 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.992 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.992 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "root", 2023-09-30T15:03:32.992 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.992 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.992 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:32.992 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm-package", 2023-09-30T15:03:32.993 INFO:teuthology.orchestra.run.smithi032.stdout: "root" 2023-09-30T15:03:32.993 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:32.993 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "mode for remote execution of cephadm", 2023-09-30T15:03:32.993 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.993 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.993 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.993 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.993 INFO:teuthology.orchestra.run.smithi032.stdout: "prometheus_alerts_path": { 2023-09-30T15:03:32.993 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus_alerts_path", 2023-09-30T15:03:32.993 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.993 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.993 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.994 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-30T15:03:32.994 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.994 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.994 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.994 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-30T15:03:32.994 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.994 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.994 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.994 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.994 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_insecure": { 2023-09-30T15:03:32.994 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_insecure", 2023-09-30T15:03:32.995 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.995 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.995 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.995 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:32.995 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.995 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.995 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.995 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-30T15:03:32.995 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.995 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.995 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.995 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.996 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_password": { 2023-09-30T15:03:32.996 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_password", 2023-09-30T15:03:32.996 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.996 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.996 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.996 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.996 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.996 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.996 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.996 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-30T15:03:32.996 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.996 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.997 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.997 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.997 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_url": { 2023-09-30T15:03:32.997 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_url", 2023-09-30T15:03:32.997 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.997 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.997 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.997 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.997 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.997 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.997 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.997 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-30T15:03:32.998 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.998 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.998 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.998 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.998 INFO:teuthology.orchestra.run.smithi032.stdout: "registry_username": { 2023-09-30T15:03:32.998 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "registry_username", 2023-09-30T15:03:32.998 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:32.998 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.998 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:32.998 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:32.998 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:32.998 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:32.999 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:32.999 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-30T15:03:32.999 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:32.999 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:32.999 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:32.999 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:32.999 INFO:teuthology.orchestra.run.smithi032.stdout: "secure_monitoring_stack": { 2023-09-30T15:03:32.999 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "secure_monitoring_stack", 2023-09-30T15:03:32.999 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:32.999 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:32.999 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.000 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.000 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.000 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.000 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.000 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-30T15:03:33.000 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.000 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.000 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.000 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.000 INFO:teuthology.orchestra.run.smithi032.stdout: "service_discovery_port": { 2023-09-30T15:03:33.000 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "service_discovery_port", 2023-09-30T15:03:33.000 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.000 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.001 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.001 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8765", 2023-09-30T15:03:33.001 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.001 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.001 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.001 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "cephadm service discovery port", 2023-09-30T15:03:33.001 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.001 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.001 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.001 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.001 INFO:teuthology.orchestra.run.smithi032.stdout: "ssh_config_file": { 2023-09-30T15:03:33.001 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssh_config_file", 2023-09-30T15:03:33.002 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.002 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.002 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.002 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.002 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.002 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.002 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.002 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-30T15:03:33.002 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.002 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.002 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: "use_agent": { 2023-09-30T15:03:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_agent", 2023-09-30T15:03:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-30T15:03:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "use_repo_digest": { 2023-09-30T15:03:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "use_repo_digest", 2023-09-30T15:03:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-30T15:03:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_failed_host_check": { 2023-09-30T15:03:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_failed_host_check", 2023-09-30T15:03:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if the host check fails", 2023-09-30T15:03:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_daemons": { 2023-09-30T15:03:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_daemons", 2023-09-30T15:03:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.006 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-30T15:03:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_on_stray_hosts": { 2023-09-30T15:03:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_on_stray_hosts", 2023-09-30T15:03:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.007 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-30T15:03:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.008 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crash", 2023-09-30T15:03:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.009 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.010 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.011 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.012 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.013 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.014 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "retain_interval": { 2023-09-30T15:03:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retain_interval", 2023-09-30T15:03:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "31536000", 2023-09-30T15:03:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.015 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-30T15:03:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_recent_interval": { 2023-09-30T15:03:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_recent_interval", 2023-09-30T15:03:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1209600", 2023-09-30T15:03:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.016 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-30T15:03:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dashboard", 2023-09-30T15:03:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.017 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-30T15:03:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-30T15:03:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_HOST": { 2023-09-30T15:03:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-30T15:03:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.019 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-30T15:03:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-30T15:03:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.020 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_ENABLED": { 2023-09-30T15:03:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_ENABLED", 2023-09-30T15:03:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.021 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.022 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.022 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.022 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.022 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.022 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.022 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.022 INFO:teuthology.orchestra.run.smithi032.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-30T15:03:33.022 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-30T15:03:33.022 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.022 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.022 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.023 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.023 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.023 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.023 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.023 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.023 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.023 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.023 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.023 INFO:teuthology.orchestra.run.smithi032.stdout: "ENABLE_BROWSABLE_API": { 2023-09-30T15:03:33.023 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-30T15:03:33.023 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.023 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.024 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.025 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.025 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.025 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.025 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.025 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.025 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.026 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.026 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.026 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-30T15:03:33.026 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-30T15:03:33.026 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.026 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.026 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.026 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.026 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.026 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.026 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.027 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.027 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.027 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.027 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.027 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-30T15:03:33.027 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-30T15:03:33.027 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.027 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.027 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.027 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.027 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.027 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.028 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.028 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.028 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.028 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.028 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.028 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-30T15:03:33.028 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-30T15:03:33.028 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.028 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.028 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.028 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.028 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.029 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.029 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.029 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.029 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.029 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.029 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.029 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-30T15:03:33.029 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-30T15:03:33.029 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.029 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.029 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.029 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.030 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.030 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.030 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.030 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.030 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.030 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.030 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.030 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-30T15:03:33.030 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-30T15:03:33.030 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.030 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.031 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.031 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.031 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.031 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.031 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.031 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.031 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.031 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.031 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.031 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-30T15:03:33.031 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-30T15:03:33.031 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.031 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.032 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.032 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.032 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.032 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.032 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.032 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.032 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.032 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.032 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.032 INFO:teuthology.orchestra.run.smithi032.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-30T15:03:33.032 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-30T15:03:33.033 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.033 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.033 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.033 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.033 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.033 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.033 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.033 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.033 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.033 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.033 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.033 INFO:teuthology.orchestra.run.smithi032.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-30T15:03:33.034 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-30T15:03:33.034 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.034 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.034 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.034 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.034 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.034 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.034 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.034 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.034 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.034 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.034 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.035 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_PASSWORD": { 2023-09-30T15:03:33.035 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-30T15:03:33.035 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.035 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.035 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.035 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-09-30T15:03:33.035 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.035 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.035 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.035 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.035 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.035 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.036 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.036 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-30T15:03:33.036 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-30T15:03:33.036 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.036 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.036 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.036 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.036 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.036 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.036 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.036 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.036 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.037 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.037 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.037 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_URL": { 2023-09-30T15:03:33.037 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_URL", 2023-09-30T15:03:33.037 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.037 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.037 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.037 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.037 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.037 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.038 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.038 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.038 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.038 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.038 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.038 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_API_USERNAME": { 2023-09-30T15:03:33.038 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-30T15:03:33.038 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.038 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.038 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.038 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-09-30T15:03:33.039 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.039 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.039 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.039 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.039 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.039 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.039 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.039 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-30T15:03:33.039 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-30T15:03:33.039 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.039 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.039 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.040 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.040 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.040 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.040 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.040 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.040 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.040 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.040 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.040 INFO:teuthology.orchestra.run.smithi032.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-30T15:03:33.040 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-30T15:03:33.040 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.041 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.041 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.041 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.041 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.041 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.041 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.041 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.041 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.041 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.041 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.041 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.041 INFO:teuthology.orchestra.run.smithi032.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-30T15:03:33.042 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-30T15:03:33.042 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.042 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.042 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.042 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.042 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.042 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.042 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.042 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.042 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.042 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.042 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.042 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.043 INFO:teuthology.orchestra.run.smithi032.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-30T15:03:33.043 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-30T15:03:33.043 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.043 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.043 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.043 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.043 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.043 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.043 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.043 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.043 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.043 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.044 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.044 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.044 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_HOST": { 2023-09-30T15:03:33.044 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-30T15:03:33.044 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.044 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.044 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.044 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.044 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.044 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.044 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.044 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.045 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.045 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.045 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.045 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.045 INFO:teuthology.orchestra.run.smithi032.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-30T15:03:33.045 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-30T15:03:33.045 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.045 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.045 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.045 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.045 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.045 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.045 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.046 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.046 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.046 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.046 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.046 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.046 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-30T15:03:33.046 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-30T15:03:33.046 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.046 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.046 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.046 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.046 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.047 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.047 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.047 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.047 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.047 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.047 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.047 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.047 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-30T15:03:33.047 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-30T15:03:33.047 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.047 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.047 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.047 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.048 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.048 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.048 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.048 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.048 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.048 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.048 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.048 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.048 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-30T15:03:33.048 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-30T15:03:33.048 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.048 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.049 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.049 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.049 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.049 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.049 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.049 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.049 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.049 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.049 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.049 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.049 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-30T15:03:33.049 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-30T15:03:33.050 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.050 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.050 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.050 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.050 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.050 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.050 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.050 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.050 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.050 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.050 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.050 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.050 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-30T15:03:33.051 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-30T15:03:33.051 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.051 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.051 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.051 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.051 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.051 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.051 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.051 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.051 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.051 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.051 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.052 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.052 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-30T15:03:33.052 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-30T15:03:33.052 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.052 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.052 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.052 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.052 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.052 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.052 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.052 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.052 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.053 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.053 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.053 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.053 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-30T15:03:33.053 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-30T15:03:33.053 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.053 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.053 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.053 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.053 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.053 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.053 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.054 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.054 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.054 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.054 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.054 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_ENABLED": { 2023-09-30T15:03:33.054 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-30T15:03:33.054 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.054 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.054 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.054 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.054 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.055 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.055 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.055 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.055 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.055 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.055 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.055 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-30T15:03:33.055 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-30T15:03:33.055 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.055 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.055 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.056 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-30T15:03:33.056 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.056 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.056 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.056 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.056 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.056 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.056 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.056 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.056 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-30T15:03:33.056 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-30T15:03:33.056 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.057 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.057 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.057 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:33.057 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.057 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.057 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.057 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.057 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.057 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.057 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.057 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.057 INFO:teuthology.orchestra.run.smithi032.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-30T15:03:33.058 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-30T15:03:33.058 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.058 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.058 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.058 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8", 2023-09-30T15:03:33.058 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.058 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.058 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.058 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.058 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.058 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.058 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.058 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.059 INFO:teuthology.orchestra.run.smithi032.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-30T15:03:33.059 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-30T15:03:33.059 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.059 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.059 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.059 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "45", 2023-09-30T15:03:33.059 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.059 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.059 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.059 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.059 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.059 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.060 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.060 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.060 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ACCESS_KEY": { 2023-09-30T15:03:33.060 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-30T15:03:33.060 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.060 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.060 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.060 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.060 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.060 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.060 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.060 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.060 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.061 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.061 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.061 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.061 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-30T15:03:33.061 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-30T15:03:33.061 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.061 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.061 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.061 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "admin", 2023-09-30T15:03:33.061 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.061 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.061 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.062 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.062 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.062 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.062 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.062 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.062 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SECRET_KEY": { 2023-09-30T15:03:33.062 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-30T15:03:33.062 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.062 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.062 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.062 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.062 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.062 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.063 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.063 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.063 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.063 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.063 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.063 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.063 INFO:teuthology.orchestra.run.smithi032.stdout: "RGW_API_SSL_VERIFY": { 2023-09-30T15:03:33.063 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-30T15:03:33.063 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.063 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.063 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.063 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.064 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.064 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.064 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.064 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.064 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.064 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.064 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.064 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.064 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-30T15:03:33.064 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-30T15:03:33.064 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.064 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.064 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.065 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-09-30T15:03:33.065 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.065 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.065 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.065 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.065 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.065 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.065 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.065 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.065 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-30T15:03:33.065 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-30T15:03:33.065 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.065 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.066 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.066 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:33.066 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.066 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.066 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.066 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.066 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.066 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.066 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.066 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.066 INFO:teuthology.orchestra.run.smithi032.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-30T15:03:33.066 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-30T15:03:33.067 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.067 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.067 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.067 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-09-30T15:03:33.067 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.067 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.067 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.067 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.067 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.067 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.067 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.067 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.067 INFO:teuthology.orchestra.run.smithi032.stdout: "cross_origin_url": { 2023-09-30T15:03:33.068 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cross_origin_url", 2023-09-30T15:03:33.068 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.068 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.068 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.068 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.068 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.068 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.068 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.068 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.068 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.068 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.068 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.068 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.069 INFO:teuthology.orchestra.run.smithi032.stdout: "crt_file": { 2023-09-30T15:03:33.069 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "crt_file", 2023-09-30T15:03:33.069 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.069 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.069 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.069 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.069 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.069 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.069 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.069 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.069 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.069 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.070 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.070 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.070 INFO:teuthology.orchestra.run.smithi032.stdout: "debug": { 2023-09-30T15:03:33.070 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "debug", 2023-09-30T15:03:33.070 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.070 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.070 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.070 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.070 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.070 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.070 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.070 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Enable/disable debug options", 2023-09-30T15:03:33.070 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.071 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.071 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.071 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.071 INFO:teuthology.orchestra.run.smithi032.stdout: "jwt_token_ttl": { 2023-09-30T15:03:33.071 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "jwt_token_ttl", 2023-09-30T15:03:33.071 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.071 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.071 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.071 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "28800", 2023-09-30T15:03:33.071 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.071 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.071 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.072 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.072 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.072 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.072 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.072 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.072 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2023-09-30T15:03:33.072 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2023-09-30T15:03:33.072 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.072 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.072 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.072 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.072 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.072 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.073 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.073 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.073 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.073 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.073 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.073 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.073 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.073 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.073 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.073 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.073 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.073 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.074 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.074 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.074 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.074 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.074 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.074 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.074 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.074 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.074 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.074 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.074 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.074 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.074 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.075 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.075 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.075 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.075 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.075 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.075 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.075 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.075 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.075 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.075 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.075 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.075 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.076 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.076 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.076 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.076 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.076 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.076 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.076 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.076 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.076 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.076 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.076 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.076 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.076 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.077 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.077 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.077 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.077 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.077 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.077 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.077 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.077 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.077 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.077 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.077 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.077 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.078 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.078 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.078 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.078 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.078 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.078 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.078 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.078 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.078 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.078 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.078 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.078 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.079 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.079 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.079 INFO:teuthology.orchestra.run.smithi032.stdout: "motd": { 2023-09-30T15:03:33.079 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "motd", 2023-09-30T15:03:33.079 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.079 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.079 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.079 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.079 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.079 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.079 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.079 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "The message of the day", 2023-09-30T15:03:33.079 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.080 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.080 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.080 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.080 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect_resolve_ip_addr": { 2023-09-30T15:03:33.080 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "redirect_resolve_ip_addr", 2023-09-30T15:03:33.080 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.080 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.080 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.080 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.080 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.080 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.080 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.080 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.081 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.081 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.081 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.081 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.081 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-09-30T15:03:33.081 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-09-30T15:03:33.081 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.081 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.081 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.081 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2023-09-30T15:03:33.081 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.081 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.082 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.082 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.082 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.082 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.082 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.082 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.082 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-09-30T15:03:33.082 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-09-30T15:03:33.082 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.082 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.082 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.082 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8080", 2023-09-30T15:03:33.083 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.083 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.083 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.083 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.083 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.083 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.083 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.083 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.083 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2023-09-30T15:03:33.083 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2023-09-30T15:03:33.083 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.083 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.083 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.084 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.084 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.084 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.084 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.084 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.084 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.084 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.084 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.084 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.084 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl_server_port": { 2023-09-30T15:03:33.084 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl_server_port", 2023-09-30T15:03:33.084 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.085 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.085 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.085 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8443", 2023-09-30T15:03:33.085 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.085 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.085 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.085 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.085 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.085 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.085 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.085 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.085 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2023-09-30T15:03:33.085 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2023-09-30T15:03:33.086 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.086 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.086 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.086 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "redirect", 2023-09-30T15:03:33.086 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.086 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.086 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.086 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.086 INFO:teuthology.orchestra.run.smithi032.stdout: "redirect" 2023-09-30T15:03:33.086 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.086 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.086 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.087 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.087 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.087 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.087 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2023-09-30T15:03:33.087 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2023-09-30T15:03:33.087 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.087 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.087 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.087 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2023-09-30T15:03:33.087 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2023-09-30T15:03:33.087 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2023-09-30T15:03:33.087 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.088 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.088 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.088 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.088 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.088 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.088 INFO:teuthology.orchestra.run.smithi032.stdout: "url_prefix": { 2023-09-30T15:03:33.088 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url_prefix", 2023-09-30T15:03:33.088 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.088 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.088 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.088 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.088 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.089 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.089 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.089 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.089 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.089 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.089 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.089 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.089 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.089 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.089 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.089 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2023-09-30T15:03:33.089 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.090 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.090 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.090 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_monitoring": { 2023-09-30T15:03:33.090 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_monitoring", 2023-09-30T15:03:33.090 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.090 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.090 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.090 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.090 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.090 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.090 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.090 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "monitor device health metrics", 2023-09-30T15:03:33.091 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.091 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.091 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.091 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.091 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.091 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.091 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.091 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.091 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.091 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.091 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.092 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.092 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.092 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.092 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.092 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.092 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.092 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.092 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.092 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.092 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.092 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.093 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.093 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.093 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.093 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.093 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.093 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.093 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.093 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.093 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.093 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.093 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.094 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.094 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.094 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.094 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.094 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.094 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.094 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.094 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.094 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.094 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.094 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.094 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.095 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.095 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.095 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.095 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.095 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.095 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.095 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.095 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.095 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.095 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.095 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.095 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.096 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.096 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.096 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.096 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.096 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.096 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.096 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.096 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.096 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.096 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.096 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.097 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.097 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.097 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.097 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.097 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.097 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.097 INFO:teuthology.orchestra.run.smithi032.stdout: "mark_out_threshold": { 2023-09-30T15:03:33.097 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mark_out_threshold", 2023-09-30T15:03:33.097 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:33.097 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.097 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.097 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "2419200", 2023-09-30T15:03:33.098 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.098 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.098 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.098 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-30T15:03:33.098 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.098 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.098 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.098 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.098 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": { 2023-09-30T15:03:33.098 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pool_name", 2023-09-30T15:03:33.098 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.098 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.099 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.099 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "device_health_metrics", 2023-09-30T15:03:33.099 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.099 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.099 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.099 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-30T15:03:33.099 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.099 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.099 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.099 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.099 INFO:teuthology.orchestra.run.smithi032.stdout: "retention_period": { 2023-09-30T15:03:33.100 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "retention_period", 2023-09-30T15:03:33.100 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:33.100 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.100 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.100 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15552000", 2023-09-30T15:03:33.100 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.100 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.100 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.100 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long to retain device health metrics", 2023-09-30T15:03:33.100 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.100 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.100 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.101 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.101 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_frequency": { 2023-09-30T15:03:33.101 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_frequency", 2023-09-30T15:03:33.101 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:33.101 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.101 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.101 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2023-09-30T15:03:33.101 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.101 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.101 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.101 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-30T15:03:33.101 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.101 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.102 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.102 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.102 INFO:teuthology.orchestra.run.smithi032.stdout: "self_heal": { 2023-09-30T15:03:33.102 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "self_heal", 2023-09-30T15:03:33.102 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.102 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.102 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.102 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.102 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.102 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.102 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.103 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-30T15:03:33.103 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.103 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.103 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.103 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.103 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-09-30T15:03:33.103 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-09-30T15:03:33.103 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:33.103 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.103 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.103 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-09-30T15:03:33.103 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.104 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.104 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.104 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how frequently to wake up and check device health", 2023-09-30T15:03:33.104 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.104 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.104 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.104 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.104 INFO:teuthology.orchestra.run.smithi032.stdout: "warn_threshold": { 2023-09-30T15:03:33.104 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "warn_threshold", 2023-09-30T15:03:33.104 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:33.104 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.105 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.105 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7257600", 2023-09-30T15:03:33.105 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.105 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.105 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.105 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-30T15:03:33.105 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.105 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.105 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.105 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.105 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.105 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.106 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.106 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "diskprediction_local", 2023-09-30T15:03:33.106 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.106 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.106 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.106 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.106 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.106 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.106 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.106 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.106 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.107 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.107 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.107 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.107 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.107 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.107 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.107 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.107 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.107 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.107 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.107 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.107 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.108 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.108 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.108 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.108 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.108 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.108 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.108 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.108 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.108 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.108 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.108 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.108 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.109 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.109 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.109 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.109 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.109 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.109 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.109 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.109 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.109 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.109 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.109 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.109 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.109 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.110 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.110 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.110 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.110 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.110 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.110 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.110 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.110 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.110 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.110 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.110 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.110 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.111 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.111 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.111 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.111 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.111 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.111 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.111 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.111 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.111 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.111 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.111 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.111 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.112 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.112 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.112 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.112 INFO:teuthology.orchestra.run.smithi032.stdout: "predict_interval": { 2023-09-30T15:03:33.112 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predict_interval", 2023-09-30T15:03:33.112 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.112 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.112 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.112 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "86400", 2023-09-30T15:03:33.112 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.112 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.112 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.113 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.113 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.113 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.113 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.113 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.113 INFO:teuthology.orchestra.run.smithi032.stdout: "predictor_model": { 2023-09-30T15:03:33.113 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "predictor_model", 2023-09-30T15:03:33.113 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.113 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.113 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.113 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "prophetstor", 2023-09-30T15:03:33.113 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.114 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.114 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.114 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.114 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.114 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.114 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.114 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.114 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-09-30T15:03:33.114 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-09-30T15:03:33.114 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.114 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.114 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.115 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "600", 2023-09-30T15:03:33.115 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.115 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.115 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.115 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.115 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.115 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.115 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.115 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.115 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.115 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.115 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.115 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "influx", 2023-09-30T15:03:33.116 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": false, 2023-09-30T15:03:33.116 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "influxdb python module not found", 2023-09-30T15:03:33.116 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.116 INFO:teuthology.orchestra.run.smithi032.stdout: "batch_size": { 2023-09-30T15:03:33.116 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "batch_size", 2023-09-30T15:03:33.116 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.116 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.116 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.116 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5000", 2023-09-30T15:03:33.116 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.116 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.116 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.117 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-30T15:03:33.117 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.117 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.117 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.117 INFO:teuthology.orchestra.run.smithi032.stdout: "database": { 2023-09-30T15:03:33.117 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "database", 2023-09-30T15:03:33.117 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.117 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.117 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.117 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "ceph", 2023-09-30T15:03:33.117 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.118 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.118 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.118 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-30T15:03:33.118 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.118 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.118 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.118 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": { 2023-09-30T15:03:33.118 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "hostname", 2023-09-30T15:03:33.118 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.118 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.118 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.119 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.119 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.119 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.119 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.119 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server hostname", 2023-09-30T15:03:33.119 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.119 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.119 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.119 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-09-30T15:03:33.119 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-09-30T15:03:33.119 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:33.119 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.120 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.120 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "30", 2023-09-30T15:03:33.120 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "5", 2023-09-30T15:03:33.120 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.120 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.120 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-30T15:03:33.120 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.120 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.120 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.120 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.120 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.121 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.121 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.121 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.121 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.121 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.121 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.121 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.121 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.121 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.121 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.121 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.121 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.122 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.122 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.122 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.122 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.122 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.122 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.122 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.122 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.122 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.122 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.122 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.122 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.123 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.123 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.123 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.123 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.123 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.123 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.123 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.123 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.123 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.123 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.123 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.124 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.124 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.124 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.124 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.124 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.124 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.124 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.124 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.124 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.124 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.124 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.124 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.124 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.125 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.125 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.125 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.125 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.125 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.125 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.125 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.125 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.125 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.125 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.125 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.126 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.126 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.126 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.126 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.126 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.126 INFO:teuthology.orchestra.run.smithi032.stdout: "password": { 2023-09-30T15:03:33.126 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "password", 2023-09-30T15:03:33.126 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.126 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.126 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.126 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.127 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.127 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.127 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.127 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "password of InfluxDB server user", 2023-09-30T15:03:33.127 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.127 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.127 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.127 INFO:teuthology.orchestra.run.smithi032.stdout: "port": { 2023-09-30T15:03:33.127 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "port", 2023-09-30T15:03:33.127 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.127 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.128 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.128 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "8086", 2023-09-30T15:03:33.128 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.128 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.128 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.128 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "InfluxDB server port", 2023-09-30T15:03:33.128 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.128 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.128 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.128 INFO:teuthology.orchestra.run.smithi032.stdout: "ssl": { 2023-09-30T15:03:33.128 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ssl", 2023-09-30T15:03:33.128 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.129 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.129 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.129 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "false", 2023-09-30T15:03:33.129 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.129 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.129 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.129 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-30T15:03:33.129 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.129 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.129 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.129 INFO:teuthology.orchestra.run.smithi032.stdout: "threads": { 2023-09-30T15:03:33.130 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threads", 2023-09-30T15:03:33.130 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.130 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.130 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.130 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-09-30T15:03:33.130 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-09-30T15:03:33.130 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "32", 2023-09-30T15:03:33.130 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.130 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-30T15:03:33.130 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.131 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.131 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.131 INFO:teuthology.orchestra.run.smithi032.stdout: "username": { 2023-09-30T15:03:33.131 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "username", 2023-09-30T15:03:33.131 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.131 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.131 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.131 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.131 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.131 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.131 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.132 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "username of InfluxDB server user", 2023-09-30T15:03:33.132 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.132 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.132 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.132 INFO:teuthology.orchestra.run.smithi032.stdout: "verify_ssl": { 2023-09-30T15:03:33.132 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "verify_ssl", 2023-09-30T15:03:33.132 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.132 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.132 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.132 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "true", 2023-09-30T15:03:33.132 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.132 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.133 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.133 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-30T15:03:33.133 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.133 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.133 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.133 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.133 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.133 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "insights", 2023-09-30T15:03:33.133 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.133 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.134 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.134 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.134 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.134 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.134 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.134 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.134 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.134 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.134 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.134 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.134 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.134 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.134 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.135 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.135 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.135 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.135 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.135 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.135 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.135 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.135 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.135 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.135 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.135 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.135 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.135 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.136 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.136 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.136 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.136 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.136 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.136 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.136 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.136 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.136 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.136 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.136 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.136 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.137 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.137 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.137 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.137 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.137 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.137 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.137 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.137 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.137 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.137 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.137 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.137 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.137 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.138 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.138 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.138 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.138 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.138 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.138 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.138 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.138 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.138 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.138 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.138 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.138 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.139 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.139 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.139 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.139 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.139 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.139 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.139 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.139 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.139 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.139 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.139 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.139 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "iostat", 2023-09-30T15:03:33.140 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.140 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.140 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.140 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.140 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.140 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.140 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.140 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.140 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.140 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.140 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.140 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.141 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.141 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.141 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.141 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.141 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.141 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.141 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.141 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.141 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.141 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.141 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.141 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.141 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.142 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.142 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.142 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.142 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.142 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.142 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.142 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.142 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.142 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.142 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.142 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.142 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.144 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.144 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.144 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.144 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.144 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.144 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.144 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.144 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.144 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.144 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.144 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.144 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.144 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.145 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.145 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.145 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.145 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.145 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.145 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.145 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.145 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.145 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.145 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.145 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.145 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.146 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.146 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "k8sevents", 2023-09-30T15:03:33.146 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.146 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.146 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.146 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_event_retention_days": { 2023-09-30T15:03:33.146 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "ceph_event_retention_days", 2023-09-30T15:03:33.146 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.146 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.146 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.146 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "7", 2023-09-30T15:03:33.146 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.147 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.147 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.147 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-30T15:03:33.147 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.147 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.147 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.147 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.147 INFO:teuthology.orchestra.run.smithi032.stdout: "config_check_secs": { 2023-09-30T15:03:33.147 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "config_check_secs", 2023-09-30T15:03:33.147 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.147 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.147 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.148 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:33.148 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "10", 2023-09-30T15:03:33.148 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.148 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.148 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-30T15:03:33.148 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.148 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.148 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.148 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.148 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.148 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.148 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.148 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.149 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.149 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.149 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.149 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.149 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.149 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.149 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.149 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.149 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.149 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.149 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.149 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.149 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.149 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.150 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.150 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.150 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.150 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.150 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.150 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.150 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.150 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.150 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.150 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.150 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.150 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.151 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.151 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.151 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.151 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.151 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.151 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.151 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.151 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.151 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.151 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.151 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.151 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.152 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.152 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.152 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.152 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.152 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.152 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.152 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.152 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.152 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.152 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.152 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.152 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.152 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.153 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.153 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.153 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.153 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.153 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.153 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.153 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.153 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.153 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.153 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.153 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.153 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.154 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.154 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.154 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.154 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.154 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.154 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.154 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "localpool", 2023-09-30T15:03:33.154 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.154 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.154 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.154 INFO:teuthology.orchestra.run.smithi032.stdout: "failure_domain": { 2023-09-30T15:03:33.154 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "failure_domain", 2023-09-30T15:03:33.155 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.155 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.155 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.155 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "host", 2023-09-30T15:03:33.155 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.155 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.155 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.155 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "failure domain for any created local pool", 2023-09-30T15:03:33.155 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-30T15:03:33.155 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.155 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.155 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.155 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.156 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.156 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.156 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.156 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.156 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.156 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.156 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.156 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.156 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.156 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.156 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.156 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.157 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.157 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.157 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.157 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.157 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.157 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.157 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.157 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.157 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.157 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.157 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.157 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.157 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.158 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.158 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.158 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.158 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.158 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.158 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.158 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.158 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.158 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.158 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.158 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.158 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.159 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.159 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.159 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.159 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.159 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.159 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.159 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.159 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.159 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.159 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.159 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.159 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.160 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.160 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.160 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.160 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.160 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.160 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.160 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.160 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.160 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.160 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.160 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.160 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.161 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.161 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.161 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.161 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.161 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.161 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.161 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.161 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.161 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": { 2023-09-30T15:03:33.161 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "min_size", 2023-09-30T15:03:33.161 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.161 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.162 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.162 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.162 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.162 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.162 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.162 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default min_size for any created local pool", 2023-09-30T15:03:33.162 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-30T15:03:33.162 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.162 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.162 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.162 INFO:teuthology.orchestra.run.smithi032.stdout: "num_rep": { 2023-09-30T15:03:33.162 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "num_rep", 2023-09-30T15:03:33.162 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.163 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.163 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.163 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3", 2023-09-30T15:03:33.163 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.163 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.163 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.163 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default replica count for any created local pool", 2023-09-30T15:03:33.163 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.163 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.163 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.163 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.163 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": { 2023-09-30T15:03:33.163 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_num", 2023-09-30T15:03:33.164 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.164 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.164 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.164 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "128", 2023-09-30T15:03:33.164 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.164 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.164 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.164 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "default pg_num for any created local pool", 2023-09-30T15:03:33.164 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.164 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.164 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.164 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.165 INFO:teuthology.orchestra.run.smithi032.stdout: "prefix": { 2023-09-30T15:03:33.165 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prefix", 2023-09-30T15:03:33.165 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.165 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.165 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.165 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.165 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.165 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.165 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.165 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "name prefix for any created local pool", 2023-09-30T15:03:33.165 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.165 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.165 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.166 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.166 INFO:teuthology.orchestra.run.smithi032.stdout: "subtree": { 2023-09-30T15:03:33.166 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "subtree", 2023-09-30T15:03:33.166 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.166 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.166 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.166 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "rack", 2023-09-30T15:03:33.166 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.166 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.166 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.166 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-30T15:03:33.166 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-30T15:03:33.167 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.167 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.167 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.167 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.167 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.167 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.167 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mds_autoscaler", 2023-09-30T15:03:33.167 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.167 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.167 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.167 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.167 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.168 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.168 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.168 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.168 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.168 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.168 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.168 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.168 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.168 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.168 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.168 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.168 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.168 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.169 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.169 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.169 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.169 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.169 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.169 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.169 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.169 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.169 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.169 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.169 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.170 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.170 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.170 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.170 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.170 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.170 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.170 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.170 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.170 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.170 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.170 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.171 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.171 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.171 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.171 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.171 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.171 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.171 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.171 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.171 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.171 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.171 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.171 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.171 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.172 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.172 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.172 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.172 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.172 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.172 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.172 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.172 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.172 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.172 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.173 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.173 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.173 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.173 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.173 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.173 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.173 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirroring", 2023-09-30T15:03:33.174 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.174 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.174 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.174 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.174 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.174 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.174 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.174 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.174 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.174 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.174 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.175 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.175 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.175 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.175 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.175 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.175 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.175 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.175 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.175 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.175 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.175 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.175 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.176 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.176 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.176 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.176 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.176 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.176 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.176 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.176 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.176 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.176 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.176 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.177 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.177 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.177 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.177 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.177 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.177 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.177 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.177 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.177 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.177 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.177 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.177 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.178 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.178 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.178 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.178 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.178 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.178 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.178 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.178 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.178 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.178 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.178 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.179 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.179 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.179 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.179 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.179 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.179 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.179 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.179 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.179 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.179 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.179 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.179 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.180 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "nfs", 2023-09-30T15:03:33.180 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.180 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.180 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.180 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.180 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.180 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.180 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.181 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.181 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.181 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.181 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.181 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.181 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.181 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.181 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.181 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.181 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.181 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.181 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.181 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.182 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.182 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.182 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.182 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.182 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.182 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.182 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.182 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.182 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.182 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.182 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.182 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.183 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.183 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.183 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.183 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.183 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.183 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.183 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.183 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.183 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.183 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.183 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.183 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.184 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.184 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.184 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.184 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.184 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.184 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.184 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.184 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.184 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.184 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.184 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.184 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.184 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.185 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.185 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.185 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.185 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.185 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.185 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.185 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.185 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.185 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.185 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.185 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.185 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.186 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.186 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.186 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.186 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.186 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.186 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2023-09-30T15:03:33.186 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.186 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.186 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.186 INFO:teuthology.orchestra.run.smithi032.stdout: "fail_fs": { 2023-09-30T15:03:33.186 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "fail_fs", 2023-09-30T15:03:33.187 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.187 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.187 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.187 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.187 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.187 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.187 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.187 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-30T15:03:33.187 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.187 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.187 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.187 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.188 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.188 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.188 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.188 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.188 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.188 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.188 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.188 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.188 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.188 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.188 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.188 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.189 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.189 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.189 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.189 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.189 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.189 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.189 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.189 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.189 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.189 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.189 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.190 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.190 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.190 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.190 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.190 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.190 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.190 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.190 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.190 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.190 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.190 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.190 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.191 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.191 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.191 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.191 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.191 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.191 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.191 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.191 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.191 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.191 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.191 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.191 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.192 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.192 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.192 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.192 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.192 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.192 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.192 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.192 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.192 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.192 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.192 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.192 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.192 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.193 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.193 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.193 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.193 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.193 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.193 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.193 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.193 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.193 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.193 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.193 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator": { 2023-09-30T15:03:33.193 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "orchestrator", 2023-09-30T15:03:33.194 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.194 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.194 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.194 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.194 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.194 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.194 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.194 INFO:teuthology.orchestra.run.smithi032.stdout: "cephadm", 2023-09-30T15:03:33.194 INFO:teuthology.orchestra.run.smithi032.stdout: "rook", 2023-09-30T15:03:33.194 INFO:teuthology.orchestra.run.smithi032.stdout: "test_orchestrator" 2023-09-30T15:03:33.194 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.194 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Orchestrator backend", 2023-09-30T15:03:33.195 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.195 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.195 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.195 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.195 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.195 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.195 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.195 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_perf_query", 2023-09-30T15:03:33.195 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.195 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.195 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.195 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.195 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.196 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.196 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.196 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.196 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.196 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.196 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.196 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.196 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.196 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.196 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.196 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.196 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.196 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.197 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.197 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.197 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.197 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.197 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.197 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.197 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.197 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.197 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.197 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.197 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.197 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.197 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.198 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.198 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.198 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.198 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.198 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.198 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.198 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.198 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.198 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.198 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.198 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.198 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.198 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.199 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.199 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.199 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.199 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.199 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.199 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.199 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.199 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.199 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.199 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.199 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.199 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.200 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.200 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.200 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.200 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.200 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.200 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.200 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.200 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.200 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.200 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.200 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.200 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.201 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.201 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.201 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.201 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.201 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.201 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.201 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "osd_support", 2023-09-30T15:03:33.201 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.201 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.201 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.201 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.201 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.202 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.202 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.202 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.202 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.202 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.202 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.202 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.202 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.202 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.202 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.202 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.202 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.203 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.203 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.203 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.203 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.203 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.203 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.203 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.203 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.203 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.203 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.203 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.203 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.203 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.204 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.204 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.204 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.204 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.204 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.204 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.204 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.204 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.204 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.204 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.204 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.204 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.204 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.205 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.205 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.205 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.205 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.205 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.205 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.205 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.205 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.205 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.205 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.205 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.205 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.206 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.206 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.206 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.206 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.206 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.206 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.206 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.206 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.206 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.206 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.206 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.206 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.207 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.207 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.207 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.207 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.207 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.207 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.207 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.207 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.207 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.207 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "pg_autoscaler", 2023-09-30T15:03:33.207 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.207 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.207 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.208 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.208 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.208 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.208 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.208 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.208 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.208 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.208 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.208 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.208 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.208 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.209 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.209 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.209 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.209 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.209 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.209 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.209 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.209 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.209 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.209 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.209 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.209 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.210 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.210 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.210 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.210 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.210 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.210 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.210 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.210 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.210 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.210 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.210 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.210 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.210 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.211 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.211 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.211 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.211 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.211 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.211 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.211 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.211 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.211 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.211 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.211 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.211 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.212 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.212 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.212 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.212 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.212 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.212 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.212 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.212 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.212 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.212 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.212 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.212 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.212 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.213 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.213 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.213 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.213 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.213 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.213 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.213 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.213 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.213 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.213 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-09-30T15:03:33.213 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-09-30T15:03:33.213 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:33.213 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.214 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.214 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-09-30T15:03:33.214 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.214 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.214 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.214 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.214 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.214 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.214 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.214 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.214 INFO:teuthology.orchestra.run.smithi032.stdout: "threshold": { 2023-09-30T15:03:33.215 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "threshold", 2023-09-30T15:03:33.215 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:33.215 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.215 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.215 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "3.0", 2023-09-30T15:03:33.215 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1.0", 2023-09-30T15:03:33.215 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.215 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.215 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "scaling threshold", 2023-09-30T15:03:33.215 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-30T15:03:33.215 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.215 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.216 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.216 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.216 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.216 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.216 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "progress", 2023-09-30T15:03:33.216 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.216 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.216 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.216 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_pg_recovery_event": { 2023-09-30T15:03:33.216 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_pg_recovery_event", 2023-09-30T15:03:33.216 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.216 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.216 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.217 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.217 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.217 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.217 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.217 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-30T15:03:33.217 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.217 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.217 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.217 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.217 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2023-09-30T15:03:33.217 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2023-09-30T15:03:33.217 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.218 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.218 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.218 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.218 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.218 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.218 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.218 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.218 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.218 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.218 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.218 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.218 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.218 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.219 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.219 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.219 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.219 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.219 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.219 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.219 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.219 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.219 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.219 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.219 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.219 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.220 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.220 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.220 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.220 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.220 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.220 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.220 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.220 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.220 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.220 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.220 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.220 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.220 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.221 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.222 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.223 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.224 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.224 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.224 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.224 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.224 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.224 INFO:teuthology.orchestra.run.smithi032.stdout: "max_completed_events": { 2023-09-30T15:03:33.224 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_completed_events", 2023-09-30T15:03:33.224 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.224 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.224 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.225 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "50", 2023-09-30T15:03:33.225 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.225 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.225 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.225 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "number of past completed events to remember", 2023-09-30T15:03:33.225 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.225 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.225 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.225 INFO:teuthology.orchestra.run.smithi032.stdout: "sleep_interval": { 2023-09-30T15:03:33.225 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "sleep_interval", 2023-09-30T15:03:33.225 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:33.226 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.226 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.226 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "5", 2023-09-30T15:03:33.226 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.226 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.226 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.226 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "how long the module is going to sleep", 2023-09-30T15:03:33.226 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.226 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.226 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.226 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.226 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.227 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "prometheus", 2023-09-30T15:03:33.227 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.227 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.227 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.227 INFO:teuthology.orchestra.run.smithi032.stdout: "cache": { 2023-09-30T15:03:33.227 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cache", 2023-09-30T15:03:33.227 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.227 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.227 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.227 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.227 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.228 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.228 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.229 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.229 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.229 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.229 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.229 INFO:teuthology.orchestra.run.smithi032.stdout: "exclude_perf_counters": { 2023-09-30T15:03:33.229 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "exclude_perf_counters", 2023-09-30T15:03:33.229 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.229 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.230 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.230 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.230 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.230 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.230 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.230 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-30T15:03:33.230 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-30T15:03:33.230 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.230 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.230 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.230 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.231 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.231 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.231 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.231 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.231 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.231 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.231 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.231 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.231 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.231 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.231 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.231 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.232 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.232 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.232 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.232 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.232 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.232 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.232 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.232 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.232 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.232 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.232 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.232 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.233 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.234 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.235 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools": { 2023-09-30T15:03:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools", 2023-09-30T15:03:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.236 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-30T15:03:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-30T15:03:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.237 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300", 2023-09-30T15:03:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: "scrape_interval": { 2023-09-30T15:03:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "scrape_interval", 2023-09-30T15:03:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.238 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15.0", 2023-09-30T15:03:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-09-30T15:03:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-09-30T15:03:33.239 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "::", 2023-09-30T15:03:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-30T15:03:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.240 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-09-30T15:03:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-09-30T15:03:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "9283", 2023-09-30T15:03:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-30T15:03:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.241 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.242 INFO:teuthology.orchestra.run.smithi032.stdout: "stale_cache_strategy": { 2023-09-30T15:03:33.242 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stale_cache_strategy", 2023-09-30T15:03:33.242 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.242 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.242 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.242 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "log", 2023-09-30T15:03:33.242 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.242 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.242 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.242 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.242 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.242 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.242 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.243 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_behaviour": { 2023-09-30T15:03:33.243 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_behaviour", 2023-09-30T15:03:33.243 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.243 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.243 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.243 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "default", 2023-09-30T15:03:33.243 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.243 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.243 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.243 INFO:teuthology.orchestra.run.smithi032.stdout: "default", 2023-09-30T15:03:33.243 INFO:teuthology.orchestra.run.smithi032.stdout: "error" 2023-09-30T15:03:33.244 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.244 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.244 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.244 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.244 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.244 INFO:teuthology.orchestra.run.smithi032.stdout: "standby_error_status_code": { 2023-09-30T15:03:33.244 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "standby_error_status_code", 2023-09-30T15:03:33.244 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.244 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.244 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.244 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "500", 2023-09-30T15:03:33.245 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "400", 2023-09-30T15:03:33.245 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "599", 2023-09-30T15:03:33.245 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.245 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.245 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.245 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.245 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.245 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.245 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.245 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2023-09-30T15:03:33.245 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.246 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.246 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.246 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.246 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.246 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.246 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.246 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.246 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.246 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.246 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.246 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.247 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.247 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.247 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.247 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.247 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.247 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.247 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.247 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.247 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.247 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.247 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.247 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.248 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.248 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.248 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.248 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.248 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.248 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.248 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.248 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.248 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.248 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.248 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.248 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.249 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.249 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.249 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.249 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.249 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.249 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.249 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.249 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.249 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.249 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.249 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.249 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.250 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.250 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.250 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.250 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.250 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.250 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.250 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.250 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.250 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.250 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.250 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.251 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.251 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.251 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.251 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.251 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.251 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.251 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.251 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.251 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.251 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.251 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_snap_create": { 2023-09-30T15:03:33.252 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_snap_create", 2023-09-30T15:03:33.252 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.252 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.252 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.252 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10", 2023-09-30T15:03:33.252 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.252 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.252 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.252 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.252 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.252 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.252 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.253 INFO:teuthology.orchestra.run.smithi032.stdout: "mirror_snapshot_schedule": { 2023-09-30T15:03:33.253 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "mirror_snapshot_schedule", 2023-09-30T15:03:33.253 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.253 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.253 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.253 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.253 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.253 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.253 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.253 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.253 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.254 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.254 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.254 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.254 INFO:teuthology.orchestra.run.smithi032.stdout: "trash_purge_schedule": { 2023-09-30T15:03:33.254 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "trash_purge_schedule", 2023-09-30T15:03:33.254 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.254 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.254 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.254 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.254 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.254 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.254 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.255 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.255 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.255 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.255 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.255 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.255 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.255 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.255 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "restful", 2023-09-30T15:03:33.255 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.255 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.255 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.255 INFO:teuthology.orchestra.run.smithi032.stdout: "enable_auth": { 2023-09-30T15:03:33.256 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enable_auth", 2023-09-30T15:03:33.256 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.256 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.256 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.256 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.256 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.256 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.256 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.256 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.256 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.256 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.256 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.257 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.257 INFO:teuthology.orchestra.run.smithi032.stdout: "key_file": { 2023-09-30T15:03:33.257 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "key_file", 2023-09-30T15:03:33.257 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.257 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.257 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.257 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.257 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.257 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.257 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.257 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.257 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.257 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.258 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.258 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.258 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.258 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.258 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.258 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.258 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.258 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.258 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.258 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.258 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.258 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.259 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.259 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.259 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.259 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.259 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.259 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.259 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.259 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.259 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.259 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.259 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.259 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.260 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.260 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.260 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.260 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.260 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.260 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.260 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.260 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.260 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.260 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.260 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.260 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.261 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.261 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.261 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.261 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.261 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.261 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.261 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.261 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.261 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.261 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.261 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.261 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.262 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.262 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.262 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.262 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.262 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.262 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.262 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.262 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.262 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.262 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.262 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.262 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.263 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.263 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.263 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.263 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.263 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.263 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.263 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.263 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.263 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.263 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.263 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.263 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.263 INFO:teuthology.orchestra.run.smithi032.stdout: "server_addr": { 2023-09-30T15:03:33.264 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_addr", 2023-09-30T15:03:33.264 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.264 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.264 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.264 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.264 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.264 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.264 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.264 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.264 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.264 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.264 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.264 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.265 INFO:teuthology.orchestra.run.smithi032.stdout: "server_port": { 2023-09-30T15:03:33.265 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "server_port", 2023-09-30T15:03:33.265 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.265 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.265 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.265 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.265 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.265 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.265 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.265 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.265 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.265 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.266 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.266 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.266 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.266 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.266 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.266 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rgw", 2023-09-30T15:03:33.266 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.266 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.266 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.266 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.266 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.266 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.267 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.267 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.267 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.267 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.267 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.267 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.267 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.267 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.267 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.267 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.267 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.267 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.267 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.268 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.268 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.268 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.268 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.268 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.268 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.268 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.268 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.268 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.268 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.268 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.268 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.269 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.269 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.269 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.269 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.269 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.269 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.269 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.269 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.269 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.269 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.269 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.269 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.269 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.270 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.270 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.270 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.270 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.270 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.270 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.270 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.270 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.270 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.270 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.270 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.270 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.271 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.271 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.271 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.271 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.271 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.271 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.271 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.271 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.271 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.271 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.271 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.271 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.271 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.272 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.272 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.272 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.272 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.272 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.272 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.272 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rook", 2023-09-30T15:03:33.272 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.272 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.272 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.272 INFO:teuthology.orchestra.run.smithi032.stdout: "drive_group_interval": { 2023-09-30T15:03:33.273 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "drive_group_interval", 2023-09-30T15:03:33.273 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:33.273 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.273 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.273 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "300.0", 2023-09-30T15:03:33.273 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.273 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.273 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.273 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-30T15:03:33.273 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.273 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.273 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.273 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.274 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.274 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.274 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.274 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.274 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.274 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.274 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.274 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.274 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.274 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.274 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.274 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.275 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.275 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.275 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.275 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.275 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.275 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.275 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.275 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.275 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.275 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.275 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.275 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.275 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: "storage_class": { 2023-09-30T15:03:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "storage_class", 2023-09-30T15:03:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "local", 2023-09-30T15:03:33.280 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.280 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.280 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.280 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-30T15:03:33.280 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.280 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.280 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.280 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.280 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.280 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.280 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.280 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "selftest", 2023-09-30T15:03:33.281 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.281 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.281 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.281 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.281 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.281 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.281 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.281 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.281 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.281 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.281 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.281 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.281 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.282 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.282 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.282 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.282 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.282 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.282 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.282 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.282 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.282 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.282 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.282 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.282 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.283 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.283 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.283 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.283 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.283 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.283 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.283 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.283 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.283 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.283 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.283 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.283 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.283 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.284 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.284 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.284 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.284 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.284 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.284 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.284 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.284 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.284 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.284 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.284 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.285 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.285 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.285 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.285 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.285 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.285 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.285 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.285 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.285 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.285 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.285 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.285 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.286 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.286 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.286 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.286 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.286 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.286 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.286 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.286 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.286 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.286 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.286 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.286 INFO:teuthology.orchestra.run.smithi032.stdout: "roption1": { 2023-09-30T15:03:33.287 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption1", 2023-09-30T15:03:33.287 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.287 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.287 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.287 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.287 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.287 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.287 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.287 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.287 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.287 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.287 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.288 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.288 INFO:teuthology.orchestra.run.smithi032.stdout: "roption2": { 2023-09-30T15:03:33.288 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "roption2", 2023-09-30T15:03:33.288 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.288 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.288 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.288 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "xyz", 2023-09-30T15:03:33.288 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.288 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.288 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.288 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.288 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.288 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.289 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.289 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.289 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption1": { 2023-09-30T15:03:33.289 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption1", 2023-09-30T15:03:33.289 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.289 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.289 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.289 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.289 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.289 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.289 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.289 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.290 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.290 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.290 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.290 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.290 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption2": { 2023-09-30T15:03:33.290 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption2", 2023-09-30T15:03:33.290 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.290 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.290 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.290 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.290 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.290 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.291 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.291 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.291 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.291 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.291 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.291 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.291 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption3": { 2023-09-30T15:03:33.291 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption3", 2023-09-30T15:03:33.291 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "float", 2023-09-30T15:03:33.291 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.291 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.291 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.292 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.292 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.292 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.292 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.292 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.292 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.292 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.292 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.292 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption4": { 2023-09-30T15:03:33.292 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption4", 2023-09-30T15:03:33.292 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.292 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.293 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.293 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.293 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.293 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.293 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.293 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.293 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.293 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.293 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.293 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.293 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption5": { 2023-09-30T15:03:33.293 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption5", 2023-09-30T15:03:33.293 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.294 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.294 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.294 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.294 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.294 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.294 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.294 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.294 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.294 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.294 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.294 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.294 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption6": { 2023-09-30T15:03:33.295 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption6", 2023-09-30T15:03:33.295 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.295 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.295 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.295 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.295 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.295 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.295 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.295 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.295 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.295 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.295 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.296 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.296 INFO:teuthology.orchestra.run.smithi032.stdout: "rwoption7": { 2023-09-30T15:03:33.296 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rwoption7", 2023-09-30T15:03:33.296 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.296 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.296 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.296 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.296 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "1", 2023-09-30T15:03:33.296 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "42", 2023-09-30T15:03:33.296 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.296 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.296 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.297 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.297 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.297 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.297 INFO:teuthology.orchestra.run.smithi032.stdout: "testkey": { 2023-09-30T15:03:33.297 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testkey", 2023-09-30T15:03:33.297 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.297 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.297 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.297 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.297 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.297 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.297 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.297 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.298 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.298 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.298 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.298 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.298 INFO:teuthology.orchestra.run.smithi032.stdout: "testlkey": { 2023-09-30T15:03:33.298 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testlkey", 2023-09-30T15:03:33.298 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.298 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.298 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.298 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.298 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.298 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.299 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.299 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.299 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.299 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.299 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.299 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.299 INFO:teuthology.orchestra.run.smithi032.stdout: "testnewline": { 2023-09-30T15:03:33.299 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "testnewline", 2023-09-30T15:03:33.299 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.299 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.299 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.299 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.300 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.300 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.300 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.300 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.300 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.300 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.300 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.300 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.300 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.300 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.300 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.300 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snap_schedule", 2023-09-30T15:03:33.300 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.301 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.301 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.301 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_m_granularity": { 2023-09-30T15:03:33.301 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "allow_m_granularity", 2023-09-30T15:03:33.301 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.301 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.301 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.301 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.301 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.301 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.301 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.301 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "allow minute scheduled snapshots", 2023-09-30T15:03:33.301 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.302 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.302 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.302 INFO:teuthology.orchestra.run.smithi032.stdout: "dump_on_update": { 2023-09-30T15:03:33.302 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "dump_on_update", 2023-09-30T15:03:33.302 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.302 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.302 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.302 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.302 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.302 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.302 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.303 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "dump database to debug log on update", 2023-09-30T15:03:33.303 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.303 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.303 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.303 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.303 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.303 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.303 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.303 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.303 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.303 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.303 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.304 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.304 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.304 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.304 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.304 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.304 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.304 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.304 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.304 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.304 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.304 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.304 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.305 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.305 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.305 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.305 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.305 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.305 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.305 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.305 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.305 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.305 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.305 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.305 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.306 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.306 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.306 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.306 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.306 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.306 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.306 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.306 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.306 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.306 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.306 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.307 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.307 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.307 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.307 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.307 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.307 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.307 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.307 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.307 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.307 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.307 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.307 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.308 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.308 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.308 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.308 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.308 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.308 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.308 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.308 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.308 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.308 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.308 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.308 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.309 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.309 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "stats", 2023-09-30T15:03:33.309 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.309 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.309 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.309 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.309 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.309 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.309 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.309 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.309 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.310 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.310 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.310 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.310 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.310 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.310 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.310 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.310 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.310 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.310 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.310 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.310 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.311 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.311 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.311 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.311 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.311 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.311 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.311 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.311 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.311 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.311 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.311 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.311 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.312 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.312 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.312 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.312 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.312 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.312 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.312 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.312 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.312 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.312 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.312 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.312 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.313 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.313 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.313 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.313 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.313 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.313 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.313 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.313 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.313 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.313 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.313 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.314 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.314 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.314 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.314 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.314 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.314 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.314 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.314 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.314 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.314 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.314 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.314 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.314 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.315 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.315 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.315 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.315 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.315 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "status", 2023-09-30T15:03:33.315 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.315 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.315 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.315 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.315 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.315 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.315 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.315 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.316 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.316 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.316 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.316 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.316 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.316 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.316 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.316 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.316 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.316 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.316 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.316 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.316 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.317 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.317 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.317 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.317 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.317 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.317 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.317 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.317 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.317 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.317 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.317 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.317 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.318 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.318 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.318 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.318 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.318 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.318 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.318 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.318 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.318 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.318 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.318 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.318 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.318 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.319 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.319 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.319 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.319 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.319 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.319 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.319 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.319 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.319 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.319 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.319 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.319 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.320 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.320 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.320 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.320 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.320 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.320 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.320 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.320 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.320 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.320 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.320 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.320 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.321 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.321 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.321 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.321 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.321 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.321 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telegraf", 2023-09-30T15:03:33.321 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.321 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.321 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.321 INFO:teuthology.orchestra.run.smithi032.stdout: "address": { 2023-09-30T15:03:33.321 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "address", 2023-09-30T15:03:33.321 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.322 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.322 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.322 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-30T15:03:33.322 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.322 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.322 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.322 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.322 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.322 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.322 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.322 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.322 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-09-30T15:03:33.323 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-09-30T15:03:33.323 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:33.323 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.323 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.323 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "15", 2023-09-30T15:03:33.323 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.323 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.323 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.323 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.323 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.323 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.323 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.323 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.324 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.324 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.324 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.324 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.324 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.324 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.324 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.324 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.324 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.324 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.324 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.324 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.325 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.325 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.325 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.325 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.325 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.325 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.325 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.325 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.325 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.325 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.325 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.325 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.326 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.326 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.326 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.326 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.326 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.326 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.326 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.326 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.326 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.326 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.326 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.326 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.327 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.327 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.327 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.327 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.327 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.327 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.327 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.327 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.327 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.327 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.327 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.327 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.327 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.328 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.328 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.328 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.328 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.328 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.328 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.328 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.328 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.328 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.328 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.328 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.328 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.329 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.329 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.329 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.329 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.329 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.329 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.329 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.329 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.329 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.329 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.329 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.329 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.330 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "telemetry", 2023-09-30T15:03:33.330 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.330 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.330 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.330 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_basic": { 2023-09-30T15:03:33.330 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_basic", 2023-09-30T15:03:33.330 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.330 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.330 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.330 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.330 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.330 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.331 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.331 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-30T15:03:33.331 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.331 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.331 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.331 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.331 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_crash": { 2023-09-30T15:03:33.331 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_crash", 2023-09-30T15:03:33.331 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.331 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.331 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.331 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.331 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.332 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.332 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.332 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-30T15:03:33.332 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.332 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.332 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.332 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.332 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_device": { 2023-09-30T15:03:33.332 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_device", 2023-09-30T15:03:33.332 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.332 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.332 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.333 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "True", 2023-09-30T15:03:33.333 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.333 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.333 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.333 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-30T15:03:33.333 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.333 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.333 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.333 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.333 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_ident": { 2023-09-30T15:03:33.333 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_ident", 2023-09-30T15:03:33.333 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.333 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.334 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.334 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.334 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.334 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.334 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.334 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-30T15:03:33.334 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.334 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.334 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.334 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.334 INFO:teuthology.orchestra.run.smithi032.stdout: "channel_perf": { 2023-09-30T15:03:33.334 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "channel_perf", 2023-09-30T15:03:33.335 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.335 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.335 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.335 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.335 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.335 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.335 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.335 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-30T15:03:33.335 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.335 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.335 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.335 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.335 INFO:teuthology.orchestra.run.smithi032.stdout: "contact": { 2023-09-30T15:03:33.336 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "contact", 2023-09-30T15:03:33.336 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.336 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.336 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.336 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.336 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.336 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.336 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.336 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.336 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.336 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.336 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.336 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.337 INFO:teuthology.orchestra.run.smithi032.stdout: "description": { 2023-09-30T15:03:33.337 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "description", 2023-09-30T15:03:33.337 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.337 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.337 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.337 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.337 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.337 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.337 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.337 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.337 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.337 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.337 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.338 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.338 INFO:teuthology.orchestra.run.smithi032.stdout: "device_url": { 2023-09-30T15:03:33.338 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "device_url", 2023-09-30T15:03:33.338 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.338 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.338 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.338 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-30T15:03:33.338 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.338 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.338 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.338 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.338 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.339 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.339 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.339 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.339 INFO:teuthology.orchestra.run.smithi032.stdout: "enabled": { 2023-09-30T15:03:33.339 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "enabled", 2023-09-30T15:03:33.339 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.339 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.339 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.339 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.339 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.339 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.339 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.339 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.340 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.340 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.340 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.340 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-09-30T15:03:33.340 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-09-30T15:03:33.340 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.340 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.340 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.340 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "24", 2023-09-30T15:03:33.340 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "8", 2023-09-30T15:03:33.340 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.340 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.341 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.341 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.341 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.341 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.341 INFO:teuthology.orchestra.run.smithi032.stdout: "last_opt_revision": { 2023-09-30T15:03:33.341 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "last_opt_revision", 2023-09-30T15:03:33.341 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.341 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.341 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.341 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "1", 2023-09-30T15:03:33.341 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.341 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.342 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.342 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.342 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.342 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.342 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.342 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard": { 2023-09-30T15:03:33.342 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard", 2023-09-30T15:03:33.342 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.342 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.342 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.342 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.343 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.343 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.343 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.343 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.343 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.343 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.343 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.343 INFO:teuthology.orchestra.run.smithi032.stdout: "leaderboard_description": { 2023-09-30T15:03:33.343 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "leaderboard_description", 2023-09-30T15:03:33.343 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.343 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.344 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.344 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.344 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.344 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.344 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.344 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.344 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.344 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.344 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.344 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.344 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.345 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.345 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.345 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.345 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.345 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.345 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.345 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.345 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.345 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.345 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.345 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.345 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.345 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.346 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.346 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.346 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.346 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.346 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.346 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.346 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.346 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.346 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.346 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.346 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.347 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.347 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.347 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.347 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.347 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.347 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.347 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.347 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.347 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.347 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.347 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.347 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.348 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.348 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.348 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.348 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.348 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.348 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.348 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.348 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.348 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.348 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.348 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.348 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.348 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.349 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.349 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.349 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.349 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.349 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.349 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.349 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.349 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.349 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.349 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.349 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.350 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.350 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.350 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.350 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.350 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.350 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.350 INFO:teuthology.orchestra.run.smithi032.stdout: "organization": { 2023-09-30T15:03:33.350 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "organization", 2023-09-30T15:03:33.350 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.350 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.350 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.350 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.350 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.351 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.351 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.351 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.351 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.351 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.351 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.351 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.351 INFO:teuthology.orchestra.run.smithi032.stdout: "proxy": { 2023-09-30T15:03:33.351 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "proxy", 2023-09-30T15:03:33.351 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.351 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.351 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.352 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.352 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.352 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.352 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.352 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.352 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.352 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.352 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.352 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.352 INFO:teuthology.orchestra.run.smithi032.stdout: "url": { 2023-09-30T15:03:33.352 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "url", 2023-09-30T15:03:33.352 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.352 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.353 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.353 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-30T15:03:33.353 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.353 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.353 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.353 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.353 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.353 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.353 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.353 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.353 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.354 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.354 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "test_orchestrator", 2023-09-30T15:03:33.354 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.354 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.354 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.354 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.354 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.354 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.354 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.354 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.354 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.354 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.355 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.355 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.355 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.355 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.355 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.355 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.355 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.355 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.355 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.355 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.355 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.355 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.355 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.356 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.356 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.356 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.356 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.356 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.356 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.356 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.356 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.356 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.356 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.356 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.356 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.357 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.357 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.357 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.357 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.357 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.357 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.357 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.357 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.357 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.357 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.357 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.357 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.357 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.358 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.358 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.358 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.358 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.358 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.358 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.358 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.358 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.358 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.358 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.358 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.358 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.359 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.359 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.359 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.359 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.359 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.359 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.359 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.359 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.359 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.359 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.359 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.359 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.359 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.360 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.360 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.360 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.360 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2023-09-30T15:03:33.360 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.360 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.360 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.360 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.360 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.360 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.360 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.360 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.361 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.361 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.361 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.361 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.361 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.361 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.361 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.361 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.361 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.361 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.361 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.361 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.361 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.362 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.362 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.362 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.362 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.362 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.362 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.362 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.362 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.362 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.362 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.362 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.362 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.363 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.363 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.363 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.363 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.363 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.363 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.363 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.363 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.363 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.363 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.363 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.363 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.364 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.364 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.364 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.364 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.364 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.364 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.364 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.364 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.364 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.364 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.364 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.364 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.364 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.365 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.365 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.365 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.365 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.365 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.365 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.365 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.365 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.365 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.365 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.365 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.365 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.366 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.366 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.366 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.366 INFO:teuthology.orchestra.run.smithi032.stdout: "max_concurrent_clones": { 2023-09-30T15:03:33.366 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "max_concurrent_clones", 2023-09-30T15:03:33.366 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.366 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.366 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.366 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "4", 2023-09-30T15:03:33.366 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.366 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.366 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-30T15:03:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "periodic_async_work": { 2023-09-30T15:03:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "periodic_async_work", 2023-09-30T15:03:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.367 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Periodically check for async work", 2023-09-30T15:03:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "snapshot_clone_delay": { 2023-09-30T15:03:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "snapshot_clone_delay", 2023-09-30T15:03:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.368 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "0", 2023-09-30T15:03:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-30T15:03:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.369 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix", 2023-09-30T15:03:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "can_run": true, 2023-09-30T15:03:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "error_string": "", 2023-09-30T15:03:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "module_options": { 2023-09-30T15:03:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "discovery_interval": { 2023-09-30T15:03:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "discovery_interval", 2023-09-30T15:03:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "uint", 2023-09-30T15:03:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "100", 2023-09-30T15:03:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.370 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "identifier": { 2023-09-30T15:03:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "identifier", 2023-09-30T15:03:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.371 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "interval": { 2023-09-30T15:03:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "interval", 2023-09-30T15:03:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "secs", 2023-09-30T15:03:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.372 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "60", 2023-09-30T15:03:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "log_level": { 2023-09-30T15:03:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_level", 2023-09-30T15:03:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.373 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.374 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster": { 2023-09-30T15:03:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster", 2023-09-30T15:03:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.375 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_cluster_level": { 2023-09-30T15:03:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_cluster_level", 2023-09-30T15:03:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "info", 2023-09-30T15:03:33.376 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [ 2023-09-30T15:03:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "", 2023-09-30T15:03:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "critical", 2023-09-30T15:03:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "debug", 2023-09-30T15:03:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "error", 2023-09-30T15:03:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "info", 2023-09-30T15:03:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "warning" 2023-09-30T15:03:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.377 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "log_to_file": { 2023-09-30T15:03:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "log_to_file", 2023-09-30T15:03:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "bool", 2023-09-30T15:03:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T15:03:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "False", 2023-09-30T15:03:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.378 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_host": { 2023-09-30T15:03:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_host", 2023-09-30T15:03:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "", 2023-09-30T15:03:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.379 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.380 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.380 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.380 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.380 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.380 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_port": { 2023-09-30T15:03:33.380 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_port", 2023-09-30T15:03:33.380 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "int", 2023-09-30T15:03:33.380 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.380 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.380 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "10051", 2023-09-30T15:03:33.380 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.380 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.381 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.381 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.381 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.381 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.381 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.381 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.381 INFO:teuthology.orchestra.run.smithi032.stdout: "zabbix_sender": { 2023-09-30T15:03:33.381 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "zabbix_sender", 2023-09-30T15:03:33.381 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "str", 2023-09-30T15:03:33.381 INFO:teuthology.orchestra.run.smithi032.stdout: "level": "advanced", 2023-09-30T15:03:33.381 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 0, 2023-09-30T15:03:33.381 INFO:teuthology.orchestra.run.smithi032.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-30T15:03:33.381 INFO:teuthology.orchestra.run.smithi032.stdout: "min": "", 2023-09-30T15:03:33.382 INFO:teuthology.orchestra.run.smithi032.stdout: "max": "", 2023-09-30T15:03:33.382 INFO:teuthology.orchestra.run.smithi032.stdout: "enum_allowed": [], 2023-09-30T15:03:33.382 INFO:teuthology.orchestra.run.smithi032.stdout: "desc": "", 2023-09-30T15:03:33.382 INFO:teuthology.orchestra.run.smithi032.stdout: "long_desc": "", 2023-09-30T15:03:33.382 INFO:teuthology.orchestra.run.smithi032.stdout: "tags": [], 2023-09-30T15:03:33.382 INFO:teuthology.orchestra.run.smithi032.stdout: "see_also": [] 2023-09-30T15:03:33.382 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.382 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.382 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.382 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.382 INFO:teuthology.orchestra.run.smithi032.stdout: "services": { 2023-09-30T15:03:33.382 INFO:teuthology.orchestra.run.smithi032.stdout: "dashboard": "https://172.21.15.32:8443/" 2023-09-30T15:03:33.383 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.383 INFO:teuthology.orchestra.run.smithi032.stdout: "always_on_modules": { 2023-09-30T15:03:33.383 INFO:teuthology.orchestra.run.smithi032.stdout: "octopus": [ 2023-09-30T15:03:33.383 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2023-09-30T15:03:33.383 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2023-09-30T15:03:33.383 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2023-09-30T15:03:33.383 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2023-09-30T15:03:33.383 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2023-09-30T15:03:33.383 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2023-09-30T15:03:33.383 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2023-09-30T15:03:33.383 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2023-09-30T15:03:33.383 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2023-09-30T15:03:33.383 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2023-09-30T15:03:33.384 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.384 INFO:teuthology.orchestra.run.smithi032.stdout: "pacific": [ 2023-09-30T15:03:33.384 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2023-09-30T15:03:33.384 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2023-09-30T15:03:33.384 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2023-09-30T15:03:33.384 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2023-09-30T15:03:33.384 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2023-09-30T15:03:33.384 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2023-09-30T15:03:33.384 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2023-09-30T15:03:33.384 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2023-09-30T15:03:33.384 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2023-09-30T15:03:33.384 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2023-09-30T15:03:33.385 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.385 INFO:teuthology.orchestra.run.smithi032.stdout: "quincy": [ 2023-09-30T15:03:33.385 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2023-09-30T15:03:33.385 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2023-09-30T15:03:33.385 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2023-09-30T15:03:33.385 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2023-09-30T15:03:33.385 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2023-09-30T15:03:33.385 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2023-09-30T15:03:33.385 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2023-09-30T15:03:33.385 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2023-09-30T15:03:33.385 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2023-09-30T15:03:33.385 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2023-09-30T15:03:33.385 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T15:03:33.386 INFO:teuthology.orchestra.run.smithi032.stdout: "reef": [ 2023-09-30T15:03:33.386 INFO:teuthology.orchestra.run.smithi032.stdout: "balancer", 2023-09-30T15:03:33.386 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2023-09-30T15:03:33.386 INFO:teuthology.orchestra.run.smithi032.stdout: "devicehealth", 2023-09-30T15:03:33.386 INFO:teuthology.orchestra.run.smithi032.stdout: "orchestrator", 2023-09-30T15:03:33.386 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscaler", 2023-09-30T15:03:33.386 INFO:teuthology.orchestra.run.smithi032.stdout: "progress", 2023-09-30T15:03:33.386 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd_support", 2023-09-30T15:03:33.386 INFO:teuthology.orchestra.run.smithi032.stdout: "status", 2023-09-30T15:03:33.386 INFO:teuthology.orchestra.run.smithi032.stdout: "telemetry", 2023-09-30T15:03:33.386 INFO:teuthology.orchestra.run.smithi032.stdout: "volumes" 2023-09-30T15:03:33.386 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T15:03:33.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.387 INFO:teuthology.orchestra.run.smithi032.stdout: "last_failure_osd_epoch": 3, 2023-09-30T15:03:33.387 INFO:teuthology.orchestra.run.smithi032.stdout: "active_clients": [ 2023-09-30T15:03:33.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.387 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "devicehealth", 2023-09-30T15:03:33.387 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T15:03:33.387 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.387 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-09-30T15:03:33.387 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2023-09-30T15:03:33.387 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3798689053 2023-09-30T15:03:33.387 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.387 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T15:03:33.387 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.388 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "libcephsqlite", 2023-09-30T15:03:33.388 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T15:03:33.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.388 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-09-30T15:03:33.388 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2023-09-30T15:03:33.388 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1806059555 2023-09-30T15:03:33.388 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.388 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T15:03:33.388 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.388 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.388 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "rbd_support", 2023-09-30T15:03:33.389 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T15:03:33.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.389 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-09-30T15:03:33.389 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2023-09-30T15:03:33.389 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3077947686 2023-09-30T15:03:33.389 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.389 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T15:03:33.389 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:33.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.389 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "volumes", 2023-09-30T15:03:33.389 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T15:03:33.389 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T15:03:33.390 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v2", 2023-09-30T15:03:33.390 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:0", 2023-09-30T15:03:33.390 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 1294075219 2023-09-30T15:03:33.390 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.390 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T15:03:33.390 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:33.390 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T15:03:33.390 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T15:03:33.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:31 smithi148 ceph-mon[102355]: pgmap v1044: 97 pgs: 97 active+clean; 785 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.8 MiB/s wr, 668 op/s 2023-09-30T15:03:33.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:31 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2571784561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T15:03:33.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:31 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/4025203489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T15:03:33.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:32 smithi148 ceph-mon[102355]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:03:33.392 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:32 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/856892262' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-30T15:03:33.392 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:31 smithi156 ceph-mon[103579]: pgmap v1044: 97 pgs: 97 active+clean; 785 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.8 MiB/s wr, 668 op/s 2023-09-30T15:03:33.392 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:31 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2571784561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T15:03:33.392 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:31 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/4025203489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T15:03:33.393 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:32 smithi156 ceph-mon[103579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:03:33.393 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:32 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/856892262' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-30T15:03:33.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.916+0000 7fc82aff0700 1 -- 172.21.15.32:0/856892262 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc8140a10a0 msgr2=0x7fc8140a3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:33.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.916+0000 7fc82aff0700 1 --2- 172.21.15.32:0/856892262 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc8140a10a0 0x7fc8140a3560 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc818009d20 tx=0x7fc818009260 comp rx=0 tx=0).stop 2023-09-30T15:03:33.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.916+0000 7fc82aff0700 1 -- 172.21.15.32:0/856892262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8241042c0 msgr2=0x7fc824070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:03:33.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.916+0000 7fc82aff0700 1 --2- 172.21.15.32:0/856892262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8241042c0 0x7fc824070cf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc81000e9f0 tx=0x7fc81000ed00 comp rx=0 tx=0).stop 2023-09-30T15:03:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.916+0000 7fc82aff0700 1 -- 172.21.15.32:0/856892262 shutdown_connections 2023-09-30T15:03:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.916+0000 7fc82aff0700 1 --2- 172.21.15.32:0/856892262 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc8140a10a0 0x7fc8140a3560 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.916+0000 7fc82aff0700 1 --2- 172.21.15.32:0/856892262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc824106c40 0x7fc824071d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.916+0000 7fc82aff0700 1 --2- 172.21.15.32:0/856892262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc824101990 0x7fc8240707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.916+0000 7fc82aff0700 1 --2- 172.21.15.32:0/856892262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8241042c0 0x7fc824070cf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:03:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.916+0000 7fc82aff0700 1 -- 172.21.15.32:0/856892262 >> 172.21.15.32:0/856892262 conn(0x7fc8240fb2e0 msgr2=0x7fc824102600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:03:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.916+0000 7fc82aff0700 1 -- 172.21.15.32:0/856892262 shutdown_connections 2023-09-30T15:03:33.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:03:31.916+0000 7fc82aff0700 1 -- 172.21.15.32:0/856892262 wait complete. 2023-09-30T15:03:33.400 DEBUG:tasks.check_counter:Getting stats from a 2023-09-30T15:03:33.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-09-30T15:03:33.401 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:32 smithi032 ceph-mon[98592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-30T15:03:33.401 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:32 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/856892262' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-30T15:03:33.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:33 smithi032 ceph-mon[98592]: pgmap v1045: 97 pgs: 97 active+clean; 735 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.6 MiB/s wr, 540 op/s 2023-09-30T15:03:34.011 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T15:03:34.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:33 smithi156 ceph-mon[103579]: pgmap v1045: 97 pgs: 97 active+clean; 735 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.6 MiB/s wr, 540 op/s 2023-09-30T15:03:34.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:33 smithi148 ceph-mon[102355]: pgmap v1045: 97 pgs: 97 active+clean; 735 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.6 MiB/s wr, 540 op/s 2023-09-30T15:03:35.342 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T15:03:35.342 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-0": { 2023-09-30T15:03:35.342 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 0, 2023-09-30T15:03:35.342 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 2, 2023-09-30T15:03:35.342 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 0, 2023-09-30T15:03:35.342 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 0, 2023-09-30T15:03:35.342 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2023-09-30T15:03:35.342 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 0, 2023-09-30T15:03:35.343 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.001691620, 2023-09-30T15:03:35.343 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.000000000, 2023-09-30T15:03:35.343 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.000333153, 2023-09-30T15:03:35.343 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-30T15:03:35.343 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:35.343 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:35.343 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:35.343 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:35.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:35.343 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:35.343 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:35.344 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:35.344 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:35.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:35.344 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-30T15:03:35.344 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 0 2023-09-30T15:03:35.344 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:35.344 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-1": { 2023-09-30T15:03:35.344 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 2, 2023-09-30T15:03:35.344 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 323, 2023-09-30T15:03:35.344 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 320, 2023-09-30T15:03:35.344 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 210032, 2023-09-30T15:03:35.345 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2023-09-30T15:03:35.345 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 2, 2023-09-30T15:03:35.345 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.019579234, 2023-09-30T15:03:35.345 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.015232667, 2023-09-30T15:03:35.345 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.001564242, 2023-09-30T15:03:35.345 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-30T15:03:35.345 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:35.345 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 321, 2023-09-30T15:03:35.345 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.029641916, 2023-09-30T15:03:35.345 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000092342 2023-09-30T15:03:35.345 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:35.346 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:35.346 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:35.346 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:35.346 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:35.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:35.346 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-30T15:03:35.346 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 210032 2023-09-30T15:03:35.346 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:35.346 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-2": { 2023-09-30T15:03:35.346 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 429, 2023-09-30T15:03:35.346 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 424, 2023-09-30T15:03:35.347 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 533728, 2023-09-30T15:03:35.347 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 255104, 2023-09-30T15:03:35.347 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 2, 2023-09-30T15:03:35.347 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 1, 2023-09-30T15:03:35.347 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.052833060, 2023-09-30T15:03:35.347 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.021287700, 2023-09-30T15:03:35.347 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.018928861, 2023-09-30T15:03:35.347 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.009660390, 2023-09-30T15:03:35.347 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:35.347 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 424, 2023-09-30T15:03:35.347 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.030335006, 2023-09-30T15:03:35.347 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000071544 2023-09-30T15:03:35.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:35.348 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:35.348 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:35.348 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:35.348 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:35.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:35.348 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 533728, 2023-09-30T15:03:35.348 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 255104 2023-09-30T15:03:35.348 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:35.348 INFO:teuthology.orchestra.run.smithi032.stdout: "cct": { 2023-09-30T15:03:35.348 INFO:teuthology.orchestra.run.smithi032.stdout: "total_workers": 0, 2023-09-30T15:03:35.348 INFO:teuthology.orchestra.run.smithi032.stdout: "unhealthy_workers": 0 2023-09-30T15:03:35.349 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:35.349 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool": { 2023-09-30T15:03:35.349 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_bytes": 0, 2023-09-30T15:03:35.349 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_items": 0, 2023-09-30T15:03:35.349 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_bytes": 0, 2023-09-30T15:03:35.349 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_items": 0, 2023-09-30T15:03:35.349 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_bytes": 0, 2023-09-30T15:03:35.349 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_items": 0, 2023-09-30T15:03:35.349 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-30T15:03:35.349 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_items": 0, 2023-09-30T15:03:35.349 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-30T15:03:35.349 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_items": 0, 2023-09-30T15:03:35.349 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_bytes": 0, 2023-09-30T15:03:35.350 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_items": 0, 2023-09-30T15:03:35.350 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-30T15:03:35.350 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_items": 0, 2023-09-30T15:03:35.350 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_bytes": 0, 2023-09-30T15:03:35.350 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_items": 0, 2023-09-30T15:03:35.350 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_bytes": 0, 2023-09-30T15:03:35.350 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_items": 0, 2023-09-30T15:03:35.350 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-30T15:03:35.350 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_items": 0, 2023-09-30T15:03:35.350 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-30T15:03:35.350 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_items": 0, 2023-09-30T15:03:35.351 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_bytes": 0, 2023-09-30T15:03:35.351 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_items": 0, 2023-09-30T15:03:35.351 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_bytes": 0, 2023-09-30T15:03:35.351 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_items": 0, 2023-09-30T15:03:35.351 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-30T15:03:35.351 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_items": 0, 2023-09-30T15:03:35.351 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_bytes": 0, 2023-09-30T15:03:35.351 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_items": 0, 2023-09-30T15:03:35.351 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_bytes": 0, 2023-09-30T15:03:35.351 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_items": 0, 2023-09-30T15:03:35.351 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_bytes": 0, 2023-09-30T15:03:35.351 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_items": 0, 2023-09-30T15:03:35.351 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_bytes": 0, 2023-09-30T15:03:35.352 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_items": 0, 2023-09-30T15:03:35.352 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_bytes": 148744, 2023-09-30T15:03:35.352 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_items": 30, 2023-09-30T15:03:35.352 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_bytes": 0, 2023-09-30T15:03:35.352 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_items": 0, 2023-09-30T15:03:35.352 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bytes": 0, 2023-09-30T15:03:35.352 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_items": 0, 2023-09-30T15:03:35.352 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_bytes": 0, 2023-09-30T15:03:35.352 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_items": 0, 2023-09-30T15:03:35.352 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_bytes": 0, 2023-09-30T15:03:35.352 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_items": 0, 2023-09-30T15:03:35.352 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_bytes": 0, 2023-09-30T15:03:35.353 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_items": 0, 2023-09-30T15:03:35.353 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_bytes": 0, 2023-09-30T15:03:35.353 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_items": 0, 2023-09-30T15:03:35.353 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_bytes": 0, 2023-09-30T15:03:35.353 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_items": 0, 2023-09-30T15:03:35.353 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_bytes": 928, 2023-09-30T15:03:35.353 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_items": 2, 2023-09-30T15:03:35.353 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_bytes": 0, 2023-09-30T15:03:35.353 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_items": 0, 2023-09-30T15:03:35.353 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_bytes": 0, 2023-09-30T15:03:35.353 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_items": 0 2023-09-30T15:03:35.353 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:35.354 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-30T15:03:35.354 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2023-09-30T15:03:35.354 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2023-09-30T15:03:35.354 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2023-09-30T15:03:35.354 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 427, 2023-09-30T15:03:35.354 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 470679, 2023-09-30T15:03:35.354 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:35.354 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 427, 2023-09-30T15:03:35.354 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2023-09-30T15:03:35.354 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2023-09-30T15:03:35.354 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 427, 2023-09-30T15:03:35.354 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 470679, 2023-09-30T15:03:35.355 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2023-09-30T15:03:35.355 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:35.355 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:35.355 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:35.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:35.355 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:35.355 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T15:03:35.767 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-09-30T15:03:35.768 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-09-30T15:03:35.768 DEBUG:tasks.check_counter:Getting stats from d 2023-09-30T15:03:35.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-09-30T15:03:36.061 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:35 smithi156 ceph-mon[103579]: pgmap v1046: 97 pgs: 97 active+clean; 687 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.8 MiB/s wr, 499 op/s 2023-09-30T15:03:36.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:35 smithi148 ceph-mon[102355]: pgmap v1046: 97 pgs: 97 active+clean; 687 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.8 MiB/s wr, 499 op/s 2023-09-30T15:03:36.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:35 smithi032 ceph-mon[98592]: pgmap v1046: 97 pgs: 97 active+clean; 687 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.8 MiB/s wr, 499 op/s 2023-09-30T15:03:36.414 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T15:03:38.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:37 smithi156 ceph-mon[103579]: pgmap v1047: 97 pgs: 97 active+clean; 687 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.2 MiB/s wr, 416 op/s 2023-09-30T15:03:38.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:37 smithi148 ceph-mon[102355]: pgmap v1047: 97 pgs: 97 active+clean; 687 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.2 MiB/s wr, 416 op/s 2023-09-30T15:03:38.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:37 smithi032 ceph-mon[98592]: pgmap v1047: 97 pgs: 97 active+clean; 687 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.2 MiB/s wr, 416 op/s 2023-09-30T15:03:38.172 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T15:03:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-0": { 2023-09-30T15:03:38.172 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 430, 2023-09-30T15:03:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 427, 2023-09-30T15:03:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 532144, 2023-09-30T15:03:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 257776, 2023-09-30T15:03:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2023-09-30T15:03:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 0, 2023-09-30T15:03:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.053752157, 2023-09-30T15:03:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.021285021, 2023-09-30T15:03:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.018940638, 2023-09-30T15:03:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.009455523, 2023-09-30T15:03:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 426, 2023-09-30T15:03:38.173 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.019643663, 2023-09-30T15:03:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000046111 2023-09-30T15:03:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 532144, 2023-09-30T15:03:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 257776 2023-09-30T15:03:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-1": { 2023-09-30T15:03:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 6, 2023-09-30T15:03:38.174 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 329, 2023-09-30T15:03:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 3264, 2023-09-30T15:03:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 211184, 2023-09-30T15:03:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2023-09-30T15:03:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 2, 2023-09-30T15:03:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.020317681, 2023-09-30T15:03:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.015441362, 2023-09-30T15:03:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.002092897, 2023-09-30T15:03:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-30T15:03:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 328, 2023-09-30T15:03:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.027803457, 2023-09-30T15:03:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000084766 2023-09-30T15:03:38.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-09-30T15:03:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 211184 2023-09-30T15:03:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-2": { 2023-09-30T15:03:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 0, 2023-09-30T15:03:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 2, 2023-09-30T15:03:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 0, 2023-09-30T15:03:38.176 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 0, 2023-09-30T15:03:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 2, 2023-09-30T15:03:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 1, 2023-09-30T15:03:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.000559371, 2023-09-30T15:03:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.000000000, 2023-09-30T15:03:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.000460424, 2023-09-30T15:03:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-30T15:03:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:38.177 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-30T15:03:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 0 2023-09-30T15:03:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "cct": { 2023-09-30T15:03:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "total_workers": 0, 2023-09-30T15:03:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "unhealthy_workers": 0 2023-09-30T15:03:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool": { 2023-09-30T15:03:38.178 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_bytes": 0, 2023-09-30T15:03:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_items": 0, 2023-09-30T15:03:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_bytes": 0, 2023-09-30T15:03:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_items": 0, 2023-09-30T15:03:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_bytes": 0, 2023-09-30T15:03:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_items": 0, 2023-09-30T15:03:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-30T15:03:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_items": 0, 2023-09-30T15:03:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-30T15:03:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_items": 0, 2023-09-30T15:03:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_bytes": 0, 2023-09-30T15:03:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_items": 0, 2023-09-30T15:03:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-30T15:03:38.179 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_items": 0, 2023-09-30T15:03:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_bytes": 0, 2023-09-30T15:03:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_items": 0, 2023-09-30T15:03:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_bytes": 0, 2023-09-30T15:03:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_items": 0, 2023-09-30T15:03:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-30T15:03:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_items": 0, 2023-09-30T15:03:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-30T15:03:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_items": 0, 2023-09-30T15:03:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_bytes": 0, 2023-09-30T15:03:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_items": 0, 2023-09-30T15:03:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_bytes": 0, 2023-09-30T15:03:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_items": 0, 2023-09-30T15:03:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-30T15:03:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_items": 0, 2023-09-30T15:03:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_bytes": 0, 2023-09-30T15:03:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_items": 0, 2023-09-30T15:03:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_bytes": 0, 2023-09-30T15:03:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_items": 0, 2023-09-30T15:03:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_bytes": 0, 2023-09-30T15:03:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_items": 0, 2023-09-30T15:03:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_bytes": 0, 2023-09-30T15:03:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_items": 0, 2023-09-30T15:03:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_bytes": 148704, 2023-09-30T15:03:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_items": 29, 2023-09-30T15:03:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_bytes": 0, 2023-09-30T15:03:38.181 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_items": 0, 2023-09-30T15:03:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bytes": 0, 2023-09-30T15:03:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_items": 0, 2023-09-30T15:03:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_bytes": 0, 2023-09-30T15:03:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_items": 0, 2023-09-30T15:03:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_bytes": 0, 2023-09-30T15:03:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_items": 0, 2023-09-30T15:03:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_bytes": 0, 2023-09-30T15:03:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_items": 0, 2023-09-30T15:03:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_bytes": 0, 2023-09-30T15:03:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_items": 0, 2023-09-30T15:03:38.182 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_bytes": 0, 2023-09-30T15:03:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_items": 0, 2023-09-30T15:03:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_bytes": 928, 2023-09-30T15:03:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_items": 2, 2023-09-30T15:03:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_bytes": 0, 2023-09-30T15:03:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_items": 0, 2023-09-30T15:03:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_bytes": 0, 2023-09-30T15:03:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_items": 0 2023-09-30T15:03:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-30T15:03:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2023-09-30T15:03:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2023-09-30T15:03:38.183 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2023-09-30T15:03:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 432, 2023-09-30T15:03:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 471329, 2023-09-30T15:03:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 432, 2023-09-30T15:03:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2023-09-30T15:03:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2023-09-30T15:03:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 432, 2023-09-30T15:03:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 471329, 2023-09-30T15:03:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2023-09-30T15:03:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:38.184 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:38.185 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:38.185 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:38.185 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T15:03:38.569 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-09-30T15:03:38.569 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-09-30T15:03:38.569 DEBUG:tasks.check_counter:Getting stats from g 2023-09-30T15:03:38.570 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-09-30T15:03:39.134 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T15:03:40.020 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:39 smithi032 ceph-mon[98592]: pgmap v1048: 97 pgs: 97 active+clean; 687 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 308 op/s 2023-09-30T15:03:40.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:39 smithi156 ceph-mon[103579]: pgmap v1048: 97 pgs: 97 active+clean; 687 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 308 op/s 2023-09-30T15:03:40.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:39 smithi148 ceph-mon[102355]: pgmap v1048: 97 pgs: 97 active+clean; 687 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 308 op/s 2023-09-30T15:03:40.642 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T15:03:40.643 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-0": { 2023-09-30T15:03:40.643 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 4, 2023-09-30T15:03:40.643 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 4, 2023-09-30T15:03:40.643 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 2944, 2023-09-30T15:03:40.643 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 448, 2023-09-30T15:03:40.643 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2023-09-30T15:03:40.643 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 0, 2023-09-30T15:03:40.643 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.002055083, 2023-09-30T15:03:40.643 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.000047982, 2023-09-30T15:03:40.643 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.000806589, 2023-09-30T15:03:40.644 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-30T15:03:40.644 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:40.644 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 3, 2023-09-30T15:03:40.644 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000943819, 2023-09-30T15:03:40.644 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000314606 2023-09-30T15:03:40.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:40.644 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:40.644 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:40.644 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:40.644 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:40.644 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:40.644 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-09-30T15:03:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 448 2023-09-30T15:03:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-1": { 2023-09-30T15:03:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 2, 2023-09-30T15:03:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 324, 2023-09-30T15:03:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 320, 2023-09-30T15:03:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 210208, 2023-09-30T15:03:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2023-09-30T15:03:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 2, 2023-09-30T15:03:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.019282605, 2023-09-30T15:03:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.014844864, 2023-09-30T15:03:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.001711810, 2023-09-30T15:03:40.645 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-30T15:03:40.646 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:40.646 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 322, 2023-09-30T15:03:40.646 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.020507718, 2023-09-30T15:03:40.646 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000063688 2023-09-30T15:03:40.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:40.646 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:40.646 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:40.646 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:40.646 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:40.646 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:40.646 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-30T15:03:40.646 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 210208 2023-09-30T15:03:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-2": { 2023-09-30T15:03:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 427, 2023-09-30T15:03:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 424, 2023-09-30T15:03:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 531328, 2023-09-30T15:03:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 255904, 2023-09-30T15:03:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 2, 2023-09-30T15:03:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 1, 2023-09-30T15:03:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.051892523, 2023-09-30T15:03:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.021014799, 2023-09-30T15:03:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.018604927, 2023-09-30T15:03:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.009063835, 2023-09-30T15:03:40.647 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:40.648 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 423, 2023-09-30T15:03:40.648 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.023477415, 2023-09-30T15:03:40.648 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000055502 2023-09-30T15:03:40.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:40.648 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:40.648 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:40.648 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:40.648 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:40.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:40.648 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 531328, 2023-09-30T15:03:40.648 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 255904 2023-09-30T15:03:40.648 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:40.649 INFO:teuthology.orchestra.run.smithi032.stdout: "cct": { 2023-09-30T15:03:40.649 INFO:teuthology.orchestra.run.smithi032.stdout: "total_workers": 0, 2023-09-30T15:03:40.649 INFO:teuthology.orchestra.run.smithi032.stdout: "unhealthy_workers": 0 2023-09-30T15:03:40.649 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:40.649 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool": { 2023-09-30T15:03:40.649 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_bytes": 0, 2023-09-30T15:03:40.649 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_items": 0, 2023-09-30T15:03:40.649 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_bytes": 0, 2023-09-30T15:03:40.649 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_items": 0, 2023-09-30T15:03:40.649 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_bytes": 0, 2023-09-30T15:03:40.649 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_items": 0, 2023-09-30T15:03:40.649 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-30T15:03:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_items": 0, 2023-09-30T15:03:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-30T15:03:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_items": 0, 2023-09-30T15:03:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_bytes": 0, 2023-09-30T15:03:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_items": 0, 2023-09-30T15:03:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-30T15:03:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_items": 0, 2023-09-30T15:03:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_bytes": 0, 2023-09-30T15:03:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_items": 0, 2023-09-30T15:03:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_bytes": 0, 2023-09-30T15:03:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_items": 0, 2023-09-30T15:03:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-30T15:03:40.650 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_items": 0, 2023-09-30T15:03:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-30T15:03:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_items": 0, 2023-09-30T15:03:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_bytes": 0, 2023-09-30T15:03:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_items": 0, 2023-09-30T15:03:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_bytes": 0, 2023-09-30T15:03:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_items": 0, 2023-09-30T15:03:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-30T15:03:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_items": 0, 2023-09-30T15:03:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_bytes": 0, 2023-09-30T15:03:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_items": 0, 2023-09-30T15:03:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_bytes": 0, 2023-09-30T15:03:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_items": 0, 2023-09-30T15:03:40.651 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_bytes": 0, 2023-09-30T15:03:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_items": 0, 2023-09-30T15:03:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_bytes": 0, 2023-09-30T15:03:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_items": 0, 2023-09-30T15:03:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_bytes": 148704, 2023-09-30T15:03:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_items": 29, 2023-09-30T15:03:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_bytes": 0, 2023-09-30T15:03:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_items": 0, 2023-09-30T15:03:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bytes": 0, 2023-09-30T15:03:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_items": 0, 2023-09-30T15:03:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_bytes": 0, 2023-09-30T15:03:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_items": 0, 2023-09-30T15:03:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_bytes": 0, 2023-09-30T15:03:40.652 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_items": 0, 2023-09-30T15:03:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_bytes": 0, 2023-09-30T15:03:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_items": 0, 2023-09-30T15:03:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_bytes": 0, 2023-09-30T15:03:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_items": 0, 2023-09-30T15:03:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_bytes": 0, 2023-09-30T15:03:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_items": 0, 2023-09-30T15:03:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_bytes": 928, 2023-09-30T15:03:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_items": 2, 2023-09-30T15:03:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_bytes": 0, 2023-09-30T15:03:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_items": 0, 2023-09-30T15:03:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_bytes": 0, 2023-09-30T15:03:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_items": 0 2023-09-30T15:03:40.653 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-30T15:03:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2023-09-30T15:03:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2023-09-30T15:03:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2023-09-30T15:03:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 429, 2023-09-30T15:03:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 470939, 2023-09-30T15:03:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 429, 2023-09-30T15:03:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2023-09-30T15:03:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2023-09-30T15:03:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 429, 2023-09-30T15:03:40.654 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 470939, 2023-09-30T15:03:40.655 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2023-09-30T15:03:40.655 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:40.655 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:40.655 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:40.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:40.655 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:40.655 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T15:03:41.167 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-09-30T15:03:41.167 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-09-30T15:03:41.167 DEBUG:tasks.check_counter:Getting stats from j 2023-09-30T15:03:41.167 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-09-30T15:03:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.a/config 2023-09-30T15:03:42.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:41 smithi156 ceph-mon[103579]: pgmap v1049: 97 pgs: 97 active+clean; 671 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.4 MiB/s wr, 329 op/s 2023-09-30T15:03:42.060 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:41 smithi032 ceph-mon[98592]: pgmap v1049: 97 pgs: 97 active+clean; 671 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.4 MiB/s wr, 329 op/s 2023-09-30T15:03:42.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:41 smithi148 ceph-mon[102355]: pgmap v1049: 97 pgs: 97 active+clean; 671 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.4 MiB/s wr, 329 op/s 2023-09-30T15:03:43.221 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T15:03:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-0": { 2023-09-30T15:03:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 0, 2023-09-30T15:03:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 2, 2023-09-30T15:03:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 0, 2023-09-30T15:03:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 0, 2023-09-30T15:03:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2023-09-30T15:03:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 0, 2023-09-30T15:03:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.001635489, 2023-09-30T15:03:43.222 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.000000000, 2023-09-30T15:03:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.000479214, 2023-09-30T15:03:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-30T15:03:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:43.223 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-30T15:03:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 0 2023-09-30T15:03:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-1": { 2023-09-30T15:03:43.224 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 432, 2023-09-30T15:03:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 429, 2023-09-30T15:03:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 532688, 2023-09-30T15:03:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 259024, 2023-09-30T15:03:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 2, 2023-09-30T15:03:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 1, 2023-09-30T15:03:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.053265430, 2023-09-30T15:03:43.225 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.021765671, 2023-09-30T15:03:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.018560478, 2023-09-30T15:03:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.009555495, 2023-09-30T15:03:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 428, 2023-09-30T15:03:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.030296325, 2023-09-30T15:03:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000070785 2023-09-30T15:03:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:43.226 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 532688, 2023-09-30T15:03:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 259024 2023-09-30T15:03:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "AsyncMessenger::Worker-2": { 2023-09-30T15:03:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_messages": 6, 2023-09-30T15:03:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages": 330, 2023-09-30T15:03:43.227 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_bytes": 3264, 2023-09-30T15:03:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_bytes": 211360, 2023-09-30T15:03:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_created_connections": 4, 2023-09-30T15:03:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_active_connections": 2, 2023-09-30T15:03:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_total_time": 0.019329238, 2023-09-30T15:03:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_send_time": 0.014726956, 2023-09-30T15:03:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_recv_time": 0.001942514, 2023-09-30T15:03:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-30T15:03:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 329, 2023-09-30T15:03:43.228 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.033837252, 2023-09-30T15:03:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000102848 2023-09-30T15:03:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-09-30T15:03:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: "msgr_send_encrypted_bytes": 211360 2023-09-30T15:03:43.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "cct": { 2023-09-30T15:03:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "total_workers": 0, 2023-09-30T15:03:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "unhealthy_workers": 0 2023-09-30T15:03:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "mempool": { 2023-09-30T15:03:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_bytes": 0, 2023-09-30T15:03:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "bloom_filter_items": 0, 2023-09-30T15:03:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_bytes": 0, 2023-09-30T15:03:43.230 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_alloc_items": 0, 2023-09-30T15:03:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_bytes": 0, 2023-09-30T15:03:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_data_items": 0, 2023-09-30T15:03:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-30T15:03:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_onode_items": 0, 2023-09-30T15:03:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-30T15:03:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_meta_items": 0, 2023-09-30T15:03:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_bytes": 0, 2023-09-30T15:03:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_other_items": 0, 2023-09-30T15:03:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-30T15:03:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_cache_buffer_items": 0, 2023-09-30T15:03:43.231 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_bytes": 0, 2023-09-30T15:03:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_extent_items": 0, 2023-09-30T15:03:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_bytes": 0, 2023-09-30T15:03:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_blob_items": 0, 2023-09-30T15:03:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-30T15:03:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_shared_blob_items": 0, 2023-09-30T15:03:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-30T15:03:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_inline_bl_items": 0, 2023-09-30T15:03:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_bytes": 0, 2023-09-30T15:03:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_fsck_items": 0, 2023-09-30T15:03:43.232 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_bytes": 0, 2023-09-30T15:03:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_txc_items": 0, 2023-09-30T15:03:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-30T15:03:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_deferred_items": 0, 2023-09-30T15:03:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_bytes": 0, 2023-09-30T15:03:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "bluestore_writing_items": 0, 2023-09-30T15:03:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_bytes": 0, 2023-09-30T15:03:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_items": 0, 2023-09-30T15:03:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_bytes": 0, 2023-09-30T15:03:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_reader_items": 0, 2023-09-30T15:03:43.233 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_bytes": 0, 2023-09-30T15:03:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "bluefs_file_writer_items": 0, 2023-09-30T15:03:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_bytes": 149713, 2023-09-30T15:03:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_anon_items": 30, 2023-09-30T15:03:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_bytes": 0, 2023-09-30T15:03:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "buffer_meta_items": 0, 2023-09-30T15:03:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_bytes": 0, 2023-09-30T15:03:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_items": 0, 2023-09-30T15:03:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_bytes": 0, 2023-09-30T15:03:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_mapbl_items": 0, 2023-09-30T15:03:43.234 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_bytes": 0, 2023-09-30T15:03:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_pglog_items": 0, 2023-09-30T15:03:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_bytes": 0, 2023-09-30T15:03:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_items": 0, 2023-09-30T15:03:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_bytes": 0, 2023-09-30T15:03:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_mapping_items": 0, 2023-09-30T15:03:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_bytes": 0, 2023-09-30T15:03:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "pgmap_items": 0, 2023-09-30T15:03:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_bytes": 928, 2023-09-30T15:03:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_co_items": 2, 2023-09-30T15:03:43.235 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_bytes": 0, 2023-09-30T15:03:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_1_items": 0, 2023-09-30T15:03:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_bytes": 0, 2023-09-30T15:03:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "unittest_2_items": 0 2023-09-30T15:03:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T15:03:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-30T15:03:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "val": 0, 2023-09-30T15:03:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "max": 104857600, 2023-09-30T15:03:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "get_started": 0, 2023-09-30T15:03:43.236 INFO:teuthology.orchestra.run.smithi032.stdout: "get": 434, 2023-09-30T15:03:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "get_sum": 471589, 2023-09-30T15:03:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "get_or_fail_success": 434, 2023-09-30T15:03:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "take": 0, 2023-09-30T15:03:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "take_sum": 0, 2023-09-30T15:03:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "put": 434, 2023-09-30T15:03:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "put_sum": 471589, 2023-09-30T15:03:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "wait": { 2023-09-30T15:03:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "avgcount": 0, 2023-09-30T15:03:43.237 INFO:teuthology.orchestra.run.smithi032.stdout: "sum": 0.000000000, 2023-09-30T15:03:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: "avgtime": 0.000000000 2023-09-30T15:03:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:43.238 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T15:03:43.238 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T15:03:43.627 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-09-30T15:03:43.627 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-09-30T15:03:43.627 DEBUG:tasks.check_counter:Getting stats from b 2023-09-30T15:03:43.627 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-09-30T15:03:43.887 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:43 smithi032 ceph-mon[98592]: pgmap v1050: 97 pgs: 97 active+clean; 667 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 5.4 MiB/s wr, 236 op/s 2023-09-30T15:03:44.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:43 smithi156 ceph-mon[103579]: pgmap v1050: 97 pgs: 97 active+clean; 667 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 5.4 MiB/s wr, 236 op/s 2023-09-30T15:03:44.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:43 smithi148 ceph-mon[102355]: pgmap v1050: 97 pgs: 97 active+clean; 667 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 5.4 MiB/s wr, 236 op/s 2023-09-30T15:03:44.355 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.b/config 2023-09-30T15:03:45.735 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-09-30T15:03:45.735 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2023-09-30T15:03:45.735 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 313668, 2023-09-30T15:03:45.736 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 310784, 2023-09-30T15:03:45.736 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 83556651, 2023-09-30T15:03:45.736 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 329755897, 2023-09-30T15:03:45.736 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 11, 2023-09-30T15:03:45.736 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2023-09-30T15:03:45.736 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 20.033525308, 2023-09-30T15:03:45.736 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 8.276156019, 2023-09-30T15:03:45.736 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 10.618549838, 2023-09-30T15:03:45.736 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 2.875402224, 2023-09-30T15:03:45.736 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:45.737 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 310782, 2023-09-30T15:03:45.737 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 108.916968587, 2023-09-30T15:03:45.737 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000350460 2023-09-30T15:03:45.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.737 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:45.737 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 471363, 2023-09-30T15:03:45.737 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.075026149, 2023-09-30T15:03:45.737 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000159 2023-09-30T15:03:45.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.737 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-30T15:03:45.737 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2023-09-30T15:03:45.737 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.738 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2023-09-30T15:03:45.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 74944, 2023-09-30T15:03:45.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 74977, 2023-09-30T15:03:45.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 24786854, 2023-09-30T15:03:45.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 289682533, 2023-09-30T15:03:45.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 145, 2023-09-30T15:03:45.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 5, 2023-09-30T15:03:45.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 6.237434886, 2023-09-30T15:03:45.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 1.759590820, 2023-09-30T15:03:45.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 1.109981173, 2023-09-30T15:03:45.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 3.069381253, 2023-09-30T15:03:45.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:45.739 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 74976, 2023-09-30T15:03:45.739 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 83.784263744, 2023-09-30T15:03:45.739 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001117481 2023-09-30T15:03:45.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.739 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:45.739 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 333, 2023-09-30T15:03:45.739 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000040895, 2023-09-30T15:03:45.739 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000122 2023-09-30T15:03:45.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.739 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 772832, 2023-09-30T15:03:45.739 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 633920 2023-09-30T15:03:45.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.740 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2023-09-30T15:03:45.740 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 572067, 2023-09-30T15:03:45.740 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 890196, 2023-09-30T15:03:45.740 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 187948163, 2023-09-30T15:03:45.740 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 836143203, 2023-09-30T15:03:45.740 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 112, 2023-09-30T15:03:45.740 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 5, 2023-09-30T15:03:45.740 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 37.859158642, 2023-09-30T15:03:45.740 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 17.153225479, 2023-09-30T15:03:45.740 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 29.131722618, 2023-09-30T15:03:45.740 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 1.450994425, 2023-09-30T15:03:45.740 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:45.741 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 890195, 2023-09-30T15:03:45.741 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 200.532889612, 2023-09-30T15:03:45.741 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000225268 2023-09-30T15:03:45.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.741 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:45.741 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1224159, 2023-09-30T15:03:45.741 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.221589960, 2023-09-30T15:03:45.741 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000181 2023-09-30T15:03:45.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.741 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 163627040, 2023-09-30T15:03:45.741 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 746796688 2023-09-30T15:03:45.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.742 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2023-09-30T15:03:45.742 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2023-09-30T15:03:45.742 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2023-09-30T15:03:45.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.742 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2023-09-30T15:03:45.742 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2023-09-30T15:03:45.742 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2023-09-30T15:03:45.742 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 132152, 2023-09-30T15:03:45.742 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 225.369623362, 2023-09-30T15:03:45.742 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001705381 2023-09-30T15:03:45.742 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:45.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.743 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2023-09-30T15:03:45.743 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2023-09-30T15:03:45.743 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2023-09-30T15:03:45.743 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 84965, 2023-09-30T15:03:45.743 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 19.197717838, 2023-09-30T15:03:45.743 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000225948 2023-09-30T15:03:45.743 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:45.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.744 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2023-09-30T15:03:45.744 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 361853, 2023-09-30T15:03:45.744 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 342713, 2023-09-30T15:03:45.744 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2023-09-30T15:03:45.744 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 342713, 2023-09-30T15:03:45.744 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1692.021828117, 2023-09-30T15:03:45.744 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.004937139 2023-09-30T15:03:45.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.744 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2023-09-30T15:03:45.744 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 19140, 2023-09-30T15:03:45.744 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2023-09-30T15:03:45.744 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2023-09-30T15:03:45.745 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 4439, 2023-09-30T15:03:45.745 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 17, 2023-09-30T15:03:45.745 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 27, 2023-09-30T15:03:45.745 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 2320, 2023-09-30T15:03:45.745 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 461, 2023-09-30T15:03:45.745 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 414, 2023-09-30T15:03:45.745 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 1445, 2023-09-30T15:03:45.745 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 1661, 2023-09-30T15:03:45.745 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 174, 2023-09-30T15:03:45.745 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 1047, 2023-09-30T15:03:45.745 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 1047, 2023-09-30T15:03:45.745 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 5, 2023-09-30T15:03:45.745 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 561387, 2023-09-30T15:03:45.746 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 308440, 2023-09-30T15:03:45.746 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 19140, 2023-09-30T15:03:45.746 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 64, 2023-09-30T15:03:45.746 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2023-09-30T15:03:45.746 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2023-09-30T15:03:45.746 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 3650, 2023-09-30T15:03:45.746 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 25244, 2023-09-30T15:03:45.746 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 17, 2023-09-30T15:03:45.746 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 7, 2023-09-30T15:03:45.746 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 68, 2023-09-30T15:03:45.746 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 5, 2023-09-30T15:03:45.746 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 546, 2023-09-30T15:03:45.747 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2023-09-30T15:03:45.747 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 23, 2023-09-30T15:03:45.747 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2023-09-30T15:03:45.747 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 134, 2023-09-30T15:03:45.747 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 14091879, 2023-09-30T15:03:45.747 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2023-09-30T15:03:45.747 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 23, 2023-09-30T15:03:45.747 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2023-09-30T15:03:45.747 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2023-09-30T15:03:45.747 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2023-09-30T15:03:45.747 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 15, 2023-09-30T15:03:45.747 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 2, 2023-09-30T15:03:45.747 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 13, 2023-09-30T15:03:45.748 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 8, 2023-09-30T15:03:45.748 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 16, 2023-09-30T15:03:45.748 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 11825, 2023-09-30T15:03:45.748 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 235771, 2023-09-30T15:03:45.748 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2023-09-30T15:03:45.748 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-30T15:03:45.748 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-30T15:03:45.748 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 82613, 2023-09-30T15:03:45.748 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 47462, 2023-09-30T15:03:45.748 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 14132, 2023-09-30T15:03:45.748 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 212842 2023-09-30T15:03:45.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.749 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2023-09-30T15:03:45.749 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 151, 2023-09-30T15:03:45.749 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 369, 2023-09-30T15:03:45.749 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 193, 2023-09-30T15:03:45.749 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 446, 2023-09-30T15:03:45.749 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 192, 2023-09-30T15:03:45.749 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 1051, 2023-09-30T15:03:45.749 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2023-09-30T15:03:45.749 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2023-09-30T15:03:45.749 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 56151, 2023-09-30T15:03:45.749 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 55100, 2023-09-30T15:03:45.749 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2023-09-30T15:03:45.749 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2023-09-30T15:03:45.750 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2023-09-30T15:03:45.750 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2023-09-30T15:03:45.750 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2023-09-30T15:03:45.750 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2023-09-30T15:03:45.750 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2023-09-30T15:03:45.750 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 2, 2023-09-30T15:03:45.750 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 10, 2023-09-30T15:03:45.750 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2023-09-30T15:03:45.750 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 44, 2023-09-30T15:03:45.750 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2023-09-30T15:03:45.750 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2023-09-30T15:03:45.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.751 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2023-09-30T15:03:45.751 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2023-09-30T15:03:45.751 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 263636, 2023-09-30T15:03:45.751 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 259663, 2023-09-30T15:03:45.751 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 259663, 2023-09-30T15:03:45.751 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 3973, 2023-09-30T15:03:45.751 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2023-09-30T15:03:45.751 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 3931, 2023-09-30T15:03:45.751 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 365, 2023-09-30T15:03:45.751 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 359, 2023-09-30T15:03:45.751 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 359, 2023-09-30T15:03:45.751 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 6, 2023-09-30T15:03:45.752 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 1, 2023-09-30T15:03:45.752 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2023-09-30T15:03:45.752 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 5, 2023-09-30T15:03:45.752 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 521297013, 2023-09-30T15:03:45.752 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 529257477, 2023-09-30T15:03:45.752 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2023-09-30T15:03:45.752 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2023-09-30T15:03:45.752 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 75455, 2023-09-30T15:03:45.752 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 762.623027827, 2023-09-30T15:03:45.752 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.010106991 2023-09-30T15:03:45.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.752 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2023-09-30T15:03:45.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.753 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2023-09-30T15:03:45.753 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 1079, 2023-09-30T15:03:45.753 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 56259, 2023-09-30T15:03:45.753 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 55180, 2023-09-30T15:03:45.753 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 83, 2023-09-30T15:03:45.753 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 517, 2023-09-30T15:03:45.753 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 434, 2023-09-30T15:03:45.753 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 2315, 2023-09-30T15:03:45.753 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 130760, 2023-09-30T15:03:45.753 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 128445, 2023-09-30T15:03:45.753 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 968, 2023-09-30T15:03:45.753 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 61840, 2023-09-30T15:03:45.754 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 60872, 2023-09-30T15:03:45.754 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 81040, 2023-09-30T15:03:45.754 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 207152 2023-09-30T15:03:45.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.754 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2023-09-30T15:03:45.754 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 534279, 2023-09-30T15:03:45.754 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 10, 2023-09-30T15:03:45.754 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 361853, 2023-09-30T15:03:45.754 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 7860, 2023-09-30T15:03:45.754 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 23316, 2023-09-30T15:03:45.754 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2023-09-30T15:03:45.754 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 55875, 2023-09-30T15:03:45.755 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 195.067916999, 2023-09-30T15:03:45.755 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003491148 2023-09-30T15:03:45.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.755 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2023-09-30T15:03:45.755 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 13288, 2023-09-30T15:03:45.755 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 119.243419651, 2023-09-30T15:03:45.755 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.008973767 2023-09-30T15:03:45.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.755 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2023-09-30T15:03:45.755 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.755 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.755 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.756 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2023-09-30T15:03:45.756 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.756 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.756 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.756 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2023-09-30T15:03:45.756 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 115383, 2023-09-30T15:03:45.756 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 227.948526906, 2023-09-30T15:03:45.756 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001975581 2023-09-30T15:03:45.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.756 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2023-09-30T15:03:45.756 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.757 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.757 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.757 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2023-09-30T15:03:45.757 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2023-09-30T15:03:45.757 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000197652, 2023-09-30T15:03:45.757 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000197652 2023-09-30T15:03:45.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.757 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2023-09-30T15:03:45.757 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.757 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.757 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.758 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2023-09-30T15:03:45.758 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.758 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.758 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.758 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2023-09-30T15:03:45.758 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.758 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.758 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.758 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2023-09-30T15:03:45.759 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.759 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.759 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.759 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2023-09-30T15:03:45.759 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 7, 2023-09-30T15:03:45.759 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.013545433, 2023-09-30T15:03:45.759 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001935061 2023-09-30T15:03:45.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.759 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2023-09-30T15:03:45.759 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.759 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.760 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.760 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2023-09-30T15:03:45.760 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.760 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.760 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.760 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2023-09-30T15:03:45.760 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.760 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.760 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.760 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2023-09-30T15:03:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 89758, 2023-09-30T15:03:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 890.938209667, 2023-09-30T15:03:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.009926003 2023-09-30T15:03:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2023-09-30T15:03:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 10659, 2023-09-30T15:03:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 46.849800125, 2023-09-30T15:03:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.004395327 2023-09-30T15:03:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2023-09-30T15:03:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.762 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.762 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2023-09-30T15:03:45.762 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.762 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.762 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.762 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2023-09-30T15:03:45.762 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 30, 2023-09-30T15:03:45.762 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.277417210, 2023-09-30T15:03:45.762 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.009247240 2023-09-30T15:03:45.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2023-09-30T15:03:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2023-09-30T15:03:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2023-09-30T15:03:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 13, 2023-09-30T15:03:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.312539533, 2023-09-30T15:03:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.024041502 2023-09-30T15:03:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2023-09-30T15:03:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2023-09-30T15:03:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1572, 2023-09-30T15:03:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 4.192110833, 2023-09-30T15:03:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002666737 2023-09-30T15:03:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2023-09-30T15:03:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2023-09-30T15:03:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 6, 2023-09-30T15:03:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.018026203, 2023-09-30T15:03:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003004367 2023-09-30T15:03:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2023-09-30T15:03:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.766 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.766 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2023-09-30T15:03:45.766 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 56121, 2023-09-30T15:03:45.766 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 207.160117905, 2023-09-30T15:03:45.766 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003691311 2023-09-30T15:03:45.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.766 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2023-09-30T15:03:45.766 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2023-09-30T15:03:45.766 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2023-09-30T15:03:45.766 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.766 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.767 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.767 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:45.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.767 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2023-09-30T15:03:45.767 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2023-09-30T15:03:45.767 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 2, 2023-09-30T15:03:45.767 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 1, 2023-09-30T15:03:45.767 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2023-09-30T15:03:45.767 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2023-09-30T15:03:45.767 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 31505, 2023-09-30T15:03:45.767 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 31505, 2023-09-30T15:03:45.767 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 1505, 2023-09-30T15:03:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2023-09-30T15:03:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2023-09-30T15:03:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2023-09-30T15:03:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2023-09-30T15:03:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2023-09-30T15:03:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2023-09-30T15:03:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2023-09-30T15:03:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2023-09-30T15:03:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-30T15:03:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2023-09-30T15:03:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-30T15:03:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2023-09-30T15:03:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2023-09-30T15:03:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2023-09-30T15:03:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-30T15:03:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2023-09-30T15:03:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2023-09-30T15:03:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2023-09-30T15:03:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2023-09-30T15:03:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2023-09-30T15:03:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-30T15:03:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2023-09-30T15:03:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-30T15:03:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2023-09-30T15:03:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2023-09-30T15:03:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2023-09-30T15:03:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2023-09-30T15:03:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2023-09-30T15:03:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-30T15:03:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2023-09-30T15:03:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2023-09-30T15:03:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2023-09-30T15:03:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2023-09-30T15:03:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2023-09-30T15:03:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2023-09-30T15:03:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2023-09-30T15:03:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2023-09-30T15:03:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2023-09-30T15:03:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 1337383, 2023-09-30T15:03:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 1550, 2023-09-30T15:03:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 704, 2023-09-30T15:03:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 8, 2023-09-30T15:03:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2023-09-30T15:03:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2023-09-30T15:03:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2023-09-30T15:03:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2023-09-30T15:03:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2023-09-30T15:03:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2023-09-30T15:03:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 7528, 2023-09-30T15:03:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 158, 2023-09-30T15:03:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2023-09-30T15:03:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2023-09-30T15:03:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2023-09-30T15:03:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2023-09-30T15:03:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 5176270, 2023-09-30T15:03:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 193679, 2023-09-30T15:03:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2023-09-30T15:03:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2023-09-30T15:03:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2023-09-30T15:03:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2023-09-30T15:03:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2023-09-30T15:03:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2023-09-30T15:03:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2023-09-30T15:03:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 172579, 2023-09-30T15:03:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 574018120, 2023-09-30T15:03:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 37, 2023-09-30T15:03:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 172579, 2023-09-30T15:03:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2023-09-30T15:03:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 172579, 2023-09-30T15:03:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 6858.526521319, 2023-09-30T15:03:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.039741373 2023-09-30T15:03:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2023-09-30T15:03:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2023-09-30T15:03:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 172579, 2023-09-30T15:03:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 248135 2023-09-30T15:03:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 172579, 2023-09-30T15:03:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 1493, 2023-09-30T15:03:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 171086, 2023-09-30T15:03:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2023-09-30T15:03:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2023-09-30T15:03:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 4222, 2023-09-30T15:03:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 32037, 2023-09-30T15:03:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 1468, 2023-09-30T15:03:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 76921, 2023-09-30T15:03:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 1199, 2023-09-30T15:03:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2023-09-30T15:03:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2023-09-30T15:03:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2023-09-30T15:03:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 17, 2023-09-30T15:03:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 55970, 2023-09-30T15:03:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2023-09-30T15:03:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2023-09-30T15:03:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2023-09-30T15:03:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 25, 2023-09-30T15:03:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 64074, 2023-09-30T15:03:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2023-09-30T15:03:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2023-09-30T15:03:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2023-09-30T15:03:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2023-09-30T15:03:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2023-09-30T15:03:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2023-09-30T15:03:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2023-09-30T15:03:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2023-09-30T15:03:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2023-09-30T15:03:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2023-09-30T15:03:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2023-09-30T15:03:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2023-09-30T15:03:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2023-09-30T15:03:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2023-09-30T15:03:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2023-09-30T15:03:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2023-09-30T15:03:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2023-09-30T15:03:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2023-09-30T15:03:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2023-09-30T15:03:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2023-09-30T15:03:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2023-09-30T15:03:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2023-09-30T15:03:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2023-09-30T15:03:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2023-09-30T15:03:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2023-09-30T15:03:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2023-09-30T15:03:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 85, 2023-09-30T15:03:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2023-09-30T15:03:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 3, 2023-09-30T15:03:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 12, 2023-09-30T15:03:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 12, 2023-09-30T15:03:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2023-09-30T15:03:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2023-09-30T15:03:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 8823, 2023-09-30T15:03:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2023-09-30T15:03:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 3379 2023-09-30T15:03:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2023-09-30T15:03:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2023-09-30T15:03:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2023-09-30T15:03:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2023-09-30T15:03:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2023-09-30T15:03:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2023-09-30T15:03:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2023-09-30T15:03:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2023-09-30T15:03:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2023-09-30T15:03:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 10, 2023-09-30T15:03:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2023-09-30T15:03:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 10, 2023-09-30T15:03:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 55100, 2023-09-30T15:03:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2023-09-30T15:03:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-30T15:03:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2023-09-30T15:03:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2023-09-30T15:03:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2023-09-30T15:03:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 960675, 2023-09-30T15:03:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 188167777, 2023-09-30T15:03:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 960675, 2023-09-30T15:03:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2023-09-30T15:03:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2023-09-30T15:03:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 960675, 2023-09-30T15:03:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 188167777, 2023-09-30T15:03:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2023-09-30T15:03:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2023-09-30T15:03:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2023-09-30T15:03:45.782 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2023-09-30T15:03:45.782 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2023-09-30T15:03:45.782 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2023-09-30T15:03:45.782 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2023-09-30T15:03:45.782 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:45.782 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2023-09-30T15:03:45.782 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 172579, 2023-09-30T15:03:45.782 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 579583220, 2023-09-30T15:03:45.782 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 172579, 2023-09-30T15:03:45.782 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 579583220, 2023-09-30T15:03:45.782 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2023-09-30T15:03:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2023-09-30T15:03:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2023-09-30T15:03:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2023-09-30T15:03:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2023-09-30T15:03:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2023-09-30T15:03:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2023-09-30T15:03:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2023-09-30T15:03:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 172579, 2023-09-30T15:03:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 172579, 2023-09-30T15:03:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 172579, 2023-09-30T15:03:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 172579, 2023-09-30T15:03:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2023-09-30T15:03:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2023-09-30T15:03:45.785 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2023-09-30T15:03:45.785 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2023-09-30T15:03:45.785 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2023-09-30T15:03:45.785 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 55100, 2023-09-30T15:03:45.785 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 5565100, 2023-09-30T15:03:45.785 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:45.785 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 55100, 2023-09-30T15:03:45.785 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2023-09-30T15:03:45.785 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2023-09-30T15:03:45.785 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1466, 2023-09-30T15:03:45.785 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 5565100, 2023-09-30T15:03:45.785 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2023-09-30T15:03:45.785 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.786 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.786 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.786 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:45.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:45.786 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x55c7bac686e0": { 2023-09-30T15:03:45.786 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 279445, 2023-09-30T15:03:45.786 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2023-09-30T15:03:45.786 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2023-09-30T15:03:45.786 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 263636, 2023-09-30T15:03:45.786 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 525063173, 2023-09-30T15:03:45.786 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:45.787 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 263636, 2023-09-30T15:03:45.787 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2023-09-30T15:03:45.787 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2023-09-30T15:03:45.787 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 75455, 2023-09-30T15:03:45.787 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 524783728, 2023-09-30T15:03:45.787 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2023-09-30T15:03:45.787 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:45.787 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:45.787 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:45.787 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:45.787 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:45.787 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-09-30T15:03:46.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:45 smithi156 ceph-mon[103579]: pgmap v1051: 97 pgs: 97 active+clean; 664 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 5.3 MiB/s wr, 203 op/s 2023-09-30T15:03:46.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:45 smithi148 ceph-mon[102355]: pgmap v1051: 97 pgs: 97 active+clean; 664 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 5.3 MiB/s wr, 203 op/s 2023-09-30T15:03:46.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:45 smithi032 ceph-mon[98592]: pgmap v1051: 97 pgs: 97 active+clean; 664 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 5.3 MiB/s wr, 203 op/s 2023-09-30T15:03:46.252 INFO:tasks.check_counter:Daemon mds.b mds.exported=7 2023-09-30T15:03:46.252 INFO:tasks.check_counter:Daemon mds.b mds.imported=5 2023-09-30T15:03:46.252 DEBUG:tasks.check_counter:Getting stats from e 2023-09-30T15:03:46.252 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-09-30T15:03:47.049 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.b/config 2023-09-30T15:03:48.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:47 smithi156 ceph-mon[103579]: pgmap v1052: 97 pgs: 97 active+clean; 663 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 5.0 MiB/s wr, 167 op/s 2023-09-30T15:03:48.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:47 smithi148 ceph-mon[102355]: pgmap v1052: 97 pgs: 97 active+clean; 663 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 5.0 MiB/s wr, 167 op/s 2023-09-30T15:03:48.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:47 smithi032 ceph-mon[98592]: pgmap v1052: 97 pgs: 97 active+clean; 663 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 5.0 MiB/s wr, 167 op/s 2023-09-30T15:03:49.807 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:49 smithi148 ceph-mon[102355]: pgmap v1053: 97 pgs: 97 active+clean; 664 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.7 MiB/s wr, 161 op/s 2023-09-30T15:03:50.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:49 smithi156 ceph-mon[103579]: pgmap v1053: 97 pgs: 97 active+clean; 664 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.7 MiB/s wr, 161 op/s 2023-09-30T15:03:50.143 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:49 smithi032 ceph-mon[98592]: pgmap v1053: 97 pgs: 97 active+clean; 664 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.7 MiB/s wr, 161 op/s 2023-09-30T15:03:50.160 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-09-30T15:03:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2023-09-30T15:03:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 1660, 2023-09-30T15:03:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 1662, 2023-09-30T15:03:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 289833623, 2023-09-30T15:03:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 516324, 2023-09-30T15:03:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 8, 2023-09-30T15:03:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2023-09-30T15:03:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.273564371, 2023-09-30T15:03:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.033971423, 2023-09-30T15:03:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.175503356, 2023-09-30T15:03:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.052692259, 2023-09-30T15:03:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1660, 2023-09-30T15:03:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.121595330, 2023-09-30T15:03:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000073250 2023-09-30T15:03:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-30T15:03:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2023-09-30T15:03:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2023-09-30T15:03:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 1604, 2023-09-30T15:03:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 1928, 2023-09-30T15:03:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 281429829, 2023-09-30T15:03:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 1144414, 2023-09-30T15:03:50.163 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 7, 2023-09-30T15:03:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 6, 2023-09-30T15:03:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.290441015, 2023-09-30T15:03:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.035097692, 2023-09-30T15:03:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.202025215, 2023-09-30T15:03:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.040656521, 2023-09-30T15:03:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1927, 2023-09-30T15:03:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.162967740, 2023-09-30T15:03:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000084570 2023-09-30T15:03:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:50.164 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-09-30T15:03:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 647120 2023-09-30T15:03:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2023-09-30T15:03:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 3684, 2023-09-30T15:03:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 3644, 2023-09-30T15:03:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 303630522, 2023-09-30T15:03:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 1258209, 2023-09-30T15:03:50.165 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 5, 2023-09-30T15:03:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2023-09-30T15:03:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.451425400, 2023-09-30T15:03:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.077049511, 2023-09-30T15:03:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.253521284, 2023-09-30T15:03:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.099124329, 2023-09-30T15:03:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3643, 2023-09-30T15:03:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.302706667, 2023-09-30T15:03:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000083092 2023-09-30T15:03:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:50.166 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 698912, 2023-09-30T15:03:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 260832 2023-09-30T15:03:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2023-09-30T15:03:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2023-09-30T15:03:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2023-09-30T15:03:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2023-09-30T15:03:50.167 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2023-09-30T15:03:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2023-09-30T15:03:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 10796, 2023-09-30T15:03:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.803665454, 2023-09-30T15:03:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000074441 2023-09-30T15:03:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2023-09-30T15:03:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2023-09-30T15:03:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2023-09-30T15:03:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.168 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2023-09-30T15:03:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 0, 2023-09-30T15:03:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 0, 2023-09-30T15:03:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2023-09-30T15:03:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2023-09-30T15:03:50.169 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 0, 2023-09-30T15:03:50.170 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2023-09-30T15:03:50.170 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2023-09-30T15:03:50.170 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 0, 2023-09-30T15:03:50.170 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2023-09-30T15:03:50.170 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2023-09-30T15:03:50.170 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 54235, 2023-09-30T15:03:50.170 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 18640, 2023-09-30T15:03:50.170 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 35580, 2023-09-30T15:03:50.170 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 15, 2023-09-30T15:03:50.170 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 27606, 2023-09-30T15:03:50.170 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 34288, 2023-09-30T15:03:50.171 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 0, 2023-09-30T15:03:50.171 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 0, 2023-09-30T15:03:50.171 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 3, 2023-09-30T15:03:50.171 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 0, 2023-09-30T15:03:50.171 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 0, 2023-09-30T15:03:50.171 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2023-09-30T15:03:50.171 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2023-09-30T15:03:50.171 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2023-09-30T15:03:50.171 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2023-09-30T15:03:50.171 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2023-09-30T15:03:50.171 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2023-09-30T15:03:50.171 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2023-09-30T15:03:50.172 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2023-09-30T15:03:50.172 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2023-09-30T15:03:50.172 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2023-09-30T15:03:50.172 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2023-09-30T15:03:50.172 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2023-09-30T15:03:50.172 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 0, 2023-09-30T15:03:50.172 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2023-09-30T15:03:50.172 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 0, 2023-09-30T15:03:50.172 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 0, 2023-09-30T15:03:50.172 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2023-09-30T15:03:50.172 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 0, 2023-09-30T15:03:50.173 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2023-09-30T15:03:50.173 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2023-09-30T15:03:50.173 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2023-09-30T15:03:50.173 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 0, 2023-09-30T15:03:50.173 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2023-09-30T15:03:50.173 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 0, 2023-09-30T15:03:50.173 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 0, 2023-09-30T15:03:50.173 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2023-09-30T15:03:50.173 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 0, 2023-09-30T15:03:50.173 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 0, 2023-09-30T15:03:50.173 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2023-09-30T15:03:50.173 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-30T15:03:50.173 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-30T15:03:50.174 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 0, 2023-09-30T15:03:50.174 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 0, 2023-09-30T15:03:50.174 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 0, 2023-09-30T15:03:50.174 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 0 2023-09-30T15:03:50.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.174 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2023-09-30T15:03:50.174 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2023-09-30T15:03:50.174 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2023-09-30T15:03:50.174 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2023-09-30T15:03:50.174 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2023-09-30T15:03:50.175 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2023-09-30T15:03:50.175 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2023-09-30T15:03:50.175 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2023-09-30T15:03:50.175 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2023-09-30T15:03:50.175 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 0, 2023-09-30T15:03:50.175 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 0, 2023-09-30T15:03:50.175 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2023-09-30T15:03:50.175 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2023-09-30T15:03:50.175 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2023-09-30T15:03:50.175 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2023-09-30T15:03:50.175 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2023-09-30T15:03:50.175 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2023-09-30T15:03:50.176 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2023-09-30T15:03:50.176 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2023-09-30T15:03:50.176 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2023-09-30T15:03:50.176 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2023-09-30T15:03:50.176 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2023-09-30T15:03:50.176 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2023-09-30T15:03:50.176 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2023-09-30T15:03:50.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.176 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2023-09-30T15:03:50.176 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2023-09-30T15:03:50.176 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 0, 2023-09-30T15:03:50.176 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2023-09-30T15:03:50.177 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2023-09-30T15:03:50.177 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 411759, 2023-09-30T15:03:50.177 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2023-09-30T15:03:50.177 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2023-09-30T15:03:50.177 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 0, 2023-09-30T15:03:50.177 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2023-09-30T15:03:50.177 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2023-09-30T15:03:50.177 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 137, 2023-09-30T15:03:50.177 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 35, 2023-09-30T15:03:50.177 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2023-09-30T15:03:50.177 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2023-09-30T15:03:50.177 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 628748540, 2023-09-30T15:03:50.178 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 876608960, 2023-09-30T15:03:50.178 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 876608361, 2023-09-30T15:03:50.178 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2023-09-30T15:03:50.178 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.178 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.178 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.178 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 411759 2023-09-30T15:03:50.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.178 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2023-09-30T15:03:50.178 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 975, 2023-09-30T15:03:50.179 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 89058, 2023-09-30T15:03:50.179 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 88083, 2023-09-30T15:03:50.179 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 84, 2023-09-30T15:03:50.179 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 315, 2023-09-30T15:03:50.179 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 231, 2023-09-30T15:03:50.179 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 54235, 2023-09-30T15:03:50.179 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 90055, 2023-09-30T15:03:50.179 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 35820, 2023-09-30T15:03:50.179 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 0, 2023-09-30T15:03:50.179 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 0, 2023-09-30T15:03:50.179 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 0, 2023-09-30T15:03:50.180 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 102572, 2023-09-30T15:03:50.180 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 207152 2023-09-30T15:03:50.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.180 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2023-09-30T15:03:50.180 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 0, 2023-09-30T15:03:50.180 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2023-09-30T15:03:50.180 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 0, 2023-09-30T15:03:50.180 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 0, 2023-09-30T15:03:50.180 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 0, 2023-09-30T15:03:50.180 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2023-09-30T15:03:50.180 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.180 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.181 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.181 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2023-09-30T15:03:50.181 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.181 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.181 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.181 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2023-09-30T15:03:50.181 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.181 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.181 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.182 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2023-09-30T15:03:50.182 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.182 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.182 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.182 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2023-09-30T15:03:50.182 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.182 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.182 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.183 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2023-09-30T15:03:50.183 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.183 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.183 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.183 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2023-09-30T15:03:50.183 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.183 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.183 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.183 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.183 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2023-09-30T15:03:50.183 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.183 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.184 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.184 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2023-09-30T15:03:50.184 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.184 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.184 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.184 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2023-09-30T15:03:50.184 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.184 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.184 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.185 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2023-09-30T15:03:50.185 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.185 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.185 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.185 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2023-09-30T15:03:50.185 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.185 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.185 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.185 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2023-09-30T15:03:50.185 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.186 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.186 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.186 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2023-09-30T15:03:50.186 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.186 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.186 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.186 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2023-09-30T15:03:50.186 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.186 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.187 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.187 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2023-09-30T15:03:50.187 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.187 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.187 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.187 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2023-09-30T15:03:50.188 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.188 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.188 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.188 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2023-09-30T15:03:50.188 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.188 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.188 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.188 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2023-09-30T15:03:50.189 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.189 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.189 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.189 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2023-09-30T15:03:50.189 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.189 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.189 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.190 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2023-09-30T15:03:50.190 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.190 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.190 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.190 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2023-09-30T15:03:50.190 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.191 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.191 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.191 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2023-09-30T15:03:50.191 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.191 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.191 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.191 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.192 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2023-09-30T15:03:50.192 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.192 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.192 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.192 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2023-09-30T15:03:50.192 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.193 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.193 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.193 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2023-09-30T15:03:50.193 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.193 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.193 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.193 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2023-09-30T15:03:50.194 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.194 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.194 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.194 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.194 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2023-09-30T15:03:50.194 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.194 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.195 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.195 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2023-09-30T15:03:50.195 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.195 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.195 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.196 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2023-09-30T15:03:50.196 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2023-09-30T15:03:50.196 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2023-09-30T15:03:50.196 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.196 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.196 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.196 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:50.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.196 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2023-09-30T15:03:50.197 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2023-09-30T15:03:50.197 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 1, 2023-09-30T15:03:50.197 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 0, 2023-09-30T15:03:50.197 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2023-09-30T15:03:50.197 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2023-09-30T15:03:50.197 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 0, 2023-09-30T15:03:50.197 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 0, 2023-09-30T15:03:50.197 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 2846, 2023-09-30T15:03:50.197 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2023-09-30T15:03:50.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.198 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2023-09-30T15:03:50.198 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2023-09-30T15:03:50.198 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2023-09-30T15:03:50.198 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2023-09-30T15:03:50.198 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2023-09-30T15:03:50.198 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2023-09-30T15:03:50.198 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2023-09-30T15:03:50.198 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-30T15:03:50.198 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2023-09-30T15:03:50.199 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-30T15:03:50.199 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2023-09-30T15:03:50.199 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2023-09-30T15:03:50.199 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2023-09-30T15:03:50.199 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-30T15:03:50.199 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2023-09-30T15:03:50.199 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2023-09-30T15:03:50.199 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2023-09-30T15:03:50.199 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2023-09-30T15:03:50.199 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2023-09-30T15:03:50.199 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-30T15:03:50.199 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2023-09-30T15:03:50.200 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-30T15:03:50.200 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2023-09-30T15:03:50.200 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2023-09-30T15:03:50.200 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2023-09-30T15:03:50.200 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2023-09-30T15:03:50.200 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2023-09-30T15:03:50.200 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-30T15:03:50.200 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2023-09-30T15:03:50.200 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2023-09-30T15:03:50.200 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2023-09-30T15:03:50.200 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2023-09-30T15:03:50.201 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2023-09-30T15:03:50.201 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2023-09-30T15:03:50.201 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2023-09-30T15:03:50.201 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2023-09-30T15:03:50.201 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2023-09-30T15:03:50.201 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 272663, 2023-09-30T15:03:50.201 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 1034, 2023-09-30T15:03:50.201 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2023-09-30T15:03:50.201 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2023-09-30T15:03:50.201 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2023-09-30T15:03:50.201 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2023-09-30T15:03:50.202 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2023-09-30T15:03:50.202 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2023-09-30T15:03:50.202 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2023-09-30T15:03:50.202 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2023-09-30T15:03:50.202 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 7528, 2023-09-30T15:03:50.202 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 158, 2023-09-30T15:03:50.202 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2023-09-30T15:03:50.202 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2023-09-30T15:03:50.202 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2023-09-30T15:03:50.202 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2023-09-30T15:03:50.202 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 40185987, 2023-09-30T15:03:50.203 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 348980, 2023-09-30T15:03:50.203 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2023-09-30T15:03:50.203 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2023-09-30T15:03:50.203 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2023-09-30T15:03:50.203 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2023-09-30T15:03:50.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.203 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2023-09-30T15:03:50.203 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2023-09-30T15:03:50.203 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2023-09-30T15:03:50.203 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 6466, 2023-09-30T15:03:50.203 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 21, 2023-09-30T15:03:50.204 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 3, 2023-09-30T15:03:50.204 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 6466, 2023-09-30T15:03:50.204 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2023-09-30T15:03:50.204 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 6466, 2023-09-30T15:03:50.204 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 15.309649674, 2023-09-30T15:03:50.204 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002367715 2023-09-30T15:03:50.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.204 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2023-09-30T15:03:50.204 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2023-09-30T15:03:50.204 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 6466, 2023-09-30T15:03:50.204 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 6467 2023-09-30T15:03:50.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.205 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 6466, 2023-09-30T15:03:50.205 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 6466, 2023-09-30T15:03:50.205 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 0, 2023-09-30T15:03:50.205 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2023-09-30T15:03:50.205 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2023-09-30T15:03:50.205 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 3344, 2023-09-30T15:03:50.205 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 0, 2023-09-30T15:03:50.205 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 3120, 2023-09-30T15:03:50.205 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 0, 2023-09-30T15:03:50.205 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 0, 2023-09-30T15:03:50.205 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2023-09-30T15:03:50.206 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2023-09-30T15:03:50.206 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2023-09-30T15:03:50.206 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2023-09-30T15:03:50.206 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 0, 2023-09-30T15:03:50.206 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2023-09-30T15:03:50.206 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2023-09-30T15:03:50.206 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2023-09-30T15:03:50.206 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 1, 2023-09-30T15:03:50.206 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 0, 2023-09-30T15:03:50.206 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2023-09-30T15:03:50.206 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2023-09-30T15:03:50.207 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2023-09-30T15:03:50.207 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2023-09-30T15:03:50.207 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2023-09-30T15:03:50.207 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2023-09-30T15:03:50.207 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2023-09-30T15:03:50.207 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2023-09-30T15:03:50.207 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2023-09-30T15:03:50.207 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2023-09-30T15:03:50.207 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2023-09-30T15:03:50.207 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2023-09-30T15:03:50.207 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2023-09-30T15:03:50.207 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2023-09-30T15:03:50.208 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2023-09-30T15:03:50.208 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2023-09-30T15:03:50.208 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2023-09-30T15:03:50.208 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2023-09-30T15:03:50.208 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2023-09-30T15:03:50.208 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2023-09-30T15:03:50.208 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2023-09-30T15:03:50.208 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2023-09-30T15:03:50.208 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2023-09-30T15:03:50.208 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2023-09-30T15:03:50.208 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2023-09-30T15:03:50.208 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2023-09-30T15:03:50.208 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 85, 2023-09-30T15:03:50.209 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2023-09-30T15:03:50.209 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 2, 2023-09-30T15:03:50.209 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 11, 2023-09-30T15:03:50.209 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 11, 2023-09-30T15:03:50.209 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2023-09-30T15:03:50.209 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2023-09-30T15:03:50.209 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 0, 2023-09-30T15:03:50.209 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 2, 2023-09-30T15:03:50.209 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 0 2023-09-30T15:03:50.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.209 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2023-09-30T15:03:50.209 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 0, 2023-09-30T15:03:50.210 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2023-09-30T15:03:50.210 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2023-09-30T15:03:50.210 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2023-09-30T15:03:50.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.210 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2023-09-30T15:03:50.210 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2023-09-30T15:03:50.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.210 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2023-09-30T15:03:50.210 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2023-09-30T15:03:50.210 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 0, 2023-09-30T15:03:50.210 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2023-09-30T15:03:50.210 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 0, 2023-09-30T15:03:50.211 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 0, 2023-09-30T15:03:50.211 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2023-09-30T15:03:50.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.211 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-30T15:03:50.211 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2023-09-30T15:03:50.211 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2023-09-30T15:03:50.211 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2023-09-30T15:03:50.211 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 6944, 2023-09-30T15:03:50.211 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 874241723, 2023-09-30T15:03:50.211 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:50.211 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 6944, 2023-09-30T15:03:50.211 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2023-09-30T15:03:50.211 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2023-09-30T15:03:50.212 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 6944, 2023-09-30T15:03:50.212 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 874241723, 2023-09-30T15:03:50.212 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2023-09-30T15:03:50.212 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.212 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.212 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.212 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:50.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.212 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2023-09-30T15:03:50.212 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2023-09-30T15:03:50.212 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2023-09-30T15:03:50.212 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2023-09-30T15:03:50.213 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2023-09-30T15:03:50.213 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2023-09-30T15:03:50.213 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:50.213 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2023-09-30T15:03:50.213 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 6466, 2023-09-30T15:03:50.213 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 872417402, 2023-09-30T15:03:50.213 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 6466, 2023-09-30T15:03:50.213 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 872417402, 2023-09-30T15:03:50.213 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2023-09-30T15:03:50.213 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.214 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.214 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.214 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:50.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.214 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2023-09-30T15:03:50.214 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2023-09-30T15:03:50.214 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2023-09-30T15:03:50.214 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2023-09-30T15:03:50.214 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2023-09-30T15:03:50.214 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2023-09-30T15:03:50.214 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:50.215 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2023-09-30T15:03:50.215 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 6466, 2023-09-30T15:03:50.215 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 6466, 2023-09-30T15:03:50.215 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 6466, 2023-09-30T15:03:50.215 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 6466, 2023-09-30T15:03:50.215 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2023-09-30T15:03:50.215 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.215 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.215 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.215 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:50.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.215 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2023-09-30T15:03:50.216 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2023-09-30T15:03:50.216 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2023-09-30T15:03:50.216 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2023-09-30T15:03:50.216 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2023-09-30T15:03:50.216 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2023-09-30T15:03:50.216 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:50.216 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2023-09-30T15:03:50.216 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2023-09-30T15:03:50.216 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2023-09-30T15:03:50.216 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2023-09-30T15:03:50.216 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2023-09-30T15:03:50.217 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2023-09-30T15:03:50.217 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.217 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.217 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.217 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:50.217 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:50.217 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x559e191ca960": { 2023-09-30T15:03:50.217 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2023-09-30T15:03:50.217 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2023-09-30T15:03:50.218 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2023-09-30T15:03:50.218 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2023-09-30T15:03:50.218 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2023-09-30T15:03:50.218 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:50.218 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2023-09-30T15:03:50.218 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2023-09-30T15:03:50.218 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2023-09-30T15:03:50.218 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2023-09-30T15:03:50.218 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2023-09-30T15:03:50.218 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2023-09-30T15:03:50.218 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:50.218 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:50.219 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:50.219 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:50.219 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:50.219 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-09-30T15:03:50.608 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-09-30T15:03:50.608 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-09-30T15:03:50.608 DEBUG:tasks.check_counter:Getting stats from h 2023-09-30T15:03:50.608 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-09-30T15:03:51.220 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.b/config 2023-09-30T15:03:51.858 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:51 smithi148 ceph-mon[102355]: pgmap v1054: 97 pgs: 97 active+clean; 650 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 6.6 MiB/s wr, 203 op/s 2023-09-30T15:03:52.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:51 smithi156 ceph-mon[103579]: pgmap v1054: 97 pgs: 97 active+clean; 650 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 6.6 MiB/s wr, 203 op/s 2023-09-30T15:03:52.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:51 smithi032 ceph-mon[98592]: pgmap v1054: 97 pgs: 97 active+clean; 650 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 6.6 MiB/s wr, 203 op/s 2023-09-30T15:03:52.551 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-09-30T15:03:52.551 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2023-09-30T15:03:52.552 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 310024, 2023-09-30T15:03:52.552 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 312914, 2023-09-30T15:03:52.552 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 79776083, 2023-09-30T15:03:52.552 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 358378715, 2023-09-30T15:03:52.552 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 11, 2023-09-30T15:03:52.552 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 5, 2023-09-30T15:03:52.552 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 19.759474736, 2023-09-30T15:03:52.552 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 8.415899806, 2023-09-30T15:03:52.552 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 10.086638558, 2023-09-30T15:03:52.553 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 2.756089397, 2023-09-30T15:03:52.553 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:52.553 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 312914, 2023-09-30T15:03:52.553 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 24.610526575, 2023-09-30T15:03:52.553 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000078649 2023-09-30T15:03:52.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.553 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:52.553 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 473814, 2023-09-30T15:03:52.553 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.074530579, 2023-09-30T15:03:52.553 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000157 2023-09-30T15:03:52.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.554 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-30T15:03:52.554 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2023-09-30T15:03:52.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.554 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2023-09-30T15:03:52.554 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 806534, 2023-09-30T15:03:52.554 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 1348108, 2023-09-30T15:03:52.554 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 271743113, 2023-09-30T15:03:52.554 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 1304673690, 2023-09-30T15:03:52.554 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 12, 2023-09-30T15:03:52.555 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 5, 2023-09-30T15:03:52.555 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 54.606619545, 2023-09-30T15:03:52.555 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 24.990561386, 2023-09-30T15:03:52.555 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 39.104336037, 2023-09-30T15:03:52.555 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 3.637213785, 2023-09-30T15:03:52.555 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:52.555 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1348108, 2023-09-30T15:03:52.555 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 199.109390427, 2023-09-30T15:03:52.555 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000147695 2023-09-30T15:03:52.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.555 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:52.555 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1672161, 2023-09-30T15:03:52.556 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.276149714, 2023-09-30T15:03:52.556 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000165 2023-09-30T15:03:52.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.556 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 249906368, 2023-09-30T15:03:52.556 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 949613040 2023-09-30T15:03:52.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.556 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2023-09-30T15:03:52.556 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 478780, 2023-09-30T15:03:52.556 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 407412, 2023-09-30T15:03:52.556 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 118649598, 2023-09-30T15:03:52.556 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 396432810, 2023-09-30T15:03:52.556 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 11, 2023-09-30T15:03:52.556 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 7, 2023-09-30T15:03:52.557 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 24.368606749, 2023-09-30T15:03:52.557 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 8.903719974, 2023-09-30T15:03:52.557 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 14.793300751, 2023-09-30T15:03:52.557 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 3.249798043, 2023-09-30T15:03:52.557 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:52.557 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 407408, 2023-09-30T15:03:52.557 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 34.973801402, 2023-09-30T15:03:52.557 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000085844 2023-09-30T15:03:52.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.557 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:52.557 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 718805, 2023-09-30T15:03:52.557 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.105453981, 2023-09-30T15:03:52.558 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000146 2023-09-30T15:03:52.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.558 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 160, 2023-09-30T15:03:52.558 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 648112 2023-09-30T15:03:52.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.558 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2023-09-30T15:03:52.558 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2023-09-30T15:03:52.558 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2023-09-30T15:03:52.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.558 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2023-09-30T15:03:52.558 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2023-09-30T15:03:52.558 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2023-09-30T15:03:52.559 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 258501, 2023-09-30T15:03:52.559 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 332.660127416, 2023-09-30T15:03:52.559 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001286881 2023-09-30T15:03:52.559 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:52.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.559 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2023-09-30T15:03:52.559 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2023-09-30T15:03:52.559 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2023-09-30T15:03:52.559 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 135393, 2023-09-30T15:03:52.559 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 37.722908400, 2023-09-30T15:03:52.559 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000278617 2023-09-30T15:03:52.559 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:52.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.560 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2023-09-30T15:03:52.560 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 541375, 2023-09-30T15:03:52.560 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 487067, 2023-09-30T15:03:52.560 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2023-09-30T15:03:52.560 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 487067, 2023-09-30T15:03:52.560 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 2710.625468249, 2023-09-30T15:03:52.560 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.005565200 2023-09-30T15:03:52.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.560 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2023-09-30T15:03:52.560 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 54308, 2023-09-30T15:03:52.560 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2023-09-30T15:03:52.560 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2023-09-30T15:03:52.560 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 342, 2023-09-30T15:03:52.561 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 20, 2023-09-30T15:03:52.561 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 24, 2023-09-30T15:03:52.561 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 3099, 2023-09-30T15:03:52.561 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 804, 2023-09-30T15:03:52.561 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 1785, 2023-09-30T15:03:52.561 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 510, 2023-09-30T15:03:52.561 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 1949, 2023-09-30T15:03:52.561 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 1872, 2023-09-30T15:03:52.561 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 777, 2023-09-30T15:03:52.561 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 777, 2023-09-30T15:03:52.561 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 5, 2023-09-30T15:03:52.562 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 754914, 2023-09-30T15:03:52.562 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 407817, 2023-09-30T15:03:52.562 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 54303, 2023-09-30T15:03:52.562 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 33, 2023-09-30T15:03:52.562 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2023-09-30T15:03:52.562 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2023-09-30T15:03:52.562 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 4287, 2023-09-30T15:03:52.562 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 35, 2023-09-30T15:03:52.562 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2023-09-30T15:03:52.562 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 14, 2023-09-30T15:03:52.562 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 1141, 2023-09-30T15:03:52.562 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 13, 2023-09-30T15:03:52.563 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 126, 2023-09-30T15:03:52.563 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2023-09-30T15:03:52.563 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 1, 2023-09-30T15:03:52.563 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2023-09-30T15:03:52.563 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 4, 2023-09-30T15:03:52.563 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 246, 2023-09-30T15:03:52.563 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2023-09-30T15:03:52.563 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 1, 2023-09-30T15:03:52.563 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2023-09-30T15:03:52.563 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2023-09-30T15:03:52.563 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2023-09-30T15:03:52.563 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 1, 2023-09-30T15:03:52.564 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2023-09-30T15:03:52.564 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 1, 2023-09-30T15:03:52.564 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 0, 2023-09-30T15:03:52.564 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 24, 2023-09-30T15:03:52.564 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 48637, 2023-09-30T15:03:52.564 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 399870, 2023-09-30T15:03:52.564 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2023-09-30T15:03:52.564 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-30T15:03:52.564 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-30T15:03:52.564 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 136598, 2023-09-30T15:03:52.564 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 61709, 2023-09-30T15:03:52.564 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 17717, 2023-09-30T15:03:52.565 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 320273 2023-09-30T15:03:52.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.565 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2023-09-30T15:03:52.565 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 19, 2023-09-30T15:03:52.565 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 21, 2023-09-30T15:03:52.565 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 14, 2023-09-30T15:03:52.565 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 41, 2023-09-30T15:03:52.565 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 405, 2023-09-30T15:03:52.565 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 834, 2023-09-30T15:03:52.565 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 125, 2023-09-30T15:03:52.565 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2023-09-30T15:03:52.565 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 88445, 2023-09-30T15:03:52.566 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 87611, 2023-09-30T15:03:52.566 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2023-09-30T15:03:52.566 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2023-09-30T15:03:52.566 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2023-09-30T15:03:52.566 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2023-09-30T15:03:52.566 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2023-09-30T15:03:52.566 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2023-09-30T15:03:52.566 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2023-09-30T15:03:52.566 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2023-09-30T15:03:52.566 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 15, 2023-09-30T15:03:52.566 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2023-09-30T15:03:52.566 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 44, 2023-09-30T15:03:52.566 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2023-09-30T15:03:52.567 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2023-09-30T15:03:52.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.567 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2023-09-30T15:03:52.567 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2023-09-30T15:03:52.567 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 412003, 2023-09-30T15:03:52.567 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 303989, 2023-09-30T15:03:52.567 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 294961, 2023-09-30T15:03:52.567 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 117042, 2023-09-30T15:03:52.567 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2023-09-30T15:03:52.567 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 9028, 2023-09-30T15:03:52.567 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 474, 2023-09-30T15:03:52.567 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 346, 2023-09-30T15:03:52.568 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 336, 2023-09-30T15:03:52.568 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 138, 2023-09-30T15:03:52.568 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 10, 2023-09-30T15:03:52.568 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2023-09-30T15:03:52.568 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 10, 2023-09-30T15:03:52.568 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 628748540, 2023-09-30T15:03:52.568 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 876945151, 2023-09-30T15:03:52.568 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2023-09-30T15:03:52.568 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2023-09-30T15:03:52.568 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 146792, 2023-09-30T15:03:52.568 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1239.805810654, 2023-09-30T15:03:52.568 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.008446003 2023-09-30T15:03:52.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.569 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2023-09-30T15:03:52.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.569 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2023-09-30T15:03:52.569 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 865, 2023-09-30T15:03:52.569 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 89199, 2023-09-30T15:03:52.569 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 88334, 2023-09-30T15:03:52.569 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 129, 2023-09-30T15:03:52.569 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 421, 2023-09-30T15:03:52.569 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 292, 2023-09-30T15:03:52.569 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 3004, 2023-09-30T15:03:52.569 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 174989, 2023-09-30T15:03:52.570 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 171985, 2023-09-30T15:03:52.570 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 705, 2023-09-30T15:03:52.570 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 93738, 2023-09-30T15:03:52.570 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 93033, 2023-09-30T15:03:52.570 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 92192, 2023-09-30T15:03:52.570 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 198960 2023-09-30T15:03:52.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.570 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2023-09-30T15:03:52.570 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 687206, 2023-09-30T15:03:52.570 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 83431, 2023-09-30T15:03:52.570 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 541375, 2023-09-30T15:03:52.570 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 11009, 2023-09-30T15:03:52.570 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 166846, 2023-09-30T15:03:52.571 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2023-09-30T15:03:52.571 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 88910, 2023-09-30T15:03:52.571 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 366.876090297, 2023-09-30T15:03:52.571 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.004126376 2023-09-30T15:03:52.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.571 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2023-09-30T15:03:52.571 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 261, 2023-09-30T15:03:52.571 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 4.031902077, 2023-09-30T15:03:52.571 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.015447900 2023-09-30T15:03:52.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.571 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2023-09-30T15:03:52.571 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.572 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.572 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.572 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2023-09-30T15:03:52.572 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.572 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.572 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.572 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2023-09-30T15:03:52.572 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 162789, 2023-09-30T15:03:52.572 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 488.580108144, 2023-09-30T15:03:52.572 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003001309 2023-09-30T15:03:52.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.573 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2023-09-30T15:03:52.573 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.573 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.573 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.573 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2023-09-30T15:03:52.573 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2023-09-30T15:03:52.573 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000124958, 2023-09-30T15:03:52.573 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000124958 2023-09-30T15:03:52.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.573 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2023-09-30T15:03:52.574 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.574 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.574 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.574 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2023-09-30T15:03:52.574 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.574 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.574 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.574 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2023-09-30T15:03:52.574 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.574 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.575 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.575 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2023-09-30T15:03:52.575 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.575 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.575 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.575 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2023-09-30T15:03:52.575 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 112, 2023-09-30T15:03:52.575 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.756955296, 2023-09-30T15:03:52.575 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.006758529 2023-09-30T15:03:52.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.575 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2023-09-30T15:03:52.576 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.576 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.576 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.576 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2023-09-30T15:03:52.576 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.576 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.576 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.576 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2023-09-30T15:03:52.576 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 9, 2023-09-30T15:03:52.576 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.014037612, 2023-09-30T15:03:52.576 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001559734 2023-09-30T15:03:52.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.577 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2023-09-30T15:03:52.577 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 125431, 2023-09-30T15:03:52.577 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1388.904822369, 2023-09-30T15:03:52.577 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.011073058 2023-09-30T15:03:52.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.577 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2023-09-30T15:03:52.577 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 18558, 2023-09-30T15:03:52.577 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 78.266222755, 2023-09-30T15:03:52.577 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.004217384 2023-09-30T15:03:52.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.577 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2023-09-30T15:03:52.578 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.578 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.578 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.578 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2023-09-30T15:03:52.578 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.578 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.578 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.578 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2023-09-30T15:03:52.578 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 64, 2023-09-30T15:03:52.578 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.537517085, 2023-09-30T15:03:52.579 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.008398704 2023-09-30T15:03:52.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.579 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2023-09-30T15:03:52.579 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.579 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.579 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.579 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2023-09-30T15:03:52.579 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.579 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.579 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.580 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2023-09-30T15:03:52.580 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 56, 2023-09-30T15:03:52.580 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.681601537, 2023-09-30T15:03:52.580 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.030028598 2023-09-30T15:03:52.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.580 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2023-09-30T15:03:52.580 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.580 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.580 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.580 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2023-09-30T15:03:52.580 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2484, 2023-09-30T15:03:52.581 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 7.739253072, 2023-09-30T15:03:52.581 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003115641 2023-09-30T15:03:52.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.581 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2023-09-30T15:03:52.581 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.581 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.581 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.581 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2023-09-30T15:03:52.581 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.581 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.581 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.582 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2023-09-30T15:03:52.582 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.582 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.582 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.582 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2023-09-30T15:03:52.582 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 88392, 2023-09-30T15:03:52.582 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 373.236833047, 2023-09-30T15:03:52.582 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.004222518 2023-09-30T15:03:52.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.582 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2023-09-30T15:03:52.583 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2023-09-30T15:03:52.583 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2023-09-30T15:03:52.583 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.583 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.583 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.583 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:52.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.583 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2023-09-30T15:03:52.583 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2023-09-30T15:03:52.583 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 1, 2023-09-30T15:03:52.583 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 0, 2023-09-30T15:03:52.583 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2023-09-30T15:03:52.584 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2023-09-30T15:03:52.584 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 33647, 2023-09-30T15:03:52.584 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 33647, 2023-09-30T15:03:52.584 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 1516, 2023-09-30T15:03:52.584 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2023-09-30T15:03:52.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.584 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2023-09-30T15:03:52.584 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 360, 2023-09-30T15:03:52.584 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 360, 2023-09-30T15:03:52.584 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2023-09-30T15:03:52.584 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2023-09-30T15:03:52.584 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2023-09-30T15:03:52.584 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2023-09-30T15:03:52.585 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-30T15:03:52.585 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2023-09-30T15:03:52.585 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-30T15:03:52.585 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2023-09-30T15:03:52.585 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2023-09-30T15:03:52.585 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2023-09-30T15:03:52.585 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-30T15:03:52.585 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2023-09-30T15:03:52.585 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2023-09-30T15:03:52.585 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2023-09-30T15:03:52.585 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2023-09-30T15:03:52.585 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2023-09-30T15:03:52.585 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-30T15:03:52.586 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2023-09-30T15:03:52.586 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-30T15:03:52.586 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2023-09-30T15:03:52.586 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2023-09-30T15:03:52.586 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2023-09-30T15:03:52.586 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2023-09-30T15:03:52.586 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2023-09-30T15:03:52.586 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-30T15:03:52.586 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2023-09-30T15:03:52.586 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2023-09-30T15:03:52.586 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2023-09-30T15:03:52.586 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2023-09-30T15:03:52.587 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2023-09-30T15:03:52.587 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2023-09-30T15:03:52.587 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2023-09-30T15:03:52.587 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2023-09-30T15:03:52.587 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2023-09-30T15:03:52.587 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 2788189, 2023-09-30T15:03:52.587 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 1688, 2023-09-30T15:03:52.587 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 440, 2023-09-30T15:03:52.587 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 5, 2023-09-30T15:03:52.587 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2023-09-30T15:03:52.587 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2023-09-30T15:03:52.587 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2023-09-30T15:03:52.588 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2023-09-30T15:03:52.588 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2023-09-30T15:03:52.588 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2023-09-30T15:03:52.588 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 7528, 2023-09-30T15:03:52.588 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 158, 2023-09-30T15:03:52.588 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2023-09-30T15:03:52.588 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2023-09-30T15:03:52.588 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2023-09-30T15:03:52.588 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2023-09-30T15:03:52.588 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 5467364, 2023-09-30T15:03:52.588 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 185050, 2023-09-30T15:03:52.588 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2023-09-30T15:03:52.588 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2023-09-30T15:03:52.589 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2023-09-30T15:03:52.589 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2023-09-30T15:03:52.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.589 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2023-09-30T15:03:52.589 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2023-09-30T15:03:52.589 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2023-09-30T15:03:52.589 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 241027, 2023-09-30T15:03:52.589 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 890812156, 2023-09-30T15:03:52.589 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 28, 2023-09-30T15:03:52.589 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 241027, 2023-09-30T15:03:52.589 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2023-09-30T15:03:52.589 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 241027, 2023-09-30T15:03:52.589 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1957.992240469, 2023-09-30T15:03:52.590 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.008123539 2023-09-30T15:03:52.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.590 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2023-09-30T15:03:52.590 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2023-09-30T15:03:52.590 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 241027, 2023-09-30T15:03:52.590 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 243413 2023-09-30T15:03:52.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.590 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 241027, 2023-09-30T15:03:52.590 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 1964, 2023-09-30T15:03:52.590 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 239063, 2023-09-30T15:03:52.590 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2023-09-30T15:03:52.590 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2023-09-30T15:03:52.591 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 95, 2023-09-30T15:03:52.591 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 614, 2023-09-30T15:03:52.591 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 1963, 2023-09-30T15:03:52.591 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 148755, 2023-09-30T15:03:52.591 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 942, 2023-09-30T15:03:52.591 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2023-09-30T15:03:52.591 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2023-09-30T15:03:52.591 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2023-09-30T15:03:52.591 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 19, 2023-09-30T15:03:52.591 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 88089, 2023-09-30T15:03:52.591 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2023-09-30T15:03:52.591 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2023-09-30T15:03:52.591 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2023-09-30T15:03:52.592 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 1, 2023-09-30T15:03:52.592 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 1228, 2023-09-30T15:03:52.592 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2023-09-30T15:03:52.592 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2023-09-30T15:03:52.592 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2023-09-30T15:03:52.592 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2023-09-30T15:03:52.592 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2023-09-30T15:03:52.592 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2023-09-30T15:03:52.592 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2023-09-30T15:03:52.592 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2023-09-30T15:03:52.592 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2023-09-30T15:03:52.592 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2023-09-30T15:03:52.593 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2023-09-30T15:03:52.593 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2023-09-30T15:03:52.593 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2023-09-30T15:03:52.593 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2023-09-30T15:03:52.593 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2023-09-30T15:03:52.593 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2023-09-30T15:03:52.593 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2023-09-30T15:03:52.593 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2023-09-30T15:03:52.593 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2023-09-30T15:03:52.593 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2023-09-30T15:03:52.593 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2023-09-30T15:03:52.593 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2023-09-30T15:03:52.593 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2023-09-30T15:03:52.594 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2023-09-30T15:03:52.594 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2023-09-30T15:03:52.594 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2023-09-30T15:03:52.594 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 85, 2023-09-30T15:03:52.594 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2023-09-30T15:03:52.594 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 2, 2023-09-30T15:03:52.594 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 12, 2023-09-30T15:03:52.594 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 12, 2023-09-30T15:03:52.594 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2023-09-30T15:03:52.594 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2023-09-30T15:03:52.594 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 1265, 2023-09-30T15:03:52.595 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2023-09-30T15:03:52.595 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 442 2023-09-30T15:03:52.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.595 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2023-09-30T15:03:52.595 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2023-09-30T15:03:52.595 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2023-09-30T15:03:52.595 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2023-09-30T15:03:52.595 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2023-09-30T15:03:52.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.595 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2023-09-30T15:03:52.595 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2023-09-30T15:03:52.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.595 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2023-09-30T15:03:52.595 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2023-09-30T15:03:52.596 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 10, 2023-09-30T15:03:52.596 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2023-09-30T15:03:52.596 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 10, 2023-09-30T15:03:52.596 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 87611, 2023-09-30T15:03:52.596 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2023-09-30T15:03:52.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.596 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-30T15:03:52.596 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2023-09-30T15:03:52.596 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2023-09-30T15:03:52.596 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2023-09-30T15:03:52.596 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 1595334, 2023-09-30T15:03:52.596 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 293716309, 2023-09-30T15:03:52.596 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:52.597 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 1595334, 2023-09-30T15:03:52.597 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2023-09-30T15:03:52.597 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2023-09-30T15:03:52.597 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1595334, 2023-09-30T15:03:52.597 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 293716309, 2023-09-30T15:03:52.597 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2023-09-30T15:03:52.597 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.597 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.597 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.597 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:52.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.598 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2023-09-30T15:03:52.598 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2023-09-30T15:03:52.598 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2023-09-30T15:03:52.598 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2023-09-30T15:03:52.598 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2023-09-30T15:03:52.598 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2023-09-30T15:03:52.598 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:52.598 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2023-09-30T15:03:52.598 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 241027, 2023-09-30T15:03:52.598 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 899660867, 2023-09-30T15:03:52.598 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 241027, 2023-09-30T15:03:52.598 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 899660867, 2023-09-30T15:03:52.599 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2023-09-30T15:03:52.599 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.599 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.599 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.599 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:52.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.599 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2023-09-30T15:03:52.599 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2023-09-30T15:03:52.599 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2023-09-30T15:03:52.599 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2023-09-30T15:03:52.599 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2023-09-30T15:03:52.599 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2023-09-30T15:03:52.600 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:52.600 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2023-09-30T15:03:52.600 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 241027, 2023-09-30T15:03:52.600 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 241027, 2023-09-30T15:03:52.600 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 241027, 2023-09-30T15:03:52.600 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 241027, 2023-09-30T15:03:52.600 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2023-09-30T15:03:52.600 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.600 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.600 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.600 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:52.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.601 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2023-09-30T15:03:52.601 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2023-09-30T15:03:52.601 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2023-09-30T15:03:52.601 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2023-09-30T15:03:52.601 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 87611, 2023-09-30T15:03:52.601 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 8848711, 2023-09-30T15:03:52.601 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:52.601 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 87611, 2023-09-30T15:03:52.601 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2023-09-30T15:03:52.601 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2023-09-30T15:03:52.601 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1963, 2023-09-30T15:03:52.601 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 8848711, 2023-09-30T15:03:52.602 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2023-09-30T15:03:52.602 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.602 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.602 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.602 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:52.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:52.602 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x561fa956a6e0": { 2023-09-30T15:03:52.602 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2023-09-30T15:03:52.602 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2023-09-30T15:03:52.602 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2023-09-30T15:03:52.602 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 412003, 2023-09-30T15:03:52.602 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 872750847, 2023-09-30T15:03:52.603 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:52.603 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 412003, 2023-09-30T15:03:52.603 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2023-09-30T15:03:52.603 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2023-09-30T15:03:52.603 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 146792, 2023-09-30T15:03:52.603 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 872750847, 2023-09-30T15:03:52.603 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2023-09-30T15:03:52.603 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:52.603 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:52.603 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:52.603 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:52.603 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:52.604 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-09-30T15:03:52.970 INFO:tasks.check_counter:Daemon mds.h mds.exported=14 2023-09-30T15:03:52.970 INFO:tasks.check_counter:Daemon mds.h mds.imported=13 2023-09-30T15:03:52.970 DEBUG:tasks.check_counter:Getting stats from k 2023-09-30T15:03:52.971 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-09-30T15:03:53.533 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.b/config 2023-09-30T15:03:54.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:53 smithi156 ceph-mon[103579]: pgmap v1055: 97 pgs: 97 active+clean; 637 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.1 MiB/s wr, 184 op/s 2023-09-30T15:03:54.091 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:53 smithi148 ceph-mon[102355]: pgmap v1055: 97 pgs: 97 active+clean; 637 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.1 MiB/s wr, 184 op/s 2023-09-30T15:03:54.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:53 smithi032 ceph-mon[98592]: pgmap v1055: 97 pgs: 97 active+clean; 637 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.1 MiB/s wr, 184 op/s 2023-09-30T15:03:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-09-30T15:03:54.809 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2023-09-30T15:03:54.809 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 0, 2023-09-30T15:03:54.809 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 2, 2023-09-30T15:03:54.809 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 0, 2023-09-30T15:03:54.809 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 0, 2023-09-30T15:03:54.809 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 4, 2023-09-30T15:03:54.809 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 0, 2023-09-30T15:03:54.809 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.001140530, 2023-09-30T15:03:54.810 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.000000000, 2023-09-30T15:03:54.810 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.000218527, 2023-09-30T15:03:54.810 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-30T15:03:54.810 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:54.810 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:54.810 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:54.810 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:54.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:54.810 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:54.810 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:54.811 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:54.811 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:54.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:54.811 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-30T15:03:54.811 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2023-09-30T15:03:54.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:54.811 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2023-09-30T15:03:54.811 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 2, 2023-09-30T15:03:54.811 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 331, 2023-09-30T15:03:54.811 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 320, 2023-09-30T15:03:54.811 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 211440, 2023-09-30T15:03:54.811 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 4, 2023-09-30T15:03:54.812 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 2, 2023-09-30T15:03:54.812 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.012871318, 2023-09-30T15:03:54.812 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.009253393, 2023-09-30T15:03:54.812 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.001237107, 2023-09-30T15:03:54.812 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-30T15:03:54.812 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:54.812 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 329, 2023-09-30T15:03:54.812 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.029915817, 2023-09-30T15:03:54.812 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000090929 2023-09-30T15:03:54.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:54.813 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:54.813 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:54.813 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:54.813 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:54.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:54.813 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-30T15:03:54.813 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 211440 2023-09-30T15:03:54.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:54.814 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2023-09-30T15:03:54.814 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 440, 2023-09-30T15:03:54.814 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 435, 2023-09-30T15:03:54.814 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 537344, 2023-09-30T15:03:54.814 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 261680, 2023-09-30T15:03:54.814 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 2, 2023-09-30T15:03:54.814 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 1, 2023-09-30T15:03:54.814 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.049342037, 2023-09-30T15:03:54.814 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.018274315, 2023-09-30T15:03:54.814 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.018737746, 2023-09-30T15:03:54.814 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.010109933, 2023-09-30T15:03:54.814 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:54.815 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 435, 2023-09-30T15:03:54.815 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.036391736, 2023-09-30T15:03:54.815 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000083659 2023-09-30T15:03:54.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:54.815 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:54.815 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:54.815 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:54.816 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:54.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:54.816 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 537344, 2023-09-30T15:03:54.816 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 261680 2023-09-30T15:03:54.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:54.816 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2023-09-30T15:03:54.816 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 0, 2023-09-30T15:03:54.816 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2023-09-30T15:03:54.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:54.816 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2023-09-30T15:03:54.817 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2023-09-30T15:03:54.817 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2023-09-30T15:03:54.817 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2023-09-30T15:03:54.817 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2023-09-30T15:03:54.817 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2023-09-30T15:03:54.817 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2023-09-30T15:03:54.817 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-30T15:03:54.817 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2023-09-30T15:03:54.818 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-30T15:03:54.818 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2023-09-30T15:03:54.818 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2023-09-30T15:03:54.818 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2023-09-30T15:03:54.818 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-30T15:03:54.818 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2023-09-30T15:03:54.818 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2023-09-30T15:03:54.818 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2023-09-30T15:03:54.819 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2023-09-30T15:03:54.819 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2023-09-30T15:03:54.819 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-30T15:03:54.819 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2023-09-30T15:03:54.819 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-30T15:03:54.819 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2023-09-30T15:03:54.819 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2023-09-30T15:03:54.819 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2023-09-30T15:03:54.820 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2023-09-30T15:03:54.820 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2023-09-30T15:03:54.820 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-30T15:03:54.820 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2023-09-30T15:03:54.820 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2023-09-30T15:03:54.820 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2023-09-30T15:03:54.820 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2023-09-30T15:03:54.820 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2023-09-30T15:03:54.821 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2023-09-30T15:03:54.821 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2023-09-30T15:03:54.821 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2023-09-30T15:03:54.821 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2023-09-30T15:03:54.821 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 149735, 2023-09-30T15:03:54.821 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 30, 2023-09-30T15:03:54.821 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2023-09-30T15:03:54.821 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2023-09-30T15:03:54.822 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2023-09-30T15:03:54.822 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2023-09-30T15:03:54.822 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2023-09-30T15:03:54.822 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2023-09-30T15:03:54.822 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2023-09-30T15:03:54.822 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2023-09-30T15:03:54.822 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 0, 2023-09-30T15:03:54.823 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 0, 2023-09-30T15:03:54.823 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2023-09-30T15:03:54.823 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2023-09-30T15:03:54.823 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2023-09-30T15:03:54.823 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2023-09-30T15:03:54.823 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 928, 2023-09-30T15:03:54.823 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 2, 2023-09-30T15:03:54.823 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2023-09-30T15:03:54.824 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2023-09-30T15:03:54.824 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2023-09-30T15:03:54.824 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2023-09-30T15:03:54.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-09-30T15:03:54.824 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-30T15:03:54.824 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2023-09-30T15:03:54.824 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2023-09-30T15:03:54.824 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2023-09-30T15:03:54.825 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 438, 2023-09-30T15:03:54.825 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 472746, 2023-09-30T15:03:54.825 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:54.825 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 438, 2023-09-30T15:03:54.825 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2023-09-30T15:03:54.825 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2023-09-30T15:03:54.825 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 438, 2023-09-30T15:03:54.825 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 472746, 2023-09-30T15:03:54.826 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2023-09-30T15:03:54.826 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2023-09-30T15:03:54.826 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2023-09-30T15:03:54.826 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2023-09-30T15:03:54.826 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:54.827 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-09-30T15:03:54.827 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-09-30T15:03:55.209 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-09-30T15:03:55.209 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-09-30T15:03:55.209 DEBUG:tasks.check_counter:Getting stats from c 2023-09-30T15:03:55.209 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-09-30T15:03:55.848 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.c/config 2023-09-30T15:03:56.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:55 smithi156 ceph-mon[103579]: pgmap v1056: 97 pgs: 97 active+clean; 618 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 6.1 MiB/s wr, 184 op/s 2023-09-30T15:03:56.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:55 smithi148 ceph-mon[102355]: pgmap v1056: 97 pgs: 97 active+clean; 618 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 6.1 MiB/s wr, 184 op/s 2023-09-30T15:03:56.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:55 smithi032 ceph-mon[98592]: pgmap v1056: 97 pgs: 97 active+clean; 618 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 6.1 MiB/s wr, 184 op/s 2023-09-30T15:03:57.409 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-09-30T15:03:57.409 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-09-30T15:03:57.409 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 3599, 2023-09-30T15:03:57.409 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 3599, 2023-09-30T15:03:57.409 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 202374738, 2023-09-30T15:03:57.410 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1118503, 2023-09-30T15:03:57.410 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 8, 2023-09-30T15:03:57.410 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2023-09-30T15:03:57.410 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.366525151, 2023-09-30T15:03:57.410 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.067450662, 2023-09-30T15:03:57.410 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.176785566, 2023-09-30T15:03:57.410 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.098300344, 2023-09-30T15:03:57.410 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:57.410 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3598, 2023-09-30T15:03:57.410 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.206767866, 2023-09-30T15:03:57.410 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000057467 2023-09-30T15:03:57.410 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.411 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:57.411 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.411 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.411 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.411 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.411 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-09-30T15:03:57.411 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 448 2023-09-30T15:03:57.411 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.411 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-09-30T15:03:57.411 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1765, 2023-09-30T15:03:57.411 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1725, 2023-09-30T15:03:57.412 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 153991108, 2023-09-30T15:03:57.412 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 661340, 2023-09-30T15:03:57.412 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2023-09-30T15:03:57.412 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 5, 2023-09-30T15:03:57.412 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.216262423, 2023-09-30T15:03:57.412 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.043346992, 2023-09-30T15:03:57.412 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.122265054, 2023-09-30T15:03:57.412 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.041871910, 2023-09-30T15:03:57.412 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:57.412 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1724, 2023-09-30T15:03:57.412 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.107641984, 2023-09-30T15:03:57.412 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000062437 2023-09-30T15:03:57.413 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.413 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:57.413 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.413 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.413 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.413 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.413 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 699968, 2023-09-30T15:03:57.413 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 261328 2023-09-30T15:03:57.413 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.413 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-09-30T15:03:57.413 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1464, 2023-09-30T15:03:57.413 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1790, 2023-09-30T15:03:57.414 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 172337437, 2023-09-30T15:03:57.414 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1104699, 2023-09-30T15:03:57.414 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2023-09-30T15:03:57.414 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 5, 2023-09-30T15:03:57.414 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.208646949, 2023-09-30T15:03:57.414 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.041123044, 2023-09-30T15:03:57.414 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.115641541, 2023-09-30T15:03:57.414 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.040373945, 2023-09-30T15:03:57.414 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:03:57.414 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1788, 2023-09-30T15:03:57.414 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.123626818, 2023-09-30T15:03:57.414 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000069142 2023-09-30T15:03:57.415 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.415 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:03:57.415 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.415 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.415 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.415 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.415 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 160, 2023-09-30T15:03:57.415 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 650016 2023-09-30T15:03:57.415 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.415 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-09-30T15:03:57.417 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2023-09-30T15:03:57.417 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-09-30T15:03:57.417 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.417 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2023-09-30T15:03:57.417 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-09-30T15:03:57.417 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-09-30T15:03:57.417 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 10748, 2023-09-30T15:03:57.417 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.806094504, 2023-09-30T15:03:57.417 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000074999 2023-09-30T15:03:57.417 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:03:57.417 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.418 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2023-09-30T15:03:57.418 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-09-30T15:03:57.418 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-09-30T15:03:57.418 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.418 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.418 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.418 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:03:57.418 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.418 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2023-09-30T15:03:57.418 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2023-09-30T15:03:57.418 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2023-09-30T15:03:57.419 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2023-09-30T15:03:57.419 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.419 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.419 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.419 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.419 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2023-09-30T15:03:57.419 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2023-09-30T15:03:57.419 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2023-09-30T15:03:57.419 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2023-09-30T15:03:57.419 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 0, 2023-09-30T15:03:57.419 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2023-09-30T15:03:57.420 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2023-09-30T15:03:57.420 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 55922, 2023-09-30T15:03:57.420 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 37942, 2023-09-30T15:03:57.420 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 17976, 2023-09-30T15:03:57.420 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 4, 2023-09-30T15:03:57.420 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 1719, 2023-09-30T15:03:57.420 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 301, 2023-09-30T15:03:57.420 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2023-09-30T15:03:57.420 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2023-09-30T15:03:57.420 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 3, 2023-09-30T15:03:57.420 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 0, 2023-09-30T15:03:57.420 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 0, 2023-09-30T15:03:57.421 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2023-09-30T15:03:57.421 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2023-09-30T15:03:57.421 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2023-09-30T15:03:57.421 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2023-09-30T15:03:57.421 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2023-09-30T15:03:57.421 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2023-09-30T15:03:57.421 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2023-09-30T15:03:57.421 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2023-09-30T15:03:57.421 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2023-09-30T15:03:57.421 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2023-09-30T15:03:57.421 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2023-09-30T15:03:57.421 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2023-09-30T15:03:57.422 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2023-09-30T15:03:57.422 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2023-09-30T15:03:57.422 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 4, 2023-09-30T15:03:57.422 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 246, 2023-09-30T15:03:57.422 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2023-09-30T15:03:57.422 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2023-09-30T15:03:57.422 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2023-09-30T15:03:57.422 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2023-09-30T15:03:57.422 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2023-09-30T15:03:57.422 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2023-09-30T15:03:57.422 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2023-09-30T15:03:57.422 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2023-09-30T15:03:57.423 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2023-09-30T15:03:57.423 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2023-09-30T15:03:57.423 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2023-09-30T15:03:57.423 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2023-09-30T15:03:57.423 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2023-09-30T15:03:57.423 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-30T15:03:57.423 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-30T15:03:57.423 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2023-09-30T15:03:57.423 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2023-09-30T15:03:57.423 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2023-09-30T15:03:57.423 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2023-09-30T15:03:57.424 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.424 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2023-09-30T15:03:57.424 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2023-09-30T15:03:57.424 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2023-09-30T15:03:57.424 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2023-09-30T15:03:57.424 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2023-09-30T15:03:57.424 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2023-09-30T15:03:57.424 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2023-09-30T15:03:57.424 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2023-09-30T15:03:57.424 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2023-09-30T15:03:57.424 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2023-09-30T15:03:57.424 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2023-09-30T15:03:57.425 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2023-09-30T15:03:57.425 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2023-09-30T15:03:57.425 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2023-09-30T15:03:57.425 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2023-09-30T15:03:57.425 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2023-09-30T15:03:57.425 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2023-09-30T15:03:57.425 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2023-09-30T15:03:57.425 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2023-09-30T15:03:57.425 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2023-09-30T15:03:57.425 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2023-09-30T15:03:57.425 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2023-09-30T15:03:57.425 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2023-09-30T15:03:57.426 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2023-09-30T15:03:57.426 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.426 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2023-09-30T15:03:57.426 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2023-09-30T15:03:57.426 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 0, 2023-09-30T15:03:57.426 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2023-09-30T15:03:57.426 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2023-09-30T15:03:57.426 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 264505, 2023-09-30T15:03:57.426 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2023-09-30T15:03:57.426 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2023-09-30T15:03:57.426 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 0, 2023-09-30T15:03:57.426 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2023-09-30T15:03:57.427 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2023-09-30T15:03:57.427 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 7, 2023-09-30T15:03:57.427 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 112, 2023-09-30T15:03:57.427 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2023-09-30T15:03:57.427 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2023-09-30T15:03:57.427 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 521297013, 2023-09-30T15:03:57.427 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 530449748, 2023-09-30T15:03:57.427 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 530448096, 2023-09-30T15:03:57.427 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2023-09-30T15:03:57.427 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.427 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.427 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.427 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.428 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 264505 2023-09-30T15:03:57.428 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.428 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2023-09-30T15:03:57.428 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 766, 2023-09-30T15:03:57.428 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 56194, 2023-09-30T15:03:57.428 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 55428, 2023-09-30T15:03:57.428 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 57, 2023-09-30T15:03:57.428 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 232, 2023-09-30T15:03:57.428 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 175, 2023-09-30T15:03:57.428 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 55922, 2023-09-30T15:03:57.428 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 56745, 2023-09-30T15:03:57.429 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 823, 2023-09-30T15:03:57.429 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2023-09-30T15:03:57.429 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2023-09-30T15:03:57.429 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2023-09-30T15:03:57.429 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 101692, 2023-09-30T15:03:57.429 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 207152 2023-09-30T15:03:57.429 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.429 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2023-09-30T15:03:57.429 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2023-09-30T15:03:57.429 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2023-09-30T15:03:57.429 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2023-09-30T15:03:57.429 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2023-09-30T15:03:57.429 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2023-09-30T15:03:57.430 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2023-09-30T15:03:57.430 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.430 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.430 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.430 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.430 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2023-09-30T15:03:57.430 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.430 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.430 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.430 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.430 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2023-09-30T15:03:57.430 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.431 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.431 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.431 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.431 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2023-09-30T15:03:57.431 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.431 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.431 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.431 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.431 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2023-09-30T15:03:57.431 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.431 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.431 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.432 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.432 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2023-09-30T15:03:57.432 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.432 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.432 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.432 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.432 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2023-09-30T15:03:57.432 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.432 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.432 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.432 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.432 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2023-09-30T15:03:57.433 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.433 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.433 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.433 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.433 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2023-09-30T15:03:57.433 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.433 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.433 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.433 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.433 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2023-09-30T15:03:57.433 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.434 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.434 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.434 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.434 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2023-09-30T15:03:57.434 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.434 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.434 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.434 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.434 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2023-09-30T15:03:57.434 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.434 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.435 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.435 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.435 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2023-09-30T15:03:57.435 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.435 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.435 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.435 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.435 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2023-09-30T15:03:57.435 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.435 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.435 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.435 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.435 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2023-09-30T15:03:57.436 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.436 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.436 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.436 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.436 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2023-09-30T15:03:57.436 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.436 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.436 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.436 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.436 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2023-09-30T15:03:57.436 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.436 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.437 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.437 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.437 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2023-09-30T15:03:57.437 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.437 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.437 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.437 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.437 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2023-09-30T15:03:57.437 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.437 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.437 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.437 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.437 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2023-09-30T15:03:57.438 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.438 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.438 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.438 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.438 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2023-09-30T15:03:57.438 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.438 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.438 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.438 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.438 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2023-09-30T15:03:57.438 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.438 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.439 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.439 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.439 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2023-09-30T15:03:57.439 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.439 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.439 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.439 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.439 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2023-09-30T15:03:57.439 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.439 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.439 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.439 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.439 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2023-09-30T15:03:57.440 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.440 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.440 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.440 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.440 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2023-09-30T15:03:57.440 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.440 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.440 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.440 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.440 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2023-09-30T15:03:57.440 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.440 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.441 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.441 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.441 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2023-09-30T15:03:57.441 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.441 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.441 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.441 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.441 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2023-09-30T15:03:57.441 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.441 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.441 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.441 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.441 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2023-09-30T15:03:57.442 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2023-09-30T15:03:57.442 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2023-09-30T15:03:57.442 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.442 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.442 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.442 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:03:57.442 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.442 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2023-09-30T15:03:57.442 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 1, 2023-09-30T15:03:57.442 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 2, 2023-09-30T15:03:57.442 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 1, 2023-09-30T15:03:57.443 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 2, 2023-09-30T15:03:57.443 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2023-09-30T15:03:57.443 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2023-09-30T15:03:57.443 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2023-09-30T15:03:57.443 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 4113, 2023-09-30T15:03:57.443 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2023-09-30T15:03:57.443 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.443 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-09-30T15:03:57.443 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-09-30T15:03:57.443 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-09-30T15:03:57.443 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-09-30T15:03:57.443 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-09-30T15:03:57.444 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-09-30T15:03:57.444 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-09-30T15:03:57.444 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-30T15:03:57.444 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-09-30T15:03:57.444 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-30T15:03:57.444 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-09-30T15:03:57.444 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-09-30T15:03:57.444 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-09-30T15:03:57.444 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-30T15:03:57.444 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-09-30T15:03:57.444 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-09-30T15:03:57.444 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-09-30T15:03:57.445 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-09-30T15:03:57.445 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-09-30T15:03:57.445 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-30T15:03:57.445 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-09-30T15:03:57.445 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-30T15:03:57.445 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-09-30T15:03:57.445 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-09-30T15:03:57.445 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-09-30T15:03:57.445 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-09-30T15:03:57.445 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-09-30T15:03:57.445 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-30T15:03:57.445 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-09-30T15:03:57.446 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-09-30T15:03:57.446 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-09-30T15:03:57.446 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-09-30T15:03:57.446 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-09-30T15:03:57.446 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-09-30T15:03:57.446 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-09-30T15:03:57.446 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-09-30T15:03:57.446 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-09-30T15:03:57.446 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 266523, 2023-09-30T15:03:57.446 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 834, 2023-09-30T15:03:57.446 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-09-30T15:03:57.447 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-09-30T15:03:57.447 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-09-30T15:03:57.447 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-09-30T15:03:57.447 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-09-30T15:03:57.447 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-09-30T15:03:57.447 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-09-30T15:03:57.447 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-09-30T15:03:57.447 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 7528, 2023-09-30T15:03:57.447 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 158, 2023-09-30T15:03:57.447 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-09-30T15:03:57.447 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-09-30T15:03:57.447 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-09-30T15:03:57.448 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-09-30T15:03:57.448 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 40832570, 2023-09-30T15:03:57.448 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 328013, 2023-09-30T15:03:57.448 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-09-30T15:03:57.448 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-09-30T15:03:57.448 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-09-30T15:03:57.448 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-09-30T15:03:57.448 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.448 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2023-09-30T15:03:57.448 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2023-09-30T15:03:57.448 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2023-09-30T15:03:57.448 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 6343, 2023-09-30T15:03:57.448 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 26, 2023-09-30T15:03:57.449 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 4, 2023-09-30T15:03:57.449 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 6343, 2023-09-30T15:03:57.449 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2023-09-30T15:03:57.449 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6343, 2023-09-30T15:03:57.449 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 11.561837241, 2023-09-30T15:03:57.449 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001822771 2023-09-30T15:03:57.449 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.449 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2023-09-30T15:03:57.449 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2023-09-30T15:03:57.449 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6343, 2023-09-30T15:03:57.449 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 6344 2023-09-30T15:03:57.449 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.450 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 6343, 2023-09-30T15:03:57.450 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 6343, 2023-09-30T15:03:57.450 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 0, 2023-09-30T15:03:57.450 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2023-09-30T15:03:57.450 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2023-09-30T15:03:57.450 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 3299, 2023-09-30T15:03:57.450 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 0, 2023-09-30T15:03:57.450 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 3041, 2023-09-30T15:03:57.450 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 0, 2023-09-30T15:03:57.450 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 0, 2023-09-30T15:03:57.450 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2023-09-30T15:03:57.450 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2023-09-30T15:03:57.451 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2023-09-30T15:03:57.451 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2023-09-30T15:03:57.451 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 0, 2023-09-30T15:03:57.451 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2023-09-30T15:03:57.451 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2023-09-30T15:03:57.451 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2023-09-30T15:03:57.451 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 2, 2023-09-30T15:03:57.451 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 0, 2023-09-30T15:03:57.451 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2023-09-30T15:03:57.451 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2023-09-30T15:03:57.451 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2023-09-30T15:03:57.451 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2023-09-30T15:03:57.451 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2023-09-30T15:03:57.452 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2023-09-30T15:03:57.452 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2023-09-30T15:03:57.452 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2023-09-30T15:03:57.452 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2023-09-30T15:03:57.452 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2023-09-30T15:03:57.452 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2023-09-30T15:03:57.452 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2023-09-30T15:03:57.452 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2023-09-30T15:03:57.452 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2023-09-30T15:03:57.452 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2023-09-30T15:03:57.452 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2023-09-30T15:03:57.452 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2023-09-30T15:03:57.453 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2023-09-30T15:03:57.453 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2023-09-30T15:03:57.453 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2023-09-30T15:03:57.453 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2023-09-30T15:03:57.453 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2023-09-30T15:03:57.453 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2023-09-30T15:03:57.453 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2023-09-30T15:03:57.453 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2023-09-30T15:03:57.453 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2023-09-30T15:03:57.453 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 85, 2023-09-30T15:03:57.453 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2023-09-30T15:03:57.453 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 2, 2023-09-30T15:03:57.454 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 11, 2023-09-30T15:03:57.454 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 11, 2023-09-30T15:03:57.454 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2023-09-30T15:03:57.454 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2023-09-30T15:03:57.454 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 0, 2023-09-30T15:03:57.454 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 2, 2023-09-30T15:03:57.454 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 0 2023-09-30T15:03:57.454 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.454 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2023-09-30T15:03:57.454 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 0, 2023-09-30T15:03:57.454 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2023-09-30T15:03:57.454 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2023-09-30T15:03:57.455 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2023-09-30T15:03:57.455 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.455 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2023-09-30T15:03:57.455 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2023-09-30T15:03:57.455 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.455 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2023-09-30T15:03:57.455 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2023-09-30T15:03:57.455 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2023-09-30T15:03:57.455 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2023-09-30T15:03:57.455 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2023-09-30T15:03:57.455 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2023-09-30T15:03:57.455 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2023-09-30T15:03:57.456 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.456 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-30T15:03:57.456 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-30T15:03:57.456 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-09-30T15:03:57.456 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-30T15:03:57.456 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 6824, 2023-09-30T15:03:57.456 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 528061747, 2023-09-30T15:03:57.456 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:57.456 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 6824, 2023-09-30T15:03:57.456 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-30T15:03:57.456 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-30T15:03:57.456 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 6824, 2023-09-30T15:03:57.456 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 528061747, 2023-09-30T15:03:57.457 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-30T15:03:57.457 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.457 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.457 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.457 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:03:57.457 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.457 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2023-09-30T15:03:57.457 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-30T15:03:57.457 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-09-30T15:03:57.457 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-30T15:03:57.457 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-30T15:03:57.457 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-30T15:03:57.458 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:57.458 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-30T15:03:57.458 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 6343, 2023-09-30T15:03:57.458 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 526259735, 2023-09-30T15:03:57.458 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 6343, 2023-09-30T15:03:57.458 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 526259735, 2023-09-30T15:03:57.458 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-30T15:03:57.458 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.458 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.458 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.458 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:03:57.458 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.459 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2023-09-30T15:03:57.459 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-30T15:03:57.459 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2023-09-30T15:03:57.459 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-30T15:03:57.459 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-30T15:03:57.459 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-30T15:03:57.459 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:57.459 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-30T15:03:57.459 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 6343, 2023-09-30T15:03:57.459 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 6343, 2023-09-30T15:03:57.459 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 6343, 2023-09-30T15:03:57.459 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 6343, 2023-09-30T15:03:57.460 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-30T15:03:57.460 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.460 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.460 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.460 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:03:57.460 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.460 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2023-09-30T15:03:57.460 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-30T15:03:57.460 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-09-30T15:03:57.460 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-30T15:03:57.460 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-30T15:03:57.460 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-30T15:03:57.461 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:57.461 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-30T15:03:57.461 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-30T15:03:57.461 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-30T15:03:57.461 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-09-30T15:03:57.461 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-09-30T15:03:57.461 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-30T15:03:57.461 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.461 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.461 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.461 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:03:57.461 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:03:57.461 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x55d73ccd6960": { 2023-09-30T15:03:57.462 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-30T15:03:57.462 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-09-30T15:03:57.462 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-30T15:03:57.462 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-30T15:03:57.462 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-30T15:03:57.462 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-30T15:03:57.462 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-30T15:03:57.462 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-30T15:03:57.462 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-30T15:03:57.462 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-09-30T15:03:57.462 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-09-30T15:03:57.462 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-30T15:03:57.463 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:03:57.463 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:03:57.463 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:03:57.463 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:03:57.463 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:03:57.463 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-09-30T15:03:57.855 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-09-30T15:03:57.855 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-09-30T15:03:57.855 DEBUG:tasks.check_counter:Getting stats from f 2023-09-30T15:03:57.856 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-09-30T15:03:58.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:57 smithi156 ceph-mon[103579]: pgmap v1057: 97 pgs: 97 active+clean; 619 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 6.0 MiB/s wr, 173 op/s 2023-09-30T15:03:58.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:57 smithi148 ceph-mon[102355]: pgmap v1057: 97 pgs: 97 active+clean; 619 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 6.0 MiB/s wr, 173 op/s 2023-09-30T15:03:58.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:57 smithi032 ceph-mon[98592]: pgmap v1057: 97 pgs: 97 active+clean; 619 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 6.0 MiB/s wr, 173 op/s 2023-09-30T15:03:58.460 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.c/config 2023-09-30T15:03:59.761 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:03:59 smithi156 ceph-mon[103579]: pgmap v1058: 97 pgs: 97 active+clean; 619 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 6.3 MiB/s wr, 185 op/s 2023-09-30T15:04:00.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:03:59 smithi148 ceph-mon[102355]: pgmap v1058: 97 pgs: 97 active+clean; 619 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 6.3 MiB/s wr, 185 op/s 2023-09-30T15:04:00.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:03:59 smithi032 ceph-mon[98592]: pgmap v1058: 97 pgs: 97 active+clean; 619 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 6.3 MiB/s wr, 185 op/s 2023-09-30T15:04:01.533 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-09-30T15:04:01.534 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-09-30T15:04:01.534 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 3085, 2023-09-30T15:04:01.534 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 3045, 2023-09-30T15:04:01.534 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 152486773, 2023-09-30T15:04:01.546 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1072571, 2023-09-30T15:04:01.547 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 7, 2023-09-30T15:04:01.547 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 3, 2023-09-30T15:04:01.547 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.331853077, 2023-09-30T15:04:01.547 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.072788570, 2023-09-30T15:04:01.547 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.151638494, 2023-09-30T15:04:01.547 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.086422134, 2023-09-30T15:04:01.547 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:04:01.547 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3044, 2023-09-30T15:04:01.547 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.204628847, 2023-09-30T15:04:01.547 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000067223 2023-09-30T15:04:01.547 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.548 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:04:01.548 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.548 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.548 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.548 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.548 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 700352, 2023-09-30T15:04:01.548 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 262416 2023-09-30T15:04:01.548 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.548 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-09-30T15:04:01.548 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1387, 2023-09-30T15:04:01.548 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1713, 2023-09-30T15:04:01.548 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 165521255, 2023-09-30T15:04:01.548 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1082088, 2023-09-30T15:04:01.549 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 7, 2023-09-30T15:04:01.549 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 6, 2023-09-30T15:04:01.549 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.205847823, 2023-09-30T15:04:01.549 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.032079779, 2023-09-30T15:04:01.549 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.126518454, 2023-09-30T15:04:01.549 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.036089702, 2023-09-30T15:04:01.549 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:04:01.549 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1712, 2023-09-30T15:04:01.549 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.117541673, 2023-09-30T15:04:01.549 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000068657 2023-09-30T15:04:01.549 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.549 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:04:01.550 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.550 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.550 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.550 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.550 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-09-30T15:04:01.550 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 652224 2023-09-30T15:04:01.550 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.550 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-09-30T15:04:01.550 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 2265, 2023-09-30T15:04:01.550 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2266, 2023-09-30T15:04:01.550 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 260625545, 2023-09-30T15:04:01.550 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 704111, 2023-09-30T15:04:01.551 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2023-09-30T15:04:01.551 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 5, 2023-09-30T15:04:01.551 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.285912874, 2023-09-30T15:04:01.551 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.043189049, 2023-09-30T15:04:01.551 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.172527653, 2023-09-30T15:04:01.551 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.056209393, 2023-09-30T15:04:01.551 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:04:01.551 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2264, 2023-09-30T15:04:01.551 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.117487898, 2023-09-30T15:04:01.551 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000051893 2023-09-30T15:04:01.551 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.551 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:04:01.552 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.552 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.552 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.552 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.552 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-30T15:04:01.552 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2023-09-30T15:04:01.552 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.552 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-09-30T15:04:01.552 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2023-09-30T15:04:01.552 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-09-30T15:04:01.552 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.552 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2023-09-30T15:04:01.552 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-09-30T15:04:01.553 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-09-30T15:04:01.553 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 10658, 2023-09-30T15:04:01.553 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.729347147, 2023-09-30T15:04:01.553 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000068431 2023-09-30T15:04:01.553 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:01.553 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.553 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2023-09-30T15:04:01.553 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-09-30T15:04:01.553 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-09-30T15:04:01.553 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.553 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.553 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.553 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:01.554 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.554 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2023-09-30T15:04:01.554 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2023-09-30T15:04:01.554 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2023-09-30T15:04:01.554 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2023-09-30T15:04:01.554 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.554 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.554 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.554 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.554 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2023-09-30T15:04:01.554 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2023-09-30T15:04:01.554 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2023-09-30T15:04:01.555 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2023-09-30T15:04:01.555 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 0, 2023-09-30T15:04:01.555 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2023-09-30T15:04:01.555 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2023-09-30T15:04:01.555 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 50692, 2023-09-30T15:04:01.555 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 17737, 2023-09-30T15:04:01.555 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 32926, 2023-09-30T15:04:01.555 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 29, 2023-09-30T15:04:01.555 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 25353, 2023-09-30T15:04:01.555 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 40877, 2023-09-30T15:04:01.555 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2023-09-30T15:04:01.555 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2023-09-30T15:04:01.555 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 1, 2023-09-30T15:04:01.556 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 0, 2023-09-30T15:04:01.556 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 0, 2023-09-30T15:04:01.556 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2023-09-30T15:04:01.556 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2023-09-30T15:04:01.556 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2023-09-30T15:04:01.556 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2023-09-30T15:04:01.556 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2023-09-30T15:04:01.556 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2023-09-30T15:04:01.556 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2023-09-30T15:04:01.556 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2023-09-30T15:04:01.556 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2023-09-30T15:04:01.556 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2023-09-30T15:04:01.556 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2023-09-30T15:04:01.557 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2023-09-30T15:04:01.557 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2023-09-30T15:04:01.557 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2023-09-30T15:04:01.557 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 0, 2023-09-30T15:04:01.557 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 0, 2023-09-30T15:04:01.557 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2023-09-30T15:04:01.557 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2023-09-30T15:04:01.557 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2023-09-30T15:04:01.557 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2023-09-30T15:04:01.557 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2023-09-30T15:04:01.557 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2023-09-30T15:04:01.557 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2023-09-30T15:04:01.557 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2023-09-30T15:04:01.558 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2023-09-30T15:04:01.558 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2023-09-30T15:04:01.558 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2023-09-30T15:04:01.558 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2023-09-30T15:04:01.558 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2023-09-30T15:04:01.558 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-30T15:04:01.558 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-30T15:04:01.558 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2023-09-30T15:04:01.558 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2023-09-30T15:04:01.558 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2023-09-30T15:04:01.558 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2023-09-30T15:04:01.558 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.559 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2023-09-30T15:04:01.559 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2023-09-30T15:04:01.559 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2023-09-30T15:04:01.559 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2023-09-30T15:04:01.559 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2023-09-30T15:04:01.559 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2023-09-30T15:04:01.559 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2023-09-30T15:04:01.559 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2023-09-30T15:04:01.559 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2023-09-30T15:04:01.559 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2023-09-30T15:04:01.559 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2023-09-30T15:04:01.560 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2023-09-30T15:04:01.560 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2023-09-30T15:04:01.560 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2023-09-30T15:04:01.560 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2023-09-30T15:04:01.560 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2023-09-30T15:04:01.560 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2023-09-30T15:04:01.560 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2023-09-30T15:04:01.560 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2023-09-30T15:04:01.560 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2023-09-30T15:04:01.560 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2023-09-30T15:04:01.560 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2023-09-30T15:04:01.560 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2023-09-30T15:04:01.561 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2023-09-30T15:04:01.561 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.561 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2023-09-30T15:04:01.561 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2023-09-30T15:04:01.561 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 0, 2023-09-30T15:04:01.561 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2023-09-30T15:04:01.561 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2023-09-30T15:04:01.561 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 316992, 2023-09-30T15:04:01.561 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2023-09-30T15:04:01.561 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2023-09-30T15:04:01.561 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 0, 2023-09-30T15:04:01.561 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2023-09-30T15:04:01.562 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2023-09-30T15:04:01.562 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 130, 2023-09-30T15:04:01.562 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 27, 2023-09-30T15:04:01.562 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2023-09-30T15:04:01.562 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2023-09-30T15:04:01.562 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 357246054, 2023-09-30T15:04:01.562 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 580401414, 2023-09-30T15:04:01.562 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 580400815, 2023-09-30T15:04:01.562 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2023-09-30T15:04:01.562 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.562 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.563 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.563 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.563 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 316992 2023-09-30T15:04:01.563 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.563 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2023-09-30T15:04:01.563 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 37, 2023-09-30T15:04:01.563 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 66358, 2023-09-30T15:04:01.563 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 66321, 2023-09-30T15:04:01.563 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 72, 2023-09-30T15:04:01.563 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 209, 2023-09-30T15:04:01.563 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 137, 2023-09-30T15:04:01.563 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 25599, 2023-09-30T15:04:01.564 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 66943, 2023-09-30T15:04:01.564 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 41344, 2023-09-30T15:04:01.564 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2023-09-30T15:04:01.564 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2023-09-30T15:04:01.564 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2023-09-30T15:04:01.564 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 98880, 2023-09-30T15:04:01.564 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 207152 2023-09-30T15:04:01.564 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.564 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2023-09-30T15:04:01.564 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2023-09-30T15:04:01.564 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2023-09-30T15:04:01.565 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2023-09-30T15:04:01.565 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2023-09-30T15:04:01.565 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2023-09-30T15:04:01.565 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2023-09-30T15:04:01.565 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.565 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.565 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.565 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.565 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2023-09-30T15:04:01.565 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.565 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.565 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.565 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.566 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2023-09-30T15:04:01.566 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.566 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.566 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.566 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.566 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2023-09-30T15:04:01.566 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.566 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.566 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.566 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.566 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2023-09-30T15:04:01.566 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.567 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.567 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.567 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.567 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2023-09-30T15:04:01.567 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.567 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.567 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.567 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.567 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2023-09-30T15:04:01.567 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.567 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.567 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.568 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.568 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2023-09-30T15:04:01.568 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.568 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.568 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.568 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.568 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2023-09-30T15:04:01.568 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.568 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.568 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.568 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.568 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2023-09-30T15:04:01.569 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.569 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.569 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.569 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.569 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2023-09-30T15:04:01.569 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.569 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.569 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.569 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.569 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2023-09-30T15:04:01.570 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.570 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.570 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.570 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.570 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2023-09-30T15:04:01.570 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.570 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.570 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.570 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.570 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2023-09-30T15:04:01.571 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.571 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.571 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.571 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.571 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2023-09-30T15:04:01.571 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.571 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.571 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.571 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.571 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2023-09-30T15:04:01.571 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.571 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.572 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.572 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.572 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2023-09-30T15:04:01.572 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.572 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.572 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.572 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.572 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2023-09-30T15:04:01.572 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.572 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.572 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.572 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.573 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2023-09-30T15:04:01.573 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.573 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.573 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.573 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.573 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2023-09-30T15:04:01.573 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.573 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.573 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.573 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.574 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2023-09-30T15:04:01.574 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.574 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.574 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.574 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.574 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2023-09-30T15:04:01.574 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.574 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.574 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.574 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.574 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2023-09-30T15:04:01.574 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.574 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.575 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.575 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.575 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2023-09-30T15:04:01.575 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.575 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.575 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.575 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.575 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2023-09-30T15:04:01.575 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.575 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.575 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.575 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.576 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2023-09-30T15:04:01.576 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.576 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.576 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.576 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.576 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2023-09-30T15:04:01.576 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.576 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.576 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.576 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.576 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2023-09-30T15:04:01.576 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.576 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.577 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.577 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.577 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2023-09-30T15:04:01.577 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.577 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.577 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.577 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.577 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2023-09-30T15:04:01.577 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2023-09-30T15:04:01.577 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2023-09-30T15:04:01.577 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.577 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.578 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.578 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:01.578 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.578 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2023-09-30T15:04:01.578 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 1, 2023-09-30T15:04:01.578 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 1, 2023-09-30T15:04:01.578 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 0, 2023-09-30T15:04:01.578 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 1, 2023-09-30T15:04:01.578 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2023-09-30T15:04:01.578 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2023-09-30T15:04:01.578 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2023-09-30T15:04:01.579 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 2857, 2023-09-30T15:04:01.579 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2023-09-30T15:04:01.579 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.579 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-09-30T15:04:01.579 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-09-30T15:04:01.579 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-09-30T15:04:01.579 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-09-30T15:04:01.579 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-09-30T15:04:01.579 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-09-30T15:04:01.579 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-09-30T15:04:01.579 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-30T15:04:01.579 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-09-30T15:04:01.579 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-30T15:04:01.580 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-09-30T15:04:01.580 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-09-30T15:04:01.580 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-09-30T15:04:01.580 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-30T15:04:01.580 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-09-30T15:04:01.580 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-09-30T15:04:01.580 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-09-30T15:04:01.580 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-09-30T15:04:01.580 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-09-30T15:04:01.580 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-30T15:04:01.580 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-09-30T15:04:01.580 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-30T15:04:01.581 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-09-30T15:04:01.581 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-09-30T15:04:01.581 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-09-30T15:04:01.581 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-09-30T15:04:01.581 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-09-30T15:04:01.581 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-30T15:04:01.581 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-09-30T15:04:01.581 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-09-30T15:04:01.581 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-09-30T15:04:01.581 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-09-30T15:04:01.581 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-09-30T15:04:01.581 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-09-30T15:04:01.581 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-09-30T15:04:01.582 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-09-30T15:04:01.582 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-09-30T15:04:01.582 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 262963, 2023-09-30T15:04:01.582 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 728, 2023-09-30T15:04:01.582 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-09-30T15:04:01.582 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-09-30T15:04:01.582 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-09-30T15:04:01.582 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-09-30T15:04:01.582 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-09-30T15:04:01.582 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-09-30T15:04:01.582 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-09-30T15:04:01.582 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-09-30T15:04:01.583 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 7528, 2023-09-30T15:04:01.583 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 158, 2023-09-30T15:04:01.583 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-09-30T15:04:01.583 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-09-30T15:04:01.583 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-09-30T15:04:01.583 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-09-30T15:04:01.583 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 36953790, 2023-09-30T15:04:01.583 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 292366, 2023-09-30T15:04:01.583 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-09-30T15:04:01.583 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-09-30T15:04:01.583 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-09-30T15:04:01.583 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-09-30T15:04:01.584 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.584 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2023-09-30T15:04:01.584 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2023-09-30T15:04:01.584 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2023-09-30T15:04:01.584 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 6251, 2023-09-30T15:04:01.584 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 21, 2023-09-30T15:04:01.584 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 3, 2023-09-30T15:04:01.584 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 6251, 2023-09-30T15:04:01.584 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2023-09-30T15:04:01.584 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6251, 2023-09-30T15:04:01.584 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 14.115801292, 2023-09-30T15:04:01.585 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002258166 2023-09-30T15:04:01.585 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.585 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2023-09-30T15:04:01.585 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2023-09-30T15:04:01.585 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 6251, 2023-09-30T15:04:01.585 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 6252 2023-09-30T15:04:01.585 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.585 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 6251, 2023-09-30T15:04:01.585 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 6251, 2023-09-30T15:04:01.585 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 0, 2023-09-30T15:04:01.585 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2023-09-30T15:04:01.585 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2023-09-30T15:04:01.586 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 3186, 2023-09-30T15:04:01.586 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 0, 2023-09-30T15:04:01.586 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 3063, 2023-09-30T15:04:01.586 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 0, 2023-09-30T15:04:01.586 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 0, 2023-09-30T15:04:01.586 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2023-09-30T15:04:01.586 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2023-09-30T15:04:01.586 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2023-09-30T15:04:01.586 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2023-09-30T15:04:01.586 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 0, 2023-09-30T15:04:01.586 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2023-09-30T15:04:01.586 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2023-09-30T15:04:01.587 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2023-09-30T15:04:01.587 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 1, 2023-09-30T15:04:01.587 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 0, 2023-09-30T15:04:01.587 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2023-09-30T15:04:01.587 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2023-09-30T15:04:01.587 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2023-09-30T15:04:01.587 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2023-09-30T15:04:01.587 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2023-09-30T15:04:01.587 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2023-09-30T15:04:01.587 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2023-09-30T15:04:01.587 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2023-09-30T15:04:01.587 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2023-09-30T15:04:01.587 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2023-09-30T15:04:01.588 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2023-09-30T15:04:01.588 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2023-09-30T15:04:01.588 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2023-09-30T15:04:01.588 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2023-09-30T15:04:01.588 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2023-09-30T15:04:01.588 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2023-09-30T15:04:01.588 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2023-09-30T15:04:01.588 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2023-09-30T15:04:01.588 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2023-09-30T15:04:01.588 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2023-09-30T15:04:01.588 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2023-09-30T15:04:01.588 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2023-09-30T15:04:01.589 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2023-09-30T15:04:01.589 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2023-09-30T15:04:01.589 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2023-09-30T15:04:01.589 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2023-09-30T15:04:01.589 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 85, 2023-09-30T15:04:01.589 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2023-09-30T15:04:01.589 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 2, 2023-09-30T15:04:01.589 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 11, 2023-09-30T15:04:01.589 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 11, 2023-09-30T15:04:01.589 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2023-09-30T15:04:01.589 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2023-09-30T15:04:01.589 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 0, 2023-09-30T15:04:01.589 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 2, 2023-09-30T15:04:01.590 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 0 2023-09-30T15:04:01.590 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.590 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2023-09-30T15:04:01.590 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 0, 2023-09-30T15:04:01.590 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2023-09-30T15:04:01.590 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2023-09-30T15:04:01.590 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2023-09-30T15:04:01.590 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.590 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2023-09-30T15:04:01.590 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2023-09-30T15:04:01.590 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.590 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2023-09-30T15:04:01.591 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2023-09-30T15:04:01.591 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2023-09-30T15:04:01.591 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2023-09-30T15:04:01.591 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2023-09-30T15:04:01.591 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2023-09-30T15:04:01.591 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2023-09-30T15:04:01.591 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.591 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-30T15:04:01.591 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-30T15:04:01.591 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-09-30T15:04:01.591 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-30T15:04:01.591 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 6733, 2023-09-30T15:04:01.592 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 578000169, 2023-09-30T15:04:01.592 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-30T15:04:01.592 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 6733, 2023-09-30T15:04:01.592 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-30T15:04:01.592 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-30T15:04:01.592 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 6733, 2023-09-30T15:04:01.592 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 578000169, 2023-09-30T15:04:01.592 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-30T15:04:01.592 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.592 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.592 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.592 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:01.592 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.593 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2023-09-30T15:04:01.593 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-30T15:04:01.593 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-09-30T15:04:01.593 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-30T15:04:01.593 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-30T15:04:01.593 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-30T15:04:01.593 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-30T15:04:01.593 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-30T15:04:01.593 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 6251, 2023-09-30T15:04:01.593 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 576211973, 2023-09-30T15:04:01.593 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 6251, 2023-09-30T15:04:01.593 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 576211973, 2023-09-30T15:04:01.594 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-30T15:04:01.594 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.594 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.594 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.594 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:01.594 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.594 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2023-09-30T15:04:01.594 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-30T15:04:01.594 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2023-09-30T15:04:01.594 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-30T15:04:01.594 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-30T15:04:01.594 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-30T15:04:01.595 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-30T15:04:01.595 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-30T15:04:01.595 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 6251, 2023-09-30T15:04:01.595 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 6251, 2023-09-30T15:04:01.595 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 6251, 2023-09-30T15:04:01.595 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 6251, 2023-09-30T15:04:01.595 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-30T15:04:01.595 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.595 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.595 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.595 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:01.595 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.595 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2023-09-30T15:04:01.596 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-30T15:04:01.596 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-09-30T15:04:01.596 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-30T15:04:01.596 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-30T15:04:01.596 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-30T15:04:01.596 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-30T15:04:01.596 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-30T15:04:01.596 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-30T15:04:01.596 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-30T15:04:01.596 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-09-30T15:04:01.596 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-09-30T15:04:01.596 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-30T15:04:01.596 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.597 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.597 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.597 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:01.597 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:01.597 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x5627b1df4960": { 2023-09-30T15:04:01.597 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-30T15:04:01.597 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-09-30T15:04:01.597 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-30T15:04:01.597 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-30T15:04:01.597 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-30T15:04:01.598 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-30T15:04:01.598 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-30T15:04:01.598 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-30T15:04:01.598 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-30T15:04:01.598 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-09-30T15:04:01.598 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-09-30T15:04:01.598 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-30T15:04:01.598 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:01.598 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:01.598 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:01.598 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:01.598 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:01.599 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-09-30T15:04:01.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:01 smithi156 ceph-mon[103579]: pgmap v1059: 97 pgs: 97 active+clean; 590 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 6.6 MiB/s wr, 228 op/s 2023-09-30T15:04:01.985 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-09-30T15:04:01.985 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-09-30T15:04:01.985 DEBUG:tasks.check_counter:Getting stats from i 2023-09-30T15:04:01.986 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-09-30T15:04:02.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:01 smithi148 ceph-mon[102355]: pgmap v1059: 97 pgs: 97 active+clean; 590 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 6.6 MiB/s wr, 228 op/s 2023-09-30T15:04:02.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:01 smithi032 ceph-mon[98592]: pgmap v1059: 97 pgs: 97 active+clean; 590 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 6.6 MiB/s wr, 228 op/s 2023-09-30T15:04:02.668 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.c/config 2023-09-30T15:04:04.015 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:03 smithi156 ceph-mon[103579]: pgmap v1060: 97 pgs: 97 active+clean; 536 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 4.9 MiB/s wr, 243 op/s 2023-09-30T15:04:04.015 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:03 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T15:04:04.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:03 smithi148 ceph-mon[102355]: pgmap v1060: 97 pgs: 97 active+clean; 536 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 4.9 MiB/s wr, 243 op/s 2023-09-30T15:04:04.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:03 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T15:04:04.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:03 smithi032 ceph-mon[98592]: pgmap v1060: 97 pgs: 97 active+clean; 536 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 4.9 MiB/s wr, 243 op/s 2023-09-30T15:04:04.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:03 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-30T15:04:04.341 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-09-30T15:04:04.341 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-09-30T15:04:04.341 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 380090, 2023-09-30T15:04:04.341 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 451787, 2023-09-30T15:04:04.341 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 96653461, 2023-09-30T15:04:04.342 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 279242059, 2023-09-30T15:04:04.342 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 12, 2023-09-30T15:04:04.342 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 6, 2023-09-30T15:04:04.342 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 20.985865406, 2023-09-30T15:04:04.342 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 8.154537842, 2023-09-30T15:04:04.342 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 12.616742041, 2023-09-30T15:04:04.342 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 1.866393188, 2023-09-30T15:04:04.342 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:04:04.342 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 451785, 2023-09-30T15:04:04.342 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 23.523561210, 2023-09-30T15:04:04.342 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000052068 2023-09-30T15:04:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:04:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 714049, 2023-09-30T15:04:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.093670861, 2023-09-30T15:04:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000131 2023-09-30T15:04:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-30T15:04:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2023-09-30T15:04:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-09-30T15:04:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 129517, 2023-09-30T15:04:04.343 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 123790, 2023-09-30T15:04:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 31314964, 2023-09-30T15:04:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 164633358, 2023-09-30T15:04:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 9, 2023-09-30T15:04:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 5, 2023-09-30T15:04:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 7.711017614, 2023-09-30T15:04:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 2.583446852, 2023-09-30T15:04:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 3.568188446, 2023-09-30T15:04:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 1.923339385, 2023-09-30T15:04:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:04:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 123791, 2023-09-30T15:04:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 11.190733963, 2023-09-30T15:04:04.344 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000090400 2023-09-30T15:04:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:04:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 149912, 2023-09-30T15:04:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.022952750, 2023-09-30T15:04:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000153 2023-09-30T15:04:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-09-30T15:04:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 737984 2023-09-30T15:04:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-09-30T15:04:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 632773, 2023-09-30T15:04:04.345 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 995383, 2023-09-30T15:04:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 220318162, 2023-09-30T15:04:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1091277455, 2023-09-30T15:04:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 14, 2023-09-30T15:04:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 7, 2023-09-30T15:04:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 41.194139781, 2023-09-30T15:04:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 19.018055067, 2023-09-30T15:04:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 27.808938331, 2023-09-30T15:04:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 2.390622218, 2023-09-30T15:04:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:04:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 995380, 2023-09-30T15:04:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 85.010048968, 2023-09-30T15:04:04.346 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000085404 2023-09-30T15:04:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:04:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1314941, 2023-09-30T15:04:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.185294960, 2023-09-30T15:04:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000140 2023-09-30T15:04:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 197700576, 2023-09-30T15:04:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 783518416 2023-09-30T15:04:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-09-30T15:04:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2023-09-30T15:04:04.347 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-09-30T15:04:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2023-09-30T15:04:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-09-30T15:04:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-09-30T15:04:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 173039, 2023-09-30T15:04:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 194.579391833, 2023-09-30T15:04:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001124482 2023-09-30T15:04:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2023-09-30T15:04:04.348 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-09-30T15:04:04.349 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-09-30T15:04:04.349 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 109522, 2023-09-30T15:04:04.349 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 13.146382065, 2023-09-30T15:04:04.349 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000120034 2023-09-30T15:04:04.349 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:04.349 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.349 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2023-09-30T15:04:04.349 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 434889, 2023-09-30T15:04:04.349 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 414702, 2023-09-30T15:04:04.349 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2023-09-30T15:04:04.349 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 414702, 2023-09-30T15:04:04.350 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1676.733845065, 2023-09-30T15:04:04.350 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.004043225 2023-09-30T15:04:04.350 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.350 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2023-09-30T15:04:04.350 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 20187, 2023-09-30T15:04:04.350 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2023-09-30T15:04:04.350 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2023-09-30T15:04:04.350 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 241, 2023-09-30T15:04:04.350 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 16, 2023-09-30T15:04:04.350 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 20, 2023-09-30T15:04:04.350 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 2391, 2023-09-30T15:04:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 707, 2023-09-30T15:04:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 1129, 2023-09-30T15:04:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 555, 2023-09-30T15:04:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 1231, 2023-09-30T15:04:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 158, 2023-09-30T15:04:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2023-09-30T15:04:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2023-09-30T15:04:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 5, 2023-09-30T15:04:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 749897, 2023-09-30T15:04:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 434682, 2023-09-30T15:04:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 20186, 2023-09-30T15:04:04.351 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 29, 2023-09-30T15:04:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2023-09-30T15:04:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2023-09-30T15:04:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 8318, 2023-09-30T15:04:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 1000, 2023-09-30T15:04:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 1, 2023-09-30T15:04:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 6, 2023-09-30T15:04:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 58, 2023-09-30T15:04:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 9, 2023-09-30T15:04:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 595, 2023-09-30T15:04:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2023-09-30T15:04:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2023-09-30T15:04:04.352 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2023-09-30T15:04:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 4, 2023-09-30T15:04:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 246, 2023-09-30T15:04:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2023-09-30T15:04:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2023-09-30T15:04:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2023-09-30T15:04:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2023-09-30T15:04:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2023-09-30T15:04:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2023-09-30T15:04:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2023-09-30T15:04:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2023-09-30T15:04:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2023-09-30T15:04:04.353 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 12, 2023-09-30T15:04:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 18776, 2023-09-30T15:04:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 263437, 2023-09-30T15:04:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2023-09-30T15:04:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-30T15:04:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-30T15:04:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 95525, 2023-09-30T15:04:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 55172, 2023-09-30T15:04:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 16108, 2023-09-30T15:04:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 278996 2023-09-30T15:04:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2023-09-30T15:04:04.354 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 168, 2023-09-30T15:04:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 334, 2023-09-30T15:04:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 179, 2023-09-30T15:04:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 330, 2023-09-30T15:04:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 220, 2023-09-30T15:04:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 6, 2023-09-30T15:04:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2023-09-30T15:04:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2023-09-30T15:04:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 66315, 2023-09-30T15:04:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 66309, 2023-09-30T15:04:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2023-09-30T15:04:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2023-09-30T15:04:04.355 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2023-09-30T15:04:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2023-09-30T15:04:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2023-09-30T15:04:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2023-09-30T15:04:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2023-09-30T15:04:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2023-09-30T15:04:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 6, 2023-09-30T15:04:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2023-09-30T15:04:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 36, 2023-09-30T15:04:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2023-09-30T15:04:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2023-09-30T15:04:04.356 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2023-09-30T15:04:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "evlrg": 0, 2023-09-30T15:04:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 317032, 2023-09-30T15:04:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 210571, 2023-09-30T15:04:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 208931, 2023-09-30T15:04:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 108101, 2023-09-30T15:04:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2023-09-30T15:04:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 1640, 2023-09-30T15:04:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 391, 2023-09-30T15:04:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 263, 2023-09-30T15:04:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 261, 2023-09-30T15:04:04.357 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 130, 2023-09-30T15:04:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: "segmjr": 9, 2023-09-30T15:04:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2023-09-30T15:04:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 2, 2023-09-30T15:04:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 357246054, 2023-09-30T15:04:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 580456362, 2023-09-30T15:04:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 0, 2023-09-30T15:04:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2023-09-30T15:04:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 95897, 2023-09-30T15:04:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 755.941971749, 2023-09-30T15:04:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.007882853 2023-09-30T15:04:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.358 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 0 2023-09-30T15:04:04.359 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.359 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2023-09-30T15:04:04.359 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 47, 2023-09-30T15:04:04.359 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 66418, 2023-09-30T15:04:04.359 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 66371, 2023-09-30T15:04:04.359 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 89, 2023-09-30T15:04:04.359 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 400, 2023-09-30T15:04:04.359 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 311, 2023-09-30T15:04:04.359 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 2391, 2023-09-30T15:04:04.359 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 117068, 2023-09-30T15:04:04.359 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 114677, 2023-09-30T15:04:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2023-09-30T15:04:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 71286, 2023-09-30T15:04:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 71286, 2023-09-30T15:04:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 85024, 2023-09-30T15:04:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 198960 2023-09-30T15:04:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2023-09-30T15:04:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 724670, 2023-09-30T15:04:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 1860, 2023-09-30T15:04:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 434889, 2023-09-30T15:04:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 10594, 2023-09-30T15:04:04.360 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 65708, 2023-09-30T15:04:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2023-09-30T15:04:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 66017, 2023-09-30T15:04:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 237.728321902, 2023-09-30T15:04:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.003601016 2023-09-30T15:04:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2023-09-30T15:04:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 37303, 2023-09-30T15:04:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 311.765384404, 2023-09-30T15:04:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.008357649 2023-09-30T15:04:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2023-09-30T15:04:04.361 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.362 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.362 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.362 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.362 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2023-09-30T15:04:04.362 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.362 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.362 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.362 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.362 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2023-09-30T15:04:04.362 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 123458, 2023-09-30T15:04:04.362 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 174.559797362, 2023-09-30T15:04:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001413920 2023-09-30T15:04:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2023-09-30T15:04:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2023-09-30T15:04:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.363 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2023-09-30T15:04:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2023-09-30T15:04:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2023-09-30T15:04:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.364 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2023-09-30T15:04:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2023-09-30T15:04:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.365 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2023-09-30T15:04:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2023-09-30T15:04:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2023-09-30T15:04:04.366 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2023-09-30T15:04:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 104086, 2023-09-30T15:04:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 727.099267386, 2023-09-30T15:04:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.006985562 2023-09-30T15:04:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2023-09-30T15:04:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 14802, 2023-09-30T15:04:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 56.595887841, 2023-09-30T15:04:04.367 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.003823529 2023-09-30T15:04:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2023-09-30T15:04:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2023-09-30T15:04:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.368 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2023-09-30T15:04:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 30, 2023-09-30T15:04:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.306084990, 2023-09-30T15:04:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.010202833 2023-09-30T15:04:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2023-09-30T15:04:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2023-09-30T15:04:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.369 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2023-09-30T15:04:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 13, 2023-09-30T15:04:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.286662305, 2023-09-30T15:04:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.022050946 2023-09-30T15:04:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2023-09-30T15:04:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.370 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2023-09-30T15:04:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2708, 2023-09-30T15:04:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 2.924784038, 2023-09-30T15:04:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001080053 2023-09-30T15:04:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2023-09-30T15:04:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2023-09-30T15:04:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.371 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2023-09-30T15:04:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2023-09-30T15:04:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 66285, 2023-09-30T15:04:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 165.467654837, 2023-09-30T15:04:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002496306 2023-09-30T15:04:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.372 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2023-09-30T15:04:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2023-09-30T15:04:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2023-09-30T15:04:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2023-09-30T15:04:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 1, 2023-09-30T15:04:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 1, 2023-09-30T15:04:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 0, 2023-09-30T15:04:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 1, 2023-09-30T15:04:04.373 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2023-09-30T15:04:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 33058, 2023-09-30T15:04:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 33058, 2023-09-30T15:04:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 1472, 2023-09-30T15:04:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: "mdthresh_evicted": 0 2023-09-30T15:04:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-09-30T15:04:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-09-30T15:04:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-09-30T15:04:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-09-30T15:04:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-09-30T15:04:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-09-30T15:04:04.374 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-09-30T15:04:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-30T15:04:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-09-30T15:04:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-30T15:04:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-09-30T15:04:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-09-30T15:04:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-09-30T15:04:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-30T15:04:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-09-30T15:04:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-09-30T15:04:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-09-30T15:04:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-09-30T15:04:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-09-30T15:04:04.375 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-30T15:04:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-09-30T15:04:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-30T15:04:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-09-30T15:04:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-09-30T15:04:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-09-30T15:04:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-09-30T15:04:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-09-30T15:04:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-30T15:04:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-09-30T15:04:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-09-30T15:04:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-09-30T15:04:04.376 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-09-30T15:04:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-09-30T15:04:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-09-30T15:04:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-09-30T15:04:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-09-30T15:04:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-09-30T15:04:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 436855, 2023-09-30T15:04:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 177, 2023-09-30T15:04:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 352, 2023-09-30T15:04:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 4, 2023-09-30T15:04:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-09-30T15:04:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-09-30T15:04:04.377 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-09-30T15:04:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-09-30T15:04:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-09-30T15:04:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-09-30T15:04:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 7528, 2023-09-30T15:04:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 158, 2023-09-30T15:04:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-09-30T15:04:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-09-30T15:04:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-09-30T15:04:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-09-30T15:04:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 2011103, 2023-09-30T15:04:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 27258, 2023-09-30T15:04:04.378 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-09-30T15:04:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-09-30T15:04:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-09-30T15:04:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-09-30T15:04:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2023-09-30T15:04:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2023-09-30T15:04:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2023-09-30T15:04:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 168222, 2023-09-30T15:04:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 589387932, 2023-09-30T15:04:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 28, 2023-09-30T15:04:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 168222, 2023-09-30T15:04:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2023-09-30T15:04:04.379 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 168222, 2023-09-30T15:04:04.380 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1415.206120310, 2023-09-30T15:04:04.380 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.008412729 2023-09-30T15:04:04.380 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.380 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2023-09-30T15:04:04.380 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2023-09-30T15:04:04.380 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 168222, 2023-09-30T15:04:04.380 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 170017 2023-09-30T15:04:04.380 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.381 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 168222, 2023-09-30T15:04:04.381 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 1878, 2023-09-30T15:04:04.381 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 166344, 2023-09-30T15:04:04.381 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2023-09-30T15:04:04.381 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2023-09-30T15:04:04.381 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 57, 2023-09-30T15:04:04.381 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 418, 2023-09-30T15:04:04.381 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 1878, 2023-09-30T15:04:04.381 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 97775, 2023-09-30T15:04:04.381 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 969, 2023-09-30T15:04:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2023-09-30T15:04:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2023-09-30T15:04:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2023-09-30T15:04:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 16, 2023-09-30T15:04:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 66628, 2023-09-30T15:04:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2023-09-30T15:04:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2023-09-30T15:04:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2023-09-30T15:04:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 0, 2023-09-30T15:04:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 836, 2023-09-30T15:04:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2023-09-30T15:04:04.382 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2023-09-30T15:04:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2023-09-30T15:04:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2023-09-30T15:04:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2023-09-30T15:04:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2023-09-30T15:04:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2023-09-30T15:04:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2023-09-30T15:04:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2023-09-30T15:04:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2023-09-30T15:04:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2023-09-30T15:04:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2023-09-30T15:04:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2023-09-30T15:04:04.383 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2023-09-30T15:04:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2023-09-30T15:04:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2023-09-30T15:04:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2023-09-30T15:04:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2023-09-30T15:04:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2023-09-30T15:04:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2023-09-30T15:04:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2023-09-30T15:04:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2023-09-30T15:04:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2023-09-30T15:04:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2023-09-30T15:04:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2023-09-30T15:04:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2023-09-30T15:04:04.384 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 85, 2023-09-30T15:04:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2023-09-30T15:04:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 2, 2023-09-30T15:04:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 12, 2023-09-30T15:04:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 12, 2023-09-30T15:04:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2023-09-30T15:04:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2023-09-30T15:04:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 1056, 2023-09-30T15:04:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 0, 2023-09-30T15:04:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 384 2023-09-30T15:04:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2023-09-30T15:04:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 1, 2023-09-30T15:04:04.385 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2023-09-30T15:04:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2023-09-30T15:04:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2023-09-30T15:04:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2023-09-30T15:04:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2023-09-30T15:04:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2023-09-30T15:04:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2023-09-30T15:04:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 10, 2023-09-30T15:04:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2023-09-30T15:04:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 10, 2023-09-30T15:04:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 66309, 2023-09-30T15:04:04.386 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2023-09-30T15:04:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-30T15:04:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-30T15:04:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-09-30T15:04:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-30T15:04:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 1142376, 2023-09-30T15:04:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 219378265, 2023-09-30T15:04:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-30T15:04:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 1142376, 2023-09-30T15:04:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-30T15:04:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-30T15:04:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 1142376, 2023-09-30T15:04:04.387 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 219378265, 2023-09-30T15:04:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-30T15:04:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2023-09-30T15:04:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-30T15:04:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-09-30T15:04:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-30T15:04:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-30T15:04:04.388 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-30T15:04:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-30T15:04:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-30T15:04:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 168222, 2023-09-30T15:04:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 596085141, 2023-09-30T15:04:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 168222, 2023-09-30T15:04:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 596085141, 2023-09-30T15:04:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-30T15:04:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.389 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2023-09-30T15:04:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-30T15:04:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2023-09-30T15:04:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-30T15:04:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-09-30T15:04:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-09-30T15:04:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-30T15:04:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-09-30T15:04:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 168222, 2023-09-30T15:04:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 168222, 2023-09-30T15:04:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 168222, 2023-09-30T15:04:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 168222, 2023-09-30T15:04:04.390 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-30T15:04:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2023-09-30T15:04:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-30T15:04:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-09-30T15:04:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-30T15:04:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 66309, 2023-09-30T15:04:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 6697209, 2023-09-30T15:04:04.391 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-30T15:04:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 66309, 2023-09-30T15:04:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-30T15:04:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-30T15:04:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 1878, 2023-09-30T15:04:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 6697209, 2023-09-30T15:04:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-30T15:04:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x561fde1d06e0": { 2023-09-30T15:04:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 189, 2023-09-30T15:04:04.392 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-09-30T15:04:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-30T15:04:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 317032, 2023-09-30T15:04:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 576262058, 2023-09-30T15:04:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-30T15:04:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 317032, 2023-09-30T15:04:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-30T15:04:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-30T15:04:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 95897, 2023-09-30T15:04:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 576261869, 2023-09-30T15:04:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-30T15:04:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:04.393 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:04.394 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:04.394 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:04.394 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:04.394 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-09-30T15:04:04.737 INFO:tasks.check_counter:Daemon mds.i mds.exported=6 2023-09-30T15:04:04.737 INFO:tasks.check_counter:Daemon mds.i mds.imported=9 2023-09-30T15:04:04.737 DEBUG:tasks.check_counter:Getting stats from l 2023-09-30T15:04:04.738 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1d10b71792f3be8887a7631e69851ac2df3585af shell --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-09-30T15:04:05.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T15:04:05.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T15:04:05.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:04 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:04:05.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:04 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T15:04:05.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:04 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T15:04:05.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:04 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:04:05.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-30T15:04:05.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-30T15:04:05.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:04 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' 2023-09-30T15:04:05.351 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/mon.c/config 2023-09-30T15:04:05.805 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:05 smithi156 ceph-mon[103579]: pgmap v1061: 97 pgs: 97 active+clean; 483 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 4.3 MiB/s wr, 277 op/s 2023-09-30T15:04:06.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:05 smithi148 ceph-mon[102355]: pgmap v1061: 97 pgs: 97 active+clean; 483 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 4.3 MiB/s wr, 277 op/s 2023-09-30T15:04:06.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:05 smithi032 ceph-mon[98592]: pgmap v1061: 97 pgs: 97 active+clean; 483 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 4.3 MiB/s wr, 277 op/s 2023-09-30T15:04:06.580 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-09-30T15:04:06.580 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-09-30T15:04:06.580 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 0, 2023-09-30T15:04:06.580 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2, 2023-09-30T15:04:06.580 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 0, 2023-09-30T15:04:06.580 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 0, 2023-09-30T15:04:06.580 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 4, 2023-09-30T15:04:06.580 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 0, 2023-09-30T15:04:06.580 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.001179907, 2023-09-30T15:04:06.580 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.000000000, 2023-09-30T15:04:06.580 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.000282980, 2023-09-30T15:04:06.581 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-30T15:04:06.581 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:04:06.581 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:06.581 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:06.581 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:06.581 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:06.581 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:04:06.581 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:06.581 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:06.581 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:06.581 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:06.581 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-30T15:04:06.582 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2023-09-30T15:04:06.582 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:06.582 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-09-30T15:04:06.582 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 6, 2023-09-30T15:04:06.582 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 336, 2023-09-30T15:04:06.582 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 3264, 2023-09-30T15:04:06.582 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 212416, 2023-09-30T15:04:06.582 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 4, 2023-09-30T15:04:06.582 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 2, 2023-09-30T15:04:06.582 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.014703336, 2023-09-30T15:04:06.582 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.010188626, 2023-09-30T15:04:06.582 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.001915741, 2023-09-30T15:04:06.583 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-30T15:04:06.583 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:04:06.583 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 335, 2023-09-30T15:04:06.583 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.022894965, 2023-09-30T15:04:06.583 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000068343 2023-09-30T15:04:06.583 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:06.583 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:04:06.583 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:06.583 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:06.583 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:06.583 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:06.583 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-09-30T15:04:06.583 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 212416 2023-09-30T15:04:06.584 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:06.584 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-09-30T15:04:06.584 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 440, 2023-09-30T15:04:06.584 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 437, 2023-09-30T15:04:06.584 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 535488, 2023-09-30T15:04:06.584 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 263728, 2023-09-30T15:04:06.584 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 2, 2023-09-30T15:04:06.584 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 1, 2023-09-30T15:04:06.584 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.051826831, 2023-09-30T15:04:06.584 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.019846353, 2023-09-30T15:04:06.584 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.019313820, 2023-09-30T15:04:06.584 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.010237476, 2023-09-30T15:04:06.585 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-09-30T15:04:06.585 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 436, 2023-09-30T15:04:06.585 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.019087706, 2023-09-30T15:04:06.585 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000043779 2023-09-30T15:04:06.585 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:06.585 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-09-30T15:04:06.585 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:06.585 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:06.585 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:06.585 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:06.585 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 535488, 2023-09-30T15:04:06.585 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 263728 2023-09-30T15:04:06.585 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:06.586 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-09-30T15:04:06.586 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 0, 2023-09-30T15:04:06.586 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-09-30T15:04:06.586 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:06.586 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-09-30T15:04:06.586 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-09-30T15:04:06.586 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-09-30T15:04:06.586 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-09-30T15:04:06.586 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-09-30T15:04:06.586 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-09-30T15:04:06.587 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-09-30T15:04:06.587 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-30T15:04:06.587 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-09-30T15:04:06.587 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-30T15:04:06.587 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-09-30T15:04:06.587 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-09-30T15:04:06.587 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-09-30T15:04:06.587 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-30T15:04:06.587 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-09-30T15:04:06.587 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-09-30T15:04:06.587 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-09-30T15:04:06.587 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-09-30T15:04:06.588 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-09-30T15:04:06.588 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-30T15:04:06.588 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-09-30T15:04:06.588 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-30T15:04:06.588 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-09-30T15:04:06.588 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-09-30T15:04:06.588 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-09-30T15:04:06.588 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-09-30T15:04:06.588 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-09-30T15:04:06.588 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-30T15:04:06.588 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-09-30T15:04:06.588 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-09-30T15:04:06.588 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-09-30T15:04:06.589 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-09-30T15:04:06.589 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-09-30T15:04:06.589 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-09-30T15:04:06.589 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-09-30T15:04:06.589 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-09-30T15:04:06.589 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-09-30T15:04:06.589 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 148744, 2023-09-30T15:04:06.589 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 30, 2023-09-30T15:04:06.589 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-09-30T15:04:06.589 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-09-30T15:04:06.589 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-09-30T15:04:06.589 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-09-30T15:04:06.590 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-09-30T15:04:06.590 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-09-30T15:04:06.590 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-09-30T15:04:06.590 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-09-30T15:04:06.590 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 0, 2023-09-30T15:04:06.590 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 0, 2023-09-30T15:04:06.590 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-09-30T15:04:06.590 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-09-30T15:04:06.590 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-09-30T15:04:06.590 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-09-30T15:04:06.590 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 928, 2023-09-30T15:04:06.590 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 2, 2023-09-30T15:04:06.590 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-09-30T15:04:06.591 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-09-30T15:04:06.591 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-09-30T15:04:06.591 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-09-30T15:04:06.591 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-09-30T15:04:06.591 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-30T15:04:06.591 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-09-30T15:04:06.591 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-09-30T15:04:06.591 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-09-30T15:04:06.591 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 442, 2023-09-30T15:04:06.591 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 473266, 2023-09-30T15:04:06.591 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-09-30T15:04:06.591 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 442, 2023-09-30T15:04:06.592 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-09-30T15:04:06.592 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-09-30T15:04:06.592 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 442, 2023-09-30T15:04:06.592 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 473266, 2023-09-30T15:04:06.592 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-09-30T15:04:06.592 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-09-30T15:04:06.592 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-09-30T15:04:06.592 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-09-30T15:04:06.592 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:06.592 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-09-30T15:04:06.592 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-09-30T15:04:07.002 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-09-30T15:04:07.003 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-09-30T15:04:07.003 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-09-30T15:04:07.014 INFO:tasks.kclient:Unmounting kernel clients... 2023-09-30T15:04:07.015 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T15:04:07.015 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-30T15:04:07.037 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T15:04:07.037 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-30T15:04:07.098 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T15:04:07.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.281+0000 7f13066ef700 1 -- 172.21.15.32:0/4013886275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1300104280 msgr2=0x7f1300106670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:07.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.281+0000 7f13066ef700 1 --2- 172.21.15.32:0/4013886275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1300104280 0x7f1300106670 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009990 tx=0x7f12e8009ca0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.281+0000 7f13066ef700 1 -- 172.21.15.32:0/4013886275 shutdown_connections 2023-09-30T15:04:07.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.281+0000 7f13066ef700 1 --2- 172.21.15.32:0/4013886275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1300106bb0 0x7f1300108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.281+0000 7f13066ef700 1 --2- 172.21.15.32:0/4013886275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1300101950 0x7f1300103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.281+0000 7f13066ef700 1 --2- 172.21.15.32:0/4013886275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1300104280 0x7f1300106670 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.281+0000 7f13066ef700 1 -- 172.21.15.32:0/4013886275 >> 172.21.15.32:0/4013886275 conn(0x7f13000fb2e0 msgr2=0x7f13000fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:07.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.281+0000 7f13066ef700 1 -- 172.21.15.32:0/4013886275 shutdown_connections 2023-09-30T15:04:07.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.281+0000 7f13066ef700 1 -- 172.21.15.32:0/4013886275 wait complete. 2023-09-30T15:04:07.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.282+0000 7f13066ef700 1 Processor -- start 2023-09-30T15:04:07.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.282+0000 7f13066ef700 1 -- start start 2023-09-30T15:04:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.282+0000 7f13066ef700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1300101950 0x7f1300196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.282+0000 7f13066ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1300104280 0x7f1300196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.282+0000 7f13066ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1300106bb0 0x7f130019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.282+0000 7f13066ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130019e5f0 con 0x7f1300106bb0 2023-09-30T15:04:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.282+0000 7f13066ef700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130019e730 con 0x7f1300101950 2023-09-30T15:04:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.282+0000 7f13066ef700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130019e8c0 con 0x7f1300104280 2023-09-30T15:04:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f1304c8c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1300106bb0 0x7f130019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f1304c8c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1300106bb0 0x7f130019e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40602/0 (socket says 172.21.15.32:40602) 2023-09-30T15:04:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f1304c8c700 1 -- 172.21.15.32:0/2117229020 learned_addr learned my addr 172.21.15.32:0/2117229020 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:04:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f12fffff700 1 --2- 172.21.15.32:0/2117229020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1300101950 0x7f1300196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:07.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f12ff7fe700 1 --2- 172.21.15.32:0/2117229020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1300104280 0x7f1300196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f1304c8c700 1 -- 172.21.15.32:0/2117229020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1300104280 msgr2=0x7f1300196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f1304c8c700 1 --2- 172.21.15.32:0/2117229020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1300104280 0x7f1300196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f1304c8c700 1 -- 172.21.15.32:0/2117229020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1300101950 msgr2=0x7f1300196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f1304c8c700 1 --2- 172.21.15.32:0/2117229020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1300101950 0x7f1300196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f1304c8c700 1 -- 172.21.15.32:0/2117229020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e80095f0 con 0x7f1300106bb0 2023-09-30T15:04:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f12ff7fe700 1 --2- 172.21.15.32:0/2117229020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1300104280 0x7f1300196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T15:04:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f1304c8c700 1 --2- 172.21.15.32:0/2117229020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1300106bb0 0x7f130019e020 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f12f8007ee0 tx=0x7f12f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f12fd7fa700 1 -- 172.21.15.32:0/2117229020 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8017070 con 0x7f1300106bb0 2023-09-30T15:04:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f12fd7fa700 1 -- 172.21.15.32:0/2117229020 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12f800f040 con 0x7f1300106bb0 2023-09-30T15:04:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f12fd7fa700 1 -- 172.21.15.32:0/2117229020 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8005740 con 0x7f1300106bb0 2023-09-30T15:04:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f13066ef700 1 -- 172.21.15.32:0/2117229020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130019eba0 con 0x7f1300106bb0 2023-09-30T15:04:07.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.283+0000 7f13066ef700 1 -- 172.21.15.32:0/2117229020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130019f060 con 0x7f1300106bb0 2023-09-30T15:04:07.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.284+0000 7f13066ef700 1 -- 172.21.15.32:0/2117229020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13000fce60 con 0x7f1300106bb0 2023-09-30T15:04:07.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.285+0000 7f12fd7fa700 1 -- 172.21.15.32:0/2117229020 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f12f800dc60 con 0x7f1300106bb0 2023-09-30T15:04:07.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.286+0000 7f12fd7fa700 1 --2- 172.21.15.32:0/2117229020 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f12ec0a0ff0 0x7f12ec0a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:07.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.286+0000 7f12fd7fa700 1 -- 172.21.15.32:0/2117229020 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f12f80cbf10 con 0x7f1300106bb0 2023-09-30T15:04:07.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.286+0000 7f12fffff700 1 --2- 172.21.15.32:0/2117229020 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f12ec0a0ff0 0x7f12ec0a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:07.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.286+0000 7f12fffff700 1 --2- 172.21.15.32:0/2117229020 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f12ec0a0ff0 0x7f12ec0a34b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009c60 tx=0x7f12f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:07.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.291+0000 7f12fd7fa700 1 -- 172.21.15.32:0/2117229020 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f12f8095720 con 0x7f1300106bb0 2023-09-30T15:04:07.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.495+0000 7f13066ef700 1 -- 172.21.15.32:0/2117229020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f130010c660 con 0x7f1300106bb0 2023-09-30T15:04:07.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.497+0000 7f12fd7fa700 1 -- 172.21.15.32:0/2117229020 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22569 (secure 0 0 0) 0x7f12f8098d40 con 0x7f1300106bb0 2023-09-30T15:04:07.497 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T15:04:07.498 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":85,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:38:13.192089+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T15:04:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.499+0000 7f13066ef700 1 -- 172.21.15.32:0/2117229020 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f12ec0a0ff0 msgr2=0x7f12ec0a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.499+0000 7f13066ef700 1 --2- 172.21.15.32:0/2117229020 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f12ec0a0ff0 0x7f12ec0a34b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009c60 tx=0x7f12f0009260 comp rx=0 tx=0).stop 2023-09-30T15:04:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.499+0000 7f13066ef700 1 -- 172.21.15.32:0/2117229020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1300106bb0 msgr2=0x7f130019e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.499+0000 7f13066ef700 1 --2- 172.21.15.32:0/2117229020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1300106bb0 0x7f130019e020 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f12f8007ee0 tx=0x7f12f800d3b0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.499+0000 7f13066ef700 1 -- 172.21.15.32:0/2117229020 shutdown_connections 2023-09-30T15:04:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.499+0000 7f13066ef700 1 --2- 172.21.15.32:0/2117229020 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f12ec0a0ff0 0x7f12ec0a34b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.499+0000 7f13066ef700 1 --2- 172.21.15.32:0/2117229020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1300106bb0 0x7f130019e020 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.499+0000 7f13066ef700 1 --2- 172.21.15.32:0/2117229020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1300101950 0x7f1300196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.499+0000 7f13066ef700 1 --2- 172.21.15.32:0/2117229020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1300104280 0x7f1300196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.499+0000 7f13066ef700 1 -- 172.21.15.32:0/2117229020 >> 172.21.15.32:0/2117229020 conn(0x7f13000fb2e0 msgr2=0x7f13001025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:07.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.499+0000 7f13066ef700 1 -- 172.21.15.32:0/2117229020 shutdown_connections 2023-09-30T15:04:07.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.499+0000 7f13066ef700 1 -- 172.21.15.32:0/2117229020 wait complete. 2023-09-30T15:04:07.512 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T14:33:20.224529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T14:37:09.573219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T14:37:10.844458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T15:04:07.513 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T15:04:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.642+0000 7fd58bcd5700 1 -- 172.21.15.32:0/2379570163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd584101fb0 msgr2=0x7fd584102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.642+0000 7fd58bcd5700 1 --2- 172.21.15.32:0/2379570163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd584101fb0 0x7fd584102430 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd574009990 tx=0x7fd574009ca0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.642+0000 7fd58bcd5700 1 -- 172.21.15.32:0/2379570163 shutdown_connections 2023-09-30T15:04:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.642+0000 7fd58bcd5700 1 --2- 172.21.15.32:0/2379570163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd584102970 0x7fd584109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.642+0000 7fd58bcd5700 1 --2- 172.21.15.32:0/2379570163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd584100e60 0x7fd584101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.642+0000 7fd58bcd5700 1 --2- 172.21.15.32:0/2379570163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd584101fb0 0x7fd584102430 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.642+0000 7fd58bcd5700 1 -- 172.21.15.32:0/2379570163 >> 172.21.15.32:0/2379570163 conn(0x7fd5840fc3d0 msgr2=0x7fd5840fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.642+0000 7fd58bcd5700 1 -- 172.21.15.32:0/2379570163 shutdown_connections 2023-09-30T15:04:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.642+0000 7fd58bcd5700 1 -- 172.21.15.32:0/2379570163 wait complete. 2023-09-30T15:04:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.642+0000 7fd58bcd5700 1 Processor -- start 2023-09-30T15:04:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.642+0000 7fd58bcd5700 1 -- start start 2023-09-30T15:04:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58bcd5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd584100e60 0x7fd584196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58bcd5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd584101fb0 0x7fd5841970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58bcd5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd584102970 0x7fd58419e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58bcd5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58419e720 con 0x7fd584102970 2023-09-30T15:04:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58bcd5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58419e860 con 0x7fd584100e60 2023-09-30T15:04:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58bcd5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58419e9f0 con 0x7fd584101fb0 2023-09-30T15:04:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58a272700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd584102970 0x7fd58419e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58a272700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd584102970 0x7fd58419e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40616/0 (socket says 172.21.15.32:40616) 2023-09-30T15:04:07.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58a272700 1 -- 172.21.15.32:0/2406960291 learned_addr learned my addr 172.21.15.32:0/2406960291 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:04:07.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd589a71700 1 --2- 172.21.15.32:0/2406960291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd584100e60 0x7fd584196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:07.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd589270700 1 --2- 172.21.15.32:0/2406960291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd584101fb0 0x7fd5841970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:07.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58a272700 1 -- 172.21.15.32:0/2406960291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd584101fb0 msgr2=0x7fd5841970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:07.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58a272700 1 --2- 172.21.15.32:0/2406960291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd584101fb0 0x7fd5841970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58a272700 1 -- 172.21.15.32:0/2406960291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd584100e60 msgr2=0x7fd584196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:07.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58a272700 1 --2- 172.21.15.32:0/2406960291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd584100e60 0x7fd584196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58a272700 1 -- 172.21.15.32:0/2406960291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5740095f0 con 0x7fd584102970 2023-09-30T15:04:07.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58a272700 1 --2- 172.21.15.32:0/2406960291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd584102970 0x7fd58419e150 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00ef30 tx=0x7fd57c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:07.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd57affd700 1 -- 172.21.15.32:0/2406960291 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c018070 con 0x7fd584102970 2023-09-30T15:04:07.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd57affd700 1 -- 172.21.15.32:0/2406960291 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd57c00f040 con 0x7fd584102970 2023-09-30T15:04:07.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58bcd5700 1 -- 172.21.15.32:0/2406960291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58419ec70 con 0x7fd584102970 2023-09-30T15:04:07.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd57affd700 1 -- 172.21.15.32:0/2406960291 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c00cd00 con 0x7fd584102970 2023-09-30T15:04:07.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.643+0000 7fd58bcd5700 1 -- 172.21.15.32:0/2406960291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58419f180 con 0x7fd584102970 2023-09-30T15:04:07.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.644+0000 7fd58bcd5700 1 -- 172.21.15.32:0/2406960291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd584108440 con 0x7fd584102970 2023-09-30T15:04:07.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.645+0000 7fd57affd700 1 -- 172.21.15.32:0/2406960291 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fd57c003a00 con 0x7fd584102970 2023-09-30T15:04:07.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.646+0000 7fd57affd700 1 --2- 172.21.15.32:0/2406960291 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd5700a1020 0x7fd5700a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:07.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.646+0000 7fd57affd700 1 -- 172.21.15.32:0/2406960291 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd57c0cc160 con 0x7fd584102970 2023-09-30T15:04:07.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.646+0000 7fd589a71700 1 --2- 172.21.15.32:0/2406960291 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd5700a1020 0x7fd5700a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:07.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.646+0000 7fd589a71700 1 --2- 172.21.15.32:0/2406960291 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd5700a1020 0x7fd5700a34e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd580009c60 tx=0x7fd580009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:07.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.651+0000 7fd57affd700 1 -- 172.21.15.32:0/2406960291 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fd57c095a00 con 0x7fd584102970 2023-09-30T15:04:07.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.855+0000 7fd58bcd5700 1 -- 172.21.15.32:0/2406960291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd58402d170 con 0x7fd584102970 2023-09-30T15:04:07.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.855+0000 7fd57affd700 1 -- 172.21.15.32:0/2406960291 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd57c099020 con 0x7fd584102970 2023-09-30T15:04:07.856 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T15:04:07.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.857+0000 7fd58bcd5700 1 -- 172.21.15.32:0/2406960291 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd5700a1020 msgr2=0x7fd5700a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:07.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.857+0000 7fd58bcd5700 1 --2- 172.21.15.32:0/2406960291 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd5700a1020 0x7fd5700a34e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd580009c60 tx=0x7fd580009260 comp rx=0 tx=0).stop 2023-09-30T15:04:07.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.857+0000 7fd58bcd5700 1 -- 172.21.15.32:0/2406960291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd584102970 msgr2=0x7fd58419e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:07.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.857+0000 7fd58bcd5700 1 --2- 172.21.15.32:0/2406960291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd584102970 0x7fd58419e150 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00ef30 tx=0x7fd57c00c3a0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.858+0000 7fd58bcd5700 1 -- 172.21.15.32:0/2406960291 shutdown_connections 2023-09-30T15:04:07.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.858+0000 7fd58bcd5700 1 --2- 172.21.15.32:0/2406960291 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fd5700a1020 0x7fd5700a34e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.858+0000 7fd58bcd5700 1 --2- 172.21.15.32:0/2406960291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd584102970 0x7fd58419e150 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.858+0000 7fd58bcd5700 1 --2- 172.21.15.32:0/2406960291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd584100e60 0x7fd584196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.858+0000 7fd58bcd5700 1 --2- 172.21.15.32:0/2406960291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd584101fb0 0x7fd5841970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:07.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.858+0000 7fd58bcd5700 1 -- 172.21.15.32:0/2406960291 >> 172.21.15.32:0/2406960291 conn(0x7fd5840fc3d0 msgr2=0x7fd584105580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:07.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.858+0000 7fd58bcd5700 1 -- 172.21.15.32:0/2406960291 shutdown_connections 2023-09-30T15:04:07.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.858+0000 7fd58bcd5700 1 -- 172.21.15.32:0/2406960291 wait complete. 2023-09-30T15:04:07.869 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:07 smithi032 ceph-mon[98592]: pgmap v1062: 97 pgs: 97 active+clean; 474 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 3.7 MiB/s wr, 263 op/s 2023-09-30T15:04:07.869 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:07 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2117229020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T15:04:07.870 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T15:04:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.999+0000 7f0834f08700 1 -- 172.21.15.32:0/2061093945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08300ff4b0 msgr2=0x7f08300ff8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.999+0000 7f0834f08700 1 --2- 172.21.15.32:0/2061093945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08300ff4b0 0x7f08300ff8d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0820009990 tx=0x7f0820009ca0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.999+0000 7f0834f08700 1 -- 172.21.15.32:0/2061093945 shutdown_connections 2023-09-30T15:04:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.999+0000 7f0834f08700 1 --2- 172.21.15.32:0/2061093945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0830101040 0x7f08301078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.999+0000 7f0834f08700 1 --2- 172.21.15.32:0/2061093945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08300ff4b0 0x7f08300ff8d0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.999+0000 7f0834f08700 1 --2- 172.21.15.32:0/2061093945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08301006a0 0x7f0830100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.999+0000 7f0834f08700 1 -- 172.21.15.32:0/2061093945 >> 172.21.15.32:0/2061093945 conn(0x7f08300faa20 msgr2=0x7f08300fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.999+0000 7f0834f08700 1 -- 172.21.15.32:0/2061093945 shutdown_connections 2023-09-30T15:04:08.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:07.999+0000 7f0834f08700 1 -- 172.21.15.32:0/2061093945 wait complete. 2023-09-30T15:04:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.000+0000 7f0834f08700 1 Processor -- start 2023-09-30T15:04:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.000+0000 7f0834f08700 1 -- start start 2023-09-30T15:04:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.000+0000 7f0834f08700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08300ff4b0 0x7f0830195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.000+0000 7f0834f08700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08301006a0 0x7f0830196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.000+0000 7f0834f08700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0830101040 0x7f083019d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.000+0000 7f0834f08700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083019d640 con 0x7f08301006a0 2023-09-30T15:04:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.000+0000 7f0834f08700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083019d780 con 0x7f08300ff4b0 2023-09-30T15:04:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.000+0000 7f0834f08700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083019d910 con 0x7f0830101040 2023-09-30T15:04:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f082dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08301006a0 0x7f0830196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f082dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08301006a0 0x7f0830196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40628/0 (socket says 172.21.15.32:40628) 2023-09-30T15:04:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f082dd9b700 1 -- 172.21.15.32:0/255853628 learned_addr learned my addr 172.21.15.32:0/255853628 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:04:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f082e59c700 1 --2- 172.21.15.32:0/255853628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08300ff4b0 0x7f0830195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f082ed9d700 1 --2- 172.21.15.32:0/255853628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0830101040 0x7f083019d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f082dd9b700 1 -- 172.21.15.32:0/255853628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0830101040 msgr2=0x7f083019d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f082dd9b700 1 --2- 172.21.15.32:0/255853628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0830101040 0x7f083019d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f082dd9b700 1 -- 172.21.15.32:0/255853628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08300ff4b0 msgr2=0x7f0830195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f082dd9b700 1 --2- 172.21.15.32:0/255853628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08300ff4b0 0x7f0830195ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f082dd9b700 1 -- 172.21.15.32:0/255853628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08200095f0 con 0x7f08301006a0 2023-09-30T15:04:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f082ed9d700 1 --2- 172.21.15.32:0/255853628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0830101040 0x7f083019d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T15:04:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f082dd9b700 1 --2- 172.21.15.32:0/255853628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08301006a0 0x7f0830196000 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f082400d950 tx=0x7f082400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f081f7fe700 1 -- 172.21.15.32:0/255853628 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0824009940 con 0x7f08301006a0 2023-09-30T15:04:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f0834f08700 1 -- 172.21.15.32:0/255853628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f083019dbf0 con 0x7f08301006a0 2023-09-30T15:04:08.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f081f7fe700 1 -- 172.21.15.32:0/255853628 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0824010460 con 0x7f08301006a0 2023-09-30T15:04:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f081f7fe700 1 -- 172.21.15.32:0/255853628 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082400f620 con 0x7f08301006a0 2023-09-30T15:04:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.001+0000 7f0834f08700 1 -- 172.21.15.32:0/255853628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083019e150 con 0x7f08301006a0 2023-09-30T15:04:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.002+0000 7f0834f08700 1 -- 172.21.15.32:0/255853628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0830106460 con 0x7f08301006a0 2023-09-30T15:04:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.003+0000 7f081f7fe700 1 -- 172.21.15.32:0/255853628 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f08240108d0 con 0x7f08301006a0 2023-09-30T15:04:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.004+0000 7f081f7fe700 1 --2- 172.21.15.32:0/255853628 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f08180a0fa0 0x7f08180a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:08.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.004+0000 7f082e59c700 1 --2- 172.21.15.32:0/255853628 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f08180a0fa0 0x7f08180a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:08.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.004+0000 7f081f7fe700 1 -- 172.21.15.32:0/255853628 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f08240cc960 con 0x7f08301006a0 2023-09-30T15:04:08.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.005+0000 7f082e59c700 1 --2- 172.21.15.32:0/255853628 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f08180a0fa0 0x7f08180a3460 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f082000b2e0 tx=0x7f0820005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:08.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.008+0000 7f081f7fe700 1 -- 172.21.15.32:0/255853628 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f0824096110 con 0x7f08301006a0 2023-09-30T15:04:08.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:07 smithi156 ceph-mon[103579]: pgmap v1062: 97 pgs: 97 active+clean; 474 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 3.7 MiB/s wr, 263 op/s 2023-09-30T15:04:08.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:07 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2117229020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T15:04:08.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:07 smithi148 ceph-mon[102355]: pgmap v1062: 97 pgs: 97 active+clean; 474 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 3.7 MiB/s wr, 263 op/s 2023-09-30T15:04:08.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:07 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2117229020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T15:04:08.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.214+0000 7f0834f08700 1 -- 172.21.15.32:0/255853628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f083010b600 con 0x7f08301006a0 2023-09-30T15:04:08.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.215+0000 7f081f7fe700 1 -- 172.21.15.32:0/255853628 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0824016020 con 0x7f08301006a0 2023-09-30T15:04:08.216 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-09-30T15:04:08.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.217+0000 7f0834f08700 1 -- 172.21.15.32:0/255853628 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f08180a0fa0 msgr2=0x7f08180a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:08.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.217+0000 7f0834f08700 1 --2- 172.21.15.32:0/255853628 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f08180a0fa0 0x7f08180a3460 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f082000b2e0 tx=0x7f0820005fb0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.217+0000 7f0834f08700 1 -- 172.21.15.32:0/255853628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08301006a0 msgr2=0x7f0830196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:08.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.217+0000 7f0834f08700 1 --2- 172.21.15.32:0/255853628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08301006a0 0x7f0830196000 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f082400d950 tx=0x7f082400dc60 comp rx=0 tx=0).stop 2023-09-30T15:04:08.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.217+0000 7f0834f08700 1 -- 172.21.15.32:0/255853628 shutdown_connections 2023-09-30T15:04:08.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.217+0000 7f0834f08700 1 --2- 172.21.15.32:0/255853628 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f08180a0fa0 0x7f08180a3460 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.217+0000 7f0834f08700 1 --2- 172.21.15.32:0/255853628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0830101040 0x7f083019d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.217+0000 7f0834f08700 1 --2- 172.21.15.32:0/255853628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08300ff4b0 0x7f0830195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.217+0000 7f0834f08700 1 --2- 172.21.15.32:0/255853628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08301006a0 0x7f0830196000 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.217+0000 7f0834f08700 1 -- 172.21.15.32:0/255853628 >> 172.21.15.32:0/255853628 conn(0x7f08300faa20 msgr2=0x7f08300fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:08.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.217+0000 7f0834f08700 1 -- 172.21.15.32:0/255853628 shutdown_connections 2023-09-30T15:04:08.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.217+0000 7f0834f08700 1 -- 172.21.15.32:0/255853628 wait complete. 2023-09-30T15:04:08.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T15:04:08.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.359+0000 7f26b56fe700 1 -- 172.21.15.32:0/1578715895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26b0104260 msgr2=0x7f26b0106650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.359+0000 7f26b56fe700 1 --2- 172.21.15.32:0/1578715895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26b0104260 0x7f26b0106650 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009990 tx=0x7f26a0009ca0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.359+0000 7f26b56fe700 1 -- 172.21.15.32:0/1578715895 shutdown_connections 2023-09-30T15:04:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.359+0000 7f26b56fe700 1 --2- 172.21.15.32:0/1578715895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b0106b90 0x7f26b0108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.359+0000 7f26b56fe700 1 --2- 172.21.15.32:0/1578715895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26b0101930 0x7f26b0103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.359+0000 7f26b56fe700 1 --2- 172.21.15.32:0/1578715895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26b0104260 0x7f26b0106650 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.359+0000 7f26b56fe700 1 -- 172.21.15.32:0/1578715895 >> 172.21.15.32:0/1578715895 conn(0x7f26b00fb2a0 msgr2=0x7f26b00fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.359+0000 7f26b56fe700 1 -- 172.21.15.32:0/1578715895 shutdown_connections 2023-09-30T15:04:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.359+0000 7f26b56fe700 1 -- 172.21.15.32:0/1578715895 wait complete. 2023-09-30T15:04:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.359+0000 7f26b56fe700 1 Processor -- start 2023-09-30T15:04:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26b56fe700 1 -- start start 2023-09-30T15:04:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26b56fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b0101930 0x7f26b01012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26b56fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26b0104260 0x7f26b00ff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26b56fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26b0106b90 0x7f26b00ffe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26b56fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b01005c0 con 0x7f26b0106b90 2023-09-30T15:04:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26b56fe700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b0100730 con 0x7f26b0104260 2023-09-30T15:04:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26b56fe700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b01a0bd0 con 0x7f26b0101930 2023-09-30T15:04:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26af7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26b0106b90 0x7f26b00ffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26af7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26b0106b90 0x7f26b00ffe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40642/0 (socket says 172.21.15.32:40642) 2023-09-30T15:04:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26af7fe700 1 -- 172.21.15.32:0/2114594085 learned_addr learned my addr 172.21.15.32:0/2114594085 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:04:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26aeffd700 1 --2- 172.21.15.32:0/2114594085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b0101930 0x7f26b01012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26ae7fc700 1 --2- 172.21.15.32:0/2114594085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26b0104260 0x7f26b00ff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26af7fe700 1 -- 172.21.15.32:0/2114594085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b0101930 msgr2=0x7f26b01012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26af7fe700 1 --2- 172.21.15.32:0/2114594085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b0101930 0x7f26b01012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26af7fe700 1 -- 172.21.15.32:0/2114594085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26b0104260 msgr2=0x7f26b00ff8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26af7fe700 1 --2- 172.21.15.32:0/2114594085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26b0104260 0x7f26b00ff8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26af7fe700 1 -- 172.21.15.32:0/2114594085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a00095f0 con 0x7f26b0106b90 2023-09-30T15:04:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26ae7fc700 1 --2- 172.21.15.32:0/2114594085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26b0104260 0x7f26b00ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T15:04:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26aeffd700 1 --2- 172.21.15.32:0/2114594085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b0101930 0x7f26b01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T15:04:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f26af7fe700 1 --2- 172.21.15.32:0/2114594085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26b0106b90 0x7f26b00ffe20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f26a800b970 tx=0x7f26a800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.360+0000 7f2697fff700 1 -- 172.21.15.32:0/2114594085 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a800c9b0 con 0x7f26b0106b90 2023-09-30T15:04:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.361+0000 7f2697fff700 1 -- 172.21.15.32:0/2114594085 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26a8014440 con 0x7f26b0106b90 2023-09-30T15:04:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.361+0000 7f26b56fe700 1 -- 172.21.15.32:0/2114594085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b01a0eb0 con 0x7f26b0106b90 2023-09-30T15:04:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.361+0000 7f2697fff700 1 -- 172.21.15.32:0/2114594085 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a80075d0 con 0x7f26b0106b90 2023-09-30T15:04:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.361+0000 7f26b56fe700 1 -- 172.21.15.32:0/2114594085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b01a12e0 con 0x7f26b0106b90 2023-09-30T15:04:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.361+0000 7f26b56fe700 1 -- 172.21.15.32:0/2114594085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b00fce20 con 0x7f26b0106b90 2023-09-30T15:04:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.363+0000 7f2697fff700 1 -- 172.21.15.32:0/2114594085 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f26a80145b0 con 0x7f26b0106b90 2023-09-30T15:04:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.363+0000 7f2697fff700 1 --2- 172.21.15.32:0/2114594085 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f26980a0fa0 0x7f26980a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.364+0000 7f26aeffd700 1 --2- 172.21.15.32:0/2114594085 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f26980a0fa0 0x7f26980a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.364+0000 7f2697fff700 1 -- 172.21.15.32:0/2114594085 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f26a8006e40 con 0x7f26b0106b90 2023-09-30T15:04:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.364+0000 7f26aeffd700 1 --2- 172.21.15.32:0/2114594085 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f26980a0fa0 0x7f26980a3460 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f26a4005950 tx=0x7f26a40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:08.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.367+0000 7f2697fff700 1 -- 172.21.15.32:0/2114594085 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f26a8095a50 con 0x7f26b0106b90 2023-09-30T15:04:08.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.567+0000 7f26b56fe700 1 -- 172.21.15.32:0/2114594085 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f26b010c820 con 0x7f26b0106b90 2023-09-30T15:04:08.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.568+0000 7f2697fff700 1 -- 172.21.15.32:0/2114594085 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f26a8099070 con 0x7f26b0106b90 2023-09-30T15:04:08.568 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-09-30T15:04:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.570+0000 7f26b56fe700 1 -- 172.21.15.32:0/2114594085 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f26980a0fa0 msgr2=0x7f26980a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.570+0000 7f26b56fe700 1 --2- 172.21.15.32:0/2114594085 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f26980a0fa0 0x7f26980a3460 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f26a4005950 tx=0x7f26a40058e0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.570+0000 7f26b56fe700 1 -- 172.21.15.32:0/2114594085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26b0106b90 msgr2=0x7f26b00ffe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.570+0000 7f26b56fe700 1 --2- 172.21.15.32:0/2114594085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26b0106b90 0x7f26b00ffe20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f26a800b970 tx=0x7f26a800bd30 comp rx=0 tx=0).stop 2023-09-30T15:04:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.570+0000 7f26b56fe700 1 -- 172.21.15.32:0/2114594085 shutdown_connections 2023-09-30T15:04:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.570+0000 7f26b56fe700 1 --2- 172.21.15.32:0/2114594085 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f26980a0fa0 0x7f26980a3460 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.570+0000 7f26b56fe700 1 --2- 172.21.15.32:0/2114594085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26b0106b90 0x7f26b00ffe20 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.570+0000 7f26b56fe700 1 --2- 172.21.15.32:0/2114594085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b0101930 0x7f26b01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.570+0000 7f26b56fe700 1 --2- 172.21.15.32:0/2114594085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26b0104260 0x7f26b00ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.570+0000 7f26b56fe700 1 -- 172.21.15.32:0/2114594085 >> 172.21.15.32:0/2114594085 conn(0x7f26b00fb2a0 msgr2=0x7f26b01025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:08.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.570+0000 7f26b56fe700 1 -- 172.21.15.32:0/2114594085 shutdown_connections 2023-09-30T15:04:08.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.570+0000 7f26b56fe700 1 -- 172.21.15.32:0/2114594085 wait complete. 2023-09-30T15:04:08.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-30T15:04:08.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.710+0000 7ffb0e532700 1 -- 172.21.15.32:0/3678727518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb08100f20 msgr2=0x7ffb08101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.710+0000 7ffb0e532700 1 --2- 172.21.15.32:0/3678727518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb08100f20 0x7ffb08101380 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ffafc0099c0 tx=0x7ffafc009cd0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.710+0000 7ffb0e532700 1 -- 172.21.15.32:0/3678727518 shutdown_connections 2023-09-30T15:04:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.710+0000 7ffb0e532700 1 --2- 172.21.15.32:0/3678727518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb081018c0 0x7ffb08108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.710+0000 7ffb0e532700 1 --2- 172.21.15.32:0/3678727518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb080ffd30 0x7ffb08100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.710+0000 7ffb0e532700 1 --2- 172.21.15.32:0/3678727518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb08100f20 0x7ffb08101380 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.710+0000 7ffb0e532700 1 -- 172.21.15.32:0/3678727518 >> 172.21.15.32:0/3678727518 conn(0x7ffb080fb2a0 msgr2=0x7ffb080fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.710+0000 7ffb0e532700 1 -- 172.21.15.32:0/3678727518 shutdown_connections 2023-09-30T15:04:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.710+0000 7ffb0e532700 1 -- 172.21.15.32:0/3678727518 wait complete. 2023-09-30T15:04:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.710+0000 7ffb0e532700 1 Processor -- start 2023-09-30T15:04:08.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.710+0000 7ffb0e532700 1 -- start start 2023-09-30T15:04:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb0e532700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb080ffd30 0x7ffb0810c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb0e532700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb08100f20 0x7ffb0810a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb0e532700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb081018c0 0x7ffb0810ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb0e532700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0810b3b0 con 0x7ffb080ffd30 2023-09-30T15:04:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb0e532700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0810b520 con 0x7ffb081018c0 2023-09-30T15:04:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb0e532700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb081a0bb0 con 0x7ffb08100f20 2023-09-30T15:04:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb07fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb080ffd30 0x7ffb0810c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb07fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb080ffd30 0x7ffb0810c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40652/0 (socket says 172.21.15.32:40652) 2023-09-30T15:04:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb07fff700 1 -- 172.21.15.32:0/3186229248 learned_addr learned my addr 172.21.15.32:0/3186229248 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:04:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb077fe700 1 --2- 172.21.15.32:0/3186229248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb08100f20 0x7ffb0810a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb0cacf700 1 --2- 172.21.15.32:0/3186229248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb081018c0 0x7ffb0810ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb07fff700 1 -- 172.21.15.32:0/3186229248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb08100f20 msgr2=0x7ffb0810a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb07fff700 1 --2- 172.21.15.32:0/3186229248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb08100f20 0x7ffb0810a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb07fff700 1 -- 172.21.15.32:0/3186229248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb081018c0 msgr2=0x7ffb0810ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb07fff700 1 --2- 172.21.15.32:0/3186229248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb081018c0 0x7ffb0810ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb07fff700 1 -- 172.21.15.32:0/3186229248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc0095f0 con 0x7ffb080ffd30 2023-09-30T15:04:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb0cacf700 1 --2- 172.21.15.32:0/3186229248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb081018c0 0x7ffb0810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T15:04:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.711+0000 7ffb07fff700 1 --2- 172.21.15.32:0/3186229248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb080ffd30 0x7ffb0810c090 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800c970 tx=0x7ffaf800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.712+0000 7ffb057fa700 1 -- 172.21.15.32:0/3186229248 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8004220 con 0x7ffb080ffd30 2023-09-30T15:04:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.712+0000 7ffb0e532700 1 -- 172.21.15.32:0/3186229248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb081a0e90 con 0x7ffb080ffd30 2023-09-30T15:04:08.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.712+0000 7ffb057fa700 1 -- 172.21.15.32:0/3186229248 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaf8007d60 con 0x7ffb080ffd30 2023-09-30T15:04:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.712+0000 7ffb057fa700 1 -- 172.21.15.32:0/3186229248 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8003d00 con 0x7ffb080ffd30 2023-09-30T15:04:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.712+0000 7ffb0e532700 1 -- 172.21.15.32:0/3186229248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb081a12b0 con 0x7ffb080ffd30 2023-09-30T15:04:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.713+0000 7ffb0e532700 1 -- 172.21.15.32:0/3186229248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb08106cb0 con 0x7ffb080ffd30 2023-09-30T15:04:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.714+0000 7ffb057fa700 1 -- 172.21.15.32:0/3186229248 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7ffaf8004380 con 0x7ffb080ffd30 2023-09-30T15:04:08.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.714+0000 7ffb057fa700 1 --2- 172.21.15.32:0/3186229248 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffaf00a0fa0 0x7ffaf00a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:08.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.715+0000 7ffb077fe700 1 --2- 172.21.15.32:0/3186229248 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffaf00a0fa0 0x7ffaf00a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:08.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.715+0000 7ffb057fa700 1 -- 172.21.15.32:0/3186229248 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffaf80cc680 con 0x7ffb080ffd30 2023-09-30T15:04:08.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.715+0000 7ffb077fe700 1 --2- 172.21.15.32:0/3186229248 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffaf00a0fa0 0x7ffaf00a3460 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ffafc000c00 tx=0x7ffafc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:08.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.718+0000 7ffb057fa700 1 -- 172.21.15.32:0/3186229248 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7ffaf8095e90 con 0x7ffb080ffd30 2023-09-30T15:04:08.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:08 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2406960291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T15:04:08.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:08 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/255853628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T15:04:08.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:08 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2114594085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T15:04:08.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.941+0000 7ffb0e532700 1 -- 172.21.15.32:0/3186229248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffb080008d0 con 0x7ffb080ffd30 2023-09-30T15:04:08.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.941+0000 7ffb057fa700 1 -- 172.21.15.32:0/3186229248 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) v1 ==== 81+0+592 (secure 0 0 0) 0x7ffaf801d260 con 0x7ffb080ffd30 2023-09-30T15:04:08.942 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/1639474065 2023-10-01T14:31:11.693545+0000 2023-09-30T15:04:08.942 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1334723693 2023-10-01T14:31:11.693545+0000 2023-09-30T15:04:08.942 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1412452188 2023-10-01T14:31:11.693545+0000 2023-09-30T15:04:08.942 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/2564784233 2023-10-01T14:30:46.025499+0000 2023-09-30T15:04:08.942 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/2564784233 2023-10-01T14:30:46.025499+0000 2023-09-30T15:04:08.943 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3422990986 2023-10-01T14:30:46.025499+0000 2023-09-30T15:04:08.943 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2379235225 2023-10-01T14:30:46.025499+0000 2023-09-30T15:04:08.943 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2971157038 2023-10-01T14:31:11.693545+0000 2023-09-30T15:04:08.943 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/1639474065 2023-10-01T14:31:11.693545+0000 2023-09-30T15:04:08.943 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/4045220015 2023-10-01T14:30:46.025499+0000 2023-09-30T15:04:08.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.944+0000 7ffb0e532700 1 -- 172.21.15.32:0/3186229248 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffaf00a0fa0 msgr2=0x7ffaf00a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:08.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.944+0000 7ffb0e532700 1 --2- 172.21.15.32:0/3186229248 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffaf00a0fa0 0x7ffaf00a3460 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ffafc000c00 tx=0x7ffafc005fb0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.944+0000 7ffb0e532700 1 -- 172.21.15.32:0/3186229248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb080ffd30 msgr2=0x7ffb0810c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:08.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.944+0000 7ffb0e532700 1 --2- 172.21.15.32:0/3186229248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb080ffd30 0x7ffb0810c090 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800c970 tx=0x7ffaf800cd30 comp rx=0 tx=0).stop 2023-09-30T15:04:08.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.944+0000 7ffb0e532700 1 -- 172.21.15.32:0/3186229248 shutdown_connections 2023-09-30T15:04:08.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.944+0000 7ffb0e532700 1 --2- 172.21.15.32:0/3186229248 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7ffaf00a0fa0 0x7ffaf00a3460 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.944+0000 7ffb0e532700 1 --2- 172.21.15.32:0/3186229248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb081018c0 0x7ffb0810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.944+0000 7ffb0e532700 1 --2- 172.21.15.32:0/3186229248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb080ffd30 0x7ffb0810c090 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.944+0000 7ffb0e532700 1 --2- 172.21.15.32:0/3186229248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb08100f20 0x7ffb0810a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:08.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.944+0000 7ffb0e532700 1 -- 172.21.15.32:0/3186229248 >> 172.21.15.32:0/3186229248 conn(0x7ffb080fb2a0 msgr2=0x7ffb080fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:08.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.944+0000 7ffb0e532700 1 -- 172.21.15.32:0/3186229248 shutdown_connections 2023-09-30T15:04:08.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:08.944+0000 7ffb0e532700 1 -- 172.21.15.32:0/3186229248 wait complete. 2023-09-30T15:04:08.946 INFO:teuthology.orchestra.run.smithi032.stderr:listed 10 entries 2023-09-30T15:04:08.957 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-09-30T15:04:08.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T15:04:08.957 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-09-30T15:04:09.063 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:08 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2406960291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T15:04:09.063 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:08 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/255853628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T15:04:09.063 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:08 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2114594085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T15:04:09.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:08 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2406960291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-30T15:04:09.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:08 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/255853628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-30T15:04:09.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:08 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2114594085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-30T15:04:09.164 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T15:04:09.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T15:04:09.164 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-30T15:04:09.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T15:04:09.357 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-09-30T15:04:09.504 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-30T15:04:09.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T15:04:09.505 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T15:04:09.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T15:04:09.663 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-09-30T15:04:09.794 INFO:tasks.kclient:Unmounting kernel clients... 2023-09-30T15:04:09.794 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T15:04:09.794 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-30T15:04:09.815 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=33,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3778 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/b89367bc2f21f71c1856a26307cf8348d64ed9af1d3e9bf2a9841c4c187478b6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7MHDKG3P2RR4ISJR34JYZLJYAC:/var/lib/containers/storage/overlay/l/QQX2KRLTQSH5P745N34YPQGT2J,upperdir=/var/lib/containers/storage/overlay/b89367bc2f21f71c1856a26307cf8348d64ed9af1d3e9bf2a9841c4c187478b6/diff,workdir=/var/lib/containers/storage/overlay/b89367bc2f21f71c1856a26307cf8348d64ed9af1d3e9bf2a9841c4c187478b6/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/721f3bb92c1c0e5f832e00233fa534e3d31df81486dc45bd390f3136a3e1b8b2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7MHDKG3P2RR4ISJR34JYZLJYAC:/var/lib/containers/storage/overlay/l/QQX2KRLTQSH5P745N34YPQGT2J,upperdir=/var/lib/containers/storage/overlay/721f3bb92c1c0e5f832e00233fa534e3d31df81486dc45bd390f3136a3e1b8b2/diff,workdir=/var/lib/containers/storage/overlay/721f3bb92c1c0e5f832e00233fa534e3d31df81486dc45bd390f3136a3e1b8b2/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/332d3368a34fca3fb46d8a600262a1bddb45c5e56a331938828892728e853e52/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7MHDKG3P2RR4ISJR34JYZLJYAC:/var/lib/containers/storage/overlay/l/QQX2KRLTQSH5P745N34YPQGT2J,upperdir=/var/lib/containers/storage/overlay/332d3368a34fca3fb46d8a600262a1bddb45c5e56a331938828892728e853e52/diff,workdir=/var/lib/containers/storage/overlay/332d3368a34fca3fb46d8a600262a1bddb45c5e56a331938828892728e853e52/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ca26d4c7404ddd7da205c321539d0dbdb83ff585d187a8f91465c5be019e967f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7MHDKG3P2RR4ISJR34JYZLJYAC:/var/lib/containers/storage/overlay/l/QQX2KRLTQSH5P745N34YPQGT2J,upperdir=/var/lib/containers/storage/overlay/ca26d4c7404ddd7da205c321539d0dbdb83ff585d187a8f91465c5be019e967f/diff,workdir=/var/lib/containers/storage/overlay/ca26d4c7404ddd7da205c321539d0dbdb83ff585d187a8f91465c5be019e967f/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e742770ac3de0bc6d77133be52e7bd18fdc2e3913b4d2e2cd58d59c9a8982238/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7MHDKG3P2RR4ISJR34JYZLJYAC:/var/lib/containers/storage/overlay/l/QQX2KRLTQSH5P745N34YPQGT2J,upperdir=/var/lib/containers/storage/overlay/e742770ac3de0bc6d77133be52e7bd18fdc2e3913b4d2e2cd58d59c9a8982238/diff,workdir=/var/lib/containers/storage/overlay/e742770ac3de0bc6d77133be52e7bd18fdc2e3913b4d2e2cd58d59c9a8982238/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/cf8caba1b270d7f8a3d217d588d8a09d9b32c1782b979006ecc7ec3583efd42f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7MHDKG3P2RR4ISJR34JYZLJYAC:/var/lib/containers/storage/overlay/l/QQX2KRLTQSH5P745N34YPQGT2J,upperdir=/var/lib/containers/storage/overlay/cf8caba1b270d7f8a3d217d588d8a09d9b32c1782b979006ecc7ec3583efd42f/diff,workdir=/var/lib/containers/storage/overlay/cf8caba1b270d7f8a3d217d588d8a09d9b32c1782b979006ecc7ec3583efd42f/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e992f95289d22ad9405811bf755b886b7c55fbb834bc07e616f0c7e6052b6aae/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7MHDKG3P2RR4ISJR34JYZLJYAC:/var/lib/containers/storage/overlay/l/QQX2KRLTQSH5P745N34YPQGT2J,upperdir=/var/lib/containers/storage/overlay/e992f95289d22ad9405811bf755b886b7c55fbb834bc07e616f0c7e6052b6aae/diff,workdir=/var/lib/containers/storage/overlay/e992f95289d22ad9405811bf755b886b7c55fbb834bc07e616f0c7e6052b6aae/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d3f94394a2fbf6c80544608833424f40fc1633a921545d3bd2290f6d0d3c2c06/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7MHDKG3P2RR4ISJR34JYZLJYAC:/var/lib/containers/storage/overlay/l/QQX2KRLTQSH5P745N34YPQGT2J,upperdir=/var/lib/containers/storage/overlay/d3f94394a2fbf6c80544608833424f40fc1633a921545d3bd2290f6d0d3c2c06/diff,workdir=/var/lib/containers/storage/overlay/d3f94394a2fbf6c80544608833424f40fc1633a921545d3bd2290f6d0d3c2c06/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6648ad6ef4f50101990139b1e3f9c181908e870dbaf01a3db2cdcd81a00b0864/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7MHDKG3P2RR4ISJR34JYZLJYAC:/var/lib/containers/storage/overlay/l/QQX2KRLTQSH5P745N34YPQGT2J,upperdir=/var/lib/containers/storage/overlay/6648ad6ef4f50101990139b1e3f9c181908e870dbaf01a3db2cdcd81a00b0864/diff,workdir=/var/lib/containers/storage/overlay/6648ad6ef4f50101990139b1e3f9c181908e870dbaf01a3db2cdcd81a00b0864/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/63237f0d43464749caec0bc4684bc526e62a35b1471fde786b2477e788f191b9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7MHDKG3P2RR4ISJR34JYZLJYAC:/var/lib/containers/storage/overlay/l/QQX2KRLTQSH5P745N34YPQGT2J,upperdir=/var/lib/containers/storage/overlay/63237f0d43464749caec0bc4684bc526e62a35b1471fde786b2477e788f191b9/diff,workdir=/var/lib/containers/storage/overlay/63237f0d43464749caec0bc4684bc526e62a35b1471fde786b2477e788f191b9/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-09-30T15:04:09.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T15:04:09.815 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-09-30T15:04:09.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T15:04:09.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-09-30T15:04:09.936 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:09 smithi032 ceph-mon[98592]: pgmap v1063: 97 pgs: 97 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 3.2 MiB/s wr, 252 op/s 2023-09-30T15:04:09.937 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:09 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3186229248' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-30T15:04:09.942 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2023-09-30T15:04:09.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T15:04:09.944 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-09-30T15:04:09.962 INFO:tasks.cephadm:Teardown begin 2023-09-30T15:04:09.963 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-30T15:04:09.963 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T15:04:10.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:09 smithi156 ceph-mon[103579]: pgmap v1063: 97 pgs: 97 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 3.2 MiB/s wr, 252 op/s 2023-09-30T15:04:10.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:09 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3186229248' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-30T15:04:10.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:09 smithi148 ceph-mon[102355]: pgmap v1063: 97 pgs: 97 active+clean; 473 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 3.2 MiB/s wr, 252 op/s 2023-09-30T15:04:10.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:09 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3186229248' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-30T15:04:10.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.111+0000 7fc20b739700 1 -- 172.21.15.32:0/766588482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2040fff80 msgr2=0x7fc204100400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:10.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.111+0000 7fc20b739700 1 --2- 172.21.15.32:0/766588482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2040fff80 0x7fc204100400 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40098f0 tx=0x7fc1f4009c00 comp rx=0 tx=0).stop 2023-09-30T15:04:10.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.111+0000 7fc20b739700 1 -- 172.21.15.32:0/766588482 shutdown_connections 2023-09-30T15:04:10.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.111+0000 7fc20b739700 1 --2- 172.21.15.32:0/766588482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc204100940 0x7fc2041081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.111+0000 7fc20b739700 1 --2- 172.21.15.32:0/766588482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2041013c0 0x7fc2041017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.111+0000 7fc20b739700 1 --2- 172.21.15.32:0/766588482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2040fff80 0x7fc204100400 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.111+0000 7fc20b739700 1 -- 172.21.15.32:0/766588482 >> 172.21.15.32:0/766588482 conn(0x7fc2040faa30 msgr2=0x7fc2040fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:10.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.111+0000 7fc20b739700 1 -- 172.21.15.32:0/766588482 shutdown_connections 2023-09-30T15:04:10.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.111+0000 7fc20b739700 1 -- 172.21.15.32:0/766588482 wait complete. 2023-09-30T15:04:10.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc20b739700 1 Processor -- start 2023-09-30T15:04:10.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc20b739700 1 -- start start 2023-09-30T15:04:10.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc20b739700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2040fff80 0x7fc204196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:10.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc20b739700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc204100940 0x7fc204197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:10.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc20b739700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2041013c0 0x7fc20419e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:10.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc20b739700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20419e650 con 0x7fc204100940 2023-09-30T15:04:10.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc20b739700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20419e790 con 0x7fc2041013c0 2023-09-30T15:04:10.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc20b739700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20419e920 con 0x7fc2040fff80 2023-09-30T15:04:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc208cd4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc204100940 0x7fc204197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc208cd4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc204100940 0x7fc204197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40666/0 (socket says 172.21.15.32:40666) 2023-09-30T15:04:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc208cd4700 1 -- 172.21.15.32:0/482037629 learned_addr learned my addr 172.21.15.32:0/482037629 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:04:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc2094d5700 1 --2- 172.21.15.32:0/482037629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2040fff80 0x7fc204196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc209cd6700 1 --2- 172.21.15.32:0/482037629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2041013c0 0x7fc20419e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc208cd4700 1 -- 172.21.15.32:0/482037629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2040fff80 msgr2=0x7fc204196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc208cd4700 1 --2- 172.21.15.32:0/482037629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2040fff80 0x7fc204196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc208cd4700 1 -- 172.21.15.32:0/482037629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2041013c0 msgr2=0x7fc20419e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc208cd4700 1 --2- 172.21.15.32:0/482037629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2041013c0 0x7fc20419e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc208cd4700 1 -- 172.21.15.32:0/482037629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f4009520 con 0x7fc204100940 2023-09-30T15:04:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.112+0000 7fc208cd4700 1 --2- 172.21.15.32:0/482037629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc204100940 0x7fc204197010 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009230 tx=0x7fc1f400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.113+0000 7fc1fa7fc700 1 -- 172.21.15.32:0/482037629 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f401c070 con 0x7fc204100940 2023-09-30T15:04:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.113+0000 7fc20b739700 1 -- 172.21.15.32:0/482037629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20419eba0 con 0x7fc204100940 2023-09-30T15:04:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.113+0000 7fc1fa7fc700 1 -- 172.21.15.32:0/482037629 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1f400f860 con 0x7fc204100940 2023-09-30T15:04:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.113+0000 7fc1fa7fc700 1 -- 172.21.15.32:0/482037629 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4004b30 con 0x7fc204100940 2023-09-30T15:04:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.113+0000 7fc20b739700 1 -- 172.21.15.32:0/482037629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20419efc0 con 0x7fc204100940 2023-09-30T15:04:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.114+0000 7fc20b739700 1 -- 172.21.15.32:0/482037629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20410b530 con 0x7fc204100940 2023-09-30T15:04:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.115+0000 7fc1fa7fc700 1 -- 172.21.15.32:0/482037629 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fc1f4020070 con 0x7fc204100940 2023-09-30T15:04:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.115+0000 7fc1fa7fc700 1 --2- 172.21.15.32:0/482037629 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1f00a0ff0 0x7fc1f00a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.115+0000 7fc1fa7fc700 1 -- 172.21.15.32:0/482037629 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc1f40cd560 con 0x7fc204100940 2023-09-30T15:04:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.115+0000 7fc2094d5700 1 --2- 172.21.15.32:0/482037629 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1f00a0ff0 0x7fc1f00a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.116+0000 7fc2094d5700 1 --2- 172.21.15.32:0/482037629 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1f00a0ff0 0x7fc1f00a34b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc200007be0 tx=0x7fc2000073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:10.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.120+0000 7fc1fa7fc700 1 -- 172.21.15.32:0/482037629 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fc1f4096e00 con 0x7fc204100940 2023-09-30T15:04:10.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.370+0000 7fc20b739700 1 -- 172.21.15.32:0/482037629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc20419f1b0 con 0x7fc204100940 2023-09-30T15:04:10.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.371+0000 7fc1fa7fc700 1 -- 172.21.15.32:0/482037629 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+11324 (secure 0 0 0) 0x7fc1f409a420 con 0x7fc204100940 2023-09-30T15:04:10.371 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T15:04:10.371 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3505064841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3505064841},{"type":"v1","addr":"172.21.15.32:6839","nonce":3505064841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2262846038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2262846038},{"type":"v1","addr":"172.21.15.32:6841","nonce":2262846038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":59,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-30T14:37:11.444993+0000","modified":"2023-09-30T15:03:31.116569+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":37,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6839/1445176203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1445176203},{"type":"v1","addr":"172.21.15.156:6839","nonce":1445176203}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/494821686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":494821686},{"type":"v1","addr":"172.21.15.148:6839","nonce":494821686}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T15:04:10.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.373+0000 7fc20b739700 1 -- 172.21.15.32:0/482037629 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1f00a0ff0 msgr2=0x7fc1f00a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:10.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.373+0000 7fc20b739700 1 --2- 172.21.15.32:0/482037629 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1f00a0ff0 0x7fc1f00a34b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc200007be0 tx=0x7fc2000073d0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.373+0000 7fc20b739700 1 -- 172.21.15.32:0/482037629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc204100940 msgr2=0x7fc204197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:10.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.373+0000 7fc20b739700 1 --2- 172.21.15.32:0/482037629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc204100940 0x7fc204197010 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009230 tx=0x7fc1f400b790 comp rx=0 tx=0).stop 2023-09-30T15:04:10.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.373+0000 7fc20b739700 1 -- 172.21.15.32:0/482037629 shutdown_connections 2023-09-30T15:04:10.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.373+0000 7fc20b739700 1 --2- 172.21.15.32:0/482037629 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc1f00a0ff0 0x7fc1f00a34b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.373+0000 7fc20b739700 1 --2- 172.21.15.32:0/482037629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2041013c0 0x7fc20419e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.373+0000 7fc20b739700 1 --2- 172.21.15.32:0/482037629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2040fff80 0x7fc204196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.373+0000 7fc20b739700 1 --2- 172.21.15.32:0/482037629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc204100940 0x7fc204197010 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.373+0000 7fc20b739700 1 -- 172.21.15.32:0/482037629 >> 172.21.15.32:0/482037629 conn(0x7fc2040faa30 msgr2=0x7fc2040feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:10.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.373+0000 7fc20b739700 1 -- 172.21.15.32:0/482037629 shutdown_connections 2023-09-30T15:04:10.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.373+0000 7fc20b739700 1 -- 172.21.15.32:0/482037629 wait complete. 2023-09-30T15:04:10.375 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 59 2023-09-30T15:04:10.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T15:04:10.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.512+0000 7f5427346700 1 -- 172.21.15.32:0/2101105819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54200ffd70 msgr2=0x7f5420100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:10.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.512+0000 7f5427346700 1 --2- 172.21.15.32:0/2101105819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54200ffd70 0x7f5420100190 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f541c0099c0 tx=0x7f541c009cd0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.512+0000 7f5427346700 1 -- 172.21.15.32:0/2101105819 shutdown_connections 2023-09-30T15:04:10.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.512+0000 7f5427346700 1 --2- 172.21.15.32:0/2101105819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5420101900 0x7f54201081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.512+0000 7f5427346700 1 --2- 172.21.15.32:0/2101105819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54200ffd70 0x7f5420100190 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.512+0000 7f5427346700 1 --2- 172.21.15.32:0/2101105819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5420100f60 0x7f54201013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.512+0000 7f5427346700 1 -- 172.21.15.32:0/2101105819 >> 172.21.15.32:0/2101105819 conn(0x7f54200fb2e0 msgr2=0x7f54200fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.512+0000 7f5427346700 1 -- 172.21.15.32:0/2101105819 shutdown_connections 2023-09-30T15:04:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.512+0000 7f5427346700 1 -- 172.21.15.32:0/2101105819 wait complete. 2023-09-30T15:04:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f5427346700 1 Processor -- start 2023-09-30T15:04:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f5427346700 1 -- start start 2023-09-30T15:04:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f5427346700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54200ffd70 0x7f5420196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f5427346700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5420100f60 0x7f5420197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f5427346700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5420101900 0x7f542019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f5427346700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542019e6b0 con 0x7f5420101900 2023-09-30T15:04:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f5427346700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542019e7f0 con 0x7f54200ffd70 2023-09-30T15:04:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f5427346700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542019e980 con 0x7f5420100f60 2023-09-30T15:04:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f54258e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5420101900 0x7f542019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f54250e2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54200ffd70 0x7f5420196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f54258e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5420101900 0x7f542019e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40682/0 (socket says 172.21.15.32:40682) 2023-09-30T15:04:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f54258e3700 1 -- 172.21.15.32:0/376259138 learned_addr learned my addr 172.21.15.32:0/376259138 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:04:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f54248e1700 1 --2- 172.21.15.32:0/376259138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5420100f60 0x7f5420197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f54258e3700 1 -- 172.21.15.32:0/376259138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5420100f60 msgr2=0x7f5420197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f54258e3700 1 --2- 172.21.15.32:0/376259138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5420100f60 0x7f5420197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f54258e3700 1 -- 172.21.15.32:0/376259138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54200ffd70 msgr2=0x7f5420196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f54258e3700 1 --2- 172.21.15.32:0/376259138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54200ffd70 0x7f5420196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f54258e3700 1 -- 172.21.15.32:0/376259138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c0095f0 con 0x7f5420101900 2023-09-30T15:04:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.513+0000 7f54250e2700 1 --2- 172.21.15.32:0/376259138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54200ffd70 0x7f5420196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T15:04:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.514+0000 7f54258e3700 1 --2- 172.21.15.32:0/376259138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5420101900 0x7f542019e0e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f541800ada0 tx=0x7f541800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:10.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.514+0000 7f54127fc700 1 -- 172.21.15.32:0/376259138 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541800f070 con 0x7f5420101900 2023-09-30T15:04:10.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.514+0000 7f54127fc700 1 -- 172.21.15.32:0/376259138 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5418007b30 con 0x7f5420101900 2023-09-30T15:04:10.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.514+0000 7f5427346700 1 -- 172.21.15.32:0/376259138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542019ec60 con 0x7f5420101900 2023-09-30T15:04:10.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.514+0000 7f54127fc700 1 -- 172.21.15.32:0/376259138 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418005280 con 0x7f5420101900 2023-09-30T15:04:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.514+0000 7f5427346700 1 -- 172.21.15.32:0/376259138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542019f170 con 0x7f5420101900 2023-09-30T15:04:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.515+0000 7f5427346700 1 -- 172.21.15.32:0/376259138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542010b580 con 0x7f5420101900 2023-09-30T15:04:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.516+0000 7f54127fc700 1 -- 172.21.15.32:0/376259138 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f541800a090 con 0x7f5420101900 2023-09-30T15:04:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.516+0000 7f54127fc700 1 --2- 172.21.15.32:0/376259138 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f540c0a0ff0 0x7f540c0a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.517+0000 7f54250e2700 1 --2- 172.21.15.32:0/376259138 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f540c0a0ff0 0x7f540c0a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.517+0000 7f54127fc700 1 -- 172.21.15.32:0/376259138 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f54180cc200 con 0x7f5420101900 2023-09-30T15:04:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.517+0000 7f54250e2700 1 --2- 172.21.15.32:0/376259138 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f540c0a0ff0 0x7f540c0a34b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f541c009f50 tx=0x7f541c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:10.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.521+0000 7f54127fc700 1 -- 172.21.15.32:0/376259138 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f54180994d0 con 0x7f5420101900 2023-09-30T15:04:10.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.770+0000 7f5427346700 1 -- 172.21.15.32:0/376259138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f542010b790 con 0x7f5420101900 2023-09-30T15:04:10.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.771+0000 7f54127fc700 1 -- 172.21.15.32:0/376259138 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+11324 (secure 0 0 0) 0x7f5418099060 con 0x7f5420101900 2023-09-30T15:04:10.771 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T15:04:10.771 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3687429810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3687429810},{"type":"v1","addr":"172.21.15.32:6835","nonce":3687429810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2728444148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2728444148},{"type":"v1","addr":"172.21.15.32:6837","nonce":2728444148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3505064841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3505064841},{"type":"v1","addr":"172.21.15.32:6839","nonce":3505064841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/2262846038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":2262846038},{"type":"v1","addr":"172.21.15.32:6841","nonce":2262846038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2184509909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2184509909},{"type":"v1","addr":"172.21.15.156:6833","nonce":2184509909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/636626698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":636626698},{"type":"v1","addr":"172.21.15.148:6833","nonce":636626698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":59,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-30T14:37:11.444993+0000","modified":"2023-09-30T15:03:31.116569+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/4151174061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4151174061},{"type":"v1","addr":"172.21.15.148:6835","nonce":4151174061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/4180205116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4180205116},{"type":"v1","addr":"172.21.15.156:6835","nonce":4180205116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/445372881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":445372881},{"type":"v1","addr":"172.21.15.156:6837","nonce":445372881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":37,"addr":"172.21.15.148:6837/3666119034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3666119034},{"type":"v1","addr":"172.21.15.148:6837","nonce":3666119034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.156:6839/1445176203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":1445176203},{"type":"v1","addr":"172.21.15.156:6839","nonce":1445176203}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6839/494821686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":494821686},{"type":"v1","addr":"172.21.15.148:6839","nonce":494821686}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T15:04:10.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.773+0000 7f5427346700 1 -- 172.21.15.32:0/376259138 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f540c0a0ff0 msgr2=0x7f540c0a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:10.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.773+0000 7f5427346700 1 --2- 172.21.15.32:0/376259138 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f540c0a0ff0 0x7f540c0a34b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f541c009f50 tx=0x7f541c00b540 comp rx=0 tx=0).stop 2023-09-30T15:04:10.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.773+0000 7f5427346700 1 -- 172.21.15.32:0/376259138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5420101900 msgr2=0x7f542019e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:10.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.773+0000 7f5427346700 1 --2- 172.21.15.32:0/376259138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5420101900 0x7f542019e0e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f541800ada0 tx=0x7f541800c330 comp rx=0 tx=0).stop 2023-09-30T15:04:10.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.773+0000 7f5427346700 1 -- 172.21.15.32:0/376259138 shutdown_connections 2023-09-30T15:04:10.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.773+0000 7f5427346700 1 --2- 172.21.15.32:0/376259138 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f540c0a0ff0 0x7f540c0a34b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.773+0000 7f5427346700 1 --2- 172.21.15.32:0/376259138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5420101900 0x7f542019e0e0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.773+0000 7f5427346700 1 --2- 172.21.15.32:0/376259138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54200ffd70 0x7f5420196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.773+0000 7f5427346700 1 --2- 172.21.15.32:0/376259138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5420100f60 0x7f5420197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.773+0000 7f5427346700 1 -- 172.21.15.32:0/376259138 >> 172.21.15.32:0/376259138 conn(0x7f54200fb2e0 msgr2=0x7f5420069000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:10.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.773+0000 7f5427346700 1 -- 172.21.15.32:0/376259138 shutdown_connections 2023-09-30T15:04:10.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.773+0000 7f5427346700 1 -- 172.21.15.32:0/376259138 wait complete. 2023-09-30T15:04:10.775 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 59 2023-09-30T15:04:10.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T15:04:10.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.912+0000 7f6c18cde700 1 -- 172.21.15.32:0/4272162539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c140fff40 msgr2=0x7f6c141003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.912+0000 7f6c18cde700 1 --2- 172.21.15.32:0/4272162539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c140fff40 0x7f6c141003c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009990 tx=0x7f6bfc009ca0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.912+0000 7f6c18cde700 1 -- 172.21.15.32:0/4272162539 shutdown_connections 2023-09-30T15:04:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.912+0000 7f6c18cde700 1 --2- 172.21.15.32:0/4272162539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c14100900 0x7f6c141081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.912+0000 7f6c18cde700 1 --2- 172.21.15.32:0/4272162539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c141012f0 0x7f6c14101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.912+0000 7f6c18cde700 1 --2- 172.21.15.32:0/4272162539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c140fff40 0x7f6c141003c0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.912+0000 7f6c18cde700 1 -- 172.21.15.32:0/4272162539 >> 172.21.15.32:0/4272162539 conn(0x7f6c140faa30 msgr2=0x7f6c140fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.912+0000 7f6c18cde700 1 -- 172.21.15.32:0/4272162539 shutdown_connections 2023-09-30T15:04:10.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.912+0000 7f6c18cde700 1 -- 172.21.15.32:0/4272162539 wait complete. 2023-09-30T15:04:10.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c18cde700 1 Processor -- start 2023-09-30T15:04:10.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c18cde700 1 -- start start 2023-09-30T15:04:10.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c18cde700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c140fff40 0x7f6c141959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:10.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c18cde700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c14100900 0x7f6c14195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:10.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c18cde700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c141012f0 0x7f6c1419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:10.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c18cde700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1419d550 con 0x7f6c140fff40 2023-09-30T15:04:10.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c18cde700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1419d690 con 0x7f6c14100900 2023-09-30T15:04:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c18cde700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1419d820 con 0x7f6c141012f0 2023-09-30T15:04:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c1259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c140fff40 0x7f6c141959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c1259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c140fff40 0x7f6c141959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40692/0 (socket says 172.21.15.32:40692) 2023-09-30T15:04:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c1259c700 1 -- 172.21.15.32:0/3663185402 learned_addr learned my addr 172.21.15.32:0/3663185402 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:04:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c11d9b700 1 --2- 172.21.15.32:0/3663185402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c14100900 0x7f6c14195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c1259c700 1 -- 172.21.15.32:0/3663185402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c141012f0 msgr2=0x7f6c1419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c1259c700 1 --2- 172.21.15.32:0/3663185402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c141012f0 0x7f6c1419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c1259c700 1 -- 172.21.15.32:0/3663185402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c14100900 msgr2=0x7f6c14195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c1259c700 1 --2- 172.21.15.32:0/3663185402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c14100900 0x7f6c14195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c1259c700 1 -- 172.21.15.32:0/3663185402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc0095f0 con 0x7f6c140fff40 2023-09-30T15:04:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c1259c700 1 --2- 172.21.15.32:0/3663185402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c140fff40 0x7f6c141959d0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400d950 tx=0x7f6c0400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:10.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.913+0000 7f6c0b7fe700 1 -- 172.21.15.32:0/3663185402 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04009b40 con 0x7f6c140fff40 2023-09-30T15:04:10.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.914+0000 7f6c0b7fe700 1 -- 172.21.15.32:0/3663185402 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c04009ca0 con 0x7f6c140fff40 2023-09-30T15:04:10.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.914+0000 7f6c18cde700 1 -- 172.21.15.32:0/3663185402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1419db00 con 0x7f6c140fff40 2023-09-30T15:04:10.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.914+0000 7f6c0b7fe700 1 -- 172.21.15.32:0/3663185402 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400f620 con 0x7f6c140fff40 2023-09-30T15:04:10.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.914+0000 7f6c18cde700 1 -- 172.21.15.32:0/3663185402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1419e010 con 0x7f6c140fff40 2023-09-30T15:04:10.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.915+0000 7f6c18cde700 1 -- 172.21.15.32:0/3663185402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1410b530 con 0x7f6c140fff40 2023-09-30T15:04:10.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.916+0000 7f6c0b7fe700 1 -- 172.21.15.32:0/3663185402 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f6c04010460 con 0x7f6c140fff40 2023-09-30T15:04:10.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.916+0000 7f6c0b7fe700 1 --2- 172.21.15.32:0/3663185402 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c000a0ff0 0x7f6c000a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:10.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.916+0000 7f6c11d9b700 1 --2- 172.21.15.32:0/3663185402 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c000a0ff0 0x7f6c000a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:10.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.917+0000 7f6c0b7fe700 1 -- 172.21.15.32:0/3663185402 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6c040cca90 con 0x7f6c140fff40 2023-09-30T15:04:10.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.917+0000 7f6c11d9b700 1 --2- 172.21.15.32:0/3663185402 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c000a0ff0 0x7f6c000a34b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc006010 tx=0x7f6bfc00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:10.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:10.921+0000 7f6c0b7fe700 1 -- 172.21.15.32:0/3663185402 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f6c040962a0 con 0x7f6c140fff40 2023-09-30T15:04:11.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:10 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/482037629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T15:04:11.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:10 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/482037629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T15:04:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.124+0000 7f6c18cde700 1 -- 172.21.15.32:0/3663185402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c1419e1a0 con 0x7f6c140fff40 2023-09-30T15:04:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.126+0000 7f6c0b7fe700 1 -- 172.21.15.32:0/3663185402 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22569 (secure 0 0 0) 0x7f6c04020090 con 0x7f6c140fff40 2023-09-30T15:04:11.126 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T15:04:11.126 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":85,"fsid":"bca3f5c6-5f9d-11ee-8db5-212e2dc638e7","created":"2023-09-30T14:30:12.351176+0000","modified":"2023-09-30T14:38:13.192089+0000","last_up_change":"2023-09-30T14:36:29.972554+0000","last_in_change":"2023-09-30T14:36:12.334230+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T14:33:20.224529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-30T14:37:09.573219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-30T14:37:10.844458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf7887c5-207a-419c-8828-18179f511646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6803","nonce":3314475588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6805","nonce":3314475588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6809","nonce":3314475588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3314475588},{"type":"v1","addr":"172.21.15.32:6807","nonce":3314475588}]},"public_addr":"172.21.15.32:6803/3314475588","cluster_addr":"172.21.15.32:6805/3314475588","heartbeat_back_addr":"172.21.15.32:6809/3314475588","heartbeat_front_addr":"172.21.15.32:6807/3314475588","state":["exists","up"]},{"osd":1,"uuid":"35766d2e-c839-4c77-839a-535465f18cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6801","nonce":2368718565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6803","nonce":2368718565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6807","nonce":2368718565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2368718565},{"type":"v1","addr":"172.21.15.148:6805","nonce":2368718565}]},"public_addr":"172.21.15.148:6801/2368718565","cluster_addr":"172.21.15.148:6803/2368718565","heartbeat_back_addr":"172.21.15.148:6807/2368718565","heartbeat_front_addr":"172.21.15.148:6805/2368718565","state":["exists","up"]},{"osd":2,"uuid":"94cc95d0-0fef-4e49-aefc-2de4626add70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6801","nonce":2843236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6803","nonce":2843236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6807","nonce":2843236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2843236580},{"type":"v1","addr":"172.21.15.156:6805","nonce":2843236580}]},"public_addr":"172.21.15.156:6801/2843236580","cluster_addr":"172.21.15.156:6803/2843236580","heartbeat_back_addr":"172.21.15.156:6807/2843236580","heartbeat_front_addr":"172.21.15.156:6805/2843236580","state":["exists","up"]},{"osd":3,"uuid":"9e9c85a2-5720-4256-9285-f6dd719f0ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6811","nonce":626048583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6813","nonce":626048583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6817","nonce":626048583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":626048583},{"type":"v1","addr":"172.21.15.32:6815","nonce":626048583}]},"public_addr":"172.21.15.32:6811/626048583","cluster_addr":"172.21.15.32:6813/626048583","heartbeat_back_addr":"172.21.15.32:6817/626048583","heartbeat_front_addr":"172.21.15.32:6815/626048583","state":["exists","up"]},{"osd":4,"uuid":"d538f510-0f4b-477a-a7e3-99f203c349cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6809","nonce":1987304559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6811","nonce":1987304559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6815","nonce":1987304559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1987304559},{"type":"v1","addr":"172.21.15.148:6813","nonce":1987304559}]},"public_addr":"172.21.15.148:6809/1987304559","cluster_addr":"172.21.15.148:6811/1987304559","heartbeat_back_addr":"172.21.15.148:6815/1987304559","heartbeat_front_addr":"172.21.15.148:6813/1987304559","state":["exists","up"]},{"osd":5,"uuid":"b7e7db08-601c-416b-be4a-e5b72a9c00a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6809","nonce":2465554180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6811","nonce":2465554180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6815","nonce":2465554180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2465554180},{"type":"v1","addr":"172.21.15.156:6813","nonce":2465554180}]},"public_addr":"172.21.15.156:6809/2465554180","cluster_addr":"172.21.15.156:6811/2465554180","heartbeat_back_addr":"172.21.15.156:6815/2465554180","heartbeat_front_addr":"172.21.15.156:6813/2465554180","state":["exists","up"]},{"osd":6,"uuid":"14fb18d1-a89b-45da-ac5a-deab8fad9681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6819","nonce":4025142433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6821","nonce":4025142433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6825","nonce":4025142433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":4025142433},{"type":"v1","addr":"172.21.15.32:6823","nonce":4025142433}]},"public_addr":"172.21.15.32:6819/4025142433","cluster_addr":"172.21.15.32:6821/4025142433","heartbeat_back_addr":"172.21.15.32:6825/4025142433","heartbeat_front_addr":"172.21.15.32:6823/4025142433","state":["exists","up"]},{"osd":7,"uuid":"15059921-bebf-46fc-87ba-415e7241af36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6817","nonce":3841614147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6819","nonce":3841614147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6823","nonce":3841614147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":3841614147},{"type":"v1","addr":"172.21.15.148:6821","nonce":3841614147}]},"public_addr":"172.21.15.148:6817/3841614147","cluster_addr":"172.21.15.148:6819/3841614147","heartbeat_back_addr":"172.21.15.148:6823/3841614147","heartbeat_front_addr":"172.21.15.148:6821/3841614147","state":["exists","up"]},{"osd":8,"uuid":"6a853c86-fce1-450d-b902-80989d4a5f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6817","nonce":2338383463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6819","nonce":2338383463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6823","nonce":2338383463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":2338383463},{"type":"v1","addr":"172.21.15.156:6821","nonce":2338383463}]},"public_addr":"172.21.15.156:6817/2338383463","cluster_addr":"172.21.15.156:6819/2338383463","heartbeat_back_addr":"172.21.15.156:6823/2338383463","heartbeat_front_addr":"172.21.15.156:6821/2338383463","state":["exists","up"]},{"osd":9,"uuid":"47eafd4b-4c81-4a0c-9496-272c10e2fb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6827","nonce":496139479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6829","nonce":496139479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6833","nonce":496139479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":496139479},{"type":"v1","addr":"172.21.15.32:6831","nonce":496139479}]},"public_addr":"172.21.15.32:6827/496139479","cluster_addr":"172.21.15.32:6829/496139479","heartbeat_back_addr":"172.21.15.32:6833/496139479","heartbeat_front_addr":"172.21.15.32:6831/496139479","state":["exists","up"]},{"osd":10,"uuid":"2c7d56d1-96b3-4d0b-8d6e-dc4587579089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6825","nonce":2093868673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6827","nonce":2093868673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6831","nonce":2093868673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2093868673},{"type":"v1","addr":"172.21.15.148:6829","nonce":2093868673}]},"public_addr":"172.21.15.148:6825/2093868673","cluster_addr":"172.21.15.148:6827/2093868673","heartbeat_back_addr":"172.21.15.148:6831/2093868673","heartbeat_front_addr":"172.21.15.148:6829/2093868673","state":["exists","up"]},{"osd":11,"uuid":"e2d6db2a-bd3d-4ed8-95fa-fe8918457114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6825","nonce":2619594507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6827","nonce":2619594507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6831","nonce":2619594507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2619594507},{"type":"v1","addr":"172.21.15.156:6829","nonce":2619594507}]},"public_addr":"172.21.15.156:6825/2619594507","cluster_addr":"172.21.15.156:6827/2619594507","heartbeat_back_addr":"172.21.15.156:6831/2619594507","heartbeat_front_addr":"172.21.15.156:6829/2619594507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:36.016137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:32:56.967882+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:17.313699+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:33:39.461375+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:00.605294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:21.268287+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:34:42.850597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:03.629362+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:24.833121+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:35:46.369657+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:07.540983+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T14:36:28.046632+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6801/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1334723693":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/1412452188":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6801/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:6800/2564784233":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/3422990986":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2379235225":"2023-10-01T14:30:46.025499+0000","172.21.15.32:0/2971157038":"2023-10-01T14:31:11.693545+0000","172.21.15.32:6800/1639474065":"2023-10-01T14:31:11.693545+0000","172.21.15.32:0/4045220015":"2023-10-01T14:30:46.025499+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T15:04:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.128+0000 7f6c18cde700 1 -- 172.21.15.32:0/3663185402 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c000a0ff0 msgr2=0x7f6c000a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.128+0000 7f6c18cde700 1 --2- 172.21.15.32:0/3663185402 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c000a0ff0 0x7f6c000a34b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc006010 tx=0x7f6bfc00b540 comp rx=0 tx=0).stop 2023-09-30T15:04:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.128+0000 7f6c18cde700 1 -- 172.21.15.32:0/3663185402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c140fff40 msgr2=0x7f6c141959d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.128+0000 7f6c18cde700 1 --2- 172.21.15.32:0/3663185402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c140fff40 0x7f6c141959d0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400d950 tx=0x7f6c0400dd10 comp rx=0 tx=0).stop 2023-09-30T15:04:11.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.128+0000 7f6c18cde700 1 -- 172.21.15.32:0/3663185402 shutdown_connections 2023-09-30T15:04:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.128+0000 7f6c18cde700 1 --2- 172.21.15.32:0/3663185402 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f6c000a0ff0 0x7f6c000a34b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.128+0000 7f6c18cde700 1 --2- 172.21.15.32:0/3663185402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c141012f0 0x7f6c1419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.128+0000 7f6c18cde700 1 --2- 172.21.15.32:0/3663185402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c140fff40 0x7f6c141959d0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.128+0000 7f6c18cde700 1 --2- 172.21.15.32:0/3663185402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c14100900 0x7f6c14195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.128+0000 7f6c18cde700 1 -- 172.21.15.32:0/3663185402 >> 172.21.15.32:0/3663185402 conn(0x7f6c140faa30 msgr2=0x7f6c140fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.128+0000 7f6c18cde700 1 -- 172.21.15.32:0/3663185402 shutdown_connections 2023-09-30T15:04:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.128+0000 7f6c18cde700 1 -- 172.21.15.32:0/3663185402 wait complete. 2023-09-30T15:04:11.137 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:10 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/482037629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T15:04:11.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-30T15:04:11.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.268+0000 7fc2ab6a4700 1 -- 172.21.15.32:0/501933994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2a41019f0 msgr2=0x7fc2a4103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:11.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.268+0000 7fc2ab6a4700 1 --2- 172.21.15.32:0/501933994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2a41019f0 0x7fc2a4103de0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009990 tx=0x7fc2a0009ca0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.268+0000 7fc2ab6a4700 1 -- 172.21.15.32:0/501933994 shutdown_connections 2023-09-30T15:04:11.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.268+0000 7fc2ab6a4700 1 --2- 172.21.15.32:0/501933994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2a4106c50 0x7fc2a4109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.268+0000 7fc2ab6a4700 1 --2- 172.21.15.32:0/501933994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2a41019f0 0x7fc2a4103de0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.268+0000 7fc2ab6a4700 1 --2- 172.21.15.32:0/501933994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a4104320 0x7fc2a4106710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.268+0000 7fc2ab6a4700 1 -- 172.21.15.32:0/501933994 >> 172.21.15.32:0/501933994 conn(0x7fc2a40fb340 msgr2=0x7fc2a40fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:11.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.268+0000 7fc2ab6a4700 1 -- 172.21.15.32:0/501933994 shutdown_connections 2023-09-30T15:04:11.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.268+0000 7fc2ab6a4700 1 -- 172.21.15.32:0/501933994 wait complete. 2023-09-30T15:04:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.269+0000 7fc2ab6a4700 1 Processor -- start 2023-09-30T15:04:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.269+0000 7fc2ab6a4700 1 -- start start 2023-09-30T15:04:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.269+0000 7fc2ab6a4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2a41019f0 0x7fc2a4101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.269+0000 7fc2ab6a4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a4104320 0x7fc2a40ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.269+0000 7fc2ab6a4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2a4106c50 0x7fc2a40ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.269+0000 7fc2ab6a4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a4100680 con 0x7fc2a4106c50 2023-09-30T15:04:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.269+0000 7fc2ab6a4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a41007f0 con 0x7fc2a4104320 2023-09-30T15:04:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.269+0000 7fc2ab6a4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a41a0bb0 con 0x7fc2a41019f0 2023-09-30T15:04:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.269+0000 7fc2a9c41700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2a4106c50 0x7fc2a40ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:11.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.269+0000 7fc2a9c41700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2a4106c50 0x7fc2a40ffee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40708/0 (socket says 172.21.15.32:40708) 2023-09-30T15:04:11.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.269+0000 7fc2a9c41700 1 -- 172.21.15.32:0/3652776598 learned_addr learned my addr 172.21.15.32:0/3652776598 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:04:11.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.269+0000 7fc2a8c3f700 1 --2- 172.21.15.32:0/3652776598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a4104320 0x7fc2a40ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:11.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.269+0000 7fc2a9440700 1 --2- 172.21.15.32:0/3652776598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2a41019f0 0x7fc2a4101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:11.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.270+0000 7fc2a9c41700 1 -- 172.21.15.32:0/3652776598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2a41019f0 msgr2=0x7fc2a4101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:11.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.270+0000 7fc2a9c41700 1 --2- 172.21.15.32:0/3652776598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2a41019f0 0x7fc2a4101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.270+0000 7fc2a9c41700 1 -- 172.21.15.32:0/3652776598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a4104320 msgr2=0x7fc2a40ff9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:11.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.270+0000 7fc2a9c41700 1 --2- 172.21.15.32:0/3652776598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a4104320 0x7fc2a40ff9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.270+0000 7fc2a9c41700 1 -- 172.21.15.32:0/3652776598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a00095f0 con 0x7fc2a4106c50 2023-09-30T15:04:11.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.270+0000 7fc2a9c41700 1 --2- 172.21.15.32:0/3652776598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2a4106c50 0x7fc2a40ffee0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fc29c007ee0 tx=0x7fc29c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:11.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.270+0000 7fc2967fc700 1 -- 172.21.15.32:0/3652776598 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c017070 con 0x7fc2a4106c50 2023-09-30T15:04:11.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.270+0000 7fc2967fc700 1 -- 172.21.15.32:0/3652776598 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc29c00f040 con 0x7fc2a4106c50 2023-09-30T15:04:11.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.270+0000 7fc2ab6a4700 1 -- 172.21.15.32:0/3652776598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a41a0e90 con 0x7fc2a4106c50 2023-09-30T15:04:11.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.270+0000 7fc2967fc700 1 -- 172.21.15.32:0/3652776598 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c005580 con 0x7fc2a4106c50 2023-09-30T15:04:11.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.270+0000 7fc2ab6a4700 1 -- 172.21.15.32:0/3652776598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a41a1310 con 0x7fc2a4106c50 2023-09-30T15:04:11.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.271+0000 7fc2ab6a4700 1 -- 172.21.15.32:0/3652776598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a40fcec0 con 0x7fc2a4106c50 2023-09-30T15:04:11.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.272+0000 7fc2967fc700 1 -- 172.21.15.32:0/3652776598 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fc29c00db20 con 0x7fc2a4106c50 2023-09-30T15:04:11.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.273+0000 7fc2967fc700 1 --2- 172.21.15.32:0/3652776598 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2900a1020 0x7fc2900a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:11.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.273+0000 7fc2a9440700 1 --2- 172.21.15.32:0/3652776598 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2900a1020 0x7fc2900a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:11.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.273+0000 7fc2967fc700 1 -- 172.21.15.32:0/3652776598 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc29c0cbe50 con 0x7fc2a4106c50 2023-09-30T15:04:11.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.273+0000 7fc2a9440700 1 --2- 172.21.15.32:0/3652776598 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2900a1020 0x7fc2900a34e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000b5c0 tx=0x7fc2a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:11.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.277+0000 7fc2967fc700 1 -- 172.21.15.32:0/3652776598 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fc29c095600 con 0x7fc2a4106c50 2023-09-30T15:04:11.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.525+0000 7fc2ab6a4700 1 -- 172.21.15.32:0/3652776598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc2a410c620 con 0x7fc2a4106c50 2023-09-30T15:04:11.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.528+0000 7fc2967fc700 1 -- 172.21.15.32:0/3652776598 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc29c098c20 con 0x7fc2a4106c50 2023-09-30T15:04:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.531+0000 7fc2ab6a4700 1 -- 172.21.15.32:0/3652776598 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2900a1020 msgr2=0x7fc2900a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.531+0000 7fc2ab6a4700 1 --2- 172.21.15.32:0/3652776598 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2900a1020 0x7fc2900a34e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000b5c0 tx=0x7fc2a0005fd0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.531+0000 7fc2ab6a4700 1 -- 172.21.15.32:0/3652776598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2a4106c50 msgr2=0x7fc2a40ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.531+0000 7fc2ab6a4700 1 --2- 172.21.15.32:0/3652776598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2a4106c50 0x7fc2a40ffee0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fc29c007ee0 tx=0x7fc29c00d3b0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.531+0000 7fc2ab6a4700 1 -- 172.21.15.32:0/3652776598 shutdown_connections 2023-09-30T15:04:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.531+0000 7fc2ab6a4700 1 --2- 172.21.15.32:0/3652776598 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fc2900a1020 0x7fc2900a34e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.531+0000 7fc2ab6a4700 1 --2- 172.21.15.32:0/3652776598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2a4106c50 0x7fc2a40ffee0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.531+0000 7fc2ab6a4700 1 --2- 172.21.15.32:0/3652776598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2a41019f0 0x7fc2a4101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.531+0000 7fc2ab6a4700 1 --2- 172.21.15.32:0/3652776598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2a4104320 0x7fc2a40ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.531+0000 7fc2ab6a4700 1 -- 172.21.15.32:0/3652776598 >> 172.21.15.32:0/3652776598 conn(0x7fc2a40fb340 msgr2=0x7fc2a4102660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.531+0000 7fc2ab6a4700 1 -- 172.21.15.32:0/3652776598 shutdown_connections 2023-09-30T15:04:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.531+0000 7fc2ab6a4700 1 -- 172.21.15.32:0/3652776598 wait complete. 2023-09-30T15:04:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-30T15:04:11.544 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-30T15:04:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.674+0000 7f958cf01700 1 -- 172.21.15.32:0/2364511670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9588102b60 msgr2=0x7f9588109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.674+0000 7f958cf01700 1 --2- 172.21.15.32:0/2364511670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9588102b60 0x7f9588109410 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f958000b0a0 tx=0x7f958000b3b0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.674+0000 7f958cf01700 1 -- 172.21.15.32:0/2364511670 shutdown_connections 2023-09-30T15:04:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.674+0000 7f958cf01700 1 --2- 172.21.15.32:0/2364511670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9588102b60 0x7f9588109410 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.674+0000 7f958cf01700 1 --2- 172.21.15.32:0/2364511670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95880ff150 0x7f95880ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.674+0000 7f958cf01700 1 --2- 172.21.15.32:0/2364511670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95881021c0 0x7f9588102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.674+0000 7f958cf01700 1 -- 172.21.15.32:0/2364511670 >> 172.21.15.32:0/2364511670 conn(0x7f95880faa80 msgr2=0x7f95880fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.674+0000 7f958cf01700 1 -- 172.21.15.32:0/2364511670 shutdown_connections 2023-09-30T15:04:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.674+0000 7f958cf01700 1 -- 172.21.15.32:0/2364511670 wait complete. 2023-09-30T15:04:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.675+0000 7f958cf01700 1 Processor -- start 2023-09-30T15:04:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.675+0000 7f958cf01700 1 -- start start 2023-09-30T15:04:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.675+0000 7f958cf01700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95880ff150 0x7f9588196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.675+0000 7f958cf01700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95881021c0 0x7f9588197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.675+0000 7f958cf01700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9588102b60 0x7f958819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.675+0000 7f958cf01700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958819e780 con 0x7f95881021c0 2023-09-30T15:04:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.675+0000 7f958cf01700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958819e8f0 con 0x7f9588102b60 2023-09-30T15:04:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.675+0000 7f958cf01700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958819eab0 con 0x7f95880ff150 2023-09-30T15:04:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.675+0000 7f9585d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95881021c0 0x7f9588197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.675+0000 7f9585d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95881021c0 0x7f9588197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40720/0 (socket says 172.21.15.32:40720) 2023-09-30T15:04:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.675+0000 7f9585d9b700 1 -- 172.21.15.32:0/2050908797 learned_addr learned my addr 172.21.15.32:0/2050908797 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:04:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.675+0000 7f958659c700 1 --2- 172.21.15.32:0/2050908797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95880ff150 0x7f9588196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.675+0000 7f9586d9d700 1 --2- 172.21.15.32:0/2050908797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9588102b60 0x7f958819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.676+0000 7f9585d9b700 1 -- 172.21.15.32:0/2050908797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95880ff150 msgr2=0x7f9588196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.676+0000 7f9585d9b700 1 --2- 172.21.15.32:0/2050908797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95880ff150 0x7f9588196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.676+0000 7f9585d9b700 1 -- 172.21.15.32:0/2050908797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9588102b60 msgr2=0x7f958819e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.676+0000 7f9585d9b700 1 --2- 172.21.15.32:0/2050908797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9588102b60 0x7f958819e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.676+0000 7f9585d9b700 1 -- 172.21.15.32:0/2050908797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9580009d00 con 0x7f95881021c0 2023-09-30T15:04:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.676+0000 7f9586d9d700 1 --2- 172.21.15.32:0/2050908797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9588102b60 0x7f958819e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T15:04:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.676+0000 7f9585d9b700 1 --2- 172.21.15.32:0/2050908797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95881021c0 0x7f9588197180 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f957800e9a0 tx=0x7f957800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.676+0000 7f95777fe700 1 -- 172.21.15.32:0/2050908797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578009790 con 0x7f95881021c0 2023-09-30T15:04:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.676+0000 7f958cf01700 1 -- 172.21.15.32:0/2050908797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958819ed90 con 0x7f95881021c0 2023-09-30T15:04:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.676+0000 7f95777fe700 1 -- 172.21.15.32:0/2050908797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f957800cbd0 con 0x7f95881021c0 2023-09-30T15:04:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.676+0000 7f958cf01700 1 -- 172.21.15.32:0/2050908797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958819f210 con 0x7f95881021c0 2023-09-30T15:04:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.676+0000 7f95777fe700 1 -- 172.21.15.32:0/2050908797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578018620 con 0x7f95881021c0 2023-09-30T15:04:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.677+0000 7f958cf01700 1 -- 172.21.15.32:0/2050908797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958810c6c0 con 0x7f95881021c0 2023-09-30T15:04:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.678+0000 7f95777fe700 1 -- 172.21.15.32:0/2050908797 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7f957800cd40 con 0x7f95881021c0 2023-09-30T15:04:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.679+0000 7f95777fe700 1 --2- 172.21.15.32:0/2050908797 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f95700a0fa0 0x7f95700a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.679+0000 7f958659c700 1 --2- 172.21.15.32:0/2050908797 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f95700a0fa0 0x7f95700a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.679+0000 7f95777fe700 1 -- 172.21.15.32:0/2050908797 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f9578014070 con 0x7f95881021c0 2023-09-30T15:04:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.679+0000 7f958659c700 1 --2- 172.21.15.32:0/2050908797 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f95700a0fa0 0x7f95700a3460 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9588198110 tx=0x7f957c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.682+0000 7f95777fe700 1 -- 172.21.15.32:0/2050908797 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7f95780963f0 con 0x7f95881021c0 2023-09-30T15:04:11.859 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-e[137823]: ignoring --setuser ceph since I am not root 2023-09-30T15:04:11.859 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-e[137823]: ignoring --setgroup ceph since I am not root 2023-09-30T15:04:11.860 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-e[137823]: starting mds.e at 2023-09-30T15:04:11.860 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-b[138505]: ignoring --setuser ceph since I am not root 2023-09-30T15:04:11.860 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-b[138505]: ignoring --setgroup ceph since I am not root 2023-09-30T15:04:11.860 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-mds[138529]: -- 172.21.15.148:0/1711170463 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564875e971e0 con 0x564875e6d000 2023-09-30T15:04:11.860 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-b[138505]: starting mds.b at 2023-09-30T15:04:11.860 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-h[138164]: ignoring --setuser ceph since I am not root 2023-09-30T15:04:11.861 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-h[138164]: ignoring --setgroup ceph since I am not root 2023-09-30T15:04:11.861 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-mds[138188]: -- 172.21.15.148:0/2906981993 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563b357831e0 con 0x563b35759000 2023-09-30T15:04:11.861 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-h[138164]: starting mds.h at 2023-09-30T15:04:11.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-mon[102355]: pgmap v1064: 97 pgs: 97 active+clean; 472 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 2.3 MiB/s wr, 228 op/s 2023-09-30T15:04:11.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/376259138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T15:04:11.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3663185402' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T15:04:11.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3652776598' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-30T15:04:11.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-mon[102355]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-30T15:04:11.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-mon[102355]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-30T15:04:11.861 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-mon[102355]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-09-30T15:04:11.862 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/3652776598' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-30T15:04:11.862 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-mon[102355]: osdmap e86: 12 total, 12 up, 12 in 2023-09-30T15:04:11.862 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:11 smithi148 ceph-mon[102355]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-09-30T15:04:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:11 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-a[98568]: 2023-09-30T15:04:11.526+0000 7f49ad3ef700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-30T15:04:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:11 smithi032 ceph-mon[98592]: pgmap v1064: 97 pgs: 97 active+clean; 472 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 2.3 MiB/s wr, 228 op/s 2023-09-30T15:04:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:11 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/376259138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T15:04:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:11 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3663185402' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T15:04:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:11 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3652776598' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-30T15:04:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:11 smithi032 ceph-mon[98592]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-30T15:04:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:11 smithi032 ceph-mon[98592]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-30T15:04:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:11 smithi032 ceph-mon[98592]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-09-30T15:04:11.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:11 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/3652776598' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-30T15:04:11.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:11 smithi032 ceph-mon[98592]: osdmap e86: 12 total, 12 up, 12 in 2023-09-30T15:04:11.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:11 smithi032 ceph-mon[98592]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-09-30T15:04:11.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:11.933+0000 7f958cf01700 1 -- 172.21.15.32:0/2050908797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9588107e30 con 0x7f95881021c0 2023-09-30T15:04:12.055 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-i[140626]: ignoring --setuser ceph since I am not root 2023-09-30T15:04:12.055 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-i[140626]: ignoring --setgroup ceph since I am not root 2023-09-30T15:04:12.055 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-i[140626]: starting mds.i at 2023-09-30T15:04:12.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-mon[103579]: pgmap v1064: 97 pgs: 97 active+clean; 472 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 2.3 MiB/s wr, 228 op/s 2023-09-30T15:04:12.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/376259138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-30T15:04:12.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3663185402' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-30T15:04:12.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3652776598' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-30T15:04:12.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-mon[103579]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-30T15:04:12.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-mon[103579]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-30T15:04:12.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-mon[103579]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-09-30T15:04:12.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/3652776598' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-30T15:04:12.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-mon[103579]: osdmap e86: 12 total, 12 up, 12 in 2023-09-30T15:04:12.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-mon[103579]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-09-30T15:04:12.057 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-c[140290]: ignoring --setuser ceph since I am not root 2023-09-30T15:04:12.057 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-c[140290]: ignoring --setgroup ceph since I am not root 2023-09-30T15:04:12.057 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-mds[140314]: -- 172.21.15.156:0/2158614964 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563e88c8b1e0 con 0x563e88c61000 2023-09-30T15:04:12.057 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-c[140290]: starting mds.c at 2023-09-30T15:04:12.057 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-f[139946]: ignoring --setuser ceph since I am not root 2023-09-30T15:04:12.058 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-f[139946]: ignoring --setgroup ceph since I am not root 2023-09-30T15:04:12.058 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-mds[139970]: -- 172.21.15.156:0/318743033 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cee0fff1e0 con 0x55cee0fd5000 2023-09-30T15:04:12.058 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 30 15:04:11 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-f[139946]: starting mds.f at 2023-09-30T15:04:12.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.529+0000 7f95777fe700 1 -- 172.21.15.32:0/2050908797 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9578099a10 con 0x7f95881021c0 2023-09-30T15:04:12.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.533+0000 7f958cf01700 1 -- 172.21.15.32:0/2050908797 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f95700a0fa0 msgr2=0x7f95700a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:12.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.533+0000 7f958cf01700 1 --2- 172.21.15.32:0/2050908797 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f95700a0fa0 0x7f95700a3460 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9588198110 tx=0x7f957c007400 comp rx=0 tx=0).stop 2023-09-30T15:04:12.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.533+0000 7f958cf01700 1 -- 172.21.15.32:0/2050908797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95881021c0 msgr2=0x7f9588197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:12.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.533+0000 7f958cf01700 1 --2- 172.21.15.32:0/2050908797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95881021c0 0x7f9588197180 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f957800e9a0 tx=0x7f957800ecb0 comp rx=0 tx=0).stop 2023-09-30T15:04:12.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.533+0000 7f958cf01700 1 -- 172.21.15.32:0/2050908797 shutdown_connections 2023-09-30T15:04:12.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.533+0000 7f958cf01700 1 --2- 172.21.15.32:0/2050908797 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7f95700a0fa0 0x7f95700a3460 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:12.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.533+0000 7f958cf01700 1 --2- 172.21.15.32:0/2050908797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9588102b60 0x7f958819e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:12.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.533+0000 7f958cf01700 1 --2- 172.21.15.32:0/2050908797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95880ff150 0x7f9588196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:12.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.533+0000 7f958cf01700 1 --2- 172.21.15.32:0/2050908797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95881021c0 0x7f9588197180 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:12.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.533+0000 7f958cf01700 1 -- 172.21.15.32:0/2050908797 >> 172.21.15.32:0/2050908797 conn(0x7f95880faa80 msgr2=0x7f95880fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:12.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.533+0000 7f958cf01700 1 -- 172.21.15.32:0/2050908797 shutdown_connections 2023-09-30T15:04:12.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.533+0000 7f958cf01700 1 -- 172.21.15.32:0/2050908797 wait complete. 2023-09-30T15:04:12.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-30T15:04:12.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.686+0000 7fb3b6326700 1 -- 172.21.15.32:0/2104340774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b0100e60 msgr2=0x7fb3b0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.686+0000 7fb3b6326700 1 --2- 172.21.15.32:0/2104340774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b0100e60 0x7fb3b0101280 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0009990 tx=0x7fb3a0009ca0 comp rx=0 tx=0).stop 2023-09-30T15:04:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.686+0000 7fb3b6326700 1 -- 172.21.15.32:0/2104340774 shutdown_connections 2023-09-30T15:04:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.686+0000 7fb3b6326700 1 --2- 172.21.15.32:0/2104340774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b0102970 0x7fb3b0109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.686+0000 7fb3b6326700 1 --2- 172.21.15.32:0/2104340774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b0100e60 0x7fb3b0101280 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.686+0000 7fb3b6326700 1 --2- 172.21.15.32:0/2104340774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b0101fb0 0x7fb3b0102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.686+0000 7fb3b6326700 1 -- 172.21.15.32:0/2104340774 >> 172.21.15.32:0/2104340774 conn(0x7fb3b00fc3d0 msgr2=0x7fb3b00fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.686+0000 7fb3b6326700 1 -- 172.21.15.32:0/2104340774 shutdown_connections 2023-09-30T15:04:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.686+0000 7fb3b6326700 1 -- 172.21.15.32:0/2104340774 wait complete. 2023-09-30T15:04:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.686+0000 7fb3b6326700 1 Processor -- start 2023-09-30T15:04:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.686+0000 7fb3b6326700 1 -- start start 2023-09-30T15:04:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3b6326700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b0100e60 0x7fb3b0074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3b6326700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b0101fb0 0x7fb3b0073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3b6326700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b0102970 0x7fb3b0073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3b6326700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b0073ef0 con 0x7fb3b0100e60 2023-09-30T15:04:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3b6326700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b0074060 con 0x7fb3b0101fb0 2023-09-30T15:04:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3b6326700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b01a0b50 con 0x7fb3b0102970 2023-09-30T15:04:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3affff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b0100e60 0x7fb3b0074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3affff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b0100e60 0x7fb3b0074bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40732/0 (socket says 172.21.15.32:40732) 2023-09-30T15:04:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3affff700 1 -- 172.21.15.32:0/27452729 learned_addr learned my addr 172.21.15.32:0/27452729 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:04:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3af7fe700 1 --2- 172.21.15.32:0/27452729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b0101fb0 0x7fb3b0073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:12.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3affff700 1 -- 172.21.15.32:0/27452729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b0102970 msgr2=0x7fb3b0073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3b48c3700 1 --2- 172.21.15.32:0/27452729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b0102970 0x7fb3b0073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3affff700 1 --2- 172.21.15.32:0/27452729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b0102970 0x7fb3b0073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3affff700 1 -- 172.21.15.32:0/27452729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b0101fb0 msgr2=0x7fb3b0073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3affff700 1 --2- 172.21.15.32:0/27452729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b0101fb0 0x7fb3b0073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3affff700 1 -- 172.21.15.32:0/27452729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a00095f0 con 0x7fb3b0100e60 2023-09-30T15:04:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.687+0000 7fb3affff700 1 --2- 172.21.15.32:0/27452729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b0100e60 0x7fb3b0074bd0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0005f50 tx=0x7fb3a00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.688+0000 7fb3ad7fa700 1 -- 172.21.15.32:0/27452729 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a001d070 con 0x7fb3b0100e60 2023-09-30T15:04:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.688+0000 7fb3ad7fa700 1 -- 172.21.15.32:0/27452729 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3a000fbf0 con 0x7fb3b0100e60 2023-09-30T15:04:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.688+0000 7fb3b6326700 1 -- 172.21.15.32:0/27452729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b01a0dd0 con 0x7fb3b0100e60 2023-09-30T15:04:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.688+0000 7fb3ad7fa700 1 -- 172.21.15.32:0/27452729 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0017650 con 0x7fb3b0100e60 2023-09-30T15:04:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.688+0000 7fb3b6326700 1 -- 172.21.15.32:0/27452729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b01a11d0 con 0x7fb3b0100e60 2023-09-30T15:04:12.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.689+0000 7fb3b6326700 1 -- 172.21.15.32:0/27452729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b010c660 con 0x7fb3b0100e60 2023-09-30T15:04:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.690+0000 7fb3ad7fa700 1 -- 172.21.15.32:0/27452729 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fb3a0021410 con 0x7fb3b0100e60 2023-09-30T15:04:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.690+0000 7fb3ad7fa700 1 --2- 172.21.15.32:0/27452729 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb3980a0fa0 0x7fb3980a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:12.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.690+0000 7fb3af7fe700 1 --2- 172.21.15.32:0/27452729 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb3980a0fa0 0x7fb3980a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.690+0000 7fb3ad7fa700 1 -- 172.21.15.32:0/27452729 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fb3a00ce460 con 0x7fb3b0100e60 2023-09-30T15:04:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.691+0000 7fb3af7fe700 1 --2- 172.21.15.32:0/27452729 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb3980a0fa0 0x7fb3980a3460 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009c60 tx=0x7fb3a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:12.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.694+0000 7fb3ad7fa700 1 -- 172.21.15.32:0/27452729 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fb3a0097c00 con 0x7fb3b0100e60 2023-09-30T15:04:12.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2050908797' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-30T15:04:12.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-30T15:04:12.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-30T15:04:12.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: Cluster is now healthy 2023-09-30T15:04:12.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2050908797' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-30T15:04:12.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.156:6835/2347286166,v1:172.21.15.156:6839/2347286166] up:boot 2023-09-30T15:04:12.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.148:6838/3028117635,v1:172.21.15.148:6839/3028117635] up:boot 2023-09-30T15:04:12.888 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.148:6834/3204275935,v1:172.21.15.148:6835/3204275935] up:boot 2023-09-30T15:04:12.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.156:6834/3891641584,v1:172.21.15.156:6837/3891641584] up:boot 2023-09-30T15:04:12.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.156:6836/2892529932,v1:172.21.15.156:6838/2892529932] up:boot 2023-09-30T15:04:12.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: mds.? [v2:172.21.15.148:6836/363158510,v1:172.21.15.148:6837/363158510] up:boot 2023-09-30T15:04:12.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: fsmap 12 up:standby 2023-09-30T15:04:12.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-30T15:04:12.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-30T15:04:12.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-30T15:04:12.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-30T15:04:12.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-30T15:04:12.889 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:12 smithi032 ceph-mon[98592]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-30T15:04:12.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:12.901+0000 7fb3b6326700 1 -- 172.21.15.32:0/27452729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb3b0107d80 con 0x7fb3b0100e60 2023-09-30T15:04:13.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2050908797' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-30T15:04:13.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-30T15:04:13.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-30T15:04:13.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: Cluster is now healthy 2023-09-30T15:04:13.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2050908797' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-30T15:04:13.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.156:6835/2347286166,v1:172.21.15.156:6839/2347286166] up:boot 2023-09-30T15:04:13.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.148:6838/3028117635,v1:172.21.15.148:6839/3028117635] up:boot 2023-09-30T15:04:13.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.148:6834/3204275935,v1:172.21.15.148:6835/3204275935] up:boot 2023-09-30T15:04:13.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.156:6834/3891641584,v1:172.21.15.156:6837/3891641584] up:boot 2023-09-30T15:04:13.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.156:6836/2892529932,v1:172.21.15.156:6838/2892529932] up:boot 2023-09-30T15:04:13.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: mds.? [v2:172.21.15.148:6836/363158510,v1:172.21.15.148:6837/363158510] up:boot 2023-09-30T15:04:13.056 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: fsmap 12 up:standby 2023-09-30T15:04:13.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-30T15:04:13.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-30T15:04:13.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-30T15:04:13.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-30T15:04:13.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-30T15:04:13.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:12 smithi156 ceph-mon[103579]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-30T15:04:13.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2050908797' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-30T15:04:13.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-30T15:04:13.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-30T15:04:13.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: Cluster is now healthy 2023-09-30T15:04:13.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2050908797' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-30T15:04:13.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.156:6835/2347286166,v1:172.21.15.156:6839/2347286166] up:boot 2023-09-30T15:04:13.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.148:6838/3028117635,v1:172.21.15.148:6839/3028117635] up:boot 2023-09-30T15:04:13.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.148:6834/3204275935,v1:172.21.15.148:6835/3204275935] up:boot 2023-09-30T15:04:13.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.156:6834/3891641584,v1:172.21.15.156:6837/3891641584] up:boot 2023-09-30T15:04:13.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.156:6836/2892529932,v1:172.21.15.156:6838/2892529932] up:boot 2023-09-30T15:04:13.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: mds.? [v2:172.21.15.148:6836/363158510,v1:172.21.15.148:6837/363158510] up:boot 2023-09-30T15:04:13.110 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: fsmap 12 up:standby 2023-09-30T15:04:13.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-30T15:04:13.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-30T15:04:13.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-30T15:04:13.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-30T15:04:13.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-30T15:04:13.111 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:12 smithi148 ceph-mon[102355]: from='mgr.14152 172.21.15.32:0/2748450750' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-30T15:04:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.780+0000 7fb3ad7fa700 1 -- 172.21.15.32:0/27452729 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb3a009b220 con 0x7fb3b0100e60 2023-09-30T15:04:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.784+0000 7fb3b6326700 1 -- 172.21.15.32:0/27452729 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb3980a0fa0 msgr2=0x7fb3980a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.784+0000 7fb3b6326700 1 --2- 172.21.15.32:0/27452729 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb3980a0fa0 0x7fb3980a3460 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009c60 tx=0x7fb3a4009260 comp rx=0 tx=0).stop 2023-09-30T15:04:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.784+0000 7fb3b6326700 1 -- 172.21.15.32:0/27452729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b0100e60 msgr2=0x7fb3b0074bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.784+0000 7fb3b6326700 1 --2- 172.21.15.32:0/27452729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b0100e60 0x7fb3b0074bd0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0005f50 tx=0x7fb3a00057b0 comp rx=0 tx=0).stop 2023-09-30T15:04:13.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.785+0000 7fb3b6326700 1 -- 172.21.15.32:0/27452729 shutdown_connections 2023-09-30T15:04:13.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.785+0000 7fb3b6326700 1 --2- 172.21.15.32:0/27452729 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fb3980a0fa0 0x7fb3980a3460 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:13.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.785+0000 7fb3b6326700 1 --2- 172.21.15.32:0/27452729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b0102970 0x7fb3b0073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:13.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.785+0000 7fb3b6326700 1 --2- 172.21.15.32:0/27452729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b0100e60 0x7fb3b0074bd0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:13.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.785+0000 7fb3b6326700 1 --2- 172.21.15.32:0/27452729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b0101fb0 0x7fb3b0073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:13.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.785+0000 7fb3b6326700 1 -- 172.21.15.32:0/27452729 >> 172.21.15.32:0/27452729 conn(0x7fb3b00fc3d0 msgr2=0x7fb3b00fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:13.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.785+0000 7fb3b6326700 1 -- 172.21.15.32:0/27452729 shutdown_connections 2023-09-30T15:04:13.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.785+0000 7fb3b6326700 1 -- 172.21.15.32:0/27452729 wait complete. 2023-09-30T15:04:13.788 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-09-30T15:04:13.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-30T15:04:13.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.959+0000 7fce045fb700 1 -- 172.21.15.32:0/3294095310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdfc0ffd30 msgr2=0x7fcdfc100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:13.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.959+0000 7fce045fb700 1 --2- 172.21.15.32:0/3294095310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdfc0ffd30 0x7fcdfc100150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009990 tx=0x7fcdf8009ca0 comp rx=0 tx=0).stop 2023-09-30T15:04:13.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.959+0000 7fce045fb700 1 -- 172.21.15.32:0/3294095310 shutdown_connections 2023-09-30T15:04:13.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.959+0000 7fce045fb700 1 --2- 172.21.15.32:0/3294095310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdfc1018c0 0x7fcdfc108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:13.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.959+0000 7fce045fb700 1 --2- 172.21.15.32:0/3294095310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdfc0ffd30 0x7fcdfc100150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:13.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.959+0000 7fce045fb700 1 --2- 172.21.15.32:0/3294095310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdfc100f20 0x7fcdfc101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:13.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.959+0000 7fce045fb700 1 -- 172.21.15.32:0/3294095310 >> 172.21.15.32:0/3294095310 conn(0x7fcdfc0fb2a0 msgr2=0x7fcdfc0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:13.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.959+0000 7fce045fb700 1 -- 172.21.15.32:0/3294095310 shutdown_connections 2023-09-30T15:04:13.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.959+0000 7fce045fb700 1 -- 172.21.15.32:0/3294095310 wait complete. 2023-09-30T15:04:13.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.959+0000 7fce045fb700 1 Processor -- start 2023-09-30T15:04:13.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.959+0000 7fce045fb700 1 -- start start 2023-09-30T15:04:13.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce045fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdfc0ffd30 0x7fcdfc195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:13.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce045fb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdfc100f20 0x7fcdfc195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:13.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce045fb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdfc1018c0 0x7fcdfc19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:13.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce045fb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc19d5c0 con 0x7fcdfc0ffd30 2023-09-30T15:04:13.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce045fb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc19d700 con 0x7fcdfc100f20 2023-09-30T15:04:13.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce045fb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc19d890 con 0x7fcdfc1018c0 2023-09-30T15:04:13.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce02397700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdfc0ffd30 0x7fcdfc195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:13.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce02397700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdfc0ffd30 0x7fcdfc195a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40746/0 (socket says 172.21.15.32:40746) 2023-09-30T15:04:13.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce02397700 1 -- 172.21.15.32:0/2118350871 learned_addr learned my addr 172.21.15.32:0/2118350871 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T15:04:13.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce02b98700 1 --2- 172.21.15.32:0/2118350871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdfc1018c0 0x7fcdfc19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:13.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce01b96700 1 --2- 172.21.15.32:0/2118350871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdfc100f20 0x7fcdfc195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:13.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce02397700 1 -- 172.21.15.32:0/2118350871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdfc1018c0 msgr2=0x7fcdfc19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:13.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce02397700 1 --2- 172.21.15.32:0/2118350871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdfc1018c0 0x7fcdfc19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:13.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce02397700 1 -- 172.21.15.32:0/2118350871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdfc100f20 msgr2=0x7fcdfc195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:13.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce02397700 1 --2- 172.21.15.32:0/2118350871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdfc100f20 0x7fcdfc195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:13.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce02397700 1 -- 172.21.15.32:0/2118350871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf80095f0 con 0x7fcdfc0ffd30 2023-09-30T15:04:13.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce02397700 1 --2- 172.21.15.32:0/2118350871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdfc0ffd30 0x7fcdfc195a40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8004710 tx=0x7fcdf8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:13.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fcdf37fe700 1 -- 172.21.15.32:0/2118350871 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf801d070 con 0x7fcdfc0ffd30 2023-09-30T15:04:13.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fcdf37fe700 1 -- 172.21.15.32:0/2118350871 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdf8021410 con 0x7fcdfc0ffd30 2023-09-30T15:04:13.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fce045fb700 1 -- 172.21.15.32:0/2118350871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdfc067fb0 con 0x7fcdfc0ffd30 2023-09-30T15:04:13.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.960+0000 7fcdf37fe700 1 -- 172.21.15.32:0/2118350871 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8017650 con 0x7fcdfc0ffd30 2023-09-30T15:04:13.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.961+0000 7fce045fb700 1 -- 172.21.15.32:0/2118350871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc068460 con 0x7fcdfc0ffd30 2023-09-30T15:04:13.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.961+0000 7fce045fb700 1 -- 172.21.15.32:0/2118350871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdfc10b540 con 0x7fcdfc0ffd30 2023-09-30T15:04:13.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.963+0000 7fcdf37fe700 1 -- 172.21.15.32:0/2118350871 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 135742+0+0 (secure 0 0 0) 0x7fcdf8021580 con 0x7fcdfc0ffd30 2023-09-30T15:04:13.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.963+0000 7fcdf37fe700 1 --2- 172.21.15.32:0/2118350871 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcde80a0fa0 0x7fcde80a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T15:04:13.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.964+0000 7fce01b96700 1 --2- 172.21.15.32:0/2118350871 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcde80a0fa0 0x7fcde80a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T15:04:13.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.964+0000 7fcdf37fe700 1 -- 172.21.15.32:0/2118350871 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 7915+0+0 (secure 0 0 0) 0x7fcdf80cdbe0 con 0x7fcdfc0ffd30 2023-09-30T15:04:13.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.964+0000 7fce01b96700 1 --2- 172.21.15.32:0/2118350871 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcde80a0fa0 0x7fcde80a3460 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009c60 tx=0x7fcdec010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T15:04:13.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:13.968+0000 7fcdf37fe700 1 -- 172.21.15.32:0/2118350871 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179998 (secure 0 0 0) 0x7fcdf8097600 con 0x7fcdfc0ffd30 2023-09-30T15:04:14.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:13 smithi156 ceph-mon[103579]: pgmap v1066: 97 pgs: 97 active+clean; 472 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 318 KiB/s wr, 104 op/s 2023-09-30T15:04:14.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:13 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/27452729' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-30T15:04:14.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:13 smithi148 ceph-mon[102355]: pgmap v1066: 97 pgs: 97 active+clean; 472 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 318 KiB/s wr, 104 op/s 2023-09-30T15:04:14.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:13 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/27452729' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-30T15:04:14.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:13 smithi032 ceph-mon[98592]: pgmap v1066: 97 pgs: 97 active+clean; 472 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 318 KiB/s wr, 104 op/s 2023-09-30T15:04:14.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:13 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/27452729' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-30T15:04:14.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:14.229+0000 7fce045fb700 1 -- 172.21.15.32:0/2118350871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcdfc068930 con 0x7fcdfc0ffd30 2023-09-30T15:04:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:14.785+0000 7fcdf37fe700 1 -- 172.21.15.32:0/2118350871 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcdf809ac20 con 0x7fcdfc0ffd30 2023-09-30T15:04:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:14.794+0000 7fce045fb700 1 -- 172.21.15.32:0/2118350871 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcde80a0fa0 msgr2=0x7fcde80a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:14.794+0000 7fce045fb700 1 --2- 172.21.15.32:0/2118350871 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcde80a0fa0 0x7fcde80a3460 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009c60 tx=0x7fcdec010040 comp rx=0 tx=0).stop 2023-09-30T15:04:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:14.795+0000 7fce045fb700 1 -- 172.21.15.32:0/2118350871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdfc0ffd30 msgr2=0x7fcdfc195a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T15:04:14.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:14.795+0000 7fce045fb700 1 --2- 172.21.15.32:0/2118350871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdfc0ffd30 0x7fcdfc195a40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8004710 tx=0x7fcdf8009580 comp rx=0 tx=0).stop 2023-09-30T15:04:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:14.801+0000 7fce045fb700 1 -- 172.21.15.32:0/2118350871 shutdown_connections 2023-09-30T15:04:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:14.801+0000 7fce045fb700 1 --2- 172.21.15.32:0/2118350871 >> [v2:172.21.15.32:6800/3273633815,v1:172.21.15.32:6801/3273633815] conn(0x7fcde80a0fa0 0x7fcde80a3460 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:14.802+0000 7fce045fb700 1 --2- 172.21.15.32:0/2118350871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdfc1018c0 0x7fcdfc19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:14.802+0000 7fce045fb700 1 --2- 172.21.15.32:0/2118350871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdfc0ffd30 0x7fcdfc195a40 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:14.802+0000 7fce045fb700 1 --2- 172.21.15.32:0/2118350871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdfc100f20 0x7fcdfc195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T15:04:14.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:14.802+0000 7fce045fb700 1 -- 172.21.15.32:0/2118350871 >> 172.21.15.32:0/2118350871 conn(0x7fcdfc0fb2a0 msgr2=0x7fcdfc0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T15:04:14.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:14.813+0000 7fce045fb700 1 -- 172.21.15.32:0/2118350871 shutdown_connections 2023-09-30T15:04:14.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T15:04:14.814+0000 7fce045fb700 1 -- 172.21.15.32:0/2118350871 wait complete. 2023-09-30T15:04:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-09-30T15:04:14.853 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-30T15:04:14.945 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-30T15:04:14.982 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-30T15:04:15.027 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-09-30T15:04:15.027 DEBUG:teuthology.orchestra.run.smithi032:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-30T15:04:15.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:14 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/27452729' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-30T15:04:15.057 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:14 smithi156 ceph-mon[103579]: osdmap e87: 12 total, 12 up, 12 in 2023-09-30T15:04:15.058 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:14 smithi156 ceph-mon[103579]: from='client.? 172.21.15.32:0/2118350871' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-30T15:04:15.070 DEBUG:teuthology.orchestra.run.smithi148:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-30T15:04:15.086 DEBUG:teuthology.orchestra.run.smithi156:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-30T15:04:15.103 INFO:tasks.cephadm:Stopping all daemons... 2023-09-30T15:04:15.103 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-09-30T15:04:15.103 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mon.a 2023-09-30T15:04:15.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:14 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/27452729' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-30T15:04:15.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:14 smithi148 ceph-mon[102355]: osdmap e87: 12 total, 12 up, 12 in 2023-09-30T15:04:15.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:14 smithi148 ceph-mon[102355]: from='client.? 172.21.15.32:0/2118350871' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-30T15:04:15.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:14 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/27452729' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-30T15:04:15.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:14 smithi032 ceph-mon[98592]: osdmap e87: 12 total, 12 up, 12 in 2023-09-30T15:04:15.138 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:14 smithi032 ceph-mon[98592]: from='client.? 172.21.15.32:0/2118350871' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-30T15:04:15.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:15 smithi032 systemd[1]: Stopping Ceph mon.a for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:04:15.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:15 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-a[98568]: 2023-09-30T15:04:15.487+0000 7f49b2bfa700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:04:15.491 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:15 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-a[98568]: 2023-09-30T15:04:15.487+0000 7f49b2bfa700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-09-30T15:04:16.066 INFO:journalctl@ceph.mon.a.smithi032.stdout:Sep 30 15:04:15 smithi032 bash[177687]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-a 2023-09-30T15:04:16.073 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mon.a.service' 2023-09-30T15:04:16.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:04:16.169 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-09-30T15:04:16.169 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-09-30T15:04:16.170 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mon.b 2023-09-30T15:04:16.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:16 smithi148 systemd[1]: Stopping Ceph mon.b for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:04:17.062 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-b[102331]: 2023-09-30T15:04:16.641+0000 7f18738e4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:04:17.062 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-b[102331]: 2023-09-30T15:04:16.641+0000 7f18738e4700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-09-30T15:04:17.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 30 15:04:17 smithi148 bash[147068]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-b 2023-09-30T15:04:17.328 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mon.b.service' 2023-09-30T15:04:17.387 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:04:17.388 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-09-30T15:04:17.388 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-09-30T15:04:17.388 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mon.c 2023-09-30T15:04:17.803 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:17 smithi156 systemd[1]: Stopping Ceph mon.c for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:04:18.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:17 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-c[103535]: 2023-09-30T15:04:17.801+0000 7f9bbc935700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:04:18.055 INFO:journalctl@ceph.mon.c.smithi156.stdout:Sep 30 15:04:17 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mon-c[103535]: 2023-09-30T15:04:17.801+0000 7f9bbc935700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-09-30T15:04:18.341 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mon.c.service' 2023-09-30T15:04:18.391 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:04:18.391 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-09-30T15:04:18.391 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-09-30T15:04:18.391 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mgr.x 2023-09-30T15:04:18.750 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 15:04:18 smithi032 systemd[1]: Stopping Ceph mgr.x for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:04:18.750 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 15:04:18 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T15:04:18.747+0000 7faa37c48700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:04:18.750 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 15:04:18 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x[98835]: 2023-09-30T15:04:18.747+0000 7faa37c48700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-09-30T15:04:19.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mgr.x.service' 2023-09-30T15:04:19.324 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 15:04:19 smithi032 bash[177832]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-x 2023-09-30T15:04:19.324 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 15:04:19 smithi032 systemd[1]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mgr.x.service: Succeeded. 2023-09-30T15:04:19.324 INFO:journalctl@ceph.mgr.x.smithi032.stdout:Sep 30 15:04:19 smithi032 systemd[1]: Stopped Ceph mgr.x for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T15:04:19.337 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:04:19.337 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-09-30T15:04:19.337 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-09-30T15:04:19.337 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mgr.y 2023-09-30T15:04:19.794 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 15:04:19 smithi148 systemd[1]: Stopping Ceph mgr.y for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:04:20.047 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 15:04:19 smithi148 bash[147218]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-y 2023-09-30T15:04:20.230 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 30 15:04:20 smithi148 systemd[1]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-09-30T15:04:20.235 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mgr.y.service' 2023-09-30T15:04:20.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:04:20.271 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-09-30T15:04:20.271 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-09-30T15:04:20.271 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mgr.z 2023-09-30T15:04:20.555 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 15:04:20 smithi156 systemd[1]: Stopping Ceph mgr.z for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:04:20.978 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 15:04:20 smithi156 bash[150676]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mgr-z 2023-09-30T15:04:21.159 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Sep 30 15:04:20 smithi156 systemd[1]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-09-30T15:04:21.163 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mgr.z.service' 2023-09-30T15:04:21.199 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:04:21.199 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-09-30T15:04:21.199 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-09-30T15:04:21.199 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.0 2023-09-30T15:04:21.558 INFO:journalctl@ceph.osd.0.smithi032.stdout:Sep 30 15:04:21 smithi032 systemd[1]: Stopping Ceph osd.0 for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:04:21.887 INFO:journalctl@ceph.osd.0.smithi032.stdout:Sep 30 15:04:21 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-0[117351]: 2023-09-30T15:04:21.556+0000 7fc6fc3d3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:04:21.888 INFO:journalctl@ceph.osd.0.smithi032.stdout:Sep 30 15:04:21 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-0[117351]: 2023-09-30T15:04:21.556+0000 7fc6fc3d3700 -1 osd.0 88 *** Got signal Terminated *** 2023-09-30T15:04:21.888 INFO:journalctl@ceph.osd.0.smithi032.stdout:Sep 30 15:04:21 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-0[117351]: 2023-09-30T15:04:21.556+0000 7fc6fc3d3700 -1 osd.0 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-30T15:04:27.638 INFO:journalctl@ceph.osd.0.smithi032.stdout:Sep 30 15:04:27 smithi032 bash[177974]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-0 2023-09-30T15:04:28.801 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.0.service' 2023-09-30T15:04:28.846 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:04:28.846 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-09-30T15:04:28.846 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-09-30T15:04:28.846 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.1 2023-09-30T15:04:29.198 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 30 15:04:28 smithi148 systemd[1]: Stopping Ceph osd.1 for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:04:29.609 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 30 15:04:29 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-1[107315]: 2023-09-30T15:04:29.193+0000 7fb99d941700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:04:29.609 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 30 15:04:29 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-1[107315]: 2023-09-30T15:04:29.193+0000 7fb99d941700 -1 osd.1 88 *** Got signal Terminated *** 2023-09-30T15:04:29.609 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 30 15:04:29 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-1[107315]: 2023-09-30T15:04:29.193+0000 7fb99d941700 -1 osd.1 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-30T15:04:35.359 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 30 15:04:34 smithi148 bash[147369]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-1 2023-09-30T15:04:36.475 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.1.service' 2023-09-30T15:04:36.516 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:04:36.516 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-09-30T15:04:36.516 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-09-30T15:04:36.517 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.2 2023-09-30T15:04:36.805 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 30 15:04:36 smithi156 systemd[1]: Stopping Ceph osd.2 for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:04:37.305 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 30 15:04:36 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-2[109388]: 2023-09-30T15:04:36.872+0000 7f777725b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:04:37.305 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 30 15:04:36 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-2[109388]: 2023-09-30T15:04:36.872+0000 7f777725b700 -1 osd.2 88 *** Got signal Terminated *** 2023-09-30T15:04:37.305 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 30 15:04:36 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-2[109388]: 2023-09-30T15:04:36.872+0000 7f777725b700 -1 osd.2 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-30T15:04:43.055 INFO:journalctl@ceph.osd.2.smithi156.stdout:Sep 30 15:04:42 smithi156 bash[150831]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-2 2023-09-30T15:04:44.219 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.2.service' 2023-09-30T15:04:44.259 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:04:44.259 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-09-30T15:04:44.260 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-09-30T15:04:44.260 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.3 2023-09-30T15:04:44.629 INFO:journalctl@ceph.osd.3.smithi032.stdout:Sep 30 15:04:44 smithi032 systemd[1]: Stopping Ceph osd.3 for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:04:44.887 INFO:journalctl@ceph.osd.3.smithi032.stdout:Sep 30 15:04:44 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-3[125952]: 2023-09-30T15:04:44.627+0000 7fd390d46700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:04:44.888 INFO:journalctl@ceph.osd.3.smithi032.stdout:Sep 30 15:04:44 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-3[125952]: 2023-09-30T15:04:44.627+0000 7fd390d46700 -1 osd.3 88 *** Got signal Terminated *** 2023-09-30T15:04:44.888 INFO:journalctl@ceph.osd.3.smithi032.stdout:Sep 30 15:04:44 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-3[125952]: 2023-09-30T15:04:44.627+0000 7fd390d46700 -1 osd.3 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-30T15:04:48.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:04:48 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:04:48.623+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:04:50.137 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:04:49 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:04:49.656+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:04:50.508 INFO:journalctl@ceph.osd.3.smithi032.stdout:Sep 30 15:04:50 smithi032 bash[178255]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-3 2023-09-30T15:04:50.805 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:04:50 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:04:50.304+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:04:51.057 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:04:50 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:04:50.640+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:04:51.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:04:50 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:04:50.698+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:04:51.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:04:51 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:04:51.269+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:04:51.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:51 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:04:51.205+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:26.517385+0000 front 2023-09-30T15:04:26.517425+0000 (oldest deadline 2023-09-30T15:04:50.617633+0000) 2023-09-30T15:04:51.773 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:04:51 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:04:51.514+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:04:51.773 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:04:51 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:04:51.641+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:04:51.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.3.service' 2023-09-30T15:04:51.821 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:04:51.822 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-09-30T15:04:51.822 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-09-30T15:04:51.822 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.4 2023-09-30T15:04:52.109 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 30 15:04:51 smithi148 systemd[1]: Stopping Ceph osd.4 for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:04:52.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:04:51 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:04:51.663+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:04:52.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:04:52 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:04:52.284+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:04:52.556 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:52 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:04:52.230+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:26.517385+0000 front 2023-09-30T15:04:26.517425+0000 (oldest deadline 2023-09-30T15:04:50.617633+0000) 2023-09-30T15:04:52.609 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 30 15:04:52 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-4[115377]: 2023-09-30T15:04:52.166+0000 7fb9f9937700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:04:52.609 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 30 15:04:52 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-4[115377]: 2023-09-30T15:04:52.166+0000 7fb9f9937700 -1 osd.4 88 *** Got signal Terminated *** 2023-09-30T15:04:52.609 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 30 15:04:52 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-4[115377]: 2023-09-30T15:04:52.166+0000 7fb9f9937700 -1 osd.4 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-30T15:04:52.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:04:52 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:04:52.473+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:04:52.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:04:52 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:04:52.687+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:04:53.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:04:52 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:04:52.712+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:04:53.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:53 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:04:53.220+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:26.517385+0000 front 2023-09-30T15:04:26.517425+0000 (oldest deadline 2023-09-30T15:04:50.617633+0000) 2023-09-30T15:04:53.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:04:53 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:04:53.290+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:04:53.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:04:53 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:04:53.510+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:04:53.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:04:53 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:04:53.675+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:04:54.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:04:53 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:04:53.694+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:04:54.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:04:54 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:04:54.263+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:04:54.556 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:54 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:04:54.268+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:26.517385+0000 front 2023-09-30T15:04:26.517425+0000 (oldest deadline 2023-09-30T15:04:50.617633+0000) 2023-09-30T15:04:54.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:04:54 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:04:54.479+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:04:54.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:04:54 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:04:54.667+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:04:55.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:04:54 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:04:54.685+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:04:55.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:55 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:04:55.292+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:26.517385+0000 front 2023-09-30T15:04:26.517425+0000 (oldest deadline 2023-09-30T15:04:50.617633+0000) 2023-09-30T15:04:55.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:04:55 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:04:55.271+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:04:55.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:04:55 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:04:55.271+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:04:55.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:04:55 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:04:55.475+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:04:55.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:04:55 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:04:55.628+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:04:56.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:04:55 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:04:55.732+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:04:56.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:04:55 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:04:55.732+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:04:56.256 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:04:55 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:04:55.980+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:04:56.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:56 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:04:56.296+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:26.517385+0000 front 2023-09-30T15:04:26.517425+0000 (oldest deadline 2023-09-30T15:04:50.617633+0000) 2023-09-30T15:04:56.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:56 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:04:56.296+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.618151+0000 front 2023-09-30T15:04:30.618164+0000 (oldest deadline 2023-09-30T15:04:55.918278+0000) 2023-09-30T15:04:56.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:04:56 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:04:56.256+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:04:56.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:04:56 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:04:56.256+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:04:56.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:04:56 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:04:56.519+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:04:56.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:04:56 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:04:56.583+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:04:57.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:04:56 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:04:56.690+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:04:57.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:04:56 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:04:56.690+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:04:57.291 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:04:56 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:04:56.989+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:04:57.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:57 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:04:57.291+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:26.517385+0000 front 2023-09-30T15:04:26.517425+0000 (oldest deadline 2023-09-30T15:04:50.617633+0000) 2023-09-30T15:04:57.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:57 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:04:57.291+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.618151+0000 front 2023-09-30T15:04:30.618164+0000 (oldest deadline 2023-09-30T15:04:55.918278+0000) 2023-09-30T15:04:57.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:04:57 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:04:57.291+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:04:57.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:04:57 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:04:57.291+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:04:57.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:04:57 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:04:57.526+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:04:57.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:04:57 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:04:57.612+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:04:57.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:04:57 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:04:57.612+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:04:58.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:04:57 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:04:57.719+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:04:58.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:04:57 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:04:57.719+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:04:58.110 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 30 15:04:57 smithi148 bash[147647]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-4 2023-09-30T15:04:58.257 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:04:57 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:04:57.949+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:04:58.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:58 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:04:58.257+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:26.517385+0000 front 2023-09-30T15:04:26.517425+0000 (oldest deadline 2023-09-30T15:04:50.617633+0000) 2023-09-30T15:04:58.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:58 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:04:58.257+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.618151+0000 front 2023-09-30T15:04:30.618164+0000 (oldest deadline 2023-09-30T15:04:55.918278+0000) 2023-09-30T15:04:58.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:04:58 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:04:58.300+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:04:58.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:04:58 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:04:58.300+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:04:58.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:04:58 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:04:58.511+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:04:58.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:04:58 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:04:58.511+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.496894+0000 front 2023-09-30T15:04:33.496849+0000 (oldest deadline 2023-09-30T15:04:57.596906+0000) 2023-09-30T15:04:58.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:04:58 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:04:58.586+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:04:58.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:04:58 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:04:58.586+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:04:59.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:04:58 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:04:58.756+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:04:59.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:04:58 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:04:58.756+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:04:59.125 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.4.service' 2023-09-30T15:04:59.163 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:04:59.163 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-09-30T15:04:59.163 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-09-30T15:04:59.164 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.5 2023-09-30T15:04:59.245 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:59 smithi156 systemd[1]: Stopping Ceph osd.5 for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:04:59.246 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:04:58 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:04:58.967+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:04:59.536 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:59 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:04:59.244+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:26.517385+0000 front 2023-09-30T15:04:26.517425+0000 (oldest deadline 2023-09-30T15:04:50.617633+0000) 2023-09-30T15:04:59.537 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:59 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:04:59.244+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.618151+0000 front 2023-09-30T15:04:30.618164+0000 (oldest deadline 2023-09-30T15:04:55.918278+0000) 2023-09-30T15:04:59.537 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:04:59 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:04:59.256+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:04:59.537 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:04:59 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:04:59.256+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:04:59.805 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:59 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:04:59.536+0000 7efcdd9ed700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:04:59.805 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:59 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:04:59.536+0000 7efcdd9ed700 -1 osd.5 88 *** Got signal Terminated *** 2023-09-30T15:04:59.805 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:04:59 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:04:59.536+0000 7efcdd9ed700 -1 osd.5 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-30T15:04:59.891 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:04:59 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:04:59.502+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:04:59.891 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:04:59 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:04:59.502+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.496894+0000 front 2023-09-30T15:04:33.496849+0000 (oldest deadline 2023-09-30T15:04:57.596906+0000) 2023-09-30T15:04:59.891 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:04:59 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:04:59.572+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:04:59.892 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:04:59 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:04:59.572+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:00.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:04:59 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:04:59.800+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:00.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:04:59 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:04:59.800+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:00.110 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:00 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:00.012+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:00.210 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:04:59 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:04:59.926+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:00.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:05:00 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:05:00.210+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:26.517385+0000 front 2023-09-30T15:04:26.517425+0000 (oldest deadline 2023-09-30T15:04:50.617633+0000) 2023-09-30T15:05:00.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:05:00 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:05:00.210+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.618151+0000 front 2023-09-30T15:04:30.618164+0000 (oldest deadline 2023-09-30T15:04:55.918278+0000) 2023-09-30T15:05:00.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:00 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:00.288+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:00.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:00 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:00.288+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:00.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:00 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:00.461+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:05:00.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:00 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:00.461+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.496894+0000 front 2023-09-30T15:04:33.496849+0000 (oldest deadline 2023-09-30T15:04:57.596906+0000) 2023-09-30T15:05:00.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:00 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:00.598+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:00.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:00 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:00.598+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:01.109 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:01 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:01.048+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:01.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:00 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:00.805+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:01.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:00 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:00.805+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:01.223 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:00 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:00.950+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:01.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:05:01 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:05:01.223+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:26.517385+0000 front 2023-09-30T15:04:26.517425+0000 (oldest deadline 2023-09-30T15:04:50.617633+0000) 2023-09-30T15:05:01.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:05:01 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:05:01.223+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.618151+0000 front 2023-09-30T15:04:30.618164+0000 (oldest deadline 2023-09-30T15:04:55.918278+0000) 2023-09-30T15:05:01.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:01 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:01.285+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:01.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:01 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:01.285+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:01.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:01 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:01.435+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:05:01.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:01 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:01.435+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.496894+0000 front 2023-09-30T15:04:33.496849+0000 (oldest deadline 2023-09-30T15:04:57.596906+0000) 2023-09-30T15:05:01.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:01 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:01.627+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:01.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:01 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:01.627+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:02.092 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:01 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:01.775+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:02.092 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:01 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:01.775+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:02.260 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:01 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:01.945+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:02.359 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:02 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:02.090+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:02.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:05:02 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:05:02.260+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:26.517385+0000 front 2023-09-30T15:04:26.517425+0000 (oldest deadline 2023-09-30T15:04:50.617633+0000) 2023-09-30T15:05:02.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:05:02 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:05:02.260+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.618151+0000 front 2023-09-30T15:04:30.618164+0000 (oldest deadline 2023-09-30T15:04:55.918278+0000) 2023-09-30T15:05:02.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:02 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:02.283+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:02.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:02 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:02.283+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:02.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:02 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:02.454+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:05:02.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:02 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:02.454+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.496894+0000 front 2023-09-30T15:04:33.496849+0000 (oldest deadline 2023-09-30T15:04:57.596906+0000) 2023-09-30T15:05:02.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:02 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:02.647+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:02.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:02 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:02.647+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:03.100 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:02 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:02.756+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:03.100 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:02 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:02.756+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:03.223 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:02 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:02.948+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:03.359 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:03 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:03.099+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:03.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:05:03 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:05:03.222+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:26.517385+0000 front 2023-09-30T15:04:26.517425+0000 (oldest deadline 2023-09-30T15:04:50.617633+0000) 2023-09-30T15:05:03.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:05:03 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:05:03.222+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.618151+0000 front 2023-09-30T15:04:30.618164+0000 (oldest deadline 2023-09-30T15:04:55.918278+0000) 2023-09-30T15:05:03.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:03 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:03.240+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:03.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:03 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:03.240+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:03.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:03 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:03.446+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:05:03.889 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:03 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:03.446+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.496894+0000 front 2023-09-30T15:04:33.496849+0000 (oldest deadline 2023-09-30T15:04:57.596906+0000) 2023-09-30T15:05:03.889 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:03 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:03.686+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:03.889 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:03 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:03.686+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:03.889 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:03 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:03.686+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:04.072 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:03 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:03.742+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:04.072 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:03 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:03.742+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:04.206 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:03 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:03.935+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:04.207 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:05:04 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:05:04.182+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:26.517385+0000 front 2023-09-30T15:04:26.517425+0000 (oldest deadline 2023-09-30T15:04:50.617633+0000) 2023-09-30T15:05:04.207 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:05:04 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:05:04.182+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.618151+0000 front 2023-09-30T15:04:30.618164+0000 (oldest deadline 2023-09-30T15:04:55.918278+0000) 2023-09-30T15:05:04.207 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:05:04 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5[117319]: 2023-09-30T15:05:04.182+0000 7efcd87e0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.120067+0000 front 2023-09-30T15:04:41.120098+0000 (oldest deadline 2023-09-30T15:05:03.420225+0000) 2023-09-30T15:05:04.359 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:04 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:04.071+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:04.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:04 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:04.206+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:04.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:04 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:04.206+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:04.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:04 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:04.495+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:05:04.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:04 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:04.495+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.496894+0000 front 2023-09-30T15:04:33.496849+0000 (oldest deadline 2023-09-30T15:04:57.596906+0000) 2023-09-30T15:05:04.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:04 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:04.702+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:04.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:04 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:04.702+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:04.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:04 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:04.702+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:05.038 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:04 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:04.720+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:05.039 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:04 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:04.720+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:05.213 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:04 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:04.887+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:05.213 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:04 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:04.887+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:05.359 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:05 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:05.037+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:05.359 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:05 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:05.037+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.374827+0000 front 2023-09-30T15:04:39.374684+0000 (oldest deadline 2023-09-30T15:05:04.074661+0000) 2023-09-30T15:05:05.555 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:05:05 smithi156 bash[151098]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-5 2023-09-30T15:05:05.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:05 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:05.213+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:05.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:05 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:05.213+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:05.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:05 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:05.538+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:05:05.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:05 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:05.538+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.496894+0000 front 2023-09-30T15:04:33.496849+0000 (oldest deadline 2023-09-30T15:04:57.596906+0000) 2023-09-30T15:05:05.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:05 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:05.700+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:05.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:05 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:05.700+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:05.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:05 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:05.700+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:06.079 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:05 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:05.731+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:06.079 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:05 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:05.731+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:06.186 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:05 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:05.881+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:06.186 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:05 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:05.881+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:06.186 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:06 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:06.185+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:06.359 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:06 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:06.078+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:06.359 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:06 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:06.078+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.374827+0000 front 2023-09-30T15:04:39.374684+0000 (oldest deadline 2023-09-30T15:05:04.074661+0000) 2023-09-30T15:05:06.518 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:06 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:06.185+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:06.518 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:06 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:06.185+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:06.766 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.5.service' 2023-09-30T15:05:06.792 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:05:06 smithi156 systemd[1]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.5.service: Succeeded. 2023-09-30T15:05:06.792 INFO:journalctl@ceph.osd.5.smithi156.stdout:Sep 30 15:05:06 smithi156 systemd[1]: Stopped Ceph osd.5 for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7. 2023-09-30T15:05:06.805 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:05:06.806 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-09-30T15:05:06.806 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-09-30T15:05:06.806 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.6 2023-09-30T15:05:06.838 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:06 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:06.586+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:05:06.839 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:06 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:06.586+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.496894+0000 front 2023-09-30T15:04:33.496849+0000 (oldest deadline 2023-09-30T15:04:57.596906+0000) 2023-09-30T15:05:06.839 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:06 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:06.681+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:06.839 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:06 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:06.681+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:06.839 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:06 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:06.681+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:07.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:06 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:06.831+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:07.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:06 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:06.831+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:07.070 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:06 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:06.743+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:07.071 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:06 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:06.743+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:07.071 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:06 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:06.743+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:07.138 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:06 smithi032 systemd[1]: Stopping Ceph osd.6 for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:05:07.359 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:07 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:07.070+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:07.359 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:07 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:07.070+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.374827+0000 front 2023-09-30T15:04:39.374684+0000 (oldest deadline 2023-09-30T15:05:04.074661+0000) 2023-09-30T15:05:07.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:07 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:07.206+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:07.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:07 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:07.206+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:07.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:07 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:07.206+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:07.605 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:07 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:07.167+0000 7fe341598700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:05:07.605 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:07 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:07.167+0000 7fe341598700 -1 osd.6 88 *** Got signal Terminated *** 2023-09-30T15:05:07.605 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:07 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:07.167+0000 7fe341598700 -1 osd.6 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-30T15:05:07.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:07 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:07.604+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:05:07.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:07 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:07.604+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.496894+0000 front 2023-09-30T15:04:33.496849+0000 (oldest deadline 2023-09-30T15:04:57.596906+0000) 2023-09-30T15:05:07.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:07 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:07.604+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.400449+0000 front 2023-09-30T15:04:41.400224+0000 (oldest deadline 2023-09-30T15:05:06.700274+0000) 2023-09-30T15:05:07.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:07 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:07.644+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:07.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:07 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:07.644+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:07.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:07 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:07.644+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:08.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:07 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:07.800+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:08.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:07 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:07.800+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:08.081 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:07 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:07.747+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:08.081 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:07 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:07.747+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:08.081 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:07 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:07.747+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:08.359 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:08 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:08.079+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:08.359 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:08 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:08.079+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.374827+0000 front 2023-09-30T15:04:39.374684+0000 (oldest deadline 2023-09-30T15:05:04.074661+0000) 2023-09-30T15:05:08.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:08 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:08.221+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:08.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:08 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:08.221+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:08.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:08 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:08.221+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:08.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:08 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:08.569+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:05:08.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:08 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:08.569+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.496894+0000 front 2023-09-30T15:04:33.496849+0000 (oldest deadline 2023-09-30T15:04:57.596906+0000) 2023-09-30T15:05:08.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:08 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:08.569+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.400449+0000 front 2023-09-30T15:04:41.400224+0000 (oldest deadline 2023-09-30T15:05:06.700274+0000) 2023-09-30T15:05:08.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:08 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:08.610+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:08.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:08 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:08.610+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:08.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:08 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:08.610+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:09.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:08 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:08.774+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:09.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:08 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:08.774+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:09.080 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:08 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:08.725+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:09.080 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:08 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:08.725+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:09.080 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:08 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:08.726+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:09.359 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:09 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:09.079+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:09.359 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:09 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:09.079+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.374827+0000 front 2023-09-30T15:04:39.374684+0000 (oldest deadline 2023-09-30T15:05:04.074661+0000) 2023-09-30T15:05:09.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:09 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:09.254+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:09.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:09 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:09.254+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:09.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:09 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:09.254+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:09.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:09 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:09.576+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:05:09.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:09 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:09.576+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.496894+0000 front 2023-09-30T15:04:33.496849+0000 (oldest deadline 2023-09-30T15:04:57.596906+0000) 2023-09-30T15:05:09.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:09 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:09.576+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.400449+0000 front 2023-09-30T15:04:41.400224+0000 (oldest deadline 2023-09-30T15:05:06.700274+0000) 2023-09-30T15:05:09.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:09 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:09.630+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:09.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:09 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:09.630+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:09.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:09 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:09.630+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:10.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:09 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:09.803+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:10.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:09 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:09.803+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:10.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:09 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:09.749+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:10.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:09 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:09.749+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:10.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:09 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:09.749+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:10.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:10 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:10.300+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:10.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:10 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:10.300+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:10.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:10 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:10.300+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:10.609 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:10 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:10.118+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:10.609 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:10 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:10.118+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.374827+0000 front 2023-09-30T15:04:39.374684+0000 (oldest deadline 2023-09-30T15:05:04.074661+0000) 2023-09-30T15:05:10.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:10 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:10.623+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:05:10.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:10 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:10.623+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.496894+0000 front 2023-09-30T15:04:33.496849+0000 (oldest deadline 2023-09-30T15:04:57.596906+0000) 2023-09-30T15:05:10.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:10 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:10.623+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.400449+0000 front 2023-09-30T15:04:41.400224+0000 (oldest deadline 2023-09-30T15:05:06.700274+0000) 2023-09-30T15:05:10.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:10 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:10.609+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:10.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:10 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:10.609+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:10.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:10 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:10.609+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:11.070 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:10 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:10.737+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:11.071 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:10 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:10.737+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:11.071 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:10 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:10.737+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:11.299 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:10 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:10.822+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:11.300 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:10 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:10.822+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:11.361 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:11 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:11.070+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:11.361 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:11 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:11.070+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.374827+0000 front 2023-09-30T15:04:39.374684+0000 (oldest deadline 2023-09-30T15:05:04.074661+0000) 2023-09-30T15:05:11.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:11 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:11.299+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:11.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:11 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:11.299+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:11.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:11 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:11.299+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:11.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:11 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:11.593+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:11.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:11 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:11.593+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:11.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:11 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:11.593+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:11.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:11 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:11.615+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.394539+0000 front 2023-09-30T15:04:25.394573+0000 (oldest deadline 2023-09-30T15:04:50.694655+0000) 2023-09-30T15:05:11.888 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:11 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:11.615+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.496894+0000 front 2023-09-30T15:04:33.496849+0000 (oldest deadline 2023-09-30T15:04:57.596906+0000) 2023-09-30T15:05:11.889 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:11 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:11.615+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.400449+0000 front 2023-09-30T15:04:41.400224+0000 (oldest deadline 2023-09-30T15:05:06.700274+0000) 2023-09-30T15:05:11.889 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:11 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6[133902]: 2023-09-30T15:05:11.615+0000 7fe33c38b700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.700778+0000 front 2023-09-30T15:04:46.700750+0000 (oldest deadline 2023-09-30T15:05:10.800874+0000) 2023-09-30T15:05:12.104 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:11 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:11.719+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:12.104 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:11 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:11.719+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:12.105 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:11 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:11.719+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:12.105 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:11 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:11.719+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:12.305 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:11 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:11.832+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:12.305 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:11 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:11.832+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:12.359 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:12 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:12.103+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:12.359 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:12 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:12.103+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.374827+0000 front 2023-09-30T15:04:39.374684+0000 (oldest deadline 2023-09-30T15:05:04.074661+0000) 2023-09-30T15:05:12.805 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:12 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:12.307+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:12.805 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:12 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:12.307+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:12.805 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:12 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:12.307+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:12.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:12 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:12.627+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:12.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:12 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:12.627+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:12.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:12 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:12.627+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:12.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:12 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:12.627+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:13.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:12 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:12.747+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:13.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:12 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:12.747+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:13.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:12 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:12.747+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:13.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:12 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:12.747+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:13.305 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:12 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:12.806+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:13.305 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:12 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:12.806+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:13.387 INFO:journalctl@ceph.osd.6.smithi032.stdout:Sep 30 15:05:13 smithi032 bash[178519]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-6 2023-09-30T15:05:13.609 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:13 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:13.146+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:13.609 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:13 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:13.146+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.374827+0000 front 2023-09-30T15:04:39.374684+0000 (oldest deadline 2023-09-30T15:05:04.074661+0000) 2023-09-30T15:05:13.797 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:13 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:13.315+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:13.797 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:13 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:13.315+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:13.797 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:13 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:13.315+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:14.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:13 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:13.796+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:14.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:13 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:13.796+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:14.104 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:13 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:13.664+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:14.104 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:13 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:13.664+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:14.105 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:13 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:13.664+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:14.105 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:13 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:13.664+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:14.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:13 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:13.774+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:14.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:13 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:13.774+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:14.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:13 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:13.774+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:14.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:13 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:13.774+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:14.426 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.6.service' 2023-09-30T15:05:14.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:05:14.509 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-09-30T15:05:14.509 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-09-30T15:05:14.510 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.7 2023-09-30T15:05:14.539 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:14 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:14.179+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:14.539 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:14 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:14.179+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.374827+0000 front 2023-09-30T15:04:39.374684+0000 (oldest deadline 2023-09-30T15:05:04.074661+0000) 2023-09-30T15:05:14.539 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:14 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:14.179+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.776204+0000 front 2023-09-30T15:04:48.775984+0000 (oldest deadline 2023-09-30T15:05:13.476011+0000) 2023-09-30T15:05:14.773 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:14 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:14.318+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:14.773 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:14 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:14.318+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:14.773 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:14 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:14.318+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:14.854 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:14 smithi148 systemd[1]: Stopping Ceph osd.7 for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:05:14.854 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:14 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:14.756+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:14.855 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:14 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:14.756+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:14.855 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:14 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:14.756+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:14.855 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:14 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:14.756+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:15.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:14 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:14.773+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:15.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:14 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:14.773+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:15.056 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:14 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:14.773+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.677095+0000 front 2023-09-30T15:04:48.677082+0000 (oldest deadline 2023-09-30T15:05:14.577134+0000) 2023-09-30T15:05:15.109 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:14 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:14.853+0000 7fcc90d2e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:05:15.109 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:14 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:14.853+0000 7fcc90d2e700 -1 osd.7 88 *** Got signal Terminated *** 2023-09-30T15:05:15.109 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:14 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:14.853+0000 7fcc90d2e700 -1 osd.7 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-30T15:05:15.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:14 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:14.685+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:15.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:14 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:14.685+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:15.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:14 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:14.685+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:15.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:14 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:14.685+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:15.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:15 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:15.299+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:15.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:15 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:15.299+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:15.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:15 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:15.299+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:15.609 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:15 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:15.161+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:15.609 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:15 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:15.161+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.374827+0000 front 2023-09-30T15:04:39.374684+0000 (oldest deadline 2023-09-30T15:05:04.074661+0000) 2023-09-30T15:05:15.609 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:15 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:15.161+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.776204+0000 front 2023-09-30T15:04:48.775984+0000 (oldest deadline 2023-09-30T15:05:13.476011+0000) 2023-09-30T15:05:16.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:15 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:15.771+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:16.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:15 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:15.771+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:16.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:15 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:15.771+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.677095+0000 front 2023-09-30T15:04:48.677082+0000 (oldest deadline 2023-09-30T15:05:14.577134+0000) 2023-09-30T15:05:16.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:15 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:15.722+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:16.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:15 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:15.722+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:16.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:15 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:15.722+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:16.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:15 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:15.722+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:16.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:15 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:15.639+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:16.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:15 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:15.639+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:16.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:15 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:15.639+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:16.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:15 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:15.639+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:16.609 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:16.148+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:16.610 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:16.148+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.374827+0000 front 2023-09-30T15:04:39.374684+0000 (oldest deadline 2023-09-30T15:05:04.074661+0000) 2023-09-30T15:05:16.610 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:16.148+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.776204+0000 front 2023-09-30T15:04:48.775984+0000 (oldest deadline 2023-09-30T15:05:13.476011+0000) 2023-09-30T15:05:16.756 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:16 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:16.310+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:16.757 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:16 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:16.310+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:16.757 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:16 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:16.310+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:16.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:16 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:16.606+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:16.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:16 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:16.606+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:16.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:16 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:16.606+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:16.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:16 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:16.606+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:17.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:16 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:16.756+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:17.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:16 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:16.756+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:17.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:16 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:16.756+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.677095+0000 front 2023-09-30T15:04:48.677082+0000 (oldest deadline 2023-09-30T15:05:14.577134+0000) 2023-09-30T15:05:17.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:16.764+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:17.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:16.764+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:17.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:16.764+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:17.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:16.764+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:17.610 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:17 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:17.190+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:17.610 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:17 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:17.190+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.374827+0000 front 2023-09-30T15:04:39.374684+0000 (oldest deadline 2023-09-30T15:05:04.074661+0000) 2023-09-30T15:05:17.610 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:17 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:17.190+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.776204+0000 front 2023-09-30T15:04:48.775984+0000 (oldest deadline 2023-09-30T15:05:13.476011+0000) 2023-09-30T15:05:17.793 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:17 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:17.334+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:17.794 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:17 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:17.334+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:17.794 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:17 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:17.334+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:17.887 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:17 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:17.629+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:17.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:17 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:17.629+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:17.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:17 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:17.629+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:17.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:17 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:17.629+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:18.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:17 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:17.793+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:18.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:17 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:17.793+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:18.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:17 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:17.793+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.677095+0000 front 2023-09-30T15:04:48.677082+0000 (oldest deadline 2023-09-30T15:05:14.577134+0000) 2023-09-30T15:05:18.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:17 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:17.723+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:18.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:17 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:17.723+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:18.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:17 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:17.723+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:18.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:17 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:17.723+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:18.609 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:18 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:18.232+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:18.609 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:18 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:18.232+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.374827+0000 front 2023-09-30T15:04:39.374684+0000 (oldest deadline 2023-09-30T15:05:04.074661+0000) 2023-09-30T15:05:18.610 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:18 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:18.232+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.776204+0000 front 2023-09-30T15:04:48.775984+0000 (oldest deadline 2023-09-30T15:05:13.476011+0000) 2023-09-30T15:05:18.745 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:18 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:18.328+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:18.745 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:18 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:18.328+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:18.745 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:18 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:18.328+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:18.746 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:18 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:18.328+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:19.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:18 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:18.745+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:19.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:18 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:18.745+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:19.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:18 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:18.745+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.677095+0000 front 2023-09-30T15:04:48.677082+0000 (oldest deadline 2023-09-30T15:05:14.577134+0000) 2023-09-30T15:05:19.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:18 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:18.713+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:19.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:18 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:18.713+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:19.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:18 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:18.713+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:19.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:18 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:18.713+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:19.137 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:18 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:18.660+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:19.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:18 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:18.660+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:19.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:18 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:18.660+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:19.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:18 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:18.660+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:19.609 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:19 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:19.223+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:34.073886+0000 front 2023-09-30T15:04:34.073860+0000 (oldest deadline 2023-09-30T15:04:59.374065+0000) 2023-09-30T15:05:19.610 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:19 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:19.223+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.374827+0000 front 2023-09-30T15:04:39.374684+0000 (oldest deadline 2023-09-30T15:05:04.074661+0000) 2023-09-30T15:05:19.610 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:19 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7[123340]: 2023-09-30T15:05:19.223+0000 7fcc8db25700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.776204+0000 front 2023-09-30T15:04:48.775984+0000 (oldest deadline 2023-09-30T15:05:13.476011+0000) 2023-09-30T15:05:19.731 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:19 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:19.349+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:19.731 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:19 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:19.349+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:19.731 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:19 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:19.349+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:19.731 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:19 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:19.349+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:20.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:19 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:19.730+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:20.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:19 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:19.730+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:20.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:19 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:19.730+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.677095+0000 front 2023-09-30T15:04:48.677082+0000 (oldest deadline 2023-09-30T15:05:14.577134+0000) 2023-09-30T15:05:20.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:19 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:19.739+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:20.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:19 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:19.739+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:20.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:19 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:19.739+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:20.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:19 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:19.739+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:20.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:19 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:19.739+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:20.137 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:19 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:19.700+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:20.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:19 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:19.700+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:20.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:19 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:19.700+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:20.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:19 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:19.700+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:20.730 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:20 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:20.339+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:20.730 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:20 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:20.339+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:20.731 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:20 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:20.340+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:20.731 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:20 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:20.340+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:20.774 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 30 15:05:20 smithi148 bash[147919]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-7 2023-09-30T15:05:21.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:20 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:20.730+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:21.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:20 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:20.730+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:21.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:20 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:20.730+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.677095+0000 front 2023-09-30T15:04:48.677082+0000 (oldest deadline 2023-09-30T15:05:14.577134+0000) 2023-09-30T15:05:21.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:20 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:20.730+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.577938+0000 front 2023-09-30T15:04:54.577905+0000 (oldest deadline 2023-09-30T15:05:20.477864+0000) 2023-09-30T15:05:21.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:20 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:20.773+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:21.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:20 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:20.773+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:21.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:20 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:20.773+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:21.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:20 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:20.773+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:21.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:20 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:20.773+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:21.137 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:20 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:20.654+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:21.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:20 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:20.654+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:21.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:20 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:20.654+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:21.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:20 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:20.654+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:21.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:21 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:21.297+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:21.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:21 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:21.297+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:21.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:21 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:21.297+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:21.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:21 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:21.297+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:21.792 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:21 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:21.764+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:21.793 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:21 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:21.764+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:21.793 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:21 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:21.764+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:21.793 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:21 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:21.764+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:21.793 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:21 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:21.764+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:21.799 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.7.service' 2023-09-30T15:05:21.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:05:21.879 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-09-30T15:05:21.879 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-09-30T15:05:21.880 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.8 2023-09-30T15:05:21.887 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:21 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:21.620+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:21.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:21 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:21.620+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:21.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:21 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:21.620+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:21.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:21 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:21.620+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:22.056 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:21 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:21.766+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:22.056 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:21 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:21.766+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:22.056 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:21 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:21.766+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.677095+0000 front 2023-09-30T15:04:48.677082+0000 (oldest deadline 2023-09-30T15:05:14.577134+0000) 2023-09-30T15:05:22.056 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:21 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:21.766+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.577938+0000 front 2023-09-30T15:04:54.577905+0000 (oldest deadline 2023-09-30T15:05:20.477864+0000) 2023-09-30T15:05:22.056 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:21 smithi156 systemd[1]: Stopping Ceph osd.8 for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:05:22.555 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:22 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:22.293+0000 7f53560ad700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:05:22.555 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:22 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:22.293+0000 7f53560ad700 -1 osd.8 88 *** Got signal Terminated *** 2023-09-30T15:05:22.555 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:22 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:22.293+0000 7f53560ad700 -1 osd.8 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-30T15:05:22.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:22 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:22.265+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:22.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:22 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:22.265+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:22.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:22 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:22.265+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:22.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:22 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:22.265+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:22.887 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:22 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:22.603+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:22.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:22 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:22.603+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:22.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:22 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:22.603+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:22.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:22 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:22.603+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:23.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:22 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:22.790+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:23.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:22 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:22.790+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:23.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:22 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:22.790+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.677095+0000 front 2023-09-30T15:04:48.677082+0000 (oldest deadline 2023-09-30T15:05:14.577134+0000) 2023-09-30T15:05:23.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:22 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:22.790+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.577938+0000 front 2023-09-30T15:04:54.577905+0000 (oldest deadline 2023-09-30T15:05:20.477864+0000) 2023-09-30T15:05:23.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:22 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:22.757+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:23.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:22 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:22.757+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:23.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:22 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:22.757+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:23.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:22 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:22.757+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:23.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:22 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:22.757+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:23.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:23 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:23.276+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:23.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:23 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:23.276+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:23.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:23 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:23.276+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:23.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:23 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:23.276+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:23.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:23 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:23.582+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:23.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:23 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:23.582+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:23.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:23 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:23.582+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:23.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:23 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:23.582+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:24.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:23 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:23.753+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:24.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:23 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:23.753+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:24.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:23 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:23.753+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.677095+0000 front 2023-09-30T15:04:48.677082+0000 (oldest deadline 2023-09-30T15:05:14.577134+0000) 2023-09-30T15:05:24.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:23 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:23.753+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.577938+0000 front 2023-09-30T15:04:54.577905+0000 (oldest deadline 2023-09-30T15:05:20.477864+0000) 2023-09-30T15:05:24.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:23 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:23.713+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:24.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:23 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:23.713+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:24.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:23 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:23.713+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:24.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:23 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:23.713+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:24.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:23 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:23.713+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:24.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:24 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:24.249+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:24.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:24 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:24.249+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:24.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:24 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:24.249+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:24.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:24 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:24.249+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:24.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:24 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:24.620+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:24.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:24 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:24.620+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:24.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:24 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:24.620+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:24.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:24 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:24.620+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:25.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:24 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:24.780+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:25.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:24 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:24.780+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:25.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:24 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:24.780+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.677095+0000 front 2023-09-30T15:04:48.677082+0000 (oldest deadline 2023-09-30T15:05:14.577134+0000) 2023-09-30T15:05:25.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:24 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:24.780+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.577938+0000 front 2023-09-30T15:04:54.577905+0000 (oldest deadline 2023-09-30T15:05:20.477864+0000) 2023-09-30T15:05:25.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:24 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:24.745+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:25.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:24 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:24.745+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:25.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:24 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:24.745+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:25.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:24 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:24.745+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:25.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:24 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:24.745+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:25.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:25 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:25.237+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:25.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:25 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:25.237+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:25.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:25 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:25.237+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:25.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:25 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:25.237+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:25.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:25 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:25.618+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:25.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:25 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:25.618+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:25.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:25 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:25.618+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:25.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:25 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:25.618+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:26.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:25 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:25.752+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:26.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:25 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:25.752+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:26.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:25 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:25.752+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.677095+0000 front 2023-09-30T15:04:48.677082+0000 (oldest deadline 2023-09-30T15:05:14.577134+0000) 2023-09-30T15:05:26.055 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:25 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:25.752+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.577938+0000 front 2023-09-30T15:04:54.577905+0000 (oldest deadline 2023-09-30T15:05:20.477864+0000) 2023-09-30T15:05:26.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:25 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:25.716+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:26.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:25 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:25.716+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:26.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:25 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:25.716+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:26.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:25 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:25.716+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:26.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:25 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:25.716+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:26.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:26 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:26.236+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:26.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:26 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:26.236+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:26.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:26 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:26.236+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:26.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:26 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:26.236+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:26.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:26 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:26.236+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:27.056 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:26 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:26.742+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:32.973350+0000 front 2023-09-30T15:04:32.973387+0000 (oldest deadline 2023-09-30T15:04:55.873441+0000) 2023-09-30T15:05:27.056 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:26 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:26.742+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:38.174644+0000 front 2023-09-30T15:04:38.174615+0000 (oldest deadline 2023-09-30T15:05:04.575844+0000) 2023-09-30T15:05:27.056 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:26 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:26.742+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:48.677095+0000 front 2023-09-30T15:04:48.677082+0000 (oldest deadline 2023-09-30T15:05:14.577134+0000) 2023-09-30T15:05:27.056 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:26 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8[125917]: 2023-09-30T15:05:26.742+0000 7f534f69d700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.577938+0000 front 2023-09-30T15:04:54.577905+0000 (oldest deadline 2023-09-30T15:05:20.477864+0000) 2023-09-30T15:05:27.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:26 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:26.724+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:27.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:26 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:26.724+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:27.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:26 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:26.724+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:27.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:26 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:26.724+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:27.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:26 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:26.724+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:27.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:26 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:26.643+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:27.139 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:26 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:26.643+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:27.139 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:26 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:26.643+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:27.139 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:26 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:26.643+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:27.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:27 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:27.284+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:27.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:27 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:27.284+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:27.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:27 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:27.284+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:27.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:27 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:27.284+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:27.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:27 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:27.284+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:28.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:27 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:27.732+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:28.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:27 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:27.732+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:28.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:27 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:27.732+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:28.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:27 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:27.732+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:28.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:27 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:27.732+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:28.137 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:27 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:27.644+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:28.137 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:27 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:27.644+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:28.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:27 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:27.644+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:28.138 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:27 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:27.644+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:28.277 INFO:journalctl@ceph.osd.8.smithi156.stdout:Sep 30 15:05:28 smithi156 bash[151365]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-8 2023-09-30T15:05:28.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:28 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:28.276+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:28.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:28 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:28.276+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:28.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:28 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:28.276+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:28.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:28 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:28.276+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:28.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:28 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:28.276+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:28.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:28 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:28.599+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:28.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:28 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:28.599+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:28.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:28 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:28.599+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:28.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:28 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:28.599+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:29.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:28 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:28.693+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:29.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:28 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:28.693+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:29.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:28 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:28.693+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:29.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:28 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:28.693+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:29.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:28 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:28.693+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:29.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:29 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:29.293+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:29.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:29 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:29.293+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:29.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:29 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:29.293+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:29.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:29 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:29.293+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:29.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:29 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:29.293+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:29.681 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.8.service' 2023-09-30T15:05:29.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:05:29.721 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-09-30T15:05:29.721 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-09-30T15:05:29.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.9 2023-09-30T15:05:29.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:29 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:29.621+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:29.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:29 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:29.621+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:29.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:29 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:29.621+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:29.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:29 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:29.621+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:29.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:29 smithi032 systemd[1]: Stopping Ceph osd.9 for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:05:30.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:29 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:29.729+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:30.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:29 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:29.729+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:30.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:29 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:29.729+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:30.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:29 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:29.729+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:30.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:29 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:29.729+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:30.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:29 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:29.729+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:04.251349+0000 front 2023-09-30T15:05:04.251447+0000 (oldest deadline 2023-09-30T15:05:28.951408+0000) 2023-09-30T15:05:30.388 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:30 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:30.096+0000 7f39ece2e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:05:30.388 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:30 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:30.096+0000 7f39ece2e700 -1 osd.9 88 *** Got signal Terminated *** 2023-09-30T15:05:30.388 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:30 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:30.096+0000 7f39ece2e700 -1 osd.9 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-30T15:05:30.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:30 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:30.252+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:30.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:30 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:30.252+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:30.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:30 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:30.252+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:30.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:30 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:30.252+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:30.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:30 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:30.252+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:30.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:30 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:30.634+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:30.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:30 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:30.634+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:30.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:30 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:30.634+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:30.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:30 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:30.634+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:30.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:30 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:30.634+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:00.476055+0000 front 2023-09-30T15:05:00.475827+0000 (oldest deadline 2023-09-30T15:05:30.476718+0000) 2023-09-30T15:05:31.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:30 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:30.764+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:31.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:30 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:30.764+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:31.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:30 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:30.764+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:31.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:30 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:30.764+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:31.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:30 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:30.764+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:31.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:30 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:30.764+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:04.251349+0000 front 2023-09-30T15:05:04.251447+0000 (oldest deadline 2023-09-30T15:05:28.951408+0000) 2023-09-30T15:05:31.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:31 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:31.242+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:31.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:31 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:31.242+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:31.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:31 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:31.242+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:31.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:31 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:31.242+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:31.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:31 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:31.242+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:31.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:31 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:31.587+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:31.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:31 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:31.587+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:31.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:31 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:31.587+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:31.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:31 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:31.587+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:31.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:31 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:31.587+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:00.476055+0000 front 2023-09-30T15:05:00.475827+0000 (oldest deadline 2023-09-30T15:05:30.476718+0000) 2023-09-30T15:05:32.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:31 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:31.716+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:32.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:31 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:31.716+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:32.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:31 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:31.716+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:32.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:31 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:31.716+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:32.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:31 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:31.716+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:32.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:31 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:31.716+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:04.251349+0000 front 2023-09-30T15:05:04.251447+0000 (oldest deadline 2023-09-30T15:05:28.951408+0000) 2023-09-30T15:05:32.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:32 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:32.242+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:32.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:32 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:32.242+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:32.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:32 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:32.242+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:32.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:32 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:32.242+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:32.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:32 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:32.242+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:32.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:32 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:32.587+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:32.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:32 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:32.587+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:32.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:32 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:32.587+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:32.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:32 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:32.587+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:32.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:32 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:32.587+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:00.476055+0000 front 2023-09-30T15:05:00.475827+0000 (oldest deadline 2023-09-30T15:05:30.476718+0000) 2023-09-30T15:05:33.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:32 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:32.677+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:33.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:32 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:32.677+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:33.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:32 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:32.677+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:33.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:32 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:32.677+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:33.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:32 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:32.677+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:33.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:32 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:32.677+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:04.251349+0000 front 2023-09-30T15:05:04.251447+0000 (oldest deadline 2023-09-30T15:05:28.951408+0000) 2023-09-30T15:05:33.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:33 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:33.274+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:33.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:33 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:33.274+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:33.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:33 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:33.274+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:33.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:33 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:33.274+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:33.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:33 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:33.274+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:33.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:33 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:33.578+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:33.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:33 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:33.578+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:33.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:33 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:33.578+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:33.889 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:33 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:33.578+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:33.889 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:33 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:33.578+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:00.476055+0000 front 2023-09-30T15:05:00.475827+0000 (oldest deadline 2023-09-30T15:05:30.476718+0000) 2023-09-30T15:05:34.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:33 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:33.680+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:34.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:33 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:33.680+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:34.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:33 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:33.680+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:34.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:33 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:33.680+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:34.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:33 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:33.680+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:34.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:33 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:33.680+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:04.251349+0000 front 2023-09-30T15:05:04.251447+0000 (oldest deadline 2023-09-30T15:05:28.951408+0000) 2023-09-30T15:05:34.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:34 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:34.229+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:34.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:34 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:34.229+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:34.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:34 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:34.229+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:34.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:34 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:34.229+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:34.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:34 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:34.229+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:34.887 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:34 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:34.622+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:20.966961+0000 front 2023-09-30T15:04:20.966677+0000 (oldest deadline 2023-09-30T15:04:48.567598+0000) 2023-09-30T15:05:34.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:34 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:34.622+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:33.169550+0000 front 2023-09-30T15:04:33.169474+0000 (oldest deadline 2023-09-30T15:04:57.269660+0000) 2023-09-30T15:05:34.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:34 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:34.622+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:39.571307+0000 front 2023-09-30T15:04:39.571288+0000 (oldest deadline 2023-09-30T15:05:03.071297+0000) 2023-09-30T15:05:34.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:34 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:34.622+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:47.772597+0000 front 2023-09-30T15:04:47.772558+0000 (oldest deadline 2023-09-30T15:05:12.472679+0000) 2023-09-30T15:05:34.888 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:34 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9[142560]: 2023-09-30T15:05:34.622+0000 7f39e8c23700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:00.476055+0000 front 2023-09-30T15:05:00.475827+0000 (oldest deadline 2023-09-30T15:05:30.476718+0000) 2023-09-30T15:05:35.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:34 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:34.665+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:35.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:34 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:34.665+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:35.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:34 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:34.665+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:35.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:34 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:34.665+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:35.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:34 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:34.665+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:35.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:34 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:34.665+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:04.251349+0000 front 2023-09-30T15:05:04.251447+0000 (oldest deadline 2023-09-30T15:05:28.951408+0000) 2023-09-30T15:05:35.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:35 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:35.198+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:35.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:35 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:35.198+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:35.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:35 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:35.198+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:35.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:35 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:35.198+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:35.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:35 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:35.198+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:35.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:35 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:35.198+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:10.129616+0000 front 2023-09-30T15:05:10.129513+0000 (oldest deadline 2023-09-30T15:05:34.829553+0000) 2023-09-30T15:05:35.887 INFO:journalctl@ceph.osd.9.smithi032.stdout:Sep 30 15:05:35 smithi032 bash[178784]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-9 2023-09-30T15:05:36.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:35 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:35.660+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:36.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:35 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:35.660+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:36.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:35 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:35.660+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:36.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:35 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:35.660+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:36.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:35 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:35.660+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:36.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:35 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:35.660+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:04.251349+0000 front 2023-09-30T15:05:04.251447+0000 (oldest deadline 2023-09-30T15:05:28.951408+0000) 2023-09-30T15:05:36.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:35 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:35.660+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:08.952202+0000 front 2023-09-30T15:05:08.952147+0000 (oldest deadline 2023-09-30T15:05:35.352534+0000) 2023-09-30T15:05:36.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:36 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:36.235+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:36.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:36 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:36.235+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:36.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:36 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:36.235+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:36.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:36 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:36.235+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:36.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:36 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:36.235+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:36.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:36 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:36.235+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:10.129616+0000 front 2023-09-30T15:05:10.129513+0000 (oldest deadline 2023-09-30T15:05:34.829553+0000) 2023-09-30T15:05:37.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:36 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:36.673+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:37.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:36 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:36.673+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:37.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:36 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:36.673+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:37.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:36 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:36.673+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:37.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:36 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:36.673+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:37.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:36 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:36.673+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:04.251349+0000 front 2023-09-30T15:05:04.251447+0000 (oldest deadline 2023-09-30T15:05:28.951408+0000) 2023-09-30T15:05:37.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:36 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:36.673+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:08.952202+0000 front 2023-09-30T15:05:08.952147+0000 (oldest deadline 2023-09-30T15:05:35.352534+0000) 2023-09-30T15:05:37.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.9.service' 2023-09-30T15:05:37.386 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:05:37.387 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-09-30T15:05:37.387 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-09-30T15:05:37.387 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.10 2023-09-30T15:05:37.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:37 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:37.202+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:37.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:37 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:37.202+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:37.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:37 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:37.202+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:37.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:37 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:37.202+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:37.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:37 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:37.202+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:37.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:37 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:37.202+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:10.129616+0000 front 2023-09-30T15:05:10.129513+0000 (oldest deadline 2023-09-30T15:05:34.829553+0000) 2023-09-30T15:05:37.743 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:37 smithi148 systemd[1]: Stopping Ceph osd.10 for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:05:37.743 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:37 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:37.635+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:37.743 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:37 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:37.635+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:37.743 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:37 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:37.635+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:37.743 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:37 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:37.635+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:37.743 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:37 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:37.635+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:37.743 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:37 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:37.635+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:04.251349+0000 front 2023-09-30T15:05:04.251447+0000 (oldest deadline 2023-09-30T15:05:28.951408+0000) 2023-09-30T15:05:37.743 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:37 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:37.635+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:08.952202+0000 front 2023-09-30T15:05:08.952147+0000 (oldest deadline 2023-09-30T15:05:35.352534+0000) 2023-09-30T15:05:38.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:37 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:37.742+0000 7fcbd4d59700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:05:38.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:37 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:37.742+0000 7fcbd4d59700 -1 osd.10 88 *** Got signal Terminated *** 2023-09-30T15:05:38.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:37 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:37.742+0000 7fcbd4d59700 -1 osd.10 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-30T15:05:38.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:38 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:38.186+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:38.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:38 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:38.186+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:38.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:38 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:38.186+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:38.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:38 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:38.186+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:38.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:38 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:38.186+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:38.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:38 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:38.186+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:10.129616+0000 front 2023-09-30T15:05:10.129513+0000 (oldest deadline 2023-09-30T15:05:34.829553+0000) 2023-09-30T15:05:39.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:38 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:38.616+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:39.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:38 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:38.616+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:39.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:38 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:38.616+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:39.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:38 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:38.616+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:39.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:38 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:38.616+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:39.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:38 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:38.616+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:04.251349+0000 front 2023-09-30T15:05:04.251447+0000 (oldest deadline 2023-09-30T15:05:28.951408+0000) 2023-09-30T15:05:39.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:38 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:38.616+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:08.952202+0000 front 2023-09-30T15:05:08.952147+0000 (oldest deadline 2023-09-30T15:05:35.352534+0000) 2023-09-30T15:05:39.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:39 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:39.210+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:39.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:39 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:39.210+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:39.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:39 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:39.210+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:39.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:39 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:39.210+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:39.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:39 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:39.210+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:39.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:39 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:39.210+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:10.129616+0000 front 2023-09-30T15:05:10.129513+0000 (oldest deadline 2023-09-30T15:05:34.829553+0000) 2023-09-30T15:05:39.859 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:39 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:39.585+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:39.860 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:39 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:39.585+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:39.860 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:39 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:39.585+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:39.860 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:39 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:39.585+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:39.860 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:39 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:39.585+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:39.860 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:39 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:39.585+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:04.251349+0000 front 2023-09-30T15:05:04.251447+0000 (oldest deadline 2023-09-30T15:05:28.951408+0000) 2023-09-30T15:05:39.860 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:39 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:39.585+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:08.952202+0000 front 2023-09-30T15:05:08.952147+0000 (oldest deadline 2023-09-30T15:05:35.352534+0000) 2023-09-30T15:05:40.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:40 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:40.201+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:40.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:40 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:40.201+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:40.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:40 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:40.201+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:40.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:40 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:40.201+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:40.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:40 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:40.201+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:40.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:40 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:40.201+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:10.129616+0000 front 2023-09-30T15:05:10.129513+0000 (oldest deadline 2023-09-30T15:05:34.829553+0000) 2023-09-30T15:05:40.859 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:40 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:40.606+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:40.860 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:40 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:40.606+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:40.860 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:40 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:40.606+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:40.860 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:40 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:40.606+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:40.860 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:40 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:40.606+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:40.860 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:40 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:40.606+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:04.251349+0000 front 2023-09-30T15:05:04.251447+0000 (oldest deadline 2023-09-30T15:05:28.951408+0000) 2023-09-30T15:05:40.860 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:40 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:40.606+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:08.952202+0000 front 2023-09-30T15:05:08.952147+0000 (oldest deadline 2023-09-30T15:05:35.352534+0000) 2023-09-30T15:05:41.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:41 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:41.164+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:41.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:41 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:41.164+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:41.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:41 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:41.164+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:41.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:41 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:41.164+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:41.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:41 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:41.164+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:41.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:41 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:41.164+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:10.129616+0000 front 2023-09-30T15:05:10.129513+0000 (oldest deadline 2023-09-30T15:05:34.829553+0000) 2023-09-30T15:05:42.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:41 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:41.637+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:42.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:41 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:41.637+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:42.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:41 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:41.637+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:42.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:41 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:41.637+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:42.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:41 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:41.637+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:42.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:41 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:41.637+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:04.251349+0000 front 2023-09-30T15:05:04.251447+0000 (oldest deadline 2023-09-30T15:05:28.951408+0000) 2023-09-30T15:05:42.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:41 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:41.637+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:08.952202+0000 front 2023-09-30T15:05:08.952147+0000 (oldest deadline 2023-09-30T15:05:35.352534+0000) 2023-09-30T15:05:42.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:42 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:42.137+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:42.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:42 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:42.137+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:42.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:42 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:42.137+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:42.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:42 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:42.137+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:42.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:42 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:42.137+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:42.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:42 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:42.137+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:10.129616+0000 front 2023-09-30T15:05:10.129513+0000 (oldest deadline 2023-09-30T15:05:34.829553+0000) 2023-09-30T15:05:43.109 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:42 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:42.652+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:25.045234+0000 front 2023-09-30T15:04:25.045316+0000 (oldest deadline 2023-09-30T15:04:50.245972+0000) 2023-09-30T15:05:43.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:42 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:42.652+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:30.246381+0000 front 2023-09-30T15:04:30.246411+0000 (oldest deadline 2023-09-30T15:04:55.546584+0000) 2023-09-30T15:05:43.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:42 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:42.652+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:41.348297+0000 front 2023-09-30T15:04:41.348155+0000 (oldest deadline 2023-09-30T15:05:06.648312+0000) 2023-09-30T15:05:43.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:42 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:42.652+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2023-09-30T15:04:46.648941+0000 front 2023-09-30T15:04:46.648922+0000 (oldest deadline 2023-09-30T15:05:10.748978+0000) 2023-09-30T15:05:43.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:42 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:42.652+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:56.650109+0000 front 2023-09-30T15:04:56.650137+0000 (oldest deadline 2023-09-30T15:05:18.950211+0000) 2023-09-30T15:05:43.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:42 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:42.652+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:04.251349+0000 front 2023-09-30T15:05:04.251447+0000 (oldest deadline 2023-09-30T15:05:28.951408+0000) 2023-09-30T15:05:43.110 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:42 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10[131384]: 2023-09-30T15:05:42.652+0000 7fcbcdb48700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:08.952202+0000 front 2023-09-30T15:05:08.952147+0000 (oldest deadline 2023-09-30T15:05:35.352534+0000) 2023-09-30T15:05:43.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:43 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:43.154+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:43.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:43 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:43.154+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:43.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:43 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:43.154+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:43.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:43 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:43.154+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:43.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:43 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:43.154+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:43.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:43 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:43.154+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:10.129616+0000 front 2023-09-30T15:05:10.129513+0000 (oldest deadline 2023-09-30T15:05:34.829553+0000) 2023-09-30T15:05:43.609 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 30 15:05:43 smithi148 bash[148193]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-10 2023-09-30T15:05:44.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:44 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:44.201+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:44.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:44 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:44.201+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:44.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:44 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:44.201+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:44.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:44 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:44.201+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:44.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:44 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:44.201+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:44.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:44 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:44.201+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:10.129616+0000 front 2023-09-30T15:05:10.129513+0000 (oldest deadline 2023-09-30T15:05:34.829553+0000) 2023-09-30T15:05:44.688 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.10.service' 2023-09-30T15:05:44.728 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:05:44.728 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-09-30T15:05:44.728 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-09-30T15:05:44.728 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.11 2023-09-30T15:05:45.056 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:44 smithi156 systemd[1]: Stopping Ceph osd.11 for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:05:45.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:45 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:45.103+0000 7fb985bb8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:05:45.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:45 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:45.103+0000 7fb985bb8700 -1 osd.11 88 *** Got signal Terminated *** 2023-09-30T15:05:45.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:45 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:45.103+0000 7fb985bb8700 -1 osd.11 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-30T15:05:45.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:45 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:45.164+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:45.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:45 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:45.164+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:45.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:45 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:45.164+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:45.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:45 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:45.164+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:45.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:45 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:45.164+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:45.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:45 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:45.164+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:10.129616+0000 front 2023-09-30T15:05:10.129513+0000 (oldest deadline 2023-09-30T15:05:34.829553+0000) 2023-09-30T15:05:46.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:46 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:46.202+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:46.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:46 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:46.202+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:46.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:46 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:46.202+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:46.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:46 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:46.202+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:46.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:46 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:46.202+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:46.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:46 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:46.202+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:10.129616+0000 front 2023-09-30T15:05:10.129513+0000 (oldest deadline 2023-09-30T15:05:34.829553+0000) 2023-09-30T15:05:46.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:46 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:46.202+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2023-09-30T15:05:17.130577+0000 front 2023-09-30T15:05:17.130558+0000 (oldest deadline 2023-09-30T15:05:45.931167+0000) 2023-09-30T15:05:47.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:47 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:47.219+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:47.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:47 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:47.219+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:47.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:47 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:47.219+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:47.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:47 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:47.219+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:47.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:47 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:47.219+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:47.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:47 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:47.219+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:10.129616+0000 front 2023-09-30T15:05:10.129513+0000 (oldest deadline 2023-09-30T15:05:34.829553+0000) 2023-09-30T15:05:47.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:47 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:47.219+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2023-09-30T15:05:17.130577+0000 front 2023-09-30T15:05:17.130558+0000 (oldest deadline 2023-09-30T15:05:45.931167+0000) 2023-09-30T15:05:48.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:48 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:48.225+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:48.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:48 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:48.225+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:48.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:48 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:48.225+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:48.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:48 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:48.225+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:48.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:48 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:48.225+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:48.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:48 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:48.225+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:10.129616+0000 front 2023-09-30T15:05:10.129513+0000 (oldest deadline 2023-09-30T15:05:34.829553+0000) 2023-09-30T15:05:48.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:48 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:48.225+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2023-09-30T15:05:17.130577+0000 front 2023-09-30T15:05:17.130558+0000 (oldest deadline 2023-09-30T15:05:45.931167+0000) 2023-09-30T15:05:49.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:49 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:49.199+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2023-09-30T15:04:23.719423+0000 front 2023-09-30T15:04:23.719262+0000 (oldest deadline 2023-09-30T15:04:49.619421+0000) 2023-09-30T15:05:49.555 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:49 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:49.199+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2023-09-30T15:04:31.821653+0000 front 2023-09-30T15:04:31.821623+0000 (oldest deadline 2023-09-30T15:04:54.721609+0000) 2023-09-30T15:05:49.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:49 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:49.199+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-09-30T15:04:40.324504+0000 front 2023-09-30T15:04:40.324543+0000 (oldest deadline 2023-09-30T15:05:05.624642+0000) 2023-09-30T15:05:49.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:49 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:49.199+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2023-09-30T15:04:54.927185+0000 front 2023-09-30T15:04:54.927214+0000 (oldest deadline 2023-09-30T15:05:17.827266+0000) 2023-09-30T15:05:49.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:49 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:49.199+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-09-30T15:05:03.728343+0000 front 2023-09-30T15:05:03.728416+0000 (oldest deadline 2023-09-30T15:05:25.428445+0000) 2023-09-30T15:05:49.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:49 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:49.199+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2023-09-30T15:05:10.129616+0000 front 2023-09-30T15:05:10.129513+0000 (oldest deadline 2023-09-30T15:05:34.829553+0000) 2023-09-30T15:05:49.556 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:49 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11[134054]: 2023-09-30T15:05:49.199+0000 7fb9829af700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2023-09-30T15:05:17.130577+0000 front 2023-09-30T15:05:17.130558+0000 (oldest deadline 2023-09-30T15:05:45.931167+0000) 2023-09-30T15:05:51.055 INFO:journalctl@ceph.osd.11.smithi156.stdout:Sep 30 15:05:50 smithi156 bash[151638]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-osd-11 2023-09-30T15:05:52.094 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@osd.11.service' 2023-09-30T15:05:52.134 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:05:52.135 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-09-30T15:05:52.135 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-09-30T15:05:52.135 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.a 2023-09-30T15:05:52.495 INFO:journalctl@ceph.mds.a.smithi032.stdout:Sep 30 15:05:52 smithi032 systemd[1]: Stopping Ceph mds.a for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:05:52.887 INFO:journalctl@ceph.mds.a.smithi032.stdout:Sep 30 15:05:52 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-a[150594]: 2023-09-30T15:05:52.494+0000 7f54e5c9c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:05:52.888 INFO:journalctl@ceph.mds.a.smithi032.stdout:Sep 30 15:05:52 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-a[150594]: 2023-09-30T15:05:52.494+0000 7f54e5c9c700 -1 mds.a *** got signal Terminated *** 2023-09-30T15:05:56.138 INFO:journalctl@ceph.mds.a.smithi032.stdout:Sep 30 15:05:55 smithi032 bash[179053]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-a 2023-09-30T15:05:56.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.a.service' 2023-09-30T15:05:56.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:05:56.233 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-09-30T15:05:56.233 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-09-30T15:05:56.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.d 2023-09-30T15:05:56.612 INFO:journalctl@ceph.mds.d.smithi032.stdout:Sep 30 15:05:56 smithi032 systemd[1]: Stopping Ceph mds.d for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:05:56.887 INFO:journalctl@ceph.mds.d.smithi032.stdout:Sep 30 15:05:56 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-d[149595]: 2023-09-30T15:05:56.611+0000 7f273e8c1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:05:56.888 INFO:journalctl@ceph.mds.d.smithi032.stdout:Sep 30 15:05:56 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-d[149595]: 2023-09-30T15:05:56.611+0000 7f273e8c1700 -1 mds.d *** got signal Terminated *** 2023-09-30T15:06:00.185 INFO:journalctl@ceph.mds.d.smithi032.stdout:Sep 30 15:06:00 smithi032 bash[179199]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-d 2023-09-30T15:06:00.382 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.d.service' 2023-09-30T15:06:00.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:06:00.419 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-09-30T15:06:00.419 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-09-30T15:06:00.419 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.g 2023-09-30T15:06:00.638 INFO:journalctl@ceph.mds.g.smithi032.stdout:Sep 30 15:06:00 smithi032 systemd[1]: Stopping Ceph mds.g for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:06:01.137 INFO:journalctl@ceph.mds.g.smithi032.stdout:Sep 30 15:06:00 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-g[150207]: 2023-09-30T15:06:00.800+0000 7fbfed52e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:06:01.138 INFO:journalctl@ceph.mds.g.smithi032.stdout:Sep 30 15:06:00 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-g[150207]: 2023-09-30T15:06:00.800+0000 7fbfed52e700 -1 mds.g *** got signal Terminated *** 2023-09-30T15:06:05.346 INFO:journalctl@ceph.mds.g.smithi032.stdout:Sep 30 15:06:05 smithi032 bash[179337]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-g 2023-09-30T15:06:05.352 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.g.service' 2023-09-30T15:06:05.387 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:06:05.387 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-09-30T15:06:05.387 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-09-30T15:06:05.387 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.j 2023-09-30T15:06:05.638 INFO:journalctl@ceph.mds.j.smithi032.stdout:Sep 30 15:06:05 smithi032 systemd[1]: Stopping Ceph mds.j for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:06:06.137 INFO:journalctl@ceph.mds.j.smithi032.stdout:Sep 30 15:06:05 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-j[149204]: 2023-09-30T15:06:05.784+0000 7fcf28cab700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:06:06.138 INFO:journalctl@ceph.mds.j.smithi032.stdout:Sep 30 15:06:05 smithi032 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-j[149204]: 2023-09-30T15:06:05.784+0000 7fcf28cab700 -1 mds.j *** got signal Terminated *** 2023-09-30T15:06:10.138 INFO:journalctl@ceph.mds.j.smithi032.stdout:Sep 30 15:06:09 smithi032 bash[179483]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-j 2023-09-30T15:06:10.246 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.j.service' 2023-09-30T15:06:10.280 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:06:10.281 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-09-30T15:06:10.281 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-09-30T15:06:10.281 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.b 2023-09-30T15:06:10.609 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 30 15:06:10 smithi148 systemd[1]: Stopping Ceph mds.b for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:06:11.109 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 30 15:06:10 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-b[138505]: 2023-09-30T15:06:10.628+0000 7ff9d06cc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:06:11.109 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 30 15:06:10 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-b[138505]: 2023-09-30T15:06:10.628+0000 7ff9d06cc700 -1 mds.b *** got signal Terminated *** 2023-09-30T15:06:12.109 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 30 15:06:11 smithi148 bash[148458]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-b 2023-09-30T15:06:12.125 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.b.service' 2023-09-30T15:06:12.160 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:06:12.161 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-09-30T15:06:12.161 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-09-30T15:06:12.161 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.e 2023-09-30T15:06:12.537 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 30 15:06:12 smithi148 systemd[1]: Stopping Ceph mds.e for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:06:12.859 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 30 15:06:12 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-e[137823]: 2023-09-30T15:06:12.536+0000 7ff88b625700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:06:12.859 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 30 15:06:12 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-e[137823]: 2023-09-30T15:06:12.536+0000 7ff88b625700 -1 mds.e *** got signal Terminated *** 2023-09-30T15:06:16.109 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 30 15:06:15 smithi148 bash[148599]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-e 2023-09-30T15:06:16.116 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.e.service' 2023-09-30T15:06:16.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:06:16.151 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-09-30T15:06:16.151 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-09-30T15:06:16.151 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.h 2023-09-30T15:06:16.519 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 30 15:06:16 smithi148 systemd[1]: Stopping Ceph mds.h for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:06:16.859 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 30 15:06:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-h[138164]: 2023-09-30T15:06:16.519+0000 7f5c4beed700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:06:16.859 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 30 15:06:16 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-h[138164]: 2023-09-30T15:06:16.519+0000 7f5c4beed700 -1 mds.h *** got signal Terminated *** 2023-09-30T15:06:20.075 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 30 15:06:19 smithi148 bash[148747]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-h 2023-09-30T15:06:20.081 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.h.service' 2023-09-30T15:06:20.114 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:06:20.115 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-09-30T15:06:20.115 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-09-30T15:06:20.115 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.k 2023-09-30T15:06:20.359 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 30 15:06:20 smithi148 systemd[1]: Stopping Ceph mds.k for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:06:20.861 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 30 15:06:20 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-k[137480]: 2023-09-30T15:06:20.485+0000 7fd3a366c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:06:20.861 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 30 15:06:20 smithi148 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-k[137480]: 2023-09-30T15:06:20.485+0000 7fd3a366c700 -1 mds.k *** got signal Terminated *** 2023-09-30T15:06:24.035 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 30 15:06:23 smithi148 bash[148893]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-k 2023-09-30T15:06:24.040 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.k.service' 2023-09-30T15:06:24.072 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:06:24.072 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-09-30T15:06:24.073 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-09-30T15:06:24.073 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.c 2023-09-30T15:06:24.440 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 30 15:06:24 smithi156 systemd[1]: Stopping Ceph mds.c for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:06:24.805 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 30 15:06:24 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-c[140290]: 2023-09-30T15:06:24.439+0000 7f80bdc9b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:06:24.805 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 30 15:06:24 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-c[140290]: 2023-09-30T15:06:24.439+0000 7f80bdc9b700 -1 mds.c *** got signal Terminated *** 2023-09-30T15:06:28.055 INFO:journalctl@ceph.mds.c.smithi156.stdout:Sep 30 15:06:27 smithi156 bash[151909]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-c 2023-09-30T15:06:28.151 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.c.service' 2023-09-30T15:06:28.186 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:06:28.186 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-09-30T15:06:28.187 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-09-30T15:06:28.187 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.f 2023-09-30T15:06:28.555 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 30 15:06:28 smithi156 systemd[1]: Stopping Ceph mds.f for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:06:29.055 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 30 15:06:28 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-f[139946]: 2023-09-30T15:06:28.564+0000 7f4476b66700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:06:29.071 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 30 15:06:28 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-f[139946]: 2023-09-30T15:06:28.564+0000 7f4476b66700 -1 mds.f *** got signal Terminated *** 2023-09-30T15:06:31.932 INFO:journalctl@ceph.mds.f.smithi156.stdout:Sep 30 15:06:31 smithi156 bash[152055]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-f 2023-09-30T15:06:32.121 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.f.service' 2023-09-30T15:06:32.156 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:06:32.157 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-09-30T15:06:32.157 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-09-30T15:06:32.157 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.i 2023-09-30T15:06:32.547 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 30 15:06:32 smithi156 systemd[1]: Stopping Ceph mds.i for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:06:32.805 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 30 15:06:32 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-i[140626]: 2023-09-30T15:06:32.546+0000 7f0035dd1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:06:32.805 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 30 15:06:32 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-i[140626]: 2023-09-30T15:06:32.546+0000 7f0035dd1700 -1 mds.i *** got signal Terminated *** 2023-09-30T15:06:36.055 INFO:journalctl@ceph.mds.i.smithi156.stdout:Sep 30 15:06:35 smithi156 bash[152200]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-i 2023-09-30T15:06:36.150 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.i.service' 2023-09-30T15:06:36.185 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:06:36.185 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-09-30T15:06:36.185 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-09-30T15:06:36.186 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.l 2023-09-30T15:06:36.555 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 30 15:06:36 smithi156 systemd[1]: Stopping Ceph mds.l for bca3f5c6-5f9d-11ee-8db5-212e2dc638e7... 2023-09-30T15:06:37.055 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 30 15:06:36 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-l[139602]: 2023-09-30T15:06:36.570+0000 7f22ecc98700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-30T15:06:37.108 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 30 15:06:36 smithi156 ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-l[139602]: 2023-09-30T15:06:36.570+0000 7f22ecc98700 -1 mds.l *** got signal Terminated *** 2023-09-30T15:06:41.195 INFO:journalctl@ceph.mds.l.smithi156.stdout:Sep 30 15:06:40 smithi156 bash[152348]: ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7-mds-l 2023-09-30T15:06:41.202 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca3f5c6-5f9d-11ee-8db5-212e2dc638e7@mds.l.service' 2023-09-30T15:06:41.237 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-30T15:06:41.237 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-09-30T15:06:41.238 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-09-30T15:06:41.554 INFO:teuthology.orchestra.run.smithi032.stdout:Deleting cluster with fsid: bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T15:06:43.198 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-09-30T15:06:43.525 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T15:06:45.162 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-09-30T15:06:45.465 INFO:teuthology.orchestra.run.smithi156.stdout:Deleting cluster with fsid: bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T15:06:47.079 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-30T15:06:47.117 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-30T15:06:47.155 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-30T15:06:47.185 INFO:tasks.cephadm:Archiving crash dumps... 2023-09-30T15:06:47.187 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627/remote/smithi032/crash 2023-09-30T15:06:47.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/crash -- . 2023-09-30T15:06:47.233 INFO:teuthology.orchestra.run.smithi032.stderr:tar: /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-09-30T15:06:47.233 INFO:teuthology.orchestra.run.smithi032.stderr:tar: Error is not recoverable: exiting now 2023-09-30T15:06:47.235 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627/remote/smithi148/crash 2023-09-30T15:06:47.236 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/crash -- . 2023-09-30T15:06:47.281 INFO:teuthology.orchestra.run.smithi148.stderr:tar: /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-09-30T15:06:47.282 INFO:teuthology.orchestra.run.smithi148.stderr:tar: Error is not recoverable: exiting now 2023-09-30T15:06:47.283 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627/remote/smithi156/crash 2023-09-30T15:06:47.284 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/crash -- . 2023-09-30T15:06:47.355 INFO:teuthology.orchestra.run.smithi156.stderr:tar: /var/lib/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-09-30T15:06:47.355 INFO:teuthology.orchestra.run.smithi156.stderr:tar: Error is not recoverable: exiting now 2023-09-30T15:06:47.356 INFO:tasks.cephadm:Checking cluster log for badness... 2023-09-30T15:06:47.357 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-09-30T15:06:47.395 INFO:teuthology.orchestra.run.smithi032.stderr:grep: /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph.log: No such file or directory 2023-09-30T15:06:47.397 INFO:tasks.cephadm:Compressing logs... 2023-09-30T15:06:47.398 DEBUG:teuthology.orchestra.run.smithi032:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-30T15:06:47.440 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-30T15:06:47.443 DEBUG:teuthology.orchestra.run.smithi156:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-30T15:06:47.470 INFO:teuthology.orchestra.run.smithi032.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.0.log 2023-09-30T15:06:47.470 INFO:teuthology.orchestra.run.smithi032.stderr:: No such file or directory 2023-09-30T15:06:47.471 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.g.log 2023-09-30T15:06:47.471 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.a.log 2023-09-30T15:06:47.471 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.6.log 2023-09-30T15:06:47.472 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.3.log 2023-09-30T15:06:47.477 INFO:teuthology.orchestra.run.smithi148.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.10.log 2023-09-30T15:06:47.477 INFO:teuthology.orchestra.run.smithi148.stderr:: No such file or directory 2023-09-30T15:06:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.g.log: /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.a.log: /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.d.log 2023-09-30T15:06:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mgr.x.log 2023-09-30T15:06:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.d.log: /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.9.log 2023-09-30T15:06:47.481 INFO:teuthology.orchestra.run.smithi032.stderr: 94.7% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.g.log.gz 2023-09-30T15:06:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.9.log: 94.7% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.a.log.gz 2023-09-30T15:06:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-09-30T15:06:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-volume.log: 94.7% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.d.log.gz 2023-09-30T15:06:47.482 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.h.log 2023-09-30T15:06:47.482 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.e.log 2023-09-30T15:06:47.482 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.4.log 2023-09-30T15:06:47.482 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.1.log 2023-09-30T15:06:47.482 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.k.log 2023-09-30T15:06:47.482 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-09-30T15:06:47.484 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.j.log 2023-09-30T15:06:47.484 INFO:teuthology.orchestra.run.smithi156.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-09-30T15:06:47.484 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.5.log 2023-09-30T15:06:47.485 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.7.log 2023-09-30T15:06:47.485 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mon.b.log 2023-09-30T15:06:47.485 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.b.log 2023-09-30T15:06:47.485 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mgr.y.log 2023-09-30T15:06:47.485 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-30T15:06:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mon.a.log 2023-09-30T15:06:47.487 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.b.log: /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mgr.y.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.k.log.gz 2023-09-30T15:06:47.489 INFO:teuthology.orchestra.run.smithi148.stderr: 93.3% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mgr.y.log.gz 2023-09-30T15:06:47.490 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.2.log 2023-09-30T15:06:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mon.a.log: 91.0% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-09-30T15:06:47.491 INFO:teuthology.orchestra.run.smithi032.stderr: 94.6% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.j.log.gz 2023-09-30T15:06:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-30T15:06:47.493 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.11.log 2023-09-30T15:06:47.493 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mgr.z.log 2023-09-30T15:06:47.494 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.8.log 2023-09-30T15:06:47.494 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.f.log 2023-09-30T15:06:47.494 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.c.log 2023-09-30T15:06:47.494 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.i.log 2023-09-30T15:06:47.494 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mon.c.log 2023-09-30T15:06:47.494 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.c.log: gzip -5 /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.i.log: --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-09-30T15:06:47.494 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.l.log 2023-09-30T15:06:47.494 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-30T15:06:47.497 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-volume.log: /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.l.log: /var/log/ceph/cephadm.log: 93.4% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mgr.z.log.gz 2023-09-30T15:06:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-30T15:06:47.498 INFO:teuthology.orchestra.run.smithi156.stderr: 94.6% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.l.log.gz 2023-09-30T15:06:47.587 INFO:teuthology.orchestra.run.smithi148.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-30T15:06:47.858 INFO:teuthology.orchestra.run.smithi032.stderr: 90.1% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mgr.x.log.gz 2023-09-30T15:06:47.956 INFO:teuthology.orchestra.run.smithi156.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-30T15:06:48.149 INFO:teuthology.orchestra.run.smithi148.stderr: 91.0% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-09-30T15:06:48.319 INFO:teuthology.orchestra.run.smithi156.stderr: 91.0% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-09-30T15:06:48.641 INFO:teuthology.orchestra.run.smithi156.stderr: 92.1% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mon.c.log.gz 2023-09-30T15:06:49.372 INFO:teuthology.orchestra.run.smithi032.stderr: 90.4% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mon.a.log.gz 2023-09-30T15:07:05.581 INFO:teuthology.orchestra.run.smithi148.stderr: 93.8% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mon.b.log.gz 2023-09-30T15:07:06.234 INFO:teuthology.orchestra.run.smithi032.stderr: 94.2% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.9.log.gz 2023-09-30T15:07:09.558 INFO:teuthology.orchestra.run.smithi032.stderr: 93.8% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.0.log.gz 2023-09-30T15:07:14.007 INFO:teuthology.orchestra.run.smithi032.stderr: 94.0% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.6.log.gz 2023-09-30T15:07:18.240 INFO:teuthology.orchestra.run.smithi032.stderr: 93.9% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.3.log.gz 2023-09-30T15:07:18.242 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-09-30T15:07:18.242 INFO:teuthology.orchestra.run.smithi032.stderr:real 0m30.784s 2023-09-30T15:07:18.242 INFO:teuthology.orchestra.run.smithi032.stderr:user 1m36.910s 2023-09-30T15:07:18.242 INFO:teuthology.orchestra.run.smithi032.stderr:sys 0m3.412s 2023-09-30T15:07:40.868 INFO:teuthology.orchestra.run.smithi156.stderr: 94.9% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.c.log.gz 2023-09-30T15:07:47.460 INFO:teuthology.orchestra.run.smithi156.stderr: 95.2% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.f.log.gz 2023-09-30T15:07:49.691 INFO:teuthology.orchestra.run.smithi156.stderr: 93.9% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.11.log.gz 2023-09-30T15:08:03.953 INFO:teuthology.orchestra.run.smithi156.stderr: 94.1% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.8.log.gz 2023-09-30T15:08:04.967 INFO:teuthology.orchestra.run.smithi156.stderr: 94.0% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.2.log.gz 2023-09-30T15:08:11.286 INFO:teuthology.orchestra.run.smithi156.stderr: 94.0% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.5.log.gz 2023-09-30T15:08:12.582 INFO:teuthology.orchestra.run.smithi148.stderr: 94.1% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.4.log.gz 2023-09-30T15:08:19.229 INFO:teuthology.orchestra.run.smithi148.stderr: 93.8% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.1.log.gz 2023-09-30T15:08:21.408 INFO:teuthology.orchestra.run.smithi148.stderr: 95.1% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.e.log.gz 2023-09-30T15:08:26.727 INFO:teuthology.orchestra.run.smithi148.stderr: 94.0% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.10.log.gz 2023-09-30T15:08:35.086 INFO:teuthology.orchestra.run.smithi148.stderr: 94.0% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-osd.7.log.gz 2023-09-30T15:12:48.233 INFO:teuthology.orchestra.run.smithi156.stderr: 93.2% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.i.log.gz 2023-09-30T15:12:48.267 INFO:teuthology.orchestra.run.smithi156.stderr: 2023-09-30T15:12:48.267 INFO:teuthology.orchestra.run.smithi156.stderr:real 6m0.807s 2023-09-30T15:12:48.267 INFO:teuthology.orchestra.run.smithi156.stderr:user 6m37.531s 2023-09-30T15:12:48.267 INFO:teuthology.orchestra.run.smithi156.stderr:sys 0m16.149s 2023-09-30T15:15:02.790 INFO:teuthology.orchestra.run.smithi148.stderr: 93.1% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.b.log.gz 2023-09-30T15:17:20.846 INFO:teuthology.orchestra.run.smithi148.stderr: 93.2% -- replaced with /var/log/ceph/bca3f5c6-5f9d-11ee-8db5-212e2dc638e7/ceph-mds.h.log.gz 2023-09-30T15:17:20.856 INFO:teuthology.orchestra.run.smithi148.stderr: 2023-09-30T15:17:20.856 INFO:teuthology.orchestra.run.smithi148.stderr:real 10m33.396s 2023-09-30T15:17:20.856 INFO:teuthology.orchestra.run.smithi148.stderr:user 12m14.638s 2023-09-30T15:17:20.856 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m30.276s 2023-09-30T15:17:20.856 INFO:tasks.cephadm:Archiving logs... 2023-09-30T15:17:20.857 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627/remote/smithi032/log 2023-09-30T15:17:20.858 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-30T15:17:34.361 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627/remote/smithi148/log 2023-09-30T15:17:34.362 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-30T15:19:25.684 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627/remote/smithi156/log 2023-09-30T15:19:25.685 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-30T15:20:25.885 INFO:tasks.cephadm:Removing cluster... 2023-09-30T15:20:25.886 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 --force 2023-09-30T15:20:26.200 INFO:teuthology.orchestra.run.smithi032.stdout:Deleting cluster with fsid: bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T15:20:26.801 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 --force 2023-09-30T15:20:27.394 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T15:20:28.498 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 --force 2023-09-30T15:20:29.189 INFO:teuthology.orchestra.run.smithi156.stdout:Deleting cluster with fsid: bca3f5c6-5f9d-11ee-8db5-212e2dc638e7 2023-09-30T15:20:30.049 INFO:tasks.cephadm:Removing cephadm ... 2023-09-30T15:20:30.049 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-30T15:20:30.071 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-30T15:20:30.108 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-30T15:20:30.133 INFO:tasks.cephadm:Teardown complete 2023-09-30T15:20:30.133 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-30T15:20:30.147 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-30T15:20:30.148 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-30T15:20:30.150 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-30T15:20:30.152 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-30T15:20:30.290 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-30T15:20:30.301 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-30T15:20:30.301 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T15:20:30.304 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T15:20:30.306 DEBUG:teuthology.orchestra.run.smithi156:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T15:20:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:bash: ntpq: command not found 2023-09-30T15:20:30.322 INFO:teuthology.orchestra.run.smithi156.stderr:bash: ntpq: command not found 2023-09-30T15:20:30.323 INFO:teuthology.orchestra.run.smithi148.stderr:bash: ntpq: command not found 2023-09-30T15:20:30.325 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T15:20:30.325 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2023-09-30T15:20:30.325 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 62 +110us[ +110us] +/- 77ms 2023-09-30T15:20:30.325 INFO:teuthology.orchestra.run.smithi032.stdout:^* hv02.front.sepia.ceph.com 3 6 377 64 -911us[ -911us] +/- 77ms 2023-09-30T15:20:30.325 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +838us[ +838us] +/- 79ms 2023-09-30T15:20:30.325 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T15:20:30.342 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T15:20:30.342 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2023-09-30T15:20:30.342 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 0 +115us[ +115us] +/- 78ms 2023-09-30T15:20:30.342 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv02.front.sepia.ceph.com 3 6 377 62 -950us[ -955us] +/- 77ms 2023-09-30T15:20:30.342 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +825us[ +825us] +/- 79ms 2023-09-30T15:20:30.342 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T15:20:30.345 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T15:20:30.345 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-09-30T15:20:30.345 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 61 +124us[ +124us] +/- 77ms 2023-09-30T15:20:30.346 INFO:teuthology.orchestra.run.smithi156.stdout:^* hv02.front.sepia.ceph.com 3 6 377 63 -902us[ -824us] +/- 77ms 2023-09-30T15:20:30.346 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 +841us[ +841us] +/- 78ms 2023-09-30T15:20:30.346 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T15:20:30.346 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-30T15:20:30.357 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-30T15:20:30.357 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-30T15:20:30.373 DEBUG:teuthology.orchestra.run.smithi032:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-30T15:20:30.481 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-30T15:20:30.689 DEBUG:teuthology.orchestra.run.smithi156:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-30T15:20:30.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T15:20:30.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T15:20:30.913 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T15:20:30.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T15:20:30.958 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T15:20:30.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T15:20:30.993 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-30T15:20:31.021 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-30T15:20:31.059 INFO:teuthology.task.internal:Duration was 3785.899720 seconds 2023-09-30T15:20:31.059 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-30T15:20:31.079 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-30T15:20:31.079 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-30T15:20:31.091 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-30T15:20:31.104 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-30T15:20:31.139 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T15:20:31.195 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T15:20:31.217 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T15:20:31.401 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-30T15:20:31.401 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T15:20:31.402 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-30T15:20:31.427 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2023-09-30T15:20:31.427 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-30T15:20:31.462 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi156.front.sepia.ceph.com 2023-09-30T15:20:31.463 DEBUG:teuthology.orchestra.run.smithi156:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-30T15:20:31.500 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-30T15:20:31.501 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T15:20:31.504 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T15:20:31.508 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T15:20:31.608 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-30T15:20:31.608 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T15:20:31.611 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T15:20:31.613 DEBUG:teuthology.orchestra.run.smithi156:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T15:20:31.722 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-30T15:20:31.776 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-30T15:20:31.776 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-30T15:20:31.804 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-30T15:20:31.835 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-30T15:20:31.879 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-30T15:20:31.897 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-30T15:20:31.900 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-30T15:20:31.902 DEBUG:teuthology.orchestra.run.smithi156:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-30T15:20:31.925 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2023-09-30T15:20:31.935 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2023-09-30T15:20:31.977 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = core 2023-09-30T15:20:31.993 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-30T15:20:32.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T15:20:32.011 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-30T15:20:32.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T15:20:32.030 DEBUG:teuthology.orchestra.run.smithi156:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-30T15:20:32.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T15:20:32.048 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-30T15:20:32.061 INFO:teuthology.task.internal:Transferring archived files... 2023-09-30T15:20:32.063 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627/remote/smithi032 2023-09-30T15:20:32.063 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-30T15:20:32.111 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627/remote/smithi148 2023-09-30T15:20:32.111 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-30T15:20:32.158 DEBUG:teuthology.misc:Transferring archived files from smithi156:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627/remote/smithi156 2023-09-30T15:20:32.159 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-30T15:20:32.210 INFO:teuthology.task.internal:Removing archive directory... 2023-09-30T15:20:32.211 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-30T15:20:32.213 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-30T15:20:32.215 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-30T15:20:32.258 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-30T15:20:32.272 INFO:teuthology.task.internal:Not uploading archives. 2023-09-30T15:20:32.272 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-30T15:20:32.284 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-30T15:20:32.284 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-30T15:20:32.287 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-30T15:20:32.289 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-30T15:20:32.301 INFO:teuthology.orchestra.run.smithi032.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 30 15:20 /home/ubuntu/cephtest 2023-09-30T15:20:32.303 INFO:teuthology.orchestra.run.smithi148.stdout: 262181 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 30 15:20 /home/ubuntu/cephtest 2023-09-30T15:20:32.314 INFO:teuthology.orchestra.run.smithi156.stdout: 262181 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 30 15:20 /home/ubuntu/cephtest 2023-09-30T15:20:32.318 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-30T15:20:32.331 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-30T15:20:32.356 INFO:teuthology.nuke:Checking targets against current locks 2023-09-30T15:20:32.385 DEBUG:teuthology.nuke:shortname: smithi032 2023-09-30T15:20:32.385 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T15:20:32.411 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 14:11:51.467913', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-30T15:20:32.415 DEBUG:teuthology.nuke:shortname: smithi148 2023-09-30T15:20:32.416 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T15:20:32.439 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 14:11:51.470920', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-30T15:20:32.444 DEBUG:teuthology.nuke:shortname: smithi156 2023-09-30T15:20:32.444 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T15:20:32.468 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:46:13-fs-main-distro-default-smithi/7407627', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 14:11:51.469733', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-30T15:20:32.477 INFO:teuthology.orchestra.console.smithi032:Power off 2023-09-30T15:20:32.477 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-30T15:20:32.499 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2023-09-30T15:20:32.500 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T15:20:32.515 INFO:teuthology.orchestra.console.smithi148:Power off 2023-09-30T15:20:32.516 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-30T15:20:32.529 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-09-30T15:20:32.539 DEBUG:teuthology.orchestra.console.smithi148:power off output: Chassis Power Control: Down/Off 2023-09-30T15:20:32.539 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T15:20:32.558 INFO:teuthology.orchestra.console.smithi156:Power off 2023-09-30T15:20:32.558 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-30T15:20:32.567 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-09-30T15:20:32.580 DEBUG:teuthology.orchestra.console.smithi156:power off output: Chassis Power Control: Down/Off 2023-09-30T15:20:32.580 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T15:20:32.603 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2023-09-30T15:20:36.530 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T15:20:36.568 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T15:20:36.604 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T15:20:36.642 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-09-30T15:20:36.680 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-09-30T15:20:36.714 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2023-09-30T15:20:40.646 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T15:20:40.681 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T15:20:40.715 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T15:20:40.756 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2023-09-30T15:20:40.791 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2023-09-30T15:20:40.827 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2023-09-30T15:20:40.857 INFO:teuthology.orchestra.console.smithi032:Power off completed 2023-09-30T15:20:40.893 INFO:teuthology.orchestra.console.smithi148:Power off completed 2023-09-30T15:20:40.928 INFO:teuthology.orchestra.console.smithi156:Power off completed 2023-09-30T15:20:41.036 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} duration: 3785.8997201919556 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2023-09-30T15:20:41.036 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-30T15:20:41.121 INFO:teuthology.run:FAIL